gpt-5.2 v1 Evaluation

Generated: 2026-07-25 12:53:22 UTC

Pairs100

Distinct lemmas100

Slope1.011

R^20.983

BLEU-423.4%

chrF++53.9%

METEOR58.7%

ROUGE-L58.5%

BERTScoreN/A

COMETN/A

BLEURTN/A

Profile version id: 2331

First translation using this version: 2026-02-14

Total usable AI runs for this version: 113 runs across 109 lemmas

Created: 2026-07-15 12:28:27.643960+10:00

Active: True

Notes: Model timeline evaluation: GPT-5.2 using prompt text from gpt-5.5 v1; approved-human-only for the 100-row Kappa corpus.

Length Regression

Formula	AI words = intercept + slope * human words
Slope	1.011
Intercept	5.096
R^2	0.983
P-value	1.12e-88
Pearson r	0.992
Mean source words	30.400
Length fallback rows	0
Mean human words	44.160
Mean AI words	49.720
Mean absolute length residual	3.993
Mean absolute percent error	19.8%

Metric vs Passage Length

These plots test whether score changes with source passage length for this prompt version. The dashed line is an ordinary least-squares fit; the annotation reports n, r, R^2, p-value, and slope.

BLEU-4 vs passage length — BLEU-4: negative; r = -0.089, R^2 = 0.008, p = 0.3809.

chrF++ vs passage length — chrF++: negative; r = -0.108, R^2 = 0.012, p = 0.2858.

METEOR vs passage length — METEOR: negative; r = -0.177, R^2 = 0.031, p = 0.0788.

ROUGE-L vs passage length — ROUGE-L: negative; r = -0.097, R^2 = 0.009, p = 0.3357.

Trigram precision vs passage length — Trigram precision: positive; r = 0.024, R^2 = 5.96e-04, p = 0.8094.

Trigram recall vs passage length — Trigram recall: negative; r = -0.035, R^2 = 0.001, p = 0.7286.

Trigram F1 vs passage length — Trigram F1: negative; r = -5.81e-04, R^2 = 3.38e-07, p = 0.9954.

Trigram Jaccard vs passage length — Trigram Jaccard: negative; r = -0.019, R^2 = 3.63e-04, p = 0.8508.

Metric	Rows	Pattern	Pearson r	R^2	P-value	Slope	Status
BLEU-4	100	negative	-0.089	0.008	0.3809	-0.00040	ok
chrF++	100	negative	-0.108	0.012	0.2858	-0.00035	ok
METEOR	100	negative	-0.177	0.031	0.0788	-0.00085	ok
ROUGE-L	100	negative	-0.097	0.009	0.3357	-0.00041	ok
BERTScore	0	not enough data	N/A	N/A	N/A	N/A	needs at least two rows with metric scores
COMET	0	not enough data	N/A	N/A	N/A	N/A	needs at least two rows with metric scores
BLEURT	0	not enough data	N/A	N/A	N/A	N/A	needs at least two rows with metric scores
Trigram precision	100	positive	0.024	5.96e-04	0.8094	0.00011	ok
Trigram recall	100	negative	-0.035	0.001	0.7286	-0.00017	ok
Trigram F1	100	negative	-5.81e-04	3.38e-07	0.9954	-2.69e-06	ok
Trigram Jaccard	100	negative	-0.019	3.63e-04	0.8508	-0.00006	ok

Translation Similarity Metrics

Mean BLEU-4	23.4%
Mean chrF++	53.9%
Mean METEOR	58.7%
Mean ROUGE-L	58.5%
Mean BERTScore	N/A
Mean COMET	N/A
Mean BLEURT	N/A
Legacy smoothed corpus BLEU	24.5%
Unigram F1	61.7%
Bigram F1	32.8%
Trigram precision	18.2%
Trigram recall	20.6%
Trigram F1	19.3%
Trigram Jaccard	10.7%
4-gram F1	11.6%

Largest Length Residuals

Lemma ID	Headword	Human words	AI words	Residual	BLEU-4	chrF++	METEOR	ROUGE-L	BERTScore	COMET	BLEURT	Human	Run	Model
2468	Καπετώλιον	138	159	14.455	13.9%	44.2%	41.0%	47.8%	N/A	N/A	N/A	reviewed/approved	11473	gpt-5.2-2025-12-11
2055	Καβασσός	109	102	-13.241	16.2%	46.5%	39.8%	44.5%	N/A	N/A	N/A	reviewed/approved	11339	gpt-5.2-2025-12-11
2626	Κατάνη	95	89	-12.094	26.8%	51.4%	51.4%	53.3%	N/A	N/A	N/A	reviewed/approved	11530	gpt-5.2-2025-12-11
3288	Κορώνεια	110	128	11.749	15.2%	51.9%	50.6%	48.5%	N/A	N/A	N/A	reviewed/approved	11545	gpt-5.2-2025-12-11
2081	Καλάθη	49	43	-11.611	33.7%	61.9%	67.3%	80.4%	N/A	N/A	N/A	reviewed/approved	11386	gpt-5.2-2025-12-11
2057	Καβελλιών	34	51	11.547	10.9%	47.5%	51.8%	44.7%	N/A	N/A	N/A	reviewed/approved	11345	gpt-5.2-2025-12-11
2056	Καβειρία	112	107	-11.272	21.4%	55.7%	63.2%	67.6%	N/A	N/A	N/A	reviewed/approved	11342	gpt-5.2-2025-12-11
7245	Κύρης	13	28	9.767	4.8%	32.2%	36.7%	39.0%	N/A	N/A	N/A	reviewed/approved	11565	gpt-5.2-2025-12-11
7243	Κύρη	22	37	9.673	16.5%	49.2%	45.1%	47.5%	N/A	N/A	N/A	reviewed/approved	11560	gpt-5.2-2025-12-11
2079	Καισάρεια	39	35	-9.506	30.2%	56.3%	67.6%	70.3%	N/A	N/A	N/A	reviewed/approved	11380	gpt-5.2-2025-12-11
2328	Καλλίπολις	43	58	9.452	8.2%	44.4%	40.7%	47.5%	N/A	N/A	N/A	reviewed/approved	11434	gpt-5.2-2025-12-11
2605	Κάσιον	56	71	9.316	17.9%	53.5%	62.5%	50.4%	N/A	N/A	N/A	reviewed/approved	11500	gpt-5.2-2025-12-11
7267	Κωνώπη	64	79	9.232	26.4%	51.8%	52.6%	55.9%	N/A	N/A	N/A	reviewed/approved	11630	gpt-5.2-2025-12-11
7249	Κύρρος	45	59	8.431	4.9%	41.0%	42.5%	46.2%	N/A	N/A	N/A	reviewed/approved	11577	gpt-5.2-2025-12-11
7255	Κυτέριον	22	35	7.673	21.6%	57.4%	65.8%	56.1%	N/A	N/A	N/A	reviewed/approved	11594	gpt-5.2-2025-12-11
2597	Καρπασία	112	111	-7.272	13.5%	50.6%	47.4%	49.3%	N/A	N/A	N/A	reviewed/approved	11485	gpt-5.2-2025-12-11
2115	Καλὴ ἀκτή	64	77	7.232	16.0%	42.6%	47.9%	51.1%	N/A	N/A	N/A	reviewed/approved	11410	gpt-5.2-2025-12-11
2119	Κάλλατις	65	78	7.221	16.0%	49.0%	50.0%	47.6%	N/A	N/A	N/A	reviewed/approved	11422	gpt-5.2-2025-12-11
7266	Κώμη	79	92	7.074	5.4%	36.4%	31.7%	31.6%	N/A	N/A	N/A	reviewed/approved	11627	gpt-5.2-2025-12-11
2802	Κάδρεμα	24	23	-6.348	24.5%	53.6%	59.6%	66.7%	N/A	N/A	N/A	reviewed/approved	11360	gpt-5.2-2025-12-11
2330	Καλύβη	24	23	-6.348	19.1%	46.7%	66.7%	68.1%	N/A	N/A	N/A	reviewed/approved	11440	gpt-5.2-2025-12-11
2078	Καινύς	22	21	-6.327	60.1%	71.9%	76.6%	74.4%	N/A	N/A	N/A	reviewed/approved	11375	gpt-5.2-2025-12-11
2061	Καδούσιοι	14	13	-6.243	54.2%	65.4%	77.2%	74.1%	N/A	N/A	N/A	reviewed/approved	11357	gpt-5.2-2025-12-11
7253	Κυρτώνιος	18	29	5.715	7.8%	45.8%	49.6%	46.8%	N/A	N/A	N/A	reviewed/approved	11588	gpt-5.2-2025-12-11
2335	Κάλυτις	26	37	5.631	3.9%	42.4%	43.2%	34.9%	N/A	N/A	N/A	reviewed/approved	11455	gpt-5.2-2025-12-11
2599	Καρπήσιοι	11	11	-5.212	15.7%	46.8%	40.5%	45.5%	N/A	N/A	N/A	reviewed/approved	11488	gpt-5.2-2025-12-11
2484	Καρία	270	273	-4.931	18.4%	47.4%	42.6%	49.0%	N/A	N/A	N/A	reviewed/approved	11482	gpt-5.2-2025-12-11
2345	Κάναθα	17	27	4.725	22.8%	57.0%	70.5%	59.1%	N/A	N/A	N/A	reviewed/approved	11461	gpt-5.2-2025-12-11
2621	Κάστνιον	25	35	4.641	18.1%	49.0%	62.0%	53.3%	N/A	N/A	N/A	reviewed/approved	11512	gpt-5.2-2025-12-11
2326	Καλλίαρος	47	57	4.410	28.4%	54.8%	59.9%	59.6%	N/A	N/A	N/A	reviewed/approved	11428	gpt-5.2-2025-12-11

Residual Predictors

Positive signed-residual terms are associated with AI translations longer than expected from the human length; negative signed-residual terms are associated with shorter AI translations. Absolute-residual models identify terms associated with larger AI-human length divergence.

Greek source terms predicting signed length residuals

Features: 176; ridge alpha: 2.894; cross-validated R^2: -0.239.

Positive term	Coefficient	Docs	Mean present	Mean absent
δια του	2.4773	10	4.006	-0.445
εν	2.2427	37	1.715	-1.007
ως	2.0840	52	0.681	-0.737
εν τη	1.9990	7	4.684	-0.353
του	1.8689	37	0.903	-0.530
δευτερα	1.8112	3	8.944	-0.277
δια	1.8078	16	2.326	-0.443
πολυβιος	1.6985	5	4.549	-0.239
την	1.5992	13	1.607	-0.240
γαρ	1.5735	9	1.381	-0.137
παρα	1.4752	7	2.696	-0.203
χωρας	1.4623	4	3.883	-0.162
κωμη	1.4421	3	5.586	-0.173
της χωρας	1.3019	3	4.624	-0.143
εν πρωτη	1.2727	3	3.675	-0.114
πρωτη	1.2727	3	3.675	-0.114
εξ	1.2199	3	7.373	-0.228
δευτερω	1.1198	3	4.076	-0.126
ος	1.1070	6	4.460	-0.285
πολιχνιον	1.1055	3	4.923	-0.152

Negative term	Coefficient	Docs	Mean present	Mean absent
καππαδοκιας	-2.1469	3	-8.032	0.248
εθνος	-1.9273	8	-3.927	0.342
και του	-1.8801	4	-7.600	0.317
λυκιας	-1.6393	3	-5.958	0.184
προς	-1.6202	15	-1.248	0.220
φησιν	-1.5716	10	-2.136	0.237
εστι και	-1.3802	20	-1.457	0.364
ουτως	-1.3425	5	-5.005	0.263
ως το	-1.2796	3	-8.101	0.251
μετα	-1.2106	3	-6.846	0.212
της	-1.1187	35	-0.117	0.063
υπο	-1.0955	4	-1.863	0.078
δεκατω	-1.0876	3	-4.021	0.124
των	-1.0862	17	-1.567	0.321
αυτην	-1.0841	9	-2.093	0.207
εισι	-1.0791	5	-3.865	0.203
πλησιον	-1.0113	8	-2.262	0.197
θηλυκον και	-0.9862	3	-5.368	0.166
θρακης	-0.9678	5	-2.239	0.118
εκαλειτο	-0.9516	7	-3.150	0.237

Greek source terms predicting large absolute residuals

Features: 176; ridge alpha: 11.938; cross-validated R^2: -0.047.

Positive term	Coefficient	Docs	Mean present	Mean absent
δε	0.6570	46	4.848	3.265
γαρ	0.5309	9	6.407	3.754
της	0.4504	35	5.473	3.196
την	0.4385	13	6.039	3.687
παρα	0.4097	7	7.325	3.742
τον	0.3980	10	6.329	3.734
κατα	0.3952	10	5.425	3.834
εκαταιος ευρωπη	0.3872	5	9.149	3.722
ευρωπη	0.3872	5	9.149	3.722
εν	0.3764	37	4.738	3.556
τριτη	0.3398	3	10.236	3.800
δυναται	0.3274	3	10.887	3.780
δια του	0.2887	10	5.386	3.838
ποντου	0.2876	3	8.358	3.858
του ποντου	0.2876	3	8.358	3.858
καππαδοκιας	0.2724	3	8.032	3.868
δευτερα	0.2686	3	8.944	3.840
τα	0.2678	12	6.582	3.640
εστι	0.2547	24	5.252	3.596
βοιωτιας	0.2509	3	8.880	3.842

Negative term	Coefficient	Docs	Mean present	Mean absent
εκ	-0.3340	6	1.765	4.135
ηροδοτος	-0.3304	9	2.846	4.107
θεοπομπος	-0.2838	5	2.985	4.046
πολις και	-0.2762	6	1.843	4.130
παιδος	-0.2580	4	1.408	4.101
εκαταιος ασια	-0.2388	5	2.211	4.087
ασια	-0.2388	5	2.211	4.087
απο	-0.1961	27	4.058	3.969
τοπος	-0.1938	4	1.715	4.088
τινες	-0.1904	6	2.414	4.094
θαλασση	-0.1900	3	1.372	4.074
οι	-0.1851	11	3.604	4.041
το εθνικον	-0.1723	56	3.839	4.189
εθνικον	-0.1723	56	3.839	4.189
οικητωρ	-0.1662	6	3.654	4.015
και το	-0.1645	9	3.606	4.031
διονυσιος	-0.1577	4	3.175	4.027
πλησιον	-0.1554	8	2.875	4.090
λιβυης	-0.1525	4	2.408	4.059
μια	-0.1507	3	2.029	4.054

AI English terms predicting large absolute residuals

Features: 404; ridge alpha: 24.245; cross-validated R^2: -0.052.

Positive term	Coefficient	Docs	Mean present	Mean absent
i	0.2473	12	6.313	3.677
with	0.2309	17	5.493	3.686
for	0.2053	11	5.224	3.841
the	0.1743	97	4.059	1.849
to the	0.1450	6	6.772	3.816
it	0.1413	51	4.591	3.371
on the	0.1403	9	6.234	3.771
or	0.1344	13	6.343	3.642
on	0.1303	16	4.848	3.830
formed	0.1273	4	8.971	3.786
but	0.1213	19	5.590	3.619
is a	0.1185	9	6.335	3.762
e	0.1173	10	4.925	3.890
cappadocia	0.1150	4	6.321	3.896
sicily	0.1122	3	9.291	3.829
a third	0.1116	3	10.236	3.800
it is	0.1109	24	4.796	3.740
according	0.1109	5	6.502	3.861
according to	0.1109	5	6.502	3.861
i e	0.1106	8	5.737	3.842

Negative term	Coefficient	Docs	Mean present	Mean absent
the ethnic gentilic	-0.1508	21	3.227	4.197
gentilic	-0.1508	21	3.227	4.197
ethnic gentilic	-0.1508	21	3.227	4.197
book of the	-0.1385	8	2.573	4.117
of the	-0.1284	47	3.615	4.328
gentilic is	-0.1194	17	3.155	4.165
ethnic gentilic is	-0.1194	17	3.155	4.165
the ethnic	-0.1074	54	3.756	4.272
also called	-0.1060	14	2.582	4.223
and	-0.1059	55	4.014	3.968
ethnic	-0.1014	58	3.837	4.209
the ethnic name	-0.0981	5	1.335	4.133
ethnic name	-0.0981	5	1.335	4.133
herodotus	-0.0912	9	2.846	4.107
an	-0.0880	19	3.232	4.172
named from	-0.0835	5	1.462	4.126
named	-0.0796	16	3.167	4.151
its	-0.0767	9	3.308	4.061
named after	-0.0766	5	2.009	4.098
in the feminine	-0.0746	4	1.847	4.083

Human English terms predicting large absolute residuals

Features: 336; ridge alpha: 10000.000; cross-validated R^2: -0.053.

Positive term	Coefficient	Docs	Mean present	Mean absent
form	0.0007	11	7.266	3.589
the	0.0006	90	4.188	2.240
with	0.0006	16	5.213	3.761
on	0.0005	16	5.135	3.776
the form	0.0004	9	7.560	3.640
on the	0.0004	10	6.481	3.717
is	0.0004	87	4.042	3.669
of	0.0004	66	4.301	3.396
has	0.0004	6	9.457	3.644
his europe	0.0004	5	9.149	3.722
in his europe	0.0004	5	9.149	3.722
from	0.0004	37	4.600	3.637
a	0.0004	91	4.139	2.523
as in	0.0004	29	4.435	3.813
in	0.0003	87	4.147	2.963
europe	0.0003	6	8.128	3.729
there	0.0003	34	5.183	3.380
one	0.0003	12	6.958	3.589
sicily	0.0003	3	9.291	3.829
is also a	0.0003	14	5.484	3.751

Negative term	Coefficient	Docs	Mean present	Mean absent
herodotos	-0.0003	9	2.846	4.107
the ethnonym is	-0.0002	52	3.521	4.505
ethnonym is	-0.0002	52	3.521	4.505
in his asia	-0.0002	5	2.211	4.087
his asia	-0.0002	5	2.211	4.087
inhabitant is a	-0.0002	6	2.694	4.076
book the	-0.0002	10	2.961	4.108
libya	-0.0002	5	2.065	4.095
in libya	-0.0002	5	2.065	4.095
theopompos in book	-0.0002	4	1.913	4.080
theopompos in	-0.0002	4	1.913	4.080
book the ethnonym	-0.0002	7	2.754	4.086
lydia	-0.0002	3	2.015	4.054
some	-0.0002	10	2.996	4.104
inhabitant is	-0.0002	7	3.320	4.044
inhabitant	-0.0002	7	3.320	4.044
an inhabitant is	-0.0002	7	3.320	4.044
an inhabitant	-0.0002	7	3.320	4.044
is the	-0.0002	3	0.901	4.089
also called	-0.0002	6	2.471	4.090

Prompt Text

Show prompt text

You are an expert classical philologist and translator specializing in Byzantine Greek geographical texts.
You will receive Greek text from a lemma entry in Stephanos of Byzantium's Ethnika.
Translate the Greek text into clear, scholarly English.
Preserve technical terminology and place names appropriately.