gpt-5.2 v1 Evaluation

Generated: 2026-09-20 11:21:25 UTC

Pairs100

Distinct lemmas100

Slope1.010

R^20.983

BLEU-423.4%

chrF++53.9%

METEOR58.8%

ROUGE-L58.5%

BERTScore91.9%

COMET71.5%

BLEURT63.6%

Profile version id: 2331

First translation using this version: 2026-02-14

Total usable AI runs for this version: 113 runs across 109 lemmas

Created: 2026-07-15 12:28:27.643960+10:00

Active: True

Notes: Model timeline evaluation: GPT-5.2 using prompt text from gpt-5.5 v1; approved-human-only for the 100-row Kappa corpus.

Length Regression

Formula	AI words = intercept + slope * human words
Slope	1.010
Intercept	5.115
R^2	0.983
P-value	9.43e-89
Pearson r	0.992
Mean source words	30.400
Length fallback rows	0
Mean human words	44.150
Mean AI words	49.720
Mean absolute length residual	3.984
Mean absolute percent error	19.9%

Metric vs Passage Length

These plots test whether score changes with source passage length for this prompt version. The dashed line is an ordinary least-squares fit; the annotation reports n, r, R^2, p-value, and slope.

BLEU-4 vs passage length — BLEU-4: negative; r = -0.088, R^2 = 0.008, p = 0.3826.

chrF++ vs passage length — chrF++: negative; r = -0.107, R^2 = 0.011, p = 0.2891.

METEOR vs passage length — METEOR: negative; r = -0.177, R^2 = 0.031, p = 0.0782.

ROUGE-L vs passage length — ROUGE-L: negative; r = -0.098, R^2 = 0.010, p = 0.3315.

BERTScore vs passage length — BERTScore: significant negative; r = -0.287, R^2 = 0.082, p = 0.0038.

COMET vs passage length — COMET: negative; r = -0.007, R^2 = 4.51e-05, p = 0.9471.

BLEURT vs passage length — BLEURT: significant negative; r = -0.640, R^2 = 0.410, p = 7.43e-13.

Trigram precision vs passage length — Trigram precision: positive; r = 0.024, R^2 = 5.96e-04, p = 0.8094.

Trigram recall vs passage length — Trigram recall: negative; r = -0.035, R^2 = 0.001, p = 0.7286.

Trigram F1 vs passage length — Trigram F1: negative; r = -5.81e-04, R^2 = 3.38e-07, p = 0.9954.

Trigram Jaccard vs passage length — Trigram Jaccard: negative; r = -0.019, R^2 = 3.63e-04, p = 0.8508.

Metric	Rows	Pattern	Pearson r	R^2	P-value	Slope	Status
BLEU-4	100	negative	-0.088	0.008	0.3826	-0.00040	ok
chrF++	100	negative	-0.107	0.011	0.2891	-0.00035	ok
METEOR	100	negative	-0.177	0.031	0.0782	-0.00085	ok
ROUGE-L	100	negative	-0.098	0.010	0.3315	-0.00041	ok
BERTScore	100	significant negative	-0.287	0.082	0.0038	-0.00020	ok
COMET	100	negative	-0.007	4.51e-05	0.9471	-0.00001	ok
BLEURT	100	significant negative	-0.640	0.410	7.43e-13	-0.00197	ok
Trigram precision	100	positive	0.024	5.96e-04	0.8094	0.00011	ok
Trigram recall	100	negative	-0.035	0.001	0.7286	-0.00017	ok
Trigram F1	100	negative	-5.81e-04	3.38e-07	0.9954	-2.69e-06	ok
Trigram Jaccard	100	negative	-0.019	3.63e-04	0.8508	-0.00006	ok

Translation Similarity Metrics

Mean BLEU-4	23.4%
Mean chrF++	53.9%
Mean METEOR	58.8%
Mean ROUGE-L	58.5%
Mean BERTScore	91.9%
Mean COMET	71.5%
Mean BLEURT	63.6%
Legacy smoothed corpus BLEU	24.5%
Unigram F1	61.8%
Bigram F1	32.8%
Trigram precision	18.2%
Trigram recall	20.6%
Trigram F1	19.3%
Trigram Jaccard	10.7%
4-gram F1	11.6%

Largest Length Residuals

Lemma ID	Headword	Human words	AI words	Residual	BLEU-4	chrF++	METEOR	ROUGE-L	BERTScore	COMET	BLEURT	Human	Run	Model
2468	Καπετώλιον	138	159	14.462	13.9%	44.2%	41.0%	47.8%	86.0%	65.6%	43.6%	reviewed/approved	11473	gpt-5.2-2025-12-11
2055	Καβασσός	109	102	-13.239	16.2%	46.5%	39.8%	44.5%	90.9%	70.6%	42.2%	reviewed/approved	11339	gpt-5.2-2025-12-11
2626	Κατάνη	95	89	-12.094	26.8%	51.4%	51.4%	53.3%	91.5%	70.8%	52.3%	reviewed/approved	11530	gpt-5.2-2025-12-11
3288	Κορώνεια	110	128	11.751	15.2%	51.9%	50.6%	48.5%	90.6%	73.3%	50.3%	reviewed/approved	11545	gpt-5.2-2025-12-11
2081	Καλάθη	49	43	-11.620	33.7%	61.9%	67.3%	80.4%	93.7%	67.9%	67.0%	reviewed/approved	11386	gpt-5.2-2025-12-11
2057	Καβελλιών	34	51	11.535	10.9%	47.5%	51.8%	44.7%	89.1%	66.9%	52.7%	reviewed/approved	11345	gpt-5.2-2025-12-11
2056	Καβειρία	112	107	-11.270	21.4%	55.7%	63.2%	67.6%	92.5%	73.0%	58.5%	reviewed/approved	11342	gpt-5.2-2025-12-11
7245	Κύρης	13	28	9.751	4.8%	32.2%	36.7%	39.0%	87.9%	56.8%	60.7%	reviewed/approved	11565	gpt-5.2-2025-12-11
7243	Κύρη	22	37	9.658	16.5%	49.2%	45.1%	47.5%	90.0%	72.6%	68.6%	reviewed/approved	11560	gpt-5.2-2025-12-11
2079	Καισάρεια	39	35	-9.517	30.2%	56.3%	67.6%	70.3%	91.9%	70.7%	60.0%	reviewed/approved	11380	gpt-5.2-2025-12-11
2328	Καλλίπολις	43	58	9.442	8.2%	44.4%	40.7%	47.5%	89.3%	72.0%	57.1%	reviewed/approved	11434	gpt-5.2-2025-12-11
2605	Κάσιον	56	71	9.308	17.9%	53.5%	62.5%	50.4%	90.4%	70.0%	57.8%	reviewed/approved	11500	gpt-5.2-2025-12-11
7267	Κωνώπη	64	79	9.225	26.4%	51.8%	52.6%	55.9%	90.7%	63.6%	62.3%	reviewed/approved	11630	gpt-5.2-2025-12-11
7249	Κύρρος	45	59	8.421	4.9%	41.0%	42.5%	46.2%	89.4%	64.7%	49.3%	reviewed/approved	11577	gpt-5.2-2025-12-11
7255	Κυτέριον	22	35	7.658	21.6%	57.4%	65.8%	56.1%	91.5%	68.5%	65.9%	reviewed/approved	11594	gpt-5.2-2025-12-11
2597	Καρπασία	112	111	-7.270	13.5%	50.6%	47.4%	49.3%	92.6%	73.3%	52.7%	reviewed/approved	11485	gpt-5.2-2025-12-11
2115	Καλὴ ἀκτή	64	77	7.225	16.0%	42.6%	47.9%	51.1%	89.1%	63.6%	45.3%	reviewed/approved	11410	gpt-5.2-2025-12-11
2119	Κάλλατις	65	78	7.215	16.0%	49.0%	50.0%	47.6%	90.0%	62.5%	60.7%	reviewed/approved	11422	gpt-5.2-2025-12-11
7266	Κώμη	79	92	7.071	5.4%	36.4%	31.7%	31.6%	88.8%	65.4%	43.5%	reviewed/approved	11627	gpt-5.2-2025-12-11
2802	Κάδρεμα	24	23	-6.362	24.5%	53.6%	59.6%	66.7%	93.7%	79.6%	75.4%	reviewed/approved	11360	gpt-5.2-2025-12-11
2330	Καλύβη	24	23	-6.362	19.1%	46.7%	66.7%	68.1%	92.6%	73.6%	67.1%	reviewed/approved	11440	gpt-5.2-2025-12-11
2078	Καινύς	22	21	-6.342	60.1%	71.9%	76.6%	74.4%	95.4%	82.7%	74.7%	reviewed/approved	11375	gpt-5.2-2025-12-11
2061	Καδούσιοι	14	13	-6.259	54.2%	65.4%	77.2%	74.1%	95.5%	74.9%	59.2%	reviewed/approved	11357	gpt-5.2-2025-12-11
7253	Κυρτώνιος	18	29	5.700	7.8%	45.8%	49.6%	46.8%	91.4%	72.0%	73.2%	reviewed/approved	11588	gpt-5.2-2025-12-11
2335	Κάλυτις	26	37	5.617	3.9%	42.4%	43.2%	34.9%	91.1%	70.1%	62.4%	reviewed/approved	11455	gpt-5.2-2025-12-11
2484	Καρία	270	273	-4.899	18.4%	47.4%	42.6%	49.0%	88.4%	74.4%	50.2%	reviewed/approved	11482	gpt-5.2-2025-12-11
2345	Κάναθα	17	27	4.710	22.8%	57.0%	70.5%	59.1%	90.9%	74.1%	60.2%	reviewed/approved	11461	gpt-5.2-2025-12-11
2621	Κάστνιον	25	35	4.627	18.1%	49.0%	62.0%	53.3%	92.3%	59.8%	67.7%	reviewed/approved	11512	gpt-5.2-2025-12-11
2326	Καλλίαρος	47	57	4.401	28.4%	54.8%	59.9%	59.6%	93.5%	70.8%	63.9%	reviewed/approved	11428	gpt-5.2-2025-12-11
3258	Κεκρυφάλεια	26	27	-4.383	52.0%	74.7%	79.9%	79.2%	96.1%	72.3%	71.2%	reviewed/approved	11542	gpt-5.2-2025-12-11

Residual Predictors

Positive signed-residual terms are associated with AI translations longer than expected from the human length; negative signed-residual terms are associated with shorter AI translations. Absolute-residual models identify terms associated with larger AI-human length divergence.

Greek source terms predicting signed length residuals

Features: 176; ridge alpha: 4.125; cross-validated R^2: -0.242.

Positive term	Coefficient	Docs	Mean present	Mean absent
δια του	1.9363	10	4.004	-0.445
εν	1.7707	37	1.708	-1.003
ως	1.6397	52	0.672	-0.728
εν τη	1.4973	7	4.675	-0.352
του	1.4829	37	0.897	-0.527
δια	1.4712	16	2.324	-0.443
δευτερα	1.3494	3	8.937	-0.276
πολυβιος	1.2791	5	4.537	-0.239
την	1.2507	13	1.609	-0.240
γαρ	1.2228	9	1.386	-0.137
χωρας	1.1070	4	3.870	-0.161
παρα	1.1027	7	2.697	-0.203
κωμη	1.0882	3	5.579	-0.173
της χωρας	0.9983	3	4.610	-0.143
εν πρωτη	0.9599	3	3.679	-0.114
πρωτη	0.9599	3	3.679	-0.114
εξ	0.9181	3	7.370	-0.228
δευτερω	0.8942	3	4.060	-0.126
ος	0.8475	6	4.459	-0.285
πολιχνιον	0.8295	3	4.916	-0.152

Negative term	Coefficient	Docs	Mean present	Mean absent
καππαδοκιας	-1.6475	3	-8.039	0.249
εθνος	-1.4421	8	-3.807	0.331
και του	-1.4420	4	-7.610	0.317
λυκιας	-1.2513	3	-5.967	0.185
προς	-1.2383	15	-1.255	0.221
φησιν	-1.2015	10	-2.136	0.237
εστι και	-1.0796	20	-1.461	0.365
ουτως	-0.9812	5	-5.000	0.263
ως το	-0.9304	3	-8.101	0.251
μετα	-0.8708	3	-6.833	0.211
της	-0.8677	35	-0.123	0.066
πλησιον	-0.8533	8	-2.273	0.198
υπο	-0.8527	4	-1.867	0.078
αυτην	-0.8342	9	-2.100	0.208
των	-0.7959	17	-1.512	0.310
εισι	-0.7906	5	-3.870	0.204
δεκατω	-0.7800	3	-4.029	0.125
θρακης	-0.7795	5	-2.251	0.118
εκαλειτο	-0.7541	7	-3.147	0.237
θηλυκον και	-0.7311	3	-5.376	0.166

Greek source terms predicting large absolute residuals

Features: 176; ridge alpha: 11.938; cross-validated R^2: -0.047.

Positive term	Coefficient	Docs	Mean present	Mean absent
δε	0.6605	46	4.845	3.251
γαρ	0.5312	9	6.400	3.746
της	0.4548	35	5.472	3.183
την	0.4392	13	6.036	3.678
παρα	0.4099	7	7.316	3.734
τον	0.3992	10	6.325	3.724
κατα	0.3967	10	5.423	3.825
εκαταιος ευρωπη	0.3880	5	9.151	3.713
ευρωπη	0.3880	5	9.151	3.713
εν	0.3799	37	4.734	3.544
τριτη	0.3402	3	10.237	3.791
δυναται	0.3274	3	10.883	3.771
ποντου	0.2894	3	8.365	3.849
του ποντου	0.2894	3	8.365	3.849
δια του	0.2886	10	5.378	3.830
καππαδοκιας	0.2740	3	8.039	3.859
τα	0.2691	12	6.578	3.631
δευτερα	0.2685	3	8.937	3.831
εστι	0.2577	24	5.251	3.585
βοιωτιας	0.2510	3	8.876	3.833

Negative term	Coefficient	Docs	Mean present	Mean absent
εκ	-0.3339	6	1.758	4.127
ηροδοτος	-0.3286	9	2.838	4.098
θεοπομπος	-0.2822	5	2.986	4.037
πολις και	-0.2743	6	1.849	4.121
παιδος	-0.2572	4	1.410	4.092
εκαταιος ασια	-0.2349	5	2.219	4.077
ασια	-0.2349	5	2.219	4.077
απο	-0.1950	27	4.054	3.959
τοπος	-0.1941	4	1.706	4.079
τινες	-0.1896	6	2.412	4.085
θαλασση	-0.1888	3	1.371	4.065
οι	-0.1837	11	3.601	4.032
το εθνικον	-0.1658	56	3.839	4.169
εθνικον	-0.1658	56	3.839	4.169
οικητωρ	-0.1650	6	3.648	4.006
και το	-0.1641	9	3.601	4.022
διονυσιος	-0.1582	4	3.168	4.019
λιβυης	-0.1526	4	2.404	4.050
πλησιον	-0.1518	8	2.884	4.080
μια	-0.1486	3	2.035	4.045

AI English terms predicting large absolute residuals

Features: 404; ridge alpha: 17.013; cross-validated R^2: -0.051.

Positive term	Coefficient	Docs	Mean present	Mean absent
i	0.3448	12	6.302	3.668
with	0.3188	17	5.486	3.677
for	0.2846	11	5.216	3.832
the	0.2420	97	4.051	1.845
to the	0.2045	6	6.770	3.807
it	0.1963	51	4.590	3.354
on the	0.1960	9	6.230	3.762
or	0.1872	13	6.338	3.633
on	0.1817	16	4.846	3.820
formed	0.1793	4	8.966	3.777
but	0.1695	19	5.586	3.609
is a	0.1664	9	6.337	3.752
e	0.1631	10	4.915	3.881
cappadocia	0.1608	4	6.325	3.887
sicily	0.1579	3	9.293	3.820
a third	0.1569	3	10.237	3.791
according	0.1563	5	6.493	3.852
according to	0.1563	5	6.493	3.852
i e	0.1544	8	5.725	3.833
it is	0.1537	24	4.793	3.729

Negative term	Coefficient	Docs	Mean present	Mean absent
the ethnic gentilic	-0.2037	21	3.226	4.186
gentilic	-0.2037	21	3.226	4.186
ethnic gentilic	-0.2037	21	3.226	4.186
book of the	-0.1921	8	2.570	4.107
of the	-0.1812	47	3.614	4.313
gentilic is	-0.1609	17	3.156	4.154
ethnic gentilic is	-0.1609	17	3.156	4.154
also called	-0.1481	14	2.580	4.213
and	-0.1461	55	4.012	3.950
the ethnic	-0.1453	54	3.755	4.254
an	-0.1383	19	3.180	4.173
the ethnic name	-0.1382	5	1.332	4.124
ethnic name	-0.1382	5	1.332	4.124
ethnic	-0.1369	58	3.837	4.188
herodotus	-0.1268	9	2.838	4.098
named from	-0.1167	5	1.458	4.117
named	-0.1120	16	3.163	4.141
named after	-0.1075	5	2.008	4.088
its	-0.1064	9	3.308	4.051
in the feminine	-0.1042	4	1.851	4.073

Human English terms predicting large absolute residuals

Features: 335; ridge alpha: 10000.000; cross-validated R^2: -0.053.

Positive term	Coefficient	Docs	Mean present	Mean absent
form	0.0007	11	7.263	3.579
the	0.0006	90	4.178	2.244
with	0.0006	16	5.208	3.751
on	0.0005	16	5.133	3.766
the form	0.0004	9	7.556	3.631
is	0.0004	87	4.042	3.598
on the	0.0004	10	6.481	3.707
of	0.0004	66	4.301	3.371
has	0.0004	6	9.460	3.635
his europe	0.0004	5	9.151	3.713
in his europe	0.0004	5	9.151	3.713
a	0.0004	91	4.140	2.416
from	0.0004	37	4.597	3.625
as in	0.0004	29	4.431	3.802
in	0.0004	87	4.147	2.895
europe	0.0003	6	8.129	3.720
there	0.0003	34	5.181	3.368
one	0.0003	12	6.961	3.579
sicily	0.0003	3	9.293	3.820
is also a	0.0003	14	5.480	3.741

Negative term	Coefficient	Docs	Mean present	Mean absent
herodotos	-0.0003	9	2.838	4.098
the ethnonym is	-0.0002	52	3.522	4.486
ethnonym is	-0.0002	52	3.522	4.486
inhabitant is a	-0.0002	6	2.691	4.067
in his asia	-0.0002	5	2.219	4.077
his asia	-0.0002	5	2.219	4.077
libya	-0.0002	5	2.059	4.086
in libya	-0.0002	5	2.059	4.086
book the	-0.0002	10	2.955	4.099
theopompos in book	-0.0002	4	1.915	4.071
theopompos in	-0.0002	4	1.915	4.071
book the ethnonym	-0.0002	7	2.751	4.077
an	-0.0002	22	3.604	4.092
lydia	-0.0002	3	2.021	4.045
some	-0.0002	10	2.991	4.095
is the	-0.0002	3	0.905	4.080
inhabitant is	-0.0002	7	3.317	4.035
inhabitant	-0.0002	7	3.317	4.035
an inhabitant is	-0.0002	7	3.317	4.035
an inhabitant	-0.0002	7	3.317	4.035

Prompt Text

Show prompt text

You are an expert classical philologist and translator specializing in Byzantine Greek geographical texts.
You will receive Greek text from a lemma entry in Stephanos of Byzantium's Ethnika.
Translate the Greek text into clear, scholarly English.
Preserve technical terminology and place names appropriately.