gpt-4.1-2025-04-14 v1 Evaluation

Generated: 2026-09-20 11:19:57 UTC

Pairs100

Distinct lemmas100

Slope0.999

R^20.977

BLEU-419.6%

chrF++51.2%

METEOR57.0%

ROUGE-L58.7%

BERTScore92.3%

COMET72.7%

BLEURT65.2%

Profile version id: 2416

First translation using this version: 2026-07-16

Total usable AI runs for this version: 100 runs across 100 lemmas

Created: 2026-07-16 06:54:03.500756+10:00

Active: True

Notes: Model timeline evaluation: GPT-4.1 (2025-04-14) using prompt text from gpt-5.5 v1; approved-human-only for the 100-row Kappa corpus.

Length Regression

Formula	AI words = intercept + slope * human words
Slope	0.999
Intercept	4.633
R^2	0.977
P-value	6.28e-82
Pearson r	0.988
Mean source words	30.400
Length fallback rows	0
Mean human words	44.150
Mean AI words	48.730
Mean absolute length residual	4.561
Mean absolute percent error	15.6%

Metric vs Passage Length

These plots test whether score changes with source passage length for this prompt version. The dashed line is an ordinary least-squares fit; the annotation reports n, r, R^2, p-value, and slope.

BLEU-4 vs passage length — BLEU-4: positive; r = 0.063, R^2 = 0.004, p = 0.5363.

chrF++ vs passage length — chrF++: positive; r = 0.118, R^2 = 0.014, p = 0.2436.

METEOR vs passage length — METEOR: negative; r = -0.124, R^2 = 0.015, p = 0.2207.

ROUGE-L vs passage length — ROUGE-L: negative; r = -0.133, R^2 = 0.018, p = 0.1856.

BERTScore vs passage length — BERTScore: significant negative; r = -0.265, R^2 = 0.070, p = 0.0077.

COMET vs passage length — COMET: negative; r = -0.043, R^2 = 0.002, p = 0.6744.

BLEURT vs passage length — BLEURT: significant negative; r = -0.638, R^2 = 0.407, p = 9.53e-13.

Trigram precision vs passage length — Trigram precision: positive; r = 0.057, R^2 = 0.003, p = 0.5764.

Trigram recall vs passage length — Trigram recall: positive; r = 0.023, R^2 = 5.52e-04, p = 0.8165.

Trigram F1 vs passage length — Trigram F1: positive; r = 0.042, R^2 = 0.002, p = 0.6795.

Trigram Jaccard vs passage length — Trigram Jaccard: positive; r = 0.009, R^2 = 7.75e-05, p = 0.9307.

Metric	Rows	Pattern	Pearson r	R^2	P-value	Slope	Status
BLEU-4	100	positive	0.063	0.004	0.5363	0.00025	ok
chrF++	100	positive	0.118	0.014	0.2436	0.00037	ok
METEOR	100	negative	-0.124	0.015	0.2207	-0.00059	ok
ROUGE-L	100	negative	-0.133	0.018	0.1856	-0.00053	ok
BERTScore	100	significant negative	-0.265	0.070	0.0077	-0.00023	ok
COMET	100	negative	-0.043	0.002	0.6744	-0.00007	ok
BLEURT	100	significant negative	-0.638	0.407	9.53e-13	-0.00182	ok
Trigram precision	100	positive	0.057	0.003	0.5764	0.00025	ok
Trigram recall	100	positive	0.023	5.52e-04	0.8165	0.00011	ok
Trigram F1	100	positive	0.042	0.002	0.6795	0.00019	ok
Trigram Jaccard	100	positive	0.009	7.75e-05	0.9307	0.00003	ok

Translation Similarity Metrics

Mean BLEU-4	19.6%
Mean chrF++	51.2%
Mean METEOR	57.0%
Mean ROUGE-L	58.7%
Mean BERTScore	92.3%
Mean COMET	72.7%
Mean BLEURT	65.2%
Legacy smoothed corpus BLEU	24.4%
Unigram F1	62.5%
Bigram F1	32.6%
Trigram precision	17.7%
Trigram recall	19.7%
Trigram F1	18.7%
Trigram Jaccard	10.3%
4-gram F1	11.3%

Largest Length Residuals

Lemma ID	Headword	Human words	AI words	Residual	BLEU-4	chrF++	METEOR	ROUGE-L	BERTScore	COMET	BLEURT	Human	Run	Model
2484	Καρία	270	254	-20.310	18.0%	48.4%	40.6%	48.5%	87.9%	75.3%	43.6%	reviewed/approved	13281	gpt-4.1-2025-04-14
7264	Κωλιάς	61	82	16.440	9.1%	41.5%	44.5%	39.2%	90.4%	64.9%	54.2%	reviewed/approved	13422	gpt-4.1-2025-04-14
2468	Καπετώλιον	138	157	14.532	9.9%	40.0%	34.7%	41.4%	85.5%	68.7%	47.4%	reviewed/approved	13272	gpt-4.1-2025-04-14
3288	Κορώνεια	110	129	14.499	17.4%	50.5%	50.5%	54.2%	90.5%	72.3%	55.4%	reviewed/approved	13344	gpt-4.1-2025-04-14
2329	Κάλπη	57	76	14.435	20.4%	49.6%	51.6%	52.2%	89.7%	69.2%	62.1%	reviewed/approved	13236	gpt-4.1-2025-04-14
7266	Κώμη	79	96	12.462	10.0%	43.1%	38.9%	36.6%	88.6%	68.0%	48.4%	reviewed/approved	13428	gpt-4.1-2025-04-14
2115	Καλὴ ἀκτή	64	81	12.444	13.9%	40.7%	51.5%	49.7%	87.5%	61.9%	45.8%	reviewed/approved	13209	gpt-4.1-2025-04-14
2055	Καβασσός	109	102	-11.502	18.4%	51.3%	45.0%	46.4%	92.4%	74.6%	47.6%	reviewed/approved	13137	gpt-4.1-2025-04-14
2477	Καρδαμύλη	43	59	11.419	20.1%	58.8%	73.0%	56.9%	91.7%	74.8%	71.9%	reviewed/approved	13278	gpt-4.1-2025-04-14
2609	Κάσπειρος	135	129	-10.471	20.1%	55.0%	49.1%	56.8%	88.3%	71.5%	58.2%	reviewed/approved	13305	gpt-4.1-2025-04-14
2455	Κάναστρον	60	55	-9.561	14.9%	47.7%	52.6%	52.2%	91.5%	67.8%	58.7%	reviewed/approved	13266	gpt-4.1-2025-04-14
2604	Καρχηδών	117	131	9.507	24.9%	57.4%	49.5%	56.8%	91.3%	76.2%	53.5%	reviewed/approved	13296	gpt-4.1-2025-04-14
2465	Κάνωπος	83	96	8.466	32.5%	65.6%	80.3%	74.9%	93.5%	74.5%	63.8%	reviewed/approved	13269	gpt-4.1-2025-04-14
2346	Κάναι	77	90	8.459	31.2%	57.3%	73.2%	73.1%	92.1%	74.7%	60.0%	reviewed/approved	13263	gpt-4.1-2025-04-14
2605	Κάσιον	56	69	8.434	12.0%	52.0%	66.7%	57.6%	92.5%	72.1%	61.3%	reviewed/approved	13299	gpt-4.1-2025-04-14
2342	Κάμιρος	54	67	8.432	29.2%	63.4%	76.9%	69.4%	93.2%	76.1%	68.0%	reviewed/approved	13257	gpt-4.1-2025-04-14
7262	Κῶβρυς	22	35	8.394	6.7%	41.4%	54.8%	49.1%	90.3%	72.0%	66.3%	reviewed/approved	13416	gpt-4.1-2025-04-14
3496	Κοτιάειον	71	68	-7.548	17.3%	46.2%	56.8%	54.7%	92.6%	71.6%	65.3%	reviewed/approved	13347	gpt-4.1-2025-04-14
2597	Καρπασία	112	109	-7.499	19.9%	57.5%	56.5%	67.0%	93.6%	72.0%	59.2%	reviewed/approved	13284	gpt-4.1-2025-04-14
2328	Καλλίπολις	43	55	7.419	5.2%	46.0%	43.4%	53.1%	90.3%	71.3%	59.0%	reviewed/approved	13233	gpt-4.1-2025-04-14
7250	Κυρταία	40	52	7.415	16.6%	41.7%	41.7%	45.7%	89.1%	69.4%	64.5%	reviewed/approved	13380	gpt-4.1-2025-04-14
2085	Καλάσιρις	13	25	7.383	6.0%	43.0%	33.8%	36.8%	90.9%	66.5%	60.1%	reviewed/approved	13197	gpt-4.1-2025-04-14
2630	Κάτρη	23	21	-6.605	9.4%	42.6%	47.6%	59.1%	94.3%	78.5%	72.4%	reviewed/approved	13338	gpt-4.1-2025-04-14
2087	Καλαύρεια	30	28	-6.597	21.1%	54.5%	59.1%	65.5%	94.1%	76.4%	71.0%	reviewed/approved	13203	gpt-4.1-2025-04-14
2079	Καισάρεια	39	37	-6.586	38.5%	58.5%	67.8%	68.4%	92.5%	73.5%	64.9%	reviewed/approved	13179	gpt-4.1-2025-04-14
2623	Κασώριον	20	31	6.391	9.7%	48.2%	49.7%	43.1%	90.2%	63.4%	60.3%	reviewed/approved	13317	gpt-4.1-2025-04-14
2624	Κατάβαθμος	20	19	-5.609	35.9%	66.6%	64.2%	71.8%	94.4%	76.5%	69.0%	reviewed/approved	13320	gpt-4.1-2025-04-14
2330	Καλύβη	24	23	-5.604	19.9%	46.3%	58.6%	68.1%	92.8%	71.8%	70.3%	reviewed/approved	13239	gpt-4.1-2025-04-14
3530	Κριώα	26	25	-5.602	15.4%	37.6%	28.5%	47.1%	89.6%	62.8%	64.1%	reviewed/approved	13353	gpt-4.1-2025-04-14
7247	Κύρνος	50	49	-5.573	20.2%	49.6%	62.7%	58.6%	92.7%	71.8%	68.7%	reviewed/approved	13371	gpt-4.1-2025-04-14

Residual Predictors

Positive signed-residual terms are associated with AI translations longer than expected from the human length; negative signed-residual terms are associated with shorter AI translations. Absolute-residual models identify terms associated with larger AI-human length divergence.

Greek source terms predicting signed length residuals

Features: 176; ridge alpha: 8.377; cross-validated R^2: -0.384.

Positive term	Coefficient	Docs	Mean present	Mean absent
και	2.0976	66	0.631	-1.226
κωμη	1.1951	3	12.108	-0.374
εστι δε	1.0213	5	9.074	-0.478
δια του	1.0128	10	3.769	-0.419
εκ	0.9234	6	5.603	-0.358
εστι δε και	0.8479	3	11.808	-0.365
γαρ	0.8236	9	2.182	-0.216
θηλυκως	0.7037	5	5.042	-0.265
και θηλυκως	0.7037	5	5.042	-0.265
εν τη	0.6924	7	3.996	-0.301
παρα	0.6622	7	1.917	-0.144
δυο	0.6441	4	7.451	-0.310
εις	0.6362	7	3.661	-0.276
του δε	0.6362	3	5.760	-0.178
λεγεται	0.6292	9	2.592	-0.256
αιγυπτου	0.6210	3	5.754	-0.178
του	0.5965	37	1.609	-0.945
αι	0.5768	3	8.454	-0.261
την	0.5521	13	1.882	-0.281
χωρας	0.5500	4	3.648	-0.152

Negative term	Coefficient	Docs	Mean present	Mean absent
και του	-0.7332	4	-6.075	0.253
περι	-0.7218	11	-1.724	0.213
καππαδοκιας	-0.6788	3	-5.898	0.182
το	-0.6643	70	0.314	-0.732
το εθνικον	-0.6525	56	-0.290	0.369
εθνικον	-0.6525	56	-0.290	0.369
προς τη	-0.6042	4	-3.854	0.161
εκαλειτο	-0.5924	7	-4.925	0.371
ελλανικος	-0.5757	4	-9.972	0.415
εθνος	-0.5726	8	-3.428	0.298
τα	-0.5462	12	-1.007	0.137
αλλ	-0.5325	3	-5.528	0.171
το θηλυκον	-0.5003	6	-3.680	0.235
μετα	-0.4980	3	-10.767	0.333
μεταξυ	-0.4962	5	-1.566	0.082
τους	-0.4653	4	-2.579	0.107
τινες δε	-0.4560	3	-4.254	0.132
εκαταιος	-0.4417	14	-1.078	0.176
μια	-0.4374	3	-3.254	0.101
νησος	-0.4258	9	-1.352	0.134

Greek source terms predicting large absolute residuals

Features: 176; ridge alpha: 2.031; cross-validated R^2: 1.44e-04.

Positive term	Coefficient	Docs	Mean present	Mean absent
γαρ	4.3238	9	11.578	3.867
δε	2.7421	46	5.879	3.437
την	2.6947	13	7.727	4.087
και	2.5328	66	5.236	3.250
εστι	2.4730	24	6.800	3.853
κωμη	1.9785	3	12.108	4.327
τα	1.8046	12	9.080	3.944
εστι δε	1.5567	5	10.073	4.270
τε	1.5290	5	9.070	4.323
περι	1.4720	11	7.048	4.253
ως	1.4556	52	5.219	3.848
ακρα	1.3737	4	9.250	4.365
εν	1.3474	37	5.687	3.899
δια	1.3426	16	6.932	4.109
εστι δε και	1.3386	3	11.808	4.336
ηρωδιανος	1.3299	3	13.816	4.274
τα τοπικα	1.2457	3	11.181	4.356
τοπικα	1.2457	3	11.181	4.356
εστι και	1.1993	20	6.367	4.109
ομηρος	1.1966	3	10.451	4.378

Negative term	Coefficient	Docs	Mean present	Mean absent
στραβων	-1.8616	15	3.235	4.794
παυσανιας	-1.4972	4	1.781	4.676
τους	-1.3184	4	2.787	4.634
ης	-1.3042	3	1.510	4.655
ουδετερως	-1.1965	3	1.568	4.653
λιβυης	-1.0910	4	4.734	4.553
πολις προς	-1.0667	3	1.865	4.644
παιδος	-1.0595	4	3.171	4.618
πολυβιος	-1.0388	5	2.074	4.691
ην	-1.0112	13	3.967	4.649
εκαταιος	-0.9930	14	3.890	4.670
αυτην	-0.9738	9	3.445	4.671
φησι	-0.8933	12	4.828	4.524
το εθνικον	-0.8713	56	4.534	4.594
εθνικον	-0.8713	56	4.534	4.594
αλλη	-0.8466	4	4.266	4.573
ιβηριας	-0.8301	3	3.895	4.581
νησος	-0.8232	9	3.735	4.642
οτι	-0.7889	5	3.698	4.606
φησι το	-0.7775	3	0.853	4.675

AI English terms predicting large absolute residuals

Features: 438; ridge alpha: 1.425; cross-validated R^2: -0.039.

Positive term	Coefficient	Docs	Mean present	Mean absent
for	4.6389	15	8.724	3.826
is	2.5005	90	4.759	2.770
usage	2.3877	4	13.412	4.192
iota	2.1654	6	11.318	4.129
among	1.9105	6	10.613	4.174
promontory	1.8634	6	9.534	4.243
there is	1.8007	26	6.549	3.862
diphthong	1.7875	3	14.114	4.265
and	1.7848	54	5.758	3.155
or	1.7700	18	7.667	3.879
people	1.6849	8	8.372	4.229
from	1.6740	41	6.174	3.439
a promontory	1.6582	3	13.500	4.284
on	1.6397	16	7.183	4.061
with	1.6353	18	7.489	3.918
there is also	1.6227	22	6.145	4.114
thus	1.6040	4	13.922	4.171
also	1.5771	43	5.234	4.052
place	1.5642	10	7.401	4.245
derived	1.5474	5	12.036	4.167

Negative term	Coefficient	Docs	Mean present	Mean absent
name	-1.5554	20	4.451	4.588
strabo	-1.2439	15	3.235	4.794
pausanias	-1.2039	4	1.781	4.676
after	-1.1335	13	4.255	4.606
a city	-1.1158	66	4.453	4.770
strabo in the	-1.1066	5	1.496	4.722
strabo in	-1.0688	7	2.480	4.717
also a city	-1.0118	7	5.196	4.513
of the	-0.9750	44	4.597	4.532
ethnic designation is	-0.9641	30	4.285	4.679
designation is	-0.9641	30	4.285	4.679
formed	-0.9054	6	2.505	4.692
hecataeus mentions	-0.9053	5	1.728	4.710
they	-0.9019	8	3.499	4.653
arcadia	-0.8860	3	1.591	4.652
named after	-0.8717	10	3.204	4.711
city	-0.8717	69	4.484	4.730
hecataeus	-0.8658	14	3.890	4.670
the ethnicon is	-0.8611	9	2.689	4.746
the ethnicon	-0.8611	9	2.689	4.746

Human English terms predicting large absolute residuals

Features: 335; ridge alpha: 2.031; cross-validated R^2: -0.043.

Positive term	Coefficient	Docs	Mean present	Mean absent
with	2.7853	16	7.921	3.921
and	2.2735	52	5.847	3.167
is	2.2701	87	4.827	2.781
there is	1.9450	24	6.839	3.841
is also	1.8186	31	6.269	3.793
there is also	1.7752	22	6.779	3.935
on	1.6892	16	7.125	4.072
there	1.4636	34	6.384	3.621
is also a	1.4404	14	8.496	3.920
as	1.4326	60	5.213	3.581
or	1.4247	18	8.338	3.731
also a	1.4025	15	7.957	3.961
among	1.4018	4	11.651	4.265
the	1.3809	90	4.682	3.468
is from	1.2867	12	6.919	4.239
place	1.2803	6	7.839	4.351
long	1.2581	3	15.768	4.214
form	1.2579	11	8.767	4.041
among the	1.2294	4	11.651	4.265
with the	1.1794	5	11.652	4.187

Negative term	Coefficient	Docs	Mean present	Mean absent
strabo	-1.2381	15	3.235	4.794
was from	-1.2103	4	1.031	4.708
was	-0.9813	20	4.608	4.549
it is also	-0.9052	3	0.751	4.678
also called	-0.8538	6	3.265	4.643
the ethnonym	-0.8455	56	4.531	4.598
pausanias	-0.8095	4	1.781	4.676
of the	-0.7739	33	5.325	4.184
strabo in	-0.7620	3	0.848	4.675
ethnonym	-0.7410	57	4.654	4.437
pausanias book	-0.7265	3	1.507	4.655
is the	-0.7193	3	1.194	4.665
which	-0.6977	20	4.345	4.615
mount	-0.6948	4	3.280	4.614
strabo book the	-0.6557	3	0.518	4.686
they	-0.6535	12	4.601	4.555
book the ethnonym	-0.6392	7	2.807	4.693
the ethnonym is	-0.6387	52	4.690	4.420
ethnonym is	-0.6387	52	4.690	4.420
an island	-0.6382	9	3.413	4.674

Prompt Text

Show prompt text

You are an expert classical philologist and translator specializing in Byzantine Greek geographical texts.
You will receive Greek text from a lemma entry in Stephanos of Byzantium's Ethnika.
Translate the Greek text into clear, scholarly English.
Preserve technical terminology and place names appropriately.