gpt-4-turbo-2024-04-09 v1 Evaluation

Generated: 2026-07-25 12:52:59 UTC

Pairs100

Distinct lemmas100

Slope0.898

R^20.964

BLEU-417.5%

chrF++47.9%

METEOR53.0%

ROUGE-L53.8%

BERTScoreN/A

COMETN/A

BLEURTN/A

Profile version id: 2404

First translation using this version: 2026-07-16

Total usable AI runs for this version: 100 runs across 100 lemmas

Created: 2026-07-16 06:54:03.500756+10:00

Active: True

Notes: Model timeline evaluation: GPT-4 Turbo (2024-04-09) using prompt text from gpt-5.5 v1; approved-human-only for the 100-row Kappa corpus.

Length Regression

Formula	AI words = intercept + slope * human words
Slope	0.898
Intercept	6.911
R^2	0.964
P-value	1.04e-72
Pearson r	0.982
Mean source words	30.400
Length fallback rows	0
Mean human words	44.160
Mean AI words	46.580
Mean absolute length residual	4.561
Mean absolute percent error	15.2%

Metric vs Passage Length

These plots test whether score changes with source passage length for this prompt version. The dashed line is an ordinary least-squares fit; the annotation reports n, r, R^2, p-value, and slope.

BLEU-4 vs passage length — BLEU-4: positive; r = 0.046, R^2 = 0.002, p = 0.6470.

chrF++ vs passage length — chrF++: positive; r = 0.066, R^2 = 0.004, p = 0.5139.

METEOR vs passage length — METEOR: negative; r = -0.178, R^2 = 0.032, p = 0.0759.

ROUGE-L vs passage length — ROUGE-L: negative; r = -0.103, R^2 = 0.011, p = 0.3058.

Trigram precision vs passage length — Trigram precision: negative; r = -0.012, R^2 = 1.45e-04, p = 0.9054.

Trigram recall vs passage length — Trigram recall: negative; r = -0.073, R^2 = 0.005, p = 0.4724.

Trigram F1 vs passage length — Trigram F1: negative; r = -0.041, R^2 = 0.002, p = 0.6861.

Trigram Jaccard vs passage length — Trigram Jaccard: negative; r = -0.059, R^2 = 0.003, p = 0.5623.

Metric	Rows	Pattern	Pearson r	R^2	P-value	Slope	Status
BLEU-4	100	positive	0.046	0.002	0.6470	0.00018	ok
chrF++	100	positive	0.066	0.004	0.5139	0.00020	ok
METEOR	100	negative	-0.178	0.032	0.0759	-0.00093	ok
ROUGE-L	100	negative	-0.103	0.011	0.3058	-0.00041	ok
BERTScore	0	not enough data	N/A	N/A	N/A	N/A	needs at least two rows with metric scores
COMET	0	not enough data	N/A	N/A	N/A	N/A	needs at least two rows with metric scores
BLEURT	0	not enough data	N/A	N/A	N/A	N/A	needs at least two rows with metric scores
Trigram precision	100	negative	-0.012	1.45e-04	0.9054	-0.00005	ok
Trigram recall	100	negative	-0.073	0.005	0.4724	-0.00034	ok
Trigram F1	100	negative	-0.041	0.002	0.6861	-0.00018	ok
Trigram Jaccard	100	negative	-0.059	0.003	0.5623	-0.00017	ok

Translation Similarity Metrics

Mean BLEU-4	17.5%
Mean chrF++	47.9%
Mean METEOR	53.0%
Mean ROUGE-L	53.8%
Mean BERTScore	N/A
Mean COMET	N/A
Mean BLEURT	N/A
Legacy smoothed corpus BLEU	21.9%
Unigram F1	58.9%
Bigram F1	28.2%
Trigram precision	15.6%
Trigram recall	16.5%
Trigram F1	16.0%
Trigram Jaccard	8.7%
4-gram F1	9.6%

Largest Length Residuals

Lemma ID	Headword	Human words	AI words	Residual	BLEU-4	chrF++	METEOR	ROUGE-L	BERTScore	COMET	BLEURT	Human	Run	Model
2484	Καρία	270	221	-28.455	13.6%	41.5%	31.2%	41.1%	N/A	N/A	N/A	reviewed/approved	12381	gpt-4-turbo-2024-04-09
2119	Κάλλατις	65	91	25.699	12.6%	47.6%	58.1%	48.7%	N/A	N/A	N/A	reviewed/approved	12321	gpt-4-turbo-2024-04-09
2597	Καρπασία	112	126	18.478	7.5%	48.5%	42.7%	47.1%	N/A	N/A	N/A	reviewed/approved	12384	gpt-4-turbo-2024-04-09
2342	Κάμιρος	54	73	17.581	10.8%	51.4%	54.4%	48.8%	N/A	N/A	N/A	reviewed/approved	12357	gpt-4-turbo-2024-04-09
7267	Κωνώπη	64	81	16.597	21.7%	45.1%	44.5%	44.1%	N/A	N/A	N/A	reviewed/approved	12531	gpt-4-turbo-2024-04-09
3288	Κορώνεια	110	122	16.275	14.0%	48.9%	43.6%	49.8%	N/A	N/A	N/A	reviewed/approved	12444	gpt-4-turbo-2024-04-09
2115	Καλὴ ἀκτή	64	78	13.597	7.0%	43.7%	45.6%	49.3%	N/A	N/A	N/A	reviewed/approved	12309	gpt-4-turbo-2024-04-09
7266	Κώμη	79	90	12.123	8.4%	34.8%	27.9%	29.6%	N/A	N/A	N/A	reviewed/approved	12528	gpt-4-turbo-2024-04-09
7254	Κύτα	84	94	11.631	31.8%	59.9%	66.8%	61.8%	N/A	N/A	N/A	reviewed/approved	12492	gpt-4-turbo-2024-04-09
2346	Κάναι	77	87	10.919	33.6%	56.6%	75.3%	71.5%	N/A	N/A	N/A	reviewed/approved	12363	gpt-4-turbo-2024-04-09
7255	Κυτέριον	22	17	-9.673	10.0%	30.6%	38.5%	46.2%	N/A	N/A	N/A	reviewed/approved	12495	gpt-4-turbo-2024-04-09
2604	Καρχηδών	117	121	8.987	18.1%	48.4%	46.4%	46.0%	N/A	N/A	N/A	reviewed/approved	12396	gpt-4-turbo-2024-04-09
2116	Κάληρος	40	34	-8.843	15.0%	44.4%	43.6%	37.8%	N/A	N/A	N/A	reviewed/approved	12312	gpt-4-turbo-2024-04-09
2055	Καβασσός	109	96	-8.827	13.0%	44.6%	34.3%	42.0%	N/A	N/A	N/A	reviewed/approved	12237	gpt-4-turbo-2024-04-09
2054	Καβαλίς	65	74	8.699	19.5%	50.9%	47.6%	59.0%	N/A	N/A	N/A	reviewed/approved	12234	gpt-4-turbo-2024-04-09
2599	Καρπήσιοι	11	9	-7.792	10.3%	36.8%	34.7%	40.0%	N/A	N/A	N/A	reviewed/approved	12387	gpt-4-turbo-2024-04-09
3496	Κοτιάειον	71	63	-7.691	7.4%	36.5%	43.6%	44.8%	N/A	N/A	N/A	reviewed/approved	12447	gpt-4-turbo-2024-04-09
2602	Καρύανδα	34	30	-7.453	21.3%	52.9%	61.4%	56.2%	N/A	N/A	N/A	reviewed/approved	12390	gpt-4-turbo-2024-04-09
2620	Κάσταξ	20	32	7.123	13.0%	45.2%	45.2%	50.0%	N/A	N/A	N/A	reviewed/approved	12408	gpt-4-turbo-2024-04-09
2621	Κάστνιον	25	36	6.632	25.1%	54.9%	67.5%	59.0%	N/A	N/A	N/A	reviewed/approved	12411	gpt-4-turbo-2024-04-09
7258	Κύτωρος	30	40	6.140	17.6%	57.5%	65.6%	65.7%	N/A	N/A	N/A	reviewed/approved	12504	gpt-4-turbo-2024-04-09
2076	Καινοί	11	11	-5.792	7.0%	43.6%	46.5%	54.5%	N/A	N/A	N/A	reviewed/approved	12270	gpt-4-turbo-2024-04-09
2084	Κάλαρνα	14	14	-5.487	8.2%	42.2%	30.5%	42.9%	N/A	N/A	N/A	reviewed/approved	12294	gpt-4-turbo-2024-04-09
3258	Κεκρυφάλεια	26	25	-5.267	45.3%	68.9%	72.2%	74.5%	N/A	N/A	N/A	reviewed/approved	12441	gpt-4-turbo-2024-04-09
2607	Κάσος	57	53	-5.114	30.6%	57.2%	58.8%	65.5%	N/A	N/A	N/A	reviewed/approved	12402	gpt-4-turbo-2024-04-09
7253	Κυρτώνιος	18	28	4.920	9.9%	35.1%	36.2%	30.4%	N/A	N/A	N/A	reviewed/approved	12489	gpt-4-turbo-2024-04-09
2468	Καπετώλιον	138	126	-4.878	7.2%	34.5%	28.2%	35.6%	N/A	N/A	N/A	reviewed/approved	12372	gpt-4-turbo-2024-04-09
2332	Καλυδών	20	20	-4.877	12.4%	41.6%	41.1%	45.0%	N/A	N/A	N/A	reviewed/approved	12345	gpt-4-turbo-2024-04-09
2074	Καικῖνον	11	12	-4.792	6.7%	37.5%	46.0%	52.2%	N/A	N/A	N/A	reviewed/approved	12264	gpt-4-turbo-2024-04-09
2078	Καινύς	22	22	-4.673	52.1%	63.4%	67.9%	68.2%	N/A	N/A	N/A	reviewed/approved	12273	gpt-4-turbo-2024-04-09

Residual Predictors

Positive signed-residual terms are associated with AI translations longer than expected from the human length; negative signed-residual terms are associated with shorter AI translations. Absolute-residual models identify terms associated with larger AI-human length divergence.

Greek source terms predicting signed length residuals

Features: 176; ridge alpha: 5.878; cross-validated R^2: -0.115.

Positive term	Coefficient	Docs	Mean present	Mean absent
και	2.4082	66	0.569	-1.105
ου	2.0416	14	6.159	-1.003
απο του	1.8060	9	8.936	-0.884
τοις	1.4197	3	15.174	-0.469
θηλυκως	1.3987	5	8.855	-0.466
και θηλυκως	1.3987	5	8.855	-0.466
πολιτης	1.3437	18	5.690	-1.249
κωμη	1.2505	3	9.061	-0.280
παρα	1.1851	7	2.404	-0.181
την	1.1824	13	3.081	-0.460
αφ ου	1.1700	5	8.865	-0.467
πολιχνιον	1.1345	3	11.360	-0.351
ως	1.0524	52	0.596	-0.646
εστι δε	1.0523	5	7.160	-0.377
απο	1.0365	27	2.371	-0.877
ει	0.9742	4	3.139	-0.131
αφ	0.9678	6	7.408	-0.473
τους	0.9213	4	2.691	-0.112
της	0.8720	35	0.370	-0.199
τριτω	0.8694	3	8.185	-0.253

Negative term	Coefficient	Docs	Mean present	Mean absent
εθνος	-1.4178	8	-5.072	0.441
το	-1.2618	70	0.120	-0.279
περι	-1.2475	11	-3.692	0.456
τω	-1.1462	16	-3.003	0.572
το εθνικον	-1.1358	56	-1.237	1.575
εθνικον	-1.1358	56	-1.237	1.575
εκαλειτο	-1.0797	7	-5.890	0.443
τινες	-0.9852	6	-4.038	0.258
τω περι	-0.9741	4	-9.593	0.400
εν τω περι	-0.9741	4	-9.593	0.400
εν τω	-0.9371	6	-6.027	0.385
ουτως	-0.8859	5	-8.440	0.444
μεν	-0.8019	4	-9.649	0.402
καππαδοκιας	-0.7977	3	-4.080	0.126
ηρωδιανος	-0.7728	3	-12.635	0.391
τινες δε	-0.7142	3	-5.346	0.165
εκαταιος	-0.6886	14	-0.705	0.115
δια	-0.6315	16	1.395	-0.266
δε απο	-0.5982	3	-4.007	0.124
γαρ	-0.5823	9	-0.646	0.064

Greek source terms predicting large absolute residuals

Features: 176; ridge alpha: 2.894; cross-validated R^2: -0.082.

Positive term	Coefficient	Docs	Mean present	Mean absent
την	3.5272	13	9.927	3.759
απο του	2.9187	9	12.493	3.776
απο	2.7261	27	8.043	3.273
ει	2.0743	4	17.367	4.027
απο της	1.8788	6	12.157	4.076
παρα	1.8285	7	11.825	4.014
γαρ	1.7832	9	9.810	4.042
εστι	1.7495	24	6.813	3.850
περι	1.6653	11	8.365	4.091
τοις	1.6363	3	15.174	4.233
και	1.4865	66	4.969	3.769
πολιχνιον	1.4245	3	13.052	4.298
ου	1.3329	14	7.941	4.011
δυναται	1.3084	3	11.592	4.343
εκαλειτο	1.2778	7	9.567	4.184
δε	1.2608	46	5.335	3.902
φησι	1.2201	12	8.756	3.989
εν	1.1150	37	5.952	3.744
εστι και	1.0876	20	6.747	4.014
αφ ου	1.0832	5	10.674	4.239

Negative term	Coefficient	Docs	Mean present	Mean absent
και το	-1.2435	9	3.688	4.647
ουδετερως	-1.1554	3	1.281	4.662
εκληθη	-1.1225	4	1.684	4.681
ης	-1.0195	3	1.770	4.647
πολις προς	-1.0153	3	1.803	4.646
τοπος	-1.0025	4	1.985	4.668
προς τη	-0.9541	4	1.695	4.680
του δε	-0.9093	3	2.050	4.639
ποσειδωνος	-0.8406	3	2.346	4.629
λιβυης	-0.8218	4	3.587	4.602
χωρα	-0.8030	6	5.889	4.476
μια	-0.7539	3	3.080	4.607
εκληθη δε	-0.7378	3	1.676	4.650
ως στραβων	-0.7323	4	3.099	4.622
και του	-0.6948	4	3.566	4.602
τον	-0.6935	10	4.700	4.545
θρακης	-0.6779	5	2.291	4.680
ηροδοτος	-0.6750	9	5.172	4.500
παυσανιας	-0.6730	4	2.955	4.628
πορρω	-0.6729	3	1.657	4.651

AI English terms predicting large absolute residuals

Features: 404; ridge alpha: 5.878; cross-validated R^2: -0.142.

Positive term	Coefficient	Docs	Mean present	Mean absent
from	1.5624	37	6.553	3.391
the	1.4330	97	4.630	2.326
said	1.0556	8	11.667	3.943
there	0.9564	34	6.733	3.442
they	0.9514	9	9.004	4.121
diphthong	0.9505	3	17.270	4.168
in the	0.8745	30	6.052	3.922
on the	0.7837	8	10.252	4.066
caria	0.7739	3	10.536	4.376
be	0.7458	4	13.689	4.181
the feminine	0.7433	11	8.981	4.015
those	0.7427	7	13.679	3.875
are	0.7402	32	6.652	3.577
it	0.7318	45	5.941	3.432
there is also	0.7206	22	6.660	3.969
there is	0.7141	24	6.451	3.964
is also	0.6991	29	6.349	3.831
in	0.6820	81	4.764	3.697
is said	0.6796	5	12.048	4.167
or	0.6787	16	7.693	3.964

Negative term	Coefficient	Docs	Mean present	Mean absent
name	-0.7588	18	2.309	5.055
region	-0.6062	6	1.237	4.773
the inhabitant is	-0.5802	7	1.464	4.794
inhabitant is	-0.5802	7	1.464	4.794
and the	-0.5226	16	4.500	4.573
the name	-0.5183	7	1.771	4.771
his	-0.5093	28	3.947	4.800
inhabitant is called	-0.4623	5	1.224	4.737
known as	-0.4147	4	1.655	4.682
of his	-0.4045	6	1.944	4.728
form is	-0.3918	11	3.357	4.710
region is	-0.3897	3	0.835	4.676
is also called	-0.3894	4	1.928	4.671
so	-0.3790	5	2.177	4.686
which the	-0.3609	4	1.964	4.669
the inhabitant	-0.3603	8	4.838	4.537
inhabitant	-0.3603	8	4.838	4.537
the region	-0.3593	3	0.730	4.679
the ethnic adjective	-0.3543	7	2.574	4.710
ethnic adjective	-0.3543	7	2.574	4.710

Human English terms predicting large absolute residuals

Features: 336; ridge alpha: 5.878; cross-validated R^2: -0.136.

Positive term	Coefficient	Docs	Mean present	Mean absent
like	1.0723	5	8.894	4.333
they	1.0206	12	7.299	4.188
on	0.9881	16	8.362	3.837
also a	0.9427	15	8.893	3.796
from	0.9341	37	6.343	3.514
there	0.9109	34	7.268	3.166
is also a	0.8786	14	9.209	3.804
genitive	0.8767	3	20.951	4.054
from the	0.8684	13	9.947	3.756
city on	0.8634	3	14.812	4.244
who	0.8414	13	8.554	3.964
and	0.8197	52	5.493	3.551
diphthong	0.7889	3	17.270	4.168
there is	0.7303	24	6.582	3.923
the	0.7114	90	4.778	2.609
however	0.7067	10	10.106	3.945
that	0.7012	29	6.948	3.586
be	0.6810	15	8.163	3.925
feminine	0.6767	16	7.723	3.959
is with	0.6744	4	11.350	4.278

Negative term	Coefficient	Docs	Mean present	Mean absent
near	-0.5764	16	3.464	4.770
the ethnonym is	-0.5404	52	3.914	5.262
ethnonym is	-0.5404	52	3.914	5.262
the ethnonym	-0.4944	56	3.985	5.294
ethnonym	-0.4635	57	4.070	5.212
herodotos	-0.4185	9	5.172	4.500
neuter	-0.4131	4	1.138	4.704
one of	-0.3917	5	2.413	4.674
just as	-0.3902	5	2.023	4.695
libya	-0.3776	5	3.045	4.641
in libya	-0.3776	5	3.045	4.641
asia	-0.3550	6	3.161	4.650
in his asia	-0.3496	5	2.889	4.649
his asia	-0.3496	5	2.889	4.649
an island	-0.3448	9	3.057	4.710
hekataios in	-0.3322	12	4.099	4.624
thrace	-0.3282	5	2.291	4.680
island	-0.3234	14	4.033	4.647
the city	-0.3229	6	3.432	4.633
one	-0.3186	12	4.195	4.611

Prompt Text

Show prompt text

You are an expert classical philologist and translator specializing in Byzantine Greek geographical texts.
You will receive Greek text from a lemma entry in Stephanos of Byzantium's Ethnika.
Translate the Greek text into clear, scholarly English.
Preserve technical terminology and place names appropriately.