gpt-4-turbo-2024-04-09 v1 Evaluation

Generated: 2026-09-20 11:20:47 UTC

Pairs100

Distinct lemmas100

Slope0.898

R^20.964

BLEU-417.5%

chrF++47.9%

METEOR53.0%

ROUGE-L53.8%

BERTScore91.6%

COMET71.1%

BLEURT62.3%

Profile version id: 2404

First translation using this version: 2026-07-16

Total usable AI runs for this version: 100 runs across 100 lemmas

Created: 2026-07-16 06:54:03.500756+10:00

Active: True

Notes: Model timeline evaluation: GPT-4 Turbo (2024-04-09) using prompt text from gpt-5.5 v1; approved-human-only for the 100-row Kappa corpus.

Length Regression

Formula	AI words = intercept + slope * human words
Slope	0.898
Intercept	6.926
R^2	0.964
P-value	9.06e-73
Pearson r	0.982
Mean source words	30.400
Length fallback rows	0
Mean human words	44.150
Mean AI words	46.580
Mean absolute length residual	4.555
Mean absolute percent error	15.1%

Metric vs Passage Length

These plots test whether score changes with source passage length for this prompt version. The dashed line is an ordinary least-squares fit; the annotation reports n, r, R^2, p-value, and slope.

BLEU-4 vs passage length — BLEU-4: positive; r = 0.047, R^2 = 0.002, p = 0.6420.

chrF++ vs passage length — chrF++: positive; r = 0.065, R^2 = 0.004, p = 0.5211.

METEOR vs passage length — METEOR: negative; r = -0.180, R^2 = 0.032, p = 0.0737.

ROUGE-L vs passage length — ROUGE-L: negative; r = -0.104, R^2 = 0.011, p = 0.3022.

BERTScore vs passage length — BERTScore: significant negative; r = -0.236, R^2 = 0.056, p = 0.0179.

COMET vs passage length — COMET: negative; r = -0.135, R^2 = 0.018, p = 0.1807.

BLEURT vs passage length — BLEURT: significant negative; r = -0.690, R^2 = 0.476, p = 2.04e-15.

Trigram precision vs passage length — Trigram precision: negative; r = -0.012, R^2 = 1.45e-04, p = 0.9054.

Trigram recall vs passage length — Trigram recall: negative; r = -0.073, R^2 = 0.005, p = 0.4724.

Trigram F1 vs passage length — Trigram F1: negative; r = -0.041, R^2 = 0.002, p = 0.6861.

Trigram Jaccard vs passage length — Trigram Jaccard: negative; r = -0.059, R^2 = 0.003, p = 0.5623.

Metric	Rows	Pattern	Pearson r	R^2	P-value	Slope	Status
BLEU-4	100	positive	0.047	0.002	0.6420	0.00018	ok
chrF++	100	positive	0.065	0.004	0.5211	0.00020	ok
METEOR	100	negative	-0.180	0.032	0.0737	-0.00093	ok
ROUGE-L	100	negative	-0.104	0.011	0.3022	-0.00041	ok
BERTScore	100	significant negative	-0.236	0.056	0.0179	-0.00019	ok
COMET	100	negative	-0.135	0.018	0.1807	-0.00023	ok
BLEURT	100	significant negative	-0.690	0.476	2.04e-15	-0.00207	ok
Trigram precision	100	negative	-0.012	1.45e-04	0.9054	-0.00005	ok
Trigram recall	100	negative	-0.073	0.005	0.4724	-0.00034	ok
Trigram F1	100	negative	-0.041	0.002	0.6861	-0.00018	ok
Trigram Jaccard	100	negative	-0.059	0.003	0.5623	-0.00017	ok

Translation Similarity Metrics

Mean BLEU-4	17.5%
Mean chrF++	47.9%
Mean METEOR	53.0%
Mean ROUGE-L	53.8%
Mean BERTScore	91.6%
Mean COMET	71.1%
Mean BLEURT	62.3%
Legacy smoothed corpus BLEU	21.9%
Unigram F1	58.9%
Bigram F1	28.3%
Trigram precision	15.6%
Trigram recall	16.5%
Trigram F1	16.0%
Trigram Jaccard	8.7%
4-gram F1	9.6%

Largest Length Residuals

Lemma ID	Headword	Human words	AI words	Residual	BLEU-4	chrF++	METEOR	ROUGE-L	BERTScore	COMET	BLEURT	Human	Run	Model
2484	Καρία	270	221	-28.431	13.6%	41.5%	31.2%	41.1%	87.0%	68.9%	32.6%	reviewed/approved	12381	gpt-4-turbo-2024-04-09
2119	Κάλλατις	65	91	25.693	12.6%	47.6%	58.1%	48.7%	91.2%	66.6%	60.8%	reviewed/approved	12321	gpt-4-turbo-2024-04-09
2597	Καρπασία	112	126	18.480	7.5%	48.5%	42.7%	47.1%	91.5%	71.7%	55.5%	reviewed/approved	12384	gpt-4-turbo-2024-04-09
2342	Κάμιρος	54	73	17.573	10.8%	51.4%	54.4%	48.8%	90.7%	71.7%	54.6%	reviewed/approved	12357	gpt-4-turbo-2024-04-09
7267	Κωνώπη	64	81	16.591	21.7%	45.1%	44.5%	44.1%	89.7%	66.9%	60.9%	reviewed/approved	12531	gpt-4-turbo-2024-04-09
3288	Κορώνεια	110	122	16.276	14.0%	48.9%	43.6%	49.8%	90.3%	69.1%	50.9%	reviewed/approved	12444	gpt-4-turbo-2024-04-09
2115	Καλὴ ἀκτή	64	78	13.591	7.0%	43.7%	45.6%	49.3%	88.7%	66.4%	52.1%	reviewed/approved	12309	gpt-4-turbo-2024-04-09
7266	Κώμη	79	90	12.119	8.4%	34.8%	27.9%	29.6%	87.2%	63.9%	38.9%	reviewed/approved	12528	gpt-4-turbo-2024-04-09
7254	Κύτα	84	94	11.628	32.6%	60.7%	68.0%	62.9%	91.2%	74.8%	57.4%	reviewed/approved	12492	gpt-4-turbo-2024-04-09
2346	Κάναι	77	87	10.915	33.6%	56.6%	75.3%	71.5%	92.6%	74.2%	56.8%	reviewed/approved	12363	gpt-4-turbo-2024-04-09
7255	Κυτέριον	22	17	-9.686	10.0%	30.6%	38.5%	46.2%	87.7%	58.4%	46.3%	reviewed/approved	12495	gpt-4-turbo-2024-04-09
2604	Καρχηδών	117	121	8.989	18.1%	48.4%	46.4%	46.0%	90.5%	73.1%	48.9%	reviewed/approved	12396	gpt-4-turbo-2024-04-09
2116	Κάληρος	40	34	-8.853	15.0%	44.4%	43.6%	37.8%	90.0%	66.5%	53.9%	reviewed/approved	12312	gpt-4-turbo-2024-04-09
2055	Καβασσός	109	96	-8.826	13.0%	44.6%	34.3%	42.0%	90.8%	72.9%	47.8%	reviewed/approved	12237	gpt-4-turbo-2024-04-09
2054	Καβαλίς	65	74	8.693	19.5%	50.9%	47.6%	59.0%	90.3%	70.1%	56.7%	reviewed/approved	12234	gpt-4-turbo-2024-04-09
3496	Κοτιάειον	71	63	-7.696	7.4%	36.5%	43.6%	44.8%	89.5%	64.8%	52.6%	reviewed/approved	12447	gpt-4-turbo-2024-04-09
2602	Καρύανδα	34	30	-7.464	21.3%	52.9%	61.4%	56.2%	94.1%	72.2%	70.3%	reviewed/approved	12390	gpt-4-turbo-2024-04-09
2620	Κάσταξ	20	32	7.111	13.0%	45.2%	45.2%	50.0%	90.3%	67.4%	62.0%	reviewed/approved	12408	gpt-4-turbo-2024-04-09
2599	Καρπήσιοι	10	9	-6.908	10.3%	39.0%	37.9%	42.1%	94.1%	71.6%	76.8%	reviewed/approved	12387	gpt-4-turbo-2024-04-09
2621	Κάστνιον	25	36	6.620	25.1%	54.9%	67.5%	59.0%	92.7%	61.6%	69.2%	reviewed/approved	12411	gpt-4-turbo-2024-04-09
7258	Κύτωρος	30	40	6.129	17.6%	57.5%	65.6%	65.7%	94.7%	74.3%	74.9%	reviewed/approved	12504	gpt-4-turbo-2024-04-09
2076	Καινοί	11	11	-5.806	7.0%	43.6%	46.5%	54.5%	93.4%	74.2%	69.9%	reviewed/approved	12270	gpt-4-turbo-2024-04-09
2084	Κάλαρνα	14	14	-5.500	8.2%	42.2%	30.5%	42.9%	90.0%	73.0%	66.2%	reviewed/approved	12294	gpt-4-turbo-2024-04-09
3258	Κεκρυφάλεια	26	25	-5.278	45.3%	68.9%	72.2%	74.5%	94.7%	75.1%	72.8%	reviewed/approved	12441	gpt-4-turbo-2024-04-09
2607	Κάσος	57	53	-5.121	30.6%	57.2%	58.8%	65.5%	93.8%	74.0%	64.9%	reviewed/approved	12402	gpt-4-turbo-2024-04-09
7253	Κυρτώνιος	18	28	4.907	9.9%	35.1%	36.2%	30.4%	89.1%	69.1%	65.3%	reviewed/approved	12489	gpt-4-turbo-2024-04-09
2332	Καλυδών	20	20	-4.889	12.4%	41.6%	41.1%	45.0%	89.2%	74.2%	61.0%	reviewed/approved	12345	gpt-4-turbo-2024-04-09
2468	Καπετώλιον	138	126	-4.873	7.2%	34.5%	28.2%	35.6%	83.8%	62.5%	42.1%	reviewed/approved	12372	gpt-4-turbo-2024-04-09
2074	Καικῖνον	11	12	-4.806	6.7%	37.5%	46.0%	52.2%	88.5%	65.0%	48.5%	reviewed/approved	12264	gpt-4-turbo-2024-04-09
2078	Καινύς	22	22	-4.686	52.1%	63.4%	67.9%	68.2%	92.5%	80.0%	67.8%	reviewed/approved	12273	gpt-4-turbo-2024-04-09

Residual Predictors

Positive signed-residual terms are associated with AI translations longer than expected from the human length; negative signed-residual terms are associated with shorter AI translations. Absolute-residual models identify terms associated with larger AI-human length divergence.

Greek source terms predicting signed length residuals

Features: 176; ridge alpha: 5.878; cross-validated R^2: -0.117.

Positive term	Coefficient	Docs	Mean present	Mean absent
και	2.3950	66	0.562	-1.091
ου	2.0390	14	6.155	-1.002
απο του	1.8052	9	8.932	-0.883
τοις	1.4193	3	15.168	-0.469
θηλυκως	1.3975	5	8.849	-0.466
και θηλυκως	1.3975	5	8.849	-0.466
πολιτης	1.3385	18	5.683	-1.247
κωμη	1.2495	3	9.054	-0.280
παρα	1.1844	7	2.403	-0.181
την	1.1830	13	3.081	-0.460
αφ ου	1.1702	5	8.865	-0.467
πολιχνιον	1.1336	3	11.354	-0.351
εστι δε	1.0515	5	7.158	-0.377
ως	1.0427	52	0.588	-0.637
απο	1.0347	27	2.367	-0.875
ει	0.9745	4	3.141	-0.131
αφ	0.9674	6	7.406	-0.473
τους	0.9191	4	2.682	-0.112
τριτω	0.8687	3	8.178	-0.253
της	0.8659	35	0.364	-0.196

Negative term	Coefficient	Docs	Mean present	Mean absent
εθνος	-1.3560	8	-4.965	0.432
το	-1.2718	70	0.112	-0.260
περι	-1.2492	11	-3.697	0.457
τω	-1.1498	16	-3.009	0.573
το εθνικον	-1.1460	56	-1.246	1.586
εθνικον	-1.1460	56	-1.246	1.586
εκαλειτο	-1.0812	7	-5.888	0.443
τινες	-0.9871	6	-4.047	0.258
τω περι	-0.9747	4	-9.594	0.400
εν τω περι	-0.9747	4	-9.594	0.400
εν τω	-0.9381	6	-6.028	0.385
ουτως	-0.8859	5	-8.437	0.444
μεν	-0.8019	4	-9.646	0.402
καππαδοκιας	-0.7990	3	-4.087	0.126
ηρωδιανος	-0.7728	3	-12.629	0.391
τινες δε	-0.7157	3	-5.356	0.166
εκαταιος	-0.6953	14	-0.714	0.116
δια	-0.6326	16	1.392	-0.265
δε απο	-0.5985	3	-4.013	0.124
γαρ	-0.5810	9	-0.643	0.064

Greek source terms predicting large absolute residuals

Features: 176; ridge alpha: 2.894; cross-validated R^2: -0.078.

Positive term	Coefficient	Docs	Mean present	Mean absent
την	3.5288	13	9.927	3.752
απο του	2.9199	9	12.492	3.770
απο	2.7294	27	8.043	3.265
ει	2.0722	4	17.357	4.022
απο της	1.8785	6	12.153	4.070
παρα	1.8286	7	11.818	4.008
γαρ	1.7818	9	9.806	4.036
εστι	1.7597	24	6.813	3.842
περι	1.6706	11	8.368	4.084
τοις	1.6346	3	15.168	4.227
και	1.5039	66	4.970	3.750
πολιχνιον	1.4262	3	13.052	4.292
ου	1.3369	14	7.939	4.004
δυναται	1.3097	3	11.596	4.337
εκαλειτο	1.2841	7	9.569	4.178
δε	1.2678	46	5.335	3.891
φησι	1.2197	12	8.754	3.982
εν	1.1327	37	5.953	3.734
εστι και	1.0977	20	6.748	4.007
αφ ου	1.0834	5	10.673	4.233

Negative term	Coefficient	Docs	Mean present	Mean absent
και το	-1.2415	9	3.689	4.641
ουδετερως	-1.1559	3	1.274	4.656
εκληθη	-1.1211	4	1.684	4.675
ης	-1.0201	3	1.763	4.641
πολις προς	-1.0155	3	1.808	4.640
τοπος	-1.0007	4	1.993	4.662
προς τη	-0.9457	4	1.707	4.674
του δε	-0.9102	3	2.050	4.632
ποσειδωνος	-0.8370	3	2.353	4.623
λιβυης	-0.8194	4	3.593	4.595
χωρα	-0.7988	6	5.887	4.470
μια	-0.7444	3	3.090	4.600
εκληθη δε	-0.7383	3	1.674	4.644
ως στραβων	-0.7293	4	3.105	4.615
τον	-0.6882	10	4.703	4.539
και του	-0.6864	4	3.570	4.596
ηροδοτος	-0.6728	9	5.170	4.494
παυσανιας	-0.6704	4	2.959	4.622
θρακης	-0.6686	5	2.294	4.674
πορρω	-0.6672	3	1.654	4.645

AI English terms predicting large absolute residuals

Features: 404; ridge alpha: 5.878; cross-validated R^2: -0.139.

Positive term	Coefficient	Docs	Mean present	Mean absent
from	1.5655	37	6.553	3.382
the	1.4306	97	4.624	2.330
said	1.0564	8	11.667	3.937
there	0.9600	34	6.734	3.433
they	0.9538	9	9.004	4.115
diphthong	0.9503	3	17.261	4.162
in the	0.8784	30	6.052	3.913
on the	0.7832	8	10.246	4.060
caria	0.7732	3	10.526	4.370
be	0.7455	4	13.683	4.175
are	0.7441	32	6.652	3.568
the feminine	0.7434	11	8.976	4.009
those	0.7432	7	13.675	3.869
it	0.7356	45	5.940	3.421
there is also	0.7223	22	6.660	3.961
there is	0.7159	24	6.451	3.956
is also	0.7016	29	6.349	3.822
in	0.6919	81	4.766	3.655
or	0.6807	16	7.692	3.958
is said	0.6806	5	12.048	4.161

Negative term	Coefficient	Docs	Mean present	Mean absent
name	-0.7556	18	2.312	5.047
region	-0.6046	6	1.238	4.767
the inhabitant is	-0.5773	7	1.473	4.787
inhabitant is	-0.5773	7	1.473	4.787
and the	-0.5212	16	4.497	4.566
the name	-0.5180	7	1.768	4.765
his	-0.5059	28	3.949	4.791
inhabitant is called	-0.4605	5	1.232	4.730
known as	-0.4134	4	1.660	4.676
of his	-0.4040	6	1.943	4.722
form is	-0.3902	11	3.358	4.703
region is	-0.3883	3	0.840	4.670
is also called	-0.3883	4	1.928	4.664
so	-0.3786	5	2.177	4.680
which the	-0.3592	4	1.968	4.663
the region	-0.3590	3	0.728	4.673
the inhabitant	-0.3576	8	4.842	4.530
inhabitant	-0.3576	8	4.842	4.530
the ethnic adjective	-0.3527	7	2.574	4.704
ethnic adjective	-0.3527	7	2.574	4.704

Human English terms predicting large absolute residuals

Features: 335; ridge alpha: 5.878; cross-validated R^2: -0.130.

Positive term	Coefficient	Docs	Mean present	Mean absent
like	1.0726	5	8.895	4.327
they	1.0229	12	7.300	4.181
on	0.9914	16	8.360	3.830
also a	0.9432	15	8.892	3.790
from	0.9360	37	6.345	3.504
there	0.9171	34	7.268	3.157
is also a	0.8796	14	9.209	3.797
genitive	0.8767	3	20.939	4.048
from the	0.8682	13	9.946	3.749
city on	0.8627	3	14.805	4.238
who	0.8390	13	8.555	3.957
and	0.8261	52	5.492	3.540
diphthong	0.7914	3	17.261	4.162
the	0.7382	90	4.770	2.617
there is	0.7342	24	6.582	3.915
however	0.7076	10	10.101	3.939
that	0.7041	29	6.949	3.577
be	0.6840	15	8.163	3.918
also	0.6795	43	5.624	3.749
is with	0.6785	4	11.346	4.272

Negative term	Coefficient	Docs	Mean present	Mean absent
near	-0.5706	16	3.469	4.762
the ethnonym is	-0.5332	52	3.916	5.247
ethnonym is	-0.5332	52	3.916	5.247
the ethnonym	-0.4876	56	3.987	5.278
ethnonym	-0.4565	57	4.072	5.195
herodotos	-0.4169	9	5.170	4.494
neuter	-0.4134	4	1.134	4.698
one of	-0.3906	5	2.417	4.668
just as	-0.3871	5	2.034	4.688
libya	-0.3756	5	3.052	4.634
in libya	-0.3756	5	3.052	4.634
asia	-0.3499	6	3.169	4.644
in his asia	-0.3443	5	2.898	4.642
his asia	-0.3443	5	2.898	4.642
an island	-0.3418	9	3.060	4.703
thrace	-0.3263	5	2.294	4.674
hekataios in	-0.3258	12	4.106	4.616
the city	-0.3213	6	3.442	4.626
island	-0.3209	14	4.036	4.640
one	-0.3134	12	4.197	4.604

Prompt Text

Show prompt text

You are an expert classical philologist and translator specializing in Byzantine Greek geographical texts.
You will receive Greek text from a lemma entry in Stephanos of Byzantium's Ethnika.
Translate the Greek text into clear, scholarly English.
Preserve technical terminology and place names appropriately.