gpt-5.3-chat-latest v1 Evaluation

Generated: 2026-09-20 11:21:38 UTC

Pairs100

Distinct lemmas100

Slope0.987

R^20.962

BLEU-423.1%

chrF++52.9%

METEOR59.6%

ROUGE-L61.2%

BERTScore92.6%

COMET72.7%

BLEURT65.6%

Profile version id: 2334

First translation using this version: 2026-07-15

Total usable AI runs for this version: 100 runs across 100 lemmas

Created: 2026-07-15 12:28:27.643960+10:00

Active: True

Notes: Model timeline evaluation: GPT-5.3 Chat using prompt text from gpt-5.5 v1; approved-human-only for the 100-row Kappa corpus.

Length Regression

Formula	AI words = intercept + slope * human words
Slope	0.987
Intercept	3.033
R^2	0.962
P-value	3.54e-71
Pearson r	0.981
Mean source words	30.400
Length fallback rows	0
Mean human words	44.150
Mean AI words	46.620
Mean absolute length residual	4.005
Mean absolute percent error	11.3%

Metric vs Passage Length

These plots test whether score changes with source passage length for this prompt version. The dashed line is an ordinary least-squares fit; the annotation reports n, r, R^2, p-value, and slope.

BLEU-4 vs passage length — BLEU-4: positive; r = 0.021, R^2 = 4.35e-04, p = 0.8369.

chrF++ vs passage length — chrF++: positive; r = 0.037, R^2 = 0.001, p = 0.7137.

METEOR vs passage length — METEOR: negative; r = -0.025, R^2 = 6.13e-04, p = 0.8068.

ROUGE-L vs passage length — ROUGE-L: negative; r = -0.065, R^2 = 0.004, p = 0.5216.

BERTScore vs passage length — BERTScore: significant negative; r = -0.342, R^2 = 0.117, p = 0.0005.

COMET vs passage length — COMET: negative; r = -0.194, R^2 = 0.037, p = 0.0536.

BLEURT vs passage length — BLEURT: significant negative; r = -0.693, R^2 = 0.481, p = 1.32e-15.

Trigram precision vs passage length — Trigram precision: positive; r = 0.022, R^2 = 4.71e-04, p = 0.8303.

Trigram recall vs passage length — Trigram recall: positive; r = 0.019, R^2 = 3.46e-04, p = 0.8542.

Trigram F1 vs passage length — Trigram F1: positive; r = 0.021, R^2 = 4.31e-04, p = 0.8375.

Trigram Jaccard vs passage length — Trigram Jaccard: positive; r = 2.27e-04, R^2 = 5.17e-08, p = 0.9982.

Metric	Rows	Pattern	Pearson r	R^2	P-value	Slope	Status
BLEU-4	100	positive	0.021	4.35e-04	0.8369	0.00010	ok
chrF++	100	positive	0.037	0.001	0.7137	0.00012	ok
METEOR	100	negative	-0.025	6.13e-04	0.8068	-0.00012	ok
ROUGE-L	100	negative	-0.065	0.004	0.5216	-0.00028	ok
BERTScore	100	significant negative	-0.342	0.117	0.0005	-0.00026	ok
COMET	100	negative	-0.194	0.037	0.0536	-0.00038	ok
BLEURT	100	significant negative	-0.693	0.481	1.32e-15	-0.00212	ok
Trigram precision	100	positive	0.022	4.71e-04	0.8303	0.00011	ok
Trigram recall	100	positive	0.019	3.46e-04	0.8542	0.00009	ok
Trigram F1	100	positive	0.021	4.31e-04	0.8375	0.00010	ok
Trigram Jaccard	100	positive	2.27e-04	5.17e-08	0.9982	7.22e-07	ok

Translation Similarity Metrics

Mean BLEU-4	23.1%
Mean chrF++	52.9%
Mean METEOR	59.6%
Mean ROUGE-L	61.2%
Mean BERTScore	92.6%
Mean COMET	72.7%
Mean BLEURT	65.6%
Legacy smoothed corpus BLEU	29.2%
Unigram F1	65.3%
Bigram F1	36.7%
Trigram precision	22.1%
Trigram recall	23.4%
Trigram F1	22.7%
Trigram Jaccard	12.8%
4-gram F1	14.8%

Largest Length Residuals

Lemma ID	Headword	Human words	AI words	Residual	BLEU-4	chrF++	METEOR	ROUGE-L	BERTScore	COMET	BLEURT	Human	Run	Model
2465	Κάνωπος	83	147	62.025	23.3%	60.5%	78.8%	59.1%	89.6%	58.8%	44.7%	reviewed/approved	11167	gpt-5.3-chat-latest
2484	Καρία	270	251	-18.592	18.7%	50.1%	52.7%	57.6%	88.7%	74.5%	46.2%	reviewed/approved	11182	gpt-5.3-chat-latest
7266	Κώμη	79	97	15.974	3.5%	34.2%	32.1%	33.0%	89.0%	63.8%	42.5%	reviewed/approved	11320	gpt-5.3-chat-latest
2056	Καβειρία	112	101	-12.605	16.6%	52.6%	54.2%	61.0%	92.3%	69.5%	54.0%	reviewed/approved	11037	gpt-5.3-chat-latest
7267	Κωνώπη	64	78	11.783	36.4%	51.6%	46.9%	54.9%	90.7%	64.5%	63.1%	reviewed/approved	11324	gpt-5.3-chat-latest
2116	Κάληρος	40	33	-9.523	21.6%	53.1%	56.2%	54.8%	93.3%	75.8%	71.7%	reviewed/approved	11111	gpt-5.3-chat-latest
2603	Κάρυστος	175	185	9.197	27.8%	57.1%	66.4%	62.8%	92.8%	76.5%	56.3%	reviewed/approved	11193	gpt-5.3-chat-latest
2328	Καλλίπολις	43	54	8.515	8.4%	48.4%	44.7%	47.4%	91.4%	71.9%	61.6%	reviewed/approved	11131	gpt-5.3-chat-latest
2609	Κάσπειρος	135	128	-8.312	35.4%	62.3%	61.7%	66.2%	89.6%	71.5%	60.8%	reviewed/approved	11204	gpt-5.3-chat-latest
7254	Κύτα	84	94	8.038	28.6%	57.3%	58.6%	59.6%	92.0%	73.6%	59.1%	reviewed/approved	11286	gpt-5.3-chat-latest
2057	Καβελλιών	34	44	7.401	16.1%	47.5%	47.0%	43.6%	89.1%	64.4%	57.2%	reviewed/approved	11038	gpt-5.3-chat-latest
2626	Κατάνη	95	90	-6.822	33.0%	56.3%	63.4%	60.5%	91.6%	72.1%	55.5%	reviewed/approved	11222	gpt-5.3-chat-latest
2342	Κάμιρος	54	63	6.656	23.4%	60.8%	66.9%	66.7%	93.9%	75.6%	65.2%	reviewed/approved	11157	gpt-5.3-chat-latest
2055	Καβασσός	109	104	-6.644	21.6%	51.5%	52.0%	49.8%	92.4%	73.1%	44.1%	reviewed/approved	11034	gpt-5.3-chat-latest
7255	Κυτέριον	22	31	6.248	14.6%	46.3%	48.7%	52.8%	91.1%	66.6%	63.4%	reviewed/approved	11289	gpt-5.3-chat-latest
7253	Κυρτώνιος	18	27	6.197	4.8%	35.7%	36.4%	35.6%	90.7%	75.3%	74.0%	reviewed/approved	11282	gpt-5.3-chat-latest
7245	Κύρης	13	22	6.133	8.7%	34.9%	50.5%	45.7%	87.7%	55.1%	63.8%	reviewed/approved	11259	gpt-5.3-chat-latest
2624	Κατάβαθμος	20	17	-5.778	29.6%	58.3%	54.9%	64.9%	92.8%	76.9%	68.1%	reviewed/approved	11213	gpt-5.3-chat-latest
7261	Κύψελα	34	31	-5.599	43.8%	65.0%	73.0%	76.9%	96.0%	76.1%	73.9%	reviewed/approved	11306	gpt-5.3-chat-latest
2081	Καλάθη	49	46	-5.408	25.4%	55.6%	70.0%	73.7%	93.7%	65.4%	65.0%	reviewed/approved	11082	gpt-5.3-chat-latest
2075	Καινίνη	15	13	-4.841	46.0%	63.3%	73.6%	78.6%	94.1%	77.3%	74.0%	reviewed/approved	11064	gpt-5.3-chat-latest
2080	Καλαβρία	18	16	-4.803	21.9%	52.2%	39.5%	47.1%	93.5%	73.6%	68.3%	reviewed/approved	11079	gpt-5.3-chat-latest
2335	Κάλυτις	26	24	-4.701	5.9%	46.6%	39.2%	56.0%	93.9%	75.1%	68.2%	reviewed/approved	11153	gpt-5.3-chat-latest
2079	Καισάρεια	39	37	-4.536	30.7%	56.4%	67.7%	68.4%	92.0%	73.5%	64.1%	reviewed/approved	11075	gpt-5.3-chat-latest
3288	Κορώνεια	110	116	4.369	16.2%	51.3%	49.6%	52.9%	90.6%	71.2%	53.9%	reviewed/approved	11239	gpt-5.3-chat-latest
2623	Κασώριον	20	27	4.222	28.1%	55.8%	59.7%	55.3%	92.6%	71.1%	73.9%	reviewed/approved	11212	gpt-5.3-chat-latest
2061	Καδούσιοι	14	13	-3.854	47.8%	61.8%	77.2%	74.1%	95.3%	74.1%	59.2%	reviewed/approved	11050	gpt-5.3-chat-latest
2115	Καλὴ ἀκτή	64	70	3.783	15.1%	41.5%	55.4%	52.2%	89.1%	61.4%	49.0%	reviewed/approved	11108	gpt-5.3-chat-latest
2620	Κάσταξ	20	19	-3.778	8.2%	43.5%	50.8%	56.4%	92.4%	67.2%	67.6%	reviewed/approved	11205	gpt-5.3-chat-latest
2078	Καινύς	22	21	-3.752	60.1%	71.7%	76.6%	74.4%	95.4%	82.1%	76.1%	reviewed/approved	11069	gpt-5.3-chat-latest

Residual Predictors

Positive signed-residual terms are associated with AI translations longer than expected from the human length; negative signed-residual terms are associated with shorter AI translations. Absolute-residual models identify terms associated with larger AI-human length divergence.

Greek source terms predicting signed length residuals

Features: 176; ridge alpha: 587.802; cross-validated R^2: -0.225.

Positive term	Coefficient	Docs	Mean present	Mean absent
και	0.0506	66	0.094	-0.182
δια του	0.0304	10	6.157	-0.684
και το	0.0300	9	7.640	-0.756
απο	0.0232	27	2.454	-0.908
και νησος	0.0201	3	21.324	-0.660
δια	0.0190	16	3.922	-0.747
και το κτητικον	0.0184	4	15.901	-0.663
λιβυης	0.0170	4	15.103	-0.629
πολις εν	0.0160	5	11.403	-0.600
εδει	0.0159	3	17.666	-0.546
τοπος	0.0158	4	16.297	-0.679
ποσειδωνος	0.0153	3	18.706	-0.579
κτητικον	0.0126	7	4.804	-0.362
το κτητικον	0.0126	7	4.804	-0.362
νησος	0.0123	9	6.254	-0.619
κωμη	0.0121	3	5.757	-0.178
εστι	0.0117	24	1.466	-0.463
του	0.0116	37	1.578	-0.927
απο του	0.0109	9	4.131	-0.409
εν	0.0095	37	2.105	-1.236

Negative term	Coefficient	Docs	Mean present	Mean absent
δε	-0.0138	46	0.448	-0.382
εκαταιος	-0.0133	14	-2.238	0.364
εκαλειτο	-0.0129	7	-4.008	0.302
εθνος	-0.0128	8	-4.802	0.418
πλησιον	-0.0128	8	-3.028	0.263
και του	-0.0114	4	-5.421	0.226
μεταξυ	-0.0110	5	-3.124	0.164
εκαταιος ασια	-0.0106	5	-2.919	0.154
ασια	-0.0106	5	-2.919	0.154
οτι	-0.0097	5	-1.882	0.099
το θηλυκον	-0.0093	6	-5.425	0.346
χωρα	-0.0086	6	-4.697	0.300
μετα	-0.0081	3	-11.242	0.348
ης	-0.0080	3	-5.281	0.163
θηλυκον και	-0.0079	3	-6.631	0.205
ποταμου	-0.0073	3	-2.852	0.088
παυσανιας	-0.0066	4	-4.235	0.176
λυκιας	-0.0066	3	-3.391	0.105
καππαδοκιας	-0.0064	3	-3.635	0.112
τινες	-0.0064	6	-2.151	0.137

Greek source terms predicting large absolute residuals

Features: 176; ridge alpha: 24.245; cross-validated R^2: -0.235.

Positive term	Coefficient	Docs	Mean present	Mean absent
και	0.9850	66	4.665	2.722
δια του	0.6059	10	10.913	3.237
δια	0.5711	16	8.070	3.230
ουτως	0.5457	5	21.055	3.107
εστι	0.5196	24	7.059	3.040
και το	0.4998	9	9.296	3.481
απο	0.4574	27	6.847	2.953
εδει	0.4084	3	23.684	3.396
τοπος	0.4074	4	19.314	3.367
πολις εν	0.3953	5	16.069	3.370
ποσειδωνος	0.3916	3	22.644	3.428
και νησος	0.3651	3	21.324	3.469
θηλυκον	0.3650	12	11.122	3.034
και το κτητικον	0.3557	4	17.150	3.457
κτητικον	0.3342	7	14.431	3.220
το κτητικον	0.3342	7	14.431	3.220
λιβυης	0.2828	4	16.139	3.499
εστι και	0.2795	20	7.504	3.130
του	0.2442	37	5.514	3.118
εν	0.2152	37	5.630	3.050

Negative term	Coefficient	Docs	Mean present	Mean absent
ως	-0.2715	52	3.595	4.448
ηροδοτος	-0.2053	9	4.563	3.949
πολιτης	-0.1811	18	3.044	4.215
δευτερω	-0.1750	3	1.664	4.077
πολις και	-0.1735	6	1.205	4.183
ην	-0.1710	13	2.370	4.249
το εθνικον	-0.1552	56	4.436	3.456
εθνικον	-0.1552	56	4.436	3.456
και πολις	-0.1460	5	1.933	4.114
δεκατω	-0.1443	3	0.717	4.106
εκ	-0.1432	6	1.626	4.156
πολις	-0.1378	68	4.629	2.677
φησιν	-0.1166	10	4.008	4.004
του δε	-0.1131	3	0.936	4.099
της	-0.1088	35	4.162	3.920
μια	-0.1018	3	1.682	4.076
τοπικα	-0.0983	3	2.533	4.050
τα τοπικα	-0.0983	3	2.533	4.050
ος	-0.0920	6	2.531	4.099
ακρα	-0.0905	4	1.351	4.115

AI English terms predicting large absolute residuals

Features: 403; ridge alpha: 8.377; cross-validated R^2: -0.211.

Positive term	Coefficient	Docs	Mean present	Mean absent
be	1.3615	8	13.716	3.160
written	1.2542	4	17.099	3.459
written with	1.2542	4	17.099	3.459
with a	1.2431	5	16.452	3.349
and the	1.0617	17	9.133	2.954
and	0.9830	56	4.963	2.784
the possessive	0.9318	3	31.074	3.167
there	0.8858	31	6.301	2.973
is a city	0.8783	4	20.388	3.322
with	0.8727	16	7.974	3.248
adjective	0.8697	3	29.552	3.214
is a	0.8620	5	17.041	3.318
possessive	0.8459	4	23.487	3.193
usage	0.8258	5	18.034	3.266
named from	0.8191	4	21.213	3.287
have	0.8170	6	16.505	3.207
is	0.8129	88	4.149	2.945
a place in	0.8032	3	25.667	3.335
feminine	0.7585	16	10.446	2.778
to be	0.7572	7	14.782	3.193

Negative term	Coefficient	Docs	Mean present	Mean absent
the ethnic	-0.4767	55	3.106	5.103
name is	-0.4649	15	1.757	4.401
ethnic name is	-0.4608	13	1.706	4.348
ethnic	-0.4255	58	3.315	4.956
in the	-0.4200	36	3.201	4.456
the ethnic name	-0.4099	15	2.060	4.348
ethnic name	-0.4099	15	2.060	4.348
at	-0.3781	7	0.948	4.235
are	-0.3558	21	3.586	4.116
the ethnic is	-0.3177	37	3.285	4.427
an iota	-0.3114	4	1.627	4.104
which	-0.3017	16	3.856	4.033
is called	-0.2834	17	3.484	4.111
also called	-0.2778	12	4.291	3.965
forms are	-0.2736	3	1.075	4.095
settlement	-0.2635	3	0.616	4.109
ethnic is	-0.2618	38	3.530	4.295
at the	-0.2495	3	0.378	4.117
strabo book	-0.2426	6	2.097	4.126
book of his	-0.2419	10	2.621	4.158

Human English terms predicting large absolute residuals

Features: 335; ridge alpha: 8.377; cross-validated R^2: -0.200.

Positive term	Coefficient	Docs	Mean present	Mean absent
written	1.8962	3	26.728	3.302
for	1.3358	11	12.878	2.908
there	1.2626	34	6.550	2.693
and the	1.0786	15	8.315	3.244
with	1.0666	16	8.086	3.227
a place	1.0059	3	25.667	3.335
the	0.9978	90	4.160	2.603
is written	0.9690	3	26.728	3.302
is a city	0.9061	3	23.121	3.413
feminine is	0.9003	12	12.138	2.895
the feminine is	0.9003	12	12.138	2.895
poseidon	0.8905	3	22.644	3.428
it should	0.8361	3	23.684	3.396
should	0.8361	3	23.684	3.396
is also	0.8229	31	6.626	2.827
there is also	0.7996	22	7.154	3.116
and	0.7930	52	4.974	2.954
sanctuary	0.7897	3	21.811	3.454
feminine	0.7818	16	10.446	2.778
the feminine	0.7818	16	10.446	2.778

Negative term	Coefficient	Docs	Mean present	Mean absent
at	-0.4348	6	0.839	4.207
of the	-0.4122	33	3.584	4.212
was	-0.3952	20	3.424	4.150
also called	-0.3816	6	1.454	4.167
city in	-0.3654	41	3.744	4.186
a city in	-0.3528	40	3.771	4.161
citizen	-0.3232	18	3.044	4.215
a citizen	-0.3232	18	3.044	4.215
the ethnonym is	-0.3013	52	4.046	3.960
ethnonym is	-0.3013	52	4.046	3.960
called	-0.2844	21	3.850	4.045
herodotos	-0.2815	9	4.563	3.949
near	-0.2809	16	3.330	4.133
in book	-0.2595	16	3.926	4.019
his	-0.2423	34	3.722	4.150
of his	-0.2410	18	3.805	4.048
herodotos book	-0.2353	5	3.285	4.042
citizen is a	-0.2346	15	3.221	4.143
citizen is	-0.2346	15	3.221	4.143
a citizen is	-0.2346	15	3.221	4.143

Prompt Text

Show prompt text

You are an expert classical philologist and translator specializing in Byzantine Greek geographical texts.
You will receive Greek text from a lemma entry in Stephanos of Byzantium's Ethnika.
Translate the Greek text into clear, scholarly English.
Preserve technical terminology and place names appropriately.