gpt-5.5 v1 Evaluation

Generated: 2026-09-20 11:22:03 UTC

Pairs100

Distinct lemmas100

Slope1.003

R^20.985

BLEU-425.1%

chrF++53.8%

METEOR56.1%

ROUGE-L58.4%

BERTScore92.3%

COMET72.2%

BLEURT63.6%

Profile version id: 1

First translation using this version: 2026-06-17

Total usable AI runs for this version: 101 runs across 101 lemmas

Created: 2026-01-21 23:08:03.772891+11:00

Active:

Notes: Initial prompt - minimal guidance

Length Regression

Formula	AI words = intercept + slope * human words
Slope	1.003
Intercept	2.626
R^2	0.985
P-value	1.85e-91
Pearson r	0.993
Mean source words	30.400
Length fallback rows	0
Mean human words	44.150
Mean AI words	46.920
Mean absolute length residual	3.554
Mean absolute percent error	10.3%

Metric vs Passage Length

These plots test whether score changes with source passage length for this prompt version. The dashed line is an ordinary least-squares fit; the annotation reports n, r, R^2, p-value, and slope.

BLEU-4 vs passage length — BLEU-4: negative; r = -0.002, R^2 = 4.52e-06, p = 0.9833.

chrF++ vs passage length — chrF++: negative; r = -0.011, R^2 = 1.32e-04, p = 0.9097.

METEOR vs passage length — METEOR: negative; r = -0.023, R^2 = 5.20e-04, p = 0.8217.

ROUGE-L vs passage length — ROUGE-L: negative; r = -0.042, R^2 = 0.002, p = 0.6788.

BERTScore vs passage length — BERTScore: significant negative; r = -0.268, R^2 = 0.072, p = 0.0070.

COMET vs passage length — COMET: negative; r = -0.125, R^2 = 0.016, p = 0.2166.

BLEURT vs passage length — BLEURT: significant negative; r = -0.570, R^2 = 0.325, p = 5.85e-10.

Trigram precision vs passage length — Trigram precision: positive; r = 0.058, R^2 = 0.003, p = 0.5653.

Trigram recall vs passage length — Trigram recall: positive; r = 0.042, R^2 = 0.002, p = 0.6757.

Trigram F1 vs passage length — Trigram F1: positive; r = 0.052, R^2 = 0.003, p = 0.6074.

Trigram Jaccard vs passage length — Trigram Jaccard: positive; r = 0.024, R^2 = 5.73e-04, p = 0.8131.

Metric	Rows	Pattern	Pearson r	R^2	P-value	Slope	Status
BLEU-4	100	negative	-0.002	4.52e-06	0.9833	-9.57e-06	ok
chrF++	100	negative	-0.011	1.32e-04	0.9097	-0.00004	ok
METEOR	100	negative	-0.023	5.20e-04	0.8217	-0.00011	ok
ROUGE-L	100	negative	-0.042	0.002	0.6788	-0.00017	ok
BERTScore	100	significant negative	-0.268	0.072	0.0070	-0.00021	ok
COMET	100	negative	-0.125	0.016	0.2166	-0.00024	ok
BLEURT	100	significant negative	-0.570	0.325	5.85e-10	-0.00179	ok
Trigram precision	100	positive	0.058	0.003	0.5653	0.00028	ok
Trigram recall	100	positive	0.042	0.002	0.6757	0.00020	ok
Trigram F1	100	positive	0.052	0.003	0.6074	0.00025	ok
Trigram Jaccard	100	positive	0.024	5.73e-04	0.8131	0.00008	ok

Translation Similarity Metrics

Mean BLEU-4	25.1%
Mean chrF++	53.8%
Mean METEOR	56.1%
Mean ROUGE-L	58.4%
Mean BERTScore	92.3%
Mean COMET	72.2%
Mean BLEURT	63.6%
Legacy smoothed corpus BLEU	26.6%
Unigram F1	62.2%
Bigram F1	33.9%
Trigram precision	20.1%
Trigram recall	21.4%
Trigram F1	20.7%
Trigram Jaccard	11.6%
4-gram F1	13.0%

Largest Length Residuals

Lemma ID	Headword	Human words	AI words	Residual	BLEU-4	chrF++	METEOR	ROUGE-L	BERTScore	COMET	BLEURT	Human	Run	Model
2484	Καρία	270	257	-16.505	18.8%	49.6%	46.4%	52.4%	88.4%	74.5%	46.8%	reviewed/approved	3592	gpt-5.5
2604	Καρχηδών	117	133	12.993	17.9%	52.0%	54.4%	54.0%	91.4%	75.7%	55.0%	reviewed/approved	3596	gpt-5.5
7264	Κωλιάς	61	76	12.175	20.8%	49.1%	44.0%	46.7%	90.0%	65.8%	53.4%	reviewed/approved	4147	gpt-5.5
3288	Κορώνεια	110	125	12.016	17.7%	51.8%	41.5%	50.8%	90.6%	70.5%	54.9%	reviewed/approved	3606	gpt-5.5
2468	Καπετώλιον	138	153	11.925	13.0%	38.9%	33.7%	40.5%	84.5%	66.5%	41.9%	reviewed/approved	3600	gpt-5.5
2328	Καλλίπολις	43	57	11.234	16.0%	48.5%	42.0%	48.0%	90.7%	72.6%	58.5%	reviewed/approved	3840	gpt-5.5
2115	Καλὴ ἀκτή	64	78	11.165	14.4%	41.4%	54.6%	49.3%	88.2%	62.3%	45.4%	reviewed/approved	3823	gpt-5.5
7266	Κώμη	79	92	10.117	3.6%	34.0%	33.8%	30.4%	89.1%	65.2%	42.5%	reviewed/approved	4235	gpt-5.5
2465	Κάνωπος	83	95	9.104	35.3%	62.1%	66.7%	67.4%	93.6%	71.6%	61.2%	reviewed/approved	4060	gpt-5.5
2621	Κάστνιον	25	36	8.292	22.1%	50.2%	60.3%	49.2%	93.0%	66.0%	71.2%	reviewed/approved	4076	gpt-5.5
2605	Κάσιον	56	66	7.191	22.1%	50.4%	53.8%	50.8%	92.0%	70.5%	58.4%	reviewed/approved	4070	gpt-5.5
2597	Καρπασία	112	108	-6.991	23.2%	57.3%	57.1%	62.7%	93.4%	70.9%	53.8%	reviewed/approved	3602	gpt-5.5
2626	Κατάνη	95	91	-6.935	30.8%	57.0%	62.8%	64.5%	92.0%	72.5%	57.0%	reviewed/approved	3610	gpt-5.5
2455	Κάναστρον	60	56	-6.822	19.0%	50.3%	59.0%	63.8%	92.5%	65.3%	59.2%	reviewed/approved	3862	gpt-5.5
7255	Κυτέριον	22	31	6.302	17.1%	46.8%	34.4%	45.3%	91.2%	66.0%	59.3%	reviewed/approved	4129	gpt-5.5
2055	Καβασσός	109	106	-5.981	18.4%	51.8%	48.8%	53.0%	91.8%	71.0%	44.4%	reviewed/approved	3608	gpt-5.5
3496	Κοτιάειον	71	68	-5.857	15.7%	43.5%	50.6%	51.8%	89.7%	57.9%	54.3%	reviewed/approved	4097	gpt-5.5
7267	Κωνώπη	64	61	-5.835	39.0%	59.3%	64.1%	66.7%	93.4%	69.0%	66.6%	reviewed/approved	4237	gpt-5.5
7261	Κύψελα	34	31	-5.737	30.1%	57.5%	53.4%	58.5%	93.7%	74.6%	66.6%	reviewed/approved	4141	gpt-5.5
2330	Καλύβη	24	21	-5.704	22.1%	48.4%	40.5%	57.8%	92.1%	72.2%	67.7%	reviewed/approved	3844	gpt-5.5
2623	Κασώριον	20	28	5.309	23.8%	53.6%	59.4%	54.2%	92.5%	72.6%	73.8%	reviewed/approved	4081	gpt-5.5
7248	Κύρου πόλις	32	40	5.270	10.8%	44.4%	28.7%	38.9%	89.6%	70.1%	65.9%	reviewed/approved	4116	gpt-5.5
2342	Κάμιρος	54	62	5.198	20.7%	48.9%	48.2%	46.6%	91.9%	72.4%	55.5%	reviewed/approved	3856	gpt-5.5
2119	Κάλλατις	65	73	5.162	16.4%	48.9%	48.3%	49.3%	90.8%	63.7%	58.5%	reviewed/approved	3832	gpt-5.5
2607	Κάσος	57	55	-4.812	27.1%	54.5%	59.6%	60.7%	93.7%	73.6%	68.8%	reviewed/approved	4072	gpt-5.5
2079	Καισάρεια	39	37	-4.753	41.6%	63.7%	76.3%	78.9%	92.7%	75.9%	67.9%	reviewed/approved	3801	gpt-5.5
7249	Κύρρος	45	52	4.227	8.4%	44.3%	50.9%	49.5%	90.4%	66.5%	48.3%	reviewed/approved	4118	gpt-5.5
2054	Καβαλίς	65	72	4.162	35.0%	60.1%	47.6%	65.7%	93.1%	73.1%	68.3%	reviewed/approved	3776	gpt-5.5
7254	Κύτα	84	91	4.100	34.4%	63.0%	69.9%	68.6%	93.2%	74.2%	67.3%	reviewed/approved	4127	gpt-5.5
2056	Καβειρία	112	111	-3.991	18.6%	56.4%	59.7%	60.1%	92.8%	71.4%	56.4%	reviewed/approved	3604	gpt-5.5

Residual Predictors

Positive signed-residual terms are associated with AI translations longer than expected from the human length; negative signed-residual terms are associated with shorter AI translations. Absolute-residual models identify terms associated with larger AI-human length divergence.

Greek source terms predicting signed length residuals

Features: 176; ridge alpha: 5.878; cross-validated R^2: -0.181.

Positive term	Coefficient	Docs	Mean present	Mean absent
δια του	1.4736	10	4.395	-0.488
και	1.2977	66	0.463	-0.899
εν τη	1.1384	7	4.931	-0.371
γαρ	1.0962	9	2.859	-0.283
εστι δε	1.0249	5	6.483	-0.341
κωμη	0.9948	3	7.172	-0.222
εξ	0.8551	3	8.106	-0.251
εστι δε και	0.8052	3	8.414	-0.260
ου	0.7745	14	1.401	-0.228
πολιτης	0.7652	18	2.691	-0.591
πολιχνιον	0.7640	3	6.506	-0.201
εν	0.7547	37	1.605	-0.943
ου και	0.7492	4	5.097	-0.212
δευτερα	0.7420	3	7.513	-0.232
δια	0.7321	16	2.598	-0.495
τοπος	0.7100	4	5.598	-0.233
δε και	0.6788	8	1.285	-0.112
θηλυκως	0.6586	5	3.570	-0.188
και θηλυκως	0.6586	5	3.570	-0.188
οι	0.6581	11	2.601	-0.321

Negative term	Coefficient	Docs	Mean present	Mean absent
το εθνικον	-1.1596	56	-1.031	1.312
εθνικον	-1.1596	56	-1.031	1.312
θρακης	-1.1005	5	-4.732	0.249
εστι και	-0.9390	20	-1.178	0.294
πολις θρακης	-0.8584	4	-4.210	0.175
το	-0.8256	70	-0.235	0.548
ηροδοτος	-0.6571	9	-2.401	0.237
τον	-0.6335	10	-1.850	0.206
εθνος	-0.6232	8	-3.095	0.269
τινες	-0.6203	6	-2.766	0.177
ην	-0.5886	13	-1.049	0.157
καππαδοκιας	-0.5721	3	-3.480	0.108
μετα	-0.5632	3	-8.502	0.263
φησιν	-0.5558	10	-0.949	0.105
οτι	-0.5529	5	-2.469	0.130
ελλανικος	-0.5102	4	-7.064	0.294
προς τη	-0.5023	4	-1.955	0.081
πλησιον	-0.4706	8	-0.880	0.077
τα	-0.4685	12	-0.803	0.110
τω	-0.4627	16	-0.083	0.016

Greek source terms predicting large absolute residuals

Features: 176; ridge alpha: 2.894; cross-validated R^2: -0.092.

Positive term	Coefficient	Docs	Mean present	Mean absent
και	2.6683	66	4.220	2.262
γαρ	2.3790	9	8.315	3.083
δε	1.8159	46	4.715	2.565
δια του	1.6545	10	8.233	3.034
εστι	1.4547	24	5.623	2.900
την	1.3822	13	6.344	3.137
εν	1.3453	37	5.027	2.689
δια	1.2948	16	6.058	3.077
τριτη	1.0183	3	9.334	3.375
εκαλειτο	1.0060	7	6.939	3.299
εστι δε	0.9985	5	8.080	3.316
εν τη	0.9391	7	5.724	3.391
της	0.8650	35	5.151	2.694
αλεξανδρος	0.8487	3	7.084	3.445
θρακης	0.8460	5	4.732	3.492
παρα	0.8298	7	7.281	3.273
κωμη	0.7940	3	7.172	3.442
περι	0.7907	11	5.871	3.268
ει	0.7875	4	9.651	3.300
εν τω περι	0.7873	4	8.315	3.356

Negative term	Coefficient	Docs	Mean present	Mean absent
δια το	-0.7691	3	0.768	3.640
του δε	-0.7633	3	1.438	3.619
νησος	-0.6971	9	2.860	3.623
δευτερω	-0.6599	3	1.122	3.629
παυσανιας	-0.6578	4	1.909	3.622
στραβων	-0.6194	15	3.318	3.596
οι	-0.6039	11	3.851	3.517
τους	-0.5728	4	2.038	3.617
αυτην	-0.5558	9	2.481	3.660
παιδος	-0.5505	4	2.588	3.594
θηλυκον και	-0.4928	3	1.970	3.603
πλησιον	-0.4911	8	2.760	3.623
τινες	-0.4714	6	2.829	3.600
ιταλιας	-0.4550	3	1.566	3.615
φασι	-0.4401	8	3.736	3.538
μια	-0.4379	3	2.749	3.579
προς τω	-0.4378	4	2.491	3.598
και του	-0.4330	4	2.687	3.590
εκαταιος	-0.4300	14	3.403	3.579
ηροδοτος	-0.4298	9	3.293	3.580

AI English terms predicting large absolute residuals

Features: 394; ridge alpha: 1.425; cross-validated R^2: -0.027.

Positive term	Coefficient	Docs	Mean present	Mean absent
with	2.6297	19	6.421	2.881
for	2.3158	12	6.251	3.186
or	2.1986	14	6.816	3.023
is a	2.0114	5	9.690	3.231
and	1.8890	56	4.505	2.343
on	1.8405	12	6.834	3.107
too	1.7753	11	7.676	3.044
at	1.7641	5	9.674	3.232
is	1.7584	89	3.731	2.124
usage	1.7363	4	8.817	3.335
caria	1.6756	3	10.471	3.340
also	1.5891	43	4.968	2.487
is a city	1.5659	4	9.584	3.303
there is	1.5579	27	5.342	2.893
he	1.5563	5	7.617	3.340
form	1.4495	21	5.503	3.036
iota	1.4137	6	7.113	3.327
to	1.4130	34	4.949	2.835
a	1.3629	89	3.840	1.242
diphthong	1.3426	3	11.807	3.299

Negative term	Coefficient	Docs	Mean present	Mean absent
is called	-1.2404	18	2.372	3.813
an	-1.1224	21	2.838	3.744
are	-0.9605	26	3.779	3.475
it in his	-0.8687	3	0.623	3.645
for the	-0.8295	6	2.356	3.630
is called a	-0.7691	12	1.923	3.776
called a	-0.7691	12	1.923	3.776
of the	-0.7457	46	3.833	3.316
book of the	-0.7415	11	2.521	3.682
a city	-0.7212	65	3.679	3.321
as	-0.7158	40	4.082	3.202
the city	-0.6897	6	2.713	3.608
an island	-0.6632	10	2.745	3.644
near	-0.6608	16	3.040	3.652
which is	-0.6472	3	2.123	3.598
ethnic	-0.6406	61	3.821	3.136
ethnic is	-0.6403	54	3.897	3.152
not	-0.6274	7	2.195	3.656
pausanias	-0.6226	4	1.909	3.622
the ethnic is	-0.6171	53	3.895	3.169

Human English terms predicting large absolute residuals

Features: 335; ridge alpha: 2.894; cross-validated R^2: -0.118.

Positive term	Coefficient	Docs	Mean present	Mean absent
with	2.1114	16	6.754	2.944
and	1.5405	52	4.593	2.428
on	1.4928	16	5.542	3.175
the	1.4180	90	3.775	1.563
at	1.1786	6	8.805	3.219
there is	1.0950	24	5.593	2.910
also	1.0682	43	4.616	2.753
or	1.0572	18	6.687	2.866
is with	1.0400	4	11.027	3.243
however	0.9846	10	7.564	3.108
karia	0.9750	3	10.471	3.340
be	0.9477	15	6.623	3.012
for	0.9281	11	5.803	3.276
this	0.9220	15	7.002	2.945
like	0.9071	5	7.952	3.322
it	0.8898	37	4.983	2.715
is also	0.8621	31	4.908	2.946
there is also	0.8608	22	5.375	3.040
when	0.8279	7	7.975	3.221
of	0.8011	66	4.012	2.664

Negative term	Coefficient	Docs	Mean present	Mean absent
the city	-0.6127	6	1.997	3.653
herodotos	-0.6034	9	3.293	3.580
the ethnonym is	-0.5842	52	3.586	3.519
ethnonym is	-0.5842	52	3.586	3.519
an island	-0.5787	9	2.273	3.681
near	-0.5616	16	2.876	3.683
is a	-0.5553	31	4.002	3.353
an	-0.5530	22	3.165	3.664
in book	-0.5325	16	3.103	3.640
in book of	-0.5292	14	2.314	3.756
the ethnonym	-0.5118	56	3.595	3.502
are	-0.5016	25	4.547	3.223
ethnonym	-0.4871	57	3.637	3.444
pausanias	-0.4383	4	1.909	3.622
and there is	-0.4311	3	2.496	3.587
and there	-0.4311	3	2.496	3.587
italy	-0.4288	3	1.566	3.615
say that	-0.4103	9	2.907	3.618
it is also	-0.3943	3	1.253	3.625
island	-0.3837	14	3.239	3.605

Prompt Text

Show prompt text

You are an expert classical philologist and translator specializing in Byzantine Greek geographical texts.
You will receive Greek text from a lemma entry in Stephanos of Byzantium's Ethnika.
Translate the Greek text into clear, scholarly English.
Preserve technical terminology and place names appropriately.