gpt-5.5 v1 Evaluation

Generated: 2026-07-25 12:53:46 UTC

Pairs100

Distinct lemmas100

Slope1.003

R^20.985

BLEU-425.1%

chrF++53.7%

METEOR56.0%

ROUGE-L58.4%

BERTScoreN/A

COMETN/A

BLEURTN/A

Profile version id: 1

First translation using this version: 2026-06-17

Total usable AI runs for this version: 101 runs across 101 lemmas

Created: 2026-01-21 23:08:03.772891+11:00

Active:

Notes: Initial prompt - minimal guidance

Length Regression

Formula	AI words = intercept + slope * human words
Slope	1.003
Intercept	2.607
R^2	0.985
P-value	1.94e-91
Pearson r	0.993
Mean source words	30.400
Length fallback rows	0
Mean human words	44.160
Mean AI words	46.920
Mean absolute length residual	3.562
Mean absolute percent error	10.2%

Metric vs Passage Length

These plots test whether score changes with source passage length for this prompt version. The dashed line is an ordinary least-squares fit; the annotation reports n, r, R^2, p-value, and slope.

BLEU-4 vs passage length — BLEU-4: negative; r = -0.003, R^2 = 7.66e-06, p = 0.9782.

chrF++ vs passage length — chrF++: negative; r = -0.009, R^2 = 8.21e-05, p = 0.9287.

METEOR vs passage length — METEOR: negative; r = -0.021, R^2 = 4.35e-04, p = 0.8368.

ROUGE-L vs passage length — ROUGE-L: negative; r = -0.041, R^2 = 0.002, p = 0.6860.

Trigram precision vs passage length — Trigram precision: positive; r = 0.057, R^2 = 0.003, p = 0.5709.

Trigram recall vs passage length — Trigram recall: positive; r = 0.044, R^2 = 0.002, p = 0.6615.

Trigram F1 vs passage length — Trigram F1: positive; r = 0.052, R^2 = 0.003, p = 0.6049.

Trigram Jaccard vs passage length — Trigram Jaccard: positive; r = 0.024, R^2 = 5.88e-04, p = 0.8107.

Metric	Rows	Pattern	Pearson r	R^2	P-value	Slope	Status
BLEU-4	100	negative	-0.003	7.66e-06	0.9782	-0.00001	ok
chrF++	100	negative	-0.009	8.21e-05	0.9287	-0.00003	ok
METEOR	100	negative	-0.021	4.35e-04	0.8368	-0.00010	ok
ROUGE-L	100	negative	-0.041	0.002	0.6860	-0.00017	ok
BERTScore	0	not enough data	N/A	N/A	N/A	N/A	needs at least two rows with metric scores
COMET	0	not enough data	N/A	N/A	N/A	N/A	needs at least two rows with metric scores
BLEURT	0	not enough data	N/A	N/A	N/A	N/A	needs at least two rows with metric scores
Trigram precision	100	positive	0.057	0.003	0.5709	0.00027	ok
Trigram recall	100	positive	0.044	0.002	0.6615	0.00021	ok
Trigram F1	100	positive	0.052	0.003	0.6049	0.00025	ok
Trigram Jaccard	100	positive	0.024	5.88e-04	0.8107	0.00008	ok

Translation Similarity Metrics

Mean BLEU-4	25.1%
Mean chrF++	53.7%
Mean METEOR	56.0%
Mean ROUGE-L	58.4%
Mean BERTScore	N/A
Mean COMET	N/A
Mean BLEURT	N/A
Legacy smoothed corpus BLEU	26.6%
Unigram F1	62.2%
Bigram F1	33.9%
Trigram precision	20.1%
Trigram recall	21.4%
Trigram F1	20.7%
Trigram Jaccard	11.6%
4-gram F1	13.0%

Largest Length Residuals

Lemma ID	Headword	Human words	AI words	Residual	BLEU-4	chrF++	METEOR	ROUGE-L	BERTScore	COMET	BLEURT	Human	Run	Model
2484	Καρία	270	257	-16.542	18.8%	49.6%	46.4%	52.4%	N/A	N/A	N/A	reviewed/approved	3592	gpt-5.5
2604	Καρχηδών	117	133	12.988	17.9%	52.0%	54.4%	54.0%	N/A	N/A	N/A	reviewed/approved	3596	gpt-5.5
7264	Κωλιάς	61	76	12.182	20.8%	49.1%	44.0%	46.7%	N/A	N/A	N/A	reviewed/approved	4147	gpt-5.5
3288	Κορώνεια	110	125	12.012	17.7%	51.8%	41.5%	50.8%	N/A	N/A	N/A	reviewed/approved	3606	gpt-5.5
2468	Καπετώλιον	138	153	11.915	13.0%	38.9%	33.7%	40.5%	N/A	N/A	N/A	reviewed/approved	3600	gpt-5.5
2328	Καλλίπολις	43	57	11.244	16.0%	48.5%	42.0%	48.0%	N/A	N/A	N/A	reviewed/approved	3840	gpt-5.5
2115	Καλὴ ἀκτή	64	78	11.171	14.4%	41.4%	54.6%	49.3%	N/A	N/A	N/A	reviewed/approved	3823	gpt-5.5
7266	Κώμη	79	92	10.119	3.6%	34.0%	33.8%	30.4%	N/A	N/A	N/A	reviewed/approved	4235	gpt-5.5
2465	Κάνωπος	83	95	9.105	35.3%	62.1%	66.7%	67.4%	N/A	N/A	N/A	reviewed/approved	4060	gpt-5.5
2621	Κάστνιον	25	36	8.306	22.1%	50.2%	60.3%	49.2%	N/A	N/A	N/A	reviewed/approved	4076	gpt-5.5
2605	Κάσιον	56	66	7.199	22.1%	50.4%	53.8%	50.8%	N/A	N/A	N/A	reviewed/approved	4070	gpt-5.5
2597	Καρπασία	112	108	-6.995	23.2%	57.3%	57.1%	62.7%	N/A	N/A	N/A	reviewed/approved	3602	gpt-5.5
2626	Κατάνη	95	91	-6.936	30.8%	57.0%	62.8%	64.5%	N/A	N/A	N/A	reviewed/approved	3610	gpt-5.5
2455	Κάναστρον	60	56	-6.815	19.0%	50.3%	59.0%	63.8%	N/A	N/A	N/A	reviewed/approved	3862	gpt-5.5
7255	Κυτέριον	22	31	6.317	17.1%	46.8%	34.4%	45.3%	N/A	N/A	N/A	reviewed/approved	4129	gpt-5.5
2055	Καβασσός	109	106	-5.985	18.4%	51.8%	48.8%	53.0%	N/A	N/A	N/A	reviewed/approved	3608	gpt-5.5
3496	Κοτιάειον	71	68	-5.853	15.7%	43.5%	50.6%	51.8%	N/A	N/A	N/A	reviewed/approved	4097	gpt-5.5
7267	Κωνώπη	64	61	-5.829	39.0%	59.3%	64.1%	66.7%	N/A	N/A	N/A	reviewed/approved	4237	gpt-5.5
7261	Κύψελα	34	31	-5.725	30.1%	57.5%	53.4%	58.5%	N/A	N/A	N/A	reviewed/approved	4141	gpt-5.5
2330	Καλύβη	24	21	-5.690	22.1%	48.4%	40.5%	57.8%	N/A	N/A	N/A	reviewed/approved	3844	gpt-5.5
2623	Κασώριον	20	28	5.324	23.8%	53.6%	59.4%	54.2%	N/A	N/A	N/A	reviewed/approved	4081	gpt-5.5
7248	Κύρου πόλις	32	40	5.282	10.8%	44.4%	28.7%	38.9%	N/A	N/A	N/A	reviewed/approved	4116	gpt-5.5
2342	Κάμιρος	54	62	5.206	20.7%	48.9%	48.2%	46.6%	N/A	N/A	N/A	reviewed/approved	3856	gpt-5.5
2119	Κάλλατις	65	73	5.168	16.4%	48.9%	48.3%	49.3%	N/A	N/A	N/A	reviewed/approved	3832	gpt-5.5
2607	Κάσος	57	55	-4.804	27.1%	54.5%	59.6%	60.7%	N/A	N/A	N/A	reviewed/approved	4072	gpt-5.5
2079	Καισάρεια	39	37	-4.742	41.6%	63.7%	76.3%	78.9%	N/A	N/A	N/A	reviewed/approved	3801	gpt-5.5
7249	Κύρρος	45	52	4.237	8.4%	44.3%	50.9%	49.5%	N/A	N/A	N/A	reviewed/approved	4118	gpt-5.5
2054	Καβαλίς	65	72	4.168	35.0%	60.1%	47.6%	65.7%	N/A	N/A	N/A	reviewed/approved	3776	gpt-5.5
7254	Κύτα	84	91	4.102	33.7%	62.4%	67.9%	67.4%	N/A	N/A	N/A	reviewed/approved	4127	gpt-5.5
2056	Καβειρία	112	111	-3.995	18.6%	56.4%	59.7%	60.1%	N/A	N/A	N/A	reviewed/approved	3604	gpt-5.5

Residual Predictors

Positive signed-residual terms are associated with AI translations longer than expected from the human length; negative signed-residual terms are associated with shorter AI translations. Absolute-residual models identify terms associated with larger AI-human length divergence.

Greek source terms predicting signed length residuals

Features: 176; ridge alpha: 5.878; cross-validated R^2: -0.177.

Positive term	Coefficient	Docs	Mean present	Mean absent
δια του	1.4742	10	4.397	-0.489
και	1.3115	66	0.471	-0.914
εν τη	1.1400	7	4.940	-0.372
γαρ	1.0939	9	2.853	-0.282
εστι δε	1.0256	5	6.484	-0.341
κωμη	0.9959	3	7.178	-0.222
εξ	0.8557	3	8.108	-0.251
εστι δε και	0.8055	3	8.415	-0.260
ου	0.7772	14	1.404	-0.229
πολιτης	0.7710	18	2.698	-0.592
πολιχνιον	0.7650	3	6.513	-0.201
εν	0.7641	37	1.612	-0.947
ου και	0.7498	4	5.099	-0.212
δευτερα	0.7431	3	7.520	-0.233
δια	0.7327	16	2.599	-0.495
τοπος	0.7103	4	5.599	-0.233
δε και	0.6795	8	1.283	-0.112
οι	0.6611	11	2.606	-0.322
θηλυκως	0.6599	5	3.576	-0.188
και θηλυκως	0.6599	5	3.576	-0.188

Negative term	Coefficient	Docs	Mean present	Mean absent
το εθνικον	-1.1479	56	-1.022	1.300
εθνικον	-1.1479	56	-1.022	1.300
θρακης	-1.0957	5	-4.720	0.248
εστι και	-0.9349	20	-1.175	0.294
πολις θρακης	-0.8533	4	-4.196	0.175
το	-0.8147	70	-0.226	0.528
εθνος	-0.6921	8	-3.215	0.280
ηροδοτος	-0.6531	9	-2.396	0.237
τον	-0.6304	10	-1.845	0.205
τινες	-0.6182	6	-2.756	0.176
ην	-0.5867	13	-1.042	0.156
καππαδοκιας	-0.5707	3	-3.472	0.107
μετα	-0.5641	3	-8.518	0.263
φησιν	-0.5551	10	-0.950	0.106
οτι	-0.5512	5	-2.465	0.130
ελλανικος	-0.5107	4	-7.073	0.295
προς τη	-0.4981	4	-1.941	0.081
τα	-0.4683	12	-0.806	0.110
πλησιον	-0.4650	8	-0.869	0.076
τινες δε	-0.4595	3	-3.406	0.105

Greek source terms predicting large absolute residuals

Features: 176; ridge alpha: 2.894; cross-validated R^2: -0.100.

Positive term	Coefficient	Docs	Mean present	Mean absent
και	2.6470	66	4.220	2.285
γαρ	2.3783	9	8.320	3.092
δε	1.8080	46	4.717	2.579
δια του	1.6549	10	8.239	3.042
εστι	1.4411	24	5.621	2.912
την	1.3811	13	6.345	3.146
εν	1.3246	37	5.028	2.701
δια	1.2928	16	6.060	3.086
τριτη	1.0184	3	9.333	3.384
εκαλειτο	0.9973	7	6.937	3.308
εστι δε	0.9960	5	8.082	3.324
εν τη	0.9411	7	5.731	3.399
της	0.8495	35	5.148	2.708
αλεξανδρος	0.8488	3	7.088	3.453
θρακης	0.8328	5	4.720	3.501
παρα	0.8289	7	7.289	3.282
κωμη	0.7926	3	7.178	3.450
ει	0.7884	4	9.661	3.308
εν τω περι	0.7864	4	8.322	3.364
τω περι	0.7864	4	8.322	3.364

Negative term	Coefficient	Docs	Mean present	Mean absent
δια το	-0.7735	3	0.755	3.649
του δε	-0.7612	3	1.444	3.628
νησος	-0.7057	9	2.853	3.632
δευτερω	-0.6642	3	1.129	3.637
παυσανιας	-0.6614	4	1.906	3.631
στραβων	-0.6280	15	3.316	3.606
οι	-0.6047	11	3.856	3.526
τους	-0.5753	4	2.041	3.625
αυτην	-0.5617	9	2.477	3.669
παιδος	-0.5508	4	2.592	3.603
πλησιον	-0.5048	8	2.754	3.632
θηλυκον και	-0.4954	3	1.969	3.611
τινες	-0.4764	6	2.821	3.609
ιταλιας	-0.4585	3	1.561	3.624
μια	-0.4485	3	2.738	3.588
φασι	-0.4472	8	3.728	3.548
προς τω	-0.4436	4	2.488	3.607
εκαταιος	-0.4427	14	3.402	3.588
και του	-0.4413	4	2.685	3.599
ηροδοτος	-0.4359	9	3.298	3.588

AI English terms predicting large absolute residuals

Features: 394; ridge alpha: 1.425; cross-validated R^2: -0.031.

Positive term	Coefficient	Docs	Mean present	Mean absent
with	2.6255	19	6.424	2.891
for	2.3236	12	6.255	3.195
or	2.1956	14	6.818	3.032
is a	2.0062	5	9.693	3.239
and	1.8737	56	4.506	2.361
on	1.8338	12	6.835	3.116
too	1.7769	11	7.678	3.053
at	1.7645	5	9.674	3.240
usage	1.7433	4	8.835	3.342
is	1.7338	89	3.729	2.208
caria	1.6752	3	10.482	3.348
also	1.5746	43	4.968	2.501
is a city	1.5613	4	9.586	3.311
he	1.5591	5	7.628	3.348
there is	1.5495	27	5.340	2.904
to	1.4753	34	4.980	2.831
form	1.4446	21	5.506	3.045
iota	1.4120	6	7.117	3.335
diphthong	1.3457	3	11.817	3.307
a	1.3230	89	3.838	1.331

Negative term	Coefficient	Docs	Mean present	Mean absent
is called	-1.2418	18	2.376	3.822
an	-1.0508	21	2.885	3.742
are	-0.9696	26	3.781	3.485
it in his	-0.8709	3	0.622	3.653
for the	-0.8249	6	2.356	3.639
is called a	-0.7721	12	1.928	3.785
called a	-0.7721	12	1.928	3.785
of the	-0.7527	46	3.834	3.331
book of the	-0.7506	11	2.525	3.690
a city	-0.7331	65	3.678	3.347
as	-0.7224	40	4.083	3.215
the city	-0.6882	6	2.706	3.617
an island	-0.6866	10	2.738	3.654
near	-0.6755	16	3.038	3.662
ethnic is	-0.6603	54	3.893	3.174
ethnic	-0.6583	61	3.818	3.162
which is	-0.6456	3	2.126	3.606
the ethnic is	-0.6373	53	3.891	3.191
not	-0.6336	7	2.192	3.665
pausanias	-0.6267	4	1.906	3.631

Human English terms predicting large absolute residuals

Features: 336; ridge alpha: 2.894; cross-validated R^2: -0.124.

Positive term	Coefficient	Docs	Mean present	Mean absent
with	2.1070	16	6.757	2.953
and	1.5299	52	4.595	2.444
on	1.4878	16	5.542	3.185
the	1.4291	90	3.784	1.566
at	1.1846	6	8.809	3.227
there is	1.0893	24	5.593	2.921
also	1.0594	43	4.616	2.767
or	1.0511	18	6.688	2.876
is with	1.0365	4	11.029	3.251
however	0.9843	10	7.568	3.117
karia	0.9734	3	10.482	3.348
be	0.9447	15	6.625	3.022
for	0.9240	11	5.808	3.284
this	0.9127	15	6.999	2.956
like	0.9075	5	7.957	3.331
it	0.8883	37	4.984	2.727
is also	0.8567	31	4.908	2.957
there is also	0.8560	22	5.375	3.051
when	0.8221	7	7.974	3.230
diphthong	0.7956	3	11.817	3.307

Negative term	Coefficient	Docs	Mean present	Mean absent
the city	-0.6162	6	1.987	3.663
herodotos	-0.6048	9	3.298	3.588
the ethnonym is	-0.5989	52	3.582	3.540
ethnonym is	-0.5989	52	3.582	3.540
an island	-0.5910	9	2.265	3.690
near	-0.5709	16	2.872	3.694
is a	-0.5602	31	4.004	3.364
in book	-0.5399	16	3.107	3.649
in book of	-0.5364	14	2.316	3.765
the ethnonym	-0.5261	56	3.591	3.525
are	-0.5028	25	4.552	3.232
ethnonym	-0.5006	57	3.633	3.468
an	-0.4947	22	3.210	3.661
pausanias	-0.4423	4	1.906	3.631
italy	-0.4337	3	1.561	3.624
and there is	-0.4293	3	2.505	3.595
and there	-0.4293	3	2.505	3.595
say that	-0.4137	9	2.902	3.627
it is also	-0.3960	3	1.250	3.634
island	-0.3924	14	3.234	3.616

Prompt Text

Show prompt text

You are an expert classical philologist and translator specializing in Byzantine Greek geographical texts.
You will receive Greek text from a lemma entry in Stephanos of Byzantium's Ethnika.
Translate the Greek text into clear, scholarly English.
Preserve technical terminology and place names appropriately.