gpt-5.3-chat-latest v1 Evaluation

Generated: 2026-07-25 12:53:30 UTC

Pairs100

Distinct lemmas100

Slope0.987

R^20.962

BLEU-423.0%

chrF++52.8%

METEOR59.5%

ROUGE-L61.1%

BERTScoreN/A

COMETN/A

BLEURTN/A

Profile version id: 2334

First translation using this version: 2026-07-15

Total usable AI runs for this version: 100 runs across 100 lemmas

Created: 2026-07-15 12:28:27.643960+10:00

Active: True

Notes: Model timeline evaluation: GPT-5.3 Chat using prompt text from gpt-5.5 v1; approved-human-only for the 100-row Kappa corpus.

Length Regression

Formula	AI words = intercept + slope * human words
Slope	0.987
Intercept	3.014
R^2	0.962
P-value	3.74e-71
Pearson r	0.981
Mean source words	30.400
Length fallback rows	0
Mean human words	44.160
Mean AI words	46.620
Mean absolute length residual	4.012
Mean absolute percent error	11.4%

Metric vs Passage Length

These plots test whether score changes with source passage length for this prompt version. The dashed line is an ordinary least-squares fit; the annotation reports n, r, R^2, p-value, and slope.

BLEU-4 vs passage length — BLEU-4: positive; r = 0.020, R^2 = 4.10e-04, p = 0.8415.

chrF++ vs passage length — chrF++: positive; r = 0.038, R^2 = 0.001, p = 0.7038.

METEOR vs passage length — METEOR: negative; r = -0.024, R^2 = 5.58e-04, p = 0.8155.

ROUGE-L vs passage length — ROUGE-L: negative; r = -0.064, R^2 = 0.004, p = 0.5292.

Trigram precision vs passage length — Trigram precision: positive; r = 0.021, R^2 = 4.36e-04, p = 0.8366.

Trigram recall vs passage length — Trigram recall: positive; r = 0.019, R^2 = 3.80e-04, p = 0.8474.

Trigram F1 vs passage length — Trigram F1: positive; r = 0.021, R^2 = 4.34e-04, p = 0.8369.

Trigram Jaccard vs passage length — Trigram Jaccard: positive; r = 3.08e-04, R^2 = 9.47e-08, p = 0.9976.

Metric	Rows	Pattern	Pearson r	R^2	P-value	Slope	Status
BLEU-4	100	positive	0.020	4.10e-04	0.8415	0.00010	ok
chrF++	100	positive	0.038	0.001	0.7038	0.00013	ok
METEOR	100	negative	-0.024	5.58e-04	0.8155	-0.00012	ok
ROUGE-L	100	negative	-0.064	0.004	0.5292	-0.00027	ok
BERTScore	0	not enough data	N/A	N/A	N/A	N/A	needs at least two rows with metric scores
COMET	0	not enough data	N/A	N/A	N/A	N/A	needs at least two rows with metric scores
BLEURT	0	not enough data	N/A	N/A	N/A	N/A	needs at least two rows with metric scores
Trigram precision	100	positive	0.021	4.36e-04	0.8366	0.00010	ok
Trigram recall	100	positive	0.019	3.80e-04	0.8474	0.00009	ok
Trigram F1	100	positive	0.021	4.34e-04	0.8369	0.00010	ok
Trigram Jaccard	100	positive	3.08e-04	9.47e-08	0.9976	9.77e-07	ok

Translation Similarity Metrics

Mean BLEU-4	23.0%
Mean chrF++	52.8%
Mean METEOR	59.5%
Mean ROUGE-L	61.1%
Mean BERTScore	N/A
Mean COMET	N/A
Mean BLEURT	N/A
Legacy smoothed corpus BLEU	29.1%
Unigram F1	65.3%
Bigram F1	36.7%
Trigram precision	22.1%
Trigram recall	23.4%
Trigram F1	22.7%
Trigram Jaccard	12.8%
4-gram F1	14.8%

Largest Length Residuals

Lemma ID	Headword	Human words	AI words	Residual	BLEU-4	chrF++	METEOR	ROUGE-L	BERTScore	COMET	BLEURT	Human	Run	Model
2465	Κάνωπος	83	147	62.027	23.3%	60.5%	78.8%	59.1%	N/A	N/A	N/A	reviewed/approved	11167	gpt-5.3-chat-latest
2484	Καρία	270	251	-18.626	18.7%	50.1%	52.7%	57.6%	N/A	N/A	N/A	reviewed/approved	11182	gpt-5.3-chat-latest
7266	Κώμη	79	97	15.977	3.5%	34.2%	32.1%	33.0%	N/A	N/A	N/A	reviewed/approved	11320	gpt-5.3-chat-latest
2056	Καβειρία	112	101	-12.609	16.6%	52.6%	54.2%	61.0%	N/A	N/A	N/A	reviewed/approved	11037	gpt-5.3-chat-latest
7267	Κωνώπη	64	78	11.789	36.4%	51.6%	46.9%	54.9%	N/A	N/A	N/A	reviewed/approved	11324	gpt-5.3-chat-latest
2116	Κάληρος	40	33	-9.512	21.6%	53.1%	56.2%	54.8%	N/A	N/A	N/A	reviewed/approved	11111	gpt-5.3-chat-latest
2603	Κάρυστος	175	185	9.182	27.8%	57.1%	66.4%	62.8%	N/A	N/A	N/A	reviewed/approved	11193	gpt-5.3-chat-latest
2328	Καλλίπολις	43	54	8.525	8.4%	48.4%	44.7%	47.4%	N/A	N/A	N/A	reviewed/approved	11131	gpt-5.3-chat-latest
2609	Κάσπειρος	135	128	-8.320	35.4%	62.3%	61.7%	66.2%	N/A	N/A	N/A	reviewed/approved	11204	gpt-5.3-chat-latest
7254	Κύτα	84	94	8.040	27.9%	56.7%	56.0%	58.4%	N/A	N/A	N/A	reviewed/approved	11286	gpt-5.3-chat-latest
2057	Καβελλιών	34	44	7.412	16.1%	47.5%	47.0%	43.6%	N/A	N/A	N/A	reviewed/approved	11038	gpt-5.3-chat-latest
2626	Κατάνη	95	90	-6.822	33.0%	56.3%	63.4%	60.5%	N/A	N/A	N/A	reviewed/approved	11222	gpt-5.3-chat-latest
2342	Κάμιρος	54	63	6.664	23.4%	60.8%	66.9%	66.7%	N/A	N/A	N/A	reviewed/approved	11157	gpt-5.3-chat-latest
2055	Καβασσός	109	104	-6.646	21.6%	51.5%	52.0%	49.8%	N/A	N/A	N/A	reviewed/approved	11034	gpt-5.3-chat-latest
7255	Κυτέριον	22	31	6.262	14.6%	46.3%	48.7%	52.8%	N/A	N/A	N/A	reviewed/approved	11289	gpt-5.3-chat-latest
7253	Κυρτώνιος	18	27	6.212	4.8%	35.7%	36.4%	35.6%	N/A	N/A	N/A	reviewed/approved	11282	gpt-5.3-chat-latest
7245	Κύρης	13	22	6.149	8.7%	34.9%	50.5%	45.7%	N/A	N/A	N/A	reviewed/approved	11259	gpt-5.3-chat-latest
2624	Κατάβαθμος	20	17	-5.763	29.6%	58.3%	54.9%	64.9%	N/A	N/A	N/A	reviewed/approved	11213	gpt-5.3-chat-latest
7261	Κύψελα	34	31	-5.588	43.8%	65.0%	73.0%	76.9%	N/A	N/A	N/A	reviewed/approved	11306	gpt-5.3-chat-latest
2081	Καλάθη	49	46	-5.399	25.4%	55.6%	70.0%	73.7%	N/A	N/A	N/A	reviewed/approved	11082	gpt-5.3-chat-latest
2075	Καινίνη	15	13	-4.826	46.0%	63.3%	73.6%	78.6%	N/A	N/A	N/A	reviewed/approved	11064	gpt-5.3-chat-latest
2080	Καλαβρία	18	16	-4.788	21.9%	52.2%	39.5%	47.1%	N/A	N/A	N/A	reviewed/approved	11079	gpt-5.3-chat-latest
2335	Κάλυτις	26	24	-4.688	5.9%	46.6%	39.2%	56.0%	N/A	N/A	N/A	reviewed/approved	11153	gpt-5.3-chat-latest
2079	Καισάρεια	39	37	-4.525	30.7%	56.4%	67.7%	68.4%	N/A	N/A	N/A	reviewed/approved	11075	gpt-5.3-chat-latest
3288	Κορώνεια	110	116	4.366	16.2%	51.3%	49.6%	52.9%	N/A	N/A	N/A	reviewed/approved	11239	gpt-5.3-chat-latest
2623	Κασώριον	20	27	4.237	28.1%	55.8%	59.7%	55.3%	N/A	N/A	N/A	reviewed/approved	11212	gpt-5.3-chat-latest
2599	Καρπήσιοι	11	10	-3.876	22.3%	55.5%	54.0%	57.1%	N/A	N/A	N/A	reviewed/approved	11187	gpt-5.3-chat-latest
2061	Καδούσιοι	14	13	-3.839	47.8%	61.8%	77.2%	74.1%	N/A	N/A	N/A	reviewed/approved	11050	gpt-5.3-chat-latest
2115	Καλὴ ἀκτή	64	70	3.789	15.1%	41.5%	55.4%	52.2%	N/A	N/A	N/A	reviewed/approved	11108	gpt-5.3-chat-latest
2620	Κάσταξ	20	19	-3.763	8.2%	43.5%	50.8%	56.4%	N/A	N/A	N/A	reviewed/approved	11205	gpt-5.3-chat-latest

Residual Predictors

Positive signed-residual terms are associated with AI translations longer than expected from the human length; negative signed-residual terms are associated with shorter AI translations. Absolute-residual models identify terms associated with larger AI-human length divergence.

Greek source terms predicting signed length residuals

Features: 176; ridge alpha: 412.463; cross-validated R^2: -0.223.

Positive term	Coefficient	Docs	Mean present	Mean absent
και	0.0722	66	0.101	-0.196
δια του	0.0433	10	6.159	-0.684
και το	0.0427	9	7.645	-0.756
απο	0.0330	27	2.457	-0.909
και νησος	0.0287	3	21.327	-0.660
δια	0.0270	16	3.924	-0.748
και το κτητικον	0.0263	4	15.905	-0.663
λιβυης	0.0242	4	15.108	-0.630
πολις εν	0.0228	5	11.409	-0.600
εδει	0.0227	3	17.669	-0.546
τοπος	0.0225	4	16.299	-0.679
ποσειδωνος	0.0219	3	18.714	-0.579
κτητικον	0.0179	7	4.803	-0.362
το κτητικον	0.0179	7	4.803	-0.362
νησος	0.0175	9	6.263	-0.619
κωμη	0.0172	3	5.764	-0.178
εστι	0.0167	24	1.469	-0.464
του	0.0165	37	1.583	-0.930
απο του	0.0156	9	4.135	-0.409
εν	0.0136	37	2.112	-1.240

Negative term	Coefficient	Docs	Mean present	Mean absent
δε	-0.0195	46	0.454	-0.386
εθνος	-0.0195	8	-4.920	0.428
εκαταιος	-0.0188	14	-2.228	0.363
εκαλειτο	-0.0183	7	-4.012	0.302
πλησιον	-0.0182	8	-3.017	0.262
και του	-0.0161	4	-5.412	0.225
μεταξυ	-0.0157	5	-3.116	0.164
εκαταιος ασια	-0.0149	5	-2.905	0.153
ασια	-0.0149	5	-2.905	0.153
οτι	-0.0137	5	-1.878	0.099
το θηλυκον	-0.0133	6	-5.426	0.346
χωρα	-0.0121	6	-4.694	0.300
ποταμου	-0.0120	3	-3.171	0.098
μετα	-0.0116	3	-11.256	0.348
ης	-0.0114	3	-5.274	0.163
θηλυκον και	-0.0112	3	-6.624	0.205
παυσανιας	-0.0094	4	-4.225	0.176
λυκιας	-0.0093	3	-3.383	0.105
καππαδοκιας	-0.0091	3	-3.628	0.112
τινες	-0.0091	6	-2.141	0.137

Greek source terms predicting large absolute residuals

Features: 176; ridge alpha: 24.245; cross-validated R^2: -0.239.

Positive term	Coefficient	Docs	Mean present	Mean absent
και	0.9793	66	4.663	2.749
δια του	0.6055	10	10.916	3.245
δια	0.5703	16	8.069	3.239
ουτως	0.5455	5	21.061	3.115
εστι	0.5179	24	7.056	3.051
και το	0.4992	9	9.295	3.489
απο	0.4562	27	6.847	2.963
εδει	0.4080	3	23.683	3.403
τοπος	0.4068	4	19.305	3.375
πολις εν	0.3949	5	16.067	3.377
ποσειδωνος	0.3910	3	22.638	3.436
και νησος	0.3651	3	21.327	3.476
θηλυκον	0.3644	12	11.123	3.042
και το κτητικον	0.3554	4	17.149	3.464
κτητικον	0.3342	7	14.438	3.227
το κτητικον	0.3342	7	14.438	3.227
λιβυης	0.2822	4	16.133	3.507
εστι και	0.2779	20	7.501	3.140
του	0.2418	37	5.514	3.129
εν	0.2127	37	5.630	3.062

Negative term	Coefficient	Docs	Mean present	Mean absent
ως	-0.2746	52	3.594	4.465
ηροδοτος	-0.2053	9	4.569	3.957
πολιτης	-0.1833	18	3.039	4.225
δευτερω	-0.1751	3	1.670	4.084
πολις και	-0.1745	6	1.199	4.191
ην	-0.1720	13	2.366	4.258
το εθνικον	-0.1580	56	4.435	3.473
εθνικον	-0.1580	56	4.435	3.473
και πολις	-0.1467	5	1.930	4.121
δεκατω	-0.1444	3	0.718	4.114
εκ	-0.1437	6	1.627	4.164
πολις	-0.1418	68	4.627	2.704
φησιν	-0.1170	10	4.010	4.012
του δε	-0.1133	3	0.936	4.107
της	-0.1112	35	4.160	3.932
μια	-0.1026	3	1.676	4.084
τοπικα	-0.0989	3	2.528	4.058
τα τοπικα	-0.0989	3	2.528	4.058
ος	-0.0923	6	2.529	4.106
ακρα	-0.0911	4	1.344	4.123

AI English terms predicting large absolute residuals

Features: 403; ridge alpha: 8.377; cross-validated R^2: -0.215.

Positive term	Coefficient	Docs	Mean present	Mean absent
be	1.3611	8	13.719	3.168
written	1.2538	4	17.098	3.467
written with	1.2538	4	17.098	3.467
with a	1.2434	5	16.461	3.357
and the	1.0598	17	9.133	2.963
and	0.9757	56	4.961	2.804
the possessive	0.9320	3	31.087	3.174
there	0.8816	31	6.298	2.985
is a city	0.8783	4	20.391	3.329
with	0.8718	16	7.977	3.257
adjective	0.8699	3	29.564	3.222
is a	0.8615	5	17.041	3.326
possessive	0.8459	4	23.495	3.200
usage	0.8261	5	18.043	3.273
named from	0.8186	4	21.211	3.295
have	0.8165	6	16.504	3.214
is	0.8028	88	4.147	3.022
a place in	0.8022	3	25.658	3.342
feminine	0.7577	16	10.448	2.786
to be	0.7566	7	14.785	3.201

Negative term	Coefficient	Docs	Mean present	Mean absent
the ethnic	-0.4814	55	3.105	5.120
name is	-0.4676	15	1.754	4.410
ethnic name is	-0.4631	13	1.704	4.357
ethnic	-0.4300	58	3.314	4.975
in the	-0.4221	36	3.202	4.467
the ethnic name	-0.4123	15	2.058	4.357
ethnic name	-0.4123	15	2.058	4.357
at	-0.3797	7	0.942	4.243
are	-0.3580	21	3.585	4.125
the ethnic is	-0.3212	37	3.284	4.439
an iota	-0.3128	4	1.627	4.111
which	-0.3038	16	3.856	4.042
is called	-0.2863	17	3.480	4.121
also called	-0.2794	12	4.291	3.974
forms are	-0.2750	3	1.066	4.103
ethnic is	-0.2653	38	3.530	4.307
settlement	-0.2640	3	0.617	4.117
at the	-0.2504	3	0.372	4.124
strabo book	-0.2449	6	2.092	4.134
book of his	-0.2439	10	2.619	4.167

Human English terms predicting large absolute residuals

Features: 336; ridge alpha: 8.377; cross-validated R^2: -0.205.

Positive term	Coefficient	Docs	Mean present	Mean absent
written	1.8953	3	26.735	3.309
for	1.3333	11	12.883	2.915
there	1.2597	34	6.548	2.705
and the	1.0759	15	8.314	3.253
with	1.0632	16	8.087	3.236
a place	1.0043	3	25.658	3.342
the	0.9873	90	4.169	2.598
is written	0.9686	3	26.735	3.309
is a city	0.9055	3	23.125	3.421
feminine is	0.8999	12	12.138	2.904
poseidon	0.8903	3	22.638	3.436
the feminine is	0.8888	11	12.511	2.961
it should	0.8347	3	23.683	3.403
should	0.8347	3	23.683	3.403
is also	0.8217	31	6.624	2.838
there is also	0.7984	22	7.151	3.127
sanctuary	0.7889	3	21.811	3.461
and	0.7855	52	4.972	2.972
feminine	0.7817	16	10.448	2.786
there is	0.7781	24	6.742	3.150

Negative term	Coefficient	Docs	Mean present	Mean absent
at	-0.4357	6	0.830	4.215
of the	-0.4152	33	3.583	4.223
was	-0.3964	20	3.421	4.159
also called	-0.3817	6	1.448	4.175
city in	-0.3696	41	3.742	4.200
a city in	-0.3571	40	3.768	4.174
citizen	-0.3250	18	3.039	4.225
a citizen	-0.3250	18	3.039	4.225
the ethnonym is	-0.3062	52	4.045	3.976
ethnonym is	-0.3062	52	4.045	3.976
called	-0.2866	21	3.846	4.056
near	-0.2857	16	3.321	4.143
herodotos	-0.2821	9	4.569	3.957
in book	-0.2630	16	3.926	4.028
his	-0.2487	34	3.720	4.162
of his	-0.2443	18	3.807	4.057
citizen is a	-0.2358	15	3.216	4.152
citizen is	-0.2358	15	3.216	4.152
a citizen is	-0.2358	15	3.216	4.152
herodotos book	-0.2351	5	3.293	4.050

Prompt Text

Show prompt text

You are an expert classical philologist and translator specializing in Byzantine Greek geographical texts.
You will receive Greek text from a lemma entry in Stephanos of Byzantium's Ethnika.
Translate the Greek text into clear, scholarly English.
Preserve technical terminology and place names appropriately.