gpt-4.1-2025-04-14 v1 Evaluation

Generated: 2026-07-25 12:52:27 UTC

Pairs100
Distinct lemmas100
Slope0.999
R^20.977
BLEU-419.6%
chrF++51.2%
METEOR56.9%
ROUGE-L58.7%
BERTScoreN/A
COMETN/A
BLEURTN/A

Profile version id: 2416

First translation using this version: 2026-07-16

Total usable AI runs for this version: 100 runs across 100 lemmas

Created: 2026-07-16 06:54:03.500756+10:00

Active: True

Notes: Model timeline evaluation: GPT-4.1 (2025-04-14) using prompt text from gpt-5.5 v1; approved-human-only for the 100-row Kappa corpus.

Prompt length scatter plot

Length Regression

FormulaAI words = intercept + slope * human words
Slope0.999
Intercept4.615
R^20.977
P-value7.20e-82
Pearson r0.988
Mean source words30.400
Length fallback rows0
Mean human words44.160
Mean AI words48.730
Mean absolute length residual4.568
Mean absolute percent error15.7%

Metric vs Passage Length

These plots test whether score changes with source passage length for this prompt version. The dashed line is an ordinary least-squares fit; the annotation reports n, r, R^2, p-value, and slope.

BLEU-4 vs passage length
BLEU-4: positive; r = 0.062, R^2 = 0.004, p = 0.5386.
chrF++ vs passage length
chrF++: positive; r = 0.120, R^2 = 0.014, p = 0.2351.
METEOR vs passage length
METEOR: negative; r = -0.124, R^2 = 0.015, p = 0.2198.
ROUGE-L vs passage length
ROUGE-L: negative; r = -0.132, R^2 = 0.018, p = 0.1892.
Trigram precision vs passage length
Trigram precision: positive; r = 0.057, R^2 = 0.003, p = 0.5764.
Trigram recall vs passage length
Trigram recall: positive; r = 0.023, R^2 = 5.52e-04, p = 0.8165.
Trigram F1 vs passage length
Trigram F1: positive; r = 0.042, R^2 = 0.002, p = 0.6795.
Trigram Jaccard vs passage length
Trigram Jaccard: positive; r = 0.009, R^2 = 7.75e-05, p = 0.9307.
Metric Rows Pattern Pearson r R^2 P-value Slope Status
BLEU-4 100 positive 0.062 0.004 0.5386 0.00025 ok
chrF++ 100 positive 0.120 0.014 0.2351 0.00038 ok
METEOR 100 negative -0.124 0.015 0.2198 -0.00060 ok
ROUGE-L 100 negative -0.132 0.018 0.1892 -0.00053 ok
BERTScore 0 not enough data N/A N/A N/A N/A needs at least two rows with metric scores
COMET 0 not enough data N/A N/A N/A N/A needs at least two rows with metric scores
BLEURT 0 not enough data N/A N/A N/A N/A needs at least two rows with metric scores
Trigram precision 100 positive 0.057 0.003 0.5764 0.00025 ok
Trigram recall 100 positive 0.023 5.52e-04 0.8165 0.00011 ok
Trigram F1 100 positive 0.042 0.002 0.6795 0.00019 ok
Trigram Jaccard 100 positive 0.009 7.75e-05 0.9307 0.00003 ok

Translation Similarity Metrics

Mean BLEU-419.6%
Mean chrF++51.2%
Mean METEOR56.9%
Mean ROUGE-L58.7%
Mean BERTScoreN/A
Mean COMETN/A
Mean BLEURTN/A
Legacy smoothed corpus BLEU24.4%
Unigram F162.5%
Bigram F132.6%
Trigram precision17.7%
Trigram recall19.7%
Trigram F118.7%
Trigram Jaccard10.3%
4-gram F111.3%

Largest Length Residuals

Lemma ID Headword Human words AI words Residual BLEU-4 chrF++ METEOR ROUGE-L BERTScore COMET BLEURT Human Run Model
2484 Καρία 270 254 -20.341 18.0% 48.4% 40.6% 48.5% N/A N/A N/A reviewed/approved 13281 gpt-4.1-2025-04-14
7264 Κωλιάς 61 82 16.447 9.1% 41.5% 44.5% 39.2% N/A N/A N/A reviewed/approved 13422 gpt-4.1-2025-04-14
2468 Καπετώλιον 138 157 14.525 9.9% 40.0% 34.7% 41.4% N/A N/A N/A reviewed/approved 13272 gpt-4.1-2025-04-14
3288 Κορώνεια 110 129 14.497 17.4% 50.5% 50.5% 54.2% N/A N/A N/A reviewed/approved 13344 gpt-4.1-2025-04-14
2329 Κάλπη 57 76 14.443 20.4% 49.6% 51.6% 52.2% N/A N/A N/A reviewed/approved 13236 gpt-4.1-2025-04-14
7266 Κώμη 79 96 12.465 10.0% 43.1% 38.9% 36.6% N/A N/A N/A reviewed/approved 13428 gpt-4.1-2025-04-14
2115 Καλὴ ἀκτή 64 81 12.450 13.9% 40.7% 51.5% 49.7% N/A N/A N/A reviewed/approved 13209 gpt-4.1-2025-04-14
2055 Καβασσός 109 102 -11.504 18.4% 51.3% 45.0% 46.4% N/A N/A N/A reviewed/approved 13137 gpt-4.1-2025-04-14
2477 Καρδαμύλη 43 59 11.429 20.1% 58.8% 73.0% 56.9% N/A N/A N/A reviewed/approved 13278 gpt-4.1-2025-04-14
2609 Κάσπειρος 135 129 -10.478 20.1% 55.0% 49.1% 56.8% N/A N/A N/A reviewed/approved 13305 gpt-4.1-2025-04-14
2455 Κάναστρον 60 55 -9.554 14.9% 47.7% 52.6% 52.2% N/A N/A N/A reviewed/approved 13266 gpt-4.1-2025-04-14
2604 Καρχηδών 117 131 9.504 24.9% 57.4% 49.5% 56.8% N/A N/A N/A reviewed/approved 13296 gpt-4.1-2025-04-14
2465 Κάνωπος 83 96 8.469 32.5% 65.6% 80.3% 74.9% N/A N/A N/A reviewed/approved 13269 gpt-4.1-2025-04-14
2346 Κάναι 77 90 8.463 31.2% 57.3% 73.2% 73.1% N/A N/A N/A reviewed/approved 13263 gpt-4.1-2025-04-14
2605 Κάσιον 56 69 8.442 12.0% 52.0% 66.7% 57.6% N/A N/A N/A reviewed/approved 13299 gpt-4.1-2025-04-14
2342 Κάμιρος 54 67 8.440 29.2% 63.4% 76.9% 69.4% N/A N/A N/A reviewed/approved 13257 gpt-4.1-2025-04-14
7262 Κῶβρυς 22 35 8.408 6.7% 41.4% 54.8% 49.1% N/A N/A N/A reviewed/approved 13416 gpt-4.1-2025-04-14
3496 Κοτιάειον 71 68 -7.543 17.3% 46.2% 56.8% 54.7% N/A N/A N/A reviewed/approved 13347 gpt-4.1-2025-04-14
2597 Καρπασία 112 109 -7.501 19.9% 57.5% 56.5% 67.0% N/A N/A N/A reviewed/approved 13284 gpt-4.1-2025-04-14
2328 Καλλίπολις 43 55 7.429 5.2% 46.0% 43.4% 53.1% N/A N/A N/A reviewed/approved 13233 gpt-4.1-2025-04-14
7250 Κυρταία 40 52 7.426 16.6% 41.7% 41.7% 45.7% N/A N/A N/A reviewed/approved 13380 gpt-4.1-2025-04-14
2085 Καλάσιρις 13 25 7.398 6.0% 43.0% 33.8% 36.8% N/A N/A N/A reviewed/approved 13197 gpt-4.1-2025-04-14
2630 Κάτρη 23 21 -6.591 9.4% 42.6% 47.6% 59.1% N/A N/A N/A reviewed/approved 13338 gpt-4.1-2025-04-14
2087 Καλαύρεια 30 28 -6.584 21.1% 54.5% 59.1% 65.5% N/A N/A N/A reviewed/approved 13203 gpt-4.1-2025-04-14
2079 Καισάρεια 39 37 -6.575 38.5% 58.5% 67.8% 68.4% N/A N/A N/A reviewed/approved 13179 gpt-4.1-2025-04-14
2623 Κασώριον 20 31 6.406 9.7% 48.2% 49.7% 43.1% N/A N/A N/A reviewed/approved 13317 gpt-4.1-2025-04-14
2599 Καρπήσιοι 11 10 -5.604 13.1% 38.7% 34.1% 47.6% N/A N/A N/A reviewed/approved 13287 gpt-4.1-2025-04-14
2624 Κατάβαθμος 20 19 -5.594 35.9% 66.6% 64.2% 71.8% N/A N/A N/A reviewed/approved 13320 gpt-4.1-2025-04-14
2330 Καλύβη 24 23 -5.590 19.9% 46.3% 58.6% 68.1% N/A N/A N/A reviewed/approved 13239 gpt-4.1-2025-04-14
3530 Κριώα 26 25 -5.588 15.4% 37.6% 28.5% 47.1% N/A N/A N/A reviewed/approved 13353 gpt-4.1-2025-04-14

Residual Predictors

Positive signed-residual terms are associated with AI translations longer than expected from the human length; negative signed-residual terms are associated with shorter AI translations. Absolute-residual models identify terms associated with larger AI-human length divergence.

Greek source terms predicting signed length residuals

Features: 176; ridge alpha: 8.377; cross-validated R^2: -0.381.

Positive termCoefficientDocsMean presentMean absent
και 2.1089 66 0.639 -1.241
κωμη 1.1959 3 12.115 -0.375
εστι δε 1.0218 5 9.076 -0.478
δια του 1.0135 10 3.772 -0.419
εκ 0.9248 6 5.611 -0.358
εστι δε και 0.8482 3 11.810 -0.365
γαρ 0.8224 9 2.177 -0.215
θηλυκως 0.7048 5 5.049 -0.266
και θηλυκως 0.7048 5 5.049 -0.266
εν τη 0.6936 7 4.005 -0.301
παρα 0.6627 7 1.917 -0.144
δυο 0.6450 4 7.456 -0.311
του δε 0.6367 3 5.769 -0.178
εις 0.6359 7 3.656 -0.275
λεγεται 0.6298 9 2.592 -0.256
αιγυπτου 0.6222 3 5.764 -0.178
του 0.6020 37 1.615 -0.949
αι 0.5781 3 8.459 -0.262
χωρας 0.5519 4 3.661 -0.153
την 0.5515 13 1.880 -0.281
Negative termCoefficientDocsMean presentMean absent
και του -0.7306 4 -6.066 0.253
περι -0.7204 11 -1.720 0.213
καππαδοκιας -0.6777 3 -5.890 0.182
το -0.6555 70 0.323 -0.753
το εθνικον -0.6434 56 -0.281 0.357
εθνικον -0.6434 56 -0.281 0.357
εθνος -0.6243 8 -3.547 0.308
προς τη -0.6014 4 -3.840 0.160
εκαλειτο -0.5913 7 -4.928 0.371
ελλανικος -0.5760 4 -9.979 0.416
τα -0.5458 12 -1.008 0.137
αλλ -0.5326 3 -5.526 0.171
το θηλυκον -0.4999 6 -3.680 0.235
μετα -0.4986 3 -10.779 0.333
μεταξυ -0.4937 5 -1.558 0.082
τους -0.4636 4 -2.569 0.107
τινες δε -0.4548 3 -4.243 0.131
εκαταιος -0.4363 14 -1.069 0.174
μια -0.4350 3 -3.243 0.100
νησος -0.4232 9 -1.343 0.133

Greek source terms predicting large absolute residuals

Features: 176; ridge alpha: 2.031; cross-validated R^2: -0.007.

Positive termCoefficientDocsMean presentMean absent
γαρ 4.3253 9 11.583 3.874
δε 2.7339 46 5.881 3.449
την 2.6942 13 7.727 4.096
και 2.5095 66 5.235 3.274
εστι 2.4561 24 6.799 3.863
κωμη 1.9772 3 12.115 4.334
τα 1.7987 12 9.081 3.952
εστι δε 1.5538 5 10.076 4.278
τε 1.5279 5 9.074 4.331
περι 1.4659 11 7.048 4.261
ως 1.4478 52 5.220 3.861
ακρα 1.3715 4 9.250 4.373
δια 1.3401 16 6.932 4.117
εστι δε και 1.3368 3 11.810 4.344
ηρωδιανος 1.3274 3 13.819 4.282
εν 1.3188 37 5.687 3.910
τα τοπικα 1.2418 3 11.180 4.363
τοπικα 1.2418 3 11.180 4.363
ομηρος 1.1954 3 10.458 4.386
θεοπομπος 1.1879 5 7.107 4.434
Negative termCoefficientDocsMean presentMean absent
στραβων -1.8705 15 3.235 4.803
παυσανιας -1.4992 4 1.777 4.684
τους -1.3262 4 2.783 4.642
ης -1.3078 3 1.509 4.662
ουδετερως -1.2055 3 1.560 4.661
λιβυης -1.0892 4 4.736 4.561
πολις προς -1.0663 3 1.859 4.652
παιδος -1.0550 4 3.181 4.626
πολυβιος -1.0434 5 2.078 4.699
ην -1.0138 13 3.966 4.658
εκαταιος -1.0099 14 3.886 4.679
αυτην -0.9828 9 3.438 4.680
φησι -0.8948 12 4.829 4.532
το εθνικον -0.8909 56 4.532 4.613
εθνικον -0.8909 56 4.532 4.613
αλλη -0.8488 4 4.263 4.580
ιβηριας -0.8355 3 3.886 4.589
νησος -0.8297 9 3.733 4.650
οτι -0.7947 5 3.698 4.614
φησι το -0.7783 3 0.851 4.683

AI English terms predicting large absolute residuals

Features: 438; ridge alpha: 1.425; cross-validated R^2: -0.045.

Positive termCoefficientDocsMean presentMean absent
for 4.6371 15 8.730 3.833
is 2.4774 90 4.757 2.861
usage 2.3926 4 13.425 4.199
iota 2.1585 6 11.320 4.137
among 1.9108 6 10.622 4.181
promontory 1.8554 6 9.530 4.251
there is 1.7947 26 6.548 3.872
diphthong 1.7892 3 14.123 4.272
and 1.7676 54 5.758 3.171
or 1.7626 18 7.667 3.887
people 1.6834 8 8.373 4.237
from 1.6716 41 6.175 3.451
a promontory 1.6598 3 13.499 4.292
on 1.6389 16 7.184 4.069
with 1.6250 18 7.490 3.926
there is also 1.6188 22 6.144 4.123
thus 1.6055 4 13.934 4.178
also 1.5737 43 5.235 4.064
place 1.5647 10 7.401 4.253
derived 1.5494 5 12.045 4.174
Negative termCoefficientDocsMean presentMean absent
name -1.5632 20 4.451 4.597
strabo -1.2487 15 3.235 4.803
pausanias -1.2039 4 1.777 4.684
after -1.1349 13 4.257 4.614
a city -1.1289 66 4.451 4.794
strabo in the -1.1112 5 1.496 4.729
strabo in -1.0727 7 2.482 4.725
also a city -1.0203 7 5.199 4.520
of the -0.9927 44 4.596 4.546
ethnic designation is -0.9753 30 4.283 4.690
designation is -0.9753 30 4.283 4.690
hecataeus mentions -0.9236 5 1.725 4.717
they -0.9118 8 3.499 4.661
formed -0.9066 6 2.504 4.700
arcadia -0.8937 3 1.578 4.660
city -0.8843 69 4.483 4.757
hecataeus -0.8818 14 3.886 4.679
named after -0.8736 10 3.203 4.719
the ethnicon is -0.8719 9 2.678 4.755
the ethnicon -0.8719 9 2.678 4.755

Human English terms predicting large absolute residuals

Features: 336; ridge alpha: 2.031; cross-validated R^2: -0.049.

Positive termCoefficientDocsMean presentMean absent
with 2.7791 16 7.923 3.929
and 2.2563 52 5.847 3.182
is 2.2437 87 4.824 2.851
there is 1.9334 24 6.838 3.851
is also 1.8092 31 6.270 3.803
there is also 1.7644 22 6.778 3.944
on 1.6833 16 7.125 4.081
there 1.4532 34 6.384 3.632
is also a 1.4390 14 8.496 3.928
as 1.4309 60 5.214 3.598
or 1.4170 18 8.338 3.740
the 1.4073 90 4.691 3.459
also a 1.4017 15 7.958 3.969
among 1.4015 4 11.666 4.272
is from 1.2857 12 6.919 4.247
place 1.2789 6 7.837 4.359
form 1.2547 11 8.768 4.049
long 1.2545 3 15.777 4.221
among the 1.2292 4 11.666 4.272
with the 1.1797 5 11.660 4.195
Negative termCoefficientDocsMean presentMean absent
strabo -1.2384 15 3.235 4.803
was from -1.2123 4 1.028 4.715
was -0.9994 20 4.606 4.558
it is also -0.9033 3 0.761 4.686
the ethnonym -0.8599 56 4.530 4.616
also called -0.8572 6 3.273 4.650
pausanias -0.8140 4 1.777 4.684
of the -0.7751 33 5.324 4.195
strabo in -0.7625 3 0.847 4.683
ethnonym -0.7540 57 4.652 4.456
pausanias book -0.7303 3 1.507 4.662
is the -0.7204 3 1.189 4.672
which -0.7025 20 4.345 4.623
mount -0.6988 4 3.275 4.622
they -0.6670 12 4.598 4.564
strabo book the -0.6563 3 0.514 4.693
the ethnonym is -0.6546 52 4.688 4.438
ethnonym is -0.6546 52 4.688 4.438
an island -0.6514 9 3.409 4.682
book the ethnonym -0.6432 7 2.804 4.701

Prompt Text

Show prompt text 
You are an expert classical philologist and translator specializing in Byzantine Greek geographical texts.
You will receive Greek text from a lemma entry in Stephanos of Byzantium's Ethnika.
Translate the Greek text into clear, scholarly English.
Preserve technical terminology and place names appropriately.
Generated by generate_translation_prompt_evaluation.py.