legacy_scholarly v1 Evaluation

Generated: 2026-06-10 11:07:53 UTC

Pairs9
Distinct lemmas9
Slope1.085
R^20.971
BLEU31.0%
Trigram F126.2%
ROUGE-L F160.3%
chrF63.1%

Profile version id: 1

First translation using this version: 2026-02-14

Total usable AI runs for this version: 30 runs across 30 lemmas

Created: 2026-01-21 23:08:03.772891+11:00

Active:

Notes: Initial prompt - minimal guidance

Prompt length scatter plot

Length Regression

FormulaAI words = intercept + slope * human words
Slope1.085
Intercept2.690
R^20.971
P-value1.29e-06
Pearson r0.985
Mean human words45.444
Mean AI words52.000
Mean absolute length residual3.191
Mean absolute percent error16.1%

Translation Similarity Metrics

Smoothed corpus BLEU31.0%
Mean sentence BLEU32.8%
Unigram F164.8%
Bigram F138.9%
Trigram precision24.4%
Trigram recall28.1%
Trigram F126.2%
Trigram Jaccard15.0%
4-gram F118.0%
Mean ROUGE-L F160.3%
Mean chrF63.1%

Largest Length Residuals

Lemma ID Headword Human words AI words Residual Sentence BLEU Trigram F1 ROUGE-L F1 Human Run Model
2115 Καλὴ ἀκτή 63 81 9.951 21.3% 18.6% 37.5% reviewed/approved 11 gpt-5.2
2346 Κάναι 77 81 -5.240 35.6% 29.9% 64.6% reviewed/approved 13 gpt-5.2
2470 Καππαδοκία 68 72 -4.474 42.7% 35.3% 74.3% reviewed/approved 1 gpt-5.2
2078 Καινύς 21 22 -3.476 55.2% 46.2% 74.4% reviewed/approved 32 gpt-5.2
7244 Κυρήνη 54 64 2.717 28.8% 22.8% 54.2% reviewed/approved 3 gpt-5.2
2059 Καδμεία 19 25 1.694 22.2% 15.0% 50.0% reviewed/approved 8 gpt-5.2
2077 Καιρή 15 18 -0.966 27.1% 13.8% 66.7% reviewed/approved 31 gpt-5.2
7251 Κύρτος 64 72 -0.134 32.5% 25.8% 61.8% reviewed/approved 4 gpt-5.2
2083 Καλαμένθη 28 33 -0.072 30.2% 21.1% 59.0% reviewed/approved 18 gpt-5.2

Residual Predictors

Residual term analysis was not run for this prompt: needs at least 30 paired rows.

Prompt Text

Show prompt text
You are an expert classical philologist and translator specializing in Byzantine Greek geographical texts.
You will receive Greek text from a lemma entry in Stephanos of Byzantium's Ethnika.
Translate the Greek text into clear, scholarly English.
Preserve technical terminology and place names appropriately.