Kappa entry length versus translation quality

All and only the 100 headwords in Gabe's frozen final Kappa review tracker are shown. Quality is reference similarity for gpt-5.6-sol prompt v3, not an expert judgment of correctness. Select a metric, search for a headword, or click a point.

Official headwords100
Mean Greek source length30.4 words
Mean four-metric quality72.9%
Length–quality Pearson r-0.486

Interactive chart

Official 100Decile-stratified review setExplicit quotationDashed line: ordinary least-squares fit
Scatter plot of Greek source word count against translation reference-similarity score for the official 100 Kappa headwords.
Select a point to see its headword, length, score, and reference-page link.

Do explicit quotations predict better translation scores?

No. In this cohort, entries with an explicit quotation scored lower, not higher. Their unadjusted four-metric mean was 62.8% versus 75.8% for entries without one.

The raw difference is statistically significant under a two-sided Welch comparison: -13.0 pp (95% CI -17.7 pp to -8.2 pp; p=1.72e-06; Cohen's d=-1.16). This is a descriptive group comparison, not evidence that quotation itself caused the lower score.

Quotation entries were also much longer: 61.4 versus 21.7 Greek words on average. After a linear adjustment for source length, the quotation association was -6.8 pp (95% CI -13.2 pp to -0.4 pp; HC3 p=0.038).

The incremental predictive value is weak. In leave-one-out cross-validation, adding the quotation flag to a linear length model changed R2 from 0.187 to 0.215, while mean absolute error stayed at 8.73% versus 8.72%. The small R2 gain does not survive alternative log-length and quadratic-length specifications.

Entries with explicit quotation22 / 100
Raw mean-score difference-13.0 pp
Raw Welch p1.72e-06
Length-adjusted difference-6.8 pp
LOOCV R2 change+0.028

Score-by-score comparison

Reference-similarity scoreQuotation meanNo-quotation meanRaw differenceRaw 95% CIWelch pLength-adjusted effectAdjusted 95% CIHC3 p
Four-metric mean 62.8% 75.8% -13.0 pp -17.7 pp to -8.2 pp 1.72e-06 -6.8 pp -13.2 pp to -0.4 pp 0.038
BLEU-4 41.1% 59.6% -18.4 pp -25.2 pp to -11.6 pp 1.61e-06 -10.1 pp -19.4 pp to -0.8 pp 0.034
chrF++ 66.3% 78.0% -11.7 pp -16.1 pp to -7.4 pp 3.05e-06 -6.5 pp -12.6 pp to -0.4 pp 0.038
METEOR 71.5% 82.9% -11.4 pp -16.5 pp to -6.3 pp 7.04e-05 -5.5 pp -12.2 pp to +1.2 pp 0.104
ROUGE-L 72.4% 82.7% -10.2 pp -14.4 pp to -6.1 pp 1.13e-05 -4.9 pp -10.1 pp to +0.2 pp 0.062

Out-of-sample sensitivity to the length model

Length specificationLength-only R2Length + quotation R2ChangeLength-only MAELength + quotation MAE
Linear words 0.187 0.215 +0.028 8.73% 8.72%
Log words 0.281 0.286 +0.005 8.13% 8.17%
Quadratic words 0.230 0.229 -0.001 8.44% 8.45%

Operational definition. “Explicit quotation” means that the Greek source contains at least one balanced “…” span. This reproducible rule excludes guillemets used around individual letters or forms. It does not identify uncited paraphrases or parallel passages, and it is only a proxy for possible prior representation in model training data. The outcome is entry-level similarity to the approved house translation for one gpt-5.6-sol v3 run per entry, not an expert correctness score. The primary regression is ordinary least squares with a linear word-count term and HC3 heteroskedasticity-robust uncertainty; individual metric rows are exploratory and not multiple-comparison adjusted.

Which source features predict translation similarity?

The best out-of-sample R2 came from Greek vocabulary + log length. It explained 42.1% of held-out score variance (nested-CV R2=0.421) with mean absolute error 7.30%. Greek vocabulary + log length had the lowest MAE at 7.30% (R2=0.421).

Source structure reached R2=0.303, slightly above log length alone at 0.273, but that block includes character count, sentence count, and words per sentence and therefore carries length-adjacent signal. The standalone rarity, citation/entity, and recogniser blocks were weaker at R2=0.068, 0.060, and 0.187.

The all-available model reached R2=0.348. Removing quotation features changed R2 to 0.327 and MAE from 7.82% to 7.74%. In paired bootstrap resampling, adding quotation did not clearly change error: ΔMAE +0.07 pp (95% CI -0.21 to +0.36).

The same result appears in the narrower check: adding quotation to log length changed R2 from 0.273 to 0.257, with ΔMAE +0.20 pp (95% CI -0.15 to +0.54). Quotation is therefore useful as a descriptive marker of difficult, long entries, but not as a stable independent predictor in this sample.

Outer validation10-fold
Inner tuning5-fold
Best held-out R20.421
Lowest held-out MAE7.30%

Nested cross-validated feature-block comparison

Each row predicts the same four-metric mean for unseen entries. The ridge penalty, vocabulary, document frequencies, imputation, and scaling are all learned inside the training folds. The highlighted row has the highest held-out R2; feature counts are training-fold medians.

Source feature setFeaturesCV R2CV MAECV RMSEMAE gain vs meanSpearman rMedian ridge α
Training-fold mean 0 -0.032 10.07% 12.45% +0.00 pp -0.342
Quotation only 3 0.132 9.39% 11.42% +0.68 pp 0.133 30
Log source length 1 0.273 8.19% 10.45% +1.88 pp 0.532 0.01
Log source length + quotation 4 0.257 8.39% 10.57% +1.67 pp 0.524 6.5
Source structure 7 0.303 7.98% 10.24% +2.08 pp 0.557 30
Lexical rarity 3 0.068 9.31% 11.83% +0.75 pp 0.357 2
Citations + named entities 9 0.060 9.62% 11.89% +0.45 pp 0.275 10
Guidance recogniser hits 78 0.187 8.57% 11.05% +1.49 pp 0.464 100
Greek vocabulary 313 0.408 7.56% 9.43% +2.51 pp 0.680 0.3
Greek vocabulary + log length 314 0.421 7.30% 9.32% +2.77 pp 0.679 0.3
All available source features, without quotation 411 0.327 7.74% 10.06% +2.32 pp 0.611 100
All available source features 414 0.348 7.82% 9.90% +2.25 pp 0.629 100

What went into the all-available model?

Only information present in the Greek source before translation: log word count; explicit-quotation count and quoted-word proportion; sentence and punctuation structure; Diorisis lexical-rarity ratios; structured citation and named-entity counts; guidance-recogniser rule hits; and Greek TF-IDF unigrams and bigrams. AI-output length and any translation-similarity metric are excluded from the predictors.

Coverage. Feature source: live PostgreSQL feature tables. Recogniser hits cover 100 / 100 entries; entity features 100 / 100; citations 95 / 100; current-version rarity and sentence segmentation 97 / 100 and 97 / 100. Source-version mismatches are treated as missing and imputed within training folds. Parsed dependency-grammar coverage is 0 / 100, so it is not included. These are predictive associations in a 100-entry observational cohort; individual ridge coefficients are not treated as stable explanations.

Ten-entry review set

The 100 entries are sorted by Greek source length into ten equal groups. The lowest four-metric score in each group is selected. This gives one challenge entry from every length decile and avoids reducing review to only the ten longest or ten lowest-scoring entries.

Length decileHeadwordGreek wordsMean qualityReview text
1 Καικῖνον 6 54.5%
Open source and translations

Greek: Καικῖνον, χωρίον Ἰταλικόν. Φίλιστος δευτέρῳ Σικελικῶν.

gpt-5.6-sol v3: Kaikinon: a locality in Italy. Philistos, book 2 in his *Sikelika*.

Human-approved: Kaikinon: an Italian locality. Philistos in book 2 of his *Sikelika*.

2 Καλάσιρις 10 63.5%
Open source and translations

Greek: Καλάσιρις, μοῖρα Αἰγύπτου. Ἡρόδοτος δευτέρῳ. οἱ καὶ Καλασίριοι καὶ Καλασιρίαι.

gpt-5.6-sol v3: Kalasiris: a region of Egypt. Herodotus, book 2. Its inhabitants are called both 'Kalasirioi' and 'Kalasiriai'.

Human-approved: Kalasiris: a part of Egypt. Herodotos, book 2. They are 'Kalasirioi' and 'Kalasiriai'.

3 Κασώριον 14 51.0%
Open source and translations

Greek: Κασώριον, τὸ πορνεῖον. Ἀριστοφάνης “ἐν κασωρίοισι λείχων”. ὁ οἰκήτωρ κασωρίτης, καὶ θηλυκῶς κασωρῖτις καὶ κασωρίς.

gpt-5.6-sol v3: Kasorion: a brothel. Aristophanes: ‘licking in brothels’. Its patron is called a ‘kasorites’, and the feminine forms are ‘kasoritis’ and ‘kasoris’.

Human-approved: Kasorion: a brothel. Aristophanes: 'licking in the kasorion...'. An inhabitant is a 'kasorites'; and in the feminine 'kasoritis' or 'kasoris'.

4 Κάθαια 16 58.0%
Open source and translations

Greek: Κάθαια, πόλις Ἰνδική. τιμῶσι δὲ τοὺς καλοὺς ἐπὶ τοσοῦτον, ὥστε βασιλέα τὸν κάλλιστον αἱροῦνται. ὁ πολίτης Καθαιαῖος.

gpt-5.6-sol v3: Kathaia: a city in India. They honour the beautiful to such an extent that they choose the most beautiful man as king. A citizen is a 'Kathaiaios'.

Human-approved: Kathaia: an Indian city. They honour handsome men to such a degree that they choose as king the most handsome. A citizen is a 'Kathaiaios'.

5 Κριώα 16 57.1%
Open source and translations

Greek: Κριώα, δῆμος τῆς Ἀντιοχίδος φυλῆς. ὁ δημότης Κριωεύς. τὰ τοπικὰ Κριῶθεν κατὰ συγκοπὴν καὶ Κριῶζε καὶ Κριῶσιν.

gpt-5.6-sol v3: Krioa: a deme in the tribe Antiochis. The deme-member is called 'Krioeus'. The locative terms are 'Kriothen' by syncope, 'Krioze', and 'Kriosin'.

Human-approved: Krioa: a deme of the Antiochis tribe. A deme-member is a 'Krioeus'. The locative terms are 'Kriothen' (through syncope) as well as 'Krioze' and 'Kriosi'.

6 Κάληρος 22 51.8%
Open source and translations

Greek: Κάληρος, οὕτως ἐκαλεῖτο ἡ Ἀλωπεκόννησος ἀπὸ Καλήρου βασιλέως, Ἀλωπεκόννησος δὲ ὅτι ἐκεῖ εἶδον σκύμνους φέρουσαν καὶ κατατιθεῖσαν ἀλώπεκα. τινὲς δ´ ὅτι δεκάληρος ἐκαλεῖτο ἡ πόλις.

gpt-5.6-sol v3: Kaleros: Alopekonnesos used to be called this after King Kaleros; it was called Alopekonnesos because there they saw a fox carrying and setting down cubs. Some, however, say that the city was called 'Dekaleros'.

Human-approved: Kaleros: this used to be what Alopekonesos was called, after the king Kaleros; it is now 'Alopekonesos' because they saw a fox carry her cubs there and put them down. Some say that the city used to be called 'Dekalaros'.

7 Καλλίπολις 31 69.0%
Open source and translations

Greek: Καλλίπολις, πολίχνιον Λαμψάκου ἐν τῇ περαίᾳ τῆς Χερρονήσου ἐπ´ ἀκτῆς κειμένη, ὡς Ἀλέξανδρος ἐν τῷ περὶ Βιθυνίας. δευτέρα κατὰ τὸν Ἀνάπλουν. τρίτη πόλις Σικελίας. τετάρτη πόλις Καρίας. οἱ πολῖται πασῶν Καλλιπολῖται.

gpt-5.6-sol v3: Kallipolis: a town in Lampsakos, situated on a headland in the territory across the water from the Cherronesos, as per Alexander in his *On Bithynia*. (2) A second, according to the *Anaplous*. (3) A third, a city in Sicily. (4) A fourth, a city in Karia. The citizens of all are called 'Kallipolitai'.

Human-approved: Kallipolis: a town of Lampsakos situated on a headland on the further bank of the Chersonese, as per Alexander in his *On Bithynia*. (2) Along the Anaplous. (3) A city in Sicily. (4) A city in Karia. The citizens of all are 'Kallipolitai'.

8 Κάναστρον 43 47.7%
Open source and translations

Greek: Κάναστρον, ἄκρον Θρᾴκης καὶ Μακεδονίας. τὸ ἐθνικὸν Καναστραῖος. Σοφοκλῆς δὲ ὑπομνηματίζων τὰ ἀργοναυτικά “Καναστραῖον” φησίν “ἀκρωτήριον τῆς Παλλήνης”. ἀλλ´ ἐναντιοῦται τὰ ἐθνικά, εἰ μὴ καὶ τοῦτο ἐκλάβοιμεν παραπλησίως τῷ Λέχαιον καὶ Λεχαῖος, καὶ Λύκειον τὸ γυμνάσιον καὶ Λυκεῖος Ἀπόλλων, καὶ Νύμφαιον καὶ Νυμφαῖος.

gpt-5.6-sol v3: Kanastron: a cape in Thrace and Makedonia. The ethnonym is 'Kanastraios'. Sophokles, commenting on the Argonautic material, says: ‘Kanastraion, a promontory of Pallene.’ But the ethnonyms are at variance, unless we understand this too in a manner similar to 'Lechaion' and 'Lechaios', 'Lykeion' the gymnasium and Apollo Lykeios, and 'Nymphaion' and 'Nymphaios'.

Human-approved: Kanastron: a cape of Thrace and Macedonia. The ethnonym is 'Kanastraios', but Sophokles has 'Kanastraion' in his commentary on *The Argonautika*, saying it is 'a cape of Pallene'. However, the ethnonyms run contrary to this, unless we were to understand this as having the same pattern as 'Lechaion' and 'Lechaios', 'Lykeion' (the gymnasium) and Apollo 'Lykeios', or 'Nymphaion' and 'Nymphaios'.

9 Κώμη 53 49.1%
Open source and translations

Greek: Κώμη, ἐν ταῖς μακραῖς ὁδοῖς μέσα χωρία ἔκτισαν πρὸς τὸ κοιμᾶσθαι νυκτὸς ἐπιγενομένης, ὅθεν καὶ ἐπικέκληται, ὡς Φιλόξενος. ὁ οἰκήτωρ ἐγκώμιος. Ἡσίοδος “εἰ γάρ τοι καὶ χρῆμ´ ἐγκώμιον ἄλλο γένοιτο”. καὶ κωμαῖος καὶ κωμήτης ἀρσενικῶς καὶ θηλυκῶς διὰ τοῦ « ι ». καὶ ὁ δῆμος κώμη. εἴρηται ἀπὸ τοῦ ἐν αὐταῖς κοιμᾶσθαί τε καὶ αὐλίζεσθαι. λέγεται καὶ κωμήτωρ.

gpt-5.6-sol v3: Kome: on long roads they built halfway points for sleeping when night came, from which it also received its name, as per Philoxenos. The inhabitant is an 'enkomios'. Hesiod: ‘For even if some other need should arise for you in the village.’ Also 'komaios', and 'kometes' in the masculine and 'kometis' in the feminine, with ι. A deme too is a 'kome'. The term is said to derive from sleeping and lodging in them. The inhabitant is also called a 'kometor'.

Human-approved: Kome: along long routes they established halfway points for the purpose of sleeping when night fell, after which they are now named, as per Philoxenos. An inhabitant is an 'enkomios'. Hesiod: 'For if some matter were to arise for you as an enkomios'. And also a 'komaios', and 'kometes' in the masculine, which in the feminine is written with ι. A deme is also a kome, named after the practice of taking shelter and sleeping there. 'Kometor' is also said.

10 Καπετώλιον 86 52.6%
Open source and translations

Greek: Καπετώλιον, ἐν Ῥώμῃ λόφος Ταρπαῖος πάλαι λεγόμενος. περὶ δὲ τῆς γραφῆς τοῦ τεμενικοῦ ὁ Εὐδαίμων διὰ τοῦ « ι » φησίν “Εἰρηναῖος προπαροξύνει διὰ τοῦ « ι » γράφων”. Ἡρωδιανὸς δὲ Καπετωλίειον ἐν ἓξ συλλαβαῖς, καὶ ἐν συναιρέσει Καπετώλειον. τὸ δὲ Καπετωλεῖα ἐπὶ τοῦ ἀγῶνος προπερισπᾶται. ὅσα γὰρ ἔχει προϋπάρχοντα εἰς « ος » καθαρόν, παραληγόμενα ἢ μόνῳ τῷ « ι » ἢ προηγουμένου αὐτοῦ τοῦ « α » ὥστε εἶναι πρὸ τέλους τὴν « αι » δίφθογγον, προπερισπᾶται, ἢ καὶ ὅσα κτητικά. Ἀσκληπιεῖον, Ἀσκληπιός γάρ· Πτολεμαεῖον, Πτολεμαῖος γάρ· Ὀλυμπιεῖα τὰ Ἀθήνησιν, Ὀλύμπιος γάρ. τὸ δὲ τέμενος Καπετώλιον καὶ Καπετώλιος Ζεὺς διὰ τοῦ « ι ». τὸ ἐθνικὸν Καπετωλῖνος.

gpt-5.6-sol v3: Kapetolion: a hill in Rome formerly called Tarpeios. On the spelling of the sanctuary name, Eudaimon says that it is written with ι: ‘Eirenaios accents it with an acute on the antepenult, writing it with ι.’ Herodianos gives Καπετωλίειον in six syllables, and, with vowel contraction, Καπετώλειον. Καπετωλεῖα, when it refers to the contest, is accented with a circumflex on the penult. For all forms that have an underlying form ending in pure -os and have in the penult either ι alone or ι preceded by α, so that the diphthong αι stands before the final syllable, are accented with a circumflex on the penult, as are possessive forms: ‘Asklepieion’, since the underlying form is ‘Asklepios’; ‘Ptolemaeion’, since it is ‘Ptolemaios’; the ‘Olympieia’ at Athens, since it is ‘Olympios’. The sanctuary is ‘Kapetolion’, and Zeus Kapetolios is written with ι. The ethnonym is ‘Kapetolinos’.

Human-approved: Kapetolion: in Rome, a hill that was long ago called 'Tarpaios'. Regarding the spelling of the sanctuary name, Eudaimon says that it is with ι: 'Eirenaios accents it with acute on the antepenult and writes it with ι.' Herodian has 'Kapetolieion' with six syllables, and 'Kapetolion' with contraction. The form 'Kapetoleia', referring to the games, is accented with circumflex on the penult. This is because forms whose base already ends in postvocalic -ος—when either a single ι is in the penultimate position or α precedes it so that the diphthong αι stands before the ultima—will be accented with a circumflex on the penult, and the same applies to possessive forms. Asklepieion (Ἀσκληπιεῖον) is thus from 'Asklepios' (Ἀσκληπιός); Ptolemaeion (Πτολεμαεῖον) is thus from 'Ptolemaios' (Πτολεμαῖος); Olympieia (Ὀλυμπιεῖα), the one at Athens, is thus from 'Olympios' (Ὀλύμπιος). The sanctuary, however, is 'Kapetolion' (Καπετώλιον), and 'Zeus Kapetolios' (Καπετώλιος), with ι. The ethnonym is 'Kapetolinos'.

Downloads

Verify all 100 official headwords
Official orderKappa entryHeadwordGreek wordsMean qualityReview set
1 1 Καβαλίς 44 80.0%
2 2 Καβασσός 69 58.0%
3 3 Καβειρία 79 62.8%
4 4 Καβελλιών 23 72.0%
5 5 Καβύλη 16 89.1%
6 6 Καδμεία πόλις 17 62.5%
7 7 Κάδοι 13 74.0%
8 8 Καδούσιοι 11 100.0%
9 9 Κάδρεμα 13 73.1%
10 10 Κάθαια 16 58.0% Decile 4
11 11 Καικῖνον 6 54.5% Decile 1
12 12 Καινίνη 9 78.0%
13 13 Καινοί 7 83.8%
14 15 Καινύς 16 72.5%
15 14 Καιρή 11 80.7%
16 16 Καισάρεια 29 76.6%
17 17 Καλαβρία 12 69.0%
18 18 Καλάθη 31 80.0%
19 19 Καλάμαι 5 93.7%
20 20 Καλαμένθη 14 63.5%
21 21 Κάλαρνα 9 86.8%
22 22 Καλάσιρις 10 63.5% Decile 2
23 23 Καλατίαι 5 89.4%
24 24 Καλαύρεια 19 70.6%
25 25 Κάλβιος 14 73.3%
26 26 Καλὴ ἀκτή 43 62.2%
27 27 Κάληρος 22 51.8% Decile 6
28 28 Καλησία 12 94.5%
29 29 Καλλάτηβος 8 100.0%
30 30 Κάλλατις 45 57.2%
31 31 Καλλίαι 16 73.4%
32 32 Καλλίαρος 30 80.3%
33 33 Καλλιόπη 11 90.2%
34 34 Καλλίπολις 31 69.0% Decile 7
35 35 Κάλπη 36 77.0%
36 36 Καλύβη 13 88.6%
37 37 Κάλυδνα 27 82.1%
38 38 Καλυδών 14 82.3%
39 39 Κάλυμνα 25 85.0%
40 40 Κάλυνδα 10 85.4%
41 41 Κάλυτις 16 62.5%
42 48 Κάμιρος 45 66.6%
43 51 Κάναθα 16 68.9%
44 52 Κάναι 59 63.1%
45 53 Κάναστρον 43 47.7% Decile 8
46 63 Κάνωπος 59 69.4%
47 66 Καπετώλιον 86 52.6% Decile 10
48 68 Καππαδοκία 57 63.3%
49 75 Καρδαμύλη 34 78.2%
50 82 Καρία 181 57.6%
51 97 Καρπασία 80 63.9%
52 99 Καρπήσιοι 7 71.2%
53 102 Καρύανδα 24 74.7%
54 103 Κάρυστος 132 64.1%
55 104 Καρχηδών 88 55.6%
56 105 Κάσιον 43 74.0%
57 107 Κάσος 44 72.6%
58 109 Κοσσαῖος γενεὴν Κασπειρόθεν 84 57.4%
59 120 Κάσταξ 14 77.0%
60 121 Κάστνιον 20 67.5%
61 122 Καστωλοῦ πεδίον 29 74.6%
62 123 Κασώριον 14 51.0% Decile 3
63 124 Κατάβαθμος 15 89.5%
64 125 Κατακεκαυμένη 34 69.5%
65 127 Κατάνειρα 11 66.1%
66 126 Κατάνη 64 59.6%
67 129 Καταννοί 8 88.3%
68 128 Καταονία 17 73.2%
69 130 Κάτρη 18 74.7%
70 151 Κεκρυφάλεια 17 77.5%
71 180 Κορώνεια 76 65.7%
72 188 Κοτιάειον 46 63.8%
73 205 Κρανίδες 11 96.0%
74 222 Κριώα 16 57.1% Decile 5
75 286 Κύρβη 7 100.0%
76 287 Κύρη 14 84.8%
77 288 Κυρήνη 33 75.3%
78 289 Κύρης 10 70.7%
79 290 Κύρις 12 79.0%
80 291 Κύρνος 34 59.9%
81 292 Κύρου πόλις 22 66.5%
82 293 Κύρρος 28 75.0%
83 294 Κυρταία 29 73.0%
84 295 Κύρτος 48 65.3%
85 296 Κύρτωνες 20 65.7%
86 297 Κυρτώνιος 14 78.9%
87 298 Κύτα 58 70.1%
88 299 Κυτέριον 18 63.8%
89 300 Κύτινα 21 96.6%
90 301 Κυτώνιον 14 91.7%
91 302 Κύτωρος 22 74.6%
92 303 Κύφος 34 62.1%
93 304 Κυχρεῖος πάγος 45 70.2%
94 305 Κύψελα 22 84.4%
95 306 Κῶβρυς 12 86.5%
96 307 Κώθων 15 94.3%
97 308 Κωλιάς 42 67.9%
98 309 Κῶλοι 17 67.7%
99 310 Κώμη 53 49.1% Decile 9
100 311 Κωνώπη 47 63.1%

Generated 2026-07-25 12:55:47 UTC. The four-metric mean is the unweighted entry-level mean of BLEU-4, chrF++, METEOR, and ROUGE-L against the approved house-style translation.