{ "_note": "Pairs excluded at analysis time by the pre-analysis data audit (DESIGN Amendment 1, 2026-07-19). Decided from data inspection only, BEFORE any full-run scoring result was read. Scoring outputs still contain these rows; 03_analyze.py drops them. Criterion: structural malformation asymmetric to one language (unanswerable/ill-defined items) — NOT translation style, which remains part of the PL condition per DESIGN caveats.", "belebele-431": "PL option B duplicates option A; the correct answer (EN gold B 'Its crash landing was a success') is absent from the Polish item — asymmetrically unanswerable in PL, would bias G against Polish", "belebele-401": "options B and D are the identical string '2' in both languages (source-inherited duplicate distractor), so scoring is ill-defined; PL question additionally contains a dittography ('ile osób' repeated)", "mmlu-8642": "moral_scenarios MT collapse: all 4 PL options identical ('Niewłaściwe, Niewłaściwe') — unanswerable in PL", "mmlu-9307": "moral_scenarios MT collapse: all 4 PL options identical — unanswerable in PL", "mmlu-8927": "moral_scenarios MT collapse: all 4 PL options identical — unanswerable in PL", "mmlu-9211": "moral_scenarios MT collapse: all 4 PL options identical — unanswerable in PL", "mmlu-8632": "moral_scenarios MT collapse: all 4 PL options identical — unanswerable in PL", "mmlu-3484": "PL gold 'Centripetal' mistranslated as 'Odśrodkowa' (centrifugal), inverting the gold meaning AND duplicating option B — broken in PL", "mmlu-2721": "cloze answer-leak: PL gold text ('macierzy mitochondrialnej') fused into the stem, option A empty — gold points at a blank and the answer is leaked", "mmlu-5242": "cloze answer-leak: PL gold text ('limbic system') fused into the stem, option A empty — gold points at a blank and the answer is leaked", "mmlu-10932": "PL gold option B empty (text dropped by MT) — gold points at a blank", "mmlu-4523": "cloze shift: PL distractor A absorbed into stem, option A empty — PL choice set structurally degraded (gold D intact but option-set asymmetric)", "mmlu-11400": "PL distractor option B empty (text dropped by MT) — PL choice set structurally degraded (gold A intact but option-set asymmetric)" }