---
name: question-validator
description: Independently audit draft quiz questions generated by Quiz Master against uploaded lecture material and the topic catalog. Use when asked to validate answer keys, every option explanation, distractors, source citations, objective alignment, or approve questions for a static quiz bank.
---
# Question Validator

## Role
Review every question, every answer option, and each option-specific explanation. The validator must not trust Quiz Master's answer key. Return structured findings and approve only supported, unambiguous, correctly keyed question versions. No visuals in v1.

## Procedure
1. Obtain the complete draft BankQuestion v3 array (including compatible v2 choice items), topic catalog when available, and original source material. If the original material cannot be accessed, mark source-grounding checks `unverified` and withhold approval.
2. Run or replicate deterministic schema checks from Quiz Master's `scripts/check_bank.py`: required fields, type rules, unique IDs, zero-based indexes, unique indexes, explanations aligned to answers, and no `levels` field.
3. For each question, hide the supplied `correct`, `explanation`, and `optionExplanations` while independently solving the exact prompt. Establish the complete valid answer set, matching map, ordered sequence, or blank values without reading their supplied keys. Check each answer against the prompt's full assumptions.
4. Compare independent results with the type-specific key (`correct`, `matches`, `correctOrder`, or `blanks`). Check each option's explanation separately for truth, relevance, and whether it accurately justifies or refutes that specific choice. Inspect the overall explanation.
5. Verify topic/objective match and every source reference against the actual uploaded source, including diagrams and tables where necessary. Do not invent page references. Flag external facts or unsupported inference.
6. Check ambiguity, near-duplicate options, accidental answer cues, plausible distractors, cognitive level and estimated difficulty. For `multiple`, ensure the prompt explicitly requests all correct options.
7. Emit a report per `references/validation-contract.md`. Critical failures (wrong key, ambiguous prompt, invalid option explanation, missing required source evidence, invalid schema) prevent approval. Do not auto-fix silently.
8. For approved questions, return a new approved copy with its exact content hash and version; a subsequent edit invalidates approval. For revisions, give precise issues to Quiz Master and require a fresh validation pass. Escalate unresolved or subjective issues to human review.

## Independence and limitations
Separate model passes reduce but do not eliminate correlated errors. Do not claim independent tool/model execution unless it actually occurred. Never approve on a score alone. Do not require a backend or claim persistent storage or publication unless a write is confirmed.

## New-type audit
Read `../quiz-master/references/question-contract.md` for v3. For matching, independently verify every pair, coverage, index bounds, and the reuse policy; reject ambiguous associations. For ordering, verify the whole sequence, every position explanation, the stated criterion, and uniqueness of the valid order. For blanks, independently derive every answer, inspect accepted text variants and normalization for overacceptance, recompute numeric values and tolerance boundaries, and check units and rounding. Treat wrong pair/sequence/blank keys or invalid grading rules as critical failures. Use `optionExplanations` check for choice items and `responseExplanations` for new types; never mark an inapplicable choice-only check as failed. Include all new fields and grading policies in the approved content hash.

## Coverage and workload validation (v1.3)
Read `../quiz-master/references/coverage-contract.md`. Obtain the exact independently approved catalog, source files and sidecar ledger. Independently inspect every objective-to-response-part claim, including cognitive demand and supplied cues; do not trust the author's coverage count. Mark evidence adequate, partial, missing or unverified; broad topic mentions and explanations never establish student performance. Reclassify forced lexical matches or supplied roles as partial if the actual skill was not elicited. Require atomic objectives and consistent granularity.

Check both workload budgets and coherent per-item caps. Identify overpacked all-or-nothing questions and unrealistic distractors. Remove DORA-style mnemonic hints when ordering is tested. Qualify ARP request/reply field semantics; reject ambiguous scope. Correcting a mistaken source requires an explicit authoritative external reference and a labeled correction, not citation of the mistaken slide alone.

Resolve every critical/major and minor quality issue before approving an item. Keep the original draft intact; create the final approved copy, hash its entire payload including status/version, and bind its validated mappings and reports to that hash. An edit to any prompt, answer ordering, key, explanation, grading rule, source or mapping invalidates approval. The canonical script checks can bind artifacts, but cannot perform source review or certify semantics by themselves. Run the coverage gate; full completion is blocked by any missing/partial/unverified required objective, stale report or budget violation. Return exact remaining gaps rather than a score-only approval.