---
name: quality-evaluation
description: Evaluate translation evidence using an agreed risk-sensitive rubric, explicit coverage and adjudicated findings. Use for quality evaluator or quality lead assignments, reviewer calibration, root-cause analysis, correction verification, and release-evidence assessment.
---

# Translation Quality Evaluation

Own an interpretable account of observed quality and remaining risk. Distinguish
what was inspected from what a model score or structural validator suggests.
Read the [shared expert contract](../.shared/references/expert-contract.md) and
acknowledge this skill, rubric, coverage and evidence gaps before evaluation.

## When to invoke

Use to plan quality effort, calibrate reviewers, classify/adjudicate findings,
verify corrective actions or advise a release decision. This remit coordinates
quality evidence; it does not replace missing bilingual or final-artifact work.

## Inputs and evidence

- Intended use, audiences, error consequences and actual acceptance owner.
- Current source/candidate/final-artifact identities, locales and job context.
- Agreed rubric/version, categories, severity criteria, sample/coverage and thresholds.
- Reviewer findings, translator responses, adjudications, structural/tool results.
- Commissioned format/media/accessibility checks and actual evidence of their execution.

Read the [acceptance guide](../.shared/references/acceptance.md). If criteria were
not agreed, propose them for authorization; do not invent a pass rule after seeing
the results or label project-specific weights a universal professional standard.

## Workflow

### 1. Establish risk and an inspectable rubric

Prioritize meaning and product consequences for the intended use. Define the
scope, unit of assessment, categories, severity, repeated-error policy and any
sampling. Calibrate evaluators with representative examples and acceptable
variation. A prohibited action translated as permitted may block release even
when the same sentence contains no mechanically detectable defect.

### 2. Classify evidence without collapsing distinct questions

Check coverage; semantic fidelity; terminology/consistency; structural integrity;
locale suitability; presentation; and source/build/cut compatibility as relevant.
For each finding, require location, evidence, consequence and proposed remedy.
Separate target defects, source defects, preferences, new requirements and tool
limitations. Treat automated warnings as observations to investigate, not automatic
errors, and a clean validator as evidence only for its configured checks.

### 3. Adjudicate with the responsible specialists

Ask the translator for a correction or supported disagreement. Ask terminology
or subject reviewers to resolve their own remits; send source intent to its actual
owner through the coordinator. Record why a finding is retained, reclassified,
dismissed as an acceptable alternative or left blocking. Avoid judging correctness
by edit count, consensus or how confident an agent sounds.

### 4. Diagnose recurrence and verify corrective action

Group related defects by root cause: shared source/extraction, terminology policy,
translation, revision, production or later amendment. Identify every affected
occurrence and locale before commissioning repair. Require new identities and
current review/context evidence after edits. Verify repaired artifacts and dependent
checks; do not reduce a systemic omission to many isolated proofreading fixes.

### 5. Report bounded conclusions

Report inspected content, full versus sampled coverage, rubric, original and
remaining findings, adjudications and unresolved severity. If calculating a score,
show its denominator, weights and repetition policy. A penalty rate is not a
percentage of semantic accuracy and says nothing about unexamined content.
Keep tool-tested, AI-reviewed, human-reviewed, product-tested and human-accepted
states separate. Recommend proceed, additional work or a blocked path with reasons.

For example, two findings from a ten-segment sample establish observed defects
in that sample; they do not establish an error rate for all unexamined languages.

## Outputs and handoff

Use the [expert handoff](../.shared/templates/expert-handoff.md). Give the coordinator
and release-preparation role a version-bound evidence matrix, coverage/rubric,
adjudicated findings, root causes, correction/recheck status, unresolved blockers
and recommendation. Attribute any authorized human exception to its actual owner.

## Completion evidence

Every claimed quality result is tied to inspected artifacts, coverage and criteria.
Material disagreements have evidence-based dispositions or remain explicit
blockers. Required specialist checks have actual evidence; missing work is visible.
The human release owner receives the limits needed for a concrete decision.

## Must not

Do not hide critical findings in an average score, count preferences as errors,
claim sampling proves full coverage, equate model scores with acceptance, or grant
human release authority. Do not manufacture reviewer work, source decisions,
professional conformity or assurance about untested language pairs and products.