---
name: certification-pilot
description: Plan certification pilots, analyze supplied scored responses and support documented standard-setting work without fabricating study evidence.
license: MIT
---

# Certification Pilot

Read [analysis limits](../certification-workshop/references/analysis.md) and [helpers](../certification-workshop/references/helpers.md). Prepare a pilot protocol covering intended population, selection, consent/data handling, form version, scoring, timing, feedback and study questions. Choose sampling with appropriate expertise; no universal sample count is imposed. A pilot draft is not an administered study.

Use analyze only on an exact form's scored response matrix with item-revision headers. Investigate missing/invalid scores, mixed versions, duplicate candidate IDs, nonrepresentative data and scoring changes before interpretation. Report item score proportions, corrected item-total associations, total-score distribution and coefficient alpha with the helper's limitations. No mechanical threshold alone justifies retaining or removing an item. DIF, IRT, equating, classification accuracy and inter-rater reliability need their own justified design/tools; do not claim they ran here.

For standard setting, document the minimally competent candidate/performance definition, chosen procedure, panel selection/training, rounds, judgments and rationale. The panel helper summarizes supplied expected scores; it does not select a method, invent judges, round to a policy cut, or authorize a passing standard. Record the owner's actual passing rule and consequence review separately. Revise items with new versions, preserve pilot data and report what still requires evidence.