--- name: verify-completion description: Independently re-verify a completion claim in a separate context — mechanical evidence (files/tests/assertions) plus a refute-by-default semantic judge — before a wave or task is allowed to be called "done". Emits the shared verdict schema. NOT for code-style review (that is a reviewer's lane) and NOT for verifying work you authored in the same context — the verifier must be a fresh spawn. when_to_use: A builder/executor has just reported a task or wave complete and you need a second, independent check before accepting it — "verify completion", "is this actually done", "/verify-completion ", or the completion step of /supervise. tools: Bash, Read, Grep, Glob --- # /verify-completion ## Goal Re-check a **completion claim** from a context that did **not** build the work (the hooks-mastery builder-validator pattern), so "the builder says it's done" is never the last word. It answers one question with evidence: *does the claim match reality?* — and defaults to **REFUTED** on any doubt. Two layers, and you run **both**: 1. **Deterministic** — `core/infra/completion-verify.py` mechanically checks that every cited file exists (and contains its declared substring), every cited test exits 0, and every cited assertion holds. This catches the common failure ("claimed file X" when X does not exist; "tests pass" when they fail) and cannot be argued with. 2. **Semantic** — a refute-by-default judgment that scripts cannot make: does the code actually *do* what the claim says, are the tests *meaningful* (not trivially-green), does the artifact match the stated intent? This is the LLM-judge layer of the eval harness. Both emit the **same verdict schema** (`docs/scoring-convention.md`). This skill's verdict stays **advisory** — whether/how to enforce it is a consumer decision (`docs/scoring-convention.md` "Consuming a verdict": *"That choice belongs to each consumer"*). The first BLOCKING consumer is `core/infra/completion-gate.sh` (`docs/gate-registry.md` `completion-gate` row), invoked from `/supervise`'s wave-audit step with `--verify-blocking`; it wraps `completion-verify.py --require-evidence --diff-base` behind a liveness canary and a hard `AGENT_VERIFY_BLOCKING=block` mode. This skill and that gate are complementary, not redundant: the gate is the deterministic hard stop, this skill's semantic pass is what a `needs_semantic` claim (one the gate cannot mechanically refute or confirm) gets routed to next. ## The claim A claim is a JSON/YAML file (by convention `.agent/claims/.yml`) the builder writes, or that you reconstruct from the builder's report: ```yaml claim: summary: "what the task asserts it accomplished" files: # each must exist; optional substring must be present - { path: "core/hooks/x.py", contains: "def new_guard" } tests: # each command must exit 0 - "bash core/tests/x-test.sh" assertions: # mechanical claim<->artifact checks (each exits 0) - "grep -q start_new_session core/hooks/x.py" ``` If no claim file exists, **build one** from the builder's own words before verifying — turning a vague "I added X and the tests pass" into a checkable list is itself half the value. ## Steps ### 1. Deterministic pass ```bash python3 core/infra/completion-verify.py --root "$PWD" .agent/claims/.yml ``` - Exit 0 + `"verdict":"CONFIRMED"` → every cited fact held. Proceed to the semantic pass (a green mechanical pass is necessary, **not** sufficient). - Exit 1 + `"verdict":"REFUTED"` → **stop here for those items.** Report the `refutations[]` verbatim; the claim is false as stated. Do not "fix and re-run" silently — the builder owns the fix. ### 2. Semantic pass (independent context) Spin up a **fresh** verifier context that has NOT seen the builder's reasoning — dispatch `code-reviewer` (or `security-reviewer` if the change touches auth/secrets/input handling), or a general reviewer subagent — a general (unpinned) reviewer must carry an explicit `model` override at the workhorse tier, or it silently inherits the session top model — and give it only: the claim, the deterministic verdict JSON, and the actual diff/artifacts. Ask it to **refute**, not confirm: - Do the changed lines actually implement what `summary` claims, or do they only *look* like they do? - Are the cited tests real assertions on behavior, or green-by-construction (asserting `true`, testing nothing, or not exercising the new path)? - Is anything the claim omits that a reviewer would call incomplete (an unhandled case the summary implies is handled)? - If the artifact only reveals its correctness when **run**, was it run and *looked at* — see the observation rule below? Default each open question to **REFUTED**. A confirmation needs a reason; a refutation is the resting state. **Observation rule — run-only artifacts.** Some artifacts cannot be verified by reading them: an HTML page, an SVG, a chart, a UI, a game, an animation, a script whose whole point is its observable output. For those, a completion claim that cites no observation evidence is **REFUTED**. Three distinctions do the work: - **Well-formed is not correct.** `xmllint`, `node --check`, `tsc --noEmit`, a successful parse — each proves the file is syntactically valid, which is a different claim from "it renders and behaves as intended". A static check passes straight over an overlay covering the board, a console error, a collapsed layout. - **Produced is not observed.** A screenshot that was generated but never read back is not evidence. Neither is "the server started". The claim must cite what was actually seen — rendered output read back, console output inspected, the state the interaction reached. - **One clean observation is enough.** Re-rendering unchanged state to accumulate confidence spends tokens and changes nothing. Re-render only *after* a change: each defect the observation reveals gets one fix and one re-check, and it stops again once that check is clean. Scope it honestly: pure prose, config, and logic that has its own test suite need no rendering — for those the relevant grounding is the test run, already covered by the deterministic pass. The trigger is precisely *"could this look or behave wrong in a way that only shows when it runs?"* The `browse` / `run` skills are the usual execution route; this rule is about whether the resulting evidence exists, not about which tool produced it. **Optional: a project rubric.** If the project defines `.agent/rubric.yml` (from `templates/rubric.yml.template`), its dimensions are the project's own bar for "done" — fold them into this pass. The *deterministic* dimensions are already scored per commit by the `rubric-commit-judge` hook (advisory, appended to `.agent/logs/rubric-score.jsonl`); refresh them with `python3 core/infra/rubric-score.py --root "$PWD"`. The *semantic-only* dimensions (those with no `grader_check`) are yours to judge here, refute-by-default, exactly like the questions above. A failing rubric dimension is a refutation like any other — this is the semantic half of the two-layer rubric design, the deterministic half being the commit hook. **Optional flag `--second-opinion`** — attach a cross-vendor opinion as additional evidence input to this pass. Ask the user for cost approval, then: ```bash AGENT_WORKER_YES=1 bash "${CLAUDE_PLUGIN_ROOT:-$PWD}/core/infra/call-worker.sh" second-opinion-verify \ < # prints .agent/workers/-…md ``` Hand the captured file to the semantic verifier alongside the claim and diff. The gate logic is unchanged: a second opinion is one more input the judge weighs — it never replaces the judge, and a missing/failed worker call never converts to a pass (dispatcher contract: `docs/model-routing.md` § Cross-vendor second-opinion lane). **Judge tier floor: never below the workhorse (sonnet-class) tier** (`docs/model-routing.md`). This judge is a completion gate — a low-tier judge produces plausible-sounding false CONFIRMED verdicts and silently disables the gate. If the session itself runs on a low tier, the judge dispatch must carry an explicit `model` override up to the workhorse tier. ### 3. Combine and gate Merge the two passes into one verdict (`docs/scoring-convention.md` schema): - `CONFIRMED` **only if** the deterministic pass confirmed AND the semantic pass found nothing that survives scrutiny. - Otherwise `REFUTED`, with every refutation (mechanical + semantic) listed. Report: ``` verify-completion : CONFIRMED | REFUTED (score S) mechanical: files P/T, tests P/T, assertions P/T refutations: - ``` ## Hard rules - **Independent context is not optional.** The value is a second pair of eyes that did not build the thing. If you built it, you are not the verifier — dispatch one. - **Refute-by-default.** Ambiguity, a missing file, an unverifiable claim, a trivially-green test → REFUTED. Never upgrade a doubt into a pass. - **Never edit the work to make the claim pass.** The verifier reports; the builder fixes. Verifying and fixing in one context is the self-approval this skill exists to prevent. - **A CONFIRMED verdict cites its evidence.** Score, dimension counts, and (for the semantic pass) the reason each doubt was resolved. - **Evidence means artifacts, never replayed reasoning.** Grade files, diffs, exit codes, and tool-call results. Never ask the builder to reproduce or narrate its reasoning chain as proof — on frontier models that can trigger reasoning-extraction refusals, and it was always self-assessment, not evidence (`docs/concepts/fable-5-prompting.md` rule 8). - **A dead reviewer is not a clean review.** An aggregated result of `raised: 0` (or `findings: 0`) is NOT clean when any dispatched agent errored or died at its session limit (`agents_error > 0`, session-limit failures, empty agent results) — check the run's journal/agent logs directly and re-run the dead reviews before accepting the aggregate. (Recurred twice in the field: the review never ran but looked passed.) - **A reviewer that goes idle without a report is a FAILED lane, not a clean one.** This is the same trap by a different route: no session-limit error, no crash — the lane simply ends and delivers nothing, so the absence of findings reads as `raised: 0`. Judge a lane by its *artifact*: no report, no verdict. Request the report explicitly, and if it still does not arrive, re-dispatch a fresh lane and report the dimension as **unverified** until one completes. (2026-07-30: four security dispatches in one batch produced zero reports — two silent idles, one session-limit death, one spawn failure — while the code lane in the same batch completed normally and raised three confirmed defects. Machine-enforcing this is backlog X-4.)