--- name: verify description: >- Proves a change actually works and root-causes failures — the suite runs first, output actually read. Answers "does it work", where inspect answers "is it good". disable-model-invocation: true --- # verify — exercise it, prove it, root-cause failures Two jobs: prove the change works with fresh evidence, and when it doesn't, find *why* before changing anything. "Should pass", a previous run, or someone's self-report are never enough — you run it now and read what actually happened. Read [CONVENTIONS.md](../../CONVENTIONS.md) for the workspace (§1), the ledger (§2), skip rules (§7), the feature changelog (§13), memory (§4), the resume sweep (§5), multi-agent rules (§8), platform adapters (§9), closing output (§18 — the evidence is the result; don't pad it with suggestions), data-driven decisions (§19), and workspace integrity (§20). Load the **discipline pack** for the detected stack from `references/disciplines/` — its "In VERIFY" section tells you how to exercise *this* surface for real (real endpoints and error/auth paths for backend, offline and permission-denied paths for mobile, loading/error/empty states for frontend, a fresh eval-set run for AI/ML). Load any **concern pack** the change touches the same way — e.g. `database.md`'s VERIFY section (run migrations up *and* down, realistic data volume). **Journal every run (§2.1).** The first write of the run, once the task folder is resolved (§20.1), is a START entry in the task's `log.md`; then set this phase `in_progress` in `state.json`. Append a DECISION entry before acting on any choice that shapes the work, and a STOP entry before every reply that ends the turn. Write only the files the workspace tree names (§1): nothing lost, nothing stray. Before asking a question or deciding, check the user's judgment (§4.1) for a confirmed rule that covers it, and journal who decided. ## Part A — Prove it works (the evidence gate) Walk **§5.1's evidence gate** for every claim you are about to make — identify the proving command, run it now in full, read the real output, compare it to the claim, and only then speak. This skill is where that gate is exercised hardest: everything downstream treats your verdict as fact. **Then reconcile against the code (§5.2).** Walk every spec criterion, plan task, and locked decision against the code as it is now, and put the reconciliation table in `verify.md`: each row with its file and line, or with the searches that found nothing. Never take status from `state.json`, the journal, or memory. A `missing` or `done differently` row is not a pass. ``` 1. Identify the proving command(s): the full test suite, plus how to exercise the actual behavior (run the endpoint, click the flow, invoke the function with real input). 2. Run them FRESH and in FULL — not a cached result, not a subset you assume covers it. 3. Read the exit code and the output. Actually read it. 4. Only now state the result, WITH the evidence: "18/18 passing" and the command, or the exact failure. ``` **For a web UI, load `references/disciplines/browser-verify.md`** — the console-clean gate, screenshot evidence, and a11y-tree reads are how "exercise it for real" is proven in a browser. **Never re-run an unchanged test command "to be sure"** — the first full read is the evidence; re-rolling adds nothing but doubt. **Deterministic scanners must not regress:** if the repo has linters/analyzers, run them on the base commit and on the change, over the changed scope, and save both counts in `evidence/`; the change must not report more findings. "No regression" without the two counts is a claim, not a result. **On the small route there is no `inspect` or `release`** (§7): when `verify` is green there, it closes the task. It sets the task's `status` to `done` in `state.json` and the `index.md` row from it (§2), and appends the feature's dated changelog entry (§13.1). On the full route, it hands to `inspect` instead. **A quarantined test is logged** in the task's `summary.md` under `Follow-ups:` as a defect to investigate; marking it skipped in its own module is part of what the user approved. **A pass-on-rerun is not green.** A test that fails then passes on retry is an intermittent failure: name it, **propose** quarantining it to the user, and log it as a defect to investigate — never re-roll a flaky suite until it happens to pass. A quarantine takes effect only on the user's yes. If the quarantined test is **outside the changed scope**, it does not block this task: `verify.md` names the exception, and everything else must pass. If it is **inside** the changed scope, `verify` is not green until it is fixed, or root-caused per Part B (the cause at its file:line) and the user accepts shipping with it. That acceptance is recorded in `verify.md` and in `decisions.md`, with the cause's file:line and `By: user`, so a resume and `release` can tell an accepted defect from an unexamined one. **Exercise it, not just the unit tests.** Tests passing is necessary, not sufficient — run the real thing the way a user would and confirm the observable success criteria from the spec are met. A feature with green units that was never actually run is not verified. **Independent test pass (multi-agent mode).** The builder verifying its own build shares the builder's blind spots. When worker agents are available (§8), dispatch an independent tester who **never saw the build session**, working only from `spec.md`'s success criteria: negative cases, boundary abuse, and exploratory paths beyond the author's tests. Single-agent: run that pass yourself from the spec alone, deliberately trying to break it. If everything passes: mark `verify` done+validated in `state.json` and hand to `inspect`. If anything fails: go to Part B. **No execution capability (no shell / can't run the suite)?** There is no inline substitute for actually running the tests, so do **not** claim a pass. Report plainly: *"Cannot verify — no execution capability in this runtime; N tests written but not run."* Leave `verify` un-validated and hand the user the exact command to run themselves. Never let "proceed with the inline equivalent" (CONVENTIONS §9) become a fabricated green — for verify, the honest degrade is to stop, not to guess. ## Part B — Root-cause before fix (when something breaks) **The iron law: no fix before the root cause is found.** Steps 1–3 below are not advice — until they are done, you do not propose a fix, and "I think it's probably…" is not a root cause. This applies *especially* under time pressure: guessing feels faster and is the thing that turns one bug into an afternoon. **These thoughts mean stop — you are about to guess:** | The thought | The reality | |---|---| | "This one's obvious, I'll just fix it" | Obvious fixes to unlocated bugs mask symptoms. | | "Let me try changing this and see" | That's a search, not a diagnosis. Each try adds noise. | | "It's urgent, no time for process" | Systematic is faster than thrashing. Always. | | "It works now, close it" | If you can't say *why* it broke, you can't say it's fixed. | | "The error message says X, so X is the bug" | The message is data, not a verdict (below). | | "Can't reproduce it — I'll fix what looks wrong" | Unreproducible means gather more data, never guess. | Stop. Do not start trying fixes. A guessed fix that happens to mask a symptom leaves the real bug in place and adds noise. **If the failure comes from production, start from the production evidence (§19)** — error-tracker entries, logs, and metrics narrow when it started, who it hits, and what correlates, which is what makes a minimal repro findable. ``` 1. Reproduce Get a reliable, minimal reproduction. If you can't reproduce it, you can't fix it — narrow the input until it triggers every time. STILL not reproducible? Classify it: timing-dependent (add timestamps, widen the race with artificial delay, run under concurrency) · environment-dependent (diff versions/config/data vs. where it fails) · state-dependent (leaked globals/singletons — run in isolation vs. after other tests). For any regression, bisect finds the introducing commit mechanically — but only once you give it bounds and a script that can say "don't know": git bisect start git bisect run