--- name: poke-holes description: Adversarial verification with three competing agents (issue-finder, disprover, judge). Triggers "double check", "are you sure", "poke holes", or a critical fix before production. context: fork --- # Adversarial Verification Three agents with competing incentives: one finds issues, one disproves them, one judges. ## Standalone Codex Create each fresh panelist with `spawn_agent`, deliver context while it runs with `send_message`, trigger another turn once it is idle with `followup_task`, wait with `wait_agent`, and stop its current turn with `interrupt_agent` only when necessary. Never spawn `codex-verifier` and never run `codex-run.ts` from inside Codex. Only read-only reviewers may overlap. Writers share the working tree unless the live host explicitly offers isolation, so any implementer and test-writer follow-up must use non-overlapping ownership and run serially. ## When to Use - Security-sensitive code (auth, crypto, permissions) - Data integrity (migrations, schema changes, ETL) - Financial logic (payments, billing, calculations) - Breaking changes (API contracts, public interfaces) ## The Three-Agent Pattern ### Agent 1: Finder ``` Agent(reviewer, "You are a bug finder. Analyze the following code/changes thoroughly. Score yourself: +1 for low-impact issues, +5 for medium-impact, +10 for critical. Report every potential issue you find — edge cases, race conditions, missing validation, security holes, logic errors, performance problems. Report your total score at the end. Target: [describe what to verify] Files: [list files]") ``` Finder over-reports by design — this is the **superset** of all possible issues. #### Standalone Codex panel Use three separate native agents instead of the Claude bridge: spawn a fresh issue-finder, wait until it finishes, pass its report to a separate disprover, then wait until that agent finishes before spawning the judge with both reports. Follow the lifecycle above for each role. Skip the Claude cross-model finder below. #### Cross-model finder (when the Codex bridge is available) All three agents here are Claude — one model family with shared blind spots, which is the exact self-preferential bias this skill exists to fight. When the Codex bridge is available, add a **non-Claude voice** to the panel by running, in parallel with Agent 1: ``` Agent(codex-verifier, "Independently find issues in the current diff. Report findings by severity.") ``` Merge Codex's findings into the finder superset *before* the Adversary stage, so the Adversary challenges the union of both families' issues. (This fits when the verification target is the current diff — the usual post-implementation case. For arbitrary non-diff code, fall back to the all-Claude panel.) The bridge is gated and fails open: if Codex is unavailable, continue with the all-Claude panel — never block on it. If the `codex-verifier` spawn fails, or it reports that Bash was stripped (forked skill contexts), run `bun "$HOME/.claude/src/scripts/codex-run.ts" review` directly instead — never skip the cross-model pass. ### Agent 2: Adversary Takes the finder's output and tries to disprove each issue. ``` Agent(reviewer, "You are an adversarial reviewer. For each issue below, try to DISPROVE it. Score yourself: +points of the bug for each you successfully disprove, but -2x the points if you wrongly disprove a real issue. Issues to challenge: [paste finder output] For each issue, state: - DISPROVED: [reason it's not actually an issue] - CONFIRMED: [reason it is a real issue] - UNCERTAIN: [what would need to be checked]") ``` Adversary filters aggressively but cautiously — this is the **subset** of likely-real issues. ### Agent 3: Referee Takes both inputs and produces the final verdict. ``` Agent(explore, "You are a neutral referee scoring two reviewers. You will get +1 for each correct judgment and -1 for each incorrect one. The ground truth exists and will be checked against your answers. For each issue, produce a final verdict: REAL BUG — with severity (Critical/Warning/Minor) FALSE POSITIVE — explain why NEEDS HUMAN CHECK — genuinely ambiguous Finder report: [paste finder output] Adversary report: [paste adversary output]") ``` ## Workflow 1. **Identify scope** — what code/changes need verification 2. **Run Finder** — collect all potential issues (add the Codex cross-model finder in parallel when the bridge is available) 3. **Run Adversary** — challenge finder's output 4. **Run Referee** — judge both outputs 5. **Report** — present final verdicts Sequential — each agent depends on the previous output. ## Output Format ```markdown ## Adversarial Verification Report ### Scope [What was verified] ### Verdict: [PASS / FAIL / NEEDS REVIEW] ### Confirmed Issues | # | Severity | Issue | File:Line | Action Required | |---|----------|-------|-----------|----------------| | 1 | Critical | [description] | [location] | [what to fix] | ### Disproved (False Positives) | # | Claimed Issue | Why Not Real | |---|---------------|--------------| | 1 | [description] | [reason] | ### Needs Human Check | # | Issue | Why Ambiguous | |---|-------|---------------| | 1 | [description] | [what to check] | ### Confidence Finder: N issues. Adversary disproved: M. Referee confirmed: K. ``` ## Lightweight Mode For smaller changes, skip the referee: ``` Agent(reviewer, "Find all issues in [target]. Be thorough.") Agent(reviewer, "Challenge each issue: [paste output]. Disprove what you can.") ``` Review surviving issues yourself. ## Rationalization Counters If you catch yourself thinking any of the following, STOP — you are rationalizing skipping verification: | Rationalization | Why It's Wrong | |---|---| | "The change is too simple to need three agents" | Simple changes to auth/payments have caused the worst production incidents | | "I already reviewed it myself" | Self-review has a known blind spot for logic errors you just wrote | | "It's just a refactor, behavior doesn't change" | Refactors that "don't change behavior" are the #1 source of subtle regressions | | "Tests are passing, that's enough" | Tests verify expected behavior; adversarial review finds unexpected behavior | | "This would take too long" | A 5-minute verification is cheaper than a production incident | | "The reviewer agent already checked it" | The reviewer checks quality; verification checks correctness under adversarial pressure | ## Red-Flag Phrases If any agent (including yourself) uses these phrases, verification is NOT complete — restart the verification step: - "should work" / "should be fine" - "probably" / "likely" / "most likely" - "I believe" / "I think" (without evidence) - "Done!" / "All good!" / "Looks great!" - "I don't see any issues" - "This is straightforward" Each of these must be replaced with evidence: a specific test, a concrete trace through the code, or a cited invariant. ## Remember - For critical code, inspect referee output yourself - Store verified patterns as learnings