--- name: harness-refinement description: >- Run a bounded, evidence-based comparison of alternative agent harness configurations for a difficult or repeatedly failing task. Use when a task's quality or cost could plausibly improve through changes to prompts, guardrails, memory, tool guidance, subagents, skills, scripts, or plugins. --- # Harness refinement ## When to use Use for complex tasks with uncertain execution strategy or after a verified failed attempt. Skip routine work where an experiment costs more than the task. ## Required context Read the approved task plan, repository verification profile, and relevant harness instructions. Record the exact base revision and fixed acceptance criteria before comparing variants. ## Preconditions The task must have a bounded outcome, repeatable checks, and an approved implementation plan. If the host cannot create isolated worktrees or report usage, record that limitation and do not claim a measured efficiency gain. ## Procedure 1. Define one baseline and at most two candidates. Each candidate changes one or a small, named set of harness elements and states a rationale. 2. Put alternatives in one `experiment.group` in the approved delegation graph. Give each the same dependencies, read scope, write scope, acceptance criteria, and base revision. Candidates are alternatives, not cumulative tasks. 3. Approve the exact graph. Start every experiment task in an isolated worktree; never run alternatives against a shared mutable workspace. 4. Run the same verification and independent review process for every alternative. Record criterion-level results, changed paths, host-reported model/effort, usage, and duration. Treat usage as host-reported evidence. 5. Record the paired measurements in the local harness-evaluation format, including shared checks, repeated runs, review outcomes and host measurements. Use the configured analyzer to validate pairing and get a deterministic recommendation. 6. Compare quality first. Reject candidates that fail a required criterion or regress a criterion the baseline passed. Compare tokens, cost, and duration only when available on both sides. 7. Keep the baseline when results tie, evidence is incomplete, or efficiency gains are not measured. Adopt a candidate only through the normal human review and change workflow. 8. Retain the experiment record and explain limits. Do not turn a single task result into a persistent rule. Repeated findings may be submitted as Gardener evidence for human review. ## Tools Use `core.repository.diff` to inspect changed files, host-managed Claude Code or Codex sessions, and Paved's delegation command. Use the repository's own verification commands for checks. ## Rules Follow the approved-plan, verification, and evidence requirements in the repository instructions and workflow. For implementation, follow `core.development.feature-development`. For a regression claim, follow `core.testing.regression-testing`. Before adopting a candidate, follow `core.code-review.change-review`. ## Failure handling If isolation, equivalent criteria, or independent review cannot be established, stop the comparison and report it as inconclusive. Never infer improvement from model choice or agent self-report alone. ## Verification Use the repository's configured checks and review procedure for every variant. The experiment comparison supplements those gates; it does not replace them. ## Evidence Record the approved graph and digest, base revision, candidate change identifiers, per-criterion outcomes and evidence references, review outcomes, host-reported usage, and the final human decision. ## Completion criteria - Every variant has equivalent checks, scopes, and an independent review. - The comparison names measured benefits and limitations or is marked inconclusive. - No persistent harness policy changes without human review. ## Failure modes | Failure | Signal | Response | |---|---|---| | Candidates contaminate one another | Shared workspace or differing base revisions | Mark comparison inconclusive and rerun isolated | | Self-reported success lacks evidence | Missing check or review references | Do not count the result as a pass | | Apparent savings lack comparable usage | One side has no host measurement | Report the metric unavailable | ## References The Core documentation describes the harness-refinement concept and paired delegation evaluation protocol. ## Paved integration Use the project-local Paved contract as the source of truth. Discover state with `node "/scripts/paved.mjs" status --json`, load only relevant `.paved/project/` context, and invoke verification with `node "/scripts/paved.mjs" verify --json`. `` is the installed Paved plugin directory, two directories above this SKILL.md. Always run the launcher from the repository being worked on, never from the plugin directory. Do not edit generated integration files or bypass Paved lifecycle and ownership checks.