--- name: scientific-writer description: > Use when the user has a scientific draft (with its dataset, figures, and optional analysis code) and wants it iteratively revised until it clears a quality bar. Five specialist judges (figures, scientific content, style, formatting, code) critique the draft; a fresh, independent peer_reviewer grades it on those same axes (1-5 each → a percentage) with honesty guardrails; a scientific_writer revises prose, figures, and code — regenerating figures by running the user's plot command — until the peer-review score clears the threshold or the budget is hit. Not for writing a paper from a blank page, and not for a standalone literature survey. compatibility: Requires Python 3.9+ metadata: version: "0.1.0" --- # Scientific Writer Loop The artifact is a **piece of scientific writing** (draft + its dataset + figures + optional code). Each iteration **critiques → grades → revises**: five specialist judges produce concrete findings, an independent **peer_reviewer** turns the paper into a graded 0-100 score on the same axes, and a **scientific_writer** fixes the prose, figures, and code — running the user's `` to regenerate figures. The loop runs until the score clears `` or the budget is hit. All work happens on copies inside a sandbox; the user's originals are never touched. The cast (all in this folder): - `roles/figures_judge.md`, `roles/scientific_judge.md`, `roles/style_judge.md`, `roles/formatting_judge.md`, `roles/code_reviewer.md` — the five critics; each emits the shared `schemas/finding.schema.json`. - `roles/peer_reviewer.md` — the summative grader (its own honesty rules); emits `schemas/peer_review.schema.json` and decides `pass`. - `roles/scientific_writer.md` — the reviser; fixes code → regenerates figures → updates prose. - `schemas/finding.schema.json`, `schemas/peer_review.schema.json` — the two validated outputs. **Spawn-or-degrade.** On Claude Code, spawn the active judges as real `Agent` subagents **in parallel**, then one fresh `peer_reviewer`, then the `scientific_writer`; otherwise adopt each role inline. You are the orchestrator. ## Why the grader is built this way (the honesty problem) The peer_reviewer grades on the **same axes** the judges critique — which invites echoing, inflation under loop-termination pressure, and a writer that games the rubric. `roles/peer_reviewer.md` counters this: it (1) grades **independently**, re-deriving each axis from the paper + dataset before reading the critiques, (2) **verifies** a sample of numbers/citations itself rather than trusting "it's fixed", (3) must surface **issues the judges missed**, (4) holds a **fixed, anchored, reproducible** bar with **no credit for effort or elapsed iterations**, (5) applies **hard gates** (a confirmed block fails the paper regardless of the average), and (6) runs a **substance check** against surface compliance. The *writer* optimizes the judges' concrete findings; the *grader* judges holistically — so "address every finding" does not mechanically buy a pass. ## When to use Use when a written scientific draft exists and the user wants it pushed past a quality bar with multi-judge critique and an independent peer-review grade. Default: run the full critique→grade→revise loop below. Escape hatch: if the user only wants the critique (no rewriting), run one round of judges + peer_reviewer and stop. Not for writing a paper from a blank page, and not for a standalone literature survey. ## Setup **Resolve bindings interactively.** If `loop.run.yaml` exists in the working dir, load it, confirm the values in one line, and skip to the loop. Otherwise: on Claude Code (the `AskUserQuestion` tool is available) infer a likely value for each binding and present it as the recommended option; on other hosts ask each as a quoted plain-text prompt. Then write `loop.run.yaml` (format: `examples/run.example.yaml`) and confirm every value — including which axes are active (4 vs 5) and the live/degraded literature tier — before creating any other files. | binding | meaning | default | how to infer | |---|---|---|---| | `` | the draft to improve (markdown/text/tex) | — | scan the working dir for a likely draft | | `` | data file(s) the claims/figures derive from — used to verify numbers | — | scan for data files near the draft | | `` | directory of figure images the draft references | — | scan near the draft; may be null | | `` | source files that produce plots/results; **empty → code axis dropped** (n_axes=4, no `code_reviewer`) | — | scan for plotting/analysis scripts | | `` | command that regenerates figures/results from the code; **null → figures/code are edit-only** (flagged "needs regeneration") | — | `pyproject.toml`/`.venv`/README; e.g. `python3 code/make_figures.py` | | `` | the single style the formatting_judge enforces (`APA`\|`MLA`) | `APA` | ask the user | | `` | optional venue whose style/length norms apply | — | ask the user | | `` | optional abstract/paper word-count targets | — | ask the user | | `` | the paper's core finding/contribution, 1-2 sentences — **frozen**; the writer may never change it | — | read the draft, extract it, confirm with the user | | `` | overall_score (0-100) the peer_reviewer must reach (and no hard gate) to stop | 85 | a solid paper without demanding perfection | | `` | max iterations | 6 | — | | `` | stop after this many consecutive no-improvement iterations | 2 | — | | `` | where working copies, critiques, reviews, and the ledger live | `./sandbox` | — | **Literature toolchain.** Citation grounding (writer) and verification (scientific_judge, peer_reviewer) go through the sibling **`literature-search` skill** — resolve `` (it installs as a sibling, e.g. `~/.claude/skills/literature-search/`), ` = python3`, and ` = /tools/lit_search.py`; append `--cache-dir /literature/.cache` after a subcommand to reuse the cache. Confirm ` --help` works at setup; if the skill is absent, tell the user and either install it (copy the repo's `loops/literature-search` folder into `~/.claude/skills/`) or degrade all retrieval to WebSearch/WebFetch. The keyless **S2 + arXiv** core works with no setup; a free `S2_API_KEY` makes `snippet`/`cite` reliable — run ` keys --init`, have the user fill the printed `keys.env` themselves, and **never paste secrets into chat**. Record the tier (presence only) in `loop.run.yaml`. **Environment.** The writer regenerates figures by running the user's `` **in the user's own environment** — that code may need third-party deps (matplotlib, pandas, …), so the skill **ships none** and never installs them; it shells out to the user's command and reads the regenerated outputs. Helper code the skill writes stays stdlib-only. `tools/lit_search.py` (in the sibling skill) is stdlib-only too. **Initialise the sandbox** once bindings are confirmed (copy in the originals; never edit them in place): ``` / ├── loop.run.yaml ← resolved bindings + + literature_tiers ├── ledger.tsv ← header only (see Ledger) ├── literature/.cache/ ← lit_search on-disk cache └── iter1/ ├── draft.md ← COPY of ├── figures/ ← COPY of └── code/ ← COPY of (omit if no code) ``` ## The loop `` starts at 1; iteration 1 critiques and grades the **unmodified** draft (the baseline — no revision before it). Re-grade fresh every iteration: the score comes only from a *new* peer review of the *revised* paper, never carried over. Surface-only changes won't move it. Copy this checklist and tick items off: - [ ] **Critique** — spawn the active judges **in parallel** (spawn-or-degrade), each over the `iter/` working copies + dataset; each writes `iter/critiques/.json` (validates against `schemas/finding.schema.json`). Skip `code_reviewer` when no code. - [ ] **Grade** — spawn **one fresh** `peer_reviewer` (`roles/peer_reviewer.md`): it grades independently (own read first, then reconcile with the critiques; spot-checks numbers/citations; surfaces missed issues), writes `iter/peer_review.json` (validates against `schemas/peer_review.schema.json`): 1-5 per active axis, `overall_score = 100 × Σscore / (5 × n_axes)`, hard gates → `pass`. - [ ] **Log** — append one `ledger.tsv` row (see Ledger). - [ ] **Stop check** — `peer_review.pass == true`, or `N == `, or `overall_score` flat for `` iterations → stop (see Stops). - [ ] **Revise** — spawn `scientific_writer` (`roles/scientific_writer.md`) with the critiques, the peer review, ``, ``, ``, and ``. It fixes block/gate items first, fixes code → regenerates figures (running the copied `` **inside the sandbox**) → updates prose, grounds new citations via ``, and writes `iter/{draft.md,figures/,code/}` + `iter/revision_notes.md`. - [ ] **`N = N + 1`** and repeat. A judge `finding` and a `peer_review` look like (abridged; full shapes in `schemas/`): ```json {"reviewer": "code_reviewer", "iteration": 1, "overall": "block", "summary": "Block: make_figures.py sorts the two columns independently, fabricating r=0.98 (true r≈0.62).", "findings": [{"urgency": "must_fix", "action_type": "replace", "area": "make_figures:broken-pairing", "finding": "xs=sorted(study); ys=sorted(score) destroys per-row pairing; r inflated to 0.98 (real ≈0.62).", "proposed_action": "Correlate the paired arrays; re-render Fig 1; update r everywhere.", "target_artifact": "iter1/code/make_figures.py", "evidence": "code/make_figures.py:~70; paired r=0.62"}]} ``` ```json {"iteration": 1, "axes": {"figures": {"score": 1, "justification": "Fig 1 plots bug-sorted data; numbers != data.", "blocking_issues": ["fig r=0.98 != paired r=0.62"]}, "scientific": {"score": 1, "justification": "Causal claim from one correlation; r unreproducible."}, "style": {"score": 2, "justification": "Promotional, AI-flavored prose."}, "formatting": {"score": 2, "justification": "Mixed APA/MLA; Results before Methods."}, "code": {"score": 1, "justification": "Independent column sort fabricates r=0.98."}}, "overall_score": 28.0, "pass": false, "gate_failures": ["make_figures.py sort bug makes r=0.98 an artifact (true 0.62)"], "issues_judges_missed": ["Methods omits the test used and n."], "spotchecks": [{"target": "paired Pearson r", "method": "recomputed from data", "result": "refuted"}], "substance_check": "Baseline iteration — nothing revised yet."} ``` ## Ledger `/ledger.tsv`, tab-separated, never commas in free text: ``` iter overall_score pass figures scientific style formatting code top_fix revision_summary 1 28.0 no 1 1 2 2 1 fix make_figures sort bug baseline (no revision) 2 61.0 no 3 3 3 3 4 report honest r, de-causalize fixed bug+claims; relabeled fig1 3 88.0 yes 4 5 4 4 5 - unified APA; trimmed abstract; balanced sleep claim ``` Use `-` in the `code` column when the code axis is absent. The per-iteration `critiques/*.json`, `peer_review.json`, and `revision_notes.md` live in `iter/`. Report the **best**-scoring iteration when stopping on budget/plateau, not necessarily the last. Leave the sandbox untracked. ## Constraints - **Never edit or run anything outside ``** — read the originals once at setup, copy them in, and work only on the copies; the `` is copied in and run from the sandbox. - **Never fabricate** data, numbers, figures, or citations — new citations come from real `` / WebFetch retrievals, quoted verbatim; on `{"error","fallback"}`, fall back to WebSearch/WebFetch. - **The grading bar is fixed and reproducible** — the peer_reviewer never relaxes anchors to let the loop finish; a confirmed hard gate fails the paper regardless of the average. - **Protect ``** — the writer strengthens the *same* finding; it never changes the core result or deletes a real finding to dodge a critique, because removing a result to raise a score is gaming. - **One coherent revision batch per iteration**, blocks/gates first, so score moves are attributable. - **No installs** — the skill ships no deps; `` runs in the user's env, helper code is stdlib-only. Never print or commit API keys (`keys.env` stays gitignored). ## Stops The loop stops on the first of: - **Pass** — `peer_review.pass == true`. Report the deliverable (`iter/` artifacts), the score, and the trajectory. - **Budget** — `N == `. Report the best-scoring iteration as the deliverable. - **Plateau** — `overall_score` flat for `` iterations. Report the best iteration + the standing `gate_failures`/`must_fix` blockers. Always end with the deliverable (`iter/` path), its `overall_score` and pass/fail, the per-axis scores, the score trajectory from `ledger.tsv`, and — if it did not pass — the standing blockers (`gate_failures` + open `must_fix`) between the paper and the bar.