# AutoScientists — Coordination-Mechanism Testbed Exemplar

A deterministic, ablatable reference implementation of the five coordination
mechanisms from Gao, Fang & Zitnik, *AutoScientists: Self-Organizing Agent Teams
for Long-Running Scientific Experimentation* (arXiv:2605.28655): a shared
champion/experiment-log state, a dead-end registry, effect-size ranking,
noise-band confirmation, and stagnation-driven team reorganization.

It is a **microscope on the coordination primitives**, not a reproduction of the
full agentic system. The deterministic core is fully tested; a live
`HermesProposer` (Ollama) drops into the same loop for an opt-in agentic demo.

## When to use this template

Use this template when you need to **isolate and measure agent-coordination
mechanisms**: coordinated teams vs single-thread baselines under matched
budgets, deterministic ablation studies, and honest per-mechanism effect
reporting (including nulls). Choose
[`template_autoresearch_project`](../template_autoresearch_project/) instead
if your focus is end-to-end AutoResearch **loop infrastructure** (evidence
registries, review gates, artifact integrity) rather than coordination
primitives. Full roster:
[`projects/AGENTS.md`](../../AGENTS.md#permanent-canonical-exemplars).

## Publication and rendering

The publishing metadata and per-platform status below are **compiled from
`manuscript/config.yaml`** by `infrastructure.publishing.status_report` — do not
hand-edit between the markers; update the config and regenerate (see the legend).

<!-- PUBLISHING-STATUS:START (generated by infrastructure.publishing.status_report) -->
**A Deterministic Testbed for Self-Organizing Agent-Team Coordination** · v1.0.2 · MIT · Daniel Ari Friedman

Concept DOI: [10.5281/zenodo.20533669](https://doi.org/10.5281/zenodo.20533669) | Version DOI: [10.5281/zenodo.20931927](https://zenodo.org/records/20931927) | Repository: [docxology/template_autoscientists](https://github.com/docxology/template_autoscientists)

Publishing surface — 20 platforms, 9 published:

| Platform | Tier | Status | Reference | Credentials |
| --- | --- | --- | --- | --- |
| zenodo | first-class | ✅ published | [10.5281/zenodo.20533669](https://doi.org/10.5281/zenodo.20533669) | `ZENODO_API_TOKEN` |
| github | first-class | ✅ published | [docxology/template_autoscientists](https://github.com/docxology/template_autoscientists) | `GITHUB_TOKEN` |
| arxiv | first-class | ⚪ available | — | — |
| pypi | first-class | ✅ published | [https://test.pypi.org/project/template-autoscientists/1.0.2/](https://test.pypi.org/project/template-autoscientists/1.0.2/) | `PYPI_TOKEN`, `TESTPYPI_TOKEN` |
| ipfs_pinata | first-class | ✅ published | [https://gateway.pinata.cloud/ipfs/QmX6F3ExeC87DxxH95YgJuHcJmjhXuf1x54BKF6V5bn5z4](https://gateway.pinata.cloud/ipfs/QmX6F3ExeC87DxxH95YgJuHcJmjhXuf1x54BKF6V5bn5z4) | `PINATA_JWT` |
| ipfs_web3storage | first-class | ⚪ available | — | `WEB3_STORAGE_TOKEN` |
| software_heritage | first-class | ✅ published | [https://archive.softwareheritage.org/browse/origin/?origin_url=https://github.com/docxology/template_autoscientists](https://archive.softwareheritage.org/browse/origin/?origin_url=https://github.com/docxology/template_autoscientists) | — |
| github_pages | first-class | ✅ published | [https://docxology.github.io/template_autoscientists/](https://docxology.github.io/template_autoscientists/) | `GITHUB_TOKEN` |
| cloudflare_pages | first-class | ⚪ available | — | `CLOUDFLARE_API_TOKEN` |
| netlify | first-class | ✅ published | [https://6a44401a17ba160047501e98--tranquil-kleicha-0c9203.netlify.app](https://6a44401a17ba160047501e98--tranquil-kleicha-0c9203.netlify.app) | `NETLIFY_AUTH_TOKEN` |
| huggingface_hub | first-class | ✅ published | [https://huggingface.co/datasets/ActiveInference/template_autoscientists](https://huggingface.co/datasets/ActiveInference/template_autoscientists) | `HUGGINGFACE_TOKEN`, `HF_TOKEN` |
| osf | first-class | ✅ published | [https://osf.io/6xcf5/](https://osf.io/6xcf5/) | `OSF_TOKEN` |
| amazon_kdp | documented | 🟡 planned | — | `AMAZON_KDP_EMAIL`, `AMAZON_KDP_PASSWORD` |
| google_play_books | documented | 🟡 planned | — | `GOOGLE_PLAY_BOOKS_SERVICE_ACCOUNT_JSON` |
| gumroad | documented | 🟡 planned | — | `GUMROAD_ACCESS_TOKEN` |
| leanpub | documented | 🟡 planned | — | `LEANPUB_API_KEY` |
| lulu | documented | 🟡 planned | — | `LULU_CLIENT_KEY`, `LULU_CLIENT_SECRET` |
| draft2digital | documented | 🟡 planned | — | `DRAFT2DIGITAL_API_TOKEN` |
| stripe | documented | 🟡 planned | — | `STRIPE_SECRET_KEY`, `STRIPE_PUBLISHABLE_KEY` |
| ingramspark | documented | 🟡 planned | — | `INGRAMSPARK_CLIENT_ID`, `INGRAMSPARK_CLIENT_SECRET` |

_Keywords: agent coordination, scientific discovery, noise-band confirmation, ablation study, reproducible research, language-model agents._

_Status legend: ✅ published (durable identifier recorded in `config.yaml`) · 🔵 reserved (identifier reserved but not yet registered by final publication) · ⚪ available (adapter implemented and locally verifiable) · 🟡 planned. This block is generated — edit `manuscript/config.yaml`, then regenerate with `uv run python -m infrastructure.publishing.status_report --project <path> --write`._
<!-- PUBLISHING-STATUS:END -->

The 3 platforms still shown ⚪ available are not automatable to "published" with
current tooling/credentials, not an oversight: **arXiv** has no submission API
in this codebase (`infrastructure.publishing.arxiv` only prepares a local
tarball — a human must upload it via arxiv.org and the resulting `arxiv` URL
would then be added to `publication.published_artifacts`); **Cloudflare
Pages** needs a `CLOUDFLARE_ACCOUNT_ID` the configured API token cannot
auto-discover; **IPFS (Web3.Storage)** has no `WEB3_STORAGE_TOKEN` configured.

- Canonical renderer: [docxology/template](https://github.com/docxology/template) with `--project templates/template_autoscientists`
- Tracked outputs: [`output/`](output/) in this project and `output/templates/template_autoscientists/` in the monorepo; public output files above 50 MB stay out of git.

To regenerate this exemplar from the public monorepo:

```bash
git clone https://github.com/docxology/template
cd template
uv sync
./run.sh --project templates/template_autoscientists --pipeline --core-only
uv run python scripts/pipeline/stage_04_validate.py --project templates/template_autoscientists
uv run python scripts/pipeline/stage_05_copy.py --project templates/template_autoscientists
```

Standalone repositories are publication mirrors for source, DOI metadata, and
tracked rendered artifacts. Use the monorepo above when you need the full shared
infrastructure, pipeline stages, or cross-template validation.

## What it honestly shows

The testbed is constructed so that **no performance claim would be honest**, and
it makes none. Under a *matched sequential experiment budget*, coordinated teams
**partition** one budget rather than adding parallel compute, so they cannot beat
a single thread on the final answer. The measured results (regenerated by the
scripts below) separate three honest axes:

| Axis | Mechanism | Measured effect on this objective |
| --- | --- | --- |
| Final answer | team partitioning | **None.** Every configuration reaches the clean optimum (clean metric `0.0`); coordinated−baseline advantage is exactly `0.0`. Partitioning across 3 teams even reaches the optimum a few experiments *later* (exp 16 vs 12) — coordination is **not** faster here. |
| Search hygiene | dead-end registry | **Real.** The registry-consulting proposer cuts redundant re-probes of retired directions from **36 → 0** and halts at **36** experiments instead of burning all **60** — same clean answer, less waste. |
| Noise robustness | noise-band confirmation | **Real.** ~**13×** smaller reported-vs-clean gap (`0.0156 → 0.0012`). |
| — | effect-size ranking, reorganization | **Null on this objective.** Identical to full coordination; reported plainly. |

These magnitudes are properties of *this* synthetic objective, budget, and
deterministic proposer — they illustrate the measurement, not a universal
constant.

## Quick Start

```bash
# Coordinated vs single-thread baseline (matched budget) + summary JSON/figure
uv run python projects/templates/template_autoscientists/scripts/run_search_comparison.py

# Per-mechanism ablation (metric figure + efficiency/hygiene figure)
uv run python projects/templates/template_autoscientists/scripts/run_ablation.py

# Deterministic test suite (live Hermes path is opt-in, see below)
uv run pytest projects/templates/template_autoscientists/tests -m "not requires_ollama"
```

## Prerequisites & verification

**Test/coverage gate (authoritative per-project command).** Exit code 0 alone is
not proof — confirm tests collected > 0 and coverage clears the 90% `src/` gate
(read the `TOTAL` line of the report for the live figure):

```bash
uv run pytest projects/templates/template_autoscientists/tests \
  -m "not requires_ollama" \
  --cov=projects/templates/template_autoscientists/src --cov-fail-under=90
```

**Full pipeline render (qualified name required).** This exemplar lives under
`projects/templates/`, so the pipeline must be invoked with the *qualified*
project name — a bare name fails environment setup:

```bash
uv run python scripts/runner/execute_pipeline.py --project templates/template_autoscientists --core-only
```

## The language-model plug-in seam

The coordination loop depends only on the `Proposer` protocol
([`src/agents.py`](src/agents.py)). Two real implementations are provided:

- `DeterministicProposer` — rule-based, fully deterministic, drives the tested
  exemplar runs and rendered figures. It consults the dead-end registry (the
  `avoid` set) so retired directions actually steer the search.
- `HermesProposer` — the live agent. It renders the shared state (champion,
  recent experiments, retired directions) to a prompt and asks a Hermes model
  served by Ollama which axis/step to try next, then parses the structured
  reply. Swapping it in is the *only* change needed to turn the deterministic
  reference run into a live agentic one.

Run the live path (requires `ollama serve` + `ollama pull hermes3`):

```bash
uv run pytest projects/templates/template_autoscientists/tests/test_hermes_live.py -m requires_ollama
```

The live network boundary is `# pragma: no cover` and excluded from the gate —
the deterministic core is the gated surface.

## No-mocks policy

Every test uses real deterministic objects and computations — no `MagicMock`,
no `mocker.patch`. The "alternating proposer" and Hermes reply-parser tests are
genuine objects exercising real code paths.

## More information

See [AGENTS.md](AGENTS.md) for module-level technical documentation and the
manuscript under [`manuscript/`](manuscript/) for the full write-up.

## Template integrity

- Forward backlog: [`TODO.md`](TODO.md).
- Copy-and-customize config: [`manuscript/config.yaml.example`](manuscript/config.yaml.example).
- Project validation: `uv run pytest projects/templates/template_autoscientists/tests/ --cov=projects/templates/template_autoscientists/src --cov-fail-under=90`.
- Repo drift validation: `uv run python scripts/audit/check_template_drift.py --strict`.