# template_autoresearch_project

Public exemplar for a deterministic bounded AutoResearch workflow over a small
local machine-learning task.

This project demonstrates a file-backed AutoResearch loop that runs inside the
normal template pipeline. The case study uses a local balanced MNIST subset, a
nearest-centroid baseline, and a bounded set of numpy-only neural-network
candidates: softmax regression, a small MLP, and a tiny patch-attention
classifier.

## When to use this template

Use this template when you need a **bounded, offline AutoResearch loop**:
deterministic ML candidate evaluation over a fixed local dataset, with
evidence-linked claims, machine-readable ledgers, artifact-integrity
manifests, and deferred human-review gates. It demonstrates how to make the
research *process* inspectable without claiming autonomous discovery.
Metric claims come from local execution artifacts: benchmark-style command
output may use exact `METRIC name=value` lines, candidate outcomes stay in
explicit keep/reject/defer-style ledgers, and confidence claims must disclose
baseline, best result, and the measurable noise floor rather than standing in
for human publication approval.

Choose [`template_autoscientists`](../template_autoscientists/) instead if
your focus is agent-team **coordination primitives** (dead-end registries,
noise-band confirmation, stagnation-driven reorganization) rather than
AutoResearch loop infrastructure. For the full exemplar roster see
[`projects/AGENTS.md`](../../AGENTS.md#permanent-canonical-exemplars).

## Publication and rendering

The publishing metadata and per-platform status below are **compiled from
`manuscript/config.yaml`** by `infrastructure.publishing.status_report` — do not
hand-edit between the markers; update the config and regenerate (see the legend).

<!-- PUBLISHING-STATUS:START (generated by infrastructure.publishing.status_report) -->
**Bounded AutoResearch for a Tiny Reproducible Machine-Learning Task** · v0.3.2 · MIT · Daniel Ari Friedman

Concept DOI: [10.5281/zenodo.20417016](https://doi.org/10.5281/zenodo.20417016) | Version DOI: [10.5281/zenodo.20931907](https://zenodo.org/records/20931907) | Repository: [docxology/template_autoresearch_project](https://github.com/docxology/template_autoresearch_project)

Publishing surface — 20 platforms, 9 published:

| Platform | Tier | Status | Reference | Credentials |
| --- | --- | --- | --- | --- |
| zenodo | first-class | ✅ published | [10.5281/zenodo.20417016](https://doi.org/10.5281/zenodo.20417016) | `ZENODO_API_TOKEN` |
| github | first-class | ✅ published | [docxology/template_autoresearch_project](https://github.com/docxology/template_autoresearch_project) | `GITHUB_TOKEN` |
| arxiv | first-class | ⚪ available | — | — |
| pypi | first-class | ✅ published | [https://test.pypi.org/project/template-autoresearch-project/0.3.2/](https://test.pypi.org/project/template-autoresearch-project/0.3.2/) | `PYPI_TOKEN`, `TESTPYPI_TOKEN` |
| ipfs_pinata | first-class | ✅ published | [https://gateway.pinata.cloud/ipfs/QmZvS2v7ZpqS4T8ZufeW9sA4gNBteAdVfmLH4LcoAqrUhv](https://gateway.pinata.cloud/ipfs/QmZvS2v7ZpqS4T8ZufeW9sA4gNBteAdVfmLH4LcoAqrUhv) | `PINATA_JWT` |
| ipfs_web3storage | first-class | ⚪ available | — | `WEB3_STORAGE_TOKEN` |
| software_heritage | first-class | ✅ published | [https://archive.softwareheritage.org/browse/origin/?origin_url=https://github.com/docxology/template_autoresearch_project](https://archive.softwareheritage.org/browse/origin/?origin_url=https://github.com/docxology/template_autoresearch_project) | — |
| github_pages | first-class | ✅ published | [https://docxology.github.io/template_autoresearch_project/](https://docxology.github.io/template_autoresearch_project/) | `GITHUB_TOKEN` |
| cloudflare_pages | first-class | ⚪ available | — | `CLOUDFLARE_API_TOKEN` |
| netlify | first-class | ✅ published | [https://6a443f868786d30c7d8ea014--tranquil-kleicha-0c9203.netlify.app](https://6a443f868786d30c7d8ea014--tranquil-kleicha-0c9203.netlify.app) | `NETLIFY_AUTH_TOKEN` |
| huggingface_hub | first-class | ✅ published | [https://huggingface.co/datasets/ActiveInference/template_autoresearch_project](https://huggingface.co/datasets/ActiveInference/template_autoresearch_project) | `HUGGINGFACE_TOKEN`, `HF_TOKEN` |
| osf | first-class | ✅ published | [https://osf.io/geykb/](https://osf.io/geykb/) | `OSF_TOKEN` |
| amazon_kdp | documented | 🟡 planned | — | `AMAZON_KDP_EMAIL`, `AMAZON_KDP_PASSWORD` |
| google_play_books | documented | 🟡 planned | — | `GOOGLE_PLAY_BOOKS_SERVICE_ACCOUNT_JSON` |
| gumroad | documented | 🟡 planned | — | `GUMROAD_ACCESS_TOKEN` |
| leanpub | documented | 🟡 planned | — | `LEANPUB_API_KEY` |
| lulu | documented | 🟡 planned | — | `LULU_CLIENT_KEY`, `LULU_CLIENT_SECRET` |
| draft2digital | documented | 🟡 planned | — | `DRAFT2DIGITAL_API_TOKEN` |
| stripe | documented | 🟡 planned | — | `STRIPE_SECRET_KEY`, `STRIPE_PUBLISHABLE_KEY` |
| ingramspark | documented | 🟡 planned | — | `INGRAMSPARK_CLIENT_ID`, `INGRAMSPARK_CLIENT_SECRET` |

_Keywords: autoresearch, reproducible research, machine learning benchmark, artifact readiness, human review, local artifact integrity._

_Status legend: ✅ published (durable identifier recorded in `config.yaml`) · 🔵 reserved (identifier reserved but not yet registered by final publication) · ⚪ available (adapter implemented and locally verifiable) · 🟡 planned. This block is generated — edit `manuscript/config.yaml`, then regenerate with `uv run python -m infrastructure.publishing.status_report --project <path> --write`._
<!-- PUBLISHING-STATUS:END -->

The 3 platforms still shown ⚪ available are not automatable to "published" with
current tooling/credentials, not an oversight: **arXiv** has no submission API
in this codebase (`infrastructure.publishing.arxiv` only prepares a local
tarball — a human must upload it via arxiv.org and the resulting `arxiv` URL
would then be added to `publication.published_artifacts`); **Cloudflare
Pages** needs a `CLOUDFLARE_ACCOUNT_ID` the configured API token cannot
auto-discover; **IPFS (Web3.Storage)** has no `WEB3_STORAGE_TOKEN` configured.

- Canonical renderer: [docxology/template](https://github.com/docxology/template) with `--project templates/template_autoresearch_project`
- Tracked outputs: [`output/`](output/) in this project and `output/templates/template_autoresearch_project/` in the monorepo; public output files above 50 MB stay out of git.

To regenerate this exemplar from the public monorepo:

```bash
git clone https://github.com/docxology/template
cd template
uv sync
./run.sh --project templates/template_autoresearch_project --pipeline --core-only
uv run python scripts/pipeline/stage_04_validate.py --project templates/template_autoresearch_project
uv run python scripts/pipeline/stage_05_copy.py --project templates/template_autoresearch_project
```

Standalone repositories are publication mirrors for source, DOI metadata, and
tracked rendered artifacts. Use the monorepo above when you need the full shared
infrastructure, pipeline stages, or cross-template validation.

## Configuration

Use [`manuscript/config.yaml`](manuscript/config.yaml) as the live metadata,
render, analysis, project-config, approval-boundary, and publication surface.
Keep [`manuscript/config.yaml.example`](manuscript/config.yaml.example) in
top-level parity with placeholder-safe values so a fork can start without
project-specific secrets or release metadata. The bounded ML task, source
ledger, human-review boundary, and generated artifact contracts remain
offline-first and deterministic.

## Quick start

```bash
./run.sh --pipeline --project templates/template_autoresearch_project --core-only --skip-infra
```

The per-project test/coverage gate (≥90% on `src/`, enforced by
`pyproject.toml` `fail_under = 90`):

```bash
uv run pytest projects/templates/template_autoresearch_project/tests/ \
  --cov=projects/templates/template_autoresearch_project/src --cov-fail-under=90
```

The analysis stage runs two thin scripts:

- `scripts/run_autoresearch_loop.py` builds the ML-loop result, plan, claims,
  stage matrix, review packet, method ledgers, benchmark scores, final figures,
  evidence registry snapshot, schema manifest, local research-object manifest,
  phase ledger, figure-quality report, artifact manifest, readiness report, and
  manuscript-hydration sidecars through `src.loop.run_autoresearch_loop`.
- `scripts/z_generate_manuscript_variables.py` hydrates manuscript variables
  into `output/manuscript/` for rendering and fails when strict run-derived
  manuscript values are not tokenized.

Reusable behavior lives under `src/` in typed packages (`loop`, `ml.data`,
`ml.models`, `ml.training`, `ml.selection`, `ml.task`, `diagnostics.records`,
`diagnostics.metrics`, `diagnostics.intervals`, `diagnostics.reports`,
`models`, `config`, `writers`, `reports`, `figures`, `manuscript_variables`).
No network calls, LLM calls, runtime dataset downloads, generated-code
execution, or autonomous approval loops are used.

The manuscript frames the exemplar as a bounded research-object analogue:
machine-readable ledgers, artifact manifests, figure registry metadata,
variable provenance, phase-settlement records, figure-quality checks, and
deferred review gates make the research process itself inspectable without
claiming autonomous discovery.
Each registered figure carries a source artifact, generation method, validation
hook, alt text, caption, and claim boundary; manuscript figure blocks and the
figure-method table are hydrated from that registry.
The local security layer adds a deterministic threat model, SBOM-style
inventory, checksum attestation, and adversarial review packet. These artifacts
support local research-artifact integrity claims only; the default run performs
no external signing and does not claim production SLSA compliance.
The reviewer-facing evidence registry report is compact by default: validation
still builds the full in-memory fact registry, while
`output/reports/evidence_registry.json` records counts, source tiers,
freshness warnings, and a bounded fact sample. Set
`TEMPLATE_EVIDENCE_REGISTRY_FULL=1` only for local debugging when a full
`output/reports/evidence_registry_full.json` dump is needed.

Loop stages are recorded as **declared** (configured intent). Claims are
**supported** only when their evidence file exists locally **and carries
substantive (non-empty, parseable) content** — an empty or hollow placeholder
does not support a claim. The same substance check guards the figure-quality and
benchmark gates, so a structurally-complete-but-hollow run is scored as
incomplete rather than silently certified.

**Validation boundary.** The claim, figure-quality, and benchmark gates bind to
substantive content and are exercised by fault-injecting negative-control tests
(`tests/test_gate_negative_controls.py`, `tests/test_gate_improvements.py`) that
prove each one fails closed. The former self-referential gates are now hardened
into production gates:

- The **schema manifest** validates field/type *conformance* for the governance
  schemas; a nonconforming payload is a **hard gate** — `write_schema_manifest`
  raises and aborts the loop rather than writing a green manifest.
- The **security/integrity attestation** fails a present-but-empty required file
  and cross-checks the input MNIST fixture against its *committed declared* hash
  (external truth, failing closed if that declared hash is absent).

One check is deliberately **opt-in, not default-enforced**, for a principled
reason: the **evidence registry** can require strict-zone manuscript numbers to
trace to trusted source tiers (external/input/declared, not the run's own
`generated_metric` output) via `validate_text_against_registry(...,
trusted_number_tiers=...)`. But an AutoResearch manuscript legitimately *reports
its own run's metrics* — the hydrated manuscript contains ~285 such strict-zone
numbers — so forcing this check on would reject the paper's own findings. The
**default** evidence gate already fails any manuscript number that matches *no*
generated artifact (fabrication); the strict-tier check is provided for
manuscripts that cite external numbers, and the registry's `source_tiers` field
discloses the provenance mix. The **research-object manifest** remains a
path/size/checksum inventory by design.

Accepted seed ideas require evidence links, candidate edits are bounded by
`edit_allowlist`, and configured review gates are recorded as human-review
inputs rather than self-approval. The generated review decisions are `deferred`
so a human reviewer still owns publication approval. A true publication
approval can only be read from the human-authored `human_review.yaml` file.

## Loop orchestration

```mermaid
flowchart TB
  plan[build_autoresearch_plan] --> intrinsic[validate phase=intrinsic]
  intrinsic --> core[write_core_loop_artifacts]
  core --> registry1[write_evidence_registry_report]
  registry1 --> ml[run_bounded_ml_task + write_ml_task_artifacts]
  ml --> claims[build_claims + finalize_loop_payloads]
  claims --> methods[write_method_contract_artifacts]
  methods --> update1[update_result_payloads provisional]
  update1 --> security1[write_security_artifacts]
  security1 --> phase1[write_phase_ledger + write_artifact_manifest]
  phase1 --> extrinsic[validate phase=extrinsic]
  extrinsic --> readiness[write_autoresearch_report combined]
  readiness --> update2[update_result_payloads final]
  update2 --> visuals[write_final_visual_artifacts]
  visuals --> hydrate[write_manuscript_hydration_artifacts]
  hydrate --> registry2[write_evidence_registry_report refresh]
  registry2 --> security2[write_security_artifacts final]
  security2 --> phase2[write_phase_ledger + write_artifact_manifest final]
```

Project-specific docs live in [`docs/`](docs/).
The project-level next-work roadmap lives in [`TODO.md`](TODO.md).

## Outputs

- `output/data/autoresearch_plan.json`
- `output/data/autoresearch_loop.json`
- `output/data/autoresearch_claims.json`
- `output/data/autoresearch_stage_matrix.csv`
- `output/data/autoresearch_review_packet.json`
- `output/data/research_program.json`
- `output/data/idea_ledger.json`
- `output/data/run_ledger.json`
- `output/data/review_decisions.json`
- `output/data/benchmark_scores.json`
- `output/data/mnist_task_config.json`
- `output/data/ml_task_results.json`
- `output/data/ml_candidate_ledger.json`
- `output/data/ml_confusion_matrix.csv`
- `output/data/ml_training_history.csv`
- `output/data/ml_error_examples.json`
- `output/data/ml_prediction_records.json`
- `output/data/ml_classification_diagnostics.json`
- `output/data/ml_candidate_intervals.json`
- `output/data/ml_class_balance.json`
- `output/data/ml_calibration_report.json`
- `output/data/ml_calibration_bin_intervals.json`
- `output/data/ml_robustness_report.json`
- `output/data/ml_probability_diagnostics.json`
- `output/data/ml_bootstrap_intervals.json`
- `output/data/ml_paired_comparison.json`
- `output/data/ml_statistical_summary.json`
- `output/data/ml_training_diagnostics.json`
- `output/data/ml_candidate_rank_stability.json`
- `output/data/ml_candidate_selection_audit.json`
- `output/data/ml_diagnostic_boundary.json`
- `output/data/autoresearch_phase_ledger.json`
- `output/data/figure_quality_report.json`
- `output/data/autoresearch_security_profile.json`
- `output/data/autoresearch_threat_model.json`
- `output/data/autoresearch_supply_chain_inventory.json`
- `output/data/autoresearch_inventory_export.json`
- `output/data/autoresearch_integrity_attestation.json`
- `output/data/autoresearch_schema_manifest.json`
- `output/data/research_object_manifest.json`
- `output/data/manuscript_variables.json`
- `output/data/manuscript_variable_provenance.json`
- `output/data/manuscript_figure_blocks.json`
- `output/data/autoresearch_evidence_overview.json`
- `output/data/benchmark_boundary.json`
- `output/reports/autoresearch_evidence_overview.md`
- `output/data/publication_ledger.json` — release-stage; written by `write_transmission_bookends()` when `manuscript/config.yaml`'s `publication.transmission_bookends.enabled` is `true` (see [`docs/guides/publishing-guide.md`](../../../docs/guides/publishing-guide.md))
- `output/data/transmission_manifest.json` — release-stage; see `publication.transmission_bookends` above
- `output/figures/autoresearch_stage_matrix.png`
- `output/figures/ml_candidate_scores.png`
- `output/figures/ml_confusion_matrix.png`
- `output/figures/ml_per_class_accuracy.png`
- `output/figures/ml_learning_curves.png`
- `output/figures/ml_complexity_accuracy.png`
- `output/figures/mnist_error_examples.png`
- `output/figures/ml_calibration_reliability.png`
- `output/figures/ml_classification_metrics_heatmap.png`
- `output/figures/ml_confusion_pairs.png`
- `output/figures/ml_generalization_gap.png`
- `output/figures/ml_robustness_matrix.png`
- `output/figures/ml_probability_margin_distribution.png`
- `output/figures/ml_bootstrap_intervals.png`
- `output/figures/ml_paired_correctness.png`
- `output/figures/ml_selective_accuracy.png`
- `output/figures/ml_probability_quality.png`
- `output/figures/ml_training_dynamics.png`
- `output/figures/ml_candidate_rank_stability.png`
- `output/figures/autoresearch_candidate_lifecycle.png`
- `output/figures/mnist_class_balance.png`
- `output/figures/mnist_subset_contact_sheet.png`
- `output/figures/autoresearch_closure_flow.png`
- `output/figures/autoresearch_security_control_matrix.png`
- `output/figures/autoresearch_integrity_chain.png`
- `output/figures/transmission_integrity_strip.png` — release-stage; see `publication.transmission_bookends` note above
- `output/figures/transmission_pairing.png` — release-stage; see `publication.transmission_bookends` note above
- `output/figures/figure_registry.json`
- `output/reports/autoresearch_loop.json`
- `output/reports/autoresearch_loop.md`
- `output/reports/autoresearch_review_packet.md`
- `output/reports/autoresearch_summary.md`
- `output/reports/autoresearch_security_review.md`
- `output/reports/ml_experiment_report.md`
- `output/reports/ml_benchmark_score.json`
- `output/reports/autoresearch_readiness.json`
- `output/reports/autoresearch_readiness.md`
- `output/reports/benchmark_readiness_smoke.json`
- `output/reports/evidence_registry.json`
- `output/reports/artifact_manifest.json`

`output/reports/evidence_registry.json` is intentionally a compact summary, not
the full validation registry. Full fact serialization is opt-in with
`TEMPLATE_EVIDENCE_REGISTRY_FULL=1` and is not a required artifact.

## Tests

```bash
uv run python scripts/pipeline/stage_01_test.py --project templates/template_autoresearch_project --project-only --quiet
```

<!-- foam-orphan-nav:start (hand-maintained: links sub-docs so they are reachable; no generator refreshes or validates this block) -->

## Directory & sub-document map

Navigation links to in-tree documents (keeps them discoverable):

- [Data Notes](data/AGENTS.md)
- [Data](data/README.md)
- [Human-authored research program](program.md)
- [Scripts - Agent Notes](scripts/AGENTS.md)
- [Scripts](scripts/README.md)

<!-- foam-orphan-nav:end -->