# openclaw-autoresearch

Autonomous experiment loop for any optimization target.

Faithful OpenClaw port of [`davebcn87/pi-autoresearch`](https://github.com/davebcn87/pi-autoresearch), including upstream statistical confidence scoring.

## How it works

The agent runs a loop: edit code, run a benchmark, measure the result, keep or discard. Each iteration is logged. The loop runs autonomously until interrupted.

Three tools drive the loop:

| Tool | What it does |
|---|---|
| `init_experiment` | Configures the session: name, primary metric, unit, direction (lower/higher). Once runs exist, starting a new segment requires `reset: true`, and the prior segment's best result is carried forward into checkpoint context. |
| `run_experiment` | Executes a shell command, times it, captures stdout/stderr, parses `METRIC name=number` lines, and opens a pending experiment window that must be logged before another run can start. |
| `log_experiment` | Records the pending run. The first logged run in a segment is tagged as the baseline automatically. `keep` auto-commits to git. `discard`/`crash` log without committing, and `discard` now requires an `idea` note that is appended to `autoresearch.ideas.md`. If the prior `run_experiment` captured the primary metric, `log_experiment` can infer `commit` and `metric` automatically. After 3+ runs in a segment, it also reports a confidence score for the best improvement versus noise. |

In OpenClaw sessions, the plugin uses the host-provided `workspaceDir` as the normal repo root. Each tool also accepts an optional `cwd` so callers can explicitly target a nested or non-session repo when needed.

All state lives in six repo-root files:

| File | Purpose |
|---|---|
| `autoresearch.md` | Session doc. The plugin keeps the Metrics, How to Run, What's Been Tried, and Plugin Checkpoint sections synchronized so resumes are less agent-dependent. |
| `autoresearch.sh` | Benchmark script. Outputs `METRIC name=number` lines. |
| `autoresearch.jsonl` | Structured log: config headers + experiment entries (metric, status, timestamp, segment, commit hash). |
| `autoresearch.ideas.md` | Backlog of promising ideas not yet tried. Optional. |
| `autoresearch.checkpoint.json` | Plugin-managed checkpoint: latest logged state, recent runs, and any pending unlogged run. |
| `autoresearch.lock` | Session lock with PID + timestamp so another agent can detect an active or stale loop before forking a second session. |

The design is file-first: any agent can pick up the repo-root files and continue the loop without prior context.

## Install

Requires OpenClaw `2026.4.25` or newer.
Needs bash, git, and a git repo.

Use OpenClaw's plugin installer:

```bash
openclaw plugins install @gianfrancopiana/openclaw-autoresearch
```

If you're running from a local OpenClaw checkout, use:

```bash
pnpm openclaw plugins install @gianfrancopiana/openclaw-autoresearch
```

For local plugin development, link your working copy instead of copying files:

```bash
openclaw plugins install --link /absolute/path/to/openclaw-autoresearch
# or from a local OpenClaw checkout:
# pnpm openclaw plugins install --link /absolute/path/to/openclaw-autoresearch
```

For a packaged local install, build the tarball and install that artifact:

```bash
npm install
npm pack
openclaw plugins install ./gianfrancopiana-openclaw-autoresearch-<version>.tgz
```

The install command records the plugin, enables it, and makes it available
after restart. OpenClaw reads the package metadata, loads the compiled runtime
entry [`dist/index.js`](dist/index.js), and finds the manifest in
[`openclaw.plugin.json`](openclaw.plugin.json).

Verify:

- skill: `autoresearch-create`
- tools: `init_experiment`, `run_experiment`, `log_experiment`, `autoresearch_status`
- command: `/autoresearch` (recommended)
- direct skill fallback: `/skill autoresearch-create`

Prefer the explicit `/autoresearch` command surface in OpenClaw. The auto-generated native skill alias `/autoresearch_create` may not trigger reliably on some hosts, so use `/skill autoresearch-create` if you need to invoke the skill directly.

## Workflow Guarantees

- `run_experiment` refuses to start a second run until the previous one is logged.
- `run_experiment` parses `METRIC name=number` lines and stores a pending run so `log_experiment` can default from the actual benchmark output.
- `init_experiment` refuses to reset a live history unless `reset: true` is passed explicitly.
- The first `log_experiment` in a segment is tagged as the baseline automatically, even if it is later discarded.
- `discard` logs must include an `idea` note, and that note is appended to `autoresearch.ideas.md`.
- During active autoresearch mode, raw benchmark execution through OpenClaw `exec`/`bash` is blocked. Use `run_experiment` instead.
- `autoresearch_status` warns when a pending run is unlogged, when the canonical branch has drifted, when a stale/live lock exists, or when git history has moved ahead of the last logged experiment. On `autoresearch/*` branches it explicitly warns not to push unlogged commits.
- After 3+ positive-metric runs in a segment, `log_experiment`, `autoresearch_status`, and the synced session doc report a MAD-based confidence score so the agent can distinguish likely wins from noise.
- The plugin updates `autoresearch.checkpoint.json`, `autoresearch.lock`, and the plugin-managed sections in `autoresearch.md` after init, run, and log transitions.

## Use

In the repo you want to optimize:

1. Load the plugin.
2. Run `/autoresearch` or `/autoresearch setup <goal>`.
3. Send a normal message with the goal, command, metric (+ direction), files in scope, and constraints.
4. If you need the raw skill invocation, use `/skill autoresearch-create`.
5. The agent writes `autoresearch.md` and `autoresearch.sh`, captures or reuses the canonical `autoresearch/*` branch, runs a baseline with `run_experiment`, then records it with `log_experiment`.
6. Use `/autoresearch` or `/autoresearch status` to re-prime context on a later turn.

To resume an existing session, a new agent reads the repo-root files and continues from where the last one stopped.

### User steers

Messages sent while an experiment is running are queued and surfaced after the next `log_experiment`. The agent finishes the current experiment before incorporating the steer.

### Ideas backlog

When the agent discovers promising but complex ideas mid-loop, it appends them to `autoresearch.ideas.md`. Discarded experiments now require an `idea` note, so failed paths leave behind concrete follow-up suggestions instead of disappearing. On resume, the agent reads the backlog, prunes stale entries, and uses the remaining ideas as experiment paths.

## Upstream reference

This port preserves upstream semantics, names, and file contracts while adapting presentation to OpenClaw. There is no Pi-style widget, dashboard, or editor shortcut layer. Remaining differences are tracked in [`docs/non-parity.md`](docs/non-parity.md).

- upstream repo: `https://github.com/davebcn87/pi-autoresearch`
- pinned upstream commit: `2227029fa5712944a36938b5fe59f709cb30ed22` (`2227029f`)
- later upstream parity cherry-pick: confidence scoring from `cf1bbf03debca8f3fb2cca2c3e799b9e23320f87` (`cf1bbf0`, March 19, 2026)

## Validation

```bash
npm install --include=dev
npm run check:release-metadata
npm run typecheck
npm test
npm run validate
npm run release:verify
npm run smoke:openclaw-host -- /absolute/path/to/openclaw
npm run smoke:registry-openclaw-host -- <published-version> /absolute/path/to/openclaw
```

Release instructions, including `npm run release:prepare -- <version> --host /absolute/path/to/openclaw` and GitHub Actions trusted publishing with npm provenance, live in [`RELEASING.md`](RELEASING.md).

The local test shim supports typechecking and tests without a full OpenClaw host checkout. Runtime behavior depends on a real OpenClaw host, so run the host smoke against a current checkout before release.

## License

MIT