A coding agent built on pi, re-engineered for long-horizon terminal tasks. Terminal-Bench 2.1 pass@1 from 0.539 to 0.773 — on par with Claude Code on the same model.
| Agent | Terminal-Bench 2.1 pass@1 |
|---|---:|
| **Crux** | **0.773** 0.793 on a rerun |
| Claude Code, same model | 0.730 |
| pi, upstream | 0.539 |
Self-hosted Qwen3.8-27B, identical weights for every agent · 89 tasks · pass@1
over whole runs.
## What changed from pi
All changes are in the agent layer; the model is untouched.
- **Context engine.** Compaction requests are sized to fit the window, retried
smaller on rejection, and summarize single-task sessions in full; the task's
original wording is carried verbatim through every compaction. Compaction
success: **14% → 100%**.
- **Agent loop.** Recovery paths live in one explicit state machine: two-phase
recovery from truncated output, the reasoning tail handed back after a cut,
stall detection, and pacing against each task's own deadline. Truncated
outputs: **18.0% → 0.7%**.
- **Runtime.** A stream idle watchdog, a default command timeout, process-tree
reaping and resume after an OOM kill keep multi-hour runs alive. Idle time in
timed-out runs: **94% → 3%**.
- **Tools.** A failed edit shows the nearest matching region of the file, long
command output keeps both its head and its tail, and the model keeps short
visible notes beside its tool calls.
## Evaluation
[`benchmark/`](benchmark) holds the harness behind these numbers: it runs the
agent in task containers via harbor, classifies every trial's outcome, and
compares versions task by task with a paired sign test.
```bash
cd benchmark && uv sync
harbor run --dataset terminal-bench/terminal-bench-2-1 \
--agent crux.pi_agent:CruxPiAgent --model openai/
```
## Acknowledgements
Built on [pi](https://pi.dev) by Earendil Works
([earendil-works/pi](https://github.com/earendil-works/pi)). Licensed as
upstream; see [`LICENSE`](LICENSE).