Crux

A coding agent built on pi, re-engineered for long-horizon terminal tasks.
Terminal-Bench 2.1 pass@1 from 0.539 to 0.773 — on par with Claude Code on the same model.

pass@1 0.773 pi 0.539 to Crux 0.773 Claude Code on the same model 0.730 built on pi

English · 简体中文

## Results

Terminal-Bench 2.1 pass@1: Crux 0.773, Claude Code on the same model 0.730, pi upstream 0.539

| Agent | Terminal-Bench 2.1 pass@1 | |---|---:| | **Crux** | **0.773** 0.793 on a rerun | | Claude Code, same model | 0.730 | | pi, upstream | 0.539 | Self-hosted Qwen3.8-27B, identical weights for every agent · 89 tasks · pass@1 over whole runs. ## What changed from pi All changes are in the agent layer; the model is untouched. - **Context engine.** Compaction requests are sized to fit the window, retried smaller on rejection, and summarize single-task sessions in full; the task's original wording is carried verbatim through every compaction. Compaction success: **14% → 100%**. - **Agent loop.** Recovery paths live in one explicit state machine: two-phase recovery from truncated output, the reasoning tail handed back after a cut, stall detection, and pacing against each task's own deadline. Truncated outputs: **18.0% → 0.7%**. - **Runtime.** A stream idle watchdog, a default command timeout, process-tree reaping and resume after an OOM kill keep multi-hour runs alive. Idle time in timed-out runs: **94% → 3%**. - **Tools.** A failed edit shows the nearest matching region of the file, long command output keeps both its head and its tail, and the model keeps short visible notes beside its tool calls. ## Evaluation [`benchmark/`](benchmark) holds the harness behind these numbers: it runs the agent in task containers via harbor, classifies every trial's outcome, and compares versions task by task with a paired sign test. ```bash cd benchmark && uv sync harbor run --dataset terminal-bench/terminal-bench-2-1 \ --agent crux.pi_agent:CruxPiAgent --model openai/ ``` ## Acknowledgements Built on [pi](https://pi.dev) by Earendil Works ([earendil-works/pi](https://github.com/earendil-works/pi)). Licensed as upstream; see [`LICENSE`](LICENSE).