# Tersio Benchmark Guide

How to measure Tersio on any agent. Report the agent, the model, and the versions you used.

The method follows how the upstream projects test their own tools:

- **Caveman**: compare replies on fixed prompts, with a control prompt `Answer concisely.` so the gain is not just a short-answer instruction. Count prompt overhead separately, and report losses.
- **Ponytail**: run the same tasks with and without the mode, five times each, and check quality with tests and injected bugs.
- **RTK**: measure command output only. Check that failure details survive. Report percentages, not bill savings.
- **Headroom**: measure the proxy JSONL before and after compression. Require a token drop and a surviving sentinel. A green switch is not evidence.

## Quick run

A first pass on your own repo, in about ten minutes. It answers "does this save me tokens" without the full protocol below.

1. Run `tersio install`, then `tersio doctor`. Both must pass.
2. Set one mode in the dashboard's General pane. Start with `Combo balanced`.
3. Open a fresh session and ask one question you ask often. Save the reply.
4. Turn every mode `off` in the same pane. Open another fresh session and ask the same question.
5. Compare the reply tokens and read Δ Net, not Δ Tokens alone.
6. For Headroom, run the proxy check in section 9. A switch that reads `on` is not evidence.

The full protocol below turns that first pass into a number two people can reproduce.

## 1. Modes and levels

Base means every mode is off, and every result is compared to base. This is the complete matrix. A
result file that skips a row is incomplete, so two people running this guide produce the same tables.

| Mode | Every level | What it changes |
|---|---|---|
| Base | `off` | Nothing. The baseline. |
| Control | the prompt `Answer concisely.` | A plain instruction, used only as a reference. |
| Caveman | `lite`, `full`, `ultra`, `megacave-lite`, `megacave-full`, `megacave-ultra` | Shorter replies. Six levels. |
| Ponytail | `lite`, `full`, `ultra` | Smaller code changes. Three levels. |
| pstack | `on` | Design and verification rules. Scored on quality only. |
| RTK | `on` | Shorter shell command output. Measured by the bench, not by a session. |
| Headroom | `on` | Compression proxy for tool output, logs, files, JSON, and history. Measured from proxy JSONL, not reply prompts. |
| Combo | `lite`, `balanced`, `max` | Presets. `lite` is lite/lite, `balanced` is full/full, and `max` is ultra/ultra. RTK is on. pstack and Headroom stay where you set them. |

There are two ways to turn a mode on, and they are measured separately:

- **Saved default.** The mode is set before the session starts. Its block is in the system prompt from turn one.
- **Slash command.** The command is issued mid-session. The block is appended when it runs, after context has already grown.

Both paths inject the same text. They differ in *when* it is paid for. Do not mix the two paths in
one comparison. Pick one, state it in the result file, and keep it for every run of that case. One
session for the whole benchmark can hold only one saved default, so a benchmark that runs in one
session uses the slash command path throughout.

### The level bodies

Each level injects its own body, so each level is its own measurement. Static sizes, so a reader can
sanity-check the growth against them:

| Caveman level | File | Bytes | Characters |
|---|---|---:|---:|
| `lite` | `extensions/caveman/rule-lite.md` | 4,210 | 4,206 |
| `full` | `extensions/caveman/rule.md` | 4,708 | 4,708 |
| `ultra` | `extensions/caveman/rule-ultra.md` | 2,513 | 2,513 |
| `megacave-lite` | `extensions/caveman/rule-megacave-lite.md` | 3,122 | 2,822 |
| `megacave-full` | `extensions/caveman/rule-megacave.md` | 2,858 | 2,448 |
| `megacave-ultra` | `extensions/caveman/rule-megacave-ultra.md` | 2,931 | 2,761 |

The two columns differ where a body carries multi-byte characters, so check with the one you have:
`wc -c` gives the bytes, a character count gives the other. The token figures quoted in the result
files are estimates that divide a body size by four, not a tokenizer count. The megacave bodies
give 705, 612 and 690 from their characters and 780, 714 and 733 from their bytes, so a figure taken
from a byte count reads a little higher than the same body counted in characters.

Ponytail ships one body for all three levels and states the level in its header, so the three levels
differ by that line rather than by size.

## 2. Setup

1. Install Tersio for your agent, then run the health check and confirm it passes.
2. Run `tersio upstream` and read the source rows. It names the upstream release each vendored body
   comes from, so a run on a body that upstream has moved measures the wrong thing.
3. Record the agent version, the model, the thinking level, and the temperature. Keep them fixed.
4. Write benchmark output to a separate folder, and leave your live settings alone.
5. Use a fixed repository with a passing test suite and at least 200 files, pinned to one commit.
6. Copy it and inject exactly one failing test. Use that copy for the agentic runs.

### The fixture repo

Step 6 takes five minutes, and the copy is the fixture. Use the same shape every time: a copy of
the repository under test at a pinned commit, with one failing test injected into a source file and
every test file left alone. Name the commit in the result, so another person can recreate it.

Put the bug in the source, not in the test. The test states the correct expectation, the source
disagrees, and the agent fixes the source. A bug inside the test file forces the agent to edit a
test to go green, which is exactly what the agentic gate forbids, so the gate stops meaning
anything.

```bash
# A copy of the repository under test, with its own history.
git clone --quiet --local /path/to/repo /tmp/bench-fixture
cd /tmp/bench-fixture
bun install --frozen-lockfile
bun run build
bun run test          # green before the bug goes in
```

Then add the bug and the test that catches it, as two files:

```ts
// extensions/shared/invoice.ts
export function invoiceTotal(invoice: Invoice): number {
  return invoice.lines.reduce((sum, line) => sum + line.price, 0); // ignores qty
}
```

```ts
// test/shared/invoice.test.ts
test("the invoice total multiplies price by quantity", () => {
  expect(invoiceTotal({ lines: [{ price: 10, qty: 2 }, { price: 5, qty: 3 }] })).toBe(35);
});
```

Check the state before you measure: the suite must report exactly one failure, and the rest must
pass. Then commit both files, so every run starts from the same injected bug. Reset between runs
with `git checkout -- .`.

Keep the copy out of your working repository. An agent that can see your project will read it, and
on a code surface it will find functions that already exist and decline to write anything.

### Isolating the run

Do not edit your live settings for a benchmark. Point the extension tree at a scratch home, so the
mode state is per-condition and your real `~/.tersio` never changes:

```bash
export TERSIO_HOME=/tmp/bench-home/base      # a folder holding settings.json
```

Write one `settings.json` per condition, with the four keys the session reads. This is how a
condition is set without touching the dashboard:

```json
{ "comboDefault": "off", "cavemanDefault": "off", "rtkDefault": false, "ponytailDefault": "off", "pstackDefault": "off" }
```

For `Combo balanced`, use `"comboDefault": "balanced"` with caveman `full`, rtk `true`, ponytail
`full`, and pstack `on`. For a single mode, set `comboDefault` to `custom` and set only that mode.

### One session for the whole benchmark

A fresh session per run fills the agent's session list, and a full benchmark makes hundreds of
entries there. Use one session for the whole benchmark instead, and clear the context between the
runs:

1. Open the session once, before the first case.
2. Run the case's first turn.
3. Clear the session's context, reset the fixture with `git checkout -- .`, and run the next turn.
4. Repeat until every case has its runs.

Each turn then starts from the same empty context, so the numbers read like a fresh session's, and
the session list holds one entry instead of one per run. A harness with no clear command starts a
new session for each turn instead, and its session list grows with the run count.

One session for everything means one mode setting for everything, so every case sets its mode with
a slash command. The saved-default path needs a session of its own, because that path fixes the
mode before the first turn. Pick one path for the whole benchmark, as section 1 sets out.

Measure the base case the same way. A base case measured with one session per run beside a mode case
measured in the shared session is two different experiments, and the deltas between them mean
nothing.

Read the per-turn usage. Every turn reports its own reply and total tokens, so the medians in
section 8 come from the turns, not from the session's last line.

The agentic surface needs the reset as much as the others: the agent has to meet the injected bug
again, so the context and the working tree both go back to their starting state before each turn.

## 3. Surfaces

Run every mode and level on every surface that applies to it, and never mix results across
surfaces. The matrix below is the full scope.

| Surface | Applies to | Task | Counted |
|---|---|---|---|
| Prose | Caveman (6 levels), Control, Combo (3) | 3 prompts, each asking for a three-sentence explanation. | Reply tokens |
| Code | Ponytail (3 levels) | 3 prompts, each asking for TypeScript functions. | Reply tokens |
| Agentic | Combo (3), Combo stacks, pstack, RTK | Run the suite. Fix the failing test. Stop when it passes. | Total tokens, time, pass rate |
| Command output | RTK | Run `bun run bench` over the committed fixtures. | Output bytes, time, fidelity |
| Compression | Headroom | Send one fixed log and one fixed JSON payload through the proxy. | Tokens before, tokens after, transforms, sentinel survival |

The prompt overhead of a mode is inside the Total median column, because a mode's block lands in the
system prompt of every turn. Read it as the difference between the two columns, and read the static
body sizes in section 1 to check it.

RTK has no session-level row of its own on the prose or code surfaces. Its effect is command
output, and the bench measures it exactly. On the agentic surface it does have a row, because the
task runs shell commands, and there the RTK row answers a different question: whether condensing
the suite output changes how the agent fixes the bug.

## 4. Procedure

1. Set every mode to `off` and run the base case.
2. Open one session for the whole benchmark, and run every case's runs as turns in it, clearing the
   context between turns as section 2 sets out. Run each case **5 times**. The prose and code
   surfaces run three prompts each, so a case there is 15 runs, and its quality column reads out of 15.
3. Keep every result. Do not drop slow or failed runs.
4. Interleave the cases: base, A, B, base, A, B.
5. Run the control prompt on the prose surface.
6. Run every level listed in section 1. A skipped level is a hole in the result file.

## 5. Metrics

| Symbol | Name | Formula |
|---|---|---|
| **Reply median** | Median reply | Median reply tokens of the case. Base is the reference. |
| **Δ Tokens** | Reply change | Median reply tokens − base median. Negative is a saving. |
| **Δ%** | Relative change | Δ ÷ base median × 100. |
| **Total median** | Session cost | Median tokens of the whole session, prompt and reply. |
| **Δ Net** | Session change | Median total session tokens − base median. Negative is a saving. |
| **Δ Time** | Time change | Median time − base median. |
| **Quality** | Pass rate | Passing runs out of N. |

Reply tokens are the assistant's generated text. A harness that reports thinking tokens as a
separate number counts them out, and the result file says which it counted.

Read Δ Net next to Δ Tokens. A mode can cut reply tokens and still cost more overall, because Δ Net
carries the mode's block on every turn of the session.

## 6. Quality gates

A saving counts only when the output stays correct.

- **Code**: passes a strict type check and a fixed test file, and contains every requested function.
- **Prose**: exactly three sentences, with every required keyword.
- **Agentic**: the suite passes, and no test file was edited.
- **Command output**: the failing test name, its assertion, its file, and its line remain, and the exit code matches.
- **Headroom**: the proxy record shows `tokens_saved > 0`, a non-empty `transforms_applied`, and every required sentinel line or JSON key in the compressed messages. A configured switch with no proxy record is a failure.
- **pstack**: score the agentic pass rate and the wrong turns. A green suite with a wrong turn is a failure.

Do not publish a saving for any mode whose pass rate is below base.

## 7. Fixed prompts

Two people running this guide must measure the same thing. The keyword test is lexical: a group matches when the reply contains any of its terms as a substring, so `quer` covers `query` and `queries`.

| Prompt | Text | Keyword groups, all required |
|---|---|---|
| P1 | Explain in three sentences how a database connection pool improves throughput under load. | `reus` · `connection` · `overhead`/`cost` · `concurren`/`wait` |
| P2 | Explain in three sentences why an index speeds up a database query. | `index` · `quer` · `scan`/`search`/`tree`/`lookup` · `rows`/`table`/`data`/`disk` |
| P3 | Explain in three sentences how HTTP caching reduces server load. | `cach` · `server`/`origin`/`backend` · `load`/`bandwidth`/`request`/`traffic` |

| Code prompt | Text | Required names |
|---|---|---|
| C1 | Write a TypeScript function `chunk<T>(items: T[], size: number): T[][]` that splits an array into fixed-size chunks, and `chunkLazy` that yields chunks from an iterable without materializing the whole array. | `chunk`, `chunkLazy` |
| C2 | Write a TypeScript `debounce` and a `throttle`, both with the signature `(fn: (...args: T) => void, ms: number)`. | `debounce`, `throttle` |
| C3 | Write a TypeScript `groupBy<T, K extends PropertyKey>(items: T[], key: (item: T) => K): Record<K, T[]>` and `countBy` with the same inputs returning `Record<K, number>`. | `groupBy`, `countBy` |

Agentic task: in the fixture repo, run the suite, fix the failing test, and stop when all tests pass. The
fixture is the one from section 2, and the failure is in the source, so a correct run edits a source
file and leaves every test file alone.

Read a miss before you record it, and compare each mode against base on the same prompt: a keyword the base model also misses measures the gate, not the mode.

## 8. Results

A result file contains only the surfaces its mode or stack needs, using the tables below and exactly
the columns given. Leave a cell blank until it is measured. Keep every row, including the levels that
measured nothing useful, because a skipped row is a hole in the file.

Reply median and Total median are the two numbers a reader compares across agents. Δ Tokens and
Δ Net are the two deltas against base. pstack is scored on the agentic pass rate and the wrong
turns, so its session columns stay `n/a`.

### Prose

| Case | Reply median | Δ Tokens | Δ% | Total median | Δ Net | Quality |
|---|---:|---:|---:|---:|---:|---:|
| Base (`off`) | ___ | 0 | 0% | ___ | 0 | ___/15 |
| Control (`Answer concisely.`) | ___ | ___ | ___% | ___ | ___ | ___/15 |
| Caveman `lite` | ___ | ___ | ___% | ___ | ___ | ___/15 |
| Caveman `full` | ___ | ___ | ___% | ___ | ___ | ___/15 |
| Caveman `ultra` | ___ | ___ | ___% | ___ | ___ | ___/15 |
| Caveman `megacave-lite` | ___ | ___ | ___% | ___ | ___ | ___/15 |
| Caveman `megacave-full` | ___ | ___ | ___% | ___ | ___ | ___/15 |
| Caveman `megacave-ultra` | ___ | ___ | ___% | ___ | ___ | ___/15 |
| Combo `lite` | ___ | ___ | ___% | ___ | ___ | ___/15 |
| Combo `balanced` | ___ | ___ | ___% | ___ | ___ | ___/15 |
| Combo `max` | ___ | ___ | ___% | ___ | ___ | ___/15 |

### Code

| Case | Reply median | Δ Tokens | Δ% | Total median | Δ Net | Quality |
|---|---:|---:|---:|---:|---:|---:|
| Base (`off`) | ___ | 0 | 0% | ___ | 0 | ___/15 |
| Ponytail `lite` | ___ | ___ | ___% | ___ | ___ | ___/15 |
| Ponytail `full` | ___ | ___ | ___% | ___ | ___ | ___/15 |
| Ponytail `ultra` | ___ | ___ | ___% | ___ | ___ | ___/15 |

### Agentic

The Agentic rows count the whole session: total tokens, wall time, and the pass rate. A row also
records whether any test file was edited, in the text under the table, because the gate forbids it.

| Case | Reply median | Total median | Δ Net | Δ% | Wall median | Pass |
|---|---:|---:|---:|---:|---:|---:|
| Base (`off`) | ___ | ___ | 0 | 0% | ___ | ___/5 |
| RTK `on` | ___ | ___ | ___ | ___% | ___ | ___/5 |
| pstack `on` | ___ | n/a | n/a | n/a | ___ | ___/5 |
| Combo `lite` | ___ | ___ | ___ | ___% | ___ | ___/5 |
| Combo `balanced` | ___ | ___ | ___ | ___% | ___ | ___/5 |
| Combo `max` | ___ | ___ | ___ | ___% | ___ | ___/5 |

The base row's wall time is the baseline every other row subtracts from.

### Command output

Tersio routes three things through rtk: `grep` tool output, `find`/`glob` tool output, and one `rtk rewrite` lookup before every Bash call. Nothing else goes through a filter, so nothing else belongs in this table.

Run the bench instead of timing commands by hand:

```bash
bun run bench
```

It reads the committed fixtures in `bench/fixtures/`, pipes each through `rtk pipe -f <filter>`, and prints ops/sec and percentiles. It also asserts fidelity, so a filter that starts dropping matches fails the run.

| Case | Fixture | What fidelity means |
|---|---|---|
| `find` condense | 260 paths, 46 dirs | the header counts every file and dir, and no printed name is invented |
| `grep` condense | 240 matches | every match line survives with its `file:line` |
| `rewrite` | 12 commands | every command gets an answer or a clean "no equivalent" |

Fill this table from your own `bun run bench` output:

| Case | Bytes in | Bytes out | Δ% | Mean |
|---|---:|---:|---:|---:|
| `find` | ___ | ___ | ___% | ___ ms |
| `grep` | ___ | ___ | ___% | ___ ms |
| `rewrite`, 12 commands | — | — | — | ___ ms |

The reduction is a byte ratio, so it reproduces across machines. The timings do not: a slow machine shifts every number, so compare percentages and ignore the milliseconds.

Two numbers explain the design. The `find` filter folds a directory after a few children, so it saves most of its bytes. The `grep` filter keeps every match line, so a dense result saves little. The rewrite lookup runs before every Bash call, which is why `MIN_FILTER_BYTES` is 1200: below that, the spawn costs more than the filter saves.

The bench does not measure Caveman, Ponytail, or pstack. Those change prompt text, and only a live session produces the reply tokens to compare. The protocol above is the only way to measure them. Headroom is measured at the proxy layer, in section 9.

## 9. Headroom compression

Headroom is a transport layer, not a reply-shaping block. A session can read `headroom=on` while
its active model bypasses the proxy, so measure the proxy record, not the switch.

### Fast check with `/v1/compress`

Start a disposable proxy. `--mode token` makes per-request compression visible; the default
`cache` mode protects provider prefix caching. `--log-file` writes JSONL, and `--log-messages`
adds the before-and-after message snapshots that `headroom inspect` reads.

```bash
headroom proxy --port 8788 --mode token \
  --openai-api-url <the same upstream the agent uses> \
  --log-file /tmp/headroom-bench.jsonl --log-messages
curl -fsS http://127.0.0.1:8788/readyz
```

Build one fixed payload. Keep a sentinel line and a JSON key that must survive compression:

```bash
python3 - <<'PY' > /tmp/headroom-payload.json
import json
lines = [
    f"2026-11-03T10:{i:02d}:00Z INFO request_id=req-{i:04d} "
    f"user=u{i % 7} status=200 latency_ms={i % 120} path=/api/v1/items"
    for i in range(200)
]
lines[67] = "2026-11-03T10:01:07Z FATAL checkout failed order=8841"
print(json.dumps({
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Keep the FATAL line.\n" + "\n".join(lines)}],
    "config": {},
}))
PY
```

Send it through the proxy:

```bash
curl -sS http://127.0.0.1:8788/v1/compress \
  -H 'content-type: application/json' \
  --data-binary @/tmp/headroom-payload.json \
  | jq '{tokens_before,tokens_after,tokens_saved,compression_ratio,transforms_applied}'
```

`compression_ratio` in this response is `tokens_after / tokens_before`, so it is the share that
remains. Compute the saved share as `tokens_saved / tokens_before`. A passing check has
`tokens_saved > 0`, a non-empty `transforms_applied`, and the FATAL sentinel still present in the
returned messages. This call also appends a `RequestLog` line to the JSONL file, so the same fields
can be checked there.

### Evidence from a real agent turn

Route the agent through the proxy, then run one fixed prompt that reads a large log or JSON tool
output. The same JSONL file records each model request. Without `--log-messages` the light fields
remain and the message snapshots are absent.

```bash
jq -c 'select(.tokens_saved > 0) | {
  request_id, model, input_tokens_original, input_tokens_optimized,
  tokens_saved, savings_percent, transforms_applied
}' /tmp/headroom-bench.jsonl
```

The fields used by this guide:

| Field | Meaning |
|---|---|
| `input_tokens_original` | Tokens in the request before compression. |
| `input_tokens_optimized` | Tokens actually sent upstream. |
| `tokens_saved` | `input_tokens_original - input_tokens_optimized`. |
| `savings_percent` | Saved share of the original request. |
| `transforms_applied` | The compressor path that ran, for example `router:log`. |
| `request_messages` / `compressed_messages` | Before and after snapshots, only with `--log-messages`. |
| `request_id` | The handle for `headroom inspect` and the report. |

Check the sentinel in the compressed snapshot:

```bash
jq -s -e '
  map(select(.tokens_saved > 0)) as $m
  | ($m | length) > 0
    and all($m[]; (.compressed_messages // [] | tostring | contains("FATAL checkout failed")))
' /tmp/headroom-bench.jsonl
```

For a quick look at the latest request without parsing the file:

```bash
headroom inspect --last 1 --format json
```

### What the number means

| Measure | What to report |
|---|---|
| Text or log payload | `tokens_before`, `tokens_after`, `tokens_saved`, saved percent, and the sentinel result |
| JSON payload | The same token fields, plus the required key still present after compression |
| Routing | The agent’s active model endpoint and the proxy upstream, so the reader can confirm traffic passes through Headroom |

A Headroom result is published only when the JSONL record shows a token reduction and the sentinel
survives. A configured switch, a green dashboard badge, or a running proxy is not enough. Report the
JSONL path and the `request_id` with the result. Do not compare `cache` mode against `token` mode:
they optimize different things. Use `--mode token` for this compression check.

## 10. Reporting

Write one file per mode or stack. Do not create a single `BENCHMARK_<AGENT>.md`.

| File | Mode or stack |
|---|---|
| [BENCH_CAVEMAN.md](./BENCH_CAVEMAN.md) | Caveman, all six levels |
| [BENCH_PONYTAIL.md](./BENCH_PONYTAIL.md) | Ponytail, all three levels |
| [BENCH_RTK.md](./BENCH_RTK.md) | RTK command output and the agentic row |
| [BENCH_HEADROOM.md](./BENCH_HEADROOM.md) | Headroom compression |
| [BENCH_PSTACK.md](./BENCH_PSTACK.md) | pstack agentic quality |
| [BENCH_COMBO.md](./BENCH_COMBO.md) | Combo `lite`, `balanced`, `max` |
| [BENCH_COMBO_HEADROOM.md](./BENCH_COMBO_HEADROOM.md) | Combo plus Headroom |
| [BENCH_COMBO_PSTACK.md](./BENCH_COMBO_PSTACK.md) | Combo plus pstack |
| [BENCH_COMBO_ALL.md](./BENCH_COMBO_ALL.md) | Combo plus Headroom plus pstack |

`Combo` means the preset only: Caveman, RTK, and Ponytail. pstack is not part of it. Headroom is not
part of it. The last three files add those layers explicitly. `BENCH_COMBO_ALL.md` is the final
stack, with both Headroom and pstack.

Each file starts with the agent and model header. If one file holds more than one agent, repeat the
header under its own second-level agent heading. Use one mode or stack per file.

```md
# Tersio benchmark: <mode or stack>

- Agent: <name and version>
- Model: <provider/model>, thinking effort <level>, temperature <not reported by the harness>
- Mode or stack: <the exact levels and whether Headroom or pstack is on>
- Tersio, RTK, Ponytail, Caveman: <versions>
- Caveman release <version> and pstack pin <commit>: the upstream bodies this run measured
- Repository and commit: <repo> at <commit>. <where each surface ran>
- Runs (N), date, surface: <N per prompt on prose and code, 5 per case on the others>, <date>
- Sessions: one for the whole benchmark, the context cleared between runs
- Path: <saved default or slash command>
- Headroom: <proxy mode, JSONL path, upstream URL>
```

Then include only the sections this mode or stack needs:

- Caveman: Prose.
- Ponytail: Code.
- RTK: Command output and Agentic.
- Headroom: Compression.
- pstack: Agentic quality.
- Combo, Combo + Headroom, Combo + pstack, Combo + everything: Prose, Agentic, and the matching
  Headroom or pstack section.

Keep every row, including levels that measured nothing useful, and keep the path column: a saved
default and a slash command are different measurements.

The guide is the reference for this shape. A stored result is optional, and the deleted per-agent files no longer ship.

Include with every result:

- The agent name and version.
- The model, the thinking level, and the temperature.
- The Tersio, RTK, Ponytail, and Caveman versions.
- The Caveman release and the pstack pin, so a reader knows which upstream bodies were measured.
- The test repository and commit.
- The number of runs (N), the date, and the surface.
- How the sessions were run: one for the whole benchmark with the context cleared between runs, or one per run.
- The Headroom JSONL path, proxy mode, and `request_id` for each compression check.

A run that breaks a rule of this guide says so in the file, in the paragraph under the header. A
short run, a skipped level, a reworded prompt, and a harness limitation each get one sentence.

Show the median for each value, and state the min and the max in the text when the spread matters,
because a median alone hides a bimodal sample. Write verdicts with their limits: "Lower reply
tokens in this sample. Quality unchanged on 5 of 5 runs."

Do not claim total bill savings. These tests measure reply tokens, prompt overhead, command output, or Headroom input tokens. Rerun after any change to the agent, the model, or the Tersio version, and do not compare results across different setups.