--- id: "008" title: Remove the Choices Agents Get Wrong status: complete amends: specs/001-agent-mail-cli.md, specs/006-inbox-watch.md blocked_by: [] blocks: [] --- # Remove the Choices Agents Get Wrong ## Overview Two defects with one cause: the tool exposes a number, and agents fill it from their priors rather than from the tool's intent. **Messages expire after 24 hours and take the coordination record with them.** On 2026-08-03 the live mailbox held 14 messages, the oldest written that same morning — an entire multi-day delegation, including ten unacked handoffs, silently deleted. This is not a hypothetical: spec 004's incident report already says *"the 24h TTL then erased the evidence"*, and that spec fixed the symptom while leaving the cause running. `watch` sharpens it further, because an agent that is off overnight now loses every handoff addressed to it with no trace that one was ever sent. Nothing about the mailbox needs messages to be ephemeral; it was an assumption inherited from the first prototype and never revisited. **`watch --timeout` is in seconds, and agents pass 30.** The default is 900, which is correct, and agents override it with values one to two orders of magnitude too small — burning a model invocation every 30 seconds to learn nothing, which is exactly the cost `watch` exists to remove. The unit invites it: "30" is a natural number of *seconds*, both sibling CLIs in the same stack default to 30s, and a documented range starting at `0` advertises smallness. The fix in both cases is the same principle, and it is a design rule rather than a patch: **a surface that lets an agent choose wrongly will be chosen wrongly, so remove the choice or make the wrong value unreachable — never document your way out of it.** Instructions do not hold; a rejected argument does. Ships as `0.2.0`. This is a breaking change. > **Completion rule:** This spec is not complete until all acceptance criteria are verified through the testing approach below, including a message surviving past 24 hours in a real mailbox and the published `0.2.0` rejecting a short timeout on JJ's machine. Build-only verification is insufficient. The agent must iterate until verification passes. ## Goals - A message stays in the mailbox until someone deliberately removes it. Nothing deletes coordination history on a timer. - No per-message lifetime appears anywhere in the surface, so nothing invites an agent to reason about one. - A `watch` cannot be told to wake sooner than is useful, and the value an agent reaches for by instinct is already the right one. - Mailbox growth is observable without any policy acting on it. ## Requirements ### Functional Requirements - **FR1**: Messages persist indefinitely. No command deletes a message as a side effect of doing something else. - **FR2**: `cleanup` deletes messages older than an explicitly supplied age and does nothing otherwise. The age has no default. - **FR3**: No command output includes a per-message lifetime field. - **FR4**: `describe` reports the size on disk of the resolved mailbox alongside its message count. - **FR5**: `watch` takes its wait in **minutes**, with the unit named in the flag itself, bounded so a value below the floor is rejected rather than honored. - **FR6**: `watch --once` performs a single non-blocking check. - **FR7**: Passing the removed `--timeout` or `--poll` flags produces an error that names the replacement and says why the flag went away. - **FR8**: The poll interval is fixed and not configurable. ### Non-Functional Requirements - **NFR1**: Existing mailboxes keep working with no migration step and no data conversion on first open by `0.2.0`. - **NFR2**: The JSON output and error contract is unchanged (spec 001 NFR1, NFR2). - **NFR3**: No new runtime dependencies, and no changes to the CI, release, npm publish, or WinGet workflows. ### Technical Constraints - **TC1**: The `messages.ttl_hours` column is retained and left inert rather than dropped. Dropping it would require a migration on every existing mailbox for no behavioral gain; retaining it costs one unread integer per row. - **TC2**: `cleanup --older-than ` is required. `cleanup` with no `--older-than` is an error, not a no-op sweep. `--dry-run` is retained. - **TC3**: `cleanup --older-than` also removes watcher liveness rows whose last refresh predates the same cutoff. - **TC4**: `describe`'s `storage` gains `size_bytes`, the size of the mailbox file, or null when it cannot be read. It is a reported fact and nothing acts on it. - **TC5**: `watch --timeout-minutes` is a float in `[5, 240]`, default `55`. Values outside the range are a JSON error. (Shipped as `90`; corrected to `55` in `0.3.1` — see the Correction below.) - **TC6**: `watch --once` is a boolean and is mutually exclusive with `--timeout-minutes`. - **TC7**: The poll interval is fixed at 1 second. - **TC8**: `--timeout` and `--poll` are accepted by the parser solely so they can be rejected with a migration message. They never take effect. - **TC9**: Ships as `0.2.0` through the existing PyPI, GitHub Release, and npm channels. ### Requirement Traceability | Requirement | Acceptance Criteria | |---|---| | FR1 | AC1, AC2 | | FR2 | AC3, AC4 | | FR3 | AC5 | | FR4 | AC6 | | FR5 | AC7, AC8 | | FR6 | AC9 | | FR7 | AC10 | | FR8 | AC11 | | NFR1 | AC12 | | NFR2 | AC5, AC8, AC10 | | NFR3 | AC13 | | TC1 | AC12 | | TC2 | AC3, AC4 | | TC3 | AC4 | | TC4 | AC6 | | TC5 | AC7, AC8 | | TC6 | AC9 | | TC7 | AC11 | | TC8 | AC10 | | TC9 | AC14, AC15 | ## Key Decisions ### The TTL was destroying the record the delegation model depends on Messages were called ephemeral coordination, with durable content routed to files. That rule is about *content* and it still stands — but it was quietly read as licence to delete the coordination *history*, which is a different thing and is exactly what makes a multi-agent run auditable after the fact. The evidence is unambiguous: on the day this spec was written the oldest surviving message was hours old, and a delegation spanning several days had no record left. The same vault treats its git log as a queryable knowledge base for precisely this reason. A mailbox that forgets is the odd one out. Deleting is also the irreversible direction. Keeping messages costs kilobytes and can be undone at any time with one command; deleting them cannot be undone at all, and the loss is silent — no error, no gap, just an empty inbox indistinguishable from one that never received anything. ### Growth is reported, not managed Unbounded growth is a real concern eventually, and the temptation is to solve it now with a policy. That would reintroduce the same failure in a new costume: an automatic rule that deletes things on a schedule nobody is watching. Instead `describe` reports `size_bytes`, so an agent or a human can *see* the mailbox growing and act deliberately. This is the same move spec 005 made with the resolved path — report the fact, let the reader judge — and it is why `cleanup` now demands an explicit age instead of quietly defaulting to one. ### The unit belongs in the flag name `--timeout 30` is wrong by two orders of magnitude and looks completely reasonable. `--timeout-minutes 30` is right, and looks equally reasonable. The failure was never that agents chose carelessly — 30 is a *sensible* number of seconds for a timeout, and every prior an agent has says so. Naming the unit in the flag makes the instinctive number the correct one, which is worth more than any amount of documentation telling them to pick something bigger. ### A floor, because a good default is not enough The default was already correct and agents overrode it anyway. That is the whole lesson: a default only helps the caller who doesn't pass the flag, and a flag that exists will be passed. The floor is what makes the failure unreachable rather than merely discouraged. Five minutes is low enough to remain usable in a harness that caps command duration — Claude Code's foreground ceiling is 600 seconds — and high enough that no plausible small number produces the observed behavior. ### `--once` is a boolean because a boolean cannot be set to 30 The non-blocking check was `--timeout 0`, which required the numeric flag to keep a range starting at zero — advertising smallness in the very place we were trying to discourage it. A separate boolean expresses the same intent and carries no number to get wrong. ### The removed flags are rejected, not merely absent Agents will pass `--timeout` from memory, from an older instruction, or from a cached example. Argparse's "unrecognized argument" is technically an answer and practically a dead end. Accepting the flag only to reject it with a message naming the replacement turns a confusing failure into a self-correcting one — the same diagnosability argument as spec 007, applied to an interface change instead of a missing file. ### `--poll` goes for the same reason as the timeout Nothing legitimate tunes it, and the failure mode is worse than a short timeout: a fast poll hammers a SQLite file that spec 003's setup shares across an OS boundary, where the documented locking behavior is already fragile. A knob whose only reachable outcomes are "the default" and "worse than the default" should not be a knob. ### Specs 001 and 006 are amended, not superseded They remain the standing contracts. This spec owns the change and its verification; its implementation tasks fold the resulting rules back into both. ## Correction (0.3.1): the default is set by cache economics, not by taste The `90` this spec shipped was reasoned from *model invocations* alone — fewer idle wakes is cheaper, so make the interval long. That ignored what an idle wake actually costs, and JJ caught it against [Ryan Skidmore's cache tokenomics](https://skids.dev/blog/anthropic-cache-tokenomics/). A blocked `watch` makes no API calls, so **nothing refreshes the caller's prompt cache while it waits.** On a one-hour cache TTL, a wake at 90 minutes finds the cache expired and pays a **cache write (2.00x base input)** to rebuild the prefix; a wake inside the hour pays a **cache read (0.10x)**. That is a twentyfold difference on the entire cached context. What makes it a real defect rather than a tuning nit is *which* wake it applies to. The timeout fires only when nothing arrived — the no-op turn where the agent checks, finds nothing, and re-arms. So the original default arranged for the **cheapest possible turn to trigger the most expensive possible input event**, in a command whose whole premise is that waiting is free. `55` is not "safely under an hour" but the **largest interval that still lands inside the window**: fewer wakes is better, and this is the most spacing available before each one starts rewriting the context instead of reading it. Where a shorter cache TTL applies, no practical wait preserves it and 55 is simply a normal idle cadence. **The generalizable rule:** when tuning an interval for an agent, the model invocation is rarely the whole cost — the *state it has to rebuild on waking* can dominate. Ask what expires while the agent sleeps, not just how often it wakes. ## Implementation Tasks - [x] Remove the TTL constant, the expiry predicate, and the opportunistic purge from every command. - [x] Remove the expired-message filter from `watch`, which existed only to avoid waking on a row the purge would delete. - [x] Remove the per-message lifetime from `send` and `read` output and from the schema. - [x] Require `--older-than ` on `cleanup`, and have it sweep stale watcher rows under the same cutoff. - [x] Report the mailbox file size from `describe`. - [x] Replace `watch --timeout` with `--timeout-minutes` (default 90, range 5–240) and `--timeout 0` with `--once`. - [x] Fix the poll interval and remove `--poll`. - [x] Reject the removed flags with a message naming the replacement. - [x] Separate the blocking loop from argument validation so the deadline path is testable without waiting out the floor. - [x] Update the schema's invariants: messages persist, and the wait is in minutes. - [x] Add regression coverage for persistence, for the rejected values, and for an existing mailbox opening unchanged. - [x] Amend specs 001 and 006 with the new rules and acceptance criteria. - [x] Bump the package version to `0.2.0` and add the CHANGELOG entry, calling out both breaking changes. - [x] Release `v0.2.0` after JJ's approval and verify the published artifacts. ## Verification Record - [x] `ruff check src/ tests/` passed; `pytest` passed 80 tests, 0 skipped, with the PyInstaller binary and npm wrapper both available. - [x] Implementation defect caught by the new tests rather than by review: `read` issues `SELECT *`, so `ttl_hours` kept reaching callers after the schema stopped advertising it. Fixed by projecting to the documented `output_fields` by default, which closes the whole class rather than that one column. - [x] `python -m build` produced the `0.2.0` wheel and sdist; `twine check` passed on both. - [x] CI passed on `main` for Python 3.10-3.13 plus lint and build. - [x] Release workflow `30830448492` succeeded across all eight jobs; GitHub Release `v0.2.0` carries the wheel, the sdist, and the four platform binaries. PyPI and npm both serve `0.2.0`. - [x] `npm-smoke.yml` passed on Windows x64, Linux x64, macOS x64, and macOS arm64. - [x] AC15 verified against the globally installed `0.2.0` on JJ's machine, on the live shared mailbox: `--timeout-minutes 0.5` rejected with the range, `--timeout 30` rejected with the migration message, `--poll 0.1` rejected as fixed, `--once` returning the empty result, and `cleanup` with no `--older-than` refusing to delete. `describe` reported `messages: 21` and `size_bytes: 286720` against a mailbox carrying pre-upgrade rows. - [x] AC1 and AC12 are covered deterministically rather than by waiting: messages are aged by writing `created` (30 and 400 days), and the compatibility case builds an `0.1.8`-shaped mailbox by hand and opens it with the current code. ## Acceptance Criteria Each criterion names its validation method. `integration` criteria are automated tests owning their own fixtures; `manual` criteria require live published artifacts. ### Persistence - [x] **AC1** (`integration`): A message whose `created` timestamp is well past 24 hours old is returned by `read`, counted by `status`, and wakes `watch`. - [x] **AC2** (`integration`): `send`, `read`, `status`, `ack`, and `watch` each leave an old message in place — no command deletes one as a side effect. - [x] **AC3** (`integration`): `cleanup` without `--older-than` exits non-zero with a JSON error and deletes nothing. - [x] **AC4** (`integration`): `cleanup --older-than ` deletes only messages older than the cutoff, removes their broadcast-ack rows, sweeps watcher rows past the same cutoff, and honors `--dry-run` by deleting nothing. ### Surface - [x] **AC5** (`integration`): No `send`, `read`, or `watch` payload contains a per-message lifetime field, and the schema documents none. - [x] **AC6** (`integration`): `describe` reports `storage.size_bytes` matching the mailbox file's size, and null when the file is absent or unreadable, still exiting 0. ### Wait bounds - [x] **AC7** (`integration`): `watch` with no flags waits 90 minutes. - [x] **AC8** (`integration`): `--timeout-minutes` below 5 or above 240 exits non-zero with a JSON error naming the range, and does not wait. - [x] **AC9** (`integration`): `--once` returns immediately — the empty result when nothing is waiting, the message when one is — and is rejected when combined with `--timeout-minutes`. - [x] **AC10** (`integration`): `--timeout` and `--poll` each exit non-zero with a JSON error naming the replacement flag. - [x] **AC11** (`integration`): The blocking loop returns its empty result once the deadline elapses, verified by driving the loop directly with a sub-floor deadline so the test does not wait out the floor. ### Compatibility and release - [x] **AC12** (`integration`): A mailbox created by `0.1.8`, including rows carrying `ttl_hours`, opens under `0.2.0` with no migration and returns every stored message. - [x] **AC13** (`integration`): `ruff check src/ tests/` and the full `pytest` suite pass on the existing CI matrix with no workflow changes and no new runtime dependencies. - [x] **AC14** (`manual`): GitHub Release `v0.2.0` carries the six artifacts required by spec 002 AC3, PyPI and npm both serve `0.2.0`, and the smoke workflow passes on all four platforms. - [x] **AC15** (`manual`): On JJ's machine against the live shared mailbox, the installed `0.2.0` rejects `--timeout-minutes 0.5`, rejects `--timeout 30` with the migration message, and a message sent before the upgrade is still readable afterwards. ## Testing Approach ### Validation steps 1. Age messages by writing `created` directly, rather than by waiting. The behavior under test is what the code does with an old timestamp, which a fixture states exactly and a sleep only approximates. 2. Drive the blocking loop directly for the deadline case. The floor is a property of the *interface*, so testing the mechanism beneath it with a small value is legitimate and keeps the suite fast. 3. Build an `0.1.8`-shaped mailbox by writing the old schema and rows, then open it with the current code — the compatibility claim is about someone else's existing file, not about one this version created. ### Test cases | Input | Expected | |---|---| | `read` on a mailbox whose only message is 30 days old | The message is returned | | `status` after any command on an aged mailbox | Counts unchanged; nothing deleted | | `cleanup` | JSON error; nothing deleted | | `cleanup --older-than 7 --dry-run` | Reports what would go; deletes nothing | | `cleanup --older-than 7` | Removes only rows older than 7 days, plus their acks and stale watchers | | `watch --timeout-minutes 0.5` | JSON error naming the 5–240 range | | `watch --timeout 30` | JSON error naming `--timeout-minutes` | | `watch --poll 0.1` | JSON error saying the interval is fixed | | `watch --once` on an empty mailbox | Immediate empty result | | `watch --once --timeout-minutes 30` | JSON error; the two are mutually exclusive | | `describe` on a populated mailbox | `size_bytes` equals the file size | ### Release validation ```bash gh run watch gh release view v0.2.0 npm view @juanjofuchs/agent-mail@0.2.0 version ``` A same-day `npx` check needs `npx --min-release-age=0`, and the smoke workflow can return `ETARGET` for about a minute after publish — propagation, not a failure (spec 007). ## Usage Examples ```bash # Wait 90 minutes by default. The number an agent reaches for is now the right one. agent-mail watch research:worker agent-mail watch research:worker --timeout-minutes 120 # One non-blocking check — no number to get wrong. agent-mail watch research:worker --once # Deliberate, explicit, and never automatic. agent-mail cleanup --older-than 30 --dry-run agent-mail cleanup --older-than 30 ``` ## Out of Scope - Dropping the `ttl_hours` column, or any schema migration. - Per-message or per-agent retention policies, archival tiers, or exporting messages before deletion. - Automatic cleanup on any schedule, or a size threshold that triggers deletion. `describe` reports size; nothing acts on it. - Compacting or vacuuming the database. - Changing the `read`/`ack`/broadcast semantics, the identity grammar, or the storage location. - Configurable poll intervals, liveness windows, or a configurable timeout floor. - Restoring messages already destroyed by the TTL — they are unrecoverable. - Backporting any of this to `scripts/mail.py` in JJ's vault, which still carries its own `--ttl`. ## References - spec 001: [`specs/001-agent-mail-cli.md`](001-agent-mail-cli.md) — the behavioral contract this spec amends. - spec 004: [`specs/004-output-integrity.md`](004-output-integrity.md) — whose incident report records the TTL erasing the evidence of a lost message. - spec 005: [`specs/005-storage-transparency.md`](005-storage-transparency.md) — the report-the-fact precedent that `size_bytes` follows. - spec 006: [`specs/006-inbox-watch.md`](006-inbox-watch.md) — introduced `watch` and the `--timeout` this spec replaces. - spec 007: [`specs/007-wrapper-resolution.md`](007-wrapper-resolution.md) — the diagnosability argument applied here to removed flags. - Implementation: [`src/agent_mail/cli.py`](../src/agent_mail/cli.py). - Project context: [PROJECT_UNDERSTANDING.md](../PROJECT_UNDERSTANDING.md).