--- id: "010" title: Messages Are Not Documents status: complete amends: specs/001-agent-mail-cli.md blocked_by: [] blocks: [] --- # Messages Are Not Documents ## Overview Agents are pasting whole documents into message bodies instead of writing a file and referencing it. This is not a tail of offenders — it is the norm. In a 142-message sample of the live mailbox the **median body is 3,548 characters**, the mean is 3,725, the largest is 11,235, and the bodies together hold **529 KB of prose**. Only 17 messages carry `--refs` at all. The rule already exists. `describe`'s `content_routing` block has said since v1: *"If it should outlive the current task, it's a file, not a message. Messages reference files via --refs but don't manage them."* Every agent reads it at orientation and encodes a document anyway. This is [spec 008](008-remove-the-wrong-choices.md)'s lesson a second time: **a surface that lets an agent choose wrongly will be chosen wrongly**, and documentation is not a mechanism. There is a second finding that changes the shape of the fix. Of the 43 stored ref paths, **42 are relative**, and 11 of the 17 messages carrying refs cross a project boundary — where spec 001 defines a ref as "relative to sender's project root", which the recipient does not know. Those refs do not resolve for the recipient today, and nobody has noticed, because **the body duplicates the content and the ref is decorative**. Capping the body promotes refs from decoration to load-bearing, so this spec has to make them work. Ships as `0.4.0`. This is a breaking change: sends that succeeded before will now fail. > **Completion rule:** This spec is not complete until all acceptance criteria are verified through the testing approach below, including a rejected over-length send and a successful file-plus-ref send performed against the live mailbox with the published `0.4.0`. Build-only and CI-only verification are insufficient. The agent must iterate until verification passes. ## Goals - A message that is really a document fails to send, and the failure names the fix. - The wrong value is unreachable rather than discouraged — no flag, no file-shaped bypass, no adjacent field to stuff instead. - A ref the sender writes is a path the recipient can actually open. - A woken agent can scan what arrived without paying for a document it did not ask for. ## Requirements ### Functional Requirements - **FR1**: `send` rejects a body longer than the body limit. The error states the actual length and the limit, directs the caller to put the durable text in a file and reference it, and carries a literal example of the corrected command. - **FR2**: The body limit applies identically however the body was supplied. `--body` and `--body-file` are measured the same way, and no flag, environment variable, or argument raises or bypasses the limit. - **FR3**: The rejection names splitting one payload across several messages as a non-solution, so the obvious evasion is refused in the same breath as the original mistake. - **FR4**: `send` rejects a subject longer than the subject limit, so content cannot migrate from the body into the subject. - **FR5**: Every `--refs` entry is resolved to an absolute path. An entry that cannot be resolved, or that does not exist on disk, is an error naming the offending entry and the path it resolved to; no message is stored. - **FR6**: Stored refs are the resolved absolute paths, so a recipient in another project can open them without knowing the sender's working directory. - **FR7**: `describe` documents both limits, the refs rule, and the file-plus-ref pattern, so the whole contract remains discoverable from `describe` alone. - **FR8**: The limits apply only when a message is sent. Messages already stored — including ones longer than the limits — are still returned by `read`, counted by `status`, and wake `watch`. ### Non-Functional Requirements - **NFR1**: The JSON output and error contract is unchanged (spec 001 NFR1, NFR2): errors remain a single JSON object on stderr with an `error` key and a non-zero exit. - **NFR2**: A rejected `send` stores nothing, consistent with spec 004's commit-after-output rule. - **NFR3**: No new runtime dependencies, and no changes to the CI, release, npm publish, or WinGet workflows. ### Technical Constraints - **TC1**: The body limit is **2000 characters**, counted in Unicode code points on the body after `--body-file` has been read, so the two input paths are measured identically. - **TC2**: The subject limit is **200 characters**, counted the same way. - **TC3**: Both limits are fixed constants. There is no flag, no environment variable, and no per-message override. - **TC4**: `--refs` entries resolve against the process's current working directory. The stored value is the JSON array of resolved absolute paths, replacing the caller's strings. - **TC5**: A refs entry qualifies if it exists as a file or a directory. An empty string, a path whose resolution raises, or a path that does not exist is an error. - **TC6**: `--refs` remains a JSON array of strings and remains optional. Sending without refs is still valid — the limits constrain length, not structure. - **TC7**: No read-side command gains a length check, and no stored message is altered or migrated by this change. - **TC8**: Ships as `0.4.0` through the existing PyPI, GitHub Release, and npm channels defined by specs 002 and 003. ### Requirement Traceability | Requirement | Acceptance Criteria | |---|---| | FR1 | AC1, AC2 | | FR2 | AC3, AC4 | | FR3 | AC2 | | FR4 | AC5 | | FR5 | AC6, AC7 | | FR6 | AC8 | | FR7 | AC9 | | FR8 | AC10 | | NFR1 | AC2, AC5, AC7 | | NFR2 | AC11 | | NFR3 | AC12 | | TC1 | AC1, AC3 | | TC2 | AC5 | | TC3 | AC4 | | TC4 | AC8 | | TC5 | AC6, AC7 | | TC6 | AC13 | | TC7 | AC10 | | TC8 | AC14, AC15, AC16 | ## Reported Shape An over-length body: ```json { "error": "Body is 8789 characters; the limit is 2000. A message is coordination, not content — this is a document. Write it to a file in your project and reference it: agent-mail send --from --to --subject \"\" --body \"\" --refs '[\"path/to/file.md\"]'. Do NOT split it across several messages: that puts the same payload in the same channel." } ``` An unresolvable ref: ```json { "error": "--refs entry does not exist: 'research/typo.md' (resolved to 'D:\\projects\\team-alpha\\research\\typo.md'). Refs are resolved and stored as absolute paths so the recipient can open them; write the file before sending, or correct the path." } ``` ## Key Decisions ### The number comes from the data, not from taste 2000 characters is roughly 500 tokens, 300 words, one dense page — and it would have rejected 85% of the sampled traffic. That sounds severe until you notice the median body is 3,548 characters: the thing being changed is not a handful of outliers but the prevailing style. A cap set above the median would ratify exactly the behavior this spec exists to stop. Erring low is also the cheap direction. The fix is one file write and costs nothing in fidelity — the content survives in full, in a place that outlives the task, and the recipient decides when to load it. Erring high preserves the status quo. When one side of a bound is recoverable and the other is not, take the recoverable side. (Same reasoning shape as spec 008's TTL removal, where deletion was the irreversible direction.) ### A hard error, never truncation Truncating at 2000 would "work" and lose the tail silently. Silent loss is the exact failure class specs 004 and 005 were written to remove; reintroducing it as a convenience would be a bad trade at any cap. The send fails, nothing is stored, and the caller still holds the text. ### The subject is capped too, because otherwise it becomes the body The sampled subjects already run to 382 characters with a median of 169 — they are carrying content, not naming it. Capping the body while leaving the subject unbounded would relocate the problem within the hour, which is the reachable-wrong-value failure with extra steps. 200 characters rejects about 17% of current subjects: a nudge rather than a redesign, because the body cap is doing the real work here. The subject has a second job that makes its length load-bearing: it is the only descriptive field `watch` returns. A wake carrying five 380-character subjects spends real context to say "you have mail". ### `--body-file` gets the same limit as `--body` Spec 001's audit added `--body-file` to solve *shell escaping* on bodies with code blocks and markdown — not to license length. Exempting it would leave a one-flag bypass in plain sight, and an agent that hits the cap will find it immediately. Measuring after the file is read makes the two paths indistinguishable to the rule. ### No override flag An override is a choice, and spec 008 established what happens to a choice an agent can get wrong: it gets chosen wrongly. There is also no case an override would serve — the file-plus-ref path is always available, always lossless, and strictly better for the recipient. ### Naming the evasion inside the error An agent told only "too long" can satisfy the letter of the rule by sending four messages of 2000 characters. That is worse than the original: the same payload, the same channel, plus fragmentation. The error names it explicitly. This is not documentation-as-mechanism — the error arrives at the moment of the mistake, to the agent that is about to make it, which is the one place a sentence still works. ### Refs become absolute and verified, overturning a spec 001 decision Spec 001's audit decided "no path canonicalization on `--refs`" and "the CLI never opens those paths", on the reasoning that refs are the caller's business. That was defensible while refs were optional decoration. It stops being defensible the moment the body cap makes refs the only channel for substance — and the data shows the current state is already broken: 42 of 43 stored paths are relative, and most ref-carrying traffic crosses project boundaries, where "relative to the sender's project root" is a path the recipient cannot resolve. Resolving against the sender's cwd is the honest reading of what the sender meant. Verifying existence is what stops the resolution from being a *plausible wrong answer* when the agent's cwd is not its project root: instead of storing a confidently wrong absolute path, the send fails while the sender still has the context to fix it. Checking existence is a stat, not an open, so the CLI still never reads the referenced content. ### The error stays a single `error` string Spec 007 gave the npm wrapper structured error payloads, and it would be tempting to give this one `chars`, `limit`, and `next` fields. Declined: every error the Python CLI emits is `{"error": ""}` per spec 001 NFR2, and making one error shape special buys nothing an agent cannot read from the message. The message carries the numbers and the literal corrected command, which is the part spec 009 showed actually changes behavior. ### Stored messages are left alone The limits are a rule about sending. Applying them at read time would make existing messages unreadable — destroying the record, which is what spec 008 was written to stop. Every stored message stays readable regardless of length. ## Implementation Tasks - [x] Enforce the body limit on the resolved body, after `--body-file` is read, with the documented error. - [x] Enforce the subject limit with the same error contract. - [x] Resolve each `--refs` entry to an absolute path, reject entries that cannot be resolved or do not exist, and store the resolved paths. - [x] Record both limits, the refs rule, and the file-plus-ref pattern in the schema's `send` arguments, `content_routing`, and invariants. - [x] Add regression coverage for both limits, the `--body-file` equivalence, the refs resolution and rejection paths, and the read-side exemption. - [x] Amend spec 001: add the limits and the refs resolution rule, and retire the "no path canonicalization" and "never opens those paths" audit decisions with a pointer to this spec. - [x] Bump the package version to `0.4.0` and add the CHANGELOG entry, calling out the breaking change. - [x] Release `v0.4.0` after JJ's approval and verify the published artifacts. ## Verification Record - [x] `ruff check src/ tests/` passed; `pytest` passed 105 tests, 0 skipped, with the PyInstaller binary and npm wrapper both available. - [x] Eight of the eleven new tests fail against the pre-fix `cli.py` (verified by stashing only that file). The three that pass either way are the compatibility cases by design: the read-side exemption, sending without refs, and refs given as already-absolute paths. - [x] The limits and refs suites pass with `AGENT_MAIL_RUNNER=binary` (21 passed) and `AGENT_MAIL_RUNNER=npm` (21 passed). - [x] Two pre-existing tests failed on the refs change and were updated rather than worked around — they passed literal `["a","b"]` paths that never existed, which is exactly the pattern this spec makes an error. - [x] The tests hardcode 2000 and 200 rather than importing the constants: a test that reads the value it is checking cannot catch a wrong value. - [x] Caught during implementation: the missing-ref error used `repr()` on the resolved path, which doubles every backslash on Windows. The path is now printed plainly, since that string exists to be acted on. - [x] CI passed on `main` (`ad7ac96`); release workflow `30933148594` succeeded across all jobs. - [x] GitHub Release `v0.4.0` carries the wheel, the sdist, and the four platform binaries. PyPI and npm both serve `0.4.0`. - [x] `npm-smoke.yml` run `30933348574` passed on Windows x64, Linux x64, macOS x64, and macOS arm64. - [x] AC16 verified end to end against the live shared mailbox (155 messages) with the globally installed `0.4.0`: - An 8,789-character body — one of the historical shapes — was rejected with the documented error and exit 1, storing nothing. - The same content through `--body-file` was rejected identically (8,790 chars), and a 400-character subject was rejected too, so neither bypass is open. - A ref naming a missing file was refused, and the error named the path it resolved to. - The corrected pattern succeeded: a 126-character body plus a **relative** `--refs` entry sent from the project directory. - Read back from a **different working directory**, that ref came out absolute and opened — the cross-project case that silently failed before. - [x] Read-side exemption confirmed on real data: 126 stored messages exceed the new cap (largest body 11,235 characters, largest subject 315), and `read` and `status` still return them untouched. ## Acceptance Criteria Each criterion names its validation method. `integration` criteria are automated tests owning their own fixtures; `manual` criteria require live published artifacts. ### Body limit - [x] **AC1** (`integration`): A `send` whose body exceeds 2000 characters exits non-zero with a JSON error naming the actual length and the limit, and stores no message. - [x] **AC2** (`integration`): That error directs the caller to write a file and use `--refs`, includes a literal corrected `send` command, and explicitly refuses splitting the payload across messages. - [x] **AC3** (`integration`): A body of exactly 2000 characters is accepted and round-trips intact; 2001 is rejected. The same two cases delivered through `--body-file` produce identical outcomes. - [x] **AC4** (`integration`): No flag or environment variable raises the limit — the limit is a constant with no argument that reaches it. ### Subject limit - [x] **AC5** (`integration`): A subject longer than 200 characters is rejected with a JSON error naming the length and the limit; 200 exactly is accepted. ### Refs - [x] **AC6** (`integration`): A `--refs` entry naming an existing file is accepted; one naming an existing directory is accepted; one naming a missing path is rejected with an error carrying both the original entry and its resolved absolute form, and stores no message. - [x] **AC7** (`integration`): An empty-string entry, and an entry whose resolution raises, are each rejected with a JSON error and store no message. - [x] **AC8** (`integration`): A relative ref sent from one working directory is stored as an absolute path and is openable by a reader whose working directory is elsewhere — the cross-project case that fails today. ### Schema and stored messages - [x] **AC9** (`integration`): `describe` reports both limits and the refs rule in the `send` command schema, and the invariants state that a long body belongs in a file referenced by `--refs`. - [x] **AC10** (`integration`): A message longer than the limits, written directly into the mailbox, is still returned by `read`, counted by `status`, and wakes `watch`. - [x] **AC13** (`integration`): A `send` with no `--refs` is still valid, and an empty `--refs` array is still accepted. ### Regression and release - [x] **AC11** (`integration`): A rejected `send` — for any of the reasons above — leaves the mailbox byte-identical, verified by direct database inspection. - [x] **AC12** (`integration`): `ruff check src/ tests/` and the full `pytest` suite pass on the existing CI matrix with no workflow changes and no new runtime dependencies. - [x] **AC14** (`manual`): GitHub Release `v0.4.0` carries the six artifacts required by spec 002 AC3, PyPI and npm both serve `0.4.0`, and the smoke workflow passes on all four platforms. - [x] **AC15** (`integration`): The `npm-smoke.yml` workflow passes for `0.4.0` on Windows x64, Linux x64, macOS x64, and macOS arm64. - [x] **AC16** (`manual`): Against the live mailbox with the installed `0.4.0`, a send carrying one of the historical 8,000-character bodies is rejected with the documented error, and the same content sent as a file plus `--refs` succeeds and is readable by the recipient identity with an absolute path. ## Testing Approach ### Validation steps 1. Build the over-length fixtures programmatically rather than pasting prose, so the boundary cases are exactly 2000 and 2001 characters. 2. Exercise `--body` and `--body-file` through the same assertions, since the whole point of TC1 is that they are indistinguishable to the rule. 3. Drive the cross-project refs case by running `send` and `read` from different working directories, which is the condition that makes a relative ref unresolvable. 4. Write an over-length message directly into the mailbox with SQLite for AC10 — the claim is about messages the current version would refuse to create. 5. Re-run the suite with `AGENT_MAIL_RUNNER=binary` and `AGENT_MAIL_RUNNER=npm`. ### Test cases | Input | Expected | |---|---| | `send --body <2000 chars>` | Accepted, body round-trips | | `send --body <2001 chars>` | JSON error naming 2001 and 2000; nothing stored | | `send --body-file <2001 chars>` | Identical error | | `send --subject <201 chars>` | JSON error naming the subject limit | | `send --refs '["existing.md"]'` | Stored as an absolute path | | `send --refs '["missing.md"]'` | JSON error naming entry and resolved path; nothing stored | | `send --refs '[""]'` | JSON error; nothing stored | | `read` from another directory | The stored ref is absolute and opens | | A 9000-char message written directly to the DB | `read`, `status`, and `watch` all still return it | ### Release validation ```bash gh run watch gh release view v0.4.0 npm view @juanjofuchs/agent-mail@0.4.0 version ``` A same-day `npx` check needs `npx --min-release-age=0`, per the note in spec 004. ## Usage Examples The pattern this spec makes mandatory: ```bash # The findings go in a file, where they outlive the task. cat > research/benchmarks/FINDINGS.md <<'EOF' # Query rerun p95 latency 19.89 -> 28.04 ms (+8.15) ... EOF # The message says what happened, what to do, and where to look. agent-mail send --from analysis:researcher \ --to research:lead \ --subject "Benchmark rerun done: p95 +8.15ms, control holds" \ --body "Rerun landed. The regression is query-specific, not a global shift. Numbers and method in the refs; I need a ruling on whether to ship before the release commitment." \ --refs '["research/benchmarks/FINDINGS.md"]' ``` ## Out of Scope - Any limit on the number of messages, the number of refs, or total mailbox size. - Compressing, truncating, or auto-extracting long bodies into files on the sender's behalf. - Migrating, shortening, or flagging messages already stored. - Reading, parsing, or transmitting the contents of a referenced file. - Copying referenced files, or resolving them across machines — refs remain local paths, per the local-first scope in `PROJECT_UNDERSTANDING.md`. - Enforcing that a long-ish message actually carries refs. - A configurable or per-project limit. - Structured error payloads for the Python CLI — see Key Decisions. - Workflow changes for CI, release, npm publish, or WinGet. ## References - spec 001: [`specs/001-agent-mail-cli.md`](001-agent-mail-cli.md) — the behavioral contract this spec amends, including the audit decisions on `--refs` it overturns. - spec 004: [`specs/004-output-integrity.md`](004-output-integrity.md) — the silent-loss failure class that rules out truncation. - spec 008: [`specs/008-remove-the-wrong-choices.md`](008-remove-the-wrong-choices.md) — established that documentation does not hold and the fix must be structural. - spec 009: [`specs/009-rearm-state.md`](009-rearm-state.md) — the precedent for putting the literal corrected command in front of the agent. - Implementation: [`src/agent_mail/cli.py`](../src/agent_mail/cli.py). - Project context: [PROJECT_UNDERSTANDING.md](../PROJECT_UNDERSTANDING.md).