---
id: "005"
title: Agent Mail Storage Transparency
status: complete
amends: specs/001-agent-mail-cli.md
blocked_by: []
blocks: []
---

# Agent Mail Storage Transparency

## Overview

Issue [#2](https://github.com/JuanjoFuchs/agent-mail-cli/issues/2) reports that `describe` advertises a hardcoded default database path instead of the one the process actually resolved. When `AGENT_MAIL_DB` is set, every other command reads the override while `describe` keeps naming the default — so the one command an agent runs to orient itself is the one command that lies about where the mailbox is.

The failure is silent rather than loud. The advertised default frequently exists, left behind by an earlier run before the override was set, so an agent that follows `describe` opens a real SQLite file, finds a stale or empty mailbox, and reports an empty inbox. Nothing distinguishes that from genuinely having no mail. This is the same family as issue #1: the tool returns a plausible answer instead of an error.

This spec makes `describe` report the storage it resolved for the current invocation, with enough context — provenance, existence, message count, and whether a leftover default is sitting alongside — that a wrong-mailbox situation is self-evident on the first command. Ships as `0.1.6`.

> **Completion rule:** This spec is not complete until all acceptance criteria are verified through the testing approach below, including the issue #2 repro re-run against a published `0.1.6` on the reporting machine with `AGENT_MAIL_DB` set. Build-only and CI-only verification are insufficient. The agent must iterate until verification passes.

## Goals

- `describe` names the mailbox the invoking process will actually use, never a path it will not open.
- An agent pointed at a stale leftover mailbox can tell from `describe` alone, without comparing timestamps by hand.
- A failed database operation names the file it was operating on.

## Requirements

### Functional Requirements

- **FR1**: `describe` reports the storage location resolved for the current invocation, not a statically composed default.
- **FR2**: The reported storage value is a structured object carrying the resolved path, the provenance of that path, the default path, whether each exists, and how many messages the resolved mailbox holds.
- **FR3**: `describe` performs no writes. It does not create the database file, its parent directory, or any mailbox content, whether or not the path exists.
- **FR4**: When the message count cannot be determined — the file is absent, unreadable, not a SQLite database, missing the expected table, or held by a concurrent writer — `describe` reports the count as null, retains every other field, and exits 0.
- **FR5**: When a command fails because of a database or filesystem error, the JSON error message names the resolved database path.

### Non-Functional Requirements

- **NFR1**: `describe` remains a single JSON document on stdout with exit 0, requires no network, and requires no existing database (spec 001 NFR3 is preserved).
- **NFR2**: `describe` never creates the database file or its parent directory and never modifies mailbox content. Ephemeral SQLite sidecar files (`-wal`, `-shm`) are managed by SQLite and are outside this guarantee.
- **NFR3**: `describe` never blocks waiting for a database lock. If the count cannot be taken promptly, it is null.
- **NFR4**: No new runtime dependencies and no changes to the CI, release, npm publish, or WinGet workflows.

### Technical Constraints

- **TC1**: The reported paths are absolute, matching what the CLI will actually open. `AGENT_MAIL_DB` resolution semantics are unchanged: no tilde expansion, no search paths, no fallbacks. A relative override is reported in its absolute form, resolved against the current working directory exactly as SQLite would resolve it.
- **TC2**: Provenance is `AGENT_MAIL_DB` when that variable is set to a non-empty value, and `default` otherwise. An empty `AGENT_MAIL_DB` resolves to the default, matching existing behavior.
- **TC3**: The message count is the number of rows in the `messages` table of the resolved database.
- **TC4**: `describe` opens the resolved database read-only, so a missing file is reported rather than created.
- **TC5**: With no database present, `describe` exits 0 and reports the resolved path as not existing with a null count.
- **TC6**: The storage value changes shape from a string to an object. `describe`'s top-level `storage` key is retained (spec 001 AC1).
- **TC7**: The fix ships as version `0.1.6` through the existing PyPI, GitHub Release, and npm channels defined by specs 002 and 003.

### Requirement Traceability

| Requirement | Acceptance Criteria |
|---|---|
| FR1 | AC1, AC2 |
| FR2 | AC1, AC2, AC3, AC4, AC5 |
| FR3 | AC6 |
| FR4 | AC5, AC7 |
| FR5 | AC9 |
| NFR1 | AC5, AC7 |
| NFR2 | AC6 |
| NFR3 | AC7 |
| NFR4 | AC11 |
| TC1 | AC1 |
| TC2 | AC1, AC2 |
| TC3 | AC5 |
| TC4 | AC6, AC7 |
| TC5 | AC5 |
| TC6 | AC1, AC10 |
| TC7 | AC12, AC13, AC14 |

## Reported Shape

```json
"storage": {
  "path": "D:\\jfuchs\\dev\\second-brain\\.agent-mail\\mail.db",
  "source": "AGENT_MAIL_DB",
  "default": "C:\\Users\\jfuchs\\.agent-mail\\mail.db",
  "exists": true,
  "default_exists": true,
  "messages": 9
}
```

| Field | Meaning |
|---|---|
| `path` | Absolute path this invocation will read and write |
| `source` | `AGENT_MAIL_DB` or `default` — which rule produced `path` |
| `default` | The default path, always reported, even when overridden |
| `exists` | Whether `path` exists on disk |
| `default_exists` | Whether `default` exists on disk — a leftover indicator when `source` is `AGENT_MAIL_DB` |
| `messages` | Row count in the resolved mailbox, or null when it cannot be determined |

## Key Decisions

### `describe` describes this invocation, not the product

The root cause is a category error: `storage` was authored as a fact about Agent Mail rather than a fact about the running process. Spec 001 AC4 codified that reading — "`describe` produces identical output regardless of whether the database file exists" — and this spec amends it. Environment-dependent fields are the point, not a regression. Command schemas, identity rules, and invariants remain static.

### Provenance is the field that resolves the ambiguity

`path` alone tells the agent where to look but not why, so an agent cannot tell a deliberate override from an accident. `source` is what makes the answer actionable, and it is the field that makes a misconfigured environment diagnosable from one command.

### The message count is worth touching the database for

`path` and `source` fix the lie, but the reported failure was an agent reading a *plausible* mailbox. What makes a stale leftover self-evident is its contents. A count is one cheap query, and it converts an invisible misconfiguration into an obvious one: an override in effect, a leftover default on disk, and a count that does not match expectations.

### A null count beats an error

`describe` is the orientation command and must answer under every condition, including a missing, corrupt, or locked mailbox. Any failure to count degrades to null while the rest of the document stands. A `describe` that fails because the mailbox is broken would withhold exactly the information needed to diagnose it.

### Read-only, and never a creator

`describe` is the one command spec 001 exempts from database creation, and the count must not change that. The database is opened read-only so a missing file is reported rather than created. SQLite's own `-wal` and `-shm` bookkeeping files are outside that guarantee because SQLite manages them; the mailbox itself is never created or modified.

### `default_exists` is data, not a warning

The reported trap is an override in effect while a stale default sits nearby looking plausible. Exposing the condition as a field lets the agent act on it. A JSON-only CLI has no warning channel, and adding one — stderr output on a successful command — would break the invariant that stderr carries errors only.

### Errors name the file

A database error that says only "unable to open database file" sends an agent looking in the wrong place, which is the same class of defect as the advertised path. Naming the resolved path costs one string and makes the failure self-locating.

### Only the total count, not per-agent counts

`describe` takes no agent identity, so it has no basis for scoping unread or unacked counts. Per-agent counts are `status`'s contract and stay there.

### Spec 001 is amended, not superseded

As with spec 004, spec 001 remains the standing behavioral contract. This spec owns the change and its verification; its implementation tasks fold the resulting rules into spec 001.

## Implementation Tasks

- [x] Resolve the database path once per invocation and report it from `describe` as the documented object.
- [x] Report the message count from a read-only open that never creates the database or its parent directory, degrading to null on any failure.
- [x] Name the resolved database path in errors caused by database or filesystem failures.
- [x] Record the storage contract in the schema's invariants so `describe` remains self-describing.
- [x] Add regression coverage for the reported repro, for the leftover-default scenario, and for the no-side-effects guarantee.
- [x] Update the parity tests so an environment-dependent storage block is compared meaningfully across the three call sites.
- [x] Amend spec 001: replace the static storage description, retire the "identical output" wording in AC4, and add acceptance criteria for the resolved storage report and the read-only guarantee.
- [x] Bump the package version to `0.1.6`; the npm package version is set by the publish workflow.
- [x] Add the `0.1.6` CHANGELOG entry referencing issue #2.
- [x] Release `v0.1.6` after JJ's approval and verify the published artifacts.
- [x] Close issue #2 referencing the released version.

## Verification Record

- [x] `ruff check src/ tests/` passed.
- [x] `pytest` passed: 33 passed, 0 skipped, with the PyInstaller binary and npm wrapper both available.
- [x] All eight storage tests fail against the pre-fix `cli.py` (verified by stashing only that file), so they are genuine regression coverage.
- [x] Full behavior, output-integrity, and storage suites pass with `AGENT_MAIL_RUNNER=binary` (27 passed) and `AGENT_MAIL_RUNNER=npm` (27 passed).
- [x] Issue #2 checked against the live second-brain mailbox with a local build: `describe` reports `path` = the override, `source` = `AGENT_MAIL_DB`, `default_exists` = true, `messages` = 14. A read-only cross-check confirms the override held 14 messages written that day while the default held 1 message last written 18 days earlier — the stale mailbox the old `describe` pointed agents at.
- [x] CI passed on `main` for Python 3.10-3.13.
- [x] Release workflow `30382210685` succeeded; GitHub Release `v0.1.6` carries the wheel, the sdist, and the four platform binaries.
- [x] PyPI `agent-mail-cli==0.1.6` is live (confirmed against the PyPI JSON API; `pip index` served a cached listing briefly).
- [x] npm publish workflow `30382361496` succeeded; `npm view @juanjofuchs/agent-mail@0.1.6 version` returns `0.1.6`.
- [x] `npm-smoke.yml` run `30382406914` passed on Windows x64, Linux x64, macOS x64, and macOS arm64.
- [x] AC14 verified against the published package on the reporting machine: `npx --min-release-age=0 -y @juanjofuchs/agent-mail@0.1.6 describe` with `AGENT_MAIL_DB` set reports `path` = the second-brain mailbox, `source` = `AGENT_MAIL_DB`, `default_exists` = true, `messages` = 16, matching a read-only row count of the same file.
- [x] Issue #2 closed referencing `0.1.6`.

## Acceptance Criteria

Each criterion names its validation method. `integration` criteria are automated tests owning their own fixtures; `manual` criteria require live published artifacts and are noted with why they cannot be automated here.

### Resolved storage report

- [x] **AC1** (`integration`): With `AGENT_MAIL_DB` set, `describe` reports `storage.path` equal to that path in absolute form and `storage.source` equal to `AGENT_MAIL_DB`. This is the issue #2 repro.
- [x] **AC2** (`integration`): With `AGENT_MAIL_DB` unset, `storage.path` is the default path and `storage.source` is `default`. With `AGENT_MAIL_DB` set to an empty value, the result is identical to unset.
- [x] **AC3** (`integration`): `storage.default` names the default path in both cases, including when overridden.
- [x] **AC4** (`integration`): `storage.exists` and `storage.default_exists` each reflect whether that path is present on disk.
- [x] **AC5** (`integration`): `storage.messages` equals the number of stored messages after a known number of sends, and is null with `storage.exists` false when no database is present — in which case `describe` still exits 0.

### No side effects

- [x] **AC6** (`integration`): `describe` pointed at a non-existent database inside a non-existent parent directory exits 0 and creates neither the file nor the directory.
- [x] **AC7** (`integration`): When the resolved path exists but is not a readable SQLite database, `describe` exits 0 with `exists` true and `messages` null, and does not modify the file.

### Wrong-mailbox diagnosis

- [x] **AC8** (`integration`): In the reported scenario — a populated default mailbox plus an override pointing at a different populated mailbox — `describe` reports the override's path, `source` `AGENT_MAIL_DB`, `default_exists` true, and the override's message count, so the two are distinguishable from `describe` alone.

### Error transparency

- [x] **AC9** (`integration`): A command that fails on a database or filesystem error exits non-zero with a JSON error on stderr whose message contains the resolved database path.

### Parity and regression suite

- [x] **AC10** (`integration`): For a shared resolved database, the importable module, the PyInstaller binary, and the npm wrapper produce identical `describe` output including the storage block.
- [x] **AC11** (`integration`): `ruff check src/ tests/` and the full `pytest` suite pass on the existing CI matrix (Python 3.10–3.13 on Linux) with no workflow changes and no new runtime dependencies.

### Released fix

- [x] **AC12** (`manual`): GitHub Release `v0.1.6` carries the six artifacts required by spec 002 AC3, PyPI serves `agent-mail-cli==0.1.6`, and `npm view @juanjofuchs/agent-mail@0.1.6 version` returns `0.1.6`. Verified by the agent against live registries, which CI cannot assert about its own unpublished build.
- [x] **AC13** (`integration`): The `npm-smoke.yml` workflow passes for `0.1.6` on Windows x64, Linux x64, macOS x64, and macOS arm64.
- [x] **AC14** (`manual`): On the reporting machine with `AGENT_MAIL_DB` set to the second-brain mailbox, `npx -y @juanjofuchs/agent-mail@0.1.6 describe` reports that path with `source` `AGENT_MAIL_DB` and a message count matching what `read` returns there. Requires the reporter's real environment and mailbox.

## Testing Approach

### Test cases

| Input | Expected output |
|---|---|
| `AGENT_MAIL_DB=<path> describe` | `path` = absolute `<path>`, `source` = `AGENT_MAIL_DB` |
| `describe` with the variable unset | `path` = default, `source` = `default` |
| `AGENT_MAIL_DB=` (empty) `describe` | Identical to unset |
| `describe` after three sends | `messages` = 3, `exists` = true |
| `describe` with no database present | `messages` = null, `exists` = false, exit 0 |
| `describe` at a missing path under a missing directory | Exit 0; neither is created |
| `describe` where the path is a text file | Exit 0, `exists` true, `messages` null |
| Populated default plus populated override | Override's path and count, `default_exists` true |
| A command failing on a database error | Non-zero exit; JSON error naming the resolved path |

### Local validation before release

```bash
ruff check src/ tests/
pytest
python -m build
twine check dist/*
```

### Release validation

```bash
gh run watch
gh release view v0.1.6
npm view @juanjofuchs/agent-mail@0.1.6 version
```

A same-day `npx` verification on JJ's machine needs `npx --min-release-age=0`, per the note in spec 004.

### Human-in-the-Loop Release Protocol

1. **Agent**: Implements, passes local validation, and reports results.
2. **Agent**: Asks JJ before pushing the release tag.
3. **Human**: Approves the release tag.
4. **Agent**: Pushes `v0.1.6`, monitors GitHub Actions, and reports PyPI, GitHub Release, and npm status.
5. **Agent**: Runs the AC14 check against the real second-brain mailbox and reports what `describe` says.
6. **Agent**: Closes issue #2 referencing `0.1.6`, or iterates if verification fails.

## Usage Examples

The repro from issue #2, expected to work after this spec:

```console
$ echo $AGENT_MAIL_DB
D:\jfuchs\dev\second-brain\.agent-mail\mail.db

$ agent-mail describe
{
  ...
  "storage": {
    "path": "D:\\jfuchs\\dev\\second-brain\\.agent-mail\\mail.db",
    "source": "AGENT_MAIL_DB",
    "default": "C:\\Users\\jfuchs\\.agent-mail\\mail.db",
    "exists": true,
    "default_exists": true,
    "messages": 9
  }
}
```

## Out of Scope

- Changing `AGENT_MAIL_DB` resolution: no tilde expansion, no search paths, no fallback to the default when the override is missing.
- Migrating, merging, or cleaning up a stale default mailbox.
- Reintroducing a `--db` flag — removed by the spec 001 audit.
- Reporting storage from `read`, `status`, or any command other than `describe`.
- Warnings on stdout or stderr for successful commands.
- Per-agent unread or unacked counts in `describe` — `status`'s contract.
- Locking, multi-mailbox, or cross-machine storage.
- Workflow changes for CI, release, npm publish, or WinGet.

## References

- Issue: [#2 — describe advertises the default DB path, not the resolved one](https://github.com/JuanjoFuchs/agent-mail-cli/issues/2).
- spec 001: [`specs/001-agent-mail-cli.md`](001-agent-mail-cli.md) — the behavioral contract this spec amends.
- spec 002: [`specs/002-packaging.md`](002-packaging.md) — PyPI and GitHub Release machinery that ships `0.1.6`.
- spec 003: [`specs/003-npm-distribution.md`](003-npm-distribution.md) — npm wrapper and smoke workflow used by AC13 and AC14.
- spec 004: [`specs/004-output-integrity.md`](004-output-integrity.md) — the sibling silent-failure fix, and the `npx --min-release-age=0` note.
- Implementation: [`src/agent_mail/cli.py`](../src/agent_mail/cli.py).
- Project context: [PROJECT_UNDERSTANDING.md](../PROJECT_UNDERSTANDING.md).