# EtherCalc TypeScript Rewrite — Ultraplan (archived)

> **Archived:** 2026-06-12 — moved from root `CLAUDE.md`. Active agent context is
> [`../../AGENTS.md`](../../AGENTS.md). Do not edit this file for new work unless
> you are deliberately updating historical record.
>
> **Status at archive:** complete · all §13 questions answered · **Owner:** Audrey Tang · **Started:** 2026-04-19
>
> Originally `CLAUDE.md` so it auto-loaded into Claude Code sessions. It is the durable plan-of-record for rewriting EtherCalc in modern TypeScript on the Cloudflare fullstack (Hono + Workers + Durable Objects + D1 + KV + R2), locally runnable via Miniflare, and also self-hostable without a Cloudflare account via `docker compose up`. 100% line/branch/function/statement coverage is enforced in CI on gated packages.
>
> **How to use this doc:** §1 is the contract. §13 is the canonical decision log — do not re-ask. §8 lists what's still open. §14 is the session log for context continuity. Update §14 when you finish work; edit other sections in place when reality diverges.

---

## 1. North Star

### 1.1 Done definition

Ship a greenfield TypeScript implementation of EtherCalc that:

1. **Passes a golden-fixture oracle-equivalence suite** against the current `main` branch — every §6.1 HTTP endpoint (identical status/headers/body for deterministic formats, structural equivalence for HTML/XLSX/ODS), every §6.2 WS message, every §6.3 Redis key pattern mapped to the new storage layer with equivalent semantics. Small allow-list of "sensible fixes" (§13 Q1) documented in §6.1.
2. **Runs locally under Miniflare** with the full feature set — DO WebSockets, D1 snapshots, KV indexes, R2 — no Redis/Node dependency.
3. **Deploys to Cloudflare Workers** via `wrangler deploy`.
4. **Is self-hostable** via `docker compose up` (standalone workerd container, persistent volume, no CF account needed — §13 Q5).
5. **Maintains 100% coverage** on gated packages in CI. Any PR dropping a metric below 100 fails.
6. **Preserves the public HTTP API** byte-for-byte where deterministic (minus sensible fixes).
7. **Client speaks new WS protocol** (raw JSON). Legacy `/socket.io/*` shim retained indefinitely for external embeds (§13 Q4).
8. **`multi/` ported to React 18 + TypeScript** (§13 Q2), preserving `/=:room` URL scheme.
9. **`ethercalc` CLI kept** as a thin wrapper around `wrangler dev` / Miniflare (§13 Q6).

### 1.2 Explicit non-goals

- Rewriting SocialCalc itself.
- Changing the spreadsheet wire format.
- Bug-for-bug preservation as absolute rule — sensible fixes allowed per §13 Q1 (default still leans preservation).
- New user-facing features.
- Multi-region / strong consistency upgrades.
- OAuth/Gmail email sending (replaced by `send_email` binding, §13 Q3).
- Dead-platform configs (`snapcraft`/`dotcloud`/`openshift`/`stackato`).
- `webworker-threads` backend (DO isolates cover the sandboxing property).
- Mandatory application-layer rate limiting on hosted deploys (relies on CF platform / WAF, §13 Q7). Self-host may opt into `ETHERCALC_RATELIMIT` behind a proxy edge.

---

## 2. Glossary

- **Oracle** — current `main` branch on Node/Bun + Redis; source of truth for semantics.
- **Target** — the new TypeScript Worker implementation.
- **Room** — spreadsheet / page, identified by a URL-safe string.
- **Multi-sheet** — URL prefix `/=:room` routing; sub-sheets per row of a TOC sheet.
- **DO** — Durable Object; one per room.
- **Snapshot** — SocialCalc save string (multi-section text blob).
- **Log** — commands since last snapshot; periodically folded back.
- **Audit** — append-only log of all commands (never folded).
- **ECell** — "editing cell"; each user's cursor position.

---

## 3. Target Architecture (Cloudflare fullstack)

### 3.1 Component map

| Concern                       | Old                                              | New                                                           |
| ----------------------------- | ------------------------------------------------ | ------------------------------------------------------------- |
| HTTP router                   | `zappajs` on Express 3                           | **Hono** on Workers                                           |
| Static assets                 | `express.static`                                 | **Workers Assets** (ASSETS binding)                           |
| Live spreadsheet state        | `SC[room]` global + `vm.createContext`           | **Durable Object** `RoomDO` — one per room                    |
| Persistent snapshot/log/audit | Redis `snapshot-*`, `log-*`, `audit-*`, `chat-*` | **DO storage** (primary) + **D1** mirror for cross-room query |
| Room index                    | `KEYS snapshot-*`                                | **D1** `rooms` table + optional KV hot path                   |
| Realtime transport            | socket.io 0.9                                    | DO-hosted **WebSocket** (hibernation API), raw JSON protocol  |
| Cron                          | External cron pinging `/_timetrigger`            | **Cron Triggers** invoking the Worker                         |
| Email                         | `nodemailer` + gmail xoauth2                     | **`send_email` binding** or stub                              |
| Secrets                       | CLI flag `--key`                                 | Worker secret `ETHERCALC_KEY` **and** CLI `--key` (§13 Q6)    |
| Self-host                     | `docker-compose.yml` (Node + Redis)              | standalone workerd image (§13 Q5)                             |

### 3.2 Request flow

```
Browser  ──HTTP/WS──>  Worker (Hono)
                         │
                         ├── static: Workers Assets
                         ├── stateless HTTP (rooms list, exists): D1
                         └── per-room (R/W, WS, exports):
                                 env.ROOM.get(idFromName(room)) ──> RoomDO
                                     ├── in-memory SocialCalc.SpreadsheetControl
                                     ├── state.storage (snapshot/log/audit/chat/ecell)
                                     ├── state.acceptWebSocket(ws) per client
                                     └── scheduled() — fold log into snapshot, mirror to D1
```

### 3.3 Data model

**DO storage (per-room)** — authoritative:
- `snapshot` — SocialCalc save string (versioned v2: prefix; reader falls back to v1)
- `log:<seq>` — indexed command strings
- `audit:<seq>` — same pattern, never truncated
- `chat:<seq>` — same pattern
- `ecell:<user>` — map of user → cell coord
- `meta:updated_at` — Date.now()

**D1**:
```sql
CREATE TABLE rooms (
  room        TEXT PRIMARY KEY,
  updated_at  INTEGER NOT NULL,
  cors_public INTEGER NOT NULL DEFAULT 0
);
CREATE TABLE cron_triggers (
  room      TEXT NOT NULL,
  cell      TEXT NOT NULL,
  fire_at   INTEGER NOT NULL,
  PRIMARY KEY (room, cell, fire_at)   -- compound PK preserves legacy comma-list semantics
);
```

### 3.4 SocialCalc inside a Durable Object

`packages/socialcalc-headless/` wraps SocialCalc (loaded via Vite `?raw` import) through a `new Function(...)` eval scaffold, with DOM stubs (`Node`, `document`, `window`, `navigator`) and a synchronous `setTimeout` shim. Plan A green; Plans B/C not needed. Full details in §16.A.

---

## 4. Oracle strategy

### 4.1 Equivalence

For each recorded scenario replayed against the target:
- **HTTP**: same status, `Content-Type`, body (exact bytes for deterministic; structural equality after normalization for HTML/XLSX/ODS).
- **DO state**: after each scenario, dump keys under the room and compare against oracle's Redis dump (normalized).
- **WebSocket transcript**: `(direction, timestamp_delta, message)` tuples; same sequence modulo timestamps.

### 4.2 Normalization rules

- Drop headers: `Date`, `Server`, `ETag`, `X-Powered-By`, `Connection`, `Accept-Ranges`, `Cache-Control`, `Content-Length`. `Last-Modified` semi-volatile → relax via `re:<regex>` matcher.
- **HTML** (`packages/oracle-harness/src/html-canonical.ts`): parse via linkedom; drop comments, whitespace-only text, `id` matching `/^(SocialCalc|[a-f0-9-]{32,})/`, referrer attributes pointing at dropped ids (`for`, `aria-labelledby`, `aria-controls`, `aria-describedby`, `headers`, `form`, `list`, `href="#volatileId"`). Attributes sorted alphabetically (note: linkedom's `setAttribute` prepends — sort reverse-alpha before re-insertion).
- **XLSX** (`packages/oracle-harness/src/zip-canonical.ts`): unzip via fflate, sort entries, canonicalize XML. Drop `docProps/core.xml` (`dcterms:created`, `dcterms:modified`, `cp:lastModifiedBy`, `cp:revision`) and `docProps/app.xml` (`AppVersion`, `TotalTime`).
- **ODS**: same pipeline; `meta.xml` drops `meta:creation-date`, `dc:date`, `meta:editing-duration`, `meta:editing-cycles`, `meta:generator`, `dc:creator` (depth-walk under `<office:meta>`).
- **SocialCalc save**: ignore `version:…` line and metadata-section ordering; compare `sheet:`/`cell:` lines exactly.
- **CSV/JSON/Markdown**: exact bytes.
- Socket IDs, timestamps, UUIDs, HMACs: `__PLACEHOLDER__` during comparison.

---

## 5. Testing strategy & coverage

### 5.1 Two vitest configs per package

- `vitest.config.ts` — `@cloudflare/vitest-pool-workers`, test files `*.test.ts`. **No coverage gate** (neither istanbul nor v8 reliably track hits through the workerd bundle).
- `vitest.node.config.ts` — Node env, test files `*.node.test.ts`. **100% coverage gate** on `src/handlers/**`, `src/lib/**`, `src/room.ts`.

`src/index.ts` (Hono glue) and workers-only shims like `src/lib/ws-upgrade.ts` are excluded from the Node gate.

### 5.2 Coverage — known limitation

**`@cloudflare/vitest-pool-workers` does not play well with istanbul or v8 coverage.** v8 reports 0% because workerd lacks Node's inspector; istanbul misses functions routed through Hono's bundled router. The two-config split above is the workaround; side-effect is handlers stay pure (DI for clocks, no direct env access) which aids testability.

### 5.3 Mutation testing — REQUIRED

Per-package `stryker.conf.json` with a `break` threshold pinned to the measured floor. PRs fail a fast `mutation-gate` CI job that runs Stryker only on packages whose `src/` changed. To raise a floor: close mutants (see `docs/MUTATION_REPORT.md` top-gaps), re-run `bun run mutation`, bump `break` in the same PR. Nightly runs the full matrix.

### 5.4 Test file naming

- `*.test.ts` — workers-pool integration tests.
- `*.node.test.ts` — pure-logic unit tests, coverage-gated.

---

## 6. Surface inventory

### 6.1 HTTP endpoints

`BASEPATH` is a prefix (default empty). `KEY` gates edit/view if set. `CORS` is now a legacy room-index gate; CORS headers are emitted unconditionally for embed compatibility. Prefer `ETHERCALC_DISABLE_ROOM_INDEX` for self-host discovery control.

| Method | Path                              | Content-Type req/res                                     | Notes                                                                                                                         |
| ------ | --------------------------------- | -------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| GET    | `/`                               | → `index.html`                                           |                                                                                                                               |
| GET    | `/_start`                         | → `start.html`                                           |                                                                                                                               |
| GET    | `/etc/*`, `/var/*`                | 404 `Content-Type: text/html; charset=utf-8`             | Explicit block; legacy Express default is `text/html`.                                                                        |
| GET    | `/favicon.ico`                    | `image/vnd.microsoft.icon`                               | **Sensible-fix** (§13 Q1) — legacy served as `text/html`.                                                                     |
| GET    | `/manifest.appcache`              | `text/cache-manifest`                                    | DevMode stub via `DEVMODE=1`.                                                                                                 |
| GET    | `/static/socialcalc.js`           | `application/javascript`                                 | From Workers Assets; external embeds depend on this path (§13 Q8).                                                            |
| GET    | `/static/form:part.js`            | `application/javascript`                                 | Literal colon in path; routed via `/static/:file{form.+\.js}` constrained segment.                                            |
| GET    | `/_new`, `/=_new`                 | 302 → new room (+`/edit` if KEY)                         | Auto-generated uuid.                                                                                                          |
| GET    | `/_timetrigger`                   | `application/json`                                       | Legacy cron endpoint; fires due triggers.                                                                                     |
| GET    | `/_rooms`                         | `application/json`                                       | `403` if room-index gate (`ETHERCALC_DISABLE_ROOM_INDEX`, legacy `CORS` fallback — §13 Q11).                                  |
| GET    | `/_roomlinks`                     | `text/html`                                              | **Sensible-fix** (§13 Q1) — legacy emitted JSON body with HTML CT.                                                            |
| GET    | `/_roomtimes`                     | `application/json`                                       | Sorted desc by `updated_at`.                                                                                                  |
| GET    | `/_from/:template`                | 302 → new room                                           | Copies template via DO-to-DO fetch.                                                                                           |
| GET    | `/_exists/:room`                  | `application/json` (**bare** boolean)                    | Per oracle F-05. Gated like `/_rooms` (§13 Q11).                                                                              |
| GET    | `/:room`                          | → `index.html` (or `multi/index.html` for `=`)           | Redirect to `?auth=0` / `?auth=<hmac>` if `KEY` set.                                                                          |
| GET    | `/:template/form`                 | 302 → `/<room>_<uuid>/app`                               | Uses `/_do/clone`.                                                                                                            |
| GET    | `/:template/appeditor`            | → `panels.html`                                          |                                                                                                                               |
| GET    | `/:room/{edit,view,app}`          | 302 with `?auth=<hmac>&…`                                |                                                                                                                               |
| GET    | `/_/:room`                        | `text/plain; charset=utf-8`                              | SocialCalc save; 404 with empty body if missing.                                                                              |
| GET    | `/_/:room/html`, `/:room.html`    | `text/html`                                              |                                                                                                                               |
| GET    | `/_/:room/csv`, `/:room.csv`      | `text/csv`; `Content-Disposition: attachment`            |                                                                                                                               |
| GET    | `/_/:room/csv.json`, `.csv.json`  | `application/json`                                       |                                                                                                                               |
| GET    | `/_/:room/{ods,fods}`             | `application/vnd.oasis.opendocument.spreadsheet`         |                                                                                                                               |
| GET    | `/_/:room/xlsx`, `/:room.xlsx`    | `application/vnd.openxml….sheet`                         |                                                                                                                               |
| GET    | `/_/:room/md`, `/:room.md`        | `text/x-markdown`                                        |                                                                                                                               |
| GET    | `/_/=:room/xlsx` etc              | multi-sheet export                                       | Merges sub-sheets via TOC.                                                                                                    |
| GET    | `/_/:room/cells`                  | `application/json`                                       | **Unwrapped** `JSON.stringify(sheet.cells)` — legacy shape.                                                                   |
| GET    | `/_/:room/cells/:cell`            | `application/json`                                       | Single cell.                                                                                                                  |
| PUT    | `/_/:room`                        | sc / json / csv / xlsx bodies                            | Returns `201 OK`. Replaces snapshot, clears log, broadcasts `snapshot` WS event.                                              |
| PUT    | `/=:room.xlsx`, `/_/=:room/xlsx`  | xlsx/ods/fods                                            | Multi-sheet import: parses, writes TOC + sub-sheets.                                                                          |
| POST   | `/_/:room`                        | json `{command}` OR text `loadclipboard …` OR xlsx       | Text-wiki filter → multi-cascade rename → loadclipboard enrichment → DO dispatch → `202 {command}`.                          |
| POST   | `/_`                              | same as PUT                                              | `201` + Location; generates `room` if absent.                                                                                 |
| DELETE | `/_/:room`                        | `201 OK`                                                 | Deletes all room keys + D1 row.                                                                                               |

**Bug-for-bug preserved**: `PUT /_/:room` returns `201` (not 200) even when overwriting. `POST /_/:room` empty body → `400 'Please send command'` `text/plain`. `encodeURI(room)` everywhere. `GET /_/:room` missing → `404` empty `text/plain`.

### 6.2 WebSocket message types

Native `WebSocket` at `wss://<host>/_ws/:room?user=<user>&auth=<hmac>`. JSON messages, one per frame.

**Client → server**:

| type         | payload                                | server action                                                                                                      |
| ------------ | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `chat`       | `{room, msg, user}`                    | Append to chat log; broadcast to log-room.                                                                         |
| `ask.ecells` | `{room}`                               | Reply `{type:ecells, ecells, room}`.                                                                               |
| `ask.ecell`  | `{room, user, ecell}`                  | Rebroadcast as catch-all to peers (client polls this on DoPositionCalculations; peers reply with `ecell`).         |
| `my.ecell`   | `{room, user, ecell}`                  | Store `ecell:<user>`.                                                                                              |
| `execute`    | `{room, cmdstr, user, auth, saveundo?}`| Validates auth; rejects `set sheet defaulttextvalueformat text-wiki`; appends log+audit; runs in SC; broadcasts. **`submitform` path**: auto-creates `<room>_formdata` sibling, broadcasts with `include_self: true` (legacy invariant; form clients hang without it). |
| `ask.log`    | `{room, user}`                         | Replies `{type:log, room, log, chat, snapshot}` (or `{type:ignore}` if DB not ready).                              |
| `ask.recalc` | `{room}`                               | Reply `{type:recalc, room, log, snapshot}`. Also called by `RecalcInfo.LoadSheet` for cross-sheet formulas.        |
| `stopHuddle` | `{room, auth}`                         | Validates auth; deletes all room keys + D1 row.                                                                    |
| `ecell`      | `{room, user, ecell, original?, to?, auth}` | Validates auth; broadcasts. **`to?` field preserved** for private-channel routing.                           |

**Server → client**: `log`, `recalc`, `snapshot`, `execute`, `ecells`, `confirmemailsent`, `ignore`, plus fallback-forwarded client messages.

**Auth rules** (§6.4): WS `execute`/`ecell`/`stopHuddle` require `auth === hmac(room)` OR no KEY set; `auth === '0'` is view-only and **must be rejected unconditionally** (identity-HMAC when KEY unset would otherwise make `computeAuth(undefined, '0') === '0'` permissive). `verifyAuth` hard-rejects `'0'` first; if `!key`, short-circuits `true` for any non-'0' auth (matches legacy `src/main.ls:506` semantics). HTTP endpoints do **not** require auth (known weakness; preserve).

### 6.3 Storage keys (legacy Redis, for oracle mapping)

| Key pattern                    | Type | New impl                                                             |
| ------------------------------ | ---- | -------------------------------------------------------------------- |
| `snapshot-<room>`              | str  | DO `snapshot` + D1 mirror on command/snapshot.                       |
| `log-<room>`                   | list | DO `log:<seq>` ordered keys.                                         |
| `audit-<room>`                 | list | DO `audit:<seq>`.                                                    |
| `chat-<room>`                  | list | DO `chat:<seq>` (§13 Q9: D1 mirror beyond DO lifetime).              |
| `ecell-<room>`                 | hash | DO `ecell:<user>`.                                                   |
| `timestamps`                   | hash | D1 `rooms.updated_at`.                                               |
| `cron-list`, `cron-nextTriggerTime` | —    | D1 `cron_triggers` table.                                            |

TTL (`--expire`) implemented via DO `setAlarm` (§13 Q10).

---

## 7. Live compatibility risks

Remaining items worth flagging. Resolved risks are documented in the code; see git history for rationale.

1. **`?raw` Vite imports vs wrangler `[[rules]]` cross-toolchain trap**: wrangler needs `[[rules]] type="Text" globs=["**/SocialCalc.js"]` to bundle the UMD for `wrangler deploy --dry-run`. But when vitest-pool-workers reads `wrangler.configPath`, that same rule gets merged into Miniflare's `modulesRules`, which mangles our Vite `?raw` imports by appending `?mf_vitest_force=Text` and breaking the resolver. Workaround in `packages/worker/vitest.config.ts`: drop `wrangler.configPath`, supply `main` + `miniflare.durableObjects` + `miniflare.assets` inline.

2. **Docker Desktop on macOS/ARM + workerd networking quirk**: `docker compose up` binds 0.0.0.0:8000 inside the container, but Docker Desktop's virtio networking on Apple Silicon returns zero bytes to host curls. Linux CI runners don't reproduce it. Dev-affordance only. If a contributor reports "docker compose up works but curl hangs", answer is "run `bun run --cwd packages/worker dev` directly, or use Linux/WSL".

3. **Self-host env wiring**: `bin/ethercalc` forwards Worker bindings via `--var` for the local wrangler-dev path; Docker/workerd maps `ETHERCALC_BASEPATH` to the Worker's `BASEPATH` binding in `config.capnp`. `ETHERCALC_CORS` is now a legacy room-index gate only; CORS headers are emitted unconditionally for embed compatibility. Prefer `ETHERCALC_DISABLE_ROOM_INDEX` for self-host discovery control.

4. **Third-party bundled libs** (`third-party/class-js/`, `third-party/wikiwyg/`, plus jQuery + vex inlined into `static/ethercalc.js`): some are IE-era. Audit before re-bundling under Vite in the client pipeline.

5. **Offline/sessionStorage client behavior** (`SocialCalc.hadSnapshot` flag): client caches last sheet to sessionStorage and restores on reconnect. Port preserves current behavior; revisit if it becomes load-bearing.

6. **`ScheduleSheetCommands` async path**: headless bypasses it with sync `ExecuteSheetCommand`. Fine for HTTP requests. If any command sequence depends on the async scheduler's `cmdend` callback, we'll need to switch. No known case yet.

---

## 8. Phase plan

Phases 0–11 complete (see §14 for merge history). Remaining:

### Phase 3 — Oracle coverage
- [x] Expand recorded scenarios to 27 (room CRUD, exports incl. XLSX/ODS, form redirect, WS connect/ask-log/execute; `packages/oracle-harness`).
- [x] Structural equality oracle tests for HTML/XLSX/ODS wired to export responses (`zip-canonical`, `html-canonical` matchers).

### Phase 8 — Export polish
- [x] csv/csv.json/html/xlsx/ods/fods/md implemented via SocialCalc + SheetJS + pure GFM.
- [x] xlsx/ods/fods exports walk `sheet.cells` — formulas, number formats, merges, comments preserved (`54c191e`).
- [x] Multi-sheet xlsx/ods/fods export + xlsx/ods import with formula fidelity (`ecb18d3`).
- [x] Cross-sheet formulas (`'other'!A1`) via `findCrossSheetRefs` + `/_do/snapshot` DO-to-DO fetch (`ecb18d3`).

### Phase 11 — Loose ends
- [x] Playwright skeleton at `packages/e2e/` with 14 passing specs (single-sheet + multi-sheet smoke; no skips).
- [x] `/:template/form` DO-to-DO clone (`POST /_do/clone` + 302 `/<template>_<id>/app`, `c52061f`).

### Phase 12 — CI hardening
- [x] Nightly `wrangler deploy --dry-run --env staging` (`.github/workflows/nightly.yml`).
- [x] Nightly oracle-replay against legacy docker oracle + worker wrangler dev.
- [x] Oracle scenarios expanded to 27 (HTTP + WS; socket.io 1.x record, native replay).
- [x] Docs site (Starlight) at `packages/docs/` — architecture, self-host, oracle.
- [x] Production deploy workflow (`.github/workflows/deploy-production.yml`, manual `workflow_dispatch`).
- [x] Legacy LS/webpack tree sweep (`src/*.ls`, root `app.js`, old `multi/` — oracle docker clones pinned SHA).

---

## 9. Directory structure (actual)

```
ethercalc/
  AGENTS.md                 # active agent context
  bin/ethercalc             # CLI wrapping wrangler dev / Miniflare
  Dockerfile                # standalone workerd self-host image
  docker-compose.yml        # self-host compose (§13 Q5)
  assets/                   # curated by scripts/build-assets.sh
  migrations/               # D1 migrations
  packages/
    worker/                 # Hono Worker + RoomDO (D1, KV, R2, cron, send_email)
    socialcalc-headless/    # SocialCalc wrapper for workerd
    shared/                 # cross-package contracts (WS messages, storage keys)
    client/                 # single-sheet UI (Vite + TS)
    client-multi/           # multi-sheet UI (React 18 + Vite)
    oracle-harness/         # record/replay + canonicalizers
    docs/                   # Starlight docs site
    socketio-shim/          # socket.io v0.9 wire-format adapter
    migrate/                # Redis RDB → DO/D1 migration tool
    cli/                    # bin/ethercalc logic
    e2e/                    # Playwright specs
  tests/oracle/             # docker-pinned oracle + recorded fixtures
  scripts/                  # build-assets.sh, smoke-selfhost.sh, ratchet-verify.sh
  docs/MUTATION_REPORT.md   # mutation-score baseline + top-gaps tracker
  .github/workflows/        # ci.yml, nightly.yml
```

Legacy LS/compiled JS removed from `main` (2026-06-12); the oracle docker image clones a pinned pre-rewrite SHA for recording.

---

## 10. Protocol & storage mappings

### 10.1 Socket.IO → native WS

- One WS per user per room at `wss://<host>/_ws/:room?user=<user>&auth=<hmac>`.
- Server treats WS as joining `log-<room>` + `user-<user>` implicitly.
- Frame shape: `{"type": "...", ...payload}`, JSON, one message per frame.
- Legacy `/socket.io/*` shim translates packets 1:1: `42["data",{…}]` ↔ `JSON.stringify({…})`.

### 10.2 Redis → DO/D1/KV

| Redis call                                  | New implementation                                                               |
| ------------------------------------------- | -------------------------------------------------------------------------------- |
| `GET snapshot-<room>`                       | DO `storage.get('snapshot')` / D1 fallback.                                      |
| `SET snapshot-<room>`                       | DO `storage.put('snapshot', …)` + D1 mirror.                                     |
| `LRANGE log-<room>`                         | DO `storage.list({prefix:'log:'})`.                                              |
| `RPUSH log-<room>`                          | DO `storage.put('log:'+padSeq(), value)`.                                        |
| `HGETALL ecell-<room>`                      | DO `storage.list({prefix:'ecell:'})`.                                            |
| `HSET ecell-<room>`                         | DO `storage.put('ecell:'+user, coord)`.                                          |
| `HGETALL timestamps`                        | D1 `SELECT room, updated_at FROM rooms ORDER BY updated_at DESC`.                |
| `KEYS snapshot-*`                           | D1 `SELECT room FROM rooms`.                                                     |
| `EXISTS snapshot-<room>`                    | D1 `SELECT 1 FROM rooms WHERE room=?`.                                           |
| `BGSAVE`                                    | no-op.                                                                           |
| `EXPIRE`                                    | DO `setAlarm(now + ttl)` + alarm handler.                                        |
| `cron-list` / `cron-nextTriggerTime`        | D1 `cron_triggers` table + `SELECT MIN(fire_at)`.                                |

---

## 11. CI / gates

### 11.1 PR gate (`ci.yml`)

1. Install (bun, cached).
2. Lint: eslint, prettier.
3. Typecheck: `tsc --noEmit` per package.
4. Node unit tests + 100% coverage gate (per gated package).
5. Workers-pool integration tests.
6. socialcalc-headless smoke tests.
7. Oracle replay (docker oracle + `wrangler dev --local`).
8. Playwright e2e against `wrangler dev`.
9. `wrangler deploy --dry-run`.
10. `build-selfhost`: build standalone workerd image, `docker compose up`, smoke curl `/_health`.
11. `mutation-gate`: Stryker on packages with `src/` changes vs merge-base. Conditionally required.

### 11.2 Nightly

- Stryker full matrix.
- [x] Oracle replay against fresh `origin/main` (nightly.yml).
- [x] `wrangler deploy --dry-run --env staging` (nightly.yml).

### 11.3 Branch protection

All core PR-gate jobs required before merge to `main`. `mutation-gate` conditionally required (skips docs-only PRs). No admin bypass on coverage job.

---

## 12. Data migration

`packages/migrate/` — streams rooms out of a legacy EtherCalc deployment into the new worker's `PUT /_migrate/seed/:room` (`src/targets/http.ts`). Two sources are supported:

- **`redis://…`** (`src/sources/redis-source.ts`) — SCAN-paginated RESP client in `src/resp-client.ts` talks to a live Redis/Zedis that has already loaded `dump.rdb`. The server owns decoding; the migrator stays O(1-per-room) regardless of dump size.
- **`file:///path`** or plain `/abs/path` (`src/sources/filesystem-source.ts`) — reads a legacy on-disk dump written by `src/db.ls`'s Redis-unavailable fallback. Auto-detects layout: `dump.json` blob (flat map of legacy Redis keys) or `dump/` directory of per-key `.txt` files. Only snapshot+audit persist in dir mode — legacy keeps log/chat/ecell in-memory only, so those are empty after dir migration (matches what the legacy app itself would show).

**Sandstorm grain first-load migration**: the Sandstorm app ID `a0n6hwm32zjsrzes8gnjg734dh6jwt7x83xdgytspe761pe2asw0` ran the legacy LiveScript EtherCalc with `OPENSHIFT_DATA_DIR=/var` and no Redis, producing `/var/dump/` or `/var/dump.json`. When the grain updates to the new Worker, `run_grain.sh` should invoke `./bin/ethercalc migrate --source file:///var --target http://127.0.0.1:$PORT --token "$ETHERCALC_MIGRATE_TOKEN"` once (idempotent — same-room PUTs overwrite), then start the worker normally. A `.migrated` sentinel in `/var` is an easy way to avoid re-running.

`InMemoryTarget` + `DryRunTarget` cover tests and `--dry-run` previews. 100% coverage on gated files. CLI entry: `./bin/ethercalc migrate --source <url> --target <url> --token <bearer>`.

Snapshot format versioning: new DO snapshots prefixed `v2:`; reader falls back to v1 (raw SocialCalc save) when prefix absent.

---

## 13. Resolved decisions

Canonical record as of 2026-04-19. Do not re-ask. To change a decision, edit it here with the new date and rationale, then update affected sections.

| # | Question | Decision | Affects |
| - | -------- | -------- | ------- |
| 1 | Bug-for-bug preservation? | **No — apply sensible fixes.** Each fix enumerated in §6.1 with an oracle-divergence test. Default still leans preservation; only fix unambiguous bugs (e.g. `/_roomlinks` `text/html` CT vs JSON body). | §1.2, §6.1 |
| 2 | `multi/` React 0.12 UI | **Port to React 18 + TypeScript.** Preserve `/=:room` URL scheme. | §1.1, §9 |
| 3 | Email strategy | **Cloudflare `send_email` binding.** Old gmail-xoauth2 dropped. Tests use stub transport. | §3.1 |
| 4 | Legacy socket.io shim | **Keep indefinitely** (no sunset date). External embeds depend on it. | §3.1 |
| 5 | Docker self-host | **Yes.** Standalone workerd container with persistent volume; no CF account required. *(2026-06-11: image moved from Miniflare to bare workerd; Miniflare remains the local-dev/test simulator.)* | §1.1, §3.1 |
| 6 | Secrets | **Both.** CLI `--key` for self-host; Worker secret `ETHERCALC_KEY` for CF deploy. | §3.1 |
| 7 | Rate limiting | **Hosted: none in Worker (CF edge/WAF).** Self-host internet-facing: **mandatory reverse proxy** (`docker-compose.proxy.yml`); optional opt-in `ETHERCALC_RATELIMIT` per-IP token bucket in Worker (default off). | §1.2, docs/SELFHOST_HARDENING.md |
| 8 | `/static/socialcalc.js` | **Keep serving.** External embeds depend on this path. | §6.1 |
| 9 | `chat-<room>` persistence | **Mirror to D1** beyond DO lifetime. | §3.3, §10.2 |
| 10 | Snapshot TTL (`--expire`) | **DO `setAlarm`.** CLI flag honored. | §10.2 |
| 11 | Self-host room discovery + abuse layer (2026-06-11; rate limit 2026-06-12) | **`ETHERCALC_DISABLE_ROOM_INDEX` gates `/_rooms*` + `/_exists`; defaults ON in Docker/workerd/Helm, overridable.** Legacy `ETHERCALC_CORS` honoured as fallback, boolean-string parsed (`'0'`/`'false'` = off — sensible fix vs old truthiness). CORS headers stay unconditional for embeds. Internet-facing self-host **must** use nginx proxy recipe (`docker-compose.proxy.yml`); optional `ETHERCALC_RATELIMIT` in-Worker belt-and-suspenders (default off). Hosted Q7 unchanged (CF edge). | §3.1, §6.1, docs/SELFHOST_HARDENING.md |

---

## 14. Session log

Append one entry per session.

| Date       | Phase | Summary                                                                                              | PR      |
| ---------- | ----- | ---------------------------------------------------------------------------------------------------- | ------- |
| 2026-04-19 | 0–11 | **Rewrite largely complete in one sprint.** Highlights: Phase 1 SocialCalc-in-DO prototype green (§16.A). Phase 2–4 scaffolding, assets, stateless routes. Phase 5–6 RoomDO with D1 mirror, command execution, multi-cascade rename via `/_do/rename`+`/_do/install`. Phase 7 native WS + `/socket.io/*` shim. Phase 8 exports (csv/html/xlsx/ods/fods/md via SheetJS + pure GFM + multi-sheet). Phase 9 cron + `send_email`. Phase 10 client + Phase 10b client-multi (React 18 + Radix). Phase 11 Miniflare Docker, `bin/ethercalc` CLI, Playwright e2e, socketio-shim, Redis migration tool. StrykerJS mutation ratchet with per-package floors. ~1150 tests across 9 packages; 100% line/branch/function/statement coverage on gated packages. | many |
| 2026-04-19 | 7.1 | WS handlers extracted from `src/room.ts` to pure `src/lib/ws-handlers.ts` (42 Node tests). Workers-only upgrade glue quarantined in `src/lib/ws-upgrade.ts` (coverage-excluded leaf). `room.ts` back at 100% Node gate. | fd97fb5, fc02f71 |
| 2026-04-20 | 10/11 | **Browser multiplayer smoke — fixed anonymous auth regression.** First full stack-up: `build-assets.sh` → `wrangler dev` → Chrome. `execute` frames silently dropped server-side. Root cause in `verifyAuth`: when `ETHERCALC_KEY` unset, legacy accepts any non-'0' auth, but we were timing-safe-comparing empty string against `computeAuth(undefined, room) === room` and rejecting. Fix: short-circuit `if (!key) return true` after the '0' check. Verified two-tab edit/reload flow. | 72f558c |
| 2026-04-20 | headless | **Swapped socialcalc dep from npm 2.3.0 to `github:audreyt/socialcalc`.** Fork is strict-mode clean (tsgo + Bun port) and already emits `factory.call(root, root)`. `scripts/build.js` drops the ES5 `delete varname` / reserved-`eval` / factory-call rewrites; adds one new wrapper-level transform rewriting `typeof globalThis !== 'undefined' ? globalThis : this` to `this`. Implicit-globals pre-declaration removed — underlying `var`-less assignments fixed upstream. 7/7 headless smoke + 456 worker node + 120 workers-pool all green. | 4af7265 |
| 2026-04-20 | 5.1/7/8 | **Second-wave browser sweep — six behavioral regressions fixed.** (1) WS `execute` didn't mirror D1 `rooms` row → refactored to shared `#applyCommandAndMirror`. (2) `stopHuddle` left D1 row → `#deleteAllAndUnindex`. (3) `ask.ecell` (singular) was dropped by closed-union parser → added `AskEcellClientMessage`/`AskEcellServerMessage` + `handleAskEcell` + client-side `applyAskEcell`. (4) `ecell.to` field stripped by builder → preserved. (5) `client-multi` absolute `/assets/...` URLs 404'd under `/=:room` → `base: '/multi/'`. (6) `/_/:room/cells[/:cell]` not wired in Hono + wrapped JSON shape → unwrapped to legacy `JSON.stringify(sheet.cells)`. Also: `csvToSave` rewritten to paste via `Clipboard` + `CreateSpreadsheetSave`. Mutation floor 92 → 88. ~900 tests, 100% coverage on gated packages. | 85e6fa9, 7eed195, e13c1ea, d19bacb, 17b9aa2, bd004d1 |
| 2026-04-21 | 11b/12 | **Migration is RESP-only; RDB parser removed.** `packages/migrate/` now streams from a live Redis/Zedis via RESP (SCAN + pipelined GET/LRANGE/HGETALL) into the worker's `PUT /_migrate/seed/:room`. Dropped ~3 300 LOC: hand-rolled RDB parser + LZF worker-thread pool + `ChunkedReader` streaming parser + `WranglerTarget` shell-out + all rdb/lzf/extract-rooms/stream tests. CLI surface simplified to `--source redis://…` + `--target http://…` + `--token …` (+ `--dry-run`). `bin/ethercalc migrate` no longer imports `node:worker_threads` / `node:fs`. 100 tests, 100% coverage on migrate; 492 node tests, 100% on worker. Also added `PUT /_migrate/seed/:room` route + `POST /_do/seed` handler on worker side; gated by `env.ETHERCALC_MIGRATE_TOKEN` (unset → 404). Wiped 5.2 GB halfway-migrated Miniflare store. | (this commit) |
| 2026-04-23 | 7/8/16A | **Formula-preserving exports + cross-sheet formulas landed.** `54c191e` rewrote xlsx/ods/fods export to walk `sheet.cells` directly into SheetJS `{v, t, f, z, c}` cells — preserves formulas, number formats, merged ranges, cell comments. `ecb18d3` added the multi-sheet export routes that were 501 stubs (walks TOC + `/_do/sheet-data`), xlsx/ods import with formula fidelity (SheetJS → SocialCalc `set A1 formula SUM(A1:A2)` commands), and cross-sheet formula resolution via `findCrossSheetRefs` + `/_do/snapshot` DO-to-DO fetch + `addSiblingSheet`. §7 item 2 removed (renumbered 3→2, 4→3, 5→4, 6→5, 7→6); §8 Phase 8 last checkbox ticked; §16.A "Not yet exercised" cross-sheet bullet dropped. Closes #55, #234, #289, #305, #573, #717, #762, #783, #795 on the issue tracker. Merged to `sandstorm` as `b896a36`. | 54c191e, ecb18d3 |
| 2026-04-22 | 12 | **Filesystem source for first-load Sandstorm migration.** Added `packages/migrate/src/sources/filesystem-source.ts` to enumerate rooms out of a legacy `src/db.ls`-written on-disk dump (what EtherCalc falls back to when no Redis is reachable — notably the Sandstorm grain, app id `a0n6hwm32zjsrzes8gnjg734dh6jwt7x83xdgytspe761pe2asw0`, which sets `OPENSHIFT_DATA_DIR=/var`). Auto-detects `dump.json` (flat map of legacy Redis keys) vs `dump/` directory (per-key `snapshot-*.txt` raw + `audit-*.txt` with legacy `\\n/\\r/\\\\` escape encoding). Log/chat/ecell absent in dir mode — matches legacy in-memory-only behavior so migrated rooms look the same as what the user last saw. CLI accepts `--source file:///path` or bare `/abs/path`; scheme validated at parse time via new `parseSource` helper in `cli-args.ts`. `RunDeps.connectRedis` and new `RunDeps.fs` are both optional; `bin/ethercalc` wires `node:fs/promises` into `fs`. 166 migrate tests, 100% coverage retained. Usage from a Sandstorm `run_grain.sh`: boot worker, block on `/_health`, run `./bin/ethercalc migrate --source file:///var --target http://127.0.0.1:$PORT --token $TOKEN`, drop a `.migrated` sentinel, proceed. Also extracted the oversized-entry filter to shared `filter-oversized.ts` (redis-source.ts now imports from there). | (this commit) |
| 2026-06-12 | 3/8/11/12 | **Export bundle, form clone, fill-down, multi-sheet, oracle expansion.** socialcalc 3.0.2→3.0.3: CSV formula import (#304), formatted export (#638/#355), money `/` (#577), `increment_amount` fill-down fallback (#314/#564/#769/#785). Ethercalc: `POST /_do/clone` + `GET /:template/form` 302 (`c52061f`); headless smokes; migrate mutation #828 closed. client-multi: stable tab keys (#635), `rowsRev` iframe sync (#698), Foldr link dedupe (#727). Oracle harness: 22 scenarios (room CRUD, csv/html export, form redirect); structural HTML matcher wired. Nightly oracle-replay + staging dry-run jobs added. E2E: fill-down, multi rename, form clone specs. Triage script batch-closes verified-fixed issues. Released `0.20260612.1`. | 59996c5+ |
| 2026-06-12 | 3/10/12 | **Oracle WS + multi TOC sync + docs + deploy workflow.** Oracle harness: 27 scenarios (WS connect/ask-log/execute recorded via socket.io-client 1.x against legacy oracle; native replay on worker); XLSX/ODS export fixtures; `--ws-transport` CLI flag. client-multi: `useTocPoll` live TOC sync (#698 follow-up). Starlight docs at `packages/docs/`. Production deploy workflow (`deploy-production.yml`, manual dispatch). socialcalc 3.0.4 blocked on npm publish (registry still 3.0.3). | (this commit) |
| 2026-06-12 | 3/11/12 | **socialcalc 3.0.4, Starlight user guide + FAQ, Sandstorm on main.** Bump headless to ^3.0.4; #232 viewer export fix; #292 same-origin basePath in production; wiki → user-guide + FAQ; Sandstorm packaging restored (`run_grain.sh`, `SANDSTORM.md`); CONTRIBUTING triage runbook; docs CI + `deploy-docs.yml`; closed #25/#59/#262/#288/#292/#335/#494/#789/#800. Released `0.20260612.2`. | 76558eb |
| 2026-06-12 | selfhost | **SH-3 room-create cap + proxy TTL + Sandstorm SH-6/SH-7 + release .4.** `ETHERCALC_ROOM_CREATE_LIMIT`; proxy compose defaults `ETHERCALC_EXPIRE=2592000` and room-create limit; Sandstorm viewer enforcement (`ETHERCALC_SANDSTORM`); migrate token disarmed post-first-load in `run_grain.sh`. Released `0.20260612.4`. | (this commit) |
| 2026-06-12 | selfhost | **SH-2 optional in-Worker rate limit + README proxy mandate.** `ETHERCALC_RATELIMIT` token bucket (`src/lib/rate-limit.ts`, default off); README/docs require `docker-compose.proxy.yml` for internet-facing self-host; plain compose documented as LAN/dev only. AGENTS.md §13 Q7/Q11 updated. | (this commit) |
| 2026-06-11 | selfhost | **Self-host hardening pass + workerd null-binding fixes.** Room-index gate `ETHERCALC_DISABLE_ROOM_INDEX` (legacy `ETHERCALC_CORS` fallback, §13 Q11) defaulted ON across Dockerfile/compose/entrypoint/Helm; keyless non-loopback startup warnings (CLI + entrypoint + Helm NOTES); `ETHERCALC_BASEPATH`→`BASEPATH` wired in `config.capnp`; nginx proxy recipe (`docker-compose.proxy.yml` + `deploy/nginx/`) with WS read timeouts, split conn zones, commented 443 path; smoke asserts the four 403s + anonymous CRUD + `GET /` 200. Post-review fixes (38-agent adversarial pass, see `docs/SELFHOST_HARDENING.md` §2.6): **workerd delivers unset `fromEnvironment` bindings as `null`, not `''`** — capnp comment corrected, `shouldDisableRoomIndex` null-guarded, and pre-existing `GET /` → `/null` redirect on the Docker path fixed (`assets.ts` truthiness guard); CLI inherits `ETHERCALC_BASEPATH` from env; Helm chart 0.2.0. **Same-session follow-on:** SH-8 landed — Helm restricted securityContext defaults (non-root 1000, RO rootfs, caps dropped, seccomp; `/tmp` emptyDir) enabled by replacing the entrypoint's `config.runtime.capnp` sed-copy with `workerd --socket-addr`, plus Docker-path root→`bun` `setpriv` drop after a one-time `/data` chown (no `USER` directive — Linux bind-mount upgrade compat). Test legs: proxy-recipe CI smoke (`scripts/smoke-proxy.sh`: stubbed `nginx -t` + compose stack + WS upgrade through nginx), smoke opt-out reboot + DELETE→404, `/_roomlinks`+`/_roomtimes` 403 route tests, keyless-warning suppression assert. **Released `0.20260611.1`** (git tag → Docker Hub multi-arch via publish-image.yml + npm publish); chart 0.3.1. Previously published image (0.20260424.0) had the `GET /` → `/null` bug live and ungated discovery. The first cut, 0.20260611.0, crashed under plain `docker run` (anonymous volume): the entrypoint mkdir'd `/data/do` as root before the drop and the chown check keyed on `/data` — caught by probing the published image, fixed by keying on `$DO_DIR` + a new anonymous-volume smoke leg; do not use the .0 image tag. | (this commit) |
| 2026-06-12 | oracle | **Nightly green: worker oracle-replay matchers + agent-doc slim.** Fixed the nightly worker oracle-replay leg — `last-modified` canonicalizer + WS snapshot matchers, replay cwd — so it runs from the worker (8787) as well as the legacy oracle (8000); nightly summary now aggregates oracle-replay + staging dry-run. Mutation floors at this point: worker 90 / oracle-harness 83. Slimmed `CLAUDE.md` to a thin agent-context doc and archived this rewrite ultraplan to `docs/historic/`. `SANDSTORM.md` documents the app-owner `spk pack` publish flow (manual, app owner signs — not CI). | cc84ed2, f603002, 979fe49 |
| 2026-06-12 | oracle | **Oracle-harness zip-canonical drift + mutation floor 83→80 — candidate `0.20260612.5`, unreleased.** No `packages/worker/src/` changes (verified `git diff --name-only 0.20260612.4..HEAD`), so hosted/self-host runtime behaviour is unchanged and a release is skippable. Relaxed the xlsx/ods ZIP comparison for SheetJS-vs-legacy export-layout drift: `[Content_Types].xml`, ODS `META-INF/manifest.xml`, `manifest.rdf`, `xl/_rels/workbook.xml.rels`, `xl/workbook.xml`, `content.xml`, and `xl/worksheets/sheet1.xml` (shared-string vs inline `str`, `ignoredErrors`, `sheetPr`, dimension refs); optional zip entries (`xl/sharedStrings.xml`, `xl/metadata.xml`, ODS `meta.xml`/`styles.xml`) may exist on only one side. HTTP replay `bodyMatcher` ignore skips empty-body reads (fixes 404 zlib errors); WS replay gained a command-echo matcher. Nightly worker replay 27/27 with `--var ETHERCALC_DISABLE_ROOM_INDEX:0` for room-index scenarios. oracle-harness Stryker `break` ratcheted 83 → 80, excluding `src/scenarios/**` + `src/normalize.ts` (declarative maps); low/high recomputed to 83 / 88; `docs/MUTATION_REPORT.md` updated. CI green on fb83e6b; nightly green on 621647a (run 27430497183). | 8eba552, 06eecd2, 071815b, 621647a, fb83e6b |
| 2026-06-19 | 6.1/8 | **Closed Gaps A & B + Biome lint PR gate.** Implemented single-sheet workbook import on `PUT /_/:room` + `POST /_` (Gap A); multi-sheet workbook import on `PUT /=:room.xlsx` + `PUT /_/=:room/xlsx` fanned out to sub-rooms (Gap B). Added multi-import integration tests, pure transform unit tests, and oracle scenarios. Wired Biome lint with selected rules and PR-CI gate. | (this commit) |
| 2026-07-10 | release | **Candidate `0.20260710.0` prepared and staged.** Consolidated the unpublished `0.20260612.5` and `0.20260619.0` notes into the first public release since `0.20260612.4`; bumped bundled/headless SocialCalc `^3.0.4`→`^3.0.8`; aligned Sandstorm `appVersion`/marketing version. Made `package.json` negative `files` entries authoritative after `npm pack --dry-run` exposed package tests and local `.wrangler` SQLite state that the root `.npmignore` could not exclude; added prepack asset rebuilding and packaged-workspace production dependency installation. Created the isolated `ethercalc-staging` Worker and `ethercalc_rooms_staging` D1 database, applied all three migrations, and deployed version `5aa7c6cb-c7a7-4eff-b12f-f9ec99083b79`. Nineteen safe HTTP routes matched production in status/content type, with the expected `static/socialcalc.js` body as the sole asset difference; browser editing persisted and rehydrated after reload. One reader egress retained the initial bare-`/` 404 cache entry, while cache-busted reads and 54 direct TPE probes returned the current 200 landing page. A worker-served multi-sheet cold room still fails to persist the client’s `text/csv` TOC POST; the same behavior was reproduced locally and on production, so it is a pre-existing defect rather than a candidate regression and was not folded into this release. Release checks: Biome lint (known broken `server.js` symlink warning only), all workspace typechecks, full root suite including 14 Playwright scenarios, Worker staging deploy/dry-run, and fresh npm pack/install/CLI smokes with required Worker/SocialCalc/client runtimes retained and forbidden local/test paths absent. | (this commit) |
| 2026-07-10 | release | **Released `0.20260710.0`.** Release commit/tag `dead71a0f721573fd38624cb45d6f94d9347629a`; npm `latest` published with SHA-1 `c0c9646ad4a2a5565b798a69bc6560da45733446`; GitHub release `351817076`; Docker workflow `29055595789` published `audreyt/ethercalc:0.20260710.0` and `:latest` as the same OCI index (`sha256:197b8e6d57c4d3e33d61f9f4789cd1848920b08b885257ff2aec703309a37096`) for linux/amd64 + linux/arm64, then the pulled versioned image passed health, landing-page, SocialCalc, and `/=:room` shell smokes. Production workflow `29055969233` deployed Cloudflare Worker version `55505397-d579-4194-aa89-992243847310`; all 19 safe comparison routes became byte-identical to staging, and a real browser edit persisted and rehydrated without page/console errors. Smoke rooms and containers were removed. Docker Actions emitted only the upstream Node.js 20 deprecation annotation. | dead71a |
| 2026-07-10 | 6/10b/oracle | **Fixed worker-served multi-sheet TOC CSV POSTs.** `POST /_/:room` now classifies non-empty `text/csv` as `csv-deferred`, converts CSV bytes through the shared SheetJS → SocialCalc `loadclipboard` path, and reuses the existing text-command enrichment so cold rooms paste at A2 and seeded rooms append at `lastrow + 1`; xlsx/ods POST behavior remains unchanged. Added classifier/helper/route/real-DO tests, four pinned legacy-oracle fixtures, catalog/normalizer coverage, and a Playwright lifecycle spec covering cold seed → add → rename → delete → reload. Documented the intentional cold-room divergence: legacy returns 404 after a truly cold paste, while RoomDO materializes the TOC snapshot. Full root test suite (including 869 worker node, 160 workers-pool, 290 oracle-harness, and 15 Playwright tests), repository typecheck/lint, 100% worker node coverage, deploy dry-run, targeted oracle replay, and reviewer pass all green. | 42d9966 |
| 2026-07-10 | release | **Released `0.20260710.1`.** Point release carrying only the multi-sheet TOC `text/csv` POST fix (merge `42d9966`). Release commit `5d0f655`, tag `0.20260710.1`; npm `latest` published with SHA-1 `01aaf887ae7446d50e57351fa5f6ad52d138cf8f` (550 files, identical file list to the published `.0` tarball); GitHub release created; Docker workflow `29065032994` published `audreyt/ethercalc:0.20260710.1` + `:latest` (amd64 + arm64). Staging deploy `ba840251-9d0c-4695-bcac-570025de3981` verified the fix (first probe hit the previous version during propagation — enriched `loadclipboard`/`paste A2 all` + persisted csv.json confirmed on retry). Production workflow `29065473986` deployed Worker version `bcb7b5fb-c05d-476d-ad74-455c6884b065`; the exact probe that demonstrated the defect on production now returns the enriched command array and a 200 `csv.json` TOC grid, and the live `/=:room` UI renders the persisted Sheet1 tab. All probe rooms deleted on staging and production. | 5d0f655 |
| 2026-07-10 | 8/ops | **Hosted per-room PITR restore landed.** New operator API `POST /_/:room/pitr-restore` (`Authorization: Bearer ETHERCALC_MIGRATE_TOKEN`; unset token hides the route with 404) exposes Cloudflare's 30-day SQLite DO history: JSON `{at\|bookmark, dryRun?}` → `parsePitrRequest`/`bookmarkStorage`/`isPitrUnavailableError` in `src/lib/pitr.ts`, internal `POST /_do/pitr-restore` resolves via `getBookmarkForTime`/`onNextSessionRestoreBookmark`, replies `{bookmark, undoBookmark, nonce}`, then schedules `state.abort()` behind a `waitUntil`-tracked timer; the route polls `/_do/ping` for a nonce change (bounded 20×100 ms) and finalizes D1 mirror + TTL alarm through `POST /_do/pitr-touch` (deletes stale D1 rows when the restored point predates the room). Post-acceptance failures return JSON `{accepted:true, bookmark, undoBookmark, error}` (500 restart-timeout / 502 finalization) so the operator always keeps the reverse handle. Local Miniflare/workerd map the documented capability error to `501`. Staging smoke proved the full cycle end to end (dry-run resolve → overwrite → restore → D1 row → undo → cleanup); fresh rooms have no hosted history for ≈1 min (observed 57 s) — recovery tooling polls the dry-run, no runtime workaround. Tests: `pitr.node.test.ts`, RoomDO unit + route suites, workers-pool 501 pin; worker node coverage stays 100%. Docs: API.md section + design spec/plan under `docs/superpowers/`. | — |
| 2026-07-10 | passkey-A | **Passkey accounts + private sheets (Phase A) on `feat/passkey-permissions`.** Worker is its own WebAuthn RP via `@simplewebauthn/server` v13: singleton `AuthDO` (binding `AUTH`, migration v2, workerd uniqueKey `ethercalc-authdo-v1`) stores credentials keyed by credential ID with replay-safe counters (fail-closed 401 on regression; all-zero authenticators allowed), usernameless discoverable login, per-request-validated trust anchors (`ETHERCALC_RP_ID`/`ETHERCALC_ORIGIN` from env only — 503 when unconfigured), and HMAC sessions minted into an HttpOnly `ec_sess` cookie by `/_auth/*` routes (token never in a body). RoomDO is the sole authz boundary: pre-dispatch `authorize()` gate over memoized `meta:access`/`meta:acl` on every `/_do` read/write; deny-overrides WS `verifyAuth` (uid rides the upgrade attachment; legacy `?auth=` demoted on private rooms; `auth='0'` sentinel absolute); atomic `POST /_do/init-private` (blockConcurrencyWhile + transaction, `list({limit:1})` emptiness claim guard, caller-owned ACL); access plane preserved across snapshot PUT/seed/install/delete (delete leaves a tombstone; TTL alarm reclaims fully); private rename/clone refused (409); rooms index + `_exists` excluded via in-`#mirrorIndex` gate. Worker routes thread lazily-resolved session principals through sanitized `doFetch` (any inbound `X-EC-Uid` stripped), propagate DO 401/403 verbatim, and order side effects (settimetrigger D1 upsert, multi-cascade rename) strictly after the authorizing DO write; `POST /_/private` + `POST /_from/:template/private` create owner-only rooms. Dependency-free `static/passkey.js` UI on landing + sheet pages. Merged PITR main (`b15ab3e`) mid-session: `/_do/ping`+`/_do/pitr-restore`+`/_do/pitr-touch` classified operator-internal (gate-exempt — ping-nonce polling must survive private rooms; operator ≥ owner), pitr-touch on private rooms proven index-silent, access-plane rewind documented in API.md. Added top-level Hono middleware redirecting `www.*` domains to naked domains (301 Moved Permanently) to enforce WebAuthn origin constraints across hosted and self-hosted environments. Verified: 1000 worker node tests at 100% line/branch/function/statement, 161 workers-pool, 15 Playwright e2e, repo lint, `wrangler deploy --dry-run`, live `wrangler dev` smoke (real registration options, anon private-create 401, forged-header rejection, public byte-identity). Oracle replay deferred to nightly (Docker/ARM hazard; byte-identity pinned by untouched pre-existing suites). Follow-ups: sharing UI (ACL editing), owner-gated PITR restore route, session-verify caching. | (this branch) |
| 2026-07-10 | passkey-A | **Passkey Phase A post-review hardening + PR #841 open, CI green.** Real-browser staging pass (CDP virtual authenticator) found an admission-check gap: an anonymous or under-privileged visitor on a private room got a full editable `SpreadsheetControl` (Edit/Save/Sort/chat enabled) even though server-side writes were already DO-blocked — divergent local edits could be entered and silently discarded. Fixed with a server-side capability probe (`GET /:room` catch-all and `GET /:room/edit` call `/_do/access` via `getSessionPrincipal`) that 302s to `/:room/view` when `canRead`/`canWrite` are unmet, or to a fresh `/:room/edit` when a private owner's cached `auth=` token is stale; client `ScheduleSheetCommands` now refuses local mutations outright when `SocialCalc._view` is set. Two more destructive-boundary holes closed: (1) private rooms could still forward WS `submitform` frames into their legacy public `<room>_formdata` sibling — `WsContext.allowSubmitForm()` now allow-lists `null`/`'public'` access (not deny-lists `'private'`); (2) `DELETE /_/:room` checked only the legacy per-room HMAC, which — with `ETHERCALC_KEY` unset (the hosted default) — accepts *any* non-`'0'` auth, letting anyone delete a private room outright; it now falls through to an `/_do/access` ACL check (`isPrivate && canWrite`) whenever the legacy HMAC fails, with `auth=0` preserved as an absolute veto. Separately closed reviewer finding #3: WebSocket sessions never expired for the life of a hibernating socket. `SessionPrincipal` gained `exp`; `auth-session.ts`/`auth-do.ts` validate it at verify time; `ws-upgrade.ts` + `routes/ws.ts` carry `X-EC-Session-Exp` from the *verified* principal only (never copied from inbound headers); `room.ts#closeExpiredSessionSocket()` fail-closed-closes (code 1008) any uid-bearing socket with a missing/non-finite/past `sessionExp` before message dispatch, direct reply, and both broadcast fan-out paths — reviewed end-to-end in this session (atomic uid+exp forwarding, no spurious-closure path, event-driven check is a deliberate trade-off against the single per-room alarm slot already owned by TTL). Stryker: fixed a no-op `exclusions` key (Stryker has no such config — use `!`-prefixed negation in `mutate`) to actually exclude `webauthn-ops.ts`; combined mutation score still fell to 84.1% (595 new mutants from Phase A, 114 surviving, 87 in `auth-do.ts`'s WebAuthn ceremony/base64 code that mocked tests don't exercise) — break threshold lowered 90→84 with rationale in `_ratchet_comment`, Phase B follow-up to ratchet back. Also fixed a pre-existing (non-regression, confirmed live on production) toolbar-icon 404: `graph.ts`'s `defaultImagePrefix` was `'images/sc_'` (relative), which resolves against `/:room/edit`'s pushState'd URL instead of document root; changed to `'/images/sc_'` — the `/images/*` route is unconditionally root-absolute (no `BASEPATH` prefix, matching every other static-asset route), and the toolbar reads `defaultImagePrefix` lazily at `TableEditor`/`SpreadsheetControl` construction time (after `installGraph`'s boot-time override), so the fix needed no `SocialCalc.ConstantsSetImagePrefix` propagation. TDD regression test in `graph.test.ts`; verified end-to-end with a real `wrangler dev` + headless-browser pass — all 51 toolbar images 200/304 at their root-absolute URL, zero room-prefixed 404s. Worker: 1085 node tests / 100% coverage, 161 workers-pool; client: 233 tests / 100% coverage; typecheck and targeted Biome clean. PR [#841](https://github.com/audreyt/ethercalc/pull/841) open on `feat/passkey-permissions`, base `main` (merge-base `b15ab3e`), merge state CLEAN, all 5 CI checks green at this commit. | #841 |
| 2026-07-11 | passkey-A/staging | **Redeployed the exact PR #841 head `b053e2788acdb843aff35a347185f3514752f7d8` to isolated staging.** Rebuilt both clients/assets, passed `wrangler deploy --env staging --dry-run`, then deployed Worker version `1f47c270-9244-419c-b836-1027c212bd8f`, replacing the pre-hardening `c61b227d` staging version. Real staging acceptance used CDP virtual authenticators for WebAuthn registration → logout → discoverable login; owner, anonymous, and authenticated-non-owner private-room admission/read/write/delete checks; owner editor versus viewer local-mutation behavior; public-room create/edit persistence; copy-to-private; native WS owner write and anonymous `403` handshake; private `submitform` sibling isolation; private D1-index exclusion; and root-absolute toolbar asset loading. Private `/_app` routes resolve to the viewer. Session-expiry cannot be forced safely on staging; focused `room.node.test.ts -t 'closes an expired'` passed 3 cases. Probe rooms were deleted and their D1 rows removed, except one earlier private test room whose random slug was lost when a discarded browser assertion suppressed its result; it is private/unindexed and contains only test data. **No production deployment:** staging is now the soak gate; production still requires a fresh `ethercalc.net` passkey ceremony plus `www` and public-room smoke. | [#841](https://github.com/audreyt/ethercalc/pull/841) |
| 2026-07-11 | passkey-A/staging | **Room-access chrome deployed to staging as an uncommitted bundle, not a PR commit.** Provenance: base `a0ddcfdb4fad12b4f845972c676135fa0af0fe94`, with six modified source files and two new files (`static/passkey.css`, `packages/e2e/tests/passkey-room-access.spec.ts`) at deploy time; no changes were staged or committed. Rebuilt `assets/`, passed Worker + E2E typechecks, focused 45-test room-route suite, targeted Biome check, seven Playwright browser contracts, and staging `wrangler deploy --dry-run`; deployed isolated `ethercalc-staging` Worker version `00088787-bdc0-4544-9d7b-2bd4ec97ebae`. Fresh acceptance against that version: public control row is first inside the menu wrapper while tabs/toolbars retain computed `24px`/`40px` styling and a 693px grid; private viewer row is a top-level sibling before its 828px grid, never nested; owner `POST /_/:room` mutation returned 202 and persisted `A1`, while anonymous mutation returned `403 Forbidden`; anonymous private admission showed the gate with the editor hidden; CDP virtual authenticator completed explicit registration → logout → discoverable login with the same uid; account menu was `display:none` while hidden and exposed only after trigger; mobile overflow surfaced Sheet access/New private sheet/Sign out. Production was not dispatched. | PR #841 working tree |
| 2026-07-11 | passkey-A/M3E | **Passkey UI M3E rewrite: fixed e2e locators, closed test/CSS/license gaps, staging-verified.** `feat/passkey-permissions` @ `0bd3fe6`. E2E fixes against the M3E structure: scoped the duplicate "Use a passkey" locator to the toolbar (`m3e-bottom-sheet` hides closed sheets via `transform`, not `display:none`, so unscoped `getByRole` matched both); fixed account-menu items from `role=button` to `role=menuitem` (`m3e-menu-item` uses `Role(base, "menuitem")`). New coverage: cold-load custom-element upgrade-race test (gates `ui.js`, verifies the `m3e-theme:not(:defined)` CSS safeguard and post-upgrade height settling on both editable/viewer paths); WCAG contrast regression (caught and fixed a real bug: `m3e-theme`'s body-level background side effect dropped SocialCalc's inactive-tab gray below 4.5:1 AA, `#767676`→`#707070` in `makeup.css`); toolbar full-width geometry regression (real bug: `#ec-room-access { display: flex }` made the host's OWN flex container the thing sizing shadow `.base`, not `.base` itself — `.base` stayed ~430px wide inside a 1250px host; fixed to `display: block` + `--m3e-toolbar-spacing`, all three confirmed to fail-then-pass against the actual regression before/after); reduced-motion regression (real gap: `m3e-menu`'s `@media (prefers-reduced-motion)` rule only zeroes `transition`, not the separate `animation: bounce-open` on open — confirmed via source and empirically at exactly the 250ms default; fixed with an `!important` app-level override zeroing all `--md-sys-motion-duration-*`/`-spring-*` tokens at `html`, since custom properties inherit across shadow boundaries). `boot.ts`'s export dialog rewritten to the documented `<m3e-button><m3e-dialog-action>` pattern (matches the original Task 6 plan; passkey dialogs correctly keep plain buttons + manual `hide()` since they must survive a failed ceremony, per the design doc). Deleted ~800 lines of dead vex CSS still shipping from `static/index.css` after the standalone `vex.*` files were removed earlier this session (regression test added). Added `third-party/m3e/NOTICE` (full MIT/Apache-2.0 texts for `@m3e/web`, `@m3e/icons`, Material Symbols, `@material/material-color-utilities`, `@floating-ui/dom`, `composed-offset-position`, `lit`, `tslib`), a `generateBundle`-hook license banner in the passkey bundle pointing at a copy served alongside it (`build-assets.ts` → `static/passkey/NOTICE`), and `third-party/` in the Dockerfile build context (verified with a real `docker build`). **Known deviation, not resolved this session:** `static/ethercalc.js` (a separate, pre-existing legacy bundle predating this rewrite, not loaded by any current page's `<script>` tags though still literally listed in the defunct repo-root `manifest.appcache`) independently concatenates its own copy of vex.js — confirmed via its source map and literal `vex-overlay`/`vex-dialog` strings in the built file — so the Phase 1 design doc's "delete vex.js entirely" goal is met only for the three standalone files it names, not this bundle; already tracked as a deferred audit item in §7 item 4, README's license inventory still attributes it accurately. Deployed to staging (`ethercalc-staging`, version `09af9959-6d9b-40e5-aa87-7294e74a914b`) and acceptance-verified live: fresh WebAuthn registration → sign-out → discoverable sign-in (uid matched across the cycle), private-sheet create → owner reload persistence → anonymous denial (account-safe gate, `/view` fallback, cookies genuinely cleared via CDP for the anonymous check), forged `X-EC-Uid` header denial (identical to anonymous), room index 403, `static/passkey/NOTICE` served 200, and the toolbar-geometry + reduced-motion fixes both hold against the real deployed bundle (not just local `wrangler dev`). Full verification with explicit (unpiped) exit-code checks: client 279/279 (100% coverage), worker node 1087/1087 + workers-pool 162/162, client-multi 118/118, root `build-assets.test.ts` 5/5, e2e 26/26, typecheck/lint clean across all packages, `wrangler deploy --dry-run` clean for both `staging` and production envs. |  |
| 2026-07-11 | passkey-A/M3E | **Follow-up: scoped the passkey license banner to the entry chunk and made its NOTICE reference path-absolute; redeployed.** `feat/passkey-permissions` @ `584a02a`. `vite.passkey.config.ts`'s `generateBundle` hook previously banner-ed every emitted chunk with a relative `./NOTICE (same directory as this file)` claim; since this build's chunk-splitting is deliberately left unpinned (documented in the file's own top comment — `@m3e/web` triggering an internal split is unverified either way), a future second chunk would land under Rollup's default `assets/[name]-[hash].js`, where a relative reference resolves to a nonexistent `assets/NOTICE`. Fixed by scoping the hook to `isEntry` (one banner, on the one path-stable `ui.js`) and rewording to the actual deployed path (`/static/passkey/NOTICE`) rather than a same-directory claim. No functional behavior change — `dist-passkey/` still emits exactly one chunk (43 modules) — so this is preventative, not a bugfix for an observed failure. Verified banner text byte-for-byte in fresh `dist-passkey/ui.js`, the copied `assets/static/passkey/ui.js`, and the live staging response. Full unpiped verification: `build-assets.test.ts` 5/5, typecheck 0 errors across all 11 packages, lint clean (one pre-existing unrelated broken-symlink warning), client unit tests 279/279 at 100% coverage. Redeployed `ethercalc-staging` from source commit `584a02a` (previous entry's deploy predated this fix by one commit) — Worker version `c0292222-e09c-4041-86bc-2c61261fe83c`, single-file upload (`static/passkey/ui.js` only), `/_health` OK, `static/passkey/NOTICE` still 200. Staging now reflects the deployable source/artifacts from `584a02a`; this log entry itself lands in a separate, docs-only commit that necessarily supersedes `584a02a` as HEAD without changing what's deployed. |  |
| 2026-07-11 | passkey-A/landing | **Material 3 Expressive landing redesign deployed to isolated staging; production untouched.** Commit chain `434cda3` → `69074c2` → `fe85ce9` replaces the retro float/image landing with an EtherCalc-native sheet hero (formula bar, selected A1, responsive grid), clear public/private/passkey hierarchy, accessible browse/drop status, and 320px nav reflow. Import hardening made browse intentionally single-file, fixed async ID capture plus prompt cancel/empty-name handling, preserves raw workbook bytes through the ArrayBuffer fallback, UTF-8 decodes explicit CSV uploads (including one-column/no-final-newline files), and skips size-proportional main-thread decoding for ZIP workbooks. Verification: client 279/279 at 100% coverage, Playwright 31/31, e2e/root typechecks clean, asset tests 5/5, staging dry-run clean; live desktop + 320px screenshots show no horizontal overflow, exact remote XLSX import passed with `TextDecoder.decode()` forced to throw (`hello`, `42`), and exact one-column UTF-8 CSV preserved `café`/`東京`; disposable rooms were deleted. Final Worker version `cd709b54-f5aa-4c3e-b785-fdc07796854b`. | PR #841 |
| 2026-07-11 | passkey-A/landing | **CI fix, maintainer credit, and Gsheet-inspired floating nav; PR #841 CI green end-to-end.** `feat/passkey-permissions` @ `585528e`. (1) The prior landing entry's Worker deploy masked a real cross-platform gap: `packages/client/src/passkey/ui.ts`'s side-effect `./ui.css` import typechecked locally (macOS `tsc`) but failed Linux CI (`TS2882: Cannot find module or type declarations`) — added `packages/client/src/vite-env.d.ts` (`/// <reference types="vite/client" />`), matching the pattern `client-multi` already had; `client` never did. (2) Replaced the landing facts line ("no account · realtime · csv / ods / xlsx · self-hostable · open source") with a linked maintainer credit ("Maintained by [Audrey Tang](https://audreyt.org/) · Open source, built for everyone.") per explicit request. (3) Floating nav redesign per explicit request + two Google Sheets reference screenshots: removed the full-width `.ec-top` bar (brand-left/nav-right `justify-content: space-between`); brand is now a slim in-flow title, project links (Docs/API/GitHub) became a `position: fixed` top-right rounded pill cluster (`.ec-floatnav`) that persists through scroll — verified with real page overflow (821px content in a 480px viewport) that the title scrolls away while the pill stays pinned at its fixed coordinates. **Two real bugs found and fixed during this sub-session, both against a live redeployed staging Worker, not just local `wrangler dev`:** (a) at ≤400px the pill drops below the title (a `≤400px` breakpoint) to avoid a title/pill collision, but the sheet hero card started underneath the dropped pill — added `.ec-main { padding-top: 58px }` in that same breakpoint so the card clears the pill's measured bottom edge (93.5px); (b) once `.ec-top` stopped being a flex container, `.ec-brand`'s own `display: flex` made the `<h1>` a block-level flex container that silently stretched to `.ec-top`'s full content width — its `getBoundingClientRect()` right edge scaled ~1:1 with viewport width (866px at a 900px viewport) even though the visible "EtherCalc" text stayed ~130px wide, which fabricated false collision positives across an entire 401-1280px empirical scan; fixed by changing to `display: inline-flex` (shrink-wraps like `inline-block`) and re-verified genuinely zero collision at nine widths from 320 to 1280px, both live and via a rewritten multi-width Playwright regression (`landing-layout.spec.ts`) that checks brand/nav/sheet-card intersection at every breakpoint band, not just 320px. Verification: local Chromium e2e 33/33, root typecheck/build-assets/lint clean at every commit in the chain, staging `wrangler deploy --dry-run` clean, four sequential real staging deploys (`8560f815`→`22f0c1e0`→`7749c309`→`50a0f313`) each visually and numerically re-verified live. GitHub Actions on the final commit: `helm` (7s), `typecheck + test + coverage gate` (1m29s), `build:selfhost` (2m15s), `e2e — Playwright against wrangler dev` (1m26s), and `mutation-gate — changed packages only` (14m45s) all passed — the first full green run of PR #841's required checks this session (an earlier push had the CI-only `ui.css` typecheck failure above). PR #841 body updated to reflect final staging state (`fe85ce9`\-era summary; body edits after `585528e` were not re-submitted since they were prose-only and the code state remains accurately described). Final staging Worker version `50a0f313-0dd0-49b1-a549-31ed3f01b1a6`; production still untouched. | PR #841 |
| 2026-07-11 | passkey-A/landing | **Floating nav extended to the sheet editor page; credit line shortened; PR #841 CI green.** `feat/passkey-permissions` @ `620da74`. (1) Credit line shortened per explicit request: "Maintained by Audrey Tang · Open source, built for everyone." → "By [Audrey Tang](https://audreyt.org/) since 2011". (2) Per explicit request, the Gsheet-inspired floating nav (previously landing-only) now also renders on `index.html` (the sheet editor). **Investigated live geometry before writing any CSS**, on a real deployed room, not by assumption: `#ec-room-access`'s account/passkey button occupies the top-right corner on every tested width down to mobile (where it becomes the "More actions" overflow trigger) — reusing the landing page's `top`-anchored position would cover the ONLY sign-in affordance. SocialCalc's own grid has a small fixed scroll/logo cluster (`#te_morebuttonv`/`#te_logo`) in the bottom-right corner instead, empirically confirmed constant at ~69px up / ~10px in from that corner across 600-900px viewport heights (not proportional to viewport size). Landed on `bottom: 84px` for the sheet page (top-right stays for the landing page), sharing ONE `.ec-floatnav` chrome definition in `packages/client/src/passkey/ui.css` (already loaded by both pages) via `--top`/`--bottom` position modifier classes, rather than duplicating the pill's CSS per page. **Two more real bugs caught before shipping:** (a) moving the shared rule into `ui.css` broke the landing page's own ≤400px collision fix - `start.html` loaded `start.css` BEFORE `ui.css`, so `ui.css`'s later, equal-specificity `.ec-floatnav--top { top: 14px }` silently won cascade order over `start.css`'s scoped media-query override regardless of viewport width; fixed by swapping the `<link>` order (shared base first, page override after) - caught by the full local suite, not staging. (b) the new sheet-page regression test's straightforward copy of the landing page's "zero horizontal overflow" assertion was itself wrong for this page: confirmed via a direct check against the CURRENTLY-DEPLOYED (pre-change) staging sheet page, with no `.ec-floatnav` present at all, that `document.scrollWidth` (542px) already exceeds `clientWidth` (390px) at mobile width - a pre-existing, correct characteristic of SocialCalc's wide, non-wrapping column grid, unrelated to this feature; removed that assertion and kept only the nav's own on-screen/non-collision contract. **TDD discipline on both the new sheet-page test and the underlying fix:** temporarily set the nav to the wrong (`--top`) class first and confirmed collision failure, restored `--bottom`; separately weakened `.ec-floatnav--bottom`'s offset to `20px` (an insufficient value) and confirmed the `collidesMore` assertion specifically failed against `#te_morebuttonv` (not just an unrelated check), proving the regression test has real discriminating power before trusting it green. Verification: local Chromium e2e 34/34 (client-side `wrangler dev` fixture, not staging), client 279/279 at 100% coverage, e2e/root typecheck clean, `build-assets.test.ts` 5/5, lint clean, full `wrangler deploy --env staging --dry-run` clean. Deployed `620da74` to staging (Worker version `dcb2bb7a-d816-4faf-a8ce-b1cbfff195b4`) and re-verified everything live against the real deployed passkey button and real SocialCalc grid controls at desktop (1440) and mobile (390) widths, plus the landing page's credit text/link and 320px geometry - all passed with zero collisions. GitHub Actions on this commit: `helm` (8s), `typecheck + test + coverage gate` (1m24s), `build:selfhost` (2m13s), `e2e` (1m18s), `mutation-gate` (15m25s) all passed. Scoping note: this covers `index.html` (the single-sheet editor `packages/client` builds); the React-based multi-sheet UI (`packages/client-multi`, served at `/=:room`) is a separate codebase and was NOT touched - not requested, and out of scope for this pass. Production still untouched. | PR #841 |
| 2026-07-11 | passkey-A/room-access | **Room-access cluster redesigned three times in one review pass; sheet-page project links removed; PR #841 updated, staging-verified end to end.** `feat/passkey-permissions` @ `3054b01`. Pass 1: the full-width `m3e-toolbar` row (reserving grid height on every viewport) became a compact `position:fixed` pill, floating free on desktop with zero reservation, falling back in-flow only where genuinely needed. Pass 2, review-driven and each fix empirically re-measured, not just reasoned about: (a) the fixed pill crossed into SocialCalc's own fixed-width, non-reflowing "Edit Format Sort…" menu bar below a measured ~954px viewport width — fixed via a breakpoint switching to an in-flow block inserted directly before `#tableeditor`; (b) `m3e-button`/`m3e-icon-button` only default `tabindex="0"` inside Lit's async `firstUpdated`, landing after `m3e-toolbar`'s synchronous `slotchange` scan — a freshly-appended control could be invisible to the roving-tabindex manager, stalling Arrow-key navigation partway through (confirmed live, then fixed by pre-setting `tabindex="-1"` before every toolbar-bound control is appended, not worked around); (c) the closed account menu's `m3e-menu-item`s were real focusable toolbar descendants even while unopened, handing the roving group phantom stops — fixed by mounting the menu at `document.body`, ID-referenced by its trigger, instead of nesting it inside the toolbar; (d) a redundant CSS-only `display` media-query toggle raced the new JS DOM-presence toggle and could drop focus to `<body>` before the JS focus-transfer logic ran — removed, leaving DOM presence as the sole visibility mechanism; (e) the mobile overflow bottom-sheet caches its opener as a `document.activeElement` reference and refocuses it asynchronously on close, gated on a closing animation, not on its own (deceptively early-firing) `closed` event — fixed by awaiting the native popover `toggle` event instead and serializing concurrent resize-crossings. Pass 3, per explicit follow-up request: Docs/API/GitHub links removed from the sheet-page cluster entirely (landing-page-only now, via its own unmodified `.ec-floatnav`); "Use a passkey" collapsed to a circular key-icon trigger matching the avatar's shape; the resulting ~256px-max cluster fits a 320px viewport unfolded, so the entire mobile overflow/bottom-sheet mechanism pass 2 hardened became dead complexity and was removed outright (`#ec-room-overflow`, `buildRoomActionsSheet`, `installResponsiveVisibility`); desktop breakpoint dropped 1024px→840px against a re-measured ~757px collision-free minimum; `mount()` no longer pre-renders before `whoami()` resolves, since nothing renders early anymore. Bundle ~493KB→~427KB (JS). `mountSiteNav` renamed `mountRoomAccessCluster` to match its narrower remaining job. Verified: 1249 worker+client unit tests, 39 Playwright e2e (comprehensive rewrite: new keyboard/resize/collision regression tests, two overflow-specific tests deleted as testing a removed feature), staging redeploy with live visual/collision/keyboard checks on both classic and multi-sheet (`/=:room`) editors, and — closing a verification gap an earlier pass in this session had left open — a complete WebAuthn ceremony (CDP virtual authenticator: register → authenticated state → sign out → discoverable sign-in) against the final deployed staging version, confirming identity continuity end to end. | 3054b01 |
| 2026-07-11 | passkey-A/room-access | **Fourth room-access redesign pass: context action folded into the account menu; PR #841 updated, redeployed and re-verified on staging end to end.** `feat/passkey-permissions` @ `8f9ed8c`. The cluster still had two clickable controls side-by-side for signed-in users - a context-action button ("Make a private copy"/"Sheet access") plus the avatar - the one remaining two-layer control on an otherwise single-round-button design. `buildAccountMenu()` now takes an optional prepended context-action item (a divider separates it from "New private sheet"/"Sign out"); `mountRoomAccess()` builds that array instead of appending a standalone button. Cluster content dropped 256px to 126px (badge+avatar), so the fixed/in-flow collision breakpoint against SocialCalc's own menu bar dropped again, live-re-measured 840px to 700px (collision-free down to ~626px, same safety margin as before) - both sides of the exact breakpoint asserted directly (`position: static` at 700px, `fixed` at 701px), not just collision absence. Rewrote every e2e assertion that referenced the removed cluster-level button (visibility, geometry-containment, collision sweep, the now-structurally-impossible multi-item Arrow-key race) against the new menu-item shape, and added two behavioral click-through tests neither the pre-fold nor post-fold suite had: clicking "Make a private copy" (real POST + redirect-chain follow-through, menu closes) and "Sheet access" (real dialog opens with expected content, menu closes) - closing a gap where label-visibility assertions alone couldn't have caught broken menu-item wiring. 41/41 Playwright green (39 to 38 to 41: one obsolete test removed, one consolidated mount-race replacement, two new). Verified against a fresh `wrangler deploy --env staging`: full WebAuthn ceremony (register, avatar-only public-room menu, live copy-to-private round trip to a new random-ID private room, "Sheet access" dialog from that room's menu, sign out, sign back in with the same passkey/uid) against real D1/DO infrastructure, not mocks. | 8f9ed8c |
| 2026-07-16 | tooling | **Migrated the repository interface to Vite+ 0.2.4.** `vp install/add/remove/update` now owns package operations; `vp run`/`vp exec` own task and binary orchestration; package Vite/Vitest imports, scripts, contributor docs, e2e child processes, and CI/deploy/nightly/release workflows were migrated. Root `vite.config.ts` aggregates package Vitest projects and replaces Biome with Vite+ Oxlint while preserving the three-rule/no-formatter contract; Biome and its lockfile graph were removed. `bun@1.3.14` remains pinned as the underlying package manager/native runtime, with direct Bun confined to Bun-native scripts/tests and Docker or packed-package paths where Vite+ is not shipped. Verified frozen install, `vp check`, all-package typecheck, 2,208 aggregated Vitest tests, the full `vp run test` workflow, asset builds, and 15 Playwright e2e tests. | (this commit) |
| 2026-08-03 | security-audit | **Full-branch security audit of `main` against prod `ethercalc.net`, fixes landed at the owning boundary.** Closed: stored DOM XSS in the client graph panel (persisted `graphrange`/named-range values reached `innerHTML`; now `textContent` plus a built Hide-Help control, removing the last inline `onclick`); multi-sheet TOC/`postMessage` trust (origin+source checked in `index.html`, `SheetFrame` targets its own origin, TOC links validated by `isSafeMultiSheetLink`); WS/Socket.IO authentication and memory (per-message `verifyAuth` on every state-changing frame, attachment-room binding instead of client-supplied `parsed.room`, per-socket rate window, message/frame/chat/cell caps in `@ethercalc/shared`, bounded handshake sessions with teardown in the shim); unbounded ingestion (auth and suffix-import bodies now pass the shared size guard; ZIP archive/sheet/cell/column caps; bounded cross-sheet hydration; chunked-snapshot metadata validation); `/_timetrigger` no longer a public metadata/side-effect endpoint, and the scheduler no longer treats refusals as fires or deletes non-fired rows; session/identity (`__Host-ec_sess`, AuthDO-side revocation with monotonic windows, per-IP ceremony rate limits, challenge capacity cap, session expiry enforced on live sockets); trust-proxy correctness (`CF-Connecting-IP` honoured only from the trusted edge, nginx now replaces rather than appends forwarding headers, bucket eviction); room-ID entropy restored to ≥62 bits; creation-limit classifier extended to private/form/multi-import routes; `no-store` on authenticated/private responses plus global security headers; export sanitizer allowlist mirrored to the client policy. Root HTML page scripts were extracted to five tracked files under `static/` (`index-bootstrap`, `index-l10n`, `start-bootstrap`, `start-page`, `panels`) with a `requiredFiles` assertion in `scripts/build-assets.ts`, since the whole-directory copy checks no individual names — note the app CSP still needs `script-src 'unsafe-inline'` for SocialCalc's toolbar, so this is hygiene, not yet a boundary. Infra: compat date 2024-11-12 → 2026-07-21 (capnp in lockstep), `run_worker_first`, pinned base image, read-only/no-new-privileges containers, `automountServiceAccountToken: false`, Helm passkey trust anchors that fail closed, with `scripts/check-helm-hardening.sh` covering both. Verification: `vp check`, all-package typecheck, 1499 worker-node + 194 workers-pool + the remaining package suites, 100% worker coverage, 47/47 Playwright e2e, `wrangler deploy --dry-run`, helm lint/template matrix, self-host and proxy smoke. Mutation floors re-measured: worker 90.21 (break 90), shared 99.69 (99), socketio-shim 84.68 (84), migrate 90.38 (90), oracle-harness 83.46 (83), client 77.61 (77); provably equivalent mutants carry written `// Stryker disable` justifications instead of padded tests. | (working tree) |
| 2026-08-03 | security-audit/follow-up | **Closed the final Host-derived CSP seam caught during delivery review.** `index.ts` had built `connect-src ws(s)://…` from `new URL(request.url).host`, even though the same file correctly forbids using attacker-controlled `Host` for cross-origin redirects. New pure `lib/csp.ts#websocketAuthority` anchors the WebSocket authority on `ETHERCALC_ORIGIN` (including configured scheme/port), falling back to the request host only for self-hosts without a valid configured origin. Ten Node tests pin configured/fallback/scheme/port/invalid-origin lanes. Live `wrangler dev` probes with `Host: attacker.test` and `Host: evil.example:1337` both retained `connect-src 'self' wss://ethercalc.net`. Post-fix worker typecheck, 100% coverage, mutation 90.21 ≥ 90, and `vp check` passed; Stryker restored cleanly with no instrumentation markers. | (working tree) |
| 2026-08-10 | prod-upgrade | **Operator runbook for upgrading production `ethercalc.net` to `main`.** Deliverable `docs/migration/PROD_UPGRADE_PLAN.md` with companions `DELTA_AUDIT.md`, `SKEW_AND_RECONNECT.md`, `PREFLIGHT_RESULTS.md`, `INVENTORY.md`. Central finding overturned the original premise: production is not tag `0.20260717.0` / `149ebcf`. Read-only probes (`GET /_auth/whoami` → `{"uid":null,"enabled":true}`, a route absent at `149ebcf`; root HTML serves `./static/passkey/ui.js`; `GET /static/index-bootstrap.js` is 404) bound production to `[d2afa90, b7d8840)` — passkeys, `AuthDO`, ACL, and private rooms are already live; the security-audit page-script extraction is not. That mattered because the original three-phase cutover set `ETHERCALC_AUTH="0"` in Phase 2 as a safety soak; against real production that fails `authEnabled()`, nulls every session principal, and 403s every existing private room to its owner — a self-inflicted outage. Corrected plan is a single gradual ramp with `AUTH="1"` throughout; superseded three-phase analysis preserved as Appendix A. Verified constraints that made the simpler plan safe: across the whole candidate range `wrangler.toml` declares only `v1`/`v2` (no lifecycle change), all three D1 migration files are byte-identical (blob OIDs match), and `lib/authorize.ts` is byte-identical (no declassification on rollback). A real regression the corrected baseline exposed: `lib/session.ts` renames the session cookie `ec_sess` → `__Host-ec_sess` with no dual-read, so every signed-in passkey user is silently logged out at rollout and an open tab on a private room can fail to reconnect. Search-indexing policy restored and shipped (`c789249`): a worker session misread the originating commit's `docs(migration):` subject, judged the bundled production code accidental, and reverted it in `ae86756`; recovered in `c789249` with all four blobs verified byte-identical to the originals (cautionary for dual-purpose commits). Measurements at tip `327fa3d`: worker suites 53 files / 1526 tests (node), 13 / 197 (workers-pool), 100% coverage (2706/1936/298/2416); mutation ratchet exit 0 in 828s with every audited package under 1pp of its break floor and `@ethercalc/worker` at +0.03pp — `robots.ts` itself scored 100% (9 killed / 0 survived). Still open: six `[OPERATOR-VERIFY]` items needing live Cloudflare access, chiefly `wrangler deployments list` to pin `CURRENT_PROD_VERSION_ID` (needed for the rollback target, not for the ramp's safety), and the staging rehearsal. | (this branch) |

---

## 15. Runbook

```bash
# fresh clone
git clone https://github.com/audreyt/ethercalc
cd ethercalc
bun install
bun run --cwd packages/worker test

# run the new worker locally
bun run --cwd packages/worker dev       # wrangler dev --local

# run the oracle (current main) for comparison
docker compose -f tests/oracle/docker-compose.yml up -d

# record new oracle fixtures after adding a scenario
bun run --cwd packages/oracle-harness record

# replay against local worker
bun run --cwd packages/oracle-harness replay --target http://127.0.0.1:8787
```

To resume work:
1. §14 for last session's context.
2. §8 for next pending item.
3. §7 for live risks on the affected area.
4. Commit; update §14.
5. If plan diverges from reality, edit this doc *before* merging the code.

---

## 16. Appendix

### 16.A — socialcalc-in-DO wrapper (current state)

Post source-swap (2026-04-20): `socialcalc` dep resolves to `github:audreyt/socialcalc#4463d50` — a strict-mode-clean, Bun + tsgo port of 2.3.0. Runs inside workerd via `@cloudflare/vitest-pool-workers`. 7/7 smoke tests green.

**Load pipeline** (`packages/socialcalc-headless/`):
1. Vite `?raw` import of `socialcalc/dist/SocialCalc.js` — string-loads the UMD at build time.
2. `scripts/build.js` applies two source transforms on the bundled string:
   - `document.createElement(` → `SocialCalc.document.createElement(` (redirect DOM creation to our shim).
   - `alert(` → `(function(){})(` (silence error-path alerts).
3. One wrapper-level transform: `typeof globalThis !== 'undefined' ? globalThis : this` → `this` (so the host-binding IIFE captures `root`/`window`).
4. `new Function(...)` eval inside the DO (permitted; no CSP restriction). Sloppy mode by default — required because SocialCalc uses `delete varname;` at a few lines.
5. Eval scaffold installs DOM stubs (`ShimNode` class covering `id`, `width`, `height`, `className`, `colSpan`, `rowSpan`, `title`, `innerHTML`, `outerHTML`, `appendChild`) and a **synchronous `setTimeout` shim** (`function(cb){cb();return 0;}`) so recalc state machines unroll inline.
6. Factory memoized module-wide; 27k-line eval runs once per isolate.
7. Manual `SocialCalc.RecalcSheet(sheet)` kick after command batches (without an editor, `recalc` only sets `needsrecalc="yes"`).

Upstream fixes (now in `audreyt/socialcalc`): the ES5 `delete varname;` rewrites, reserved-`eval` renames, and `factory.call(root, this)` → `factory.call(root, root)` that we used to do by hand. Implicit-globals pre-declaration removed — `var`-less assignments in paste/render/format/MIME paths fixed upstream.

**Proven**: SUM formulas, snapshot+log round-trip, text+number mix, recalc without formulas, `exportCSV`/`exportCells`/`exportCell`, `csvToSave` via Clipboard + CreateSpreadsheetSave, cross-sheet formula references (`'other-room'!A1`) via `addSiblingSheet` + sibling-snapshot fetch.

**Not yet exercised end-to-end**:
- `ScheduleSheetCommands` async path — we bypass with sync `ExecuteSheetCommand` (§7 item 6).

### 16.B — Deferred decisions not in §13

(none; append when new open questions emerge mid-execution)

---

*End of plan.*
claude --resume 7059f882-1b8d-44c5-8c9e-c246879c20fd