# agent-stream-hub Streaming bridge that lets you peek at AI-CLI sessions running on Windows `agent-term` instances from a MacBook browser or iPhone PWA. ## Architecture ``` [Windows agent-term × N] [MacBook browser / iPhone PWA] │ HTTPS POST │ HTTPS GET │ (register + push snapshots) │ (list runs, view one) ▼ ▼ Cloudflare edge (named tunnel → your hub's public URL) │ ▼ cloudflared on the host (supervised, e.g. LaunchDaemon / systemd) │ ▼ localhost:9000 ── agent-stream-hub (this repo, Node) ├── POST /runs (register / update meta) ├── POST /runs/:id/snapshot (push a viewport) ├── POST /runs/:id/heartbeat (liveness + isWorking) ├── POST /runs/:id/input (viewer → source prompt) ├── DELETE /runs/:id (cleanup) ├── GET /runs (list) ├── GET /runs/:id/latest (most recent snapshot) ├── GET /runs/:id/history (older snapshots) └── GET / (viewer SPA) │ ▼ in-memory state (ring buffer per runId, text-similarity compaction) ``` Three tiers: - **Source** — `agent-term` on Windows (or macOS), multiple concurrent instances. Outbound HTTPS only. POSTs viewport snapshots to the hub. - **Hub** — this repo, deployed on an always-on host. Schema-agnostic ring buffer keyed by `runId`. Also serves the viewer SPA. - **Viewer** — static HTML+JS served by the hub. Loaded by a browser or installed PWA via the hub's public URL. Why three tiers: - **Corporate firewall** — the source can only outbound HTTPS to a Cloudflare CDN host, never accept inbound. The hub gives it a reachable endpoint. - **Clients sleep, the host doesn't** — hub state lives on the always-on host; a viewer reconnects when its device wakes. - **Multi-instance** — hub aggregates N sources, presents one list to the viewer. - **Host stays low-touch** — the relay is schema-agnostic, so adding fields, styles, or new view types never touches the host. ## Data model — snapshots, not blocks The hub stores raw viewport snapshots; the viewer stitches them into a continuous buffer per run. There are no blocks, no per-prompt segmentation, no marker/divider tracking. The source's terminal viewport is the source of truth, full stop. ### RunMeta (registration payload, hub-cached for the run's lifetime) ```js type RunMeta = { runId: string; // random hex, generated by source at startup cli: string; // "claude" | "codex" | "copilot" | ... title: string; // human-readable; the first user prompt host: string; // hostname of the source machine startedAt: number; // ms epoch hue?: number; // 0..360, optional session color for chrome }; ``` ### Snapshot (the per-push payload) ```js type Snapshot = { runId: string; cols: number; rows: Row[]; isWorking?: boolean; // "AI is computing"; drives viewer's working indicator }; type Row = { text: string; styles?: StyleRun[]; }; type StyleRun = { start: number; end: number; fg?: string; bg?: string; bold?: true; italic?: true; underline?: true; dim?: true; inverse?: true; strike?: true; }; ``` ### Hub-side ring entry ```js type Entry = { seq: number; // monotonic per run, set by source ts: number; // ms epoch, hub-assigned on receipt payload: Snapshot; }; ``` ## Endpoints All under the hub's public base URL. Bodies are JSON unless noted. ### Source → hub (auth-open; sources have no shared secret) | Method | Path | Body | Returns | |---|---|---|---| | POST | `/runs` | `RunMeta` | `204` | | POST | `/runs/:runId/snapshot` | `{ seq, payload: Snapshot }` | `{ viewerAgeMs }` | | POST | `/runs/:runId/heartbeat` | `{ isWorking?: boolean, touchedAt?: number }` | `{ inputs: string[], voiceInputs: string[], viewerAgeMs }` | `voiceInputs` are voice-origin transcripts the source prefixes with the guide reference before typing (see [voice.md](voice.md)); `inputs` are typed viewer prompts, written as-is. `viewerAgeMs` is ms since a viewer last polled this run's detail view (`/latest` or `/history`); `null` if never. An age, not a timestamp — hub and source clocks can't be compared. The source uses it to pace its heartbeat by viewer presence: top pace (2s) while someone is watching — even mid-work — so interjections land in seconds; back to 30s-while-working once nobody has looked for ~5 min. Snapshot acks carry it too, which is how a working source (slow heartbeats, frequent snapshots) learns of a new viewer within a second or two. `touchedAt` is when the session last drew the user, in ms on the source's clock: their input, at the desk or from a viewer, or the agent finishing a turn. agent-term sends the timer its auto-hide ages windows out by, so the phone ranks sessions the way the desktop keeps them in view. The viewer lists a machine's sessions by state (your turn first), then most recently touched first; a run that never sent `touchedAt` ranks by `startedAt`. It is compared only among one machine's runs, which share a clock. It moves on input and finished turns, never on a heartbeat, so the list holds still while sessions wait. ### Viewer / debug → hub (auth-gated: `X-Hub-Secret` required when configured) | Method | Path | Returns | |---|---|---| | GET | `/runs` | `{ runs: RunMeta[] }` (all known), each with `lastSeenAt`, `isWorking`, `touchedAt` | | GET | `/runs/:runId/latest` | `{ snapshot, lastSeenAt, isWorking }` or `404` | | GET | `/runs/:runId/history?before=&limit=` | `{ entries: Entry[] }` oldest→newest | | POST | `/runs/:runId/input` | viewer-submitted prompt (`source:"voice"` for reviewed transcripts); drained on next heartbeat | | POST | `/runs/:runId/voice` | raw audio in → transcript + gate verdict; auto-queues on confidence ([voice.md](voice.md)) | | DELETE | `/runs/:runId` | `204` | ### Auth model - **Viewer reads and control**: when `STREAM_HUB_SECRET` is set, require `X-Hub-Secret` on every connection, including loopback. Proxy headers do not bypass authentication. The viewer prompts once and stores the secret in `localStorage`. - **Source POSTs**: `/runs`, `/runs/:id/snapshot`, `/runs/:id/heartbeat` remain open so source machines do not need the viewer secret. A run ID acts as a capability to publish snapshots and drain queued replies; keep it private. Anonymous registration is possible, so this is a hub for mutually trusted machines and users, not isolated tenants. - **Local development**: with no secret, API requests require a loopback Host and no Cloudflare forwarding header. Always set a secret before exposing a proxy, including proxies that rewrite Host to localhost. - **Browser boundary**: API requests from other origins are rejected; there is no wildcard CORS. Serve the viewer from the hub and preserve Host at the proxy. Native AgentTerm sources do not send Origin headers. - **JSON POSTs**: require `Content-Type: application/json`; `/voice` accepts raw audio. Static viewer assets are public and include no session data. ### Resource limits The hub retains at most 128 runs. Registration metadata is limited to 16 KiB; POST bodies to 4 MiB. Each run keeps up to 400 snapshots or 8 MiB of serialized snapshot data, whichever is smaller. A 64 MiB global snapshot budget removes the oldest snapshots under pressure. These are serialized byte limits, not a bound on total JavaScript heap usage. Runs expire after 24 hours without activity. Each run queues at most 64 replies totaling 64 KiB across typed and voice input. A full queue returns `429` without accepting the input; heartbeat drains it. Registration at capacity returns `503`; metadata over its limit returns `413`. Snapshot and metadata fields consumed by the viewer are validated before storage; extra fields remain allowed. Voice processing permits two concurrent requests, limits decoding to 15 seconds and transcription to 30 seconds, and rejects recordings over two minutes. Request-rate and connection limits belong at the proxy. See [Security](SECURITY.md) for deployment and data handling. ## Source-side stream lifecycle A run does **not** register at agent-term startup. It registers when the first user prompt is captured by `prompt-capture.js`. Pre-CLI shell activity and pre-first-prompt CLI welcome output are not streamed. ``` agent-term starts ─► generate runId in memory; wire prompt-capture; no hub traffic yet User submits first prompt to AI CLI: ─► POST /runs (register; RunMeta.title = first prompt) ─► start heartbeat timer renderer-watch detects viewport change: ─► POST /runs//snapshot (current visible rows) Subsequent prompts: ─► local state updates only (no extra hub traffic) ─► next viewport change pushes a new snapshot as usual Alt-screen flip (copilot, codex resume picker, ...): ─► POST /runs//snapshot with new viewport ─► viewer's stitcher inserts a DROPPED marker (alt-screen-replace path) Periodically: ─► POST /runs//heartbeat — adaptive pace: 2s while awaiting input or while a viewer is watching (viewerAgeMs from acks), 30s while working unwatched; 200ms burst after an input drains agent-term exits cleanly: ─► heartbeat stops; run becomes stale after 60s; hub evicts after 24h idle agent-term crashes or net drops: ─► same — heartbeat stops, run ages out ``` ### What scrolling and alt-screen mean to the source The renderer encodes only the **visible viewport** (`buffer.baseY` through `baseY + terminal.rows`). Content that scrolled off the top is not in the snapshot — and is not in any later snapshot. The viewer's stitcher captures rows that pass through the viewport over time, accumulating them into a buffer; rows that scrolled off **before** the viewer connected are lost. Alt-screen enter/exit produces a snapshot of the new viewport. The viewer's stitcher detects the discontinuity (no alignment between old buffer and new snapshot, similar sizes) and emits a `DROPPED` marker as a visible cue. ## Hub-side compaction The ring is text-similarity compacted: two adjacent snapshots whose rows have ≥ `COMPACT_TEXT_MATCH` (default 0.9) of text matching → the newer one **overwrites** the older in place. Keeps the ring shallow when the source streams spinner-frame churn. Net effect: every distinct ring entry represents a meaningful viewport change. Spinner cycling collapses to one entry. New content (added lines, text changes) appends. ## Viewer-side stitching `public/stitcher.js` merges incoming snapshots into a single buffer per run. - **Alignment**: for each substantive row of the new snapshot, find all buffer positions where it matches, then score each candidate by how many more substantive rows continue to align. Pick max-score (ties → highest offset = "furthest match"). This handles repeating CLI templates (codex's per-turn `Done. Rebuilt / AgentTerm / Release` shell) because the correct offset is the one with the longest extended match, regardless of how small a sub-window of rows is reused across turns. - **No alignment + similar sizes** → REPLACE buffer with snapshot, prepend a `DROPPED` sentinel as visible "previous viewport gone" indicator. Catches alt-screen flips, picker close, modal dismiss. - **No alignment + different sizes** → append `GAP` sentinel + snapshot. Preserves history when the two snapshots are genuinely disjoint pieces of content the viewer should both show. - **Cap** at `DEFAULT_MAX_ROWS = 1000` rows. Bounded buffer growth. ## Source connection status The agent-term stream client tracks its state and surfaces it to the renderer for UI indication. | State | Meaning | |---|---| | `idle` | no first prompt yet; nothing to send; indicator hidden | | `disabled` | no hub URL configured; hollow grey dot | | `connecting` | first prompt fired; register POST in flight | | `connected` | recent successful POST (register / snapshot / heartbeat) | | `retrying` | last POST failed; backoff retry in progress | | `disconnected` | ≥ N consecutive POST failures; retry queue building | Failure is never silent: persistent dot in the agent-term window, toast on disconnect, toast on recovery. ### Retry queue - In-memory ring of pending POSTs (default cap 50). - On reconnect: drain oldest-first. On overflow during disconnect: drop oldest. - v1 does not signal gaps to the hub or viewer. ### Async / non-blocking Network I/O must never block agent-term's UI or PTY work. All hub POSTs are fire-and-forget from the producer's perspective. A single worker loop owns the network state. Producers (stream-state, heartbeat timer) never await a `fetch`. ## URL + secret discovery Source reads from `~/.agent-term/config.json`: ```json { "hubUrl": "https://your-hub.example.com", "hubSecret": "" } ``` - `hubUrl` empty → streaming `disabled` (hollow dot, no requests). - `hubSecret` is unused by the source (all source POSTs are auth-open). Set when running viewer-side curl debugging. Viewer prompts for the hub secret on first auth-gated request and persists it in `localStorage`. Optional seed links use `#s=` so the secret is not sent in the HTTP URL; query-string seeding is unsupported. ## Iterative roadmap — current state Boxes checked = shipped. - [x] **Hub + source minimal** — register, snapshot, heartbeat, latest. - [x] **Viewer SPA** — list runs, click into detail, stitched buffer with `GAP` / `DROPPED` markers. - [x] **Adaptive polling** — 1.5s while live, 10s while idle, pause when tab hidden. - [x] **Hub-side compaction** — text-similarity collapse of spinner churn. - [x] **Named tunnel** — stable public URL, no rotation. - [x] **Daemon supervision** — hub + cloudflared survive a host reboot without a graphical login. - [x] **Auth secret** — gated reads + destructive writes. - [x] **Run grouping** — viewer groups streams by `(host, title)`; per-card delete; stale tiles dimmed; drill-in for past streams in a group. - [x] **Viewer → source input channel** — `/runs/:id/input` queue, drained on heartbeat, written into PTY as typed characters + CR. - [x] **`/history` endpoint** — older snapshots paginated by `before`+`limit`. - [x] **Voice input, hub side** — `/runs/:id/voice`: transcribe via resident whisper-server, confidence-gate, queue as `voiceInputs`. Phone UX and source-side injection still open — see [voice.md](voice.md). - [ ] **Hub-side persistence** — survive daemon restart without losing rings. Deferred until format and schema settle. - [ ] **History UI in viewer** — frame-replay through `/history` results. - [ ] **iPhone PWA polish** — manifest, apple-touch-* tags, swipe-friendly wide-block rendering. - [ ] **Clickable file/line patterns in viewer** — port from agent-term's `src/renderer.js` for IDE handoff. - [ ] **SSE** — `GET /runs/:id/stream`; viewer prefers SSE, falls back to polling. Only worth doing if polling cost becomes noticeable. ## Locked decisions (don't relitigate without good reason) - The iOS/iPadOS home-screen icon (`public/icon-180.png`) intentionally keeps the original terminal artwork at 108×108 (60%), anchored top-left on an opaque-white 180×180 canvas. The asymmetric field makes the black tile read as a terminal pane opening into the larger viewer; its smaller prompt mark and breathing room tested better than an optically enlarged 62% redraw. Do not crop, upscale, center, or round the inner pane, and do not make it match the deliberately full-bleed `public/icon.svg` browser/favicon variant. - Snapshot-based protocol, no blocks, no markers/dividers. Viewer stitches. - One agent-term process = one run = one `runId`. Resume = new run + new tile. - Hub keeps last 24h of activity (`EVICT_MS`). User deletes stale tiles manually via the viewer's per-card `✕`. - Ring cap = 400 entries per run, post-compaction. Loss beyond that is acceptable — viewer's accumulated buffer holds anything that passed through during a viewing session. - Schema-agnostic relay (Node, ~300 LOC), rarely changes. - Hub + viewer live in this repo; the source (agent-term) has a one-line pointer. Deployment specifics are kept separate from this public repo. - Public URL comes from a named tunnel (stable); never hardcoded as a constant in agent-term (source reads `~/.agent-term/config.json`). - `cli` is run-level (in `RunMeta`), not snapshot-level.