# Notifications, Activity & Watches Map Three interlocking subsystems that the root `CLAUDE.md` predates and does not describe: an append-only activity log, a per-user watch registry, and a single notification funnel that both feed. They are wired together inside one helper, which is the part most likely to surprise you. --- ## 1. The shape ``` a route or the collab WS │ ├── processMentionsOnSave() ──────────────┐ routes/helpers/mentions.js │ │ └── logActivity() │ routes/helpers/activity.js │ │ ├── INSERT activity_log │ ├── emitEvent() ─► webhook outbox │ services/webhooks.js (section 2a) ├── applyAutoWatch() ─► watches │ └── fanOutToWatchers() ────────────┤ ▼ createNotification() services/notifications.js │ ┌──────────────┼──────────────┐ ▼ ▼ ▼ INSERT WS push email notifications user-channel.js email.js + templates ``` The load-bearing surprise: **`logActivity` is not just a logger.** It also enrols the actor as a watcher and fans notifications out to every other watcher (`routes/helpers/activity.js:102-103`). Adding a `logActivity` call to a new route therefore starts sending people email. That is usually what you want, but it is not what the name suggests. ## 2. `logActivity` (`routes/helpers/activity.js:51-104`) Fire-and-forget by construction: it returns `void`, wraps everything in a promise chain, and swallows errors into `console.error` (`routes/helpers/activity.js:51-57`). A failed activity write never breaks the user's request. Callers do not `await` it. ```js logActivity({ user, action, resourceType, resourceId, metadata, workspaceId?, squadId? }) ``` Steps: 1. Bail unless `user.id`, `action`, `resourceType`, `resourceId` are all present (`routes/helpers/activity.js:60`). 2. If `workspaceId` was not passed, `resolveScope()` walks the parent chain (`routes/helpers/activity.js:233-291`): log → archive → squad → workspace, with dedicated branches for `archive`, `comment`, `squad`, and `version` resource types. **If the scope cannot be resolved, the event is dropped silently** (`routes/helpers/activity.js:66`, `routes/helpers/activity.js:69`). An archive with `squad_id NULL`, such as the GitHub PR-session system archive, produces no activity at all. The same function is exported as `resolveActivityScope` (`routes/helpers/activity.js:293`) for a caller that deletes the resource: after the delete nothing leads back to a workspace, so that caller resolves first and passes `workspaceId` and `squadId` in. `DELETE /api/archives/:id` is the one such caller (`routes/archives.js:205`), and it skips `logActivity` when the archive has no workspace. 3. `log.update` events from the same user for the same log inside a **5-minute** window are coalesced away entirely (`routes/helpers/activity.js:23`, `routes/helpers/activity.js:72-83`). No other action type is coalesced. 4. Insert the row, then emit the outbound event (section 2a), then auto-watch, then fan out. ### The action taxonomy Eighteen values are currently emitted across the codebase: ``` archive.create archive.delete archive.rename log.create log.delete log.rename log.update log.move log.publish log.restore comment.create comment.delete comment.reply comment.reopen version.delete squad.invite_create squad.member_join squad.member_leave ``` `log.rename` has three sources: `PUT /api/document/:logId/title`, the collab `title` message, and the tree route `PUT /api/archives/:archiveId/logs/:logId` (`routes/archives.js:542`). The tree route is the only source of `log.move`, with `title`, `parent_id` and `previous_parent_id` in its metadata (the title is what the feed names the document by). It reads the row first and logs each event only when the value actually changes (`routes/archives.js:593-658`). A new parent must be a log in the same archive and not the log itself or one of its descendants, checked by a recursive walk up the new parent's ancestry that stays in the archive (`routes/archives.js:611-627`), so a `log.move` never names a parent outside the tree. The read, the walk and the UPDATE run in one transaction behind `SELECT ... FOR UPDATE` on the archive row (`routes/archives.js:594`), so moves in one archive take turns: two opposite moves sent at once cannot both pass the walk, and `previous_parent_id` is the parent the move actually replaced. The two routes that create a log, `POST /api/archives/:archiveId/logs` and the upload route, hold a new log's parent to the same archive through `isLogInArchive` (`routes/helpers/shared.js:112`). Neither action auto-watches or notifies. `archive.delete` lands in `activity_log` but is invisible to both read paths: the workspace feed's access clause needs an `EXISTS` on the `archives` row (`routes/activity.js:98-101`), which the delete removed, and the per-document read lists no archive rows. Its consumer is the outbound-events stream, not the feed. `resourceType` is one of `log`, `archive`, `comment`, `squad`, `version`. Note the asymmetry: a comment event carries `resourceId = comment.id` and puts the document id in `metadata.log_id`, which the fan-out then reads back (`routes/helpers/activity.js:187`). ## 2a. Outbound events (`services/webhooks.js`, W6-CDX-13) `doLogActivity` calls `emitEvent(ctx, { workspaceId, squadId })` directly after the `activity_log` insert and before auto-watch, in a `try` of its own (`routes/helpers/activity.js:93-100`). So a coalesced `log.update` (step 3 returns before the insert) is never emitted, an event is emitted only for an activity row that was written, and the outbox and the watchers cannot fail each other. `emitEvent` also never throws: it catches, logs `webhook emit failed` and resolves `null` (`services/webhooks.js:374`). What it does, in order (`services/webhooks.js:315`): 1. Return unless the action is one of the eight in `EMITTED_TYPES` (`log.update`, `log.publish`, `log.restore`, `log.rename`, `log.move`, `log.delete`, `archive.rename`, `archive.delete`). 2. Match the **in-process subscription cache** by workspace (a subscription with no `workspace_id` matches every workspace) and by `event_types`, and return if nothing matches. **No query runs up to here**, so an install with no subscription, and every unit test that never calls `loadSubscriptions`, issues nothing new (`services/webhooks.js:324`). 3. Build `data` per type. Document types read `SELECT archive_id, title, parent_id FROM logs WHERE id = ?` once (`services/webhooks.js:285`) and emit nothing if the row is gone. `title` is the document's: from `metadata.title` where the caller logged the document's title, and from the row for `log.publish`, whose `metadata.title` is the name given to the version (`services/webhooks.js:298`). `log.delete`'s activity row names the archive with `metadata.log_id`, so its data needs no read; the archive types need none either. The route records `log.delete` only when its `DELETE ... WHERE id = ? AND archive_id = ?` removed a row (`routes/archives.js:691`), so an id from another archive can never be announced as deleted. Every id field is an integer or `null`. 4. In one `withTransaction` (`services/webhooks.js:339`), after taking the order lock (`webhook_sequence_lock FOR UPDATE`, `:343`): read the instant (`occurred_at`, `:347`), insert the `webhook_events` row with an empty body to get its id (the envelope's `sequence`), serialize the envelope with a fresh UUID v4 as `id`, store its UTF-8 bytes, and insert the deliveries with `INSERT ... SELECT` over the matched ids that **re-checks** `enabled`, the workspace and the type in SQL, so a subscription deleted, disabled or narrowed since the cache last loaded gets nothing. When that leaves no delivery at all, the transaction throws a private sentinel and rolls the event row back, and `emitEvent` resolves `null` without logging (`services/webhooks.js:369`). Otherwise it resolves `{ eventId, deliveries }`, the count the database wrote. `serializeEnvelope` bounds `data.title` and `data.name` to 255 code points (never splitting a surrogate pair) and `actor.name` to 32, so no v1 body exceeds 4 KiB however its text escapes; the worst case measures 3,631 bytes (`tests/services/webhooks.test.js`). The envelope is public: [`docs/api/webhooks.md`](../api/webhooks.md). **The cache** (`loadSubscriptions`, `services/webhooks.js:116`) holds id, workspace and types of every enabled subscription and **no secret** (the query never selects it). `server.js` loads it at boot and every 60 seconds, and every admin write reloads it. A failed load keeps the previous cache. Loads can overlap, so each is numbered when it starts and a result older than the one already applied is dropped: a slow minute refresh that read the table before an admin write cannot put back the cache that write's reload replaced. **There is no env subscription** (removed in CDX-SA-3, 0.15.0). Every subscription is an admin row (`source = 'admin'`). The 0.15.0 migration deleted the old `source = 'env'` rows with their deliveries, and the worker never sends a row that is not `admin` (below). Boot loads the cache and nothing else. Ids are allocated in commit order: before its insert, the transaction takes `webhook_sequence_lock FOR UPDATE` (the one-row table from `2026-10-10-webhook-sequence-lock.sql`), so an emit that starts while another is uncommitted waits for it. The delivery worker that sends these rows is section 2b. ### Auto-watch (`routes/helpers/activity.js:33-37`, `routes/helpers/activity.js:112-137`) Three rules, all `INSERT IGNORE` so they are idempotent: | Action | Actor starts watching | `watches.source` | |---|---|---| | `log.create` | the new log | `auto_create` | | `log.update` | the log | `auto_edit` | | `comment.create` | the comment's parent log | `auto_comment` | Because auto-watch runs *after* the `log.update` coalescing check, a user whose update was coalesced away does not get auto-watched on that edit. They will on the next non-coalesced one. ### Watcher fan-out (`routes/helpers/activity.js:181-222`) Only five actions map to a notification type (`routes/helpers/activity.js:25-31`): ``` log.update -> watched_log_update log.publish -> watched_publish log.restore -> watched_publish comment.create -> watched_comment comment.reply -> watched_comment ``` `collectWatchers(logId)` (`routes/helpers/activity.js:145-168`) unions direct watchers of the log with watchers of its parent **archive**, so watching an archive cascades to every document in it. The actor is excluded (`routes/helpers/activity.js:208`), and `createNotification` self-suppresses again as a second line of defence. Fan-out is a **sequential `await` loop**, one `createNotification` per watcher. For a heavily-watched archive that is a lot of serial round trips inside a fire-and-forget promise. Fine at current scale, worth knowing before someone watches a 500-document archive. ## 2b. The delivery worker (`services/webhook-worker.js`, W6-CDX-14) Sends the rows section 2a writes. Started by `server.js` after `loadSubscriptions()` (`startWebhookWorker`, a `setInterval` of `TICK_MS` = 2 s, `.unref()`ed, never two ticks at once; a tick that throws is logged). Every dependency is injected into `createWebhookWorker({ query, send, now, owner, resolve, targetOptions })`, so the tick is testable without timers or a network. **One tick** (`tick`, `services/webhook-worker.js:299`). It reads every enabled, unpaused subscription (`SUBS_SQL`, no `LIMIT`: an instance holds at most 50, see `MAX_WEBHOOK_SUBSCRIPTIONS`). It is round-robin: each round takes one step for every subscription still going, so an endless backlog on one cannot starve another. The first subscription of a round rotates from tick to tick. A tick stops at 100 sends (`MAX_SENDS_PER_TICK`) or 5 seconds (`TICK_BUDGET_MS`), checked before each step. A step that is `delivered`, `dead` (422) or `expired` lets its subscription go on; anything else ends its run for the tick. Every query has a 10-second deadline (`QUERY_TIMEOUT_MS`, `withDeadline`), so a hung query fails the tick rather than the loop. **One head** (`deliverHead`, `:223`). The head is the subscription's oldest `pending` delivery (`HEAD_SQL`, `ORDER BY d.event_id LIMIT 1`). Nothing behind it is read, which is the ordering guarantee. The lease is a guarded `UPDATE` (`LEASE_SQL`): `leased_by` is the worker's owner token, `lease_expires_at` 30 seconds out, taken only when the row is pending and unleased or expired. After the DNS check and before the send it is renewed (`RENEW_SQL`), guarded, so the send starts with its full window; a renewal that touches no row sends nothing. `affectedRows !== 1` means another worker holds it: the subscription is skipped this tick (`held`). Then, in order: - `occurred_at` older than 72 hours, compared in SQL with `UTC_TIMESTAMP(3)`: dead-lettered with `expired after 72h`, nothing sent; the run goes on to the next head (`:230`). - A row whose `source` is not `admin` is never sent, whatever it holds: one skip line per window, its deliveries stay pending (`admin()` in `services/webhook-worker.js`, called from `tick`). - `checkWebhookTarget` runs again at send time (`:239`). A refusal that is not a DNS failure releases the lease, disables the subscription with the guard's sentence and leaves the delivery pending for an administrator. A name that does not resolve yet is a failure. - The send (`:271`): `postWebhook` (`:72`) POSTs the stored bytes with `Content-Type`, `Content-Length`, `X-Codex-Signature-256` (`signBody`, `:60`, HMAC-SHA256 over the bytes, keyed by the UTF-8 secret), `X-Codex-Event` and `X-Codex-Delivery`. It uses `node:https` or `node:http` with `lookup: pinnedLookup(addresses)` from the check that just ran, a 10-second `AbortSignal.timeout` (reported as `ETIMEDOUT`), and never reads or follows the response. **Outcomes** (every write after the lease is `... AND leased_by = ?`; a write that touches no row is counted `stale` and changes nothing else, so a lapsed lease writes nothing): | Answer | Delivery | Subscription | |---|---|---| | any 2xx | `delivered`, `delivered_at`, `last_status`, lease released | `consecutive_failures = 0`, `paused_until = NULL`, only when no pause is live (`RESET_SQL`), so a slow success cannot clear another worker's backoff | | 410 (`:240`) | pending, `last_status = 410`, lease released | `enabled = FALSE`, `disabled_reason = 'receiver answered 410'` | | 422 (`:246`) | `dead`, `last_status = 422` | untouched | | anything else (3xx, 5xx, a timeout, a connection error, a missing secret, a DNS failure) (`:250`) | `attempts + 1`, `last_status`, `last_error` (at most 255 code points, never the body), lease released | paused until `UTC_TIMESTAMP(3) + LEAST(10 * 2^consecutive_failures, 3600)` seconds, `consecutive_failures + 1` (`BACKOFF_SQL`) | A 3xx is a failure with `last_error` `HTTP 302 (redirects are not followed)`: `node:https` does not follow redirects. The backoff reads the count before the increment, so the first failure pauses 10 s, then 20, 40, and so on, capped at one hour. **Clock.** Every stored or compared time the worker touches is `UTC_TIMESTAMP(3)`, matching the UTC `occurred_at`. `occurred_at` is written from the application clock, the same instant the envelope carries, and the 72-hour test compares it with the database clock, so the two clocks can differ only by the application-to-database skew, which matters only at the 72-hour edge. The prune's `created_at` is a `TIMESTAMP`, which MySQL converts to the session zone on comparison, so it uses `NOW()`; `UTC_TIMESTAMP` there would be wrong. **Logging.** `startWebhookWorker` counts a tick that is still running when the next is due and logs the count at most once a minute; a tick that throws is logged. **Prune** (`pruneWebhookRows`, `:370`, run daily from `server.js`, `pruneWebhooks`). Deletes `delivered` and `dead` deliveries older than 30 days by `created_at`, then events older than 30 days that no delivery references. **Admin.** `GET /api/admin/webhooks/:id/deliveries` (the last 50) and `POST /api/admin/webhooks/deliveries/:id/redeliver` (a `dead` delivery back to `pending` with `attempts = 0`; 409 for any other status, and for an event older than 72 hours) are in `routes/webhooks.js`, documented in `docs/api/admin.md`. **Console (W6-CDX-15).** The Webhooks panel is `WebhooksPanel` in `src/components/WebhooksPanel.jsx`, mounted from `AdminPage` as its `webhooks` nav entry. It reads `GET /api/admin/webhooks` and loads the deliveries of every stalled subscription (`consecutive_failures > 0`) up front, because a stalled card states the last failure. A subscription is Stalled while it is enabled and `consecutive_failures > 0`, Off when disabled (the reason is shown), otherwise Delivering. The secret appears only in the reveal box returned by create or rotate; the panel never fetches it. Enable, disable, rotate, delete and redeliver all go through `ConfirmDialog`. A dead delivery whose event is `expired` (the server's own 72-hour test, the one the redeliver route answers 409 by) offers no Redeliver. The console reads the UTC columns as text and the TIMESTAMP columns as epoch seconds, so the driver's local-time parse never touches them (see `docs/maps/open-questions.md` C9). Tests: `tests/services/webhook-worker.test.js` (the three signature vectors, and each outcome, the backoff SQL, the 72-hour expiry, the budgets, the lapsed-lease write count, the send-time guard and the timer) and `tests/integration/webhooks-worker.test.js` against a real receiver on 127.0.0.1: the body is byte-identical to the outbox row and its signature verifies; event 3 waits for event 2; two workers on one database send each delivery once; a lapsed lease writes nothing after another worker delivers; 410, 422, 302 and a timeout; the backoff gaps read back from `paused_until`; the 72-hour dead-letter; a name that resolves to a private address is refused, and the connect goes to the approved address; the prune; and the save latency with a subscriber that never answers. --- ### Outbound events: deferred, not built These were out of scope for W6-CDX-12 to W6-CDX-16 and are not planned in Wave 6. Each would be its own spec: - **GitHub pull writes** (`github.js`, the three pull-request writes): emitting them means adding `logActivity`, which emails watchers. - **Squad and workspace deletes:** they detach archives and delete no document, so they emit nothing today. - **Comment and membership events:** they carry people's data, so they would be a later additive event type. - **Cross-archive moves:** nothing user-reachable does one today. - **Encrypting admin-API subscription secrets at rest:** they are stored in MySQL, write-only through the API, and a database dump lets its holder forge events to that one receiver. (The environment-declared subscription that avoided this was removed in 0.15.0.) - **Delivery to more than one receiver per instance, and moving an instance between workspaces:** the provisioning era. ## 3. Watches (`routes/watches.js`) Four routes, all `requireAuth`. `resource_type` is allow-listed to `log` and `archive` (`watches.js:25`). - `GET /api/watches` lists with a resolved `resource_name` via correlated subqueries (`watches.js:37-51`). - `GET /api/watches/:type/:id` returns `{ watching, source }`. - `POST /api/watches` **checks read access first** through `userCanReadResource` (`watches.js:27-32`, `watches.js:84-87`), then upserts with `source = 'manual'`. Idempotent. - `DELETE /api/watches/:type/:id` is idempotent and does not check access, which is correct: removing your own watch needs no permission. A manual watch overwrites an auto one (`ON DUPLICATE KEY UPDATE source = 'manual'`, `watches.js:90-93`), but an auto-watch never overwrites a manual one because it uses `INSERT IGNORE`. Unwatching does not prevent re-auto-watching: edit the document again and `auto_edit` puts the watch straight back. ## 4. `createNotification` (`services/notifications.js:86-159`) The single funnel. Everything that alerts a user goes through it. ```js createNotification({ recipientId, actorId, type, title, body, linkUrl, resourceType, resourceId, metadata, emailData }) ``` In order: 1. Require `recipientId`, `type`, `title`; else `null` (`services/notifications.js:100`). 2. **Self-suppression**: `actorId === recipientId` returns `null` (`services/notifications.js:101`). 3. **Coalescing**: if `resourceType` and `resourceId` are both set and a row with the same `(user_id, type, resource_type, resource_id)` was created within **60 seconds**, return `null` (`services/notifications.js:20`, `services/notifications.js:104-114`). A notification without a resource is never coalesced. 4. Sanitise: title goes through `sanitizeHtml` then a tag strip and a 255-char cap; body through `sanitizeHtml` and a 2000-char cap (`services/notifications.js:119-120`). This is defence in depth, since React and the email templates already escape. 5. Insert, then **fire-and-forget** the WS push (`services/notifications.js:147-151`) and the email (`services/notifications.js:154-156`). Neither is awaited, so a dead SMTP server cannot slow a save. **Two independent coalescing windows exist and are easy to conflate:** activity's is 5 minutes and applies only to `log.update` (`routes/helpers/activity.js:23`); notifications' is 60 seconds and applies to every type with a resource (`services/notifications.js:20`). ### Types and email preferences Six types, defined by `DEFAULT_EMAIL_PREFS` (`services/notifications.js:27-34`): | Type | Email by default | Emitted from | |---|---|---| | `mention` | yes | `mentions.js:142`, `comments.js:196` | | `comment_on_my_doc` | yes | `comments.js:216` | | `watched_comment` | yes | `routes/helpers/activity.js:209` | | `watched_publish` | yes | `routes/helpers/activity.js:209` | | `watched_log_update` | **no** | `routes/helpers/activity.js:209` | | `squad_invite` | yes (but see below) | `squads.js:428` | Resolution (`services/notifications.js:36-46`): look up `email_` in the user's `users.notification_prefs` JSON; if that key is absent, fall back to the default. `setPrefs` (`services/notifications.js:273-286`) whitelists to exactly the six known keys and coerces to boolean, so a client cannot inject arbitrary JSON. **`squad_invite` has a preference key but no email template.** `builders` in `services/email-templates.js:63` defines only `mention`, `comment_on_my_doc`, `watched_log_update`, `watched_publish`, `watched_comment`. `buildNotificationEmail` returns `null` for anything else and `deliverEmail` bails (`services/notifications.js:170`). Squad invites do still send email, but through a direct `sendEmail` call in `routes/squads.js`, outside the funnel. That send now calls `getPrefs()` and honours `email_squad_invite` itself; until 2026-08-09 it consulted nothing, so the preference toggle had no effect (B7 in [open-questions.md](open-questions.md)). **Any future send that bypasses the funnel inherits the same obligation.** ### Reads `listForUser` (`services/notifications.js:225-251`) is cursor-paginated on `created_at < before`, limit clamped to 1..100, joined to `users` for actor name and avatar. `getUnreadCount` (`services/notifications.js:212`) backs the badge. `markRead` (`services/notifications.js:184-195`) is scoped `AND user_id = ?` so one user cannot mark another's notification read, and it pushes `{type:'read', id}` to the owner's tabs. ## 5. Mentions (`routes/helpers/mentions.js`) Mentions are `` nodes emitted by the Tiptap `Mention` extension (`src/extensions/Mention.jsx`). - `extractMentions(html)` (`mentions.js:40-51`) regex-matches the attribute after stripping `
`, ``, `