# Architecture A developer-facing description of how the pieces fit together. It used to live in `CLAUDE.md`, where it was loaded on every change; it moved here because most changes touch none of it. `CLAUDE.md` keeps the **invariants** that must not be broken and points here for the explanation. Update this file when you change a subsystem it describes. **Everything ships as one Docker image, one container** — no multi-container orchestration to reason about. The container needs its own routable LAN IP so WebRTC has no NAT/ICE problems (MediaMTX advertises that IP to browsers). There are two supported ways to give it one: **host networking** (the README quick-start default — the container shares the host's IP), or an **ipvlan network with a dedicated IP**. The Unraid prod + staging deployments both use the latter (both on `br0.10` — see Branching and deploy pipeline in `CLAUDE.md`); the host-networking path is what an out-of-the-box `docker run` uses. ## The video pipeline (the core thing to understand before touching camera code) RTSP cannot be played directly in a browser, and many IP cameras send audio as G711 (a codec HLS can't carry at all — WebRTC can). This drives a specific pipeline, in order: 1. **FFmpeg** (`backend/src/lib/transcoder.js`, one process per camera) pulls each camera's RTSP feed, copies video untouched, and produces **two audio tracks**: track 0 copied as-is (for WebRTC, which can't decode AAC), track 1 transcoded to AAC (for HLS, which can't carry the original codec). It publishes the result into MediaMTX via RTSP on `127.0.0.1:8554`. 2. **MediaMTX** (`backend/src/lib/mediamtxProcess.js`, spawned as a child process, config baked into the image at `mediamtx/mediamtx.yml`) re-publishes that stream as WebRTC (WHEP) and HLS. Each camera's path has no pull source configured — it's publisher-only; FFmpeg pushes into it rather than MediaMTX pulling the camera directly (`backend/src/lib/mediamtx.js`). 3. **Backend** (`backend/src/index.js`) reverse-proxies `/live` (WHEP) and `/hls` to MediaMTX's local ports, and serves the built frontend + REST API on the single public port (4000). 4. **Frontend** picks WebRTC (`WhepPlayer.jsx`, "Low latency") or HLS (`HlsPlayer.jsx`, "Compatibility") per camera tile, toggled by the user. ## Child process environment (`backend/src/lib/childEnv.js`, #553) Camera streams are untrusted input, and ffmpeg/ffprobe parse whatever a (possibly compromised) camera sends. So **no child is handed the backend's environment**. Every ffmpeg, ffprobe and MediaMTX process is started through `spawnChild()`, which builds the environment from an allowlist; `JWT_SECRET`, `FIREBASE_CREDENTIALS`, `DATA_DIR`, `PUBLIC_HOST` and everything else the operator set are not passed to a child. This narrows what a child *inherits*; it does not stop a child reading the backend's own environment (see "What this does not do" below). | Variable | ffmpeg / ffprobe | MediaMTX | Why | |---|---|---|---| | `PATH` | yes | yes | finds the binary and what it execs | | `TZ` | yes | yes | segment file names and log times use the container's zone (the app's own times follow Settings → General) | | `LANG`, `LC_ALL` | yes | yes | locale only | | `MTX_*` (operator's own) | **no** | yes | MediaMTX's documented override mechanism, e.g. `MTX_WEBRTCLOCALUDPADDRESS` | | `MTX_WEBRTCADDITIONALHOSTS` | no | yes | computed by `mediamtxProcess.js` at every launch from `PUBLIC_HOST` and the detected host IPs; it replaces an inherited value whenever that list is non-empty | | `MTX_WEBRTCLOCALUDPADDRESS` | no | yes | the operator's own value when they set one: it wins, and `WEBRTC_UDP_PORT` is then ignored with a warning; otherwise `:` from a valid `WEBRTC_UDP_PORT`; otherwise unset, so MediaMTX keeps `:8189` from `mediamtx.yml` | | `HOME`, `LIBVA_*`, `NVIDIA_*`, `HTTP(S)_PROXY`, `LD_LIBRARY_PATH`, `RTSP_*` | no | no | nothing in the app documents or sets them (ffmpeg runs with software decoding; cameras are reached directly on the LAN; `RTSP_*` is MediaMTX's legacy override prefix, which it still reads: use the `MTX_*` spelling) | A name that is unset in the backend stays unset in the child. A caller-supplied `env` option is ignored. **To add a variable:** add the name to `ALLOWED_NAMES` in `childEnv.js` with a comment saying which setup needs it, add a row to the table above, and update `backend/test/child-env.test.js` (it pins the list on purpose). For a per-launch computed value (MediaMTX's) pass it as `extra` instead. **To add a new spawn site:** call `spawnChild()`, never `child_process` directly; the source test in `child-env.test.js` fails on any other import of it under `backend/src`. **What the allowlist does not do:** - **It is not a sandbox.** It narrows what a child *inherits*: what a bug that discloses the child's own memory or environment (in a demuxer, a crash report or log output) could expose, and what a command run by a MediaMTX hook inherits from MediaMTX. Taken alone it does not stop a child reading the backend's own environment through the process table (`/proc` on Linux), `JWT_SECRET` included, or the files on disk. - **The separation that does is [media isolation](#media-isolation-the-layout-and-the-fence-553-phases-2b-and-4-on-by-default)** (#553), which is **on by default**: the children run as a second operating-system user (uid 1700, no capabilities), so a child a camera took over cannot read the database, `.jwt_secret` or the app's `/proc//environ`. (An earlier version of this page said a separate user was considered and not done; the owner reversed that on 2026-10-25.) With isolation **off** (`NIGHTLIGHT_MEDIA_ISOLATION=0`, or a host where it cannot start) the children run as the backend's user and everything in the first bullet applies in full. The limits of isolation are listed at the end of that section. Keep cameras on their own VLAN, as [docs/cameras.md](cameras.md) advises. ## ONVIF, PTZ, and camera credentials (`backend/src/lib/onvif.js`, `rtspProbe.js`) Cameras are added/edited by **components** (IP / port / path / username / password), not a raw RTSP URL. The route layer (`routes/cameras.js`) assembles those into the stored `rtsp_url` that the transcoder uses, and splits an existing `rtsp_url` back into fields for the edit form. **The password is never returned to the client** — GET responses carry the address in fields plus a credential-free display URL and a `rtsp_has_password` flag; on edit, a blank password means "keep the existing one." Keep that invariant. - **`lib/onvif.js`** — `onvif@0.8.1` client, deliberately resilient to minimal ONVIF servers (the sonoff-hack Sonoff faults on `GetCapabilities`/`GetServices`): it tries the normal connect, then falls back to hitting the media service directly at known paths, and reconstructs the RTSP URL from the connect host + user's creds + the discovered path (never the host/creds the camera returns — often empty/bogus). `probeOnvifCamera()` powers add-by-IP (`POST /api/cameras/onvif-probe`) and also reports two-way-audio (`getAudioOutput- Configurations`) and PTZ capability. **Discovery is add-by-IP, not multicast** — WS-Discovery doesn't cross VLANs. - **PTZ** — `ptzNudge()` does start → hold `PTZ_NUDGE_MS` → stop in one call, so each press moves a fixed distance regardless of tap/network timing (`POST /api/cameras/:id/ptz/nudge`; the tile sends one per tap and repeats while held). ONVIF creds for control are the same single credential set stored with the camera; capability/creds/profile-token are captured at add time (`ptz_supported`, `onvif_*` columns) so control reconnects without re-querying. - **`lib/rtspProbe.js`** — `validateRtspStream()` (ffprobe over TCP) runs on add/edit so a camera that can't be reached (bad creds/path/IP) isn't silently saved; the UI can override with `force` for a camera that's just momentarily offline. ## Self-healing / reconciliation (`backend/src/index.js`) MediaMTX only learns about a camera when it's added/edited through the API, or via periodic reconciliation. Three independent mechanisms keep the pipeline alive without manual restarts: - `reconcileCameraPaths()` runs at startup and every 5 minutes: re-creates any MediaMTX path that's missing/misconfigured and restarts any camera whose transcoder isn't running. It only *writes* to MediaMTX when something is actually wrong, since every write forces a path reload that disconnects the current publisher. - The same pass decides each camera's pixel-diff motion leg through `reconcileMotionLeg()` (`lib/motionDetector.js`; it lives there, not inline, because `index.js` spawns MediaMTX at import so no test can import it). A frame-diff camera with alerts on runs around the clock; an *activity-only* camera (alerts off, or MQTT/ONVIF motion, assigned to a child) runs only while `childSamplingActiveNow` (`lib/sleepAnalysis.js`) is open: `ONSET_LOOKBEHIND_MS` before the child's window to `WAKE_LOOKAHEAD_MS` (+ the 5-minute reconcile margin) after it, the inference's own span, so the morning departure is observed (#353). The lead and tail are absolute milliseconds, so a DST night gives 3 real hours. - A watchdog (15s interval) tracks how long each camera's MediaMTX path has been "not ready" and force-restarts that camera's transcoder past a 30s threshold — a second, independent layer of defense beyond FFmpeg's own stream error handling. - A MediaMTX that stays alive but stops answering its API is restarted too, after 3 minutes of silence (`lib/mediamtxApiWatchdog.js`, #479: a probe every 30 s, a doubling backoff of 3, 6, 12, 24, then 30 minutes, back to 3 after 5 minutes of answers; local MediaMTX only; Camera history gets one row per enabled camera, and the camera paths are reconciled as soon as the new MediaMTX is up). - FFmpeg and MediaMTX processes both auto-restart on unexpected exit (`transcoder.js`, `mediamtxProcess.js`), and `transcoder.js` also watches for a specific known bad-camera symptom ("DTS discontinuity") and proactively restarts rather than let a session run poisoned. **One start at a time per camera (#478).** The reconcile pass, the camera watchdog, the audio watchdog and the camera routes (add, an edit with a new address, enable, **Restart stream**, a failed delete putting the camera back) all start a camera's transcoder through `startTranscoder` (`lib/transcoder.js`), and so does the optional sub-stream, under its own key. Starts for one camera run one after another, each with its own arguments: two that overlapped used to both launch FFmpeg and leave one running unseen (two publishers on one path knock each other off it). Different cameras never wait for each other. What differs from the neighbouring mechanisms: - A request identical to one still **waiting** its turn (same address, path and name, no stop in between) shares it, so pressing **Restart stream** repeatedly queues one restart, not one per press. It never joins the start already running, nor merges different addresses: an edit with a new address gets a start of its own with that address. The narrow stale-row limit (#780) is at the end of this list. - A stop (switching the camera off, deleting it, shutting down) reaches a start that is queued or part-way through: it launches nothing and finishes quietly, logging `start abandoned …` at INFO. The stop itself never waits in the queue, which keeps shutdown inside its 9 s bound; shutdown does wait for a stop that is already in progress, in parallel with the others. The other way round, a start waits for a stop of the same camera that is still in progress, for at most that stop's 3 s fallback: until the old process exits, or until SIGKILL is sent to one that ignored SIGTERM. After the SIGKILL the old process is gone within a moment, not instantly, so the new FFmpeg can start while it is still exiting; a process that survives SIGKILL (stuck in the kernel) is not waited for at all. Deliberately: waiting on, or giving up the start for, a process that cannot be killed would leave the camera down for good. - While a start is queued or running, the camera counts as running for the reconcile pass, which therefore does not queue another one behind it. - Known cost: a request now waits for every start ahead of it to finish. Each start is bounded (the old process's stop 3 s, the audio probe 9 s, the MediaMTX checks 5 s and 10 s: about 27 s at the very worst), so an edit with a new address that arrives while one other start of the same camera is running, with MediaMTX timing out throughout, can take about 70 s before it answers (its own checks included; worked out from those bounds, not measured), under Cloudflare's 100 s proxy limit. With more starts ahead of it (a watchdog's and the reconcile pass's at once) it can take longer than that. - An edit with a new address first registers the MediaMTX path, then saves the camera row, then queues the transcoder start (#780). An ONVIF backchannel re-probe usually finishes in under 1 s, but a hanging camera can delay that restart by about 25 s. A failed path registration still answers 502 without saving, and a failed save, including a stale child assignment that answers 409, does not start the transcoder with the new address. Registering the MediaMTX path can still briefly interrupt its publisher. A camera disabled or deleted during the probe cannot be started by the edit after it resumes. If the transcoder start rejects, the route answers 500 after saving the whole edited row. It skips the subsequent sub-stream, detector and clip-ring steps and `subscribeAllCameraTopics`; a changed MQTT topic is not subscribed until the MQTT client reconnects. Reconcile and the watchdogs can restart the camera from the saved row because `isRunning` is false after a rejected start. - During that probe, a second camera save can finish first (#787). The waiting save now preserves each column it did not change from the latest row; its own changed columns win, including a changed address or talk/ONVIF credentials. A sub-stream path entered against an unchanged main address follows a newer main address. Deletion during the probe or transcoder start answers 404. A request re-sending a column's old value cannot be distinguished from leaving it untouched, so it preserves a newer value in that column. Credential pairs can still be mixed across overlapping saves: when one save leaves a password blank, its talk or ONVIF username can come from one save and the password from the other because the merge is per column. - Known limit (#780): the transcoder never reads the database. A watchdog or reconcile pass that fetched the old camera row before the edit's write can queue its stale-address start after the edit's start; it then runs last. The window is the other reader's own wait after fetching the row, not just the time between the edit's write and queue: the camera watchdog can await MediaMTX status for up to 5 s, and the audio watchdog can await that status plus a 6 s audio-flow probe. Reconcile reads all camera rows before looping, though its `isRunning` check usually prevents a duplicate start. This window remains open for several seconds in a stalled case (not measured). Fully closing it would require the transcoder to re-read the address when its queued start runs; that is outside this fix. When editing this area, preserve the "only write when actually broken" invariant — an unconditional reconcile-on-every-tick would cause constant disconnects. **Reading readiness without reconciling (`backend/src/lib/recordingReadiness.js`, #394).** The camera settings page's read-only "Recording and alerts" panel asks `GET /api/cameras/:id/recording-readiness` (admin only) whether the ring buffer, the detectors and the alert window are up *right now*. The route only reads: it asks `clipCapture`, `motionDetector`, `onvifMotion`, `soundDetector` and `detectSchedule` for their own answers and never starts, stops or reconciles anything, so it cannot cause the path reloads the invariant above is about. It returns booleans and fixed words only. `alerting_now` is false for a silenced camera, one outside its schedule, and a disabled one (which runs no detector); `alert_pause_reason` is `null` while alerts fire, else `"silenced"`, `"schedule"` or `"disabled"` (checked in that order of precedence: disabled, silenced, schedule), computed from the same instant as `alerting_now`. It is an added field: `alerting_now` is unchanged in meaning for existing clients, and the panel falls back to guessing the reason from the camera row only if the field is absent. Its one restated condition, manual recording availability, mirrors `can_record` in `routes/cameras.js`, and `recording-readiness.test.js` fails if the two disagree. ## Auth (`backend/src/middleware/auth.js`, `backend/src/routes/auth.js`) JWT-based, but a valid JWT alone isn't sufficient — every request also checks a `sessions` row in SQLite still exists (`backend/src/db.js`). This is what makes "sign out this device" and "delete this caregiver" take effect immediately rather than waiting for token expiry. The same lookup checks the session's age: a session lasts `SESSION_TTL_DAYS` (default 30, range 7-30) from sign-in, judged by `sessions.created_at`, not by the token's `exp`, so lowering the setting ends older sessions at their next request even though their tokens say 30 days (#562). One helper, `sessionAgeCondition` in `middleware/auth.js`, supplies that condition to every place that asks: the request check (and so media tokens and the talk socket's `verifyToken`), push recipients (`lib/push.js`), the WebRTC live-view sweep (`isLiveSessionId`), the daily purge (`purgeExpiredSessions`) and the two session lists. Two auth middlewares exist: `requireAuth` (Bearer header) and `requireAuthQueryOrHeader` (also accepts `?token=`, needed because Safari's native `