# Live performance budgets

The picture is the product. Chrome, scopes, and SET traffic ride around it.
Target hardware is mid/high-end: iPhone 13-class and newer; Android API 33+
with ≥4 GB RAM (the Kyant glass gate). Prove **physical** after any live-path
change.

Numbers that already have a home stay there. This file is the SLO index and the
rules that are not in those homes. Changing a budget is a code + docs change in
the same PR.

## Budgets

| Surface | Budget | Owner |
| --- | --- | --- |
| Live picture | Present at the camera’s live rate. Typical Pocket/Nano SoftAP is ~25 fps 720p. Do not pace decode at 30 fps. A 4K 50p body may present 50 Hz 720p. Skip duplicate timestamps; latest-wins if a LUT bake is busy. Runtime grade cap 1440 px (`FeedPresentPolicy.maxWorkingWidth`). | [`live-session.md`](live-session.md), `FeedPresentPolicy` |
| Playback LUT | 720p proxy with the official cube stays at a usable rate on iPhone 13-class (picture first; same class as LUT-off). Native 420 IOSurface → GPU cube at `maxWorkingWidth`. Pull clock follows the display (24–120 Hz); `hasNewPixelBuffer` gates the cube — do not cap the display link at 24. No `AVVideoComposition` for preview. Scope tap stays off the present thread. Android playback is the live GLES session (OES → cube); no TextureView `getBitmap`. | `PlaybackFeedSession`, `PlaybackFeedView` |
| Watcher relay (iOS) | Same camera Wi-Fi for host and watchers; no peer-to-peer fallback. One encode; two admitted source frames; two video sends per authorized watcher, separate from control traffic; one pending state send at 5 Hz. Encode and fan-out off MainActor. Forced relay keyframes ≤1 Hz. | [`watcher-relay.md`](watcher-relay.md) |
| Window ACK | pktType `0x04` at **40 Hz**, three groups: video `0x02` seq, ackedData `0x03` seq, telemetry extra | [`live-session.md`](live-session.md) |
| Live enable | **Enable-once.** Further enables follow the watchdog only | `AGENTS.md`, [`feed-watchdog.md`](feed-watchdog.md) |
| Stall / recover | 2 s UDP silence is a stall; 8 s GOP grace after `0x09/0xa8`; 4 s after an AF-C SET; 5 s between enables; 60 s UDP rebuild backoff. Encoder pause permits two enables, then one rebuild that negotiates a fresh handshake. Full-session automatic recovery has a separate 180 s total cap | `FeedWatchdog`, [`feed-watchdog.md`](feed-watchdog.md), `SessionRecoveryPolicy` |
| HUD chrome | 5 Hz (`LiveChromeThrottle.statusInterval` = 0.2 s). REC, format, color, zoom, and the other `isImmediate` fields bypass | `LiveChromeThrottle` |
| Scope tap | 25 Hz with 1–2 scopes, 10 Hz with 3+ (`PocketScopeSampler`). 200-wide downsample (213×120 on 720p SoftAP). Thermal ×3 serious / ×5 critical. A 50 Hz proxy still skips. Assists-off is one blit — no 1280×720 histogram or readback per frame | [`ANDROID.md`](../ANDROID.md) I/O; iOS present path matches the rate |
| HUD glass sample | PixelCopy ~20 Hz when Kyant cannot sample the SurfaceView | [`ANDROID.md`](../ANDROID.md) |
| Zoom pinch | Distinct lens ticks at 20 Hz, no ACK wait | [`PARITY.md`](PARITY.md) |
| Gimbal stick | `0x04/0x01` notify at **25 Hz** on the UDP ACK queue while held; one rest packet on lift. Not MainActor `sendUntracked` (that starved window ACK). AirPods IMU samples ~100 Hz off main; a 25 Hz pump publishes native targets to a latest-only mailbox. Native wire emission has a 40 ms minimum interval on the 25 ms ACK timer (typically 20 Hz). Duplicate targets are suppressed; a not-ready socket cannot accumulate a backlog. HUD at the 5 Hz chrome budget. Head-track yaw/pitch rings (head + gimbal arrows) follow the 25 Hz pump while Head Tracking is on (not the 5 Hz HUD). Motion Control waypoint letters follow the 25 Hz stick budget — not 60 Hz `TimelineView.animation` / `withFrameNanos` on the live canvas (that starved ingest and flashed Reconnecting). Motion Control session progress is 5 Hz; no debug overlay is drawn. Timed-path ticks use monotonic elapsed time; a gap over 120 ms or attitude receipt age over 300 ms aborts the take. Physical precision remains unqualified ([Motion Control takes](programmed-moves.md)). | `GimbalStick.streamInterval`, iOS `DatalinkDriver.tickGimbalStick`, `HeadphoneMotionBridge` |
| Gimbal mode readback | At most 1 Hz tilt/speed GET, driven by existing attitude receipts; no extra timer | `GimbalParamPoll` |
| Battery | Sticky `ACTION_BATTERY_CHANGED` (Android); no 1 Hz poll | [`ANDROID.md`](../ANDROID.md) |
| Watch preview | Ack-paced JPEG, drop-stale, **3** outstanding across wrist wake/resume (fps ≈ depth/RTT; one in flight was ~12 fps). Encode on a detached queue so the three slots overlap. Identity JPEG is `VTCreateCGImageFromCVPixelBuffer` (same family as the phone layer — a DeviceRGB CI bake was a Rec.709 contrast shift). LUT cubes stay unmanaged. Adaptive 320 / 416 / 512 px. A paired, installed companion requests the existing VT decoder even with AF-S and assists off; wrist sleep stops JPEG work without restarting decode. Rec/tally uses `updateApplicationContext` when not reachable. | `WatchRelay` |

Programmed takes run on the background transport scheduler, using complete-frame
attitude receipts before the UI hop. Smoothed paths write 20 Hz native targets
directly at monotonic deadlines under exclusive ownership; UI progress is 5 Hz. Marker/curve projection uses the existing 25 Hz overlay timeline;
measured motion prediction is display-only. No new ACK timer or video enable
is introduced. Native stream targets are not logged individually at 20 Hz.

## Image anchoring experiment

A temporary iPhone 16 Pro Max benchmark of Vision homography registration at
320 pixels wide and 5 Hz measured 30 jobs: mean 42.6 ms, p95 68.9 ms, maximum
800.3 ms including cold start. A concurrent movement test interrupted on timing.
This does not establish causation, but the cost is unsuitable for enabling by
default. The probe was removed; production waypoint projection performs no
image registration. Background transport checks without the probe sustained
approximately 25 fps through normal and fast smoothed takes.

## Threading

UDP receive re-arms on the network queue, not after a main-actor hop. A busy HUD
must not stop the socket.

Depacketize and scope accumulation stay off the UI thread. Compose/SwiftUI
invalidates at the HUD budget, not per video packet.

Keep the last decoded frame through recover, and across extra-mirror
(3 frames / 120 ms) so TT180 does not X-flip the on-screen picture in
place. Empty samples and
`flushAndRemoveImage` before the next picture are a black well, not a stall.
A 2 s gap with no present is a **freeze** (`FeedPresentPolicy.isFrozen`) —
UDP still alive means do not send `0x09/0xa8`. Skip duplicate timestamps
on the GPU path; if a LUT bake is still in flight, drop to the latest sample.
Metal present is latest-wins with **one drawable in flight**
(`FeedPresentPolicy.maxInFlightMetalPresents`). Do not block MainActor on
`nextDrawable` — LUT 50/50 plus PEAK / FALSE / ZEBRA pipelined baker
completions and froze ingest until force-quit (#218). Acquire on a dedicated
serial worker; keep one reservation through GPU completion. Skip new acquisitions
while busy and adopt the newest coherent bake when the slot becomes available.
Only successful GPU completion advances the Metal presentation clock; it does
not measure physical display scanout. Keep source time, bake identity and layer
style together through completion, including resize and LUT replacement.
Runtime grade stays at `FeedPresentPolicy.maxWorkingWidth` (1440 px) on the
720p proxy — do not memcpy a 4K original to apply a cube. Live LUT replace
hides the HEVC layer once Metal owns the picture. Playback does the same:
`AVPlayerItemVideoOutput` pulls Metal-compatible 420 (not 32BGRA),
`LiveAssistEngine` grades on a pull queue, and `CIFeedView` is a **sibling**
of `AVPlayerLayer` under a plain UIView (same stacking as live
`DisplayLayerView`). Nesting `CAMetalLayer` inside `AVPlayerLayer` presents
LUT replace as a black plate; overlay stripes still showed through. Once
Metal owns the cube, hide the player — a second HEVC present under the grade
is the hitch. Overlay (PEAK / FALSE / ZEBRA) keeps the identity layer.
Export bake stays `AVVideoComposition`. The cube bakes at feed resolution and
bilinear-fits the panel; Lanczos / MetalFX stay opt-in Quality/AI. Android
live matches that order (Vulkan / GLES cube 720p, then stretch Rec.709). The baker
pipelines the next cube while the GPU finishes the last. Next/prev clip does
not recreate the playback `CAMetalLayer` — a slide `.id` rebuild stole the
session and left LUT off until the chip was cycled. Playback Auto reads
2 MiB from the **original** take's tail once per item
(`ClipColorProfile.fileTailBytes`) — never the LRF/XRF sidecar, not the 4K
`mdat`, not per frame. When the original is not cached, that tail is an HTTP
Range.

Offscreen / hidden processed feeds set `isEnabled = false` (no Metal/GLES).
Replace-grade unhides the drawable **before** `nextDrawable`; overlay stays
hidden until the transparent bake lands. `nextDrawable` must not block the
UI thread (`allowsNextDrawableTimeout` on iOS).

## Hardware

Kyant liquid glass: API 33+ and ≥4 GB, not `isLowRamDevice`. FULL stays FULL —
no frame-budget demote. Older / low-RAM devices stay on solid frost.

Decoder prefers hardware (`c2.qti` / Exynos, VideoToolbox) over a software
fallback. GLES `FeedEffectsGlProgram` is the Android decode fallback when
Vulkan cannot init.

`WIFI_MODE_FULL_LOW_LATENCY` stays on while live.

## When this pointer fires

A live-path, HUD, scope, ACK, playback-LUT, or smoothness change. After the
edit, the row you touched still matches its owner, and the picture is
**physical** at the camera’s live rate on mid/high-end hardware. Playback LUT
on a 720p proxy is the same bar.