# Architecture CueTake is an agent-facing local recording runtime. Settled decisions live in [docs/adr](adr/). ## Pipeline Agent + skill → business scenario → CLI → Playwright context → Chromium tab capture → a VP9 Source per Tab + Timeline → Render in headless Chromium (WebCodecs decode, WebGL compose, H.264 encode) → Final + verification. The scenario controls story and timing. All pointer/camera coordinates are CSS pixels; the source uses device pixels. Device scale does not change the page layout. Render samples the latest Source frame at or before each output moment's Source time, turning the variable-rate Source into a constant-rate Final and holding the last frame to the measured Take duration. The Edit's fast-forwards map Final time to Source time (`time-map.js`, ADR 0007); without one the two are the same. The Source holds only page pixels, since tab capture never shows the pointer; `probe.js` records the cursor shape the page asks for. The probe runs in every frame, iframes that load during Capture included, and sends its records to Node as they happen, so a frame removed mid-Take keeps what it saw. A frame knows only its own coordinates, and a cross-origin one cannot ask where it sits, so each document measures the iframes it holds every animation frame (where their content starts, how much they are scaled) and Node maps each record up through its parents. Screen coordinates were tried and rejected: Chromium leaves fixed and sticky positioning out of a scrolled iframe's screen position. The Director aims with the same measure: Playwright's own box for an element in a cross-origin iframe starts at the iframe's outer edge and is unscaled, so it misses the iframe's border, padding and scale. The Director passes Playwright a press position that lands where its glide ended, and checks what the press hits itself wherever Playwright's box is off. Render draws the Cursor from the Timeline's pointer path, in that shape once the page has asked for it the Edit's `cursor.settle` or at a press (a shape asked for only in passing keeps the one before), pressed while a button is held, and any click effect at each press, as WebGL sprites under the same camera, so a Take can be restyled without recording again (ADR 0002). Motion blur (`shutter.js`) exposes each output frame over the Edit's `output.shutter`, a fraction of the frame: while the camera moves, the window and what moves with it are composed under the camera at several moments of the exposure and averaged in a half-float buffer, and the Cursor likewise at several points of its path; the Background stays still. Each moment needs no extra Source frame, since the camera track and the pointer path are known at any time, so motion inside the page is not blurred. The Background, window effects, chrome, cursor, click effect and Subtitles are agent-written shaders, drawing code or images, with built-in defaults in `src/styles/` (ADR 0005); the render page has no network. A Take shows one Tab or more (ADR 0008). `follow-tabs.js` follows each page a Tab opens (Playwright's `popup` event) and which Tab is in front; one recorder page records each Tab as its own stream until it closes, and the probe runs in each. The recorder has a window of its own: on macOS `getDisplayMedia` starts only from the page in front, so the recorder comes in front for each start, and as a tab of the Tabs' window it would hide the tab being started, which then paints no first frame for a second or more, and give that window a tab strip that leaves each tab 87 CSS px short of the viewport, which tab capture records. Render moves every Tab's input into the window's CSS pixels by where its popup sits (`tabs.js`), shows a Tab the size of the viewport in the window and any other as a popup window over its opener, and samples each shown Tab's Source at its own start. A Take of Apps has one Tab, its Desktop (ADR 0009). Its Source shows only the windows of the Take's Apps, in their real order across them, on nothing, and H.264 keeps no alpha, so the helper samples the windows with every frame instead: the window list gives their order and bounds, Accessibility the visible frame of a menu (whose window is larger than the menu, for its shadow), and the frame's alpha each window's corner radius before encoding. Render draws the Desktop's Source once per window showing, clipped to its frame, back to front, so the Background shows between them; only a window over no other gets the window effects, since the Source already shows the shadow a menu or sheet casts on its window. A Mask (ADR 0010) follows an element the Scenario names: the probe in that element's frame measures every element its locator matches each animation frame and sends their boxes when they change, Node places them in top-level CSS pixels like input, and the Director matches the locator again ten times a second so a replaced element is followed. Render draws blur and cover with the page, in its CSS pixels, so they move with the camera and fade with their Tab; a blur reads a mip level of the Source, past what text survives. Spotlights dim the frame around their regions in one pass after the last window, popups included, so several showing at once each stay lit. A Narration is voiced before Capture by a separate speech tool (ADR 0006). The Director holds each said Line for its audio; Render decodes the Lines with Web Audio, mixes them into one mono track (AAC, or Opus where Chromium has no AAC encoder) and draws Subtitles outside the camera. WebCodecs gives no AAC encoder's priming (2112 samples on macOS), so Render measures the encoder's delay on a burst of noise and stamps the packets back by it; the MP4's edit list then skips the priming, and packets that start after the video are left out. ## Modules - `pipeline.js`: scenario validation, artifact lifecycle, render and verification. - `director.js`: timed interactions, scrolling to an element through each container that holds it, focus targets, said Lines and fast-forwarded waits. - `motion.js`: how the Director moves like a person (durations, path, pauses, typing rhythm, how long a scroll takes); pure and seeded. - `recorder.js` / `tab-capture.js`: supported HD capture only, one stream per Tab from one recorder page. - `follow-tabs.js`: the Tabs a Take shows, which is in front, and starting each one's capture. - `app-capture.js` / `helper/`: Capture of a Desktop of macOS Apps (ADR 0009) through `cuetake-capture`, a Swift helper run as a child process over JSON lines, and the `doctor` checks of its grants. The helper starts the Take's Apps and records their windows (ScreenCaptureKit, filtered to those Apps, no pointer) to an H.264 Source and samples each window's frame, layer and corner radius with every frame. - `desktop.js`: which App windows show at each moment, where Render draws them, the warnings about the Desktop, and where on a title bar `d.switchTo` clicks; shared by Node and the render page. - `tabs.js`: where popup windows sit, which Tabs the Final shows at each moment, and moving input between Tabs and the window; shared by Node and the render page. - `cursor.js`: the Cursor's position, press, fade and shape over time, and click-effect timing; shared by Node and the render page. - `camera.js`: camera chains and the crop at any time; shared by Node and the render page. - `shutter.js`: motion blur: each frame's exposure, the moments it averages, and the camera between frames; shared by Node and the render page. - `stage.js`: output size, window and page layout, camera view and brand color; shared by Node and the render page. - `narration.js`: Line problems, subtitle rows and cues, and subtitle files; shared by Node and the render page. - `masks.js`: which Masks show at each moment, where their element's boxes are, and what stops one from being drawn; shared by Node and the render page. - `time-map.js`: the map between Final and Source time that fast-forwards make, and what moves to Final time; shared by Node and the render page. - `styles.js` / `styles/`: built-in styling files, and the file refs an Edit or Look names (copying, serving). - `home.js`: the Home, where Scenarios, Takes and Looks live: `CUETAKE_HOME`, which the skill's launcher sets for a skill installed in a project, else the user Home (ADR 0011). - `look.js`: saving, listing and adopting Looks, in the Home and then the user Home. - `environment.js`: `doctor`, `install-browser`, and removing the browser. - `uninstall.js`: the `uninstall` command; it removes only what CueTake owns, and the user Home's files only when asked. - `render.js` / `render-page.js`: the Render driver and the in-browser compositor, encoder and inspector. - `probe.js`: Timeline recording in every frame of the page, and placing each frame's input in top-level CSS pixels. - `frames.js`: where an iframe's content shows in its parent, where a frame shows on the page, and an element's box on the page through all its iframes; the probe and the Director both measure with it. - `edit.js`: the Edit's defaults, merging, and its first version from a Scenario. - `auth.js`: the `login` command and Auth profile storage. - `cli.js`: the command interface, the only entry point. Source capture is variable frame rate and does not guarantee 60 distinct frames per second. Login and full navigations may happen during preparation; after recording begins, the document must remain alive (SPA changes are supported). Capture runs in Chromium's new headless mode by default (`--headed` shows the window); both are verified on macOS, without page audio. Login always uses a headed window. No persistent background service, global browser profile modifications, desktop capture, or software GUI is required. Publishing and clean-machine installation remain release work; see [distribution.md](distribution.md).