# Testing ```sh cargo test --workspace # everything; oracle tests skip without ffmpeg cargo test -p filmcraft-h264 # one crate cargo test --release -p filmcraft-hevc # codec tests are much faster in release cargo xtask ci # all gates (fmt, clippy, tests, layers, assets, wasm) cargo xtask fixtures # pre-generate the ffmpeg fixture matrix FILMCRAFT_REQUIRE_ORACLES=1 cargo test --workspace # CI: missing ffmpeg fails instead of skipping ``` ## 1. Kinds of tests | Kind | Where | Examples | |---|---|---| | Unit | `src/` `#[cfg(test)]` modules, `src/tests.rs` | CABAC engines, VLC tables (prefix-freeness, Kraft sums), colour matrices, frame cache | | Property (`proptest`) | `time`, `bitstream`, `edit`, `isobmff/tests/roundtrip.rs`, `interchange/tests/*` | tick↔frame round trips for every rate, DF timecode, edit invariants (no overlaps, durations conserved), mux→demux round trips, interchange export→import | | Synthetic streams | `h264/src/synth_tests.rs`, `hevc/src/synth_tests.rs` | hand-built bitstreams for features the reference encoders don't emit; expected output known exactly | | Scripted UI | `crates/ui-egui/tests/scripted.rs` | headless app (egui_kittest) driven over the control channel: razor/undo, insert, apply effect, playback, click by automation id (§4) | | Monitor view / Graphics menu | `crates/ui-egui/tests/view_ui.rs`, `crates/engine/src/graphics_tests.rs` | View menu items and checkmarks, playback/paused resolution, channel / comparison / multi-camera / waveform display modes, magnification and Hand-tool panning, guides dragged from the rulers (move, lock, remove, Add Guide…, templates in the preferences), snapping a graphic to the frame centre and a guide; align to frame / as group / to selection, distribute (centres and gaps), arrange, select next/previous graphic and layer, reset parameters / duration, vertical text, New Layer from file (`FILMCRAFT_UI_SHOTS=` writes `view-*.png`) | | Settings | `crates/engine/src/settings_tests.rs`, `crates/ui-egui/tests/settings_ui.rs` | schema keys and defaults, persistence in the data directory, v1 → v2 migration and repair of bad values, validation, per-category reset; each wired setting (still / transition durations, step many, label defaults and colours, media scaling, timebase, recent projects, media cache policy, output mapping, smart quotes, auto-transcribe); the dialog: every category from its command and Cmd+,, edits by automation id, OK / Cancel / Escape / Reset…, theme, label names in Edit ▸ Label, tooltips, frame cache (`FILMCRAFT_UI_SHOTS=` writes `settings-*.png`) | | Scopes and panels (M8.9 / M12.6) | `crates/scopes/src/tests.rs`, `crates/engine/src/{scopes,panels}_tests.rs`, `crates/ui-egui/tests/panels_ui.rs` | scope maths on generated frames (flat colours → exact histogram bins, ramps → waveform row = code value, 75 % bars → parade levels and vectorscope target cells for BT.601/709/2020, HLS hue angles, YC chroma, Clamp Signal, decimation, NaN); `scopes.read` on a colour matte (exact bins, 78.43 %) and on Bars and Tone (the six targets); metadata edits (one undo step, read-only fields refused, saved in the project), the event log (failed / disabled commands, repeats, jobs started / finished / failed / cancelled); every panel from Window ▸, the scopes' wrench menu, presets, five-scope grid, typing into a Metadata field, Timecode rows and modes, Events filter / Clear All, Progress cancel, Reference Monitor park / gang / scopes (`FILMCRAFT_UI_SNAPSHOT_DIR=` writes `panels-*.png`); `perf_scopes_at_1080p` (ignored) times each scope | | Golden images | `crates/golden/tests/golden.rs` | CPU renders vs committed PNGs; GPU vs CPU on the same scenes (§3) | | Engine / command | `crates/engine/src/tests.rs` | run commands on the demo project, assert the sequence, undo/redo, disabled cases | | Render | `crates/render/src/tests.rs` | compositing, opacity, Motion, cross dissolve midpoint, ½-res vs full, GPU plan vs reference, audio mix, audio-effect continuity | | Multi-camera / sync | `crates/audio-dsp/src/sync.rs`, `crates/engine/src/multicam_tests.rs`, `crates/ui-egui/tests/multicam_ui.rs` | audio offsets within one sample: Rust-generated multi-mic speech (gains −20…+10 dB, SNR 40/25/10/3 dB, zero-phase colouring, fractional delays, a 0 dB-SNR room with a −4 dB echo) and ffmpeg-made camera/recorder files (pink-noise bursts, independent noise; exact lags recovered); timecode (± hours) and marker sync within a frame; Merge Clips; timeline Synchronize keeps frame-aligned starts; multicam structure, angle render (PSNR vs the camera), Switch Audio mix = the angle's track, live switching from a simulated playback clock (cuts, healing, one undo step), Ctrl+N cuts, flatten (frames identical), save/load; 20 angles: paging (auto / 2×2 / 3×3), page-relative keys, absolute cameras beyond 16, a page decoding only its angles at the cell scale (recording source), Auto-Adjust scale, Selection Top Down; headless: page arrows, angle 17–20 cells, grid-only view, Edit Cameras thumbnails; `perf_grid_of_four_1080p_angles` (ignored) prints the grid cost | | Project panel and Media Browser (M12.7) | `crates/engine/src/{project_panel,media_browser}_tests.rs`, `crates/ui-egui/tests/project_panel_ui.rs` | view settings and their preference round trip; Metadata Display columns (custom fields, Freeform keys hidden), numeric vs text sorting, view presets (save / save as / restore / rename / delete, ten slots, persisted in `preferences.json`); Freeform moves keeping offsets, snapping, Align to Grid, Clip Size, stacks, arrangements, Reset (each one undo step, saved in the project); bin rename / nesting; the Media Browser against a fake filesystem (folders first, hidden and unsupported files left out, file-type filter, back / forward / up, recent directories, Favorites, import of a selection or a folder, Open in Source Monitor, ingest, probe columns). Headless: List view (sort by header, drag a column wider, inline rename), Icon view (hover scrub + I / O set the clip's In / Out, Shift+] / [), Freeform (drag a card, stack from the card menu, options dialog), panel menu, Metadata Display dialog, view presets, bins in place / tab / window, footer buttons, Media Browser navigation, Favorites, import, Shift+O, Edit Columns…, thumbnails, drag to the Timeline (`FILMCRAFT_UI_SNAPSHOT_DIR=` writes `project-*.png`, `media-browser-*.png`) | | Menu long tail (M3.11) | `crates/render/src/scene.rs`, `crates/engine/src/{scene_detect,sequence_extras,project_tools}_tests.rs`, `crates/interchange/src/ale.rs`, `crates/project/src/find.rs`, `crates/ui-egui/tests/menus_ui.rs` | Scene Edit Detection on synthetic frames (hard cuts vs pans, flashes, sensitivity) and on generated movies with known cut points (our own ProRes render of demo scenes; an ffmpeg-made H.264 concat of testsrc2 / SMPTE bars / Mandelbrot): cuts with linked audio, clip markers, subclips, one undo step, background job and cancel; Normalize Mix Track to a target peak; Simplify Sequence; Find / Find Next and search bins; Automate to Sequence (overlap, transitions, insert, unnumbered markers); ALE round trip; Project Settings and scratch disks; menu order and shortcuts against Premiere's; the dialogs by automation id | | Mixer | `crates/render/src/mixer_tests.rs`, `crates/engine/src/mixer_tests.rs` | sample-exact fader gain and pan laws, automation at block boundaries (bit-identical however requests are cut), solo/mute/solo-safe, sends and submix routing, latency-compensation alignment, render-vs-playback identity, Touch ramp-back, recorder modes (Latch/Touch/Write), thinning, Audio Gain, transition curves; `perf_24_tracks_3_effects_realtime_factor` prints the realtime factor | | 5.1 / multichannel | `crates/audio-dsp/src/channels.rs`, `crates/render/src/mixer_tests.rs`, `crates/export/src/surround_tests.rs`, `crates/engine/src/mixer_tests.rs`, `crates/ui-egui/tests/audio51_ui.rs` | BS.775 downmix / upmix coefficients exact, mixdown types; 5.1 panner point gains equal-power (Σg² = 1 over a 21×21 grid × 3 centre settings), speaker positions exact, 5.1 through the default puck bit-exact identity; graph: stereo/mono tracks into a 5.1 Mix (L/R, C, rear right, room centre −6 dB each, LFE = mean), 5.1 track through a 5.1 Mix and folded into stereo, a 5.1 submix, `pan51` automation block-invariant, six-channel meters; export: 6-ch WAV (`WAVE_FORMAT_EXTENSIBLE`, mask 0x3F) and ProRes MOV (6 × 24-bit PCM) decoded by ffmpeg **sample-exact**, ffprobe `channels` 6 / `channel_layout` 5.1; AAC 5.1 in MP4: every decoded channel's own tone ≥ 30 dB above the others' (no swapped channels), stereo export of a 5.1 Mix = BS.775 within 2·10⁻⁶; `sequence.settings {mix}`, `file.newSequence {mix, trackType}`, panner lanes, Write pass records the puck; UI: puck drag (one undo step), Center / LFE, six meter bars | | Audio Clip Mixer automation, Track Mixer view | `crates/engine/src/mixer_tests.rs`, `crates/ui-egui/tests/audio51_ui.rs` | Touch: a 201-point fader ramp thins to ≤ 6 clip keyframes, values within 0.1 dB, live override while held, values outside the gesture unchanged, one undo step; Latch holds until stop; Write records volume and pan from the start; a gesture across two clips writes both; Read commits once without a pass; UI: Clip Mixer mode dropdown, a recorded fader drag writes clip keyframes; Show/Hide Tracks, Meter Input(s) Only, transport record button | | Voice-over | `crates/engine/src/voiceover_tests.rs`, `crates/ui-egui/tests/voiceover_ui.rs` | synthetic click train lands in `mix_sequence` exactly at R + k·period (pre-roll dropped, exact sample count), punch-in between In/Out (ramp source, every sample checked), one undo step + undo/redo, take numbering, record-arm/targeting track choice, sync/discard/zero-length/locked cases, cue times; UI: right-click Mic → Voice-Over Record Settings dialog (edits persist on OK, Cancel discards), the Mic button starts and stops a take on its track with pre-roll and overlay | | Remix | `crates/audio-dsp/src/remix.rs`, `crates/render/src/remix.rs`, `crates/engine/src/remix_tests.rs`, `crates/ui-egui/tests/remix_ui.rs` | generated rhythmic music (Rust-synthesised drums, bass, chord sections at 90–128 BPM, 22.05/44.1/48 kHz): tempo within 0.3 BPM, every beat within 10 ms (measured ≤ 8.9 ms); remix targets from 0.4× to 2.2× the source within one beat (measured ≤ 12 ms at default sliders), every cut on a detected beat, intro and outro kept, pieces ≥ 4 beats, deterministic; silence, tone and noise refused; rendering independent of request cuts, source-exact outside the 20 ms equal-power crossfades, no clicks; the engine's mix equals the source pieces (< 1e-5); undo, redo and revert exact; overlaps refused; Remix Properties dialog and Remix tool drag driven by automation id | | Oracle | `crates/*/tests/*oracle*.rs`, `conformance.rs` | compare with ffmpeg/ffprobe (§2) | | Robustness / fuzz | `*/tests/robustness.rs`, `*/tests/fuzz.rs` | seeded mutation and truncation of real and synthetic files; nothing may panic | | Performance | `*/tests/perf.rs` (`#[ignore]`) | §5 | Commit `*.proptest-regressions` files so failing cases are re-run. ## 2. ffmpeg oracle tests ffmpeg and ffprobe are **external processes** used to generate fixtures and to check results. They are never linked or shipped ([AGENTS.md](../AGENTS.md) §2). - The tools are found by `filmcraft-testkit` (a dev-dependency-only crate, `crates/testkit`), in this order: `FILMCRAFT_FFMPEG` / `FILMCRAFT_FFPROBE` (explicit paths; a wrong path is an error), then every directory on `PATH` (`ffmpeg` and `ffmpeg.exe`), then well-known install directories (`/opt/homebrew/bin`, `/usr/local/bin`, `/usr/bin`, `C:\ffmpeg\bin`, …). - If a tool is missing the test prints `SKIPPED (): ffmpeg not found …` and passes. Set **`FILMCRAFT_REQUIRE_ORACLES=1`** (CI) to make every such skip a failure. In new tests use `let ff = filmcraft_testkit::require_ffmpeg!();` (also `require_ffprobe!`, `require_oracles!`). - Fixtures are generated on first use into `/target/fixtures//` (`filmcraft_testkit::fixtures_dir`; independent of `CARGO_TARGET_DIR`, so agents with private target dirs share them; `FILMCRAFT_FIXTURES_DIR` overrides the root) and reused afterwards. Generators write to a per-process/thread temporary name (`testkit::temp_path`) and rename it into place, so concurrent tests never see half-written files. Delete the directory to regenerate. Never commit media. - Every crate's `tests/common/mod.rs` delegates discovery and fixture paths to testkit, except `crates/codecs/src/tests.rs`, which still has its own lookup (the crate was being edited concurrently; migrate it when convenient). - `cargo xtask fixtures [crate…]` pre-generates the whole fixture matrix up front (useful before a parallel test run or on a fresh machine). It runs each crate's ignored `generate_fixtures` test — the same generators the oracle tests call — for `h264`, `hevc`, `isobmff`, `matroska`, `dnx`, `prores`, `mpeg2v`, `ac3` and `codecs` (MXF, Ogg and MPEG TS / PS), and prints one `made` / `cached` / `skipped` line per fixture plus a summary. The `aac`, `opus` and `h264enc` oracles generate small per-test signals on demand and are not part of the matrix. - Fixture sources are synthetic: `testsrc2`, `mandelbrot`, SMPTE bars, noise and fades, sine tones. H.264 and HEVC fixtures need ffmpeg built with libx264 and libx265. VideoToolbox fixtures are generated only on macOS. ### Pass criteria per codec | Crate | Oracle check | Criterion | |---|---|---| | `h264` | decode every fixture single-threaded and frame-threaded; compare with `ffmpeg -f rawvideo -pix_fmt yuv420p` | **bit-exact**, every frame | | `hevc` | same, `yuv420p` / `yuv420p10le` | **bit-exact** (8- and 10-bit) | | `h264enc` | `ffmpeg -v warning -err_detect +crccheck+bitstream+buffer+explode` decodes our stream | no output at all (a `corrupt decoded frame` warning fails; a negative control checks dropped and truncated slices are reported), exact frame count, **bit-identical to the encoder's reconstruction**; B-frame order checked with ffprobe; rate-control targets | | `prores` decode | ffmpeg `prores_ks` / `prores_aw` fixtures, compared at native depth | within **±1 LSB** (different integer IDCT); alpha bit-exact | | `prores` encode | ffmpeg decodes with `-xerror` | no errors; agrees with our decoder within ±1 | | `dnx` decode | ffmpeg VC-3 fixtures (13 DNxHD CIDs, DNxHR LB/SQ/HQ/HQX/444), compared at native depth | within **±2 LSB**, ≤ 100 samples per million beyond ±1 (ffmpeg's integer IDCT; we evaluate the exact IDCT) | | `dnx` encode | ffmpeg decodes with `-xerror` | no errors; agrees with our decoder within ±2; HQ luma PSNR ≥ 45 dB | | `aac` decode | ffmpeg's decode of the same stream | max abs error ~1e-7 (float) | | `aac` encode | ffmpeg decodes our stream | no errors, no clipping, CBR within ±5% of target; SNR reported | | `isobmff` | `ffprobe -show_packets` on ffmpeg-made MP4/MOV | packet offsets, sizes, pts/dts, durations, key flags and stream parameters equal; remuxed files decode with `ffmpeg -v error` silent | | `matroska` | `ffprobe -show_packets` | every packet (stream, size, key flag, pts, duration) equal; seeks land on the latest keyframe ≤ target | | `audio-dsp` loudness | `tests/loudness_oracle.rs`: signals generated in Rust (997 Hz sine, pink noise at 48/96 kHz, speech-like bursts at 48/44.1 kHz, stereo with silence and sub-gate passages, an fs/4 inter-sample-peak tone) written as float WAV and measured with `ffmpeg -af ebur128=peak=true:metadata=1` | momentary and short-term every 100 ms **±0.1 LU**, integrated **±0.1 LU**, LRA **±0.5 LU**, true peak **±0.2 dB** (against the analytic value when one exists). Measured: ΔM/ΔS ≤ 0.0005 LU, ΔI ≤ 0.007 LU, ΔLRA ≤ 0.04 LU, ΔTP ≤ 0.045 dB against ffmpeg. On the fs/4 tone ffmpeg's own true peak is +0.6 dB high (−0.32 vs the analytic −0.92 dBTP); ours is −0.05 dB | | `audio-dsp` effects | `src/effects/premiere_tests.rs`, `src/design.rs`, registry tests in `src/effects/mod.rs`: generated tones, noise and impulses | every effect: neutral settings = delayed identity (≤ 2·10⁻⁴), bit-exact determinism, latency constant and equal to where an impulse lands, block-size independence, finite / denormal-free tails at extreme settings, > 1× realtime; per effect a level or frequency-response check (measured tone vs analytic `response_db` within 0.1–0.2 dB, Butterworth vs closed form 10⁻⁶ dB, Chebyshev/elliptic ripple and stop-band bounds, LR4 crossover sum flat within 0.02 dB, comb-notch depth, echo positions to the sample, Schroeder RT60 of generated impulses within 25 %, click repair −20 dB) | | `mxf` (via `codecs/tests/mxf_oracle.rs`) | ffmpeg-written OP1a / OP-Atom / D-10 files decoded by ffmpeg | H.264 (long GOP, B pictures without temporal offsets) **bit-exact** every frame and at 25 random seeks; DNxHR ±2, ProRes ±1; PCM / AES3 **sample-exact**; frame count = ffprobe packets; timecode (25 fps, 29.97 DF); MPEG-2 long GOP, XDCAM HD422 (1080i 4:2:2) and D-10 / IMX (720×608 4:2:2) within ±4 (`mpeg2v` criterion); truncation/mutation never panics | | `mpeg2v` | ffmpeg-encoded MPEG-1/2 elementary streams (progressive, TFF/BFF interlaced, 4:2:2 intra and long GOP, 1080i, escapes + custom matrices, MPEG-1) and our synthetic streams with field pictures / 16x8 / dual prime / concealment vectors, decoded by ffmpeg (`-xerror`) | per frame max ±4 and PSNR ≥ 58 dB, ≤ 3 % of samples differing (ffmpeg's integer IDCT vs our IEEE 1180-accurate one; measured max ±1-3, 65-75 dB); frame count, display order, picture types, interlacing and field order **exact** (ffprobe); IEEE 1180 procedure passes | | MPEG TS / PS (via `codecs/tests/mpeg_oracle.rs`) | ffmpeg-written TS (MPEG-2 + MP2, H.264 + AAC, HEVC + LATM AAC), BDAV `.m2ts` / `.mts` (H.264 + Blu-ray LPCM / AC-3), VOB (MPEG-2 + MP2 / DVD LPCM 16 and 24-bit / AC-3), MPEG-1 system streams, `.m2v`; MPEG-2 in MOV (XDCAM `xd5c`), MP4 (`mp4v`), MKV | frame count and every frame's **PTS exact** (ffprobe); H.264 / HEVC **bit-exact**, MPEG-2 within the `mpeg2v` criterion; random seeks = sequential decode; LPCM **sample-exact**; AAC ≤ 2.1·10⁻⁷, MP2 (symphonia) ≤ 3.2·10⁻⁵, AC-3 ≤ 5·10⁻³ (decoder-specific dither; measured 2.3·10⁻³); random-access audio = sequential | | `ac3` | ffmpeg AC-3 encodes (2/0 with coupling and rematrixing, 1/0 32 kHz, 3/2+LFE 448 kb/s, noise) decoded by ffmpeg | SNR per channel ≥ 40-60 dB on tonal content (measured 66-94 dB), ≥ 18 dB on noise (27.7 dB, the same as between two of our own dither seeds) | | `ogg` (via `codecs/tests/ogg_oracle.rs`) | ffmpeg (libopus, Vorbis) files decoded by libopus / ffmpeg | length **exact** (pre-skip, end trimming); SNR ≥ 50 dB CELT (measured 77.7), ≥ 10 dB SILK, Vorbis 139 dB; random seeks within 6·10⁻⁶ of the continuous decode | | MP4 audio at another rate (`codecs/tests/mp4_oracle.rs`) | ffmpeg-written AAC in MP4 (48 kHz, edit list skipping the priming) | reads resampled to 44.1 kHz in frame-sized requests equal the native decode interpolated at the same times (difference < 10⁻⁴, peak ≤ the source level) | | image sequences / BWF (`codecs/tests/media_oracle.rs`) | ffmpeg-written PNG/TIFF/BMP/JPEG stills read by ffmpeg's image2 demuxer; ffmpeg BWF (`-write_bext`) | PNG/TIFF/BMP **exact**, JPEG ±12 (measured 3); missing frames hold the previous one; BWF TimeReference = ffprobe's, start timecode, samples exact | | `export` | our own demuxer/decoder reads the file back; ffprobe counts frames when present | expected size, duration, colour, audio level; exact frame count | | `export` settings and presets (M6.5) | `export/src/settings_tests.rs`, engine `export_tests.rs`: every built-in preset exports a slice of the demo sequence; ffprobe reads codec, profile, size, rate, audio format and data rate; our decoders read pixels back | codec / profile / container / size / `r_frame_rate` / sample format exact; H.264 bitrate ≤ VBR max × 1.1 and ≥ ½ target over 3 s; ProRes ≤ 1.5 × nominal; DNxHR within 25 % of nominal; loudness normalization within **0.5 LU** of the target (WAV and AAC), true peak ≤ ceiling + 0.1 dB; image sequences numbered `000…`; limiter, overlays, metadata (`title`/`copyright` tags), MOV multiplexer, two-pass/CBR progress | Each codec README has the full fixture matrix and the measured results. ## 3. Rendering: CPU reference and GPU parity - The CPU compositor in `filmcraft-render` is the reference. Its tests check computed values (opacity blends, dissolve midpoints, Motion placement, ½-resolution against downsampled full resolution) and that `render::plan` matches `render_sequence`. - `crates/gpu/src/tests.rs` composites the same frame plan (a YUV layer plus a transformed, semi-transparent RGBA layer) on the GPU and on the CPU. It requires a 99th-percentile channel difference of ≤ 6 and a mean of < 1.5 (8-bit levels). It skips when no GPU adapter is available. - Effects implemented only on the CPU need no GPU test: the plan pre-renders those layers on the CPU. - **Golden images** (`crates/golden`, a test-only crate): `tests/golden.rs` builds small procedural projects (demo-generator footage, bars, no media files) at 320×180 and renders one frame of each through the CPU compositor: Motion transform + opacity, four blend modes (Multiply, Screen, Overlay, Difference), Gaussian Blur, Lumetri basic correction, Crop, Cross Dissolve at 50 %, Dip to Black at 25 %, Wipe at 50 %, Timecode / Clip Name burn-in text, graphic clips (also a built-in graphics template with overridden properties and per-character styles with a pinned box), and colour management: S-Log3/S-Gamut3.Cine footage interpreted into Rec. 709 (`log_to_rec709`) and Rec. 2100 PQ footage tone mapped into Rec. 709 (`hdr_tone_map`; the footage is demo frames re-encoded by the test's `Encoded` source). Each frame is compared with `crates/golden/goldens/.png` by `filmcraft_testkit::golden`: **PSNR ≥ 45 dB, max abs ≤ 12, 99th-percentile per-pixel max channel difference ≤ 2** (8-bit sRGB levels; `Tolerance::RENDER`). On failure the actual frame and a ×8 difference image go to `/target/golden-failures/`. Six `vfx_*` scenes cover the M5.11 effects (Corner Pin over Turbulent Displace, Ultra Key, Wonder Glow + Glint, Spacer + Stroke + Long Shadow over Gradient, VR Rotate Sphere, Channel Mix + Color Emboss + Rounded Crop); per-effect unit tests live in `crates/render/src/vfx/tests.rs` (identity at neutral parameters, determinism, range, a known value each, temporal/track effects through the compositor). `scenes_are_distinct` guards against blank or duplicate scenes. - **Video transitions** (`crates/render/src/transitions/tests.rs`) are checked for every transition (84 modern, 21 Legacy, 7 hidden Obsolete): p = 0 is exactly the outgoing frame and p = 1 exactly the incoming one, near both ends the output is close to that frame (continuity), the midpoint differs from both, the output is finite and deterministic (also on 1×1 / odd sizes and NaN progress), every parameter visibly changes the picture, and every direction differs. Golden *fingerprints* (4×4 mean RGB grid at p = 0.25/0.5/0.75, ±4 levels) live in `src/transitions/goldens.txt`; re-bless with `FILMCRAFT_BLESS=1 cargo test -p filmcraft-render golden_fingerprints` and review the diff. `FILMCRAFT_TRANSITION_SHEETS= cargo test -p filmcraft-render contact_sheets -- --ignored` writes one contact-sheet PNG per folder for eyeballing (never commit them). - **Blessing:** `FILMCRAFT_BLESS=1 cargo test -p filmcraft-golden` rewrites the references (and writes a `.attribution` sidecar for any new one; it prints the `ATTRIBUTION.md` row to add). References are original work rendered by FilmCraft, must stay under 50 KB (enforced on bless), and every one needs its sidecar and index row (`cargo xtask assets`). Look at a re-blessed PNG before committing it. - The same scenes run through `plan_frame` + `GpuCompositor` when a GPU adapter exists (`gpu_matches_cpu_on_golden_scenes`, same p99 ≤ 6 / mean < 1.5 criterion; skipped otherwise). This found a real bug: the GPU upload cache was keyed by pixel-buffer address without keeping the buffer alive, so a new frame allocated at a freed frame's address was drawn with the stale texture (fixed; regression test `upload_cache_keeps_buffers_alive`). ## 4. UI: control channel and screenshots ### Scripted UI tests (headless) `crates/ui-egui/tests/scripted.rs` runs the real `FilmcraftApp` under [`egui_kittest`](https://docs.rs/egui_kittest) (`build_eframe`): no window, no GPU, no OS event loop. A small `Driver` opens the demo project, sends requests through the same control channel agents use (`ControlRequest` → `control::handle`, exactly as the TCP server does) and steps egui frames until each reply arrives. Synthetic input queued by `ui.click` / `ui.drag` / `ui.key` is moved into the next frame by the app's own `raw_input_hook`, so clicks by automation id work headless. Covered: | Test | Asserts | |---|---| | `demo_project_opens_headless_and_registers_widgets` | active sequence, > 50 registered widgets, panel and tool ids, 6 clips on V1 (`sequence.inspect`) | | `razor_then_undo_through_the_control_channel` | `timeline.razor` adds a clip, the timeline can locate it on screen, `edit.undo` via `ui.menu.invoke`, redo | | `insert_from_source_ripples_the_sequence` | `source.open` + marks + `source.insert` lengthens the sequence | | `apply_effect_appears_in_effect_controls` | `effects.apply` adds Gaussian Blur in the model and `effectControls.effect.gaussian_blur` appears in the panel; undo removes it | | `playback_toggle_and_stop` | `playback.toggle` / `ui.playback stop` and `ui.inspect` playback state | | `clicking_a_tool_button_by_automation_id` | `ui.click {id: "tools.Razor"}` changes the tool (real egui input path) | | `unknown_methods_and_commands_fail_cleanly` | errors come back as `{"ok": false}` | `crates/ui-egui/tests/mixer_ui.rs` drives the Audio Track Mixer, Audio Clip Mixer, timeline track keyframes and the Audio Gain dialog by automation id and with multi-frame pointer drags (press, move over several frames, release). With `FILMCRAFT_UI_SNAPSHOT_DIR=` it also renders the window offscreen through wgpu (`Harness::render`) and writes `mixer-*.png`; this works without a visible window (for example on a locked screen, where `ui.screenshot` cannot capture). `crates/ui-egui/tests/essential_sound_ui.rs` does the same for the Essential Sound panel (type buttons, switches, a slider drag as one undo step, section bypass, Auto-Match, ducking, Browse presets; `essential-sound-*.png`). Essential Sound engine tests (`crates/engine/src/essential_sound_tests.rs`) build projects from generated speech-like and tonal WAVs: Auto-Match lands within ±0.5 LU of the target (measured: 0.000 LU, with and without a repair/clarity chain); ducking on a dialogue + music project gives keyframes within 60 ms of the expected times (measured 20 ms) and −15.00 dB in the mix; each repair stage improves its metric through the render path (hum −39 dB, rumble −22 dB, noise floor −13 dB, sibilance −19 dB, reverb tail −10 dB); the mix is bit-identical however requests are cut and the WAV export equals it. `perf_full_dialogue_chain_realtime_factor` (ignored; run with `--release`) prints the realtime factor of all nine Dialogue effects on one clip (22× on one core). `crates/ui-egui/tests/export_ui.rs` drives Export mode: the settings column and Summary, editing a setting turning the preset into Custom, the Preset Manager (search, favourite star, save, apply, delete), the queue panel (Send to Queue, Up/Down, Cancel, Start, Retry, Clear) with real encodes, the Export button and the header's Quick Export popup (`export-*.png` snapshots). `crates/ui-egui/tests/color_ui.rs` drives colour features the same way (`color-*.png` snapshots): Lumetri Input/Look LUT menus and section switches, the Interpret Footage ▸ Color Management and Sequence Color Management dialogs, and the HDR scopes of a PQ sequence. Colour science tests: `filmcraft-color` (curve round trips and published reference values per camera log curve, BT.709/BT.2087 matrices, BT.2390 EETF, gamut mapping, LUT tetrahedral vs a brute-force barycentric reference, `.cube`/`.3dl` round trips), `filmcraft-gpu` (`gpu_lut_matches_cpu_tetrahedral`: WGSL vs CPU, max |Δ| ≈ 2e-7), `render::colorman`, `render::color_match`, engine `color_tests.rs`, and `export::tests::hdr_exports_signal_pq_and_hlg` (PQ/HLG exports read back by our demuxer and by ffprobe: `color_transfer`, `color_primaries`, `color_space`, mastering display and content light side data). Not covered headless: `ui.screenshot` (needs a real viewport), the wgpu monitor path (the harness runs the CPU texture path), audio output, and wall-clock playback advance (kittest frames do not advance real time). Playing sound is covered without a device: `play_ahead` tests drive the mixed-ahead ring with a wall-clock device (order, underruns stay in time, mixer panic, stop), and `cargo run --release -p filmcraft-ui-egui --example bench_audio -- ` plays a real project's sequences through a simulated 48 kHz device and counts dropouts, mixing in the callback (`direct`, as before) against mixing ahead (`ahead`). Use `Driver` for new UI regressions: `d.exec(command, params)`, `d.ok(method, params)`, `d.frames(n)`. ### Interactive checks For visual work, drive the real app: ```sh cargo run --release -p filmcraft -- --control 9876 ``` Then script it over the control channel or MCP: run commands, click by automation id, `ui.inspect` / `ui.elements` to assert state, and `ui.screenshot {"path": "…"}` to capture the window or a single panel (`{"panel": "Timeline"}`). Look at every screenshot. Examples are in [agents.md](agents.md). The headless path (`filmcraft-cli run script.jsonl`, MCP `--demo`) covers engine behaviour without a window. ## 5. Performance | Benchmark | Command | |---|---| | H.264 decode, 1080p, threads sweep | `cargo test --release -p filmcraft-h264 --test perf -- --ignored --nocapture` | | HEVC decode, 1080p / 2160p | `cargo test --release -p filmcraft-hevc --test perf -- --ignored --nocapture` | | ProRes decode/encode | `cargo test --release -p filmcraft-prores --test perf -- --ignored --nocapture` | | Encoder speed / PSNR / bitrate | `cargo run --release -p filmcraft-h264enc --example h264enc_synth -- …` | | Decode a real file through the media stack | `cargo run --release -p filmcraft-cli -- bench-decode file.mp4 --frames 120` | | Coding-tool coverage of the fixtures | `cargo test --release -p filmcraft-h264 --test conformance coverage_report -- --ignored --nocapture` (same for `hevc`) | | **Whole-app suite**: decode fps per codec, playback, scrubbing, timeline UI, export, project save/open, peak RSS | `cargo xtask bench` ([performance.md](performance.md)) | `cargo xtask bench` (the example `crates/ui-egui/examples/bench/`) runs six sections, each in its own process under `/usr/bin/time` so peak RSS is per section: `decode` (every frame of 1080p and 2160p H.264, HEVC, VP9, AV1 and ProRes through the media stack), `playback` (the scenarios below), `scrub` (random seeks and playhead drags: time until the exact frame is on screen), `timeline` (the real app under `egui_kittest` with a 1000-clip / 20-track sequence: update, tessellation and wgpu render ms per frame), `export` (H.264 + AAC and ProRes) and `project` (save / open 5000 clips). Options: `--sections decode,scrub`, `--only ` (fixture or scenario), `--repeat N`, `--quick`, `--cpu`, `--hw auto|off` (Settings ▸ Playback ▸ Hardware decoding; default `auto`, as the app), `--label NAME` (output `target/bench/bench-