# Issue 89: reproduce the video spectrum failure through the real AVPlayer audio tap This probe exercises production `VideoPlayerService`, its private `VideoAudioTapProcessor`, `VideoAudioSampleRing`, `VideoSpectrumAccumulator`, `SpectrumAnalyzer`, `SpectrumFFT`, and the production video EQ. It plays a generated 55 Hz left / 110 Hz right video, pauses, seeks to 4 seconds, and replaces the source with a silent video. No app UI is launched. The recorded host was macOS 27.0.1, arm64, macOS SDK 27.0, Apple Swift 6.4 in Swift 6 language mode, and FFmpeg 8.0. Requirements are the selected Xcode toolchain, FFmpeg with `libx264` and AAC encoders, Python 3, Git, `patch`, and `rg`. ## Recreate and run Keep the six reproduction files together: this README, `setup.sh`, `instrument.py`, `production-fix.patch`, `Support.swift`, and `Probe.swift`. Run from a separate copy if the original observation logs must be preserved; the script overwrites its own `before-*.log` and `after-*.log` files. ```bash bash setup.sh /path/to/simple-media-player ``` The script obtains the ten required Swift files from fixed repository revision `f5756158f6fd51ff8ee76e3af310512cfbee1d0f`, generates the two videos, compiles the before and after executables, and runs all three response modes sequentially. It writes sources, the module cache, binaries, and logs beside the script and does not modify the supplied repository checkout. To prepare source copies without compiling or playing audio: ```bash bash setup.sh /path/to/simple-media-player prepare-only ``` The `prepare-only` path was executed and all 20 regenerated, instrumented source files matched the two snapshots used for the recorded runs exactly. To rerun just one prepared executable: ```bash ./probe-before "$PWD" normal ./probe-after "$PWD" slow ./probe-after "$PWD" fast ``` The first argument is the directory containing `tones.mp4` and `silence.mp4`; the second is `slow`, `normal`, or `fast`. Each run takes roughly 5 seconds plus asset preparation. The tones are audible through the default output device. Within the Codex restricted command sandbox, the Observation macro's plugin subprocess failed to launch, and AVPlayer's clock advanced without preparing its audio tap. Compilation and playback for the recorded runs therefore used the approved execution context outside that restriction. A run without `TAP_FORMAT` and playing frames does not verify the real tap. No `xcodebuild` was used here. ## Video generation These are the exact FFmpeg filters and codecs used by `setup.sh`: ```bash ffmpeg -hide_banner -loglevel error \ -f lavfi -i 'color=c=black:s=160x90:r=10:d=8' \ -f lavfi -i 'aevalsrc=0.35*sin(2*PI*55*t)|0.35*sin(2*PI*110*t):s=48000:d=8:c=stereo' \ -c:v libx264 -preset ultrafast -pix_fmt yuv420p \ -c:a aac -b:a 192k -shortest -y tones.mp4 ffmpeg -hide_banner -loglevel error \ -f lavfi -i 'color=c=black:s=160x90:r=10:d=8' \ -f lavfi -i 'anullsrc=r=48000:cl=stereo:d=8' \ -c:v libx264 -preset ultrafast -pix_fmt yuv420p \ -c:a aac -b:a 192k -shortest -y silence.mp4 ``` ## Source selection and compiler command `source-before/` contains these unmodified production files from the fixed revision, then receives only the diagnostic instrumentation described below: - `Services/SpectrumAnalyzer.swift`, `SpectrumFFT.swift`, `SpectrumSmoothing.swift`, `AudioFrameData.swift`. - `Services/VideoPlayerService.swift`, `VideoAudioSampleRing.swift`, `VideoSpectrumAccumulator.swift`, `VideoEqualizerProcessor.swift`. - `Models/EqualizerSettings.swift`, `MediaFormatInfo.swift`. All paths are under `SimpleMediaPlayer/` in the repository. `source-after/` starts from the same files and additionally receives `production-fix.patch` before the same instrumentation is applied. That patch reconstructs the exact after snapshot used for these observations; it is the pre-submission working implementation, not an assertion that every later change in a final PR was exercised by this probe. The patch changes only the following five production files: - `SpectrumAnalyzer.swift`: accumulate enough history for the requested frequency resolution across every response mode. - `SpectrumFFT.swift`: derive the window from sample rate and logarithmic band width, use the latest complete FFT window, and distinguish unavailable bins. - `VideoAudioSampleRing.swift`: carry the source asset range end with the PCM batch. - `VideoPlayerService.swift`: populate that end from `timeRange.end`. - `VideoSpectrumAccumulator.swift`: compare consecutive asset range boundaries independently of the PCM chunk clock. `Support.swift` supplies only the response-mode enum (10/30/60 Hz), localization identity function, settings key, and marker protocol needed to compile these production files without the rest of the UI. The actual spectrum, PCM channel copying, source epochs, EQ, and transport operations remain production code. EQ is disabled by the production default settings. The compile command used by the script is: ```bash swiftc -parse-as-library -swift-version 6 -O \ -module-cache-path "$probe_dir/ModuleCache" \ "$probe_dir/source-$probe_variant/"*.swift \ "$probe_dir/Support.swift" "$probe_dir/Probe.swift" \ -o "$probe_dir/probe-$probe_variant" ``` Compiler diagnostics are retained in `compile-before.log` and `compile-after.log`. The driver produces a compiler warning about a phase variable mutated after capture; all phase updates and the frame callback run on MainActor in this diagnostic driver. The existing notification deprecation warning is unrelated to this issue. ## Diagnostic instrumentation `instrument.py` modifies only the isolated copies: - In `VideoAudioTapProcessor.prepare`, immediately before `equalizer.prepare`, add `TAP_FORMAT` with sample rate, channel count, interleaving, PCM common format, and maximum callback frames. - In the real-time `process` callback, immediately after successful `MTAudioProcessingTapGetSourceAudio`, print `TAP_BATCH` for only the first 16 accepted callbacks per processor. A producer-local counter limits output. Each line includes frame count, source range start/duration, flags, epoch, and analyzer generation. - In `VideoSpectrumAccumulator.append`, print the first 32 reset decisions with actual/expected source time, frame count, sample rate, and `startsStream`. - In the ring-full branch, print `RING_DROP` when visualization PCM is discarded. The real-time callback `print` calls are diagnostic-only and unsuitable for the production audio path: they can allocate and block. They are absent from the production fix and are not a proposed logging design. This probe cannot establish a real-time execution budget or absence of production dropouts while using them. ## Observations in the saved logs The tap actually prepared `48000 Hz`, `2 channels`, `interleaved=false`, `commonFormat=1` (Float32), and `maxFrames=4096` in every mode. The source callback repeatedly provided 4096 PCM frames (85.333 ms), but asset time ranges had durations of 81.375 or 93 ms. Their boundaries were consecutive within about one sample: | Start | Asset duration | Next start | |---:|---:|---:| | -0.0232291667 | 0.081375 | 0.058125 | | 0.058125 | 0.093 | 0.151125 | | 0.151125 | 0.081375 | 0.2325 | | 0.2325 | 0.081375 | 0.313875 | The old comparison against `start + PCMFrames / sampleRate` differed by several milliseconds, so it reset on every callback. `startsStream` was false in these callbacks; no `RING_DROP` occurred. The SDK header for `MTAudioProcessingTapGetSourceAudio` calls `timeRangeOut` the asset time range corresponding to the provided frames; it does not promise equality with PCM duration. Recorded results: | Mode | Before | After | |---|---|---| | slow | 0 startup playing frames; 4096-frame input never reached the required 4800-frame chunk before reset | 17 startup frames, 100 ms apart; distinct left/right tone bands | | normal | Right 110 Hz lower four bands repeatedly exactly equal, around 0.82; a reset on every callback | One startup reset, 33.333 ms frame cadence; left 55 Hz peaks at band 2 and right 110 Hz at band 3 | | fast | Right 110 Hz lower four bands repeatedly exactly equal, for example 0.6561 | 16.667 ms frame cadence; the same separated tone peaks | At normal steady state, left band 2 was approximately 0.96–1.00 and right band 3 approximately 0.943; right bands 0–2 settled to 0.0002–0.0005. At fast steady state, right bands 0–2 settled to effectively zero. Every fixed mode emitted zero playing frames after pause. Switching to the silent source yielded only silent playing frames: slow 7/7, normal 22/22, fast 45/45. `AudioFrameData.isSilent` also checks held peaks. Seek discarded history and resumed steady analysis; a zero-duration range at the seek boundary caused one additional valid discontinuity reset before the subsequent normal ranges. The driver reports the first five and last three playing frames of each phase. Startup frame counts vary with asset preparation time; the cadence and frequency separation above are the useful comparisons. ## Limits and suggested Wiki bundle This is a macOS production-service integration probe, not a visual UI test or an iPhone/iPad device run. It covers the generated 48 kHz AAC input and default audio output; other codecs, sample rates, output devices, long playback drift, and sustained-load responsiveness need separate coverage. It checks pause, one seek, and one source replacement, not every possible concurrent transport ordering. The first AVPlayer callback had an invalid time range and the production ring rejected it. Some AAC preroll frames have negative source times. Those behaviors were preserved. Initial FFT warmup and release smoothing temporarily broaden the levels; the tone-separation result above is for steady state. For the Wiki, retain the six reproduction files listed above plus these six logs: - `before-normal-resets.log` (also contains the first 16 source ranges), `before-slow.log`, `before-fast.log`. - `after-normal.log`, `after-slow.log`, `after-fast.log`. `wiki-reproduction/` contains exactly that selection, and `issue-89-reproduction.tar.gz` packages it. Full `before/`, `after/`, `source-*/` snapshots, generated videos, binaries, module caches, and the redundant exploratory logs are unnecessary. A checkout that contains the fixed baseline revision is required to regenerate the selected sources.