# clip-storyboard-director
Local-First AI Storyboard Director & Scene Continuity Orchestrator
Bridging AI video generators, browser automation (CDP), 4D persistence buffers, and multi-track audio assembly into a coherent directorial pipeline.
English |
Deutsch
---
### 🧭 Quick Navigation
- [1. Executive Summary & Why This Exists](#1-executive-summary--why-this-exists)
- [2. Architecture & System Flow](#2-architecture--system-flow)
- [3. Director & Editor Duo](#3-director--editor-duo)
- [4. 4D Persistence & Bounded Continuity](#4-4d-persistence--bounded-continuity)
- [5. Clue-Frame Continuity Chain](#5-clue-frame-continuity-chain)
- [6. Dual-Pane Director Cockpit](#6-dual-pane-director-cockpit)
- [7. Multi-Track Audio Staging & Ducking](#7-multi-track-audio-staging--ducking)
- [8. Key Governance & Runtime Invariants](#8-key-governance--runtime-invariants)
- [9. End-to-End Media Production Lifecycle](#9-end-to-end-media-production-lifecycle)
- [10. Target Personas & Discoverability](#10-target-personas--discoverability)
- [11. Comparative Matrix vs. Alternatives](#11-comparative-matrix-vs-alternatives)
- [12. Third-Party Licenses, Level 1 SBOM & RunAsInvoker](#12-third-party-licenses-level-1-sbom--runasinvoker)
- [13. Sibling Tools & Ecosystem Matrix](#13-sibling-tools--ecosystem-matrix)
- [14. Installation & CLI Usage](#14-installation--cli-usage)
- [15. Reference Production "Sternenseufzer"](#15-reference-production-sternenseufzer)
- [16. Security Policy & Privacy](#16-security-policy--privacy)
- [17. Development, Verification & Quality Gates](#17-development-verification--quality-gates)
- [18. Statutory Notice, Liability Limitation & License (§ 521 BGB)](#18-statutory-notice-liability-limitation--license--521-bgb)
- [19. Sister Project: `ai-media-editor`](#19-sister-project-ai-media-editor)
---
## 1. Executive Summary & Why This Exists
Generating individual AI video clips using models such as Veo, Kling, Sora, or Runway has become effortless. However, turning isolated clips into a **coherent narrative film** remains difficult. Creators face persistent challenges:
1. **Continuity Drift**: Characters alter clothing, facial features, or hairstyle from shot to shot.
2. **Context Amnesia**: Background environments and props disappear or morph between cuts.
3. **Manual Drag-and-Drop Fatigue**: Users constantly download clips from web browsers, rename them, extract final frames, and re-upload them as reference images.
4. **Dialogue & Audio Desynchronization**: Synthesized voices, background atmospheres, and music are often assembled as an afterthought without proper ducking or beat-matching.
`clip-storyboard-director` solves these problems through an automated, local-first directorial pipeline. It maintains a **4D persistence buffer** across all scenes, automatically extracts optical **clue-frames**, orchestrates web-based video generators via **Chrome DevTools Protocol (CDP)**, watches downloads, and assembles multi-track audio masters with zero cloud telemetry.
---
## 2. Architecture & System Flow
```mermaid
flowchart TD
subgraph Input["1. Input & Script Breakdown"]
Script["Story Script & Concept"] --> Init["storyboard_init.py"]
Init --> Config["project.yaml
(4D Persistence Buffer & Shots)"]
end
subgraph Directing["2. Directorial Control & Automation"]
Config --> Server["board_server.py
(Port 8765)"]
Server --> Cockpit["start_cockpit.py
(Dual-Pane Edge Cockpit)"]
Server --> Bridge["edge_bridge.py
(CDP Automation :9222)"]
Bridge --> Gen["AI Video Generator
(Gemini / Runway / Kling)"]
end
subgraph Ingestion["3. Ingestion & Continuity"]
Gen --> DL["Downloads Watcher"]
DL --> Ingest["ingest.py"]
Ingest --> Clue["Frame Extractor
(Shot N Clue-Frame)"]
Clue -.->|"Next Frame Prompt Injection"| Bridge
end
subgraph Assembly["4. Audio Staging & Assembly"]
Config --> Voice["TTS / Microphone Staging"]
Ingest --> Assemble["assemble.py
(FFmpeg Video Stitcher)"]
Voice --> Assemble
Assemble --> Master["Master Video (.mp4)
+ Audio Staging"]
Master -.-> Handoff["ai-media-editor
(Timeline Post-Production)"]
end
classDef core fill:#2563eb,stroke:#1d4ed8,color:#ffffff
classDef buffer fill:#7c3aed,stroke:#6d28d9,color:#ffffff
classDef io fill:#059669,stroke:#047857,color:#ffffff
class Server,Bridge,Ingest,Assemble core
class Config,Clue,Master buffer
class Cockpit,Gen,DL,Voice io
```
### ASCII Four-View Architectural Projection
```text
+====================================================================================================+
| CLIP-STORYBOARD-DIRECTOR: ARCHITECTURAL TOPOLOGY & SUBSYSTEMS |
+====================================================================================================+
[VIEW 1: USER INTERACTION, CLI COCKPIT & AGENTIC ORCHESTRATION]
+-------------------------+ +--------------------------+ +--------------------------------+
| Human Creator / Director| | Autonomous Coding Agents | | Pipeline / Studio Integrations |
| (Narrative Storyboard) | | (Gemini / Claude / Codex)| | (Scripts / CI Automation) |
+------------+------------+ +------------+-------------+ +---------------+----------------+
| | |
v v v
+-----------------------------------------------------------------------------------------------+
| CLI Entrypoint (clip-director) / Module Entrypoint (python -m clip_director) |
| * init: bootstrap project.yaml & 4D persistence buffer |
| * doctor: probe FFmpeg, Edge CDP port 9222, edge-tts, Python env |
| * serve: local HTTP board dashboard server (127.0.0.1:8765) |
| * cockpit: spawn synchronized dual-pane Microsoft Edge browser workspace |
| * autopilot: run CDP prompt injection & downloads watcher loop |
| * assemble: stitch multi-track video master with side-chain audio ducking |
+-----------------------------------------------------------------------------------------------+
[VIEW 2: DIRECTORIAL CONTROL & EDGE CDP BROWSER AUTOMATION ENGINE]
+-------------------------------------+ +----------------------------------------+
| board_server.py (Local HTTP :8765) | | edge_bridge.py (CDP Engine :9222) |
| - Serves storyboard.html & cockpit | JSON IPC | - Queries http://127.0.0.1:9222/json |
| - Shot status, take selection |<-------------->| - WebSocket CDP Target Binding |
| - Audio layer mixing staging | (Loopback) | - Automated prompt input injection |
| - Real-time director dashboard | | - Multi-tab orchestration (Veo/Kling) |
+-------------------------------------+ +-------------------+--------------------+
| DevTools Protocol
v
+----------------------------------------+
| Microsoft Edge Browser Running CDP |
| [Tab 1: Cockpit UI] [Tab 2: Gen AI UI] |
+-------------------+--------------------+
| Browser Video Download
v
[VIEW 3: 4D PERSISTENCE BUFFER, CLUE-FRAME EXTRACTION & MEDIA STAGING] |
+-------------------------------------+ +-------------------v--------------------+
| 4D Persistence Buffer (project.yaml)| | Ingestion & Watcher Engine (ingest.py) |
| * Characters: attire, facial, build| Continuity | - Polls ~/Downloads for generated takes|
| * Locations: lighting, architecture| Constraints | - Moves & hashes files to projects/ |
| * Props: persistent item tracking |--------------->| - Extracts Shot N tail clue-frame |
| * Palette: color mood, audio style | | - Feeds optical reference into Shot N+1|
+-------------------------------------+ +-------------------+--------------------+
|
v
+--------------------------------------------------------------------------+--------------------+
| Multi-Track Audio Staging & Master Assembly (assemble.py + FFmpeg) |
| - Track 1: In-Video Generated Audio / Veo Ambiance |
| - Track 2: Dialogue Voiceover (Synthesized locally via edge-tts or studio microphone) |
| - Track 3: Atmospheric Drone / Ambient Bed |
| - Track 4: Melodic Soundtrack & SFX Foley |
| => Dynamic Side-Chain Audio Ducking (lowers background music beneath dialogue) |
| => Lossless H.264 / AAC Concat Stitching to Master Video (projects//render/master.mp4) |
| => Optional EDL / XML Handoff to ai-media-editor for timeline trimming & color grading |
+-----------------------------------------------------------------------------------------------+
[VIEW 4: SECURITY BOUNDARY, AIR-GAP & RUNASINVOKER ZERO-EGRESS]
+-----------------------------------------------------------------------------------------------+
| Local Workstation Process Perimeter (INV-LOCAL-01 / INV-UNPRIV-02) |
| [Non-Elevation] Standard unprivileged user token; zero UAC or root requirement (RunAsInvoker)|
| [Air-Gap IPC] All sockets bound to 127.0.0.1; zero external LAN or WAN listeners |
| [Zero-Egress] Zero cloud analytics, tracking, telemetry, or third-party phone-home |
| [Subprocess] FFmpeg/FFprobe & edge-tts invoked with parameterized arrays (no shell=True) |
| [Sync-Defense] Multi-host collision guards (.gitignore LOCK*.txt, *conflicted copy*, etc.) |
| [SLA Guarantee] Formal 48h vulnerability response & 5-day triage commitment (INV-SLA-10) |
+-----------------------------------------------------------------------------------------------+
```
---
## 3. Director & Editor Duo
`clip-storyboard-director` is purposefully designed as the **Director** in a specialized two-agent media production model:
| Role | Tool | Focus & Responsibilities |
| :--- | :--- | :--- |
| **Director (Regisseur)** | `clip-storyboard-director` | **Pre-Production & Generation**: Script breakdown, shot timing, prompt engineering, 4D persistence buffer management, Edge CDP browser injection, clue-frame optical continuity, download auto-ingestion, and rough-cut master assembly. |
| **Cutter (Editor)** | `ai-media-editor` | **Post-Production**: Non-linear timeline editing, multi-track trim and split, color grading, fine transition curves, dynamic audio ducking, sub-frame cut synchronization, and final delivery rendering. |
> **Standalone Guarantee**: `clip-storyboard-director` operates 100% autonomously. It does not require `ai-media-editor` to generate, orchestrate, or assemble complete films. When `ai-media-editor` is available, seamless project handoff is supported via standard XML/EDL or project folder sharing.
---
## 4. 4D Persistence & Bounded Continuity
In filmmaking, continuity spans 3 spatial dimensions plus time ($3D + T = 4D$). `clip-storyboard-director` enforces continuity constraints through its declarative `persistence_buffer` in `project.yaml`:
- **Characters**: Persistent identifiers, attire details, physical builds, hair color, and facial traits.
- **Locations**: Spatial layouts, architectural styles, lighting conditions, and camera perspectives.
- **Props & Objects**: Key items, materials, wear and tear, and persistent physical state across cuts.
- **Atmospheric Palette**: Color grading mood, weather conditions, time-of-day progression, and acoustic acoustics.
Every prompt generated by the director automatically inherits the active persistence parameters, preventing model hallucination and style drift.
---
## 5. Clue-Frame Continuity Chain
The fundamental breakdown in multi-shot video generation occurs at the cut boundary. `clip-storyboard-director` introduces the **Clue-Frame Continuity Chain**:
1. **Deterministic Last-Frame Extraction**: Upon ingesting take $N$, `ingest.py` invokes `ffprobe` and `ffmpeg` to extract the exact final display frame ($T_{\text{end}}$).
2. **Optical Reference Anchoring**: The frame is stored under `projects//frames/shot_lastframe.png`.
3. **Automated Prompt Chaining**: When shot $N+1$ is staged, the director injects shot $N$'s clue-frame as the initial image prompt into the generator tab via CDP or copies it directly to the desktop workspace.
4. **Seamless Motion Handoff**: The AI model uses the clue-frame as its visual starting point, ensuring zero visual pop, identical camera framing, and seamless physical motion across cuts.
---
## 6. Dual-Pane Director Cockpit
The director features an integrated web-based timeline cockpit rendered via `start_cockpit.py` in Microsoft Edge App Mode (`--app=http://localhost:8765/cockpit.html`):
- **Left Pane (Storyboard Timeline)**: Complete scene cards with durations, active prompts, clue-frame previews, voiceover playback, and take selection controls.
- **Right Pane (Generation Studio)**: Embedded or parallel browser workspace connected via CDP for one-click prompt injection into Gemini, Kling, or Runway.
- **Real-Time Synchronized State**: Timeline modifications made in the browser cockpit instantly synchronize to `project.yaml` via local REST endpoints (`/api/projects/update_shot`).
### Cockpit Split-View Layout & Ergonomics
```text
+----------------------------------------------------------------------------------------------------+
| DUAL-PANE COCKPIT (Microsoft Edge App Mode - 127.0.0.1:8765/cockpit.html) |
+--------------------------------------------------+-------------------------------------------------+
| LEFT PANE: Storyboard Timeline & Directorial Deck| RIGHT PANE: Generative AI Studio & CDP Bridge |
+--------------------------------------------------+-------------------------------------------------+
| [Timeline Header] Shots 1..N | Total 01:20 | [CDP Bridge :9222] Status: Connected (Target AI)|
| ------------------------------------------------ | ----------------------------------------------- |
| [Current Shot Card: Shot 02 - Observatory Set] | [AI Generator Tab: Google Veo / Runway / Kling] |
| Duration: 10s | Camera: Dolly In, Slow Pace | Prompt Input Field: |
| Clue-Frame: [ shot01_lastframe.png ] | "Interior ancient mountaintop observatory... |
| Voiceover: [ de-DE-ConradNeural | Staged ] | Dr. Aris stands beside the bronze astrolabe..."|
| | |
| [Single-Source Prompt Engine (prompt_builder.py)]| [Image Prompt Reference]: |
| Mode: [x] Continuity [ ] Dialogue [ ] Clue-Prime| [ shot01_lastframe.png ] |
| Canonical Text: | |
| "Interior ancient mountaintop observatory..." | [Action Buttons]: |
| Buttons: [Inject via CDP] [Copy Prompt] [Handoff| [Generate Video] |
| ------------------------------------------------ | ----------------------------------------------- |
| [Hero Takes Matrix] Take 01 [x] Take 02 [ ] | [Downloads Watcher] Status: Listening |
| [Audio Deck] Track 1: Ambience | Track 2: Speech | Auto-ingest -> projects/film/video/shot02_v01 |
+--------------------------------------------------+-------------------------------------------------+
```
### Single-Source Prompt Engine (`prompt_builder.py`)
A critical bottleneck in generative video directing is prompt drift between the UI display, clipboard buffers, automated browser injection, and local file handoffs. `clip-storyboard-director` introduces a dedicated **Single-Source Prompt Engine** (`src/clip_director/prompt_builder.py`):
1. **Zero Divergence Guarantee**: All textual variants—interactive UI preview cards, one-click copy buttons, automated Chrome DevTools Protocol (CDP) socket payloads, and desktop handoff files (`~/Desktop/clip-director-next-prompt.txt`)—are compiled by a unified prompt builder function (`build_prompt`).
2. **Context-Aware Variant Modes**:
- `continuity`: Standard visual directive infused with 4D persistence buffer tokens (consistent character wardrobe, location architecture, color palette).
- `clue_prime`: Continuity directive explicitly anchored to the previous shot's optical tail-frame (`frames/shot_lastframe.png`).
- `dialogue`: Dedicated spoken script extraction for speech synthesis and voiceover actors.
3. **Live Server-Side Refresh**: The cockpit dynamically fetches freshly compiled prompt variants via `/api/cowork/focus`, completely eliminating stale client-side caches and ensuring that prompt adjustments in `project.yaml` propagate instantly across all execution paths.
---
## 7. Multi-Track Audio Staging & Ducking
Narrative cinema requires layered soundscapes. `clip-storyboard-director` provisions a 3-tier audio architecture:
1. **Dialogue & Voiceover Track**: High-fidelity speech synthesis via local `edge-tts` or live microphone recordings captured directly from the cockpit UI.
2. **Ambient Atmosphere & Foley**: Environmental sounds staged per scene (e.g. observatory hum, night wind, footsteps).
3. **Musical Score**: Emotional soundtrack with automatic gain ducking during spoken dialogue passages via FFmpeg `sidechaincompress` and `amix` filters.
---
## 8. Key Governance & Runtime Invariants
The following 10 invariants govern every execution of `clip-storyboard-director`:
| Guarantee | Invariant Code | Enforcement Mechanism | Verification Rule |
| :--- | :--- | :--- | :--- |
| **1. 100% Local-First Privacy** | `INV-LOCAL-01` | Zero telemetry, analytics, or external logging. | Source audit confirms zero remote data egress. |
| **2. Unprivileged Execution** | `INV-UNPRIV-02` | Standard user space (`RunAsInvoker`), no elevation. | `SECURITY.md` and runtime permissions verify non-root. |
| **3. Loopback IPC Isolation** | `INV-LOOPBACK-03` | Board server binds strictly to `127.0.0.1:8765`. | Port binding audit rejects `0.0.0.0` or external IPs. |
| **4. Subprocess Sandboxing** | `INV-SANDBOX-04` | Parameterized arrays for `ffmpeg` and `msedge`. | Zero `shell=True` invocations across all modules. |
| **5. 4D Persistence Continuity** | `INV-CONTINUITY-05` | Character/location buffer enforced in `project.yaml`. | Prompt compilation validates persistence inclusion. |
| **6. Clue-Frame Chaining** | `INV-CLUEFRAME-06` | Deterministic last-frame extraction per video take. | `ingest.py` verifies last frame creation on ingest. |
| **7. Standalone Decoupling** | `INV-STANDALONE-07` | Full operation without `ai-media-editor`. | `test_standalone_without_media_editor` passes. |
| **8. Multi-OS Parity** | `INV-MULTIOS-08` | `pathlib.Path` abstraction across Windows/Linux/macOS. | GitHub Actions CI matrix verifies multi-OS suite. |
| **9. Cloud-Sync Resilience** | `INV-SYNC-09` | Ignore patterns for locks (`LOCK.*`) and conflicts. | `.gitignore` contains conflict and mutex patterns. |
| **10. 48h Security SLA** | `INV-SLA-10` | 48h acknowledgment and 5-day triage commitment. | Documented in `SECURITY.md` and contract tests. |
---
## 9. End-to-End Media Production Lifecycle
```mermaid
sequenceDiagram
autonumber
actor Director as Human / LLM Director
participant Engine as clip-director Engine
participant Builder as PromptBuilder (prompt_builder.py)
participant Server as Board Server (:8765)
participant Edge as Edge Browser (CDP :9222)
participant AI as AI Generator (Gemini/Runway)
participant Watcher as Downloads Watcher
participant FFmpeg as FFmpeg Assembler
Director->>Engine: "clip-director init --name film --duration 60"
Engine->>Server: Start local board server & watcher
Director->>Server: Open Dual-Pane Cockpit
Server->>Edge: Launch Edge in App Mode with CDP
Server->>Builder: "Build prompt variants (Shot 1, active_mode)"
Builder-->>Server: Canonical prompt text
Director->>Edge: Inject Shot 1 Prompt via CDP
Edge->>AI: Generate Video Clip (Shot 1)
AI-->>Watcher: Download completed clip to ~/Downloads
Watcher->>Engine: Auto-ingest video to projects/film/video/
Engine->>FFmpeg: Extract last frame as Shot 1 Clue-Frame
Engine->>Server: Update timeline UI & clue-frame preview
Server->>Builder: "Build clue-primed prompt (Shot 2 + Clue-Frame)"
Builder-->>Server: Synchronized Shot 2 prompt
Director->>Edge: Inject Shot 2 Prompt + Clue-Frame Reference
Edge->>AI: Generate Consistent Video Clip (Shot 2)
AI-->>Watcher: Download completed clip
Watcher->>Engine: Ingest Shot 2 & extract next clue-frame
Director->>Engine: "clip-director assemble --project film"
Engine->>FFmpeg: Concatenate takes + mix voiceover & audio
FFmpeg-->>Director: film_master.mp4 ready for playback or editing
```
---
## 10. Target Personas & Discoverability
`clip-storyboard-director` is engineered to empower four primary user profiles across the generative media ecosystem:
| Target Persona | Key Pain Points | Core Solution & Value Proposition | Typical Workflow |
| :--- | :--- | :--- | :--- |
| **AI Filmmakers & Narrative Directors** | Face and wardrobe drift across shots, disjointed scene cuts, manual prompt juggling. | Declarative 4D Persistence Buffer (`project.yaml`) and automated optical clue-frame chaining between shot ends and starts. | `clip-director init` -> define 4D buffer -> review clue-frames -> assemble cut. |
| **Generative Media Engineers** | Web-based generator portals lack scriptable automation and structured asset ingestion. | Chrome DevTools Protocol (CDP) bridge on port 9222 and real-time downloads watcher with automatic asset sorting. | Launch Edge CDP bridge -> execute scripted take loop -> auto-ingest clips. |
| **Content Creators & YouTubers** | Manual speech synthesis, timing, and mixing with music beds in heavy editing suites is slow. | Integrated speech synthesis via `edge-tts` and automated side-chain audio ducking across four dedicated audio tracks. | Script scene dialogue -> `clip-director voice` -> `clip-director assemble`. |
| **Autonomous Coding Agents** | Cloud-dependent tools require OAuth/API tokens, elevate permissions, or fail without GUI. | 100% Local-first zero-egress architecture (`INV-LOCAL-01`), unprivileged `RunAsInvoker` CLI, and machine-readable `llms.txt`. | Headless execution via `clip-director auto` -> system check via `clip-director doctor`. |
### High-Intent Search Queries
- **English**: `ai storyboard director`, `scene continuity python`, `clue-frame video generator`, `local-first ai video orchestrator`, `veo kling runway automation`, `edge cdp video workflow`, `4d persistence buffer`, `automated audio staging and ducking python`.
- **German**: `clip-storyboard-director lokale KI-Regie`, `KI Storyboard Regisseur Python`, `Szenen-Kontinuität generative Videomodelle`, `Clue-Frame optische Bildverkettung`, `Lokale Video-Orchestrierung ohne Cloud-Zwang`, `Edge CDP Browser-Automatisierung KI Video`, `4D-Persistenzpuffer Charakter-Kontinuität`, `Mehrspur-Audio-Staging und Ducking FFmpeg`.
---
## 11. Comparative Matrix vs. Alternatives
The table below benchmarks `clip-storyboard-director` across 10 architectural and governance dimensions against 4 standard industry alternatives:
| Dimension / Capability | `clip-storyboard-director` | Commercial Video SaaS (Runway / Pika / Kling) | Manual Web Browser Workflow | Heavy Desktop NLEs (Premiere / DaVinci) | Ad-Hoc Python Scripts & Shell Glue |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **1. Local-First & Zero-Egress** | **100% Local-First (`INV-LOCAL-01`)**; zero cloud telemetry or media leaks. | ❌ Mandatory cloud SaaS; user media stored on remote servers. | ❌ Web-bound manual uploads; prone to cloud data leakage. | Local desktop app; heavy background telemetry services. | Varies; typically relies on unpinned cloud APIs. |
| **2. Scene Continuity & 4D Buffer** | **Built-in 4D persistence buffer (`INV-CONTINUITY-05`)** for characters, locations, objects. | ⚠️ Ephemeral prompt history; frequent character & wardrobe drift. | ❌ Manual prompt copy-pasting; severe context loss between shots. | ❌ No native generative continuity engine. | ❌ Fragile custom scripts; lack bounded 4D context models. |
| **3. Clue-Frame Optical Chaining** | **Automated tail-frame extraction (`INV-CLUEFRAME-06`)** seeding next shot prompts. | ❌ Manual end-frame extraction and re-upload per shot. | ❌ Tedious manual download, frame grab, and upload loop. | ❌ None built-in for AI generator prompts. | ⚠️ Ad-hoc FFmpeg scripts; lack automated seed validation. |
| **4. Browser Generator Automation** | **Direct Chrome DevTools Protocol (CDP)** bridge on port `9222`. | ❌ Proprietary closed web UIs without scriptable hooks. | ❌ 100% manual clicking, dragging, and parameter re-entry. | ❌ No direct browser-to-generator automation bridge. | ⚠️ Fragile Selenium or Playwright browser scrapers. |
| **5. Audio Staging & Multi-Track Ducking** | **Integrated 4-track mix + automated speech ducking** (`assemble.py`). | ❌ Minimal or single-track audio; no side-chain ducking. | ❌ No audio staging; requires external post-processing tools. | Full manual multi-track mixing; high manual time overhead. | ❌ Complex filtergraphs prone to audio desynchronization. |
| **6. Dual-Pane Visual Cockpit** | **Integrated split-screen dashboard** (Timeline & Generator Bridge). | ❌ Single tab view; disjointed asset previews. | ❌ Cluttered multi-window desktop and browser tabs. | Complex multi-panel editing interfaces requiring deep training. | ❌ Headless only; zero visual interactive timeline controls. |
| **7. Runtime Footprint & Level 1 SBOM** | **Lightweight Python stdlib + PyYAML / edge-tts**; Level 1 SBOM audited. | Heavy web frontend requiring continuous internet bandwidth. | N/A (human effort and browser memory consumption). | 5 GB+ proprietary desktop installers with heavy GPU footprint. | Unpinned virtualenvs; unpredictable dependencies. |
| **8. Agentic Discoverability (LLM-Ready)** | **Native `llms.txt` + `ellmos-module.v2.json`**; deterministic CLI interfaces. | ❌ Anti-bot scraping protections and cloud CAPTCHAs. | ❌ Incompatible with autonomous agentic execution. | ❌ Proprietary closed scripting APIs (Lua/ExtendScript). | ⚠️ Ad-hoc CLI flags without structured machine metadata. |
| **9. Privilege Model & User Safety** | **Unprivileged `RunAsInvoker` (`INV-UNPRIV-02`)**; loopback isolation (`127.0.0.1`). | Cloud SaaS account requiring billing and subscription tokens. | User browser session subject to session hijack risks. | Often requires administrative elevation during installation. | Varies; occasionally run as root or elevated shell. |
| **10. Security SLA & Governance** | **Formal 48h response & 5-day triage SLA (`INV-SLA-10`)**; § 521 BGB disclaimer. | Standard commercial SaaS Terms of Service. | N/A; individual browser user responsibility. | Vendor enterprise patch cycles; slow turnaround times. | ❌ Zero vulnerability response commitments or triage SLAs. |
---
## 12. Third-Party Licenses, Level 1 SBOM & RunAsInvoker
`clip-storyboard-director` adheres to strict open-source governance, permissive licensing, and zero-egress runtime invariants:
- **Core Module**: Licensed under the permissive [MIT License](LICENSE) with attribution declared in [NOTICE](NOTICE).
- **Runtime Dependencies**:
- `PyYAML` (MIT License): Declarative project configuration and persistence buffer parsing.
- `edge-tts` (GNU GPL-3.0): Standalone local speech synthesis subprocess.
- `websocket-client` (Apache-2.0): Loopback Chrome DevTools Protocol socket communication.
- `requests` (Apache-2.0): Loopback HTTP target discovery on `127.0.0.1:9222`.
- **External Binaries**:
- `FFmpeg / FFprobe` (LGPL-2.1+ / GPL-2.0+): System PATH media probing, clue-frame extraction, and audio-video stitching.
- `Microsoft Edge`: Optional host browser for dual-pane cockpit and CDP automation.
- **Compliance & Level 1 SBOM**:
- `INV-LOCAL-01`: 100% offline, zero cloud tracking, zero external telemetry.
- `INV-UNPRIV-02`: Operates strictly within unprivileged user mode (`RunAsInvoker`).
- Full Level 1 SBOM and Invariant Cross-Reference Matrix are cataloged in [THIRD_PARTY_LICENSES.md](THIRD_PARTY_LICENSES.md).
---
## 13. Sibling Tools & Ecosystem Matrix
`clip-storyboard-director` is part of the **ellmos-ai** infrastructure and the broader **open-bricks** open-source software family:
| Repository | Domain & Focus | Relationship to Director |
| :--- | :--- | :--- |
| [ai-media-editor](https://github.com/ellmos-ai/ai-media-editor) | Post-Production & Timeline Editor | **Cutter Partner**: Receives assembled cuts for non-linear finishing. |
| [ellmos-voice-io](https://github.com/ellmos-ai/ellmos-voice-io) | Speech Synthesis & Audio I/O | Provides advanced voice models and multi-character audio staging. |
| [system-auditor](https://github.com/ellmos-ai/system-auditor) | Multi-Host Audit Engine | Validates code quality, lock governance, and metadata parity. |
| [system-explorer](https://github.com/ellmos-ai/system-explorer) | Local Environment Introspection | Inspects hardware encoders (NVENC/VAAPI) and local codecs. |
| [ellmos-controlcenter-mcp](https://github.com/ellmos-ai/ellmos-controlcenter-mcp) | Governance & Capability Control | Enforces agent permissions and tool routing. |
| [ellmos-delegation-authority](https://github.com/ellmos-ai/ellmos-delegation-authority) | Autonomous Task Delegation | Orchestrates autonomous multi-shot production runs. |
| [sqlite-transit-sync](https://github.com/ellmos-ai/sqlite-transit-sync) | Zero-Conflict Database Sync | Synchronizes storyboard state across distributed hosts. |
| [memoryhooker-provenance](https://github.com/ellmos-ai/memoryhooker-provenance) | Context Hooking & Memory | Preserves narrative decisions across long-form video projects. |
| [workflowhooker-provenance](https://github.com/ellmos-ai/workflowhooker-provenance) | Workflow Tracking & Audit Trails | Captures prompt lineage, take variations, and model versions. |
| [automation-master](https://github.com/ellmos-ai/automation-master) | Enterprise Workflow Automation | Schedules background rendering and automated asset ingestion. |
| [automizer-for-claude-desktop](https://github.com/ellmos-ai/automizer-for-claude-desktop) | Claude Desktop Integration | Connects LLM directors directly to the timeline cockpit. |
| [WikiStub-Seed](https://github.com/ellmos-ai/WikiStub-Seed) | World-Building Knowledge Graphs | Generates detailed lore and character profiles for 4D buffers. |
| [ProSync](https://github.com/file-bricks/prosync) | Resilient File Mirroring | Mirrors video assets across editing workstations. |
| [CleanMarkdown](https://github.com/doc-bricks/cleanmarkdown) | Documentation Sanitizer | Normalizes scripts, screenplays, and directorial notes. |
| [PrivacyMailDesk](https://github.com/dev-bricks/privacymaildesk) | Local Mail Processing | Notifies producers upon render completions without cloud telemetry. |
| [prompt-archaeology-casestudy2](https://github.com/research-line/prompt-archaeology-casestudy2) | Prompt Engineering Research | Provides proven continuity prompts for video generation models. |
| [open-bricks](https://github.com/open-bricks/open-bricks) | Umbrella Open-Source Ecosystem | Architectural governance, security SLAs, and release standards. |
---
## 14. Installation & CLI Usage
### Requirements
- **Python**: 3.10, 3.11, 3.12, or 3.13
- **FFmpeg & FFprobe**: Required in system `PATH` for frame extraction and video assembly
- **Microsoft Edge** (or Chrome): For optional CDP browser automation
- **Optional Python Packages**: `edge-tts` (speech synthesis), `websocket-client` (CDP bridge)
### Installation
```bash
# Clone the repository
git clone https://github.com/ellmos-ai/clip-storyboard-director.git
cd clip-storyboard-director
# Install package in editable mode with development dependencies
pip install -e ".[dev]"
```
### System Health Check (`doctor`)
Verify all binaries, Python dependencies, and browser connections:
```bash
clip-director doctor
```
### CLI Command Reference
```bash
# 1. Initialize a new storyboard project (60s total, 10s per shot)
clip-director init --name my_film --title "My Narrative Film" --duration 60 --step 10
# 2. Render the interactive HTML timeline dashboard
clip-director render --project projects/my_film
# 3. Start the local board server (default port 8765)
clip-director serve --project projects/my_film --port 8765
# 4. Launch the dual-pane director cockpit in Microsoft Edge
clip-director cockpit --project my_film
# 5. Check Edge CDP connection status
clip-director cdp-status --port 9222
# 6. Watch downloads folder and automatically ingest new clips
clip-director ingest --project projects/my_film --watch
# 7. Synthesize dialogue voiceover for a specific shot
clip-director voice --project projects/my_film --step 1
# 8. Assemble final master video with mixed audio
clip-director assemble --project projects/my_film
# 9. Autonomous end-to-end production run
clip-director auto --project projects/my_film
```
---
## 15. Reference Production "Sternenseufzer"
The repository includes a complete reference production under `projects/sternenseufzer/` demonstrating the entire workflow:
- **Genre**: Sci-Fi Drama / Narrative Short
- **Plot**: An astronomer in a high-altitude observatory detects an anomalous acoustic resonance in starlight data.
- **Continuity Matrix**: Pre-configured 4D persistence buffer tracking Dr. Alistair Vance, the observatory dome, and the quantum spectrum analyzer.
- **Voiceover**: Bilingual directives (German source, English translation) and silent directives pre-generated.
Explore the reference project directly:
```bash
clip-director render --project projects/sternenseufzer
clip-director assemble --project projects/sternenseufzer
```
---
## 16. Security Policy & Privacy
- **Zero-Egress Guarantee**: `clip-storyboard-director` contains zero network telemetry, tracking pixels, or remote error reporting (`INV-LOCAL-01`).
- **RunAsInvoker**: Operates strictly within standard unprivileged user permissions (`INV-UNPRIV-02`).
- **Localhost Isolation**: Web services bind strictly to `127.0.0.1` (`INV-LOOPBACK-03`).
- **Security SLA**: We acknowledge vulnerability reports within 48 hours and provide initial triage within 5 business days (`INV-SLA-10`). See [SECURITY.md](SECURITY.md) for reporting channels.
---
## 17. Development, Verification & Quality Gates
### Running Linting & Type Checks
```bash
ruff check src tests
python -m compileall -q src tests
```
### Running Test Suite
```bash
pytest -v
```
---
## 18. Statutory Notice, Liability Limitation & License (§ 521 BGB)
### Statutory Notice & Limitation of Liability (§ 521 BGB)
The provision of this software is made free of charge as a statutory courtesy (*Gefälligkeit* / *unentgeltliche Schenkung* pursuant to **§ 521 BGB** of the German Civil Code). Under German statutory law, liability of the author and contributors is strictly limited to intent and gross negligence (*Vorsatz und grobe Fahrlässigkeit*).
### License
This project is licensed under the terms of the **MIT License**. See [LICENSE](LICENSE) and [NOTICE](NOTICE) for complete copyright and attribution details.
---
## 19. Sister Project: `ai-media-editor`
[`ai-media-editor`](https://github.com/ellmos-ai/ai-media-editor) is the **Cutter** to this
tool's **Director** in the same two-agent production model (see
[Director & Editor Duo](#director--editor-duo) above): it takes the assembled rough cut this
project produces and finishes it (non-linear editing, color grading, transitions, audio
ducking, final render). `clip-storyboard-director` operates 100% standalone and does not
require `ai-media-editor`; where present, project handoff works via standard XML/EDL or
shared project folders.
`ai-media-editor` declares `clip-storyboard-director` as its optional, commit-pinned
`storyboard` dependency (see its own README for the exact `pip install` command), so the
two repositories together cover both halves of the duo.
---
Part of the ellmos-ai suite under the open-bricks umbrella.