--- name: create-video-seedance-2-fal description: Generate a single 4-15s vertical video clip with ByteDance Seedance 2.0 reference-to-video via fal.ai. Multi-image reference (avatar + product + setting), native lip-synced VO + ambient audio (generate-audio on by default), internal multi-cut handling within one render. Routes through the GooseWorks FAL proxy (bills the Ads agent). The default clip atom for AI-creator UGC ads built on the NB2 + Seedance architecture. Validated on beauty-by-earth/video-01. owner: team status: superseded version: "2.0.2" created: 2026-05-26 updated: 2026-10-06 superseded_by: video-seedance-2@1.0.1 --- > **Superseded** in the video kit by the video-seedance-2 part (1.0.1; scripted lines use creator-h3). This atom still runs, unchanged, for skills outside the kit. # create-video-seedance-2-fal > ⚠️ **REQUIRED PREFLIGHT (2026-06-04): every Seedance prompt that names a branded product MUST be paired with a real reference image of that exact product** as one of the `--image-ref` inputs. Text-only product description without a ref invites Seedance to invent geometry that does not match the real SKU. If none exists, take one from the brand's assets, existing ads or product page; clean an occluded one with a gpt-image-2 edit first. ## Purpose Wraps the FAL endpoint `bytedance/seedance-2.0/reference-to-video`. Pass 1-N image references (typically: portrait + product + optional setting) + a structured prompt → get back a 4-15s clip with native lip-synced VO and ambient audio. Seedance 2.0 renders **multiple internal cuts within a single 15s call** when the prompt specifies sub-scene structure (e.g. WIDE HOOK / PRODUCT HERO / SIDESTEP / REACTION). This is the architectural win over the older NB2 + i2v multi-keyframe approach — fewer calls, native lip-sync, no Soul ID required. Use this atom when: - The clip needs a talking-head avatar reviewing or demonstrating a product - Native lip-synced dialogue is required (set `--generate-audio`) - Identity must hold via image reference, not video reference (see Decision Rule 1) - Length 4-15s, vertical 9:16 (or other aspect ratios) - Multi-image conditioning (face + product, or face + multiple products for a hook scene) **Do NOT use** for: - > 15s clips → split across multiple calls and stitch (`assembly/stitch-videos-ffmpeg`) - Pure cinematic camera moves without an on-screen person → `create-video-veo3` is often better - Product-only B-roll with no person → `create-product-videos-higgsfield-ms` or text-to-video Seedance - Clips that need video reference for continuity from a prior AI-gen scene → not supported (content_policy_violation) ## Inputs Required: - `--prompt` — structured prompt block (see "Prompt template" below). Long, multi-block. Reference run example: `beauty-by-earth/video-01-three-product-grwm/working/fire_seedance_facewash.py`. - `--output` — local MP4 destination. - `--image-url` (alias `--image-ref`) — at least one **PUBLIC** reference image URL (repeatable), passed as `image_urls`. Order matters — first = `@Image1` in prompt addressing. The proxy does NOT upload local files: host local refs via MCP `get_upload_url` → `get_download_url` and pass the URL (identical to `create-video-fal`). Optional: - `--resolution` — `480p` | `720p` | `1080p` (default). 1080p meaningfully better for product label fidelity. - `--duration` — `4`–`15` seconds (default `15`). Passed as an **int** (`seedance-2.0/reference-to-video` enum {auto,4..15}); a string 400s with `invalid_request` (validated 2026-07-18). - `--aspect-ratio` — `9:16` (default), `16:9`, `1:1`, etc. - `--generate-audio` — bool, default true. Set false for silent B-roll where VO is added post. - `--seed` — integer for deterministic re-runs (FAL returns a seed; pass it back to reproduce). - `--input-digest` — a stable id of the WHOLE request: `media_proxy.input_digest(model, payload)` over the full payload (prompt, resolution, duration, aspect ratio, audio, seed) with each ref URL replaced by its file's sha256 or ingredient key. Pass it when refs are re-hosted between runs: a recorded rejection then still matches, and the proxy re-attaches to a running job instead of paying twice. **It becomes a permanent refusal key after a policy rejection**, so it must cover every input: leave the prompt or an image out and a run that changed only that input is refused. Credentials: - **No FAL key.** Routes through the GooseWorks FAL proxy (`media_proxy.py`, bundled) and bills the Ads agent, using `~/.gooseworks/credentials.json` (written by the `gooseworks` CLI). Your `cal_`/agent token is not a FAL key — the old direct-key path 401'd; that's why this capability was rerouted through the proxy. ## Decision Rules **0. Prefer FEWER, LONGER calls with internal multi-scene prompting over many short calls.** Every new Seedance call drifts character, wardrobe, lighting, and background slightly — even with the same portrait reference. 5 short calls = 5 drift points the viewer registers as continuity breaks. Use the proven multi-scene template inside ONE 10-15s call to get internal hard-feeling cuts from a single render: ``` SCENE 1 (0-3s) — WIDE HOOK [framing + action + dialogue] SCENE 2 (3-7s) — PRODUCT HERO [framing + camera move + dialogue] SCENE 3 (7-11s) — SIDESTEP / PROOF [framing + action + dialogue] SCENE 4 (11-15s) — REACTION + CTA [framing + action + dialogue] ``` **Setting CAN vary across internal sub-scenes within one Seedance call.** Character identity is locked by the portrait reference image, not by the setting. Validated reference (ref-01 Bristle UGC, 26s) shows the same creator across 5+ different room/background settings within a single multi-scene render. Don't artificially split into multiple calls just to change rooms — bake the setting changes INTO the SCENE N (X-Ys) sub-scene structure of a single call. Distribute calls based on STORY ARC and the 15s max-duration cap, not setting count. See `prompt-example.md` for the validated template. Memory: `feedback_seedance_long_calls_not_short.md`. **0a. WARDROBE must be FULLY specified — top + bottom + hair + accessories.** Validated 2026-05-23 (Bristle): if only the top is specified ("cream sweatshirt"), Seedance hallucinates the bottom (denim shorts appeared from nowhere). Every prompt must lock all of: - Top (with negations) - Bottom (with negations) — "light denim cuffed shorts, mid-thigh, NOT jeans, NOT pajama shorts" - Hair (specific styling — "loose low ponytail at the nape with face-framing strands" works better than vague "bun") - Accessories (e.g. "thin gold hoop earrings, nothing else") For confessional / single-sitting UGC, wardrobe should be IDENTICAL across all calls of the same ad (validated user preference + ref-01 + ad-03 patterns). Vary wardrobe across calls ONLY for GRWM / transformation / multi-day formats. Memory: `feedback_seedance_long_calls_not_short.md`. **0b. PASS A BACKGROUND REFERENCE IMAGE as `image_urls[2]` for locked settings.** Validated 2026-05-23 (Bristle). Text-only setting description yields generic-rental aesthetic — Seedance fills with its own room prior. A background hero image (generated via NB2 Pro at `fal-ai/nano-banana-pro`, $0.15/1K, text-to-image with "no bokeh, infinite focus, no people, no faces" directives) LOCKS the room across all sub-scenes. Use the SAME background image across all calls of the same ad — different bg image per call reads as "she moved." Memory: `feedback_seedance_long_calls_not_short.md`. **0c. PRE-CLEAN PRODUCT REFERENCE IMAGES via NB2 edit if they contain accessories.** Validated 2026-05-23 (Bristle hi_res-12 had vials + funnel beside the box; Seedance fused them onto the box face). Inspect source product images at intake. If accessories / secondary products / score badges / lifestyle elements visible, route through `fal-ai/nano-banana/edit` (quote the current proxy price first) to isolate the hero product before passing to Seedance. Approve the cleanup cost before calling it; compare with the current quoted cost of a Seedance retry. Memory: `feedback_nb2_clean_product_refs.md`. **0d. SEQUENTIAL CALL CONTINUITY — use seed reuse, NOT video_urls.** Validated 2026-05-23 (Bristle Variant A/B test). For Call N>1 in a multi-call ad, pass the prior call's `seed` (via `--seed`) with the same `image_urls`. NEVER pass the prior call's mp4 as `video_urls` for sequential continuity — it HURTS continuity (model tries to "continue" prior video; wardrobe/product drift more than they otherwise would). `video_urls` is ONLY for style transfer from a REAL HUMAN reference video (e.g. take a real-creator UGC ad and recreate with AI creator). Memory: `feedback_seedance_video_urls_vs_seed.md`. **0e. WHEN USING `video_urls` (style transfer from real reference): set client timeout to 30+ min AND trim reference to ≤15s.** Validated 2026-05-23: - Render time: ~18-20 min (vs ~10 min for image-only). The default `fal_client.subscribe()` 10-min timeout will fail. Use `fal_client.submit()` + manual polling. - **Combined `video_urls` duration cap: 15s.** FAL rejects with `input_value_error: Combined video duration must not exceed 15.0s`. Trim reference videos with ffmpeg `-t 15` before upload. - **Combined files cap: 50 MB total.** Our `fal_helpers.upload_file()` cap bumped from 10 MB to 50 MB on 2026-05-23. - Detect validation errors (`input_value_error` / "must not exceed") in polling exceptions and bail after 1-2 retries — terminal errors don't surface via `status=Failed`. - **Video reference transfers MORE than style/pacing — it transfers wardrobe styling, accessories, hair to a non-trivial degree** (validated 2026-05-23 Probiotic Recreation: hair flipped to match reference; tablets appeared despite cleaned product image). Text-side overrides are PARTIALLY effective; pick a reference whose visuals already align with your target. - FAL DOES accept AI-gen video as input (the old "instant content_policy_violation" assumption from BBE 2026-05 early is outdated). Memory: `feedback_seedance_video_urls_vs_seed.md`. **1. NEVER pass AI-generated video as `video_urls`.** FAL's content-policy validator rejects this with `partner_validation_failed`. The 60% video-input discount is real but only applies to REAL HUMAN reference video. For AI-avatar continuity across multiple scenes, use the same identity portrait as `image_urls` in every scene. Validated on BBE. **2. NSFW reject → STOP and surface, do not auto-retry (exit 4).** Body-application + female + water/lather hits the classifier reliably. If FAL returns `content_policy_violation` or `status: failed` with `nsfw` reason, do NOT retry the same prompt. Surface to the caller with 3 options: rewrite the application beat as smell-test / fingertips-show, reframe as POV (no face), or skip the scene. Follows the project-wide moderation policy (see memory `feedback_hf_moderation_surface.md`). **3. `duration` as an INT.** Pass `15`, not `"15"`. `seedance-2.0/reference-to-video` rejects a string duration with `invalid_request` (validated 2026-07-18; the old "string only" note was the deprecated v1 i2v endpoint). **4. Single product per call (except hook).** Multi-product hero scenes within a single Seedance call degrade label fidelity. The exception is the hook/intro scene where all products appear together but no single label is hero. **5. 1080p for hero quality.** Skin texture and product label crispness scale noticeably with resolution. Use 720p only for budget/probe calls. ## Workflow ```bash python3 scripts/generate.py \ --prompt "$(cat prompt-scene-1.txt)" \ --output /path/to/scene-1.mp4 \ --image-url "https:///portrait.png" \ --image-url "https:///product.png" \ --resolution 1080p \ --duration 15 \ --aspect-ratio 9:16 \ --generate-audio ``` The script: 1. Routes through the bundled `media_proxy.py` (GooseWorks FAL proxy) — no FAL key; bills the Ads agent via `~/.gooseworks/credentials.json`. 2. Passes the `--image-url` values straight through as `image_urls` (they must already be PUBLIC URLs; the orchestrator hosts local refs via MCP). 3. Submits `bytedance/seedance-2.0/reference-to-video` through the proxy and polls to completion with: ```python { "prompt": , "image_urls": [], "resolution": , "duration": , # int "aspect_ratio": , "generate_audio": , "seed": , } ``` 4. Downloads the result MP4 to `--output`. 5. Writes `.meta.json` with prompt, refs, video URL, seed, duration, cost estimate. 6. On a policy rejection: one submit, no retry. Prints the reason, type, request id and charge state, writes `.rejection.json`, and exits **3** (likeness, `content_policy_violation`, `partner_validation_failed`) or **4** (NSFW / safety checker). See "Provider rejection and scene acceptance". Exit codes: `0` done · `1` other error · `3` policy rejection, or the same request already rejected · `4` NSFW rejection. The MCP relay also exits 3 when it needs the agent to make a call; it prints `[mcp-relay]` instead of "surface, do not retry". ## Output - `` — MP4 clip at requested duration, resolution, aspect ratio. - `.meta.json` — request + result metadata + seed for reproducibility + cost estimate. - `.rejection.json` — only after a policy rejection: reason, kind, type, request id, HTTP status, charge state, exit code. Removed by the next successful render to the same path. ## Pricing (2026-05) | Resolution | Duration | Cost per call | |---|---|---| | 720p std | 15s | ~$4.54 | | 1080p std | 15s | ~$10.20 | | 1080p std | 8s | ~$5.44 | | 1080p std | 4s | ~$2.72 | Fast tier is ~20% cheaper but caps at 720p. ## Prompt template The prompt is the single biggest quality lever. Use the proven BBE template structure: ``` 📱 UGC INFLUENCER [PRODUCT] REVIEW REEL (SELFIE-STYLE, NOT A COMMERCIAL) [BLOCK 0 — anti-pattern opener] This is NOT a cinematic commercial, NOT a [category] ad, NOT an editorial campaign. This is a real Instagram Reel filmed by a 20-something [demographic] creator on her iPhone front camera, talking directly to her followers in a relaxed, confiding tone... [IMAGE REF ASSIGNMENTS] Use @Image1 as the influencer's face/identity. Use @Image2 as the exact [product] she is reviewing — it is a [VISUAL DESCRIPTION]. It is NOT a [common confusion]. The product and label MUST be clearly identifiable. [FORMAT block] - Vertical 9:16 phone-shot selfie aesthetic, slight handheld micro-shake - WARDROBE (STRICT): [wardrobe]. ABSOLUTELY NOT [list of negations]. - Setting: [setting] - Lighting: [lighting] - Real, slightly imperfect skin (natural barely-any makeup), light golden tan [IDENTITY block] Same young woman as @Image1 — [age range, hair, eyes, skin tone] [DYNAMIC FRAMING block] The video must vary its framing across scenes like a real Reel — do NOT stay on one focal length the whole time. Mix wide / medium / close-ups. [SCENE 1 (0-3s)] WIDE HOOK — [script line] + [emotion + action] [SCENE 2 (3-7s)] PRODUCT HERO with rotation — [script line] + [emotion + action] [SCENE 3 (7-11s)] SIDESTEP application or smell-test — [script line] + [emotion + action] [SCENE 4 (11-15s)] REACTION + recommendation — [script line] + [emotion + action] [AUDIO DIRECTION] [Voice character: tone, accent, pace per DELIVERY_STYLES.md] NO music in the audio (we add music in post). [GLOBAL RULES] - Wardrobe identical across the scene - Product stays identical and clearly visible in her hand - Lips remain CLOSED between dialogue beats - [NSFW sidestep negations if relevant] [STRICT REALISM RULES — Block F] - Tiny natural imperfections, subtle peach fuzz, faint fine lines - NOT plastic, NOT airbrushed, NOT doll-like, NOT porcelain - Skin highlights ONLY on nose tip, cheekbones, cupid's bow - Hair: individual flyaway strands, slight frizz - Image grain matches iPhone front camera ``` Full template reference: `prompt-example.md` at the repo root, plus all four scripts in `beauty-by-earth/video-01-three-product-grwm/working/fire_seedance_*.py`. ## Quality Checks - Output MP4 exists, plays end-to-end, duration within ±0.5s of `--duration`. - Aspect ratio and resolution match the request. - Avatar identity matches `--image-ref` portrait reference (visual check). - Product label readable in the hero sub-scene (zoom in on the t=mid frame). - Lip-sync within ~100ms of the audio. - No NSFW visual leaked (no water/lather/body-application unless explicitly approved). - `meta.json` records `gateway: "fal"`, `model: "bytedance/seedance-2.0/reference-to-video"`, and the FAL seed. ## Failure Modes | Failure | Cause | Recovery | |---|---|---| | FAL 422 `content_policy_violation: partner_validation_failed` | AI-gen scene passed as `video_urls` | Remove `video_urls`. Use `image_urls` only for identity continuity. | | FAL 422 "may contain likenesses of real people" (exit 3) | A photoreal face in `image_urls` read as a real person | STOP. Surface the reason, request id and charge state. Re-running the same command is refused locally. Offer a permitted route with its cost (see below). | | FAL response `status: failed`, `nsfw` reason | Body-application + female + water/lather hit classifier | STOP. Surface to caller. Rewrite sidestep beat as smell-test or fingertips-show. NEVER auto-retry. | | FAL 404 on endpoint path | Wrong endpoint slug | Use `bytedance/seedance-2.0/reference-to-video` exactly — no `fal-ai/` prefix. | | `invalid_request` on submit | `duration` sent as a **string** | Send an **int**: `"duration": 15` (enum {auto,4..15}). | | Product label garbled / wrong | Multi-image refs drifted, or 720p | Move to 1080p, ensure product ref is sharp + correctly cropped, add "label MUST be sharp and clearly readable" to the hero sub-scene. | | Mouth moving when not speaking | Default Seedance behavior | Add to Global Rules: "Lips remain CLOSED between dialogue beats — no mouth movement when not speaking." | | Cinematic slow pacing despite "iPhone selfie" prompt | Seedance prior leans cinematic | Strengthen anti-pattern block: "Casual real-time pace, NOT slow-motion, NOT dolly moves, NOT cinematic." Optionally post-process with `ffmpeg setpts=0.66*PTS` for 1.5x speed. | | Heavy freckles / blotchy skin | Block B language too aggressive | Use v8 soft-skin language. Drop "freckles prominent", "real-person imperfections". Replace with "tiny natural imperfections, subtle peach fuzz, faint fine lines". | | Wrapper RuntimeError "fal subscribe exceeded timeout" AFTER long elapse (>600s on 1080p) | Shared `fal_helpers.subscribe()` post-call timeout check raises even when the result returned successfully (latent bug; default bumped to 2400s on 2026-05-26 but recovery still applies for older runs) | **The render likely completed on FAL — DO NOT re-fire.** See Recovery Workflow below. | | FAL `downstream_service_unavailable` after long wait (~30 min) | Seedance backend overloaded / transient infra issue | Not billed (FAL bills on completion). Retry once. If retry also fails, wait 30 min then retry. Don't change prompt — it's infra, not your input. | | **Brand name mispronounced in native-audio output** (validated 2026-06-08 on Alitu A3: rendered "Alitu" as "al-too" instead of "ah-LEE-too") | Seedance picks its own voice + interprets non-English / made-up brand names phonetically without dictionary knowledge | **Preflight:** use phonetic spelling directly in the dialogue (NOT a parenthetical). E.g. `SPOKEN LINE: "Ali-too does the engineering for you."` Avoid `Dialogue: "Alitu (pronounced ah-LEE-too)"` — Seedance reads the parenthetical out loud (see next row). **Post-render QC:** /watch:watch and confirm the Whisper transcript matches expected pronunciation. **Recovery:** re-roll the affected sub-scene with phonetic spelling baked into the line, OR splice an ElevenLabs reading of the brand-name beat over the Seedance visual. | | **Prompt metadata leaks into spoken line** (validated 2026-06-08 on Alitu A3 v2: rendered "Alitu, pronounced Ali-tu, three syllables, does the engineering for you" — Seedance read the parenthetical voice-coaching out loud as if it were part of the dialogue) | Seedance does not distinguish "spoken text" from "voice coaching" inside a `Dialogue:` field. Parentheticals, notes, "(pronounced X)", "(softly)", "(stage whisper)" all get verbalized. | **Prevention:** keep voice-coaching OUTSIDE the dialogue field entirely. Use a separate `SPOKEN LINE:` heading whose content is the EXACT verbatim utterance, no parens, no notes. Put coaching under `NEGATIONS:` as "DO NOT say X / DO NOT explain pronunciation". Bake phonetic intent into the dialogue spelling itself (`Ali-too` not `Alitu`). | ## Recovery Workflow — recover a lost result via FAL share URL When a Seedance render is paid for but the result was discarded by the wrapper / shell / timeout, the mp4 is still accessible via FAL's dashboard for 24-48+ hours. 1. **Ask the user** to open `https://fal.ai/dashboard/requests` (or the share-link view of the model: `https://fal.ai/models/bytedance/seedance-2.0/reference-to-video?share=`) signed in to the FAL account that holds `FAL_API_KEY`. 2. Filter by model = `bytedance/seedance-2.0/reference-to-video` and look for the request matching the lost run's timestamp. 3. The user copies the share URL (contains `?share=`) and pastes it back. 4. `WebFetch` the share URL extracting "the v3.fal.media/files/.../*.mp4 URL plus the seed and any other metadata". The page renders all of this verbatim. 5. `curl -sSL -o .mp4 ""` to grab the file. Zero extra cost. 6. Save metadata to `.mp4.meta.json` (seed, share URL, mp4 URL, recovery note) so downstream calls can use the seed for continuity. **Prevention:** Fire long Seedance calls with `--with-logs` so queue-update messages including the request_id stream to the atom's stdout. If the wrapper later errors, the request_id is recoverable from the background task output file at `/private/tmp/claude-501/.../tasks/.output` and can be passed to `fal_client.result(request_id)` directly. Memory: `feedback_fal_helpers_timeout_bug.md`. Validated: Lineage Video 01 Call 1 (2026-05-26) — recovered $8.83 render via share URL. ## References - FAL model page: https://fal.ai/models/bytedance/seedance-2.0/reference-to-video - Reference runs: `beauty-by-earth/video-01-three-product-grwm/working/fire_seedance_{facewash,deo_probe,hook,tanner}.py` - Proven prompt template: `prompt-example.md` - Sibling NB2 atom (avatar / product stills): `coworkers/video/atoms/image-generation/create-image-nano-banana-2-fal` - Composing molecule: `coworkers/video/molecules/ugc-ad/create-ugc-style-video` - Shared helpers: `coworkers/video/atoms/_shared/fal_helpers.py` ## Provider rejection and scene acceptance `generate.py` enforces the first part. A rejection in any body shape (fal's `detail` with `msg` and/or `type`, or the GooseWorks `provider_validation_failed` wrapper) exits 3 after exactly one submit. The reason, request id and charge state go to stderr and `.rejection.json`. The exact request is recorded (media-proxy's rejected-request ledger), so running the same command again exits 3 without sending anything. Changing the prompt, an image, the seed or the model makes a new request. `charged: false` means the GooseWorks proxy did not debit the explicit 4xx (read from the proxy code; one paid reproduction is still needed to confirm it). Preserve the exact rejection reason, request id and charged/uncharged/unknown state. Stop identical rejected submissions; do not batch them or assume a provider switch is a policy bypass. Check which references the selected provider permits and review a permitted original character or non-likeness route, including changed cost, before another paid step. Historical incidents do not establish current universal policy or billing. Record wearable counts/body locations, holding hands, object contacts and permitted gestures before generation. Review every generated frame against that checklist. Pin a reference per cast member and map speakers to lines. Compare fewer people per shot, shared referenced scenes and separately composed plates. Multi-character success rates and a six-character/two-attempt target remain unmeasured until a separately approved benchmark retains both successes and failures.