"""Character and location reference sheets from a MiniMax-H3 camera move. A reference sheet generated angle-by-angle from text drifts: the same courtyard comes back with a different arch count, the same character with a different collar. H3 does not drift, because every angle is the *same shot* — one continuous move around a still subject yields views that genuinely agree with each other. The remaining job is picking the useful frames, which is what this pack does. Four nodes, each covering a step ComfyUI has no answer for: * Location Sheet Prompt writes a 4-view location prompt to H3's spec * Character Turnaround Prompt same for a figure, plus an expression shot * Frame Select picks the frames worth keeping * Contact Sheet lays them out as a sheet Generation stays ordinary graph wiring — an image model for the opening reference, `MiniMaxH3ImageToVideo` plus a sampler for the move — so seeds, steps and models remain tunable where they belong. See `example_workflows/` for the two complete graphs. Frame selection asks a vision-language model to compare the candidates as one contact sheet, and falls back to sharpness-and-spread scoring whenever no such model is reachable. Nothing here requires a specific server: any OpenAI-compatible vision endpoint works, and the pack has no dependencies outside ComfyUI's own runtime. """ from __future__ import annotations import base64 import io import json import logging import re import numpy as np import torch import torch.nn.functional as F #: Frames the vision model is asked to compare in one montage. Beyond roughly #: this many the tiles get too small for it to judge sharpness reliably. DEFAULT_CANDIDATES = 16 #: Framing for the turnaround. Generous margin is the default because it is #: the one a reference sheet usually fails without: a dragon's wingspan, a tail #: or a held weapon outstretches a 16:9 frame, and H3 faithfully crops whatever #: the prompt did not insist stays inside it. _FRAMINGS = ["full body, generous margin", "full body, tight"] _FRAMING_CLAUSES = { "full body, generous margin": ( " Every full-body shot is framed with generous empty margin on every " "side of the figure, so that the whole figure and anything extending " "beyond its body stays fully inside the frame at all times. No part " "of the figure may ever touch, cross or be cut off by the edges of " "the frame." ), "full body, tight": ( " Every full-body shot is framed with tight cropping, the figure " "nearly filling the frame from edge to edge but never cropped at any " "point." ), } #: How far the camera turns, in ascending order. Stating the angle is what #: replaces H3's "large/small amplitude" here: the angle *is* the amplitude, #: and saying both invites the model to average two instructions. The angle and #: the take length between them also fix the speed, so there is no speed widget #: either — a turn is written "at slow speed" or it is not written at all. _TURNS = [ (90.0, "a quarter turn of 90 degrees"), (180.0, "a half turn of 180 degrees"), (360.0, "a complete turn of all 360 degrees"), ] #: Degrees per second above which the move stops being a camera move and #: becomes a whip. H3's own worked example budgets a full 360 across fifteen #: seconds — 24 deg/s — and asking for three times that is what makes the #: frames tumble: with no way to turn that fast and stay level, the model #: substitutes the roll and tilt it *can* render at that rate. Nothing past #: this limit is written into a prompt; `_fit_turn` clamps it first. _ROTATION_RATE_LIMIT = 40.0 #: The widget. `None` means "as far as this take can hold", which is the #: default because it is the only setting that cannot be wrong. The others say #: what they need in the label, since the label is the only documentation a #: dropdown ever gets read with. _ROTATION_CHOICES = { "auto (as far as the take allows)": None, "quarter turn (90 degrees)": 90.0, "half turn (180 degrees)": 180.0, "full turn (360 degrees, needs a 9s take)": 360.0, } def _fit_turn(rotation, take_seconds): """Pick the turn to write. Never returns one the take cannot hold. Returns (degrees, phrase, note): `note` is None when the request was honoured and a sentence explaining the clamp when it was not. """ span = max(float(take_seconds), 1.0) ceiling = _ROTATION_RATE_LIMIT * span fits = [t for t in _TURNS if t[0] <= ceiling] or _TURNS[:1] wanted = _ROTATION_CHOICES.get(rotation, None) if wanted is None: # auto, or a stale widget value return fits[-1][0], fits[-1][1], None degrees, phrase = next(t for t in _TURNS if t[0] == wanted) if degrees <= ceiling: return degrees, phrase, None capped, capped_phrase = fits[-1] return capped, capped_phrase, ( f"{rotation} across {span:.1f}s is {degrees / span:.0f} deg/s, past " f"the {_ROTATION_RATE_LIMIT:.0f} deg/s this model holds level. " f"Writing {capped:.0f} degrees instead, which fits. For the full " f"{degrees:.0f}, give the take more frames: {degrees / _ROTATION_RATE_LIMIT:.0f}s " f"needs length {int(round(degrees / _ROTATION_RATE_LIMIT * 24 / 4)) * 4} " f"on MiniMaxH3ImageToVideo, and take_seconds to match." ) #: How long the sheet's video runs, in seconds. 124 frames on H3's 17k+5 grid #: is 5.17s at 24fps, and the shots divide that span rather than each taking a #: whole second — six shots simply make each one shorter, which H3 follows #: perfectly well and which costs no extra frames. SHEET_SECONDS = 5.0 def _timecode(seconds: float) -> str: """Seconds -> H3's MM:SS.mmm cut marker.""" minutes, rest = divmod(seconds, 60.0) return f"{int(minutes):02d}:{rest:06.3f}" # ---------------------------------------------------------------- utilities def _tensor_to_pils(images: torch.Tensor) -> list: """ComfyUI IMAGE (B,H,W,C float 0..1) -> list of PIL images.""" from PIL import Image arr = (images.detach().cpu().numpy() * 255.0).clip(0, 255).astype(np.uint8) return [Image.fromarray(arr[i]) for i in range(arr.shape[0])] def _pils_to_tensor(pils: list) -> torch.Tensor: """PIL images -> ComfyUI IMAGE. Sizes must already agree.""" stack = [np.asarray(p.convert("RGB"), dtype=np.float32) / 255.0 for p in pils] return torch.from_numpy(np.stack(stack, axis=0)) def _load_font(size: int): """A legible truetype face if the box has one, else PIL's bitmap default. The default face does not scale, so labels on a 512px tile would be unreadable; every common Linux image ships at least one of these. """ from PIL import ImageFont for path in ( "/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf", "/usr/share/fonts/truetype/liberation/LiberationSans-Bold.ttf", "/usr/share/fonts/TTF/DejaVuSans-Bold.ttf", ): try: return ImageFont.truetype(path, size) except Exception: continue return ImageFont.load_default() def _montage(pils: list, columns: int, cell_width: int, padding: int, labels: list | None, background: tuple = (18, 18, 20)): """Grid the images, optionally stamping a label on each tile.""" from PIL import Image, ImageDraw if not pils: raise ValueError("No frames to lay out.") columns = max(1, min(columns, len(pils))) rows = (len(pils) + columns - 1) // columns ratio = pils[0].height / pils[0].width cell_h = max(1, int(round(cell_width * ratio))) sheet_w = columns * cell_width + padding * (columns + 1) sheet_h = rows * cell_h + padding * (rows + 1) sheet = Image.new("RGB", (sheet_w, sheet_h), background) draw = ImageDraw.Draw(sheet) font = _load_font(max(14, cell_width // 16)) for index, pil in enumerate(pils): row, col = divmod(index, columns) x = padding + col * (cell_width + padding) y = padding + row * (cell_h + padding) sheet.paste(pil.convert("RGB").resize((cell_width, cell_h)), (x, y)) if not labels: continue text = str(labels[index]) # A filled plate behind the text: a bare glyph vanishes against a # bright sky or a pale wall, and these tiles are exactly that. try: box = draw.textbbox((0, 0), text, font=font) tw, th = box[2] - box[0], box[3] - box[1] except Exception: tw, th = 8 * len(text), 14 pad = max(4, cell_width // 96) draw.rectangle( [x, y, x + tw + pad * 2, y + th + pad * 2], fill=(0, 0, 0) ) draw.text((x + pad, y + pad), text, fill=(255, 255, 255), font=font) return sheet def _pil_to_data_uri(pil) -> str: buf = io.BytesIO() pil.convert("RGB").save(buf, format="JPEG", quality=90) return "data:image/jpeg;base64," + base64.b64encode(buf.getvalue()).decode() # ------------------------------------------------------- classical scoring def _sharpness(images: torch.Tensor) -> torch.Tensor: """Variance of the Laplacian per frame — the standard blur detector. An orbit spends part of its arc mid-motion, and those frames are soft. A soft frame is useless as a reference no matter how good its angle is. """ gray = images.mean(dim=3).unsqueeze(1) # B,1,H,W kernel = torch.tensor( [[0.0, 1.0, 0.0], [1.0, -4.0, 1.0], [0.0, 1.0, 0.0]], dtype=gray.dtype, device=gray.device, ).view(1, 1, 3, 3) lap = F.conv2d(gray, kernel, padding=1) return lap.view(lap.shape[0], -1).var(dim=1) def _descriptors(images: torch.Tensor) -> torch.Tensor: """A coarse, brightness-normalised thumbnail per frame. Contrast-normalising means two views separate by *layout*, not by one being a stop brighter — which is what "a different angle" actually means. """ gray = images.mean(dim=3).unsqueeze(1) small = F.interpolate(gray, size=(16, 16), mode="area").view(gray.shape[0], -1) small = small - small.mean(dim=1, keepdim=True) return small / (small.std(dim=1, keepdim=True) + 1e-6) def _view_descriptors(images: torch.Tensor) -> torch.Tensor: """A per-frame signature fine enough to tell two camera angles apart. Colour is kept and the resolution is higher than `_descriptors` because this one has to separate views a coarse grey thumbnail merges: a left profile from a right profile (mirror images, identical silhouette area) and a front from a back (same outline, different content). Each frame is normalised on its own so the comparison is of layout, not exposure. """ small = images.permute(0, 3, 1, 2) # B,C,H,W small = F.interpolate(small, size=(32, 32), mode="area") small = small.reshape(small.shape[0], -1) small = small - small.mean(dim=1, keepdim=True) return small / (small.std(dim=1, keepdim=True) + 1e-6) def _cluster_views(images: torch.Tensor, k: int) -> list[list[int]]: """Group frames by what they show, returning k groups of frame indices. This is the answer to shot boundaries that move. Splitting the timeline by time — evenly, or even at correctly detected cuts — only works if each slice happens to hold a different view, and across runs it does not: the model gives one view three seconds and another half a second, so two slices come back with the same angle and a third view never reaches the sheet at all. Grouping by appearance cannot make that mistake. Five distinct views are five clusters wherever the cuts happen to fall, and near-duplicate frames land in one cluster instead of consuming two slots. Plain k-means over `_view_descriptors`, with farthest-point seeding rather than random, so the same video always yields the same sheet. """ total = int(images.shape[0]) k = max(1, min(k, total)) desc = _view_descriptors(images) # Farthest-point seeding: start from the anchor frame and repeatedly take # the frame least like anything seeded so far. That lands one seed on each # genuinely distinct view, which random seeding routinely fails to do — # and it is deterministic, so a re-run reproduces the sheet exactly. seeds = [0] while len(seeds) < k: dist = torch.cdist(desc, desc[seeds]).min(dim=1).values dist[torch.tensor(seeds, device=dist.device)] = -1.0 seeds.append(int(torch.argmax(dist).item())) centres = desc[seeds].clone() labels = torch.zeros(total, dtype=torch.long) for _ in range(25): labels = torch.cdist(desc, centres).argmin(dim=1) moved = False for c in range(k): members = labels == c if not bool(members.any()): # An emptied cluster would silently cost a view, so re-seed it # on the frame currently worst served by any centre. worst = int(torch.cdist(desc, centres).min(dim=1).values.argmax()) centres[c] = desc[worst] moved = True continue mean = desc[members].mean(dim=0) if not torch.allclose(mean, centres[c]): centres[c] = mean moved = True if not moved: break labels = torch.cdist(desc, centres).argmin(dim=1) return [ [i for i in range(total) if int(labels[i]) == c] for c in range(k) ] def _greedy_spread(images: torch.Tensor, count: int, sharpness_weight: float, forced_first: bool) -> list[int]: """Farthest-point selection, biased toward frames that are in focus.""" total = images.shape[0] if total <= count: return list(range(total)) sharp = _sharpness(images) desc = _descriptors(images) span = sharp.max() - sharp.min() sharp_norm = (sharp - sharp.min()) / (span + 1e-9) chosen = [0] if forced_first else [int(torch.argmax(sharp).item())] while len(chosen) < count: dist = torch.cdist(desc, desc[chosen]).min(dim=1).values dist = dist / (dist.max() + 1e-9) score = sharpness_weight * sharp_norm + (1.0 - sharpness_weight) * dist score[torch.tensor(chosen, device=score.device)] = float("-inf") chosen.append(int(torch.argmax(score).item())) return sorted(chosen) # ------------------------------------------------------------- vision call #: Where an OpenAI-compatible vision endpoint usually lives. Probed in order #: when the node is left on its default, so the common setups work untouched: #: llama-server, LM Studio, Ollama, vLLM and friends all speak this API. _LLM_CANDIDATES = ( "http://127.0.0.1:8010", # llama-server, the default in these workflows "http://127.0.0.1:1234", # LM Studio "http://127.0.0.1:11434", # Ollama "http://127.0.0.1:8000", # vLLM / generic ) def _llm_base_url(override: str) -> str: """Resolve the vision endpoint, preferring an explicit address. Self-contained by design: this pack asks the network what is listening rather than importing from a sibling node pack, so it works on a plain ComfyUI install with any OpenAI-compatible server. """ if override.strip(): return override.strip().rstrip("/") import requests for base in _LLM_CANDIDATES: try: response = requests.get(f"{base}/v1/models", timeout=1.5) if response.status_code < 500: return base except Exception: continue # Nothing answered. Return the primary anyway so the caller's error names a # concrete address instead of an empty string. return _LLM_CANDIDATES[0] #: What to look for when the caller says nothing. Tuned for a location orbit, #: where every frame is the same place and the only question is which views #: are sharp and genuinely different. DEFAULT_BRIEF = ( "Prefer: sharp, well-exposed frames; clearly distinct camera angles that " "each reveal a different side or aspect.\n" "Reject: motion-blurred or smeared frames; near-duplicates of a frame you " "already chose; frames where the subject is cropped, occluded, or the " "camera is too close to read the space." ) def _vlm_pick(pils: list, count: int, hint: str, base_url: str, timeout: int, brief: str = "") -> tuple[list[int] | None, str]: """Ask the vision model to choose, comparing every candidate at once. One montage rather than one call per frame: scoring frames in isolation cannot tell a near-duplicate from a genuinely new angle, and a 31B vision pass per frame would cost more than the orbit did. Returns indices into [pils], or None when the model is unreachable or unusable. """ import requests labels = [str(i + 1) for i in range(len(pils))] board = _montage(pils, columns=4, cell_width=384, padding=10, labels=labels) subject = hint.strip() or "a location" instruction = ( f"This is a numbered contact sheet of {len(pils)} frames taken from a " f"single continuous camera move around {subject}. Choose exactly " f"{count} frames that together document the subject best.\n" f"{brief.strip() or DEFAULT_BRIEF}\n" 'Reply with JSON only, no prose: {"picks": [numbers], "why": "one short sentence"}' ) payload = { "messages": [ { "role": "user", "content": [ {"type": "text", "text": instruction}, {"type": "image_url", "image_url": {"url": _pil_to_data_uri(board)}}, ], } ], "max_tokens": 400, "temperature": 0.2, } try: response = requests.post( f"{base_url}/v1/chat/completions", json=payload, timeout=timeout ) if response.status_code != 200: return None, f"vision model returned HTTP {response.status_code}" content = response.json()["choices"][0]["message"]["content"] except Exception as exc: return None, f"vision model unreachable ({type(exc).__name__})" picks = _parse_picks(content, len(pils)) if not picks: return None, "vision model gave no usable frame numbers" why = "" try: why = str(json.loads(_json_slice(content)).get("why", ""))[:160] except Exception: pass # Short answers are salvageable; top up by spread rather than failing. return picks[:count], (why or "chosen by vision model") def _json_slice(text: str) -> str: cleaned = re.sub(r"```(?:json)?", "", text).strip() start, end = cleaned.find("{"), cleaned.rfind("}") return cleaned[start:end + 1] if start >= 0 and end > start else cleaned def _parse_picks(text: str, total: int) -> list[int]: """Pull frame numbers out of the reply, tolerating chatty models.""" numbers: list[int] = [] try: payload = json.loads(_json_slice(text)) raw = payload.get("picks", []) numbers = [int(n) for n in raw if isinstance(n, (int, float, str)) and str(n).strip().lstrip("-").isdigit()] except Exception: numbers = [int(n) for n in re.findall(r"\b\d{1,3}\b", text)] seen, out = set(), [] for number in numbers: index = number - 1 # labels are 1-based if 0 <= index < total and index not in seen: seen.add(index) out.append(index) return out def _clip_pick(clip, pils, count, hint, brief, max_length, temperature): """Judge the montage with an in-graph vision-language CLIP (Qwen3-VL etc.). Mirrors core's `TextGenerate` path — tokenize the instruction with the image attached, autoregressively generate, decode — so the very CLIP that encoded the anchor's prompt can pick the frames, with no external server and no second model resident. Only CLIPs whose tokenizer accepts images and whose model can generate will work; anything else is caught by the caller and falls back to the HTTP path. """ labels = [str(i + 1) for i in range(len(pils))] board = _montage(pils, columns=4, cell_width=384, padding=10, labels=labels) subject = hint.strip() or "a location" instruction = ( f"This is a numbered contact sheet of {len(pils)} frames taken from a " f"single continuous camera move around {subject}. Choose exactly " f"{count} frames that together document the subject best.\n" f"{brief.strip() or DEFAULT_BRIEF}\n" 'Reply with JSON only, no prose: {"picks": [numbers], "why": "one short sentence"}' ) image = _pils_to_tensor([board]) tokens = clip.tokenize(instruction, image=image, min_length=1) ids = clip.generate( tokens, do_sample=float(temperature) > 0.0, max_length=int(max_length), temperature=float(temperature), top_k=64, top_p=0.95, min_p=0.05, repetition_penalty=1.05, seed=0, ) return clip.decode(ids), "in-graph CLIP" # ------------------------------------------------------- attention backend #: Shown when a model should be left exactly as the loader produced it. _ATTENTION_DEFAULT = "default (unchanged)" #: Registry names are terse; these read better in a menu. Anything not listed #: is offered under its own name, so a backend added to ComfyUI later still #: appears here without this pack being updated. _ATTENTION_LABELS = { "comfy_kitchen_int8": "comfy kitchen (int8)", "pytorch": "pytorch (SDPA)", "sage": "sage", "sage3": "sage 3", "flash": "flash", "xformers": "xformers", "sub_quad": "sub-quadratic", "split": "split", } def _attention_registry() -> dict: """Attention backends this ComfyUI actually has, name -> callable. Read live rather than hardcoded: which backends exist depends on what is installed (sage, flash, xformers) and on the launch flags. Empty on a build with no registry, which is what keeps this node harmless there. """ try: from comfy.ldm.modules.attention import REGISTERED_ATTENTION_FUNCTIONS return dict(REGISTERED_ATTENTION_FUNCTIONS) except Exception: return {} def _attention_choices() -> list: registry = _attention_registry() ordered = [n for n in _ATTENTION_LABELS if n in registry] ordered += [n for n in sorted(registry) if n not in _ATTENTION_LABELS] return [_ATTENTION_DEFAULT] + [_ATTENTION_LABELS.get(n, n) for n in ordered] class LumosAttentionBackend: """Swap the attention kernel for one model, without touching the rest. Attention dominates the cost of a video model: H3 attends over every frame at once, so the kernel choice moves wall-clock far more than it would for a still image. ComfyUI can select one globally with `--use-ck-attention`, but that is a launch flag applying to every workflow on the server. Patching the model object instead keeps the choice inside the graph, where it can be changed per run and cannot surprise anything else. Core ships `ModelAttentionBackend`, which does the same patching but offers only pytorch and comfy kitchen. This lists whatever is registered — sage, flash and xformers included when they are installed — so there is no need for a separate pack just to reach the other kernels. Note that comfy kitchen's kernel is *int8*: it quantizes the attention computation. It is the fastest option here and the one worth trying first, but it is lossy in a way pytorch and sage are not, so compare a render before committing to it. Selecting the default leaves the model untouched and is always safe. """ @classmethod def INPUT_TYPES(cls): return { "required": { "model": ("MODEL",), "attention": (_attention_choices(),), }, } RETURN_TYPES = ("MODEL",) RETURN_NAMES = ("model",) FUNCTION = "patch" CATEGORY = "OrbitSheets" @classmethod def VALIDATE_INPUTS(cls, attention): # A graph saved on a box with sage must still load on one without it, # rather than failing validation before the user can change the widget. return True def patch(self, model, attention): if attention == _ATTENTION_DEFAULT: return (model,) registry = _attention_registry() wanted = next( (name for name, label in _ATTENTION_LABELS.items() if label == attention and name in registry), attention if attention in registry else None, ) if wanted is None: logging.warning( "[OrbitSheets] attention backend %r is not available here; " "leaving the model on its default kernel.", attention, ) return (model,) patched = model.clone() try: patched.set_model_optimized_attention(registry[wanted]) except Exception as exc: logging.warning( "[OrbitSheets] this ComfyUI cannot patch attention (%s); " "leaving the model unchanged.", exc, ) return (model,) logging.info("[OrbitSheets] attention backend: %s", wanted) return (patched,) # ------------------------------------------------------------------- nodes class LumosOrbitPrompt: """Write H3's prompt for a location sheet, by rotation or by hard cuts. A location is not a character, and the turnaround structure that makes the character sheet work is the wrong shape for it. A cut asks the model to re-establish the subject from a camera position it has never seen. For a figure on a plain backdrop that is easy — the model knows what a back looks like. For a specific building it is not: told to cut to "the rear", with no idea what this rear looks like, the model either re-frames the view it already has (the sheet fills with the same facade three times) or invents somewhere else entirely and wanders inside the building. Rotating the camera is a different matter, and the one move H3 does well here: every frame overlaps the last, so nothing has to be invented, only continued. The first version of this node rotated and produced genuinely different directions — its failure was that the frames tumbled and the horizon rolled, which is a stability problem, not a coverage one. That tumble had a cause, and it was arithmetic. A complete 360 asked of a 124-frame take is 70 degrees a second, against the 24 deg/s of H3's own worked example ("one full rotation across fifteen seconds"), and asking for it "with large amplitude at fast speed" pushed it further still. No camera turns that fast and stays level, so the model rendered the move it could render at that rate: a roll, tilting up into the vault. `_fit_turn` is the answer to that: no angle is ever written into a prompt that the take, at `take_seconds`, cannot hold. Asking for more clamps and says so. The wording it replaced made the same failure worse from the other end. "The camera stays level ... and never tilts up, tilts down, rolls or leans" names four of H3's documented motion types — Tilt Up, Tilt Down, Roll Clockwise, Roll Counterclockwise — in a model whose guide asks for what *does* happen, not what does not. The stability constraint is now positive: the horizon stays level, the verticals stay vertical. None of which was enough, and the honest verdict on a cathedral interior is that no continuous move beat cuts. A pan rolled at every rate it was tried at; a translational glide held level but only ever went right, and six frames of the same wall sliding past is not a location sheet. Locked tripod frames have no camera motion to get wrong, and that is what won. Hence `coverage`: * "cut views" (default) — the locked-off shots below. The tumbling is structurally impossible here: every shot is a static frame. * "continuous move" — one unbroken take, the camera panning on the spot (interior) or arcing round the subject (exterior). Kept because it is the only thing that works on a location the model cannot extrapolate, and because Frame Select's view clustering can pull distinct frames out of a take that a cut list would have had to invent. [Shot 1] the front, straight on (the anchor, = the first frame) [Shot 2] the right side, 90 degrees round [Shot 3] the rear, from directly behind [Shot 4] the left side, 90 degrees the other way [Shot 5] a wide view of the whole place [Shot 6] a close look at the main feature `space` decides what the continuous move is: inside, the camera pivots on its own axis to reach the wall behind it; outside, it travels round the subject and is told where it stays, not where it may not go. It needs telling that nothing moves, too: left to itself the model animates flags, water and passers-by, and a reference sheet wants none of that. """ @classmethod def INPUT_TYPES(cls): return { "required": { "location_description": ("STRING", {"multiline": True, "default": ""}), "visual_style": ("STRING", {"default": "Cinematic, live-action"}), # Decides move-around versus shoot-across. Getting this wrong # is what makes an interior sheet show one wall four times. "space": (["interior", "exterior"],), # Cuts first, because cuts are what won. Six locked-off # tripod frames cannot roll, and on a cathedral interior they # beat every continuous move tried against them. "coverage": (["cut views", "continuous move"],), # How far the camera comes round on the continuous move: the # exterior arc, or the interior pan. A turn the take cannot # hold is clamped, not obeyed. "rotation": (list(_ROTATION_CHOICES),), }, "optional": { # The take this prompt is written for, which has to match the # `length` on MiniMaxH3ImageToVideo: 124 frames is 5.17s at # 24fps, 260 frames is 10.8s. It sets the cut times, and it is # what the rotation is rationed against — a full turn wants # the longer take, and is cut down to fit if it does not get # one. "take_seconds": ("FLOAT", {"default": SHEET_SECONDS, "min": 1.0, "max": 30.0, "step": 0.1}), # The fifth shot. Worth having on a big or irregular location # where four orthogonal views miss how the parts sit together, # and worth dropping on a small one, where it is a duplicate. "wide_establishing_shot": ("BOOLEAN", {"default": True}), # The sixth: the closest thing the sheet has to a detail # reference, since every other view is framed at the same # distance and reads the place only as a shape. "detail_shot": ("BOOLEAN", {"default": True}), # Length of each shot but the last, which runs to the end of # the take — six shorter shots fit the same 124 frames. "shot_seconds": ("FLOAT", {"default": 0.75, "min": 0.25, "max": 2.0, "step": 0.05}), "time_of_day": ("STRING", {"default": ""}), "ambient_sound": ("STRING", {"default": ""}), }, } RETURN_TYPES = ("STRING", "STRING", "STRING") RETURN_NAMES = ("prompt", "soundscape", "music") FUNCTION = "build" CATEGORY = "OrbitSheets" def build(self, location_description, visual_style, space, coverage="cut views", rotation="auto (as far as the take allows)", take_seconds=SHEET_SECONDS, wide_establishing_shot=True, detail_shot=True, shot_seconds=0.75, time_of_day="", ambient_sound=""): description = location_description.strip().rstrip(".") style = visual_style.strip().rstrip(".") or "Cinematic, live-action" subject = description or "the location" when = time_of_day.strip().rstrip(".") when_clause = ( f" The time of day is {when} and it does not change." if when else "" ) # Repeated on every shot. The tumbling frames were the model reading a # moving camera as licence to tilt and roll, so each view is nailed # down as a tripod frame rather than a moment in a move. locked = ( "a locked-off static camera at eye level, the horizon level and " "centred, no camera movement of any kind" ) if space == "interior": # Inside there is nothing to walk around, so the camera crosses to # the far side and shoots back — that is what reveals the wall # behind the opening frame. views = [ ("the front wall of the space straight on, shot square from " "the middle of the room"), ("the right-hand wall of the space straight on, the camera " "turned 90 degrees from the opening view"), ("the rear wall of the space straight on — the wall directly " "behind the opening frame — the camera now facing the exact " "opposite direction to Shot 1, so that none of the wall seen " "in Shot 1 appears anywhere in this frame"), ("the left-hand wall of the space straight on, the camera " "turned 90 degrees the other way"), ] overview = ( "a wide corner view taking in the whole space at once, shot " "from one corner across to the far corner, the far walls and " "the full width of the floor all inside the frame, showing " "how the walls, floor and ceiling meet. This is the widest " "shot of the sequence" ) detail = ( "a tight close-up of the surface and construction of the " "space's main feature, the camera close enough that the " "material fills the frame and no wall, floor or sky is " "visible behind it. This is by far the closest shot of the " "sequence and looks nothing like the wide views before it" ) else: views = [ ("the front of the location straight on, the facade square to " "the camera and entirely inside the frame"), ("the right side of the location straight on, the camera moved " "90 degrees round, the full side elevation in frame"), ("the rear of the location straight on, shot from directly " "behind — the side hidden in the opening frame — the camera " "now facing the exact opposite direction to Shot 1, so that " "none of the facade seen in Shot 1 appears anywhere in this " "frame, the full rear elevation filling it instead"), ("the left side of the location straight on, the camera moved " "90 degrees round the other way, the full side elevation in " "frame"), ] overview = ( "a wide three-quarter establishing view of the whole location " "from further back, front and one side both visible, the " "entire place inside the frame with generous margin" ) detail = ( "a tight close-up of the main entrance and the wall surface " "around it, the camera close enough that the doorway and its " "masonry fill the frame with no sky and no surrounding " "ground visible. This is by far the closest shot of the " "sequence and looks nothing like the wide views before it" ) if coverage.startswith("continuous"): # Deliberately NOT wrapped in the I2VA envelope that # the cut path uses. That envelope was found to change the camera # behaviour on this node long before the cut rewrite, and carrying # it into the rotation path is what brought the rolling back: a # cathedral interior pan came out as a spin under the vault. The # plain style-subject-motion sentence is what produces a clean # rotation, and the move is stated in H3's own camera terms — # one move, its direction, its speed, and what stays stable. degrees, turn_phrase, clamped = _fit_turn(rotation, take_seconds) if clamped: logging.warning("[OrbitSheets] %s", clamped) # What the turn is *for*, said as the thing that comes into view. if degrees >= 360.0: reveal = ( "Every wall comes into view in turn, including the wall " "directly behind the opening frame, and the turn carries " "all the way round to where it started." ) elif degrees >= 180.0: reveal = ( "The wall directly behind the opening frame comes fully " "into view, and the take ends facing it." ) else: reveal = ( "The take ends facing the wall to the right of the " "opening frame, square on." ) # Positive throughout: H3's guide asks for what happens, and the # wording this replaced listed the model's own Tilt Up / Tilt Down # / Roll motion types under a "never" it does not reliably read. steady = ( "The horizon stays level and the vertical lines of the walls " "and pillars stay vertical from the first frame to the last. " "The camera turns on its own axis alone, at one constant " "height, the floor along the bottom of the frame and the " "ceiling along the top exactly as in the opening frame." ) if space == "interior": motion = ( "The camera holds its position in the middle of the space " "and pans right at slow speed, one single continuous " f"unbroken take turning steadily through {turn_phrase} " f"across the whole take and stopping there. {reveal}" ) else: motion = ( "The camera performs an arc shot around the location at " "slow speed, one single continuous unbroken take travelling " f"{turn_phrase} around it at a constant radius. " + ("The far side of the location — the side hidden in the " "opening frame — is fully revealed before the shot " "ends. " if degrees >= 180.0 else "The right-hand side of the location is brought fully " "into view before the shot ends. ") + "The camera stays outside at ground level for the whole " "take, at the same eye-level height and the same distance " "from the building in every frame." ) steady = ( "The horizon stays level and the vertical lines of the " "walls stay vertical from the first frame to the last." ) prompt = ( f"{style}. {subject}. {motion}" f"{when_clause} {steady} " "The location is completely empty: no people, no animals and " "no vehicles are present, and nothing within the environment " "moves. Lighting, weather and atmosphere stay exactly as " "established. Architecture, materials, colours, and the " "position of every object remain identical from every angle. " "A single continuous take with no cuts, no transitions, no " "on-screen text and no titles." ) return (prompt, ambient_sound.strip() or "N/A", "N/A") if wide_establishing_shot: views = views + [overview] if detail_shot: views = views + [detail] # Same timing rule as the turnaround: each shot but the last runs # `shot_seconds`, the last keeps the remainder, so the shot count can # change without the video needing more frames. Clamped against the # take the user actually rendered, not a fixed five seconds, so a # longer `length` spreads the cuts instead of stacking them all in # the opening half. span = max(float(take_seconds), 1.0) step = max(0.25, min(float(shot_seconds), span / len(views))) at = [_timecode(i * step) for i in range(len(views))] shots = [ f"[Shot 1] {style}, {views[0]}, {locked}.{when_clause}" ] for index, view in enumerate(views[1:], start=2): shots.append( f" [Shot {index}] At {at[index - 1]}, the shot cuts to " f"{view}, {locked}." ) body = ( "".join(shots) + " Every shot is a different framing of the place and no two " "shots repeat the same view: the camera position, direction and " "distance are visibly different in each one. " "The location is completely empty: no people, no animals and no " "vehicles are present, and nothing within the environment moves. " "Lighting, weather and atmosphere stay exactly as established and " "identical in every shot. Architecture, materials, colours, and the " "position of every object remain identical from every angle — it is " "the same place seen from a different side each time. No on-screen " "text and no titles." ) ambient = ambient_sound.strip() soundscape = ambient if ambient else "N/A" # I2VA per H3's prompt guide: first-frame instruction first, then the # three core fields — the same envelope the character sheet uses, which # is what anchors Shot 1 to the image the sheet starts from. prompt = ( "For the target video, at 0.00 seconds into the target video, " " (from [Shot 1]) is fully referenced.\n\n" f"integrated_multimodal_description: {subject}. {body}\n\n" f"overall_soundscape: {soundscape}\n\n" "non_diegetic_music: N/A" ) return (prompt, soundscape, "N/A") class LumosCharacterTurnaroundPrompt: """Write H3's prompt for a 6-shot character turnaround with hard cuts. The reference sheet is one I2VA sequence of six distinct views, each about a second long, joined by hard cuts per H3's prompt guide: [Shot 1] full body, facing the camera (the anchor, = the first frame) [Shot 2] tight close-up of the face [Shot 3] left side profile [Shot 4] right side profile [Shot 5] rear view of the body [Shot 6] the face again, frightened (optional, `scared_shot`) Shot 6 earns its slot by being the one thing the other five cannot show: they are all deliberately neutral, and a neutral face is no use as a reaction reference. Six shots need six seconds, so `length` wants 158 frames (~6.6s at 24fps) rather than the 124 a five-shot sheet uses. Cuts, not a continuous orbit: a cut forces the model to re-establish the figure at each angle, which is exactly what a turnaround sheet needs, and the identity stays locked because every shot reuses the same description. The first frame from the image model is the full body, so the identity is anchored before the face close-up arrives. Framing is its own lever because a 16:9 frame crops wide subjects: a dragon with spread wings, a tail or a held weapon outstretches the shot, and H3 will cut off whatever the prompt did not insist stays inside it. The default framing demands generous empty margin on every full-body shot. """ @classmethod def INPUT_TYPES(cls): return { "required": { "character_description": ("STRING", {"multiline": True, "default": ""}), "visual_style": ("STRING", {"default": "Cinematic, live-action"}), }, "optional": { "backdrop": ("STRING", {"default": "plain seamless neutral grey studio backdrop"}), # Framing + margin. Generous margin is the default because # reference sheets fail on wide subjects (wings, tails) when the # frame crops them. "framing": (_FRAMINGS,), # H3 renders audio alongside video from the same latent, so a # spoken line in the opening shot costs no extra sampling and # yields a voice-timbre sample for the story's