English | 한국어

# CuePrecise > **Find the exact moment in a long YouTube video—and see the evidence behind the answer.** A long video in a language you do not speak should still be searchable. CuePrecise is an open-source MCP server for Claude Desktop, Codex, Cursor, and other AI clients. It turns a YouTube video's original speech, captions, speakers, and relevant frames into a searchable local reference. CuePrecise analyzes videos longer than an hour in about three minutes. It does not send the whole video to your AI client for every question. It transcribes the original audio in chunks, indexes the evidence, and retrieves only the passages and frames related to your question. The transcription starts with the original audio. When original-language YouTube captions are available, they fill matching gaps for Latin-script names and terms missed by the transcription. You can ask about a video in your own language and still get the original words, speaker information, relevant frames, and a timestamp that takes you back to YouTube. Precise retrieval may require terms in the video's original language. Actual processing time depends on the video, network, and API response time. [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) [![Python](https://img.shields.io/badge/Python-3.11%2B-blue.svg)](https://www.python.org/) [![CI](https://github.com/Nattentia/cueprecise/actions/workflows/ci.yml/badge.svg)](https://github.com/Nattentia/cueprecise/actions/workflows/ci.yml) [![Listed on mcpservers.org](https://mcpservers.org/badge.svg)](https://mcpservers.org/servers/nattentia/cueprecise) [![Glama](https://glama.ai/mcp/servers/Nattentia/cueprecise/badges/score.svg)](https://glama.ai/mcp/servers/Nattentia/cueprecise) ## See it in action The attached demo uses a Polish-language interview. Ask what it is about in your own language; CuePrecise returns relevant moments, the original transcript, and matching frames as evidence. ~~~text You: I do not speak Polish. What is this interview about? Your AI + CuePrecise: Explains the interview in your language, points to relevant moments, and provides the original transcript and matching frames as evidence. ~~~ https://github.com/user-attachments/assets/ce7d595b-871f-469a-bcb8-798713751ffd Demo source: “Czym jest prompt injection i jak chronić firmę przed złośliwą instrukcją dla AI? Gośc. Tomasz Bartel” by Daniel Bartosiewicz | Content i Automatyzacja, licensed under CC BY. [**Download CuePrecise → GitHub Releases**](https://github.com/Nattentia/cueprecise/releases) - **Jump to the answer.** Get a timestamp for each item in a summary. - **Keep speakers separate.** Compare what each person said and why. - **Find a referenced screen again.** Connect a relevant frame to what was being explained. - **Check the source.** Open the original YouTube video at the cited moment. - **Watch with Korean subtitles.** Play the video with subtitles translated from the same evidence used in the analysis. Try questions like these: ~~~text What is the main argument of this video? Include timestamps for each point. Find where the speaker explains self-supervised learning. When does that phrase appear on screen? Compare each speaker's position on basic income and include the supporting passages. If these speakers debated a new issue, what arguments and counterarguments would follow from what they actually said in the video? ~~~ In Claude Desktop, clicking a timestamp opens YouTube at that moment. Other AI clients may render timestamp links differently. ## It keeps answers tied to evidence CuePrecise is not just a summarizer. It gives your AI client the material it needs to answer a question and lets you check where the answer came from: - Gemini's word-level transcription of the original speech. - Original-language YouTube captions, when available, to recover missed Latin-script terms. - Frames from screen references, recovered terms, or explicitly requested timestamps. - Speaker labels, confidence, and provenance for retrieved words. When evidence is absent or weak, CuePrecise reports insufficient evidence. The analysis bundle stays on your computer, so a later conversation can search the same video without starting over. Audio chunks used for transcription are sent to Gemini; see [PRIVACY.md](PRIVACY.md). ### Recovering a term the transcription missed Names and technical terms are easy to lose in a long transcription. In one 23-minute Korean technical lecture, the phrase `self supervised learning` disappeared from all four Gemini-only transcription runs. ~~~text Gemini transcription: So how did they learn this ability? It is a way of learning. CuePrecise merge: So how did they learn this ability? self supervised learning is a way of learning. ~~~ CuePrecise checks the YouTube captions from the same time range and fills only a matching gap. It does not rewrite Gemini's words. Words supplied by captions keep their origin. The measured example looked like this: - The YouTube original-language captions contained 91 Latin-script words and the missing phrase. - The Gemini runs contained 28–29 Latin-script words, but missed the phrase every time. - The merged result contained 38 Latin-script words and recovered the phrase while keeping the Korean transcription quality. This is one measured edge case, not a general accuracy claim. The exact merge rules and validation criteria are documented in [CONTRACT.md](CONTRACT.md). ### Compare speakers without pretending to know their names CuePrecise carries speaker information across chunks of a long video. Speaker labels are identifiers, not real names. Confirmed and inferred identities are kept separate, and weak evidence remains unresolved instead of being presented as a fact. That lets your AI client: - collect one speaker's claims and supporting passages; - compare the positions of several speakers; - simulate a debate about a new issue using the speakers' actual statements as evidence. A simulated debate is generated from the video. It is not a claim that those people actually discussed the new issue. ### Watch with Korean subtitles built from the same evidence CuePrecise can overlay Korean subtitles on a foreign-language lecture using the evidence already collected during analysis. First, it fixes the terminology. Gemini's transcript is checked against YouTube captions and on-screen text to resolve spellings such as `by torch → PyTorch` and `combine JS → ConvNetJS`. The terminology is locked before translation so the same term stays consistent across batches. Your AI client translates the lecture in small batches. CuePrecise checks the returned translations for numbers, terminology, reading speed, missing lines, and style, then returns only the flagged lines for review. Translation makes no additional Gemini calls; usage or charges from your AI client still depend on that client's plan. In one measured run on a 68-minute lecture with 934 sentences, terminology review took about two minutes and 19 translation batches took about 14 minutes, with six batches needing review. In that run, subtitles were produced about five times faster than playback. Actual speed depends on the video and AI client. The local browser viewer provides: - Subtitles over the YouTube player, including in fullscreen. - Korean, Korean with the original text, or original-language subtitles. - Saved preferences for font size, subtitle position, shadows, and box style. - Optional short term explanations shown when each term appears. - A searchable transcript, terminology list, and quality report in the side panel. Click a sentence to play from its timestamp. - Shortcuts: `Space` to play/pause, `←` and `→` to seek five seconds, `C` for subtitle language, `T` for term explanations, and `F` for fullscreen. The quality report excludes purely cosmetic differences such as `MinGPT → minGPT` from its correction count. Korean is the only translation target currently supported; other languages are on the roadmap. ## Quick start > **Windows Smart App Control:** v0.2.5 was blocked on PCs where Smart App Control is on > (`[WinError 4551] An Application Control policy has blocked this file`). Starting with > v0.2.6, the Claude Desktop extension ships only widely used runtime files (the official > Python embeddable package and an unmodified FFmpeg release build) and runs with Smart App > Control on. `v0.2.10` ships the other AI clients installer the same way, as a plain zip > instead of an unsigned installer executable. There is no remaining known Smart App Control > issue. ### Claude Desktop on Windows — one-file extension 1. Open [Releases](https://github.com/Nattentia/cueprecise/releases) and download `cueprecise-windows.mcpb`. 2. In Claude Desktop, open **Settings → Extensions → Advanced settings → Install Extension**. 3. Select the file. Claude will ask for a Gemini API key and a folder for local video data. 4. Enable CuePrecise and ask Claude about a YouTube link. The approximately 97 MiB bundle includes CuePrecise, an embedded Python runtime, `yt-dlp`, FFmpeg, and FFprobe. You do not need to install Python, Git, or the video tools separately. The extension is currently available for Claude Desktop on Windows. ### Other AI clients on Windows 1. Open [Releases](https://github.com/Nattentia/cueprecise/releases) and download `cueprecise-windows.zip`. 2. Extract the zip and double-click `CuePrecise 설치.cmd`. Windows may ask you to confirm running it. 3. Paste a Gemini API key from [Google AI Studio](https://aistudio.google.com/api-keys) when asked, then pick the AI clients found on your computer. 4. Fully quit and reopen the connected clients. An AI client can also run the script non-interactively instead of walking through the prompts, for example `CuePrecise 설치.cmd --api-key-stdin --targets claude-desktop,codex`. It adds only the CuePrecise entry and backs up existing configuration when it can do so safely; a secret-bearing Codex TOML may be left without a backup to avoid copying the key. On Windows, the API key is encrypted with Windows DPAPI for the current user. Older plaintext CuePrecise keys are moved into the protected store during an upgrade. > **Unsigned preview:** `v0.2.10` is not digitally signed, so Windows may show an > unknown-publisher warning. Download it only from this repository's Releases page and > verify `SHA256SUMS.txt` if you want to check the file before installing it. ### macOS, Linux, and command-line installation With [uv](https://docs.astral.sh/uv/getting-started/installation/): ~~~bash uv tool install git+https://github.com/Nattentia/cueprecise cueprecise setup ~~~ The setup command configures detected supported AI clients, including Claude Desktop, and creates the default data directory `~/.cueprecise/data`. It keeps a timestamped `.bak` file when an existing configuration can be backed up safely. If a secret-bearing configuration cannot be parsed safely, it avoids making an unsafe backup. Install `ffmpeg` and `ffprobe`, then check the environment: ~~~bash cueprecise doctor ~~~ Create a [Gemini API key](https://aistudio.google.com/api-keys), then pass it through standard input or a file so it does not appear in the command or shell history: ~~~bash cueprecise setup --api-key - # paste the key, then press Enter cueprecise setup --api-key-file ~/.gemini-key # read it from a file pass show gemini/api-key | cueprecise setup --api-key - cueprecise run "https://www.youtube.com/watch?v=VIDEO_ID" --language en-US cueprecise status VIDEO_ID ~~~ If a key is exposed, delete it in [Google AI Studio](https://aistudio.google.com/api-keys) and create a new one. See [PRIVACY.md](PRIVACY.md) for the full procedure. For source development only: ~~~bash git clone https://github.com/Nattentia/cueprecise.git cd cueprecise python -m pip install -r requirements.txt python src/pipeline.py --help ~~~ ## Supported AI clients `cueprecise setup` can detect and configure: - Claude Desktop - Codex - Claude Code - VS Code - Cursor - Windsurf - Gemini CLI Configuration paths for Cursor, Windsurf, and Gemini CLI are included, but their complete setup flows have not yet been verified on the development PC. Run it for every detected client, for one named client, or to inspect the result: ~~~bash cueprecise setup cueprecise setup --client codex cueprecise doctor ~~~ An application counts as installed when its executable is on `PATH`. A leftover configuration folder is not treated as proof that the application is present. An undetected client can still be named explicitly with `--client `. CuePrecise skips an existing `cueprecise` entry if CuePrecise did not create it. It does not overwrite another MCP server's settings, and a failure for one client does not stop the others. ChatGPT connectors and Claude.ai on the web are not supported. They accept remote MCP servers over HTTPS with OAuth, while CuePrecise runs as a local stdio server on your computer. ## Connect another MCP host CuePrecise accepts both the request-per-request MCP revision `2026-07-28` and the earlier `initialize` handshake. `cueprecise setup` is preferred because it preserves existing configuration and handles credentials for supported clients. The JSON below is for a source checkout or an MCP host not listed above. It stores the key in plaintext, so use the setup command when possible: ~~~json { "mcpServers": { "cueprecise": { "command": "python", "args": [ "C:/path/to/cueprecise/src/mcp_server.py", "--bundle-root", "C:/path/to/cueprecise/data" ], "env": { "GEMINI_API_KEY": "..." } } } } ~~~ Use absolute paths. On every platform, literal `--api-key VALUE` is rejected to keep the key out of process listings and shell history. The server can start without `GEMINI_API_KEY`: existing analyses remain searchable, while new transcription requests stop with a configuration message. ## MCP tools The tools fall into five groups. Analysis and status: - `cueprecise_register` — register and analyze a YouTube video. Choose the stages to run; new transcription chunks call Gemini, while saved responses are reused when settings match. - `cueprecise_status` — report progress, generated artifacts, and estimated local usage. Search and evidence: - `cueprecise_outline` — return a timestamped outline, recovered terms, and speaker state. - `cueprecise_query` — search transcript evidence and related frames. - `cueprecise_excerpt` — return transcript and frames for a specific time range. - `cueprecise_frames` — extract frames around screen-reference moments or requested timestamps, downloading the video again if needed. Saved results: - `cueprecise_summary` — create or retrieve a summary. - `cueprecise_set_summary` — validate and save a host-improved summary. - `cueprecise_set_chapter_titles` — validate and save host-written chapter titles. Subtitles (Korean only for now): - `cueprecise_subtitle` — return the next subtitle work packet (term review, translation, or checks) and the local viewer link. - `cueprecise_set_subtitle` — validate and save the AI client's response, then return the next packet. Cleanup: - `cueprecise_purge` — explicitly remove chunks, source video, derived results, raw material, or all data. After fetching material from YouTube and transcribing audio with Gemini, assembly, caption merging, chapter generation, rendering, frame extraction, and indexing run locally. Chapter titles and summaries are written by the host AI from retrieved evidence; they do not create another Gemini call. ## Local evidence bundle The installer and `cueprecise setup` use `~/.cueprecise/data`. A source checkout can use `data` unless you set `--bundle-root`. ~~~text data// job.json chunk plan and progress raw/ captions.json original-language YouTube captions metadata.json metadata used for language checks audio/ audio chunks used for transcription transcripts/ per-chunk transcripts and raw responses frames/ extracted frames derived/ transcript.json assembled Gemini transcription merged.json transcription plus caption evidence chapters.json timestamped outline frames.json frame index output.srt, output.txt optional render output index.sqlite3 transcript, chapter, frame index, and summary ~~~ Each indexed evidence span keeps its timestamp, speaker status, evidence confidence, and origin. The merged word data also keeps per-word timestamps and origin: ~~~json { "text": "supervised", "start": 208.93, "end": 209.87, "speaker": "speaker:0", "speaker_status": "confirmed", "origin": "youtube" } ~~~ A query result includes the time range, text, source, confidence, and any related frame: ~~~json { "start": 1728.4, "end": 1740.2, "timecode": "00:28:48", "text": "The experiment was stopped after eight participants had seizures.", "source_path": "derived/merged.json", "source_kind": "transcript", "speaker": "speaker:3", "speaker_status": "inferred", "speaker_confidence": 0.75 } ~~~ Frames are not sampled uniformly across the whole video. CuePrecise prioritizes screen references in the transcript, recovered terms, and timestamps requested by the user. It does not semantically classify every code sample, table, or diagram. Text recognized on those frames is stored as separate provenance instead of replacing the transcript. ## How it works CuePrecise does not try to make your AI client watch the entire video in one pass. It builds a knowledge bundle that can be searched again: 1. Fetch audio, a low-resolution video stream, original-language captions when available, and metadata from YouTube. `--skip-video` defers the video download until frames are requested. 2. Split the audio into chunks and request word-level transcription and speaker information from Gemini. 3. Assemble completed chunks and use original-language captions only to fill matching gaps for missed Latin-script terms. 4. Extract frames at screen-reference moments, recovered terms, and timestamps requested by the user. 5. Index transcript, chapters, speakers, and frames in SQLite. 6. Let the AI client retrieve the relevant evidence and write the answer with timestamps. Stages communicate through JSON files and can be rerun independently. Completed chunks are reused when the input and settings match. Raw transcription responses are saved before validation, so a parse failure does not automatically spend another Gemini call on the same response. The optional `render` stage creates SRT and TXT files. Overflowing text is carried into the next cue instead of being silently dropped, preserving 100% of the words in the rendered output. ### Language selection and translation guard Pass the video's original language when possible, for example `--language en-US`, `--language ko-KR`, or another BCP-47 code. Without it, Gemini can occasionally return a translation instead of a verbatim transcript. CuePrecise checks each chunk against original-language captions, the requested language, and video metadata. If it detects a translation, it stops before spending calls on the remaining chunks. When there is no basis for the check, it records that the check was skipped. The check uses material already fetched and does not make an extra API call. ## Command-line reference ~~~bash python src/pipeline.py run [options] python src/pipeline.py status python src/pipeline.py purge --scope ~~~ Common `run` options: - `--language` — comma-separated BCP-47 language codes; specifying the original language is recommended - `--stages` — stages to run; `all` includes optional stages - `--bundle-root` — directory for video bundles - `--force` — ignore cached results and rebuild - `--skip-video` — skip the video download - `--keep-video` — keep the video after frame extraction - `--at` — timestamps, in seconds, at which to extract frames - `--max-frames` — maximum number of frames; default 40 - `--chunk-max-secs` — maximum chunk length; default 1790 seconds - `--overlap-secs` — overlap between chunks; default 10 seconds - `--daily-limit`, `--rpm-limit`, `--request-interval` — local Gemini usage limits and the minimum gap between calls - `--width` — subtitle line width; default 20, with 42 recommended for English Rebuild selected derived output or clean up source material: ~~~bash python src/pipeline.py run --stages render python src/pipeline.py run --stages visual python src/pipeline.py purge --scope chunks ~~~ `--scope` accepts `chunks`, `video`, `derived`, `raw`, or `all`. Deletion is explicit. ## Performance and usage The pipeline is designed to resume. Completed chunks are reused when the input and settings are unchanged, and local post-processing such as merge, visual extraction, and indexing does not call Gemini. The source video is removed after frame extraction; use `--keep-video` to retain it. The split transcription audio is removed once every chunk is transcribed. The original audio is kept. If two apps ask for the same video at once, the second reports that it is busy. A local usage ledger records attempts by an API-key hash and Pacific date, never the original key. CuePrecise shows the expected calls before a job starts and stops if a configured limit would be exceeded. Google AI Studio remains authoritative for server-side usage. ## Requirements - Python 3.11+ - `ffmpeg` and `ffprobe` for audio chunking and frame extraction - Frame OCR is built into Windows. On macOS and Linux, install `pytesseract`, `Pillow`, and the `tesseract` binary (optional) ~~~bash python -m pip install -r requirements.txt python -m pip install -r requirements-optional.txt # optional OCR and timezone support ~~~ `google-genai` is needed only for the `transcribe` stage. The installable distribution is `cueprecise-mcp`, with `cueprecise` and `cueprecise-mcp` as its command-line entry points. It has not been published to PyPI yet, so install from the GitHub URL above. ## Tests ~~~bash python -m unittest discover -s tests ~~~ The suite uses the standard-library `unittest` runner. Tests do not access the network or call the Gemini API. Tests that require `google-genai` are skipped when the SDK is not installed. ## Known limitations - YouTube caption spelling errors can remain in recovered terms, and captions may be unavailable. - The caption-merge threshold was tuned on a limited set of real videos and needs broader validation. - Across three or more chunks, a speaker absent from the overlap can remain `unresolved`. CuePrecise avoids assigning a potentially wrong identity. - The interruption/resume path for a long-running job on the real API still needs end-to-end validation. - Visual-reference phrase matching currently focuses on Korean and English. - Visual search is candidate-based: it prioritizes transcript screen references and requested timestamps rather than inspecting every frame semantically. - Cross-language retrieval is lexical rather than translation- or embedding-based; original terms may be needed when the host AI does not translate the query. - Timestamps assigned to caption-recovered words are placed inside the missing transcription gap and should be treated as approximate within that interval. - The subtitle tools translate into Korean only, and the viewer that plays those subtitles is in Korean. Other target languages are planned. - That viewer runs on this computer (`127.0.0.1`) while the MCP server is running, and is designed for desktop browsers. ## Documentation - [`README.ko.md`](README.ko.md) — Korean README - [`CODE_SIGNING_POLICY.md`](CODE_SIGNING_POLICY.md) — Windows release review and signing policy - [`PRIVACY.md`](PRIVACY.md) — API keys, local data, external services, and uninstall behavior - [`CONTRACT.md`](CONTRACT.md) — authoritative data contracts and validation rules - [`DECISIONS/`](DECISIONS/) — design decisions and rejected alternatives - [`CONTRIBUTING.md`](CONTRIBUTING.md) — development environment and pull request process - [`SECURITY.md`](SECURITY.md) — private vulnerability reporting ## Roadmap - [x] Korean subtitles laid over the YouTube video, built from the analyzed evidence - [ ] A browser extension for videos that block external playback - [ ] The same subtitles in languages other than Korean - [ ] Host-specific timestamp links - [ ] Pipelined chunk transcription - [ ] Multi-video research with source-specific evidence ## Acknowledgements The initial transcription flow was informed by the MIT-licensed [`gemini-transcribe-wrapper`](https://pypi.org/project/gemini-transcribe-wrapper/0.0.13/). CuePrecise is an independently written project. ## License CuePrecise is released under the [MIT License](LICENSE). CuePrecise is not affiliated with or endorsed by YouTube or Google. YouTube is a supported service.