# live-audio-intelligence-mcp **MCP server for live financial webcast transcription and heuristic vocal stress analysis.** Turns any live webcast URL (earnings calls, CNBC, investor days) into a real-time pipeline that feeds an LLM two things simultaneously: 1. A **rolling transcript** via `faster-whisper` (CPU, int8). 2. A **heuristic vocal stress score (0–100)** derived from F0 pitch jitter, hesitation ratio, and voiced-frame fraction. These prosodic features are well-established correlates of speaker arousal in the vocal-analysis literature; their composition into the score below is heuristic and has **not** been empirically validated against market outcomes. Treat it as a coarse signal, not an oracle. Built on the [Model Context Protocol](https://modelcontextprotocol.io). Exposes 4 tools over stdio; drop it into Claude Desktop, Claude Code, or any MCP client. --- ## Why this exists Sell-side analysts and hedge-fund PMs don't just want to read the earnings transcript after the fact — they want a real-time signal about **how confident the CFO sounds when asked about Q4 guidance**. This server wires a Whisper pipeline and a pYIN-based prosody analyzer directly into an LLM's tool loop, so the model can ask *"what did the CEO just say about China?"* and *"how stressed did they sound saying it?"* in the same conversation. --- ## Install ### 1. System prerequisite — FFmpeg FFmpeg is a **system binary**, not a Python package. The `ffmpeg-python` wrapper is *not* a dependency here — we drive the binary directly via `subprocess`. You must install it yourself. **macOS** (Homebrew): ```bash brew install ffmpeg ``` **Linux** (Debian / Ubuntu): ```bash sudo apt-get update && sudo apt-get install -y ffmpeg ``` **Linux** (Fedora / RHEL): ```bash sudo dnf install -y ffmpeg ``` **Windows** — choose one: ```powershell # Option A — winget (Windows 10/11) winget install --id=Gyan.FFmpeg -e # Option B — Chocolatey choco install ffmpeg # Option C — Scoop scoop install ffmpeg ``` Confirm it's on your PATH: ```bash ffmpeg -version ``` If the command errors with "not found", reopen the terminal (PATH changes don't propagate to already-open shells) or add the ffmpeg `bin/` directory to your PATH manually. ### 2. Python package Requires **Python ≥ 3.10**. ```bash pip install live-audio-intelligence-mcp ``` Or run directly without installing with `uv`: ```bash uvx live-audio-intelligence-mcp ``` The first run will download the `faster-whisper base.en` model (~140 MB) from Hugging Face and cache it under `~/.cache/huggingface/`. --- ## Run it Stdio MCP server: ```bash live-audio-intelligence-mcp ``` Or equivalently: ```bash python -m live_audio_intelligence_mcp ``` ### Claude Desktop Add to `claude_desktop_config.json`: ```json { "mcpServers": { "live-audio-intelligence": { "command": "live-audio-intelligence-mcp" } } } ``` ### Claude Code ```bash claude mcp add live-audio-intelligence -- live-audio-intelligence-mcp ``` --- ## Tools | Tool | Purpose | |---|---| | `monitor_live_stream(url, disable_vad=False)` | Resolve the audio URL, spawn ffmpeg, start chunking + transcription. Returns a `stream_id`. | | `get_rolling_transcript(stream_id, minutes_back=10)` | Get the last N minutes of concatenated transcript text. | | `analyze_speaker_stress(stream_id, time_window_seconds=60)` | Run prosody analysis over the last N seconds of audio. Returns stress score, pitch jitter, hesitation ratio, pause stats, and a human-readable interpretation. | | `stop_monitor(stream_id)` | Kill ffmpeg, clean up temp files, drop the transcript buffer. | ### The stress score | Score | Interpretation | |---|---| | 0–20 | Confident, fluent delivery | | 20–45 | Normal variation | | 45–75 | Elevated stress — worth monitoring | | 75–100 | High stress — potential market-moving signal | Composite of: - **Pitch jitter** (coefficient of variation of F0) — 50% weight, saturating at jitter = 0.12 - **Hesitation ratio** (fraction of audio in pauses > 400 ms) — 35% weight, saturating at 0.30 - **Unvoiced fraction** (speaker trailing off) — 15% weight The three features are literature-backed correlates of speaker arousal (see pYIN for F0 tracking, and the broad "disfluency is a correlate of cognitive load" line of work). The *weights* and *saturation points* are hand-picked defaults, chosen so that a calm speaker scores in the 0–20 band on clean studio audio and visibly stressed speech scores ≥ 45 — they are not fit to any labeled dataset. Consumers who care about absolute numbers should recalibrate thresholds against their own recordings. A synthetic-audio calibration harness lives at [scripts/validate_stress_score.py](scripts/validate_stress_score.py). It generates controlled audio (smooth sine, jittered pitch, silence-padded speech) and asserts that the score responds in the expected direction. This is *calibration evidence*, not market-outcome validation. ### Low-SNR mode For speakerphone audio (most earnings Q&A), pass `disable_vad=true` to `monitor_live_stream`. Silero VAD tends to aggressively classify muddy conference-call speech as silence; disabling it preserves more of the speech at the cost of transcribing a bit more ambient noise. ### Concurrency limits By default the server caps concurrent streams at 4 (each stream holds an ffmpeg subprocess, a yt-dlp subprocess, a thread, and a temp directory). Override via env var for high-throughput deployments: ```bash LAI_MAX_CONCURRENT_STREAMS=16 live-audio-intelligence-mcp ``` Exceeding the cap raises `StreamLimitExceededError` rather than silently queuing. --- ## Architecture ``` ┌──────────────────┐ URL ─────▶ │ yt-dlp resolve │ └────────┬─────────┘ │ audio URL ▼ ┌──────────────────┐ ┌────────────────┐ │ ffmpeg (bg) │ ───▶ │ 15s WAV chunk │ │ 16kHz mono PCM │ │ queue │ └──────────────────┘ └───────┬────────┘ │ ┌──────────────────┴────────────────┐ ▼ ▼ ┌──────────────────┐ ┌──────────────────┐ │ faster-whisper │ │ librosa.pyin │ │ (int8 / CPU) │ │ + pause detect │ └────────┬─────────┘ └────────┬─────────┘ │ rolling transcript │ stress score ▼ ▼ ┌────────────── MCP stdio ───────────────┐ │ LLM (Claude) — calls tools freely │ └────────────────────────────────────────┘ ``` All blocking work (Whisper inference, ffmpeg I/O, librosa DSP) is dispatched to threads via `asyncio.to_thread` so the MCP event loop stays responsive. --- ## Development ```bash git clone https://github.com/ykshah1309/live-audio-intelligence-mcp cd live-audio-intelligence-mcp python -m venv .venv source .venv/bin/activate # Windows: .venv\Scripts\activate pip install -e ".[dev]" pytest live-audio-intelligence-mcp ``` ### Running the tests The pytest suite in [tests/](tests/) covers the pure-Python logic that doesn't require network or ffmpeg: - URL syntactic validation (scheme allow-list, host presence) - Concurrency-cap enforcement in `StreamManager` - Custom exception hierarchy (backward-compat with `ValueError` / `RuntimeError`) - Prosody analyzer on synthetic audio (sine tone, silence, jittered pitch) ```bash pytest -q ``` ### Calibration benchmark ```bash python scripts/validate_stress_score.py ``` This generates synthetic audio with known acoustic properties and verifies the stress score responds in the expected direction. It's a sanity check for the weighting heuristics — not a replacement for empirical validation against real earnings-call outcomes. --- ## Troubleshooting **`ffmpeg: command not found`** — ffmpeg isn't on PATH. See the install section above. On Windows, reopen your terminal after installing. **`yt-dlp could not resolve URL`** — The site isn't supported by yt-dlp or the URL is malformed. Test with `yt-dlp -F ` from the command line; if that fails, the server will too. **Whisper downloads hang on first run** — The ~140 MB model download goes to `~/.cache/huggingface/`. Check your network and Hugging Face access. **"Insufficient voiced frames"** in stress output — The audio window is mostly silence or noise. Usually means the stream is still buffering; wait 30s and retry. For speakerphone Q&A, start the monitor with `disable_vad=true`. --- ## Contributing See [CONTRIBUTING.md](CONTRIBUTING.md). ## Changelog See [CHANGELOG.md](CHANGELOG.md). ## License MIT — see [LICENSE](LICENSE).