tldw Server

Too Long; Didn't Watch - API-first media analysis and research platform

Made with love License: multi-license E2E Critical Smoke

Process media with 16+ LLM providers and OpenAI-compatible APIs for Chat, Embeddings, and Evals.

Browser extension support is included in-repo; hosted deployment tooling is still evolving.

Your local platform for media analysis, knowledge work, and LLM-backed creation.

Server: GPL-3.0-only
Frontend: source-available under PolyForm Perimeter 1.0.1
OpenAPI contract: Apache-2.0
Detailed licensing scope and terms

--- ## Table of Contents
Expand table of contents - [Overview](#overview) - [Start Here](#start-here) - [Current Status](#current-status) - [What's New (in the last few releases)](#whats-new-in-the-last-few-releases) - [Privacy & Security](#privacy--security) - [Highlights](#highlights) - [Feature Status](#feature-status) - [Quickstart](#quickstart) - [Preflight Check (Recommended)](#preflight-check-recommended) - [At-a-Glance Commands](#at-a-glance-commands) - [No-Docker Path (Makefile, Development)](#no-docker-path-makefile-development) - [No-Make Path (Windows-Friendly)](#no-make-path-windows-friendly) - [Manual Setup](#manual-setup) - [Local Profile: Add the WebUI](#local-profile-add-the-webui) - [Run the Web UI (WIP)](#run-the-web-ui-wip) - [Docker Compose](#docker-compose) - [Supporting Services via Docker](#supporting-services-via-docker) - [Usage Examples](#usage-examples) - [Key Endpoints](#key-endpoints) - [Architecture & Repo Layout](#architecture--repo-layout) - [Architecture Diagram](#architecture-diagram) - [Networking & Limits](#networking--limits) - [Running Tests](#running-tests) - [CI Status & Smoke Tests](#ci-status--smoke-tests) - [Documentation & Resources](#documentation--resources) - [Resource Governor Config](#resource-governor-config) - [OpenAI-Compatible Strict Mode (Local Providers)](#openai-compatible-strict-mode-local-providers) - [Chatbook Tools Guide](#chatbook-tools-guide) - [Deployment](#deployment) - [Monitoring](#monitoring) - [PostgreSQL Content Mode](#postgresql-content-mode) - [Troubleshooting](#troubleshooting) - [Contributing & Support](#contributing--support) - [Developer Guides](#developer-guides) - [Ingestion & Media Processing Docs](#ingestion--media-processing-docs) - [About](#about) - [Getting Help](#getting-help) - [Security Disclosures](#security-disclosures) - [Project Guidelines](#project-guidelines) - [Appendix (Optional)](#appendix-optional) - [License](#license) - [Credits](#credits)
## Overview **tldw_server** is an API-first platform for ingesting media, transcribing, analyzing, and retrieving knowledge from video, audio, documents, websites, and more. It runs a FastAPI server with OpenAI-compatible Chat, Audio, Embeddings, and Evals APIs, a unified RAG pipeline, and integrations with local or hosted LLM providers. The primary clients are the Next.js WebUI and Admin UI, with browser-extension support in the repo as well. Long-term vision: a personal research assistant inspired by "The Young Lady's Illustrated Primer" that helps people learn, reason about, and retain what they watch or read. Good fit for: - Turning videos, podcasts, and documents into searchable, citable knowledge. - Running local or hosted LLMs behind a consistent OpenAI-compatible API. - Building research workflows with RAG, evaluation, and prompt tooling. ## Start Here 1. **Clone this repository and pick one peer setup profile.** | Profile | Best for | Prepare | Start | Verify | |---------|----------|---------|-------|--------| | [Docker single-user + WebUI](Docs/Getting_Started/Profile_Docker_Single_User.md) | Most users (recommended) | `make setup-docker-single` | `make start-docker-single` | `make verify-docker-single` | | [Docker multi-user + Postgres](Docs/Getting_Started/Profile_Docker_Multi_User_Postgres.md) | Teams, public deployments | Export generated `ADMIN_USERNAME` / `ADMIN_PASSWORD`, then `make setup-docker-multi` | `make start-docker-multi` | `make verify-docker-multi` | | [Local single-user](Docs/Getting_Started/Profile_Local_Single_User.md) | Development, debugging | `make install-local` then `make setup-local-single` | `make start-local-single` | `make verify-local-single` | `make quickstart` remains the shortest Docker single-user + WebUI alias. It runs setup, start, and verification for the first profile. For user-facing workflows across setup, the WebUI, browser extension, local models, APIs, and administration, start with the [User Wiki](Docs/Wiki/User_Wiki.md). Contributors should start with the [Developer Wiki](Docs/Wiki/Developer_Wiki.md). After cloning, you can run the optional Makefile helper checks with `make quickstart-prereqs`, or verify Python 3.11+, ffmpeg, and Docker manually for your chosen profile. On a fresh checkout, the setup targets are still the source of truth because they create the lightweight setup environment they need. 2. **Follow your profile guide** end-to-end. It covers prepare, start, verify, first value, audio path, troubleshoot, and optional add-ons. Developers working on the WebUI, extension, or shared app packages should also start with [apps/DEVELOPMENT.md](apps/DEVELOPMENT.md). Optional add-ons (apply AFTER your base profile is healthy): - [First-time audio setup: CPU systems](Docs/Getting_Started/First_Time_Audio_Setup_CPU.md) - [First-time audio setup: GPU/accelerated systems](Docs/Getting_Started/First_Time_Audio_Setup_GPU_Accelerated.md) ## Current Status Current release line: - `0.1.46` Beta status. Expect rough edges and please report issues. - Primary client surfaces are the Next.js WebUI, Admin UI, and browser extension. - Package metadata is prepared under the canonical PyPI name `tldw-server`; use a repository checkout until publishing is complete. - The `dev` branch remains the home for ongoing work beyond `0.1.46`. - This release candidate includes merged development changes since `v0.1.45` through frozen PR #3073 and requires Python 3.12 or newer. See [CHANGELOG.md](CHANGELOG.md) for the PR rollup and [Docs/Published/RELEASE_NOTES.md](Docs/Published/RELEASE_NOTES.md) for the published release entry point.
Current focus and migration notes from the old Gradio version ### Active Work-in-Progress (not in order) - Multi-User setup - Workflows - [Web App frontend & Browser Extension UIs](./apps/) - Unified Admin Dashboard ([admin-ui](./admin-ui)) - Collections (read-it-later) - Better TTS support - Better Documentation ### Migrating From Gradio Version (pre-0.1.0) - Backup: - `cp -a ./Databases ./Databases.backup` - Update configuration: - Copy provider keys to `.env`. - For AuthNZ setup: `cp tldw_Server_API/Config_Files/.env.example tldw_Server_API/Config_Files/.env && python -m tldw_Server_API.app.core.AuthNZ.initialize` - Database migration: - Inspect: `python -m tldw_Server_API.app.core.DB_Management.migrate_db status` - Migrate: `python -m tldw_Server_API.app.core.DB_Management.migrate_db migrate` - Optional: `--db-path /path/to/media.db` if not using defaults - If migrating content to Postgres later, use the tools under `tldw_Server_API/app/core/DB_Management/` (e.g., migration_tools.py) - API changes: - Use FastAPI routes; see http://127.0.0.1:8000/docs. OpenAI-compatible endpoints are available (e.g., `/api/v1/chat/completions`). - Frontend: - Next.js WebUI: `apps/tldw-frontend/` (run separately) - Or integrate directly against the API
## What's New (in the last few releases)
0.1.44 release candidate - Chat Macros v1.1 authoring/output profiles, local Workspace Persona provenance and owner-validated Chat history selection. - Reproducible VN failed-slot recipes; native-fork projection/storage groundwork (public native forks are not enabled yet). - PostgreSQL tenant isolation, authenticated diagnostic access, core reliability, Sync/OCR/cancellation and post-0.1.43 review repairs. - All 272 commits and 37 merged PRs are listed in the [change inventory](Docs/Development/releases/0.1.44-change-inventory.md). - Back up and drain all database writers before migrating. PostgreSQL needs a dedicated NOSUPERUSER/NOBYPASSRLS application role that does not own content tables. Mixed-version rolling upgrades and old-binary rollback are unsupported. - See [release notes](Docs/Published/RELEASE_NOTES.md) and the [release plan](Docs/superpowers/plans/2026-09-27-release-0.1.44-plan.md) for migration details, limits and pending final release/date approval.
0.1.43 release - OSCE practice, advanced quiz metrics, structured prompt recipes and Writing Predict/Fill service prompts. - Usable-provider scheduled-task overrides, Explainer navigation and visual-novel generation recovery. - Fresh-install and post-merge repairs across Chat, Notes, Study, World Books, account isolation and PostgreSQL lifecycle handling. - macOS guest mismatch, timeout and launchd recovery drills. - All 616 commits through PR #2970 are listed in the [change inventory](Docs/Development/releases/0.1.43-change-inventory.md). - Published and verified; main-to-dev synchronization and post-publication review repairs are complete. See the [release plan](Docs/superpowers/plans/2026-09-20-release-0.1.43-plan.md). - Back up persistent data before upgrading. UAT261 remains open and a fresh complete UAT matrix is outstanding.
0.1.42 release Included in the `0.1.42` release candidate (frozen through PR #2941): - Chat macros and service prompts, notes/personal-context sync, Research Workspace improvements, and standalone HTML presentations. - Audio/persona and buddy workflows, MCP HTTP/SSE transports, durable webhooks, administration improvements, and production backup/rollback checks. - Provider credential resolution is shared across Chat, RAG, embeddings, and audio, with fail-closed BYOK handling and safer streaming. - Inline embeddings workflows, Jobs admission transactions and quotas, and strict Skills live-integration certification were hardened. - The protected frontend source-available boundary, trusted license gate, protected-branch prerequisites, and license-first PR sequencing were established. - Package, FastAPI, README, release-note, and MkDocs metadata were bumped to `0.1.42`. - Protected frontend source remains under PolyForm Perimeter 1.0.1; the `0.1.42` Countdown grant adds `AGPL-3.0-only` on September 10, 2028 at 12:00 UTC. See [the immutable release record](LICENSES/releases/0.1.42/release.json). No protected frontend binary is published. Still active on `dev`: - Treat [CHANGELOG.md](CHANGELOG.md) as the authoritative branch-level history for work after this frozen release. License-first CI follow-ups are merged and included in the 0.1.44 candidate. Currently landing on `dev` (post-`0.1.46` branch work): - Follow the backlog for separately tracked UAT, native-fork recovery/synchronization and certification work. See [CHANGELOG.md](CHANGELOG.md) for the full running history and [Docs/Published/RELEASE_NOTES.md](Docs/Published/RELEASE_NOTES.md) for published release notes.
0.1.41 release Included in the `0.1.41` release: - Research and source-grounded learning expanded through spaced repetition, advanced quizzes, Paperless source workflows, Research Workspace discovery/PDF handoff foundations, beginner UAT, and status/reconciliation work. - Skills gained MCP catalog rendering, render-tool binding, UX and extension parity, and focused UAT gates, with service-prompt customization groundwork documented for follow-on delivery. - Quick Ingest, Chatbooks, authentication recovery, Parakeet ONNX configuration, extension launch behavior, frontend CodeQL paths, Watchlists briefings, Jobs operations, and local/custom provider egress were hardened. - Package, FastAPI, README, release-note, and MkDocs metadata were bumped to `0.1.41`. Still active on `dev`: - `0.1.41` is superseded by the `0.1.42` release metadata above. - Treat [CHANGELOG.md](CHANGELOG.md) as the authoritative branch-level history for what has entered the release train. See [CHANGELOG.md](CHANGELOG.md) for the full running history and [Docs/Published/RELEASE_NOTES.md](Docs/Published/RELEASE_NOTES.md) for published release notes.
0.1.40 release Included in the `0.1.40` release: - Chatbooks Backup & Import now covers full user-account backup/restore flows with redacted inventory previews, bundled tldw-stored artifacts, embeddings, generated documents, OpenWebUI hydration reuse, and post-write verification metadata. - Chat document uploads gained explicit processing choices, queue replay support, sidepanel/WebUI handoff plumbing, backend preflight contracts, and client/service coverage. - Release hardening fixed Guardian timestamp mutation, visual identity ZIP path validation on Windows, legacy audio WebSocket config handling, macOS PyAudio setup, sandbox concurrency tests, CodeQL alert paths, and YouTube/media ingest worker startup defaults. - Package, FastAPI, README, and MkDocs metadata were bumped to `0.1.40`. Still active on `dev`: - `0.1.40` is superseded by the `0.1.41` release metadata above. - Treat [CHANGELOG.md](CHANGELOG.md) as the authoritative branch-level history for what has entered the release train. See [CHANGELOG.md](CHANGELOG.md) for the full running history and [Docs/Published/RELEASE_NOTES.md](Docs/Published/RELEASE_NOTES.md) for published release notes.
0.1.39 release Included in the `0.1.39` release: - Character expression images and emote directives landed across backend metadata, streaming recovery, WebUI editing, onboarding nudges, and sidepanel/chat rendering. - Settings UX follow-ups added the split settings IA, chat backdrop transparency controls, setup/recovery hardening, and TTS voice preview. - Release hardening stabilized chat cockpit CI gates, strict audio stream protocol flows, OpenAPI fingerprints, frontend review fixes, and notification timestamp handling. - Package, FastAPI, README, and MkDocs metadata were bumped to `0.1.39`. Still active on `dev`: - `0.1.39` is superseded by the `0.1.40` release metadata above. - Treat [CHANGELOG.md](CHANGELOG.md) as the authoritative branch-level history for what has entered the release train. See [CHANGELOG.md](CHANGELOG.md) for the full running history and [Docs/Published/RELEASE_NOTES.md](Docs/Published/RELEASE_NOTES.md) for published release notes.
0.1.38 corrective release Included in the `0.1.38` corrective release: - Docker release publishing now targets GitHub Container Registry only and no longer attempts a Docker Hub login. - PyPI release gating installs PortAudio before installing development dependencies, fixing the PyAudio build failure on hosted Linux runners. - Package, FastAPI, README, and MkDocs metadata were bumped to `0.1.38`. Still active on `dev`: - `0.1.38` is superseded by the `0.1.39` release metadata above. - Treat [CHANGELOG.md](CHANGELOG.md) as the authoritative branch-level history for what has entered the release train. See [CHANGELOG.md](CHANGELOG.md) for the full running history and [Docs/Published/RELEASE_NOTES.md](Docs/Published/RELEASE_NOTES.md) for published release notes.
0.1.37 release Included in the `0.1.37` release: - Research Workspace media outputs landed with media capabilities, output job APIs, worker startup contracts, persisted artifacts, infographic generation, video overview generation, frontend client contracts, and rendered output surfaces. - Research output handling was hardened for owner isolation, status lookup failures, worker errors, artifact persistence, context excerpts, PNG byte validation, and encoded/embedded/delimited metadata path rejection. - Package, FastAPI, README, and MkDocs metadata were bumped to `0.1.37`. Still active on `dev`: - `0.1.37` is superseded by the `0.1.38` corrective release metadata above. - Treat [CHANGELOG.md](CHANGELOG.md) as the authoritative branch-level history for what has entered the release train. See [CHANGELOG.md](CHANGELOG.md) for the full running history and [Docs/Published/RELEASE_NOTES.md](Docs/Published/RELEASE_NOTES.md) for published release notes.
0.1.36 corrective release Included in the `0.1.36` corrective release: - MCP external resource discovery/read parity landed for external servers, including redacted `external://` resource URIs, profile-aware grants, gateway bootstrap wiring, and stdio/websocket list/read support. - The `0.1.35` release merge was synced back into `dev` so `main` remains the release branch and `dev` remains the forward working branch. - Package, FastAPI, README, and MkDocs metadata were bumped to `0.1.36` for a clean patch release after `0.1.35`. Still active on `dev`: - `0.1.36` is superseded by the `0.1.37` release metadata above. - Treat [CHANGELOG.md](CHANGELOG.md) as the authoritative branch-level history for what has entered the release train. See [CHANGELOG.md](CHANGELOG.md) for the full running history and [Docs/Published/RELEASE_NOTES.md](Docs/Published/RELEASE_NOTES.md) for published release notes.
0.1.35 release-prep rollup Included in the `0.1.35` release-prep rollup: - Visual identity expression packs added backend storage, APIs, import jobs, chat runtime integration, frontend management surfaces, provenance bridging, and casting resolver controls. - MCP docs source synchronization added local and URL source registries, source sync tools, approved URL tracking, redirect handling, tombstone cleanup, and privacy redaction hardening. - Research workspace source-worker follow-ups surfaced ingest failure codes, while backend PyPI auto-publish wiring was prepared for main-branch releases. - User/developer documentation wikis were split, Docker publishing actions were updated, llama.cpp character-chat provider aliases were fixed, and TTS/Embeddings/Character_Chat_NEW suites were unquarantined. - Post-PR #2571 CI follow-ups fixed Guardian notification payload timestamp mutation and retargeted a stale UX smoke assertion to a current shell route. - Chat Workspace live flow, WebUI setup handoff, first-run MCP tool packs, RAG MCP module, Research Workspace share/UAT/NotebookLM parity, and API/WebUI/extension docs landed in the release train. - CI/release gates were expanded with OpenAPI drift checks, env/minimal startup smokes, property/contract coverage, singleton/isolation guards, test-quality triage, and PyPI test-suite gating/fail-closed detection. - Hardening landed for audio transcription inputs, Parakeet ONNX dependencies, runtime auth persistence, integrations egress, redirect credential stripping, ACP reconnect cleanup, VZ guest validation, PostgreSQL setup timestamps, and tracked `node_modules` cleanup. - Current release branch follow-ups moved Visual Identity idempotency workflow logic into core service code and kept main-only `node_modules` cleanup in the merge path. Still active on `dev`: - `0.1.35` is superseded by the `0.1.36` corrective release metadata above. - Treat [CHANGELOG.md](CHANGELOG.md) as the authoritative branch-level history for what has entered the release train. See [CHANGELOG.md](CHANGELOG.md) for the full running history and [Docs/Published/RELEASE_NOTES.md](Docs/Published/RELEASE_NOTES.md) for published release notes.
## Privacy & Security - Self-hosted by design; no telemetry or data collection. - Users own and control their data; see hardening guidance for production. - Auth modes: single-user API key or multi-user JWT. - Security reporting and hardening docs: `SECURITY.md`, `Docs/Published/User_Guides/Server/Production_Hardening_Checklist.md`. - Outbound URL egress policy blocks SSRF to private networks and disallowed ports for media downloads (audio/video/doc URLs), with test-mode DNS relaxations for hostnames. ## Highlights - Ingest and process media (video, audio, docs, web) with OCR and chunking. - OpenAI-compatible APIs for Chat, Audio (STT/TTS), Embeddings, and Evals. - Hybrid search + RAG (BM25 + vector + rerank) with ChromaDB or pgvector. - Multi-provider LLM and TTS/STT support, including local models. - MCP server with JWT/RBAC, tool execution, WebSockets, and metrics. - Next.js WebUI and Admin UI (both in active development).
Full highlights (expanded) - Media ingestion & processing: video, audio, PDFs, EPUB, DOCX, HTML, Markdown, XML, MediaWiki dumps; OCR for PDFs/images; metadata extraction; configurable chunking. - Custom-built Chunking library, tldw_Chunker, supporting token, word, sentence, paragraph, semantic, hierarchical and template chunking approaches. - Audio & speech: real-time and file STT via faster_whisper, NVIDIA NeMo (Canary/Parakeet), Qwen2Audio; diarization/VAD; TTS backends — commercial: OpenAI, ElevenLabs; local: Kokoro, PocketTTS (Python/ONNX), PocketTTS.cpp (native binary), LuxTTS, Higgs, Chatterbox, Dia, VibeVoice, VibeVoice Realtime, NeuTTS, IndexTTS2, Supertonic, Supertonic2, Qwen3-TTS, EchoTTS; voice catalog + audio jobs queue. - Audiobooks: parse + chapter detection, per-chapter voice settings, optional TTS provider overrides (alignment/subtitles Kokoro-only), and M4B packaging (API-only). - Search & retrieval (RAG): hybrid BM25 + vector (ChromaDB/pgvector), re-ranking, contextual retrieval, OpenAI-compatible embeddings, vector stores API, and media embeddings ingestion. 50+ optional parameters available for tuning. - Chat & providers: `/api/v1/chat/completions` (OpenAI-compatible), 16+ providers (commercial + self-hosted), character chat, budgets/allowlists, moderation endpoint. - Knowledge management: notes, prompt library, character cards, soft-delete with recovery, Chatbooks import/export including OpenWebUI chat JSON migration, OpenWebUI webui.db migration, and post-import attachment hydration, flashcards (.apkg), reading items/highlights. - Prompt Studio & evaluations: projects, prompt testing/optimization, unified evaluation APIs (G-Eval, RAG, OCR, embeddings A/B tests, batch metrics). Full evaluations and prompt management. - Research & web scraping: multi-provider web search, academic paper search (arXiv/PubMed/etc.), scraping jobs with cookies/progress, aggregation/final answers. - Connectors: Google Drive + Notion OAuth import; connector policies/quotas. - Outputs & artifacts: templated outputs (Markdown/HTML/MP3 via TTS), persisted artifacts with retention controls. - Automation: watchlists (sources/groups/tags, OPML import/export, scheduled runs) and workflows engine/scheduler (WIP). - MCP Server: production MCP with JWT/RBAC, tool execution, WebSockets, metrics, and health endpoints. Use existing tools or add your own using a handy guide. Setup tool categories and collections, allowing for easier context management. - Admin/Ops: orgs/teams/roles, API key management, audit logging, usage/LLM cost reporting, Prometheus metrics + Grafana dashboards, PostgreSQL content mode + backup/restore helpers.
## Feature Status See the full [Feature Status Matrix at `Docs/Published/Overview/Feature_Status.md`](./Docs/Published/Overview/Feature_Status.md). ## Quickstart `pip install tldw-server` is not generally available yet; the package metadata and artifact checks are prepared, but PyPI publishing is still in progress. For now, use this repository checkout and the setup paths below (`make` and no-`make`). ### Preflight Check (Recommended)
Host prerequisite checks After cloning the repository, you can run the optional Makefile helper checks: ```bash make quickstart-prereqs ``` This helper checks your host Python and optional media tooling. If it reports missing project Python packages on a fresh checkout, continue with the selected setup profile; `make setup-docker-single`, `make setup-docker-multi`, and `make install-local` create the setup environments they need. If `make` is unavailable (common on Windows), run equivalent host checks manually: ```powershell py -3.12 --version # or py -3.13 / -3.11 ffmpeg -version docker --version # only if using Docker paths ```
### At-a-Glance Commands
Public setup profiles and command shortcuts Choose one public setup profile: | Profile | Prepare | Start | Verify | |------|---------|-------|--------| | Docker single-user + WebUI | `make setup-docker-single` | `make start-docker-single` | `make verify-docker-single` | | Docker multi-user + Postgres | Export generated `ADMIN_USERNAME` / `ADMIN_PASSWORD`, then `make setup-docker-multi` | `make start-docker-multi` | `make verify-docker-multi` | | Local single-user | `make install-local` then `make setup-local-single` | `make start-local-single` | `make verify-local-single` | ```bash git clone https://github.com/rmusser01/tldw_server.git && cd tldw_server # Shortest Docker single-user + WebUI alias: make quickstart # Equivalent explicit Docker single-user + WebUI lifecycle: make setup-docker-single make start-docker-single make verify-docker-single # Docker multi-user + Postgres: export ADMIN_USERNAME=tldw-admin export ADMIN_PASSWORD="$(python3 -c 'import secrets; print(secrets.token_urlsafe(24))')" make setup-docker-multi make start-docker-multi make verify-docker-multi # Local single-user: make install-local make setup-local-single make start-local-single make verify-local-single # Compatibility aliases: # make quickstart-docker-webui # same as make quickstart # make quickstart-docker # Docker single-user API only # make quickstart-install # local install only; does not start the server # If `python3` is older than 3.11 on your machine: # make install-local PYTHON=python3.13 # or python3.12 / python3.11 # Force a full image rebuild when needed: # make start-docker-single DOCKER_BUILD=true ``` If `make` is unavailable, use [No-Make Path (Windows-Friendly)](#no-make-path-windows-friendly).
### Default Docker Path (Makefile)
Docker single-user + WebUI quickstart details ```bash # from repo root make quickstart ``` This target: - Runs `make setup-docker-single`, `make start-docker-single`, and `make verify-docker-single`. - Matches the existing `quickstart-docker-webui` compatibility alias. - Brings up the API at `http://localhost:8000` and WebUI at `http://localhost:8080`. - Keeps the default browser path on same-origin browser API requests through the WebUI proxy. Want a more advanced deployment? - Team/public deployment: [Docker Multi-User + Postgres Profile](Docs/Getting_Started/Profile_Docker_Multi_User_Postgres.md) - API-only Docker deployment: `make quickstart-docker` - Use the advanced/custom-host path for LAN, reverse-proxy, or custom-domain browser access only when you need browsers to call a non-default host.
### No-Docker Path (Makefile, Development)
Local single-user setup with Makefile targets ```bash # from repo root make install-local make setup-local-single make start-local-single # If `python3` is older than 3.11 on your machine: # make install-local PYTHON=python3.13 # or python3.12 / python3.11 ``` These targets: - Creates `.venv` if missing and installs dependencies. - Creates `tldw_Server_API/Config_Files/.env` from `.env.example` if missing. - Initializes AuthNZ (non-interactive). - Starts the local API server at `http://127.0.0.1:8000`. Verify with: ```bash make verify-local-single ``` `make quickstart-install` remains a compatibility alias for local installation only. It does not start the server.
### No-Make Path (Windows-Friendly)
Shortcut scripts and manual compose commands Use these paths when `make` is not available. Local single-user: - Shortcut scripts from the repository root: - macOS/Linux terminal: `./quick-launch.sh [api|webui|all]` - macOS Finder: double-click `quick-launch.command` for the default `all` mode. - Windows PowerShell: `.\quick-launch.ps1 [api|webui|all]` - These scripts create or update `.venv`, run the `local-single` setup wizard when the API is started, and default to `all`: API at `http://127.0.0.1:8000` plus WebUI at `http://127.0.0.1:8080`. - Use `api` for backend-only startup or `webui` when the API is already running. - For manual control, follow [Manual Setup](#manual-setup) below, then start with `python -m uvicorn tldw_Server_API.app.main:app --reload`. Docker single-user + WebUI: ```powershell # from repo root if (!(Test-Path "tldw_Server_API/Config_Files/.env")) { Copy-Item "tldw_Server_API/Config_Files/.env.example" "tldw_Server_API/Config_Files/.env" } # For non-localhost browser access, uncomment both advanced/custom-host overrides: # $env:NEXT_PUBLIC_TLDW_DEPLOYMENT_MODE="advanced" # $env:NEXT_PUBLIC_API_URL="http://YOUR_HOST_OR_DOMAIN:8000" docker compose --env-file tldw_Server_API/Config_Files/.env -f Dockerfiles/docker-compose.single-user.yml -f Dockerfiles/docker-compose.webui.yml up -d --build curl http://localhost:8000/health ``` Docker multi-user + Postgres: ```powershell # from repo root if (!(Test-Path "tldw_Server_API/Config_Files/.env")) { Copy-Item "tldw_Server_API/Config_Files/.env.example" "tldw_Server_API/Config_Files/.env" } $env:ADMIN_USERNAME="tldw-admin" $env:ADMIN_PASSWORD = py -3.12 -c "import secrets; print(secrets.token_urlsafe(24))" # Run the tldw-setup init command from the multi-user profile guide first. $env:TLDW_ENV_FILE=(Resolve-Path "tldw_Server_API/Config_Files/.env").Path docker compose -f Dockerfiles/docker-compose.multi-user-postgres.yml up -d --build ```
### Manual Setup
Manual virtualenv, environment, auth, and startup steps Supported Python versions: - Minimum: Python 3.11+ - CI-tested: Python 3.11, 3.12, and 3.13 - Recommended for local development: Python 3.12 Prerequisites: - Required: Python + ffmpeg - Optional (for audio capture paths): PyAudio/PortAudio Platform setup examples: ```bash # macOS (Homebrew) brew install ffmpeg portaudio # Ubuntu/Debian sudo apt update sudo apt install -y ffmpeg portaudio19-dev python3-pyaudio # Fedora/RHEL (RPM Fusion may be needed for ffmpeg) sudo dnf install -y ffmpeg portaudio-devel python3-pyaudio ``` Windows notes: - Install ffmpeg (for example with `winget` or Chocolatey). - For PyAudio, run `pip install pyaudio` in the activated venv. If wheel/build fails, install Microsoft C++ Build Tools and retry. 1) **Create and activate a virtual environment** ```bash # macOS/Linux: choose a supported interpreter explicitly (3.12 recommended) python3.12 -m venv .venv # or python3.13 / python3.11 source .venv/bin/activate # Windows (PowerShell) py -3.12 -m venv .venv # or -3.13 / -3.11 .venv\Scripts\Activate.ps1 # Confirm venv interpreter version python --version # Install project dependencies pip install -e . ``` 2) **Configure environment** ```bash # macOS/Linux cp tldw_Server_API/Config_Files/.env.example tldw_Server_API/Config_Files/.env # Windows (PowerShell) # Copy-Item tldw_Server_API/Config_Files/.env.example tldw_Server_API/Config_Files/.env # Edit tldw_Server_API/Config_Files/.env and set: # SINGLE_USER_API_KEY= ``` 3) **Initialize auth** (use `--non-interactive` to skip prompts) ```bash python -m tldw_Server_API.app.core.AuthNZ.initialize --non-interactive ``` 4) **Start the server** ```bash python -m uvicorn tldw_Server_API.app.main:app --reload ``` 5) **Verify** ```bash curl http://localhost:8000/health # Expected: {"status":"ok",...} ``` - API docs: http://127.0.0.1:8000/docs - Quickstart redirect: http://127.0.0.1:8000/api/v1/config/quickstart
Optional extras and pyaudio setup ```bash # Optional pip extras pip install -e ".[multiplayer]" # multi-user/PostgreSQL features pip install -e ".[tooling]" # unified tooling smoke helpers/scripts pip install -e ".[dev]" # tests and linters pip install -e ".[otel]" # OpenTelemetry metrics/tracing # pyaudio (needed for audio capture and some processing paths) # Linux (Ubuntu/Debian): sudo apt install -y portaudio19-dev python3-pyaudio # Linux (Fedora/RHEL): sudo dnf install -y portaudio-devel python3-pyaudio # macOS: brew install portaudio && pip install pyaudio # Windows (PowerShell): pip install pyaudio # if build fails, install Microsoft C++ Build Tools ``` MCP secrets (`MCP_JWT_SECRET`, `MCP_API_KEY_SALT`) are auto-generated per-process when MCP runs with local-only defaults (loopback IP/origin restrictions). These ephemeral secrets do not persist across restarts. For shared or production deployments (when MCP is accessible beyond localhost), set these explicitly in `tldw_Server_API/Config_Files/.env` and rotate regularly. Run these commands in your shell and paste the resulting values into `tldw_Server_API/Config_Files/.env` (do not paste the command templates themselves): ```bash # Generate secure secrets — run each command, then copy the output into tldw_Server_API/Config_Files/.env openssl rand -base64 32 # paste the output as the value for MCP_JWT_SECRET openssl rand -base64 32 # paste the output as the value for MCP_API_KEY_SALT ``` See [MCP System Admin Guide](Docs/MCP/Unified/System_Admin_Guide.md) for details.
### Next Steps by Goal
Goal-oriented guide links | I want to... | Guide | |--------------|-------| | Choose the right onboarding path | [Getting Started Index](Docs/Getting_Started/README.md) | | Start the default Docker + WebUI setup | `make quickstart` | | Start from local single-user, then launch the WebUI | [Local Profile: Add the WebUI](#local-profile-add-the-webui) | | Build apps against the API locally | [Local Single-User Profile](Docs/Getting_Started/Profile_Local_Single_User.md) | | Run on my home server with Docker | [Docker Single-User Profile](Docs/Getting_Started/Profile_Docker_Single_User.md) | | Deploy for a team with proper security | [Docker Multi-User + Postgres Profile](Docs/Getting_Started/Profile_Docker_Multi_User_Postgres.md) | | Develop the WebUI or browser extension | [Extension & Web UI Development Guide](apps/DEVELOPMENT.md) |
### Local Profile: Add the WebUI
Launch the WebUI alongside a local API The repo-root quick-launch scripts start the local API and WebUI together by default: ```bash ./quick-launch.sh all ``` ```powershell .\quick-launch.ps1 all ``` If you already completed the [Local Single-User Profile](Docs/Getting_Started/Profile_Local_Single_User.md) and your API is running at `http://127.0.0.1:8000`, this is the shortest add-on path: ```bash # from the repo root cd apps/tldw-frontend cp .env.local.example .env.local bun install bun run dev -- -p 8080 ``` Confirm `.env.local` contains: ```bash NEXT_PUBLIC_API_URL=http://127.0.0.1:8000 NEXT_PUBLIC_API_VERSION=v1 # Optional in single-user mode: # NEXT_PUBLIC_X_API_KEY=your_single_user_api_key ``` Open `http://localhost:8080`. Need Bun installation steps, npm fallback, or LAN/mobile access setup? See [Run the Web UI (WIP)](#run-the-web-ui-wip).
Sidecar workers (optional) Run the API and Jobs workers as separate processes (recommended for heavier workloads or when using multiple Uvicorn workers): ```bash ./start-sidecars.sh ``` Notes: - Sets `TLDW_WORKERS_SIDECAR_MODE=true` to keep the API process from starting in-process workers. - Default worker list is documented in `Docs/Deployment/Sidecar_Workers.md` (regen via `python Helper_Scripts/Deployment/generate_sidecar_files.py`). - Customize worker list with `TLDW_SIDECAR_WORKERS=chatbooks,files,data_tables,prompt_studio,privilege_snapshots,audio,media_ingest,evals_abtest` or point to a different manifest with `TLDW_WORKERS_MANIFEST=/path/to/manifest.json`. - Logs are written to `.logs/sidecars/` by default. - Compose overlay: `docker compose -f Dockerfiles/docker-compose.yml -f Dockerfiles/docker-compose.workers.yml up -d --build`. - systemd and launchd examples: `Docs/Deployment/Sidecar_Workers.md`. In-process workers are fine for light dev use, but SQLite + multiple Uvicorn workers can increase lock contention; use sidecars or Postgres for heavier workloads.
Model container cycling (optional, Docker) For low-VRAM dev rigs, you can cycle model containers instead of running all model stacks at once: ```bash make model-cycle \ MODEL_CYCLE_FIRST=your-llm-container \ MODEL_CYCLE_SECOND=your-tts-container \ MODEL_CYCLE_EXCLUDED=postgres,redis \ MODEL_CYCLE_FIRST_BOOT_WAIT=30 \ MODEL_CYCLE_SECOND_BOOT_WAIT=10 ``` Notes: - Runs `Helper_Scripts/model_container_cycle.py`. - Workflow is: stop all running containers except exclusions, start first model container, stop it, then start second model container. - Use `MODEL_CYCLE_DRY_RUN=true` to print Docker commands without executing them. - Container names are whatever Docker reports via `docker ps --format '{{.Names}}'`.
### Run the Web UI (WIP)
WebUI quickstart, development server, and LAN notes The current Next.js UI is a work in progress and may be unstable, buggy, or rough around the edges. Use Bun (recommended) or Node.js with npm/yarn/pnpm. Quickest full-stack path (containerized API + WebUI): ```bash make quickstart-docker-webui # API: http://localhost:8000 # WebUI: http://localhost:8080 # Optional advanced/custom-host path for LAN, reverse-proxy, or custom-domain browser access: # make quickstart-docker-webui NEXT_PUBLIC_TLDW_DEPLOYMENT_MODE=advanced NEXT_PUBLIC_API_URL=http://YOUR_HOST_OR_DOMAIN:8000 # Optional (force image rebuild instead of cached layers): # make quickstart-docker-webui DOCKER_BUILD=true ``` No-`make` equivalent: use [No-Make Path (Windows-Friendly)](#no-make-path-windows-friendly). Default quickstart behavior keeps same-origin browser API requests through the WebUI proxy. When you intentionally move the browser onto an advanced/custom-host path for LAN, reverse-proxy, or custom-domain browser access, set both `NEXT_PUBLIC_TLDW_DEPLOYMENT_MODE=advanced` and `NEXT_PUBLIC_API_URL`. For the Docker single-user WebUI quickstart, the WebUI container reads `AUTH_MODE` and `SINGLE_USER_API_KEY` at runtime and exposes the key to the browser only through the local runtime bootstrap path. You do not need to rebuild the WebUI image when the single-user key changes. `NEXT_PUBLIC_X_API_KEY` remains available for advanced/static WebUI builds where the operator deliberately wants to bake a public browser credential into the client bundle. Do not use it for the normal Docker quickstart path. Local WebUI development (API should already be running, for example via `make quickstart`): If Bun is not installed yet, install it first: ```bash # macOS/Linux curl -fsSL https://bun.sh/install | bash # Windows (PowerShell) powershell -c "irm bun.sh/install.ps1 | iex" # Open a new terminal and verify: bun --version ``` 1) From the repo root: ```bash cd apps/tldw-frontend cp .env.local.example .env.local ``` 2) Set your API URL (defaults shown): ```bash # .env.local NEXT_PUBLIC_API_URL=http://127.0.0.1:8000 NEXT_PUBLIC_API_VERSION=v1 # Optional for single-user mode: # NEXT_PUBLIC_X_API_KEY=your_api_key ``` 3) Install and run the dev server (use port 8080 to match the server's built-in local browser defaults): ```bash bun install bun run dev -- -p 8080 # npm fallback: # npm install # npm run dev -- -p 8080 ``` Open http://localhost:8080
LAN/mobile access setup (optional) Access from another device on your network (for example, phone/tablet): ```bash # API (from repo root) python -m uvicorn tldw_Server_API.app.main:app --host 0.0.0.0 --port 8000 # tldw_Server_API/Config_Files/.env # Only needed when the browser connects from a non-localhost origin: # ALLOWED_ORIGINS=http://YOUR_SERVER_IP:8080 # apps/tldw-frontend/.env.local # NEXT_PUBLIC_API_URL=http://YOUR_SERVER_IP:8000 # WebUI bun run dev -- -H 0.0.0.0 -p 8080 ``` Then open `http://YOUR_SERVER_IP:8080` from your mobile device on the same network.
Tip: http://127.0.0.1:8000/api/v1/config/quickstart redirects to your configured quickstart target. Security note: avoid exposing this quickstart setup directly to the public internet. For internet-facing access, use HTTPS with a reverse proxy and follow `Docs/Deployment/First_Time_Production_Setup.md`.
### Docker Compose
Manual Docker Compose reference For the fastest Docker path, use the quickstart targets in [At-a-Glance Commands](#at-a-glance-commands). This section is the canonical manual/no-`make` compose reference. Quickstart targets skip forced rebuilds by default; pass `DOCKER_BUILD=true` to force a rebuild. Or manually: ```bash # Docker single-user + WebUI docker compose --env-file tldw_Server_API/Config_Files/.env \ -f Dockerfiles/docker-compose.single-user.yml \ -f Dockerfiles/docker-compose.webui.yml up -d --build curl http://localhost:8000/health # Verify ``` On first start in `single_user` mode, the app container now: - Ensures `SINGLE_USER_API_KEY` is secure (generates one if missing/placeholder). - Runs `AuthNZ.initialize --non-interactive` once per persisted Docker data volume. - View the configured key with: ```bash grep '^SINGLE_USER_API_KEY=' tldw_Server_API/Config_Files/.env ```
More Docker options (multi-user, overlays) ```bash # Multi-user + Postgres users DB export TLDW_ENV_FILE="$(pwd)/tldw_Server_API/Config_Files/.env" docker compose -f Dockerfiles/docker-compose.multi-user-postgres.yml up -d --build # Dev overlay — unified streaming (non-prod) docker compose -f Dockerfiles/docker-compose.yml -f Dockerfiles/docker-compose.dev.yml up -d --build # WebUI overlay with single-user profile (Next.js container on :8080) # PowerShell advanced/custom-host browser access: # $env:NEXT_PUBLIC_TLDW_DEPLOYMENT_MODE="advanced" # $env:NEXT_PUBLIC_API_URL="http://YOUR_HOST_OR_DOMAIN:8000" docker compose -f Dockerfiles/docker-compose.single-user.yml -f Dockerfiles/docker-compose.webui.yml up -d --build # Check status docker compose -f Dockerfiles/docker-compose.single-user.yml ps docker compose -f Dockerfiles/docker-compose.single-user.yml logs -f app # Proxy overlays # - Dockerfiles/docker-compose.proxy.yml (Caddy) # - Dockerfiles/docker-compose.proxy-nginx.yml (Nginx) # pgvector + pgbouncer for Postgres docker compose -f Dockerfiles/docker-compose.yml -f Dockerfiles/docker-compose.pg.yml up -d --build ```
Notes - Run compose commands from the repository root. The public profile compose files are `Dockerfiles/docker-compose.single-user.yml` and `Dockerfiles/docker-compose.multi-user-postgres.yml`. - For `Dockerfiles/docker-compose.webui.yml`, the default quickstart leaves `NEXT_PUBLIC_API_URL` empty so browsers stay on same-origin browser API requests through the WebUI proxy. Set `NEXT_PUBLIC_API_URL` only for the advanced/custom-host path for LAN, reverse-proxy, or custom-domain browser access. - Single-user WebUI auth is runtime-configured in the default Docker quickstart. The `webui` service receives `AUTH_MODE`, `SINGLE_USER_API_KEY`, and the explicit local-quickstart `TLDW_WEBUI_EXPOSE_RUNTIME_AUTH=1` opt-in written by setup; browsers do not require `NEXT_PUBLIC_X_API_KEY`. The shared WebUI compose overlay defaults runtime-auth exposure to disabled when this opt-in is absent. - `NEXT_PUBLIC_API_VERSION` and `NEXT_PUBLIC_X_API_KEY` are build-time public values in the client bundle for advanced/static WebUI builds only. - The primary UI is the Next.js WebUI in `apps/tldw-frontend/`. - Run it separately for development, or use `Dockerfiles/docker-compose.webui.yml` for a containerized WebUI. - For unified streaming validation in non-prod, prefer the dev overlay above. You can also export `STREAMS_UNIFIED=1` directly in your environment.
### Supporting Services via Docker
Supporting services (Postgres, Redis, Prometheus, Grafana) Run only infrastructure services without the app. Postgres + Redis (base compose) ```bash docker compose -f Dockerfiles/docker-compose.yml up -d postgres redis ``` Prometheus + Grafana (embeddings compose, monitoring profile) ```bash docker compose -f Dockerfiles/docker-compose.embeddings.yml --profile monitoring up -d prometheus grafana ``` All four together ```bash docker compose -f Dockerfiles/docker-compose.yml up -d postgres redis docker compose -f Dockerfiles/docker-compose.embeddings.yml --profile monitoring up -d prometheus grafana ``` Manage and verify ```bash # Status docker compose -f Dockerfiles/docker-compose.yml ps docker compose -f Dockerfiles/docker-compose.embeddings.yml ps # Logs docker compose -f Dockerfiles/docker-compose.yml logs -f postgres redis docker compose -f Dockerfiles/docker-compose.embeddings.yml logs -f prometheus grafana # Stop docker compose -f Dockerfiles/docker-compose.yml stop postgres redis docker compose -f Dockerfiles/docker-compose.embeddings.yml stop prometheus grafana # Remove docker compose -f Dockerfiles/docker-compose.yml down docker compose -f Dockerfiles/docker-compose.embeddings.yml down ``` Ports - Postgres: 5432 - Redis: 6379 - Prometheus: 9091 (container listens on 9090) - Grafana: 3000 Prometheus config - Create `Config_Files/prometheus.yml` to define scrape targets. Minimal self-scrape example: ```yaml global: scrape_interval: 15s scrape_configs: - job_name: 'prometheus' static_configs: - targets: ['localhost:9090'] ``` See Docs/Operations/monitoring/README.md for examples that scrape the API and worker orchestrator. Tip: See multi-user setup and production hardening in Docs/User_Guides/Server/Authentication_Setup.md and Docs/Published/Deployment/First_Time_Production_Setup.md.
## Usage Examples
Usage Examples Use the single-user API key with the `X-API-KEY` header. Chat (OpenAI-compatible) ```bash curl -s http://127.0.0.1:8000/api/v1/chat/completions \ -H "X-API-KEY: $SINGLE_USER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-4", "messages": [{"role": "user", "content": "Hello!"}], "stream": false }' ``` Embeddings (OpenAI-compatible) ```bash curl -s http://127.0.0.1:8000/api/v1/embeddings \ -H "X-API-KEY: $SINGLE_USER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "text-embedding-3-small", "input": "hello world" }' ``` Media Ingest (URL) ```bash curl -s -X POST http://127.0.0.1:8000/api/v1/media/add \ -H "X-API-KEY: $SINGLE_USER_API_KEY" \ -F "media_type=document" \ -F "title=Example Article" \ -F "keywords=demo,quickstart" \ -F "urls=https://www.example.com/some-article" ``` Media Search ```bash curl -s -X POST http://127.0.0.1:8000/api/v1/media/search \ -H "X-API-KEY: $SINGLE_USER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "query": "keyword" }' ``` Audio Transcription (file) ```bash curl -s -X POST http://127.0.0.1:8000/api/v1/audio/transcriptions \ -H "X-API-KEY: $SINGLE_USER_API_KEY" \ -F "file=@sample.wav" -F "model=whisper-1" ```
## Key Endpoints
Key Endpoints - Media: `POST /api/v1/media/add` - ingest/process media (URLs/files) with DB persistence ([docs](Docs/Code_Documentation/Ingestion_Media_Processing.md)) - Media Search: `POST /api/v1/media/search` - search ingested content ([docs](Docs/API-related/API_Design.md)) - Chat: `POST /api/v1/chat/completions` - OpenAI-compatible chat ([docs](Docs/API-related/Chat_API_Documentation.md)) - Chat Commands: `GET /api/v1/chat/commands` - list available slash commands ([docs](Docs/API-related/Chatbook_Features_API_Documentation.md#chat-tools-slash-commands)) - Chat Dictionary Validate: `POST /api/v1/chat/dictionaries/validate` - validate a chat dictionary ([docs](Docs/API-related/Chatbook_Features_API_Documentation.md#chat-dictionary-api)) - Embeddings: `POST /api/v1/embeddings` - OpenAI-compatible embeddings ([docs](Docs/API-related/Embeddings_API_Documentation.md)) - RAG: `POST /api/v1/rag/search` - unified RAG search ([docs](Docs/API-related/RAG-API-Guide.md)) - Audio STT: `POST /api/v1/audio/transcriptions` - file-based transcription ([docs](Docs/API-related/Audio_Transcription_API.md)) - Audio STT (WS): `WS /api/v1/audio/stream/transcribe` - real-time transcription ([docs](Docs/API-related/Audio_Transcription_API.md)) - Audio TTS: `POST /api/v1/audio/speech` - text-to-speech (streaming and non-streaming) ([docs](Docs/API-related/TTS_API.md)) - TTS Voices: `GET /api/v1/audio/voices/catalog` - voice catalog across providers ([docs](Docs/API-related/TTS_API.md)) - Voice Assistant: `POST /api/v1/voice/command` (REST) and `WS /api/v1/voice/assistant` (real-time) ([docs](Docs/API/Voice_Assistant.md)) - Audiobooks Parse: `POST /api/v1/audiobooks/parse` - parse + chapter preview ([docs](Docs/Product/Completed/Audiobook_Creation_PRD.md)) - Audiobooks Jobs: `POST /api/v1/audiobooks/jobs` - create audiobook job; `GET /api/v1/audiobooks/jobs/{job_id}` - status ([docs](Docs/Product/Completed/Audiobook_Creation_PRD.md)) - Audiobooks Subtitles: `POST /api/v1/audiobooks/subtitles` - render subtitle export ([docs](Docs/Product/Completed/Audiobook_Creation_PRD.md)) - Vector Stores: `POST /api/v1/vector_stores` - create; `POST /api/v1/vector_stores/{id}/query` - query ([docs](Docs/API-related/Vector_Stores_Admin_and_Query.md)) - OCR Backends: `GET /api/v1/ocr/backends` - available OCR providers ([docs](Docs/API-related/OCR_API_Documentation.md)) - VLM Backends: `GET /api/v1/vlm/backends` - available VLM providers ([docs](Docs/Code_Documentation/VLM_Backends.md)) - Connectors: `GET /api/v1/connectors/providers` - Drive/Notion providers ([docs](Docs/Product/External_Connectors_PRD.md)) - Outputs: `POST /api/v1/outputs` - generate output artifact (md/html/mp3) ([docs](Docs/Product/Completed/Content_Collections_PRD.md)) - Tasks: `POST /api/v1/tasks` + `GET /api/v1/tasks` - reminder task create/list ([docs](Docs/API-related/Reminder_Notifications_API.md)) - Notifications: `GET /api/v1/notifications` + `GET /api/v1/notifications/stream` - inbox list + realtime SSE ([docs](Docs/API-related/Reminder_Notifications_API.md)) - Metrics: `GET /api/v1/metrics/text` - Prometheus metrics (text format) ([docs](Docs/Deployment/Monitoring/Metrics_Cheatsheet.md)) - Providers: `GET /api/v1/llm/providers` - provider/models list ([docs](Docs/API-related/Providers_API_Documentation.md)) - MCP: `GET /api/v1/mcp/status` - MCP server status ([docs](Docs/MCP/Unified/System_Admin_Guide.md)) - MCP Hub Management: `GET /api/v1/mcp/hub/acp-profiles`, `GET /api/v1/mcp/hub/external-servers`, plus mutation endpoints for authorized principals ([docs](Docs/MCP/mcp_hub_management.md)) Admin maintenance - Chat model aliases cache reload: `POST /api/v1/admin/chat/model-aliases/reload` - Single-user (API key) ```bash curl -s -X POST http://127.0.0.1:8000/api/v1/admin/chat/model-aliases/reload \ -H "X-API-KEY: $SINGLE_USER_API_KEY" ``` - Multi-user (JWT) ```bash curl -s -X POST http://127.0.0.1:8000/api/v1/admin/chat/model-aliases/reload \ -H "Authorization: Bearer $JWT" ``` Examples - GET `/api/v1/chat/commands` response ```json { "commands": [ {"name": "time", "description": "Show the current time (optional TZ).", "required_permission": "chat.commands.time"}, {"name": "weather", "description": "Show current weather for a location.", "required_permission": "chat.commands.weather"} ] } ``` - POST `/api/v1/chat/dictionaries/validate` request ```json { "data": { "name": "Example", "entries": [ {"type": "literal", "pattern": "today", "replacement": "It is {{ now('%B %d') }}."}, {"type": "regex", "pattern": "User:(\\w+)", "replacement": "Hello, {{ match.group(1) }}!"} ] }, "schema_version": 1, "strict": false } ``` Minimal success response ```json { "ok": true, "schema_version": 1, "errors": [], "warnings": [], "entry_stats": {"total": 2, "regex": 1, "literal": 1}, "suggested_fixes": [] } ```
## Architecture & Repo Layout
Architecture & Repo Layout ```text / ├── tldw_Server_API/ # Main API server implementation │ ├── app/ │ │ ├── api/v1/ │ │ │ ├── endpoints/ # REST endpoints (media, chat, audio, rag, evals, etc.) │ │ │ ├── schemas/ # Pydantic models │ │ │ └── API_Deps/ # Shared dependencies (auth, DB, rate limits) │ │ ├── core/ # Core logic (AuthNZ, RAG, LLM, DB, TTS, MCP, etc.) │ │ ├── services/ # Background services │ │ ├── Setup_UI/ # Setup UI assets │ │ ├── static/ # Static assets │ │ └── main.py # FastAPI entry point │ ├── cli/ # CLI entry points │ ├── Config_Files/ # config.txt, example YAMLs, migration helpers │ ├── Databases/ # Default DBs (runtime data; some are gitignored) │ ├── tests/ # Pytest suite │ └── requirements.txt # Legacy pin set (prefer pyproject extras) ├── admin-ui/ # Admin dashboard ├── apps/tldw-frontend/ # Next.js WebUI (primary web client) ├── Docs/ # Documentation (API, Development, RAG, AuthNZ, TTS, etc.) ├── Helper_Scripts/ # Utilities (installers, prompt tools, doc generators) ├── mock_openai_server/ # Mock OpenAI-compatible API server for tests/dev ├── Dockerfiles/ # Docker images and compose files │ └── Monitoring/ # Monitoring stack assets ├── Databases/ # DBs (AuthNZ defaults here; content DBs per-user under user_databases/) ├── models/ # Optional model assets (if used) ├── pyproject.toml # Project configuration ├── README.md # Project README (this file) ├── start-sidecars.sh # Run API + Jobs workers as sidecar processes └── Project_Guidelines.md # Development philosophy ``` Notes - The primary UI is the Next.js WebUI in `apps/tldw-frontend/` (run separately). - SQLite is default for local dev; PostgreSQL is supported for AuthNZ, the content DBs (Media/ChaCha), Jobs, and the Scheduler. - `mock_openai_server/` is handy for local OpenAI-compatible API testing.
## Architecture Diagram Two views are provided: a **high-level overview** of how clients, the FastAPI app, the core module layers, and external providers fit together; and a detailed **module → database map** that shows every backend datastore as its own node (no single icon stands in for multiple databases). ### High-Level Overview
High-level Mermaid architecture diagram ```mermaid flowchart TB subgraph CLIENTS["Clients"] direction LR WebUI["Next.js WebUI"]:::client AdminUI["Admin UI"]:::client MCPClients["MCP Clients (IDE / tools)"]:::client ACPClients["ACP Clients (agents)"]:::client APIClients["CLI / HTTP Clients"]:::client end subgraph API_STACK["FastAPI Application"] direction LR API["FastAPI App /api/v1"]:::api Endpoints["Endpoints + Schemas"]:::module Deps["API Deps: Auth, DB, Rate Limits, Resource Governor"]:::module Workers["Background Services / Jobs Workers"]:::module Streaming["Streaming / SSE / WebSocket"]:::module end subgraph SEC["Auth & Security"] direction LR AuthNZ["AuthNZ"]:::core Security["Security"]:::core ResourceGov["Resource Governance"]:::core Moderation["Moderation / Guardian"]:::core Governance["Governance"]:::core PrivilegeMaps["Privilege Maps"]:::core end subgraph ING["Ingestion & Media"] direction LR Ingestion["Ingestion & Media Processing"]:::core IngestionSources["Ingestion Sources"]:::core Chunking["Chunking"]:::core WebScraping["Web Scraping"]:::core WebClipper["Web Clipper"]:::core ExternalSources["External Sources"]:::core end subgraph LLMLAYER["LLM & Conversation"] direction LR LLMCalls["LLM Calls"]:::core LocalLLM["Local LLM"]:::core Chat["Chat"]:::core ChatWorkflows["Chat Workflows"]:::core CharacterChat["Character Chat"]:::core Persona["Persona"]:::core end subgraph KNOW["Knowledge & RAG"] direction LR RAG["RAG"]:::core Embeddings["Embeddings"]:::core Notes["Notes"]:::core Collections["Collections"]:::core PromptMgmt["Prompt Management"]:::core Chatbooks["Chatbooks"]:::core end subgraph AUD["Audio & Speech"] direction LR Audio["Audio (STT)"]:::core TTS["TTS"]:::core Audiobooks["Audiobooks"]:::core VoiceAssistant["Voice Assistant"]:::core Meetings["Meetings"]:::core end subgraph EVAL["Evaluation & Study"] direction LR Evaluations["Evaluations"]:::core Flashcards["Flashcards"]:::core StudyPacks["Study Packs"]:::core Explainer["Explainer"]:::core Claims["Claims Extraction"]:::core Text2SQL["Text2SQL"]:::core end subgraph AG["Agents & Automation"] direction LR AgentOrch["Agent Orchestration"]:::core ACP["Agent Client Protocol"]:::core MCP["MCP Unified"]:::core Tools["Tools"]:::core Skills["Skills"]:::core Workflows["Workflows"]:::core Scheduler["Scheduler"]:::core Sandbox["Sandbox"]:::core CodeGraph["Code Graph"]:::core end subgraph RES["Research & Monitoring"] direction LR Research["Research"]:::core WebSearch["Web Search"]:::core Watchlists["Watchlists"]:::core Calendar["Calendar"]:::core Reminders["Reminders"]:::core Monitoring["Monitoring"]:::core end subgraph OUT["Outputs & Artifacts"] direction LR FileArtifacts["File Artifacts"]:::core Slides["Slides"]:::core ImageGen["Image Generation"]:::core VN["VN Assets / Play"]:::core Writing["Writing"]:::core end subgraph INFRA["Platform & Infrastructure"] direction LR DBM["DB Management"]:::core Jobs["Jobs"]:::core Storage["Storage"]:::core Notifications["Notifications"]:::core Metrics["Metrics"]:::core Billing["Billing"]:::core Usage["Usage"]:::core Sync["Sync"]:::core Infrastructure["Infrastructure"]:::core end DATA[("Datastores — see Module → Database map below")]:::db subgraph EXT["External Providers"] direction LR LLMCloud["LLM APIs (OpenAI, Anthropic, ...)"]:::ext LLMOnPrem["Local LLMs (vLLM, Ollama, llama.cpp)"]:::ext AudioProv["STT / TTS Providers"]:::ext OCRVLM["OCR / VLM"]:::ext MediaDL["yt-dlp / ffmpeg"]:::ext WebProv["Web Search / Scrapers"]:::ext end %% Clients to API WebUI --> API AdminUI --> API MCPClients --> API ACPClients --> API APIClients --> API %% Inside API stack API --> Endpoints API --> Deps API --> Workers API --> Streaming %% Endpoints to module layers Endpoints --> SEC Endpoints --> ING Endpoints --> LLMLAYER Endpoints --> KNOW Endpoints --> AUD Endpoints --> EVAL Endpoints --> AG Endpoints --> RES Endpoints --> OUT Endpoints --> INFRA %% Module layers to storage SEC --> DATA ING --> DATA LLMLAYER --> DATA KNOW --> DATA AUD --> DATA EVAL --> DATA AG --> DATA RES --> DATA OUT --> DATA INFRA --> DATA %% Module layers to external providers LLMLAYER --> EXT AUD --> EXT ING --> EXT RES --> EXT classDef client fill:#e8f3ff,stroke:#5b8def,color:#1f3b6e; classDef api fill:#fff4e6,stroke:#ff9800,color:#5d3d00; classDef module fill:#f4f6f8,stroke:#9aa5b1,color:#2d3748; classDef core fill:#eefbea,stroke:#34a853,color:#1e4620; classDef db fill:#f0eaff,stroke:#8e6cf1,color:#3a2a87; classDef ext fill:#fff0f0,stroke:#e57373,color:#7b1f1f; ```
### Core Modules → Databases
Module-to-database Mermaid map Each backend datastore is a separate node. Content DBs are **per-user** (under `Databases/user_databases//`); platform DBs are **shared/global** (under `Databases/`). SQLite is the default; PostgreSQL is supported for AuthNZ, the content DBs (Media/ChaCha, via `TLDW_CONTENT_DB_BACKEND=postgresql`), Jobs, and the Scheduler. Redis is an optional backend for caching, queues, and rate limiting. > Note: Collections, Watchlists, and Meetings persist **inside the per-user media content DB**, and VN Assets / > VN Play persist **inside `ChaChaNotes.db`** — they are modules that write to those DBs, not > separate database files. ```mermaid flowchart LR %% ---- Modules ---- AuthNZ["AuthNZ"]:::core Audit["Audit"]:::core Ingestion["Ingestion & Media"]:::core Collections["Collections"]:::core Watchlists["Watchlists"]:::core Meetings["Meetings"]:::core SearchRes["Search & Research"]:::core RAG["RAG"]:::core Embeddings["Embeddings"]:::core Chat["Chat"]:::core CharChat["Character Chat"]:::core Notes["Notes"]:::core Persona["Persona"]:::core Research["Research"]:::core Chatbooks["Chatbooks"]:::core VNAssets["VN Assets"]:::core VNPlay["VN Play"]:::core PromptMgmt["Prompt Management"]:::core Evaluations["Evaluations"]:::core Personalization["Personalization"]:::core Moderation["Moderation"]:::core Monitoring["Monitoring"]:::core Workflows["Workflows"]:::core ChatWF["Chat Workflows"]:::core KanbanM["Notes Tasks / Kanban"]:::core SlidesM["Slides"]:::core ExplainerM["Explainer"]:::core AgentOrch["Agent Orchestration"]:::core TTS["TTS"]:::core CalendarM["Calendar"]:::core ACP["Agent Client Protocol"]:::core Sandbox["Sandbox"]:::core Infra["Infrastructure"]:::core Jobs["Jobs"]:::core Scheduler["Scheduler"]:::core ResourceGov["Resource Governance"]:::core subgraph PERUSER["Per-user SQLite (Databases/user_databases//)"] dbMedia[("Media content DB")]:::db dbChaCha[("ChaChaNotes.db")]:::db dbPrompts[("prompts_user_dbs/user_prompts_v2.sqlite")]:::db dbPromptStudio[("prompt_studio_dbs/prompt_studio.db")]:::db dbEval[("evaluations/evaluations.db")]:::db dbAuditUser[("audit/unified_audit.db")]:::db dbPers[("Personalization.db")]:::db dbGuardian[("Guardian.db")]:::db dbWF[("workflows/workflows.db")]:::db dbWFSched[("workflows/workflows_scheduler.db")]:::db dbResSess[("ResearchSessions.db")]:::db dbKanban[("Kanban.db")]:::db dbSlides[("Slides.db")]:::db dbExplainer[("Explainer.db")]:::db dbOrch[("orchestration.db")]:::db dbVoice[("voices/voice_registry.db")]:::db dbCalendar[("calendar.db")]:::db end subgraph VECTOR["Vector Store (per-user)"] dbVec[("chroma_storage/ — ChromaDB / pgvector")]:::db end subgraph SHARED["Shared / Global SQLite (Databases/)"] dbUsers[("users.db — AuthNZ")]:::db dbAuditShared[("audit_shared.db")]:::db dbJobs[("jobs.db")]:::db dbJobsAudit[("jobs_audit.db")]:::db dbSched[("scheduler.db")]:::db dbAcpAudit[("acp_audit.db")]:::db dbAcpSess[("acp_sessions.db")]:::db dbSandbox[("sandbox_store.db")]:::db dbCircuit[("circuit_breaker_registry.db")]:::db dbMonAlerts[("monitoring_alerts.db")]:::db dbAnalytics[("Analytics.db — RAG")]:::db end subgraph BACKENDS["Optional Backends"] dbRedis[("Redis")]:::ext dbPostgres[("PostgreSQL")]:::ext end %% ---- Per-user mappings ---- AuthNZ --> dbUsers Audit --> dbAuditUser Audit --> dbAuditShared Ingestion --> dbMedia Collections --> dbMedia Watchlists --> dbMedia Meetings --> dbMedia SearchRes --> dbMedia RAG --> dbMedia RAG --> dbVec RAG --> dbAnalytics Embeddings --> dbVec Chat --> dbChaCha CharChat --> dbChaCha Notes --> dbChaCha Persona --> dbChaCha Persona --> dbPers Research --> dbChaCha Research --> dbResSess Chatbooks --> dbChaCha Chatbooks --> dbPrompts Chatbooks --> dbExplainer Chatbooks --> dbJobs VNAssets --> dbChaCha VNPlay --> dbChaCha PromptMgmt --> dbPrompts PromptMgmt --> dbPromptStudio Evaluations --> dbEval Personalization --> dbPers Moderation --> dbGuardian Monitoring --> dbGuardian Monitoring --> dbMonAlerts Workflows --> dbWF Workflows --> dbWFSched ChatWF --> dbWF KanbanM --> dbKanban SlidesM --> dbSlides ExplainerM --> dbExplainer AgentOrch --> dbOrch TTS --> dbVoice CalendarM --> dbCalendar %% ---- Shared mappings ---- ACP --> dbAcpAudit ACP --> dbAcpSess Sandbox --> dbSandbox Infra --> dbCircuit Jobs --> dbJobs Jobs --> dbJobsAudit Scheduler --> dbSched %% ---- Optional backends ---- %% AuthNZ, Jobs, and the Scheduler can run on PostgreSQL instead of their SQLite files; %% the content DBs (Media/ChaCha) can too via TLDW_CONTENT_DB_BACKEND=postgresql. AuthNZ -.-> dbPostgres dbMedia -.-> dbPostgres dbChaCha -.-> dbPostgres Jobs -.-> dbPostgres Scheduler -.-> dbPostgres Jobs -.-> dbRedis ResourceGov -.-> dbRedis classDef core fill:#eefbea,stroke:#34a853,color:#1e4620; classDef db fill:#f0eaff,stroke:#8e6cf1,color:#3a2a87; classDef ext fill:#fff0f0,stroke:#e57373,color:#7b1f1f; ```
## Networking & Limits
HTTP client, egress, and rate-limit references - HTTP client and TLS/pinning configuration: `tldw_Server_API/Config_Files/README.md` (timeouts, retries, redirects/proxies, JSON limits, TLS min version, cert pinning, SSE/download helpers). - Egress/SSRF policy and security middleware: `tldw_Server_API/app/core/Security/README.md`. - Resource Governor (rate limits, tokens, streams; Redis backend optional): `tldw_Server_API/app/core/Resource_Governance/README.md`. ### API Rate Limits (High-Level) - **Characters & Character Chat** - Per-user caps (configurable via env/settings; defaults documented in `Docs/API-related/CHARACTER_CHAT_API_DOCUMENTATION.md`): - Operations per hour (character operations). - Per-user character limit. - Concurrent chats per user. - Messages per chat. - Per-minute limits for chat completions and message sends. - Status endpoint: - `GET /api/v1/characters/rate-limit-status` → returns a simple snapshot: - `operations_used`, `operations_remaining`, `reset_time` (Unix timestamp or `null`). - Enforcement is handled by `CharacterRateLimiter` with Redis-backed ZSETs when `REDIS_ENABLED=true`, or per-process in-memory counters otherwise. - **Core Chat / RAG / Embeddings** - Per-user RPM/TPM limits enforced via the Resource Governor (optional Redis backend). - See: - `tldw_Server_API/app/core/Chat/rate_limiter.py` - `tldw_Server_API/app/core/Resource_Governance/README.md` All limits are designed to be conservative by default and can be tuned using the various `*_RATE_LIMIT_*`, `MAX_*`, and RG policy settings in `Config_Files/` and environment variables.
## Running Tests
Running tests locally - `python -m pytest -v` - full test suite (skips heavy optional suites by default). - `python -m pytest --cov=tldw_Server_API --cov-report=term-missing` - coverage report. - Use markers (`unit`, `integration`, `e2e`, `external_api`, `performance`) to focus specific areas. - Enable optional suites with environment flags such as `RUN_MCP_TESTS=1`, `TLDW_TEST_POSTGRES_REQUIRED=1`, or `RUN_MOCK_OPENAI=1`. - `make tooling-smoke` - runs unified tooling smoke checks (`streaming_unified_smoke.py` + `watchlists_audio_smoke.py`) against a running local API (`http://127.0.0.1:8000` by default).
## Frontend Integration Testing
Frontend integration test helper Use the helper script to run frontend unit tests plus smoke checks against a live backend: ```bash cd apps/tldw-frontend npm run test:integration ``` Notes: - Starts the backend (uvicorn) by default and runs `pytest -m integration`, then `npm run test:run` + `npm run smoke`. - Set `TLDW_X_API_KEY=...` for single-user mode (a temporary key is generated if missing). - Use `--backend-docker` to start the backend via Docker Compose, or `--skip-backend` if you already have it running. - Use `--no-backend-tests` to skip backend integration tests.
## CI Status & Smoke Tests
CI status and smoke tests | Workflow | Status | | --- | --- | | E2E Critical Smoke (In-Process) | [![E2E Critical Smoke](https://github.com/rmusser01/tldw_server/actions/workflows/e2e-smoke.yml/badge.svg)](https://github.com/rmusser01/tldw_server/actions/workflows/e2e-smoke.yml) | Required gate policy and check-name contract: - `Docs/Development/CI_REQUIRED_GATES.md` Run locally - In-process (no open port): - `export E2E_INPROCESS=1 AUTH_MODE=single_user TEST_MODE=1` - `export SINGLE_USER_API_KEY=test-api-key-for-e2e-testing-12345` - `export SINGLE_USER_TEST_API_KEY=$SINGLE_USER_API_KEY` - `python -m pytest tldw_Server_API/tests/e2e/ --critical-only -q` - Live server (normal): - `python -m uvicorn tldw_Server_API.app.main:app --reload` - `export E2E_TEST_BASE_URL=http://localhost:8000` - `python -m pytest tldw_Server_API/tests/e2e/ --critical-only -q`
## Documentation & Resources
Documentation and resources **Getting Started Guides:** - [Public documentation site](https://tldwproject.com/server/docs/) - canonical docs hosted on tldwproject.com - [User Wiki](Docs/Wiki/User_Wiki.md) - install, run, configure, and use tldw_server - [Developer Wiki](Docs/Wiki/Developer_Wiki.md) - contribute to, test, package, and understand the codebase - [User Guides Documentation Map](Docs/User_Guides/index.md) - task-oriented map for setup, WebUI, extension, API, and admin workflows - [Getting Started Index](Docs/Getting_Started/README.md) - choose the right setup path - [Docker Single-User + WebUI Profile](Docs/Getting_Started/Profile_Docker_Single_User.md) - self-host with Docker and the WebUI - [Docker Multi-User + Postgres Profile](Docs/Getting_Started/Profile_Docker_Multi_User_Postgres.md) - team deployment baseline - [Local Single-User Profile](Docs/Getting_Started/Profile_Local_Single_User.md) - local API development path - [First-time audio setup: CPU systems](Docs/Getting_Started/First_Time_Audio_Setup_CPU.md) - local-first STT and TTS for CPU boxes - [First-time audio setup: GPU/accelerated systems](Docs/Getting_Started/First_Time_Audio_Setup_GPU_Accelerated.md) - NVIDIA and Apple Silicon speech setup - [GPU/STT Add-on](Docs/Getting_Started/GPU_STT_Addon.md) - legacy pointer to the accelerated guide **Reference:** - `Docs/Documentation.md` - documentation index and developer guide links - `Docs/About.md` - project background and philosophy - Module deep dives: `Docs/Code_Documentation/AuthNZ-Developer-Guide.md`, `Docs/Code_Documentation/RAG-Developer-Guide.md`, `Docs/MCP/Unified/Developer_Guide.md` - Packaging and releases: `Docs/Development/PyPI_Publishing.md` - Distribution strategy (API + WebUI): `Docs/Development/Packaging_and_Distribution_Strategy.md` - API references: `Docs/API-related/RAG-API-Guide.md`, `Docs/API-related/OCR_API_Documentation.md`, `Docs/API-related/Prompt_Studio_API.md` - Deployment/Monitoring: `Docs/Published/Deployment/First_Time_Production_Setup.md`, `Docs/Published/Deployment/Reverse_Proxy_Examples.md`, `Docs/Deployment/Monitoring/` - Speech quickstart (STT + TTS): `Docs/User_Guides/WebUI_Extension/Getting-Started-STT_and_TTS.md` - TTS provider onboarding: `Docs/User_Guides/WebUI_Extension/TTS_Getting_Started.md` - TTS deep runbooks index: `Docs/User_Guides/WebUI_Extension/TTS-SETUP-GUIDE.md` - Design notes (WIP features): `Docs/Design/` - e.g., `Docs/Design/Audio_Pipeline.md` ### Resource Governor Config For complete Resource Governor setup and examples (env, DB store bootstrap, YAML policy, middleware, diagnostics, and tests), see `tldw_Server_API/app/core/Resource_Governance/README.md`. ### OpenAI-Compatible Strict Mode (Local Providers) Some self-hosted OpenAI-compatible servers reject unknown fields (like `top_k`). For local providers you can enable a strict mode that filters non-standard keys from chat payloads. - Set `strict_openai_compat: true` in the relevant provider section (`local_llm`, `llama_api`, `ooba_api`, `tabby_api`, `vllm_api`, `aphrodite_api`, `ollama_api`). - For `local_llm`, you can also use `LOCAL_LLM_STRICT_OPENAI_COMPAT=1`. - When enabled, only standard OpenAI Chat Completions parameters are sent: `messages, model, temperature, top_p, max_tokens, n, stop, presence_penalty, frequency_penalty, logit_bias, seed, response_format, tools, tool_choice, logprobs, top_logprobs, user, stream`. ### Chatbook Tools Guide - User guide: `Docs/User_Guides/WebUI_Extension/Chatbook_User_Guide.md` covers Chatbook backup/restore, OpenWebUI "Export Chats" JSON import, OpenWebUI `webui.db` database import, and post-import OpenWebUI attachment hydration for restoring referenced images/files from a server-local data root. - Getting started: `Docs/User_Guides/WebUI_Extension/Chatbook_Tools_Getting_Started.md` - Product spec (PRD): `Docs/Product/Completed/Chatbook-Tools-PRD.md` - API reference: `Docs/API-related/Chatbook_API_Documentation.md` - Related endpoints (also listed above under Key Endpoints): - `GET /api/v1/chat/commands` — list slash commands (RBAC-filtered when enabled; returns empty list when disabled) - `POST /api/v1/chat/dictionaries/validate` — validate chat dictionaries (schema, regex, templates)
## Deployment
Deployment resources - Dockerfiles and compose templates live under `Dockerfiles/` (see `Dockerfiles/README.md`). - Reverse proxy samples: `Helper_Scripts/Samples/Nginx/`, `Helper_Scripts/Samples/Caddy/`. - Monitoring: `Docs/Deployment/Monitoring/` and `Helper_Scripts/Samples/Grafana/`. - Prometheus metrics exposed at `/metrics` and `/api/v1/metrics`. - Production hardening: `Docs/Published/User_Guides/Server/Production_Hardening_Checklist.md`.
## Monitoring
Monitoring resources - Monitoring docs and setup: `Docs/Deployment/Monitoring/README.md` - Grafana dashboards and samples: `Helper_Scripts/Samples/Grafana/README.md` - Prometheus scrape endpoints: `GET /metrics` and `GET /api/v1/mcp/metrics/prometheus` (both require authenticated principals with appropriate permissions; MCP Prometheus uses `system.logs`) ### PostgreSQL Content Mode - Content DBs (Media, ChaChaNotes, Workflows) can run on Postgres. - See: `Docs/Published/User_Guides/Server/Multi-User_Postgres_Setup.md`, `Docs/Published/Deployment/Postgres_Migration_Guide.md`, and `Docs/Published/User_Guides/Server/Backups_Using_Litestream.md`.
## Troubleshooting
Common setup and runtime fixes | Issue | Solution | |-------|----------| | `Connection refused` | Server not running - check `uvicorn` or `docker compose ps` | | `401 Unauthorized` | API key mismatch - verify `.env` matches your `X-API-KEY` header | | `API key too short` | Key must be at least 16 characters | | `ffmpeg not found` | Install: `brew install ffmpeg` (macOS) or `apt install ffmpeg` (Linux) | | `ModuleNotFoundError` | Activate venv: `source .venv/bin/activate` | | Torch/CUDA mismatch | Install CUDA-compatible PyTorch or use CPU wheels | | SQLite locks | Use short-lived transactions; consider Postgres for concurrency | | Docker issues | Check `docker compose ps` and `docker compose logs -f` | Quick verify: `curl http://localhost:8000/health` (no auth needed)
## Contributing & Support - Read `CONTRIBUTING.md`, `Project_Guidelines.md`, and `AGENTS.md` before submitting changes. - File bugs or feature requests via GitHub Issues; longer-form discussions live in GitHub Discussions. - Respect the project philosophy: incremental progress, clear intent, and kindness toward contributors. --- ## Developer Guides
Developer Guides + Links - Documentation index: `Docs/Documentation.md` (see the "Developer Guides" section) - Core module guides: - Chat Module: `Docs/Code_Documentation/Chat_Developer_Guide.md` - Chunking Module: `Docs/Code_Documentation/Chunking-Module.md` - RAG Module: `tldw_Server_API/app/core/RAG/README.md` and `tldw_Server_API/app/core/RAG/API_DOCUMENTATION.md` ### Ingestion & Media Processing Docs - Overview: `Docs/Code_Documentation/Ingestion_Media_Processing.md` - Pipelines: - Audio: `Docs/Code_Documentation/Ingestion_Pipeline_Audio.md` - Video: `Docs/Code_Documentation/Ingestion_Pipeline_Video.md` - PDF: `Docs/Code_Documentation/Ingestion_Pipeline_PDF.md` - EPUB: `Docs/Code_Documentation/Ingestion_Pipeline_Ebooks.md` - Documents: `Docs/Code_Documentation/Ingestion_Pipeline_Documents.md` - MediaWiki: `Docs/Code_Documentation/Ingestion_Pipeline_MediaWiki.md`
## About
Project background and support links tldw_server started as a tool to transcribe and summarize YouTube videos and has evolved into a comprehensive media analysis and knowledge management platform for researchers, students, and professionals. For extended background and vision, see `Docs/About.md`. ### Getting Help - API Documentation: `http://localhost:8000/docs` - GitHub Issues: [Report bugs or request features](https://github.com/rmusser01/tldw_server/issues) - Discussions: [Community forum](https://github.com/rmusser01/tldw_server/discussions) ### Security Disclosures See `SECURITY.md` for reporting guidelines and disclosures. ### Project Guidelines See [Project_Guidelines.md](Project_Guidelines.md) for development philosophy and contribution guidelines.
## Appendix (Optional)
Background and reading list Optional background reading for deeper context; not required to use the project. - See `Docs/About.md` for the extended project background, vision, and notes. - https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5049562 - Purpose of this section is to help bring awareness to certain concepts and terms that are used in the field of AI/ML/NLP, as well as to provide some resources for learning more about them. - Also because some of those things are extremely relevant and important to know if you care about accuracy and the effectiveness of the LLMs you're using. - Some of this stuff may be 101 level, but I'm going to include it anyways. This repo is aimed at people from a lot of different fields, so I want to make sure everyone can understand what's going on. Or at least has an idea. - LLMs 101(coming from a tech background): https://vinija.ai/models/LLM/ - LLM Fundamentals / LLM Scientist / LLM Engineer courses(Free): https://github.com/mlabonne/llm-course - **Phrases & Terms** - **LLM** - Large Language Model - A type of neural network that can generate human-like text. - **API** - Application Programming Interface - A set of rules and protocols that allows one software application to communicate with another. - **API Wrapper** - A set of functions that provide a simplified interface to a larger body of code. - **API Key** - A unique identifier that is used to authenticate a user, developer, or calling program to an API. - **GUI** - Graphical User Interface - **CLI** - Command Line Interface - **DB** - Database - **SQLite** - A C-language library that implements a small, fast, self-contained, high-reliability, full-featured, SQL database engine. - **Prompt Engineering** - The process of designing prompts that are used to guide the output of a language model. - **Quantization** - The process of converting a continuous range of values into a finite range of discrete values. - **GGUF Files** - GGUF is a binary format that is designed for fast loading and saving of models, and for ease of reading. Models are traditionally developed using PyTorch or another framework, and then converted to GGUF for use in GGML. https://github.com/ggerganov/ggml/blob/master/docs/gguf.md - **Inference Engine** - A software system that is designed to execute a model that has been trained by a machine learning algorithm. Llama.cpp and Kobold.cpp are examples of inference engines. - **Papers & Concepts** 1. Lost in the Middle: How Language Models Use Long Contexts(2023) - https://arxiv.org/abs/2307.03172 - `We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models` 2. [Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models(2024)](https://arxiv.org/abs/2402.14848) - `Our findings show a notable degradation in LLMs' reasoning performance at much shorter input lengths than their technical maximum. We show that the degradation trend appears in every version of our dataset, although at different intensities. Additionally, our study reveals that the traditional metric of next word prediction correlates negatively with performance of LLMs' on our reasoning dataset. We analyse our results and identify failure modes that can serve as useful guides for future research, potentially informing strategies to address the limitations observed in LLMs.` 3. Why Does the Effective Context Length of LLMs Fall Short?(2024) - https://arxiv.org/abs/2410.18745 - ` Advancements in distributed training and efficient attention mechanisms have significantly expanded the context window sizes of large language models (LLMs). However, recent work reveals that the effective context lengths of open-source LLMs often fall short, typically not exceeding half of their training lengths. In this work, we attribute this limitation to the left-skewed frequency distribution of relative positions formed in LLMs pretraining and post-training stages, which impedes their ability to effectively gather distant information. To address this challenge, we introduce ShifTed Rotray position embeddING (STRING). STRING shifts well-trained positions to overwrite the original ineffective positions during inference, enhancing performance within their existing training lengths. Experimental results show that without additional training, STRING dramatically improves the performance of the latest large-scale models, such as Llama3.1 70B and Qwen2 72B, by over 10 points on popular long-context benchmarks RULER and InfiniteBench, establishing new state-of-the-art results for open-source LLMs. Compared to commercial models, Llama 3.1 70B with \method even achieves better performance than GPT-4-128K and clearly surpasses Claude 2 and Kimi-chat.` 4. [RULER: What's the Real Context Size of Your Long-Context Language Models?(2024)](https://arxiv.org/abs/2404.06654) - `The needle-in-a-haystack (NIAH) test, which examines the ability to retrieve a piece of information (the "needle") from long distractor texts (the "haystack"), has been widely adopted to evaluate long-context language models (LMs). However, this simple retrieval-based test is indicative of only a superficial form of long-context understanding. To provide a more comprehensive evaluation of long-context LMs, we create a new synthetic benchmark RULER with flexible configurations for customized sequence length and task complexity. RULER expands upon the vanilla NIAH test to encompass variations with diverse types and quantities of needles. Moreover, RULER introduces new task categories multi-hop tracing and aggregation to test behaviors beyond searching from context. We evaluate ten long-context LMs with 13 representative tasks in RULER. Despite achieving nearly perfect accuracy in the vanilla NIAH test, all models exhibit large performance drops as the context length increases. While these models all claim context sizes of 32K tokens or greater, only four models (GPT-4, Command-R, Yi-34B, and Mixtral) can maintain satisfactory performance at the length of 32K. Our analysis of Yi-34B, which supports context length of 200K, reveals large room for improvement as we increase input length and task complexity.` 5. Abliteration (Uncensoring LLMs) - [Uncensor any LLM with abliteration - Maxime Labonne(2024)](https://huggingface.co/blog/mlabonne/abliteration) 6. Retrieval-Augmented-Generation - [Retrieval-Augmented Generation for Large Language Models: A Survey](https://arxiv.org/abs/2312.10997) - https://arxiv.org/abs/2312.10997 - `Retrieval-Augmented Generation (RAG) has emerged as a promising solution by incorporating knowledge from external databases. This enhances the accuracy and credibility of the generation, particularly for knowledge-intensive tasks, and allows for continuous knowledge updates and integration of domain-specific information. RAG synergistically merges LLMs' intrinsic knowledge with the vast, dynamic repositories of external databases. This comprehensive review paper offers a detailed examination of the progression of RAG paradigms, encompassing the Naive RAG, the Advanced RAG, and the Modular RAG. It meticulously scrutinizes the tripartite foundation of RAG frameworks, which includes the retrieval, the generation and the augmentation techniques. The paper highlights the state-of-the-art technologies embedded in each of these critical components, providing a profound understanding of the advancements in RAG systems. Furthermore, this paper introduces up-to-date evaluation framework and benchmark. At the end, this article delineates the challenges currently faced and points out prospective avenues for research and development. ` 7. Prompt Engineering - Prompt Engineering Guide: https://www.promptingguide.ai/ & https://github.com/dair-ai/Prompt-Engineering-Guide - 'The Prompt Report' - https://arxiv.org/abs/2406.06608 8. Bias and Fairness in LLMs - [ChatGPT Doesn't Trust Chargers Fans: Guardrail Sensitivity in Context](https://arxiv.org/abs/2407.06866) - `While the biases of language models in production are extensively documented, the biases of their guardrails have been neglected. This paper studies how contextual information about the user influences the likelihood of an LLM to refuse to execute a request. By generating user biographies that offer ideological and demographic information, we find a number of biases in guardrail sensitivity on GPT-3.5. Younger, female, and Asian-American personas are more likely to trigger a refusal guardrail when requesting censored or illegal information. Guardrails are also sycophantic, refusing to comply with requests for a political position the user is likely to disagree with. We find that certain identity groups and seemingly innocuous information, e.g., sports fandom, can elicit changes in guardrail sensitivity similar to direct statements of political ideology. For each demographic category and even for American football team fandom, we find that ChatGPT appears to infer a likely political ideology and modify guardrail behavior accordingly.` - **Tools & Libraries** 1. `llama.cpp` - A C++ inference engine. Highly recommend. * https://github.com/ggerganov/llama.cpp 2. `kobold.cpp` - A C++ inference engine. GUI wrapper of llama.cpp with some tweaks. * https://github.com/LostRuins/koboldcpp 3. `sillytavern` - A web-based interface for text generation models. Supports inference engines. Ignore the cat girls and weebness. This software is _powerful_ and _useful_. Also supports just about every API you could want. * https://github.com/SillyTavern/SillyTavern 4. `llamafile` - A wrapper for llama.cpp that allows for easy use of local LLMs. * Uses libcosomopolitan for cross-platform compatibility. * Can be used to run LLMs on Windows, Linux, and MacOS with a single binary wrapper around Llama.cpp. 5. `pytorch` - An open-source machine learning library based on the Torch library. 6. `ffmpeg` - A free software project consisting of a large suite of libraries and programs for handling video, audio, and other multimedia files and streams. 7. `pandoc` - A free and open-source document converter, widely used as a writing tool (especially by scholars) and as a basis for publishing workflows. * https://pandoc.org/ 8. `marker` - A tool for converting PDFs(and other document types) to markdown. * https://github.com/VikParuchuri/marker 9. `faster_whisper` - A fast, lightweight, and accurate speech-to-text model. * https://github.com/SYSTRAN/faster-whisper
Local model recommendations (opinionated) Personal recommendations; optional reading. - These are just the 'standard smaller' models I recommend, there are many more out there, and you can use any of them with this project. - One should also be aware that people create 'fine-tunes' and 'merges' of existing models, to create new models that are more suited to their needs. - This can result in models that may be better at some tasks but worse at others, so it's important to test and see what works best for you. - Mistral Nemo Instruct 2407 - https://huggingface.co/QuantFactory/Mistral-Nemo-Instruct-2407-GGUF - Magistral Small: https://huggingface.co/mistralai/Magistral-Small-2509-GGUF - Qwen 3/VL Series - Qwen/Qwen3-VL-4B-Instruct (Qwen3 4B+Vision): https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct-GGUF - Qwen3-30B-A3B-Instruct-2507: https://huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF For commercial API usage for use with this project: GPT-5.1, Anthropic's models(The Temu of AI services/models), Kimi k2, DeepSeek Originally written: Flipside I would say none, honestly. The (largest players) will gaslight you and charge you money for it. Fun. That being said they obviously can provide help/be useful(helped me make this app), but it's important to remember that they're not your friend, and they're not there to help you. They are there to make money not off you, but off large institutions and your data. You are just a stepping stone to their goals. 2025 Nov: I would say service quality has improved enough to the point where it can make sense to use a 'premium' subscription/usage of API services without expecting to be screwed 7-8/10 times. 2025 Dec: I spoke too soon. I have the opinion of GPT-5.1 High > Sonnet/Opus. The Anthropic models are just too 'independent/lazy'. I personally have never had gpt5 fill out a PRD with completely hallucinated bullshit, or gaslight me repeatedly across multiple different sessions. Sonnet/Opus 4.0-4.5? Complete opposite. From @nrose 05/08/2024 on Threads: ``` No, it’s a design. First they train it, then they optimize it. Optimize it for what- better answers? No. For efficiency. Per watt. Because they need all the compute they can get to train the next model.So it’s a sawtooth. The model declines over time, then the optimization makes it somewhat better, then in a sort of reverse asymptote, they dedicate all their “good compute” to the next bigger model.Which they then trim down over time, so they can train the next big model… etc etc. None of these companies exist to provide AI services in 2024. They’re only doing it to finance the things they want to build in 2025 and 2026 and so on, and the goal is to obsolete computing in general and become a hidden monopoly like the oil and electric companies. 2024 service quality is not a metric they want to optimize, they’re forced to, only to maintain some directional income ```
Quotes - `I like the lies-to-children motif, because it underlies the way we run our society and resonates nicely with Discworld. Like the reason for Unseen being a storehouse of knowledge - you arrive knowing everything and leave realising that you know practically nothing, therefore all the knowledge you had must be stored in the university. But it's like that in "real Science", too. You arrive with your sparkling A-levels all agleam, and the first job of the tutors is to reveal that what you thought was true is only true for a given value of "truth". Most of us need just "enough" knowledge of the sciences, and it's delivered to us in metaphors and analogies that bite us in the bum if we think they're the same as the truth.` * Terry Pratchett - `The first principle is that you must not fool yourself - and you are the easiest person to fool.` * Richard Feynman
--- ## License - Server: GPL-3.0-only - Frontend: source-available under PolyForm Perimeter 1.0.1 - OpenAPI contract: Apache-2.0 See the root [LICENSE](LICENSE) for the authoritative scope map and detailed terms. --- ## Credits - [The original version of tldw by @the-crypt-keeper](https://github.com/the-crypt-keeper/tldw/tree/main/tldw-original-scripts) - [yt-dlp](https://github.com/yt-dlp/yt-dlp) - [ffmpeg](https://github.com/FFmpeg/FFmpeg) - [faster_whisper](https://github.com/SYSTRAN/faster-whisper) - [pyannote](https://github.com/pyannote/pyannote-audio) - Thank you cognitivetech for the summarization system prompt: https://github.com/cognitivetech/llm-long-text-summarization/tree/main?tab=readme-ov-file#one-shot-prompting - [Fabric](https://github.com/danielmiessler/fabric) - [Llamafile](https://github.com/Mozilla-Ocho/llamafile) - For the local LLM inference engine - [Mikupad](https://github.com/lmg-anon/mikupad) - Because I'm not going to write a whole new frontend for non-chat writing. - The people who have helped me get to this point(SC & CS), and especially for those not around to see it(DT & CC).