site_name: DeltaLLM Documentation site_description: Open-source LLM gateway with unified API, RBAC, guardrails, and observability # Preserve the language/version path used by Read the Docs in canonical URLs and the sitemap. site_url: !ENV [READTHEDOCS_CANONICAL_URL, 'https://docs.deltallm.io/en/latest/'] repo_url: https://github.com/deltawi/deltallm repo_name: deltallm # Internal plans live beside the public sources for repository-local review, but # they must never be rendered, indexed, or included in the public sitemap. exclude_docs: | internal/ # Shared snippets are pulled into pages by pymdownx.snippets, not published on their own. includes/ theme: name: material # Reuse the UI's built-in assets so the product and docs have one brand source. custom_dir: ui/public logo: brand/deltallm-delta.svg favicon: favicon.svg palette: - scheme: default primary: custom accent: custom toggle: icon: material/brightness-7 name: Switch to dark mode - scheme: slate primary: custom accent: custom toggle: icon: material/brightness-4 name: Switch to light mode features: - navigation.sections - navigation.indexes - navigation.path - navigation.instant - navigation.instant.progress - navigation.tracking - navigation.top - navigation.footer - toc.follow - search.suggest - search.highlight - search.share - content.code.copy - content.tabs.link markdown_extensions: - admonition - pymdownx.details - pymdownx.superfences - pymdownx.tabbed: alternate_style: true - pymdownx.highlight: anchor_linenums: true - pymdownx.inlinehilite - pymdownx.snippets: base_path: - docs/includes check_paths: true - tables - toc: permalink: true - attr_list - def_list extra_css: - stylesheets/brand.css nav: - Home: index.md - Get started: - getting-started/index.md - Start with Docker: getting-started/docker.md - Check or add your first model: getting-started/first-model.md - Send your first request: getting-started/quickstart.md - Create an application key: getting-started/first-api-key.md - Set up a development environment: getting-started/installation.md - Video tutorials: tutorials/index.md - Build with DeltaLLM: - features/index.md - Models and providers: - Choose and connect a model: guides/models-and-providers.md - Routing and reliability: - Route traffic and handle failures: guides/routing-and-failover.md - Cache repeated requests: features/caching.md - Process work in batches: guides/batching.md - Realtime audio over WebSockets: guides/realtime.md - Tools and safety: - Connect your first MCP server: getting-started/mcp-quickstart.md - Add your first guardrail: guides/guardrails.md - Build complete workflows: guides/build-workflows.md - Use coding agents: guides/coding-agents.md - Administer DeltaLLM: - admin-ui/index.md - Start administering: - Sign in and get access: admin-ui/access-requirements.md - Configure the gateway: - Add and manage models: guides/admin-models.md - Reuse provider credentials: guides/admin-provider-credentials.md - Manage route groups: guides/admin-route-groups.md - Prompts: admin-ui/prompt-registry.md - MCP servers, bundles and approvals: admin-ui/mcp.md - Connected accounts: admin-ui/connected-accounts.md - Test a model: admin-ui/playground.md - Manage organizations and access: - Organizations: admin-ui/organizations.md - Teams: admin-ui/teams.md - People and permissions: admin-ui/people-and-access.md - Application keys: admin-ui/api-keys.md - Coding agents: admin-ui/coding-agents.md - Plans and tiers: guides/admin-tiers.md - Set up sign-in and SSO: guides/admin-authentication.md - External customer sign-in: configuration/external-customer-auth.md - Connect a Console: guides/console-gateway-connection.md - Monitor and control: - Dashboard: admin-ui/dashboard.md - Usage and spending: admin-ui/usage.md - Set budgets: guides/admin-budgets.md - Set rate limits: guides/admin-rate-limits.md - Guardrails: admin-ui/guardrails.md - Audit logs: admin-ui/audit-logs.md - Run operations: - Administration workflows: guides/admin-workflows.md - Batch jobs: admin-ui/batch-jobs.md - Settings: admin-ui/settings.md - Remove an organization: features/organization-deletion.md - Deploy and operate: - deployment/index.md - Choose where to run DeltaLLM: - Railway: deployment/railway.md - Docker: deployment/docker.md - Kubernetes: guides/kubernetes-deployment.md - Prepare for production: - Choose what to turn on: deployment/feature-switches.md - Production checklist: deployment/production-checklist.md - Size production for RPS targets: deployment/production-rps-requirements.md - Secure the deployment: security/hardening.md - Tune provider connections: guides/provider-connections.md - Monitor the system: guides/monitoring.md - Maintain the system: - Upgrade or roll back: guides/deployment-workflow.md - Move from LiteLLM: guides/migrate-from-litellm.md - Check a deployment: deployment/live-smoke-suite.md - Update the database: deployment/database-migrations.md - Recover the v0.1.48 model migration: deployment/model-api-identity-migration-recovery.md - Back up and restore data: deployment/backup-and-restore.md - Respond to incidents: deployment/incident-runbooks.md - Understand the system: - concepts/index.md - Architecture: concepts/architecture.md - What happens to a request: concepts/request-lifecycle.md - Accounts, teams, and access: concepts/tenancy-and-access.md - Reference: - reference/index.md - API: - Overview: api/index.md - Common API behavior: api/conventions.md - OpenAPI file: api/openapi.md - Model and file endpoints: api/proxy.md - OpenAI SDK compatibility: api/openai-sdk-compatibility.md - OpenAI Responses: api/responses.md - Anthropic Messages: api/messages.md - MCP endpoints: api/mcp.md - Administration endpoints: api/admin.md - Health and metrics: api/health.md - Configuration: - Overview: configuration/index.md - Model deployments: configuration/models.md - Request shaping: configuration/request-shaping.md - Routing settings: configuration/router.md - Model router administration: configuration/model-router-admin.md - General settings guide: configuration/general.md - All settings: configuration/general-settings-reference.md - Providers: - Overview: providers/index.md - Models and features: providers/capabilities.md - Compatible chat providers: providers/compatible-chat.md - Mistral AI: providers/mistral.md - SGLang: providers/sglang.md - OpenAI-compatible: providers/openai-compatible.md - Feature details: - Routing and failover: features/routing.md - Batch processing: features/batching.md - Caching: reference/caching.md - MCP tools: features/mcp.md - Guardrails: features/guardrails.md - Administration details: - Access and permission matrix: reference/admin-access.md - Model administration: admin-ui/models.md - Plans and tiers: admin-ui/tiers.md - Provider credentials: admin-ui/named-credentials.md - Route-group administration: admin-ui/route-groups.md - Authentication and SSO: features/authentication.md - Supplied API key design: design/supplied-api-keys.md - Budgets and spending: features/budgets.md - Rate limiting: features/rate-limiting.md - Audit records: features/audit-log.md - Operations details: - Kubernetes and Helm: deployment/kubernetes.md - Metrics and callbacks: features/observability.md - Alert destinations: features/alerting.md - Measure gateway concurrency: deployment/concurrency-measurement.md - Gateway ingress admission: deployment/ingress-admission.md - Dependency capacity: deployment/dependency-capacity.md - Database pooler (PgBouncer): deployment/database-pooler.md - Concurrency runtime design: design/concurrency-runtime.md - Process lifecycle: deployment/process-lifecycle.md - Saturation autoscaling: deployment/saturation-autoscaling.md - Capacity profiles: deployment/capacity-profiles.md - Budget and prompt performance: deployment/budget-prompt-performance.md - Spend recovery: deployment/spend-recovery.md - Accounting protocol v2: deployment/accounting-v2.md - Accounting v2 acceptance and soak: deployment/accounting-v2-acceptance.md - Accounting protocol v2 design: design/accounting-protocol-v2.md - Bounded in-memory state: deployment/bounded-state.md - Request deadlines: deployment/request-deadlines.md - Provider connection tuning: deployment/upstream-http.md - Provider connection pool design: design/upstream-http-pool.md - Selector session affinity design: design/selector-session-affinity.md - Upgrade and rollback behavior: deployment/upgrade-and-rollback.md - Release-specific rollouts: deployment/rollout-index.md - Governance migration: deployment/governance.md - Organization tiers rollout: deployment/organization-tiers-rollout.md - Batch scheduler rollout: deployment/batch-scheduler-rollout.md - Batch webhook rollout: deployment/batch-webhook-rollout.md - Router state cutover: deployment/router-state-schema-cutover.md - Telemetry ingestion rollout: deployment/telemetry-ingestion-rollout.md - Usage reporting rollout: deployment/usage-reporting-v2.md - Performance evidence: - RPS results: project/issue-320-rps-report.md - Reproduce RPS tests: project/issue-320-rps-reproduction.md - Concrete concurrency measures: project/issue-320-50-to-500-rps-measures-and-integration.md - Policies and support: - Security model: security/index.md - Report a security problem: security/vulnerability-reporting.md - Verify release images: security/release-verification.md - Versions and compatibility: project/versioning-and-compatibility.md - Documentation policy: project/documentation-governance.md extra: social: - icon: fontawesome/brands/github link: https://github.com/deltawi/deltallm - icon: fontawesome/brands/youtube link: https://www.youtube.com/@deltallm name: DeltaLLM on YouTube