--- name: config-architecture description: LiteLLM-RS Configuration Architecture. Covers YAML loading with ${VAR} environment substitution, strict serde config models, startup validation via the Validate trait, and the LITELLM_* env-only fallback. Use when changing config models or serde defaults, debugging env variable substitution failures, updating validation rules, altering config loading behavior, or troubleshooting gateway.yaml. --- # Configuration Architecture Guide ## Overview LiteLLM-RS loads a single YAML file into type-safe Rust models (`GatewayConfig` in `src/config/models/gateway.rs` and its children), substitutes `${VAR}` / `$VAR` tokens from the process environment during load, fills missing fields from serde defaults, then validates the whole tree once at startup. There is **no hot reload**. Configuration is read once at startup; `src/utils/config/optimized.rs` states this explicitly ("Hot reload is not supported"). Restart the process to pick up changes. `AppState.config` (`src/server/state.rs`) is atomically swappable by explicit code paths, but nothing watches the file. ### Load Order ``` ┌─────────────────────────────────────────────────────────────────┐ │ 1. YAML file │ │ --config PATH (default: "config/gateway.yaml", src/main.rs) │ └─────────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────┐ │ 2. Environment substitution (substitute_env_vars) │ │ ${VAR} = required (hard error if unset) │ │ $VAR = best-effort (left literal if unset) │ └─────────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────┐ │ 3. serde parse + defaults │ │ #[serde(default ...)] fills omitted fields; │ │ deny_unknown_fields rejects keys on structs that opt in │ └─────────────────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────┐ │ 4. Validation (Validate trait) │ │ runs inside Config::from_file / Config::from_env │ │ failure aborts startup │ └─────────────────────────────────────────────────────────────────┘ ``` After loading, explicit `--host` / `--port` CLI values overwrite `server.host` / `server.port`, then the config is re-validated (`load_config_with_overrides` in `src/main.rs`). Fallback rule: when the gateway starts **without** an explicit `--config` and the default file fails to load, `run_server_with_default_config_overrides` retries with `Config::from_env()`, which builds the config from `LITELLM_*` environment variables only (`src/config/models/gateway.rs::GatewayConfig::from_env`). An explicit `--config` path that fails is a hard error — no env fallback (`load_explicit_config` in `src/server/builder.rs`). There is no search-path chain: no `/etc/litellm/...`, no `gateway.yaml` in cwd. The only default is the single constant `DEFAULT_CONFIG_PATH = "config/gateway.yaml"` relative to the process working directory (`src/main.rs`). --- ## YAML Configuration Structure The canonical example is `config/gateway.yaml.example`; unit tests assert it parses against the current schema. Top-level sections of `GatewayConfig`: ```yaml schema_version: "1.0" # only "1.0" is accepted by validation server: # host/port/workers/timeout/max_body_size/tls/cors/features/... providers: # LIST of provider entries, not a map - name: "openai-primary" provider_type: "openai" api_key: "${OPENAI_API_KEY}" base_url: "https://api.openai.com/v1" models: ["gpt-4o"] priority: 0 # lower wins under priority_based routing model_aliases: # public alias -> canonical model or another alias production-chat: "gpt-4o" router: # strategy, circuit_breaker, load_balancer storage: # database, redis, vector_db auth: # enable_jwt, enable_api_key, jwt_secret, rbac, allow_anonymous monitoring: # metrics, tracing, health, callbacks cache: # deterministic response cache rate_limit: # token bucket strategy, redis_failure_mode guardrails: # content safety; prompt-injection protection on by default ip_access: # allowlist/blocklist; empty/default rules allow all enterprise: # sso, audit_logging pricing: # source, unpriced_model_policy ``` Unknown top-level fields are rejected at parse time (`#[serde(deny_unknown_fields)]` on `GatewayConfig`), as are unknown keys in the nested YAML models that carry the same attribute. This is not a universal serde guarantee: `providers[].settings` is intentionally open-ended, and some provider/callback backend payload structs do not deny unknown fields. There are no top-level `logging`, `routing`, or `observability` sections — logging lives under `monitoring.logging`, routing under `router`. --- ## Type-Safe Configuration Models ### Root Configuration `src/config/models/gateway.rs`: ```rust #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(deny_unknown_fields)] pub struct GatewayConfig { pub schema_version: String, // default "1.0" pub server: ServerConfig, pub providers: Vec, // Vec, not a map pub model_aliases: HashMap, // default empty pub router: GatewayRouterConfig, pub storage: StorageConfig, pub auth: AuthConfig, pub monitoring: MonitoringConfig, pub cache: CacheConfig, // default pub rate_limit: RateLimitConfig, // default pub guardrails: GuardrailConfig, // default = enabled pub ip_access: IpAccessConfig, // default pub enterprise: EnterpriseConfig, // default pub pricing: GatewayPricingConfig, // default } ``` ### Server Configuration `src/config/models/server.rs`: ```rust #[serde(deny_unknown_fields)] pub struct ServerConfig { pub host: String, // default "0.0.0.0" pub port: u16, // default 8000 pub workers: Option, // None -> CPU count at runtime pub max_connections: Option, pub timeout: u64, // seconds, default 30 pub max_body_size: usize, // bytes, default 10 MiB pub dev_mode: bool, // default false pub tls: Option, pub cors: CorsConfig, pub features: Vec, pub trusted_proxies: Vec, // X-Forwarded-For trust list pub stream_idle_timeout: u64, // SSE idle timeout, default 300s; 0 disables } ``` Shared defaults live in `src/config/models/mod.rs`: `default_port()` returns `8000`, `default_timeout()` returns `30`, `default_max_body_size()` returns `10 * 1024 * 1024`. There is no `keep_alive`, `request_timeout`, or string-form `max_request_size` field. ### Provider Configuration `src/config/models/provider.rs` — one entry per deployment, stored as `Vec`: ```rust #[serde(deny_unknown_fields)] pub struct ProviderConfig { pub name: String, // must be unique across entries pub provider_type: String, // e.g. "openai", "anthropic" pub api_key: String, // required unless registry skips key checks pub base_url: Option, pub endpoint_access: ProviderEndpointAccess, // default PublicOnly; "private_network" opt-in pub api_version: Option, pub organization: Option, pub project: Option, pub weight: f32, // default 1.0, valid range (0, 100] pub priority: u32, // default 0; lower wins in priority_based pub rpm: u32, // default 1000 pub tpm: u32, // default 100_000 pub max_concurrent_requests: u32, // default 10 pub timeout: u64, // seconds, default 30, max 300 pub max_retries: u32, // default 3 pub retry: RetryConfig, // base_delay/max_delay/backoff_multiplier/jitter pub health_check: ProviderHealthCheckConfig, pub settings: HashMap, // provider-specific keys pub models: Vec, pub tags: Vec, pub enabled: bool, // default true } ``` This is the YAML-facing model. The separate `ProviderConfig` *trait* that provider implementations satisfy lives in `src/core/traits/provider/config.rs` — see [reference/provider-config-errors.md](reference/provider-config-errors.md). --- ## Environment Variable Substitution `substitute_env_vars` in `src/config/mod.rs` processes the raw file text before parsing: - `${VAR_NAME}` — substituted with the env value. If unset, loading **fails**: all missing braced variables are collected, deduplicated, sorted, and reported in one error (`Missing environment variables referenced by config: A, B`). Placeholders can never silently reach runtime as literal strings. - `$VAR_NAME` — shell-style bare form, also substituted. If unset, the token is **left literal** so ordinary dollar-containing values pass through untouched. - `${VAR:-default}` is **not supported** and is silently dangerous: neither recognized pattern matches a token containing `:-`, so the whole string stays literal in the parsed value — no substitution and no error. - Substitution is line-based and quote/comment aware: text after an unquoted `#` (a YAML comment) is never substituted, and values pulled from the environment are not re-expanded. - Errors surface as `GatewayError::Config` (the runtime loader does not use `ConfigError`; see [reference/provider-config-errors.md](reference/provider-config-errors.md)). ```yaml # Works api_key: "${OPENAI_API_KEY}" # fails startup if OPENAI_API_KEY is unset url: "$REDIS_URL" # left literal if REDIS_URL is unset # Does NOT work - no fallback syntax; url becomes the literal # string "${REDIS_URL:-redis://localhost:6379}" url: "${REDIS_URL:-redis://localhost:6379}" ``` Separately, when the implicit-default serve path falls back to env-only configuration, `GatewayConfig::from_env` reads dedicated `LITELLM_*` variables (`LITELLM_HOST`, `LITELLM_PORT`, `LITELLM_DATABASE_URL`, `LITELLM_JWT_SECRET`, `LITELLM_PROVIDERS`, ... — see the constants atop `src/config/models/gateway.rs`). This is independent of `${VAR}` substitution inside YAML. --- ## Configuration Loading Actual chain, `Config::from_file` in `src/config/mod.rs`: ```rust pub async fn from_file>(path: P) -> Result { let content = tokio::fs::read_to_string(path).await .map_err(|e| GatewayError::Config(format!("Failed to read config file: {}", e)))?; let content = substitute_env_vars(&content)?; // hard error on missing ${VAR} let gateway: GatewayConfig = serde_yml::from_str(&content) .map_err(|e| GatewayError::Config(format!("Failed to parse config: {}", e)))?; let config = Self { gateway }; config.validate()?; // Validate trait, single entry point Ok(config) } ``` Entry points: | Path | Behavior | |------|----------| | `gateway --config FILE ...` | loads exactly that file; failure is fatal | | `gateway serve` (no `--config`) | tries `config/gateway.yaml`; on any load error retries via `Config::from_env()` | | `gateway validate-config` | loads + validates only, prints result | Secret redaction happens only on export: `to_json` / `to_yaml` replace known secrets (provider API keys, `jwt_secret`, HMAC secret, S3/vector/SSO credentials) with `[REDACTED]` before serializing. ## References - [reference/validation.md](reference/validation.md) — Validate trait, validation module layout, and per-section validation rules. - [reference/provider-config-errors.md](reference/provider-config-errors.md) — Provider configuration trait and ConfigError variants. - [reference/best-practices.md](reference/best-practices.md) — Defaults, early validation, secret handling, type safety, and documentation practices.