--- title: Authentication --- # Authentication SMG supports multiple authentication methods for securing access to inference APIs and the control plane, including JWT/OIDC integration, API keys, and role-based access control. --- ## Overview
### :material-key-chain: Multiple Auth Methods Support for JWT/OIDC, API keys, and worker authentication to fit your deployment model.
### :material-shield-account: Role-Based Access The admin role has full control plane access; the user role is read-only (`GET` and `HEAD`). Data plane APIs use API keys, not roles.
### :material-office-building: Enterprise SSO Integrate with Keycloak, Auth0, Azure AD, Okta, and other OIDC providers.
### :material-clipboard-text: Audit Logging Track all control plane operations for security monitoring and compliance.
--- ## Authentication Methods | Method | Use Case | Configuration | |--------|----------|---------------| | **Control plane JWT/OIDC** | Enterprise SSO integration with identity providers (admin routes) | `--jwt-issuer`, `--jwt-audience` | | **Control plane API keys** | Service accounts and programmatic access (admin routes) | `--control-plane-api-keys` | | **Data plane API key** | Shared bearer token gating data plane routes; also the default API key of startup and discovered workers | `--api-key` | | **Data plane tenant keys** | Per-tenant bearer tokens for data plane routes, each resolving to its own tenant identity | `--tenant-api-key` | ### When to Use Each Method - **Control plane JWT/OIDC**: Use for enterprise deployments with existing identity providers (Keycloak, Auth0, Azure AD, Okta). Provides centralized user management and SSO for control plane operations. - **Control plane API keys**: Use for service-to-service automation against admin endpoints (CI/CD pipelines, tooling). Simpler to set up but requires manual key management. - **Data plane API key**: Use when you want a single shared secret that clients present on data plane routes (chat, completions, responses, and so on), and that workers started with the same key accept. - **Data plane tenant keys**: Use when each team or application needs its own data plane key, so that [tenant rate limits](../reliability/tenant-rate-limiting.md) and priority scheduling can tell callers apart. Each `--tenant-api-key tenant_id:key` resolves to the tenant `auth:`. Tenant keys never unlock control plane routes. JWTs and control plane API keys are checked only on control plane routes. Data plane routes accept only `--api-key` and `--tenant-api-key` credentials, and are open when neither is set. --- ## JWT/OIDC Authentication JWT (JSON Web Token) authentication allows integration with OIDC-compliant identity providers for enterprise single sign-on on the control plane. ### Configuration Options | Option | Environment Variable | Description | |--------|---------------------|-------------| | `--jwt-issuer` | `JWT_ISSUER` | OIDC issuer URL (required for JWT auth) | | `--jwt-audience` | `JWT_AUDIENCE` | Expected audience claim (required for JWT auth) | | `--jwt-jwks-uri` | `JWT_JWKS_URI` | Explicit JWKS URI (auto-discovered if not set) | | `--jwt-role-claim` | - | Claim name containing roles (default: `roles`) | | `--jwt-role-mapping` | - | Map IDP roles to gateway roles | The `JWT_*` environment variables belong to the Rust `smg` binary; the Python launcher (`smg launch` from pip, and the container image) takes the JWT settings as flags only. See [Python Launcher Differences](../../reference/configuration.md#python-launcher-differences). ### Basic Setup Enable JWT authentication by providing the issuer and audience: ```bash smg \ --worker-urls http://worker:8000 \ --jwt-issuer "https://auth.example.com/realms/myrealm" \ --jwt-audience "smg-gateway" ``` `--jwt-issuer` and `--jwt-audience` are required together: SMG refuses to start with only one of them. ### JWKS Discovery By default, SMG discovers the JWKS (JSON Web Key Set) endpoint automatically via OIDC discovery (`/.well-known/openid-configuration`). You can override this with an explicit JWKS URI: ```bash smg \ --worker-urls http://worker:8000 \ --jwt-issuer "https://auth.example.com" \ --jwt-audience "smg-gateway" \ --jwt-jwks-uri "https://auth.example.com/.well-known/jwks.json" ``` Without `--jwt-jwks-uri`, SMG fetches the discovery document at startup. Either way, SMG fetches the key set on first use and caches it for an hour. The discovery document's `issuer` must be identical to the configured `--jwt-issuer`: a document naming another issuer — even one differing only in a trailing slash — is refused, and JWT initialization fails. Tokens must carry a `kid` header that names a key in the set. When a token names a key the cached set doesn't have, SMG fetches the set again, which is how a rotated key gets picked up. This refetch happens at most once a minute, and a failed fetch also counts against the minute. Until the minute has passed, tokens naming unknown keys are rejected without a request to the identity provider. The discovery URL and the JWKS URI must use HTTPS (plain HTTP is allowed only for `localhost`, `127.0.0.1`, and `::1`), must not point at a private, loopback, link-local, or other internal IP address literal, and must not use a host name ending in `.internal` or `.local`. SMG does not follow redirects on these requests. If SMG cannot set up JWT validation at startup (for example, OIDC discovery fails or a URL is rejected), control plane authentication stays required: SMG logs `Failed to initialize JWT authentication: ` and runs with JWT validation unavailable. Keys from `--control-plane-api-keys` keep working, while JWTs, the shared `--api-key`, and any other token get `401` on control plane routes. SMG does not retry JWT initialization while running; restart the gateway after correcting the configuration or restoring the identity provider. ### Role Mapping Map identity provider roles to SMG gateway roles: ```bash smg \ --worker-urls http://worker:8000 \ --jwt-issuer "https://auth.example.com" \ --jwt-audience "smg-gateway" \ --jwt-role-mapping "Gateway.Admin=admin" \ --jwt-role-mapping "Gateway.User=user" ``` **Role Mapping Format**: `idp_role=gateway_role` SMG refuses to start on a malformed mapping: one without `=`, or with a gateway role other than `admin` or `user`. | Gateway Role | Permissions | |--------------|-------------| | `admin` | Full access to all control plane APIs (workers, WASM modules, tokenizers) | | `user` | Read-only control plane access: `GET` and `HEAD`, except the admin-only `GET /config`; any other method answers `403` | ### Supported Claims SMG reads role values from the configured `--jwt-role-claim` (default: `roles`). Only when the token has no such claim does it fall back to the `role`, `roles`, `groups`, and `group` claims, collecting the values of all of them in that order. Each claim may be a string or an array of strings. - Without `--jwt-role-mapping`, the first value equal to `admin` or `user` (ignoring case) sets the role. - With `--jwt-role-mapping`, the first value that has a mapping sets the role; unmapped values are ignored. If no role is found, the user defaults to the `user` role. ### Supported Algorithms SMG supports the following JWT signing algorithms: - **RSA**: RS256, RS384, RS512 - **ECDSA**: ES256, ES384 --- ## Identity Provider Setup ### Keycloak
**1. Create a Client** - Navigate to Clients > Create - Client ID: `smg-gateway` - Client Protocol: `openid-connect` - Access Type: `confidential` or `public`
**2. Configure Mappers** - Add a mapper of type "User Realm Role" - Token Claim Name: `roles` - Add to ID token: Yes - Add to access token: Yes
```bash smg \ --worker-urls http://worker:8000 \ --jwt-issuer "https://keycloak.example.com/realms/myrealm" \ --jwt-audience "smg-gateway" \ --jwt-role-mapping "admin=admin" \ --jwt-role-mapping "user=user" ``` ### Auth0
**1. Create an API** - Navigate to Applications > APIs > Create API - Name: `SMG Gateway` - Identifier: `https://smg.example.com/api`
**2. Add Roles Action** ```javascript exports.onExecutePostLogin = async (event, api) => { const namespace = 'https://smg.example.com'; if (event.authorization) { api.accessToken.setCustomClaim( `${namespace}/roles`, event.authorization.roles ); } }; ```
```bash smg \ --worker-urls http://worker:8000 \ --jwt-issuer "https://your-tenant.auth0.com/" \ --jwt-audience "https://smg.example.com/api" \ --jwt-role-claim "https://smg.example.com/roles" \ --jwt-role-mapping "smg-admin=admin" \ --jwt-role-mapping "smg-user=user" ``` ### Azure AD / Entra ID
**1. Register an Application** - Navigate to Azure Portal > App registrations - Name: `SMG Gateway` - Configure app roles: `Gateway.Admin`, `Gateway.User`
**2. Expose an API** - Navigate to Expose an API - Set Application ID URI: `api://smg-gateway`
```bash smg \ --worker-urls http://worker:8000 \ --jwt-issuer "https://login.microsoftonline.com/{tenant-id}/v2.0" \ --jwt-audience "api://smg-gateway" \ --jwt-role-mapping "Gateway.Admin=admin" \ --jwt-role-mapping "Gateway.User=user" ``` ### Okta ```bash smg \ --worker-urls http://worker:8000 \ --jwt-issuer "https://your-org.okta.com/oauth2/default" \ --jwt-audience "api://smg" \ --jwt-role-mapping "smg_admins=admin" \ --jwt-role-mapping "smg_users=user" ``` --- ## API Key Authentication API keys provide a simpler authentication method for service accounts and programmatic access. ### Control Plane API Keys Configure API keys for control plane access: ```bash smg \ --worker-urls http://worker:8000 \ --control-plane-api-keys "key1:Service Account:admin:sk-your-secret-key-here" ``` **Format**: `id:name:role:key` | Component | Description | |-----------|-------------| | `id` | Unique identifier for the key | | `name` | Human-readable name/description | | `role` | Gateway role (`admin` or `user`) | | `key` | The secret API key value | SMG refuses to start on a malformed entry, including one with a role other than `admin` or `user`. The error does not echo the entry, since it carries the key. ### Multiple API Keys ```bash smg \ --worker-urls http://worker:8000 \ --control-plane-api-keys "admin1:Admin Service:admin:sk-admin-key-12345" \ --control-plane-api-keys "user1:Read Only Service:user:sk-readonly-key-67890" ``` A `user` key can read the control plane (`GET` and `HEAD`, except the admin-only `GET /config`); any other control plane method answers it with `403` (see [Role-Based Access Control](#role-based-access-control)). ### Environment Variable Configuration For security, pass API keys via environment variable: ```bash export CONTROL_PLANE_API_KEYS="admin1:Admin Service:admin:sk-admin-key-12345" smg --worker-urls http://worker:8000 ``` `CONTROL_PLANE_API_KEYS` holds one or more `id:name:role:key` entries, separated by commas or newlines, and both the Rust `smg` binary and the Python launcher (`smg launch` from pip, and the container image) read it. The flag beats the environment, and a variable that is set but holds no entry fails startup. ### Key Files Each key setting — the control plane keys, the shared data plane key, and the tenant keys — can also come from a file, such as a mounted Kubernetes Secret: | Setting | Flag | Environment | File flag | File's environment | |---------|------|-------------|-----------|--------------------| | Control plane keys | `--control-plane-api-keys` | `CONTROL_PLANE_API_KEYS` | `--control-plane-api-keys-file` | `CONTROL_PLANE_API_KEYS_FILE` | | Shared data plane key | `--api-key` | `SMG_API_KEY` | `--api-key-file` | `SMG_API_KEY_FILE` | | Tenant keys | `--tenant-api-key` | `SMG_TENANT_API_KEYS` | `--tenant-api-keys-file` | `SMG_TENANT_API_KEYS_FILE` | A key-list file holds one entry per line (`id:name:role:key` or `tenant_id:key`); blank lines and lines starting with `#` are skipped, and a file that holds no entry is refused — it was meant to close the routes its keys guard. `--api-key-file` holds the key alone on one line, surrounding whitespace dropped; an empty file or one with more than one line is refused. For each setting, the flag beats its environment variable, and a value together with a file is refused rather than ranked. A refusal names the setting and the path, never the key. ### Reloading Keys Without a Restart `POST /secrets/reload` reads every key file given at start again and puts the new sets in force in one swap; keys that came from a flag or an environment variable stay as the start read them. The route requires the `admin` role and exists only when control plane authentication is configured — without it the path answers `404`. It answers: - `200` with a report of the files read and whether the serving and control plane key sets changed - `409` when no key came from a file, so there is nothing to reload - `422` when a file cannot be used — unreadable, malformed, or breaking a startup rule such as a duplicate key — with every set standing as it was Requests in flight finish under the keys they were admitted with. With `--secrets-auto-reload`, SMG watches the key files and runs the same reload when one changes, so a mounted Secret rotated in place takes effect without a restart or an API call. ### Using API Keys Clients authenticate by including the API key in the Authorization header: ```bash curl -H "Authorization: Bearer sk-admin-key-12345" \ https://smg.example.com/workers ``` ### Security Features - **Hashed Storage**: Keys are SHA-256 hashed immediately; plaintext keys are never stored in memory - **Constant-Time Comparison**: Key verification uses constant-time comparison to prevent timing attacks - **Role-Based Access**: Each key is assigned a specific role limiting its permissions --- ## Data Plane API Key (`--api-key`) The `--api-key` option configures a single bearer token that does two things: ```bash smg \ --worker-urls http://worker:8000 \ --api-key "shared-secret-key" ``` 1. **Gates incoming data plane requests.** The gateway requires every client request to data plane routes (`/v1/chat/completions`, `/v1/completions`, `/v1/responses`, `/v1/embeddings`, `/v1/rerank`, `/v1/messages`, `/v1/realtime/*`, etc.) to present `Authorization: Bearer ` (or a `--tenant-api-key` key). Requests without a valid token receive `401 Unauthorized`. Public routes such as `/health` and `/v1/models` stay open. 2. **Becomes the API key of startup and discovered workers.** Workers from `--worker-urls` (and from `--prefill` and `--decode`) and workers that Kubernetes service discovery registers get the `--api-key` value as their worker API key. A worker added through `POST /workers` uses only the `api_key` in its own spec; SMG logs a warning when one arrives without a key while `--api-key` is set. Like the other key settings, the key can come from the `SMG_API_KEY` environment variable or a file (`--api-key-file`), and tenant keys from `SMG_TENANT_API_KEYS` or `--tenant-api-keys-file`; see [Key Files](#key-files). What reaches a worker depends on the path: | Path | What the worker receives | |------|--------------------------| | HTTP regular router | The client's `Authorization` header as sent, else the worker's API key as `Authorization: Bearer `. A matched gateway key (`--api-key` or a tenant key) is removed at authentication, so the worker receives its own API key or, when it has no key, no `Authorization` header at all. | | HTTP PD router | The client's allowlisted headers on both the prefill and the decode request, with the same `Authorization` precedence as the regular router: the client's header when one reaches the router, else each worker's own API key as `Authorization: Bearer `. | | gRPC and ZMQ workers | No API key and no `Authorization` metadata. | | SMG's own calls to HTTP workers (health checks, metadata discovery, load polling) | The worker's API key, as `Authorization: Bearer `. | | External providers | See [External Providers](../../getting-started/external-providers.md#api-key-handling). | Service discovery registers each pod as a bare `host:port`, so SMG reaches it over plain HTTP (or plaintext gRPC), and a worker API key crosses the network unencrypted. This is useful when: - Clients and workers share a common token - Workers require authentication (e.g., deployed with API key protection) - Using DP-aware scheduling that requires authenticated worker queries - Workers are behind an authentication proxy Control plane routes (`/workers`, `/wasm`, `/v1/tokenizers`, etc.) use their own middleware stack. When `--control-plane-api-keys` or `--jwt-*` are configured they take over as the admin auth backend (with role-based access control). With this backend, the Rust binary enables audit logging by default; the Python launcher (pip and container image) requires `--control-plane-audit-enabled`. When neither auth option is set, admin routes fall back to the same `--api-key` bearer check that guards the data plane (tenant keys are never accepted there, and with only `--tenant-api-key` set, admin routes reject every request with `401`). If you run with **only** `--api-key`, the same shared secret therefore gates both the data plane and the control plane — which is rarely what you want in production. See [Control Plane Auth](../../getting-started/control-plane-auth.md) for configuring JWT/OIDC or dedicated control-plane keys. --- ## Role-Based Access Control SMG implements role-based access control (RBAC) with two primary roles: ### Admin Role Full access to all control plane APIs: - Worker management (`/workers`, `/workers/{id}`) - WASM module management (`/wasm/*`) - Tokenizer configuration - System administration - Configuration view and key reload (`GET /config`, `POST /secrets/reload`) — two routes that exist only when control plane authentication is configured ### User Role Read-only control plane access: `GET` and `HEAD` on the control plane routes. Every other method answers a `user` credential with `403 Admin role required for this control plane operation`. One read is kept to the `admin` role: `GET /config`, which names the store addresses, answers a `user` credential with the same `403`. Roles don't apply to data plane routes (chat completions, completions, embeddings, and the rest): those accept only `--api-key` and `--tenant-api-key` credentials, never JWTs or control plane API keys. `/v1/models` is public. ### Role Assignment Roles are assigned through: 1. **JWT Claims**: Via `--jwt-role-mapping` configuration 2. **API Key Configuration**: Via the role component in `--control-plane-api-keys` If no role can be determined, the user defaults to `user` role for safety. --- ## Audit Logging SMG provides audit logging for control plane operations to support security monitoring and compliance. ### Configuration With control plane authentication configured (`--jwt-issuer` with `--jwt-audience`, or control plane API keys from any source), the Rust `smg` binary logs audit events **by default**. To disable: ```bash smg \ --worker-urls http://worker:8000 \ --jwt-issuer "https://auth.example.com" \ --jwt-audience "smg-gateway" \ --disable-audit-logging ``` The Python launcher (`smg launch` from pip, and the container image) works the other way around: audit logging is off unless you pass `--control-plane-audit-enabled`, and it has no `--disable-audit-logging` flag. Neither launcher audits admin requests checked by the `--api-key` fallback. ### Audit Log Format Each audit event is an `INFO` log record with target `smg::audit`; the layout of the record follows your log format (`--log-json` for JSON). A control-plane call writes up to two lines: - **`control_plane_audit`** — the admission line, written before the handler runs, for every request that reaches the control plane authentication check: successes, `403` role denials, and `401` authentication failures. - **`control_plane_mutation`** — the outcome line, written after the handler of an admitted request other than a `GET` or `HEAD` has answered. It records what the handler answered: the HTTP `status`, an `outcome` of `success` for a 2xx status and `failed` for any other, and the `resource` the handler acted on, when it named one. Both lines of a mutation carry the same `principal`, `auth_method`, `role`, `method`, `path`, and `request_id`, so one request ID correlates them. A read (`GET` or `HEAD`) writes the admission line alone. A denied or unauthenticated request also writes only the admission line, recording the denial — no handler ran, so there is no outcome line. `--disable-audit-logging` silences both lines. ### Audit Event Fields | Field | Description | |-------|-------------| | `timestamp` | RFC 3339 timestamp of the event | | `principal` | JWT subject (or its `email`, then `preferred_username`, when `sub` is missing), the API key's `id`, or `unauthenticated` when authentication failed | | `auth_method` | Authentication method (`jwt`, `api_key`, or `none` when authentication failed) | | `role` | Role of the principal (`admin`, `user`) | | `method` | HTTP method (GET, POST, DELETE, etc.) | | `path` | Request path | | `resource` | What the handler acted on: a worker ID, a tokenizer ID, WASM module UUIDs (comma-joined for a batch), or `secrets` for a key-file reload (`POST /secrets/reload`). Only on a `control_plane_mutation` line whose handler named one; left out everywhere else | | `outcome` | On the admission line, `success` (admitted) or `denied` — decided before the handler runs; on the outcome line, `success` (a 2xx status) or `failed` (any other status) | | `status` | The HTTP status the handler answered. Only on the `control_plane_mutation` line; the admission line carries none | | `request_id` | Correlation ID for request tracing; a mutation's admission and outcome lines share it | | `details` | Why the request was denied or failed authentication | ### Viewing Audit Logs ```bash # Filter for audit logs RUST_LOG=smg::audit=info smg ... # Or view in combined logs (both the admission and the outcome lines) kubectl logs -n inference -l app=smg | grep -E "control_plane_(audit|mutation)" ``` --- ## Production Configuration A production setup combining JWT and API key authentication: ```bash smg \ --worker-urls http://worker1:8000 http://worker2:8000 \ --host 0.0.0.0 \ --port 8443 \ --tls-cert-path /etc/certs/server.crt \ --tls-key-path /etc/certs/server.key \ --jwt-issuer "https://auth.example.com/realms/production" \ --jwt-audience "smg-gateway" \ --jwt-role-mapping "Gateway.Admin=admin" \ --jwt-role-mapping "Gateway.User=user" \ --control-plane-api-keys "ci-cd:CI/CD Pipeline:admin:${CI_CD_API_KEY}" \ --api-key "${DATA_PLANE_API_KEY}" ``` JWT and the control plane key protect the control plane routes. `--api-key` protects the data plane routes, which are open without it. The example serves on `8443`, not `443`: the published image runs as uid 65532 and the Helm chart drops every capability, so a gateway in that image cannot bind a port below 1024. Put a Service, an ingress or a load balancer in front for `443`, or grant the container `CAP_NET_BIND_SERVICE`. --- ## Kubernetes Deployment ### Secret for API Keys ```yaml apiVersion: v1 kind: Secret metadata: name: smg-auth namespace: inference type: Opaque stringData: CONTROL_PLANE_API_KEYS: "admin1:Admin:admin:sk-secret-key" ``` ### ConfigMap for JWT Configuration ```yaml apiVersion: v1 kind: ConfigMap metadata: name: smg-config namespace: inference data: JWT_ISSUER: "https://auth.example.com/realms/production" JWT_AUDIENCE: "smg-gateway" ``` ### Deployment ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: smg namespace: inference spec: template: spec: containers: - name: smg image: lightseekorg/smg:latest envFrom: - configMapRef: name: smg-config - secretRef: name: smg-auth args: - --service-discovery - --selector - app=sglang-worker - --jwt-issuer - $(JWT_ISSUER) - --jwt-audience - $(JWT_AUDIENCE) - --jwt-role-mapping - "Gateway.Admin=admin" - --jwt-role-mapping - "Gateway.User=user" - --control-plane-audit-enabled ``` The container image runs the Python launcher, which does not read `JWT_ISSUER` or `JWT_AUDIENCE` itself, so the `args` pass them as flags; Kubernetes expands each `$(VAR)` reference from the container's environment. `CONTROL_PLANE_API_KEYS` needs no flag: the launcher reads the variable, with entries separated by commas or newlines. `--control-plane-audit-enabled` turns on audit logging, which the Python launcher leaves off by default. To rotate keys without a restart, mount the Secret as a file instead and pass `--control-plane-api-keys-file` with `--secrets-auto-reload` (see [Key Files](#key-files)). --- ## Troubleshooting ### JWT Validation Failures **Symptom**: `Invalid JWT` or `Token validation failed` errors **Solutions**: 1. Verify issuer URL matches exactly (including trailing slash): ```bash curl https://auth.example.com/.well-known/openid-configuration ``` 2. Verify audience claim matches your configuration: ```bash echo "YOUR_JWT" | cut -d. -f2 | base64 -d | jq . ``` 3. Check clock synchronization (JWT validation uses time-based claims and allows 30 seconds of clock skew) 4. Verify JWKS endpoint is accessible from the SMG pod ### API Key Not Working **Symptom**: `Invalid authentication token` errors **Solutions**: 1. Verify key format is correct: `id:name:role:key` 2. Check for special characters that may need escaping 3. Ensure the Authorization header format is correct: `Bearer ` ### Role Mapping Issues **Symptom**: Users getting wrong permissions **Solutions**: 1. Check which claim contains roles in your JWT 2. Verify role mapping syntax: `idp_role=gateway_role` 3. Check if role claim name needs to be specified with `--jwt-role-claim` --- ## Security Best Practices
### :material-shield-lock: Use HTTPS Always enable TLS for the gateway in production.
### :material-key-change: Rotate Keys Regularly rotate API keys and use short-lived JWT tokens.
### :material-account-lock: Least Privilege Assign `user` role by default, `admin` only when needed.
### :material-clipboard-text: Enable Auditing Keep audit logs for security monitoring and compliance.
--- ## What's Next?
### :material-traffic-light: Rate Limiting Protect against overload and abuse. [Rate Limiting →](../reliability/rate-limiting.md)
### :material-shield-check: High Availability Deploy SMG in a highly available configuration. [High Availability →](../architecture/high-availability.md)
### :material-chart-box: Metrics Reference Count `401` and `403` responses by path with `smg_http_responses_total`. [Metrics Reference →](../../reference/metrics.md)