specification: API Commons Rate Limits
specificationVersion: '0.1'
schema: https://raw.githubusercontent.com/api-evangelist/interface-research/main/schema/api-commons.yml#/$defs/RateLimits
provider: vLLM
providerId: vllm
created: '2026-05-08'
# Provenance stamped 2026-08-11: this artifact was written by the API Evangelist
# bulk sweep dated 2026-05-08, not harvested from the provider. See roadmap#35.
method: generated
modified: '2026-05-08'
reconciled: true
tags:
- LLM
- Inference
- Open Source
- GPU
- OpenAI Compatible
- Self-Hosted
- Rate Limiting
- Quotas
- Throttling
description: >-
  vLLM does not impose project-level API rate limits. Throughput is bounded by GPU memory, model size,
  batch settings (--max-num-seqs, --max-model-len), and tensor/pipeline parallelism. Optionally, set
  --api-key to require auth and put a reverse proxy (Nginx, Envoy) in front to enforce per-client
  throttles.
notes: No built-in rate limits; throughput is GPU-bound.
sources:
- https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html
responseCodes:
  throttled: 429
limits:
- name: Project-level
  scope: n/a
  metric: n/a
  limit: no built-in cap
  notes: Throughput is bounded by GPU and batching.
- name: Per-deployment (operator-set)
  scope: deployment
  metric: concurrent-requests
  limit: configured via --max-num-seqs
  notes: Operator tunes max concurrency at server start.
policies:
- name: Reverse-Proxy Throttling
  description: Front vLLM with Nginx/Envoy to enforce per-API-key or per-IP rate limits.
- name: Batch Tuning
  description: Tune --max-num-seqs, --max-model-len, and tensor parallelism for throughput vs. latency trade-offs.
maintainers:
- FN: Kin Lane
  email: kin@apievangelist.com