specification: API Commons Rate Limits
specificationVersion: '0.1'
schema: https://raw.githubusercontent.com/api-evangelist/interface-research/main/schema/api-commons.yml#/$defs/RateLimits
provider: DataHub
providerId: datahub
created: '2026-05-04'
# Provenance stamped 2026-08-11: this artifact was written by the API Evangelist
# bulk sweep dated 2026-05-04, not harvested from the provider. See roadmap#35.
method: generated
modified: '2026-05-05'
reconciled: false
tags:
  - Data Catalog
  - Metadata
  - Open Source
  - Rate Limiting
description: DataHub Core is self-hosted, so there are no vendor-imposed rate limits — throughput
  is bounded by the operator's GMS (Generalized Metadata Service) replicas, the underlying Kafka /
  Elasticsearch / SQL backends, and any reverse-proxy throttling the operator configures. DataHub
  Cloud (the managed Acryl/datahub.com service) does not publish numeric request-per-second limits
  on the public site; tenant capacity is sized to the contract.
sources:
  - https://datahubproject.io/docs/
  - https://datahub.com/products/cloud-vs-core/
notes: No public numeric limits for either Core (operator-defined) or Cloud (gated). Reconciled
  false because there are no published vendor numbers.
responseCodes:
  throttled: 429
  serviceUnavailable: 503
limits:
  - name: DataHub Core operator-configured limit
    scope: deployment
    metric: requests_per_second
    limit: operator-defined
    notes: Self-hosted; throughput is gated by GMS replica count, Kafka throughput, and
      Elasticsearch / SQL backend capacity. Use a reverse proxy (Nginx, Envoy) for external
      throttling.
  - name: DataHub Cloud tenant capacity
    scope: tenant
    metric: requests_per_second
    limit: see datahub.com — sized to contract
    notes: DataHub Cloud capacity is provisioned per customer; numeric ceilings are not published.
policies:
  - name: Self-Hosted Throttling (Core)
    description: For DataHub Core, scale GMS pods and use a reverse-proxy rate-limit module if
      needed. Backoff on 503 from the GMS.
  - name: Bulk Ingestion Pacing
    description: Metadata ingestion via the Python emitter or CLI should batch events and pace
      throughput to avoid saturating the metadata change log topic.
  - name: GraphQL Query Cost
    description: Prefer narrow GraphQL queries with field selection; deep lineage walks can spike
      backend load and induce 503s.
  - name: Cloud SLA
    description: DataHub Cloud is SLA-backed at 99.5% availability; tenants should still implement
      retry with exponential backoff on transient errors.
maintainers:
  - FN: Kin Lane
    email: kin@apievangelist.com