specification: API Commons Rate Limits
specificationVersion: '0.1'
schema: https://raw.githubusercontent.com/api-evangelist/interface-research/main/schema/api-commons.yml#/$defs/RateLimits
provider: Amazon SageMaker
providerId: amazon-sagemaker
created: '2026-05-04'
# Provenance stamped 2026-08-11: this artifact was written by the API Evangelist
# bulk sweep dated 2026-05-04, not harvested from the provider. See roadmap#35.
method: generated
modified: '2026-05-05'
reconciled: true
tags:
  - Rate Limiting
  - AWS
  - Machine Learning
  - SageMaker
description: Amazon SageMaker exposes a control-plane API (CreateTrainingJob, CreateEndpoint, etc.) that
  follows AWS API throttling per account/region, plus a runtime InvokeEndpoint surface whose throughput
  scales with the underlying instance count and instance type. Endpoint-specific quotas (concurrent invocations,
  payload size, timeout) are configurable. ServiceQuotas governs the maximum number and type of instances
  per account.
sources:
  - https://docs.aws.amazon.com/sagemaker/latest/dg/regions-quotas.html
  - https://docs.aws.amazon.com/sagemaker/latest/APIReference/Welcome.html
  - https://docs.aws.amazon.com/general/latest/gr/api-retries.html
headers:
  requestId: x-amzn-RequestId
  retryAfter: Retry-After
responseCodes:
  throttled: 400
  quotaExceeded: 400
  serviceUnavailable: 503
limits:
  - name: SageMaker control-plane API
    scope: account/region
    metric: varies
    limit: see Service Quotas console for SageMaker
    notes: Standard AWS API throttling envelope.
  - name: InvokeEndpoint (real-time)
    scope: endpoint
    metric: requests_per_second
    limit: scales with instance count and type
    notes: Default soft limit per endpoint; configure auto-scaling on the production variant. Payload up
      to 6 MB synchronous, 1 GB asynchronous.
  - name: InvokeEndpoint payload size
    scope: endpoint
    metric: bytes
    limit: 6291456
    notes: 6 MB max synchronous payload; use AsynchronousInferenceConfig for larger payloads (up to 1 GB).
  - name: Synchronous invocation timeout
    scope: endpoint
    metric: seconds
    limit: 60
    timeFrame: second
    notes: Default 60s; can be raised on async endpoints up to 1 hour.
  - name: ML instances per type per region
    scope: account/region
    metric: count
    limit: see Service Quotas console for SageMaker
    notes: Soft limits; raise via Service Quotas before training/deploying at scale.
policies:
  - name: Backoff with jitter
    description: AWS SDKs default to standard retry mode (truncated exponential backoff with jitter, max
      20s, 3 attempts).
  - name: Auto-scaling
    description: Configure target-tracking scaling on production variants (InvocationsPerInstance) to absorb
      load.
  - name: Quota increases
    description: ML instance counts, training-job concurrency, and notebook quotas are all soft limits;
      raise via Service Quotas before campaigns.
  - name: Async inference for large payloads
    description: Use SageMaker Asynchronous Inference for payloads >6 MB or processing >60s, queuing requests
      to a SageMaker-managed S3 location.
maintainers:
  - name: Kin Lane
    email: kin@apievangelist.com
    url: https://apievangelist.com
  - name: Amazon Web Services
    email: support@aws.amazon.com
    url: https://aws.amazon.com