asyncapi: 3.0.0 info: title: Amazon Kinesis Data Streams version: '2013-12-02' description: >- Amazon Kinesis Data Streams is a massively scalable and durable real-time data streaming service. This AsyncAPI specification describes the event-driven consumer patterns for Kinesis Data Streams, including enhanced fan-out via SubscribeToShard (HTTP/2 push), polling via GetRecords, and integration with AWS Lambda as an event source. Kinesis Data Streams captures data from producers and makes it available to consumers in real time. Data records are organized into shards, and consumers read records sequentially from shards using shard iterators or enhanced fan-out subscriptions. contact: name: Amazon Web Services url: https://aws.amazon.com/kinesis/data-streams/ license: name: Apache 2.0 url: https://www.apache.org/licenses/LICENSE-2.0 externalDocs: description: Amazon Kinesis Data Streams Developer Guide url: https://docs.aws.amazon.com/streams/latest/dev/introduction.html tags: - name: streaming description: Real-time data streaming operations. - name: enhanced-fan-out description: Dedicated throughput consumer via SubscribeToShard. - name: polling description: Pull-based consumption using GetRecords. - name: event-source description: AWS Lambda event source mapping for Kinesis. defaultContentType: application/json servers: production: host: kinesis.{region}.amazonaws.com protocol: https description: >- Amazon Kinesis Data Streams regional endpoint. Enhanced fan-out uses HTTP/2 server push over this endpoint. Polling consumers use standard HTTPS POST requests. variables: region: default: us-east-1 description: AWS region identifier enum: - us-east-1 - us-east-2 - us-west-1 - us-west-2 - eu-west-1 - eu-west-2 - eu-west-3 - eu-central-1 - eu-north-1 - ap-northeast-1 - ap-northeast-2 - ap-southeast-1 - ap-southeast-2 - ap-south-1 - sa-east-1 - ca-central-1 security: - $ref: '#/components/securitySchemes/awsSigV4' channels: kinesisStreamShard: address: 'kinesis/{streamName}/{shardId}' title: Kinesis Stream Shard description: >- A shard within a Kinesis data stream. Each shard is an append-only, ordered sequence of data records. Producers write records to shards via partition keys, and consumers read records sequentially. Each shard provides 1 MiB/sec write capacity and 2 MiB/sec read capacity (shared among polling consumers), or 2 MiB/sec dedicated throughput per enhanced fan-out consumer. parameters: streamName: description: >- The name of the Kinesis data stream. Must be between 1 and 128 characters, matching the pattern [a-zA-Z0-9_.-]+. shardId: description: >- The unique identifier of the shard within the stream, for example shardId-000000000001. messages: kinesisRecord: $ref: '#/components/messages/KinesisRecord' subscribeToShardEvent: $ref: '#/components/messages/SubscribeToShardEvent' enhancedFanOutSubscription: address: 'kinesis/{streamName}/{shardId}/consumer/{consumerName}' title: Enhanced Fan-Out Subscription description: >- A dedicated HTTP/2 push channel between a registered consumer and a specific shard. Established via the SubscribeToShard API, this channel pushes SubscribeToShardEvent messages containing data records at up to 2 MiB/sec per shard per consumer. Each subscription lasts up to 5 minutes before requiring renewal. parameters: streamName: description: The name of the Kinesis data stream. shardId: description: The shard ID to subscribe to. consumerName: description: >- The name of the registered consumer (from RegisterStreamConsumer). messages: subscribeToShardEvent: $ref: '#/components/messages/SubscribeToShardEvent' lambdaEventSource: address: 'kinesis/{streamName}/lambda-event-source' title: Lambda Event Source Mapping description: >- When configured as an AWS Lambda event source, Kinesis Data Streams automatically invokes a Lambda function with batches of records from each shard. The Lambda service manages shard iterator lifecycle, checkpointing, and retry logic. Each shard is processed by one Lambda invocation at a time (unless parallelization is configured). parameters: streamName: description: The name of the Kinesis data stream. messages: lambdaKinesisEvent: $ref: '#/components/messages/LambdaKinesisEvent' operations: publishRecord: action: send channel: $ref: '#/channels/kinesisStreamShard' title: Publish Record to Stream summary: >- A producer writes a data record to a Kinesis data stream shard. Each record includes a data blob (up to 1 MiB), a partition key, and optionally an explicit hash key. The partition key determines which shard receives the record. description: >- Producers publish records using the PutRecord or PutRecords API. Kinesis assigns each record a unique sequence number and distributes records across shards based on the partition key hash. Each shard supports up to 1,000 records/sec or 1 MiB/sec of write throughput. messages: - $ref: '#/channels/kinesisStreamShard/messages/kinesisRecord' tags: - name: streaming consumeRecordsByPolling: action: receive channel: $ref: '#/channels/kinesisStreamShard' title: Consume Records by Polling (GetRecords) summary: >- A consumer polls for data records from a shard using GetRecords with a shard iterator obtained from GetShardIterator. Each call returns up to 10,000 records or 10 MiB. description: >- Polling consumers use the GetShardIterator API to obtain a starting position, then repeatedly call GetRecords to retrieve batches of records. The shard iterator advances automatically. Each shard supports up to 5 GetRecords calls per second, sharing 2 MiB/sec of read throughput among all polling consumers. The consumer must handle checkpointing and iterator management. messages: - $ref: '#/channels/kinesisStreamShard/messages/kinesisRecord' tags: - name: polling consumeRecordsByEnhancedFanOut: action: receive channel: $ref: '#/channels/enhancedFanOutSubscription' title: Consume Records via Enhanced Fan-Out (SubscribeToShard) summary: >- A registered consumer receives data records pushed over an HTTP/2 connection at a dedicated rate of 2 MiB/sec per shard. description: >- Enhanced fan-out consumers register using RegisterStreamConsumer, then call SubscribeToShard to establish an HTTP/2 connection to a specific shard. Kinesis pushes SubscribeToShardEvent messages containing batches of records over the connection. Each subscription lasts up to 5 minutes. Unlike polling, each enhanced fan-out consumer gets its own dedicated 2 MiB/sec read throughput per shard, enabling multiple consumers to read from the same shard simultaneously without contention. Up to 20 consumers can be registered per stream. messages: - $ref: '#/channels/enhancedFanOutSubscription/messages/subscribeToShardEvent' tags: - name: enhanced-fan-out consumeRecordsByLambda: action: receive channel: $ref: '#/channels/lambdaEventSource' title: Consume Records via AWS Lambda Event Source summary: >- AWS Lambda automatically polls the stream and invokes a function with batches of records from each shard. description: >- When a Kinesis data stream is configured as a Lambda event source, the Lambda service polls each shard and invokes the target function with batches of records. Configurable settings include batch size (up to 10,000 records), batch window (up to 5 minutes), starting position, bisect-on-error behavior, retry attempts, and parallelization factor (up to 10 concurrent invocations per shard). Lambda manages checkpointing automatically. messages: - $ref: '#/channels/lambdaEventSource/messages/lambdaKinesisEvent' tags: - name: event-source components: securitySchemes: awsSigV4: type: httpApiKey name: Authorization in: header description: >- AWS Signature Version 4 authentication. All requests to the Kinesis Data Streams API must be signed with IAM credentials. messages: KinesisRecord: name: KinesisRecord title: Kinesis Data Record summary: >- A single data record in a Kinesis data stream. Each record contains a data blob, partition key, and is assigned a unique sequence number and approximate arrival timestamp by the service. contentType: application/json payload: $ref: '#/components/schemas/KinesisRecord' SubscribeToShardEvent: name: SubscribeToShardEvent title: Subscribe to Shard Event summary: >- An event pushed to an enhanced fan-out consumer over an HTTP/2 connection. Contains a batch of data records along with a continuation sequence number and lag metrics. contentType: application/json payload: $ref: '#/components/schemas/SubscribeToShardEvent' LambdaKinesisEvent: name: LambdaKinesisEvent title: Lambda Kinesis Event summary: >- The event payload delivered to an AWS Lambda function when a Kinesis data stream is configured as an event source. Contains an array of Kinesis event records with metadata including the event source ARN, region, and event name. contentType: application/json payload: $ref: '#/components/schemas/LambdaKinesisEvent' schemas: KinesisRecord: type: object description: >- The unit of data stored in a Kinesis data stream. A record includes a sequence number, approximate arrival timestamp, data blob, partition key, and optionally the encryption type. required: - SequenceNumber - Data - PartitionKey properties: SequenceNumber: type: string pattern: '0|([1-9]\d{0,128})' description: >- The unique identifier of the record within its shard. Sequence numbers increase over time, and each record has a unique sequence number within its shard. ApproximateArrivalTimestamp: type: number format: double description: >- The approximate time that the record was inserted into the stream, as a Unix epoch timestamp in seconds with millisecond precision. Data: type: string format: byte maxLength: 1048576 description: >- The data blob. The data is opaque to Kinesis Data Streams, which does not inspect, interpret, or change the data. Base64-encoded when serialized to JSON. Maximum size is 1 MiB. PartitionKey: type: string minLength: 1 maxLength: 256 description: >- Identifies which shard in the stream the data record is assigned to. Partition key is a Unicode string with a maximum length limit of 256 characters. EncryptionType: type: string enum: - NONE - KMS description: >- The encryption type used on the record. If the stream uses server-side encryption, this is set to KMS. SubscribeToShardEvent: type: object description: >- After you call SubscribeToShard, Kinesis Data Streams sends events of this type over the HTTP/2 connection to the registered consumer. required: - Records - ContinuationSequenceNumber - MillisBehindLatest properties: Records: type: array items: $ref: '#/components/schemas/KinesisRecord' description: The data records from the shard. ContinuationSequenceNumber: type: string pattern: '0|([1-9]\d{0,128})' description: >- Use this value as the StartingSequenceNumber in the next SubscribeToShard call to continue reading from where you left off. MillisBehindLatest: type: integer minimum: 0 description: >- The number of milliseconds the read records are from the tip of the stream, indicating how far behind current time the consumer is. A value of 0 indicates the consumer is caught up. ChildShards: type: array items: $ref: '#/components/schemas/ChildShard' description: >- The list of the current shard's child shards, returned only at the end of the current shard after a shard split or merge. ChildShard: type: object description: A child shard resulting from a shard split or merge operation. required: - ShardId - ParentShards - HashKeyRange properties: ShardId: type: string minLength: 1 maxLength: 128 pattern: '[a-zA-Z0-9_.-]+' description: The shard ID of the child shard. ParentShards: type: array items: type: string description: The list of parent shard IDs. HashKeyRange: $ref: '#/components/schemas/HashKeyRange' HashKeyRange: type: object description: The range of possible hash key values for the shard. required: - StartingHashKey - EndingHashKey properties: StartingHashKey: type: string description: The starting hash key of the hash key range. EndingHashKey: type: string description: The ending hash key of the hash key range. LambdaKinesisEvent: type: object description: >- The event payload delivered to a Lambda function by a Kinesis event source mapping. Contains an array of event records with Kinesis data and metadata. required: - Records properties: Records: type: array items: $ref: '#/components/schemas/LambdaKinesisEventRecord' description: An array of Kinesis event records. LambdaKinesisEventRecord: type: object description: >- A single Kinesis event record within a Lambda invocation payload. Wraps the Kinesis data record with event source metadata. required: - kinesis - eventSource - eventSourceARN - eventID - eventName - eventVersion - invokeIdentityArn - awsRegion properties: kinesis: $ref: '#/components/schemas/LambdaKinesisData' eventSource: type: string const: 'aws:kinesis' description: The AWS service that is the source of the event. eventSourceARN: type: string pattern: 'arn:aws.*:kinesis:.*:\d{12}:stream/\S+' description: The ARN of the Kinesis data stream. eventID: type: string description: >- A unique identifier for the event, composed of the shard ID and sequence number. examples: - 'shardId-000000000001:21269319989652663814458848515492872193' eventName: type: string const: 'aws:kinesis:record' description: The event type identifier. eventVersion: type: string description: The version of the event structure. examples: - '1.0' invokeIdentityArn: type: string description: The ARN of the IAM role used to invoke the Lambda function. awsRegion: type: string description: The AWS region where the event originated. examples: - us-east-1 LambdaKinesisData: type: object description: >- The Kinesis-specific data within a Lambda event record, containing the data blob, partition key, sequence information, and arrival timestamp. required: - data - partitionKey - sequenceNumber - kinesisSchemaVersion properties: kinesisSchemaVersion: type: string description: The Kinesis schema version for this record. examples: - '1.0' partitionKey: type: string minLength: 1 maxLength: 256 description: The partition key for the record. sequenceNumber: type: string description: The unique sequence number for the record within the shard. data: type: string format: byte description: >- The data blob, Base64-encoded. Your Lambda function must decode this value to access the original data. approximateArrivalTimestamp: type: number format: double description: >- The approximate time that the record was inserted into the stream, as a Unix epoch timestamp.