--- title: DNA wire protocol description: "DNA is a protocol built on top of gRPC to stream onchain data." diataxis: explanation updatedAt: 2024-10-10 --- # DNA wire protocol ## `Cursor` message Before explaining the DNA protocol in more detail, we're going to discuss the `Cursor` message type. This type is used by all methods discussed later and plays a central role in how DNA works. DNA models a blockchain as a sequence of blocks. The distance of a block from the first block in the chain (the genesis block) is known as chain height. The genesis block has height `0`. Ideally, a blockchain should always build a block on top of the most recent block, but that's not always the case. For this reason, a block's height isn't enough to uniquely identify a block in the blockchain. A _chain reorganization_ is when a chain produces blocks that are not building on top of the most recent block. As we will see later, the DNA protocol detects and handles chain reorganizations. A block that can't be part of a chain reorganization is _finalized_. DNA uses a _cursor_ to uniquely identify blocks on the chain. A cursor contains two fields: - `order_key`: the block's height. - `unique_key`: the block's unique identifier. Depending on the chain, it's the block hash or state root. ## `Status` method The `Status` method is used to retrieve the state of the DNA server. The request is an empty message. The response has the following fields: - `last_ingested`: returns the last block ingested by the server. This is the most recent block available for streaming. - `finalized`: the most recent finalized block. - `starting`: the first available block. Usually this is the genesis block, but DNA server operators can prune older nodes to save on storage space. ## `StreamData` method The `StreamData` method is used to start a DNA stream. It accepts a `StreamDataRequest` message and returns an infinite stream of `StreamDataResponse` messages. ### Request The request message is used to configure the stream. All fields except `filter` are optional. - `starting_cursor`: resume the stream from the provided cursor. The first block received in the stream will be the block following the provided cursor. If no cursor is provided, the stream will start from the genesis block. Notice that since `starting_cursor` is a cursor, the DNA server can detect if that block has been part of a chain's reorganization while the indexer was offline. - `finality`: the stream contains data with at least the specified finality. Possible values are _finalized_ (only receive finalized data), _accepted_ (receive finalized and non-finalized blocks), and _pending_ (receive finalized, non-finalized, and pending blocks). - `filter`: a non-empty list of chain-specific data filters. - `heartbeat_interval`: the stream will send an heartbeat message if there are no messages for the specified amount of time. This is useful to detect if the stream hangs. Value must be between 10 and 60 seconds. ### Response Once the server validates and accepts the request, it starts streaming data. Each stream message can be one of the following message types: - `data`: receive data about a block. - `invalidate`: the specified blocks don't belong to the canonical chain anymore because they were part of a chain reorganization. - `finalize`: the most recent finalized block moved forward. - `heartbeat`: an heartbeat message. - `system_message`: used to send messages from the server to the client. #### `Data` message Contains the requested data for a single block. All data messages cursors are monotonically increasing, unless an `Invalidate` message is received. The message contains the following fields: - `cursor`: the cursor of the block before this message. If the client reconnects using this cursor, the first message will be the same as this message. - `end_cursor`: this block's cursor. Reconnecting to the stream using this cursor will resume the stream. - `finality`: finality status of this block. - `production`: how the block was produced. Either `backfill` or `live`. - `data`: a list of encoded block data. Notice how the `data` field is a _list of block data_. This sounds counter-intuitive since the `Data` message contains data about a _single block_. The reason is that, as we've seen in the _"Request"_ section, the client can specify a list of filters. The `data` field has the same length as the request's `filters` field. In most cases, the client specifies a single filter and receives a single block of data. For advanced use cases (like tracking contracts deployed by a factory), the client uses multiple filters to have parallel streams of data synced on the block number. #### `Invalidate` message This message warns the client about a chain reorganization. It contains the following fields: - `cursor`: the new chain's head. All previously received messages where the `end_cursor.order_key` was greater than (`>`) this message `cursor.order_key` should be considered invalid/recalled. - `removed`: a list of cursors that used to belong to the canonical chain. #### `Finalize` message This message contains a single `cursor` field with the cursor of the most recent finalized block. All data at or before this block can't be part of a chain reorganization. This message is useful to prune old data. #### `Heartbeat` message This message is sent at regular intervals once the stream reaches the chain's head. Clients can detect if the stream hang by adding a timeout to the stream's _receive_ method. #### `SytemMessage` message This message is used by the server to send out-of-band messages to the client. It contains text messages such as data usage, warnings about reaching the free quota, or information about upcoming system upgrades. ## protobuf definition This section contains the protobuf definition used by the DNA server and clients. If you're implementing a new SDK for DNA, you can use this as the starting point. ```proto syntax = "proto3"; package dna.v2.stream; import "google/protobuf/duration.proto"; service DnaStream { // Stream data from the server. rpc StreamData(StreamDataRequest) returns (stream StreamDataResponse); // Get DNA server status. rpc Status(StatusRequest) returns (StatusResponse); } // A cursor over the stream content. message Cursor { // Key used for ordering messages in the stream. // // This is usually the block or slot number. uint64 order_key = 1; // Key used to discriminate branches in the stream. // // This is usually the hash of the block. bytes unique_key = 2; } // Request for the `Status` method. message StatusRequest {} // Response for the `Status` method. message StatusResponse { // The current head of the chain. Cursor current_head = 1; // The last cursor that was ingested by the node. Cursor last_ingested = 2; // The finalized block. Cursor finalized = 3; // The first block available. Cursor starting = 4; } // Request data to be streamed. message StreamDataRequest { // Cursor to start streaming from. // // If not specified, starts from the genesis block. // Use the data's message `end_cursor` field to resume streaming. optional Cursor starting_cursor = 1; // Return data with the specified finality. // // If not specified, defaults to `DATA_FINALITY_ACCEPTED`. optional DataFinality finality = 2; // Filters used to generate data. repeated bytes filter = 3; // Heartbeat interval. // // Value must be between 10 and 60 seconds. // If not specified, defaults to 30 seconds. optional google.protobuf.Duration heartbeat_interval = 4; } // Contains a piece of streamed data. message StreamDataResponse { oneof message { Data data = 1; Invalidate invalidate = 2; Finalize finalize = 3; Heartbeat heartbeat = 4; SystemMessage system_message = 5; } } // Invalidate data after the given cursor. message Invalidate { // The cursor of the new chain's head. // // All data after this cursor should be considered invalid. Cursor cursor = 1; // List of blocks that were removed from the chain. repeated Cursor removed = 2; } // Move the finalized block forward. message Finalize { // The cursor of the new finalized block. // // All data before this cursor cannot be invalidated. Cursor cursor = 1; } // A single block of data. // // If the request specified multiple filters, the `data` field will contain the // data for each filter in the same order as the filters were specified in the // request. // If no data is available for a filter, the corresponding data field will be // empty. message Data { // Cursor that generated this block of data. optional Cursor cursor = 1; // Block cursor. Use this cursor to resume the stream. Cursor end_cursor = 2; // The finality status of the block. DataFinality finality = 3; // The block data. // // This message contains chain-specific data serialized using protobuf. repeated bytes data = 4; // The production mode of the block. DataProduction production = 5; } // Sent to clients to check if stream is still connected. message Heartbeat {} // Message from the server to the client. message SystemMessage { oneof output { // Output to stdout. string stdout = 1; // Output to stderr. string stderr = 2; } } // Data finality. enum DataFinality { DATA_FINALITY_UNKNOWN = 0; // Data was received, but is not part of the canonical chain yet. DATA_FINALITY_PENDING = 1; // Data is now part of the canonical chain, but could still be invalidated. DATA_FINALITY_ACCEPTED = 2; // Data is finalized and cannot be invalidated. DATA_FINALITY_FINALIZED = 3; } // Data production mode. enum DataProduction { DATA_PRODUCTION_UNKNOWN = 0; // Data is for a backfilled block. DATA_PRODUCTION_BACKFILL = 1; // Data is for a live block. DATA_PRODUCTION_LIVE = 2; } ```