# MPC Indexer Breakout This document outlines the design and efforts for breaking out the indexer into its own crate. ## Background ### Current State The MPC node relies heavily on the NEAR blockchain for coordination. It fetches the following information from chain: - pending signature and CKD requests (read from non-finalized block state) - MPC protocol state (read from contract-state) - Node migration instructions and data - Foreign Chain Transaction Data - TEE related information, such as the current docker image hashes of launcher and node Additionally, an MPC node writes data to chain, such as: - responses to signature and CKD requests - confirmations during critical operations such as key generation and resharing (key events) - its own TEE attestation - migration confirmation The MPC nodes monitor and interact with the NEAR blockchain by spawning a neard node in the same process (but in a different thread, with its own tokio runtime, c.f. [`spawn_real_indexer`](https://github.com/near/mpc/blob/dacc610b92b8ef4d80b389d86e450a3488ae72ed/crates/node/src/indexer/real.rs#L49). The part of the code that is responsible for spawning the neard node, fetching data from it and forwarding transactions to it, is what we currently refer to as _MPC Indexer_. Conceptually, the following graph depicts information flow: ```mermaid --- title: MPC Orchestration --- flowchart TB CORE[MPC Core] subgraph INDEXER[MPC Indexer] direction TB WRITE[ MPC State Write

Signature Responses CKD Responses TEE Attestation Migration Foreign Chain Transactions ] VIEW[ MPC State View

Protocol State Key Events Migration Info Foreign Chain Policy TEE Allowed Images Signature Requests CKD Requests ] end subgraph NEAR[NEAR Blockchain] direction LR subgraph MEMPOOL[NEAR Non-finalized blocks] direction LR STREAMER[ NEAR Block Stream

Non-finalized Transactions ] end subgraph CONTRACT[MPC Smart Contract] direction TB CONTRACT_WRITE[ Write Methods

Signature Responses respond

CKD Responses respond_ckd

Key Events start_keygen_instance vote_pk start_reshare_instance vote_reshared vote_abort_key_event_instance

Foreign Chain Transactions vote_foreign_chain_policy respond_verify_foreign_tx

TEE Attestation submit_participant_info verify_tee

Migration conclude_node_migration ] CONTRACT_VIEW[ Read Methods

Protocol State state

Migration Info migration_info

Foreign Chain Policy get_foreign_chain_policy

TEE Allowed Images allowed_docker_image_hashes allowed_launcher_compose_hashes get_tee_accounts ] end end CORE --> VIEW CORE --> WRITE %% Indexer --> NEAR Blockchain VIEW --> CONTRACT_VIEW VIEW -->STREAMER %%|Monitor Blocks for
Signature & CKD Requests| STREAMER WRITE --> CONTRACT_WRITE %% ------------------------ %% Styling %% ------------------------ classDef core stroke:#1b5e20,stroke-width:4px; classDef indexer stroke:#2563eb,stroke-width:4px; classDef near stroke:#7c3aed,stroke-width:4px; classDef contract stroke:#d97706,stroke-width:2px; classDef mempool stroke:#0f766e,stroke-width:2px; class CORE core; class INDEXER indexer; class NEAR near; class CONTRACT contract; class MEMPOOL mempool; ``` The MPC Indexer currently offers the following API: ```rust pub struct IndexerAPI { /// Provides the current contract state as well as updates to it. pub contract_state_receiver: watch::Receiver, /// Provides block updates (signature requests and other relevant receipts). /// It is in a mutex, because the logical "owner" of this receiver can /// change over time (specifically, when we transition from the Running /// state to a Resharing state to the Running state again, two different /// tasks would successively "own" the receiver). /// We do not want to re-create the channel, because while resharing is /// happening we want to buffer the signature requests. pub block_update_receiver: Arc>>, /// Handle to transaction processor. pub txn_sender: TransactionSender, /// Watcher that keeps track of allowed [`DockerImageHash`]es on the contract. pub allowed_docker_images_receiver: watch::Receiver>, /// Watcher that keeps track of allowed [`LauncherDockerComposeHash`]es on the contract. pub allowed_launcher_compose_receiver: watch::Receiver>, /// Watcher that tracks node IDs that have TEE attestations in the contract. pub attested_nodes_receiver: watch::Receiver>, pub my_migration_info_receiver: watch::Receiver, pub foreign_chain_policy_reader: ForeignChainPolicyReader, } ``` ##### Issues with the current design Currently, the MPC indexer tries to achieve two things: 1. Interacting with the neard node: - spawning the neard node - generating and forwarding of transactions - viewing contract state - monitoring blocks and filtering for relevant events (signature requests, ckd requests, ...) 2. Providing context to the MPC network: - Acting as an abstraction layer for the blockchain, ensuring no blockchain-internals are leaked to the node; - informing the MPC node about jobs such as resharings, signature requests, ckd requests, etc. - informing the MPC node about relevant TEE information such as allowed docker images etc. The main concern with our current implementation is the lack of separation between abstraction layer 1 (neard-node-abstraction) and abstraction layer 2 (MPC orchestration). We would like to re-use the chain-specific abstraction layer for our backup service implementation (c.f.[#1891](https://github.com/near/mpc/issues/1891)), as well as the HOT wallet TEE application (c.f. [#2062](https://github.com/near/mpc/issues/2062)), but this is not possible with the current design. Additionally, the second abstraction layer is not enforced coherently. Looking at the `IndexerAPI` above, it reads pretty much verbatim like the corresponding contract endpoints and the `TransactionSender` trait is too low-level. Overall, this interface makes testing the node non-trivial, as evidenced by our `FakeIndexer` implementation. ### Design Goals An improved indexer design should achieve the following goals: 1. Re-usability of components: - We have planned works for other applications that would greatly profit from one or multiple functionalities: - The TEE backup service will need to monitor the same MPC smart contract, requiring viewing state and submitting transactions (c.f.[#1891](https://github.com/near/mpc/issues/1891); - The HOT wallet TEE application will want to monitor a different smart contract, but require similar methods (viewing Attestations and docker image hashes, c.f. [#2062](https://github.com/near/mpc/issues/2062)); - We could leverage parts of the MPC indexer to monitor our production deployments; 2. Improved testing: - we currently have a `FakeIndexer` to isolate the core logic of our node. This mostly achieves its goal, but if we had a more mature interface, we could increase test coverage; 3. Isolation of nearcore internals: - This ties into point 1., but is worth mentioning as a stand-alone goal. The interface exposed by the neard node has experienced breaking changes in the past and due to the usage of nearcore internals in our code, we have no guarantee that this won't happen again. Since we will require this functionality in other applications, it makes sense to have a stand-alone crate that has an isolated dependency on the nearcore internals. This way, in case we do experience breaking changes, we only need to fix them once. ### Design Proposal We propose to split the two functionalities of the current indexer (MPC orchestration and Chain Indexing) into two separate components: The **Chain Gateway**: This component is responsible for: - spinning-up a neard node; - abstracting the neard indexer interface such that no nearcore internals are exposed. - providing a convenient interface to: - call arbitrary view methods on arbitrary contracts - subscribe to view methods of contracts - forward transactions to the NEAR blockchain - monitor non-finalized blocks for transactions matching a user-specified pattern; This is the first step and primary goal. This abstraction is a huge enabler for migrating the backup service into a TEE [(#1891)](https://github.com/near/mpc/issues/1891) and for our long-term support of legacy keys [(#2062)](https://github.com/near/mpc/issues/2062) As a secondary goal, we propose the **MPC Context**. This component is responsible for informing the MPC node about: - pending jobs (signature, CKD, foreign chain verification requests) - network state (peers) - protocol state (resharing) ```mermaid --- title: MPC Orchestration --- flowchart TB CORE[MPC Node Core] subgraph CONTEXT[MPC Context] direction TB WRITE[ MPC State Write ] VIEW[ MPC State View ] WRITE -.->|verify success| VIEW end subgraph CHAIN[Chain Gateway] direction TB TX_SUBSCRIBER[**Block Event Subscriber**

**Filters** non-finalized NEAR blocks for specific transactions **Returns** matching args in a stream ] CONTRACT_STATE_VIEWER[**Contract State Subscriber**

**Queries** view functions of smart contracts **Returns** the result to the MPC Context ] TX_SENDER[**Transaction Sender**

**Submits** transaction to the neard node **Returns** transaction hash ] subgraph NEARD[**Neard node**] direction TB BLOCK_STREAMER[**Streamer**] VIEW_CLIENT[**View Client**] RPC_HANDLER[**RPC Handler**] end end subgraph NEAR[NEAR Blockchain] direction TB subgraph MEMPOOL[NEAR Mempool] end subgraph CONTRACT[MPC Smart Contract] direction TB CONTRACT_VIEW[ Read Methods ] CONTRACT_WRITE[ Write Methods ] end end %% CORE --> Context CORE --> VIEW CORE --> WRITE %% Context --> Chain Gateway VIEW --> CONTRACT_STATE_VIEWER VIEW --> TX_SUBSCRIBER WRITE --> TX_SENDER %% Chain Gateway --> Neard Node TX_SUBSCRIBER --> BLOCK_STREAMER CONTRACT_STATE_VIEWER --> VIEW_CLIENT TX_SENDER --> RPC_HANDLER %% Neard --> Smart Contract RPC_HANDLER --> MEMPOOL MEMPOOL -.-> CONTRACT_WRITE VIEW_CLIENT --> CONTRACT_VIEW BLOCK_STREAMER --> MEMPOOL %% ------------------------ %% Styling %% ------------------------ classDef core stroke:#1b5e20,stroke-width:4px; classDef indexer stroke:#2563eb,stroke-width:4px; classDef near stroke:#7c3aed,stroke-width:4px; classDef contract stroke:#d97706,stroke-width:2px; classDef mempool stroke:#0f766e,stroke-width:2px; classDef chain stroke-width:2px; class CORE core; class CONTEXT indexer; class NEAR near; class CONTRACT contract; class MEMPOOL mempool; class CHAIN chain; ``` ### Crate Dependencies Below is a graph depicting dependencies of the envisioned crate dependencies as it relates to NEAR indexer functionality. ```mermaid --- title: MPC Dependencies --- flowchart TB subgraph SERVICES[MPC Services] direction TB subgraph MPC_NODE[MPC Node] direction TB CORE[MPC Node Core] CONTEXT[MPC Context] CORE --> CONTEXT end BACKUP_SERVICE[MPC Backup and Migration Service] HOT_SERVICE[HOT MPC Service] end subgraph CHAIN[Chain Gateway] direction TB TX_SUBSCRIBER[**Block Event Subscriber**

**Filters** non-finalized NEAR blocks for specific transactions **Returns** matching args in a stream ] CONTRACT_STATE_VIEWER[**Contract State Subscriber**

**Queries** view functions of smart contracts **Returns** stream for state ] TX_SENDER[**Transaction Sender**

**Submits** transaction to the neard node **Returns** transaction result ] end subgraph NEAR[NEAR Blockchain] end CHAIN --> NEAR CONTEXT --> CONTRACT_STATE_VIEWER CONTEXT --> TX_SENDER CONTEXT --> TX_SUBSCRIBER BACKUP_SERVICE --> CONTRACT_STATE_VIEWER BACKUP_SERVICE --> TX_SENDER HOT_SERVICE --> CONTRACT_STATE_VIEWER HOT_SERVICE --> TX_SENDER ``` ### API Proposal In this section, we propose API designs for the Chain Gateway and MPC Context. #### Chain Gateway The Chain Gateway provides three functionalities: - **State viewing:** allows to: - subscribe to arbitrary view methods on arbitrary contracts on the NEAR blockchain - query arbitrary view methods on arbitrary contracts on the NEAR blockchain - **Block Events:** Filter the mempool for transactions matching a specific pattern (receipient or executor id and method names). Receive a stream of all matching transactions. - **Transaction Sender:** send transactions to the NEAR blockchain. ##### State Viewer The Chain Gateway offers the following traits for viewing and subscribing to contract state: ```rust /// One-shot typed view call with JSON serialization/deserialization. pub trait ViewMethod: ViewRaw { async fn view( &self, contract_id: AccountId, method_name: &str, args: &Arg, ) -> Result, ChainGatewayError>; } /// Polls every 200ms; emits change only when returned bytes differ. pub trait SubscribeContractState: ViewRaw + Clone { async fn subscribe( &self, contract: AccountId, view_method: &str, ) -> impl WatchContractState + Send; } pub trait WatchContractState { /// Returns the last observed value and the block height at which it was observed. fn latest(&mut self) -> Result, ChainGatewayError>; /// Waits until the observed value changes. async fn changed(&mut self) -> Result<(), ChainGatewayError>; } pub struct ObservedState> { pub observed_at: BlockHeight, pub value: T, } /// Empty arguments for view calls that take no parameters. pub struct NoArgs {} pub struct BlockHeight(u64); ``` Note that the above traits derive from `ViewRaw`, which in turn derives from two low-level traits. ```rust /// Waits for sync then delegates to QueryViewFunction. /// Supertraits provide the raw RPC plumbing. pub trait ViewRaw: IsSyncing + QueryViewFunction { async fn view_raw( &self, contract_id: &AccountId, method_name: &str, args: &[u8], ) -> Result; } // queries the actual state pub trait QueryViewFunction: Send + Sync + 'static { async fn view_function_query( &self, contract_id: &AccountId, method_name: &str, args: &[u8], ) -> Result; } // returns true if the node is still syncing with the blockchain pub trait IsSyncing: Send + Sync + 'static { /// Returns whether the node is currently syncing. async fn is_syncing(&self) -> Result; } ``` ##### Block Event Subscriber The purpose of this interface is to enable easy subscription to block events. In the MPC node, we use this to monitor the mempool for requests to the MPC network and responses from the MPC network. Specifically, we filter for receipts that match one of the following pattern: - They are executed on a specific contract, call a specific method of that contract and successfully spawn a promise. We will call this `ExecutorFunctionCallSuccessWithPromise`: - in case of our MPC node, we are looking for any calls to `sign`, `request_app_private_key` or `verify_foreign_chain_transaction` of our MPC contract. - They are addressed to a specific contract and call a specific method of that contract. We call those `ReceiverFunctionCall`: - in the case of our MPC node, we are looking for calls to `return_signature_and_clean_state_on_success`, `return_ck_and_clean_state_on_success` or `return_verify_foreign_tx_and_clean_state_on_success` that originate from the contract. If we want this interface to be re-usable in other parts of our code, we can create a more or less generic filter interface: ```rust impl BlockEventSubscriptions { /// Create a new subscriber with the given channel buffer size. pub fn new(buffer_size: usize) -> Self; /// Add a subscription and get a unique identifier for it. /// Can be called multiple times before passing the subscriber to `ChainGateway::start()`. /// The returned identifier can be used to match returned events to the given subscription. pub fn subscribe(&mut self, filter: BlockEventSubscription) -> BlockEventId; } /// An identifier for a subscription, returned by `subscribe()`. pub struct BlockEventId(pub u64); pub enum BlockEventSubscription { /// Filter for events where a receipt outcome was executed by `transaction_outcome_executor_id` and called `method_name`. ExecutorFunctionCallSuccessWithPromise { transaction_outcome_executor_id: AccountId, method_name: String, }, /// Filter for events where a receipt was addressed to `receipt_receiver_id` and called `method_name`. ReceiverFunctionCall { receipt_receiver_id: AccountId, method_name: String, }, } ``` > **Note:** Block replay from a specific height (`SubscriptionReplay`) is planned but not yet implemented. Replay leverages the NEAR indexer's `sync_from_block_height` config option and comes essentially for free in the chain-gateway implementation — we just need to expose it through the `BlockEventSubscriptions` API. See [#236](https://github.com/near/mpc/issues/236). The subscriber is passed to `ChainGateway::start()`, which returns an `Option>`: ```rust let (chain_gateway, node_handle, block_update_receiver) = ChainGateway::start(indexer_config, Some(subscriber)).await?; ``` Example usage: ```rust let mut subscriber = BlockEventSubscriptions::new(100); let signature_requests_id = subscriber.subscribe( BlockEventSubscription::ExecutorFunctionCallSuccessWithPromise { transaction_outcome_executor_id: "v1.signer".parse()?, method_name: "sign".to_string(), } ); let ckd_request_id = subscriber.subscribe( BlockEventSubscription::ExecutorFunctionCallSuccessWithPromise { transaction_outcome_executor_id: "v1.signer".parse()?, method_name: "request_app_private_key".to_string(), } ); let (chain_gateway, node_handle, block_update_receiver) = ChainGateway::start(indexer_config, Some(subscriber)).await?; let mut block_stream_receiver = block_update_receiver.unwrap(); while let Some(update) = block_stream_receiver.recv().await { for matched in update.events { match matched.id { id if id == signature_requests_id => { /* handle signature request */ } id if id == ckd_request_id => { /* handle ckd request */ } _ => {} } } } ``` Specific types (c.f. [Appendix](#current-block-update) and `indexer/handler.rs` for justification). ```rust /// The BlockUpdate returned by the Chain indexer. Similar to the current `BlockUpdate` pub struct BlockUpdate { pub context: BlockContext, pub events: Vec, } /// Context for a single block pub struct BlockContext { pub hash: CryptoHash, pub height: BlockHeight, pub prev_hash: CryptoHash, pub last_final_block: CryptoHash, pub block_entropy: CryptoHash, pub block_timestamp_nanosec: u64, } pub struct MatchedEvent { /// Identifies which subscription matched this event. pub id: BlockEventId, /// Data associated with the event. pub event_data: EventData, } /// This can be extended if required. pub enum EventData { ExecutorFunctionCallSuccessWithPromise(ExecutorFunctionCallSuccessWithPromiseData), ReceiverFunctionCall(ReceiverFunctionCallData), } /// This event is associated to a transaction that matched a specific (transaction_outcome_executor_id: AccountId, method_name: String) pattern. struct ExecutorFunctionCallSuccessWithPromiseData { /// the receipt_id of the receipt this event came from receipt_id: CryptoHash, /// predecessor_id who signed the transaction predecessor_id: AccountId, /// the receipt that will hold the outcome of this receipt next_receipt_id: CryptoHash, /// raw bytes used for function call args_raw: Vec, } /// This event is associated to a transaction that matched a specific BlockEventSubscription. struct ReceiverFunctionCallData { /// the receipt id for the matched transaction receipt_id: CryptoHash, } ``` Note that this will be subject to changes in [#2680](https://github.com/near/mpc/issues/2680) ##### Transaction Sender We propose the following API for the transaction sender: ```rust /// Default impl fetches the latest final block, signs, and submits. pub trait SubmitFunctionCall: FetchLatestFinalBlockInfo + SubmitSignedTransaction { async fn submit_function_call_tx( &self, signer: Arc, receiver_id: AccountId, method_name: String, args: Vec, gas: Gas, ) -> Result; } ``` `TransactionSigner` handles nonce management and ED25519 signing: ```rust pub struct TransactionSigner { signing_key: SigningKey, account_id: AccountId, nonce: Mutex, } impl TransactionSigner { pub fn from_key(account_id: AccountId, signing_key: SigningKey) -> Self; pub fn public_key(&self) -> VerifyingKey; } ``` #### MPC Context TODO(#2138): handle in a separate discussion. This is not of priority right now. ## Appendix ### Current Block Update Currently, the MPC indexer monitors the mempool for transactions of interest to the MPC node. It does so by filtering all receipts for: 1. signature and CKD requests, as well as foreign transaction verifications. We do so by: 1. matching the `executor_id` of a receipt with the MPC contracts `AccountId`. This means that this receipt is executed by the MPC contract; 2. matching the method called in the contract to one of the expected methods (in the case of the MPC network, we are looking for calls to `sign`, `request_app_private_key` and `verify_foreign_transaction`). 3. if we have a match, we are interested in the following data: - receipt id - predecessor id - next receipt id - arguments for the function call (note: we currently deserialize the json and return a type). - block entropy - block timestamp 2. signature responses, CKD responses and foreign transaction verification responses. We do so by: 1. matching on the `receiver_id` of the receipt (must match the contract); 2. matching the method called in the contract to one of the expected values (`return_signature_and_clean_state_on_success`, `return_ck_and_clean_state_on_success` or `return_verify_foreign_tx_and_clean_state_on_success`, for the MPC contract). 3. In case we match, we track the following data: - receipt id (this will match the _next receipt id_ of the corresponding request from point 3 above) _Open Question: we should figure out if `receipt_id` and `executor_id` can be different from one another. If not, then we could simplify our design slightly_ For each block, the indexer composes a `BlockUpdate` for all of the above and sends that to the MPC node, together with some information about the block, such as: - block height - block hash - previous block hash ```rust /// This is a block update - containing all matched events for the latest block pub struct ChainBlockUpdate { pub block: BlockViewLite, pub signature_requests: Vec, pub completed_signatures: Vec, pub ckd_requests: Vec, pub completed_ckds: Vec, pub verify_foreign_tx_requests: Vec, pub completed_verify_foreign_txs: Vec, } pub struct BlockViewLite { pub hash: CryptoHash, pub height: u64, pub prev_hash: CryptoHash, pub last_final_block: CryptoHash, } ``` ## Related issues https://github.com/near/mpc/issues/1956 https://github.com/near/mpc/issues/592 https://github.com/near/mpc/issues/950 https://github.com/near/mpc/issues/913 https://github.com/near/mpc/issues/1187 https://github.com/near/mpc/issues/439 https://github.com/near/mpc/issues/1957 https://github.com/near/mpc/issues/155 https://github.com/near/mpc/issues/236 potentially: https://github.com/near/mpc/issues/1643