![SP1 Cluster](./.github/assets/header.png) [![Docker CI][docker-badge]][docker-url] [![Lint CI][lint-badge]][lint-url] [![Telegram Chat][tg-badge]][tg-url] The official multi-GPU proving service implementation for SP1, designed for the cloud and bare metal deployment. [docker-badge]: https://img.shields.io/github/actions/workflow/status/succinctlabs/sp1-cluster/docker.yml?branch=main [docker-url]: https://github.com/succinctlabs/sp1-cluster/actions/workflows/docker.yml [lint-badge]: https://img.shields.io/github/actions/workflow/status/succinctlabs/sp1-cluster/lint.yml?branch=main&label=lint [lint-url]: https://github.com/succinctlabs/sp1-cluster/actions/workflows/lint.yml [tg-badge]: https://img.shields.io/endpoint?color=neon&logo=telegram&label=chat&url=https%3A%2F%2Ftg.sumanjay.workers.dev%2F%2B9jkmKpjkJ1U5MTc5 [tg-url]: https://t.me/+9jkmKpjkJ1U5MTc5 [docs-badge]: https://img.shields.io/github/deployments/succinctlabs/sp1-cluster/Production?label=docs [docs-url]: https://cluster-docs.succinct.xyz
**Disclaimer.** Use of SP1 Cluster is also subject to the disclaimers and limitations of liability described in the [Legal / Disclaimer](https://docs.succinct.xyz/legal/disclaimer) page. ## Overview This project implements a high performance SP1 proving cluster that can scale to thousands of GPUs. The system consists of several key components: - **API**: REST API for interfacing with the cluster backed by Postgres - **Coordinator**: Central service that manages task and worker coordination - **CPU/GPU Node**: Distributed worker nodes that execute proof generation tasks - **Fulfiller**: Service that syncs the cluster API with assigned proofs from the SP1 Prover Network - **Bidder**: Service that bids on proofs from the SP1 Prover Network - **Network Gateway**: SDK-compatible gRPC/HTTP gateway that terminates the `sp1-sdk` network contract against a self-hosted cluster (see [bin/network-gateway/README.md](bin/network-gateway/README.md)) - **Artifact Store**: Redis or S3 store used for storing ephemeral proving data For detailed information on the architecture and deployment instructions, see the [docs](https://docs.succinct.xyz/docs/provers/introduction). ## Development To get started, you will need to have the following installed: * [Rust](https://www.rust-lang.org/tools/install) * [SP1](https://docs.succinct.xyz/docs/sp1/getting-started/install) ```bash git clone https://github.com/succinctlabs/sp1-cluster.git cd sp1-cluster # Build cargo build --release ``` ## Benchmarking Start `gpu0`. Compose also starts its API, coordinator, Redis, and Postgres dependencies: ```bash docker compose -f infra/docker-compose.yml up -d gpu0 ``` The CLI connects to the **API** service, not the coordinator. It must use the workers' artifact store: ```bash export CLI_CLUSTER_RPC=http://127.0.0.1:50051 export CLI_REDIS_NODES=redis://:redispassword@127.0.0.1:6379/0 # One 20M-cycle Fibonacci proof. ./target/release/sp1-cluster-cli bench fibonacci 20 # Four at once, for aggregate cluster throughput. ./target/release/sp1-cluster-cli bench fibonacci 20 --count 4 ``` Inside the Compose network, use service names: `CLI_CLUSTER_RPC=http://api:50051` and `CLI_REDIS_NODES=redis://:redispassword@redis:6379/0`. ## Deploying To deploy to a Kubernetes cluster, you can use the provided Helm chart in the `infra/charts/sp1-cluster` directory. ```bash cd infra/charts/sp1-cluster cp values-example.yaml values-testnet.yaml ``` Edit the values-testnet.yaml file to configure your cluster. ```bash helm install sp1-cluster . -f values-testnet.yaml ```