# Rust Package

The Rust package target is `sipp-rs`. It publishes the `sipp` library crate
for Rust applications and re-exports the high-level client API plus selected
runtime, backend, lifecycle, shard, provider, and gateway types.

`sipp-rs` depends on `sipp-sys`, the native llama.cpp FFI crate. Installing
`sipp-rs` from crates.io builds the native backend from source on the target
machine; it is not a binary wheel-style package.

See the [Library API Overview](../api) for the shared `add`, `query`,
`chat`, and `embed` contracts.

## Install

```bash
cargo add sipp-rs
```

The release workflow publishes `sipp-sys` first, then publishes `sipp-rs`.
Applications depend on the `sipp-rs` package and import the `sipp` crate.

## Build Requirements

Rust applications that depend on `sipp-rs` need the normal Rust toolchain plus
the native build tools used by `sipp-sys`:

- A C/C++ compiler for the target platform.
- CMake.
- Ninja or a compatible CMake generator.
- Platform SDKs required by the selected backend.

The CPU native backend is the baseline and does not require a Cargo feature.
Backend features add their own requirements:

- `cuda`: CUDA Toolkit plus a compatible NVIDIA driver.
- `metal`: macOS with Xcode command line tools.
- `vulkan`: Vulkan SDK or system Vulkan development libraries.
- `openmp`: OpenMP compiler/runtime support for the target platform.

## Use It For

- Rust applications that need local GGUF inference.
- Gateway-backed query, chat, and embedding calls.
- Direct provider descriptors behind the `providers` feature.
- Shared Sipp value types across application boundaries.

## Local GGUF Query

```rust
use sipp::{
    LocalDescriptor, LocalTextOptions, SippClient, SippQueryRequest,
    SippTextOptions,
};
use sipp::engine::{
    CacheRuntimeConfig, ContextRuntimeConfig, KvReuseMode, NativeRuntimeConfig,
    ObservabilityRuntimeConfig, SchedulerRuntimeConfig,
};

async fn run(
    model_path: std::path::PathBuf,
) -> Result<(), Box<dyn std::error::Error>> {
    let mut client = SippClient::new()?;
    let model = client.models().add([model_path]).await?;
    let mut descriptor = LocalDescriptor::new(model.id);
    descriptor.runtime = runtime_config();
    let endpoint = client.add("default", descriptor).await?;

    let response = client
        .query(SippQueryRequest {
            endpoint: Some(endpoint),
            prompt: "Explain Sipp in one sentence.".to_string(),
            options: SippTextOptions {
                max_tokens: Some(64),
                ..Default::default()
            },
            local: LocalTextOptions {
                context_key: Some("rust-local".to_string()),
                ..Default::default()
            },
            ..Default::default()
        })
        .await?;
    println!("{}", response.text);
    Ok(())
}

fn runtime_config() -> NativeRuntimeConfig {
    NativeRuntimeConfig {
        context: ContextRuntimeConfig {
            n_ctx: Some(2048),
            ..Default::default()
        },
        scheduler: SchedulerRuntimeConfig {
            continuous_batching: true,
            prefill_chunk_size: 0,
            ..Default::default()
        },
        cache: CacheRuntimeConfig {
            mode: KvReuseMode::LiveSlotPrefix,
            ..Default::default()
        },
        observability: ObservabilityRuntimeConfig {
            runtime_metrics: true,
            backend_profiling: false,
        },
        ..Default::default()
    }
}
```

See [Runtime Options](../reference/runtime-options.md) for the shared runtime
config groups and request option boundaries.

## Gateway Query

```rust
use sipp::{
    GatewayAuthentication, GatewayDescriptor, GatewayRoutes, GatewaySecret,
    GatewayTimeoutPolicy, SippClient, SippQueryRequest, SippTextOptions,
};

let mut client = SippClient::new()?;
let endpoint = client
    .add(
        "gateway",
        GatewayDescriptor {
            target: std::env::var("SIPP_GATEWAY_TARGET")?,
            base_url: std::env::var("SIPP_GATEWAY_URL")?,
            routes: GatewayRoutes::default(),
            authentication: GatewayAuthentication::Bearer(GatewaySecret::new(
                std::env::var("SIPP_GATEWAY_TOKEN")?,
            )),
            static_headers: Default::default(),
            timeouts: GatewayTimeoutPolicy::default(),
            protocol_options: Default::default(),
        },
    )
    .await?;

let response = client
    .query(SippQueryRequest {
        endpoint: Some(endpoint),
        prompt: "Explain gateway inference.".to_string(),
        options: SippTextOptions {
            max_tokens: Some(64),
            ..Default::default()
        },
        ..Default::default()
    })
    .await?;
println!("{}", response.text);
```

## Related Docs

- [Gateway Server](../gateway/server.md)
- [Gateway Toolkit](../gateway/toolkit.md)
- [Local Inference](../guides/local-inference.md)
- [Providers](../guides/providers.md)
- [Runtime Options](../reference/runtime-options.md)
- [Gateway And Hybrid Inference](../guides/gateway-hybrid.md)
- [Architecture](../architecture.md)
- [Maintainer source builds](../maintainers/source-builds.md)