# Safari SDK: the SDK for Google DeepMind Gemini Robotics models 🦓🦄🐘🐒🐍

## Disclaimer

This is not an officially supported Google product.

Safari SDK provides full lifecycle toolings necessary for using Gemini Robotics
models, including but not limited to, access checkpoint, serving a model,
evaluate the model on robot and in sim, upload data, finetuning the model,
download the finetuned checkpoint, etc. Most of the functionality requires you
to join Gemini Robotics Trusted Tester Program to use. See details in Gemini
Robotics [main page](https://deepmind.google/models/gemini-robotics/).

## Source Code

The source code can be found in
[GitHub](https://github.com/google-deepmind/gemini-robotics-sdk).

Note: Unless stated otherwise, the commands below assume that your working
directory is the root of your repository clone (ie, the same directory which
contains the `pyproject.toml` file).

### Code Structure

The safari-sdk is structured and distributed as a single Python pip package. It
contains both the Python source code and compiled C++ logging extensions.

## Installing the SDK

### Installing Pip Dependencies

The pyproject.toml file specifies the dependencies for the Safari SDK, including
allowed version ranges for some packages. The requirements.txt file specifies
exact versions and hashes of each of those dependencies; internal unit tests use
these versions, and installing them in a virtual environment before installing
the SDK is recommended to avoid failures due to upstream package changes:

```shell
pip install -r requirements.txt
```

### Updating Pip Dependencies

A script is provided which updates the requirements.txt file to the most recent
versions of dependencies which satisfy the constraints in the pyproject.toml
file. This allows those updates to be done at discrete times (rather than
whenever those updates are pushed to PyPi) and tested in isolation before being
pushed to all users. To run that script:

```shell
scripts/update_pip_dependencies.sh
```

### Installing from PyPi

The Safari SDK can be easily installed via PyPI.

```shell
pip install safari-sdk
```

### Installing from Source Code

The Safari SDK can also be installed from the source code.

```shell
pip install -e .[dev]
```

## Building the Wheel

To build a Python wheel, run the following command from the root of the
repository.

```shell
scripts/build_wheel.sh
```

This script will build a pip installable wheel for the Safari SDK, and print the
file's path to stdout.

## Model support

Safari SDK aims to support all models in the Gemini Robotics model series.

Trusted Testers can access the Gemini Robotics On Device model from SDK v2.4.1.

## Libraries

Libraries related to robot data logging is in `safari_sdk/logging`.

Libraries related to model inference and interface with model servers are in
`safari_sdk/model`.

Libraries and binary related to accessing model checkpoints, upload data and
request of model finetune can be found in `safari_sdk/flywheel`.

Examples, including robot and simulation evaluation of models are in
`examples/`. Aloha specific eval code are in `examples/aloha`.

## Flywheel CLI

The flywheel CLI is a convenient CLI tool available after installation of the
pip package. It provides a set of commands to interact with the Gemini Robotics
platform, such as training models, serving models, managing data, and
downloading artifacts.

To use the CLI

```
flywheel-cli <command> [--flags] [--flags]
```

Supported commands are:

*   `train`: Train a model. Requires specifying task ID, start date, and end
    date.
*   `serve`: Serve a model. Requires specifying the training job ID.
*   `list`: List available training jobs.
*   `list_serve`: List available serving jobs.
*   `data_stats`: Show data statistics available for training.
*   `download`: Download artifacts from a training job or a specific artifact
    ID.
*   `upload_data`: Upload data to the data ingestion service.
*   `version`: Show the version of the SDK.
*   `help`: Show this help message with all the available commands and flags.

## Agent

The Safari SDK includes a comprehensive agent framework for building interactive
robotics agents powered by Gemini models. See
[YouTube Video: Gemini Robotics 1.5: Using agentic capabilities](https://youtu.be/AMRxbIO04kQ?si=UFILQ9IOgfw7RTus).
The framework is located in `safari_sdk/agent/framework` and provides a modular
architecture for creating agents that can perceive their environment, reason
about tasks, and control robot hardware.

### Key Components

**Agents** (`safari_sdk/agent/framework/agents/`): Base agent classes that integrate
with the Gemini Live API to provide conversational interaction and tool use
capabilities.

**Embodiments** (`safari_sdk/agent/framework/embodiments/`): Hardware-specific
interfaces that connect agents to physical robot systems (e.g., Aloha robot).
Each embodiment provides tools for robot control.

**Tools** (`safari_sdk/agent/framework/tools/`): Modular capabilities that agents
can use, including:

*   Run instruction
*   Success detection
*   Scene description
*   etc.

**Event Bus** (`safari_sdk/agent/framework/event_bus/`): Asynchronous
publish-subscribe system for communication between agent components.

**Configuration** (`safari_sdk/agent/framework/config.py`): Centralized
configuration management using `AgentFrameworkConfig`, supporting both
programmatic configuration and flag-based setup.

### Aloha Agent Example

The `examples/aloha/agent/` directory contains agent implementations for the
Aloha robot platform.

The primary example is `simple_agent.py`, which provides a conversational agent
that can control the Aloha robot using natural language instructions.

To run the Aloha agent, use the provided `run.py` script:

```shell
python examples/aloha/agent/run.py --agent_name=simple_agent
```

The Aloha agent demonstrates integration of vision-based robot control,
multi-camera perception, and conversational interaction with Gemini models.

Alternatively, you build your own agent using the agent framework.

The codebase is still in active development. We will update our most updated
user guide with Trusted Testers of Gemini Robotics.