OpenEnv-banner # OpenEnv: Agentic Execution Environments An end-to-end framework for creating, deploying and using isolated execution environments for agentic RL, with a simple Gymnasium-style API.

PyPI License Docs Hugging Face Discord Open In Colab

## What you get - **One API for every environment.** `reset()`, `step()` and `state()`, sync or async, over a WebSocket. - **Isolated, deployable environments.** Each environment is a Docker image that runs locally, on a cloud sandbox, or as a [Hugging Face Space](https://huggingface.co/docs/openenv/getting_started/contributing-envs). - **40+ ready-to-use environments**, from games and coding sandboxes to browsers, finance and simulators, in the [environment catalog](https://huggingface.co/docs/openenv/environments). - **Train with your framework.** TRL, Unsloth, SkyRL, ART, Oumi, torchforge, Miles and more (see [Integrations](#integrations)). - **Train real coding agents.** The [Harbor integration](https://huggingface.co/docs/openenv/environments/harbor) runs Claude Code, Codex, OpenCode, mini-swe-agent and other harnesses on Harbor tasks, and captures the exact tokens for RL. - **Serve tools to agents.** MCP environments expose their tools to agents over `/mcp` in [production mode](https://huggingface.co/docs/openenv/guides/simulation-vs-production), while training keeps using `step()` and rewards. - **Rewards and evals built in.** Compose rewards with [rubrics](https://huggingface.co/docs/openenv/guides/rewards) and evaluate with [Inspect AI](https://huggingface.co/docs/openenv/tutorials/evaluation-inspect). ## Quick Start Install the OpenEnv package: ```bash pip install openenv ``` Install an environment client (e.g., Echo): ```bash pip install git+https://huggingface.co/spaces/openenv/echo_env ``` Then use the environment: ```python import asyncio from echo_env import CallToolAction, EchoEnv async def main(): # Connect to a running Space (async context manager) async with EchoEnv(base_url="https://openenv-echo-env.hf.space") as client: # Reset the environment result = await client.reset() print(result.observation.metadata["message"]) # "Echo environment ready!" # Send messages result = await client.step( CallToolAction( tool_name="echo_message", arguments={"message": "Hello, World!"}, ) ) print(result.observation.result["data"]) # "Hello, World!" print(result.reward) asyncio.run(main()) ``` **Synchronous usage** is also supported via the `.sync()` wrapper: ```python from echo_env import CallToolAction, EchoEnv # Use .sync() for synchronous context manager with EchoEnv(base_url="https://openenv-echo-env.hf.space").sync() as client: result = client.reset() result = client.step( CallToolAction( tool_name="echo_message", arguments={"message": "Hello, World!"}, ) ) print(result.observation.result["data"]) ``` For a detailed quick start, check out the [docs page](https://huggingface.co/docs/openenv/getting-started). ## Train an agent Any training framework that can call an environment can train on it. [Training with OpenEnv](https://huggingface.co/docs/openenv/guides/training) maps the ways to train and the frameworks that support each one. - **Environments as tools (white-box).** The trainer runs the multi-turn tool loop and the environment supplies the tools and the reward. Each framework under [Integrations](#integrations) has its own example. With TRL, `GRPOTrainer` takes an `environment_factory`: start with the [Wordle GRPO tutorial](https://huggingface.co/docs/openenv/tutorials/wordle-grpo) or [TRL's OpenEnv guide](https://huggingface.co/docs/trl/openenv). - **Real agent harnesses (loop-owning).** The agent runs its own loop, and OpenEnv's [Harbor integration](https://huggingface.co/docs/openenv/environments/harbor) captures every model call as a framework-neutral `TrainingTrace`. TRL's `AsyncGRPOTrainer` trains on those captures today: see [`examples/async_grpo_harbor`](https://github.com/huggingface/trl/tree/main/examples/async_grpo_harbor) and [The ultimate guide to multi-harness RL](https://huggingface.co/spaces/FineEnvs/multi-harness-rl). ## Build your own environment ```bash openenv init my_env # scaffold an environment openenv validate my_env --level static --skip-build # quick check against the OpenEnv contract openenv push my_env # deploy it to Hugging Face Spaces ``` For local development, run `openenv serve my_env` after installing its dependencies. This runs on the host and binds to `127.0.0.1`; use Docker or Spaces for isolated training and deployment. See [Your First Environment](https://huggingface.co/docs/openenv/guides/first-environment) and [Deploying an Environment](https://huggingface.co/docs/openenv/getting_started/environment-builder). `openenv import` wraps an existing environment from ORS/OpenReward or Verifiers, or an AgentEnv bundle. ## Environments A few to start with: | Environment | What it is | |---|---| | [Echo](https://huggingface.co/docs/openenv/environments/echo) | Minimal MCP environment, for learning the API and testing a deployment | | [Coding](https://huggingface.co/docs/openenv/environments/coding) | Sandboxed Python execution with stdout, stderr and exit codes | | [TextArena (Wordle and more)](https://huggingface.co/docs/openenv/environments/textarena) | Text games for multi-turn RL | | [OpenSpiel](https://huggingface.co/docs/openenv/environments/openspiel) | Board and card games from DeepMind's OpenSpiel | | [BrowserGym](https://huggingface.co/docs/openenv/environments/browsergym) | Web navigation tasks (MiniWoB++, WebArena, ...) | | [Harbor](https://huggingface.co/docs/openenv/environments/harbor) | Harbor task datasets through coding-agent harnesses, with token capture for training | Browse all of them in the [environment catalog](https://huggingface.co/docs/openenv/environments), or on the [OpenEnv Hub organization](https://huggingface.co/openenv). ## Integrations OpenEnv works with a growing ecosystem of RL frameworks and platforms. If your project supports OpenEnv, open a PR to add it here. | Framework | Example | |---|---| | ART | [ART integration](https://art.openpipe.ai/integrations/openenv-integration) | | Lightning AI | [Templates](https://lightning.ai/templates?section=featured&query=openenv) | | Miles | [Terminal-Bench-2 GRPO](https://github.com/radixark/miles/tree/main/examples/experimental/openenv) | | Oumi | [GRPO notebook](https://github.com/oumi-ai/oumi/blob/main/notebooks/Oumi%20-%20OpenEnv%20GRPO%20with%20trl.ipynb) | | SkyRL | [SkyRL example](https://skyrl.readthedocs.io/en/latest/examples/openenv.html) | | torchforge | [GRPO BlackJack](https://github.com/openenv/OpenEnv/tree/main/examples/grpo_blackjack) | | TorchTitan-RL (TitanRL) | [OpenEnv bridge and Muse Glimmer 30B chess recipe](examples/titanrl_openenv/) for GRPO/DAPO, with a CPU demo and tests | | TRL | [OpenEnv guide](https://huggingface.co/docs/trl/openenv) (GRPO with `environment_factory`, and harness training) | | Unsloth | [2048 with gpt-oss](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/OpenEnv_gpt_oss_(20B)_Reinforcement_Learning_2048_Game.ipynb) | ## Learn more - [Documentation](https://huggingface.co/docs/openenv): concepts, guides and tutorials - [Core Concepts](https://huggingface.co/docs/openenv/guides/concepts) and the [CLI reference](https://huggingface.co/docs/openenv/reference/cli) - [Tutorials](https://huggingface.co/docs/openenv/tutorials/index), and the [Zero to Hero tutorial](https://github.com/openenv/OpenEnv/tree/main/tutorial) from our GPU Mode lecture - [FineEnvs](https://github.com/adithya-s-k/FineEnvs): end-to-end projects that build, deploy and train against OpenEnv environments - [RFCs](https://github.com/openenv/OpenEnv/tree/main/rfcs), the proposals behind major changes - [Contributing](https://github.com/openenv/OpenEnv/blob/main/CONTRIBUTING.md): development setup, tests and the PR process > [!NOTE] > OpenEnv is in early development, so APIs may still change. Bug fixes are welcome. For larger changes, open or claim an issue first so the change can be discussed. ## Community Support & Acknowledgments OpenEnv is governed by a technical committee that coordinates project direction, major technical decisions, RFCs, and release planning through the public issue tracker, pull requests, and RFC process. Current committee members: Meta-PyTorch, Reflection, Unsloth, Modal, Prime Intellect, Nvidia, Mercor, Fleet AI, Microsoft, Hugging Face, RadixArk, and Nebius. The project is also supported by a broader community of organizations. If you would like to add your project or organization here, please open a pull request for maintainer review. Supporters include: [Meta-PyTorch](https://github.com/meta-pytorch), [Hugging Face](https://huggingface.co), [Scaler AI Labs](https://scalerailabs.com), [Patronus AI](https://patronus.ai), [Surge AI](https://surgehq.ai), [LastMile AI](https://www.lastmileai.dev), [Unsloth](https://unsloth.ai), [Reflection](https://reflection.ai), [vLLM](https://vllm.ai), [SkyRL](https://skyrl.readthedocs.io) (UC-Berkeley), [Lightning AI](https://lightning.ai), [Axolotl AI](https://github.com/axolotl-ai-cloud/axolotl), [Stanford Scaling Intelligence Lab](https://scalingintelligence.stanford.edu/), [Mithril](https://mithril.ai), [OpenMined](https://openmined.org/), [Fleet AI](https://fleetai.com), [Halluminate](https://halluminate.ai/), [Turing](https://www.turing.com/), [Scale AI](https://scale.com/), [Scorecard](https://www.scorecard.io/), [Snorkel AI](https://snorkel.ai/), [SGLang](https://github.com/sgl-project/sglang), [Miles](https://github.com/radixark/miles), [Nebius](https://nebius.com) And we'd also like to acknowledge the team at Farama Foundation as the OpenEnv API was heavily inspired by the work you all have done on Gymnasium. Cheers! ## License BSD 3-Clause License (see [LICENSE](./LICENSE) file)