Long-WAM: Scaling the Context of World-Action Models

Wei Huang1,3,*, Bohan Zhang2,*, Chenzhi Liu3, Isabella Liu1,4, Shuai Yang1, Weian Mao1, Luozhou Wang1, Yicheng Xiao3, Weifeng Lin1, Qixin Hu, Bryan Chu1, Sifei Liu1, Jim (Linxi) Fan1, Xiaojuan Qi3, Song Han1,2, Yukang Chen1

1NVIDIA    2MIT    3HKU    4UCSD
* Equal contribution.

Long-context world-action modeling with streaming visual memory.

Long-WAM paper on arXiv Official Long-WAM code Long-WAM project page Public checkpoints on Hugging Face Watch Long-WAM on YouTube Documentation: getting started Original code: Apache 2.0; see third-party notices

## Demo video

Watch the Long-WAM video: Scaling the Context of World-Action Models
▶ Watch the Long-WAM video on YouTube

## Overview Long-WAM connects long-video autoregressive modeling with robot action learning. It provides portable YAML configs, training/evaluation scripts for five benchmarks, and real-robot deployment interfaces. | Stage | What is included | Start here | | --- | --- | --- | | **1 · Robot video pretraining** | LongLive 2.0 Robot autoregressive training and fixed-checkpoint video validation | [Stage 1 guide](stage1/README.md) | | **2 · World-action learning** | Benchmark-specific training, streaming policy inference, and simulator evaluation | [Inference and evaluation](#inference-and-evaluation) | > See [Verification](docs/VERIFICATION.md) for tested paths and known > LIBERO/RoboTwin integration issues; full benchmark reproduction remains unverified. ## Contents - [Checkpoints & Downloads](#checkpoints) - [Quick Start & Installation](#quick-start) - [Inference & Evaluation](#inference-and-evaluation) - [Infra: Accelerated Inference & Robot Interfaces](#infra-accelerated-inference-and-robot-interfaces) - [Supported Hardware & Compilation](#supported-hardware-and-compilation) - [Install, Check & Run](#install-check-and-run) - [YAM, Franka & Unitree G1 Demos](#real-robot-deployment-demos) - [Data Preparation](#data-preparation) - [Training](#training) - [Benchmark Recipes](#benchmark-recipes) - [RoboCasa GR1 Context Scaling](#robocasa-gr1-context-scaling) - [Hardware Adaptation & Fine-tuning](#adapt-the-hardware-or-fine-tune-a-policy) - [AR Video: LongLive 2.0 Robot](#ar-video-longlive-20-robot) - [Image-to-video Inference](#image-to-video-inference) - [Agentic API + Long-WAM](#agentic-api--long-wam) - [Documentation & Repository Layout](#documentation) - [Paper & Citation](#paper-and-citation) - [License](#license) - [References & Acknowledgments](#references-and-acknowledgments) ## Checkpoints Public checkpoints are hosted by [Efficient-Large-Model (ELM)](https://huggingface.co/Efficient-Large-Model). **Download → inference → evaluation**; no retraining required. ### Robot policies | Model | Inference mode / use | Hugging Face | Download key | | --- | --- | --- | --- | | Long-WAM LIBERO | IDM | [Long-WAM-LIBERO-IDM](https://huggingface.co/Efficient-Large-Model/Long-WAM-LIBERO-IDM) | `libero_idm` | | Long-WAM LIBERO | COD | [Long-WAM-LIBERO-COD](https://huggingface.co/Efficient-Large-Model/Long-WAM-LIBERO-COD) | `libero_cod` | | Long-WAM RoboTwin 2.0 | IDM | [Long-WAM-RoboTwin2.0-IDM](https://huggingface.co/Efficient-Large-Model/Long-WAM-RoboTwin2.0-IDM) | `robotwin2_idm` | | Long-WAM RoboTwin 2.0 | COD | [Long-WAM-RoboTwin2.0-COD](https://huggingface.co/Efficient-Large-Model/Long-WAM-RoboTwin2.0-COD) | `robotwin2_cod` | | Long-WAM RoboCasa365 | 2.4-second context | [Long-WAM-RoboCasa365](https://huggingface.co/Efficient-Large-Model/Long-WAM-RoboCasa365) | `robocasa365` | | Long-WAM YAM | Bimanual YAM pretraining on ABC 130K | [Long-WAM-YAM](https://huggingface.co/Efficient-Large-Model/Long-WAM-YAM) | `yam` | **COD** uses `inference_mode=codenoise` with matching weights/config. YAM is a pretraining release; see its model card for the camera/action interface. ### RoboCasa GR1: context-length scaling All variants use 20-Hz control and a 16-step action chunk. | History | Past control steps | Clean latent frames | Hugging Face | Download key | | --- | ---: | ---: | --- | --- | | 0 s | P0 | 1 | [Long-WAM-RoboCasa-GR1-0s](https://huggingface.co/Efficient-Large-Model/Long-WAM-RoboCasa-GR1-0s) | `robocasa_gr1_p0` | | 2.4 s | P48 | 4 | [Long-WAM-RoboCasa-GR1-2.4s](https://huggingface.co/Efficient-Large-Model/Long-WAM-RoboCasa-GR1-2.4s) | `robocasa_gr1_p48` | | 4.8 s | P96 | 7 | [Long-WAM-RoboCasa-GR1-4.8s](https://huggingface.co/Efficient-Large-Model/Long-WAM-RoboCasa-GR1-4.8s) | `robocasa_gr1_p96` | | 9.6 s | P192 | 13 | [Long-WAM-RoboCasa-GR1-9.6s](https://huggingface.co/Efficient-Large-Model/Long-WAM-RoboCasa-GR1-9.6s) | `robocasa_gr1_p192` | | 19.2 s | P384 | 25 | [Long-WAM-RoboCasa-GR1-19.2s](https://huggingface.co/Efficient-Large-Model/Long-WAM-RoboCasa-GR1-19.2s) | `robocasa_gr1_p384` | The 19.2 s bundle was refreshed on November 9, 2026. P0 uses only the current frame; other variants add the listed history. Use matching weights and config. ### LongLive 2.0 Robot video models | Model | Training sequence lengths | Hugging Face | Download key | | --- | --- | --- | --- | | Robot-S | Short sequences, within a few seconds | [LongLive2.0-Robot-S](https://huggingface.co/Efficient-Large-Model/LongLive2.0-Robot-S) | `stage1_robot_s` | | Robot-M | Short and medium-length sequences | [LongLive2.0-Robot-M](https://huggingface.co/Efficient-Large-Model/LongLive2.0-Robot-M) | `stage1_robot_m` | | Robot-L | Long sequences; continued from Robot-M | [LongLive2.0-Robot-L](https://huggingface.co/Efficient-Large-Model/LongLive2.0-Robot-L) | `stage1_robot_l` | S/M/L denote sequence-length coverage, not model size. Robot-L is the final Stage-L checkpoint at optimizer step **3,972**. These are video generators; see the [Stage 1 guide](stage1/README.md). ### Download a bundle Use a download key above; revisions are pinned in [`configs/checkpoints.yaml`](configs/checkpoints.yaml). ```bash python scripts/download_checkpoint.py robocasa365 --output checkpoints/robocasa365 python scripts/download_checkpoint.py robocasa_gr1_p192 --output checkpoints/gr1-9.6s # Optional integrity check after downloading a bundle. (cd checkpoints/gr1-9.6s && sha256sum -c SHA256SUMS) ``` Policy bundles include `model.pt`, `config.yaml` and `dataset_stats.json`; keep them together. Video bundles include weights, a model card and checksums. VAE/text assets, GR1 text cache and simulators are installed separately. ## Quick start Download **only Long-WAM**; the other LongLive projects are not required: ```bash git clone --filter=blob:none --sparse --single-branch --branch main --depth 1 https://github.com/NVlabs/LongLive.git cd LongLive git sparse-checkout set Long-WAM cd Long-WAM ``` Partial clone fetches Git metadata and sparse checkout downloads this project's files, plus the small shared root files—not the other projects' source or assets. Run all commands below from **`LongLive/Long-WAM/`**, the independent project root. ### 1. Install the policy environment Use Python ≥3.10 and compatible CUDA PyTorch (policy target: 2.7.x; see [dependencies](pyproject.toml)). ```bash # Deployment environment: LIBERO/RoboTwin/Domino and LeRobot control interfaces. pip install -e '.[infra]' # Use a SEPARATE environment for training and its pinned data stack: # pip install -e '.[train,test]' # # RoboCasa GR1/365 socket serving without LeRobot only needs: pip install -e . export LONGWAM_DATA_ROOT="$PWD/data" export LONGWAM_CHECKPOINT_ROOT="$PWD/checkpoints" export LONGWAM_MODEL_ROOT="$PWD/models" export LONGWAM_OUTPUT_ROOT="$PWD/outputs" ``` Set the roots to your storage paths. Install simulators via the [benchmark setup guide](docs/BENCHMARKS.md). Stage 2 training and inference share **NumPy 2.2.x** (validated with 2.2.6). **Keep environments separate:** `.[infra]` uses `datasets` 4.x; `.[train]` pins 3.6.0. Stage 1 also has its own environment. GR1 and RoboCasa365 use incompatible `robocasa` packages; select each simulator with `simulator_python`. ### 2. Inspect a recipe without loading a model ```bash bash scripts/robocasa365/eval.sh --dry-run bash scripts/robocasa_gr1/train.sh context=p96 --dry-run ``` `--dry-run` checks configuration without loading models/data or using a GPU. ### 3. Evaluate a downloaded checkpoint After installing the simulator and model assets: ```bash python scripts/download_checkpoint.py robocasa365 --output checkpoints/robocasa365 bash scripts/robocasa365/eval.sh \ checkpoint=checkpoints/robocasa365/model.pt \ model_config=checkpoints/robocasa365/config.yaml \ stats=checkpoints/robocasa365/dataset_stats.json \ simulator_python=/path/to/robocasa-env/bin/python \ output_dir=outputs/robocasa365-eval ``` Use the bundle's **inference config**, your simulator path and a fresh `output_dir`. ## Inference and evaluation Edit [`configs/eval/`](configs/eval/) or pass `key=value` overrides. Add `--dry-run` to inspect settings; rollout videos are off by default.
LIBERO · IDM and COD (CodeDenoise) ```bash python scripts/download_checkpoint.py libero_idm --output checkpoints/libero-idm bash scripts/libero/eval.sh inference_mode=idm \ checkpoint=checkpoints/libero-idm/model.pt \ model_config=checkpoints/libero-idm/config.yaml \ stats=checkpoints/libero-idm/dataset_stats.json \ output_dir=outputs/libero-idm-eval ``` For COD, use `libero_cod` and `inference_mode=codenoise` with its matching bundle. Covers Spatial, Object, Goal and Long. Configuration: [`configs/eval/libero.yaml`](configs/eval/libero.yaml).
RoboTwin 2.0 · IDM and COD (CodeDenoise) ```bash python scripts/download_checkpoint.py robotwin2_cod --output checkpoints/robotwin2-cod bash scripts/robotwin2/eval.sh inference_mode=codenoise \ checkpoint=checkpoints/robotwin2-cod/model.pt \ model_config=checkpoints/robotwin2-cod/config.yaml \ stats=checkpoints/robotwin2-cod/dataset_stats.json \ simulator_root=/path/to/RoboTwin \ output_dir=outputs/robotwin-codenoise-eval ``` For IDM, use `robotwin2_idm` and `inference_mode=idm` with its matching bundle. Covers clean/randomized settings; follow the [simulator setup](docs/BENCHMARKS.md). Configuration: [`configs/eval/robotwin2.yaml`](configs/eval/robotwin2.yaml).
Domino · OOD and fine-tuned evaluation ```bash bash scripts/domino/eval.sh \ checkpoint=/path/to/policy.pt \ model_config=/path/to/resolved-config.yaml \ stats=/path/to/stats.json \ simulator_root=/path/to/DOMINO \ output_dir=outputs/domino-eval ``` OOD uses a RoboTwin policy without Domino fine-tuning; fine-tuned evaluation uses a Domino-trained policy. Compare within each setting using identical tasks/seeds/episodes. Reports SR/MS. Configuration: [`configs/eval/domino.yaml`](configs/eval/domino.yaml).
RoboCasa GR1 · History-specific checkpoints ```bash python scripts/download_checkpoint.py robocasa_gr1_p384 --output checkpoints/gr1-19.2s (cd checkpoints/gr1-19.2s && sha256sum -c SHA256SUMS) bash scripts/robocasa_gr1/eval.sh expected_history=384 \ checkpoint=checkpoints/gr1-19.2s/model.pt \ model_config=checkpoints/gr1-19.2s/config.yaml \ stats=checkpoints/gr1-19.2s/dataset_stats.json \ text_cache=/path/to/text_cache \ simulator_python=/path/to/gr1-env/bin/python \ output_dir=outputs/gr1-p384-eval ``` Set `expected_history` to `0`, `48`, `96`, `192` or `384` and use the matching bundle/text cache. This flag checks history; it does not resize memory. Configuration: [`configs/eval/robocasa_gr1.yaml`](configs/eval/robocasa_gr1.yaml).
RoboCasa365 · Target50 Use [Quick start](#3-evaluate-a-downloaded-checkpoint). Defaults: `task_set=target50 split=pretrain episodes=50 env_seed=7`. The [Agentic experiment](#agentic-api--long-wam) uses a separate protocol. Configuration: [`configs/eval/robocasa365.yaml`](configs/eval/robocasa365.yaml).
### Default evaluation protocols | Benchmark | Coverage | Seed / reset protocol | Action horizon / executed prefix | | --- | --- | --- | --- | | LIBERO | 4 suites × 10 tasks × 50 episodes | Seed 1 | 80 / 10 | | RoboTwin 2.0 | 50 tasks × clean/randomized × 100 episodes | Native clean/randomized protocol | 32 / 24 | | Domino | 35 level-1 tasks × 5 episodes | Seed 41 | 32 / 24 | | RoboCasa GR1 | 24 tasks × 10 episodes; 5 environments | Recorded unseeded resets | 16 / 16 | | RoboCasa365 | Target50 × 50 episodes; pretrain split | Environment seed 7 | 32 / 32 | These are project defaults; checkpoint-specific metadata takes precedence. LIBERO uses 20 action-denoising steps; others use 10. IDM adds 4 partial video-denoising steps. ### Denoising modes and streaming history | Mode | Inference behavior | Public benchmark variants | | --- | --- | --- | | **IDM** | Partial video denoising, then conditioned action denoising | Default benchmark recipes | | **CodeDenoise** | Joint video/action denoising | LIBERO and RoboTwin 2.0 | Match the checkpoint's denoising mode and context. `P` counts raw control steps (GR1: P48 = 2.4 s); startup history repeats the oldest observation. GR1/365 evaluation manages the policy server automatically. For custom clients, use [`scripts/infer.sh`](scripts/infer.sh) with the same bundle arguments. ## Infra: accelerated inference and robot interfaces `longwam.runtime` provides sync/async inference, streaming VAE and accelerated hardware backends. See the [infra guide](infra/README.md). ### Supported hardware and compilation | Device | GPU compute capability | Compile on the target device | Evaluation setting | | --- | --- | --- | --- | | NVIDIA GeForce RTX 4090 (opt-in BF16) | 8.9 | No native build | `hardware=rtx4090` | | NVIDIA GeForce RTX 5090 | 12.0 | `longwam build rtx5090` | `hardware=rtx5090` | | NVIDIA DGX Spark | 12.1 | `longwam build spark` | `hardware=spark` | | NVIDIA Jetson AGX Thor | 11.0 | `longwam build thor` | `hardware=thor` | The RTX4090 profile retains original BF16/FP32 dtypes and currently validates synchronous P48/H32/V4/A10 inference. See [its contract, tests and measurements](docs/RTX4090.md). RTX5090, Spark and Thor profiles use **NVFP4 video / BF16 actions and KV**. Build on the target device with compatible PyTorch, CUDA and C++ tools; follow the [platform instructions](infra/README.md#build), especially on Spark/Thor. ### Install, check and run ```bash # In a separate deployment environment: pip install -e '.[infra,native]' # Add --check-only to validate patches without compiling or requiring a GPU. longwam build rtx5090 --check-only longwam build spark --check-only longwam build thor --check-only # Inspect an async recipe without loading weights or starting a simulator. longwam eval libero --dry-run execution=async \ action_horizon=32 execute_steps=24 trigger_stride=16 streaming_vae=true ``` Remove `--check-only` to compile; select `hardware=` for inference. `hardware=reference` needs no native build. Python entry point: `from longwam.runtime import create_policy` (LeRobot `reset` / `select_action`). Async is opt-in; preserve the benchmark settings for reproduction. GR1/365 retain socket inference without these edge backends. Checks above validate configuration/patches, not device performance. ### Real-robot deployment demos YAM, Franka and G1 share `create_policy` and the existing infra backends. | G1 · Dynamic cup stacking (1×) | G1 · Moving-cup grasping (1×) | | --- | --- | | [![G1 stacks cups on a moving conveyor](assets/demos/g1_cup_stacking.gif)](https://nvlabs.github.io/LongLive/Long-WAM/#demos) | [![G1 grasps moving cups at four conveyor speeds](assets/demos/g1_speeds.gif)](https://nvlabs.github.io/LongLive/Long-WAM/#demos) | **YAM · Bowl stacking and brick sorting (2×)** [![YAM long-horizon bowl stacking and brick sorting](assets/demos/yam_long_horizon.gif)](https://nvlabs.github.io/LongLive/Long-WAM/#demos)
G1 runtime comparisons (1×) **Synchronous execution vs. accelerated infra** [![G1 synchronous execution compared with accelerated infrastructure](assets/demos/g1_sync_infra.gif)](https://nvlabs.github.io/LongLive/Long-WAM/#demos) **Synchronous (top) vs. asynchronous (bottom), across four speeds** [![G1 synchronous and asynchronous execution across four conveyor speeds](assets/demos/g1_sync_async.gif)](https://nvlabs.github.io/LongLive/Long-WAM/#demos)
Real-robot recordings from the [project page](https://nvlabs.github.io/LongLive/Long-WAM/#demos); click a GIF to watch on the project page. | Robot | Supported policy interface | Deployment configuration | Inspect without hardware | | --- | --- | --- | --- | | YAM | 14D bimanual state/action; top and two wrist cameras | [yam.yaml](configs/deploy/yam.yaml) | `longwam deploy yam --dry-run` | | Franka | 8D state with 7D Cartesian-delta/gripper or 8D joint/gripper actions | [franka.yaml](configs/deploy/franka.yaml) | `longwam deploy franka --dry-run` | | Unitree G1 + Dex1 | 16D dual-arm/three-camera or 8D right-arm/two-camera manipulation | [g1.yaml](configs/deploy/g1.yaml) | `longwam deploy g1 --dry-run` | **Train on your own robot/task data**, then supply matching weights, model config and statistics. Deployment checkpoint paths are **blank**; no public model is selected. Provide a calibrated driver (G1 includes an optional `ial_g1d` bridge wrapper). ```bash # Fill a copy of the robot YAML with your own policy bundle and calibrated driver. longwam deploy g1 --config /path/to/your-g1.yaml --dry-run # Read observations and run your policy, but do not arm or send robot commands. longwam deploy g1 --config /path/to/your-g1.yaml ``` Default: read-only inference, **no robot commands**. Motion requires explicit enablement, calibrated limits and a working emergency stop. These examples have offline tests, not new physical-robot validation. See the [deployment guide](infra/README.md#real-robot-deployment-demos). ## Data preparation Evaluation needs the policy bundle, model/simulator assets and released statistics; training demonstrations are not required. | Benchmark | Training data used by this release | Preparation | | --- | --- | --- | | LIBERO | Four LeRobot suites: Spatial, Object, Goal, Long | [Download and directory layout](docs/DATA.md#libero) | | RoboTwin 2.0 | Recorded RoboTwin 2.0 LeRobot release | [Download and archive extraction](docs/DATA.md#robotwin-20) | | Domino | Domino + RoboTwin replay, sampled 50:50 | [Subset selection and conversion](docs/DATA.md#domino) | | RoboCasa GR1 | Good Teleop release; 24 tasks / 24,000 demos at 20 Hz | [Pinned dataset and statistics](docs/DATA.md#robocasa-gr1-use-the-good-teleop-release) | | RoboCasa365 | Human300; 100 demos/task | [Download, selection and statistics](docs/DATA.md#robocasa365) | ```bash python scripts/download_data.py libero --output data/downloads/libero ``` Follow [`docs/DATA.md`](docs/DATA.md) for extraction, layouts, statistics and text caches. GR1 requires the good Teleop release; Domino requires prepared LeRobot data. ## Training Each benchmark has its own `train.sh` and `eval.sh`. Prepare data, text embeddings and initialization weights; select variants with `context=` or `denoising=`. Fresh benchmark training initializes the video expert from public [LongLive2.0-Robot-S](https://huggingface.co/Efficient-Large-Model/LongLive2.0-Robot-S), not general LongLive 2.0. Download it once: ```bash python scripts/download_checkpoint.py stage1_robot_s \ --output "${LONGWAM_CHECKPOINT_ROOT:-./checkpoints}/LongLive2.0-Robot-S" # LIBERO / RoboCasa: generate the ActionDiT initializer from Robot-S on CPU. python tools/weights/preprocess_action_dit_backbone.py ``` No separate ActionDiT download is needed. RoboTwin uses random ActionDiT initialization and does not require this conversion. See [initialization and fine-tuning](docs/TRAINING.md#initialization-fine-tuning-and-resume) for task-specific settings. ### Benchmark recipes | Benchmark / launcher | Default GPUs × batch/GPU × accumulation | Global batch | Peak LR | Training length | History / action chunk | | --- | --- | --- | --- | --- | --- | | [LIBERO](scripts/libero/train.sh) | 8 × 8 × 2 | 128 | 1e-4 | 10 epochs | P48 / 32 | | [RoboTwin 2.0](scripts/robotwin2/train.sh) | 8 × 16 × 8 | 1024 | 1e-4 | 5 epochs | P48 / 32 | | [Domino](scripts/domino/train.sh) | 8 × 16 × 8 | 1024 | 3e-6 | 300 updates | P48 / 32 | | [RoboCasa GR1](scripts/robocasa_gr1/train.sh) | 16 × 30 × 1; P768 below | 480 | 3e-5 | 30K updates | Selected P / 16 | | [RoboCasa365](scripts/robocasa365/train.sh) | 16 × 16 × 1 | 256 | 1e-4 | Fresh 100K updates | P48 / 32 | All recipes use BF16, seed 42 and gradient clipping 1.0. Exact schedules and initialization: [training guide](docs/TRAINING.md). ```bash # Inspect the 100K recipe. bash scripts/robocasa365/train.sh --dry-run # Start training after preparing the required data and initializers. bash scripts/robocasa365/train.sh output_dir=outputs/robocasa365-100k # LIBERO and RoboTwin support both training regimes. bash scripts/libero/train.sh denoising=idm --dry-run bash scripts/robotwin2/train.sh denoising=codenoise --dry-run ``` For Domino, set `LONGWAM_ROBOTWIN_CHECKPOINT` to the full RoboTwin initializer. ### RoboCasa GR1 context scaling All six contexts support training/inference with the same Teleop data, 30K updates, H16 action chunk and global batch 480. | Context | History at 20 Hz | Raw / sampled training frames | Batch/GPU × accumulation at 16 GPUs | | --- | --- | --- | --- | | `context=p0` | 0 s | 33 / 9 | 30 × 1 | | `context=p48` | 2.4 s | 81 / 21 | 30 × 1 | | `context=p96` | 4.8 s | 129 / 33 | 30 × 1 | | `context=p192` | 9.6 s | 225 / 57 | 30 × 1 | | `context=p384` **(default)** | 19.2 s | 417 / 105 | 30 × 1 | | `context=p768` | 38.4 s | 801 / 201 | 6 × 5 | Frames count the full training window. [Context configs](configs/context/) update both the data window and model. ```bash bash scripts/robocasa_gr1/train.sh context=p384 --dry-run # Robot-S video initialization is the default at every context length. bash scripts/robocasa_gr1/train.sh context=p192 \ output_dir=outputs/gr1-p192-robot ``` GR1 retains its own action architecture and mapped action-backbone initializer. ### Adapt the hardware or fine-tune a policy Keep global batch = GPU workers × batch/GPU × accumulation. For GR1 on four GPUs (global batch 480): ```bash NPROC_PER_NODE=4 bash scripts/robocasa_gr1/train.sh context=p96 --dry-run \ batch_size=15 gradient_accumulation_steps=8 ``` `resume=/path/to/weights.pt` starts a new optimization run from policy weights; a training-state directory restores optimizer/scheduler/data state. Use matching model/data configs; see [multi-GPU and resume details](docs/TRAINING.md). ## AR video: LongLive 2.0 Robot Based on [LongLive 2.0](https://github.com/NVlabs/LongLive/tree/main/LongLive2.0), with separate training and video-validation launchers in [`stage1/`](stage1/README.md). Use its own environment. Recorded training uses **128 workers**; validation uses 8 GPUs. | Shoe → container (1×) | Bun → steamer (1×) | Lid → pan (1×) | | --- | --- | --- | | [![Generated video of placing a shoe into a container](assets/demos/ar_shoe.gif)](https://nvlabs.github.io/LongLive/Long-WAM/#demos) | [![Generated video of putting a bun into a steamer](assets/demos/ar_bun.gif)](https://nvlabs.github.io/LongLive/Long-WAM/#demos) | [![Generated video of covering a frying pan](assets/demos/ar_pan.gif)](https://nvlabs.github.io/LongLive/Long-WAM/#demos) | | **Pour water (1×)** | **Fold clothing (2×)** | **Empty measuring cup (1×)** | | [![Generated video of pouring water into a bowl](assets/demos/ar_pour.gif)](https://nvlabs.github.io/LongLive/Long-WAM/#demos) | [![Generated video of folding clothing](assets/demos/ar_fold.gif)](https://nvlabs.github.io/LongLive/Long-WAM/#demos) | [![Generated video of emptying a measuring cup](assets/demos/ar_empty_cup.gif)](https://nvlabs.github.io/LongLive/Long-WAM/#demos) | Generated from an image and instruction, **not executed robot rollouts**. Click a GIF to watch on the [project page](https://nvlabs.github.io/LongLive/Long-WAM/#demos). ### Image-to-video inference Published Robot-S, Robot-M and Robot-L weights are listed [above](#checkpoints). The example command below uses Robot-S; see the [Stage 1 guide](stage1/README.md#published-checkpoints) for Robot-L downloads and its long-context validation settings. Seven RGB first frames and their original instructions are bundled: | `short_00` · Bun → steamer | `short_04` · Lid → pan | `long_00` · Shoe → container | | --- | --- | --- | | First-frame input: bun and steamer | First-frame input: frying pan | First-frame input: shoe and container | ```bash bash stage1/infer.sh --example all --dry-run bash stage1/infer.sh --example short_00 \ --checkpoint checkpoints/robot-s/model.pt --assets-root /path/to/video-assets \ --output-dir outputs/video-demo ``` Output: `outputs/video-demo/short_00.mp4`. Uses 50 steps, CFG 3 and 24 FPS. For installation, downloads and your own `--image` / `--prompt`, see the [inference guide](stage1/README.md#image-to-video-inference). Sample images: [RoVid-X, CC-BY-4.0](https://huggingface.co/datasets/DAGroup-PKU/RoVid-X). ### Video training and validation ```bash # CPU-only planning; does not launch distributed training or validation. export LONGWAM_VIDEO_DATA_ROOT=/path/to/robot-video-data export LONGWAM_VIDEO_INDEX_CACHE=/path/to/robot-video-index bash stage1/train.sh --stage S --output outputs/video-S \ --assets-root /path/to/video-assets --generator /path/to/initial-generator.pt --plan bash stage1/eval.sh --stage S --training-step 28602 \ --generator /path/to/S/model.pt --assets-root /path/to/video-assets \ --output outputs/video-S-validation --plan ``` `--plan` checks settings without launching jobs. For data, curriculum and real-run commands, see the [Stage 1 guide](stage1/README.md); weights are listed [above](#checkpoints). ## Agentic API + Long-WAM [`longwam.agentic`](src/longwam/agentic) provides a reusable model/tool runtime. The [RoboCasa365 adapter](src/longwam/benchmarks/robocasa/agentic) adds instruction decomposition and action review at inference time. Codex defaults: `gpt-6-astra` / `xhigh`; model and effort are configurable. ```bash bash scripts/robocasa365/eval.sh --agentic --dry-run # Select another exact Codex model ID without changing the framework: bash scripts/robocasa365/eval.sh --agentic --dry-run \ agentic.model=YOUR_EXACT_CODEX_MODEL_ID agentic.effort=high ``` Protocol: **50 tasks × 5 episodes**, pretrain split, seed 7; use 5 episodes/task for student-only comparisons. Live runs require model access, observation-sharing consent and a token budget. Dry-runs make no model calls. Authenticate with your own account/key; no credentials are included. See [configuration and new adapters](docs/AGENTIC.md). ## Documentation | Guide | Contents | | --- | --- | | [Benchmark setup](docs/BENCHMARKS.md) | Pinned simulators, compatibility patches and external prerequisites | | [Data preparation](docs/DATA.md) | Downloads, directory layouts, subsets, statistics and text caches | | [Training recipes](docs/TRAINING.md) | Exact parameters, GR1 scaling, initializers, resume and multi-GPU launch | | [Stage 1](stage1/README.md) | LongLive 2.0 Robot training and long-video validation | | [Agentic API](docs/AGENTIC.md) | Configurable Codex models, benchmark adapters and RoboCasa365 hybrid inference | | [Infra](infra/README.md) | Accelerated inference, device builds, async control and LeRobot/robot interfaces | | [Verification](docs/VERIFICATION.md) | CPU regression checks and outstanding end-to-end validation | | [Import notes](docs/MERGE.md) | Source provenance and intentionally retained functionality | ### Repository layout ```text Long-WAM/ ├── configs/ # Benchmark recipes, GR1 contexts and denoising modes ├── scripts/ # Shared helpers + one train/eval pair per benchmark ├── infra/ # Deployment guide and hardware/runtime integration ├── src/longwam/ # Models, training, data and benchmark adapters ├── stage1/ # LongLive 2.0 Robot curriculum and video validation ├── third_party/ # Attributed FourOverSix/CUTLASS build sources; no compiled artifacts ├── tools/ # Reusable data preparation and weight conversion ├── tests/ # CPU regression tests └── docs/ # Detailed setup and reproduction guides ``` ## Paper and citation [**Long-WAM: Scaling the Context of World-Action Models**](https://arxiv.org/abs/2610.10528) ```bibtex @misc{huang2026longwamscalingcontextworldaction, title={Long-WAM: Scaling the Context of World-Action Models}, author={Wei Huang and Bohan Zhang and Chenzhi Liu and Isabella Liu and Shuai Yang and Weian Mao and Luozhou Wang and Yicheng Xiao and Weifeng Lin and Qixin Hu and Bryan Chu and Sifei Liu and Linxi Fan and Xiaojuan Qi and Song Han and Yukang Chen}, year={2026}, eprint={2610.10528}, archivePrefix={arXiv}, primaryClass={cs.RO}, url={https://arxiv.org/abs/2610.10528}, } ``` ## License Long-WAM is released under [Apache 2.0](LICENSE). Third-party components retain their [original licenses](THIRD_PARTY_NOTICES.md). ## References and acknowledgments We thank [FastWAM](https://github.com/yuantianyuan01/FastWAM) for the foundation of parts of the model/data implementation. **Long-WAM is independently maintained** with its own interfaces and no runtime dependency on the FastWAM framework. We also acknowledge [LongLive 2.0](https://github.com/NVlabs/LongLive/tree/main/LongLive2.0), Wan, LeRobot, and the LIBERO, RoboTwin 2.0, Domino, RoboCasa GR1 and RoboCasa benchmark teams. External models, datasets and simulator assets retain their own terms.