# Changelog This document lists major updates which change UX and require adaptation. It should be sorted by date (more recent on top) and link to MRs which introduce the changes. ## July 2026 public sync updates (12.07.26) This sync expands the public scene catalog, improves rollout reliability and scheduling, and adds new telemetry, evaluation, and Slurm controls. ### Scene catalogs and artifact loading The public 26.04 catalog is available as the `public_2604` suite, containing 1,606 scenes after removing known-invalid artifacts. Scene-suite CSV rows now pin an exact artifact with `test_suite_id`, `scene_id`, and `uuid`; suite lookup no longer selects the newest artifact sharing a scene ID. Artifacts whose parquet vector map is stored under `fastmap/` are now supported. AlpaSim loads it after the preferred `map_data/` and `clipgt/` sources, aligns its height to the reconstruction ground mesh, and retains XODR as the final fallback. **Migration**: Add a matching `uuid` column to custom suite CSVs. Every `(scene_id, uuid)` pair must exist in the corresponding scene catalog. ### Runtime reliability and scheduling Failed rollouts are retried up to `runtime.max_rollout_retries` times, with a default of two retries. Renderer startup now waits for channel readiness and retries known transient NRE initialization failures. Invalid vector maps return the structured `ROLLOUT_ERROR_CODE_INVALID_SCENE_MAP` error code. Scene-affine renderer dispatch now spreads non-affine work across the least-loaded renderers and bounds scene replication. New controls include `max_renderers_per_scene`, `max_scenes_per_renderer`, and cache refresh timing. **Migration**: Replace a boolean override such as `runtime.scene_affine_dispatch=true` with `runtime.scene_affine_dispatch.enabled=true`. Public NRE releases without the loaded-scene introspection RPC should leave this disabled. ### Traffic simulation and evaluation Traffic simulation is explicitly disabled by default through `trafficsim=disabled`; select `trafficsim=catk` to enable CATK. CATK now batches requests, limits per-request CPU threading, and derives worker counts from the selected topology. Evaluation adds the `open_loop_collision` metric, which checks planned ego trajectories against recorded actors over a configurable horizon. Force-GT warmup exclusion is now anchored to the actual renderer handover, preventing warmup frames from leaking into aggregated metrics. Route generation can be disabled with `runtime.simulation_config.route_generator_type=NONE`. The public Alpamayo 1 driver can derive deterministic per-inference seeds from the session seed with `driver.model.force_determinism=true`. The low-speed MPC linearization also corrects two erroneous derivative terms. **Migration**: Custom CATK configs should use `catk.loader.prediction_steps` instead of `minimum_future_steps`. ### Telemetry and Slurm execution Per-process Prometheus exporters can now be scraped by an external Prometheus without starting the wizard-managed sidecar. Set `wizard.prometheus.start_prometheus=false`; discovery publication and exporter lifecycle remain active. Slurm telemetry adds GPU ownership metrics for joining AlpaSim processes with DCGM data, and the Grafana dashboard includes the new scheduler, GPU, process, and throughput panels. Slurm deployments can opt into CUDA MPS with `wizard.enable_mps=true` to allow co-located GPU services to execute kernels concurrently. This option fails fast for non-Slurm run methods. Slurm submission also supports automatic resubmission of timed-out jobs. ## Runtime telemetry with Prometheus and Grafana (30.06.26) AlpaSim runs now start Prometheus telemetry by default. The wizard allocates metrics ports, starts Prometheus support services, writes scrape configuration, and preserves the Prometheus TSDB under the run directory. Runs still generate `metrics_plot.png`. Grafana dashboard resources and a helper script are available for inspecting local or shared Prometheus file-SD targets: ```bash src/tools/scripts/start-prometheus-grafana.sh ``` Eval aggregation no longer reads Prometheus runtime metric summaries or adds runtime performance fields to driving metric outputs. Runs no longer generate `prometheus/runtime_metrics_summary.json`; query Prometheus or use Grafana for runtime performance analysis. ## SMART/CATK trafficsim integration (29.06.26) Added an integrated CATK-backed traffic simulation service. Use `trafficsim=catk` to run the SMART/CATK traffic predictor from the AlpaSim base image with model weights stored under `data/trafficsim-models`. ## Upgrade OSS renderer to NRE-GA 26.04 (11.06.26) Bumped the `base_config.yaml` renderer image to `nvcr.io/nvidia/nre/nre-ga:26.04` and repointed `defines.renderer_entrypoint` to the `/app/run` symlink (the old path was renamed in 26.04). ## Eval Modifications (05.06.26) Eval aggregation now writes `results-summary.json` with per-rollout pass/fail status, scene scores, failed rollout rows, and driver `drive` RPC latency from telemetry. Ground-truth scoring adds `progress_rel_to_total` and `gt_dist_traveled_m`; tune or disable scoring with `eval.scene_score`, and set `eval.allow_aggregation_with_failed_rollouts=true` to aggregate successful rollouts while preserving failed rollout records. ## May/June 2026 public sync updates (02.06.26) This sync focuses on renderer unification, video-model support, scene catalog updates, and runtime robustness improvements. ### Renderer and video-model workflow Wizard-managed renderer services now use the `renderer` service key for both the default NRE-backed renderer and video-model renderers. The runtime selects the active renderer with `runtime.renderer.kind`, and generated configs now use `runtime.endpoints.renderer` / `network.renderer` instead of `sensorsim`. Video-model renderer support is now built into the main runtime and wizard configuration. New public configs include `deploy=external_video_model`, `deploy=managed_flashdreams`, `+chunking=<8frame|12frame|16frame>`, and `docs/VIDEO_MODEL.md`. Docker Compose deployments now stop backing services when `runtime-0` exits. **Migration**: Replace `services.sensorsim` and `wizard.run_sim_services=["sensorsim", ...]` with `services.renderer` and `wizard.run_sim_services=["renderer", ...]`. Replace `physics/implemented_in_sensorsim.yaml` with `physics/implemented_in_renderer.yaml`. Replace custom `runtime.endpoints.sensorsim` / `network.sensorsim` entries with `renderer`. For one-shot Compose runs, use `docker compose up --exit-code-from runtime-0`. ### Scene catalog and scene loading The default public Hugging Face scene catalog now uses the 26.01 release and the `public_2601` suite. The previous 25.05 catalog remains available through `sim_scenes_2505.csv` / `sim_suites_2505.csv`. Runtime scene discovery now uses `scene_provider` configuration instead of passing artifact globs through the runtime CLI. **Migration**: Existing 25.05 cached artifacts are not reused by the new default. Pin `scenes.scenes_csv` / `scenes.suites_csv` to the legacy CSVs if you need the old suite. For direct `python -m alpasim_runtime.simulate` usage, remove `--usdz-glob` and configure `scene_provider.kind` plus `scene_provider.usdz.data_dir`. ### Runtime, daemon, and evaluation robustness The runtime daemon now exposes `RuntimeService.get_runtime_info` for client-side capacity and scene discovery. Startup logs now report connection progress while waiting for services. Closed-loop and RL workflows also gained `DriveResponse.terminate_session`, single-rollout `RolloutSpec.session_uuid`, explicit driver precondition errors, spawned evaluation workers, empty driver-response handling, and continuous min-distance eval scorers. **Migration**: Existing clients remain compatible. Daemon clients can call `get_runtime_info`, set `RolloutSpec.session_uuid` for precise single-rollout abort handling, or set `DriveResponse.terminate_session=true` for early episode termination. Eval configs can opt into `min_distance_to_obstacle_m` and `min_distance_to_lane_boundary_m`. ### Simulation timing and logging Rollout timing now starts from the first GT camera frame timestamp. Force-GT startup blends recorded GT trajectories into physics-derived trajectories, and `runtime.simulation_config.skip_driver_during_force_gt` can skip expensive driver policy queries during force-GT warmup. `alpasim_utils.asl_to_frames` now exports video-model RGB / HD-map streams and names output frames by end-of-frame timestamp. `alpasim_utils.print_asl` redacts large video-model image payloads. **Migration**: Remove custom `time_start_offset_us` and camera `first_frame_offset_us` settings. Update downstream scripts that parse frame filenames to expect end-of-frame timestamps. ### Developer workflow The `src/grpc` package now builds generated protobuf artifacts during package builds, so downstream Git installs no longer need a manual `compile-protos` step. `trajdata-alpasim` is pinned to a specific Git commit for consistent workspace resolution. **Migration**: Local proto development can still use `uv run compile-protos`. Re-run dependency sync after pulling this change. ## Wizard runtime server mode and run mode rename (28.04.26) Wizard run modes now distinguish one-shot runtime execution from long-running runtime daemon deployment. `wizard.run_mode=BATCH` has been renamed to `wizard.run_mode=ONESHOT`, and `wizard.run_mode=SERVER` starts the runtime as a gRPC server for request-scoped simulations. **Migration**: If you explicitly set `wizard.run_mode=BATCH`, replace it with `wizard.run_mode=ONESHOT`. The default behavior remains unchanged for standard one-shot execution workflows. ### External driver configuration changes External driver ownership is now expressed through `driver_source` config groups rather than deployment targets: * Default managed driver behavior remains unchanged. * `driver_source=external_static` uses configured external driver addresses, replacing the removed `deploy=local_external_driver` deployment target. * `driver_source=external_dynamic` supports per-request drivers via `SimulationRequest.available_drivers`. **Migration**: Replace `deploy=local_external_driver` with `deploy=local driver_source=external_static driver=manual`. ## Move evaluation to a separate thread (03.04.26) Runtime evaluation now runs in its own thread instead of inline in the simulation loop. This decouples eval latency from the simulation step, improving throughput when evaluation is expensive. ## Dependency fix: override `torchmetrics` pin (03.04.26) Added `torchmetrics>=1.8.2` to `override-dependencies` in the root `pyproject.toml` to resolve a conflict between upstream driver dependencies. ## Duplicate config detection across providers (01.04.26) The Hydra config discovery plugin now detects YAML files that exist at the same relative path in multiple config providers (e.g. both `wizard` and an installed plugin). Duplicate paths raise a `ValueError` at startup, preventing silent config shadowing. ## Rename driver configs: ar1 → alpamayo1, a15 → alpamayo1_5 (31.03.26) Driver config names, entry points, and `model_type` values now use explicit names instead of abbreviations: | Before | After | |--------|-------| | `driver=ar1` | `driver=alpamayo1` | | `driver=a15` | `driver=alpamayo1_5` | **Migration**: Replace `driver=ar1` with `driver=alpamayo1` and `driver=a15` with `driver=alpamayo1_5` in CLI invocations, SLURM scripts, and any custom configs that reference these drivers. ## Upgrade OSS sensorsim to NRE-GA 26.02 and unify entrypoint (30.03.26) The OSS sensorsim image has been upgraded from `docker.io/carlasimulator/nvidia-nurec-grpc:0.2.0` to `nvcr.io/nvidia/nre/nre-ga:26.02`. * The sensorsim entrypoint (`/app/run serve-grpc`) and all shared flags are now defined once in `base_config.yaml`. * New flag `--enable-editing-actors` added to the base sensorsim command, required by NRE 26.3 for render requests that include dynamic object updates. **Migration**: If you override `services.sensorsim.command` in a custom manifest, add `--enable-editing-actors` to the argument list. ## Config refactoring: three-axis composition, per-service images, unified exp/ group (30.03.26) ### Three-axis config model The wizard config is now composed from three required, independent axes instead of monolithic deploy configs: ```bash uv run alpasim_wizard deploy=local topology=1gpu driver=vavam wizard.log_dir=./out ``` | Group | Purpose | Examples | |-------|---------|----------| | `deploy=` | Where to run (filesystem, run method) | `local`, `local_external_driver` | | `topology=` | GPU layout, replicas, concurrency | `1gpu`, `2gpu`, `8gpu_64rollouts` | | `driver=` | Which driving model | `vavam`, `ar1`, `a15`, `manual` | All three are required. Omitting any prints a helpful error listing available options. ### Driver configs simplified Each driver config now includes its own runtime settings via the Hydra defaults list. Specify a single config instead of a list: | Before | After | |--------|-------| | `driver=[vavam,vavam_runtime_configs]` | `driver=vavam` | | `driver=[ar1,alpamayo_runtime_configs]` | `driver=ar1` | | `driver=[a15,alpamayo_runtime_configs]` | `driver=a15` | ### stable_manifest removed, images derived from pyproject.toml The `stable_manifest` config group (`oss.yaml`, `oss_gitlab.yaml`) has been removed. Its content has been merged into `base_config.yaml`: * Services built from the repo (driver, physics, controller, trafficsim, runtime) use `${defines.base_image}`, which reads the version from `pyproject.toml` at runtime via a `repo-version:` OmegaConf resolver. * The external sensorsim image (`nvcr.io/nvidia/nre/nre-ga:26.02`) is set directly in `base_config.yaml`. * A default OSS scene ID is now in `base_config.yaml`, so new users can run without specifying scenes. ### Runtime endpoint config moved to topology `runtime.nr_workers` and all `runtime.endpoints.*.n_concurrent_rollouts` values are now set by topology configs instead of `base_config.yaml`. Each topology preset defines capacity to match its GPU layout. `base_config.yaml` retains only behavioral settings (`do_shutdown`, `enable_autoresume`, etc.). ### Unified exp/ config group The scattered `model/`, `experiment/`, `sim/`, and `exp/` config directories have been consolidated under a single `exp/` group. Presets (e.g., `vavam_4hz`) moved to `exp/presets/`. ### New optional config groups New optional groups in `base_config.yaml` defaults allow overriding service-specific settings: * `controller=` — override controller config * `sensorsim=` — override NRE image * `trafficsim=` — override trafficsim config ### SLURM submit.sh changes * `submit.sh` no longer defaults to any deploy target. All three axes (`deploy=`, `topology=`, `driver=`) must be specified. * Early sanity check rejects submissions with missing required config groups before allocating SLURM resources. * Example: `sbatch submit.sh deploy=ord topology=8gpu_64rollouts driver=vavam` ### Breaking changes summary * `+deploy=` syntax is now `deploy=` (no `+` prefix). Same for `topology` and `driver`. * `driver=[,]` list syntax is now just `driver=`. * `cameras/wide_only_cam.yaml` removed (use `cameras/1cam.yaml`). * `stable_manifest` config group removed entirely. * Deleted monolithic deploy configs: `iad_oss`, `ord_oss`, `ord_oss_single`, `local_2gpus`, `iad` (OSS). Use `deploy= topology=` instead. * `runtime.nr_workers` and `runtime.endpoints.*` defaults removed from `base_config.yaml` (set by topology). * `defines.nre_cache_size` removed from `base_config.yaml` (set by topology). ## Alpamayo 1.5 driver support (24.03.26) [Alpamayo 1.5](https://github.com/NVlabs/alpamayo1.5) is now available as a driver (`model_type: a15`). Use `driver=a15` to run with the 10B model. * New `A15Model` driver with camera-index-aware inference and optional classifier-free guidance navigation (`use_classifier_free_guidance_nav: true`, ~60 GB VRAM). * AR1 and A1.5 now share a common `AlpamayoBaseModel` base class, reducing code duplication. * `planner_delay_us` now defaults to `0` everywhere; the legacy `alpamayo_runtime_configs` file (which set 200ms delay) has been removed. ## Make ~/.netrc optional for public users (17.03.26) References to `~/.netrc` in the Dockerfile and wizard's Docker Compose generation were unconditional, requiring all users to have the file. The Dockerfile now conditionally sets `NETRC` only when the secret is provided, and the wizard only includes the `netrc` secret in the compose config when `~/.netrc` exists on the host. ## Composable dependency management (12.03.26) The root `pyproject.toml` now exposes every workspace member as a named optional-dependency extra, enabling composable installs from the repo root. A bare `uv sync` installs nothing (avoiding heavy deps like torch by default). * `uv sync --extra wizard` — wizard and its transitive deps only * `uv sync --extra all` — all core packages * `source setup_local_env.sh` still works and installs all core packages (plugins must be added separately). See [Onboarding — Dependency management](docs/ONBOARDING.md#dependency-management) for details. ## Overridable Hydra config groups (12.03.26) Wizard config groups (e.g. `driver`, `deploy`) can now be extended by any installed package. Packages register an `alpasim.configs` entry point pointing to a Python package that contains YAML files, and the wizard automatically adds it to Hydra's search path at startup via `SearchPathPlugin`. * `model_type` in driver config is now a plain string (e.g. `"ar1"`, `"manual"`) instead of an enum. * The transfuser driver configs have been moved out of the wizard into the transfuser plugin — when installed, `driver=[transfuser,transfuser_runtime_configs]` resolves automatically. ## Plugin system (12.03.26) Alpasim is now extensible via Python [entry points](https://packaging.python.org/en/latest/specifications/entry-points/). Any installed package can register models, controllers, configs, or tools without modifying the core codebase. * New `alpasim-plugins` package (`src/plugins`) provides a `PluginRegistry` that discovers entry points lazily at runtime. * Driver models (ar1, transfuser, vam, manual) and controller MPCs (linear, nonlinear) are registered as entry points and resolved by name. * Run `uv run alpasim-info` to list all installed plugins. See [Plugin System](docs/PLUGIN_SYSTEM.md) for the full architecture, entry-point groups, and how to create new plugins. ## Runtime event-based simulation loop and config cleanup (10.03.26) - The runtime simulation loop is now event-based instead of a fixed sequential control-step loop. - `pose_reporting_interval_us` is the active pose-reporting setting; older `egopose_*` configuration naming has been removed from the active runtime path. - The active egomotion noise model path was removed, so configs and tooling should no longer expect `egomotion_noise` behavior in standard runtime execution. ## Runtime daemon mode for on-demand simulation (10.03.26) - The runtime can now run as a long-lived gRPC daemon that accepts simulation requests on demand. - The gRPC API changed: `RolloutSpec.random_seed` was replaced by `nr_rollouts`, structured rollout results are returned, and a `shut_down` RPC was added for graceful shutdown. - One-shot CLI execution still works, but now routes through the same daemon engine internally. ## NRE 26.02 update, compatibility matrix removal, and sensorsim worker scaling (10.03.26) - The manual scene artifact compatibility matrix was removed. Scene selection now treats newer NRE versions as backwards-compatible and chooses the newest available artifact per scene. - Sensorsim/NRE scaling now relies on internal workers (`--max-workers`) rather than multiple replicas per container in the common OSS deploy configs. - If you tune throughput, update your expectations for sensorsim capacity: `replicas_per_container` alone no longer tells the full story. ## Add Higher Frequecy Reporting (18.02.26) Added higher frequency pose/state information for when model updates are more sparse. Additionally, changed the way that the `HF_HOME` environment variable is handled to be more like the public repo. ## ARM64 support and unified SLURM submit script (17.02.26) * **ARM64 support**: AlpaSim can now run on aarch64 (DGX Spark, DGX Station, IPP5 GB300). Build with `docker build --secret id=netrc,src=$HOME/.netrc -t alpasim-base:arm64 .` and deploy with `+deploy=local_arm` (Docker Compose) or `+deploy=ipp5` (SLURM). * **Unified SLURM script**: `src/tools/run-on-slurm/` is the single entry point; previous per-site directories have been consolidated into `src/tools/run-on-slurm/submit.sh`. **Migration**: Update SLURM submit commands: - `cd src/tools/run-on-slurm && sbatch --account= --partition= submit.sh +deploy=ord_oss` - `cd src/tools/run-on-ipp5 && sbatch submit.sh` → `cd src/tools/run-on-slurm && sbatch --account= --partition= --gpus-per-node=4 submit.sh +deploy=ipp5` ## Output directory structure changes (03.02.26) The wizard output directory structure has been reorganized for clarity: * `./asl/` directory renamed to `./rollouts/` - contains rollout logs organized by scene and session * `0.asl` and `0.rclog` files renamed to `rollout.asl` and `rollout.rclog` * `./metrics/` directory renamed to `./telemetry/` - contains Prometheus telemetry data (not to be confused with evaluation metrics stored in rollouts) * Videos are now saved next to ASL files: `rollouts///