--port ` evaluates a running
[server](#openpi-compatible-serving) instead, on this machine or another. The
model flags belong to the server. At connection, the evaluator checks
`--precision` and any published image, state, and prompt field names against
`franka_libero`. A mismatch stops the run; unpublished fields are not checked.
Pass `--norm-stats /norm_stats.json` to `apxinf-robo serve`
when serving this checkpoint.
- Runs are resumable: completed task/trial rows in the JSONL ledger are skipped,
and the summary reports success rate alongside per-segment latency.
## Benchmark
Measure latency with a checkpoint:
```bash
python scripts/bench_pi05.py --model-dir --precision bf16 \
--layer l1,l2 --warmup 10 --samples 30 --out devlocal/benchmark/results.json
```
| Layer | Measurement |
|---|---|
| `l1` | Model inference from resized RGB, including the Python/Rust call |
| `l2` | Engine policy, including its preprocessing and postprocessing |
| `l3` | Client-to-server round trip, including the served policy and transport |
L2 excludes robot-specific processing added by `build_robot_policy`. To include
that processing, start a server with the required robot preset and measure L3:
```bash
python scripts/bench_pi05.py --layer l3 --precision bf16 \
--host 127.0.0.1 --port 8000 --out devlocal/benchmark/server.json
```
- `--random-weights` runs without a checkpoint. Use `--views`, `--image-size`,
`--action-horizon`, `--num-flow-steps`, and `--token-count` to set its workload.
Synthetic L2 uses a fixed-length tokenizer and identity normalization.
- `--action-horizon` also overrides a checkpoint's native horizon.
- `--calibration` applies to synthetic FP8 runs; checkpoint runs read
`calibration.json` from the checkpoint directory.
- `--warmup` and `--samples` default to 10 and 30. The command prints latency
statistics; `--out` also saves JSON. Compare runs with matching layers,
precision, input shapes, and sampling settings.
For LIBERO task success and per-segment latency, use [LIBERO evaluation](#libero-evaluation).
For one-step evaluation, see [Run warm-start with one-step](https://github.com/infinigence/ApxInf/blob/ba968f63c9820f7db368bea0ce17bb890aa90781/doc/run_warmstart_with_onestep.md).
## NVIDIA build environment
A complete CUDA toolkit is required: `nvcc`, CUDA headers and runtime, cuBLAS
and cuBLASLt development libraries, and NVTX (`libnvToolsExt` on Jetson,
`libnvtx3interop` on desktop CUDA). Also a C/C++ compiler, linker, `ar`, Git,
`pkg-config`, and Python 3. The CUDA kernels, CUTLASS, and FlashAttention
sources are vendored — no external checkout needed.
Install the driver and toolkit through the JetPack, DRIVE OS, or CUDA
distribution for the machine, then check `nvcc --version`. If CUDA does not live
at `/usr/local/cuda`, point `CUDA_PATH` at it.
| Device | Architecture | Validated toolkit |
|---|---:|---:|
| Jetson AGX Thor | `sm_110` | CUDA 13.0 |
| Thor-U | `sm_101` | CUDA 12.8 |
| Jetson AGX Orin | `sm_87` | CUDA 12.6, 13.2 |
| RTX 4090 | `sm_89` | CUDA 12.8 |
## Rust toolchain
```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"
rustup default stable
```
Built with Rust 1.95 and 1.96; no minimum supported version is declared.
## The engine submodule
Initialize or restore the engine version recorded by this checkout:
```bash
git submodule update --init --recursive
```
Record the Robo and engine revisions alongside benchmark or evaluation results:
```bash
git rev-parse HEAD
git -C apxinf rev-parse HEAD
git submodule status apxinf
```
If the output starts with `+`, the engine is checked out at a different commit
from the one recorded by this Robo checkout.
Use `git submodule update --remote` only for a deliberate engine upgrade; it
follows `main`, as configured in `.gitmodules`. Before submitting
an upgrade, review the gitlink diff and run `tests/test_parity.py` with
`APXINF_PARITY_CHECKPOINT` set to a compatible checkpoint.
If you use onestep with warm-start in APXInf-robo, you can cite our paper:
```bibtex
@article{li2026step,
title={STEP: Warm-started visuomotor policies with spatiotemporal consistency prediction},
author={Li, Jinhao and Cong, Yuxuan and Wang, Yingqiao and Xia, Hao and Huang, Shan and Zhang, Yijia and Xu, Ningyi and Dai, Guohao},
journal={arXiv preprint arXiv:2602.08245},
year={2026}
}
```
## Acknowledgement
The development of APXInf has been inspired by, and benefits from, the ideas and tooling of the broader open-source community.
In particular, we would like to thank the teams and contributors behind
[flashinfer](https://github.com/flashinfer-ai/flashinfer),
[FasterTransformer](https://github.com/NVIDIA/FasterTransformer),
[TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM),
[llama.cpp](https://github.com/ggml-org/llama.cpp),
[FlashAttention](https://github.com/Dao-AILab/flash-attention),
[FlashRT](https://github.com/flashrt-project/FlashRT/tree/main),
[vLLM](https://github.com/vllm-project/vllm),
[sgLang](https://github.com/sgl-project/sglang),
and if we have inadvertently missed your project or contribution,
please open an issue or a pull request so we can properly credit you.
## License
Apache 2.0. Vendored third-party components retain their own licenses.
## Community
Scan the QR Code to join our Wechat Group
