:toc:
:toclevels: 4

N/A : Benchmark not present in a round

X: Change in benchmark. Submission results can be compared across rounds when there has been no change in the benchmark

== Training

|===
|Model |0.5 |0.6 |0.7 |1.0 |1.1 |2.0 |2.1 |3.0 | 3.1 | 4.0 | 4.1 | 5.0 | 5.1 | 6.0 | 6.1
|ResNet-50 v1.5 |X 9+|X 5+|N/A
|SSD-ResNet34 |X 4+|X 10+|N/A
|RetinaNet-ResNeXt50 5+|N/A 8+|X 2+|N/A
|MaskRCNN |X 8+|X 6+|N/A
|NCF |X 14+|N/A
|NMT 3+|X 12+|N/A
|Transformer 3+|X 12+|N/A
|MiniGo 7+|X 8+|N/A
|DLRM 2+|N/A 5+|X 8+|N/A
|DLRM-dcnv2 7+|N/A 7+|X |N/A
|BERT 3+|N/A 9+|X 3+|N/A
|RNN-T 3+|N/A |X 5+|X 6+|N/A
|3D U-Net 3+|N/A 7+|X 5+|N/A
|GPT3 7+|N/A 4+|X 4+|N/A
|Stable Diffusionv2 8+|N/A 4+|X 3+|N/A
|LLama70B-LoRA 9+|N/A 6+|X
|RGAT 9+|N/A 4+|X 2+|N/A
|Llama3.1 405b 11+|N/A 3+|X |N/A
|Llama3.1 8b 12+|N/A 3+|X
|FLUX.1 12+|N/A 3+|X
|DeepSeekv3 13+|N/A |X |X
|GPT-OSS-20B 13+|N/A 2+|X
|Qwen3.5-397B-GRPO 14+|N/A |X
|DLRMv4-HSTU 14+|N/A |X
|===

Metric: Time-to-train (measured in minutes)

Note: v0.6 ResNet-50 v1.5, SSD-ResNet34, NMT increased accuracy targets, all v0.6 benchmarks changed initializition timing, and v0.7 MiniGo moved to 19x19 board

== HPC

|===
|Model |0.7 |1.0 |2.0 
|CosmoFlow |X |X |X 
|DeepCAM 2+|X |X 
|Open Catalyst |N/A |X |X
|===

Metrics: Time-to-train (measured in minutes) and throughput (weak scaling - measured in models/minute)

== Inference
Datacenter
|===

|Model | 0.5 | 0.7 | 1.0 | 1.1 | 2.0 | 2.1 | 3.0 | 3.1 | 4.0 | 4.1 | 5.0 | 5.1 | 6.0
|MobileNet-v1 |X 12+|N/A
|ResNet-50 v1.5 11+|X 2+|N/A
|SSD-MobileNets 8+| X 5+|N/A
|SSD-ResNet34 5+|X 8+|N/A 
|RetinaNet-ResNeXt50 5+|N/A 7+|X |N/A
|NMT |X 12+|N/A
|DLRM |N/A 6+|X 6+|N/A
|DLRM-v2 7+|N/A 5+|X |N/A
|DLRM-v3 12+|N/A 1+|X
|BERT |N/A 9+|X 3+|N/A
|RNN-T |N/A 8+|X 4+|N/A
|3D U-Net |N/A 10+|X 2+|N/A
|GPT-J 7+|N/A 4+|X 2+|N/A
|Llama2-70b 8+|N/A 5+|X
|Stable-diffusion-xl 8+|N/A 4+|X |N/A
|Mixtral-8x7b 9+|N/A 4+|X
|Llama 3.1 405B 10+|N/A 3+|X
|RGAT 10+|N/A 3+|X
|Llama 3.1 8B 11+|N/A 2+|X
|Whisper 11+|N/A 2+|X
|DeepSeek R1 11+|N/A 2+|X
|GPT-OSS-20B 12+|N/A 1+|X
|WAN2.2 12+|N/A 1+|X
|Qwen3 12+|N/A 1+|X

|===

Edge
|===
| Model | 0.5 | 0.7 | 1.0 | 1.1 | 2.0 | 2.1 | 3.0 | 3.1 | 4.0 | 4.1 | 5.0 | 5.1 | 6.0
| MobileNet-v1 | X 12+|N/A
| ResNet-50 v1.5 13+|X 
| SSD-MobileNets 8+|X 5+|N/A
| SSD-ResNet34 5+|X 8+|N/A
| RetinaNet-ResNeXt50 5+|N/A 7+|X |N/A
| NMT |X 12+|N/A
| DLRM|N/A 6+|X 7+|N/A
| DLRM-v2 7+|N/A |X 5+|N/A
| BERT|N/A 12+|X
| RNN-T|N/A 8+|X 4+|N/A
| 3D U-Net|N/A 12+|X
| GPT-J 7+|N/A 4+|X 2+|N/A 
| Stable-diffusion-xl 8+|N/A 5+|X 
| Automotive PointPainting 10+|N/A 3+|X
| Llama 3.1 8B 11+|N/A 2+|X
| Whisper 11+|N/A 2+|X
| Yolo11 12+|N/A 1+|X
|===

Metrics: Queries/second (server), Samples/second (offline),  Latency (measured in milliseconds) (single stream), Streams (multi-stream v0.5-v1.1), Latency (measured in milliseconds) (multi-stream 2.0+)

Additional power metrics: System power (measured in watts) (server and offline), system energy per stream (measured in joules) (single stream and multi-stream)

Note: Performance metrics for inference and power submissions are not comparable

Note: Multistream v0.5-v1.1 is not compatible with v2.0 and newer

Note: Inference over Network scenario introduced in v2.1

== Mobile

|===
|Model |0.7 |1.0 |1.1 |2.0 |2.1 |3.0
|MobileNetEdge 6+|X
|SSD-MobileNetsV2 |X 5+|N/A 
|MobileDET |N/A 5+|X 
|DeeplabV3 4+|X 2+|N/A
|MOSAIC 3+|N/A 3+|X
|MobileBERT 6+|X
|EDSR 5+|N/A |X
|===

Primary metrics: Latency (measured in milliseconds) (single stream), Samples/second (offline)

Note: Submission requires all benchmarks in single stream and MobileNetEdge in single stream and offline

== Storage

|===
|Model |0.5 |1.0 |2.0
|BERT |X 2+|N/A
|3D-Unet 3+|X 
|ResNet-50 v1.5 |N/A 2+|X 
|Cosmoflow |N/A 2+|X
|Llama 3 8B Checkpoints 2+|N/A |X
|Llama 3 70B Checkpoints 2+|N/A |X
|Llama 3 405B Checkpoints 2+|N/A |X
|Llama 3 1.2T Checkpoints 2+|N/A |X
|===

Metrics: Number of accelerators that can be supported without dropping below X% Accelerator Utilization (where X depends upon the model), Throughput (aggregate read bandwidth)

== Tiny

|===
|Model |0.5 |0.7 |1.0 
|MobileNetV1 2+|X |X
|ResNet-V1 2+|X* |X
|DSCNN 2+|X |X
|FC Autoencoder 2+|X |X
|===

Primary metric: Latency (measured in milliseconds)

Secondary metric: Energy per inference (measured in microjoules)

*Latency Compatible, not accuracy: v0.5 and v0.7 use the same model, but changed the evaluation set to improve balance.