# EfficientViT Classification
## Datasets
ImageNet: https://www.image-net.org/
```python
Our code expects the ImageNet dataset directory to follow the following structure:
imagenet
├── train
├── val
```
## Pretrained EfficientViT Classification Models
Latency/Throughput is measured on NVIDIA Jetson Nano, NVIDIA Jetson AGX Orin, and NVIDIA A100 GPU with TensorRT, fp16. Data transfer time is included.
### ImageNet
All EfficientViT classification models are trained on ImageNet-1K with random initialization (300 epochs + 20 warmup epochs) using supervised learning. Please put the downloaded checkpoints under *${efficientvit_repo}/assets/checkpoints/efficientvit_cls/*
| Model | Resolution | ImageNet Top1 Acc | ImageNet Top5 Acc | Params | MACs | A100 Throughput | Checkpoint |
|----------------------|:----------:|:----------:|:---------:|:------------:|:---------:|:------------:|:------------:|
| EfficientNetV2-S | 384x384 | [83.9](https://github.com/google/automl/tree/master/efficientnetv2#2-pretrained-efficientnetv2-checkpoints) | - | 22M | 8.4G | 2869 image/s | - |
| EfficientNetV2-M | 480x480 | [85.2](https://github.com/google/automl/tree/master/efficientnetv2#2-pretrained-efficientnetv2-checkpoints) | - | 54M | 25G | 1160 image/s | - |
| |
| EfficientViT-L1 | 224x224 | 84.484 | 96.862 | 53M | 5.3G | 6207 image/s | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_l1_r224.pt) |
| |
| EfficientViT-L2 | 224x224 | 85.050 | 97.090 | 64M | 6.9G | 4998 image/s | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_l2_r224.pt) |
| EfficientViT-L2 | 256x256 | 85.366 | 97.216 | 64M | 9.1G | 3969 image/s | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_l2_r256.pt) |
| EfficientViT-L2 | 288x288 | 85.630 | 97.364 | 64M | 11G | 3102 image/s | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_l2_r288.pt) |
| EfficientViT-L2 | 320x320 | 85.734 | 97.438 | 64M | 14G | 2525 image/s | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_l2_r320.pt) |
| EfficientViT-L2 | 384x384 | 85.978 | 97.518 | 64M | 20G | 1784 image/s | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_l2_r384.pt) |
| |
| EfficientViT-L3 | 224x224 | 85.814 | 97.198 | 246M | 28G | 2081 image/s | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_l3_r224.pt) |
| EfficientViT-L3 | 256x256 | 85.938 | 97.318 | 246M | 36G | 1641 image/s | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_l3_r256.pt) |
| EfficientViT-L3 | 288x288 | 86.070 | 97.440 | 246M | 46G | 1276 image/s | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_l3_r288.pt) |
| EfficientViT-L3 | 320x320 | 86.230 | 97.474 | 246M | 56G | 1049 image/s | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_l3_r320.pt) |
| EfficientViT-L3 | 384x384 | 86.408 | 97.632 | 246M | 81G | 724 image/s | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_l3_r384.pt) |
EfficientViT B series
| Model | Resolution | ImageNet Top1 Acc | ImageNet Top5 Acc | Params | MACs | Jetson Nano (bs1) | Jetson Orin (bs1) | Checkpoint |
|----------------------|:----------:|:----------:|:---------:|:------------:|:---------:|:------------:|:------------:|:------------:|
| EfficientViT-B1 | 224x224 | 79.390 | 94.346 | 9.1M | 0.52G | 24.8ms | 1.48ms | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_b1_r224.pt) |
| EfficientViT-B1 | 256x256 | 79.918 | 94.704 | 9.1M | 0.68G | 28.5ms | 1.57ms | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_b1_r256.pt) |
| EfficientViT-B1 | 288x288 | 80.410 | 94.984 | 9.1M | 0.86G | 34.5ms | 1.82ms | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_b1_r288.pt) |
| |
| EfficientViT-B2 | 224x224 | 82.100 | 95.782 | 24M | 1.6G | 50.6ms | 2.63ms | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_b2_r224.pt) |
| EfficientViT-B2 | 256x256 | 82.698 | 96.096 | 24M | 2.1G | 58.5ms | 2.84ms | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_b2_r256.pt) |
| EfficientViT-B2 | 288x288 | 83.086 | 96.302 | 24M | 2.6G | 69.9ms | 3.30ms | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_b2_r288.pt) |
| |
| EfficientViT-B3 | 224x224 | 83.468 | 96.356 | 49M | 4.0G | 101ms | 4.36ms | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_b3_r224.pt) |
| EfficientViT-B3 | 256x256 | 83.806 | 96.514 | 49M | 5.2G | 120ms | 4.74ms | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_b3_r256.pt) |
| EfficientViT-B3 | 288x288 | 84.150 | 96.732 | 49M | 6.5G | 141ms | 5.63ms | [link](https://huggingface.co/han-cai/efficientvit-cls/resolve/main/efficientvit_b3_r288.pt) |
## Usage
```python
# classification
from efficientvit.cls_model_zoo import create_efficientvit_cls_model
model = create_efficientvit_cls_model(name="efficientvit-l3-r384", pretrained=True)
```
## Evaluation
Please run [eval_efficientvit_cls_model.py](eval_efficientvit_cls_model.py) to evaluate our models.
Examples: [classification](../../assets/eval_efficientvit_cls_model.sh)
## Export
### Onnx
To generate ONNX files, please refer to [onnx_export.py](../../assets/onnx_export.py).
Example:
```bash
python assets/onnx_export.py --export_path assets/export_models/efficientvit_cls_l3_r224.onnx --model efficientvit-l3 --resolution 224 224 --bs 1
```
### TFLite
To generate TFLite files, please refer to [tflite_export.py](../../assets/tflite_export.py).
Example:
```bash
python assets/tflite_export.py --export_path assets/export_models/efficientvit_cls_b3_r224.tflite --model efficientvit-b3 --resolution 224 224
```
## Training
Please refer to [train_efficientvit_cls_model.py](train_efficientvit_cls_model.py) for training models on imagenet.
### EfficientViT L Series
```bash
torchrun --nnodes 1 --nproc_per_node=8 \
python applications/efficientvit_cls/train_efficientvit_cls_model.py applications/efficientvit_cls/configs/imagenet/efficientvit_l1.yaml --amp bf16 \
--data_provider.data_dir ~/dataset/imagenet \
--path .exp/efficientvit_cls/imagenet/efficientvit_l1_r224/
```
```bash
torchrun --nnodes 1 --nproc_per_node=8 \
python applications/efficientvit_cls/train_efficientvit_cls_model.py applications/efficientvit_cls/configs/imagenet/efficientvit_l2.yaml --amp bf16 \
--data_provider.data_dir ~/dataset/imagenet \
--path .exp/efficientvit_cls/imagenet/efficientvit_l2_r224/
```
```bash
torchrun --nnodes 1 --nproc_per_node=8 \
python applications/efficientvit_cls/train_efficientvit_cls_model.py applications/efficientvit_cls/configs/imagenet/efficientvit_l3.yaml --amp bf16 \
--data_provider.data_dir ~/dataset/imagenet \
--path .exp/efficientvit_cls/imagenet/efficientvit_l3_r224/
```
### EfficientViT B Series
```bash
torchrun --nnodes 1 --nproc_per_node=8 \
python applications/efficientvit_cls/train_efficientvit_cls_model.py applications/efficientvit_cls/configs/imagenet/efficientvit_b1.yaml \
--data_provider.data_dir ~/dataset/imagenet \
--path .exp/efficientvit_cls/imagenet/efficientvit_b1_r224/
```
```bash
torchrun --nnodes 1 --nproc_per_node=8 \
python applications/efficientvit_cls/train_efficientvit_cls_model.py applications/efficientvit_cls/configs/imagenet/efficientvit_b1.yaml \
--data_provider.image_size "[128,160,192,224,256,288]" \
--data_provider.data_dir ~/dataset/imagenet \
--run_config.eval_image_size "[288]" \
--path .exp/efficientvit_cls/imagenet/efficientvit_b1_r288/
```
```bash
torchrun --nnodes 1 --nproc_per_node=8 \
python applications/efficientvit_cls/train_efficientvit_cls_model.py applications/efficientvit_cls/configs/imagenet/efficientvit_b2.yaml \
--data_provider.data_dir ~/dataset/imagenet \
--path .exp/efficientvit_cls/imagenet/efficientvit_b2_r224/
```
```bash
torchrun --nnodes 1 --nproc_per_node=8 \
python applications/efficientvit_cls/train_efficientvit_cls_model.py applications/efficientvit_cls/configs/imagenet/efficientvit_b2.yaml \
--data_provider.image_size "[128,160,192,224,256,288]" \
--data_provider.data_dir ~/dataset/imagenet \
--run_config.eval_image_size "[288]" \
--data_provider.data_aug "{n:1,m:5}" \
--path .exp/efficientvit_cls/imagenet/efficientvit_b2_r288/
```
```bash
torchrun --nnodes 1 --nproc_per_node=8 \
python applications/efficientvit_cls/train_efficientvit_cls_model.py applications/efficientvit_cls/configs/imagenet/efficientvit_b3.yaml \
--data_provider.data_dir ~/dataset/imagenet \
--path .exp/efficientvit_cls/imagenet/efficientvit_b3_r224/
```
## Reference
If EfficientViT is useful or relevant to your research, please kindly recognize our contributions by citing our paper:
```bibtex
@inproceedings{cai2023efficientvit,
title={Efficientvit: Lightweight multi-scale attention for high-resolution dense prediction},
author={Cai, Han and Li, Junyan and Hu, Muyan and Gan, Chuang and Han, Song},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
pages={17302--17313},
year={2023}
}
```