Sapiens2

Scale. Semantics. Fidelity.

Rawal Khirodkar · He Wen · Julieta Martinez · Yuan Dong · Su Zhaoen · Shunsuke Saito

ICLR 2026

Project Page Paper PDF HuggingFace Hub

01 03

02 04

A family of high-resolution transformers pretrained on 1 billion human images, achieving state-of-the-art performance across diverse human-centric tasks — pose estimation, body-part segmentation, surface normals, pointmaps, and human matting.

🤗 Demos: Pose · Seg · Normal · Pointmap · Matting

## 📣 News - May 15, 2026: Sapiens2-1B human matting model is released. - April 24, 2026: Initial Sapiens2 release — pose, body-part segmentation, surface normals, and pointmaps. ## ⚡ Quick Start Run a pretrained backbone forward pass — only `torch` and `safetensors` needed: ```python import os import torch from safetensors.torch import load_file from sapiens.backbones.standalone.sapiens2 import Sapiens2 # Build the model and load a pretrained checkpoint model = Sapiens2(arch="sapiens2_1b", img_size=(1024, 768), patch_size=16).eval().cuda() # img_size is (H, W) ckpt = os.path.expanduser("~/sapiens2_host/pretrain/sapiens2_1b_pretrain.safetensors") model.load_state_dict(load_file(ckpt)) # Forward pass on a single image (RGB; ImageNet normalization recommended) x = torch.randn(1, 3, 1024, 768).cuda() with torch.no_grad(): features = model(x)[0] # dense backbone features ``` ## 🪶 Zero-Dependency Usage The Quick Start snippet above imports from a single self-contained file — `torch` (plus `safetensors` for checkpoint loading) is all you need. Drop the file into your project and you're done: ```bash curl -O https://raw.githubusercontent.com/facebookresearch/sapiens2/main/sapiens/backbones/standalone/sapiens2.py ``` For Sapiens v1, grab [`sapiens.py`](sapiens/backbones/standalone/sapiens.py) instead. ## 🧬 Model Card | Model | Params | FLOPs | Embed dim | Layers | Heads | |-------|--------|-------|-----------|--------|-------| | Sapiens2-0.1B | 0.114 B | 0.342 T | 768 | 12 | 12 | | Sapiens2-0.4B | 0.398 B | 1.260 T | 1024 | 24 | 16 | | Sapiens2-0.8B | 0.818 B | 2.592 T | 1280 | 32 | 16 | | Sapiens2-1B | 1.462 B | 4.715 T | 1536 | 40 | 24 | | Sapiens2-1B (4K) | 1.607 B | — | 1536 | 40 | 24 | | Sapiens2-5B | 5.071 B | 15.722 T | 2432 | 56 | 32 | All models use patch size 16 and are trained at 1024×768 (H×W) resolution, except Sapiens2-1B (4K) which is trained at 4096×3072 with `use_tokenizer=True`. ## 📦 Getting Started **Clone the repository:** ```bash git clone https://github.com/facebookresearch/sapiens2.git cd sapiens2 export SAPIENS_ROOT=$(pwd) ``` **Install** (requires Python ≥3.12 and PyTorch ≥2.7): ```bash pip install -e . ``` **Download checkpoints** from [MODEL_ZOO.md](docs/MODEL_ZOO.md). Place downloaded files under `$SAPIENS_CHECKPOINT_ROOT` (default: `~/sapiens2_host`): ```plaintext sapiens2_host/ ├── pretrain/ │ ├── sapiens2_{0.1b,0.4b,0.8b,1b,5b}_pretrain.safetensors │ └── sapiens2_1b_4k_pretrain.safetensors ├── pose/ │ └── sapiens2_{0.4b,0.8b,1b,5b}_pose.safetensors ├── seg/ │ └── sapiens2_{0.4b,0.8b,1b,5b}_seg.safetensors ├── normal/ │ └── sapiens2_{0.4b,0.8b,1b,5b}_normal.safetensors ├── pointmap/ │ └── sapiens2_{0.4b,0.8b,1b,5b}_pointmap.safetensors ├── matting/ │ └── sapiens2_1b_matting.safetensors └── detector/ # [optional] only needed for pose inference └── detr-resnet-101-dc5/ ``` ## 🎯 Vision Tasks | Task | Description | Inference | Train | |------|-------------|-----------|-------| | Pose Estimation | 308 whole-body keypoints | [docs/POSE.md](docs/POSE.md) | [docs/train/POSE.md](docs/train/POSE.md) | | Body-Part Segmentation | 29 body parts | [docs/SEG.md](docs/SEG.md) | [docs/train/SEG.md](docs/train/SEG.md) | | Surface Normal Estimation | per-pixel normals | [docs/NORMAL.md](docs/NORMAL.md) | [docs/train/NORMAL.md](docs/train/NORMAL.md) | | Pointmap Estimation | per-pixel 3D points | [docs/POINTMAP.md](docs/POINTMAP.md) | [docs/train/POINTMAP.md](docs/train/POINTMAP.md) | | Human Matting | alpha matte + foreground | [docs/MATTING.md](docs/MATTING.md) | [docs/train/MATTING.md](docs/train/MATTING.md) | ## ✨ Acknowledgements We would like to acknowledge the contributions of [DINOv3](https://github.com/facebookresearch/dinov3), [OpenMMLab](https://github.com/open-mmlab), and [Accelerate](https://github.com/huggingface/accelerate), which this project benefits from. ## 🤝 Contributing For questions or issues, please open an issue on GitHub. See [CONTRIBUTING](CONTRIBUTING.md) and the [Code of Conduct](CODE_OF_CONDUCT.md). ## License This project is licensed under the [Sapiens2 License](LICENSE.md). ## 📚 Citation If you use Sapiens2 in your research, please consider citing us. ```bibtex @article{khirodkarsapiens2, title={Sapiens2}, author={Khirodkar, Rawal and Wen, He and Martinez, Julieta and Dong, Yuan and Su, Zhaoen and Saito, Shunsuke}, journal={arXiv preprint arXiv:2604.21681}, year={2026} } ```