# F<sup>4</sup>Splat: Feed-Forward Predictive Densification for Feed-Forward 3D Gaussian Splatting

<h2 align="center">ECCV 2026</h2>

<p align="center">
  <a href="https://mlvlab.github.io/F4Splat/"><img src="https://img.shields.io/badge/Project-Page-2f80ed?style=for-the-badge" alt="Project Page"></a>
  <a href="https://arxiv.org/abs/2603.21304"><img src="https://img.shields.io/badge/arXiv-2603.21304-b31b1b?style=for-the-badge" alt="arXiv Paper"></a>
</p>

## 🔧 Installation

The project is tested with Python 3.11 and CUDA 12.8 builds of PyTorch.

```bash
conda create -y -n f4splat python=3.11
conda activate f4splat

pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128
pip install --no-build-isolation -r requirements.txt
```

If you use a different CUDA version, install the matching PyTorch wheels first, then install the remaining requirements.

## 📦 Pre-trained Checkpoints

We provide all Hugging Face Hub-format weights in one model repository.

| Checkpoint | Dataset | Context views | Subfolder |
| --- | --- | ---: | --- |
| F4Splat ACID 2-view | ACID | 2 | `acid-2view` |
| F4Splat RE10K 2-view | RealEstate10K | 2 | `re10k-2view` |
| F4Splat RE10K 24-view | RealEstate10K | 24 | `re10k-24view` |

Download one checkpoint:

```bash
huggingface-cli download Knowing/F4Splat \
    --repo-type model \
    --include "re10k-2view/*" \
    --local-dir hub/f4splat
```

Or download all released checkpoints:

```bash
huggingface-cli download Knowing/F4Splat \
    --repo-type model \
    --local-dir hub/f4splat
```

Load a downloaded checkpoint with `PyTorchModelHubMixin`:

```python
from src.model.f4splat_hub import F4SplatHubModel

model = F4SplatHubModel.from_pretrained("hub/f4splat/re10k-2view")
model = model.eval().cuda()
```

To load directly from Hugging Face, first snapshot the target subfolder:

```python
from pathlib import Path
from huggingface_hub import snapshot_download
from src.model.f4splat_hub import F4SplatHubModel

local_dir = snapshot_download(
    repo_id="Knowing/F4Splat",
    repo_type="model",
    allow_patterns=["re10k-2view/**"],
)
model = F4SplatHubModel.from_pretrained(str(Path(local_dir) / "re10k-2view"))
model = model.eval().cuda()
```

## 🚀 Demo

Run the local CUDA demo with the default RE10K 2-view Hub-format weight:

```bash
python demo/app.py
```

Or set the Hub-format weight explicitly:

```bash
F4SPLAT_MODEL_ID=Knowing/F4Splat F4SPLAT_MODEL_SUBFOLDER=re10k-2view python demo/app.py
```

To use a downloaded Hub-format folder:

```bash
F4SPLAT_MODEL_PATH=hub/f4splat F4SPLAT_MODEL_SUBFOLDER=re10k-2view python demo/app.py
```

The demo can also load a local raw Lightning `.ckpt` or tensor-only
`.safetensors` file for development:

```bash
F4SPLAT_MODEL_PATH=/path/to/checkpoint.ckpt python demo/app.py
```

The demo accepts 2-16 uploaded images, shows the preprocessed model inputs,
reports the available Gaussian budget range, and renders interpolation-only
videos for user-editable Gaussian budget counts. The default budgets correspond
to ratios 0.2, 0.4, 0.6, and 0.8. The video length is derived from the number
of uploaded views, using 24 frames per input interval and up to 192 frames by
default. Override this with `F4SPLAT_DEMO_FRAMES_PER_INPUT_INTERVAL` and
`F4SPLAT_DEMO_MAX_VIDEO_FRAMES` if needed. Each video column is labeled by the
actual Gaussian count in thousands and stacks the red Gaussian-location overlay
below the novel-view render.

## 🗂️ Data Preparation

F4Splat uses a preprocessed PyTorch chunk format similar to the layouts used by prior feed-forward NVS pipelines. Each split should contain chunk files and an `index.json`. Place datasets under `datasets/` by default:

```text
datasets/
├── re10k/
│   ├── train/
│   │   ├── 000000.torch
│   │   ├── ...
│   │   └── index.json
│   └── test/
│       ├── 000000.torch
│       ├── ...
│       └── index.json
├── dl3dv/
│   ├── train/
│   │   ├── 000000.torch
│   │   ├── ...
│   │   └── index.json
│   └── test/
│       ├── 000000.torch
│       ├── ...
│       └── index.json
└── acid/
    ├── train/
    │   ├── 000000.torch
    │   ├── ...
    │   └── index.json
    └── test/
        ├── 000000.torch
        ├── ...
        └── index.json
```

The main experiment configs are:

| Experiment | Config | Default Dataset Root |
| --- | --- | --- |
| RealEstate10K 2-view | `+experiment=re10k_2view` | `datasets/re10k` |
| RealEstate10K 24-view | `+experiment=re10k_24view` | `datasets/re10k` |
| ACID 2-view | `+experiment=acid_2view` | `datasets/acid` |

Dataset roots can be overridden from the command line:

```bash
python -m src.main +experiment=re10k_2view \
    dataset.re10k.roots='[/path/to/re10k]'
```

### References

Detailed upstream preparation notes are available in [NoPoSplat's dataset guide](https://github.com/cvg/NoPoSplat/blob/main/DATASETS.md). Please also follow the original dataset licenses and access instructions for [RealEstate10K](https://google.github.io/realestate10k/), [DL3DV](https://dl3dv-10k.github.io/DL3DV-10K/), and [ACID](https://infinite-nature.github.io/).

## 🏋️ Training

Use the provided scripts with a GPU id:

```bash
bash scripts/train/train_re10k_full_2view.sh 0
bash scripts/train/train_re10k_full_24view.sh 0
bash scripts/train/train_acid_full_2view.sh 0
bash scripts/train/train_re10k_dl3dv_full_2view.sh 0
```

Or call Hydra directly:

```bash
CUDA_VISIBLE_DEVICES=0 python -m src.main \
    +experiment=re10k_2view \
    wandb.mode=online \
    wandb.name=RE10k_FULL_2view
```

To disable Weights & Biases logging:

```bash
CUDA_VISIBLE_DEVICES=0 python -m src.main \
    +experiment=re10k_2view \
    wandb.mode=disabled
```

Checkpoints and logs are written under `outputs/` according to each experiment's Hydra run directory.

## 📊 Evaluation

Run test-time NVS evaluation with a checkpoint. The evaluation scripts take
the GPU id and checkpoint path as required arguments and write outputs under
`eval_results/` by default.

| Script | Checkpoint | Default output |
| --- | --- | --- |
| `scripts/eval/re10k_2view.sh` | RE10K 2-view checkpoint | `eval_results/re10k_2view` |
| `scripts/eval/acid_2view.sh` | ACID 2-view checkpoint | `eval_results/acid_2view` |
| `scripts/eval/re10k_Nview.sh` | RE10K 24-view checkpoint | `eval_results/re10k_24view/{8,16,24}view` |

```bash
bash scripts/eval/re10k_2view.sh 0 /path/to/checkpoint.ckpt
bash scripts/eval/acid_2view.sh 0 /path/to/checkpoint.ckpt
bash scripts/eval/re10k_Nview.sh 0 /path/to/re10k_24view_checkpoint.ckpt "8 16 24"
```

You can override the output path as the last argument:

```bash
bash scripts/eval/re10k_2view.sh 0 /path/to/checkpoint.ckpt eval_results/custom_re10k
bash scripts/eval/re10k_Nview.sh 0 /path/to/checkpoint.ckpt "8 16" eval_results/custom_re10k_24view
```

Available RealEstate10K evaluation indices include:

```text
assets/evaluation_index_re10k.json
assets/evaluation_index_re10k_ctx_8.json
assets/evaluation_index_re10k_ctx_16.json
assets/evaluation_index_re10k_ctx_24.json
```

## 📚 Citation

If you find our work helpful, please consider citing:

```bibtex
@misc{kim2026f4splat,
  title         = {F4Splat: Feed-Forward Predictive Densification for Feed-Forward 3D Gaussian Splatting},
  author        = {Kim, Injae and Kim, Chaehyeon and Bae, Minseong and Joo, Minseok and Kim, Hyunwoo J.},
  year          = {2026},
  eprint        = {2603.21304},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2603.21304}
}
```

## 🙏 Acknowledgement

We thank the authors of [VGGT](https://github.com/facebookresearch/vggt), [NoPoSplat](https://github.com/cvg/NoPoSplat), [AnySplat](https://github.com/InternRobotics/AnySplat), and [gsplat](https://github.com/nerfstudio-project/gsplat) for their excellent work and open-source contributions.