# NeRF-SLAM > Rosinol 2023 · [Paper](https://arxiv.org/abs/2210.13641) **One-line summary** — Coupled a dense monocular SLAM frontend (DROID-SLAM) with an Instant-NGP radiance-field backend, weighting the NeRF depth loss by the SLAM depth marginal covariances — real-time, geometrically and photometrically accurate dense monocular reconstruction. ## Problem The first neural-implicit SLAM systems (iMAP, NICE-SLAM) required RGB-D input, and radiance fields trained on photometric loss alone are prone to "floaters" — ghost geometry from bad initialisation or bad local minima; adding depth supervision removes it and speeds convergence. Dense monocular SLAM can supply that depth in real time, but its depth maps "are extremely noisy due to their density, since even textureless regions are given a depth value". NeRF-SLAM's insight: dense monocular SLAM provides exactly the right information to fit a NeRF in real time — accurate poses plus dense depth maps *with associated uncertainty*, so the map can trust each depth only as much as the estimator does. ## Method & architecture Two threads run in parallel on one GPU (RTX 2080 Ti, 11 GB, PyTorch + CUDA): **Tracking: dense SLAM with covariances.** DROID-SLAM computes dense optical flow $\mathbf{p}_{ij}$ between frame pairs with a RAFT-style ConvGRU that also outputs per-measurement weights $\mathbf{\Sigma}_{\mathbf{p}_{ij}}$, then solves a dense bundle adjustment with geometry parametrised as per-keyframe inverse depth maps. Linearising gives the block-sparse system $$H\mathbf{x}=\mathbf{b}, \quad \begin{bmatrix} C & E \\ E^{T} & P \end{bmatrix} \begin{bmatrix} \Delta\boldsymbol{\xi} \\ \Delta\mathbf{d} \end{bmatrix} = \begin{bmatrix} \mathbf{v} \\ \mathbf{w} \end{bmatrix},$$ with $C$ the camera block, $P$ the (diagonal) inverse-depth block, $E$ the camera/depth coupling, $\Delta\boldsymbol{\xi}$ the $SE(3)$ pose updates and $\Delta\mathbf{d}$ the per-pixel inverse-depth updates. The Schur complement gives the reduced camera matrix $H_T$, solved by Cholesky factorisation $H_T = LL^{T}$. Following Rosinol et al.'s probabilistic volumetric fusion (WACV 2022), the marginal covariances of depths and poses come from the same factorisation: $$\mathbf{\Sigma}_{\mathbf{d}} = P^{-1} + P^{-T}E^{T}\mathbf{\Sigma}_{\mathbf{T}}EP^{-1}, \qquad \mathbf{\Sigma}_{\mathbf{T}} = (LL^{T})^{-1}.$$ **Mapping: probabilistic volumetric NeRF.** An Instant-NGP hash-grid radiance field is trained on all keyframes (no sliding window) with the mapping loss, minimised over both poses $\mathbf{T}$ and network parameters $\Theta$: $$\mathcal{L}_{M}(\mathbf{T},\Theta) = \mathcal{L}_{\text{rgb}}(\mathbf{T},\Theta) + \lambda_{D}\,\mathcal{L}_{\text{D}}(\mathbf{T},\Theta), \qquad \lambda_D = 1.0,$$ where the depth loss is Mahalanobis-weighted by the tracking covariance, $$\mathcal{L}_{\text{D}}(\mathbf{T},\Theta) = \|D - D^{\star}(\mathbf{T},\Theta)\|^{2}_{\Sigma_{D}},$$ and $\mathcal{L}_{\text{rgb}} = \|I - I^{\star}(\mathbf{T},\Theta)\|^{2}$. Rendered depth is the expected ray-termination distance under standard volume rendering, $$d^{\star} = \sum_{i}\mathcal{T}_{i}\bigl(1-\exp(-\sigma_{i}\delta_{i})\bigr)d_{i}, \qquad \mathcal{T}_{i} = \exp\Bigl(-\sum_{j