# BARF > Lin 2021 · [Paper](https://arxiv.org/abs/2104.06405) **One-line summary** — Bundle-Adjusting NeRF: jointly optimises a NeRF scene representation *and* the camera poses from imperfect or unknown initialisation, using a coarse-to-fine positional encoding schedule — the key enabling insight for NeRF-based SLAM. ## Problem NeRF synthesises photorealistic novel views, but it has a hard prerequisite: accurate camera poses for every training image, typically pre-computed with an SfM package. When poses are noisy or unknown, naive pose optimisation with NeRF "is sensitive to initialization" and "may lead to suboptimal solutions of the 3D scene representation". Reconstruction and registration form a chicken-and-egg problem: recovering 3D structure requires known poses, while localizing requires reliable correspondences from the reconstruction. BARF tackles training NeRF from imperfect (or even unknown) camera poses — the joint problem of learning neural 3D representations and registering camera frames — as a form of photometric bundle adjustment with view synthesis as the proxy objective. ## Method & architecture BARF first analyses 2D image alignment: registering images by gradient descent on $\min_{\mathbf{p}}\sum_{\mathbf{x}}\|\mathcal{I}_1(\mathcal{W}(\mathbf{x};\mathbf{p}))-\mathcal{I}_2(\mathbf{x})\|_2^2$ works only if the "steepest descent image" (the Jacobian chaining image gradients through the warp) gives *coherent* per-pixel updates — which is why classical Lucas-Kanade alignment blurs images coarse-to-fine to widen the basin of attraction. The same structure appears in 3D with NeRF. A pixel's colour is volume-rendered through the MLP $f$: $$\hat{\mathcal{I}}(\mathbf{u})=\int_{z_{\text{near}}}^{z_{\text{far}}}T(\mathbf{u},z)\,\sigma(z\bar{\mathbf{u}})\,\mathbf{c}(z\bar{\mathbf{u}})\,\mathrm{d}z\;,\qquad T(\mathbf{u},z)=\exp\Big(-\int_{z_{\text{near}}}^{z}\sigma(z'\bar{\mathbf{u}})\,\mathrm{d}z'\Big)$$ and BARF jointly optimises the $M$ camera poses $\mathbf{p}_i\in\mathbb{R}^6$ (parameterised in the Lie algebra $\mathfrak{se}(3)$) and NeRF weights $\boldsymbol{\Theta}$ over the synthesis-based objective $$\min_{\mathbf{p}_1,\dots,\mathbf{p}_M,\boldsymbol{\Theta}}\;\sum_{i=1}^{M}\sum_{\mathbf{u}}\big\|\hat{\mathcal{I}}(\mathbf{u};\mathbf{p}_i,\boldsymbol{\Theta})-\mathcal{I}_i(\mathbf{u})\big\|_2^2\;.$$ The obstacle is positional encoding. NeRF lifts inputs with $\gamma_k(\mathbf{x})=\big[\cos(2^k\pi\mathbf{x}),\sin(2^k\pi\mathbf{x})\big]$, whose Jacobian $$\frac{\partial\gamma_k(\mathbf{x})}{\partial\mathbf{x}}=2^k\pi\cdot\big[-\sin(2^k\pi\mathbf{x}),\cos(2^k\pi\mathbf{x})\big]$$ amplifies gradients by $2^k\pi$ while flipping direction at the same frequency, so pose gradients from sampled 3D points "are incoherent … and can easily cancel out each other". BARF's fix is a dynamic low-pass filter: weight the $k$-th band as $\gamma_k(\mathbf{x};\alpha)=w_k(\alpha)\cdot\big[\cos(2^k\pi\mathbf{x}),\sin(2^k\pi\mathbf{x})\big]$ with $$w_k(\alpha)=\begin{cases}0 & \text{if } \alpha