Season 10 · Article 03

NeRFs for the Web

Neural Radiance Fields. An MLP is queried along each ray to produce color. Photoreal from photos. Largely superseded by Gaussian Splatting in 2024-25, but still has niches.

1. The original NeRF idea

Scene = function (x, y, z, θ, φ) → (RGB, σ). Learnable neural net.

For each pixel: shoot a ray. Sample 64-256 points. Query MLP for color + density at each. Composite via volume rendering.

2. Volume rendering formula

C(r) = Σ T_i · (1 - exp(-σ_i · δ_i)) · c_i
T_i = exp(-Σ σ_j · δ_j)   // transmittance

3. Training

100+ photos, known camera poses (COLMAP). Render predicted image → MSE loss against real photo → backprop through the ray → update MLP weights. Hours on a GPU.

4. Speed-ups

• Instant-NGP (NVIDIA): hash grid, 5-second training.
• Plenoxels: voxel grid, no MLP. Fast training.
• TensoRF: tensor decomposition of voxels.
• Nerfstudio: unified framework for all variants.

5. Why splats won

• NeRF renders slow: every pixel 64 MLP queries per frame. Even with tricks, tens of ms.
• Splats render in 1-2ms for the same quality.
• NeRFs better for SLAM and sparse-view. Splats dominate dense capture.

6. NeRFs in browser

• Google's View SIBR
• Mobile-NeRF: bake NeRF → textured mesh → runs anywhere.
• SMERF: streamable NeRF block for AR/VR.
• Nerfstudio web viewer: experimental.

7. Bake-to-mesh path

Train NeRF → extract textured mesh → load as regular glTF in Three.js. MobileNeRF, Bake-a-NeRF, Instant-NGP's mesh export.

Quality: good. Perf: excellent (just mesh rendering). Fidelity: ~80% of original NeRF view synthesis.

8. When to use NeRF today

• Sparse-view (only 5-10 photos): NeRF generalizes better than splats.
• Relighting: NeRF separates geometry + appearance; splats bake appearance.
• Research reproduction: papers still reference NeRF variants.
• Else: use Gaussian Splatting (S10-02).

9. Takeaways

• NeRF = MLP queried along rays + volume rendering.
• Largely replaced by Gaussian Splatting for speed.
• Still wins for sparse-view and relighting.
• Bake-to-mesh path gets NeRF-quality assets into any engine.