Source-linked AI summary
RadSplat: Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPS
Michael Niemeyer, Fabian Manhardt, Marie-Julie Rakotosaona, Michael Oechsle, Daniel Duckworth, Rama Gosula, Keisuke Tateno, John Bates, Dominik Kaeser, Federico Tombari
TL;DR
Large scenes challenge efficient rendering: point counts affect speed, and growing counts can strain memory. RadSplat guides a point-based representation with a radiance-field prior and uses pruning and test-time filtering. It achieves state-of-the-art results on common benchmarks while rendering up to 3,000× faster than prior works.
Problem
Large scenes challenge efficient rendering: increasing point counts can make memory use unmanageable and reduce speed, while real-time 3DGS still requires powerful GPUs.
Source: §1 · p. 2, §3.3 · p. 5Method
RadSplat uses radiance fields as a prior and supervision for point-based 3DGS, with pruning and test-time visibility filtering to reduce scene size and speed rendering.
Source: §1 · p. 2Results
RadSplat achieves state-of-the-art results on common benchmarks while rendering up to 3,000× faster than prior works.
Source: §5 · p. 8Takeaways & Limitations
The approach combines robust real-time rendering of complex scenes with more compact representations and faster test-time rendering without a quality drop.
Source: §5 · p. 8RadSplat retains a small performance gap to Zip-NeRF on large-scale scenes, which the authors leave for future investigation.
Source: §4.2 · p. 8
Abstract
from arXiv · showhide
Recent advances in view synthesis and real-time rendering have achieved photorealistic quality at impressive rendering speeds. While Radiance Field-based methods achieve state-of-the-art quality in challenging scenarios such as in-the-wild captures and large-scale scenes, they often suffer from excessively high compute requirements linked to volumetric rendering. Gaussian Splatting-based methods, on the other hand, rely on rasterization and naturally achieve real-time rendering but suffer from brittle optimization heuristics that underperform on more challenging scenes. In this work, we present RadSplat, a lightweight method for robust real-time rendering of complex scenes. Our main contributions are threefold. First, we use radiance fields as a prior and supervision signal for optimizing point-based scene representations, leading to improved quality and more robust optimization. Next, we develop a novel pruning technique reducing the overall point count while maintaining high quality, leading to smaller and more compact scene representations with faster inference speeds. Finally, we propose a novel test-time filtering approach that further accelerates rendering and allows to scale to larger, house-sized scenes. We find that our method enables state-of-the-art synthesis of complex captures at 900+ FPS.
1. Introduction
Neural fields offer strong view-synthesis quality but are costly to render, while 3D Gaussian Splatting is fast but difficult to optimize robustly on complex scenes. RadSplat uses a radiance field to guide point-based representations, then prunes and filters them to target compact, fast rendering.
1. Introduction
- RadSplat reaches up to 907 FPS, over 3.6× faster than 3DGS and more than 3,000× faster than Zip-NeRF, with PSNR up to 1.87 dB higher than 3DGS and SSIM exceeding Zip-NeRF.These results cover medium and large scenes.Source: Figure 1 · p. 1, §1 · p. 2
- Neural-field methods deliver high view-synthesis quality but volumetric rendering is computationally expensive, and faster alternatives trade quality or storage for speed.Prior approaches reduce network complexity, cache outputs, or extract meshes, but do not maintain state-of-the-art quality in real time.Source: §1 · p. 1
- 3DGS renders in real time, but its brittle optimization heuristics struggle with exposure variation, motion blur, and moving objects in large scenes.Growing primitive counts can also make memory use unmanageable and reduce rendering speed.Source: §1 · pp. 1–2
- RadSplat uses radiance fields as a prior and supervision signal for point-based 3DGS, and adds pruning and viewpoint-based filtering to reduce scene size and accelerate rendering.The filtering step is described as speeding rendering without a corresponding quality loss.Source: §1 · p. 2
2. Related Work
Related work positions neural fields as robust, high-quality representations, while noting the rendering costs and storage trade-offs of approaches that accelerate them. RadSplat combines a neural-field prior with rasterized point representations and adds pruning and test-time visibility rendering.
Neural Fields
- Neural fields are valued for simplicity, strong performance, and robust optimization, and are used across view synthesis, reconstruction, and generative modeling.NeRF's volumetric rendering is described as more robust than earlier surface-based rendering approaches.Source: §2 · p. 2
Neural Fields for Real-Time Rendering
- Neural-field acceleration methods use voxel grids, meshes, or cacheable assets, but can require powerful GPUs, sacrifice quality, consume substantial storage, or need custom rendering.RadSplat instead optimizes lightweight point-based representations suited to rasterization.Source: §2 · p. 2
Point-Based Representations
- Point-based methods rasterize efficiently, and 3DGS combines adaptive density control with fast kernels; RadSplat adds a NeRF prior, pruning, and visibility rendering for complex captures.The passage reports 10× fewer Gaussian primitives at higher quality than 3DGS and inference above 900 FPS.Source: §2 · p. 3
3. Method
RadSplat combines radiance fields as a robust prior for initializing and supervising 3D Gaussian Splatting (3DGS) with pruning and viewpoint-based filtering. The method targets compact, high-quality scenes and faster rendering, including larger scenes.
3. Method
- The pipeline trains radiance fields, uses them to initialize and supervise 3DGS, then prunes points and filters them by viewpoint at test time.Source: §3 · p. 3
3.1. Neural Radiance Fields as a Robust Prior
- Per-image GLO vectors let the radiance field model image- and view-dependent appearance changes, with zero-latent rendering used for stable view synthesis.Source: §3.1 · p. 3
3.2. Radiance Field-Informed Gaussian Splatting
- 3DGS is initialized by unprojecting NeRF median-depth samples (the first sample with accumulated transmittance τ > 0.5) and trained on zero-GLO NeRF renderings for cleaner supervision.NeRF ray casting also supports arbitrary camera lens types, unlike 3DGS's pinhole-camera gradient formulation.Source: §3.2 · pp. 4–5
3.3. Ray Contribution-Based Pruning
- During optimization, pruning removes Gaussians whose maximum exact ray contribution across training rays falls below a threshold, reducing point count while maintaining quality.Source: §3.3 · p. 5
3.4. Viewpoint-Based Visibility Filtering
- Visibility masks are built per camera cluster, with random camera samples for test-view robustness; rendering only active points boosts FPS by up to 45% without quality loss.Source: §3.4 · p. 5
3.5. Implementation Details
- Experiments use 1 million initial points, 64 visibility clusters, and threshold 0.001; radiance fields train on 8 V100 GPUs and 3DGS on an A100, about 1 hour each.Source: §3.5 · pp. 5–6
4. Experiments
The experiments evaluate RadSplat on unbounded MipNeRF360 scenes and large-scale Zip-NeRF captures, comparing reconstruction quality, rendering speed, and Gaussian counts. Results cover benchmark comparisons, component ablations, optimization progression, visibility filtering, and a remaining large-scene quality gap to Zip-NeRF.
4. Experiments
- Evaluation uses nine unbounded indoor and outdoor MipNeRF360 scenes and four large-scale Zip-NeRF apartment and house scenes, reporting PSNR, SSIM, LPIPS, FPS, and Gaussian counts.FPS is measured on an RTX 3090 GPU; comparison methods include 3DGS, MERF, SMERF, and Zip-NeRF, with additional baselines on MipNeRF360.Source: §4 · p. 6
4.1. Real-Time View Synthesis
- Ours Light reaches 907 FPS on Mip-NeRF360; on Zip-NeRF it achieves 0.838 mean SSIM at 748 FPS, versus SMERF’s 0.829 SSIM at 221 FPS.Table 1 also reports a 10× Gaussian reduction for Ours Light versus 3DGS and up to 3.3× faster rendering than top-performing real-time methods on large-scale scenes.Source: Table 1 · p. 6, §4.1 · pp. 6–7
- RadSplat’s default model exceeds Zip-NeRF in SSIM and LPIPS while rendering 1,600× faster, and its large-scene variant achieves high-quality results on challenging in-the-wild captures.The reported large-scale comparison gives Ours Light 0.838 mean SSIM at 748 FPS, versus SMERF’s 0.829 SSIM at 221 FPS.Source: §4.1 · pp. 6–7
4.2. Ablation Study and Limitations
- Removing NeRF initialization can lose small geometric and texture details, while removing NeRF supervision introduces floating artifacts under lighting or exposure variation.The supervision comparison is against optimizing the scene representation directly on input views.Source: §4.2 · pp. 7–8, Figure 5 · p. 8
- Without pruning, the Gaussian count is 1.5× larger without quality improvement; pruning maintains high quality and can match 3DGS with roughly 10× fewer Gaussians.The reported pruning thresholds below 0.1 maintain quality while reducing point count by 4× on the mip-NeRF 360 Bicycle scene.Source: Figure 5 · p. 8, §4.2 · p. 8, Figure 6 · p. 8
- On mip-NeRF 360 Bicycle, RadSplat’s SSIM rises faster and reaches 3DGS’s final quality after less than 8k steps.The progression comparison is against 3DGS.Source: Figure 7 · p. 8, §4.2 · p. 8
- Visibility-list rendering increases mean FPS by up to 10% on MipNeRF360 scenes and up to 45% on larger Zip-NeRF scenes while keeping quality fixed.The authors identify this postprocessing step as particularly important for scaling to larger scenes.Source: Table 2 · p. 8, §4.2 · p. 8
- The authors report a small remaining quality gap to Zip-NeRF on large-scale scenes and identify it as future work.Source: §4.2 · p. 8
5. Conclusion
RadSplat combines neural fields and point-based representations for robust real-time rendering of complex scenes. Its radiance-field supervision, pruning, and test-time filtering improve optimization, compactness, quality, and speed.
5. Conclusion
- Up to 3,000× faster rendering than prior works accompanies state-of-the-art results on common benchmarks.Source: §5 · p. 8
- Radiance fields used as a prior and supervision signal improve results and stabilize optimization of point-based 3DGS representations.Source: §5 · p. 8
- Novel pruning produces more compact scenes with significantly smaller scene size while improving quality.Source: §5 · p. 8
- Novel test-time filtering increases rendering speed without a quality drop.Source: §5 · p. 8