PaperlayerView exact revision on arXiv

Source-linked AI summary

Faster-GS: Analyzing and Improving Gaussian Splatting Optimization

Florian Hahlbohm, Linus Franke, Martin Eisemann, Marcus Magnor

arXiv:2602.09999v1cs.CVcs.GR

TL;DR

3DGS optimization research is fragmented, complicating fair assessment of training-performance improvements that preserve reconstruction quality. Faster-GS consolidates and evaluates prior strategies with novel memory-efficient optimizations, achieving faster 3D and 4D Gaussian reconstruction while maintaining quality.

  • Problem

    Fragmented 3DGS extensions and entangled algorithmic changes make it difficult to establish an updated training-performance baseline while preserving reconstruction quality.

  • Method

    Faster-GS surveys and evaluates prior performance improvements, integrates effective techniques, and adds memory-coalescing and fused gradient-update optimizations into an optimized 3DGS framework.

  • Results

    Faster-GS trains 3D and 4D Gaussian scenes up to 5× faster than prior work while maintaining full quality and all Gaussians.

  • Takeaways & Limitations

    Faster-GS provides a cost-effective baseline that enables full 3DGS reconstruction in less than two minutes and reduces VRAM requirements by up to 30%.

  • Takeaways & Limitations

    Remaining bottlenecks are tied to parameter updates, while further fusion and mixed-precision optimizations involve tradeoffs among simplicity, robustness, and performance.

Abstract

from arXiv · show

Recent advances in 3D Gaussian Splatting (3DGS) have focused on accelerating optimization while preserving reconstruction quality. However, many proposed methods entangle implementation-level improvements with fundamental algorithmic modifications or trade performance for fidelity, leading to a fragmented research landscape that complicates fair comparison. In this work, we consolidate and evaluate the most effective and broadly applicable strategies from prior 3DGS research and augment them with several novel optimizations. We further investigate underexplored aspects of the framework, including numerical stability, Gaussian truncation, and gradient approximation. The resulting system, Faster-GS, provides a rigorously optimized algorithm that we evaluate across a comprehensive suite of benchmarks. Our experiments demonstrate that Faster-GS achieves up to 5$\times$ faster training while maintaining visual quality, establishing a new cost-effective and resource efficient baseline for 3DGS optimization. Furthermore, we demonstrate that optimizations can be applied to 4D Gaussian reconstruction, leading to efficient non-rigid scene optimization.

1. Introduction

Faster-GS addresses the fragmented 3DGS optimization landscape by systematically evaluating prior performance improvements and integrating compatible novel optimizations. The resulting pipeline targets faster training while preserving reconstruction quality and supports extensions to 4D dynamic-scene reconstruction.

  • 3DGS combines point-based rendering with gradient-based optimization and achieved broad adoption through visual fidelity and real-time performance.
  • The rapid growth of specialized extensions makes cohesive integration and evaluation difficult, especially for estimating training-performance improvements without changing the original algorithm.
  • Faster-GS surveys and evaluates prior 3DGS performance improvements, then integrates them into an optimized training framework to establish an updated performance baseline.
  • The framework introduces memory-coalescence and fused gradient-update optimizations while retaining compatibility with the widely used CUDA-based differentiable rasterization pipeline.
  • Faster-GS achieves up to 5× faster training while maintaining reconstruction quality and extends the implementation to 4D dynamic-scene reconstruction.

2. Preliminaries and Related Work

3DGS represents scenes with anisotropic Gaussian primitives and optimizes them through differentiable rasterization, adaptive density control, and gradient-based updates. Follow-up work targets rendering, optimization, representation, portability, and GPU efficiency through varied algorithmic and implementation changes.

  • 3D Gaussian Splatting: Each 3DGS primitive contains a 3D mean, anisotropic covariance, opacity, and spherical-harmonics coefficients for view-dependent color.
  • 3D Gaussian Splatting: Rendering transforms relevant Gaussian parameters into camera space, projects them into 2D Gaussians, and evaluates them at pixels for alpha blending.
  • 3D Gaussian Splatting: Pixel colors are formed by front-to-back alpha blending, while object-level depth sorting approximates exact per-pixel sorting before rasterization.
  • 3D Gaussian Splatting: 3DGS uses activations and parameterizations that constrain opacity, color, and covariance to valid optimization domains.
  • Optimization: Adaptive density control clones, splits, and prunes Gaussians using 2D-gradient magnitudes, but the growing set can cause memory fragmentation.
  • Improvements and Follow-up Works: Follow-up research improves rendering, densification, pruning, storage, level of detail, dynamic-scene reconstruction, and GPU performance through diverse methods and trade-offs.

3. Method

Faster-GS consolidates compatible 3DGS optimizations and adds implementation improvements targeting memory access, gradient computation, optimizer updates, and data locality. It preserves the original optimization paradigm while extending the optimized framework to 4D Gaussian rendering.

  • Scope and basis: Faster-GS follows the original 3DGS paradigm while consolidating recent optimization techniques and integrating additional improvements.The framework is designed as a compatible, optimized baseline rather than a fundamental replacement of the original pipeline.
  • Memory and rasterization: Memory-bound tiled rasterization motivates reducing Gaussian duplication, tile intersections, sorting overhead, and per-Gaussian gradient memory accesses.The implementation targets the dominant cost of loading many Gaussian parameters and processing per-tile splat lists.
  • Gradient computation: The backward pass is accelerated by parallelizing over Gaussians, reducing atomic operations by a factor of 256 for 16×16 tiles, and using shared memory to lower VRAM costs.Faster-GS improves the Gaussian-parallel backward design while addressing its additional memory allocations.
  • Parameter updates: Faster-GS accelerates parameter updates with a fused CUDA Adam implementation that matches PyTorch behavior while reducing overhead and instruction count.The implementation uses kernel fusion, fast math operations, and fused-multiply additions.
  • Data locality: Z-ordering places neighboring 3D Gaussians near one another in parameter buffers, reducing warp divergence and cache misses during densification.The optimization is applied regularly while densification is active.
  • Scope and limitations: Skipping updates for invisible Gaussians is optional because it can cause inconsistencies and performance regressions relative to the original 3DGS implementation.The optimization fits the fused design but is not treated as universally safe.
  • 4D extension: The optimized framework extends to 4D Gaussian rendering by adapting the data model, rasterizer kernels, gradients, and multi-image training schedule.The prior optimizations transfer directly after these changes.

4. Evaluation

The evaluation compares Faster-GS with established implementations across quality, training speed, memory use, component ablations, GPU generations, and 4D dynamic reconstruction. Faster-GS preserves reconstruction quality while delivering substantial training-speed improvements, including up to 5.2× on Deep Blending and up to 3× for 4D scenes.

  • Overall comparison: All evaluated methods achieve similar image quality and converge to roughly the same number of Gaussians, while image-quality metrics vary across runs.The reported quality metrics are averaged across five runs because floating-point arithmetic and random gradient-accumulation ordering can change PSNR by up to 0.5 dB for some scenes.
  • Overall comparison: 5.2× faster training than 3DGS and 2.4× faster than Taming-3DGS are achieved on the Deep Blending scenes.The main differences between methods are training time and VRAM consumption.
  • Component ablations: Fused Adam techniques produce particularly strong speedups, while the per-Gaussian alpha-blending backward pass substantially accelerates training but increases VRAM usage.The optimized fused Adam implementation consistently outperforms the PyTorch and apex alternatives.
  • Component ablations: Load-balanced Gaussian/tile instance creation increasingly slows training as primitive counts grow because warp divergence leaves more than half of each warp inactive.Z-ordering mitigates this issue for scenes with many Gaussians but can slow small scenes through increased atomic contention; repeated z-ordering is beneficial with the per-Gaussian backward pass.
  • GPU comparison: 5× improvement over the original implementation is obtained on an RTX 5090, where average training takes 163 seconds across the evaluated scenes.Newer consumer GPUs exhibit greater speedups, suggesting performance gains on newer hardware generations.
  • Dynamic scenes: 3× faster training is achieved for dynamic 4D reconstruction while using less VRAM and maintaining quality.The extension is evaluated against the reference implementation on eight synthetic D-NeRF scenes.

5. Discussion, Limitations, and Future Work

Faster-GS substantially accelerates Gaussian Splatting, but parameter updates remain a bottleneck. The authors identify several promising optimizations and scope boundaries for future work.

  • Remaining Bottlenecks: Parameter-update computation remains a principal training bottleneck after the framework’s other optimizations.The authors identify second-order optimizers and more compact view-dependent appearance representations as possible future directions.
  • Future Work: Further optimizations, including fused forward and backward passes or mixed-precision training, may trade simplicity and robustness against performance.These approaches are presented as possible extensions rather than components of the current system.
  • Scope Boundaries: The evaluation excludes training improvements targeting artifacts, controllability, and informed pruning.The authors also leave integration of state-of-the-art anti-aliasing and densification techniques outside the main scope.

6. Conclusion

Faster-GS combines a systematic evaluation of 3DGS performance improvements with memory-efficient adaptations for faster 3D and 4D Gaussian scene training. It reaches up to 5× faster training, reduces VRAM requirements by up to 30%, and enables full 3DGS reconstruction in under two minutes.

  • Framework: Faster-GS surveys and systematically evaluates recent 3DGS performance improvements before integrating memory-efficient adaptations.The resulting framework targets both 3D and 4D Gaussian scenes.
  • Performance: 5× faster training is achieved for 3D and 4D Gaussian scenes than prior work.This is the maximum speedup reported for the optimized framework.
  • Efficiency: 30% lower VRAM requirements make the approach more feasible on lower-end hardware.The conclusion characterizes the framework as especially cost-effective.
  • Practical Impact: Full 3DGS reconstruction takes less than two minutes with Faster-GS.The authors release code to support future Gaussian Splatting-based view synthesis research.

A. 3DGS Training Details

The appendix details the original 3DGS training schedule, including opacity resets, the weighted L1 and D-SSIM loss, Adam optimization, and learning-rate settings.

  • Opacity Reset: 3DGS resets every Gaussian’s opacity every 3000 iterations while densification is active, clipping opacity values to 0.01 from above.The reset occurs four times in total and encourages pruning of floaters and incorrectly placed Gaussians.
  • Optimization Schedule: The loss combines L1 and D-SSIM terms with weights 0.8 and 0.2, respectively.Parameter updates use Adam with β1 = 0.9, β2 = 0.999, and ϵ = 1e-15.
  • Learning Rates: Scale, rotation, and opacity learning rates are set to 0.005, 0.001, and 0.025, respectively.The opacity learning rate reflects a change from the original 0.05 setting.
  • Learning Rates: Gaussian-mean learning rates decay exponentially from 1.6e-4 to 1.6e-6 during optimization.An additional scene-size-dependent constant scales these learning rates.

B. Implementation Details

This section describes the refactored 3DGS implementation used as the testbed and the implementation of prior-work improvements.

  • Testbed Implementation: The testbed is based on a refactored 3DGS implementation with implementations of improvements proposed in prior work.The section provides detailed implementation information for the refactored system.

B.1. Testbed Basis

The testbed refactors 3DGS around rewritten CUDA kernels and targeted frontend, sorting, and backward-pass optimizations. These changes improve efficiency while preserving the original rasterization pipeline and reconstruction behavior.

  • Testbed implementation: The implementation builds a clean 3DGS testbed on NeRFICG and rewrites the CUDA rasterizer with simplifications and small optimizations.The rewrite also replaces OpenGL projection-matrix reliance with direct intrinsic camera parameters.
  • Numerical stability: Numerical stability is improved by capping fragment opacity at 0.99 and stopping blending when transmittance falls below τ = 1e-4.These choices stabilize repeated division in backward transmittance computation and avoid unnecessary work.
  • Frontend and densification: Persistent non-differentiable buffers replace implicit rasterizer returns for densification metrics, reducing VRAM use and slightly improving speed.The buffers store 2D mean gradients and visibility masks.
  • Frontend and densification: Avoiding redundant Gaussian copies reduces overhead in cloning, splitting, and pruning, while 145 additional optimizer steps have only a small runtime effect.The extra steps do not add rasterizer forward or backward calls.
  • Separate sorting: Separate 32-bit depth and 16-bit tile sorting reduces sorting-key storage and radix-sort complexity for images up to roughly 16 megapixels.The approach replaces sorting a large 64-bit-key buffer with two specialized routines.
  • Backward pass: Per-Gaussian backward processing and collaborative warp loads reduce gradient-computation overhead by restructuring stored blending state and global-memory access.The method stores intermediate color and transmittance states, then lets all 32 warp threads load alpha-blending batches collaboratively.

C. Further Experiments and Results

The experiments also assess inference rendering speed across 13 scenes and report that the inference-optimized forward pass substantially accelerates rendering. The broader evaluation is presented as complementary to the main-paper results.

  • The supplementary experiments complement the evaluation reported in the main paper.
  • Inference rendering: Inference frame rates are measured across 13 scenes from Mip-NeRF360, Tanks and Temples, and Deep Blending at native resolution.Each test set is rendered 100 times, with activation functions baked to avoid PyTorch-related overhead.
  • Inference rendering: More than 3× faster rendering is achieved during inference with an optimized version of the testbed forward pass.The optimized pass enables the improvements that accelerate inference.

C.2. Efficient Anti-Aliasing

Faster-GS integrates Mip-Splatting’s 3D and 2D anti-aliasing extensions with cheaper smoothing and more stable gradients. The resulting implementation supports anti-aliased training and rendering while improving efficiency relative to the original implementation.

  • Mip-Splatting extensions: Mip-Splatting combines a 3D smoothing filter and a 2D Mip filter to prevent aliasing when rendering at sampling rates different from training.The 3D filter limits Gaussian sizes, while the 2D filter addresses projected Gaussians that become too small under changed viewpoints or focal lengths.
  • 3D smoothing: The 3D smoothing filter is recomputed from visibility, focal length, and depth information, with its sampling-rate estimate updated every 100 iterations.Rendering applies the filter to Gaussian scales and opacity using κ3D = 0.2 by default.
  • 3D smoothing: Faster-GS enforces the 3D smoothing size constraint by clipping Gaussian scales in place after each optimizer step, independently of gradient computation.This direct update retains the filter’s advantages while being significantly cheaper to compute.
  • 2D anti-aliasing: The 2D Mip filter compensates opacity to avoid increasingly blurry renderings when Gaussians shrink with distance or larger focal lengths.Faster-GS uses the compensated opacity in opacity-aware bounding boxes and offers a stable analytical derivative option for Σ2D.
  • Results: Anti-aliased Faster-GS trains faster and uses less VRAM than the original Mip-Splatting implementation while reaching the same quality.The revised backward pass also yields a more consistent number of Gaussians.
  • Results: Fully anti-aliased training and rendering are enabled in Faster-GS, although some configurations can create tiny elongated Gaussians that slightly reduce reconstruction quality.The effectiveness of the anti-aliasing approach is validated through multi-scale evaluation and renderings at changed resolutions.
  • MCMC densification: The optimized MCMC densification implementation is significantly faster and uses less VRAM while achieving similar quality to the original implementation.The implementation fuses noise injection into a single CUDA kernel and reports results on nine Mip-NeRF360 scenes.

C.4. About Gaussian Truncation and Opacity

The paper examines how opacity-dependent Gaussian truncation limits the minimum effective support and proposes checking Gaussian response before opacity multiplication. This enables more aggressive, opacity-independent truncation but can affect training speed and quality.

  • Existing truncation: Original 3DGS truncates projected 2D Gaussians at roughly 3.33σ while incorporating opacity into the truncation rule.For near-unit opacity, the effective limit is approximately 3.03 standard deviations.
  • Opacity-independent truncation: Checking Gaussian response before multiplying by opacity enables truncation at fewer standard deviations and changes how opacity gradients are treated.The modification addresses analytical opacity gradients that remain non-zero even when the Gaussian value is zero.
  • Truncation results: At 1/2σ, densification creates fewer Gaussians and reconstruction quality decreases, while opacity-independent truncation slows training.The most aggressive 1σ setting produces contributing fragments with minimum opacity approximately 0.61.
  • Scope and future work: The authors identify 2σ truncation with opaque 2D ellipse rendering as a future direction rather than an established result.The proposed direction would avoid depth-ordered alpha blending, but the paper does not report it as evaluated.

C.5. Additional 4D Reconstruction Results

The 4D Gaussian extension is evaluated against the reference implementation across three real scenes from a neural 3D video dataset.

  • Both implementations use consistent preprocessing, hyperparameters, resolution, train/test splits, and view-dependent color parametrization.
  • The comparison covers Coffee Martini, Cook Spinach, and Flame Steak using the reference implementation as baseline.
Loading 2602.09999v1…