Source-linked AI summary
TruncGradGS: Improved 3D Gaussian Splatting via Truncated Gradient Updates
Theo Morales, Nhat-Quynh Le-Pham, Robin Atkins, Binh-Son Hua
TL;DR
TruncGradGS addresses vanishing gradients in tiled Gaussian Splatting, which limit learning when pixels lie far from Gaussian primitives. It uses piecewise truncated gradients and enlarged tile coverage, improving reconstruction quality and robustness across static and dynamic settings while introducing computational overhead.
Problem
Vanishing gradients from Gaussian locality prevent distant pixels from meaningfully updating primitives, limiting reconstruction when scenes diverge from initialization.
Method
TruncGradGS replaces near-zero Gaussian derivative tails with a continuous piecewise surrogate and expands splat tile coverage during training.
Results
Truncated gradients consistently improve reconstruction quality and robustness across random and COLMAP initializations, static and dynamic Gaussian Splatting, and challenging synthetic dynamic scenes.
Takeaways & Limitations
The method broadens effective optimization support, making Gaussian Splatting more robust across initialization settings and scene dynamics while providing a synthetic six-scene dynamic benchmark.
Takeaways & Limitations
Enlarging optimization coverage increases computation, so TruncGradGS generally requires longer training times than the original rasterization pipeline.
Abstract
from arXiv · showhide
3D Gaussian Splatting has become a de facto scene representation for novel view synthesis, yet robustly learning 3D Gaussian primitives from visual input remains challenging. Standard optimization relies on gradient-based updates, but a common issue is the gradient vanishing phenomenon: a pixel far from a Gaussian primitive often has diminishing gradient magnitudes to influence primitive attributes, resulting in suboptimal scene reconstruction. In this paper, we propose a method to address gradient vanishing with a piecewise truncated gradient formulation that improves the optimization stability and robustness to initializations. We show that our method consistently improves 3D Gaussian Splatting with random and COLMAP initializations while being generalizable across static and dynamic Gaussian Splatting. As a by-product, we also examine the limitations of current benchmarks for dynamic scenes, and introduce a novel dataset for benchmarking dynamic Gaussian Splatting using synthetic 3D scenes. We demonstrate the effectiveness of our method in both static and dynamic settings for the public benchmarks and our proposed dataset.
1. Introduction
3D Gaussian Splatting learns scene primitives through differentiable rasterization, but local tile support can cause vanishing gradients and unstable reconstruction. TruncGradGS enlarges gradient support with a continuous piecewise surrogate and demonstrates improvements across static and dynamic settings.
- 3D Gaussian Splatting represents scenes with optimized ellipsoidal primitives parameterized by means, covariances, opacity, and appearance attributes.
- Differentiable rendering enables stochastic-gradient optimization of Gaussian attributes through camera projection and tiled primitive rasterization.
- Local tile support can leave primitives without gradients from distant relevant pixels, degrading reconstruction when initialization is poor.
- TruncGradGS modifies Gaussian derivatives with a linear tail surrogate to strengthen primitive–tile coverage while preserving derivative continuity.
- The method improves scene quality across initializations and works with both vanilla and dynamic Gaussian Splatting.
- The paper introduces six synthetic multi-view video scenes as a benchmark for challenging dynamic Gaussian Splatting reconstruction.
2. Related Works
Related work addresses initialization, densification, pruning, feed-forward reconstruction, and dynamic deformation, while TruncGradGS targets locality and vanishing gradients directly. The paper also motivates a harder dynamic benchmark with longer, photorealistic sequences and complex motion.
- Existing 3DGS methods face sensitivity to sparse initialization, vanishing gradients, floaters, and inefficient Gaussian proliferation.
- Prior approaches improve initialization through sparse-large-variance strategies, progressive filtering, bound-expanding splits, depth alignment, and importance-aware resampling.
- Other methods redesign densification and pruning using importance scoring, budgeting, viewing-cone guidance, pixel-aware control, or alternating phases.
- Feed-forward Gaussian reconstruction improves scalability and inference speed but typically requires large training datasets and may generalize less robustly to sparse or out-of-distribution views.
- TruncGradGS instead modifies the gradient field to enlarge Gaussian optimization regions and remains complementary to initialization and densification methods.
- Dynamic Gaussian Splatting uses spatio-temporal or deformation models, yet large temporal divergence makes distant frames difficult to model with one Gaussian set.
- The proposed benchmark targets shortcomings of existing datasets through long-duration, photorealistic sequences with complex non-rigid motion and diverse scenes.
3. Methodology
3DGS optimizes projected Gaussian primitives through differentiable tiled rasterization, but local support can cause vanishing spatial gradients. TruncGradGS replaces far-tail derivatives with a piecewise linear surrogate while preserving the true derivative near the Gaussian.
- 3.1. Background: 3DGS represents a radiance field with Gaussian primitives whose attributes are optimized against rendered image error.Each primitive uses a mean, covariance, opacity, and view-dependent color.
- 3.1. Background: The differentiable renderer projects and rasterizes Gaussians into 16×16 image tiles, restricting each primitive's gradient signals to pixels within its tile support.Bounding boxes assign splats to intersecting tiles, reducing the number of rasterized Gaussians per CUDA kernel.
- 3.2. Gradient analysis: Pixels outside a 2D Gaussian's isocontour receive near-zero gradients, limiting primitive movement toward relevant regions and making dynamic scenes especially difficult to optimize.The analysis focuses on the loss derivative with respect to the projected 2D mean because it controls Gaussian movement and coverage.
- 3.3. Piecewise truncated gradient: The method truncates the Gaussian partial derivatives below a density threshold, replacing them with a linear surrogate while retaining the true derivative within the threshold region.The surrogate is designed to expand the effective support of Gaussian derivatives while preserving derivative continuity.
- 3.3. Piecewise truncated gradient: The truncated gradient introduces a convergence risk when accumulated surrogate gradients exceed the true gradient for poor fits or near-optimal Gaussians.This creates a trade-off between moving badly placed Gaussians and preserving reconstruction of good fits.
4. Evaluation
The evaluation tests TruncGradGS across static and dynamic scenes, initialization settings, public benchmarks, and a newly introduced synthetic dynamic dataset. Results show consistent reconstruction improvements, while ablations and discussion identify the components, compatibility, and computational trade-offs behind the method.
- Evaluation setup: The evaluation covers static and dynamic scenes across public benchmarks and a newly introduced dynamic dataset designed for challenging spatio-temporal divergence and initialization conditions.The experiments include Mip-NeRF360, Neural 3D Video, and six synthetic dynamic scenes.
- Evaluation setup: The synthetic benchmark contains six 300-frame scenes rendered at 1600×900 and 30fps, with 25–45 cameras and one to four test viewpoints.Scenes span fluid simulation and hand-crafted animation.
- Static reconstruction: TruncGradGS consistently improves reconstruction quality across 3DGS and 2DGS with both random and COLMAP initialization, with especially clear gains from random starts.The method also tends to produce more compact representations with fewer Gaussians.
- Dynamic reconstruction: The truncated gradient improves 4DGS, CEM-4DGS, and 4D-Scaffold on dynamic benchmarks, including recovery of complex motion, fluid effects, and fine geometric details.Baselines exhibit floaters, blurring, or missing structures in qualitative comparisons.
- Ablation study: The full method combines truncated gradients, radius padding, and delayed pruning, with removing truncated gradients causing the largest performance drop.Radius padding supports relevant tile assignment, while delayed pruning preserves recovery opportunities for weak Gaussians.
- Ablation study: Applying truncated gradients indiscriminately or without alpha-gradient sign filtering can destabilize training or repel Gaussians, whereas the full method achieves the best performance.The surrogate linear model can bias optimization when applied to all Gaussians.
- Discussion and limitations: The method improves robustness by enlarging Gaussian support regions, but this increases the number of Gaussian-pixel pairs in backpropagation and generally lengthens training.The paper characterizes this as a trade-off between reconstruction robustness and optimization efficiency.
- Discussion and limitations: Except for delayed pruning, the authors retain baseline densification and pruning hyperparameters, leaving potential gains from jointly tuning these policies with truncated gradients.The paper specifically identifies SSIM as a metric that may be sensitive to densification strategies.
5. Conclusion
TruncGradGS improves Gaussian splatting through piecewise truncated gradient updates, with reported gains in reconstruction quality and robustness across static and dynamic settings. The conclusion also introduces a synthetic benchmark for challenging dynamic scenes.
- TruncGradGS replaces near-zero Gaussian-derivative tails with a continuous surrogate to enlarge primitives’ effective optimization support.This alleviates vanishing gradients in tiled Gaussian rasterization.
- Experiments report consistent improvements in reconstruction quality and robustness across static and dynamic benchmarks and different initialization settings.
- The paper introduces a synthetic dynamic Gaussian-splatting benchmark containing scenes with large spatio-temporal divergence.