Source-linked AI summary
DropGaussian: Structural Regularization for Sparse-view Gaussian Splatting
Hyunwoo Park, Gun Ryu, Wonjun Kim
TL;DR
Sparse-view 3DGS can overfit because limited views provide insufficient scene cues, reducing novel-view quality. DropGaussian randomly removes Gaussians during training, progressively increasing the dropping ratio to improve visibility and gradients; it achieves competitive benchmark performance without added computational complexity, though its scaling hyperparameter requires dataset-specific tuning.
Problem
Sparse-view 3DGS overfits to limited training views because insufficient appearance and geometric cues hinder scene understanding.
Method
DropGaussian randomly removes Gaussians during training and progressively increases the dropping ratio to improve remaining Gaussians' visibility and updates.
Results
DropGaussian achieves the highest PSNR of 20.76 in the 3-view LLFF setting and competitive performance in 6-view and 9-view settings without increased computational complexity.
Takeaways & Limitations
DropGaussian provides a prior-free, simple regularization strategy that improves sparse-view 3DGS rendering while retaining the original framework's computational complexity.
Takeaways & Limitations
The dropping-rate scaling factor gamma is sensitive to dataset-specific tuning, and its optimal value varies across datasets and tasks.
Abstract
from arXiv · showhide
Recently, 3D Gaussian splatting (3DGS) has gained considerable attentions in the field of novel view synthesis due to its fast performance while yielding the excellent image quality. However, 3DGS in sparse-view settings (e.g., three-view inputs) often faces with the problem of overfitting to training views, which significantly drops the visual quality of novel view images. Many existing approaches have tackled this issue by using strong priors, such as 2D generative contextual information and external depth signals. In contrast, this paper introduces a prior-free method, so-called DropGaussian, with simple changes in 3D Gaussian splatting. Specifically, we randomly remove Gaussians during the training process in a similar way of dropout, which allows non-excluded Gaussians to have larger gradients while improving their visibility. This makes the remaining Gaussians to contribute more to the optimization process for rendering with sparse input views. Such simple operation effectively alleviates the overfitting problem and enhances the quality of novel view synthesis. By simply applying DropGaussian to the original 3DGS framework, we can achieve the competitive performance with existing prior-based 3DGS methods in sparse-view settings of benchmark datasets without any additional complexity. The code and model are publicly available at: https://github.com/DCVL-3D/DropGaussian release.
1. Introduction
3DGS enables efficient, high-quality novel-view rendering but overfits when only sparse input views provide insufficient appearance and geometry cues. DropGaussian addresses this without strong priors by randomly removing Gaussians during training and progressively increasing the dropping ratio.
- 3DGS provides real-time rendering while maintaining novel-view quality, but sparse input views leave insufficient cues for scene appearance and geometry.
- Sparse-view optimization remains challenging, motivating methods that use monocular depth, generative context, or optical-flow priors despite consistency, cost, and stability limitations.
- DropGaussian randomly removes Gaussians during training so remaining Gaussians become more visible and receive larger gradients, balancing optimization under sparse views.
- Overfitting predominantly occurs during later training, so the method progressively increases the dropping ratio to strengthen regularization without affecting initial rendering performance.
2. Related Works
Prior work established efficient 3DGS for novel-view synthesis but highlighted overfitting under sparse inputs. Existing solutions use transfer learning, semantic or patch-based regularization, frequency constraints, depth handling, generative supervision, and optical flow, while DropGaussian offers a different framework-level strategy.
- 2.1. Novel View Synthesis: 3DGS represents scenes with 3D Gaussians and differentiable rasterization, improving training and rendering efficiency for real-time applications.
- 2.1. Novel View Synthesis with Sparse-Views: Sparse input views degrade NeRF and 3DGS performance through overfitting because high-quality rendering typically requires many input images.
- 2.2. Novel View Synthesis with Sparse-Views: Prior sparse-view methods use transfer learning, CLIP semantic consistency, patch-based color-depth regularization, and frequency regularization to improve unseen-view generalization.
- 2.2. Novel View Synthesis with Sparse-Views: DropGaussian randomly removes Gaussians during training and renders all Gaussians at test time, improving visibility while using the complete scene representation for novel views.
- 2.2. Novel View Synthesis with Sparse-Views: 3DGS-specific approaches address sparse views through learnable depth handling, global-local depth normalization, score distillation, and optical-flow correspondence regularization.
3. Proposed Method
DropGaussian regularizes sparse-view 3DGS by randomly removing Gaussians during training, improving the visibility and gradients of those remaining. Its progressively increasing dropping rate targets later-stage overfitting while preserving the standard rendering-loss framework.
- 3.2. DropGaussian: DropGaussian randomly removes Gaussians during training so remaining Gaussians receive more update opportunities under sparse-view conditions.The method is designed to improve Gaussian visibility and parameter updating when distant Gaussians are frequently occluded.
- 3.1. Preliminaries: 3DGS represents scenes with point-based 3D Gaussians defined by centers, scales, rotations, opacities, and color features.The Gaussian basis uses a covariance determined by scale and rotation; color features are typically spherical-harmonic coefficients.
- 3.1. Preliminaries: Rendered pixel colors accumulate Gaussian contributions weighted by RGB colors, projected Gaussian values, opacities, and accumulated transparency.The transparency and alpha terms determine how each Gaussian contributes along the view direction.
- 3.2. DropGaussian: The compensation factor assigns 1/(1-r) to retained Gaussians and 0 to removed ones, maintaining total pixel-color contribution after dropping.This scaling compensates for the reduced cumulative opacity caused by randomly removing Gaussians.
- 3.2. DropGaussian: DropGaussian increases large-gradient Gaussians at greater camera distances and improves robustness to sparse-view novel-view overfitting.The cited analysis counts gradients above the 3DGS densification threshold of 0.0005.
- 3.2. DropGaussian: Because overfitting strengthens later in training, the dropping rate increases with iteration, intensifying regularization over time.The adjustment uses the current iteration t relative to the total training iterations t_total, with scaling factor gamma.
- 3.3. Loss Function: Training uses the conventional 3DGS color reconstruction loss, combining L1 and D-SSIM terms with weighting factor lambda set to 0.2.The loss compares rendered and ground-truth images.
4. Experimental Results
Experiments evaluate DropGaussian for sparse-view synthesis across LLFF, Mip-NeRF360, Blender, and Replica, including quantitative, qualitative, and ablation studies. The method achieves strong rendering performance while retaining simple training changes, though its dropping-rate scale requires dataset-specific tuning.
- Quantitative evaluation: 20.76 PSNR is achieved in the LLFF 3-view setting, surpassing all compared NeRF-based and 3DGS-based methods.For 6-view and 9-view settings, the method remains competitive with state-of-the-art approaches without increased computational complexity.
- Quantitative evaluation: 23.92 PSNR, 0.755 SSIM, and 0.242 LPIPS are reported on Mip-NeRF360, significantly outperforming state-of-the-art methods.These values are reported together for the Mip-NeRF360 evaluation.
- Quantitative evaluation: 25.42 PSNR is obtained on Blender, while the other two metrics are slightly below their best scores.The method is nevertheless described as effective and competitive without additional modules or algorithms.
- Qualitative evaluation: Qualitative results show higher precision, artifact-free renderings, preserved fine details, and fewer artifacts than baseline methods on LLFF.The qualitative evaluation also includes novel-view renderings on Blender and comparisons involving FSGS, CoR-GS, and 3DGS.
- Ablation and limitations: Progressive dropping improves rendering performance over fixed dropping in the LLFF 3-view ablation, while the dropping-rate scale remains a tuning variable.The ablation reports a baseline row of 19.22 PSNR, 0.649 SSIM, and 0.229 LPIPS, and the method temporarily deactivates rather than permanently removing Gaussians.
5. Conclusion
The conclusion presents DropGaussian as a simple regularizer for sparse-view 3DGS that randomly removes Gaussians during training. Progressive application improves visibility and gradient feedback for remaining Gaussians, helping mitigate overfitting and enhance rendering quality.
- 5. Conclusion: DropGaussian randomly removes 3D Gaussians during training to mitigate overfitting in sparse-view 3DGS.The method is described as a simple approach for sparse-view rendering.
- 5. Conclusion: Dropping Gaussians increases visibility and gradient magnitudes for remaining Gaussians, improving their contribution to optimization.The conclusion specifically relates this mechanism to Gaussians far from the camera under sparse-view conditions.
- 5. Conclusion: Progressively applying DropGaussian during training further enhances the visual quality of rendering results.The conclusion identifies progressive application as an additional component of the proposed approach.