Source-linked AI summary
UGOD: Uncertainty-Guided Opacity and Dropout for Sparse-View 3D Gaussian Splatting
Zhihao Guo, Peng Wang, Zidong Chen, Xiangyu Kong, Yan Lyu, Guanyu Gao, Chenghao Qian, Ziyang Wang, Xinqi Fan, Liangxiu Han
TL;DR
Sparse-view observations leave some Gaussians weakly constrained, while alpha blending still accumulates their contributions. UGOD learns view-dependent uncertainty to modulate opacity and regularise training, and experiments on Mip-NeRF 360 and LLFF report improved quality–compactness trade-offs, especially under severe view sparsity.
Problem
Sparse-view reconstruction leaves some Gaussians weakly constrained, but conventional opacity cannot distinguish reliable from ambiguous contributions for the current view during cumulative alpha blending.
Source: §1 · p. 2Method
UGOD predicts view-dependent per-Gaussian uncertainty and uses it for differentiable opacity modulation before compositing and gradient-detached soft dropout during training.
Source: §1 · p. 3, Notation and overview · p. 6Results
Experiments on Mip-NeRF 360 and LLFF show improved quality–compactness trade-offs in evaluated sparse-view protocols, with the most consistent gains under severe view sparsity.
Source: §5 · p. 18Takeaways & Limitations
The results support view-dependent uncertainty as a rendering-time control for sparse-view reconstruction.
Source: §1 · p. 3, §5 · p. 18The predicted score is not a calibrated epistemic-uncertainty estimate; it is learned through photometric loss via opacity modulation without an uncertainty target or calibration loss.
Source: Network · p. 8
Abstract
from arXiv · showhide
Sparse-view 3D Gaussian Splatting is prone to overfitting because limited observations leave many Gaussian primitives weakly constrained, yet their contributions are still accumulated through alpha blending. Without uncertainty estimation, the renderer cannot distinguish unreliable primitives from well-constrained ones, allowing their erroneous contributions to corrupt novel-view synthesis. We introduce UGOD, an uncertainty-guided framework that estimates a view-dependent uncertainty score for each Gaussian and uses it to regulate its rendering contribution. A lightweight uncertainty head conditioned on Gaussian attributes and viewing direction predicts this score, which then drives a differentiable opacity-modulation mechanism that attenuates high-uncertainty primitives before compositing. During training, a detached soft-dropout branch applies an uncertainty-controlled continuous keep mask to discourage the model from relying on poorly constrained Gaussians and thereby reduce overfitting. Crucially, detaching the uncertainty score prevents gradients from this stochastic regulariser from biasing or collapsing the uncertainty prediction. Experiments on Mip-NeRF~360 and LLFF show that UGOD improves sparse-view novel-view synthesis while producing more compact Gaussian representations than the compared methods. These results demonstrate that Gaussian uncertainty provides an effective rendering-time control for sparse-view reconstruction.
1 Introduction
Sparse observations leave some Gaussians weakly constrained, yet conventional alpha blending can still give them substantial influence. UGOD estimates view-dependent uncertainty and uses it to regulate opacity during rendering and training; the paper reports improved synthesis quality with compact representations across the evaluated sparse-view protocols.
1 Introduction
- Sparse-view observations can leave Gaussians weakly supervised, while learned opacity still lets unreliable primitives inject incorrect colour during alpha blending.This can reinforce a training-view explanation that does not generalise.Source: §1 · p. 2
- Existing sparse-view approaches add priors, densify geometry, or regularise primitives, but do not attenuate opacity according to current-view uncertainty before blending.UGOD targets the rendering contribution of weakly supported Gaussians directly.Source: §1 · pp. 2–3
- UGOD leads all six average-PSNR and final-Gaussian-reduction comparisons against 3DGS* across Mip-NeRF 360 and LLFF sparse-view protocols.The figure covers Mip-NeRF 360 at 24 and 8 views and LLFF at 3 views.Source: Figure 1 · p. 3, §1 · p. 3
- UGOD predicts view-dependent uncertainty from Gaussian attributes, appearance features, and viewing direction, then differentiably attenuates high-uncertainty opacity before compositing.A detached soft-dropout branch also uses uncertainty for training-time regularisation without allowing that stochastic branch to distort the uncertainty prediction.Source: §1 · p. 3
2 Related Work
Related work improves Gaussian representations through rendering and point-management advances, sparse-view geometric guidance and regularisation, and uncertainty estimation. The paper distinguishes UGOD by learning view-dependent uncertainty for online opacity control during reconstruction.
2.1 3D Gaussian Splatting
- 3DGS uses projected, depth-sorted anisotropic Gaussians for rasterisation, but its quality remains sensitive to observation coverage and initialisation accuracy.Prior opacity work addresses physical interpretation or material dependence rather than view-dependent reliability under weak constraints.Source: §2.1 · pp. 3–4
2.2 Sparse 3DGS Reconstruction
- Sparse-view methods use geometric priors, densification, or primitive-level regularisation, but do not estimate current-view uncertainty to control compositing contributions.The cited approaches include depth and diffusion priors, point-cloud densification, and selective dropping or pruning.Source: §2.2 · p. 4
2.3 Uncertainty Estimation in Gaussian Splatting
- Uncertainty methods estimate parameter or predictive uncertainty for view selection, maps, or downstream decisions; UGOD instead learns uncertainty jointly and uses it to regulate reconstruction.The closest cited post-hoc method fits uncertainty to residuals after freezing 3DGS, whereas UGOD modulates opacity online and drives detached soft dropout during training.Source: §2.3 · p. 5
3.1 Preliminaries
3DGS represents scenes with anisotropic Gaussians and renders their projected footprints through ordered alpha compositing. Its learned opacity controls each primitive’s compositing weight, but a view-independent scalar cannot distinguish reliable from ambiguous contributions under sparse or noisy initialisation.
3.1 Preliminaries
- 3DGS initialises anisotropic Gaussians from an SfM point cloud and parameterises each by position, rotation, scale, opacity, and appearance.The covariance uses a rotation matrix and diagonal scale matrix, with compact quaternion and scale-vector parameterisations in the implementation.Source: §3.1 · p. 5
- Projected Gaussian footprints are rasterised and ordered front to back, with learned opacity determining screen-space compositing weights and the rendered RGB colour.Very low-opacity primitives are pruned during density control, while remaining primitives continue to refine the representation.Source: §3.1 · pp. 5–6
- Under sparse or noisy initialisation, view-independent opacity cannot distinguish a reliably supported contribution from one ambiguous for the current camera.Opacity is jointly optimised with the other Gaussian attributes and influences density control.Source: §3.1 · p. 6
3.2 Per-Gaussian Uncertainty Prediction
UGOD predicts view-dependent uncertainty per visible Gaussian and uses it to modulate opacity and regularize training. Its uncertainty head combines spatial, geometric, appearance, and viewing-direction inputs, while the predicted score is not calibrated epistemic uncertainty.
3.2 Per-Gaussian Uncertainty Prediction
- UGOD’s uncertainty score drives differentiable opacity modulation and detached, training-only soft dropout; inference instead uses deterministic raw-uncertainty modulation.The framework jointly optimizes Gaussians using supervised depth, pseudo-view depth, and photometric losses.Source: Notation and overview · p. 6, §3.2 · p. 6, Figure 2 · p. 7
SH compression.
- The head compresses non-DC SH coefficients into a three-channel magnitude summary, preserving view-dependent strength while discarding directional detail.Together with DC colour, this forms a six-dimensional appearance descriptor.Source: SH compression · pp. 7–8
Input feature.
- The head input is a 40-dimensional concatenation whose listed components have respective dimensions 3, 24, 4, 3, 3, and 3; the first two are unit view direction and hashed position encoding.Source: Input feature · p. 8
Network.
- A fully fused MLP maps the 40-dimensional input to one view-conditioned uncertainty score per visible Gaussian.It uses two 32-neuron hidden layers with LeakyReLU activations and a multi-resolution hash-grid position encoding.Source: Network · p. 8
- The score is not calibrated epistemic uncertainty: it is learned through photometric loss via opacity modulation, without an uncertainty target or calibration loss.The head is also frozen after validation PSNR decreases across two consecutive evaluations.Source: §3.2 · p. 6, Network · p. 8
3.2.2 Uncertainty-Guided Opacity Modulation
UGOD attenuates uncertain Gaussians before compositing and supplements this deterministic gate with detached stochastic dropout during training. Relative gating is used for optimization, while inference uses raw uncertainty for per-Gaussian attenuation.
3.2.2 Uncertainty-Guided Opacity Modulation
- The differentiable opacity gate reduces uncertain primitives’ compositing contribution while preserving their photometric supervision and uncertainty-learning gradient.Relative-uncertainty gating is used after warm-up during training; evaluation uses the raw-uncertainty rule instead.Source: §3.2.2 · pp. 8–9
Rasterised uncertainty map.
- The rasterised uncertainty map uses standard, unmodulated compositing weights and is for analysis only.It is distinct from the weights used for training and inference.Source: Rasterised uncertainty map · pp. 9–10
3.2.3 Detached Soft Dropout (Tail Regulariser)
- Detached Concrete soft dropout stochastically regularizes high-uncertainty Gaussians during training, discouraging co-adaptation to weakly constrained evidence.Dropout is disabled at test time, when every primitive is retained and deterministically attenuated by raw uncertainty.Source: §3.2.3 · p. 10
Gradient flow.
- The dropout branch uses detached uncertainty, so its probability path does not update the head; the opacity branch remains differentiable and learns through photometric loss.This separation prevents the head from inflating uncertainty merely to trigger dropout.Source: §3.2.2 · p. 9, Gradient flow · pp. 10–11
3.2.4 Training Objective
The training objective combines observed-view monocular depth regularization, pseudo-view depth regularization, and photometric supervision. A frozen DPT-Hybrid model supplies the depth priors.
3.2.4 Training Objective
- A frozen DPT-Hybrid estimator supplies monocular depth priors for training images, with Pearson correlation accommodating scale and sign ambiguity.This depth regularization is adopted from FSGS rather than presented as a UGOD contribution.Source: §3.2.4 · p. 11
- From iterations 500 to 5500, sampled pseudo cameras provide rendered RGB and depth for an additional DPT-based pseudo-view depth prior.The pseudo-view loss is ramped during its first 500 active iterations, and gradients flow through rendered depth rather than DPT preprocessing.Source: §3.2.4 · p. 11
- The Gaussians and uncertainty head are jointly optimized; the head receives implicit supervision through differentiable opacity modulation without auxiliary uncertainty-calibration loss, and pseudo-view weight λp is 0.5 by default or 0.03 for outdoor MiP-NeRF 360 scenes.Source: §3.2.4 · p. 11
Default hyper-parameters.
- The complete uncertainty-head, gating, dropout, and early-stopping settings are reported in Supplementary Table 1.Source: Default hyper-parameters · p. 12
4 Experiments
UGOD is evaluated across sparse-view protocols on MiP-NeRF 360 and LLFF, with scene-average results, component and encoding ablations, qualitative comparisons, and uncertainty visualizations. The results show a quality–compactness trade-off that varies by protocol and scene, alongside uncertainty patterns linked to opacity modulation.
4.1 Experimental Setup
- The evaluation uses FSGS-style 24-view MiP-NeRF 360 and 3-view LLFF protocols, adds an 8-view MiP-NeRF 360 setting, and reports held-out-view image metrics and Gaussian counts.The camera splits and image resolutions define the protocol, so absolute metrics may differ from prior reports using other splits or resolutions.Source: §4.1 · p. 12
4.2 Quantitative Comparison
- Across LLFF and MiP-NeRF 360 protocols, Full UGOD generally improves scene-average quality and compactness, with scene- and metric-specific exceptions.On LLFF, it achieves PSNR 19.33, SSIM 0.637, and LPIPS 0.245 with 102,280 Gaussians, 2.5× fewer than 3DGS*; on 8-view MiP-NeRF 360, it has the highest average PSNR (14.14) and SSIM (0.368) with 95,662 Gaussians, 9.0× fewer than 3DGS*. At 24 views, it has the highest average PSNR (19.09) and 129,498 Gaussians, 8.0× fewer than 3DGS*.Source: §4.2 · pp. 12–13, Table 1 · p. 13
4.3 Ablation Study
- The full configuration improves all three image metrics and reduces Gaussian count on LLFF, while the 8-view and 24-view ablations show quality–compactness trade-offs.At 8 views, adding soft dropout alone reduces average Gaussian count from 108,008 to 102,644 but lowers PSNR from 13.94 to 12.97; the full model also includes depth supervision, so this is not an isolated dropout effect.Source: §4.3 · pp. 13–14, Table 2 · p. 14
- In the controlled 24-view kitchen ablation, position-only HashGrid encoding with six levels achieves the best PSNR (15.0757), SSIM (0.4628), and LPIPS (0.6396).Allocating levels to scale or rotation, or increasing position encoding to seven levels, degrades all three metrics; the selected configuration encodes only position and concatenates other attributes in raw form.Source: §4.3 · p. 14, Table 3 · p. 16
4.4 Qualitative Comparison
- Qualitative examples show Full UGOD recovering bicycle spokes and fence more clearly, and outperforming FSGS in displayed-view PSNR while using fewer Gaussians on three LLFF scenes.The bicycle example reports 13.11 dB and 59.5K Gaussians for Full UGOD, versus 11.01 dB and 1460.8K for 3DGS*; the LLFF comparisons are horns, flower, and fern.Source: Figure 3 · p. 15, §4.4 · p. 15
4.5 Uncertainty Dynamics and Opacity Response
- On three-view LLFF fern, most predicted uncertainties shift toward low values during training while a high-uncertainty tail persists, and higher uncertainty corresponds to smaller opacity-gate multipliers at 6k iterations.The fraction with relative uncertainty above 0.8 rises from approximately 27% at 1k iterations to 40% at 2k, then falls to approximately 36% at 6k; the gate response approaches its 0.70 floor for high-uncertainty Gaussians.Source: §4.5 · p. 16, Figure 4 · p. 17
4.6 Photometric Residuals and Uncertainty Visualisation
- For three-view LLFF horns and fern and 24-view MiP-NeRF 360 bonsai, the visualization contrasts baseline image-space residual maps with UGOD’s predicted uncertainty map.The baseline residual heatmaps measure reconstruction error, whereas UGOD’s map represents predicted uncertainty.Source: Figure 5 · p. 17, §4.6 · p. 17
5 Conclusion
UGOD uses view-dependent per-Gaussian uncertainty to regulate rendering and training, improving the quality–compactness trade-off in evaluated sparse-view protocols, with gains most consistent under severe view sparsity.
5 Conclusion
- UGOD learns view-dependent per-Gaussian uncertainty from Gaussian attributes and viewing direction.The uncertainty signal is used in two complementary mechanisms: opacity modulation before compositing and gradient-detached soft dropout during training.Source: §5 · p. 18
- Uncertainty-guided opacity modulation attenuates high-uncertainty contributions before alpha compositing, while detached soft dropout regularises high-uncertainty primitives without directly altering the uncertainty estimator.Source: §5 · p. 18
- Experiments on MiP-NeRF 360 and LLFF show improved quality–compactness trade-offs, with the most consistent gains under severe view sparsity.Per-scene benefits vary by metric and scene, especially when more input views reduce reconstruction ambiguity.Source: §5 · p. 18
S1 Supplementary Material
The supplementary material provides mathematical derivations, implementation details, per-scene comparisons, and component ablations for UGOD; it explains the main-method operators without adding a training objective.
S1 Supplementary Material
- The supplement covers derivations, implementation, per-scene comparisons, and component ablations, with its mathematical analysis explaining main-method operators rather than adding a training objective.Source: S1 Supplementary Material · p. 21, S1.1 Supplementary Mathematical Analysis · p. 21
S1.1.1 SH Compression: Compact Appearance Descriptor
- The appearance descriptor uses channel-wise L1 mean non-DC SH coefficients, while the uncertainty input combines hashed position with view direction and Gaussian appearance and shape attributes.The rest-energy vector is a compact magnitude surrogate, not an exact spectral-energy or Parseval-equivalent quantity.Source: S1.1.1 SH Compression: Compact… · p. 21, S1.1.2 The Input Feature Vector:… · p. 21
S1.1.3 Robust Normalisation: Non-Parametric Order Statistics
- Median/MAD normalization makes modulation and dropout depend on relative uncertainty among visible Gaussians, while a positive δ stabilizes near-identical values and clipping limits extremes.Source: S1.1.3 Robust Normalisation:… · pp. 21–22
S1.1.4 Alpha-Compositing Sensitivity
- Opacity changes affect both a Gaussian’s direct colour contribution and later transmittance; the uncertainty-based multiplier decreases with uncertainty and remains bounded below by β.The positive floor preserves a direct gradient path through modulation.Source: S1.1.4 Alpha-Compositing Sensitivity · p. 22, S1.1.5 Opacity-Modulation Bounds and… · pp. 22–23
S1.1.6 Detached Soft Dropout
- Detached soft dropout applies an uncertainty-controlled continuous keep mask during training without directly updating the uncertainty head or incentivising uncertainty inflation to increase dropout.The Concrete-mask temperature controls relaxation sharpness; as it approaches zero, the mask approaches a binary keep decision.Source: S1.1.6 Detached Soft Dropout · p. 23
S1.1.7 Depth-Regularisation Schedule
- Pseudo-view depth regularisation operates within a prescribed interval, ramps to full weight over 500 iterations, and samples a pseudo camera every S iterations for online DPT-Hybrid depth estimation.At inference, the procedure uses raw-uncertainty opacity and disables relative-uncertainty modulation and stochastic dropout.Source: S1.1.7 Depth-Regularisation Schedule · pp. 23–24, S1.1.8 Training and Inference Weights · p. 24
S1.1.9 Validation-Based Early Stopping
- Early stopping freezes the uncertainty head after P consecutive validation evaluations with decreasing PSNR while Gaussian parameters continue training; implementation tables list default hyperparameters and training/inference procedure.Source: S1.1.9 Validation-Based Early Stopping · p. 24, S1.2 Implementation Details · pp. 24–25, Table 1 · p. 25
S1.3 Per-Scene Quantitative Results
- The supplement reports scene-wise quality and Gaussian counts across sparse-view protocols and compares modulation-only, dropout, and full-model ablations, exposing scene-dependent quality–compactness trade-offs.The result tables cover MiP-NeRF 360 at 8 and 24 views and LLFF at 3 views.Source: S1.3 Per-Scene Quantitative Results · pp. 25–26, S1.4 Component Ablation Details · p. 26, Table 2 · p. 27