PaperlayerView exact revision on arXiv

Source-linked AI summary

NeuDonatello: Uncertainty-Aware Framework for Accurate Neural SDF Learning

Alvin Jinsung Choi, Wanhee Kim, Taeyun Kim, Dasol Hong, Wooju Lee, Hyun Myung

arXiv:2608.26504v1cs.CV

TL;DR

RGB-only neural surface reconstruction is challenged by ambiguous geometry and spatially varying uncertainty that existing approaches often overlook. NeuDonatello estimates SDF uncertainty with Monte Carlo sampling, adapts regularization and SDF-to-density conversion accordingly, and reports state-of-the-art reconstruction across diverse scenes using posed RGB images.

  • Problem

    RGB-only multi-view reconstruction is under-constrained because textureless regions, occlusions, lighting variations, and limited views create geometrically ambiguous surfaces.

  • Method

    NeuDonatello estimates spatially varying SDF uncertainty with Monte Carlo sampling, then conditions geometric regularization and SDF-to-density scale on that uncertainty.

  • Results

    NeuDonatello achieves state-of-the-art performance with consistent quantitative and perceptual gains, particularly in low-texture and sparsely observed regions.

  • Takeaways & Limitations

    Explicit SDF uncertainty modeling supports high-fidelity RGB-only reconstruction in geometrically challenging regions without auxiliary depth or normal priors.

  • Takeaways & Limitations

    Reconstruction quality depends on early geometry estimation, while Monte Carlo sampling increases training cost and uncertainty-weighted losses require hyperparameter tuning.

Abstract

from arXiv · show

Neural surface reconstruction has emerged as a powerful paradigm for recovering high-quality 3D surfaces from multi-view images. However, recovering accurate geometry solely from RGB images remains challenging due to uncertainties arising from textureless regions, occlusions, and inherent scene ambiguities. Existing methods often overlook such uncertainties, leading to inaccurate estimates of the signed distance function (SDF). We introduce NeuDonatello, a novel framework that models and leverages SDF uncertainty to improve surface reconstruction. Central to our approach is to model spatially varying uncertainty using a Monte Carlo sampling strategy. Using this uncertainty, we develop an adaptive regularization that selectively strengthens geometric constraints where RGB supervision is unreliable, avoiding incorrect surface reconstruction. We further introduce an uncertainty-aware scale parameter for the SDF-to-density conversion. Conditioned on uncertainty, this design enables more accurate modeling of spatially varying densities. Extensive experiments demonstrate that NeuDonatello achieves state-of-the-art reconstruction accuracy, with robust performance across diverse scenes using only posed RGB images.

1 Introduction

NeuDonatello addresses ambiguity in RGB-only surface reconstruction by estimating spatially varying SDF uncertainty and using it to adapt geometric constraints and density conversion. Experiments report state-of-the-art performance and improvements in challenging regions.

  • Multi-view RGB reconstruction is under-constrained because different surfaces can produce similar renderings, especially in textureless, variably lit, or sparsely viewed regions.
  • Experiments on ScanNet++ and Tanks and Temples report consistent quantitative and visual gains, including substantial improvements in textureless and sparsely observed regions.
  • NeuDonatello estimates spatially varying SDF uncertainty from posed multi-view RGB images using Monte Carlo sampling.The framework is designed to distinguish geometrically ambiguous regions from well-constrained areas.
  • Adaptive regularization strengthens geometric constraints where RGB supervision is unreliable, enabling reconstruction in ambiguous regions without depth or normal priors.
  • The uncertainty-aware SDF-to-density conversion conditions its scale parameter on position, viewing direction, and uncertainty to mitigate density bias and improve surface fidelity.

2 Related Works

Neural surface reconstruction builds on implicit SDF representations and volume rendering, while NeuDonatello extends this line by explicitly modeling uncertainty. Its overview combines uncertainty estimation, adaptive regularization, and uncertainty-aware density conversion.

  • Neural surface methods represent geometry as an SDF and extract surfaces through its zero-level set within differentiable volume-rendering frameworks.
  • NeuDonatello estimates SDF uncertainty through Monte Carlo sampling supervised by a negative log-likelihood loss.
  • The framework uses uncertainty to strengthen regularization in uncertain regions and weaken it in confident regions.
  • Its uncertainty-aware scale parameter incorporates spatial and directional ambiguity into SDF-to-density conversion, addressing limitations of prior scaling approaches.
  • Prior methods apply geometric regularization uniformly or use deterministic SDF predictions, disregarding spatially varying uncertainty.

3 Proposed Method

NeuDonatello models spatially varying SDF uncertainty from multi-view RGB images and uses it to guide geometric regularization and SDF-to-density conversion. This uncertainty-aware pipeline targets ambiguous regions while preserving details in well-constrained areas.

  • 3.3 Uncertainty-Aware Adaptive Geometric Regularization: This spatially varying weighting guides ambiguous regions toward plausible geometry while preserving fine details where reliable cues are available.The approach addresses the weakness of uniform regularization, which does not account for spatial uncertainty and is less effective in ambiguous regions.
  • 3.2 Uncertainty Modeling: NeuDonatello represents SDF predictions probabilistically and estimates their uncertainty using Monte Carlo sampling through nonlinear volume rendering.The geometry and uncertainty networks predict SDF means and variances; sampled SDFs produce multiple rendered colors whose variance measures pixel-space uncertainty.
  • 3.2 Uncertainty Modeling: The method propagates sampled SDF predictions into rendered color estimates, avoiding analytical uncertainty derivatives through the nonlinear rendering pipeline.Each sampled SDF is converted to density and rendered with view-dependent radiance, producing Nmc pixel-color estimates; their sample variance captures uncertainty.
  • 3.3 Uncertainty-Aware Adaptive Geometric Regularization: Adaptive regularization increases geometric constraint strength in high-uncertainty regions and decreases it where RGB supervision is reliable.The uncertainty is normalized across the current batch and used to weight eikonal and smoothness losses through a parameterized weighting function.
  • 3.4 Uncertainty-Aware SDF-to-Density Conversion: The uncertainty-aware SDF-to-density conversion conditions its scale parameter on geometric features, viewing direction, and SDF uncertainty.A scale network predicts the parameter from z(r(t)), d, and β_f^2(r(t)), allowing density transitions to respond to spatial and directional ambiguity.

4 Experiments

Experiments on ScanNet++ and Tanks and Temples show that NeuDonatello reconstructs surfaces accurately across challenging scenes, while uncertainty-aware components improve ambiguous-region geometry and remain computationally tractable.

  • Datasets and evaluation: NeuDonatello is evaluated on eight ScanNet++ scenes and six Tanks and Temples training-subset scenes using standard mesh-reconstruction benchmarks.Meshes are extracted with marching cubes at resolution 2,048 and assessed using Accuracy, Completeness, Chamfer Distance, Precision, Recall, and F1-score.
  • ScanNet++ results: NeuDonatello achieves state-of-the-art ScanNet++ performance and surpasses RGB-only methods using uniform regularization, as well as MonoSDF with monocular priors.The comparison uses ScanNet++ version 2, which differs from the version used in NeuRodin.
  • ScanNet++ results: Uncertainty-aware modules reduce artifacts in ambiguous ScanNet++ regions, preserving textureless ceilings and reflective floors where other methods produce holes or collapse.Neuralangelo produces a ceiling hole, while NeuRodin exhibits floor collapse in the cited qualitative comparison.
  • Tanks and Temples results: NeuDonatello achieves the highest mean F1-score on Tanks and Temples and consistently outperforms prior methods across indoor and outdoor scenes.Performance is particularly strong on sparse-view and textureless scenes such as Barn and Meetingroom.
  • Ablations: Ablations show that NLL loss enables uncertainty estimation, adaptive regularization improves ambiguous-region fidelity, and uncertainty-conditioned scale improves density modeling.The NLL loss alone does not directly enhance surface quality and can produce artifacts in highly uncertain regions; conditioning scale on uncertainty better reflects SDF confidence.
  • Uncertainty and efficiency analysis: Uncertainty-guided sparsification outperforms random removal, approaches the ensemble baseline, and is substantially faster, while GPU-parallel Monte Carlo sampling adds only modest overhead.Low AUSE values support correspondence between predicted uncertainty and geometric error, and parallel sampling avoids repeated full-network forward passes.

Conclusion

NeuDonatello estimates SDF uncertainty from posed multi-view images and uses it to guide geometric regularization and SDF-to-density conversion. Experiments show improved reconstruction, especially in low-texture and sparse-view regions, while future work targets non-posed inputs and dynamic scenes.

  • Conclusion: NeuDonatello estimates spatially varying SDF uncertainty from posed multi-view images using Monte Carlo sampling and uses it to guide geometry reconstruction.The framework identifies geometrically ambiguous regions and adaptively modulates geometric regularization strength.
  • Conclusion: The framework conditions its SDF-to-density scale parameter on position, direction, and uncertainty to reduce bias and improve surface accuracy.This design complements uncertainty-guided geometric regularization.
  • Conclusion: Experiments on ScanNet++ and Tanks and Temples show that NeuDonatello outperforms prior methods, particularly in low-texture or sparse-view regions.The conclusion highlights uncertainty modeling as important for neural reconstruction.
  • Conclusion: Future work will extend the framework to non-posed inputs and dynamic scenes for broader real-world applicability.The current framework is described as operating on posed multi-view images.

A.2 Training Details

NeuDonatello uses a staged training pipeline with uncertainty-aware SDF-to-density conversion, adaptive regularization, gradient estimation, and stage-specific auxiliary losses. Dataset-specific sampling and evaluation protocols complement these architectural and optimization choices.

  • The framework uses a compact hash-grid proposal network, learned appearance embeddings, and a dedicated background network for outdoor scenes.
  • Training and evaluation: ScanNet++ uses 512 sampled pixels per iteration, while Tanks and Temples increases refinement-stage sampling from 1024 to 8192 pixels.
  • Initial stage: The initial stage learns uncertainty with NLL loss and uses VolSDF conversion with uncertainty-aware scale su, stochastic gradients, and bias correction.
  • Adaptive regularization: Uncertainty is normalized using batchwise minimum and maximum values, with ε added for numerical stability.
  • Gradient estimation: The stochastic x-gradient uses symmetric SDF evaluations with εx sampled uniformly between 0 and εmax.
  • Refinement stage: The refinement stage freezes the uncertainty network, removes NLL and explicit bias correction, and adds smoothness and Lipschitz color regularization.

B.1 Analysis on Uncertainty Estimation

The uncertainty estimates correspond to unreliable depth predictions in ambiguous image regions. Uncertainty-aware optimization then adapts training in those regions, producing more accurate and consistent geometry.

  • High SDF uncertainty aligns with inaccurate or inconsistent depth predictions caused by lighting variation and textureless surfaces.
  • Uncertainty-aware optimization identifies ambiguous regions and adaptively adjusts the optimization process, improving geometric accuracy and depth consistency.

B.2 Analysis on Monte Carlo Sampling

Increasing Monte Carlo samples generally improves uncertainty estimation and reconstruction quality, but gains peak at a moderate sample count. NeuDonatello therefore selects Nmc = 10 to balance quality and computational cost.

  • Nmc = 10 provides the peak mean reconstruction quality across eight representative ScanNet++ scenes.
  • Nmc = 20 adds computational complexity, slows optimization, and introduces training-signal variance without noticeable reconstruction gains.
  • With Nmc = 3, uncertainty localization is noisy and incomplete, while more samples improve identification of geometrically ambiguous regions.

B.3 Analysis on Uncertainty-Aware Adaptive Geometric Regularization

Adaptive geometric regularization is most effective after initial geometry and uncertainty estimates stabilize. Applying it too early can over-regularize noisy geometry and distort surfaces in uncertain regions.

  • Applying adaptive regularization in both stages reduces performance compared with applying it only during refinement.
  • The ablation compares regularization settings using reconstructed meshes and uncertainty maps across different Monte Carlo sample counts.
  • Early strong regularization can distort surfaces because geometry, predicted normals, and uncertainty estimates remain unreliable.

B.4 Analysis on Uncertainty-Aware SDF-to-Density Conversion

The uncertainty-aware SDF-to-density conversion assigns scale values according to geometric reliability and uses scheduling to localize surfaces more accurately. Ablation evidence also shows that applying adaptive geometric regularization only during refinement preserves details better than applying it in both stages.

  • Adaptive regularization ablation: Applying adaptive geometric regularization in both stages oversmooths complex structures, whereas refinement-only application better preserves details.The refinement-only strategy leads to more accurate reconstruction in the illustrated complex-geometry region.
  • Uncertainty-aware scale behavior: Ambiguous regions receive high scale values, while reliable edges and texture-rich regions receive low values during the initial stage.The scale parameter is visualized as bright in ambiguous areas and dark in geometrically reliable areas.
  • Uncertainty-aware scale behavior: Upper-bound scheduling reduces the scale parameter across all regions during refinement, enabling accurate surface localization.The upper bound decreases exponentially during both initial and refinement stages.

C.1 Comparison with Explicit Methods

NeuDonatello is compared with explicit 3D Gaussian Splatting mesh methods and DebSDF across ScanNet++ and Tanks and Temples. It consistently reports stronger quantitative and qualitative reconstruction performance, including better detail preservation in ambiguous regions.

  • Comparison with 3DGS-based methods: NeuDonatello achieves the highest performance on every metric on ScanNet++, outperforming the strongest 3DGS-based baselines.Qualitative results show cleaner surfaces, fewer artifacts, and better preservation of geometric details.
  • Comparison with 3DGS-based methods: NeuDonatello achieves the top F1-score across all methods on Tanks and Temples.The result is reported across complex environments in the quantitative comparison.
  • Comparison with uncertainty-aware methods: Compared with DebSDF, NeuDonatello preserves thin structures such as cables instead of oversmoothing fine geometric details.NeuDonatello uses uncertainty as an intrinsic property of the RGB-only inverse-rendering process.
  • Additional ScanNet++ results: Per-scene ScanNet++ results show consistently improved reconstructions, particularly on textureless surfaces, lighting variations, and occlusions.The improvements are reported in both quantitative and qualitative evaluations.
  • Additional Tanks and Temples results: Additional Tanks and Temples results show visually compelling reconstructions that capture fine-grained details in indoor and outdoor scenes.The method maintains surface fidelity in challenging regions.

D Limitation

NeuDonatello’s reconstruction quality depends on the initial geometry estimate, while its uncertainty estimation increases training cost and introduces hyperparameter sensitivity. The paper also notes deployment concerns involving computational accessibility and privacy.

  • Method limitations: Severely incorrect early geometry may not be fully recovered during the refinement stage.Final reconstruction quality is influenced by the initial stage.
  • Method limitations: Monte Carlo uncertainty estimation increases computational cost during training.This may limit broader accessibility.
  • Method limitations: Uncertainty-weighted loss terms make the model sensitive to hyperparameters and require careful tuning for stable training.The limitation follows from directly modifying loss-term weights based on uncertainty.
  • Societal considerations: Reconstructing real-world environments without consent raises potential privacy concerns.The paper identifies this as a societal consideration for deployment.
Loading 2608.26504v1…