Source-linked AI summary
Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction
Qiancheng Fu, Qingshan Xu, Yew-Soon Ong, Wenbing Tao
TL;DR
Neural implicit surface methods for multi-view reconstruction often lack explicit geometry constraints, leaving a gap between volume-rendering supervision and point-based SDF modeling. Geo-Neus directly locates the SDF zero-level set and applies sparse SFM geometry plus MVS photometric consistency, achieving high-quality reconstruction across thin and smooth structures while outperforming prior methods.
Problem
Existing volume-rendering approaches lack explicit multi-view geometry constraints and can fail to produce geometry-consistent surfaces.
Method
Geo-Neus directly locates SDF zero-level sets and explicitly supervises them with sparse SFM points and MVS photometric consistency.
Results
Geo-Neus produces high-quality surfaces for complex thin structures and large smooth regions and outperforms traditional and neural implicit state-of-the-art methods by a large margin.
Takeaways & Limitations
Explicit multi-view geometry constraints focus SDF optimization on the true surface and support geometry-consistent reconstruction.
Takeaways & Limitations
The method assumes image colors approximate object colors and uses the first ray-surface intersection when multiple intersections occur because of occlusion.
Abstract
from arXiv · showhide
Recently, neural implicit surfaces learning by volume rendering has become popular for multi-view reconstruction. However, one key challenge remains: existing approaches lack explicit multi-view geometry constraints, hence usually fail to generate geometry consistent surface reconstruction. To address this challenge, we propose geometry-consistent neural implicit surfaces learning for multi-view reconstruction. We theoretically analyze that there exists a gap between the volume rendering integral and point-based signed distance function (SDF) modeling. To bridge this gap, we directly locate the zero-level set of SDF networks and explicitly perform multi-view geometry optimization by leveraging the sparse geometry from structure from motion (SFM) and photometric consistency in multi-view stereo. This makes our SDF optimization unbiased and allows the multi-view geometry constraints to focus on the true surface optimization. Extensive experiments show that our proposed method achieves high-quality surface reconstruction in both complex thin structures and large smooth regions, thus outperforming the state-of-the-arts by a large margin.
1 Introduction
Existing neural implicit surface methods rely on volume-rendered color supervision, creating a gap between rendered colors and true geometry. Geo-Neus directly locates SDF zero-level sets and applies explicit multi-view constraints for geometry-consistent reconstruction.
- 1 Introduction: Existing approaches optimize SDF representations mainly through volume-rendered color fields rather than explicit multi-view geometry constraints.Volume rendering aggregates sampled colors along rays, which can diverge from surface geometry.
- 1 Introduction: Rendered colors can be high quality while estimated surface colors fail to preserve object geometry, producing imprecise surfaces.The NeuS example illustrates the gap between rendered colors and colors predicted at the SDF zero-level surface.
- 1 Introduction: Geo-Neus directly locates the SDF zero-level set and optimizes it using sparse SFM geometry and multi-view stereo photometric consistency.These constraints explicitly supervise the SDF network rather than relying only on rendering loss.
- 1 Introduction: The method is designed to reconstruct both complex thin structures and large smooth regions while producing geometry-consistent surfaces.The paper reports that Geo-Neus greatly outperforms traditional and neural implicit surface reconstruction methods.
2 Related work
Related work spans traditional multi-stage reconstruction, continuous implicit surface representations, and neural implicit methods optimized through volume rendering. Geo-Neus is presented against these approaches as an explicit geometry-supervised alternative.
- Traditional multi-view 3D reconstruction: Traditional reconstruction uses SFM, dense MVS, point-cloud fusion, and surface reconstruction in multiple stages to recover surfaces from multi-view images.Intermediate depth maps and point clouds can accumulate errors and leave surfaces incomplete.
- Implicit representation of surface: Implicit representations model surfaces continuously and allow extraction at arbitrary resolution, unlike resolution-limited voxel or mesh representations.The related-work discussion distinguishes explicit and implicit surface representations by their geometric encoding.
- Neural implicit surface reconstruction: Neural implicit methods represent geometry with neural fields or SDF zero-level sets and use volume rendering to connect colors with geometry.Methods discussed include IDR, MVSDF, VolSDF, NeuS, and UNISURF.
- Neural implicit surface reconstruction: Neural implicit reconstruction can improve completeness, especially for non-Lambertian cases, but complex structures, flat planes, and sharp corners remain challenging.These limitations motivate explicit geometry constraints in Geo-Neus.
3 Method
Geo-Neus analyzes the gap between volume-rendered colors and point-based SDF geometry, then directly supervises the SDF zero-level surface using sparse SfM geometry and multi-view photometric consistency.
- Bias in color rendering: Volume rendering optimizes estimated colors through weighted samples along camera rays, while surface reconstruction focuses on the object's surface rather than color.The rendered color is formed by a weighted sum or integral of sampled colors along each ray.
- Bias in color rendering: The weighted volume-rendering integral introduces bias between rendered colors and implicit geometry, especially when weights do not concentrate exactly on the true surface or under occlusion.When the dominant weight approaches one, only sampling bias remains; otherwise weighted-sum bias also contributes.
- Explicit supervision on SDF network: Geo-Neus directly locates the SDF zero-level set and adds explicit geometry supervision to optimize the surface rather than relying only on rendered-color differences.The SDF network estimates signed distance to the object surface and is supervised directly with 3D points.
- Explicit supervision on SDF network: Sparse SfM points provide low-cost geometry priors assumed to lie on the surface, and view-aware SDF loss supervises only points visible from the rendered view.The loss varies with the view so its supervision remains consistent with color rendering and occlusion visibility.
- Geometry-consistent supervision with multi-view constraints: Explicit SDF supervision can accelerate convergence and capture more meticulous geometry, while photometric consistency is used to guarantee geometric consistency across views.The geometry prior is particularly concentrated around complex structures with strong textures.
- Geometry-consistent supervision with multi-view constraints: The method extracts the first SDF sign-change intersection along each ray and uses surface patches, normals, and plane-induced homographies for multi-view photometric consistency.Rendered-image patches are compared with source-view patches using NCC, retaining the best four scores to handle occlusions.
4 Experiments
Experiments evaluate Geo-Neus on DTU and BlendedMVS against learning-based and traditional reconstruction baselines, including ablations of its geometry constraints. Results show complementary benefits from sparse SDF supervision and photometric consistency, with the full model achieving the best performance and reducing runtime.
- Datasets: Evaluation uses 15 DTU scans and 7 challenging BlendedMVS scenes with calibrated multi-view images.DTU images have 1200 × 1600 resolution, while BlendedMVS images have 768 × 576 resolution.
- Baselines: Geo-Neus is compared with IDR, VolSDF, NeuS, NeuralWarp, and colmap using trimming value 7.The comparison covers learning-based methods and the traditional reconstruction method colmap.
- Ablation study: With very sparse 3D supervision, Model-A outperforms colmap on DTU with a score of 0.62 versus 0.65.The ablation uses NeuS as the baseline and progressively adds the proposed modules.
- Ablation study: The SDF loss improves complex thin structures, while the photometric loss improves large smooth regions.Model-A reconstructs the apple stem finely but less smoothly; Model-B reconstructs a smooth surface but loses the stem.
- Ablation study: Geo-Neus combines the individual advantages of the SDF and photometric losses and achieves the best performance.Figure 5 reports the corresponding surface-quality differences among ablation models.
- Efficiency: Runtime decreases from around 16 hours to around 10 hours.The supplied passage reports this reduction without specifying the exact experimental configuration.
5 Conclusion
The conclusion presents Geo-Neus as an explicit SDF-optimization method using sparse SFM points and multi-view photometric consistency. It reports high-quality reconstruction across thin and smooth structures, while noting that efficiency remains limited.
- 5 Conclusion: Geo-Neus enforces explicit SDF optimization using sparse 3D points from SFM and photometric consistency from multi-view stereo.The method is motivated by a theoretically analyzed gap between volume rendering integration and neural SDF learning.
- 5 Conclusion: Geo-Neus produces high-quality surface reconstruction in both complex thin structures and large smooth regions.The conclusion states that this performance applies across the two cited geometry types.
- 5 Conclusion: The method outperforms state-of-the-art traditional and neural implicit surface-learning methods by a large margin.The conclusion makes this comparison across both categories of competing methods.
- 5 Conclusion: Efficiency remains limited, motivating future work on accelerating neural implicit surface learning by volume rendering.The conclusion specifically points to super-fast per-scene radiance-field optimization methods as a future direction.