PaperlayerView exact revision on arXiv

Source-linked AI summary

MicroSDF: Microfacet-Driven Hybrid Neural SDFs for Mixed-Reflectance Surface Reconstruction

Lejia Ye, Yuhua Xu, Yulan Guo, Lian Xu, Xiaohu Zhang

arXiv:2610.19681v1cs.CV

TL;DR

Mixed reflective and non-reflective scenes challenge reconstruction methods that often model the surface types separately. MicroSDF integrates microfacet priors into neural SDF geometry and color modeling, and the paper reports state-of-the-art results across several benchmarks.

  • Problem

    Existing methods often handle reflective and non-reflective surfaces separately, while reflective appearance can be difficult to disentangle from true geometry.

    Source: §I · pp. 1–2

  • Method

    MicroSDF integrates microfacet priors into neural SDF geometry and dual-branch color modeling, with detection-guided blending of branch outputs.

    Source: §I · p. 2

  • Results

    MicroSDF reports state-of-the-art results across DTU, Shiny Blender, Ref-NeRF, and DeepVoxels, including an average Chamfer Distance of 0.61 on DTU.

    Source: §V · p. 15

  • Takeaways & Limitations

    The reported results support a unified framework for high-fidelity reconstruction of reflective and non-reflective surfaces.

    Source: §V · p. 15

Abstract

from arXiv · show

Accurate 3D reconstruction in real-world environments remains a significant challenge due to the coexistence of reflective and non-reflective surfaces, which pose distinct modeling demands. Existing methods often treat these surface types separately, limiting their generalizability and physical plausibility. To bridge this gap, we propose MicroSDF, a novel neural implicit framework that facilitates geometry and reflectance modeling through microfacet theory. Our approach incorporates three core innovations: (i) a microfacet-guided geometry model that extracts multi-scale surface normals (macroscopic and microfacet) from a signed distance field (SDF), regularized by a proposed microfacet normal consistency loss to enforce physically plausible surface orientations; (ii) an enhanced dual-branch color model, where the specular branch leverages the microfacet normals to model high-frequency reflectance, and the vanilla branch, unlike prior works, uses reflection direction (instead of viewing direction) to better model diffuse and low-frequency specular components; and (iii) a detection-guided color blending strategy that adaptively fuses the color outputs based on reflection priors, providing more physically intuitive blending than implicitly learned blending weights. Combined with a tailored multi-stage optimization scheme, the proposed MicroSDF achieves robust and high-fidelity reconstruction across reflective and non-reflective surfaces. Extensive experiments on DTU, Shiny Blender, Ref-NeRF, and DeepVoxels datasets demonstrate state-of-the-art performance, establishing a new direction for physically grounded neural reconstruction.

I. INTRODUCTION

The introduction frames mixed reflective and non-reflective scenes as a gap for reconstruction methods that typically handle the two surface types separately. MicroSDF integrates microfacet priors into neural SDF geometry and color modeling, and the paper reports strong performance across mixed-surface benchmarks.

I. INTRODUCTION

  • Existing methods often target either diffuse or highly reflective surfaces independently, limiting generalization in mixed scenes.Reflective regions also challenge MVS photometric consistency and neural SDF methods that neglect reflective-surface physics.Source: §I · p. 1, A. Multi-view Stereo · p. 3
  • MicroSDF unifies reconstruction of reflective and non-reflective surfaces by explicitly integrating microfacet theory into a neural implicit framework.The framework introduces microfacet-guided geometry, microfacet-driven color modeling, and detection-guided blending.Source: §I · p. 2
  • MicroSDF uses reflection direction in the vanilla color branch, microfacet normals in the specular branch, and detector-guided weights to blend their outputs.The design targets diffuse and low-frequency reflections, high-frequency highlights, and physically intuitive blending, respectively.Source: §I · p. 2
  • Microfacet normal consistency aligns microfacet and local surface normals in specular regions to support physically meaningful geometry.The paper presents the loss as part of its microfacet-guided geometry model.Source: §I · p. 2
  • MicroSDF reports state-of-the-art performance across DTU, Shiny Blender, Ref-NeRF, and DeepVoxels, with effectiveness on reflective and non-reflective surfaces.The introduction also reports robust, high-fidelity reconstruction across the two surface types.Source: §I · p. 2

A. Multi-view Stereo

  • Traditional and learning-based MVS methods face difficulty on reflective surfaces because their photometric-consistency assumption breaks down for glossy materials.The cited related-work discussion also notes that deep-learning MVS methods typically require large training datasets.Source: A. Multi-view Stereo · pp. 2–3

B. Neural Implicit Surface Based Reconstruction

Neural implicit representations improve reconstruction, but reflective appearance and geometry remain difficult to disentangle. Related approaches address different parts of this problem; MicroSDF instead focuses on geometric reconstruction with an explicit SDF surface and microfacet priors.

B. Neural Implicit Surface Based Reconstruction

  • Neural Radiance Fields and 3D Gaussian Splatting advance rendering, but 3DGS geometry remains challenging without an explicit surface representation.The cited surface-aligned 3DGS variants are described as generally less geometrically accurate in practice than neural SDF methods.Source: B. Neural Implicit Surface Based… · p. 3
  • SDF-based formulations have been explored to improve geometric precision in NeRF-based methods, with VolSDF linking SDF values to volumetric density.NeuralAngelo is described as further refining surface fidelity and reconstruction quality.Source: B. Neural Implicit Surface Based… · p. 3
  • External geometric priors can supervise SDF-based networks, but their effectiveness depends on supervision quality and multi-view consistency may fail in reflective scenes.NeRF view-dependent color prediction also struggles with reflective light interactions.Source: B. Neural Implicit Surface Based… · p. 3

C. Reflective Surface Reconstruction

  • MicroSDF and Neural Microfacet Fields both use microfacet-based priors, but MicroSDF targets explicit-SDF geometry while NMF targets volumetric inverse rendering and material–illumination decomposition.MicroSDF uses surface microfacet normals as specular-branch inputs and as a surface-orientation consistency prior, without explicit material–illumination decomposition.Source: C. Reflective Surface Reconstruction · p. 3
  • Neural implicit reconstruction methods can struggle to separate view-dependent appearance from true geometry in reflective regions, producing inaccurate reconstructions.The paper attributes this difficulty to rendering formulations that neglect the physical properties of highly reflective surfaces.Source: C. Reflective Surface Reconstruction · p. 4
  • UniSDF uses a dual-branch architecture for reflective and non-reflective regions, while AniSDF uses dual-granularity geometry; MicroSDF retains a single geometry network.The paper says UniSDF can show artifacts under near-field reflection and AniSDF requires separate geometric networks at different granularities.Source: C. Reflective Surface Reconstruction · p. 4

III. THE PROPOSED MICROSDF

MicroSDF is a unified neural surface reconstruction framework that combines microfacet-guided geometry and dual-branch color modeling. Its overview places these components within rendering formulations for volume-based NeRFs and surface reflectance.

III. THE PROPOSED MICROSDF

  • MicroSDF combines a geometry network with a dual-branch color network and detection-guided blending to reconstruct reflective and non-reflective surfaces.The method also uses a multi-stage optimization strategy for progressive refinement and physically faithful reconstruction.Source: §III · p. 4

1) Volume Rendering in Neural Radiance Fields:

  • NeRF maps 3D position and view direction to density and color, then computes pixel color by volume rendering along each camera ray.Its training minimizes color reconstruction loss against ground-truth pixel color.Source: 1) Volume Rendering in Neural Radiance… · p. 4

2) Surface Rendering:

  • The surface-rendering formulation describes outgoing radiance using incoming radiance, a BRDF, and emitted light integrated over the hemisphere.The Cook-Torrance specular BRDF is based on microfacet theory and combines normal-distribution, Fresnel, and geometry functions.Source: 2) Surface Rendering · p. 5

B. Microfacet-Guided Geometry Modeling

MicroSDF derives geometry and surface normals from an SDF-based geometry network. Its dual-scale normal representation captures both overall surface orientation and finer normal variation for reflectance modeling.

1) Geometry Network:

  • The geometry network maps sampled coordinates and an external point cloud to surface normals, geometric features, and SDF values.Coordinates are encoded by an iNGP before an MLP predicts the SDF, while the point cloud directly supervises early SDF training.Source: 1) Geometry Network · p. 5

2) Dual-Scale Normal Extraction:

  • MicroSDF extracts macroscopic and microfacet normals at different scales to represent global surface structure and fine-grained normal variation.The microfacet normals are concatenated with the macroscopic normal and later used to predict the specular color component.Source: 2) Dual-Scale Normal Extraction · p. 5
  • Microfacet normal variance is lower on specular surfaces than on diffuse surfaces, consistent with microfacet theory.This distinction motivates using normal variation to represent differing surface reflectance behavior.Source: Figure 3 · p. 5, 2) Dual-Scale Normal Extraction · p. 5
  • The method samples multiple normal scales within one geometry network, unlike AniSDF’s separate networks for different geometric granularities.The fine-scale normals are derived from the SDF at a finer hierarchy level.Source: 2) Dual-Scale Normal Extraction · p. 5

3) Microfacet Normal Consistency:

MicroSDF uses microfacet-normal consistency to regularize geometry in specular regions and uses microfacet normals to inform its specular color branch. Its vanilla branch uses reflection direction to model low-frequency specular cues.

3) Microfacet Normal Consistency:

  • Specular regions have low microfacet-normal variance, while rough regions have higher variance; the consistency loss aligns microscopic and macroscopic orientations in specular regions.The metric measures average angular discrepancy between the coarse surface normal and each microfacet normal.Source: 3) Microfacet Normal Consistency · p. 5

1) Dual-branch color network:

  • The vanilla branch takes reflection direction, rather than viewing direction, to model low-frequency specular cues and reduce the burden on the specular branch.The reflection direction is computed from the surface normal and ray direction.Source: 1) Dual-branch color network · pp. 5–6
  • The specular branch adds sampled microfacet normals to its inputs, enabling empirical estimation of D(x) for more physically grounded reconstruction of specular regions.It also uses position, reflection direction, the macroscopic normal, and geometric features.Source: 1) Dual-branch color network · p. 6

2) Detection-guided blending:

MicroSDF blends its vanilla and specular color branches using spatially varying specular confidence from an external detector. Its optimization combines geometry initialization, multistage training, and loss design.

2) Detection-guided blending:

  • The blending addresses ambiguity in assigning color to low- versus high-frequency specular components.The detector scores indicate each pixel’s likelihood of exhibiting high-frequency specular reflection.Source: 2) Detection-guided blending · p. 6
  • StableDelight detector scores provide spatially varying blending weights between the vanilla branch’s low-frequency effects and the specular branch’s high-frequency highlights.Unlike the auxiliary-MLP blending used by UniSDF and AniSDF, the detector supplies explicit specular confidence maps from input images.Source: 2) Detection-guided blending · p. 6

D. Optimization

  • The optimization process has three components: geometry initialization, a multistage training strategy, and loss-function design.The supplied passage names these components but does not specify their individual procedures.Source: D. Optimization · p. 6

1) Geometry Initialization:

Geometry initialization uses point-cloud and image-mask information to constrain the SDF, followed by a four-stage training strategy. Loss terms change across stages, with the final stage using analytical gradients.

1) Geometry Initialization:

  • Geometry initialization uses dense point clouds to constrain surface positions and empty space, while avoiding normal constraints because initial normal errors can harm microfacet-based rendering.Surface SDF values are encouraged toward zero, and off-surface SDF gradients toward zero; foreground masks also constrain background rays.Source: 1) Geometry Initialization · p. 6

2) Four-stage Training Strategy:

  • The four stages are geometry initialization, thorough first-level training, progressive coarse-to-fine processing, and final finest-level training.This strategy reduces total training iterations versus conventional coarse-to-fine training while yielding superior quantitative results; microfacet sampling occurs mainly in the first-level and coarse-to-fine stages.Source: 2) Four-stage Training Strategy · p. 6

3) Loss function design:

  • During first-level and coarse-to-fine training, the objective combines color, Eikonal, curvature, and microfacet-consistency losses.The loss design activates terms by training stage; Table I summarizes the stage-specific terms.Source: 3) Loss function design · p. 7, Table I · p. 7
  • In the final stage, analytical gradients replace numerical ones, making curvature and microfacet losses unnecessary.The geometry-initialization objective separately combines SDF and mask losses.Source: 3) Loss function design · p. 7

A. Implementation Settings

On DTU, the qualitative comparison reports finer geometric detail than NeuS and stronger performance than UniSDF. The evaluation also covers reflective and non-reflective datasets, with metrics selected according to available ground truth.

A. Implementation Settings

  • Evaluation spans DTU, Shiny Blender, Ref-NeRF, and DeepVoxels, covering real-world, synthetic reflective, and handheld-camera scenes.Ref-NeRF and DeepVoxels lack ground-truth geometry in this evaluation, so they are assessed qualitatively.Source: A. Implementation Settings · p. 7
  • For DTU, Chamfer Distance is the primary geometry metric; Shiny Blender uses normal MAE and mesh accuracy, while Ref-NeRF and DeepVoxels receive qualitative evaluation.For Shiny Blender, only accuracy is reported because invisible inner-layer ground-truth points would bias Chamfer completeness; accuracy is not reported for the ball and teapot scenes.Source: A. Implementation Settings · p. 7
  • On DTU, MicroSDF preserves finer geometric details than NeuS [3] and significantly outperforms UniSDF [5] in the qualitative comparison.The comparison is qualitative and does not provide numerical values in the supplied passage.Source: Figure 4 · p. 8
  • Training uses 37,500 iterations and takes approximately 40 minutes on an NVIDIA A100 GPU, excluding pose estimation and preprocessing.The final model is extracted with Marching Cubes at resolution 1024.Source: A. Implementation Settings · p. 8

B. Comparison with the State-of-the-Art

MicroSDF performs strongly across mixed-material and reflective benchmarks, with quantitative gains on DTU and Shiny Blender and qualitative fidelity on Ref-NeRF and DeepVoxels.

B. Comparison with the State-of-the-Art

  • MicroSDF takes approximately 1 hour per DTU scene and achieves the best mean Chamfer distance among compared methods.The comparison spans neural implicit and 3D Gaussian Splatting methods, whose optimization times range from approximately 0.5 to over 16 hours per scene.Source: B. Comparison with the State-of-the-Art · pp. 8–9
  • On Shiny Blender, the proposed method achieves the best mean MAE (4.58°) and mean ACC (0.61) among compared methods.Source: B. Comparison with the State-of-the-Art · p. 9
  • On Ref-NeRF and DeepVoxels, MicroSDF preserves complex geometry and reconstructs specular regions under challenging lighting and reflections.Compared with UniSDF and AniSDF, it has higher success rates and better geometric fidelity; AniSDF exhibits texture-replication artifacts.Source: Figure 5 · p. 9, B. Comparison with the State-of-the-Art · p. 10

C. Ablation Studies

The ablations examine the color-model components, blending priors, normal consistency, initialization, sampling scale, and image inputs. Their results show that component choices and scene conditions affect reconstruction quality.

Component-wise Analysis of Color Modeling

  • Adding RD, then MS, then DC progressively improves qualitative reconstruction, with the full model recovering the spherical object accurately.RD improves surface continuity, while MS further enhances geometric fidelity in reflective regions.Source: C. Ablation Studies · p. 11, Figure 6 · p. 11
  • On DTU, the full configuration reaches average CD 0.607; removing MS, RD, or DC degrades performance by 7.08% (CD 0.65), 5.44% (CD 0.64), or 1.32% (CD 0.615), respectively.Removing DC raises CD on metallic DTU 110 by 13.64%, from 0.66 to 0.75.Source: C. Ablation Studies · p. 11

Effect of Specular Blending Weights

  • Stable detector-derived blending achieves DTU mean Chamfer Distance 0.607, outperforming All Ones (0.65), All Zeros (0.63), and MLP (0.615).On Shiny Blender, All Ones performs best (mean MAE 4.58°, mean ACC 0.61), while Stable scores mean MAE 5.37° amid a detector-to-synthetic-domain gap.Source: C. Ablation Studies · pp. 11–12

Effect of Microfacet Normal Consistency Loss

  • Removing the microfacet normal consistency loss degrades DTU performance by 7.08% (CD 0.65) with MS, RD, and DC enabled.The loss aligns microfacet and local surface normals in specular regions.Source: C. Ablation Studies · p. 12, Table VII · p. 12

On the Use of Diffuse v.s. Original Images in the Vanilla branch

  • Using diffuse images as vanilla-branch input loses fine structural detail and reduces CD from 0.81 to 0.65 and from 0.72 to 0.61 in the reported comparisons.Examples include scissors blade edges, screwdriver helical patterns, and buddha-statue details.Source: C. Ablation Studies · p. 13, Figure 10 · p. 13

Geometry Initialization Strategies

  • Dense initialization produces smoother reconstructions than sparse initialization, especially in darker regions, and improves CD from 0.67 to 0.61.The comparison uses OpenMVS dense points and COLMAP sparse points; the method outperforms OpenMVS across scenes except DTU scan 55, where OpenMVS has a slightly better metric.Source: Table VIII · p. 13, C. Ablation Studies · p. 14, Figure 12 · p. 14

Effect of Microfacet Normal Scale

  • Using cur+4 rather than the finest cur+8 microfacet scale improves CD from 0.79 to 0.65 with sparse initialization and from 0.62 to 0.61 with dense initialization.The gain is larger with sparse input, where overly fine scales may amplify noise or destabilize normal estimation.Source: C. Ablation Studies · pp. 14–15

V. CONCLUSION

MicroSDF integrates microfacet theory into a unified reconstruction framework for reflective and non-reflective surfaces. Experiments report strong benchmark performance, while the proposed priors are identified as a possible direction for reflective-surface reconstruction with 3D Gaussian Splatting.

V. CONCLUSION

  • MicroSDF combines microfacet consistency, microfacet normals for specular color learning, and detection-guided blending to model reflective and non-reflective surfaces in one framework.The conclusion describes these components as providing physically grounded geometry and color modeling.Source: §V · p. 15
  • MicroSDF achieves DTU average Chamfer Distance 0.61 and state-of-the-art quantitative results on Shiny Blender, with qualitative high-fidelity reconstructions on Ref-NeRF and DeepVoxels.The authors describe these results as generalizing across controlled synthetic and real-world scenarios.Source: §V · p. 15
  • The authors identify compatibility of microfacet-driven priors with 3D Gaussian Splatting as a promising future direction for reflective and specular reconstruction.This is presented as a potential direction, not an evaluated result.Source: §V · p. 15

Related papers

Loading 2610.19681v1…