PaperlayerView exact revision on arXiv

Source-linked AI summary

FlashNormal: Detailed Surface Normal Estimation from Flash and No-Flash Images

Ruiyang Chen, Feiran Li, Heng Guo, Zhanyu Ma

arXiv:2608.25360v1cs.CV

TL;DR

FlashNormal addresses the limited detail recovery and shape-reflectance ambiguity of single-image normal estimation while avoiding photometric stereo’s multi-illumination capture requirements. It uses flash/no-flash pairs with diffusion priors and curvature-guided detail enhancement, and evaluates the approach with EvalFlash. Ablations report MAE reductions of 3.4%, 3.5%, and 4.1% for three modules, while the method remains challenged by transparent objects and textured flat objects.

  • Problem

    Single-image normal estimation struggles with fine details and shape-reflectance ambiguity, while photometric stereo requires multi-illumination capture.

  • Method

    FlashNormal is a diffusion-based estimator using flash/no-flash images, curvature-guided geometry enhancement, and zoomed-pixels refinement.

  • Results

    FlashNormal outperforms state-of-the-art methods, with its three ablated modules reducing EvalFlash MAE by 3.4%, 3.5%, and 4.1%.

  • Takeaways & Limitations

    FlashNormal provides fine-grained normal estimation while mitigating shape-reflectance ambiguity in a practical flash/no-flash setup.

  • Takeaways & Limitations

    The method struggles with transparent objects and can misinterpret flat objects with complex realistic textures as 3D, partly because transparent examples are limited in training.

Abstract

from arXiv · show

High-quality surface normal estimation is preferred for detailed surface shape recovery and image editing. Existing single image-based methods, though being a practical setup, often struggle to recover fine surface details and are sensitive to inherent shape-reflectance ambiguity. While photometric stereo achieves high-fidelity surface normal estimation from images under varying lights, its applicability is strictly limited by requiring a multi-illumination capture setup. To this end, we propose FlashNormal, a diffusion-based surface normal estimator from flash/no-flash image pairs. While retaining high practicability on modern smartphones, our proposal takes advantage of flash-induced shading variations, and leverages curvature-guided detail enhancement strategy, improving surface detail recovery and mitigating shape-reflectance ambiguity effectively. To evaluate our proposed method, we further present EvalFlash, the first real-world flash/no-flash evaluation dataset containing 20 objects aligned with ground-truth surface normals for quantitative benchmarking. Extensive experiments demonstrate the effectiveness of FlashNormal over state-of-the-art single image-based methods and show a significant out-performance over flash/no-flash-based normal estimation method on EvalFlash.

I. INTRODUCTION

FlashNormal addresses the detail loss and shape-reflectance ambiguity of practical single-image normal estimation by using flash/no-flash shading cues, diffusion priors, and detail-enhancement strategies. The paper also introduces EvalFlash, a real-world benchmark with aligned ground-truth normals.

  • Context: The paper positions FlashNormal against single-image and photometric-stereo approaches while retaining a practical setup.Photometric stereo requires varying illuminations, whereas flash/no-flash images are widely available on modern smartphones and cameras.
  • Motivation: Single-image normal estimation is practical but struggles with geometric details and shape-reflectance ambiguity.A single RGB image may not distinguish geometric details from texture details, producing insufficient or over-smoothed normals.
  • Method: FlashNormal uses flash/no-flash image pairs to provide shading variations that reduce shape estimation ambiguity.The method draws on photometric stereo while retaining a minimal, accessible capture setup and uses diffusion priors for zero-shot generalization.
  • Method: Curvature-guided detail enhancement and zoomed-pixels refinement target regions and scales with pronounced geometric features.The two strategies are designed to produce detail-rich surface normal maps.
  • Evaluation: EvalFlash is presented as the first real-world flash/no-flash dataset with labeled ground-truth surface normals.It contains 20 objects with diverse shapes and materials, using scanned and carefully aligned meshes for quantitative evaluation.

III. FLASHNORMAL

FlashNormal combines flash/no-flash image cues and a coarse normal initialization in a diffusion-based pipeline to estimate detailed surface normal maps. Separate VAE encodings are concatenated as guidance before denoising and decoding.

  • III. FLASHNORMAL: A diffusion prior denoises the combined latent representation in a single step to generate a latent surface normal, which the VAE decoder converts into the final map.The single-step estimator is designed to generate detailed surface normals from the combined guidance representation.
  • III. FLASHNORMAL: FlashNormal encodes flash/no-flash images separately, concatenates their latent representations, and uses the resulting feature map as estimator guidance.Separate encoding preserves compatibility with pretrained 3-channel VAEs while producing an 8-channel guidance feature map.
  • III. FLASHNORMAL: The pipeline combines flash/no-flash guidance with a coarse surface normal initialization before diffusion-based normal estimation.The coarse initialization comes from Metric3Dv2 and is encoded into a latent representation before combination with flash/no-flash features.

B. Diffusion prior guided normal estimator

The diffusion prior guided estimator uses deterministic single-step denoising and curvature-aware supervision to emphasize geometric details while preserving smooth regions. Zoomed pixels further target small regions of interest during training and testing.

  • B. Diffusion prior guided normal estimator: The Diffusion U-Net deterministically denoises the guided latent feature map in a single step, producing a latent surface normal for decoding.The process uses t = 999 and text guidance describing a detailed surface normal map from a DSLR image.
  • B. Diffusion prior guided normal estimator: Curvature-guided supervision emphasizes regions with strong normal variation, where higher curvature indicates richer geometric detail.Curvature is derived from the first and second fundamental forms and used to construct a detail mask.
  • B. Diffusion prior guided normal estimator: A curvature loss compares ground-truth and predicted curvature maps within detail-rich regions, while angular-error supervision gives additional weight to smooth areas.This combination is intended to recover both detailed and smooth regions of the surface normal map.
  • B. Diffusion prior guided normal estimator: The total training objective combines mean angular error and curvature losses with λ empirically set to 100.The VAE decoder maps the predicted latent representation back to pixel-space surface normals in the end-to-end pipeline.
  • B. Diffusion prior guided normal estimator: Zoomed-pixels training enlarges small regions of interest because limited image area can otherwise produce blurry surface normal estimates.The strategy uses bounding boxes and adapts an object-detection idea for fine-scale surface detail enhancement.

C. Analyses regarding flash inputs

Flash/no-flash pairs provide illumination and shading cues that make fine geometric details more visible and help reduce shape-reflectance ambiguity.

  • Flash illumination makes fine surface details and concave-convex structures more visible than no-flash lighting.No-flash ambient lighting can obscure details, whereas flash provides brighter and more uniform illumination.
  • Flash cues help distinguish actual objects from textures by revealing whether surfaces exhibit diverse shading variations.The earphone-poster example uses concentrated flash response and absent broader shading variation as evidence for a poster.
  • EvalFlash presents ground-truth normals alongside flash and no-flash observations for real-world benchmarking.
  • Shading differences between flash and no-flash images clarify the underlying geometric structure.

IV. DATASET

The dataset section introduces Flash100K, a photorealistic rendered training resource designed to support detailed flash/no-flash normal estimation and reduce rendered-to-real domain gaps.

  • A. A large-scale training dataset: Flash100K: Flash100K addresses limitations of an existing dataset whose random material-shape combinations fail to reflect real-world object appearances.
  • A. A large-scale training dataset: Flash100K: Figure 6 illustrates 3D assets, corresponding HDR environment maps, and rendered training data from Flash100K.
  • A. A large-scale training dataset: Flash100K: Flash100K provides 100,000 flash/no-flash image pairs with corresponding ground-truth surface normals.The dataset uses carefully composed shapes and materials to better reflect real-world appearances.
  • A. A large-scale training dataset: Flash100K: Each rendered sample pairs flash and no-flash observations produced under ambient environmental illumination and a point light.The rendering process selects one of 15 environment maps and captures images with the point light activated and deactivated.

B. A comprehensive large-scale synthetic evaluation dataset: EvalFlash-synth

EvalFlash-synth is introduced as a large-scale synthetic test set for evaluating robustness and generalizability under flash/no-flash conditions, alongside real-world data collection and alignment validation.

  • B. A comprehensive large-scale synthetic evaluation dataset: EvalFlash-synth: EvalFlash-synth contains 839 high-quality test samples designed specifically for flash/no-flash normal estimation evaluation.
  • C. A real-world evaluation dataset: EvalFlash: The real-world EvalFlash collection covers 20 objects with intricate shapes and diverse materials for detailed reconstruction evaluation.
  • C. A real-world evaluation dataset: EvalFlash: Mesh-to-image alignment across the 20 objects achieves average IoU above 99.0%, supporting quantitative evaluation.The reported residual alignment errors are considered unlikely to substantially affect overall MAE trends.
  • B. A comprehensive large-scale synthetic evaluation dataset: EvalFlash-synth: The evaluation compares single-image, photometric-stereo, and flash/no-flash methods using released baseline implementations.
  • B. A comprehensive large-scale synthetic evaluation dataset: EvalFlash-synth: MAE, RMSE, and MSE measure differences between estimated and ground-truth surface normal maps.

B. Benchmark evaluation on EvalFlash

On EvalFlash and EvalFlash-synth, FlashNormal outperforms comparison methods, including single-image and flash/no-flash baselines, while producing detailed normals across diverse reflectances.

  • B. Benchmark evaluation on EvalFlash: 6.6%: FlashNormal improves surface normal estimation accuracy over the second-best E2E-FT baseline on EvalFlash.
  • B. Benchmark evaluation on EvalFlash: 58.4%: FlashNormal improves performance over the state-of-the-art flash/no-flash method MV20 on EvalFlash.
  • B. Benchmark evaluation on EvalFlash: FlashNormal achieves the smallest MAE and RMSE across most tested EvalFlash objects, indicating stable estimation for richly detailed surfaces.
  • B. Benchmark evaluation on EvalFlash: FlashNormal estimates plausible normals for both specular PINEAPPLE and diffuse LADY while capturing fine geometric details.E2E-FT and StableNormal perform well on only one of the two surfaces, while Metric3Dv2 and MV20 produce coarse estimates.
  • B. Benchmark evaluation on EvalFlash: 72.1%: FlashNormal gains performance over MV20 on the EvalFlash-synth test set.FlashNormal achieves the smallest average MAE and RMSE across 839 synthetic test samples.
  • B. Benchmark evaluation on EvalFlash: FlashNormal is also evaluated on smartphone-captured flash/no-flash images to assess robustness across capturing devices.
  • B. Benchmark evaluation on EvalFlash: Flash inputs and curvature-map supervision support detailed surface-shape recovery beyond the coarse estimates of single-image baselines.

E. Comparison with single-image-based mesh generation methods

FlashNormal produces more faithful and detailed meshes than single-image-based methods, while ablations show that flash input and detail-enhancement components each improve normal estimation.

  • Single-image methods can generate plausible geometry but may hallucinate or distort visible structures, producing less faithful front views and smoother side surfaces.
  • Ablation study: Flash input, curvature-guided enhancement, and zoomed pixels each reduce EvalFlash MAE by 3.4%, 3.5%, and 4.1%, respectively.Removing any module increases MAE and loses details, particularly around the neck of PINKCAT and head of LADY.
  • Ablation study: Flash inputs help distinguish flat, shape-reflectance-ambiguous surfaces from true 3D structures using shading variations.The effect is illustrated on DRUM POSTER and MIRROR.
  • FlashNormal estimates richer surface details and more geometrically structured meshes than peer methods for detail-rich objects.
  • Ablation study: Coarse surface-normal initialization improves predicted normals at the same 10-epoch training budget and accelerates convergence.

G. Analyses regarding zoomed-pixels for inputting image to E2E-FT

Adding a bounding box improves E2E-FT predictions, but FlashNormal remains more accurate on both synthetic and real data because E2E-FT was not trained with zoomed pixels.

  • The comparison uses default peer-method settings in the main evaluation, without applying zoomed pixels to those methods.
  • Bounding-box zooming improves E2E-FT surface-normal quality compared with using its no-flash input without a bounding box.
  • E2E-FT still has higher MAE than FlashNormal on synthetic BUDDHA and real PINEAPPLE data after adding the bounding box.

H. Robustness analyses

FlashNormal remains robust across camera configurations and illumination changes, supports automatic ROI-based predictions, and improves multi-view 3D reconstruction quality over E2E-FT.

  • Camera robustness: FlashNormal maintains reliable normal estimation under varying focal lengths and camera-to-object distances, while E2E-FT becomes blurred when the valid surface area is small.
  • Illumination robustness: FlashNormal consistently estimates reliable surface normals under varying illumination intensities and color temperatures in synthetic and real-world experiments.
  • Automatic ROI acquisition: SAM 2-derived masks can be converted into valid ROIs that produce coherent predictions for single-object and correctly segmented multi-object scenes.The experiments include three single-object and two multi-object cases.
  • Single-image fallback: A single ambient-light image and its StableDelight-processed version can still produce high-quality normal maps with FlashNormal.
  • Multi-view reconstruction: FlashNormal’s multi-view normal maps yield detailed, geometrically consistent meshes for SHAKESPEARE and MONKEY.For MONKEY, average multi-view MAE is 12.86 versus 17.83 for E2E-FT, a 27.8% improvement.

VI. LIMITATIONS AND FUTURE WORKS

FlashNormal improves detailed normal estimation and reconstruction, but remains challenged by special materials and highly textured flat objects; future work expands data and application scope.

  • Limitations: Transparent objects remain difficult because reflective properties obscure the object surface and training data contain relatively few transparent examples.
  • Limitations: Highly realistic textures on flat objects can be misinterpreted as three-dimensional structure because flash-induced shading variations are minimal in textured regions.
  • Future work: Future work will augment training data with more diverse examples and investigate surface-normal estimation for special materials.
  • Conclusion: The conclusion reports fine-grained normal estimation, reduced shape-reflectance ambiguity, and improved downstream multi-view 3D reconstruction quality.
Loading 2608.25360v1…