PaperlayerView exact revision on arXiv

Source-linked AI summary

FlashNormal: Detailed Surface Normal Estimation from Flash and No-Flash Images

Ruiyang Chen, Feiran Li, Heng Guo, Zhanyu Ma

arXiv:2608.25360v1cs.CV

TL;DR

FlashNormal addresses the limited detail recovery and shape-reflectance ambiguity of single-image normal estimation while avoiding photometric stereo’s multi-illumination capture requirements. It uses flash/no-flash pairs with diffusion priors and curvature-guided detail enhancement, and evaluates the approach with EvalFlash. Ablations report MAE reductions of 3.4%, 3.5%, and 4.1% for three modules, while the method remains challenged by transparent objects and textured flat objects.

  • Problem

    Single-image normal estimation struggles with fine details and shape-reflectance ambiguity, while photometric stereo requires multi-illumination capture.

    Source: §I · pp. 1–2

  • Method

    FlashNormal is a diffusion-based estimator using flash/no-flash images, curvature-guided geometry enhancement, and zoomed-pixels refinement.

    Source: §I · p. 2, B. Diffusion prior guided normal… · p. 4

  • Results

    FlashNormal outperforms state-of-the-art methods, with its three ablated modules reducing EvalFlash MAE by 3.4%, 3.5%, and 4.1%.

    Source: F. Ablation study · p. 8, §VII · p. 13

  • Takeaways & Limitations

    FlashNormal provides fine-grained normal estimation while mitigating shape-reflectance ambiguity in a practical flash/no-flash setup.

    Source: §VII · pp. 12–13

    The method struggles with transparent objects and can misinterpret flat objects with complex realistic textures as 3D, partly because transparent examples are limited in training.

    Source: §VI · p. 12

Abstract

from arXiv · show

High-quality surface normal estimation is preferred for detailed surface shape recovery and image editing. Existing single image-based methods, though being a practical setup, often struggle to recover fine surface details and are sensitive to inherent shape-reflectance ambiguity. While photometric stereo achieves high-fidelity surface normal estimation from images under varying lights, its applicability is strictly limited by requiring a multi-illumination capture setup. To this end, we propose FlashNormal, a diffusion-based surface normal estimator from flash/no-flash image pairs. While retaining high practicability on modern smartphones, our proposal takes advantage of flash-induced shading variations, and leverages curvature-guided detail enhancement strategy, improving surface detail recovery and mitigating shape-reflectance ambiguity effectively. To evaluate our proposed method, we further present EvalFlash, the first real-world flash/no-flash evaluation dataset containing 20 objects aligned with ground-truth surface normals for quantitative benchmarking. Extensive experiments demonstrate the effectiveness of FlashNormal over state-of-the-art single image-based methods and show a significant out-performance over flash/no-flash-based normal estimation method on EvalFlash.

I. INTRODUCTION

FlashNormal addresses the detail loss and shape-reflectance ambiguity of practical single-image normal estimation by using flash/no-flash shading cues, diffusion priors, and detail-enhancement strategies. The paper also introduces EvalFlash, a real-world benchmark with aligned ground-truth normals.

Context

  • The paper positions FlashNormal against single-image and photometric-stereo approaches while retaining a practical setup.Photometric stereo requires varying illuminations, whereas flash/no-flash images are widely available on modern smartphones and cameras.Source: §I · p. 1, §II · p. 2

Motivation

  • Single-image normal estimation is practical but struggles with geometric details and shape-reflectance ambiguity.A single RGB image may not distinguish geometric details from texture details, producing insufficient or over-smoothed normals.Source: §I · pp. 1–2

Method

  • FlashNormal uses flash/no-flash image pairs to provide shading variations that reduce shape estimation ambiguity.The method draws on photometric stereo while retaining a minimal, accessible capture setup and uses diffusion priors for zero-shot generalization.Source: §I · p. 2
  • Curvature-guided detail enhancement and zoomed-pixels refinement target regions and scales with pronounced geometric features.The two strategies are designed to produce detail-rich surface normal maps.Source: §I · p. 2

Evaluation

  • EvalFlash is presented as the first real-world flash/no-flash dataset with labeled ground-truth surface normals.It contains 20 objects with diverse shapes and materials, using scanned and carefully aligned meshes for quantitative evaluation.Source: §I · p. 2

III. FLASHNORMAL

FlashNormal combines flash/no-flash image cues and a coarse normal initialization in a diffusion-based pipeline to estimate detailed surface normal maps. Separate VAE encodings are concatenated as guidance before denoising and decoding.

III. FLASHNORMAL

  • A diffusion prior denoises the combined latent representation in a single step to generate a latent surface normal, which the VAE decoder converts into the final map.The single-step estimator is designed to generate detailed surface normals from the combined guidance representation.Source: §III · pp. 2–3, Figure 2 · p. 3
  • FlashNormal encodes flash/no-flash images separately, concatenates their latent representations, and uses the resulting feature map as estimator guidance.Separate encoding preserves compatibility with pretrained 3-channel VAEs while producing an 8-channel guidance feature map.Source: A. Flash/no-flash VAE · p. 3
  • The pipeline combines flash/no-flash guidance with a coarse surface normal initialization before diffusion-based normal estimation.The coarse initialization comes from Metric3Dv2 and is encoded into a latent representation before combination with flash/no-flash features.Source: A. Flash/no-flash VAE · p. 3, §III · p. 3

B. Diffusion prior guided normal estimator

The diffusion prior guided estimator uses deterministic single-step denoising and curvature-aware supervision to emphasize geometric details while preserving smooth regions. Zoomed pixels further target small regions of interest during training and testing.

B. Diffusion prior guided normal estimator

  • The Diffusion U-Net deterministically denoises the guided latent feature map in a single step, producing a latent surface normal for decoding.The process uses t = 999 and text guidance describing a detailed surface normal map from a DSLR image.Source: B. Diffusion prior guided normal… · p. 3
  • Curvature-guided supervision emphasizes regions with strong normal variation, where higher curvature indicates richer geometric detail.Curvature is derived from the first and second fundamental forms and used to construct a detail mask.Source: B. Diffusion prior guided normal… · pp. 3–4
  • A curvature loss compares ground-truth and predicted curvature maps within detail-rich regions, while angular-error supervision gives additional weight to smooth areas.This combination is intended to recover both detailed and smooth regions of the surface normal map.Source: B. Diffusion prior guided normal… · p. 4
  • The total training objective combines mean angular error and curvature losses with λ empirically set to 100.The VAE decoder maps the predicted latent representation back to pixel-space surface normals in the end-to-end pipeline.Source: B. Diffusion prior guided normal… · pp. 3–4
  • Zoomed-pixels training enlarges small regions of interest because limited image area can otherwise produce blurry surface normal estimates.The strategy uses bounding boxes and adapts an object-detection idea for fine-scale surface detail enhancement.Source: B. Diffusion prior guided normal… · p. 4

C. Analyses regarding flash inputs

Flash/no-flash pairs provide illumination and shading cues that make fine geometric details more visible and help reduce shape-reflectance ambiguity.

  • Flash illumination makes fine surface details and concave-convex structures more visible than no-flash lighting.No-flash ambient lighting can obscure details, whereas flash provides brighter and more uniform illumination.Source: C. Analyses regarding flash inputs · p. 4
  • Flash cues help distinguish actual objects from textures by revealing whether surfaces exhibit diverse shading variations.The earphone-poster example uses concentrated flash response and absent broader shading variation as evidence for a poster.Source: C. Analyses regarding flash inputs · pp. 4–5
  • EvalFlash presents ground-truth normals alongside flash and no-flash observations for real-world benchmarking.Source: Figure 4 · p. 5, Figure 5 · p. 5
  • Shading differences between flash and no-flash images clarify the underlying geometric structure.Source: C. Analyses regarding flash inputs · p. 5

IV. DATASET

The dataset section introduces Flash100K, a photorealistic rendered training resource designed to support detailed flash/no-flash normal estimation and reduce rendered-to-real domain gaps.

A. A large-scale training dataset: Flash100K

  • Flash100K addresses limitations of an existing dataset whose random material-shape combinations fail to reflect real-world object appearances.Source: A. A large-scale training dataset:… · p. 5
  • Figure 6 illustrates 3D assets, corresponding HDR environment maps, and rendered training data from Flash100K.Source: Figure 6 · p. 5
  • Flash100K provides 100,000 flash/no-flash image pairs with corresponding ground-truth surface normals.The dataset uses carefully composed shapes and materials to better reflect real-world appearances.Source: A. A large-scale training dataset:… · p. 5
  • Each rendered sample pairs flash and no-flash observations produced under ambient environmental illumination and a point light.The rendering process selects one of 15 environment maps and captures images with the point light activated and deactivated.Source: A. A large-scale training dataset:… · p. 6

B. A comprehensive large-scale synthetic evaluation dataset: EvalFlash-synth

EvalFlash-synth is introduced as a large-scale synthetic test set for evaluating robustness and generalizability under flash/no-flash conditions, alongside real-world data collection and alignment validation.

B. A comprehensive large-scale synthetic evaluation dataset: EvalFlash-synth

C. A real-world evaluation dataset: EvalFlash

  • The real-world EvalFlash collection covers 20 objects with intricate shapes and diverse materials for detailed reconstruction evaluation.Source: C. A real-world evaluation dataset:… · p. 6
  • Mesh-to-image alignment across the 20 objects achieves average IoU above 99.0%, supporting quantitative evaluation.The reported residual alignment errors are considered unlikely to substantially affect overall MAE trends.Source: C. A real-world evaluation dataset:… · p. 6

B. A comprehensive large-scale synthetic evaluation dataset: EvalFlash-synth

B. Benchmark evaluation on EvalFlash

On EvalFlash and EvalFlash-synth, FlashNormal outperforms comparison methods, including single-image and flash/no-flash baselines, while producing detailed normals across diverse reflectances.

B. Benchmark evaluation on EvalFlash

E. Comparison with single-image-based mesh generation methods

FlashNormal produces more faithful and detailed meshes than single-image-based methods, while ablations show that flash input and detail-enhancement components each improve normal estimation.

Ablation study

  • Flash input, curvature-guided enhancement, and zoomed pixels each reduce EvalFlash MAE by 3.4%, 3.5%, and 4.1%, respectively.Removing any module increases MAE and loses details, particularly around the neck of PINKCAT and head of LADY.Source: F. Ablation study · p. 8
  • Flash inputs help distinguish flat, shape-reflectance-ambiguous surfaces from true 3D structures using shading variations.The effect is illustrated on DRUM POSTER and MIRROR.Source: F. Ablation study · p. 8
  • FlashNormal estimates richer surface details and more geometrically structured meshes than peer methods for detail-rich objects.Source: Figure 9 · p. 9

Ablation study

G. Analyses regarding zoomed-pixels for inputting image to E2E-FT

Adding a bounding box improves E2E-FT predictions, but FlashNormal remains more accurate on both synthetic and real data because E2E-FT was not trained with zoomed pixels.

H. Robustness analyses

FlashNormal remains robust across camera configurations and illumination changes, supports automatic ROI-based predictions, and improves multi-view 3D reconstruction quality over E2E-FT.

Camera robustness

Illumination robustness

Automatic ROI acquisition

  • SAM 2-derived masks can be converted into valid ROIs that produce coherent predictions for single-object and correctly segmented multi-object scenes.The experiments include three single-object and two multi-object cases.Source: H. Robustness analyses · p. 11

Single-image fallback

  • A single ambient-light image and its StableDelight-processed version can still produce high-quality normal maps with FlashNormal.Source: §I · p. 11

Multi-view reconstruction

  • FlashNormal’s multi-view normal maps yield detailed, geometrically consistent meshes for SHAKESPEARE and MONKEY.For MONKEY, average multi-view MAE is 12.86 versus 17.83 for E2E-FT, a 27.8% improvement.Source: J. Application: Multi-view normal… · pp. 11–12

VI. LIMITATIONS AND FUTURE WORKS

FlashNormal improves detailed normal estimation and reconstruction, but remains challenged by special materials and highly textured flat objects; future work expands data and application scope.

Limitations

  • Transparent objects remain difficult because reflective properties obscure the object surface and training data contain relatively few transparent examples.Source: §VI · p. 12
  • Highly realistic textures on flat objects can be misinterpreted as three-dimensional structure because flash-induced shading variations are minimal in textured regions.Source: §VI · p. 12

Future work

  • Future work will augment training data with more diverse examples and investigate surface-normal estimation for special materials.Source: §VI · p. 12, §VII · p. 12

Conclusion

  • The conclusion reports fine-grained normal estimation, reduced shape-reflectance ambiguity, and improved downstream multi-view 3D reconstruction quality.Source: §VII · p. 13

Related papers

Loading 2608.25360v1…