Neuralangelo: High-Fidelity Neural Surface Reconstruction

Motivation

• 3D surface reconstruction recovers dense geometric scene structures from multi-view images for applications including digital twins, environment mapping, and immersive 3D assets.Monocular RGB photogrammetry is especially attractive because ubiquitous mobile devices can support casual real-world capture.Source: §1 · p. 1

Limitations of Existing Methods

• Classical multi-view stereo produces noisy or missing surfaces when observations contain homogeneous colors, repetitive textures, or strong color variations.These ambiguities limit reliable reconstruction from image correspondences.Source: §1 · p. 1

Neural Surface Reconstruction

• Neural surface methods use continuous MLP-based implicit functions and neural volume rendering to interpolate surfaces between observed image evidence.Occupancy fields and signed distance functions are representative scene representations.Source: §1 · p. 2

Scalable Representations

• Current neural methods do not scale reconstruction fidelity effectively with MLP capacity, motivating scalable multi-resolution hash encodings with lightweight MLPs.Instant NGP increases neural-field representation power while keeping memory growth loglinear in resolution.Source: §1 · p. 2

Neuralangelo

• Neuralangelo incorporates multi-resolution hash encoding into neural SDF reconstruction and optimizes it from multi-view images through neural surface rendering.The framework is designed for high-fidelity surface reconstruction.Source: §1 · p. 2

Key Techniques

• Numerical gradients for higher-order derivatives and coarse-to-fine optimization across detail levels are the two techniques that improve hash-encoded surface reconstruction.Experiments report significant improvements over previous methods in reconstruction accuracy and view synthesis quality.Source: §1 · p. 2

Classical Reconstruction

• Classical volumetric photogrammetry relies on strict cross-view color constancy, which often fails under autoexposure and non-Lambertian materials.Relaxing this assumption is important for realistic 3D reconstruction.Source: §2 · p. 2

Point Clouds and Cost Volumes

• Point-cloud-based pipelines inherit missing or noisy surfaces from multi-view stereo, while learned cost volumes remain limited by their resolution.These limitations prevent reliable recovery of fine geometric details.Source: §2 · p. 2

Neural Radiance Fields

• NeRF supports photorealistic view synthesis but does not directly define geometry, so heuristic density thresholding can yield noisy or inaccurate surfaces.Direct surface representations are therefore preferred for photogrammetric reconstruction.Source: §2 · p. 2

Neural Surface Reconstruction

• Occupancy grids and SDFs provide better-defined surfaces while retaining neural volume-rendering view synthesis capabilities.Neural implicit functions reparameterize these representations into volume density for rendering.Source: §2 · p. 2

Extensions and Auxiliary Inputs

• Some follow-up methods trade surface fidelity for real-time operation, use auxiliary information, or remain bounded by sparse point-cloud quality.These approaches include patch warping, point-cloud supervision, depth, and segmentation.Source: §2 · pp. 2–3

Neuralangelo

• Neuralangelo builds on hash encodings to recover surfaces without the auxiliary inputs used by several prior methods.Its approach differs from concurrent coarse-to-fine work by using hierarchical hash grids rather than a displacement network.Source: §2 · p. 3

Neural surface representation

• Neuralangelo encodes sampled 3D positions with multi-resolution hash features, feeds them to SDF and color MLPs, and renders images using an SDF-based formulation.The hash features are formed by trilinear interpolation at multiple grid resolutions and concatenation.Source: §3 · p. 3, §3.1 · p. 3
• Density-based representations do not clearly define surfaces, whereas SDF geometry uses the zero-level set and can be optimized through volume rendering.The SDF-based formulation computes opacity from SDF values for rendering.Source: §3.1 · p. 3

Numerical gradient computation

• Analytical hash-encoding gradients are discontinuous across grid cells, so eikonal-loss updates affect only locally sampled hash entries.This locality prevents neighboring cells spanning continuous surfaces from being jointly optimized for coherent normals.Source: §3.2 · pp. 3–4
• Numerical gradients sample neighboring SDF locations so multiple grid cells participate in surface-normal computation and receive updates simultaneously.With step sizes larger than a hash-grid cell, the numerical gradient acts as a smoothing operation on the analytical gradient.Source: §3.2 · pp. 3–4, Figure 2 · p. 4

Progressive optimization

• Neuralangelo progressively activates coarse-to-fine hash grids because activating all levels initially can cause fine grids to lose geometric details during coarse optimization.Coarse-to-fine optimization shapes the loss landscape and progressively reconstructs levels of detail.Source: §3.3 · p. 4

Datasets and evaluation

• Experiments use 15 DTU object-centric scenes and 6 Tanks and Temples indoor/outdoor scenes, evaluated with Chamfer distance, F1 score, and PSNR.DTU images come from a monocular RGB camera with structured-light ground truth; Tanks and Temples uses monocular RGB images with LiDAR ground truth.Source: §4 · p. 5

DTU Benchmark

• Neuralangelo achieves the lowest average Chamfer distance and highest average PSNR on DTU, despite not leading in every individual scene.The result is reported without using auxiliary inputs.Source: §4.1 · p. 5

Ablations

• Numerical gradients improve coarse surface smoothness but can smooth details, whereas combining them with progressive hash activation produces smooth surfaces and fine details.Analytical-gradient variants often produce noisy or artifact-laden coarse surfaces.Source: §4.1 · pp. 5–6, Figure 4 · p. 6

Tanks and Temples Benchmark

• On Tanks and Temples, Neuralangelo achieves the highest average PSNR and best average F1 score, recovering intricate details better than NeuS, NeuralWarp, and COLMAP.COLMAP surfaces are sensitive to sparse-point-cloud outliers, while NeuralWarp can predict background surfaces that worsen F1 scores.Source: §4.2 · p. 6

Level of Details

• Finer hash resolutions recover structures missed at coarse levels, while sufficiently fine resolutions are required to predict flat continuous surfaces.The observed behavior motivates numerical-gradient updates that back-propagate beyond local grid cells.Source: §4.3 · pp. 6–7, Figure 6 · p. 8

Ablations

• Curvature regularization removes surface noise, while topology warmup helps form concave regions that curvature regularization otherwise makes difficult to represent.Without curvature regularization, surfaces tend to develop undesirable sharp transitions.Source: §4.4 · p. 8, Figure 7 · p. 8

• Neuralangelo is a photogrammetric neural surface reconstruction approach that models scenes as SDFs using multi-resolution hash encoding.Source: §5 · p. 8
• Numerical gradients for higher-order derivatives and coarse-to-fine optimization are the method's central techniques.Source: §5 · p. 8
• The method recovers dense scene structures with extremely high fidelity in object-centric and large-scale indoor/outdoor scenes from RGB videos.Source: §5 · p. 8
• Neuralangelo currently uses long training iterations because random pixel sampling does not track sampling statistics and errors.Source: §5 · p. 8

A. Additional Hyper-parameters

• Training runs for 500k iterations with AdamW optimization and weight decay of 10^-2.The learning rate is initialized at 1 × 10^-3, warmed up for 5k iterations, and decayed by a factor of 10 at 300k and 400k iterations.Source: A. Additional Hyper-parameters · p. 9
• The region of interest is assumed to lie inside a unit sphere.Source: A. Additional Hyper-parameters · p. 9
• Inactive hash-grid resolutions use zero feature vectors, while the eikonal and curvature regularization settings are explicitly weighted.The eikonal weight is 0.1, and the curvature regularization strength uses linear warmup.Source: A. Additional Hyper-parameters · p. 9

• The in-the-wild scenes are captured by a consumer drone, and COLMAP recovers the camera intrinsics and poses.Source: B. Additional In-the-wild Results · p. 9

Surface reconstruction

• Neuralangelo achieves the best surface reconstruction quality and performs best on average for image synthesis on Tanks and Temples.Source: Table 3 · p. 9
• Geo-NeuS performs worse than NeuS and Neuralangelo in most Tanks and Temples scenes.The authors attribute this to noisy COLMAP point clouds degrading reconstruction.Source: C. Additional Tanks and Temples Results · p. 9

RGB image synthesis

• Neuralangelo produces high-fidelity renderings with recovered details on buildings and objects compared with NeuS.PSNR is evaluated on temporally subsampled video frames and foreground objects.Source: C. Additional Tanks and Temples Results · p. 9

Surface reconstruction

• Neuralangelo reconstructs smoother surfaces in several DTU scans and sharper details such as a pumpkin vine and statue face.Source: D. Additional DTU Results · pp. 9–10
• Neuralangelo misses button structures and eyes on highly reflective Scan 69, recovering fewer details than NeuS.This limitation is consistent with the reported weakness of multi-resolution hash encoding on reflective surfaces.Source: D. Additional DTU Results · pp. 10, 14

RGB image synthesis

• Neuralangelo generates high-fidelity RGB renderings with minute details compared with NeuS on DTU.Source: Figure 11 · p. 13
• Neuralangelo recovers detailed textures while NeuS produces overly smoothed images in the RGB synthesis comparison.Source: D. Additional DTU Results · p. 14

Evaluation

• Foreground masks are used to remove background regions for proper DTU evaluation, but the IDR masks omit part of Scan 83's object.The authors re-annotate the masks to fix this issue.Source: D. Additional DTU Results · p. 14

Color network

• Per-image appearance embedding greatly reduces floating objects used to explain exposure variation across frames.Source: E. Additional Ablations · p. 14, Figure 13 · p. 15

Curvature regularization

• Decaying curvature regularization as finer hash grids activate preserves details better.The regularization is reduced as step size ϵ decreases to avoid smoothing fine structures.Source: E. Additional Ablations · p. 14, Figure 14 · p. 15

Gradient comparison

• Numerical and analytical gradients produce nearly identical surface normals at the end of optimization when ϵ is sufficiently small.Source: E. Additional Ablations · p. 14, Figure 15 · p. 15

Color network

• The decomposed color network uses view-invariant albedo and view-dependent shading branches, with the rendered image formed from their sum.The sigmoid function normalizes predictions to the range 0 to 1.Source: E. Additional Ablations · p. 14
• The decomposed color network's shading branch explains view-dependent effects, but flat surfaces may be carved away because reflection computation is unstable.Source: E. Additional Ablations · p. 14

Computation time

• 1.2 times slowdown in training occurs with numerical gradients versus analytical gradients, while the method remains faster than NeuS.Inference for both Neuralangelo gradient variants is more than 2 times faster than NeuS at 128^3 surface extraction resolution.Source: Table 5 · p. 15, E. Additional Ablations · p. 15

Gradient locality

• Analytical derivatives of multi-resolution hash encoding suffer from gradient locality, whereas Fourier frequency encoding does not.Hash-grid locality restricts eikonal-loss backpropagation to locally sampled entries.Source: F. Derivation of Frequency Encoding · p. 15

Frequency encoding

• Fourier frequency encoding represents position with sine and cosine features at frequency 2^l.Source: F. Derivation of Frequency Encoding · p. 15

Color decomposition

• The intrinsic color decomposition uses albedo and shading images, but the implemented design lacks smoothness in homogeneous-color regions.Source: Figure 16 · p. 16, Figure 17 · p. 16

Derivatives

• Its derivative remains continuous across space and retains position information, enabling second-order derivatives for curvature regularization.Source: F. Derivation of Frequency Encoding · p. 16

Coarse-to-fine optimization

• 0.84 to 0.79 Chamfer distance is achieved when coarse-to-fine optimization is applied with Fourier encoding.The result supports annealing different frequencies through a varying step size for more robust optimization.Source: F. Derivation of Frequency Encoding · p. 16