PaperlayerView exact revision on arXiv

Source-linked AI summary

Boosting Metric Depth Completion via Training-Free Adaptive Response Geometry

Mia Zhang, Jizong Peng

arXiv:2609.36168v1cs.CV

TL;DR

Depth completion must convert dense relative-depth priors into metric depth despite fixed-coordinate affine alignments that can leave depth-dependent errors. The paper estimates an image-level response coordinate and completes the calibrated prior with hard-constrained residual reconstruction. Its training-free pipeline improves both reported aggregate measures over PriorDA, LDCM, and Any2Full.

  • Problem

    Global affine alignment in a fixed coordinate can leave systematic depth-dependent deviations, limiting conversion of relative-depth priors to metric depth.

    Source: §1 · p. 1

  • Method

    The method estimates an image-level monotone response from metric observations, then uses hard-constrained reconstruction to propagate residual corrections while enforcing those observations.

    Source: §1 · pp. 2–3, §3.3 · p. 6

  • Results

    Macro AbsRel 0.0301 and NMed 14.04° improve on PriorDA, LDCM, and Any2Full.

    Source: §1 · p. 3, §4.3 · p. 8

  • Takeaways & Limitations

    Unobserved regions receive the same dense conversion estimated from available pairs, while spatial correction does not refit that global relation.

    Source: §5 · p. 10

    A single monotone response cannot resolve arbitrary spatial disagreements or correct depth-order reversals in the prior; the hard constraints use geometry-screened benchmark measurements.

    Source: §5 · p. 10

Abstract

from arXiv · show

Depth completion aims to recover dense metric depth from sparse sensor measurements, increasingly leveraging visual foundation models as geometric priors. However, aligning these priors to true metric scale typically relies on rigid affine assumptions in predefined coordinate systems, leaving systematic calibration errors. Linearity in depth calibration depends on the response coordinate. We introduce adaptive response geometry, which makes the fixed choice of depth, log depth, or disparity an image-level unknown. A continuous response family unifies these coordinates and defines an explicit depth-dependent gain. We derive the response-gradient relation and estimate the response parameters in metric space. Hard-Dirichlet residual reconstruction completes the calibrated prior. Under deliberately incomplete metric observations, the training-free pipeline achieves macro AbsRel 0.0301 and macro NMed 14.04{\deg}, improving both aggregate measures over PriorDA, LDCM, and Any2Full. Linearity diagnostics examine how the selected response changes the depth relation and its metric error.

1 INTRODUCTION

Monocular depth priors provide relative scene structure, but their fixed alignment coordinates can leave systematic depth-dependent errors when converted to metric depth. The paper adapts a global response coordinate and combines metric calibration with hard-constrained residual reconstruction; its training-free method improves both reported aggregate measures over trained comparison systems.

1 INTRODUCTION

  • Monocular depth foundation models provide dense geometric priors, but scale ambiguity and a potentially unresolved shift mean their predictions need not be distances in meters.Models use different supervision coordinates, including disparity, log-depth, and metric depth.Source: §1 · p. 1
  • Sparse metric observations constrain conversion from prior depth to physical depth, but depth-dependent drift can remain after a single scale and offset.Region-wise affine fits can be underconstrained when sparse observations leave regions with too few anchors.Source: §1 · p. 1
  • Global affine alignment in metric depth, log-depth, or disparity can leave systematic depth-dependent deviations because one scale and offset compromise across depth ranges.Figure 2 illustrates these residuals; on its iBims example, the estimated response is λ = 2.00, outside the conventional cases λ ∈ {1, 0, −1}.Source: §1 · pp. 1–2, Figure 2 · p. 2
  • Adaptive response geometry makes the alignment coordinate an image-level unknown and uses a monotone family that includes metric depth, log-depth, and disparity as special cases.Changing λ changes the relative gain for near and far depth increments while preserving depth ordering; the response is estimated from observed pairs without model weights, training data, or the original loss.Source: §1 · pp. 2–3
  • The method estimates shared response and affine parameters from metric residuals, then propagates remaining log-depth residuals through hard-constrained reconstruction that enforces observations exactly.This separates global calibration from spatial correction rather than assigning separate affine parameters to unobserved objects.Source: §1 · p. 3
  • Macro AbsRel 0.0301 and NMed 14.04° improve on PriorDA, LDCM, and Any2Full under distributed and spatially missing observations.The evaluation covers DIODE, ETH3D, iBims-1, ARKitScenes, and five Tanks and Temples scenes, without training a completion network.Source: §1 · p. 3

2 RELATED WORK

Prior work addresses relative-to-metric alignment and depth completion through fixed-coordinate mappings, gradient-domain reconstruction, and trained completion networks. This paper situates its approach among these families while retaining a global affine relation and adapting its response coordinate.

2 RELATED WORK

  • Monocular foundation models use different prescribed coordinates for relative depth, including disparity, log-depth, and metric depth with scale-and-shift alignment.These choices support learning across heterogeneous datasets but prescribe the coordinate of linear alignment.Source: §2 · p. 3
  • Depth completion combines images with incomplete LiDAR or RGB-D observations, using prior relations, spatial regularizers, gradient-domain methods, relative-to-metric mappings, or trained networks.These methods differ in how they infer depth at unobserved pixels.Source: §2 · p. 3
  • Gradient-domain methods integrate monocular guidance with sparse anchors, but commonly fix the structural coordinate in advance.OGNI-DC reconstructs from depth gradients, OMNI-DC integrates log-depth, and LDCM uses log-gradients before learned refinement; the completion method separates this structural coordinate from the global map's response.Source: §2 · pp. 3–4
  • Relative-to-metric methods range from global disparity rescaling to spatially varying affine fits and richer nonlinear or segment-based mappings.Local parameters can be underconstrained with sparse metric support; this work instead retains one image-global affine relation while adapting its response coordinate.Source: §2 · p. 4

3 METHOD

The method adapts the response coordinate for each image, estimates its alignment to metric observations, then spatially propagates residuals while enforcing those observations exactly.

3.1 LINEARITY AS A COORDINATE CHOICE

  • A single parameter λ adapts the response coordinate to depth-dependent calibration drift, avoiding separate affine parameters for regions that become underconstrained with sparse observations.The response family connects metric depth, log-depth, and disparity; its depth-dependent gain absorbs smooth deformation before affine alignment.Source: §3.1 · pp. 4–5

3.2 METRIC-SPACE ESTIMATION

  • The method estimates λ, α, and β by minimizing metric residuals over observed pairs, using Levenberg–Marquardt for the nonlinear objective.The optimization parameterization enforces α > 0.Source: §3.2 · pp. 5–6

3.3 HARD-CONSTRAINED COMPLETION

  • Hard-constrained Poisson reconstruction propagates the remaining log-depth residual while matching each observed depth exactly.Magnitude and curvature regularization constrain the propagated correction without changing the estimated global parameters.Source: §3.3 · p. 6

4 EXPERIMENTS

Experiments evaluate depth completion under incomplete metric support and examine how response choice affects alignment. The reported diagnostics show that an adaptive response can improve held-out metric error even when fixed-coordinate correlations are already high.

4.1 INCOMPLETE METRIC SUPPORT

  • Evaluation covers indoor datasets and five Tanks and Temples scenes, combines sparse observations with central and outer removal, and excludes observed pixels from evaluation.Scores are reported for nine subsets and as their unweighted mean; metrics are AbsRel and per-image median angular normal error (NMed).Source: §4.1 · pp. 6–7

4.2 RESPONSE COORDINATE AND METRIC ALIGNMENT

  • On one iBims image, λ = 2.00 lies outside the fixed choices, and its local-median path is closer to metric identity than those choices.Scene error changes with the selected coordinate.Source: §4.2 · p. 7
  • 0.028 m held-out MAE with adaptive λ = 7.66, versus 0.044–0.047 m for λ ∈{1, 0, −1}; Pearson correlation is 0.9984 versus 0.9980–0.9982.Figure 3 shows that the metric-depth change remains visible despite correlations near 1.Source: Figure 3 · p. 7, §4.2 · p. 7

Appendix A.4 explains the diagnostic measures and display conventions.

Ours improves on the trained comparison systems in both macro depth and normal error, while the section also separates the effects of response estimation and residual completion. It reports diagnostic and implementation details alongside these comparisons.

4.3 COMPARISON WITH TRAINED COMPLETION METHODS

  • Macro AbsRel is 0.0301 and NMed is 14.04◦ for Ours, both lower than PriorDA, LDCM, and Any2Full.The strongest competing aggregates are PriorDA for AbsRel and Any2Full for NMed.Source: Table 1 · p. 8, §4.3 · p. 8
  • Ours has the lowest AbsRel on five subsets and NMed on seven; Figure 4 shows macro comparisons and per-subset p50 depth and normal errors.LDCM has lower AbsRel on iBims, ARKit, Barn, and Meetingroom; Any2Full has lower NMed on Caterpillar and Truck.Source: Figure 4 · p. 9, §4.3 · p. 9
  • On one ARKit scene, held-out MAE is 0.029 m for Ours, versus 0.131 m for PriorDA, 0.055 m for LDCM, and 0.112 m for Any2Full.The paired maps show broad wall-orientation differences and a remaining localized lamp error.Source: §4.3 · p. 9, Figure 5 · p. 10

4.4 RESPONSE ESTIMATION AND RESIDUAL COMPLETION

  • Response estimation alone yields macro AbsRel 0.0382 and NMed 11.46◦, versus 0.0524 and 13.38◦ for fixed disparity; Dirichlet completion lowers AbsRel to 0.0301 but raises NMed to 14.04◦.The residual stage improves depth accuracy on every subset while changing local surface orientation.Source: §4.4 · p. 10

5 DISCUSSION AND CONCLUSION

  • The exponent adjusts the depth-dependent gain while affine parameters remain global, but one monotone response cannot resolve arbitrary spatial disagreements or prior depth-order reversals.The hard constraints use reference measurements from the geometry-screened benchmark.Source: §5 · p. 10

A.1 NUMERICAL EVALUATION AND DENSE-DOMAIN FEASIBILITY

  • For nonzero λ, the response uses expm1(λq)/λ for numerical stability near zero, and the implementation checks dense-domain feasibility at the relevant prior endpoint.It switches to the logarithmic branch when |λ| < 10^-5; the logarithmic member has no finite response-domain boundary.Source: §A.1 · p. 14

A.2 MASKED RESIDUAL SYSTEM AND LINEAR SOLVER

  • The masked residual system fixes observed residuals and solves over free valid pixels using graph operators restricted to valid neighbors.The system remains positive definite when λdata > 0; an entirely unobserved connected component retains the calibrated prediction.Source: §A.2 · p. 14

A.3 OBSERVATION AND SCORING DETAILS

This appendix specifies how incomplete observations are formed and scored, then describes the sampling and measures used for linearity diagnostics.

A.3 OBSERVATION AND SCORING DETAILS

  • Observations are averaged within a regular grid of nominal stride 7.5; central and outer removal retain complementary regions and score the removed area.The three cases are pooled, with central and outer removal affecting 25% and 75% of image area, respectively.Source: §A.3 · p. 14
  • Depth metrics exclude anchor pixels, while normal errors use back-projected geometry and score valid pixels in the evaluation region, excluding anchors and the outer pixel ring.Macro is the unweighted mean of nine subset p50 values; p50 NMed is the median across per-image median angular errors.Source: §A.3 · pp. 14–15

A.4 LINEARITY DIAGNOSTICS

  • Linearity scatter diagnostics sample up to 6,000 held-out pixels and display supported local-median paths; adaptive panels use recorded response parameters.Predicted-depth bins are 0.05 m wide with at least 30 samples, and the same pixels retain their colors across panels with axes in different coordinates.Source: §A.4 · p. 15
  • Trend error is the equal-weighted mean absolute distance of the supported local-median path from metric identity, expressed in centimeters.Pearson measures linear association, whereas metric identity also fixes scale and offset; scatter statistics and full held-out MAE use different supports.Source: §A.4 · p. 15

A.5 COMPLETE BASELINE COMPARISONS

The comparison reports p50 scores across nine subsets and examines how method rankings vary across percentiles. Ours leads on several subset comparisons, while LDCM and Any2Full lead on others.

A.5 COMPLETE BASELINE COMPARISONS

  • The comparison includes PriorDA, LDCM, Any2Full, PromptDA, and Ours; KNN, LWLR, and Poisson provide additional reconstruction baselines.KNN uses five nearest observations, LWLR fits affine depth maps on a 24×24 grid, and Poisson is an LDCM-style multiscale log-depth reconstruction baseline.Source: §A.5 · pp. 15–16
  • Ours has the lowest AbsRel on DIODE, ETH3D, Caterpillar, Ignatius, and Truck.LDCM has lower AbsRel on iBims (0.0203 versus 0.0208), ARKit (0.0145 versus 0.0154), Barn (0.0207 versus 0.0251), and Meetingroom (0.0507 versus 0.0569).Source: §A.5 · p. 15
  • Caterpillar reverses its p50 AbsRel ordering by p93: Ours leads at p50 (0.0469 versus 0.0571), while Any2Full leads at p93 (0.1069 versus 0.1174).LDCM also leads Ours on Meetingroom at p50 and p93, while Ours remains below LDCM on Ignatius at p93 (0.0399 versus 0.5687).Source: §A.5 · p. 16
  • Figure 6 compares eight methods across p7–p93, with rows for percentiles and columns for AbsRel and median normal error.The p50 table does not show the full distributions; accompanying CSVs contain numerical percentiles and failure counts.Source: §A.5 · p. 16, Figure 6 · p. 17

B RESPONSE ESTIMATION AND RESIDUAL COMPLETION

Response estimation provides a dense depth relation, and residual completion enforces observed metric pairs spatially. The residual stage lowers p50 AbsRel for all nine subsets.

B RESPONSE ESTIMATION AND RESIDUAL COMPLETION

  • Response estimation selects one dense depth relation from observed metric pairs; residual completion then enforces those pairs spatially.Both stages use the same monocular prior, isolating the effect of the residual solve.Source: B RESPONSE ESTIMATION AND RESIDUAL… · p. 18
  • Dirichlet completion lowers AbsRel on every subset, including Caterpillar (0.0701 to 0.0469) and ETH3D (0.0269 to 0.0180).MAE decreases on eight subsets, but iBims changes slightly from 0.0604 m to 0.0610 m; the two depth metrics weight errors differently.Source: B RESPONSE ESTIMATION AND RESIDUAL… · p. 18, Table B.1 · p. 18
  • All nine response-only versus complete p50 AbsRel points fall below the equality line in the stage scatter.Figure 7 shows the depth effect of residual completion; all seven percentiles and failure counts are in the accompanying CSVs.Source: B RESPONSE ESTIMATION AND RESIDUAL… · p. 18, Figure 7 · p. 18

C.1 GEOMETRY OF THE COMPLETE METHOD

The complete method has lower median normal error than the trained complete baselines on most subsets. Response-only outputs have still lower normal errors across all nine subsets, making them a target for the spatial reconstruction stage.

C.1 GEOMETRY OF THE COMPLETE METHOD

  • Complete Ours has the lowest p50 NMed on seven of nine subsets; Any2Full leads on Caterpillar (21.1007° versus 22.6456°) and Truck (14.0926° versus 14.5495°).Per-image mean angular error complements median error by giving greater influence to large angular deviations; both are reported at p50.Source: §C.1 · p. 19, Table C.2 · p. 19

C.2 HIGHER NORMAL QUALITY ALREADY PRESENT IN THE RESPONSE STAGE

  • Response-only NMed is lower than that of the four trained complete baselines on every subset, and lower than complete Ours on all nine.Meetingroom changes from 19.3755° in the final result to 13.1470° before residual completion, and Truck from 14.5495° to 9.9316°; preserving more response-stage normal quality is a target for spatial reconstruction.Source: §C.2 · p. 19, Table C.2 · p. 19, Figure 8 · p. 20

D RESIDUAL-RECONSTRUCTION WEIGHT SENSITIVITY

The weight sweep finds a depth–normal trade-off when the data weight increases, while changing the Laplacian weight has little effect over the tested range.

D RESIDUAL-RECONSTRUCTION WEIGHT SENSITIVITY

  • Raising λdata from 10^-4 to 10^-2 increases MAE from 0.1326 m to approximately 0.1353–0.1354 m while lowering NMed from 14.31° to approximately 13.86–13.87°.The stronger proximity penalty keeps the solution closer to the calibrated prior.Source: D RESIDUAL-RECONSTRUCTION WEIGHT… · p. 21, Table D.1 · p. 21
  • The backend sweep can change depth and normal quality in different directions; tested weights yield modest changes, while the main experiments use (λdata,λlap)=(10^-3,10^-3), fixed independently of the sweep.Source: D RESIDUAL-RECONSTRUCTION WEIGHT… · p. 21

E TRANSFER ACROSS MONOCULAR PRIORS

Across additional monocular priors, residual completion lowers AbsRel on every evaluated subset, while final depth error still depends on the prior. Response-only calibration beats the best fixed-coordinate entry across all 45 prior–subset combinations, although normal-error rankings vary and PromptDA shows a broad spread across percentiles.

E TRANSFER ACROSS MONOCULAR PRIORS

F.1 COORDINATE SELECTION ACROSS PRIORS AND SUBSETS

  • 45 prior–subset combinations show lower response-only AbsRel than the best fixed-coordinate entry; on MoGe-3 DIODE-in, response-only is 0.0356 versus 0.0416 for GLog.The fixed-coordinate reference is the best of GDisp, GMetric, and GLog for each prior and subset.Source: §F.1 · p. 24

F.2 DEPTH AND NORMAL BEHAVIOR NEED SEPARATE COMPARISONS

  • Normal-error rankings vary: on MoGe-3 ETH3D, response-only p50 NMed is 10.6771° versus 10.6612° for GLog, and Meetingroom is 13.1470° versus 13.0269°.With MoGe-2 Small, fixed coordinates also provide lower NMed on several subsets.Source: §F.2 · p. 24

F.3 SUBSET COMPARISONS AND UNSUCCESSFUL ENTRIES

  • On DIODE-in, DAV2-B has 25 GMetric failures and 21 GLog failures; on ETH3D, the counts are 23 and 19.Tables F.1 and F.2 group standard benchmarks and Tanks and Temples scenes; failure counts are also retained in the CSVs.Source: §F.3 · p. 24, Table F.1 · p. 25, Table F.2 · p. 26

G.2 PROMPTDA INFERENCE PROTOCOL AND SUPPORT DIAGNOSTIC

G.1 BENCHMARK INCLUSION, SUPPORT STRUCTURE, AND DEPTH RANGE

The benchmark combines reference geometry from several sources with varied scene content and valid-support structure. Courthouse is excluded because vegetation-related projection mismatches would require substantial scene-specific filtering; retained support can be highly connected yet cover little of an image.

Benchmark composition

  • The benchmark combines laser-based, scan-based, FARO, and projected laser reference geometry across indoor scenes and object-centered scans with localized valid support.The five Tanks and Temples scenes share a calibration source and projection procedure; Church lacks calibration in that bundle, while Courthouse is excluded after auditing.Source: §G.1 · p. 27

Benchmark inclusion

  • Courthouse is excluded because vegetation caused projected reference mismatches, and correcting the remainder would require substantial scene-specific filtering.Using the stated far-pixel criterion, 20.8% of frames had more than 10% far pixels; min-pooling recovered approximately 23–57% of these pixels.Source: §G.1 · p. 27
  • The retained sample totals 3,698 unique images and 11,094 image–observation-case entries before method-specific validity filtering; low coverage alone is not an exclusion criterion.ARKit uses at most four uniformly selected frames per video at 384×512; ARKit and iBims reference depth is capped at 10 m.Source: §G.1 · p. 27

Support structure

  • Ignatius has 15.54% median image coverage, yet 99.99% of valid pixels belong to one component and its interior-hole fraction is numerically zero.ETH3D has greater median coverage (68.01%) but a smaller largest component (58.16% of valid pixels) and a 9.00% interior-hole fraction.Source: §G.1 · pp. 27–28

Depth range

  • Reference depth spans different scales: ARKit pooled p50 is 1.198 m, Barn p50 is 5.964 m, ETH3D reaches 15.967 m at p93, and Ignatius reaches 3.822 m.These measurements precede further observation removal; removed image-area fraction is not the remaining valid-anchor fraction.Source: §G.1 · p. 28
Loading 2609.36168v1…