PaperlayerView exact revision on arXiv

Source-linked AI summary

One Scene, Two Depths: Probing Geometric Ambiguity in Monocular Foundation Models

Xiaohao Xu, Feng Xue, Xiang Li, Haowei Li, Shusheng Yang, Tianyi Zhang, Matthew Johnson-Roberson, Xiaonan Huang

arXiv:2606.29600v1cs.CVcs.AI

TL;DR

Monocular depth models collapse layered geometry into one depth value, making transparent-scene labels dependent on annotation and training conventions. This paper introduces MD-3k and evaluates whether LVP can expose alternative layer hypotheses, with DAv2-L reaching 75.5% ML-SRA using RGB/LVP outputs.

  • Problem

    Monocular depth models reduce rays containing multiple valid surfaces to one depth value, making transparent-scene supervision a layer convention rather than scene-intrinsic truth.

  • Method

    The paper introduces MD-3k, a sparse two-layer ordinal benchmark, and compares standard RGB predictions with frozen-model outputs after training-free Laplacian Visual Prompting.

  • Results

    75.5% Multi-Layer Spatial Relationship Accuracy (ML-SRA) is achieved by DAv2-L’s RGB/LVP pair, while leading models show diverse RGB layer preferences.

  • Takeaways & Limitations

    The findings suggest that frozen depth models can express complementary geometric hypotheses and that depth systems should represent and evaluate multiple plausible scene depths.

  • Takeaways & Limitations

    Because all benchmark images come from GDD, broader validation across capture domains, transparent materials, and object categories remains necessary.

Abstract

from arXiv · show

A faithful 3D world representation should account for layered geometry, where a single camera ray may contain multiple visible and geometrically valid surfaces. Monocular depth estimation, however, reduces this structure to one scalar depth per pixel. Transparent scenes make this ambiguity measurable: the same ray can pass through foreground glass and observe the background, turning the supervised target into a convention of annotation, data, and training rather than a scene-intrinsic truth. A learned predictor exposes this convention as its depth-layer preference. We introduce MultiDepth-3k (MD-3k), a sparse two-layer ordinal benchmark for measuring depth-layer preference and multi-layer spatial relationship accuracy (ML-SRA). On MD-3k, leading depth foundation models exhibit diverse layer preferences under standard RGB input, showing that the same layered geometry can be resolved differently across models. We further find that Laplacian Visual Prompting (LVP), a training-free spectral input transformation, can substantially change the reported layer for certain frozen models. The strongest RGB/LVP pair, DAv2-L, reaches 75.5% ML-SRA. These results suggest that depth foundation models may express complementary geometric hypotheses that standard RGB inference leaves unexpressed. We invite the community to rethink depth supervision and evaluation through an ambiguity-aware lens, where multiple valid 3D interpretations are treated as geometric structure to be measured, preserved, and expressed.

1 Introduction

Transparent scenes reveal that monocular depth models collapse multiple valid surfaces along one ray into a dataset-shaped single-depth preference. The paper introduces MD-3k and LVP to measure and modulate these preferences, finding diverse model behaviors and 75.5% ML-SRA for DAv2-L’s RGB/LVP pair.

  • Geometric ambiguity: Transparent scenes can contain multiple visible, geometrically valid surfaces along one ray, but monocular foundation models report one depth value per pixel.Single-depth supervision therefore records only one potentially biased layer.
  • Geometric ambiguity: Mixed sensor and synthetic supervision collapses layered geometry into a dataset-shaped scalar target, causing models to learn depth-layer preferences rather than layer-free geometry.Different sensing and rendering conventions can emphasize different physical surfaces.
  • MD-3k: MD-3k measures depth-layer preference and multi-layer spatial relationship accuracy using sparse ordinal labels for transparent foreground and visible background surfaces.The benchmark evaluates whether a single-layer output satisfies either the foreground or background spatial relation.
  • Layer modulation: Leading depth foundation models exhibit diverse layer preferences under standard RGB input, while LVP can change the predicted layer for certain frozen models without retraining.LVP is a deterministic high-frequency input-space transform that exposes candidate complementary ordinal behavior.
  • Results: 75.5% ML-SRA is achieved by DAv2-L’s RGB/LVP pair on MD-3k, exceeding the strict 56.4% duplicated single-hypothesis ceiling.The result is reported as the strongest case among the evaluated model/input pairs.
  • Contributions: Fixed-weight, single-output models can express different valid ordinal relations when only the input representation changes, although LVP’s modulation remains model-dependent.The paper frames LVP as a training-free spectral probe of complementary behavior.

2 Related Work

Related work spans the evolution from domain-specific monocular depth estimators to general-purpose foundation models, alongside emerging multi-layer inference and input-space prompting. This work differs by probing how frozen single-output models’ depth-layer preferences change under controlled, nonlearnable spectral input modulation.

  • Monocular depth estimation: Monocular depth estimation has evolved from domain-specific models trained on KITTI and NYUv2 to general-purpose systems trained on large-scale mixed data.The passage describes modern MDE foundation models as providing broad generalization, while noting that single-output methods do not explicitly represent multiple geometric interpretations.
  • Multi-layer inference: Multi-layer methods retrain models to predict separate geometric layers, whereas this work studies frozen single-output models’ depth-layer preference under controlled input modulation.This distinction separates predicting multiple layers from examining how an existing model expresses one layer when its input is modified.
  • Input-space prompting: Input-space prompting transforms inputs without parameter optimization, and this work extends the paradigm to 3D geometry through nonlearnable spectral prompting.Prior vision prompting includes learnable prompts and input-space prompting; the proposed direction applies the latter to geometric inference.

3 Methodology

The methodology represents transparent-scene ambiguity with sparse two-layer ordinal labels, measures single-output layer preference and paired-output complementarity, and tests whether LVP changes the convention expressed by frozen depth models. It introduces MD-3k as a focused transparent-scene benchmark and reports SRA, α, and ML-SRA to separate these behaviors.

  • Two-layer ordinal representation: The framework defines single-output depth models as frozen predictors emitting one scalar depth per pixel, while ambiguous scenes are represented by ordered layers D^(1) and D^(2).The ordered representation satisfies D^(1)(x) ≤ D^(2)(x) at each image location.
  • Two-layer ordinal representation: Sparse ordinal constraints replace dense metric labels because physical sensors cannot reliably capture a complete, sensor-neutral stack of visible surfaces along transparent-scene rays.The formulation follows the sparse ordinal approach of DIW and DA-2K.
  • Layer preference and complementarity: Depth-layer preference α(fθ) is the expected background-minus-foreground ordinal correctness, with α > 0 indicating background preference and α < 0 indicating foreground preference.The absolute value of α denotes preference strength.
  • Layer preference and complementarity: Paired-hypothesis complementarity evaluates whether two predicted depth maps can jointly match both annotated layers under one dataset-level assignment fixed across images.The assignment is calibrated by maximizing joint ordinal correctness across the benchmark.
  • MD-3k benchmark: MD-3k contains 3,161 RGB images with one annotated point pair per image, partitioned into Same and Reverse subsets according to whether layer orderings agree or conflict.The Same subset has 1,783 pairs, while the Reverse subset has 1,378 pairs; the benchmark remains a focused diagnostic rather than general transparent-scene coverage.
  • Laplacian Visual Prompting: LVP applies a Laplacian-transformed version of the same RGB image as an alternative input to the frozen model, probing whether selectively emphasized frequency components express a different layer convention.The RGB and LVP outputs are evaluated with a deterministic complementary-layer assignment based on benchmark-level SRA.

4 Experiments

Experiments measure depth-layer preference and paired RGB/LVP performance across monocular depth models on MD-3k. Results show model-dependent layer biases and spectral responses, with DAv2-L achieving 75.5% overall ML-SRA.

  • Experimental setup: The study evaluates diverse pretrained monocular depth models training-free, including Depth Anything, DPT, ZoeDepth, Marigold, GeoWizard, Depth Pro, UniK3D, and UniDepth-v2.RGB and LVP candidate pairs are evaluated using depth-layer preference and ML-SRA, alongside scale, training-distribution, prompt, and paired-hypothesis analyses.
  • Per-layer ordinal behavior: Depth foundation models exhibit strong but inconsistent RGB layer preferences, with DAv2 general-purpose variants favoring foreground while DAv2-O and DAv1 favor background.Preference is computed as α = SRA(2)−SRA(1), with α < 0 indicating foreground preference and α > 0 indicating background preference.
  • Paired-output complementarity: 75.5% overall ML-SRA is achieved by the DAv2-L RGB/LVP pair, exceeding the 56.4% strict ceiling for duplicated single-map pairs.The duplicated baseline scores 100% on Same but 0% on Reverse, yielding 56.4% overall from the benchmark partition.
  • Model-family trend: LVP response is model-dependent rather than determined by discriminative versus generative architecture, with DAv2 and DPT showing stronger modulation than DAv1, Marigold, and GeoWizard.The results attribute this variation to model spectral receptivity shaped by architecture and training regime, while leaving direct causal analysis for future work.
  • Scaling behavior: Scaling improves Reverse complementarity when RGB and LVP select different layers, but paired outputs offer less benefit when both inputs retain the same bias.On consistent relations, the RGB/LVP ML-SRA gap often shrinks as model size grows.
  • Prompt-design ablations: Gaussian prompting outperforms LVP across models on unambiguous scenes, whereas high-frequency prompting helps conflicting layer orderings; results remain stable across kernel, sign, and grayscale variants.Across alternative spectral prompts, no operator dominates every model and subset: Sobel matches DAv2-S on Reverse, while LVP is strongest for DAv2-B/L.

5 Discussion and Conclusions

Ambiguous layered scenes expose a fundamental limitation of single-depth estimation: scalar targets collapse multiple geometrically valid interpretations into dataset-dependent layer conventions. MD-3k makes this convention explicit, revealing diverse RGB layer preferences and LVP-based modulation of some frozen models without retraining.

  • Discussion and Conclusions: Single-depth estimation collapses multiple geometrically valid ray-wise interpretations into one dataset-dependent layer convention.A single scalar target cannot represent layered geometric ambiguity.
  • Discussion and Conclusions: MD-3k makes the dataset-dependent layer convention explicit.The benchmark frames annotation convention as an observable property of depth estimation.
  • Discussion and Conclusions: Leading depth foundation models exhibit diverse layer preferences under standard RGB input.The models resolve the same ambiguity through different preferred layers.
  • Discussion and Conclusions: LVP can modulate some frozen models to express complementary layer hypotheses without retraining.The modulation is achieved through a training-free input transformation rather than model updates.

Model LVP Sobel Fourier high-pass Wavelet · A More Quantitative Results · A.1 Alternative High-Frequency Prompts

Table A shows that several high-frequency prompts can alter the layer preference expressed by models on MD-3k. LVP performs best overall for five of nine DAv2 variants, including DAv2-B/L, but no operator is universally superior.

  • A.1 Alternative High-Frequency Prompts: LVP, Sobel, Fourier high-pass, and wavelet prompts are compared on MD-3k.The comparison evaluates alternative high-frequency input transformations.
  • Model LVP Sobel Fourier high-pass Wavelet: The alternative prompts also modulate the layer preference expressed by models.This supports treating high-frequency prompting as a family of diagnostics.
  • A. More Quantitative Results: The findings support a family of high-frequency diagnostics rather than a Laplacian-specific hidden-state claim.The conclusion follows from the effects observed across LVP, Sobel, Fourier high-pass, and wavelet prompts.
  • A.1 Alternative High-Frequency Prompts: No operator is universally best across the evaluated DAv2 variants.Performance varies by model variant and prompt operator.
  • A. More Quantitative Results: LVP gives the highest overall score for five of the nine DAv2 variants.This is the aggregate result reported in Table A.
  • A.1 Alternative High-Frequency Prompts: The DAv2-B/L variants are among those for which LVP gives the highest overall score.The passage specifically identifies DAv2-B/L as included in LVP’s five winning variants.

A.2 Zero-Shot Dense Depth Fidelity on Non-Ambiguous Benchmarks · A.3 Comparison between LVP and Canny Binary Edge Prompts · A.4 Depth-Layer Preference Values

LVP probes alternate depth hypotheses but generally sacrifices conventional single-depth fidelity on non-ambiguous benchmarks. On MD-3k Reverse, it outperforms Canny prompts for DAv2 models, while α and ∆α quantify layer preference and its change under LVP.

  • A.2 Zero-Shot Dense Depth Fidelity on Non-Ambiguous Benchmarks: LVP increases sensitivity to alternate depth hypotheses but usually reduces accuracy under conventional single-depth metrics.Across 18 RGB/LVP comparisons, AbsRel increases by 0.003–0.060, while δ1, δ2, and δ3 show mixed or negative changes.
  • A.2 Zero-Shot Dense Depth Fidelity on Non-Ambiguous Benchmarks: AbsRel increases by 0.003–0.060 across the 18 RGB/LVP comparisons.The reported comparisons cover NYU-D, KITTI, and ETH3D.
  • A.2 Zero-Shot Dense Depth Fidelity on Non-Ambiguous Benchmarks: δ1 ranges from a 0.09-point gain to a 10.91-point drop, while δ2 drops by 0.85–3.82 points and δ3 ranges from a 0.11-point gain to a 1.33-point drop.Losses are small on KITTI but substantial on ETH3D for several models, supporting LVP as an ambiguity probe rather than a universal RGB replacement.
  • A.3 Comparison between LVP and Canny Binary Edge Prompts: LVP consistently outperforms the best Canny variant on the MD-3k Reverse subset for the non-finetuned DAv2-S/B/L family.The controlled comparison evaluates Edge-1 through Edge-4 with increasing hysteresis thresholds: (50, 150), (60, 180), (70, 210), and (80, 240).
  • A.3 Comparison between LVP and Canny Binary Edge Prompts: The comparison suggests that high-frequency prompting helps under conflicting layer orderings.It also suggests that the continuous Laplacian residual contains richer structural information than a binarized edge map.
  • A.4 Depth-Layer Preference Values: Depth-layer preference is quantified as α = SRA(2) − SRA(1) on the MD-3k Reverse subset.Positive α indicates stronger agreement with the visible background layer, whereas negative α indicates stronger agreement with the transparent foreground layer.
  • A.4 Depth-Layer Preference Values: ∆α = αLVP − αRGB summarizes how much LVP changes the expressed layer preference.The final column of Table D reports this change relative to standard RGB input.

B More Qualitative Results

Qualitative examples show that Laplacian Visual Prompting can alter depth ordering in transparent regions while preserving recognizable scene structure for receptive models. The figures also document failure cases and limitations of semantic-prior variants.

  • Qualitative modulation: LVP often changes output ordering in transparent regions while preserving recognizable scene structure in receptive models.The examples use RGB- and LVP-conditioned outputs, including extended examples with DAv2-L.
  • Failure cases: Failure cases show that LVP-conditioned output modulation has limitations at challenging boundaries.Figure H presents paired RGB and Laplacian inputs and outputs for these failures.
  • Qualitative modulation: The RGB/LVP figures compare each frozen model’s standard RGB depth estimate with its corresponding Laplacian-conditioned output.Each case presents the RGB or Laplacian input alongside the output from the same frozen model.
  • Semantic-prior variant: Successful semantic-prior examples show GT-mask interpolation for reference and predicted-mask interpolation as the deployable variant.The semantic prior is illustrated in Figure J through multi-layer depth outputs.
  • Semantic-prior variant: Semantic-prior failure cases indicate that the deployable predicted-mask variant does not consistently reproduce the reference behavior.Figure K contrasts GT-mask interpolation with predicted-mask interpolation in failure examples.

C Implementation Details … D.1 Motivation

The implementation compares a semantics-guided multi-layer depth method with LVP, uses RGB/LVP depth hypotheses for geometry-conditioned generation, and evaluates them under fixed benchmark-level layer assignments. The MD-3k datasheet motivates the benchmark by documenting shortcomings of existing transparent-scene depth data and the need for multi-layer spatial relationship labels.

  • C.1 Multi-layer Depth via Semantic Prior for Comparison: DAv1-L predictions are combined with semantic segmentation to estimate nearer transparent surfaces by interpolating depth from transparent-region boundaries.DAv1-L’s tendency to predict greater depths in ambiguous regions motivates estimating the nearer layer from region boundaries.
  • C.1 Multi-layer Depth via Semantic Prior for Comparison: The hybrid DAv1-L and semantic-segmentation approach achieves higher quantitative multi-layer depth precision than training-free LVP, but relies on task-specific model combinations.The authors emphasize developing foundation models that directly handle multi-layer depth estimation.
  • C.2 Depth-Conditioned Image Generation: RGB/LVP depth outputs condition ControlNet-based image generation, with depth controlling geometry and text prompts controlling appearance.The illustrations test conditioning diversity rather than metric geometric accuracy, and selected outputs represent alternative scene hypotheses.
  • C.2 Depth-Conditioned Image Generation: The benchmark-level assignment protocol computes per-layer RGB accuracies and assigns RGB and LVP to layers according to the sign of αRGB = SRARGB(2) − SRARGB(1).The resulting RGB/LVP-to-layer assignment is selected once per model and fixed for all images.
  • C.3 Output Assignment and Controllability: ML-SRA treats RGB and LVP as an unordered candidate pair and evaluates the fixed benchmark-level permutation across every image.The protocol does not select assignments per instance or use a per-image oracle.
  • C.3 Output Assignment and Controllability: The protocol measures candidate-pair complementarity after dataset-level label matching, not user-controlled layer selection; deployment would require labeled calibration or an external signal.Possible external signals include semantic, material, or uncertainty information.
  • D.1 Motivation: MD-3k addresses noisy sensor depth, inaccurate human-curated depth, and the absence of multi-layer spatial relationship labels in ambiguous transparent scenes.The benchmark focuses on evaluating multi-layer spatial perception and depth disentanglement in ambiguous 3D scenes.
  • D Datasheet for MD-3k Benchmark: The MD-3k datasheet follows established dataset-documentation guidelines and identifies the benchmark’s creators as the paper’s authors.The creation is reported as unfunded, with funding listed as N/A.

D.2 Composition … D.5 Uses

MD-3k is a 3,161-image benchmark built from GDD to evaluate layered spatial understanding in ambiguous scenes, using expert-annotated masks and two-layer ordinal relationships. It is self-contained in annotations but requires the original GDD images, has no predefined splits, and supports evaluation of depth-layer preference, disentanglement, and related tasks.

  • D.2 Composition: MD-3k contains 3,161 high-resolution RGB images and 3,161 annotated sparse-point pairs, each with ordinal labels for transparent foreground and visible background layers.Images depict indoor and outdoor scenes with ambiguous regions, especially transparent objects.
  • D.2 Composition: The dataset is a targeted, nonrandom sample from GDD selected for scenes rich in ambiguous regions, particularly transparent objects.It is intended to address multi-layer spatial understanding in challenging scenarios.
  • D.2 Composition: Each instance includes a 720p PNG RGB image, binary ambiguity-region masks, and JSON near/far labels for sparse point pairs in two layers.The released annotations are the segmentation masks and spatial relationship labels; the original RGB images remain available through GDD.
  • D.2 Composition: MD-3k provides no predefined train, development, or test splits because it is primarily treated as an exploratory diagnostic benchmark.Users may define splits according to their research needs.
  • D.2 Composition: MD-3k is distributed with annotations, masks, and image lists but requires users to obtain the underlying GDD images separately, without guaranteed long-term availability or archival copies.Users must follow GDD’s separate licensing terms.
  • Q15 Is it possible to identify individuals (i.e., one or more natural: The benchmark does not identify subpopulations, individuals, or sensitive information, and its images depict ordinary indoor and outdoor scenes without known offensive content.The provided documentation marks the person-related questions as not applicable or negative.
  • D.4 Preprocessing, Cleaning, and/or Labeling: Labeling is the benchmark’s core contribution, while no other RGB preprocessing or cleaning was performed and the annotation tools are not publicly released.The documentation instead provides descriptions of the annotation process and data format.

Q4 Is there anything about the composition of the dataset or the way it · D.6 Distribution and License

MD-3k targets ambiguous transparent scenes and may not represent ordinary unambiguous scenes, so results should be interpreted within its multi-layer spatial focus. The benchmark is publicly released for research through GitHub, without an assigned DOI.

  • Q4 Is there anything about the composition of the dataset or the way it: MD-3k focuses on ambiguous scenes, particularly those with transparent objects.Users should account for this specialized scenario when applying the benchmark and interpreting results.
  • Q4 Is there anything about the composition of the dataset or the way it: The dataset might not represent general scenes without ambiguity.Its focused composition should inform how broadly benchmark results are interpreted.
  • Q4 Is there anything about the composition of the dataset or the way it: Because MD-3k does not involve people or sensitive attributes, the risk of unfair treatment or other harms is considered low.Responsible and ethical use remains encouraged.
  • Q4 Is there anything about the composition of the dataset or the way it: MD-3k has no identified task exclusions, but using it for tasks unrelated to spatial reasoning or depth perception might be inappropriate.Its primary focus is multi-layer spatial understanding in ambiguous regions.
  • D.6 Distribution and License: MD-3k will be publicly available for research purposes.The benchmark is intended for distribution to third parties outside the creating entity.
  • D.6 Distribution and License: The benchmark is distributed through GitHub.No DOI has been assigned.
  • D.6 Distribution and License: The dataset has already been released publicly.Its release timing is described as current rather than prospective.

Q4 Will the dataset be distributed under a copyright or other intellec- … I Public Model and Code Resources Used

MD-3k is released openly with separate obligations for its underlying GDD images, while the authors plan ongoing maintenance, versioned access, and community contributions. The work is positioned as a lightweight, model-dependent diagnostic rather than a complete or safety-certified multilayer estimator, and it documents the public models and code resources used.

  • Q4 Will the dataset be distributed under a copyright or other intellec-: MD-3k annotations and code are released under the Apache-2.0 license for research and commercial use with attribution.The license permits free use, modification, and distribution.
  • Q4 Will the dataset be distributed under a copyright or other intellec-: MD-3k relies on GDD RGB images, so users must comply with GDD’s separate licensing terms and any applicable restrictions.The authors report no other known third-party IP restrictions on the annotations and benchmark data.
  • D.7 Maintenance: The authors will support, host, and maintain MD-3k, with contact available through the GitHub issue tracker and repository email.They report no erratum and may announce benchmark updates in the dataset repository.
  • Q1 Who will be supporting/hosting/maintaining the dataset?: All MD-3k versions are intended to remain accessible in the GitHub repository for reproducibility and comparison, including versions marked obsolete.The repository will clearly identify obsolete versions while retaining access to them.
  • Q1 Who will be supporting/hosting/maintaining the dataset?: Community members may extend MD-3k by reporting issues, submitting pull requests, or proposing new annotations and dataset extensions.Suggested contributions include evaluation scripts, new baselines, and other benchmark improvements.
  • E Broader Impact: The work is a lightweight diagnostic for transparent-scene ambiguity, not a complete or safety-certified multilayer estimator.Its response is model-dependent, may degrade standard depth accuracy, and ordinal benchmark success does not guarantee metric correctness.
  • F Availability and Maintenance: The public release includes LVP code, MD-3k annotations and image-retrieval instructions, evaluation scripts, baselines, and a reproduction guide.The materials are publicly available through the Ambiguity-in-Space GitHub repository, with planned versioned releases and documentation.
  • I Public Model and Code Resources Used: The study acknowledges public resources including Depth-Anything-v2, Depth-Anything, DPT, ZoeDepth, and Marigold with their associated licenses.The listed licenses include Apache-2.0+CC-BY-NC-4.0, Apache-2.0, and MIT.
Loading 2606.29600v1…