AnyDepth: Depth Estimation Made Easy

1 INTRODUCTION

• Depth maps support scene-distance measurement and serve as conditioning information in 3D reconstruction, generation, and embodied-AI models.Source: §1 · p. 1
• Prior work includes DPT as a mainstream decoder and Depth Anything’s use of massive datasets to improve model understanding and generalization in complex scenarios.Source: §1 · p. 2
• DPT’s per-layer reassembly and alignment add complexity, parameters, and latency, while bilinear upsampling can blur edges and lose fine details.The paper also identifies costly, noisy large-scale data collection as a limitation of purely data-driven approaches.Source: §1 · p. 2
• The proposed decoder fuses features before one-shot resolution restoration, avoiding multi-branch alignment and repeated reconstruction to better preserve detail and geometric consistency.Source: §1 · p. 2
• AnyDepth reports comparable accuracy and generalization to DPT with significantly fewer parameters and lower training overhead.The paper pairs this decoder contribution with filtering low-quality training samples.Source: §1 · p. 3
• The authors propose two sample-quality metrics and use them to filter low-quality samples, reducing dataset size while improving overall data quality.Source: §1 · p. 3

Zero-Shot Monocular Depth Estimation

• Zero-shot methods often predict affine-invariant depth, recovering relative structure while remaining invariant to scale and translation.Source: §2 · p. 3
• Prior approaches include transformer and diffusion models, larger-scale training with 62 million unlabeled images, and joint depth-normal prediction.Depth Anything used 62 million unlabeled images; Geowizard jointly predicts depth and normals.Source: §2 · p. 3

Decoder for Dense Prediction

• Dense-prediction decoders commonly fuse multi-scale features; DPT uses ViT features for finer representations, but multi-branch reassembly adds overhead, especially at high resolution.DPT uses a ViT backbone.Source: §2 · p. 3

3.1 OVERVIEW

• DINOv3’s four intermediate transformer layers provide multi-scale representations that SDT fuses and refines before generating a dense depth map.Source: §3.1 · p. 3, Figure 4 · p. 4

3.2 SIMPLE DEPTH TRANSFORMER (SDT)

• SDT projects encoder tokens into 256 dimensions, then uses learnable weighted fusion to combine representations from different layers.The projection uses a linear layer followed by GELU; softmax-normalized weights balance the layer contributions.Source: §3.2 · p. 4
• The Spatial Detail Enhancer applies local spatial modeling after token reshaping, addressing the loss of continuity and shallow texture detail in fused features.Its operation uses a 3 × 3 depthwise convolution, batch normalization, a residual connection, and an activation.Source: §3.2 · p. 4
• Instead of bilinear interpolation, SDT uses a learnable dynamic sampler; progressive two-stage ×4 upsampling with scale-2 DySamples supports local refinement and detail recovery.The design avoids one-shot ×16 upsampling, which the authors say can amplify errors and destabilize gradients.Source: §3.2 · pp. 4–5

3.3 SDT VS. DPT

• Unlike DPT’s reassemble-fusion order, SDT fuses projected tokens before spatial reassembly and upsampling, avoiding per-layer reassembly and cross-scale alignment overhead.The paper describes this single-path strategy as more efficient and stable, especially for high-resolution inputs.Source: §3.3 · p. 5

4.1 DATASETS AND METRICS

• Training uses five synthetic datasets and 369K samples; evaluation covers five zero-shot datasets and reports AbsRel and δ1 accuracy thresholds.The training set spans indoor and outdoor scenes, and the evaluation datasets are NYUv2, KITTI, ETH3D, ScanNet, and DIODE.Source: §4.1 · p. 5

4.2 IMPLEMENTATION DETAILS

• The encoder remains frozen at 768 × 768 input resolution, and decoder inputs come from four intermediate DINOv3 layers; training lasts five epochs.The reported setup uses scale- and shift-invariant loss with gradient matching loss at a 1:2 weight ratio.Source: §4.2 · pp. 5–6

4.3.1 RESULTS OF DATA CENTRIC LEARNING

• 20% of the lowest-scoring samples for each metric were filtered after excluding maps with under 20% valid depth; Hypersim had the highest overall quality score.Outdoor datasets including VKITTI2, BlendedMVS, and TartanAir had lower Depth Distribution Scores.Source: §4.3.1 · p. 6

4.3.2 QUANTITATIVE COMPARISONS

• SDT uses 5–13M parameters and outperforms DPT across encoder sizes, but AnyDepth does not yet surpass fully data-driven state-of-the-art methods.The authors describe AnyDepth as designed around lightweight model design and data quality, and report using 369K high-quality training samples.Source: Table 2 · p. 7, §4.3.2 · p. 7

4.4 EFFICIENCY

• 37% lower FLOPs and 10% shorter average training iterations accompany reduced parameters across model sizes, with inference latency comparable to or slightly faster than DPT.The FLOPs reduction is reported for models of varying sizes, particularly at high resolutions.Source: §4.4 · pp. 7–8, Table 5 · p. 7, Table 4 · p. 8

4.5 REAL WORLD EVALUATION

• At 256×256 on Jetson Orin Nano (4GB), SDT uses approximately 33% less peak memory; it also outperforms DPT in latency and frame rate at 256×256 and 512×512.The memory comparison is reported at 256×256 resolution.Source: Table 7 · p. 8, §4.5 · pp. 8–9, Table 6 · p. 9

4.6 ABLATION STUDY

• Ablations test data filtering, SDE, and DySample, supporting data-centric learning, SDE detail enrichment, and an additional gain from DySample over bilinear upsampling.Source: §4.6 · p. 9, Table 8 · p. 9

• The current pipeline has not been evaluated in large-scale fully supervised settings.Source: §5 · p. 9
• The current pipeline has not been evaluated in fine-tuned settings.Source: §5 · p. 9
• Further dataset analysis could help optimize the filtering strategy.Source: §5 · p. 9
• Future work could extend the framework to metric depth and normal estimation.Source: §5 · p. 9

• AnyDepth pairs a powerful self-supervised visual backbone with a single-path lightweight decoder for zero-shot monocular depth estimation.Source: §6 · p. 9
• In the authors’ setup, this pairing achieves competitive performance without large-scale, costly training.Source: §6 · p. 9
• The framework aims for practical and academically valuable use through lightweight design and improved data quality, rather than surpassing large-scale state-of-the-art methods.Source: §6 · p. 9

• ChatGPT was used exclusively to improve the clarity and fluency of English writing.Source: §A.1 · p. 15
• ChatGPT was not involved in research ideation, experimental design, data analysis, or interpretation.Source: §A.1 · p. 15
• The authors take full responsibility for all content.Source: §A.1 · p. 15

• The proposed sample-quality criteria require depth values to be evenly distributed across the image and gradients to vary mildly on continuous surfaces but more strongly near object edges.Source: §A.2 · p. 15
• The authors define two metrics from these criteria to measure sample quality for monocular depth estimation.Source: §A.2 · p. 15
• The metrics are intended to reduce low-quality samples, facilitate model training, and reduce dataset size and training cost.Source: §A.2 · p. 15

A.2.1 DEPTH DISTRIBUTION SCORE

• Uneven depth values can bias learning toward a subset of the valid range, causing unstable training and poor generalization.Outdoor datasets commonly show this concentration pattern.Source: §A.2.1 · p. 15
• The chi-square deviation score compares binned depth counts with uniform occupancy; its exponential transformation maps the statistic to [0, 1], with higher scores indicating greater uniformity.The depth range is divided into K equal-width bins, with K = 20 by default.Source: §A.2.1 · p. 15
• The maximum-concentration score tolerates bin occupancy up to 2/K without penalty, then decreases linearly as concentration rises.It penalizes excessive concentration in any single depth interval.Source: §A.2.1 · p. 15
• Range utilization is the fraction of depth bins that contain values, penalizing samples that use only a narrow portion of the available range.It is defined as Srange = K+/K.Source: §A.2.1 · p. 15
• The Depth Distribution Score combines chi-square deviation, maximum concentration, and range utilization as a weighted sum.The reported weights are λ1 = 0.5, λ2 = 0.3, and λ3 = 0.2.Source: §A.2.1 · p. 15

A.2.2 GRADIENT CONTINUITY SCORE

• Pixels in the top 10% of gradient magnitudes are treated as edges, while gradient variation in smooth regions is assessed with the coefficient of variation.The score targets abrupt changes on otherwise smooth surfaces, which can teach incorrect depth changes and affect prediction quality.Source: §A.2.2 · p. 16

A.2.3 TOTAL SCORE

• The Total Score averages gradient continuity and depth distribution to quickly filter samples, rather than precisely assess sample quality.The authors avoid Canny/Sobel maps that introduce artifacts and learned edge methods that are misaligned and time-consuming.Source: §A.2.3 · p. 16

A.3 VISUALIZATION OF LOW-QUALITY SAMPLES

• Figures 8 and 9 show uneven depth distributions and noisy or inconsistent edges in examples from five training datasets, supporting the two quality indicators.The highlighted areas indicate severe gradient noise or inconsistent edges that can negatively affect training stability.Source: §A.3 · p. 16, Figure 8 · p. 17, Figure 9 · p. 18