Source-linked AI summary
Survey on Monocular Metric Depth Estimation
Jiuling Zhang
TL;DR
Recent comprehensive survey coverage of MMDE remains limited. This survey reviews datasets, methods, and open challenges, and synthesizes trade-offs among single-inference, patch-based, and generative approaches.
Problem
Method
The survey synthesizes MMDE datasets, methodological advances, and open challenges.
Source: §1 · p. 2Results
Single-inference methods trade detail for speed, patch-based methods improve accuracy at high cost, and generative models offer superior fidelity but face scalability challenges.
Source: §6.4 · p. 6Takeaways & Limitations
Architectural, training, and inference advances have boosted accuracy and stability, but achieving high precision and strong generalization in dynamic settings remains open.
Source: §6 · p. 4Generalization to unseen domains remains a primary challenge, with blur and degraded high-resolution or cross-domain performance limiting real-world reliability.
Source: §6 · p. 4
Abstract
from arXiv · showhide
Monocular Depth Estimation (MDE) enables spatial understanding, 3D reconstruction, and autonomous navigation, yet deep learning approaches often predict only relative depth without a consistent metric scale. This limitation reduces reliability in applications such as visual SLAM, precise 3D modeling, and view synthesis. Monocular Metric Depth Estimation (MMDE) overcomes this challenge by producing depth maps with absolute scale, ensuring geometric consistency and enabling deployment without additional calibration. This survey reviews the evolution of MMDE, from geometry-based methods to state-of-the-art deep models, with emphasis on the datasets that drive progress. Key benchmarks, including KITTI, NYU-D, ApolloScape, and TartanAir, are examined in terms of modality, scene type, and application domain. Methodological advances are analyzed, covering domain generalization, boundary preservation, and the integration of synthetic and real data. Techniques such as unsupervised and semi-supervised learning, patch-based inference, architectural innovations, and generative modeling are evaluated for their strengths and limitations. By synthesizing current progress, highlighting the importance of high-quality datasets, and identifying open challenges, this survey provides a structured reference for advancing MMDE and supporting its adoption in real-world computer vision systems.
1 Preliminary
The paper situates monocular metric depth estimation as a shift toward absolute-scale predictions for real-world requirements and addresses limited survey coverage with a review of datasets, methods, and open challenges.
1 Preliminary
- Depth estimation supports 3D reconstruction, autonomous navigation, self-driving vehicles, video understanding, and emerging AI-generated content applications.The listed AIGC applications include image synthesis, video generation, and 3D scene reconstruction.Source: §1 · p. 1
- Classical depth methods relied on parallax, stereo vision, or multiple cameras, while MDE predicts depth from a single image.The Monocular Depth Estimation Challenge was hosted at CVPR in 2023 and 2024.Source: §1 · p. 1
- MMDE provides absolute depth values rather than scale-inconsistent maps, addressing practical requirements for accurate, generalizable, detail-preserving predictions.The passage notes advances in zero-shot generalization and reconstruction fidelity through datasets, computing, and architectures.Source: §1 · pp. 1–2
- Recent survey literature on MMDE is limited, and this paper reviews its datasets, methodological advances, open challenges, and future directions.Most comprehensive reviews predate 2020, while newer work often focuses on domain-specific settings or relative depth.Source: §1 · p. 2
2 Depth Estimation
Depth estimation infers dense scene distances from images, an inherently ill-posed task because projection discards geometric information. The section traces the move from hardware-dependent methods to scalable deep-learning approaches.
2 Depth Estimation
- Monocular depth estimation is ill-posed because image projection discards geometric information and provides no parallax or auxiliary cues.Depth maps support scene understanding and applications including autonomous driving, robotics, AR/VR, and computational photography.Source: §2 · p. 2
2.1 Traditional Methods
- Traditional geometric and sensor-based approaches could be accurate in controlled conditions but faced hardware demands and environmental constraints.These limitations motivated a shift toward more scalable, flexible, and robust deep-learning solutions.Source: §2.1 · p. 2, §2.1.3 · p. 3
2.1.1 Sensors
- Structured-light and Time-of-Flight sensors captured spatial information directly but were costly and sensitive to ambient light and surface properties.These constraints limited their use in dynamic or portable applications.Source: §2.1.1 · p. 2
2.1.2 Stereo Vision
- Stereo vision estimates depth from disparities between calibrated cameras, but correspondence failures in low-texture, poorly lit, or dynamic scenes degrade performance.The hardware complexity of stereo rigs also restricted deployment.Source: §2.1.2 · p. 2
2.1.3 Geometric Multi-Frame Methods
- SfM and SLAM infer depth from multi-frame parallax by estimating camera poses and reconstructing point clouds, but remain sensitive to illumination and texture changes.Indirect methods minimize reprojection error; direct methods use photometric consistency (Wofk et al., 2023).Source: §2.1.3 · p. 2
2.2 Deep Learning
- Deep-learning models predict depth from one image and use dataset-learned scene priors and multi-scale features to infer structure in ambiguous regions.CNNs combine fine textures and high-level semantics, supporting depth prediction in sparse-texture, cluttered, and dynamic scenes.Source: §2.2 · p. 3
3 Monocular Depth Estimation
Monocular depth estimation predicts depth from a single RGB image, with neural models learning visual cues and architectural features to recover plausible depth. The section describes early supervised models and later encoder–decoder designs with geometric priors.
3 Monocular Depth Estimation
- MDE predicts scene depth from a single RGB image without multi-view setups or specialized sensors.Neural networks extract visual cues directly, reducing system complexity and cost relative to geometry-based methods.Source: §3 · p. 3
- Early supervised models used labeled data: Eigen et al. introduced multi-scale CNN depth prediction, then combined depth with surface normals and semantic labels (Eigen et al., 2014; Eigen & Fergus, 2015).Source: §3 · p. 3
- Modern encoder–decoder architectures fuse multi-scale features and may incorporate perspective or object-size priors to improve depth plausibility and generalization.Universal feature extraction and domain-invariant learning have also expanded cross-domain applicability.Source: §3 · p. 3
4 Zero-shot Monocular Depth Estimation
Zero-shot MDE seeks transfer across datasets despite scale and camera-intrinsic differences. The section describes relative-depth methods, MiDaS’s cross-domain training, and efforts to combine generalization with metric scale.
4 Zero-shot Monocular Depth Estimation
- Early zero-shot methods reformulated depth as ordinal pixel relationships, using scale-invariant losses to generalize across heterogeneous datasets (Fu et al., 2018).This relative-depth formulation does not directly predict absolute distances.Source: §4 · p. 3
- MiDaS unified multi-dataset training under scale-invariant objectives and showed strong cross-domain performance, but relative depth cannot provide metric scale (Birkl et al., 2023).The passage identifies metric consistency and temporal coherence as important for SLAM, AR, and autonomous driving.Source: §4 · p. 3
- Recent work seeks unified zero-shot models that balance cross-domain generalization with scale fidelity for real-world deployment.Source: §4 · p. 4
5 Monocular Metric Depth Estimation
MMDE produces absolute depth for geometrically consistent perception, while methods have evolved from relying on known camera intrinsics toward estimating or avoiding them. Adaptive depth binning and scene-aware routing support cross-domain metric prediction.
5 Monocular Metric Depth Estimation
- MMDE predicts absolute depth in physical units, supporting geometric stability and temporal coherence for reconstruction, novel-view synthesis, and SLAM.These properties make it more practical for real-world deployment than relative-depth methods, according to the passage.Source: §5 · p. 4
- Early methods assumed known camera intrinsics; newer approaches estimate intrinsics with auxiliary networks or predict depth in spherical representations.Metric3D uses canonical-space mapping with focal-length corrections, while ZeroDepth uses variational inference with camera-specific embeddings.Source: §5 · p. 4
- Adaptive depth bins progressed from dynamic allocation to spatial partitioning and Transformer-based global-local optimization, while NeW CRFs enforce pixel-wise consistency.The cited methods are AdaBins, LocalBins, BinsFormer, and NeW CRFs, respectively.Source: §5 · p. 4
- ZoeDepth extends MiDaS with adaptive metric binning and scene-aware routing, achieving strong zero-shot generalization across indoor and outdoor datasets.The survey describes it as a benchmark for scalable, robust metric depth estimation.Source: §5 · p. 4
6 Challenges and Improvements
The section surveys generalization, detail preservation, sensor assistance, and efficiency trade-offs in MMDE. Architectural, training, and inference changes address these challenges, but high precision and strong generalization in dynamic settings remain open.
6 Challenges and Improvements
- Dynamic settings remain an open challenge for achieving both high precision and strong generalization, despite architectural, training, and inference improvements.The surveyed strategies include multi-scale feature fusion, domain-invariant learning, multi-dataset supervision, and patch-based or adaptive inference.Source: §6 · p. 4
6.1 Generalizability
- Depth Anything uses 62M self-annotated images for cross-domain adaptation, while UniDepth targets camera variation and DepthAnything-AC regularizes consistency; DMD is the only reported generative method producing absolute metric depth.DMD's public release is absent, limiting independent validation of its role in MMDE.Source: §6.1 · pp. 4–5, Table 1 · p. 5
6.2 Blurriness
- Blurred edges and lost fine detail affect boundaries, occlusions, and textures; patch-based fusion, synthetic supervision, and diffusion refinement are surveyed as responses.PatchFusion and BoostingDepth improve sharpness at high computational cost; PatchRefiner targets boundaries and efficiency, while DepthPro trades some distant-region accuracy for detail and speed.Source: §6.2 · p. 5
6.3 Incremental Sensor Assistance
- Sparse LiDAR prompts within a depth decoder guide pre-trained foundation models toward more accurate, high-resolution metric predictions while retaining vision-based scalability.The passage characterizes this as a hybrid strategy using minimal sensor input.Source: §6.3 · p. 5
6.4 Analysis and Comparison
- Single-inference methods are efficient but can lose detail and rely on high-quality annotations; patch methods improve resolution at latency cost, while generative methods face inference overhead.The section notes substantial performance variation and a lack of standardized datasets, training protocols, and model settings for fair comparison.Source: §6.4 · pp. 5–6, Table 2 · p. 6
7 Datasets for MMDE
The survey describes real-world driving datasets as dominant in MMDE, with indoor datasets providing complementary supervision and synthetic datasets adding diversity and precision. The dataset landscape spans different scene types, modalities, and supervision types.
7 Datasets for MMDE
- Driving benchmarks such as KITTI, Waymo Open Dataset, and nuScenes provide RGB, LiDAR, and radar data, while indoor sets such as ScanNet and NYU Depth V2 supply RGB-D imagery and ground truth.The passages connect driving collections to autonomous driving and indoor datasets to robotics and AR.Source: §7 · p. 6
- TartanAir, Hypersim, and vKITTI provide pixel-accurate synthetic depth that complements real-world data and can help mitigate annotation errors and domain gaps.Figure 1 compares MMDE models across scene types and method categories, distinguishing generative relative-depth predictions from other methods' absolute-depth outputs.Source: §7 · p. 6, Figure 1 · p. 7
- Real-world driving datasets dominate, indoor collections complement them, and synthetic datasets add diversity and precision to MMDE training and benchmarking.The survey reports that most reviewed datasets, 32 of 38, provide true metric depth.Source: §7 · p. 6
- Figure 2 presents inference time and memory usage for different model types on a logarithmic scale in seconds.The passage identifies the plotted quantities and scale but does not state a performance winner.Source: Figure 2 · p. 7
8 Summary and Outlook
The survey presents MMDE as shifting toward generative modeling and domain-generalizable frameworks while retaining challenges in detail, geometry, and efficiency. It highlights mixed data and loss strategies, dataset resources, and priorities for future work.
8 Summary and Outlook
- MMDE has moved toward generative modeling and domain-generalizable frameworks, but fine-detail preservation, geometric consistency, and accuracy-efficiency trade-offs remain challenges.The survey links this evolution to broader applications in reconstruction, navigation, and interactive perception.Source: §8 · p. 8
- Edge-aware and gradient-based losses, generative supervision, and real-synthetic data combinations are presented as strategies for structural fidelity, robustness, and addressing annotation scarcity and domain gaps.The cited data examples include Wang et al. (2020) and Roberts et al. (2021).Source: §8 · p. 8
- Marigold and GeoWizard are noted for detail and geometry, while DMD uses field-of-view conditioning and log-scale parameterization; diffusion methods remain computationally demanding.The passage says optimization of multi-step inference is advancing toward practical deployment.Source: §8 · p. 8
- Zero-shot transfer is a research trend; the survey identifies computational efficiency, multi-view geometric consistency, and domain adaptation as priorities.ZoeDepth and UniDepth are cited as approaches demonstrating promising transferability through architectural innovations and large-scale training.Source: §8 · p. 8
- The survey frames loss functions, hybrid data pipelines, and generative modeling as ongoing directions toward more accurate and efficient depth estimation.The stated trajectory is toward universal estimation, but the passage describes this as a direction rather than an achieved outcome.Source: §8 · p. 8
- Table 3 distinguishes metric RGB-depth datasets for supervised MMDE from relative-depth datasets for ordinal prediction, and identifies synthetic and hybrid datasets as resources for domain adaptation.Its columns cover scene type, driving relevance, origin, tasks, modalities, supervision, and dataset features.Source: Table 3 · p. 8