PaperlayerView exact revision on arXiv

Source-linked AI summary

Monocular Depth Estimation from a Single Image: Progress and Opportunities

Muxin Liu, Xiaoyang Lyu, Yang-Tian Sun, Yi-Hua Huang, Ziyi Yang, Peng Dai, Xiaojuan Qi

arXiv:2609.01172v1cs.CV

TL;DR

Monocular depth estimation seeks flexible, reliable depth from a single image despite challenges such as varying camera intrinsics. This survey organizes prior and foundation-model methods, introduces a unified benchmark, and finds that scaled heterogeneous data and pretrained encoders support robust, detail-preserving estimation, while generative models complement discriminative models with fine-grained geometry.

  • Problem

    Monocular depth estimation must infer reliable spatial information from one image while addressing challenges such as camera-intrinsic variation and the hardware costs of multi-image or sensor-based methods.

  • Method

    The survey reviews pre-foundation methods, categorizes discriminative and generative depth foundation models, and evaluates them across 10 datasets using a unified protocol.

  • Results

    Foundation models combine heterogeneous data and pretrained encoders for robust, detail-preserving depth, while discriminative models provide efficient estimation and diffusion models excel at fine-grained geometry.

  • Takeaways & Limitations

    The field is progressing toward unified depth foundation models spanning relative and metric estimation across images and temporally consistent video.

  • Takeaways & Limitations

    Most foundation models lack built-in temporal or multi-view consistency, causing flickering and inconsistency in video or multi-camera setups.

Abstract

from arXiv · show

Monocular depth estimation has long stood as a fundamental challenge in computer vision, enabling a wide range of applications including 3D reconstruction, robotics, autonomous driving, and augmented reality. This survey traces the field's evolution from early learning-based methods to the emergence of transformative foundation models. We begin by framing the problem, distinguishing between relative and metric depth estimation, and highlighting the key challenges that have shaped a decade of research. We then present common problem formulations and introduce the most widely used datasets, covering indoor, outdoor, and synthetic data. Following this, we review major advances prior to the foundation model era, distilling core insights from influential methods that contributed to improvements in accuracy, efficiency, and robustness. The survey then turns to the recent surge of foundation-model-based approaches, categorizing them into discriminative and generative paradigms and emphasizing the critical roles of large-scale pretraining (e.g., DINOv3) and synthetic data. We compare representative models using both quantitative benchmarks and qualitative examples, and discuss natural extensions to video-based depth estimation. Further, to illustrate real-world impact, we highlight the integration of depth estimation into applications such as visual SLAM, content generation, and robot perception. Finally, we outline open challenges and promising research directions as the field advances further into the era of foundation models.

1 Introduction and Background

Monocular depth estimation addresses the hardware cost and rigidity of multi-image or specialized-sensor methods while enabling depth from a single RGB image. The field has evolved from early learning-based models toward foundation models designed for broad zero-shot generalization, motivated by severe cross-domain failures and dataset-specific biases.

  • Motivation: Monocular depth estimation provides flexible depth perception for robotics, autonomous driving, 3D modeling, virtual and augmented reality, and computational photography without stereo cameras or Kinect.Traditional alternatives require multiple images or specialized sensors, increasing hardware complexity and costs.
  • Historical evolution: The modern MDE era began with data-driven Markov Random Fields on hand-crafted features and Eigen et al.’s multi-scale CNN for dense depth regression from a single RGB image.Deep convolutional networks marked the breakthrough from earlier learning-based approaches.
  • Challenges: Models often generalized poorly across indoor and outdoor domains because scene scale, camera intrinsics, and dataset-specific biases differed between datasets such as NYU Depth v2 and KITTI.High accuracy on individual datasets could reflect limited depth ranges or other dataset-specific characteristics.
  • Foundation-model transition: Depth foundation models train on vast, diverse datasets, including unlabeled or synthetic data, to achieve strong zero-shot performance on varied in-the-wild images.Their goal is one broadly applicable model rather than specialized models for each dataset; early efforts combined multiple datasets and designed losses or used vision transformers for global context.
  • Survey scope: The survey formalizes MDE and datasets, reviews pre-foundation supervised and self-supervised methods, and then centers on the emergence of depth foundation models.The stated organization covers problem setup in Section 2, pre-foundation developments in Section 3, and foundation models in Section 4.

2 Problem Setup and Datasets

Monocular depth estimation predicts a dense depth map from one RGB image, with outputs formulated as either metric depth in physical units or relative depth up to scale and possibly shift. Its progress depends on diverse datasets spanning synthetic, indoor, outdoor, and internet imagery, while camera-intrinsic variation and dataset-specific limitations remain central challenges.

  • Depth Estimation Paradigms: Metric depth predicts real-world distances, whereas relative depth preserves scene ordering up to an unknown scale and, for affine-invariant estimation, an additive shift [22] [26] [33].If D is valid, αD+β is also valid for α > 0 in affine-invariant formulations.
  • Key Challenges: Camera-intrinsic variation, especially focal-length changes, directly affects absolute-depth mapping, challenging methods trained with fixed or narrow intrinsic ranges [3, 11, 24, 39, 49–51].Recent approaches condition models on camera intrinsics to generalize across unknown camera parameters.
  • Datasets: Datasets span synthetic, indoor, outdoor/driving, and depth-in-the-wild settings, trading controlled complete annotations against realism, diversity, sparsity, noise, or sensor-dependent quality [52–67].Synthetic datasets provide perfectly labeled depth under controlled conditions but can differ from real-world data; LiDAR annotations are often sparse and noisy, particularly for distant or dynamic objects.
  • Datasets and Training: Modern frameworks use hybrid or multi-stage training, pretraining on extensive multi-domain data before fine-tuning with synthetic data, teacher-generated pseudo-labels, or domain-specific datasets [22].This strategy aims to learn generalizable representations that improve robustness and adaptability across real-world scenarios.

3 Depth Estimation Prior to Foundation Models

Before foundation models, monocular depth estimation progressed from supervised CNNs and diverse depth objectives toward architectures that better captured global context and self-supervised methods using view consistency. Key advances addressed dense prediction, scale ambiguity, structural detail, and dynamic scenes.

  • 3.2 Self-Supervised Learning: Self-supervised learning removed reliance on ground-truth depth through view-consistency losses, and Monodepth2 used auto-masking, occlusion handling, and ImageNet-pretrained ResNet features to match or outperform some supervised models.The paradigm was subsequently extended through architectural and uncertainty-modeling innovations.
  • 3.1 Supervised Learning: Supervised learning dominated early monocular depth estimation, mapping one RGB image to depth-related 3D representations with pixel-wise, geometric, and structural losses.Predictions included depth maps, point clouds, or discrete depth bins, guided by losses ranging from pixel-wise regression to geometric and structural consistency.
  • 3.1.1 Backbone Architectures: DPT combined a pretrained Vision Transformer encoder with a multi-scale convolutional decoder, reducing relative error by up to 28% versus CNN baselines.The design addressed CNN limitations in global context while recovering fine-grained details through multi-scale features.
  • 3.1.2 Training Objectives: Regression losses often blurred depth discontinuities, motivating inverse Huber and hierarchical losses that improved convergence, boundary accuracy, and multi-scale consistency.The inverse Huber loss behaves like L1 for small errors and L2 for larger discrepancies; CLIFFNet extended supervision through intermediate features.
  • 3.1.2 Training Objectives: Scale-invariant, affine-invariant, classification, ordinal, and geometric-consistency objectives addressed scale ambiguity, relative depth structure, image-specific depth ranges, and depth–surface-normal relationships.Ordinal regression preserved depth ordering, while local or image-specific bins captured spatially varying depth distributions; joint depth–normal estimation enforced geometric constraints.

4 Depth Foundation Model · 4.1 Dataset Collection and Unification · 4.2 Discriminative Methods

Depth Foundation Models combine large-scale pretrained models with extensive RGB-D data to achieve zero-shot robustness and detailed depth prediction across diverse cameras and scenes. The survey organizes these advances around dataset unification and discriminative methods spanning relative and metric depth estimation.

  • 4 Depth Foundation Model: Depth Foundation Models leverage pretrained models such as DINOv2, DINOv3, and Stable Diffusion, fine-tuned on extensive RGB-D datasets for in-the-wild depth estimation.Their goals include zero-shot generalization across camera types and scenes without fine-tuning, while preserving fine-grained geometric detail.
  • 4.1 Dataset Collection and Unification: Dataset unification addresses heterogeneous annotations by using affine-invariant losses, while photorealistic synthetic data and depth-aware diffusion models expand coverage where real labels are noisy, missing, or impractical.These approaches accommodate absolute depth, unknown-scale depth, disparity, and challenging conditions such as underwater, adverse-weather, reflective, or translucent scenes.
  • 4.2 Discriminative Methods: Discriminative methods directly map images to depth, evolving from CNN-based models such as MegaDepth, MiDaS, and LRSI to ViT-based architectures.MiDaS v3.1 [33] introduced DPT-based depth foundation models, whose global-context and long-range-relation modeling helped establish ViTs as the prevailing backbone.
  • 4.2.1 Relative Depth Estimation: Relative-depth models increasingly scale data and representations: MiDaS trained on 1.1 million diverse images, while Depth Anything used 1.5 million labeled and 62 million unlabeled images with DINOv2 features.MoGe instead predicts affine-invariant 3D pointmaps, avoiding camera-intrinsic requirements and supporting high-quality point reconstruction.
  • 4.2.2 Metric Depth Estimation: Metric depth is harder because unknown camera intrinsics and domain shifts create ambiguity across indoor, outdoor, object-centric, real, and synthetic scenes.Different camera models, fields of view, positions, and distortion can produce visually similar images with different metric depths or different appearances for identical geometry.
  • 4.2.2 Metric Depth Estimation: Metric-depth methods model camera effects through canonical-camera normalization in Metric3D, self-prompted intrinsic inference in UniDepth and UniDepthV2, or focal-length prediction in DepthPro.These designs aim to disentangle camera-specific effects from depth and improve zero-shot generalization across imaging conditions.
  • 4.2 Discriminative Methods: Across discriminative metric-depth strategies, camera transformations, intrinsic estimation, focal-length prediction, relative-to-metric transfer, and targeted synthetic data collectively support robust estimation across diverse imaging conditions.The approaches address both global scale ambiguity and camera-dependent geometric distortions rather than relying on a single architectural solution.
  • 4.2.2 Metric Depth Estimation: Other approaches transfer relative depth to metric prediction: ZoeDepth [22] uses adaptive metric bins, MoGe-2 separates relative depth from global scale, and FoundationGeo [26] corrects spatial scale and ray-direction errors.MoGe-2 refines noisy real-world data with synthetic labels, whereas FoundationGeo [26] targets focal-length coverage mismatch with multifocal synthetic data and two-stage training.

4.3 Generative Methods

Generative depth-estimation methods increasingly use pretrained diffusion models to capture complex distributions, preserve detail, and improve prediction realism and reliability. Their multi-step sampling remains computationally expensive, motivating deterministic one-step and hybrid approaches that combine generative detail with discriminative efficiency.

  • Diffusion Models: Diffusion models capture complex data distributions and fine details, with Marigold representing a depth estimator fine-tuned from Stable Diffusion.Multiple works leverage pretrained diffusion models to improve the realism and reliability of depth predictions.
  • Efficiency and Detail Enhancement: Marigold [27] requires approximately 24 seconds for a 578×578 input because multi-step denoising and test-time ensembling address sampling uncertainty.GenPercept and Diffusion-E2E therefore investigate deterministic one-step alternatives and the role of the denoising scheduler.
  • Detail Preservation: Lotus uses a task switcher to alternate depth/normal prediction with input-image reconstruction, helping preserve structural and textural cues that may degrade through catastrophic forgetting.GeoWizard further exploits diffusion-model flexibility for joint depth and surface-normal estimation.
  • Beyond Diffusion: VAR-inspired methods reformulate depth estimation as next-scale prediction across multiple resolutions, but their robustness and performance ceilings remain under early investigation [107, 172].Hybrid methods such as BetterDepth instead use diffusion as a plug-and-play refiner for coarse outputs from discriminative networks such as DPT.
  • Efficiency and Hybridization: GenPercept and Diffusion-E2E [34, 35] show that single-step diffusion with discriminative-style losses can improve performance while bridging generative flexibility and discriminative efficiency.This reflects a broader shift toward using diffusion models as deterministic predictors or feature extractors rather than generative samplers.

4.4 Benchmark Performance

The section establishes a unified zero-shot evaluation framework spanning ten diverse datasets, consistent affine-invariant alignment, and standardized comparisons across metric, relative, and disparity depth methods.

  • Evaluation protocol: Evaluation uses the MoGe-2 benchmark, covering 10 datasets across indoor, street-view, object-scan, and synthetic-animation domains while excluding ambiguous reflective regions.The protocol standardizes assessment across multiple domains and applies consistent depth-map evaluation.
  • Method categories: The comparison categorizes methods into metric, relative, and disparity estimation, including representative models such as UniDepth, MoGe, MiDaS v3.1 [33], and DepthAnythingV2.The listed methods span the three estimation types evaluated under the unified framework.
  • Evaluation protocol: All methods undergo uniform affine-invariant scale-and-shift alignment via least-squares fitting before standard depth metrics are computed, including metric models despite their absolute-depth objective.Relative and disparity models receive the same alignment because they lack absolute scale or require comparable calibration.
  • Evaluation protocol: Methods are evaluated using their official preprocessing pipelines and best-performing open-source implementations, with commonly used metrics including AbsRel and δ1 Accuracy.Table 2 presents the unified evaluation, primarily for zero-shot comparison; bold values denote best results, underlined values second-best, and gray values excluded from ranking.

4.5 Extensions to Videos

Video depth estimation must maintain temporal consistency under scarce annotations and long-sequence efficiency constraints, motivating generative diffusion and discriminative streamable models with temporal mechanisms. Recent methods support coherent, high-fidelity, and streaming depth prediction without pose or flow supervision, while addressing scale–shift drift and temporal instability.

  • Challenges: Temporal consistency, dense video annotations, and efficient long-sequence processing are the central challenges distinguishing video from single-image depth estimation.Naïve frame-by-frame models can flicker and produce inconsistent scale across frames because they lack cross-frame awareness.
  • Generative Video Diffusion Model: Generative models DepthCrafter, ChronoDepth, and Depth Any Video use pretrained video diffusion priors to jointly model spatial detail and temporal coherence.They avoid pose or flow supervision, generalize in zero-shot settings, and produce high-fidelity depth maps with strong temporal stability; DepthCrafter processes up to 110 frames, while Depth Any Video uses 40K+ synthetic clips and supports up to 150 frames.
  • Discriminative (Streamable) Models: Discriminative models Video Depth Anything and FlashDepth extend the Depth Anything V2 backbone with temporal modeling for stable, streamable predictions.Video Depth Anything combines a lightweight spatio-temporal head with temporal gradient consistency loss, while video models introduce temporal layers or recurrent latent states to enforce cross-frame consistency.
  • Discriminative (Streamable) Models: DyFN stabilizes streaming depth by attributing inconsistency to scale–shift drift from fluctuating latent feature statistics and introducing lightweight causal recurrent normalization.This approach targets temporal feature-statistics stability rather than inaccurate per-frame geometry.

4.6 Extensions to Any Camera

Extending depth foundation models beyond rectified perspective cameras requires handling arbitrary projections and large fields of view, where distortion, missing context, limited supervision, and representation variability undermine direct transfer.

  • 4.6 Extensions to Any Camera: Any-camera depth estimation is challenging because perspective-centric models fail under spherical distortions and lose global context in fisheye and 360° imagery.Real-world systems often use arbitrary projection models and large fields of view, unlike the rectified perspective assumptions used for most depth foundation models.
  • 4.6 Extensions to Any Camera: Changing pixel-to-ray mappings across camera models make learned pinhole priors brittle under wide-field-of-view distortions.
  • 4.6 Extensions to Any Camera: Cropping or undistortion trades long-range context for spatially varying warps, while scarce metric supervision and variable FoV resolutions complicate training and deployment.Paired metric depth is much less available for fisheye and panoramic imagery than for perspective RGBD or LiDAR data; spherical or ERP representations also produce uneven sampling densities and resolution mismatches.

4.7 Summary

Recent depth foundation models achieve robust, detail-preserving zero-shot estimation by scaling heterogeneous data and using strong pretrained encoders. Discriminative models offer efficient accuracy, while diffusion-based generative models provide fine-grained geometry and are increasingly distilled for efficiency.

  • 4.7 Summary: Scaling heterogeneous LiDAR, RGB-D, synthetic, and pseudo-labeled data with strong pretrained encoders enables zero-shot robustness and detail-preserving depth.This trend underpins recent depth foundation models.
  • 4.7 Summary: Discriminative ViT-based architectures efficiently estimate relative, metric, and point-based depth using affine-invariant objectives, camera-aware designs, and lightweight global scale heads.These design choices support both efficiency and accuracy.
  • 4.7 Summary: Generative diffusion-based approaches inject strong image priors for fine-grained geometry and are increasingly distilled into one-step or hybrid refiners.Distillation narrows their efficiency gap with discriminative approaches.

5 Downstream Applications of Depth Estimation

Depth estimation supports sensor-free 3D mapping, neural reconstruction, creative image manipulation, robotics, and autonomous driving. Across these applications, depth acts as geometric supervision that improves spatial understanding, rendering, navigation, and generation.

  • SLAM and 3D/4D Scene Reconstruction: Depth priors regularize neural scene representations, improving reconstruction and robustness for NeRF-like models, especially under sparse or unconstrained observations.Methods use depth order, metric cues, or explicit depth supervision to constrain geometry and align neural or Gaussian primitives.
  • SLAM and 3D/4D Scene Reconstruction: Monocular depth enables scalable, accurate, sensor-free 3D mapping across SLAM, SfM, neural reconstruction, and dynamic 4D scene understanding.It provides geometric cues for correspondence triangulation, depth-sensor replacement, and reconstruction from in-the-wild monocular videos.
  • Image Editing and Creative Applications: Depth-guided warping, inpainting, and generation enable viewpoint synthesis, stereoscopic imagery, 3D video, spatial editing, relighting, and portrait effects from 2D inputs.These applications require sharp boundaries and accurate ordinal relations to reduce distortions, disocclusion artifacts, and edge errors.
  • Robotic Perception and Navigation: Monocular depth provides robots with a lightweight, low-cost alternative to LiDAR and stereo cameras for obstacle perception, free-space segmentation, trajectory planning, and grasping.Learned models support onboard navigation and improve real-time grasp success and 6-DoF pose estimation without additional sensors.
  • Autonomous Driving: In autonomous driving, depth maps can form pseudo-LiDAR point clouds that improve vehicle detection, or jointly support object identity and distance estimation.These approaches integrate monocular depth into perception pipelines as a cost-effective alternative to LiDAR.

6 Future Research Directions

Future work targets universal metric depth, reliable high-fidelity predictions, robustness in difficult regions and domains, temporal and multi-view consistency, stronger evaluation, and tighter integration with broader 3D understanding and generation.

  • Closing the Gap between Relative and Metric Depth: Universal metric depth remains open, requiring camera-adaptive models that infer intrinsic parameters beyond focal length despite progress from Metric3D, UniDepth, and DepthPro.
  • High-Fidelity Depth and Uncertainty: Depth systems should quantify uncertainty, preserve thin structures and boundaries, and scale toward 4K or 8K through multi-scale or implicit representations.Per-pixel confidence or depth distributions could improve reliability, while wires, branches, and other fine structures remain difficult; diffusion samples still require actionable statistics such as mean and variance.
  • Challenging Regions: Sky, Boundaries, and Non-Lambertian Surfaces: Robust models must handle sky, horizons, discontinuities, thin structures, and transparent or reflective surfaces using open-set predictions, calibrated uncertainty, and boundary-aware constraints.These regions provide weak geometric cues, suffer from over-smoothing or non-Lambertian effects, and often lack RGB-D or LiDAR supervision.
  • Temporal and Multi-View Consistency: Future systems should enforce temporal and multi-view consistency, adapt to specialized domains, and incorporate multimodal information with minimal supervision.Video-trained temporal losses and recurrent or transformer propagation could reduce flickering, while physics-guided adaptation is suggested for domains such as underwater imagery.
  • Benchmarking and Evaluation Metrics: Evaluation should expand beyond RMSE toward cross-dataset, continual, perceptual, structural, and multi-view geometric consistency measures that reward globally coherent 3D predictions.
  • Integration with 3D Understanding: Depth research should integrate recognition, segmentation, detection, and generative modeling, enabling jointly grounded 3D systems and depth-guided generation with NeRFs or 3D GANs.Generative models can synthesize rare training cases, while depth can initialize NeRFs and accelerate convergence; future systems may jointly perceive and imagine depth.

7 CONCLUSION

Monocular depth estimation has progressed from early learning-based methods to foundation models, improving accuracy, robustness, and generalization while narrowing the gap between relative and metric depth. These advances enable broad applications, but temporal consistency, domain adaptation, uncertainty estimation, and scene understanding remain open challenges.

  • Foundation-model-era monocular depth estimation improves accuracy, robustness, and generalization while bridging relative and metric depth through vision transformers, synthetic and pseudo-labeled data, and pretrained vision foundation models.
  • Recent advances support applications in 3D reconstruction, visual SLAM, image editing, robotics, and multimedia content creation.
  • Key open challenges include temporal and multi-view consistency, specialized domain adaptation, predictive uncertainty, and comprehensive scene understanding.
  • The benchmark uses publicly released evaluation datasets, and the authors provide code and scripts through CVMI-Lab/Depth-Survey.

Graphical abstract

The section presents a graphical abstract of the survey.

  • The graphical abstract provides a visual overview of the survey.
Loading 2609.01172v1…