DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimation

1. Introduction

• Regression approaches can overfit, while bin-center discretization is associated with discontinuities and blur in predicted depth.Earlier work commonly framed monocular depth as per-pixel regression or classification.Source: §1 · p. 1
• DiffusionDepth starts from a random depth distribution and iteratively denoises it with monocular visual guidance in latent depth space.A dedicated depth encoder and decoder define the latent space, and visual conditions guide denoising through a hierarchical structure.Source: §1 · pp. 1–2
• The authors describe this as the first work introducing diffusion models into depth estimation.They also report extensive ablations and frame their findings as potentially relevant to other 3D vision tasks.Source: §1 · p. 2
• Self-diffusion adds noise to refined depth latents from the current denoising output instead of directly diffusing sparse ground-truth depth.KITTI Depth has ground-truth values for only 3.75–5% of pixels.Source: §1 · p. 2
• 0.298 NYU-Depth-V2 RMSE and 1.452 KITTI RMSE are reported as exceeding state-of-the-art performance, with affordable inference costs also claimed.Both values are on the official offline test split.Source: §1 · p. 2

2. Related Works

• Monocular depth methods include per-pixel regression, transformer-based feature extraction, uncertainty and planarity constraints, CRFs, and adaptive-bin classification-regression,, [29],,, [47],,,, [33].VA-Depth [35] introduces variational inference for refining depth prediction.Source: §2 · p. 2
• Diffusion models have been explored for segmentation and detection, while DiffusionDepth applies them to conditioned depth refinement rather than a conventional generative head [58] [17] [27] [3] [8] [7].The authors describe this as the first use of diffusion for monocular depth estimation.Source: §2 · pp. 2–3
• Figure 2 depicts visual feature extraction and multiscale aggregation guiding the MCDB as it iteratively refines depth from noise to a prediction.The overview presents the model's main flow from monocular input to refined depth.Source: Figure 2 · p. 3

3.1. Task Reformulation

• Depth estimation is reformulated as visually conditioned denoising that iteratively refines a noisy depth latent toward the final depth map, with deterministic DDIM inference.The model predicts the previous latent from the current latent and visual condition; DDIM uses zero transition variance for deterministic output.Source: §3.1 · p. 3

3.2. Network Architecture

• A visual backbone extracts multiscale features that hierarchical aggregation and a feature-pyramid neck combine into a monocular condition; multiple backbone families are supported.Swin Transformer is the illustrated example, while ResNet, Efficient, and ViT are also reported to achieve competitive performance.Source: §3.2 · p. 3

3.3. Monocular Conditioned Denoising Block

• The MCDB upsamples the lower-resolution visual condition to the depth-latent shape, fuses it with the latent, and processes the result through CNN and attention layers.The block predicts the previous depth latent from the current latent and visual condition.Source: §3.3 · pp. 3–4, Figure 3 · p. 4

3.4. Diffusion-Denosing Process

• Diffusion and denoising operate in encoded depth-latent space, which a decoder maps to pixel depth; pixel and masked latent losses supervise predictions against sparse valid GT.Self-diffusion adds noise to the current refined depth latent instead of sparse encoded GT; the paper reports that either target is feasible with dense indoor GT.Source: §3.4 · p. 4

4.1. Benchmark Comparison with SOTA Methods

• On KITTI’s official offline split, 0.041 absolute related error and 1.452 RMSE beat URCDC-Depth (0.049 rel, 1.528 RMSE); on NYU, 0.85 absolute related error and 0.295 RSME exceed prior SOTA.The KITTI Eigen split also reports 0.050 absolute error and 2.016 RMSE, ahead of URCDC-Depth (RSME 2.032) and VA-Depth (RSME 2.090).Source: Table 1 · p. 5, §4.1 · p. 6, Table 3 · p. 7
• On KITTI’s official online evaluation, DiffusionDepth slightly underperformed VA-Depth and URCDC-Depth, although its qualitative results show sharper object edges and shapes than the compared methods.The paper highlights a signpost that is blurred by BinsFormer and VA-Depth but predicted with a sharp shape by DiffusionDepth.Source: Figure 4 · p. 5, Table 2 · p. 6, §4.1 · p. 6

4.2. Ablation Study

• The denoising visualization shows early steps (t < 10) initializing scene shapes and edges, followed by guided refinement of depth values and distance relations.The paper describes this progression as recognizing scene shape before resolving depth relations between objects.Source: §4.2 · p. 7, Figure 6 · p. 7
• Changing inference steps after training with 1000 diffusion steps causes a severe performance drop, while training directly for the desired inference setting produces only a slight drop.The latter setting is presented as a feasible way to accelerate inference; lower inference steps also reduce GPU memory use and inference time.Source: §4.2 · p. 7, Table 4 · p. 8
• With 20 inference steps, DiffusionDepth reaches 14 FPS on ResNet backbones and 5 FPS on Swin backbones using an RTX 3090 GPU.The paper notes that acceleration could make inference faster.Source: §4.2 · p. 8, Figure 7 · p. 8
• On KITTI, diffusing sparse ground-truth depth does not converge, whereas both diffusion approaches achieve competitive results on NYU with dense ground-truth depth.The ablation motivates self-diffusion, which adds noise to refined depth latent rather than directly to sparse ground-truth depth.Source: §4.2 · p. 8, Table 5 · p. 8
• Both ×4 and ×2 depth encoder-decoder pairs are competitive on KITTI, but the higher-resolution latent space slightly outperforms the alternatives.Table 6 evaluates these encoder-decoder settings on the KITTI official offline split (0–50m).Source: Table 6 · p. 8, §4.2 · p. 8
• The DiffusionDepth head achieves good results with ResNet and Swin backbones and can also use Bins as visual-condition guidance with competitive performance.The reported ablation covers CNN-based Res34 and Res50 as well as the transformer-based Swin backbone.Source: §4.2 · p. 8, Table 6 · p. 8

• DiffusionDepth reformulates monocular depth estimation as a diffusion-denoising approach.Source: §5 · p. 8
• Iterative refinement of the depth latent helps DiffusionDepth generate accurate and highly detailed depth maps.Source: §5 · p. 8
• The reported experiments suggest state-of-the-art performance and verify the feasibility of diffusion-denoising for 3D perception tasks.The paper also provides an ablation study intended to give insights for follow-up work.Source: §5 · p. 8

• The forward process adds scheduled Gaussian noise to refined depth latent x0 over successive steps until xt approaches a normal noise distribution.The prose describes the transitions as a Markov process and gives xT as the noise initialization for reverse sampling.Source: A. Formulation of Diffusion Model · p. 12
• A neural-network posterior approximates the reverse transition, and the model predicts x0 rather than noise to parameterize denoising.The approximate posterior predicts a mean and diagonal covariance matrix.Source: A. Formulation of Diffusion Model · p. 12
• Visual condition c guides denoising, while setting random variance ϑt to 0 removes randomness for deterministic depth prediction.Source: A. Formulation of Diffusion Model · p. 12

• Training uses batch size 16 for 30 epochs on eight NVIDIA A100 40G GPUs, AdamW, learning-rate warm-up and cosine decay, plus auxiliary L1 and L2 pixel-wise depth losses during the first half of training.The initial learning rate decays from 1e−4 to 1e−8.Source: B. Extended Implementation details · pp. 12–13
• Augmentation uses random crops of 706 × 352 for KITTI and 512 × 340 for NYU-Depth-V2, alongside color jitter, scale changes, and horizontal flips.The implementation description also specifies random rotation by a small angle.Source: B. Extended Implementation details · p. 13
• The visual-condition setup supports ResNet and Swin backbones, using HAHI for cross-scale feature enhancement and a feature pyramid neck to aggregate monocular visual conditions.The specified channel dimensions are [64, 128, 256, 512] for ResNet and [192, 384, 768, 1536] for transformer backbones.Source: B. Extended Implementation details · p. 13
• The diffusion head uses 1000 training steps and 20 inference steps, with a learning rate 10 times the backbone rate and encoded depth-latent dimension 16.The decoder’s maximum depth value is 1e6 for all experiments.Source: B. Extended Implementation details · p. 13

B.1. Evaluation Metrics

• Evaluation includes SILog, Sq Rel, Abs Rel, RMS, and RMS log error measures.The passages name these metrics but do not provide enough intact formula text to explain their calculations.Source: §B.1 · p. 13
• Threshold accuracy (δk) measures the percentage of predictions satisfying a stated threshold condition.The condition is incomplete in the supplied passage.Source: §B.1 · p. 13

C. Extensive Qualitative Comparison on NYU-Depth-V2 Dataset.

• DiffusionDepth reaches state-of-the-art performance on NYU-Depth-V2; the extended visual comparison uses BinsFormer and DepthFormer.URCDC-depth and VA-Depth are excluded because they have not been open-sourced.Source: C. Extensive Qualitative Comparison on… · p. 13

D. Illustration of the Denoising Process.

• On KITTI, unpredictable upper regions correspond to road ends or sky with very large depths that the model does not predict.The denoising process initializes the predictable area's upper and lower bounds at t=2 and t=5.Source: D. Illustration of the Denoising Process · p. 13, Figure 9 · p. 14, Figure 10 · p. 15
• Later KITTI steps refine the map's shapes and structure, then adjust distance correlations and real depth values as predicted objects become sharper.The described refinement occurs at t=8 and t=10 before gradual adjustment.Source: D. Illustration of the Denoising Process · p. 14
• On NYU-Depth-V2, dense ground truth means the visualizations show no unpredictable areas like those seen on KITTI.The subsequent steps refine the depth map's shapes and basic structure.Source: D. Illustration of the Denoising Process · p. 14, Figure 11 · p. 16
• On NYU-Depth-V2, denoising adjusts distance correlations and real depth values, handling edges' extreme depth variances better than previous baselines.Predicted object shapes also become sharper and clearer through subsequent steps.Source: D. Illustration of the Denoising Process · p. 14