DAG: Depth-Aware Guidance with Denoising Diffusion Probabilistic Models

1. Introduction

• Diffusion models can produce geometrically implausible images with ambiguous depth and cluttered object layouts.The paper notes that such samples can be visually unappealing and unsuitable for downstream tasks such as robotics and autonomous driving.Source: §1 · pp. 1–2
• DAG trains depth predictors from pretrained diffusion representations to reduce reliance on ground-truth depth annotations and joint model training.The paper describes training from scratch with depth-image pairs as annotation-intensive and computationally demanding.Source: §1 · p. 2
• DAG introduces depth awareness to diffusion sampling through two strategies: depth consistency guidance and depth prior guidance.Consistency guidance uses a better depth prediction as a pseudo-label; prior guidance uses a pretrained diffusion U-Net on the depth domain.Source: §1 · p. 2
• Experiments on indoor and outdoor scene datasets report qualitative and quantitative effectiveness, supported by ablation studies.The paper also introduces a depth-estimation-based metric to capture geometric awareness.Source: §1 · p. 2

Sampling guidance for diffusion models

• Diffusion models generate images by reversing a noise process, while classifier and classifier-free guidance support conditional generation and trade off fidelity and diversity.Prior applications include text-to-image and semantic-map-guided image generation; geometric-awareness guidance is described as unexplored.Source: §2 · pp. 2–3

Internal representations in diffusion models

• Prior studies use intermediate diffusion U-Net representations for semantic segmentation, while this work proposes using them to capture depth information.The cited segmentation approaches use shallow prediction modules or decoder pretraining with limited labels.Source: §2 · p. 3

Monocular depth estimation

• Monocular depth estimation predicts a dense depth map from one image, and recent Transformer-based approaches exploit a global receptive field.The paper notes that recent methods increase model complexity and reliance on large paired supervision.Source: §2 · p. 3

Denoising diffusion probabilistic models

• DDPM adds Gaussian noise to an input image according to a timestep and variance schedule, then generates images through iterative denoising.The forward process can be written in closed form for an arbitrary timestep.Source: §3 · p. 3
• The reverse DDPM process uses a denoising U-Net to predict noise and compute the mean for sampling.Some variants also predict the reverse-process variance.Source: §3 · p. 3

Classifier guidance for diffusion model

• Classifier guidance modifies sampling using the gradient of a classifier that predicts a class from a noised image.The guidance scale controls the fidelity–diversity trade-off, but the cited discussion says dense predictors for geometric awareness had not been explored.Source: §3 · p. 3

4.1. Motivation and Overview

• DAG addresses geometrically implausible generated layouts by training depth predictors on diffusion-model representations and using their depth maps to guide intermediate images.The predictors use a small amount of depth-labeled data.Source: §4.1 · pp. 3–4, Figure 2 · p. 4

4.2. Label-Efficient Training of Depth Predictors

• A pixel-wise shallow MLP estimates depth from intermediate U-Net features, with features from multiple decoder layers concatenated as predictor input.The predictor is trained using frozen diffusion U-Net features and ground-truth depth with an L1 loss.Source: §4.2 · pp. 4–5
• The predictor uses timestep conditioning to estimate depth for intermediate images across sampling steps.The paper reports reasonable label-efficient depth-prediction performance and shows predictions at arbitrary sampling steps.Source: §4.2 · pp. 4–5

4.3. Depth Guided Sampling for Diffusion Model

• Depth consistency guidance treats the stronger multi-feature prediction as a pseudo-label for the weaker prediction, then guides sampling using their consistency loss.A stop-gradient operation prevents the strong prediction from learning from the weak prediction.Source: §4.3 · pp. 5–6
• Depth prior guidance noises the strong-branch depth prediction and uses a pretrained depth-domain diffusion model’s noise-prediction loss gradient to guide image sampling.The method can combine depth prior guidance with depth consistency guidance.Source: §4.3 · p. 6

5.1. Experimental Settings

• Experiments use LSUN-Bedroom and LSUN-Church for indoor and outdoor scenes, train depth predictors with DPT pseudo-labels, and assess depth awareness using DPT-based dFID.For dFID, depth maps are predicted with DPT and compared against depth predictions for dataset images.Source: §5.1 · pp. 6–7

5.2. Experimental Results

• DAG's DCG and DPG improve dFID over unguided ADM on LSUN-Bedroom and LSUN-Church, while qualitative results show better-preserved geometric characteristics.The comparisons use 5,000 generated samples per dataset.Source: Table 1 · p. 7, §5.2 · p. 8, Table 2 · p. 8

5.3. Ablation Study

• Both DCG and DPG achieve their best dFID at guidance scale 40, which becomes the default for later experiments.The scales are ωdc = 40 for DCG and ωdp = 40 for DPG.Source: §5.3 · p. 8, Figure 8 · p. 8
• The 128 × 128 prior diffusion network outperforms the tested 32 × 32 and 64 × 64 alternatives, with 64 × 64 output depth maps interpolated for its input.The prior-network resolution is chosen in light of sampling computation cost.Source: §5.3 · p. 8, Table 3 · p. 8

5.4. Application for Monocular Depth Estimation

• DAG-generated images are more helpful than unguided generated samples for training a depth predictor evaluated on NYU-Depth.Evaluation uses threshold accuracy (δ < 1.25) and absolute relative error (AbsRel).Source: §5.4 · p. 9, Table 4 · p. 9

6. Conclusion

• The proposed method predicts depth maps of reverse-process diffusion images from internal representations in a label-efficient manner.The conclusion also introduces guidance intended to produce images with plausible depth maps.Source: §6 · p. 9
• The guidance scheme steers generated images toward plausible depth maps.Source: §6 · p. 9
• A new metric is presented to represent depth awareness using pretrained depth estimation networks.Source: §6 · p. 9

A.1. Implementation Details

• ADM pretrained weights are used for LSUN-Bedroom and DDIM weights for LSUN-Church, leading to slight variation in feature-extraction block selection.The appendix provides the detailed hyperparameter settings in Table 5.Source: §A.1 · p. 13, Table 5 · p. 13

A.2. PyTorch-like Pseudo Code.

• The guidance method is detailed in Algorithm 1, and the depth estimator is a time-embedded three-layer MLP that predicts depth from concatenated pixel features.The estimator uses SiLU activation.Source: §A.2 · p. 13
• The depth estimator's time-embedding module follows the ADM U-Net implementation.Source: §A.2 · p. 13

A.3. Proposed Metric

• dFID computes Fréchet distance between Inception v3 embeddings of DPT-estimated depth maps.The reference batch uses DPT-Hybrid predictions for images from the dataset.Source: §A.3 · p. 13
• Examples in Table 6 show that good FID can coexist with limited geometric realism, so FID does not adequately capture geometric plausibility.Source: Table 6 · p. 13, §A.3 · p. 14
• DAG improves both geometric realism and dFID, whereas the paper notes that FID does not capture geometric plausibility adequately.Source: §A.3 · p. 14

• Guided images show reduced perspective ambiguity and a more accurate portrayal of scene layout than baseline images.Source: B. More Visualizations · p. 14
• Visualizations of the related depth maps indicate more geometrically plausible outcomes for guided samples than for unguided samples.Source: B. More Visualizations · p. 14
• The comparison presents both generated images and their corresponding depth maps to illustrate the layout differences.Source: B. More Visualizations · p. 14

B.1. Point Cloud Visualization

• Guided point-cloud representations are more consistent and realistic across the same viewing perspective, showing robust geometric consistency.The visualizations use front, left, right, and upward perspectives.Source: §B.1 · p. 14
• The point-cloud comparison derives 3D representations from generated images and their related depth maps, viewing samples from multiple perspectives.Source: §B.1 · p. 14

B.2. Surface Normal Visualization

• Surface-normal predictions from the guided method show more defined layouts and bounds, with lower uncertainty along wall borders.Source: §B.2 · p. 14

C. Limitations and Future Works

• The study focuses on sampling strategies for generating images with higher depth awareness, rather than training diffusion models directly for that goal.Source: C. Limitations and Future Works · p. 14
• Experiments use pretrained diffusion models because training is computationally costly, and data selection was limited.Future work plans to train on indoor and outdoor scene datasets to demonstrate and improve the guidance technique directly.Source: C. Limitations and Future Works · p. 14
• Using the diffusion U-Net in the depth estimator increases guidance cost because gradients require backpropagation through the whole U-Net.Depth-map conditioning with classifier-free guidance is an alternative, but requires many depth-image pairs and substantial joint-training resources.Source: C. Limitations and Future Works · p. 14