Source-linked AI summary
BRIDGE -- Building Reinforcement-Learning Depth-to-Image Data Generation Engine for Monocular Depth Estimation
Dingning Liu, Haoyu Guo, Jingyi Zhou, Tong He
TL;DR
MDE training is constrained by scarce, low-quality ground-truth depth. BRIDGE uses RL-optimized depth-to-image generation and hybrid supervision to build a large RGB-D training set. It reports state-of-the-art benchmark performance with 20M rather than 62M training data, alongside detail capture and robust zero-shot generalization.
Problem
Training robust MDE models remains challenging because precise ground-truth depth annotations are scarce and existing labels lack sufficient detail and diversity.
Source: §1 · p. 2Method
BRIDGE uses RL-optimized depth-to-image generation to create over 20M RGB images from source depth maps, then trains with teacher pseudo-labels and high-precision ground-truth depth.
Source: §1 · pp. 2–3Results
20M vs. 62M training data: BRIDGE reports state-of-the-art performance, consistently outperforming methods like Depth Anything V2 on benchmarks.
Source: §6 · p. 10Takeaways & Limitations
BRIDGE demonstrates fine-grained detail capture, geometric consistency, and robust zero-shot generalization to complex in-the-wild scenes.
Source: §6 · p. 10
Abstract
from arXiv · showhide
Monocular Depth Estimation (MDE) is a foundational task for computer vision. Traditional methods are limited by data scarcity and quality, hindering their robustness. To overcome this, we propose BRIDGE, an RL-optimized depth-to-image (D2I) generation framework that synthesizes over 20M realistic and geometrically accurate RGB images, each intrinsically paired with its ground truth depth, from diverse source depth maps. Then we train our depth estimation model on this dataset, employing a hybrid supervision strategy that integrates teacher pseudo-labels with ground truth depth for comprehensive and robust training. This innovative data generation and training paradigm enables BRIDGE to achieve breakthroughs in scale and domain diversity, consistently outperforming existing state-of-the-art approaches quantitatively and in complex scene detail capture, thereby fostering general and robust depth features. Code and models are available at https://dingning-liu.github.io/bridge.github.io/.
1 INTRODUCTION
MDE training remains constrained by scarce, low-quality, and insufficiently diverse ground-truth depth. BRIDGE addresses this with RL-optimized depth-to-image generation and hybrid supervision.
1 INTRODUCTION
- Prior real-world, synthetic, and mixed-data approaches face sensor sparsity or annotation difficulty, domain gaps or artifacts, and limits in coverage or pseudo-label fidelity.These constraints include difficulty annotating transparent or reflective objects and dependence on large real-world datasets.Source: §1 · p. 2
- BRIDGE generates over 20 million diverse RGB images from synthetic source depth maps and pairs them with high-precision ground-truth depth screened using similarity detection.The screening methods include SSIM and gradient analysis.Source: §1 · p. 2
- BRIDGE outperforms state-of-the-art methods across indoor, outdoor, and synthetic-animation benchmarks, with reported detail capture and robustness in complex scenes.Ablations support the effectiveness of both RL-D2I-generated data and hybrid supervision.Source: §1 · p. 3
- Hybrid supervision combines teacher-generated pseudo-labels with high-precision ground-truth depth to train on generated images.The strategy aims to combine broad coverage with reliable depth supervision.Source: §1 · p. 3
- With approximately 20M data versus Depth Anything V2’s 62M, BRIDGE surpasses state-of-the-art models while improving training efficiency.The authors also report stronger fine-detail capture and robustness.Source: §1 · p. 3
2 RELATED WORKS
Related work covers discriminative and generative MDE, weakly supervised and pseudo-label approaches, and synthetic-data generation. The section positions BRIDGE’s hybrid supervision and RL-optimized depth-to-image generation against limits in label precision, geometric alignment, and domain gap.
2.1 MONOCULAR DEPTH ESTIMATION
- MDE research includes discriminative image-to-depth and generative depth-map methods, but accurate generalization to complex unseen scenarios remains challenging.The section also notes transformer, self-supervised, and boundary-focused approaches.Source: §2.1 · p. 3
2.2 SELF-SUPERVISED AND HYBRID SUPERVISION LEARNING
- Pseudo-labels offer broad coverage but can be noisy around boundaries and fine details; BRIDGE combines them with similarity-guided ground-truth depth for more reliable supervision.The ground truth provides precision alongside pseudo-label breadth.Source: §2.2 · pp. 3–4
2.3 DEPTH-TO-IMAGE GENERATION AND SYNTHETIC DATA
- Conventional depth-to-image generation can produce geometric misalignment and artifacts, while BRIDGE uses RL optimization to target visual realism and geometric consistency.The section contrasts this with synthetic rendering’s domain gap and low generation efficiency.Source: §2.3 · p. 4
3 METHOD
BRIDGE combines reinforcement-learning-optimized depth-to-image generation, teacher pseudo-labels, and similarity-guided use of original ground-truth depth in a multi-stage training pipeline. The MDE model is trained on generated data and then refined using high-precision depth labels in similar regions.
3.1 REINFORCEMENT LEARNING-OPTIMIZED DEPTH-TO-IMAGE GENERATION
- The RL-optimized D2I model generates diverse RGB images from source depth maps while aiming to preserve geometric structure.Reward-gradient-driven direct optimization guides visual quality without complex proxy objectives and is described as improving training memory efficiency.Source: §3.1 · pp. 4–5, Figure 3 · p. 6
3.2 DEPTH PSEUDO-LABEL GENERATION AND MULTI-SOURCE DEPTH FUSION STRATEGY
- A teacher trained on synthetic data supplies pseudo-labels for generated images, while similarity masks identify regions where original ground-truth depth can refine supervision.The mask combines registered and direct SSIM-based similarity regions, then undergoes morphological opening and closing.Source: §3.2 · p. 6
- The hybrid training set combines generated RGB images with teacher-derived depth pseudo-labels and original synthetic datasets’ high-precision depth labels selected by the similarity-guided strategy.The original ground truth is used for masked regions during fine-tuning after pre-training primarily on pseudo-labels.Source: §3.2 · pp. 6–7
3.3 TRAINING MONOCULAR DEPTH ESTIMATION MODEL
- The MDE model uses a pretrained DINOv2-Giant encoder with a DPT head, scale- and shift-invariant and gradient-matching losses, and a separately trained scale head for zero-shot metric depth estimation.The loss terms are weighted 1:4, and the largest 10% of loss regions are ignored during training as potentially noisy pseudo-labels.Source: §3.3 · p. 7
4 IMPLEMENTATION DETAILS
The RL-D2I generator is fine-tuned from Flux.1 Depth[dev] using synthetic depth data and generates four images per source depth map. Image selection applies SSIM, ORB-match, mask-area, erosion, and crop criteria.
4 IMPLEMENTATION DETAILS
- The generator uses Flux.1 Depth[dev], a 1e-5 learning rate, and a loss weighted 0.9 for depth and 0.1 for aesthetic reward.It generates four RGB images per source depth map using different random seeds, with the prompt set to None.Source: §4 · p. 7
- Selected images must pass registered and direct SSIM thresholds above 0.85, with at least 10 ORB matches for registered SSIM.Only masks covering over 50% of pixels are retained; a 3x3 erosion filters small regions before a 518x518 crop is taken around the largest valid region’s bounding box.Source: §4 · p. 7
- The generator produces approximately 20 million visually realistic, geometrically accurate RGB images from source depth maps.The generation loss combines depth loss with aesthetic reward.Source: §4 · p. 7
5 EXPERIMENTS
The experiments evaluate zero-shot depth estimation across indoor, outdoor, and synthetic benchmarks, then examine metric-depth transfer and the contributions of generated data and hybrid supervision.
5.1 EVALUATION DATASETS
- The five evaluation benchmarks cover indoor NYUv2 and ScanNet, indoor/outdoor ETH3D, outdoor KITTI, and synthetic-animation Sintel.The experiments use 1,400 ScanNet test frames and 600 Sintel training samples for evaluation.Source: §5.1 · p. 7
5.3 ZERO-SHOT DEPTH ESTIMATION
- BRIDGE establishes SOTA on several benchmarks, leads on NYUv2, ScanNet, and ETH3D, slightly surpasses Depth Anything V2 on DA2K, but not KITTI.The authors attribute the KITTI result primarily to the dataset's inherent sparsity.Source: Table 1 · p. 8, §5.3 · p. 8
5.2 EVALUATION METRICS
- Zero-shot evaluations optimize scale and shift for affine-invariant inverse depth, reporting AbsRel and δ1 accuracy.δ1 measures the share of pixels whose maximum predicted-to-true inverse-depth ratio is below 1.25.Source: §5.2 · p. 8
5.3 ZERO-SHOT DEPTH ESTIMATION
- Qualitative comparisons report fine-grained depth detail and robustness on challenging objects, including reflective surfaces, distant structures, and transparent umbrellas.The authors also report generalization to in-the-wild data.Source: §5.3 · pp. 8–9, Figure 4 · p. 9
5.4 FINE-TUNED TO METRIC DEPTH ESTIMATION
- For metric-depth transfer, the pretrained encoder is paired with an additional scale head designed for indoor and outdoor domains.The head uses a DINOv2 encoder and an MLP to predict metric scale; the text reports strong performance on NYUv2 and KITTI.Source: §5.4 · p. 9
5.5 ABLATION STUDIES
- With 1M training examples, RL-D2I-generated data improves performance across all evaluated metrics over standard synthetic training data.The comparison uses ViT-L and generated data comprising 0.95M pseudo-labels and 0.05M masked original ground-truth depth.Source: §5.5 · p. 9, Table 2 · p. 10
- On 1M examples, the full hybrid strategy outperforms training generated RGB images solely with teacher pseudo-labels.The strategy combines broad teacher pseudo-label coverage with similarity-guided high-precision ground-truth depth.Source: §5.5 · p. 10, Table 3 · p. 10
6 CONCLUSION
BRIDGE combines RL-optimized depth-to-image generation with hybrid supervision to train MDE using less data than Depth Anything V2. The paper reports strong benchmark results and fine-detail capture with robust zero-shot generalization.
6 CONCLUSION
- BRIDGE uses an RL-optimized depth-to-image engine to generate over 20M RGB-D data and combines teacher pseudo-labels with high-precision ground truth.The conclusion compares its 20M training examples with Depth Anything V2's 62M.Source: §6 · p. 10
- The authors report state-of-the-art performance and consistent outperformance of methods like Depth Anything V2 on benchmarks with significantly less training data.The conclusion gives the comparison as 20M versus 62M.Source: §6 · p. 10
- BRIDGE is reported to capture fine-grained details, maintain geometric consistency, and generalize robustly to complex in-the-wild scenes.The conclusion frames these findings as supporting more efficient and generalizable MDE solutions.Source: §6 · p. 10
A.1 MODEL SELECTION
The model-selection comparison finds that larger DINOv2 encoders learn effectively from BRIDGE data and generalize well to real-world test datasets; the authors select DINOv2-Giant.
A.1 MODEL SELECTION
- DINOv2-Giant and DINOv2-Large show excellent learning ability on synthetic data and generalize remarkably well to real-world test datasets.The comparison uses non-register versions for Giant and Large.Source: §A.1 · p. 16, Table 6 · p. 16
- DINOv2-Small and DINOv2-Base do not achieve satisfactory results in the reported comparison.The authors suggest limited model capacity might make it difficult to capture the generated data's fine-grained information and diversity.Source: §A.1 · p. 16, Table 6 · p. 16
- The authors select DINOv2-Giant for BRIDGE to leverage its reported generalization capabilities and accuracy.Source: §A.1 · p. 16
A.2 ADDITIONAL QUALITATIVE COMPARISON
BRIDGE’s qualitative comparisons show higher-fidelity depth maps across indoor, outdoor, and non-real scenes, with particular gains in distinguishing distant objects from the sky. Its depth estimates also support image synthesis that more closely resembles the source image.
A.2 ADDITIONAL QUALITATIVE COMPARISON
- From indoor-focused Hypersim data, BRIDGE generates outdoor samples largely consistent with their source depth maps, expanding training scene coverage.The training setup generates 20M synthetic samples from seven synthetic datasets.Source: §A.2 · pp. 16–17
- BRIDGE produces higher-fidelity depth maps across indoor, outdoor, and non-real scenes, especially distinguishing distant objects from the sky where Depth Pro and Marigold fail.The comparisons also include open-world and in-the-wild images.Source: §A.2 · p. 17, Figure 5 · p. 18, Figure 6 · p. 19
- Depth-conditioned images synthesized from BRIDGE’s estimates more closely resemble the source image than those based on Depth Anything V2.The comparison uses the same basic pre-trained ControlNet and Stable Diffusion 1.5 setup.Source: §A.2 · p. 17, Figure 7 · p. 20