Source-linked AI summary
Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimation
Shanshan Zhao, Huan Fu, Mingming Gong, Dacheng Tao
TL;DR
Monocular depth estimation needs either costly depth annotations or less reliable unsupervised cues, while synthetic-data adaptation can overlook real-domain geometry. GASDA jointly uses synthetic depth labels and real stereo epipolar geometry through symmetric, end-to-end translation and depth estimation, with experiments showing effectiveness and comparable state-of-the-art performance.
Problem
Synthetic depth labels are accessible but domain-adaptation methods overlook the target domain's geometric structure, while real depth annotations are difficult to obtain.
Method
GASDA jointly learns symmetric synthetic-to-realistic and realistic-to-synthetic translation and monocular depth estimation using synthetic ground-truth labels and real stereo epipolar geometry.
Results
Experiments on KITTI and Make3D demonstrate GASDA's effectiveness, with the paper reporting comparable performance against the state of the art.
Takeaways & Limitations
Combining synthetic supervision, real-domain geometry, bidirectional style transfer, and end-to-end training provides the paper's supported approach to unsupervised monocular depth adaptation.
Abstract
from arXiv · showhide
Supervised depth estimation has achieved high accuracy due to the advanced deep network architectures. Since the groundtruth depth labels are hard to obtain, recent methods try to learn depth estimation networks in an unsupervised way by exploring unsupervised cues, which are effective but less reliable than true labels. An emerging way to resolve this dilemma is to transfer knowledge from synthetic images with ground truth depth via domain adaptation techniques. However, these approaches overlook specific geometric structure of the natural images in the target domain (i.e., real data), which is important for high-performing depth prediction. Motivated by the observation, we propose a geometry-aware symmetric domain adaptation framework (GASDA) to explore the labels in the synthetic data and epipolar geometry in the real data jointly. Moreover, by training two image style translators and depth estimators symmetrically in an end-to-end network, our model achieves better image style transfer and generates high-quality depth maps. The experimental results demonstrate the effectiveness of our proposed method and comparable performance against the state-of-the-art. Code will be publicly available at: https://github.com/sshan-zhao/GASDA.
1. Introduction
The paper addresses the cost and unreliability of obtaining real depth labels by combining synthetic ground-truth depth with geometric cues from real stereo images. GASDA uses symmetric, end-to-end style translation and depth estimation to improve domain adaptation for monocular depth estimation.
- Motivation: Supervised methods require expensive ground-truth depth, while unsupervised methods are vulnerable to illumination changes, occlusion, and blurring.Synthetic data offers easier access to depth maps, but models trained on it often suffer from synthetic-to-real domain shift.
- Motivation: Unpaired image translation can introduce structural distortions that degrade subsequent depth prediction, whereas real stereo pairs provide additional geometric constraints.The paper motivates jointly exploiting synthetic labels and real-image geometry to address these limitations.
- Method: GASDA combines synthetic-to-realistic and realistic-to-synthetic translation with geometry consistency based on the epipolar geometry of real stereo images.Its learning uses ground-truth synthetic labels, real-domain epipolar geometry, and consistency between depth predictions.
- Method: The framework trains style translation and depth prediction end to end so translated images retain geometric and semantic content while supporting depth estimation.The model contains symmetric translation directions and is designed to improve both image translation and depth prediction.
- Evaluation: GASDA is evaluated on KITTI and its generalization performance is demonstrated on Make3D.The supplied introduction reports effectiveness on KITTI and generalization on Make3D without providing numerical results.
2. Related Work
Prior work advances monocular depth estimation through deep networks, stereo-based unsupervised learning, and domain adaptation, but domain-adaptive methods overlook target-domain geometric structure. GASDA addresses this gap with bidirectional translation that exploits real stereo epipolar geometry alongside synthetic depth supervision.
- Monocular Depth Estimation: Deep convolutional networks improved monocular depth estimation, but supervised training requires substantial annotated data.Earlier graphical-model and non-parametric approaches also faced performance and efficiency limitations from hand-crafted features and slow inference.
- Unsupervised Depth Estimation: Unsupervised monocular depth methods use stereo correspondence and epipolar geometry, with later work adding left-right consistency, semi-supervision, and temporal photometric constraints.These methods recast depth estimation as image reconstruction without requiring paired image-depth labels.
- Domain Adaptation: Domain adaptation reduces dataset bias by learning domain-invariant representations or mappings between source and target domains.Mappings may operate at the feature or pixel level.
- Depth Domain Adaptation: Earlier depth-adaptation methods used stylization, content-congruent regularization, or jointly optimized translation and depth estimation, but overlooked target-domain geometric structure.The paper identifies this omission as important because geometric structure has been shown to matter for depth estimation.
- Depth Domain Adaptation: Figure 2 contrasts prior approaches with GASDA, which uses real stereo data and both synthetic-to-real and real-to-synthetic translation.The diagram labels synthetic data, real data, extracted features, generated data, estimated depth, adversarial loss, and monocular depth estimation.
3. Method
GASDA jointly adapts synthetic and real domains through bidirectional style translation, supervised depth learning, and real-image epipolar geometry. Two complementary depth estimators are trained end-to-end and averaged during inference.
- Depth estimation: Two complementary estimators, Fs and Ft, learn from original synthetic images and translated realistic images using synthetic ground-truth depth.Fs trains directly on Xs, while Ft trains on Gs2t(Xs); both use supervised ℓ1 depth losses.
- Geometry consistency: Real stereo pairs provide epipolar geometry that encourages Fs and Ft to capture target-domain structure during training.The geometry consistency mechanism uses inverse warping based on estimated depth, camera baseline, and focal length.
- Style translation: GASDA uses bidirectional generators Gs2t and Gt2s with discriminators Dt and Ds to translate between synthetic and real image domains.Cycle consistency and identity mapping losses help preserve meaningful image content during translation.
- Additional losses: An edge-aware smoothness loss is applied to real data and translated real data, while an ℓ1 depth consistency loss aligns predictions from the two estimators.The consistency loss addresses regional disagreement between Ft(xt) and Fs(Gt2s(xt)).
- Optimization and inference: The complete objective jointly optimizes translators, discriminators, and depth estimators in an end-to-end network.At inference, real images follow direct and translated paths, and the final depth prediction averages the two outputs.
4. Experiments
Experiments evaluate GASDA on KITTI and Make3D, including benchmark comparisons, qualitative results, and ablations of domain adaptation, geometry consistency, symmetry, and depth consistency. The results report strong KITTI performance, competitive KITTI Stereo 2015 and Make3D results, and benefits from the proposed components.
- KITTI Dataset: GASDA obtains significant improvements across KITTI benchmark metrics against prior state-of-the-art methods and synthetic-only and real-only baselines.The evaluation uses the Eigen split with KITTI and vKITTI training data and compares domain adaptation, supervised, semi-supervised, and unsupervised methods.
- KITTI Dataset: On KITTI Stereo 2015, GASDA yields lower errors than state-of-the-art approaches and competitive accuracy compared with [2].The benchmark uses 200 training images, and S* denotes synthetic data captured from GTA5 that is more similar to real data than vKITTI.
- KITTI Dataset: GASDA cannot outperform on the Eigen split, partly because uses GTA5 images with smaller domain shift and roughly three times more training data.The comparison is more favorable on the official KITTI Stereo 2015 dataset and Make3D.
- KITTI Dataset: Qualitative KITTI outputs preserve more details, recover depth for small objects such as distant cars and rails, and produce clear boundaries.The comparison includes methods by Eigen et al. and Zheng et al.; top regions without ground-truth depth are masked for visualization.
- Make3D Dataset: Without training or fine-tuning on Make3D, GASDA performs well despite the large domain shift from KITTI and remains comparable with models trained on Make3D.The Make3D evaluation includes quantitative and qualitative results from Table 4 and Figure 8.
- Ablation Study: Ablations show that geometry consistency substantially improves standard domain adaptation, while depth consistency and joint symmetric training improve GASDA.The comparisons include SYN, REAL, SYN2REAL, REAL2SYN, GASDA-w/oDC, and the two GASDA depth-estimator outputs.
5. Conclusion
GASDA combines labelled synthetic depth data with epipolar geometry from real stereo data in a unified symmetric network. Experiments on KITTI and Make3D show desirable quantitative and qualitative results and generalization to unseen datasets.
- Conclusion: GASDA jointly uses labelled synthetic image-depth pairs and epipolar geometry from real stereo data for unsupervised monocular depth estimation.The framework also uses bidirectional image style transfer to reduce domain discrepancy and trains translation and depth estimation end to end.
- Conclusion: Experiments on KITTI and Make3D report desirable quantitative and qualitative results and good generalization to unseen datasets.