Source-linked AI summary
Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepth
Doyeon Kim, Woonghyun Ka, Pyungwhan Ahn, Donggyu Joo, Sehwan Chun, Junmo Kim
TL;DR
Monocular depth estimation remains difficult because a single image is ambiguous, motivating architectures that capture both global context and local connectivity. The paper combines a hierarchical transformer encoder, lightweight decoder, selective feature fusion, and vertical CutDepth, achieving state-of-the-art NYU Depth V2 performance with stronger generalization and robustness.
Problem
Monocular depth estimation is an inherently ambiguous single-image prediction problem, so the paper seeks improved accuracy through a new architecture and training strategy.
Method
The method combines a hierarchical transformer encoder, global-local feature paths with selective feature fusion, and vertical CutDepth augmentation.
Results
The network achieves state-of-the-art performance on NYU Depth V2 and shows improved generalization and robustness against comparative networks.
Takeaways & Limitations
Global-local feature integration and vertical-position-aware augmentation provide the paper’s supported route to more accurate monocular depth estimation.
Abstract
from arXiv · showhide
Depth estimation from a single image is an important task that can be applied to various fields in computer vision, and has grown rapidly with the development of convolutional neural networks. In this paper, we propose a novel structure and training strategy for monocular depth estimation to further improve the prediction accuracy of the network. We deploy a hierarchical transformer encoder to capture and convey the global context, and design a lightweight yet powerful decoder to generate an estimated depth map while considering local connectivity. By constructing connected paths between multi-scale local features and the global decoding stream with our proposed selective feature fusion module, the network can integrate both representations and recover fine details. In addition, the proposed decoder shows better performance than the previously proposed decoders, with considerably less computational complexity. Furthermore, we improve the depth-specific augmentation method by utilizing an important observation in depth estimation to enhance the model. Our network achieves state-of-the-art performance over the challenging depth dataset NYU Depth V2. Extensive experiments have been conducted to validate and show the effectiveness of the proposed approach. Finally, our model shows better generalisation ability and robustness than other comparative models.
1 Introduction
The paper proposes a global-local path network and depth-specific training strategy for monocular depth estimation. It combines hierarchical global context, local feature fusion, and vertical CutDepth, achieving state-of-the-art NYU Depth V2 performance with improved generalization and robustness.
- Monocular depth estimation is challenging because a single image provides inherently ambiguous depth cues.
- The network uses a global-local path architecture to extract multi-scale features and deliver them throughout the network.Its hierarchical transformer encoder models long-range dependencies, while the decoder and feature-fusion paths preserve local detail.
- The proposed variant of CutDepth crops only along the horizontal axis to help the model learn vertical long-range information.
- The architecture and training strategy achieve state-of-the-art performance on NYU Depth V2.
- The network shows improved generalization and robustness compared with previously developed networks.The paper reports cross-dataset generalization and robustness against image corruption.
2 Related Work
Prior work established CNN, transformer, and depth-specific augmentation approaches for monocular depth estimation. The paper positions its method against computationally heavier transformer-based systems and limited depth-specific augmentation research.
- Monocular depth estimation predicts depth maps from input images and has progressed from Markov random fields to deep CNNs.
- Transformers have been adopted for dense prediction, including segmentation and depth estimation, to capture global receptive fields.
- AdaBins and DPT combine CNN-based encoders with transformers, increasing computational complexity; DPT also uses an extra large-scale dataset.
- Depth-specific augmentation has been rarely studied, with CutDepth identified as an early method for depth estimation.
3 Methods
The framework combines global context from a hierarchical transformer encoder with local feature paths and selective fusion in a lightweight decoder. Its training strategy uses a vertically constrained CutDepth augmentation to preserve vertical information during depth estimation.
- 3 Methods: The framework predicts a depth map from an RGB image using a hierarchical transformer encoder for global dependencies and a decoder with local skip paths.The encoder produces multi-scale features and a bottleneck representation for reconstruction into the target depth map.
- 3 Methods: The lightweight decoder restores the bottleneck feature with fewer convolution and bilinear upsampling layers than conventional decoders.The decoder ultimately applies convolution layers and a sigmoid function to produce the depth map.
- 3 Methods: The Selective Feature Fusion module adaptively integrates multi-scale local features with decoded global features using attention maps.Local features are dimension-reduced, concatenated, processed by convolutional layers, and used to generate a two-channel attention map.
- 3 Methods: Vertical CutDepth replaces a horizontally bounded region while maintaining the input image's full vertical range to preserve long-range vertical information.The replacement region begins at the top and spans the image height, with p set to 0.75 after evaluating different settings.
- 3 Methods: Training uses scale-invariant log loss to measure the distance between predicted and ground-truth depth values.The loss uses per-pixel log differences defined as d_i = log y_i − log y_i^*.
4 Experiments
Experiments evaluate the proposed model on NYU Depth V2 and SUN RGB-D, including comparisons, decoder and augmentation ablations, and corruption robustness. The model achieves strong performance with a compact decoder, generalizes without fine-tuning, and benefits from vertical CutDepth.
- Experimental Setup: The evaluation covers NYU Depth V2 and SUN RGB-D using quantitative and qualitative comparisons, ablations, and robustness testing.SUN RGB-D is used to evaluate models trained on NYU Depth V2, while corruption experiments use NYU Depth V2 test images.
- Comparison with State-of-the-Arts: The proposed model achieves state-of-the-art performance on most NYU Depth V2 metrics and exceeds recent models with fewer parameters.The comparison includes Adabins and DPT; DPT uses 1.4M training images, whereas the proposed model uses fewer parameters.
- Cross-Dataset Generalization: The model produces accurate NYU Depth V2 depth maps under varied illumination and outperforms competing methods on all SUN RGB-D metrics without fine-tuning.SUN RGB-D evaluation uses a model trained on NYU Depth V2, and qualitative results show reasonable depth maps without additional training.
- Decoder Ablation: The proposed decoder outperforms simpler and existing multi-scale decoders while using only 0.66M parameters.The MLP-decoder, BTS, and DPT use 3.19M, 5.79M, and 14.15M parameters, respectively; selective feature fusion further improves performance.
- Data-Augmentation Ablation: Vertical CutDepth improves performance beyond the baseline and basic CutDepth, with the best result at p = 0.75.The ablation attributes the further improvement to using vertical features rather than cropping a random area.
- Robustness: The model is more robust than compared models across the tested corruption types, supporting its use in safety-critical applications.The robustness evaluation uses corrupted images from the NYU Depth V2 test set.
5 Conclusion
The paper proposes a global-local architecture and vertical-position-aware augmentation strategy for monocular depth estimation. It reports improved NYU Depth V2 performance alongside demonstrated effectiveness and generalization ability.
- The method combines a new architecture for global and local features with depth-specific augmentation that uses vertical position.The architecture targets precise depth maps, while the augmentation exploits vertical position as a crucial depth-estimation property.
- The proposed method improves over state-of-the-art performance on NYU Depth V2.
- Extensive experiments demonstrate the effectiveness and generalization ability of the network.
6 Appendix: Additional dataset results
Additional experiments evaluate the model on KITTI and iBims-1, covering outdoor and indoor depth estimation settings. The KITTI evaluation uses a defined training crop, test split, and 80m maximum depth, while iBims-1 is evaluated without fine-tuning.
- KITTI is an outdoor depth estimation dataset, while iBims-1 is an indoor dataset used for additional evaluation.
- On KITTI, the network trains on approximately 23K images and tests on 697 images using a 704 × 352 random crop.Evaluation follows the crop defined by Garg et al. and uses an 80m maximum depth.
- The appendix reports KITTI and iBims-1 performance in Tables 6 and 7.
- iBims-1 provides high-quality RGB-D data with accurate edges, planar regions, consistent depth values, and absolute distances.The NYU Depth V2-trained model is evaluated on iBims-1 without fine-tuning.
7 Appendix: Robustness of the Model
The appendix reports a full robustness evaluation on corrupted NYU Depth V2 images. The results are presented in Table 8.
- Table 8 reports robustness experiment results on corrupted NYU Depth V2 images.
8 Appendix: Detailed structure of baseline decoder
The appendix details Baseline-DConv and Baseline-UNet decoder structures. Both use transposed convolution for 2× upscaling, while Baseline-UNet concatenates encoder features channel-wise.
- The baseline decoders use transposed convolution with K = 3, S = 2, and P = 1 to upscale features to twice their size.
- Baseline-UNet concatenates encoder features in the channel dimension.
- Figure 5 illustrates the detailed structures of Baseline-DConv and Baseline-UNet.