Source-linked AI summary
HR-Depth: High Resolution Self-Supervised Monocular Depth Estimation
Xiaoyang Lyu, Liang Liu, Mengmeng Wang, Xin Kong, Lina Liu, Yong Liu, Xinxin Chen, Yi Yuan
TL;DR
High-resolution monocular depth estimation has not consistently improved accuracy, partly because large-gradient regions require better spatial and semantic features. HR-Depth redesigns skip connections and adds fSE fusion, while Lite-HR-Depth uses a lightweight strategy; the authors report strong performance with substantially fewer parameters.
Problem
High-resolution inputs have produced limited accuracy gains, while semantic and spatial information remain difficult to integrate and complex models require many parameters.
Method
HR-Depth combines redesigned dense skip connections with a feature fusion Squeeze-and-Excitation block, alongside a lightweight network design strategy.
Results
HR-Depth outperforms existing self- and semi-supervised methods and is competitive with supervised methods, while Lite-HR-Depth performs on par with large models using fewer parameters.
Takeaways & Limitations
The reported results support high-resolution self-supervised depth estimation with sharper predictions and lightweight models suitable for parameter-constrained deployment.
Abstract
from arXiv · showhide
Self-supervised learning shows great potential in monoculardepth estimation, using image sequences as the only source ofsupervision. Although people try to use the high-resolutionimage for depth estimation, the accuracy of prediction hasnot been significantly improved. In this work, we find thecore reason comes from the inaccurate depth estimation inlarge gradient regions, making the bilinear interpolation er-ror gradually disappear as the resolution increases. To obtainmore accurate depth estimation in large gradient regions, itis necessary to obtain high-resolution features with spatialand semantic information. Therefore, we present an improvedDepthNet, HR-Depth, with two effective strategies: (1) re-design the skip-connection in DepthNet to get better high-resolution features and (2) propose feature fusion Squeeze-and-Excitation(fSE) module to fuse feature more efficiently.Using Resnet-18 as the encoder, HR-Depth surpasses all pre-vious state-of-the-art(SoTA) methods with the least param-eters at both high and low resolution. Moreover, previousstate-of-the-art methods are based on fairly complex and deepnetworks with a mass of parameters which limits their realapplications. Thus we also construct a lightweight networkwhich uses MobileNetV3 as encoder. Experiments show thatthe lightweight network can perform on par with many largemodels like Monodepth2 at high-resolution with only20%parameters. All codes and models will be available at https://github.com/shawLyu/HR-Depth.
Introduction
HR-Depth targets the limited gains of high-resolution monocular depth estimation by improving the integration of spatial and semantic information. It combines redesigned skip connections with fSE feature fusion and introduces a lightweight design that preserves accuracy with fewer parameters.
- Motivation: High-resolution depth methods have achieved limited performance improvements, while earlier self-supervised approaches were constrained by the memory demands of low-resolution training.
- Motivation: Large semantic gaps between encoder and decoder features hinder the integration of spatial and semantic information needed for sharp object boundaries.
- Evaluation: The method is evaluated on KITTI, where the authors report sharper edges and state-of-the-art performance.
- Lightweight design: Lite-HR-Depth uses a lightweight design strategy and achieves Monodepth2-level accuracy with only 3.1M parameters.
- HR-Depth: HR-Depth redesigns skip connections to produce high-resolution semantic feature maps that help predict sharper edges.
- HR-Depth: The fSE block improves feature-fusion efficiency and effect by replacing basic convolution and reducing the number of parameters.
Related Work
Prior monocular depth work progressed from supervised RGB-D learning to self-supervision from stereo pairs and monocular image sequences. HR-Depth addresses feature-fusion limitations in this line of work while targeting accurate and efficient inference.
- Supervised methods: Supervised monocular depth estimation faces poor generalization and difficult ground-truth acquisition, motivating self-supervised approaches.
- Self-supervised methods: Self-supervised methods use stereo consistency or relative pose estimation from adjacent monocular frames instead of requiring ground-truth depth.
- Feature fusion: HR-Depth fuses encoder information to obtain features containing spatial and semantic information, improving overall performance and producing sharper edges without many additional parameters.
- Lightweight networks: Lightweight depth networks reduce parameters, but earlier compact models suffered substantial performance decreases compared with complex networks.
- Evaluation: Abs Rel is an evaluation index for depth estimation in which lower values are better; HR and LR denote high- and low-resolution settings.
HR-Depth Network
HR-Depth targets inaccurate depth estimation at large-gradient regions by improving high-resolution feature fusion in DepthNet. It combines redesigned dense skip connections with an fSE module and supports a lightweight MobileNetV3 variant.
- Self-Supervised Training: HR-Depth is trained with photometric re-projection and edge-aware smoothness losses across multiple image scales.The setup predicts depth from a target RGB image and relative pose from adjacent source frames.
- High-Resolution Motivation: High-resolution inputs do not significantly improve prior depth-estimation performance because low-resolution predictions are interpolated to high resolution, with errors concentrated near large depth gradients.The paper identifies local prediction errors around instance edges as especially harmful to global accuracy.
- Redesign Skip Connection: Dense skip connections aggregate encoder, intermediate, and decoder features to reduce semantic and resolution gaps and produce sharper depth maps.Intermediate nodes are added to aggregate features before decoding.
- Feature Fusion SE Block: The fSE module uses global average pooling, fully connected layers, sigmoid re-weighting, and 1×1 convolution to improve feature fusion efficiency.With Resnet-18, fSE reduces HR-Depth parameters from 16.06M to 14.62M, below Monodepth2's 14.84M.
- Lightweight Network: The lightweight Lite-HR-Depth uses a teacher-student strategy and achieves performance better than Monodepth2 at high resolution with 20% of its parameters.The student network S is trained to minimize disparity differences relative to teacher network T.
Experiments
Experiments on KITTI evaluate HR-Depth and Lite-HR-Depth across resolutions, datasets, qualitative outputs, and architectural ablations. The results show stronger depth estimation, sharper edges, and efficient feature fusion with reduced parameter counts.
- Datasets: 39810 training, 4424 validation, and 697 evaluation images were used from KITTI, with optional CityScapes pre-training and KITTI fine-tuning.The evaluation follows the Eigen split and removes static frames.
- Qualitative Results: HR-Depth and Lite-HR-Depth predict sharper edges than Monodepth2 and achieve performance comparable to PackNet-SfM with fewer parameters.Figure 4 presents qualitative comparisons on KITTI frames.
- Depth Estimation Performance: HR-Depth outperforms existing self-supervised state-of-the-art approaches and improves further with CityScapes pre-training and higher-resolution inputs.The best results use higher-resolution inputs with minimum parameters.
- Depth Estimation Performance: Lite-HR-Depth surpasses Monodepth2 with teacher supervision at low resolution and without additional supervision at high resolution.The comparison uses the KITTI benchmark and the same evaluation metrics as Table 2.
- Ablation Studies: Dense skip-connections reduce the semantic gap between encoder and decoder features, producing high-resolution maps with richer semantic information.Intermediate features are progressively enriched as semantic information is added through the dense skip connection.
Conclusion
HR-Depth addresses high-resolution self-supervised monocular depth estimation with dense skip-connections and an fSE block. The paper reports state-of-the-art performance, while Lite-HR-Depth performs on par with larger models using fewer parameters.
- Conclusion: HR-Depth uses dense skip-connections and an fSE block to reduce the gap between feature resolution and semantic information.The architecture is designed for high-resolution self-supervised monocular depth estimation.
- Conclusion: HR-Depth outperforms existing self- and semi-supervised methods and is competitive with supervised methods despite training only on image sequences.The conclusion reports both theoretical and empirical evidence for improving high-resolution estimation.
- Conclusion: Lite-HR-Depth performs on par with large models while using fewer parameters.The lightweight design strategy is presented as simple and efficient.