Source-linked AI summary
Deep Ordinal Regression Network for Monocular Depth Estimation
Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, Dacheng Tao
TL;DR
Monocular depth estimation from a single image is ill-posed, and regression-based methods face slow convergence while pooling-based architectures produce low-resolution features. The paper introduces SID ordinal regression and a simpler multi-scale network, achieving state-of-the-art results across four challenging benchmarks.
Problem
Monocular depth estimation is ill-posed, while regression training converges slowly and existing pooling-based architectures require complex mechanisms for high-resolution outputs.
Method
The method discretizes depth with spacing-increasing discretization and trains an end-to-end ordinal regression network with a multi-scale architecture that avoids unnecessary subsampling.
Results
The proposed method achieves state-of-the-art results on KITTI, ScanNet, Make3D, and NYU Depth v2.
Takeaways & Limitations
DORN provides a clean monocular depth-estimation network that addresses high-resolution prediction and regression-optimization concerns within one end-to-end framework.
Abstract
from arXiv · showhide
Monocular depth estimation, which plays a crucial role in understanding 3D scene geometry, is an ill-posed problem. Recent methods have gained significant improvement by exploring image-level information and hierarchical features from deep convolutional neural networks (DCNNs). These methods model depth estimation as a regression problem and train the regression networks by minimizing mean squared error, which suffers from slow convergence and unsatisfactory local solutions. Besides, existing depth estimation networks employ repeated spatial pooling operations, resulting in undesirable low-resolution feature maps. To obtain high-resolution depth maps, skip-connections or multi-layer deconvolution networks are required, which complicates network training and consumes much more computations. To eliminate or at least largely reduce these problems, we introduce a spacing-increasing discretization (SID) strategy to discretize depth and recast depth network learning as an ordinal regression problem. By training the network using an ordinary regression loss, our method achieves much higher accuracy and \dd{faster convergence in synch}. Furthermore, we adopt a multi-scale network structure which avoids unnecessary spatial pooling and captures multi-scale information in parallel. The method described in this paper achieves state-of-the-art results on four challenging benchmarks, i.e., KITTI [17], ScanNet [9], Make3D [50], and NYU Depth v2 [42], and win the 1st prize in Robust Vision Challenge 2018. Code has been made available at: https://github.com/hufu6371/DORN.
1. Introduction
Monocular depth estimation from a single image is ill-posed and remains difficult despite progress from deep features. DORN addresses slow regression training and low-resolution outputs with ordinal learning and a simpler multi-scale architecture.
- Monocular depth estimation is ill-posed because one 2D image can correspond to infinitely many distinct 3D scenes.
- DCNN-based methods improve monocular depth estimation but typically formulate it as continuous regression optimized with MSE in log-space.
- Regression training can converge slowly and reach unsatisfactory final solutions.
- Repeated pooling reduces feature-map resolution, while recovering high-resolution depth with deconvolution, multi-scale features, or skip-connections complicates training.
- DORN discretizes continuous depth with SID, casts learning as ordinal regression, and uses an ordinal loss that accounts for label ordering.SID allows relatively larger errors at greater depths, where prediction uncertainty increases.
- DORN avoids unnecessary subsampling, extracts multi-scale information more simply, and achieves state-of-the-art results on four challenging benchmarks.The network is trained end-to-end without stage-wise training or iterative refinement.
2. Related Work
Prior depth-estimation work progressed from geometry-based stereo methods and monocular cues to DCNNs that capture contextual and structural information. Related approaches also include unsupervised reconstruction, weakly supervised ranking, and ordinal regression formulations.
- Early depth estimation used stereo geometry, while later monocular methods learned from cues such as image structure and scene appearance.Handcrafted features and global models were used to incorporate local and long-range information.
- DCNNs improved depth estimation by providing multi-level contextual and structural information from very deep networks.
- Repeated pooling in deep feature extractors rapidly decreases feature-map spatial resolution, motivating multi-scale and related architectural responses.
- Unsupervised and semi-supervised methods use reconstruction losses to estimate disparity by recovering one view from another.
- Weakly supervised methods use pair-wise ranking information to roughly estimate and compare depth.
- Ordinal regression predicts labels on an ordered scale and can be formulated as multiple threshold decisions or binary classification subproblems.
3. Method
DORN combines a dense multi-scale architecture with SID-based ordinal regression to estimate depth while preserving spatial detail and modeling global context.
- DORN uses a dense feature extractor and scene understanding module to output multi-channel dense ordinal labels from an input image.
- The architecture limits resolution loss by avoiding unnecessary subsampling, using dilated convolutions, and extracting multi-scale features in parallel.ASPP uses dilation rates 6, 12, and 18, while additional branches model cross-channel interactions and global context.
- A full-image encoder captures global contextual information with fewer parameters than fully connected alternatives.The encoder uses average pooling before subsequent processing, reducing the parameter burden of pure fully connected designs.
- SID discretizes depth in log space rather than uniformly, down-weighting losses for large depths where estimation errors are generally larger.The depth interval [α, β] is shifted before SID is applied, with α∗ set to 1.0.
- The method casts depth prediction as ordinal regression, using an ordinal loss that penalizes predictions farther from the true label more strongly.The loss averages pixelwise ordinal losses over the image and is optimized by backpropagation; predicted ordinal labels are decoded into depth values during inference.
- On KITTI, DORN predictions are presented alongside ground truth, Eigen, and LRC, with visualization-specific interpolation and masking noted.Pixels beyond 80m in LRC are masked out in the displayed comparison.
4. Experiments
Experiments evaluate DORN across outdoor and indoor benchmarks, online servers, qualitative examples, and targeted ablations. Results show strong benchmark performance and identify depth discretization, ordinal regression, interval count, and the full-image encoder as important design factors.
- Benchmark performance: DORN significantly outperforms officially provided baselines on the online KITTI and ScanNet evaluation servers.The evaluation also covers KITTI, Make3D, and NYU Depth v2, alongside qualitative results on Make3D.
- Benchmark performance: DORN improves accuracy by 5%–30% across all metrics and settings on KITTI and Make3D compared with previous works.On NYU Depth v2, DORN also outperforms other methods, indicating applicability to indoor and outdoor data.
- Depth discretization and ordinal regression: Ordinal regression with depth discretization achieves the best performance, while continuous-depth regression converges to a poorer solution.The ablation compares regression, multi-class classification, SID, UD, and BerHu-based training.
- Depth discretization and ordinal regression: SID improves performance over uniform discretization, and ordinal correlation further improves convergence relative to regression-based alternatives.The study concludes that discretization with classification and ordinal losses is more effective than regression losses.
- Full-image encoder: The full-image encoder performs better than comparable fully connected encoders while requiring fewer computational resources.The reported parameter analysis gives approximately 753M parameters for one comparison and 51M for the proposed alternative.
- Number of intervals: Between 40 and 120 SID intervals, DORN scores 0.908–0.915 on δ < 1.25 and 3.056–3.125 RMSE, showing robustness to interval count.Too few intervals cause large quantization error, whereas too many lose the advantage of discretization.
5. Conclusion
The paper develops DORN for monocular depth estimation, combining a clean CNN architecture with depth discretization and ordinal-regression optimization strategies. It reports state-of-the-art performance on four benchmark datasets while identifying extensions to other dense prediction problems as future work.
- DORN combines a clean CNN architecture with depth discretization and ordinal regression to improve monocular depth estimation training and accuracy.The network also uses dilated convolutions and a full-image encoder to directly obtain high-resolution depth maps.
- The architecture targets high-resolution depth estimation while reducing the complexity and computational cost associated with multi-scale and full-image feature integration.It uses dilated convolutions and a full-image encoder rather than relying on complex prior architectures.
- The proposed method achieves state-of-the-art performance on the KITTI, ScanNet, Make3D, and NYU Depth v2 datasets.
- Future work will investigate new approximations to depth and extend the framework to other dense prediction problems.