Source-linked AI summary
AdaBins: Depth Estimation using Adaptive Bins
Shariq Farooq Bhat, Ibraheem Alhashim, Peter Wonka
TL;DR
The paper addresses high-quality dense depth estimation from a single RGB image, where scene depth distributions vary substantially. It introduces AdaBins, a transformer-based block that adaptively divides the depth range into image-specific bins and combines their centers to estimate depth. The authors report decisive improvements over the state of the art on NYU and KITTI, supported by ablations of the module.
Problem
The paper addresses high-quality dense depth estimation from a single RGB image, a difficult problem whose scene depth distributions can vary substantially.
Method
AdaBins uses global processing to predict image-specific depth-bin widths within an encoder-decoder architecture, with final depths formed from bin centers.
Results
The authors report decisive improvement across all metrics on NYU and KITTI, and ablations show that removing AdaBins performs worse than its variants.
Takeaways & Limitations
Adaptive binning provides the paper's reported state-of-the-art depth-estimation architecture across the two evaluated indoor and outdoor datasets.
Takeaways & Limitations
A comparison caveat is that DAV computes depth maps at one-quarter resolution and downsamples ground truth for evaluation, unlike the full-resolution evaluation used by other methods.
Abstract
from arXiv · showhide
We address the problem of estimating a high quality dense depth map from a single RGB input image. We start out with a baseline encoder-decoder convolutional neural network architecture and pose the question of how the global processing of information can help improve overall depth estimation. To this end, we propose a transformer-based architecture block that divides the depth range into bins whose center value is estimated adaptively per image. The final depth values are estimated as linear combinations of the bin centers. We call our new building block AdaBins. Our results show a decisive improvement over the state-of-the-art on several popular depth datasets across all metrics. We also validate the effectiveness of the proposed block with an ablation study and provide the code and corresponding pre-trained weights of the new state-of-the-art model.
1. Introduction
AdaBins addresses dense monocular depth estimation by adapting depth-bin placement to each input image and applying global processing to refine encoder-decoder predictions. The proposed block improves supervised single-image depth estimation on NYU and KITTI.
- AdaBins targets high-quality dense depth estimation from a single RGB image.
- The method uses global statistical analysis and high-resolution learned post-processing to refine a traditional encoder-decoder output.
- Because scene depth distributions vary widely, AdaBins focuses adaptively on depth-range regions more probable for each input image.
- The architecture combines an encoder-decoder block with an adaptive bin-width estimator, producing a single-channel depth image at reduced spatial resolution.
- AdaBins divides the predicted depth range into image-dependent bins and estimates final depths as linear combinations of bin centers.
- The authors report decisive improvement across all metrics on the NYU and KITTI depth-estimation datasets.
2. Related Work
Prior monocular depth-estimation work uses CNN-based regression and encoder-decoder architectures, with competitors incorporating planar guidance or bottleneck attention. AdaBins instead explores adaptive depth-bin choices and transformer-based non-local processing.
- Single-image depth estimation is an ill-posed 3D reconstruction problem, though CNNs can produce reasonable depth maps in real time.
- Monocular depth methods commonly regress dense depth maps from single RGB images using CNNs.
- BTS uses local planar guidance during decoding, whereas DAV exploits object co-planarity through attention at the bottleneck.
- Unlike pre-determined or dataset-specific bin widths, adaptive bins vary for each input image.
- Encoder-decoder networks have been successful across vision tasks and form the baseline architecture adapted in this paper.
- Transformer-based CNN combinations motivate using a Transformer encoder for non-local processing of CNN outputs.
3. Methodology
AdaBins augments an encoder-decoder depth estimator with image-adaptive binning, transformer-based global processing, and hybrid regression. Its design combines globally estimated depth intervals with smooth per-pixel depth prediction.
- AdaBins design: AdaBins generalizes ordinal depth estimation by adapting bin widths per image and predicting depth as a linear combination of bin centers.This combines classification-like depth reasoning with regression to avoid discretization artifacts.
- AdaBins design: The depth interval is evaluated using fixed uniform bins, fixed log-scale bins, trained dataset-wide widths, and image-specific AdaBins widths.The paper recommends AdaBins and reports that ablation validates its superiority over these alternatives.
- Range attention: Range-Attention-Maps combine decoded high-resolution features with transformer outputs used as convolutional kernels for pixel-level depth computation.The mini-ViT block is designed to estimate both adaptive bin widths and the representations needed for range attention.
- Hybrid regression: Hybrid regression uses softmax scores over bin centers, producing smoothly varying depth values instead of selecting the most likely bin center.The paper connects this choice to smoother depth maps without discretization artifacts.
- Architecture: The architecture uses an EfficientNet B5 encoder, feature upsampling decoder, and AdaBins module that converts decoded features into a depth map.The AdaBins module operates on decoded features and produces a lower-resolution output that is bilinearly upsampled to the final image resolution.
- Mini-ViT: Mini-ViT converts decoded features into patch embeddings, applies a transformer encoder, and predicts a normalized bin-width vector for each image.A small transformer supplies global information while the MLP head outputs the bin-width parameters.
4. Experiments
The experiments evaluate AdaBins on indoor and outdoor depth datasets against prior methods, then examine its design choices through ablations. AdaBins outperforms prior state of the art across reported metrics, while adaptive binning and the Chamfer loss improve performance.
- Datasets and metrics: The study evaluates supervised monocular depth estimation on NYU-Depth-v2 and KITTI using standard metrics and dataset-specific evaluation settings.NYU-Depth-v2 covers indoor scenes, while KITTI covers outdoor scenes with depths up to 80 meters.
- Comparison to the state-of-the-art: NYU-Depth-v2 results significantly outperform the previous state of the art across all metrics.
- Comparison to the state-of-the-art: 13.5% RMS improvement and 22.4% Squared Relative Difference improvement are reported over the previous state of the art on KITTI.
- Ablation study: Removing AdaBins makes the architecture perform worse than all evaluated AdaBins variants.
- Ablation study: Adaptive bins outperform the alternative bin-width choices, while trained-but-fixed bins perform worst.
- Ablation study: Absolute Relative Error varies non-monotonically with bin count, and performance gains diminish above N = 256, the final model’s choice.
- Ablation study: Adding the Chamfer loss reduces Absolute Relative Error from 10.6% to 10.3%.
5. Conclusion
The paper concludes that AdaBins improves single-image depth estimation on NYU and KITTI. It identifies high-resolution global processing as a direction for future investigation in other vision tasks.
- AdaBins delivers a decisive state-of-the-art improvement for single-image depth estimation on NYU and KITTI.
- Future work will investigate high-resolution global processing for segmentation, normal estimation, and multi-image 3D reconstruction.
A.1. Geometric Consistency
The paper qualitatively evaluates geometric consistency by visualizing surface normals derived from predicted depth maps. The comparison includes AdaBins, DAV, and BTS.
- Surface-normal visualizations are used to assess predicted surface orientations and texture details.
- The qualitative comparison includes depth maps from AdaBins, DAV, and BTS.
A.2. Generlization Analysis
The generalization analysis tests models trained on NYU-Depth-v2 on unseen SUN RGB-D data without retraining. AdaBins produces consistent results in this qualitative comparison, while BTS shows conspicuous artifacts.
- Models trained on NYU-Depth-v2 are evaluated on SUN RGB-D to assess generalization to unseen data.The comparison is performed without fine-tuning.
- BTS predictions contain conspicuous artifacts, whereas AdaBins provides consistent results on SUN RGB-D.
A.3. More Results on KITTI dataset
On KITTI, the method is qualitatively compared with BTS, showing stronger predictions for vehicle, sign-board, and thin-pole extents and edges. BTS more often blends distant objects with the background.
- On KITTI, the method qualitatively outperforms BTS in predicting the extents and edges of on-road vehicles, sign-boards, and thin poles.
- The comparison removes sky regions from visualized depth maps using masks from a pretrained segmentation model.
- BTS tends to blend farther-away objects with the background, unlike the sharper qualitative predictions described for the method.
A.4. MLP Head Details
The mini-ViT module uses a three-layer MLP head attached to its first transformer output embedding, with architecture parameters organized by embedding dimension and bin count.
- The mini-ViT module applies a three-layer MLP to the transformer's first output embedding.
- Table 7 reports the MLP head architecture and its parameters.
- The architecture notation defines FC as fully connected layer, E as embedding dimension, and N as number of bins.