Source-linked AI summary
Geometry-based Distance Decomposition for Monocular 3D Object Detection
Xuepeng Shi, Qi Ye, Xiaozhi Chen, Chuangrong Chen, Zhixiang Chen, Tae-Kyun Kim
TL;DR
Monocular 3D detection must estimate object distance from RGB images without explicit depth. The paper decomposes distance into physical and projected visual heights, then uses this geometry-based representation in a compact detector. It reports state-of-the-art KITTI performance across monocular 3D detection and Bird’s Eye View tasks, with generalization to different camera intrinsics.
Problem
The central problem is estimating object distance for monocular 3D detection when RGB images provide no explicit depth.
Method
The method decomposes distance into physical height and projected visual height, regresses these factors and their uncertainties, and reconstructs distance through camera geometry.
Results
The method achieves state-of-the-art performance on KITTI monocular 3D Object Detection and Bird’s Eye View tasks and generalizes to different camera intrinsics.
Takeaways & Limitations
The decomposition makes distance prediction interpretable and robust while enabling uncertainty causes to be traced for different scenarios.
Takeaways & Limitations
The method assumes yaw-only 3D bounding boxes and known per-image camera intrinsics during training and inference.
Abstract
from arXiv · showhide
Monocular 3D object detection is of great significance for autonomous driving but remains challenging. The core challenge is to predict the distance of objects in the absence of explicit depth information. Unlike regressing the distance as a single variable in most existing methods, we propose a novel geometry-based distance decomposition to recover the distance by its factors. The decomposition factors the distance of objects into the most representative and stable variables, i.e. the physical height and the projected visual height in the image plane. Moreover, the decomposition maintains the self-consistency between the two heights, leading to robust distance prediction when both predicted heights are inaccurate. The decomposition also enables us to trace the causes of the distance uncertainty for different scenarios. Such decomposition makes the distance prediction interpretable, accurate, and robust. Our method directly predicts 3D bounding boxes from RGB images with a compact architecture, making the training and inference simple and efficient. The experimental results show that our method achieves the state-of-the-art performance on the monocular 3D Object Detection and Birds Eye View tasks of the KITTI dataset, and can generalize to images with different camera intrinsics.
1. Introduction
Monocular 3D detection must infer object distance from RGB images without explicit depth. The paper proposes decomposing distance into physical and projected visual heights, yielding interpretable, robust prediction and strong KITTI results.
- Monocular 3D detection infers object distance without explicit depth, making geometric priors about object size, scene layout, and camera imaging essential.
- The proposed decomposition recovers distance from an object's physical height and projected visual height using pinhole-camera geometry.The physical height is an intrinsic object attribute, while projected visual height reflects its extrinsic scene position.
- The decomposition preserves self-consistency between the two heights, supporting robust distance prediction when both predicted heights are inaccurate.
- The method makes distance prediction interpretable, accurate, and robust while enabling causes of distance uncertainty to be traced.
- The compact architecture directly predicts 3D bounding boxes from RGB images, simplifying training and inference.
- The method achieves state-of-the-art performance on KITTI monocular 3D Object Detection and Bird’s Eye View tasks and adapts to different camera intrinsics.
2. Related Work
Prior monocular methods commonly regress distance directly or derive it from depth, proposals, or reprojection, whereas MonoRCNN uses a simpler factorization based on geometry. Its uncertainty-aware regression operates on decomposed variables to make uncertainty interpretable.
- Faster R-CNN with FPN provides the paper's two-stage 2D detection framework because of its accuracy and flexibility.
- Most monocular methods regress distance as a single variable, while pseudo-LiDAR methods estimate depth first and anchor-based methods split proposals from offset regression.
- Prior factor-based approaches use projected box boundaries or eight projected corners, whereas this method adopts a simpler decomposition.
- MonoRCNN's architecture adds a 3D distance head to Faster R-CNN and recovers distance from decomposed geometric variables.
- Uncertainty-aware regression is applied to decomposed distance variables, enabling the causes of distance uncertainty to be traced across scenarios.
3. Proposed MonoRCNN
MonoRCNN predicts monocular 3D boxes with separate distance and attribute heads, using geometry-based decomposition of distance into physical height and reciprocal projected visual height. The design improves interpretability and robustness by exploiting height correlation and tracing uncertainty across object distances.
- Geometry-based Distance Decomposition: The distance head decomposes distance Z into physical height H and reciprocal projected visual height hrec, then recovers Z from both predictions.The focal length f connects these geometric factors to distance.
- Geometry-based Distance Decomposition: Physical height is preferred over full physical size because its prediction error is smaller and is not affected by yaw-angle ambiguity like physical length and width.The method uses only physical height to recover distance, while visual height remains easier to estimate than eight projected corners under challenging viewing conditions.
- Geometry-based Distance Decomposition: The network learns an inverse correlation between H and hrec, enabling self-consistent distance prediction when both predicted heights are inaccurate.For different objects, the expected reciprocal projected height decreases as physical height increases; the learned correlation compensates during inference.
- Uncertainty-aware Regression: Uncertainty is separately estimated for H and hrec to identify distance-dependent error sources and improve detection-box ranking.Both uncertainties decrease and then increase with distance; truncation dominates nearby errors, while coarse faraway views particularly increase hrec uncertainty.
- 3D Attribute Head: During inference, MonoRCNN uses the projected 3D center while the eight projected corners serve as auxiliary training losses.The attribute head predicts physical size, yaw, and projected center and corners, with specified weights for the detection losses.
4. Experiments
Experiments evaluate MonoRCNN through ablations, KITTI benchmark comparisons, self-consistency analysis, and cross-dataset testing. The results support the effectiveness of the decomposition, uncertainty modeling, score sorting, benchmark accuracy, efficiency, and generalization to different camera intrinsics.
- 4.1. Datasets: The experiments use KITTI validation and official test splits for BEV and 3D object detection, plus a nuScenes cross-test set for cross-dataset evaluation.KITTI experiments focus on cars and report AP|R40 under the official IoU criterion; the nuScenes cross-test uses front-camera images converted to KITTI format.
- 4.2. Ablation Studies: The ablation study compares direct distance regression, keypoint-based recovery, the proposed decomposition, uncertainty-aware regression, and score-based sorting.The ablation labels are L, K, D, U, and S, respectively.
- 4.2. Ablation Studies: The geometry-based decomposition outperforms the keypoint-based model by a large margin, supporting its effectiveness.This comparison is made between models D and K.
- 4.2. Ablation Studies: Uncertainty-aware regression improves accuracy, and sorting by score fHσhrec improves accuracy while better representing 3D prediction quality.These effects are reported by comparing D+U with D and D+U+S with D+U.
- 4.3. Self-Consistency: Self-consistency is supported because replacing MonoRCNN’s predicted physical height with ground truth decreases accuracy, whereas the keypoint model improves when ground-truth physical size replaces its prediction.The results indicate that MonoRCNN maintains consistency between predicted quantities used for distance recovery, unlike the keypoint-based alternative.
- 4.4. Comparisons on the KITTI benchmark: MonoRCNN achieves state-of-the-art KITTI accuracy, outperforming image-only and depth-based methods while remaining faster than several monocular depth-estimation pipelines.Reported gains include 3.17/1.75 in AP3D and 2.72/1.08 in APBEV over image-only methods on easy/moderate subsets, plus 1.59/0.93 in AP3D and 0.45/0.79 in APBEV over depth-based methods.
- 4.5. Cross-Dataset Test: When trained on KITTI and tested on nuScenes images with different camera intrinsics, MonoRCNN achieves lower distance-prediction errors, with better generalization at longer distances.The authors attribute this to reasoning from local geometric variables of objects.
5. Conclusion
The method uses geometry-based distance decomposition to make monocular 3D distance prediction interpretable, accurate, and robust, while directly predicting 3D boxes efficiently from RGB images.
- The proposed geometry-based distance decomposition makes distance prediction interpretable, accurate, and robust.
- The compact architecture directly predicts 3D bounding boxes from RGB images, making training and inference simple and efficient.
- The method achieves state-of-the-art performance on KITTI monocular 3D object detection and bird’s-eye-view tasks.
- The method generalizes to images with different camera intrinsics.