Source-linked AI summary
Geometry Uncertainty Projection Network for Monocular 3D Object Detection
Yan Lu, Xinzhu Ma, Lei Yang, Tianzhu Zhang, Yating Liu, Qi Chu, Junjie Yan, Wanli Ouyang
TL;DR
Projection-based depth estimation in monocular 3D detection amplifies height-estimation errors, harming depth reliability and training stability. GUP Net combines geometry-guided uncertainty modeling with Hierarchical Task Learning, achieving state-of-the-art KITTI performance while relying on a Laplace-distribution assumption.
Problem
Projection-based monocular 3D detectors amplify errors in estimated 3D heights when inferring depth, making depth inference unreliable and training unstable.
Method
GUP Net models projected depth as a distribution with geometry-guided uncertainty and adaptively weights hierarchical task losses according to pre-task learning status.
Results
GUP Net achieves state-of-the-art KITTI testing performance of 20.11% for cars and 14.72% for pedestrians.
Takeaways & Limitations
The model provides more reliable object-depth estimates and improves training stability and efficiency within monocular 3D detection.
Takeaways & Limitations
The geometry uncertainty formulation assumes that predicted 3D height and depth distributions follow a Laplace distribution.
Abstract
from arXiv · showhide
Geometry Projection is a powerful depth estimation method in monocular 3D object detection. It estimates depth dependent on heights, which introduces mathematical priors into the deep model. But projection process also introduces the error amplification problem, in which the error of the estimated height will be amplified and reflected greatly at the output depth. This property leads to uncontrollable depth inferences and also damages the training efficiency. In this paper, we propose a Geometry Uncertainty Projection Network (GUP Net) to tackle the error amplification problem at both inference and training stages. Specifically, a GUP module is proposed to obtains the geometry-guided uncertainty of the inferred depth, which not only provides high reliable confidence for each depth but also benefits depth learning. Furthermore, at the training stage, we propose a Hierarchical Task Learning strategy to reduce the instability caused by error amplification. This learning algorithm monitors the learning situation of each task by a proposed indicator and adaptively assigns the proper loss weights for different tasks according to their pre-tasks situation. Based on that, each task starts learning only when its pre-tasks are learned well, which can significantly improve the stability and efficiency of the training process. Extensive experiments demonstrate the effectiveness of the proposed method. The overall model can infer more reliable object depth than existing methods and outperforms the state-of-the-art image-based monocular 3D detectors by 3.74% and 4.7% AP40 of the car and pedestrian categories on the KITTI benchmark.
1. Introduction
Monocular 3D detection is ill-posed because single images lack depth cues, while projection-based depth estimation amplifies 3D-height errors. GUP Net addresses inference reliability and training instability through uncertainty modeling and hierarchical task learning.
- Monocular 3D detection remains challenging because single images lack depth cues, making object-level depth estimation ill-posed.
- Projection methods infer depth as depth = h3d·f/h2d, so errors in estimated heights propagate directly into depth.
- A 0.1m 3D-height bias can cause a depth shift of up to 4m, making projection outputs difficult to control and reducing inference reliability and training efficiency.
- The GUP module models depth through a distribution derived from estimated 3D-height uncertainty, producing confidence values that indicate depth reliability.
- Hierarchical Task Learning adaptively weights losses according to pre-task learning status, so dependent tasks are trained after their prerequisites improve.
- 20.11% and 14.72% are the reported car and pedestrian 3D detection performances of GUP Net on the KITTI testing set.
2. Related works
Prior monocular 3D detection work combines learned representations, geometric priors, and uncertainty modeling, but projection-based methods overlook error amplification. Existing multi-task loss weighting also assumes task independence, unlike the hierarchical dependencies addressed here.
- Monocular 3D detectors use deep representation learning and geometry priors to predict 3D bounding boxes from a single image.
- Several methods introduce geometry projection for depth or center reasoning, but they do not account for error amplification.
- Uncertainty-based depth methods model regression uncertainty but generally neglect the relationship between object height and depth.
- Multi-task learning studies include task-relation modeling and adaptive loss weighting methods such as GradNorm and task-uncertainty strategies.
- Existing loss-weighting methods assume task independence, whereas the proposed framework contains hierarchical dependencies among its tasks.
3. Geometry Uncertainty Projection Network
GUP Net models geometry-based depth uncertainty and schedules hierarchical task learning to improve inference reliability and training stability. Its task hierarchy progresses from 2D detection to 3D heads and finally depth inference.
- Network overview: GUP Net processes images through 2D detection, basic 3D heads, geometry-based depth inference, and final 3D bounding-box construction.The depth stage uses estimated 2D and 3D heights from earlier stages.
- Geometry Uncertainty Projection: The GUP module predicts a depth distribution by combining mathematical projection priors with uncertainty modeling instead of regressing a single depth value.This produces an inferred depth and theoretical uncertainty for assessing reliability and supporting depth learning.
- Geometry Uncertainty Projection: The 3D height prediction is modeled as a Laplace distribution whose parameters are learned by the 3D size stream, with uncertainty reflecting difficult or noisy samples.The learned height distribution is projected into a depth distribution, and a learned bias distribution modifies the initial projection.
- Geometry Uncertainty Projection: Geometry based Uncertainty, denoted GeU, combines projection uncertainty with bias-learning uncertainty and is optimized through uncertainty regression.The final uncertainty reflects uncertainty from both the projected height and learned depth bias.
- Inference confidence: During inference, GeU is mapped to a 0–1 Uncertainty-Confidence score and combined with 2D confidence to form conditional 3D bounding-box scores.The resulting score represents both 2D detection confidence and depth inference confidence.
- Hierarchical Task Learning: Hierarchical Task Learning controls epoch-specific loss weights so tasks begin training after their prerequisite tasks have learned sufficiently.The hierarchy places 2D detection before 3D heads, and 3D size plus 2D tasks before depth inference.
- Hierarchical Task Learning: HTL increases each task’s loss weight from 0 to 1 using polynomial scheduling, with the adjust parameter determined by pre-task learning indicators.The indicator compares recent loss-trend derivatives with the initial training trend and approaches 1 as a task tends to converge.
4. Experiments
Experiments on KITTI show that GUP Net improves monocular 3D detection across car, pedestrian, and high-precision validation settings while maintaining efficient inference. Ablations and qualitative results support the contributions of coordinate maps, hierarchical task learning, and uncertainty modeling.
- Main Results: 3.74%, 3.19%, and 2.25% gains are achieved for car detection on easy, moderate, and hard KITTI test settings, respectively.The method also exceeds CaDNN, which uses LiDAR supervision, by 0.94%, 0.79%, and 0.31% on these settings.
- Main Results: 4.16%/4.77% improvements over MonoPair are obtained for moderate car 3D/BEV detection at the 0.7 IoU threshold on KITTI validation.At the 0.5 IoU threshold, performance is nearly the same as MonoPair.
- Main Results: GUP Net outperforms all competing methods across pedestrian difficulty levels and ranks first among methods without extra data for cyclist detection.MonoPSR and CaDNN remain stronger for cyclist detection when extra LiDAR-derived depth supervision is available.
- Main Results: 29.4 FPS is achieved on a single Nvidia TiTan XP GPU, indicating an efficient inference pipeline.The reported runtime is averaged over the tested system.
- Ablation Study: Coordinate maps improve ablation performance with negligible additional computing overhead by supplying location and size cues to RoI features.The comparison is reported for the KITTI validation set car category.
- Ablation Study: HTL increases the third depth-inference-stage loss weight after its pre-tasks have made progress, including heatmap, 2D offset, 2D size, and 3D size.The loss weight begins increasing at about the 11th epoch.
- Ablation Study: HTL achieves the best performance among compared loss-weight controllers because the model's tasks form a hierarchy rather than an independent multitask structure.GradNorm is additionally affected by changing loss magnitudes caused by error amplification.
- Qualitative Results: GUP Net assigns high uncertainty to bad cases such as occlusion and far distance, whereas the baseline projection model can assign similarly low uncertainty to such cases.The method's uncertainty generally decreases as prediction quality improves.
5. Conclusion
The conclusion presents GUP Net as a monocular 3D detector that addresses error amplification in geometry projection through reliable uncertainty computation and hierarchical task learning. Experiments validate its superior performance and the effectiveness of its components.
- Conclusion: GUP Net combines mathematical projection priors with deep regression to compute more reliable object uncertainty for learning and testing confidence.The model is designed to address error amplification in conventional geometry projection models.
- Conclusion: Hierarchical Task Learning reduces training instability caused by error amplification and improves learning of the overall model.The conclusion reports extensive experimental validation of this strategy and the model's components.