Delving into Localization Errors for Monocular 3D Object Detection

• The diagnostic experiments progressively replace predictions with ground truth and evaluate performance by range to quantify subtask impacts and investigate localization-error causes.The paper characterizes its strategies as operating on annotations, training samples, and optimization losses.Source: §1 · pp. 1–2
• Using ground-truth location raises performance to a level similar to state-of-the-art LiDAR methods, identifying localization error as the key constraint; the work revisits 2D-box/projected-center misalignment.It also retains 2D detection branches as an auxiliary task, unlike prior work that discards 2D detection [24].Source: §1 · pp. 1–2
• Because distant objects are nearly impossible to localize accurately and ‘bad’ samples create a domain gap, the paper proposes removing distant training samples or reducing their loss weights.Source: §1 · p. 2
• Even with ground-truth location, dimension estimation leaves 27.4% room for improvement; an IoU-oriented size loss dynamically weights each side by its sample-level contribution to 3D IoU.Source: §1 · p. 2
• At least 1.6 points AP40 improvements on bird’s view detection and 3D object detection are reported on KITTI versus existing best-performing monocular methods.Source: §1 · p. 2

Standard monocular 3D detection

• Standard monocular 3D detectors use KITTI RGB images, annotations, and calibration, with geometric constraints, orthographic features, joint 2D/3D losses, or 3D-aware proposals [28] [33] [30] [36] [2] [44].The paper describes these works as orthogonal to its localization-error analysis and proposed strategies.Source: §2 · p. 2

Monocular 3D detection using additional data

• Additional-data methods use CAD templates [27], keypoint labels, estimated depth [12], pseudo-LiDAR [38], PatchNet’s image representation [25], or video cues [3].In contrast, this method uses no extra data or annotation and achieves better or competitive performance.Source: §2 · p. 2

Misalignment between the definitions of object’s center

• 2D-box-based methods derive 3D position from the 2D center [44] or regress an offset [10] [36], while SMOKE [24] predicts projected 3D centers directly.The paper reports that replacing the 2D center with the projected 3D center improves localization accuracy, while 2D detection helps learn shared features for 3D detection.Source: §2 · pp. 2–3

3.1. Problem Definition

• The detector represents each object with its category, image-plane 2D box, and world-space 3D box, including center, dimensions, and heading.Source: §3.1 · p. 3

3.2. Baseline Model

• The CenterNet [44] baseline uses a DLA-34 backbone and lightweight heads for 2D and 3D predictions, with L1 regression, uncertainty-based depth, multi-bin heading, and a proposed IoU size loss.The baseline uses L1 loss for 3D size; the proposed model replaces it with IoU-oriented loss.Source: §3.2 · p. 3

3.3. Error Analysis

• Replacing the predicted projected 3D center raises moderate KITTI-val AP40 from 11.12% to 18.97%, while correcting depth raises it to 38.01%.The analysis attributes low monocular 3D detection accuracy mainly to localization error; at 60 meters, a 4–8-pixel image shift can prevent correct detection.Source: §3.3 · pp. 3–4, Table 1 · p. 3

3.4. Revisiting Center Detection

• The method supervises the coarse center with the projected 3D center, while retaining 2D detection branches to learn offsets and size cues that support 3D-aware features.Unlike methods that use the 2D box center as the coarse-center label, this ties that prediction to 3D geometry; 2D detection also supplies perspective cues for depth.Source: §3.4 · pp. 4–5

3.5. Training Samples

• To reduce misleading hard cases, training either excludes objects beyond 60 meters or applies reverse sigmoid-like weights, with both schemes reported as similarly effective.The soft-weight scheme defaults to c = 60 and T = 1; its parameters control the symmetry center and curve steepness.Source: §3.5 · p. 5

3.6. IoU Oriented Optimization

• The 3D size loss weights each dimension by its partial derivative with respect to 3D IoU, assuming all other predicted quantities are correct.A dynamically computed compensation weight keeps the modified loss magnitude comparable to standard L1 size loss.Source: §3.6 · p. 5

3.7. Implementation

• Training runs end-to-end for 140 epochs on two GTX 1080Ti GPUs, and inference uses center-based NMS before discarding predictions below confidence 0.2.Training uses Adam, random cropping/scaling for 2D detection, and horizontal flipping; inference decodes 2D/3D boxes from parallel decoders.Source: §3.7 · p. 5

4.1. Setup

• The experiments use KITTI, with 3,712 training and 3,769 validation images for ablations, and report final results from training on all 7,481 training images.They report AP40 for 3D, BEV, and AOS under easy, moderate, and hard settings, focusing on Cars and also reporting Pedestrians and Cyclists.Source: §4.1 · p. 6

4.2. Main Results

• On KITTI test, the method improves Car 3D AP40 by 2.47/2.27/1.64 on easy/moderate/hard and reaches 18.89 BEV and 90.23 AOS AP40 at moderate, gains of 4.06/4.12.The comparison is under fair conditions; the method also achieves comparable performance to methods using extra data.Source: §4.2 · p. 6
• On KITTI validation, excluding methods using DORN-pretrained depth estimators because their training data overlaps the KITTI 3D validation set, the proposed model outperforms all previous methods in 3D detection.For BEV detection, it outperforms all methods except MonoPair, performing better at 0.7 IoU while MonoPair is slightly better at 0.5 IoU; its validation and test performance is more consistent. On a single GTX 1080Ti GPU with batch size 1, it runs at 25 FPS, 4× faster than two-stage M3D-RPN and saving 16 ms per image versus MonoPair; SMOKE is faster but performs only 3D detection, whereas the proposed method jointly performs 2D and 3D detection.Source: §4.2 · pp. 6–7

4.3. Pedestrian/Cyclist Detection

• On KITTI test, the method performs better than [2] and comparably to [10] for Pedestrian/Cyclist detection, though results may fluctuate because these categories have few training samples.Only [2] [10] report test-set performance for these categories.Source: §4.3 · p. 7, Table 5 · p. 7

4.4. Analysis

• Projected-3D-center supervision improves 3D detection accuracy by 1.5; the IoU loss adds 0.3, and discarding distant samples adds 0.7.These experiments accumulate the proposed designs for the task.Source: §4.4 · p. 7, Table 6 · p. 7
• Correcting the 2D/projected-3D center misalignment and then supervising coarse detection with the projected 3D center improves 3D detection; joint 2D/3D learning also benefits 3D detection.The 2D detection accuracy is also better when using the proposed joint-learning setting than when generating 2D boxes from 3D detection.Source: §4.4 · p. 7, Table 7 · p. 7
• Removing or downweighting distant training samples improves overall performance; samples beyond 60/65 meters are 1,301/767, just 4.5%/2.7% of 28,742 samples.The paper reports these distant samples are only a small part of the training set and do not affect its overall representation learning.Source: §4.4 · pp. 7–8, Table 8 · p. 8

4.5. Qualitative Results

• Qualitative results show accurate 3D boxes at reasonable distances, but false positives mainly arise from imprecise depth or center estimates, while dimension and orientation estimates remain accurate.The visualizations show LiDAR signals for clarity; the proposed model runs at 25 FPS.Source: §4.5 · p. 8, Figure 5 · p. 8

5. Conclusion

• The analysis identifies localization error as the bottleneck in monocular 3D detection.The paper argues that directly detecting the projected 3D center can reduce localization error, while 2D detection helps optimize 3D detection.Source: §5 · p. 8
• Discarding distant training samples that existing technologies cannot accurately detect can prevent them from distracting the network.The paper finds accurate detection of these distant samples is almost impossible with existing technologies.Source: §5 · p. 8
• The paper proposes an IoU-oriented loss for 3D size estimation, and KITTI experiments show the proposed strategies are effective.Source: §5 · p. 8

A. Overview

• Section B provides additional implementation details for the depth-estimation sub-task.Source: A. Overview · p. 11

C. IoU Oriented Loss

• Section C presents details and ablations of the proposed IoU oriented loss.Source: A. Overview · p. 11

Supplementary Material

• Section D provides discussion omitted from the main paper.Source: A. Overview · p. 11
• Section E presents additional visual results.The supplement as a whole provides additional technical details, experimental results, theoretical analysis, and qualitative results.Source: A. Overview · p. 11

B. Depth Estimation

• The depth-estimation model simultaneously predicts depth d and standard deviation σ (or variance σ2), following [10].Source: B. Depth Estimation · p. 11
• The supplementary material models depth uncertainty with a Laplace likelihood and also gives a Gaussian-likelihood loss formulation.Source: B. Depth Estimation · p. 11
• Uncertainty-based depth estimation improves depth-map accuracy and overall monocular 3D detection performance; Laplace modeling is more suitable than Gaussian modeling for this task.These findings are reported with reference to Figure 6 and Table 9; the passage provides no numerical values.Source: B. Depth Estimation · p. 11
• Uncertainty modeling is not claimed as a contribution of this work.Source: B. Depth Estimation · p. 11

• Assuming all predictions except 3D size are correct, the proposition defines each predicted side’s contribution to 3D IoU.The proof applies this contribution ratio to the IoU-oriented optimization described in the main paper.Source: §C.1 · p. 11
• For uniformly undersized or oversized predictions, the proof derives 3D IoU and its partial derivatives with respect to each size dimension.It states that the all-oversized case yields the same conclusion as the all-undersized case.Source: §C.1 · pp. 11–12
• For mixed over- and under-estimated dimensions, the proof formulates 3D IoU and derives each side’s contribution ratio using its partial derivatives.It covers cases with one oversized and two undersized dimensions, and with two oversized and one undersized dimension.Source: §C.1 · p. 12
• When predicted height, width, and length are approximately their ground-truth values, the proof recovers Equation 5 used in the main paper.Source: §C.1 · p. 12

C.2. Experiments

• For Car AP40 under the moderate setting on KITTI validation, the proposed IoU-oriented optimization improves accuracy, whereas 3D GIoU loss does not across tested weights.The 3D GIoU loss was added as a regularization item with different weights in the baseline model.Source: Table 10 · p. 12, §C.2 · p. 12

D. Performance for the Close Objects

• Very-close objects tend to have greater center misalignment, which contributes to their lower performance.The paper points to Figure 3 for the misalignment statistics.Source: D. Performance for the Close Objects · p. 12
• Closer-range objects are usually more truncated, adding hard cases to detection.Examples include a red car at depth 3.7 with truncation 0.88 and a black car at depth 6.2 with truncation 0.34.Source: D. Performance for the Close Objects · p. 12
• The KITTI trainval set has 5,979 cars in [5m, 15m] and 6,707 cars in [10m, 20m].After excluding samples tagged ‘un-Known,’ the effective counts are 4,522 and 6,149, respectively.Source: D. Performance for the Close Objects · p. 12
• The paper attributes low performance for very-close objects to limited training samples and a large proportion of hard cases.Hard cases arise from greater center misalignment and truncation.Source: D. Performance for the Close Objects · p. 12
• KITTI difficulty labels reflect 2D bounding-box size, occlusion, and truncation.Instances tagged ‘un-Known’ are usually extremely difficult to detect and are ignored in evaluation.Source: D. Performance for the Close Objects · p. 12
• The close-range sample distribution is summarized in Table 11.The paper gives counts for the [5m, 15m] and [10m, 20m] ranges.Source: D. Performance for the Close Objects · p. 12

E.1. Learned features

• Using the projected 3D center as ground truth makes coarse center detection more accurate and improves localization.The qualitative results compare features supervised by the 2D center with those supervised by the projected 3D center.Source: §E.1 · p. 12, Figure 7 · p. 13
• The 2D bounding-box center and projected 3D object center are misaligned, especially for close objects.The paper presents this misalignment as the motivation for its proposed solution.Source: §E.1 · p. 12

Visualizations in the image plane

• Image-plane visualizations show the proposed method performing better than M3D-RPN for dense objects.Figure 8 compares their 3D bounding boxes on the KITTI validation set.Source: §E.2 · p. 13, Figure 8 · p. 14

Visualizations in the 3D world space

• 3D world-space visualizations show better results for the proposed method than M3D-RPN, especially in orientation estimation.Figure 9 presents the 3D boxes alongside the corresponding 2D image for reference.Source: §E.2 · p. 13, Figure 9 · p. 14

Representative failure case

• A representative failure occurs when inaccurate depth makes a tightly image-fitted projected 3D box a false positive.The paper notes that this error pattern is common in monocular 3D detection and identifies depth estimation as a key limiting factor.Source: §E.2 · p. 13, Figure 10 · p. 15