PaperlayerView exact revision on arXiv

Source-linked AI summary

Triangulation Learning Network: from Monocular to Stereo 3D Object Detection

Zengyi Qin, Jinglu Wang, Yan Lu

arXiv:1906.01193v1cs.CV

TL;DR

Stereo 3D detection must exploit image-pair information beyond pixel-level depth or disparity maps. The paper combines a monocular baseline with anchor-based object-level triangulation and channel reweighting, reporting strong KITTI detection results.

  • Problem

    Monocular 3D detection is ill-posed, and prior stereo-learning approaches mainly use pixel-level information rather than object-level correspondences.

    Source: §1 · p. 1

  • Method

    TLNet uses 3D anchors to form correspondences between stereo-image regions of interest and triangulate objects, with channel reweighting integrated into a monocular baseline.

    Source: §1 · pp. 1–2, §5 · p. 8

  • Results

    TLNet outperforms the monocular baseline by a significant margin on KITTI, while the monocular baseline is comparable to a state-of-the-art stereo counterpart.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The integrated detector uses stereo geometry without pixel-level depth maps and achieves state-of-the-art performance.

    Source: §1 · p. 1, §5 · p. 8

Abstract

from arXiv · show

In this paper, we study the problem of 3D object detection from stereo images, in which the key challenge is how to effectively utilize stereo information. Different from previous methods using pixel-level depth maps, we propose employing 3D anchors to explicitly construct object-level correspondences between the regions of interest in stereo images, from which the deep neural network learns to detect and triangulate the targeted object in 3D space. We also introduce a cost-efficient channel reweighting strategy that enhances representational features and weakens noisy signals to facilitate the learning process. All of these are flexibly integrated into a solid baseline detector that uses monocular images. We demonstrate that both the monocular baseline and the stereo triangulation learning network outperform the prior state-of-the-arts in 3D object detection and localization on the challenging KITTI dataset.

1. Introduction

The introduction frames stereo 3D detection as a way to exploit image-pair geometry at object level, and presents TLNet and channel reweighting alongside a monocular baseline.

1. Introduction

  • Passive imagery offers a low-cost alternative to active 3D scans, which are costly and limited in scalability.Source: §1 · p. 1
  • Monocular 3D detection is ill-posed, while prior deep stereo-learning work focuses mainly on pixel-level rather than object-level information.Source: §1 · p. 1
  • TLNet uses 3D anchors to construct object-level correspondences between stereo-image regions of interest and learns to triangulate the target.Source: §1 · p. 1
  • Left-right coherence reweights informative channels, filters noisy or mismatched signals, and directs attention to key object parts without added parameters.Source: §1 · p. 2
  • The monocular baseline has performance comparable to its state-of-the-art stereo counterpart.Source: §1 · p. 2
  • TLNet outperforms the monocular baseline by a significant margin on the challenging KITTI dataset.Source: §1 · p. 2

2. Related Work

Related work spans monocular and multi-view detection, pixel-level stereo learning, and geometric triangulation; the paper positions its approach at the object level, using anchors instead of pixel-level disparity.

Monocular 3D Detection

  • Prior monocular methods may require extra segmentation, disparity training data, or CAD templates, whereas the proposed baseline trains using only ground-truth 3D boxes.Source: §2 · p. 2

Multi-View based 3D Detection

  • Multi-view detectors fuse image and LiDAR features, while 3DOP relies on disparity maps, with high computational cost and imprecise estimates at distant regions.Source: §2 · p. 2

Learning based Stereo

  • Existing learning-based stereo methods mainly estimate pixel-level disparity, while this work explores instance-level learning for 3D object detection.Source: §2 · p. 2

Triangulation

  • Pixel-level feature matching for triangulation can be computationally expensive; the proposed method uses 3D anchors to localize objects through learnable triangulation in one forward pass.Source: §2 · p. 2

Front view anchor generation

  • Only high-objectness potential anchors from the front view are fed into the RPN to reduce its search space and computational cost.Source: Figure 2 · p. 3

3. Approach

The approach pairs a staged monocular 3D detector with TLNet, which uses 3D anchors to connect stereo RoIs and learn object-level triangulation. Coherence-based channel reweighting helps the network focus on corresponding, informative features; implementation details specify the backbone and staged training.

3.1. Baseline Monocular Network

  • The monocular baseline uses a three-stage pipeline—front-view anchor generation, 3D box proposal, and refinement—to progressively reduce the search space.The stages select confident anchors before proposing and refining 3D boxes.Source: §3.1 · p. 3

3.1.1 Front View Anchor Generation

  • Front-view cells predict objectness, and foreground cells generate anchors sampled along viewing rays from a pool of 3D locations and class-specific sizes.The pool samples the ground plane at 0.25 m intervals, within the view frustum and depths from 0 to 70 m; each location has two BEV orientations.Source: §3.1.1 · p. 3

3.1.2 3D Box Proposal and Refinement

  • The proposal and refinement stages crop RoI features to classify objects and regress 3D box offsets, with the refinement stage also predicting orientation.The proposal stage retains the top K=1024 proposals using NMS.Source: §3.1.2 · p. 3

3.2. Triangulation Learning Network

  • TLNet adds object-level triangulation to the monocular baseline, using 3D anchors rather than pixel-level disparity estimation.It is integrated into the baseline for stereo image pairs.Source: §3.2 · p. 3, §3.2.1 · p. 4

3.2.1 Anchor Triangulation

  • A 3D anchor projects to left and right RoIs that establish geometric correspondence; mismatch between the projections reflects differences between the anchor and target in 3D.The network uses the anchor as a reference to estimate objectness and regress offsets toward the target.Source: Figure 3 · p. 3, §3.2.1 · p. 4

3.2.2 TLNet Architecture

  • TLNet computes per-channel coherence between paired RoI features, reweights channels by that score, then fuses the features to predict objectness and 3D box offsets.The described channel scores are lower for noisy, mismatched activations and higher for clear, coherent ones.Source: §3.2.2 · pp. 4–5
  • TLNet replaces post-RoIAlign fully connected layers in the RPN and refinement stages, while shared backbone parameters and parameter-free reweighting limit additional memory burden.In refinement, the classification output is object class rather than RPN objectness confidence.Source: §3.2.2 · p. 5

3.3. Implementation Details

  • The implementation uses a shared-weight VGG-16 backbone and trains in stages, then applies a final SGD optimization phase.Training proceeds through 20K, 40K, and 60K-iteration stages, followed by 20K iterations of SGD; the batch size is 1.Source: §3.3 · p. 5

4. Experiment

On KITTI, TLNet improves stereo 3D detection, BEV detection, and 3D localization over the monocular baseline, while the monocular baseline itself performs competitively. Qualitative results and the reweighting ablation examine how these outcomes relate to distant or truncated objects and feature fusion.

4. Experiment

  • Evaluation uses Car objects on KITTI's train1/val1 split and compares monocular methods with 3DOP for stereo input.The benchmark has 7,481 training images and 7,518 test images; evaluation distinguishes easy, moderate, and hard cases by occlusion and truncation.Source: §4 · pp. 5–6

4.1. 3D Object Detection

  • TLNet improves AP3D by ∼10% over 3DOP across easy, moderate, and hard regimes, and by ∼10% over the baseline at IoU thresholds 0.3 and 0.5.It also outperforms both comparators across all 3D IoU thresholds and scenarios.Source: Table 1 · p. 6, §4.1 · p. 6

4.2. BEV Object Detection

  • TLNet raises APBEV by ∼8% over the monocular baseline at IoU thresholds 0.3 and 0.5, and surpasses 3DOP especially at IoU 0.7.The reported comparison uses oriented BEV boxes; the authors attribute improvement to the baseline's competitive performance and the added localization capability of stereo triangulation.Source: Table 2 · p. 6, §4.2 · p. 7

4.3. 3D Localization

  • At distance thresholds of 2m and 1m, TLNet gains ∼10% in APLOC over the baseline and increases both precision and recall.The table reports Average Location Precision for 3D boxes under different distance thresholds on KITTI validation.Source: Table 3 · p. 6, §4.3 · p. 8

4.1. 3D Object Detection

  • Moderate-case AP3D exceeds MF3D by 4.50% at IoU 0.5 and 4.65% at IoU 0.7; the baseline also outperforms MF3D and Mono3D.The authors suggest that proposing directly in 3D may contribute to the gap, particularly for partly unseen, occluded, or truncated objects.Source: §4.1 · p. 6, Table 1 · p. 6

4.5. Ablation Study

  • The ablation compares reweighting with concatenation and element-wise addition; reweighting dynamically scores channel coherence for each RoI pair.Shared left-right backbone weights mean corresponding RoI channels are expected to represent the same feature types, and reweighting strengthens features suited to triangulation.Source: Table 4 · p. 8, §4.5 · p. 8, Figure 9 · p. 8

4.4. Qualitative Results

  • Qualitative examples show reduced depth error for distant targets, recovered missed detections, and detection of a heavily truncated car.The figure uses orange boxes for detections and green boxes for ground truth; displayed lidar point clouds are for reference only and were not used in training or evaluation.Source: Figure 6 · p. 7, §4.4 · p. 8

5. Conclusion

The paper extends a monocular detector to stereo 3D detection by using 3D anchors to form geometric correspondences and triangulate objects. It combines this with channel reweighting, reporting state-of-the-art performance.

5. Conclusion

  • 3D anchors form correspondences between stereo projections, allowing the network to triangulate target objects in a forward pass.The stereo network flexibly extends the monocular baseline detector.Source: §5 · p. 8
  • Channel reweighting strengthens informative features and weakens noisy signals in the integrated detector.Source: §5 · p. 8
  • The integrated method achieves state-of-the-art performance in 3D object detection.Source: §5 · p. 8

Related papers

Loading 1906.01193v1…