PaperlayerView exact revision on arXiv

Source-linked AI summary

Ground-aware Monocular 3D Object Detection for Autonomous Driving

Yuxuan Liu, Yuan Yixuan, Ming Liu

arXiv:2102.00690v1cs.CVcs.RO

TL;DR

Monocular 3D detection is limited when accurate car models are unavailable for autonomous-driving scenes. This paper explicitly reasons over ground-plane priors through anchor processing and a ground-aware convolution module, achieving state-of-the-art KITTI detection and depth-prediction performance.

  • Problem

    Monocular 3D detectors are limited in autonomous-driving scenes because accurate car models for each vehicle are unavailable.

  • Method

    The paper injects ground-plane priors through anchor filtering and a ground-aware convolution module that incorporates prior depth and features from pixels below.

  • Results

    The proposed networks achieve state-of-the-art performance on KITTI 3D object detection and depth prediction benchmarks.

  • Takeaways & Limitations

    Ground-plane priors provide useful information for 3D reasoning from images in urban scenes and support both detection and depth prediction.

  • Takeaways & Limitations

    The floor-wall assumption applies only to scenes with specific camera poses and holds only partially in complex driving scenes.

Abstract

from arXiv · show

Estimating the 3D position and orientation of objects in the environment with a single RGB camera is a critical and challenging task for low-cost urban autonomous driving and mobile robots. Most of the existing algorithms are based on the geometric constraints in 2D-3D correspondence, which stems from generic 6D object pose estimation. We first identify how the ground plane provides additional clues in depth reasoning in 3D detection in driving scenes. Based on this observation, we then improve the processing of 3D anchors and introduce a novel neural network module to fully utilize such application-specific priors in the framework of deep learning. Finally, we introduce an efficient neural network embedded with the proposed module for 3D object detection. We further verify the power of the proposed module with a neural network designed for monocular depth prediction. The two proposed networks achieve state-of-the-art performances on the KITTI 3D object detection and depth prediction benchmarks, respectively. The code will be published in https://www.github.com/Owen-Liuyuxuan/visualDet3D

I. INTRODUCTION

Monocular 3D detection is difficult because a single image lacks direct depth, while driving scenes provide ground-plane priors that can support geometric reasoning. The paper uses these priors through anchor filtering and a ground-aware convolution module, yielding an efficient detector and competitive KITTI results.

  • Motivation: Single-camera 3D estimation is ill-posed, although monocular systems are cheaper and more versatile than lidar or stereo setups.Lidar and stereo methods obtain depth or distance information directly through measurements or triangulation.
  • Motivation: Ground-plane assumptions provide additional geometric information because important dynamic objects generally lie on the ground and cameras are mounted above it.These perspective priors are specific to driving scenes and are absent from general monocular 6D pose-estimation settings.
  • Method: The framework filters 3D anchors far from the ground during training and testing, focusing predictions on likely object locations.The procedure back-projects anchors into 3D using their prior camera distance before applying the ground constraint.
  • Method: The ground-aware convolution module encodes per-pixel prior depth and guides feature aggregation toward pixels below each location.This supports the sequence of identifying object–ground contact points, computing their 3D positions, and gathering downward-focused context.
  • Results: The proposed one-stage framework runs at about 20 FPS on a modern GPU and achieves state-of-the-art performance on KITTI 3D detection and depth prediction.The module is also incorporated into a U-Net-based monocular depth-prediction network.

II. RELATED WORKS

Pseudo-LiDAR methods reconstruct point clouds from monocular or stereo images, but their pipelines depend on depth-prediction networks that limit inference speed.

  • Pseudo-LiDAR: Pseudo-LiDAR reconstructs point clouds from mono or stereo images using off-the-shelf depth-prediction networks.These methods have driven recent advances in 3D detection.
  • Pseudo-LiDAR: Current state-of-the-art monocular depth-prediction networks generally take about 0.05s per frame, limiting pseudo-LiDAR detection-pipeline speed.The passage identifies depth inference time as the bottleneck for these pipelines.

B. One-Stage Detection for Monocular 3D Object Detection

One-stage monocular 3D detectors use direct prediction or optimization strategies, while anchor-based methods embed useful geometric priors. This paper explicitly applies ground-plane reasoning in an efficient anchor-based framework and extends the module to depth prediction.

  • One-stage detection: Recent one-stage methods directly regress 3D bounding boxes, with optimization-based approaches combining predictions for depth, orientation, dimensions, and 3D corners.Examples include SS3D, Shift-RCNN, SMOKE, and RTM3D.
  • Prior methods: M3D-RPN and D4LCN use 3D or depth-related priors, but their ground-plane use is implicit and their convolution techniques are costly.M3D-RPN emphasizes the ground-plane hypothesis, whereas D4LCN introduces depth-guided convolution.
  • Anchor-based design: Anchor-based detectors can encode the relationship between apparent object size and distance during anchor preprocessing, unlike anchor-free detectors.The paper therefore develops its framework on an anchor-based detector.
  • Contribution: The proposed framework is presented as the first 3D anchor-based method to explicitly use the ground-plane hypothesis in monocular driving-scene detection.The authors report state-of-the-art performance at the time of writing.
  • Network structure: The network extracts image features, predicts classification and regression tensors, filters ground-inconsistent anchors, and produces final bounding boxes.The same ground-aware module is also relevant to monocular depth prediction, where existing multi-dataset methods often lack camera-extrinsic and absolute-scale priors.

A. Anchors Preprocessing

The method preprocesses anchors with object-size-dependent 3D statistics and ground-plane filtering, then uses perspective-derived depth priors and ground-aware convolution to query features below object centers.

  • Anchor statistics: Anchors are treated as 3D proposal distributions with shape-specific depth means and variances collected from training objects with high IoU.
  • Anchor filtering: Ground-plane back-projection filters anchors whose 3D positions are too far from the ground, focusing training and testing on likely object regions.
  • Depth priors: Perspective geometry converts image pixels into ground-plane depth priors using camera calibration and the assumed camera elevation.
  • Ground-aware convolution: Ground-aware convolution predicts vertical offsets and samples features and depth priors from pixels below each object-center feature.
  • Ground-aware convolution: The differentiable module merges sampled features through a residual connection, leaving the network to predict depth residuals relative to the geometric priors.

C. Network Architecture for Monocular 3D Detection

The detector uses a ResNet-101 backbone with separate classification and regression branches, embedding ground-aware convolution in the regression path for anchor-level 3D predictions.

  • Backbone and branches: ResNet-101 features at scale 1/16 feed separate classification and regression branches.
  • Backbone and branches: The classification branch uses two convolutional layers, while the regression branch places ground-aware convolution before its output layer.
  • Outputs: The regression output has shape (B, W/16, H/16, 12 ∗#anchors) and predicts box, center, dimension, and observation-angle parameters per anchor.

1) Loss Functions:

Training combines objectness classification with regression losses for the remaining 3D detection parameters.

  • The total loss is L = Lcls + Lreg, combining classification and regression objectives.
  • Focal loss handles objectness, cross-entropy handles multi-bin dimensions, and smoothed-L1 loss optimizes the remaining normalized parameters.

2) Post Optimization:

A hillclimbing post-optimization adjusts the observation angle using projected 3D-to-2D box overlap, while experiments indicate that optimizing depth simultaneously can hurt validation accuracy.

  • Hillclimbing perturbs observation angle and depth to maximize IoU between the estimated 2D box and the projected 3D box.
  • Optimizing only observation angle performed better on validation than jointly optimizing angle and depth, which could overfit sparse 3D-2D constraints.

D. Network Architecture for Monocular Depth Prediction

The depth prediction network uses a U-Net with a pretrained ResNet-34 encoder and inserts ground-aware convolution into the decoder. It predicts log depth densely and combines scale-invariant and smoothness losses.

  • A U-Net structure with a pretrained ResNet-34 backbone is used for supervised dense depth prediction.
  • The decoder bilinearly upsamples features, applies convolution layers, concatenates skip connections, and inserts ground-aware convolution beforehand.
  • The network predicts logarithmic depth y = log z for every image pixel with an output tensor shaped (B, 1, H, W).
  • Training uses scale-invariant loss together with smoothness loss because KITTI supervision is sparse and lacks local consistency.
  • The KITTI detection experiments use 7,481 training frames and 7,518 test frames, with Chen’s split providing 3,712 training and 3,769 validation frames.
  • The final network is retrained on the entire training set after hyperparameter selection on Chen’s split, while ablations use its validation set.

B. Evaluation Metric and Results for 3D Detection

The evaluation reports KITTI 3D detection results using AP40 and includes qualitative predictions, failure cases, and depth-prediction outcomes. The proposed network outperforms existing methods on easy and moderate vehicles, while small or obscured objects remain difficult.

  • Results are reported with AP40, which uses 40 recall positions to avoid problematic lowest-recall-bin behavior associated with AP11.
  • The proposed network significantly outperforms existing monocular methods on easy and moderate vehicles.
  • Qualitative validation examples show mostly accurate car detection and estimation, alongside visualized effects of the GAC module.
  • Failure cases include missed heavily obscured, truncated, and distant cars, plus inaccurate 3D dimension estimates.
  • The depth network produces one of the best KITTI performances, improves significantly over the baseline U-Net, and remains generally consistent within LiDAR range.

V. MODEL ANALYSIS AND DISCUSSION

The analysis evaluates anchor filtering and ground-aware convolution through ablations. Anchor filtering improves 3D inference while leaving 2D detection nearly unchanged relative to the compared alternatives.

  • The ablation studies assess the contributions of anchor preprocessing and the ground-aware convolution module.
  • Anchor-filtering experiments compare the full model with no filtering and with Online Hard Example Mining after removing half of the easy negative anchors.
  • The baseline model performs better for 3D inference than both the ablated model and OHEM, with almost no difference in 2D detection.
  • Filtering off-ground anchors reduces the classification branch’s learning burden and lets it focus on selecting suitable foreground anchors.

B. Ground-Aware Convolution Module

The ground-aware convolution module adapts feature processing using geometric priors, outperforming alternative convolution designs while supporting the paper’s KITTI results and stated scope.

  • The ground-aware convolution module fixes the search direction and permits a larger search range than deformable convolutions.This design explicitly adapts feature aggregation to the ground-aware setting.
  • The proposed module achieves better results by a considerable margin than basic, disparity-conditioned, and deformable convolution alternatives.Deformable convolutions improve 2D detection, while disparity-conditioned convolutions add depth priors but lack sufficient receptive-field coverage.
  • The module provides geometric priors for ground-plane reasoning in the proposed monocular 3D detection network.The authors report testing the resulting network on the KITTI detection benchmark.
  • The paper also evaluates the module for monocular depth prediction, where it produces competitive results on the KITTI depth prediction benchmark.
  • The ground-aware approach does not explicitly reason from ground and object boundaries, instead encoding priors into a data-driven network.The authors identify this as a limitation of the proposed methods.
Loading 2102.00690v1…