PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection

Di Feng, Lars Rosenbaum, Klaus Dietmayer

arXiv:1804.05132v2cs.ROcs.CV

TL;DR

Deep-learning detectors do not explicitly represent uncertainty, limiting their ability to communicate confidence for safe autonomous driving. This paper develops a probabilistic Lidar 3D vehicle detector that models epistemic and aleatoric uncertainty, finding distinct behavioral patterns and a 1%–5% detection improvement from aleatoric modeling.

  • Problem

    Deep-learning object detectors do not explicitly estimate uncertainty in classification and bounding-box regression, although prediction confidence matters for safe autonomous driving.

  • Method

    The paper uses Bayesian dropout sampling to capture epistemic uncertainty and an auxiliary output layer to model aleatoric uncertainty in Lidar vehicle recognition and 3D box regression.

  • Results

    Epistemic uncertainty relates to detection accuracy, whereas aleatoric uncertainty is influenced by vehicle distance and occlusion; modeling aleatoric uncertainty improves detection performance by 1%–5%.

  • Takeaways & Limitations

    Epistemic uncertainty indicates detector limitations, while aleatoric uncertainty reflects sensor limitations and can support vehicle tracking and online deployment.

  • Takeaways & Limitations

    Extracting epistemic uncertainty requires multiple dropout feed-forward passes, costing 0.3 fps on a Titan X GPU and making it infeasible for online autonomous driving.

Abstract

from arXiv · show

To assure that an autonomous car is driving safely on public roads, its object detection module should not only work correctly, but show its prediction confidence as well. Previous object detectors driven by deep learning do not explicitly model uncertainties in the neural network. We tackle with this problem by presenting practical methods to capture uncertainties in a 3D vehicle detector for Lidar point clouds. The proposed probabilistic detector represents reliable epistemic uncertainty and aleatoric uncertainty in classification and localization tasks. Experimental results show that the epistemic uncertainty is related to the detection accuracy, whereas the aleatoric uncertainty is influenced by vehicle distance and occlusion. The results also show that we can improve the detection performance by 1%-5% by modeling the aleatoric uncertainty.

I. INTRODUCTION

The paper addresses the lack of explicit uncertainty estimates in deep-learning object detectors by modeling both model and sensor uncertainty for Lidar 3D vehicle detection. It relates epistemic uncertainty to detection accuracy and aleatoric uncertainty to distance and occlusion.

  • Motivation: Deep-learning object detectors generally report detections without explicitly estimating uncertainty in classification or bounding-box regression.Softmax scores do not necessarily represent classification uncertainty.
  • Uncertainty types: Epistemic uncertainty reflects uncertainty in explaining the observed dataset, whereas aleatoric uncertainty captures noise inherent in sensors.Abnormal objects can produce high epistemic uncertainty, while distant objects can produce high aleatoric uncertainty.
  • Approach: The proposed probabilistic detector predicts object classes, 3D bounding boxes, model uncertainty, and sensor observation uncertainty from Lidar point clouds.Shannon Entropy and Mutual Information quantify classification uncertainty, while Total Variance quantifies localization uncertainty.
  • Contributions: The methods extract model and observation uncertainty for vehicle recognition and 3D bounding-box regression.
  • Contributions: Modeling aleatoric uncertainty improves vehicle detection performance.
  • Contributions: Epistemic uncertainty is associated with detection accuracy, while aleatoric uncertainty is influenced by vehicle distance and occlusion.

II. RELATED WORKS

Related work covers Lidar and multimodal 3D object detection alongside Bayesian neural networks for uncertainty estimation. Existing approaches include voxelized point-cloud representations, sensor fusion, and probabilistic weight modeling.

  • 3D object detection: Prior work applies deep learning to Lidar-based and multimodal 3D object detection in autonomous driving.The paper situates its method among camera, Lidar, and combined-sensor detectors.
  • Lidar representations: Voxel-based methods discretize point clouds into grids, learn voxel features, or use voxel-derived features for region proposal generation.
  • Bayesian neural networks: Bayesian neural networks place distributions over weights and infer posterior distributions to estimate predictive uncertainty.Related uncertainty methods include variational inference and sampling techniques.
  • Bayesian neural networks: Dropout sampling has been applied to uncertainty estimation in active learning, cancer diagnosis, semantic segmentation, and open-set image object detection.

III. NETWORK ARCHITECTURE

The detector converts Lidar point clouds into bird’s-eye-view features, generates 3D region proposals, and processes ROI features for objectness and bounding-box regression. Intermediate and fully connected layers support uncertainty extraction.

  • Detection pipeline: A region proposal network generates ROI candidates from Lidar BEV features, which intermediate layers process before objectness scoring and 3D box regression.
  • Input representation: The network projects 3D point clouds onto a 0.1 m 2D grid and encodes height, intensity, and density maps as input features.Height maps divide the point cloud into M slices, producing M + 2 feature channels.
  • Bounding-box regression: The detector encodes oriented 3D bounding boxes using 24 normalized corner offsets relative to proposal anchors.
  • Backbone: The ResNet-8 backbone uses four residual blocks with kernels increasing from 64 to 512, then upsamples features by a factor of two.This supports detection of small vehicles occupying 5–40 grid cells.
  • Region proposals: Following the Faster-RCNN pipeline, the network generates nine 2D anchors per feature-map pixel and projects them into 3D using a fixed height.
  • Intermediate layers: Three fully connected hidden layers with 512 units and following dropout layers further process Lidar features and extract uncertainties.

IV. CAPTURING UNCERTAINTY IN LIDAR 3D VEHICLE DETECTION

The detector captures epistemic uncertainty through Bayesian treatment of fully connected-layer weights and aleatoric uncertainty through observation-likelihood modeling. Dropout sampling approximates the weight posterior during testing.

  • Uncertainty types: The detector distinguishes epistemic uncertainty in the model from aleatoric uncertainty caused by observation noise.
  • Bayesian formulation: Bayesian inference marginalizes predictions over fully connected-layer weights to estimate the posterior predictive distribution.The weight posterior yields epistemic uncertainty, while the observation likelihood yields aleatoric uncertainty.
  • Epistemic uncertainty: Dropout approximates Bayesian inference by sampling model outputs across multiple stochastic forward passes at test time.These samples are used to obtain epistemic uncertainty for each region proposal.
  • Epistemic uncertainty: During testing, each region proposal is processed N times with dropout, producing softmax scores and regression outputs for epistemic-uncertainty measurement.

1) Extracting Vehicle Probability and Epistemic Classification Uncertainty:

The detector estimates vehicle probability by averaging softmax scores across dropout forward passes, then uses Shannon entropy and mutual information to quantify classification uncertainty.

  • Vehicle probability is approximated by the mean softmax score from N forward passes.With N = 1 and p = 0, this reduces to the point-wise softmax prediction.
  • Dropout-based multiple forward passes are used to extract more accurate model uncertainty than a single point estimate.The method requires N > 1 and p > 0 for this uncertainty estimation.
  • Shannon entropy measures uncertainty in the prediction output p(veh|x∗), ranging from 0 to ln(2).It is zero for probabilities of 0 or 1 and peaks at the most uncertain prediction.
  • Mutual information measures the information difference between the prediction probability and the posterior of model parameters.Its range is [0,1], with larger values indicating higher epistemic classification uncertainty.

2) Extracting 3D Bounding Box and Epistemic Spatial Uncertainty:

The detector estimates 3D bounding boxes in the Lidar coordinate frame by averaging regression outputs across forward passes, and quantifies epistemic spatial uncertainty from their covariance.

  • Bounding box position is estimated in the Lidar coordinate frame using the mean regression output from N forward passes.The bounding box prediction is first transformed into that coordinate frame.
  • Epistemic spatial uncertainty is estimated from the total variance of the covariance matrix across N forward-pass regressions.The total variance is computed as the trace of the covariance matrix.
  • A larger total-variance score indicates higher epistemic spatial uncertainty.The score ranges from [0,+∞).

B. Capturing Aleatoric Uncertainty

Aleatoric uncertainty is modeled as observation noise: softmax outputs represent classification likelihoods, while diagonal-covariance Gaussian outputs represent regression noise.

  • Classification aleatoric uncertainty is modeled through the softmax function over object-label outputs.The likelihood is written as p(y∗|gW(x∗)) = softmax(gW(x∗)) = sx∗.
  • For 3D bounding box regression, the observation likelihood is modeled as a multivariate Gaussian with diagonal covariance.The network predicts observation noises for bounding-box regression outputs.
  • The regression output represents a 24-dimensional vector whose elements encode observation noise for bounding-box predictions.The noise is parameterized as λx∗ := log(σ2x∗) for numerical stability.
  • The modified regression loss reduces the contribution of data with high aleatoric uncertainty.When uncertainty is high, the residual term is downweighted because exp(−λT) becomes small.

C. Implementation

The implementation trains the detector end to end with multiple losses and uses dropout sampling at test time to isolate or combine epistemic and aleatoric uncertainty.

  • The network is trained end to end with classification and regression losses for both proposals and final object outputs.Smooth L1 and cross-entropy losses are used for oriented 3D box regression and classification, respectively.
  • Dropout with rate p = 0.5 is used for posterior-distribution approximation and over-fitting prevention.Training uses L2 regularization and Adam optimization with staged learning rates.
  • Epistemic uncertainty is extracted with N = 40 dropout forward passes during testing.A single non-dropout pass is used to extract aleatoric uncertainty, while multiple dropout passes with the modified loss extract both types.
  • Modeling aleatoric uncertainty improves vehicle detection performance, while epistemic and aleatoric uncertainties show different empirical relationships.Epistemic uncertainty is influenced by detection accuracy; aleatoric uncertainty is associated with vehicle distance and occlusion.

A. Experimental Setup

The detector was evaluated on KITTI data against variants modeling epistemic and aleatoric uncertainty, using F1 across IoU thresholds. Aleatoric modeling improved detection, while epistemic uncertainty tracked accuracy and highlighted atypical detections.

  • A. Experimental Setup: The evaluation used 9,918 training frames and 2,010 testing frames from the KITTI raw dataset.Twenty-nine drives were used for training and six for testing.
  • A. Experimental Setup: Detections with probability scores above 0.5 were evaluated using F1 scores across IoU thresholds from 0.1 to 0.8.IoU measures similarity between predicted vehicles and ground truth; higher IoU indicates greater accuracy.
  • A. Experimental Setup: 1%−5% F1 improvement was achieved by detectors modeling aleatoric uncertainty compared with the Non-Bayesian baseline.The Aleatoric and Epistemic+Aleatoric detectors consistently outperformed the baseline.
  • A. Experimental Setup: The Epistemic detector slightly underperformed the baseline, attributed to reduced network capability from dropping hidden units.The baseline did not explicitly model observation noise or the weight posterior distribution.
  • A. Experimental Setup: Shannon Entropy and Mutual Information decreased as IoU increased, associating epistemic classification uncertainty with prediction accuracy.High-IoU detections showed low classification uncertainty, whereas low-IoU detections showed higher and more dispersed uncertainty.
  • A. Experimental Setup: High epistemic uncertainty appeared for big vehicles, ghost objects, and detections with abnormal bounding boxes.These detections differed from the training data and could be used to query unseen samples during active learning.

D. Understanding the Aleatoric Uncertainty in 3D Object Detection

The study examined aleatoric spatial uncertainty in Lidar bounding-box regression across IoU, distance, and occlusion conditions. It found little relationship with detection accuracy, but higher uncertainty for distant vehicles and occluded corners.

  • D. Understanding the Aleatoric Uncertainty in 3D Object Detection: Aleatoric uncertainty was evaluated for 3D bounding-box regression, quantified by σ2, while aleatoric classification uncertainty was not explicitly modeled.The authors leave aleatoric classification uncertainty for future research.
  • D. Understanding the Aleatoric Uncertainty in 3D Object Detection: Aleatoric spatial uncertainty showed little relation to detection accuracy across IoU intervals.Total variance was computed by summing observation-noise variances along the x, y, and z axes.
  • D. Understanding the Aleatoric Uncertainty in 3D Object Detection: Aleatoric uncertainty was positively correlated with vehicle distance from the ego-vehicle.Uncertainty in the x, y, and z axes increased as vehicles moved away, while epistemic uncertainty showed no tendency.
  • D. Understanding the Aleatoric Uncertainty in 3D Object Detection: Occluded bounding-box corners consistently had higher aleatoric spatial uncertainty than corners facing the Lidar sensor.The comparison summed observation noises separately for front-facing and occluded corners.

VI. CONCLUSIONS AND DISCUSSIONS

The probabilistic Lidar detector captures epistemic and aleatoric uncertainty in vehicle recognition and 3D bounding box regression. Epistemic uncertainty relates to detection accuracy, while aleatoric uncertainty reflects sensor noise, improves performance, and is more suitable for online use; extracting epistemic uncertainty remains computationally costly.

  • Conclusions and discussions: The network captures epistemic uncertainty through repeated dropout predictions and aleatoric uncertainty through an auxiliary output layer.Both uncertainties are modeled for vehicle recognition and 3D bounding box regression.
  • Conclusions and discussions: Epistemic uncertainty is associated with detection accuracy and is high for samples unlike the training data, including ghost objects, big vehicles, and abnormal bounding box regressions.It can support active learning by identifying unseen samples for offline model improvement.
  • Conclusions and discussions: Aleatoric uncertainty is influenced by detection distance and occlusion rather than detection accuracy, reflecting higher observation noise in distant or partially occluded vehicles.It can also improve vehicle-position tracking.
  • Conclusions and discussions: 1%−5% improved detection performance resulted from modeling aleatoric uncertainty, indicating increased robustness to noisy data.Computing aleatoric uncertainty requires only one inference, supporting online deployment.
  • Conclusions and discussions: Epistemic uncertainty extraction is computationally expensive, requiring multiple dropout feed-forward passes and achieving 0.3 fps on a Titan X GPU.The authors identify the tradeoff between epistemic-uncertainty performance and the number of passes as future work.
  • Conclusions and discussions: Future work will examine uncertainty across other Lidar detection architectures and additional factors affecting aleatoric uncertainty.Planned applications include active learning and object tracking.
Loading 1804.05132v2…