Source-linked AI summary
Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving
Nemanja Djuric, Vladan Radosavljevic, Henggang Cui, Thi Nguyen, Fang-Chieh Chou, Tsung-Han Lin, Nitin Singh, Jeff Schneider
TL;DR
Autonomous vehicles need reliable predictions of nearby traffic actors’ future states for safe operation. The paper rasterizes current actor-centered context and uses deep convolutional models to predict short-term motion and uncertainty, with real-world evaluation suggesting accurate, well-calibrated predictions and successful onboard testing. Longer-horizon prediction remains limited by traffic multimodality.
Problem
Safe autonomous driving requires predicting future trajectories of tracked traffic actors in the vehicle’s surroundings.
Method
The method rasterizes high-definition maps and actor surroundings into images, then trains CNNs to predict short-term trajectories and their uncertainty.
Results
Large-scale real-world evaluation showed accurate predictions and well-calibrated uncertainties, followed by successful onboard testing on self-driving vehicles.
Takeaways & Limitations
Actor-centered raster context and uncertainty-aware CNN prediction provide a practically tested approach to short-term traffic-motion prediction.
Takeaways & Limitations
The unimodal model provides useful short-term predictions, but longer horizons require modeling multimodal behavior beyond the considered 3s horizon.
Abstract
from arXiv · showhide
We address one of the crucial aspects necessary for safe and efficient operations of autonomous vehicles, namely predicting future state of traffic actors in the autonomous vehicle's surroundings. We introduce a deep learning-based approach that takes into account a current world state and produces raster images of each actor's vicinity. The rasters are then used as inputs to deep convolutional models to infer future movement of actors while also accounting for and capturing inherent uncertainty of the prediction task. Extensive experiments on real-world data strongly suggest benefits of the proposed approach. Moreover, following completion of the offline tests the system was successfully tested onboard self-driving vehicles.
1. Introduction
Safe autonomous driving requires predicting tracked actors’ future trajectories amid rapidly changing, multi-actor scenes. The paper proposes rasterized scene context and CNN-based short-term prediction with uncertainty, reporting accurate predictions, calibrated uncertainties, and onboard testing.
- Traffic-actor trajectory prediction is a critical component of the self-driving pipeline, alongside detection, tracking, and safe navigation.
- The approach rasterizes high-definition maps and each vehicle’s surroundings to provide contextual inputs for trajectory prediction.
- A deep CNN predicts short-term vehicle trajectories while accounting for inherent uncertainty in road-traffic motion.
- Large-scale evaluation on real-world data found accurate predictions and well-calibrated uncertainties, indicating practical benefits.
- Following extensive offline testing, the system was successfully tested onboard self-driving vehicles.
- Figure 1 illustrates the pipeline from a 3D scene to an actor-centered bird’s-eye raster and overlaid 3-second trajectories.
2. Related work
Prior motion-prediction systems include engineered kinematic, map-constrained, and maneuver-based models, as well as learned methods for scene context and interactions. The paper positions uncertainty estimation as a critical safety feature for autonomous-driving prediction.
- Engineered predictors propagate tracked states with physical or kinematic models, but may ignore surrounding context and degrade over longer horizons.
- Map-constrained systems associate vehicles with lanes and generate paths using map topology, lane connectivity, and vehicle state.
- Manually designed models can impose unrealistic assumptions, while maneuver-based models often predict each object independently.
- Bayesian networks can mitigate independence assumptions but are computationally more expensive and infeasible for real-time tasks.
- Learned approaches include RNN, social LSTM, and inverse-reinforcement-learning methods that incorporate scene context or interactions.
- Uncertainty estimation distinguishes observation noise, or aleatoric uncertainty, from uncertainty in the prediction model itself, or epistemic uncertainty.
3. Proposed approach
The approach predicts each tracked actor’s future positions from map and state context rasterized into actor-centric images, using CNN-based models with optional actor-state inputs. It trains point or uncertainty-aware trajectory predictors from observed actor states and map data.
- The system assumes real-time sensor streams and an existing tracker that provides actor state estimates, including position, velocity, acceleration, heading, and heading-change rate.
- Future x- and y-positions over a prediction horizon are inferred from high-definition maps and tracked actor states in an actor-centric coordinate system.The formulation predicts future positions rather than complete future state estimates.
- Each actor’s dynamic and static context is rasterized into an RGB image, allowing a CNN to learn trajectory-relevant features instead of relying on manually defined context features.The raster uses map elements and surrounding actors, while the current actor state can optionally be supplied as a vector.
- Rasterization assigns distinct colors to vector layers and encodes lane direction through HSV hue before compositing map and actor layers into the scene image.Lane hue represents driving direction, while layers are rasterized from larger-area structures toward finer structures.
- Historical actor bounding boxes are rendered with reduced brightness to create fading, preserving past motion information in the raster.The experiments use either one time step without fading or five time steps with fading, with brightness reduced by δ = 0.1.
- The feed-forward architecture extracts raster features with a base CNN, optionally concatenates a three-dimensional actor-state vector, and predicts trajectory outputs through fully connected layers.The actor-state vector contains velocity, acceleration, and heading-change rate; an alternative recurrent decoder uses an LSTM.
4. Experiments
Experiments on real-world tracked-vehicle data compare baselines and CNN variants, then examine uncertainty calibration, case studies, error dynamics, and long-horizon limitations.
- Experimental setup: The evaluation uses tracked vehicle data collected from 240 hours of manually driven SDV operation in Pittsburgh and Phoenix.Data were collected at 10 Hz under varied traffic conditions and split into training, validation, and test sets.
- Experimental setup: The study compares UKF, linear, and lane-association baselines with CNN models using raster, fading, and vehicle-state inputs.CNN backbones include AlexNet, VGG-19, ResNet-50, and MobileNet-v2.
- Ablation results: Adding fading and state inputs progressively improves the AlexNet model, with their combination providing an additional performance boost.Without fading or state, the model performs worse than UKF; fading alone enables a large improvement, while direct state inputs perform better still.
- Architecture comparison: The MNv2 model outperforms the other CNN variants, uncertainty outputs further improve results, and an LSTM decoder achieves the best performance.The uncertainty outputs also estimate trajectory uncertainty and mitigate adverse effects of noisy training data.
- Uncertainty evaluation: Reliability diagrams show that predicted error distributions are well calibrated, especially at 3 seconds, while 1-second predictions are slightly underconfident.The paper interprets a one-sigma estimate as containing 68% of actual errors when calibration is accurate.
- Case studies: Case studies show accurate short-term trajectories, intuitive aleatoric and epistemic uncertainty, and raster attention focused on relevant traffic elements.Epistemic uncertainty is several orders of magnitude lower than aleatoric uncertainty, while occlusion analysis highlights lanes, vehicles, crosswalks, and obstacles.
- Case studies: During a turning scenario, MNv2 adapts nearly when the vehicle initiates its turn, whereas UKF retains higher-error predictions for nearly 1.5 seconds longer.The proposed approach also maintains lower along-track error in the analyzed example.
- Long-horizon analysis: For longer horizons, multimodality causes exponential error increase for both UKF and the proposed unimodal method.The authors retrained a model with H = 60 to examine how prediction quality changes across horizons.
5. Conclusion
The paper presents a deep learning-based method for traffic-actor motion prediction that estimates future positions and their uncertainties. It rasterizes actor contexts, uses CNNs for short-term trajectory prediction, and reports practical benefits including onboard testing.
- The method addresses motion prediction of traffic actors, a critical part of the self-driving vehicle problem.
- It rasterizes actor contexts and trains CNNs to predict short-term trajectories and corresponding uncertainty.
- Extensive evaluation strongly suggests practical benefits, and the framework was successfully tested onboard self-driving vehicles after offline testing.