Source-linked AI summary
IntentNet: Learning to Predict Intention from Raw Sensor Data
Sergio Casas, Wenjie Luo, Raquel Urtasun
TL;DR
Self-driving vehicles need to predict other traffic participants’ intent, including discrete behaviors and continuous future trajectories. IntentNet jointly processes LiDAR and maps in a one-stage multi-task model, achieving higher performance across detection and intent-prediction tasks than prior approaches while sharing computation.
Problem
Autonomous vehicles must predict human drivers’ difficult-to-predict maneuvers to understand other traffic participants’ intent.
Method
IntentNet is an end-to-end one-stage network that uses LiDAR point clouds and HD-map scene knowledge to jointly predict detection, discrete behavior, and future trajectories.
Results
IntentNet achieves higher performance than previous work across detection, continuous trajectory forecasting, and discrete intention prediction.
Takeaways & Limitations
A single neural network can combine LiDAR and map information for detection and intent prediction while sharing heavy feature computation across tasks.
Abstract
from arXiv · showhide
In order to plan a safe maneuver, self-driving vehicles need to understand the intent of other traffic participants. We define intent as a combination of discrete high-level behaviors as well as continuous trajectories describing future motion. In this paper, we develop a one-stage detector and forecaster that exploits both 3D point clouds produced by a LiDAR sensor as well as dynamic maps of the environment. Our multi-task model achieves better accuracy than the respective separate modules while saving computation, which is critical to reducing reaction time in self-driving applications.
1 Introduction
IntentNet addresses the difficulty of predicting human road-user maneuvers by jointly modeling detection, discrete intentions, and long-term trajectories from LiDAR and maps. The system reports stronger detection and intent-prediction performance than prior real-time detectors and intent approaches.
- Human drivers’ difficult-to-predict maneuvers make understanding other traffic participants’ intent fundamental to autonomous driving.
- IntentNet uses LiDAR, semantic road-topology maps, and a fully convolutional network to predict detection, discrete action probabilities, and current and future bounding boxes.
- Joint optimization addresses distribution mismatch between separately solved tasks while allowing uncertainty propagation across components.
- Shared heavy feature computation makes the multi-task approach computationally efficient.
- IntentNet surpasses real-time state-of-the-art detectors and previous approaches for both continuous and discrete intent prediction.
2 Related Work
Prior work includes efficient single-stage detection, LiDAR-based motion forecasting, and models of discrete driving intention. IntentNet extends joint detection and forecasting with longer horizons, map-based traffic constraints, and a temporally discounted loss.
- Object detection: Single-stage detectors provide a more efficient alternative to computationally demanding proposal-based object detectors.
- Motion Forecasting: Motion forecasting predicts future actor locations from current and past information, using approaches that include recurrent, game-theoretic, rasterized, and LiDAR-based models.
- Intention Prediction: Intention prediction treats an actor’s intention as the sequence of actions taken to achieve an objective.
- IntentNet improves joint detection and trajectory forecasting, extends the prediction horizon, and estimates future high-level driver behavior.
- Its performance gains use early fusion of more LiDAR sweeps, map parametrization for traffic constraints, and a temporal discount factor for future ambiguity.
3 Learning to Predict Intention
IntentNet jointly predicts vehicle detections, discrete high-level actions, and continuous trajectories from voxelized LiDAR and rasterized dynamic maps. Its shared architecture combines separate sensor streams with task-specific branches and trains them using a multi-task objective.
- Input parametrization: IntentNet uses voxelized bird’s-eye-view LiDAR and rasterized dynamic maps containing roads, lanes, intersections, crossings, signs, and traffic-light states.LiDAR is represented as a 3D tensor, while map semantics are encoded as binary masks.
- Network architecture: A two-stream backbone processes LiDAR and map inputs separately, concatenates their features, and feeds shared representations to three task-specific branches.The branches handle detection, discrete intention classification, and motion estimation conditioned on embedded intention scores.
- Output parametrization: The model predicts detection scores, eight-class high-level intentions, and future vehicle trajectories in a single forward pass.The intention classes include keep lane, turns, lane changes, stopping, parking, and other actions.
- Learning: The training objective combines trajectory regression, binary detection classification, and multiclass intention classification across T prediction steps.A temporal discount factor weights future losses so more difficult distant predictions do not dominate optimization.
- Learning: IntentNet uses focal loss for detection, cross entropy for intentions, and weighted smooth L1 regression for positive trajectory targets.The intention-training procedure downsamples dominant keep-lane, stopping/stopped, and parked examples by 95% without discarding them for detection or trajectory regression.
- Inference: Inference thresholds classification probabilities at 0.1, examines regression outputs, and applies non-maximum suppression to deduplicate detections.The resulting detections include continuous and discrete intentions for vehicles in the scene.
4 Experimental Evaluation
IntentNet is evaluated as a joint detector, intention predictor, and motion forecaster on a large LiDAR dataset. It outperforms real-time and prior intent-prediction baselines, while ablations show benefits from longer temporal context and map-conditioned discrete intention.
- Dataset: The dataset contains over 1 million LiDAR frames from more than 5,000 scenarios, with 250-frame sequences captured at 10 Hz.
- Implementation: The model uses a 144 × 80 × 5.8 meter BEV region, 10 past LiDAR sweeps, and predicts 3 seconds of future motion.
- Detection: IntentNet is clearly superior to SqueezeNet, SSD, MobileNet, FaF, and FaF′ across all evaluated detection IoU levels under real-time inference constraints.
- Trajectory regression: IntentNet outperforms FaF and FaF′ in along-track, across-track, and heading errors, with metrics computed over shared true-positive detections covering over 90% of validation data.The model also maintains low along-track error on turns and lane changes while learning complex velocity profiles.
- Intention prediction: IntentNet outperforms MLP and LSTM baselines, particularly for underrepresented turns and lane changes, despite extreme behavior imbalance.
- Ablation study: A 2D CNN with early fusion is more robust than the compared 3D CNN, while increasing temporal context from 0.5 to 1 second slightly reduces long-term L2 error.
- Ablation study: Adding discrete-intention loss alone degrades detection and regression, but map input enables behavior prediction and makes the added loss improve overall performance.
- Qualitative results: Qualitative examples show lane-change prediction, large-vehicle detection, high precision and recall in clutter, and turn prediction.
5 Conclusion
IntentNet is an end-to-end vehicle detection and intent-prediction model using LiDAR point clouds and HD-map scene information. The authors report higher performance than previous work across all tasks with a single neural network.
- IntentNet is a learnable end-to-end model for vehicle detection and intent prediction in self-driving cars.
- The model exploits 3D LiDAR point clouds and HD-map scene knowledge in a single neural network.
- The authors report higher performance than previous work across all tasks.
- Future work includes modeling statistical dependencies between discrete and continuous intention and extending the approach to pedestrians and bicyclists.