PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs

Nachiket Deo, Mohan M. Trivedi

arXiv:1805.05499v1cs.CV

TL;DR

Autonomous vehicles need reliable surrounding-vehicle predictions despite nonlinear, interacting, and multi-modal driver behavior. The paper proposes an interaction-aware LSTM that predicts maneuver-conditioned motion distributions, and reports lower RMS prediction error on two real freeway trajectory datasets than prior approaches.

  • Problem

    Surrounding-vehicle prediction is difficult because driver behavior is nonlinear and multiple maneuvers can be plausible under the same traffic circumstances.

  • Method

    An interaction-aware LSTM uses vehicle histories, freeway lane structure, and six maneuver classes to produce maneuver-specific probabilistic future trajectories.

  • Results

    The proposed M-LSTM achieves lower RMS prediction error than compared approaches on the NGSIM US-101 and I-80 freeway datasets.

  • Takeaways & Limitations

    Ablations indicate that adjacent-vehicle motion and maneuver classification contribute to future-motion prediction accuracy.

Abstract

from arXiv · show

To safely and efficiently navigate through complex traffic scenarios, autonomous vehicles need to have the ability to predict the future motion of surrounding vehicles. Multiple interacting agents, the multi-modal nature of driver behavior, and the inherent uncertainty involved in the task make motion prediction of surrounding vehicles a challenging problem. In this paper, we present an LSTM model for interaction aware motion prediction of surrounding vehicles on freeways. Our model assigns confidence values to maneuvers being performed by vehicles and outputs a multi-modal distribution over future motion based on them. We compare our approach with the prior art for vehicle motion prediction on the publicly available NGSIM US-101 and I-80 datasets. Our results show an improvement in terms of RMS values of prediction error. We also present an ablative analysis of the components of our proposed model and analyze the predictions made by the model in complex traffic scenarios.

I. INTRODUCTION

The paper addresses freeway trajectory prediction when driver behavior is nonlinear and inherently multi-modal. It proposes an LSTM that uses maneuver classes to produce probabilistic, interaction-aware future trajectories.

  • Motivation: Reliable surrounding-vehicle trajectories are needed for autonomous vehicles to balance safety with efficient motion in complex traffic.Existing tactical path-planning algorithms depend on future trajectory estimates.
  • Limitations of prior approaches: Motion models can become unreliable over longer horizons because driver decisions make vehicle trajectories highly nonlinear.Data-driven approaches address this limitation by learning from trajectory datasets.
  • Multi-modality: Regression methods can average distinct driver decisions, producing trajectories that do not correspond to plausible maneuvers.A driver may slow down or change lanes and accelerate when approaching a slower lead vehicle.
  • Proposed approach: The proposed LSTM assigns probabilities to six maneuver classes and predicts maneuver-specific multi-modal future motion from surrounding-vehicle histories and freeway lane structure.The model is trained and evaluated on real trajectories from the NGSIM US-101 and I-80 freeway datasets.
  • Related work: The work builds on maneuver-recognition and maneuver-specific trajectory-prediction approaches, including classifiers based on heuristic, Bayesian, hidden-Markov, random-forest, and recurrent models.Prior LSTM work includes maneuver classification and occupancy-grid prediction.

III. PROBLEM FORMULATION

The problem is formulated as estimating a vehicle’s future-position distribution from its own and nearby vehicles’ track histories. The formulation uses a coordinate-based representation of the predicted vehicle and its surrounding context.

  • Problem formulation: Motion prediction estimates the probability distribution of future vehicle positions conditioned on the predicted vehicle’s and nearby vehicles’ track histories at time t.
  • Representation: The coordinate system represents the predicted vehicle in relation to neighboring vehicles considered for trajectory prediction.Figure 2 depicts the predicted vehicle and neighboring vehicles in the formulation.

A. Frame of reference

The model uses a stationary frame centered on the vehicle being predicted and represents its surrounding context through track-history coordinates. Its outputs describe future vehicle coordinates, while the architecture uses encoded context and maneuver information.

  • A. Frame of reference: The stationary frame places its origin at the predicted vehicle at time t, with axes aligned longitudinally and laterally to freeway motion.This representation is independent of track-collection method and road curvature, given onboard lane estimation.
  • B. Inputs and outputs: The input tensor contains x- and y-coordinates for the predicted vehicle and six surrounding vehicles at the current time.The six surrounding vehicles are selected because they seem to have the most effect on vehicle motion.
  • B. Inputs and outputs: The model outputs a probability distribution over the predicted vehicle’s future coordinates.
  • B. Inputs and outputs: The trajectory encoder LSTM converts vehicle histories and relative positions into context for a decoder that produces maneuver-specific future position distributions.The proposed architecture also includes maneuver encodings and a maneuver-classification branch assigning maneuver probabilities.

C. Probabilistic motion prediction

The probabilistic model expands future-position prediction over maneuver classes. For each maneuver, it produces a conditional trajectory distribution whose parameters describe future location uncertainty.

  • Probabilistic prediction: The model decomposes P(Y|X) into maneuver-conditioned trajectory distributions weighted by maneuver probabilities: P(Y|X) = PΘ(Y|mi, X)P(mi|X).This maneuver expansion is used to produce multi-modal future-position distributions.
  • Probabilistic prediction: The predicted distribution parameters define a bivariate Gaussian at each future time step, including means and variances of future locations.

D. Maneuver classes

The model uses three lateral and two longitudinal maneuver classes to represent lane changes, lane keeping, normal driving, and braking.

  • Lateral maneuvers: The three lateral maneuvers are left lane change, right lane change, and lane keeping.A vehicle is labeled as changing lanes from 4 seconds before to 4 seconds after the actual crossover.
  • Longitudinal maneuvers: The two longitudinal maneuvers are normal driving and braking.Braking is defined when average speed over the prediction horizon is below 0.8 times the speed at prediction.
  • Operational rationale: These maneuver classes correspond to cues such as turn signals and brake lights that vehicles can communicate.The paper identifies these cues as inputs for future work.

IV. MODEL

The model uses an encoder-decoder LSTM to encode the predicted vehicle and six adjacent vehicles, then generate maneuver-conditioned future trajectories.

  • Trajectory encoding: The trajectory encoder LSTM processes frame-by-frame past locations of the predicted vehicle and six adjacent vehicles.Its final state represents track histories and relative positions of all seven vehicles.
  • Trajectory decoding: The decoder LSTM uses the encoded context vector to update its state and produce future vehicle locations over the prediction horizon.Decoder updates occur at each future frame using the context vector and the previous LSTM state.
  • Maneuver conditioning: Maneuver context is formed by appending one-hot lateral and longitudinal maneuver vectors to the encoder context.This enables the decoder to generate maneuver-specific trajectories for six defined maneuver combinations.
  • Implementation: The encoder, decoder, and maneuver-classification branch each use LSTMs with 128 units.Input vectors are first embedded through a 64-unit fully connected layer with leaky ReLU activation.
  • Training: Trajectory prediction and maneuver classification are trained separately but used together during testing.The trajectory model minimizes negative log likelihood for ground-truth future locations, using ground-truth maneuver classes during training.

V. EXPERIMENTAL EVALUATION

Experiments use the publicly available NGSIM US-101 and I-80 datasets, which contain real freeway trajectories across varied traffic conditions.

  • Datasets: The evaluation uses the publicly available NGSIM US-101 and I-80 freeway-traffic datasets.Both datasets contain trajectories of real vehicles.
  • Collection sites: The datasets record real vehicle trajectories on multi-lane freeways with entry and exit ramps.The collection sites include varying traffic densities.
  • Traffic conditions: Trajectories were captured at 10 Hz over 45 minutes, with 15-minute segments representing mild, moderate, and congested traffic.Vehicle coordinates are projected into a local coordinate system.
  • Data split: A quarter of the trajectories from each traffic-condition subset in both datasets is assigned to the test set.The remaining trajectories are used for training.

B. Models compared

The evaluation compares the proposed Maneuver-LSTM with conventional, maneuver-based, and recurrent baselines using RMS prediction error, while ablations test interaction history and maneuver classification.

  • Evaluation setup: RMS prediction error is evaluated over a 5-second prediction horizon.The compared models include Constant Velocity, C-VGMM + VIM, GAIL-GRU, and Maneuver-LSTM.
  • Baseline comparison: The proposed M-LSTM and GAIL model considerably outperform the Constant Velocity and C-VGMM + VIM baselines.The comparison uses RMS values of prediction error.
  • Ablation: The ablation evaluates the significance of adjacent-vehicle histories and the maneuver-classification branch.These components are compared through RMS prediction error across four system settings.
  • Recurrent comparison: M-LSTM has lower prediction error than GAIL at all prediction intervals.GAIL appears to approach M-LSTM as the prediction horizon increases, but its evaluation gives it access to ground-truth adjacent-vehicle trajectories.

D. Ablative Analysis

The ablation shows that adjacent-vehicle histories and maneuver classification each improve trajectory prediction, especially at longer horizons. Ground-truth maneuvers indicate further gains are possible with more accurate maneuver recognition.

  • Adjacent-vehicle histories improve prediction over using only the predicted vehicle’s history.The comparison is between Surround LSTM and Vanilla LSTM.
  • Maneuver classification further improves prediction accuracy beyond the Surround LSTM.M-LSTM represents the complete proposed model, while S-LSTM excludes maneuver recognition.
  • Both adjacent-vehicle modeling and maneuver classification effects become more pronounced at longer prediction intervals.
  • Ground-truth maneuver labels show considerable potential for further trajectory-prediction improvement through better maneuver recognition.M-LSTM (GT) uses ground-truth maneuver classes to gauge this potential.

E. Qualitative analysis of predictions

The qualitative analysis examines multimodal predictions and how leading and adjacent vehicles influence predicted trajectories. The model assigns maneuver probabilities and adapts its predictions to surrounding traffic behavior.

  • Each scenario displays past 3 seconds of track history, mean predictions over the next 5 seconds for each maneuver class, and a probability-weighted trajectory thickness.A heat map additionally represents the complete predicted distribution.
  • The model represents lane-changing vehicles with modes for both changing lanes and keeping lane.It becomes more confident in the lane change further into the maneuver and predicts merging into the target lane.
  • Leading-vehicle motion changes predictions from braking behind a slowing leader to accelerating in stop-and-go traffic.The model predicts braking when the leader slows and acceleration when a previously stationary vehicle’s leader begins moving.
  • Adjacent-vehicle proximity changes the probability of a predicted vehicle changing into the left lane.Lane-change probability rises when the adjacent vehicle is far behind, drops when it is alongside, and rises again after it passes.

VI. CONCLUSIONS

The paper presents an interaction-aware LSTM that makes multimodal trajectory predictions using maneuver classes. It reports lower prediction error on two large freeway-trajectory datasets and identifies adjacent-vehicle motion and maneuver modeling as significant components.

  • The proposed interaction-aware LSTM makes multimodal trajectory predictions based on maneuver classes.
  • The model achieves lower prediction error on two large datasets of real freeway vehicle trajectories than two existing state-of-the-art approaches.
  • Ablation results identify adjacent-vehicle motion and maneuver detection and exploitation as significant for future motion prediction.
Loading 1805.05499v1…