Source-linked AI summary
Intention-aware Long Horizon Trajectory Prediction of Surrounding Vehicles using Dual LSTM Networks
Long Xin, Pin Wang, Ching-Yao Chan, Jianyu Chen, Shengbo Eben Li, Bo Cheng
TL;DR
The paper addresses prediction of surrounding-vehicle trajectories from incomplete, noisy observations available to an autonomous vehicle. It uses dual LSTMs to recognize driver intention and predict future motion, reporting better accuracy than some existing methods over a five-second horizon, including RMSE below 5.77m longitudinally and 0.49m laterally.
Problem
Autonomous vehicles need future surrounding-vehicle trajectories, but sensing is often partial or occluded and motion depends on interactions, driver behavior, and road geometry.
Method
A dual-LSTM regression framework recognizes driver intention from sequential observations before predicting future surrounding-vehicle trajectories using ego-vehicle sensor-feasible features.
Results
The model reports better overall longitudinal prediction and lateral performance comparable to the best method, with improved prediction timeliness for lane-change events.
Takeaways & Limitations
Intention recognition provides semantic information that narrows prediction boundaries and supports long-horizon trajectory prediction under limited observation.
Abstract
from arXiv · showhide
As autonomous vehicles (AVs) need to interact with other road users, it is of importance to comprehensively understand the dynamic traffic environment, especially the future possible trajectories of surrounding vehicles. This paper presents an algorithm for long-horizon trajectory prediction of surrounding vehicles using a dual long short term memory (LSTM) network, which is capable of effectively improving prediction accuracy in strongly interactive driving environments. In contrast to traditional approaches which require trajectory matching and manual feature selection, this method can automatically learn high-level spatial-temporal features of driver behaviors from naturalistic driving data through sequence learning. By employing two blocks of LSTMs, the proposed method feeds the sequential trajectory to the first LSTM for driver intention recognition as an intermediate indicator, which is immediately followed by a second LSTM for future trajectory prediction. Test results from real-world highway driving data show that the proposed method can, in comparison to state-of-art methods, output more accurate and reasonable estimate of different future trajectories over 5s time horizon with root mean square error (RMSE) for longitudinal and lateral prediction less than 5.77m and 0.49m, respectively.
I. INTRODUCTION
Surrounding-vehicle trajectories are difficult to predict because observations are noisy and incomplete, while driver behavior and road geometry vary. The paper therefore uses dual LSTMs to recognize driver intention before predicting five-second future motion.
- Motivation: Noisy, partial observations and interactions among road users make surrounding-vehicle motion difficult to predict for autonomous driving.The paper identifies driver randomness, road geometry, occlusion, and sensing noise as relevant challenges.
- Existing approaches: Prototype-trajectory methods require similarity matching and may need many prototypes to represent diverse real-world driving patterns.Gaussian approaches also incur expensive online probability calculations and time-dependent handling of stopped vehicles.
- Existing approaches: LSTM-based studies address temporal structure but can lose positional accuracy, require unavailable neighboring-vehicle information, or confuse lane changes with curved-road driving.These limitations motivate a method that uses feasible observations and adapts to road geometry.
- Contribution: The proposed geography-adaptive method first estimates maneuver intention and then predicts future vehicle states using two LSTMs.It uses sequentially transformed surrounding-vehicle coordinates and predicts future locations over the next 5 seconds.
II. PROBLEM STATEMENT
The task is to predict surrounding vehicles’ future trajectories from historic, limited observations available to the ego vehicle. It is formulated as continuous regression rather than occupancy-grid classification, under realistic sensing constraints.
- Problem formulation: The predictor uses historic but limited on-board sensor observations to estimate surrounding vehicles’ future highway trajectories.The formulation targets more accurate future values for decision-making and path planning.
- Problem formulation: Unlike occupancy-grid classification, the paper formulates trajectory prediction as regression whose outputs should match actual values.This formulation is intended to preserve continuous position accuracy.
- Problem formulation: Future trajectories are modeled as sequential outputs Y conditioned on observable feature inputs X across historical and future time intervals.The notation distinguishes historical input times, future prediction times, target outputs, and acquired feature sets.
- Assumptions: The model assumes future surrounding-vehicle trajectories depend only on information observed from the ego vehicle, which may be limited by sensors and occlusion.This avoids relying on information from vehicles surrounding the target vehicle.
III. PROPOSED METHOD
The proposed architecture uses two LSTM networks to connect driver-intention recognition with future trajectory prediction. It incorporates semantic intention information, road-structure constraints, and unified treatment of lane keeping and lane changes.
- System Architecture: The first LSTM recognizes driver intentions such as lane keeping and lane change, while the second extracts sequential representations for trajectory prediction.A dense output layer produces the future trajectory from the second LSTM’s representation and embedded intention information.
- System Architecture: Semantic driving-intention information provides instructive input for inferring surrounding-vehicle motion.The architecture uses intention recognition as an intermediate signal for subsequent trajectory prediction.
- System Architecture: Road-structure knowledge supports upper and lower lateral prediction boundaries, including lane-center travel and one-lane-at-a-time changes.These constraints are intended to keep lateral predictions consistent with structured-road driving.
- System Architecture: Timely intention recognition can reduce lane-change prediction delay, while lane keeping and lane change data are handled in one learning framework.The unified treatment is described as reducing the negative effect of imbalanced maneuver data.
B. Input Features
The model uses sensor-measurable trajectory features represented in an ego-vehicle-aligned local coordinate system. Features include lane-relative lateral quantities, derivatives, and longitudinal motion variables, with the target lane determined by recognized intention.
- Input features are measured using onboard sensors and represented in a local coordinate system parallel to the global one.
- Relative lateral position is defined against the nearest lane marking, with its first and second derivatives supporting time-series intention recognition.
- Lateral deviation is measured from the center line of the target lane.
- The target lane is the current lane for lane keeping or the intended neighboring lane for a lane change.
- Additional inputs include lateral derivatives and longitudinal velocity and acceleration, selected from sensor observations and likely human driving decision factors.
C. LSTM Network
The network uses LSTM recurrence to learn temporal features through memory cells and gates. Its internal states are updated over time, allowing the architecture to manage information flow and extract higher-level features.
- LSTMs analyze sequential information and learn temporal features through recurrent internal states, while addressing vanishing gradients over time.
- An LSTM cell stores interpretations of past inputs in a memory cell updated from the current input and previous cell state.
- Input, forget, output, and state-update vectors, together with weights and biases, define the LSTM's learned state transformations.
- Input and output gates control data flow, while the forget gate determines whether stored memory is retained or discarded.
- The gating mechanism is learned from data and helps overcome training problems caused by exponentially decaying or increasing inputs in general RNNs.
D. Prediction Outpus
The prediction stage outputs longitudinal acceleration and lateral deviation over the future horizon, then reconstructs vehicle positions using the inferred target lane and time step.
- Future longitudinal accelerations are output over the prediction horizon instead of directly predicting large longitudinal positions.
- The future longitudinal trajectory is calculated from the predicted accelerations.
- The time step δt is used in calculating the future longitudinal trajectory.
- Because lateral position is bounded, lateral deviation is directly output over the prediction horizon and used to calculate lateral position.
- The lateral position calculation uses the centerline of the inferred target lane produced from the first LSTM's intention-recognition output.
A. Data
The study uses naturalistic highway trajectories from NGSIM's I-80 freeway segment in Emeryville, California. The data cover transitional and congested traffic and include more than 5,000 vehicle trajectories sampled at 10 Hz.
- The dataset is NGSIM, an open naturalistic-driving dataset collected and published by the US Federal Highway Administration in 2005.
- The study area is an approximately 500m-long, six-lane segment of I-80 in Emeryville, California.
- The 45-minute collection period spans transitional and congested traffic conditions.
- The dataset contains more than 5000 individual-vehicle trajectories sampled at 10 Hz, with speed, acceleration, position, vehicle-length, and vehicle-type information.
- A total of 914 successful lane changes were automatically identified using SAE J2944.
B. Training
The network is trained on 5 seconds of past observations to recognize driving intentions and predict future vehicle states through two LSTM stages.
- B. Training: The network uses a 50-time-step window representing 5s of past observations, updated every second at 10 Hz.Training batches contain 100 samples shuffled within batches to improve efficiency and avoid backpropagation issues.
- B. Training: The first LSTM treats lane keeping, left lane change, and right lane change as a three-class classification problem.Intentions are encoded as a 1x3 one-hot vector, and a softmax layer maps the hidden state to intention probabilities.
- B. Training: The intention prediction is represented by y, with weights assigned to each category and k=3 intention classes.The softmax output gives the driving intention prediction for the three categories.
- B. Training: The second LSTM directly processes defined features and minimizes the quadratic difference between predicted and actual values through BPTT.Its weights are updated using the calculated loss during backpropagation through time.
V. RESULTS
The evaluation uses NGSIM I-80 data and compares the proposed model with LSTM-based alternatives using trajectory and lane-change prediction results. The model performs best overall for longitudinal position and recognizes lane changes before lane crossings.
- V. RESULTS: 70% of the NGSIM I-80 dataset is used for training and 30% for validation, with GPU training in TensorFlow for 5 epochs.The learning rate starts at 1.0 and decreases until validation error stops improving.
- V. RESULTS: The model directly accepts trajectory features, recognizes driver intention probabilistically, and predicts future trajectories simultaneously.Examples for vehicles 831, 967, and 928 show individualized predicted trajectories over a 2s difference.
- V. RESULTS: RMSE is computed for each trajectory and compared with two LSTM-based methods, including occupancy-grid prediction and a model using surrounding-vehicle information.The evaluation assesses both learning performance and generalization across different drivers.
- V. RESULTS: The proposed model provides the best overall longitudinal position prediction and results similar to the best lateral position prediction.The comparison is summarized in Table I through RMSE for trajectory predictions.
- V. RESULTS: Lane-change events are underrepresented in the dataset, so RMSE does not properly account for their errors, especially at longer horizons.The paper therefore evaluates lane-change prediction separately from aggregate trajectory error.
- V. RESULTS: All lane-change events are recognized in advance, allowing the model to reduce prediction delay before vehicles cross lane markings.The analysis computes relations between lane-change probabilities and the time to the actual lane-change point for left and right lane changes.
VI. CONCLUSION
The proposed intention-aware LSTM architecture predicts 5-second highway vehicle trajectories with improved accuracy and timeliness, while offering semantic motion guidance for higher-level decisions. The authors identify broader scenarios, interactive vehicle relations, and stochastic outputs as directions for future work.
- 5s highway trajectory prediction achieved smaller longitudinal and lateral RMSE across different horizons, with minimized prediction delay for lane-change events.
- The model is designed to provide semantic understanding of surrounding-vehicle motion for high-level decision-making systems.
- Future work includes generalizing the model to intersections and unstructured roads, modeling interactive vehicle relations, and producing distributions with confidence intervals.