Source-linked AI summary
An LSTM Network for Highway Trajectory Prediction
Florent Altché, Arnaud de La Fortelle
TL;DR
Autonomous driving needs medium-term forecasts of surrounding vehicles, which current ADAS often lack. The paper trains an LSTM trajectory predictor using local observations and evaluates it on the whole NGSIM US101 dataset, reporting accurate long-horizon lateral and longitudinal predictions but noting delayed responses as a limitation.
Problem
The paper addresses the limited medium-term prediction capabilities of current ADAS and the need to forecast surrounding vehicles for autonomous driving.
Method
The study trains an LSTM regression network to predict a target vehicle’s future trajectory from previously observed local vehicle information.
Results
Using the whole US101 dataset, the network achieved roughly 70 cm lateral-position RMS error and below 3 m s−1 longitudinal-velocity RMS error 10 s ahead.
Takeaways & Limitations
The results provide a promising basis for computing probable trajectories of surrounding vehicles for future motion-planning improvements.
Takeaways & Limitations
The work is preliminary and exhibits delayed responses that should be addressed before broader generalization to other highways and driving scenarios.
Abstract
from arXiv · showhide
In order to drive safely and efficiently on public roads, autonomous vehicles will have to understand the intentions of surrounding vehicles, and adapt their own behavior accordingly. If experienced human drivers are generally good at inferring other vehicles' motion up to a few seconds in the future, most current Advanced Driving Assistance Systems (ADAS) are unable to perform such medium-term forecasts, and are usually limited to high-likelihood situations such as emergency braking. In this article, we present a first step towards consistent trajectory prediction by introducing a long short-term memory (LSTM) neural network, which is capable of accurately predicting future longitudinal and lateral trajectories for vehicles on highway. Unlike previous work focusing on a low number of trajectories collected from a few drivers, our network was trained and validated on the NGSIM US-101 dataset, which contains a total of 800 hours of recorded trajectories in various traffic densities, representing more than 6000 individual drivers.
I. INTRODUCTION
The paper addresses highway trajectory prediction with LSTM networks, motivated by the need for autonomous vehicles to anticipate surrounding traffic. It emphasizes evaluation on comprehensive naturalistic data rather than selected trajectories.
- Reliable prediction of surrounding vehicles supports tactical decisions such as overtaking and crossing unsignalized intersections.
- Current ADAS often act reactively, while autonomous vehicles without predictive capabilities may behave conservatively around other traffic.
- Behavior classification is easier to train and evaluate but provides rough future states that are difficult to use directly for trajectory planning.
- Direct trajectory regression can support motion planning, although many regression methods output only a single trajectory without confidence estimates.
- The proposed LSTM network targets highway car trajectories, where little prior literature exists and memory-based recurrent modeling suits time-series prediction.
- Using the entire NGSIM US101 dataset, the study reports average RMS errors below 0.7 m laterally and 2.5 m s−1 longitudinally at a 10 s horizon.
II. PROBLEM STATEMENT
The paper formulates highway trajectory prediction as forecasting a target vehicle’s future motion from previously observed data. The model uses observable local context and learns a regression function mapping past features to future outputs.
- The task is to predict future highway vehicle trajectories from previously observed data for autonomous-vehicle motion planning.
- The formulation separates observable features I from target outputs O across past and future time steps.
- A learned regression function f maps the history tensor X to predicted outputs Ŷ that match actual future values.
- The predictor models one target vehicle using information from immediately surrounding vehicles that can realistically be gathered.
- Highway data are imbalanced because constant-velocity phases are much more frequent than events such as lane changes.
A. Dataset
The study uses the NGSIM US101 naturalistic-driving dataset, a publicly available highway trajectory source collected during a rush-hour transition. It provides road-aligned positions and additional vehicle-level context.
- NGSIM US101 contains 45 minutes of highway trajectories recorded from 7:50am to 8:35am during the transition from fluid traffic to saturated rush-hour conditions.
- The dataset is a large publicly available source of naturalistic driving data collected by the United States Federal Highway Administration in 2005.
- Vehicle trajectories include global and road-aligned local coordinates for each vehicle’s front-center point.
- The local representation defines x as lateral position relative to the road’s left edge and y as longitudinal position.
- Additional records include lane identifiers, vehicle dimensions and type, and the preceding-vehicle identifier when applicable.
B. Data preparation
The data preparation smooths noisy vehicle trajectories, identifies vehicles surrounding a target vehicle, and limits contextual inputs to nine nearby vehicles for realistic sensing.
- A first-order Savitzky-Golay filter with window length 11 smooths longitudinal and lateral positions and computes corresponding velocities.The window corresponds to a 1-second time window and addresses noise in video-derived positions and numerically differentiated velocities.
- The model represents nine vehicles of interest according to their relative positions around the target vehicle.The selected vehicles include nearby vehicles in the target’s lane and adjacent lanes, along with their leaders and followers.
- Vehicle identities are computed during preprocessing, and missing vehicles receive zero-valued data columns.
- Information from the vehicle directly ahead is supplemented by its leader because that vehicle’s state can help infer future traffic evolution in a jam.For example, acceleration by the leader may indicate that a currently stopped vehicle will accelerate rather than remain stopped.
- The context is restricted to nine vehicles because on-board sensors may not realistically provide sufficient data for more vehicles.
C. Features
The feature design combines absolute state variables for the target vehicle with relative distances, speeds, and types for surrounding vehicles, reflecting information available to a driver and onboard sensors.
- The feature set is designed around quantities that can be reasonably measured using onboard sensors such as GNSS and LiDAR.The target vehicle and surrounding vehicles use different feature sets.
- Target-vehicle features include local lateral and longitudinal positions, lateral and longitudinal velocities, and vehicle type.Vehicle type encodes motorcycle, car, and truck as −1, 0, and +1.
- Surrounding-vehicle features include longitudinal distance and relative longitudinal velocity with respect to the target vehicle, plus vehicle type.Relative velocity is defined as the target vehicle’s longitudinal velocity minus the surrounding vehicle’s longitudinal velocity.
- Distances and longitudinal velocities are divided by 10, generally scaling values into [−2, 2] for the network activations.Missing-vehicle values can exceed this range in absolute value.
- Surrounding-vehicle features are expressed relative to the target, while target speed uses an absolute road-relative frame.This reflects drivers’ use of perceived distances and relative speeds alongside speedometer information.
D. Outputs
The model predicts future lateral positions and longitudinal velocities at multiple time horizons, using velocity rather than longitudinal position because the region of interest spans roughly 1 km.
- The output is the target vehicle’s future trajectory, represented by lateral position and longitudinal velocity predictions.
- Future longitudinal velocities are predicted instead of longitudinal positions because longitudinal position values become large across the roughly 1 km region of interest.
- Future lateral position is predicted directly because lateral position is bounded.
- The output vector contains predicted values at horizons of k seconds in the future.
IV. LEARNING MODEL
The learning model uses an LSTM network for regression-based highway trajectory prediction, with a reference architecture built around one recurrent layer and direct target-state inputs to the output.
- Because future vehicle positions form a regression problem, the paper uses an LSTM artificial neural network suited to time series.
- LSTM cells combine current inputs with recurrent state through forget, input, and output gates.The internal memory state allows the network to learn long-term relations between features.
- The reference design uses a first layer of 256 LSTM cells followed by dense and time-distributed layers.The architecture is presented as a reference design for comparison with variations.
- Stacked LSTM layers did not improve performance over a single layer in the authors’ experiments for this application.
- The first four target-state inputs bypass the LSTM and are fed directly to the dense output layer.This bypass lets the recurrent layer focus on variations from current states rather than steady constant-speed driving.
V. RESULTS
The study evaluates LSTM trajectory predictors on held-out US101 vehicles, comparing architectures, input variants, and prediction horizons. The bagged predictor performs strongly overall, but delayed responses and aggregate metrics limit interpretation.
- Evaluation setup: 80% of vehicles, comprising 4892 trajectories, were used for training, while 20%, comprising 1209 trajectories, were withheld for testing.
- Training setup: The models use 100-input windows representing 10 s of past observations, with overlapping windows and a batch tensor shaped B × 100 × N.
- Model comparisons: The evaluation compares a reference architecture with vehicle-type information, removed inputs, bypass variations, an additional LSTM layer, and alternative dense-layer configurations.
- Model comparisons: The bagged predictor averages the outputs of the four best models and almost always performs best on testing data; longer horizons also favor LSTM-based prediction relative to the cited multilayer perceptron.
- Prediction results: The reference architecture gives the best overall lateral-position results but is less precise for velocity, while vehicle type improves longitudinal-speed forecasting without improving lateral prediction.
- Error characterization: RMSE and error distributions can underrepresent meaningful events because constant-speed driving dominates the data, while prediction delays become more prominent at longer horizons.
- Error characterization: For a 10 s horizon, delay can reach 8 s or 9 s in the worst cases, indicating occasional failure to interpret observed behaviors.
- Future improvements: Separately training each network output appears to improve results but increases training time, motivating alternative architectures and event-specific evaluation metrics.
VI. CONCLUSION
The paper proposes an LSTM architecture for predicting highway vehicle trajectories from naturalistic NGSIM data. It reports improved accuracy over prior state-of-the-art results, while identifying delayed responses and broader generalization as unresolved issues.
- The proposed LSTM network predicts future highway vehicle trajectories using naturalistic driving data from the NGSIM dataset.
- The model achieves roughly 70 cm average RMS error for lateral position 10 s ahead and below 3 m s−1 for longitudinal velocity at the same horizon.
- Using the whole US101 dataset avoids the selection bias associated with handpicked trajectories and should improve suitability for real-world scenarios.
- The work remains preliminary, with delayed responses, limited generalizability, and no current modeling of stochasticity or confidence levels.