Source-linked AI summary
Probabilistic Vehicle Trajectory Prediction over Occupancy Grid Map via Recurrent Neural Network
ByeoungDo Kim, Chang Mook Kang, Seung Hi Lee, Hyunmin Chae, Jaekyum Kim, Chung Choo Chung, Jun Won Choi
TL;DR
Vehicle trajectories are hard to predict because they reflect latent factors such as driver intention, traffic situations, and road structure, while conventional dynamics models degrade for longer horizons. The paper uses LSTM networks trained on long-term sensor-derived trajectory data to produce probabilistic future locations on occupancy grids, achieving reasonably accurate highway predictions and outperforming Kalman-filter-based prediction.
Problem
Vehicle behavior is difficult to predict because it depends on latent factors, and dynamics-based models match true trajectories poorly for long-term prediction.
Method
An LSTM learns temporal vehicle-motion dynamics from sensor-derived coordinates and velocities, producing occupancy probabilities for future locations.
Results
The proposed method provides reasonably accurate highway trajectory predictions and outperforms the Kalman filter for all considered cases.
Takeaways & Limitations
Probabilistic occupancy-grid prediction offers a data-driven framework for estimating surrounding-vehicle behavior from real-road trajectory data.
Abstract
from arXiv · showhide
In this paper, we propose an efficient vehicle trajectory prediction framework based on recurrent neural network. Basically, the characteristic of the vehicle's trajectory is different from that of regular moving objects since it is affected by various latent factors including road structure, traffic rules, and driver's intention. Previous state of the art approaches use sophisticated vehicle behavior model describing these factors and derive the complex trajectory prediction algorithm, which requires a system designer to conduct intensive model optimization for practical use. Our approach is data-driven and simple to use in that it learns complex behavior of the vehicles from the massive amount of trajectory data through deep neural network model. The proposed trajectory prediction method employs the recurrent neural network called long short-term memory (LSTM) to analyze the temporal behavior and predict the future coordinate of the surrounding vehicles. The proposed scheme feeds the sequence of vehicles' coordinates obtained from sensor measurements to the LSTM and produces the probabilistic information on the future location of the vehicles over occupancy grid map. The experiments conducted using the data collected from highway driving show that the proposed method can produce reasonably good estimate of future trajectory.
I. INTRODUCTION
Vehicle trajectory prediction is difficult because traffic behavior depends on latent factors and dynamics-based models degrade beyond the near future. The paper proposes learning these dynamics with LSTM from trajectory data and predicting probabilistic future locations.
- Traffic participants are difficult to predict because driver intention, traffic situations, and road structure affect their behavior.
- Dynamics-based prediction is accurate only for the very near future and does not match true trajectories well beyond one second.
- The proposed framework uses an LSTM-based deep neural network to learn complex vehicle-motion dynamics from sequential data.
- Sensor-derived coordinates and velocities are input to the LSTM, which predicts surrounding vehicles’ future locations probabilistically on an occupancy grid map.
- The method differs from related LSTM work by predicting future trajectories over occupancy grids rather than classifying maneuvers or tracking objects.
II. SYSTEM DESCRIPTION
The system represents each surrounding vehicle relative to the ego-vehicle and derives trajectories from time-sequenced sensor measurements. It uses a defined highway-oriented detection range and records relative positions and velocities.
- The ego-vehicle is the coordinate origin, and each surrounding vehicle is represented by its relative position.
- Relative coordinates use longitudinal and lateral directions, while velocity measurements represent each vehicle’s relative velocity.
- Vehicle trajectories are formed from coordinate sequences acquired every Ts second from camera, radar, and Lidar measurements.
- The system considers surrounding vehicles from 0 to 180 meters longitudinally and -9.2 to 9.2 meters laterally.
B. Occupancy Grid Map
The proposed system predicts future vehicle coordinates as occupancy probabilities over a grid aligned with the ego-vehicle’s heading. Grid sizing is chosen to represent lane-relative movement and vehicle extent.
- The trajectory objective is to estimate each surrounding vehicle’s future coordinate and express the result probabilistically.
- The occupancy grid partitions the considered spatial range into Mx × My elements to represent uncertainty in predicted trajectories.
- Grid elements approximately cover a quarter lane and the vehicle length, supporting recognition of same-lane movement.
- The occupancy grid is aligned with the ego-vehicle’s heading because surrounding-vehicle coordinates come from ego-mounted sensors.
III. STRUCTURE OF LSTM
LSTM addresses recurrent-network gradient problems by maintaining a cell state with learned gates that regulate memory updates and information flow. Its state summarizes sequential past inputs.
- LSTM uses a memory cell whose state summarizes the sequence of past input data.
- The cell state is updated from the current input, output, and previous cell state through recursive equations.
- Input, forget, and output gates regulate information entering memory, being discarded, and reaching the output.
- A zero forget-gate configuration can discard the information stored in the previous memory state.
- The gate-control behavior is learned from data, and unfolding the LSTM over time yields a deep model.
IV. PROPOSED VEHICLE TRAJECTORY PREDICTION
This section presents the proposed vehicle trajectory prediction framework.
- The paper introduces the proposed vehicle trajectory prediction framework.
- The framework is described as the central subject of the proposed prediction system.
- The section provides details of the proposed trajectory prediction framework.
A. Proposed Trajectory Prediction Technique
The system uses separate, parameter-shared LSTMs to predict surrounding vehicles’ future occupancy over a grid, while also allowing deterministic coordinate prediction.
- N LSTMs process the estimated coordinates and velocities of the N nearest surrounding vehicles, using shared network parameters.Each LSTM produces a prediction for one surrounding vehicle, with ego-vehicle yaw rate used to compensate coordinate changes.
- For different prediction horizons, the system trains independent LSTM parameter sets for ∆=0.5s, 1.0s, and 2.0s.The trained parameter set corresponding to the selected future time step can then be loaded.
- The occupancy-grid output assigns each grid element a future occupancy probability for the surrounding vehicles.The softmax layer in each LSTM produces the probability for a grid element.
- Po(ix, iy) combines predictions for all N vehicles into one map that summarizes their behavior after ∆seconds.This provides a comprehensive view of the predicted surrounding traffic participants.
- A deterministic alternative directly predicts future x- and y-coordinate values when probabilistic trajectory information is unnecessary.This version uses the same LSTM architecture with a regression loss instead of a softmax layer.
B. Training of LSTM
Training data are generated automatically from long-term sensor-derived trajectories, with future occupancy labels or coordinate targets used to train the LSTM.
- Long-term real-road trajectories are separated by vehicle to generate individual training data for each surrounding vehicle.Vehicle coordinates are obtained using a localization algorithm based on sensor fusion.
- The coordinate ∆seconds later automatically supplies each training example’s label, avoiding human labeling.One-hot encoding marks the occupied element in the occupancy grid map.
- The occupancy-grid model is trained by minimizing a negative log-likelihood objective with regularization.The network parameters are optimized using back propagation through time with mini-batches.
- The deterministic model uses a coordinate-regression loss for directly predicted future positions.Its predicted x- and y-values are produced by the neural network, while the remaining training procedure is similar.
A. Experiment Description
Training data were collected during extended highway driving using a sensor-equipped Hyundai Genesis, producing relative coordinates of surrounding vehicles.
- Training data came from long hours of highway driving in the suburbs of Seoul, Korea.
- The test vehicle was a Hyundai Genesis equipped with Delphi long-range radar and a Mobileye front camera.
- Sensor measurements were processed to obtain surrounding vehicles’ relative coordinates, while ego-vehicle yaw rate and velocity were also logged.
B. System Configurations
The system combines fully connected layers, a two-layer LSTM, output layers, and softmax to predict probabilistic vehicle occupancy. Evaluation uses weighted MAE and compares LSTM predictions with Kalman-filter and regression baselines across prediction horizons.
- Occupancy grid: The occupancy grid map has 36 longitudinal by 21 lateral elements, with each element covering 0.875 meters in width and 5 meters in height.Vehicles leaving the map boundary are assigned to an “out of boundary” class.
- System architecture: The architecture concatenates input fully connected layers, a two-layer LSTM cell memory, output fully connected layers, and a softmax layer.The system uses separately trained LSTM parameter sets for prediction horizons of 0.5, 1.0, and 2.0 seconds.
- Evaluation metric: Weighted mean absolute error (MAE) evaluates the predicted occupancy probabilities, with separate longitudinal and lateral MAE measures reported.The occupancy probability for each grid element is represented by P_o(i_x, i_y).
- Experimental results: Prediction accuracy is better for short-term horizons, with MAE below 1.0 grid for Δ=0.5s and 1.0s and up to 1.5 grid for Δ=2.0s.The paper characterizes these MAE levels as sufficient for risk assessment and path planning.
- Experimental results: The LSTM outperforms the Kalman filter in all considered cases, with a larger performance gap for long-term prediction and lower lateral MAE.The results indicate stronger prediction of complex, long-term vehicle motion and lateral movement.
- Regression evaluation: The regression-trained proposed system also achieves a significant prediction-accuracy gain over the Kalman filter.This evaluation tests deterministic future-coordinate prediction in addition to probabilistic occupancy prediction.
- Qualitative results: The method successfully predicts lane changes after one second and performs reasonably well for the challenging Δ=2.0s case.The examples include two front vehicles changing lanes simultaneously.
VI. CONCLUSIONS
The paper proposes an LSTM-based probabilistic vehicle trajectory prediction method trained on long-term real-road trajectory data. Experiments on real highway data show reasonably accurate predictions of surrounding-vehicle trajectories.
- The proposed method uses LSTM to learn complex surrounding-vehicle motion dynamics from trajectory data processed from sensor measurements.It predicts future vehicle locations probabilistically over an occupancy grid map.
- Experiments using real highway data show reasonably accurate predictions of surrounding vehicles’ trajectories.