PaperlayerView exact revision on arXiv

Source-linked AI summary

Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture

Seong Hyeon Park, ByeongDo Kim, Chang Mook Kang, Chung Choo Chung, Jun Won Choi

arXiv:1802.06338v3cs.LG

TL;DR

The paper addresses real-time prediction of complex surrounding-vehicle trajectories for autonomous-driving safety. It uses an LSTM encoder-decoder with beam search to generate multiple likely future trajectories on an occupancy grid map, and reports significantly improved accuracy over conventional methods in highway traffic scenarios.

  • Problem

    Complex, dynamically changing traffic makes long-term surrounding-vehicle trajectory prediction difficult, although such predictions support collision-minimizing path planning.

  • Method

    An LSTM encoder summarizes recent sensor trajectories and ego state, while an LSTM decoder recursively generates future occupancy-grid trajectories using beam search to retain K candidates.

  • Results

    The proposed method significantly improves prediction accuracy over existing methods; increasing Ω from 1 to 3 decreases MAE by around 0.2 grid for higher Δ.

  • Takeaways & Limitations

    Generating multiple trajectory hypotheses provides more informative and reliable predictions for subsequent autonomous-driving path-planning and control steps.

Abstract

from arXiv · show

In this paper, we propose a deep learning based vehicle trajectory prediction technique which can generate the future trajectory sequence of surrounding vehicles in real time. We employ the encoder-decoder architecture which analyzes the pattern underlying in the past trajectory using the long short-term memory (LSTM) based encoder and generates the future trajectory sequence using the LSTM based decoder. This structure produces the $K$ most likely trajectory candidates over occupancy grid map by employing the beam search technique which keeps the $K$ locally best candidates from the decoder output. The experiments conducted on highway traffic scenarios show that the prediction accuracy of the proposed method is significantly higher than the conventional trajectory prediction techniques.

I. INTRODUCTION

The paper targets real-time prediction of complex surrounding-vehicle trajectories for safer autonomous driving and proposes an LSTM encoder-decoder with beam search to generate multiple likely futures.

  • Motivation: Trajectory prediction is critical because autonomous vehicles can use predicted surrounding-vehicle paths to plan responses that minimize collision probability.Traffic participants’ motion is governed by latent factors that can change dynamically in real time.
  • Prior approaches: Traditional Kalman and extended Kalman filters may inadequately model complicated vehicle motion and often perform poorly for long-term prediction.The cited example of long-term prediction is Δ = 2 sec.
  • Prior approaches: Dynamic Bayesian networks explicitly model physical trajectory-generating processes but can be limited by designer-selected structures and high inference complexity for real-time applications.The model structure may not capture the variety of dynamic traffic scenarios.
  • Prior approaches: Existing RNN-based trajectory models either cannot generate the full trajectory sequence or lack a simple probabilistic framework without extra components such as CVAE.The paper positions its approach as addressing these limitations with an encoder-decoder design and beam search.
  • Proposed approach: The proposed LSTM encoder summarizes recent surrounding-vehicle trajectories and ego state, while the decoder recursively generates future OGM trajectories.Beam search retains K locally best candidates at each step, yielding K probable trajectory hypotheses.

A. Long-Short Term Memory (LSTM)

LSTM is an RNN variant that summarizes past inputs in cell memory and regulates information flow through input, forget, and output gates.

  • LSTM properties: LSTM addresses the vanishing-gradient issue that affects naively designed recurrent neural networks.It is introduced as a recurrent neural network variant for sequence modeling.
  • LSTM properties: The cell memory stores a summary of the past input sequence, while gating controls information flow among input, output, and cell memory.The architecture uses recursive updates to maintain and transform sequence information.
  • LSTM notation: The LSTM formulation uses linear transformation matrices, bias vectors, gating vectors, a cell-memory state vector, and a state-output vector.The passage identifies these symbol groups as components of the recursive equations.
  • LSTM operation: The gating vectors determine how much information the cell memory updates, forgets, and outputs.The forget gate can reset or restore cell state, while the input and output gates regulate corresponding information flows.

B. LSTM Encoder-Decoder Architecture

The encoder-decoder reads an input sequence into a final cell state and recursively generates an output sequence by feeding each previous output into the next decoding step.

  • Encoder: The encoder processes an arbitrary-length input sequence and compresses it into the final cell state vector c_T.After T recursive updates, c_T summarizes the whole input sequence and initializes sequence generation.
  • Decoder: The decoder starts from a dummy input and recursively generates an output sequence of arbitrary length.At each update, the previous decoder output becomes the next input.
  • Probabilistic formulation: The architecture models the conditional probability of an output sequence given an input sequence.The encoder state provides the input-sequence summary used to approximate this conditional distribution.
  • Autoregressive decoding: Because the true previous output is unavailable during decoding, the decoder uses a tentative decision from its predicted distribution for the next update.This autoregressive feedback determines the subsequent decoder state.

C. The Beam-Search Algorithm

Greedy decoding can propagate early mistakes and represent only one outcome, so beam search retains multiple high-probability hypotheses during sequence generation.

  • Greedy search: Greedy search selects the most probable current output and feeds it back to generate the next sample.Its single-decision strategy can propagate a wrong choice through later time steps.
  • Need for beam search: A multimodal decoder distribution may contain several equally promising candidates, making one generated hypothesis insufficient to represent probable outcomes.The multiple peaks correspond to alternative plausible next outputs.
  • Beam search: Beam search keeps the K most probable hypotheses at each sequence-generation step, where K is the beam width.For K previous sequences, it evaluates K × |S| candidate extensions and retains the best K.
  • Beam search: Beam width K = 1 reduces beam search to greedy search.Larger beam widths preserve multiple candidate sequences rather than committing to one at each step.

III. PROPOSED TRAJECTORY ANALYSIS TECHNIQUE

The system represents surrounding-vehicle motion on an occupancy grid map and uses an LSTM encoder-decoder with beam search to generate multiple future trajectory hypotheses.

  • System Description: The system predicts each surrounding vehicle’s future trajectory from recent observations of relative position, relative velocity, and ego-vehicle speed.The ego vehicle’s motion is obtained from IMU measurements, while surrounding-vehicle state is estimated from sensors.
  • System Description: The occupancy grid map divides the region around the ego vehicle into 36 × 21 rectangular elements, each spanning 5.0 meters longitudinally and 0.875 meters laterally.The relative coordinate system fixes the ego vehicle at (0, 0), with sensor-determined ranges of x ∈[0, 180] and y ∈[−9.2, 9.2] meters.
  • System Description: The encoder processes the latest M observations, after which the decoder sequentially predicts the trajectory for ∆ time steps.Beam search generates K likely trajectory sequences in parallel by retaining locally best candidates at each decoding step.
  • System Description: At each future step, the decoder produces occupancy probabilities for each surviving hypothesis and selects K trajectory candidates with the largest occupancy probability.Selected grid indices are embedded and fed back into the decoder for the next step, repeating until the prediction horizon is reached.
  • System Description: Sharing encoder and decoder parameters across vehicles yields K×N future trajectory hypotheses on a single occupancy grid map.This provides a unified view of how N surrounding vehicles may develop ∆ time steps ahead.

1) Encoder:

The encoder transforms recent vehicle observations into a fixed-dimensional representation for the decoder, while the decoder uses stacked LSTMs and occupancy-grid probabilities to generate candidate trajectories.

  • Encoder: The encoder uses three fully connected layers followed by two stacked LSTM layers to transform six-dimensional inputs into 256-dimensional trajectory features.The fully connected layers use affine transformations with ReLU activation and expand representation capacity for complex trajectory patterns.
  • Decoder: The decoder contains two stacked LSTM layers followed by three fully connected layers, a Softmax layer, and an embedding layer.The decoder LSTMs are initialized with cell-state vectors passed from the encoder.
  • Decoder: The final fully connected layer has 757 outputs corresponding to 36 × 21 grid cells plus one out-of-map class.Softmax converts these outputs into an occupancy probability map of size 757.
  • Decoder: Beam search selects the most probable K trajectory candidates, whose longitudinal and lateral grid indices are embedded into 256-dimensional decoder inputs.The embedding matrices have dimensions 128 × 37 and 128 × 22, and the resulting vectors are fed back for the next decoding step.

C. Training Methodology

The model is trained end to end on cropped trajectory sequences using subsequent occupancy-grid indices as supervised labels and minimizing negative log likelihood.

  • Training Data: Training data are generated by cropping trajectory samples of length M + ∆ from recorded trajectories of surrounding vehicles.Subsequent ∆ measurements provide the future occupancy-grid labels for supervised learning.
  • Objective: The loss is a negative log likelihood over training samples, prediction steps, and Q occupancy-grid classes.The one-hot label identifies the occupied grid element, while the Softmax output supplies its predicted probability.
  • Optimization: Optimization uses the ADAM optimizer with momentum and mini-batch size B, stopping when validation error from 15% of the training data no longer decreases.The validation subset is used as the stopping criterion during training.

IV. EXPERIMENTS

Experiments evaluate the proposed predictor on real highway-driving data using Top-Ω MAE, including longitudinal and lateral trajectory errors.

  • Data Collection: The dataset contains 1,325 trajectory sequences from 26 highway scenarios, including lane changes, cut-ins, and junction merging.Data were collected around Seoul using a Hyundai Genesis equipped with Delphi long-range front radars.
  • Data Collection: The data split uses 85% of sequences for training and 15% for validation, corresponding to 1,126 and 199 sequences.Raw 10 ms sensor samples were averaged over 100 ms because of noise and asynchronous-sampling cut-offs.
  • Experiment Setup: The maximum prediction range is ∆ = 20, corresponding to 2 seconds, with decoder samples generated every 0.2 seconds.This reduces the horizon to 10 decoder updates instead of 20 updates at a 0.1-second sampling interval.
  • Experiment Setup: The evaluated network uses three fully connected layers, 256-dimensional LSTM states, a two-layer LSTM stack, beam width K = 10, and observation length M = 30.These settings were selected through empirical hyperparameter optimization.
  • Evaluation Metric: Top-Ω MAE selects the candidate trajectory closest to ground truth among the top Ω predictions and measures their absolute grid-index error.Errors are also reported separately in longitudinal and lateral directions as Top-ΩMAE_X(δ) and Top-ΩMAE_Y(δ).

C. Experiment Results

The proposed method’s prediction accuracy improves as the number of trajectory hypotheses increases, outperforming simpler baselines and providing full trajectory sequences. Gains are especially pronounced for longer prediction horizons.

  • Baseline comparison: The proposed method with Ω=3 and 5 exceeds baseline [14] because it generates more likely trajectory samples.With Ω=1, performance is comparable to baseline [14], while the proposed method generates full sequences rather than only a future location.
  • Baseline comparison: Prediction accuracy decreases as the prediction horizon Δ increases across the evaluated methods.The comparison includes MAE, MAE_X, and MAE_Y at multiple prediction time steps.
  • Parameter analysis: Figure 6 plots MAE against Ω for several Δ values and also reports MAE averaged across the considered horizons.Use the curves to compare how prediction error changes as the number of hypotheses increases.
  • Parameter analysis: MAE improves for every evaluated Δ as Ω increases, with larger gains at higher Δ.Increasing Ω from 1 to 3 decreases MAE by around 0.2 grid for higher prediction horizons.
  • Parameter analysis: Including more trajectory hypotheses increases the probability that one candidate is close to the ground-truth trajectory.The resulting multiple trajectories are intended to provide more informative and reliable predictions for subsequent planning and control.

V. CONCLUSIONS

The paper proposes an LSTM encoder-decoder vehicle trajectory predictor that uses beam search to generate multiple future trajectory hypotheses. Experiments report improved prediction accuracy while generating full predicted trajectories in one shot.

  • Conclusions: The LSTM encoder analyzes past sensor measurements, while the LSTM decoder generates future trajectory samples from the encoder output.The architecture is designed for vehicle trajectory prediction.
  • Conclusions: Beam search keeps the K best trajectory candidates during iterative decoding to alleviate error propagation.This produces multiple hypotheses that can develop in different ways from the same previous situation.
  • Conclusions: The proposed method improves prediction accuracy over existing methods while generating the full trajectory sequence in one shot.The conclusion reports this outcome without restricting it to a single metric or baseline.
Loading 1802.06338v3…