PaperlayerView exact revision on arXiv

Source-linked AI summary

Graph-Based Spatial-Temporal Convolutional Network for Vehicle Trajectory Prediction in Autonomous Driving

Zihao Sheng, Yunwen Xu, Shibei Xue, Dewei Li

arXiv:2109.12764v1cs.LGcs.AIcs.RO

TL;DR

Autonomous vehicles need forecasts of nearby vehicles to assess risks and plan motions, while prior approaches face challenges modeling spatial-temporal interactions efficiently. GSTCN combines graph convolution, temporal convolution, and GRU encoding-decoding with weighted vehicle interactions to predict all neighbors’ future trajectory distributions, improving accuracy across prediction horizons and showing potential for deployment through small model size and fast inference.

  • Problem

    Autonomous vehicles need nearby-vehicle trajectory forecasts to assess risks and plan appropriate motions, including detailed future information for all neighbors.

  • Method

    GSTCN uses graph convolution for spatial dependencies, CNN-based temporal extraction, GRU encoder-decoding, and a weighted adjacency matrix to predict all vehicles’ future trajectory distributions.

  • Results

    22.4% average accuracy improvement over GRIP-ALL was achieved when predicting trajectories of all vehicles, while GSTCN outperformed CS-LSTM across all prediction horizons.

  • Takeaways & Limitations

    GSTCN’s smaller model size and faster inference speed indicate potential for real-time deployment in autonomous vehicles.

Abstract

from arXiv · show

Forecasting the trajectories of neighbor vehicles is a crucial step for decision making and motion planning of autonomous vehicles. This paper proposes a graph-based spatial-temporal convolutional network (GSTCN) to predict future trajectory distributions of all neighbor vehicles using past trajectories. This network tackles the spatial interactions using a graph convolutional network (GCN), and captures the temporal features with a convolutional neural network (CNN). The spatial-temporal features are encoded and decoded by a gated recurrent unit (GRU) network to generate future trajectory distributions. Besides, we propose a weighted adjacency matrix to describe the intensities of mutual influence between vehicles, and the ablation study demonstrates the effectiveness of our proposed scheme. Our network is evaluated on two real-world freeway trajectory datasets: I-80 and US-101 in the Next Generation Simulation (NGSIM).Comparisons in three aspects, including prediction errors, model sizes, and inference speeds, show that our network can achieve state-of-the-art performance.

I. INTRODUCTION

Vehicle trajectory prediction supports autonomous-vehicle risk assessment and motion planning, but existing deep-learning approaches struggle with non-Euclidean spatial interactions and real-time computation. GSTCN addresses these limitations with graph, convolutional, and recurrent components, weighted vehicle interactions, and probabilistic trajectory outputs.

  • Trajectory prediction helps autonomous vehicles assess future risks and plan appropriate motions from nearby vehicles’ movement information.
  • Existing LSTM-based approaches model vehicle interactions inefficiently because spatial relations are non-Euclidean and require intensive computation.
  • GSTCN simultaneously predicts future trajectory distributions for all neighbor vehicles using a spatial graph convolutional module and temporal dependency extractor.
  • GCN and CNN backbones give GSTCN smaller model size and faster inference than LSTM-based models, supporting real-time prediction of nearby vehicles’ trajectories.
  • A weighted adjacency matrix represents different mutual-influence intensities between vehicles, and ablation results show better performance than an unweighted matrix.
  • GSTCN predicts future trajectory distributions rather than deterministic trajectories, representing stochastic human-driver behavior especially over long horizons.

II. PROBLEM DESCRIPTION OF VEHICLE TRAJECTORY PREDICTION

The paper formulates vehicle trajectory prediction as estimating future trajectory distributions for all observed neighbor vehicles from their past positions. The setup uses spatial-temporal vehicle information, defined observation limits, and a bi-variable Gaussian assumption for predicted coordinates.

  • The formulation predicts future positions for all neighbor vehicles simultaneously, providing more detailed information about future situations.
  • Vehicle positions X are represented as a sequence of position states P_1 through P_T across the past time horizon.
  • The autonomous vehicle observes nearby motions within ±L meters longitudinally and two adjacent lanes laterally, collecting trajectories at a certain frequency.
  • Predicted coordinates are modeled as random variables satisfying a bi-variable Gaussian distribution, including a correlation term.
  • The prediction task estimates future trajectory distributions Y from all neighbor vehicles’ past positions X over horizons T and F.

III. GRAPH-BASED SPATIAL-TEMPORAL CONVOLUTIONAL NETWORK

GSTCN represents vehicle interactions as a spatial-temporal graph and uses weighted edges to encode how strongly vehicles influence one another. The graph combines vehicle coordinates across past timestamps as the network input.

  • The spatial graph convolution module is designed to capture spatial dependencies among vehicles directly from the graph.
  • GSTCN models vehicle interactions as a spatial-temporal graph whose vertices represent vehicles and edges represent mutual effects.
  • Each vertex represents an individual vehicle, with its attribute defined by the vehicle’s two-dimensional coordinate.
  • The weighted adjacency matrix assigns stronger interaction weights to closer vehicles, reflecting their greater mutual influence.
  • At each past timestamp, the method constructs a spatial graph and stacks these graphs into a spatial-temporal graph.

2) Spatial graph convolution:

The spatial graph convolution aggregates weighted features from each vehicle and its neighbors, while the CNN-based temporal dependency extractor learns evolving interactions across past timestamps. Its output is organized by future prediction steps.

  • Spatial graph convolution: The convolution uses normalized adjacency information, with self-loops added through the identity matrix to support GCN learning.
  • Spatial graph convolution: Graph convolution applies a weighted sum to a target vehicle’s features and those of surrounding vehicles before passing the result to the next layer.
  • Temporal dependency extractor: The temporal dependency extractor receives spatial features arranged by timestamp and learns each vehicle’s evolving pattern from past motions and dynamic neighbor interactions.
  • Temporal dependency extractor: Its filters integrate temporal information across channels, and the number of filters equals the prediction-horizon length.

C. Trajectory prediction module

The trajectory prediction module uses a GRU encoder-decoder to transform extracted spatial-temporal features into future trajectory distributions. Evaluation uses real freeway data and RMSE-based comparisons over prediction horizons from 1 to 5 seconds.

  • Trajectory prediction module: The GRU encoder strengthens correlations between spatial and temporal dependencies, while the decoder generates probability distributions for future trajectories.
  • Trajectory prediction module: Encoder and decoder GRUs share weights across vehicles, supporting generalization when the number of neighboring vehicles varies.
  • Evaluation: The model is evaluated on I-80 and US-101 NGSIM freeway trajectories recorded at 10Hz across mild, moderate, and heavy traffic.
  • Evaluation: The evaluation uses 8-second trajectory segments, with 3 seconds of history and 5 seconds for prediction, producing 13,218 segments.
  • Evaluation metrics: RMSE measures predicted coordinates against ground truth and is reported for prediction horizons from 1 to 5 seconds.

C. Implementation details

The implementation specifies preprocessing, observation limits, convolutional settings, and GRU encoder-decoder details for GSTCN.

  • Data preprocessing: Abnormal sensor data are removed and missing data are inferred using cubic Hermite interpolation.
  • Observation range: The autonomous vehicle observes vehicles within ±100 meters longitudinally and two adjacent lanes laterally.
  • Input embedding: A 32-channel 1 × 1 convolutional layer increases the dimensions of spatial coordinates.
  • Hyperparameter comparison: Fig. 7 compares RMSE across layer configurations, with (α+β) denoting spatial graph-convolution and TDE layer counts.
  • Spatial processing: Residual convolutions use 3 × 3 kernels with padding 1, preserving feature-map shape.
  • Trajectory prediction: The encoder and decoder are one-layer GRUs, followed by a linear layer and dropout with probability 0.5.

6) Training loss:

GSTCN is trained with negative log-likelihood for predicted position distributions, followed by experiments on hyperparameters and module effectiveness.

  • Training loss: GSTCN minimizes negative log-likelihood because its outputs are probability distributions over future trajectories.
  • Optimization: Training uses SGD for 250 epochs with batch size 128, initial learning rate 0.1, and tenfold reductions every 80 epochs.
  • Hyperparameter selection: The best tested configuration uses one spatial graph-convolution layer and five TDE layers.
  • Hyperparameter selection: The lowest RMSE occurs with 32 GRU hidden units among the tested sizes.
  • Ablation study: Removing GCN, TDE, or GRU increases RMSE, supporting the effectiveness of all three modules.
  • Ablation study: The GCN and TDE provide complementary spatial and temporal dependencies, while the GRU strengthens their correlation for trajectory generation.
  • Location analysis: Vehicles in the middle of the scene have the lowest RMSE across prediction horizons.
  • Location analysis: Rear-position vehicles have lower prediction errors than front-position vehicles because drivers are more affected by vehicles ahead.

3) GSTCN with different weighted adjacency matrices:

The adjacency-matrix study tests how interaction weights affect GSTCN, comparing reciprocal-distance weighting with alternative schemes and baseline performance.

  • Weight design: Adjacency weights represent interaction intensity, with reciprocal distance encoding stronger mutual effects for closer vehicles.
  • Weight comparison: The reciprocal-distance weighted adjacency matrix outperforms the alternative weighted matrices.
  • Weight comparison: All weighted adjacency matrices outperform the baseline, demonstrating the effectiveness of weighted adjacency matrices.
  • Weight comparison: Direct distance weighting still yields lower prediction errors than the baseline despite conflicting with the stated intuition.
  • Baselines: The evaluation compares GSTCN against CV, V-LSTM, C-VGMM+VIM, CS-LSTM-M, CS-LSTM, MATF, GRIP-ALL, and GRIP.

F. Quantitative analysis for prediction results

GSTCN is evaluated for single- and all-vehicle prediction, qualitative trajectory distributions, and deployment-oriented model size and inference speed.

  • Single-vehicle prediction: GSTCN-ONE achieves the lowest prediction errors for almost all horizons when predicting one vehicle at a time.
  • Single-vehicle prediction: 7.45% lower average RMSE than GRIP is reported for long prediction horizons, although GRIP performs better at one second.
  • Single-vehicle prediction: GSTCN outperforms CS-LSTM at all horizons, while surrounding-vehicle information improves deep-learning-based prediction relative to V-LSTM.
  • All-vehicle prediction: 22.4% average accuracy improvement over GRIP-ALL is reported when predicting all vehicles simultaneously.
  • All-vehicle prediction: GSTCN outperforms MATF despite not using scene images, while MATF considers spatial relationships at only one timestamp.
  • Model size and speed: GSTCN has about one tenth of GRIP-ALL’s model size and predicts one vehicle in 0.044 ms, about 7.3 times faster.
  • Qualitative analysis: Qualitative visualizations use three seconds of observed trajectories and five seconds of predicted distributions across traffic scenarios.
  • Qualitative analysis: Predicted trajectory distributions capture ground-truth patterns across mild, moderate, and congested traffic scenarios.

H. Robustness to imperfect data

The robustness analysis compares GSTCN under perfect data, partial missingness, and complete vehicle undetection. Imperfect data increase RMSE, with complete undetection producing the larger long-horizon degradation.

  • Partially missing data: 20% randomly deleted trajectory points are reconstructed with cubic Hermite interpolation before GSTCN receives the imperfect sequences.Half of the testing sequences are selected for this partially missing-data case.
  • Totally missing data: Completely discarding one vehicle’s trajectory represents an extreme case in which reasonable input data cannot be inferred.The vehicle is randomly selected from the model input.
  • Results: RMSE is greater with imperfect than perfect data, but the increase is described as acceptable, supporting GSTCN’s robustness and practical potential.The comparison is reported in Table VII, whose values are in meters.
  • Results: Complete undetection causes much larger RMSE increments over the 3-5s horizon than partial undetection of several vehicles.The result indicates that entirely missing one vehicle has a greater effect on prediction accuracy than partial missingness across several vehicles.

APPENDIX

Table VIII reports the speeds of each vehicle during the past time horizon to support interpretation of the predictions in Fig. 9.

  • Appendix: Table VIII presents vehicle speeds for interpreting the predictions shown in Fig. 9.
  • Appendix: The table covers each vehicle represented in the predictions.
  • Appendix: The reported speeds correspond to the past time horizon used for the predictions.
Loading 2109.12764v1…