PaperlayerView exact revision on arXiv

Source-linked AI summary

GRIP++: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous Driving

Xin Li, Xiaowen Ying, Mooi Choo Chuah

arXiv:1907.07792v2cs.CVcs.RO

TL;DR

Autonomous-driving safety requires accurate prediction of surrounding agents’ future trajectories, but urban traffic involves heterogeneous agents and complex interactions that challenge existing approaches. GRIP++ combines fixed and dynamic graphs with an encoder-decoder GRU-based predictor to forecast all observed objects simultaneously, achieving higher reported accuracy and substantially faster execution than existing schemes.

  • Problem

    Autonomous-driving safety depends on trajectory prediction, while prior GRIP used a fixed interaction graph that may degrade in complex urban traffic.

  • Method

    GRIP++ uses fixed and dynamic graphs to model interactions among traffic agents and an encoder-decoder GRU-based model to predict their trajectories simultaneously.

  • Results

    GRIP++ achieves higher prediction accuracy than existing schemes across highway and urban datasets and runs 21.7 times faster than CS-LSTM.

  • Takeaways & Limitations

    The scheme provides a faster trajectory-prediction approach that models interactions among multiple surrounding traffic agents for autonomous driving.

Abstract

from arXiv · show

Despite the advancement in the technology of autonomous driving cars, the safety of a self-driving car is still a challenging problem that has not been well studied. Motion prediction is one of the core functions of an autonomous driving car. Previously, we propose a novel scheme called GRIP which is designed to predict trajectories for traffic agents around an autonomous car efficiently. GRIP uses a graph to represent the interactions of close objects, applies several graph convolutional blocks to extract features, and subsequently uses an encoder-decoder long short-term memory (LSTM) model to make predictions. Even though our experimental results show that GRIP improves the prediction accuracy of the state-of-the-art solution by 30%, GRIP still has some limitations. GRIP uses a fixed graph to describe the relationships between different traffic agents and hence may suffer some performance degradations when it is being used in urban traffic scenarios. Hence, in this paper, we describe an improved scheme called GRIP++ where we use both fixed and dynamic graphs for trajectory predictions of different types of traffic agents. Such an improvement can help autonomous driving cars avoid many traffic accidents. Our evaluations using a recently released urban traffic dataset, namely ApolloScape showed that GRIP++ achieves better prediction accuracy than state-of-the-art schemes. GRIP++ ranked #1 on the leaderboard of the ApolloScape trajectory competition in October 2019. In addition, GRIP++ runs 21.7 times faster than a state-of-the-art scheme, CS-LSTM.

I. INTRODUCTION

Trajectory prediction is central to autonomous-driving safety, but existing approaches struggle with complex interactions, heterogeneous agents, and urban conditions. GRIP++ addresses these gaps by modeling inter-agent interactions and targeting simultaneous prediction across traffic agents.

  • Autonomous-driving safety depends on accurately predicting the future locations of surrounding vehicles, pedestrians, and bicycles.
  • Trajectory prediction is challenging because many factors affect future motion, while maneuver-based methods can fail when maneuver types are inferred incorrectly.
  • Existing approaches often neglect surrounding-object states, despite heterogeneous agents having different shapes, dynamics, movement patterns, and mutual behavioral effects.
  • GRIP predicts nearby-object locations simultaneously and is trainable end-to-end, but its prior evaluation covered highway environments rather than urban driving.
  • GRIP++ uses both fixed and dynamic graphs to capture complex interactions among different traffic-agent types for improved urban trajectory prediction.
  • The paper evaluates the scheme on highway and urban datasets, reporting higher accuracy and order-of-magnitude faster execution than existing schemes.

III. PROBLEM FORMULATION

The problem formulation estimates future positions for all observed objects from their trajectory histories. Unlike approaches that predict one object at a time, the proposed model outputs future positions for every observed object over the prediction horizon.

  • The model input X consists of trajectory histories for all observed objects across th time steps.
  • At each time t, the representation contains the x and y coordinates of all n observed objects in global coordinates.
  • The model outputs Y, predicted future positions for all observed objects from time step th + 1 through th + tf.

IV. PROPOSED SCHEME

GRIP++ preprocesses object trajectories, represents inter-object interactions with a graph, and predicts trajectories using the proposed architecture. Its graph construction combines spatial proximity and temporal continuity, with a manually defined fixed graph representation.

  • GRIP++ comprises an input preprocessing model, graph convolutional model, and trajectory prediction model.
  • Input Preprocessing Model: Preprocessing stores n objects observed over th time steps in an n×th×c array, using x and y coordinates and calculated velocities as inputs.
  • Graph Construction:: The model represents inter-object interactions with an undirected graph whose nodes correspond to object states at particular time steps.
  • Graph Construction:: Spatial edges connect objects closer than Dclose, while temporal edges connect each object across consecutive time steps.
  • Graph Construction:: The graph uses adjacency matrices A0 and A1, where A0 represents temporal self-connections and A1 represents spatial connections.
  • Graph Construction:: Because its adjacency structure is manually designed from the input and remains unchanged during training, this representation is called a fixed graph.

B. Graph Convolutional Model

The Graph Convolutional Model maps trajectory inputs into higher-dimensional features, then combines spatial interaction modeling with temporal feature extraction. Its graph operations use both manually constructed and trainable graphs to better represent object interactions.

  • A 1×1 convolution maps the input representation into 64-channel features while preserving the object-by-time structure.The resulting feature map has shape (n × th × C), with C = 64.
  • Graph operations model inter-object spatial interactions, while temporal convolutions capture motion patterns over time.Each illustrated Graph Operation layer is followed by a Temporal Convolution layer.
  • Each Graph Operation layer combines a fixed adjacency-based graph with a same-shaped trainable graph.The fixed graph is constructed from the current input, whereas the trainable graph is learned.
  • The trainable graph is added to the fixed graph because manually designed connectivity may represent object interactions imperfectly.This learned component is intended to alleviate the fixed graph’s limitation.
  • Graph operations preserve the feature-map size, producing fgraph with shape (n × th × C).

2) Temporal Convolutional Layer:

The temporal prediction module processes graph features with recurrent encoder-decoder networks and converts predicted velocities into future coordinates. GRIP++ differs from GRIP through velocity inputs, trainable graph support, a lighter graph model, and recurrent ensemble averaging.

  • Temporal Convolutional Layer:: Graph features are fed across time into the encoder cells of Seq2Seq recurrent networks.The trajectory model uses multiple networks sharing the same Seq2Seq structure but trained with different weights.
  • Trajectory Prediction Model: The decoder uses encoder hidden features and previous coordinates to predict current positions recursively through the future horizon.A residual connection makes the decoder predict velocity changes rather than assuming constant velocity.
  • Trajectory Prediction Model: Predicted velocities from the Seq2Seq networks are averaged at each time step and added to the last historical location to obtain coordinates.
  • Differences from GRIP: GRIP++ takes velocity changes as input, whereas GRIP takes x,y coordinates as input.
  • Differences from GRIP: GRIP++ uses fixed and trainable graphs, three graph-convolution blocks with batch normalization and skip connections, and GRUs with three averaged encoder-decoder blocks.GRIP uses fixed graphs, ten graph-convolution blocks without batch normalization, LSTMs, and one encoder-decoder block.

D. Implementation Details

Implementation processes nearby traffic within a defined spatial range, extracts graph-temporal features, predicts trajectories with GRUs, and trains the model by minimizing per-time-step regression loss.

  • The implementation uses Python and PyTorch.
  • Input Preprocessing Model: Scenes are processed within 180 feet, and objects less than 25 feet apart are connected by spatial graph edges.The observed region extends ±90 feet around the relevant scene area.
  • Graph Convolutional Model: The graph-convolution model uses three graph-operation layers, each followed by a temporal convolution with a (1×3) kernel.A 1×1 convolution first increases the input channel count to 64.
  • Trajectory Prediction Model: The trajectory predictor uses two-layer GRU encoder and decoder networks with hidden size r×2×n, choosing r = 30.Here n is the number of objects and the factor 2 represents x,y coordinates.
  • Optimization: Training treats prediction as a regression task at each time step and minimizes the overall loss.The loss aggregates the losses over future time steps.
  • Training Process: The model is trained with Adam using PyTorch defaults and batch size 64.

V. EXPERIMENTS

GRIP++ is evaluated on freeway and urban trajectory datasets using dataset-specific protocols and metrics. The experiments cover real traffic conditions, historical-to-future trajectory splits, and ApolloScape’s weighted displacement measures.

  • Datasets: The evaluation uses NGSIM I-80, US-101, and the ApolloScape Trajectory dataset.
  • NGSIM Datasets: NGSIM I-80 and US-101 contain real freeway vehicle trajectories captured at 10 Hz across mild, moderate, and congested traffic.
  • NGSIM Datasets: For NGSIM, each 8-second clip uses 3 seconds of observed history and 5 seconds of prediction ground truth.Testing data includes one-fourth of each traffic-condition subset, with downsampling by a factor of 2 for fair comparison.
  • ApolloScape Trajectory Dataset: ApolloScape contains urban rush-hour sequences captured at 2 frames per second, with 53 minutes for training and 50 minutes for testing.It provides object identity, type, position, size, and heading information.
  • ApolloScape Trajectory Dataset: ApolloScape training uses 80% of training sequences, reserves 20% for validation, and submits test predictions to the ApolloScape website.
  • Metrics: Freeway results use RMSE over predicted trajectories across 5-second horizons.
  • Metrics: ApolloScape evaluation uses WSADE and WSFDE, weighted sums of average and final displacement errors for vehicles, pedestrians, and cyclists.The weights are based on reciprocals of average agent velocities and are set to 0.20, 0.58, and 0.22.

C. Ablation Study

The ablations examine neighborhood distance, object context, prediction horizon, and baseline comparisons. Nearby-object context improves predictions, while GRIP++ outperforms CS-LSTM and improves long-horizon forecasts over GRIP.

  • Considering surrounding objects improves trajectory prediction compared with using none.
  • Prediction error increases when Dclose expands from 25 feet to 50 feet because more surrounding objects are included.The study therefore uses Dclose = 25 feet by default.
  • Prediction error is lower at location −90 than at 90, indicating that front objects are more important than objects behind the target.Error decreases from −90 to −45 and increases afterward as fewer front objects are observed.
  • GRIP++ improves prediction performance by at least 30% over CS-LSTM and is at least half a meter shorter in error after 3 seconds.GRIP++ predicts all observed objects simultaneously, whereas the listed comparison schemes predict one middle object at a time.
  • GRIP++ is comparable to GRIP for the first three seconds and performs better at 4 and 5 seconds on the NGSIM datasets.The NGSIM datasets contain vehicle trajectories from freeway traffic with similar and straightforward motion patterns.

E. Experiments on the ApolloScape Trajectory Datasets

ApolloScape experiments compare GRIP++ with published methods and isolate the effects of its architectural choices. GRIP++ performs strongly on the urban benchmark, while velocity inputs, residual connections, trainable graphs, and GRUs contribute to the reported design.

  • GRIP++ achieves 85% improvement over TrafficPredict and better performance than StarNet on all ADE and FDE metrics.
  • Using velocity instead of normalized positions produces a significant improvement in prediction performance.The authors attribute this to velocity being more constant across object locations than physical position.
  • Adding a residual connection between Decoder GRU input and output improves performance by letting the model learn acceleration rather than velocity.The ablation describes acceleration as the change in velocity.
  • A trainable graph improves prediction over the fixed-graph configuration by addressing the fixed graph’s shortcomings.Each Graph Operation layer combines the fixed graph with a trainable graph before graph operation.
  • Replacing LSTM with GRU yields a surprising improvement, which the authors associate with limited ApolloScape training data for the LSTM model.
  • Reducing the graph and temporal-convolution layers from 10 to 3 gives similar performance and supports a faster, simpler model.The paper uses fewer layers for faster training and testing speed.
  • Performance improves as the RNN hidden-size ratio increases up to r = 30, after which additional parameters can cause overfitting.
  • GRIP++ achieves 83% improvement over the B1 configuration on the ApolloScape Trajectory Dataset, while BatchNorm, extra Seq2Seq models, and augmentation help only slightly.The authors characterize the resulting model as more robust and useful in real-world scenarios.

F. Computation Time

The computation-time experiment compares CS-LSTM, GRIP, and GRIP++ under matched implementations. GRIP++ is substantially faster, especially at batch size 1, while predicting trajectories for all observed objects simultaneously.

  • The experiment measures computation time for CS-LSTM, GRIP, and GRIP using PyTorch implementations.
  • At batch size 128, GRIP++ predicts trajectories for 1,000 objects in 0.02s, 14.5x faster than CS-LSTM.CS-LSTM takes 0.29s and GRIP takes 0.05s under the same batch-size setting.
  • At batch size 1, GRIP++ runs 21.7 times faster than CS-LSTM, while GRIP runs 5.5 times faster.Batch size 1 is used because the autonomous-driving application has limited resources.
  • GRIP and GRIP++ are faster partly because they predict all observed objects simultaneously, unlike CS-LSTM’s one-object prediction.GRIP++ also uses 3 Graph Operation and 3 Temporal Convolution layers, compared with 10 of each in GRIP.

G. Visualization of Prediction Results

GRIP++ predicts trajectories over a 5-second future horizon from 3 seconds of observed history and outperforms CS-LSTM in the visualized scenarios. It simultaneously predicts all observed objects, including interacting vehicles and scenes with differing speeds.

  • GRIP++ predicts 5-second trajectories after observing 3 seconds of history in mild, moderate, and congested traffic conditions.The visualizations use NGSIM I-80 and US-101 highway scenarios.
  • GRIP++ produces predictions closer to ground truth than CS-LSTM for the central object across the visualized scenarios.The comparison uses identical history trajectories for all objects in each scene.
  • GRIP++ correctly predicts a lane-changing car and the trajectory of another car affected by it.This example demonstrates prediction of interacting vehicles during a future lane change.
  • GRIP++ predicts all objects simultaneously, including a 15-car scene containing slower middle-lane vehicles and faster right-lane cars.CS-LSTM predicts only the object located in the middle.
  • The visualized results indicate improved trajectory prediction over existing methods, and the approach also works on curved roads.Figure 5 itself shows straight highway scenarios, while the text states equivalent performance on curved roads.

VI. CONCLUSION

GRIP++ predicts future trajectories for all observed objects using graph-based interaction modeling and an encoder-decoder GRU model. Experiments across highway and urban datasets report better results than existing methods, faster execution, and planned integration with perception and route planning.

  • GRIP++ represents interactions among nearby objects with a graph and uses an encoder-decoder GRU-based model for trajectory prediction.
  • GRIP++ predicts trajectories for all observed objects simultaneously rather than only one traffic agent at a time.
  • 21.7 times faster than one state-of-the-art scheme, GRIP++ achieves better prediction results across two highway and one urban traffic scenario datasets.Compared with GRIP, it achieves similar highway performance and an 83% improvement in urban scenarios.
  • The authors plan to integrate GRIP++ with route planning and deep-learning-based perception before evaluating the combined system in prototype robotic cars.
Loading 1907.07792v2…