Source-linked AI summary
TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents
Yuexin Ma, Xinge Zhu, Sibo Zhang, Ruigang Yang, Wenping Wang, Dinesh Manocha
TL;DR
Autonomous vehicles need accurate trajectories for interacting vehicles, bicycles, and pedestrians in complex urban traffic. TrafficPredict uses an LSTM-based 4D Graph with instance and category layers, and reports higher accuracy than prior methods on a newly collected dataset. Its accuracy varies with traffic conditions and the duration of past trajectories.
Problem
Autonomous vehicles need trajectory predictions that account for diverse traffic-agents and their interactions in complex urban environments.
Method
TrafficPredict uses an LSTM-based 4D Graph with an instance layer for agent movements and interactions and a category layer for same-type movement similarities.
Results
TrafficPredict outperforms prior state-of-the-art approaches in trajectory-prediction accuracy on a newly collected heterogeneous-traffic dataset.
Takeaways & Limitations
The method provides a unified framework for predicting trajectories of different kinds of traffic-agents in urban traffic.
Takeaways & Limitations
Accuracy varies with traffic conditions and the duration of past trajectories, and evaluation in denser scenarios remains future work.
Abstract
from arXiv · showhide
To safely and efficiently navigate in complex urban traffic, autonomous vehicles must make responsible predictions in relation to surrounding traffic-agents (vehicles, bicycles, pedestrians, etc.). A challenging and critical task is to explore the movement patterns of different traffic-agents and predict their future trajectories accurately to help the autonomous vehicle make reasonable navigation decision. To solve this problem, we propose a long short-term memory-based (LSTM-based) realtime traffic prediction algorithm, TrafficPredict. Our approach uses an instance layer to learn instances' movements and interactions and has a category layer to learn the similarities of instances belonging to the same type to refine the prediction. In order to evaluate its performance, we collected trajectory datasets in a large city consisting of varying conditions and traffic densities. The dataset includes many challenging scenarios where vehicles, bicycles, and pedestrians move among one another. We evaluate the performance of TrafficPredict on our new dataset and highlight its higher accuracy for trajectory prediction by comparing with prior prediction methods.
Introduction
TrafficPredict addresses trajectory prediction in heterogeneous urban traffic, where diverse agents interact and exhibit different motion patterns. It combines instance- and category-level modeling, introduces a dataset, and reports improved accuracy over prior approaches.
- Heterogeneous urban traffic contains agents with different shapes, dynamics, behaviors, and interactions, making trajectory prediction important for safe autonomous navigation.
- Prior motion-model and LSTM-based methods have limited ability to model interacting heterogeneous traffic, often focusing on a single agent type.
- TrafficPredict is an LSTM-based trajectory-prediction algorithm with an instance layer for movements and interactions and a category layer for same-type behavior similarities.
- 20% accuracy improvement is reported for TrafficPredict over prior prediction schemes in a heterogeneous urban traffic scenario.
- The authors provide a new trajectory dataset covering complex urban traffic with interactions among different categories of traffic-agents.
- The paper is organized around related work, algorithm and problem definition, and evaluation on the new traffic dataset.
Related Work
Prior trajectory methods model object motion, human interactions, or sequence dependencies, but existing datasets and approaches leave heterogeneous urban interactions insufficiently covered. TrafficPredict’s 4D Graph represents instance interactions and category structure for this setting.
- Classical trajectory methods include Bayesian networks, Monte Carlo simulation, HMMs, Kalman filters, and Gaussian-process regression.
- Social Force, continuum-dynamics, Gaussian-process, and ensemble-filter methods address pedestrian or crowd behavior and interactions.
- RNN and LSTM methods support sequence generation and trajectory prediction, while related approaches commonly target pedestrians, vehicles, or other single groups.
- RNN networks for sequence prediction: TrafficPredict’s 4D Graph separates instance-level spatial and temporal relations from category-level temporal relations among super nodes.
- Traffic datasets: Existing datasets cover street scenes, highways, short vehicle tracklets, or pedestrian crowds, but do not adequately provide complex heterogeneous urban trajectories.
TrafficPredict
TrafficPredict defines heterogeneous traffic-agent trajectory prediction by observing agent features over time and forecasting future discrete positions. The formulation represents each agent with spatial coordinates and a category label.
- Each traffic-agent feature contains x-axis and y-axis coordinates plus its category.The dataset distinguishes pedestrians, bicycles, and cars using category values 1, 2, and 3.
- The model currently represents three traffic-agent types: pedestrians, bicycles, and cars.The approach is described as extensible to additional agent types.
- TrafficPredict observes all traffic-agent features during [1:Tobs] and predicts discrete positions during [Tobs+1:Tpred].
4D Graph Generation
TrafficPredict builds a 4D Graph that combines agent interactions, temporal information, and category-level similarities. Instance and category layers exchange information through nodes, edges, recurrent processing, and attention mechanisms to refine trajectory prediction.
- Instance Layer: Spatial edges transfer interaction information between agents, while temporal edges pass historical information across adjacent frames.The instance layer models each traffic-agent as a node connected through spatial and temporal relationships.
- Category Layer: The category layer groups same-type instances under a super node, summarizes their movement similarities, and sends guidance back to the instances.Self-attention explores internal sequence patterns, and the super node receives averaged movement features from category members.
- 4D Graph Generation: The 4D Graph represents traffic-agents and interactions, time series, and high-level categories as four dimensions.Nodes represent valid instances and categories, while edges carry spatial and temporal relationships.
- Instance Layer: The instance layer assigns LSTMs to agent nodes and edges, sharing parameters among instances or edges with the same relevant type or role.Three instance LSTMs correspond to vehicles, bicycles, and pedestrians; spatial and temporal edges also share parameters within their groups.
- Instance Layer: Soft attention weights spatial edges according to their differing influence on an instance’s behavior.The weighted interaction information is combined with temporal information and embedded before entering the instance LSTM.
- Position Estimation: The predicted next-frame position is modeled with a bivariate Gaussian whose parameters are estimated from the agents’ second hidden state.Training minimizes the negative log-likelihood and jointly back-propagates through nodes and spatial and temporal edges.
Experiments
TrafficPredict is evaluated on a large-scale heterogeneous urban-traffic dataset using displacement metrics and comparisons with prior methods. It achieves lower prediction error, with improvements especially evident when its category layer and self-attention are included, and produces accurate trajectories in varied and curved scenarios.
- Dataset: The dataset contains heterogeneous urban traffic trajectories collected with camera, LiDAR, radar, maps, and localization sensors.It includes RGB videos with 100K 1920 × 1080 images and around 1000km of trajectories for moving traffic agents.
- Metrics: Average displacement error and final displacement error are measured for pedestrians, bicycles, vehicles, and all traffic agents.The benchmark uses 2 seconds of observation and 3 seconds of prediction.
- Quantitative comparison: TrafficPredict outperforms ED, Social LSTM, and Social Attention across the reported metrics, with about 20% accuracy improvement.Adding the category layer improves both metrics, while self-attention makes the improvement more evident.
- Qualitative results: In camera-based illustrations, TrafficPredict trajectories are closest to ground truth in most of six scenarios with different road conditions and traffic situations.The figure compares ground truth with ED, SL, SA, and TrafficPredict trajectories.
- Qualitative results: Figure 6 visualizes vehicle, bicycle, and pedestrian ground-truth trajectories with yellow-star predicted locations after five observed positions.Pedestrian trajectories have some overlaps in the illustration.
- Qualitative results: TrafficPredict computes accurate predicted trajectories in cross-road scenarios containing overlapping paths and high-curvature left turns.Prediction accuracy also increases when prior trajectories are available over a longer duration.
Conclusion
TrafficPredict predicts heterogeneous traffic-agent trajectories in urban environments using instance- and category-level modeling within a 4D Graph. The authors report improved accuracy on a newly collected dataset, realtime operation, and limitations tied to traffic conditions and past-trajectory duration.
- Conclusion: TrafficPredict uses an instance layer for trajectories and interactions and a category layer for same-type movement similarities to guide prediction.The 4D Graph transfers spatial and temporal information through the model.
- Conclusion: The method outperforms previous state-of-the-art approaches on the authors’ heterogeneous urban-traffic dataset.The evaluation covers dense urban scenarios with heterogeneous traffic agents.
- Conclusion: TrafficPredict is realtime and makes no assumption about traffic conditions or the number of agents.The authors describe it as applicable to heterogeneous traffic conditions.
- Limitations: Accuracy varies with traffic conditions and the duration of past trajectories.The authors propose adding lane direction, traffic signals, and traffic rules, and evaluating denser scenarios.