Source-linked AI summary
TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions
Rohan Chandra, Uttaran Bhattacharya, Aniket Bera, Dinesh Manocha
TL;DR
TraPHic addresses near-term trajectory prediction for dense, heterogeneous traffic involving varied road-agent types and interactions. It uses weighted horizon-based and heterogeneous-based interactions in an LSTM-CNN hybrid network, achieving up to 30% improvement over prior methods on dense traffic datasets while remaining comparable on sparse or homogeneous data.
Problem
Near-term trajectory prediction must handle dense traffic involving heterogeneous road agents and support applications including safe autonomous navigation, traffic forecasting, vehicle routing, and congestion analysis.
Method
TraPHic predicts future trajectories from 2-4 seconds of observed history over a 3-5-second horizon using weighted horizon-based and heterogeneous-based interactions in an end-to-end LSTM-CNN hybrid network.
Results
30% improvement in RMSE over prior methods on dense traffic datasets with 10-30 road agents, while achieving comparable accuracy on sparse or homogeneous datasets such as NGSIM.
Takeaways & Limitations
Weighted interactions, particularly the horizon-based formulation, are associated with the largest ablation gains, and the approach extends evaluation to a new dense, heterogeneous Asian traffic dataset.
Takeaways & Limitations
TraPHic does not outperform prior methods on sparse or homogeneous traffic videos, and heterogeneous modeling requires collecting road-agent shape and size information.
Abstract
from arXiv · showhide
We present a new algorithm for predicting the near-term trajectories of road-agents in dense traffic videos. Our approach is designed for heterogeneous traffic, where the road-agents may correspond to buses, cars, scooters, bicycles, or pedestrians. We model the interactions between different road-agents using a novel LSTM-CNN hybrid network for trajectory prediction. In particular, we take into account heterogeneous interactions that implicitly accounts for the varying shapes, dynamics, and behaviors of different road agents. In addition, we model horizon-based interactions which are used to implicitly model the driving behavior of each road-agent. We evaluate the performance of our prediction algorithm, TraPHic, on the standard datasets and also introduce a new dense, heterogeneous traffic dataset corresponding to urban Asian videos and agent trajectories. We outperform state-of-the-art methods on dense traffic datasets by 30%.
1. Introduction
TraPHic targets realtime trajectory prediction in dense, heterogeneous traffic, where prior methods struggle to model interactions among agents with different physical and behavioral properties. It introduces horizon-based and heterogeneous-based weighted interactions, evaluates them on traffic datasets, and reports a 30% RMSE improvement on dense scenarios.
- Trajectory prediction supports safe autonomous navigation, traffic forecasting, vehicle routing, and congestion-related applications.
- Dense heterogeneous traffic includes agents with different shapes, dynamic constraints, and behaviors, making their interactions distinct.
- Prior methods often target homogeneous agents or relatively sparse heterogeneous scenarios, limiting their interaction modeling for dense traffic.
- TraPHic predicts 3–5 seconds of future trajectory from 2–4 seconds of observed history and models horizon-based and heterogeneous-based weighted interactions.
- Heterogeneous-Based: The heterogeneous formulation incorporates agent sizes, aspect ratios, driver behaviors, dynamics, and constraints such as velocity, turning radius, neighbor distance, and local density.
- Results: 0.78 meters RMSE improvement, equivalent to 30% over prior methods, is reported on dense traffic datasets with 10–30 road agents.
2. Related Work
Prior trajectory-prediction research spans classical probabilistic and dynamical models, interaction-aware methods, recurrent networks, and hybrid deep architectures. Available datasets often specialize in highways, pedestrians, or non-trajectory annotations, motivating TRAF for dense heterogeneous Asian traffic.
- Classical trajectory-prediction approaches include Bayesian formulations, Monte Carlo simulation, Hidden Markov Models, and Kalman Filters.
- Interaction-aware methods are generally more accurate than methods that ignore road-agent interactions, including social-force, velocity-obstacle, and LTA approaches.
- Many prior interaction models focus on homogeneous pedestrian crowds, while other methods address traffic with few heterogeneous interactions.
- Deep trajectory-prediction methods commonly use RNN or LSTM variants for sequence modeling, with hybrid networks combining multiple architectures.
- Traffic Datasets: NGSIM is limited to highway cars and trucks, pedestrian datasets focus on crowds, and ApolloScape lacks trajectory information.
- Traffic Datasets: TRAF provides 2D/3D trajectories for dense, heterogeneous traffic captured in Asian cities.
3. TraPHic: Trajectory Prediction in Heterogeneous Traffic
TraPHic predicts future agent coordinates using weighted interactions tailored to dense, heterogeneous traffic. It combines horizon-based attention to nearby influential agents with heterogeneous state representations and local traffic concentration.
- Horizon Interactions: The horizon is a semi-elliptical field-of-view region whose key interactions receive priority because agents need not react equally to every neighbor.
- Heterogeneous Interactions: Heterogeneous interactions differ across buses, pedestrians, cars, and other agents because their shapes, sizes, and maneuverability create different dynamic constraints.
- Problem Setup and Notation: The model maps observed agent histories, sizes, and local traffic concentration to predicted spatial coordinates for the next τ frames.
- Problem Setup and Notation: Traffic concentration counts agents within a predefined local spatial interval, providing a mesoscopic representation alongside macroscopic traffic density.
- Interaction Visualization: Figure 2 distinguishes prioritized horizon interactions in blue from heterogeneous interactions involving pedestrians and motorcycles in green.
- Hybrid Architecture: The horizon and neighbor representations are pooled into separate maps and processed by ConvNets, with larger tensors representing more relevant interactions.
4. Hybrid Architecture for Traffic Prediction
TraPHic combines LSTMs and ConvNets to learn weighted horizon and neighborhood interactions for trajectory prediction in dense, heterogeneous traffic. Agent embeddings encode trajectories and dynamic constraints, while separate interaction maps are fused with the ego-agent representation before prediction.
- Hybrid Architecture: TraPHic uses a hybrid architecture in which LSTMs capture temporal dependencies and ConvNets process locally dependent relationships among heterogeneous road agents.The architecture addresses the limitation that independently parameterized LSTMs cannot learn relationships among different agents.
- Weighted Interactions: The horizon layer processes agents in a semi-elliptical region ahead of the ego agent, while the neighbor layer processes agents in its neighborhood.Each layer applies fully connected transformations and LSTMs to produce hidden vectors for its respective agents.
- Map Construction: Hidden vectors are pooled into horizon and neighbor maps, processed by separate ConvNets, and combined with the ego agent’s hidden vector.The horizon branch applies an additional fully connected layer before pooling, whereas the neighbor branch pools its hidden vectors directly.
- Agent Representation: Input embeddings encode trajectory information and constraints including agent shape, velocity, traffic concentration, turning radius, and spatial distance from neighbors.These state-space inputs implicitly represent maneuverability and driver behavior through agent dimensions, velocity or acceleration, and neighbor distance.
- Trajectory Prediction: The concatenated representation is passed through an LSTM to generate the ego agent’s trajectory prediction.Training jointly back-propagates through the linear blocks, ConvNets, LSTMs, and interaction maps, with higher weights assigned to prioritized interactions.
- Evaluation: Evaluation reports average RMSE error as ADE and final RMSE error as FDE after 5 seconds, with lower values indicating better performance.Table 1 evaluates the method on NGSIM and Beijing traffic datasets.
5. Experimental Evaluation
Experiments evaluate TraPHic on standard and dense, heterogeneous traffic datasets using ADE, FDE, RMSE curves, qualitative trajectories, and ablations. TraPHic performs best in dense TRAF traffic, while its advantage is smaller on sparse NGSIM data.
- Dataset: The TRAF dataset contains 50 dense, heterogeneous traffic videos with cars, buses, trucks, rickshaws, pedestrians, scooters, motorcycles, carts, and animals.Videos are categorized by viewpoint, camera motion, time of day, and difficulty level.
- Evaluation setup: Table 2 reports ADE and FDE after 5 seconds in meters, comparing original and learned settings with lower values preferred.The learned setting trains comparison methods on TRAF for a fair comparison.
- Results: TraPHic outperforms all compared prior methods on TRAF after those methods are trained on the dataset.The prior methods generalize poorly to dense, heterogeneous, front-view traffic, whereas TraPHic models heterogeneous and horizon-based interactions.
- Results: TraPHic is comparable to prior methods on sparse NGSIM data but does not outperform the state of the art there.The paper attributes its relatively high NGSIM RMSE to averaging with a sampling rate twice that used by prior methods.
- Ablation study: 71% lower ADE and 66% lower FDE result when both weighted-interaction formulations are incorporated compared with TraPHic-B.The ablation also reports larger gains from horizon-based interactions than from heterogeneous-based interactions.
6. Conclusion, Limitations, and Future Work
TraPHic predicts trajectories in dense, heterogeneous traffic using an end-to-end LSTM-CNN model with horizon-based and heterogeneous-based weighted interactions. It improves prediction accuracy by up to 30%, while remaining limited on sparse or homogeneous traffic and requiring agent shape and size information.
- TraPHic predicts road-agent trajectories in dense and heterogeneous traffic using an end-to-end approach without assuming lane-based driving.
- The LSTM-CNN hybrid network models horizon-based and heterogeneous-based weighted interactions among cars, buses, pedestrians, two-wheelers, three-wheelers, and animals.
- Up to 30% improvement in prediction accuracy is observed against state-of-the-art methods on standard and novel dense traffic datasets.
- The model does not outperform prior methods on sparse or homogeneous traffic videos, although its prediction results are comparable.
- Modeling heterogeneous constraints requires knowledge of different road agents’ shapes and sizes, which may be tedious to collect.