Source-linked AI summary
Forecasting Trajectory and Behavior of Road-Agents Using Spectral Clustering in Graph-LSTMs
Rohan Chandra, Tianrui Guan, Srujan Panuganti, Trisha Mittal, Uttaran Bhattacharya, Aniket Bera, Dinesh Manocha
TL;DR
Accurate trajectory prediction supports safe autonomous navigation, while behavior prediction remains less studied despite its relevance to anticipating nearby agents. The paper combines weighted dynamic geometric graphs with a two-stream graph-LSTM and spectral regularization, reducing average RMSE by approximately 75%, achieving 91.2% weighted average behavior accuracy, and improving long-term prediction by up to 70%.
Problem
Trajectory prediction is important for safe navigation, but behavior prediction has received significantly less research despite helping autonomous vehicles anticipate neighboring agents and avoid overly conservative behavior.
Method
The paper uses weighted dynamic geometric graphs and a two-stream graph-LSTM: one stream predicts trajectories, while the other predicts behavior and regularizes long-term forecasts through spectral information.
Results
The approach reduces average RMSE by approximately 75% versus the next best method, achieves 91.2% weighted average behavior accuracy, and reduces long-term FDE by up to 70%.
Takeaways & Limitations
Joint trajectory and behavior forecasting with spectral regularization improves long-term prediction across four urban driving datasets while forecasting overspeeding, underspeeding, or neutral behavior.
Takeaways & Limitations
The stated conclusion bounds the contribution to trajectory and behavior prediction with long-term error reduction; it identifies no further concrete limitation in the supplied passages.
Abstract
from arXiv · showhide
We present a novel approach for traffic forecasting in urban traffic scenarios using a combination of spectral graph analysis and deep learning. We predict both the low-level information (future trajectories) as well as the high-level information (road-agent behavior) from the extracted trajectory of each road-agent. Our formulation represents the proximity between the road agents using a weighted dynamic geometric graph (DGG). We use a two-stream graph-LSTM network to perform traffic forecasting using these weighted DGGs. The first stream predicts the spatial coordinates of road-agents, while the second stream predicts whether a road-agent is going to exhibit overspeeding, underspeeding, or neutral behavior by modeling spatial interactions between road-agents. Additionally, we propose a new regularization algorithm based on spectral clustering to reduce the error margin in long-term prediction (3-5 seconds) and improve the accuracy of the predicted trajectories. Moreover, we prove a theoretical upper bound on the regularized prediction error. We evaluate our approach on the Argoverse, Lyft, Apolloscape, and NGSIM datasets and highlight the benefits over prior trajectory prediction methods. In practice, our approach reduces the average prediction error by approximately 75% over prior algorithms and achieves a weighted average accuracy of 91.2% for behavior prediction. Additionally, our spectral regularization improves long-term prediction by up to 70%.
I. INTRODUCTION
The paper targets joint trajectory and behavior forecasting in urban traffic, emphasizing safe navigation, socially aware driving, and accurate 3–5-second prediction. It combines dynamic-graph modeling, two-stream forecasting, and spectral regularization to address these challenges.
- Motivation: Accurate trajectory prediction is crucial for safe navigation, while behavior recognition remains difficult for autonomous vehicles in dense, heterogeneous traffic.
- Motivation: Behavior prediction is less studied than trajectory prediction, although anticipating overtaking or sudden braking supports safer and more socially aware navigation.
- Challenge: 3–5-second forecasting becomes less accurate as temporal correlations weaken, increasing the long-term prediction error margin.
- Approach: The method uses weighted dynamic geometric graphs whose vertices represent road-agents and whose edge weights encode proximity.
- Approach: A two-stream graph-LSTM separately predicts future spatial coordinates and road-agent behavior, while spectral clustering regularizes long-term sequence prediction.
- Results: Approximately 75% lower average RMSE than the next-best method and 91.2% weighted-average behavior accuracy are reported across four urban driving datasets.
B. Road-Agent Behavior Prediction
The paper situates behavior prediction within socially aware autonomous driving and develops an online graph-based formulation that predicts trajectory and behavior separately. It frames these tasks as simultaneous but distinct forecasting problems.
- Motivation: Overly conservative autonomous-vehicle behavior can reduce efficiency and make travel uncomfortable, motivating socially aware behavior prediction.
- Prior work: Existing behavior-modeling approaches often rely on offline questionnaires or longitudinal data, limiting their suitability for online prediction.
- Proposed approach: The proposed behavior algorithm is presented as computationally online, independent of offline data and manually tunable parameters.
- Behavior modeling: Behavior studies classify aggressive, conservative, or moderate driving using driver, environmental, psychological, and vehicular factors.
- Proposed approach: The method extends graph-based eigenvector prediction by forecasting eigenvector sequences for future time-steps.
- Problem formulation: Traffic forecasting is formulated as separate trajectory-prediction and behavior-prediction problems solved simultaneously through two streams.
B. Weighted Dynamic Geometric Graphs (DGGs)
The method represents traffic as weighted dynamic geometric graphs whose vertices are road-agents and whose proximity-based edges encode interactions. Graph Laplacians evolve as agents observe new neighbors, supporting spectral analysis over time.
- Road-agent trajectories are provided as input for constructing dynamic geometric graphs at each time-step.
- Each graph uses road-agents as vertices and connects pairs whose Euclidean distance is below the threshold μ, set to 10 meters in experiments.
- Edge weights decrease with distance, assigning greater interaction weight to nearby road-agents than to distant ones.
- The graph Laplacian is formed from the adjacency and degree matrices, with L = D − A.
- The next-time-step Laplacian is updated from the current one using a sparse rank-2 perturbation that represents newly observed neighbors.
IV. TRAJECTORY AND BEHAVIOR FORECASTING
The forecasting architecture uses two parallel LSTM streams: one predicts future coordinates from trajectory history, while the other predicts future graph spectra for interaction-aware behavior forecasting.
- The approach consists of two parallel LSTM networks operating separately on trajectory and graph-spectrum information.
- Stream 1 encodes trajectory history and predicts future spatial coordinates for each road-agent.
- Stream 2 converts each input DGG into a Laplacian spectrum and organizes corresponding eigenvectors into temporal sequences.
- The second stream predicts future eigenvectors, reconstructs future Laplacian matrices, and uses them to assign road-agent behavior labels.
B. Trajectory Prediction
Trajectory and behavior forecasting are separated across two streams: the first predicts coordinates without neighbor interactions, while the second uses predicted graph spectra for behavior classification.
- Stream 1 predicts future spatial coordinates with an LSTM encoder-decoder that does not account for neighboring road-agents.
- The behavior stream uses predicted DGG eigenvectors because many data-driven behavior methods require large, well-annotated behavior datasets.
- The degree θ_i measures an agent’s accumulated distinct neighbors, and its temporal increase rate θ′_i is used to characterize behavior.
- Future spectra are converted into Laplacian matrices, whose diagonal entries provide θ_i and whose changes over time provide θ′_i.
- Heuristic thresholds λ1 and λ2 assign overspeeding, underspeeding, or neutral behavior labels.
- Behavior is predicted over several frames because human behavior does not change instantaneously at every time-step.
D. Spectral Clustering Regularization
Spectral cluster regularization addresses the growing long-horizon trajectory error by encouraging predictions toward preferred trajectories obtained from graph structure.
- Standard trajectory loss can produce gradually diverging predictions whose error increases with the prediction horizon.
- The method treats a road-agent’s ground-truth trajectory as its preferred trajectory in the absence of other dynamic road-agents.
- Preferred trajectories are obtained by minimizing DGG Dirichlet energy through spectral clustering on road-agents.
- Regularization clusters the second stream’s predicted eigenvectors, computes cluster centers and deviations, and incorporates them into stream 1’s loss.
- The regularized loss is used to backpropagate weights corresponding to the predicted mean μ_t in stream 1.
- Equation 6 can also support multimodal trajectory prediction by computing maneuver probabilities from μ and σ.
V. UPPER BOUND FOR PREDICTION ERROR
The paper bounds successive eigenvector changes in dynamic traffic graphs, linking spectral regularization to more predictable sequences and controlled long-term error. The bound scales with traffic density and local graph changes.
- Motivation: LSTM prediction is difficult when eigenvectors lack temporal correlation, motivating spectral regularization of the graph sequence.The regularization aims to make successive eigenvectors more correlated and therefore easier to predict.
- Graph perturbation: The perturbation between consecutive Laplacian matrices is represented by the sparse outer product δδ⊤, reflecting newly observed neighboring agents.The sparse matrix δ records road-agents that acquire new neighbors in the dynamic geometric graph.
- Theoretical bound: The analysis bounds the angular difference between successive jth Laplacian eigenvectors by O(Nδmax).Here, N is the number of road-agents and δmax bounds the maximum component of the graph-change matrix.
- Numerical implication: For Lyft with N = 270 and δmax = 0.049, the theoretical upper bound on the eigenvector error is 0.8 meters.A smaller φj indicates greater similarity between successive eigenvectors and stronger sequence correlation.
- Alternative computation: Computing future spectra from trajectories predicted by the first stream is sub-optimal because its bound becomes O(NLmax), with Lmax much larger than δmax.The proposed approach instead uses the spectral structure directly to avoid this looser bound.
VI. EXPERIMENTS AND RESULTS
The experiments evaluate trajectory and behavior forecasting across sparse and dense urban-driving datasets using standard displacement and weighted behavior-accuracy metrics. Comparisons include prior trajectory methods and architectures relevant to the proposed two-stream design.
- Datasets: The evaluation uses sparse NGSIM and dense Lyft Level 5, Argoverse Motion Forecasting, and Apolloscape Trajectory datasets.This covers both sparse and dense traffic conditions.
- Metrics: Trajectory forecasting is evaluated with Average Displacement Error and Final Displacement Error using metrics standard in prior work.These metrics assess predicted positions across the window and at its endpoint.
- Metrics: Behavior forecasting reports weighted classification accuracy across overspeeding, neutral, and underspeeding labels.The three labels represent the behavior classes used in evaluation.
- Baselines: The comparison set is selected using both ADE/FDE performance and the deep-learning architecture used by each competing method.The authors therefore compare both trajectory scores and architectural advantages of the two-stream network.
- Baselines: Compared methods include CS-LSTM, TraPHic, and Social-GAN, spanning CNN-LSTM, spatial-attention, and GAN-based trajectory predictors.These baselines cover different modeling strategies and traffic settings.
C. Analysis and Discussion
The experiments report strong trajectory gains from the spectrally regularized two-stream model, especially for long-term prediction, while showing density and parameter choices affect the error bound. The theoretical bound broadly agrees with empirical FDE on three datasets but cannot be computed for NGSIM.
- Main results: Approximately 75% average RMSE reduction is achieved relative to the next-best method, GRIP.The comparison concerns the average RMSE across the evaluated trajectory-prediction results.
- Ablation: Up to 70% FDE reduction results from adding spectral regularization to Stream 1.The ablation compares the unregularized S1 Only model with the two-stream S1 + S2 model.
- Ablation: The regularized model has lower RMSE than Stream 1 across the entire five-second NGSIM prediction window.Figure 3 compares the spectrally regularized both-streams curve with the Stream 1 curve.
- Density effects: Traffic density affects RMSE because the theoretical increase bound scales as O(Nδmax), with NGSIM reporting RMSE values of 0.40/1.08.NGSIM is the sparsest dataset, whereas the other datasets contain denser urban traffic.
- Upper-bound analysis: The theoretical FDE is computed as T-FDE = φn × (T − τ), summing the bounded per-step RMSE increase across the prediction window.The analysis compares this theoretical quantity with empirical FDE results.
- Limitations: The theoretical bound cannot be computed for NGSIM because the total number of lanes is unknown.Lane count is needed to determine the relevant graph-change parameter for the calculation.
- Upper-bound analysis: The theoretical and empirical FDE agree by 73%, 82%, and 100% on Apolloscape, Lyft, and Argoverse, respectively.Disagreement is attributed partly to using δmax = 0.049 for predominantly three-lane roads when some videos have one or two lanes.
E. Behavior Prediction Results
The model predicts overspeeding, neutral, and underspeeding behavior across road-agents, achieving high weighted accuracy on Lyft, Argoverse, and Apolloscape. Prediction similarity varies with agents’ positions in the traffic video and available neighboring agents.
- Behavior prediction accuracy: 92.96% weighted accuracy is achieved on Lyft, 84.11% on Argoverse, and 96.72% on Apolloscape.The behavior labels are generated using the algorithm described in Section IV-C.
- Behavior prediction accuracy: The behavior classes are overspeeding, underspeeding, and neutral, with predictions compared against ground-truth labels for each road-agent.The vertical axis shows θ′, while the horizontal axis identifies road-agents.
- Behavior variation across agents: Similar ground-truth and predicted label patterns indicate more correct classifications across the evaluated datasets.The Lyft plots provide an example in which aggressive and conservative regions are nearly identical; NGSIM is excluded because it lacks diverse behaviors.
- Behavior variation across agents: Behavior-label variation decreases for road-agents appearing later in the traffic video because θ′ depends on the number of distinct neighbors observed.Agents near the beginning of the x-axis appear earlier in the video, while those near the end appear later.
F. Ablation Study of the Radius Parameter (µ)
The radius parameter μ controls which neighboring road-agents enter the spectral clustering procedure. Performance is best for 0 < μ ≤10 meters; ignoring neighbors or including distant agents worsens RMSE.
- Radius ablation: 0 < μ ≤10 meters produces the best performance across the evaluated radius ranges.The ablation measures averaged RMSE over μ = 0, 0 < μ ≤10, and 10 < μ ≤20.
- Radius ablation: Setting μ = 0 severely degrades performance in dense traffic by ignoring neighborhood information.The radius parameter determines which road-agents contribute to the graph-based representation.
- Radius ablation: Increasing μ beyond 10 meters increases RMSE by including road-agents that are too far to interact with the ego-agent.These distant agents expand and shift cluster centers, acting as outliers in spectral clustering.
- Method context: The method jointly predicts trajectories and behavior using two streams, while spectral regularization reduces long-term prediction errors.The first stream predicts trajectories and the second predicts road-agent behavior.
- Limitations: The method may not generalize to new traffic scenarios because its behavior model currently uses only one feature.Training also requires several hours because of traffic-graph and Laplacian-matrix computations.