PaperlayerView exact revision on arXiv

Source-linked AI summary

Dynamic Trend Fusion Module for Traffic Flow Prediction

Jing Chen, Haocheng Ye, Zhian Ying, Yuntao Sun, Wenqiang Xu

arXiv:2501.10796v1cs.LG

TL;DR

Traffic prediction requires modeling complex spatio-temporal correlations, yet existing methods incompletely interact across spatial and temporal dimensions and often depend on predefined graphs. DTRformer combines DST2former dynamic-trend generation with multi-view graph fusion, and experiments on four real-world datasets show significantly better performance than existing methods.

  • Problem

    Existing traffic-prediction methods do not fully capture complex spatial-temporal interactions, including lagged patterns and similar behavior among distant nodes, while relying on predefined graphs.

  • Method

    DTRformer uses temporal and spatial encoders with cross spatial-temporal attention to generate dynamic trends, then fuses them with compressed static graph information.

  • Results

    Experiments on four real-world traffic datasets show that DTRformer significantly outperforms existing methods, with DST2former alone exceeding traditional transformer accuracy and MGFM further improving performance.

  • Takeaways & Limitations

    The framework provides a robust approach for capturing multi-view dynamic information in traffic networks while integrating dynamic and static information.

  • Takeaways & Limitations

    Existing dynamic and spatio-temporal modeling approaches remain constrained by predefined graphs and incomplete spatial-temporal interaction, defining the scope addressed by this work.

Abstract

from arXiv · show

Accurate traffic flow prediction is essential for applications like transport logistics but remains challenging due to complex spatio-temporal correlations and non-linear traffic patterns. Existing methods often model spatial and temporal dependencies separately, failing to effectively fuse them. To overcome this limitation, the Dynamic Spatial-Temporal Trend Transformer DST2former is proposed to capture spatio-temporal correlations through adaptive embedding and to fuse dynamic and static information for learning multi-view dynamic features of traffic networks. The approach employs the Dynamic Trend Representation Transformer (DTRformer) to generate dynamic trends using encoders for both temporal and spatial dimensions, fused via Cross Spatial-Temporal Attention. Predefined graphs are compressed into a representation graph to extract static attributes and reduce redundancy. Experiments on four real-world traffic datasets demonstrate that our framework achieves state-of-the-art performance.

1. Introduction

Traffic prediction must capture nonlinear, dynamic spatio-temporal patterns, but existing approaches often separate spatial and temporal modeling or incur costly dynamic-graph construction. The proposed framework fuses dynamic trends with static graph information to improve traffic prediction.

  • Traffic prediction supports road-network optimization, congestion management, real-time signal adjustment, route planning, and congestion pricing.
  • Static graphs miss dynamic flow distributions, whereas dynamic graphs improve adaptation but require costly adjacency generation at every time step.
  • Existing methods struggle to model lagged traffic patterns, similar behaviors among distant nodes, and interactions between spatial and temporal information.
  • The multi-view graph integration method fuses static distance and dynamic trend features, reduces static-graph redundancy, and learns time-specific connectivity.
  • The Transformer-based trend module generates fine-grained multi-step spatio-temporal trends and uses cross-attention to fuse global temporal and spatial information.
  • Experiments on four public datasets against 18 baselines significantly reduce prediction error and achieve state-of-the-art accuracy.

2. Related Works

Traffic forecasting research has progressed from statistical and shallow-learning methods to spatio-temporal graph and attention-based models. However, existing approaches remain limited by predefined spatial graphs and incomplete modeling of fine-grained dynamic spatio-temporal context.

  • Traffic forecasting evolved from statistical models and shallow machine learning toward deep models integrating spatial and temporal information.
  • RNN- and CNN-based spatio-temporal graph networks decompose traffic dependencies into temporal and spatial domains but remain limited in their joint modeling.
  • Attention-based models offer broad receptive fields, but examples such as GMAN separately process spatial and temporal inputs and may overlook node-specific traffic patterns.
  • Existing approaches rely heavily on predefined graphs and insufficiently model dynamic spatio-temporal context, intertemporal effects, and fine-grained changes.

3. Preliminaries

The traffic network is modeled as a weighted graph whose nodes are sensors and whose adjacency matrix encodes spatial distances. Historical traffic observations are used to predict traffic at future time steps.

  • The traffic network is represented by a weighted graph G = (V, E, A), where V contains sensors and A describes spatial distances between nodes.
  • Given observed traffic flow over T time steps on N sensors, the task is to predict the next T time steps.

4.1. Framework structure

DTRformer takes the traffic graph and historical signals as inputs, embeds the signals, generates dynamic trends with parallel temporal and spatial encoders, and fuses them with predefined-graph connectivity.

  • DTRformer accepts the traffic-network adjacency matrix and historical traffic signals as its two inputs.
  • The historical traffic signal is transformed into hidden space through an embedding layer before entering DST2former.
  • DST2former uses parallel stacked Temporal Encoder and Spatial Encoder modules to process hidden traffic representations.
  • Cross Spatial-Temporal Attention generates T-step dynamic trends, which are fused with predefined-graph connectivity to form a representation graph.

4.2. Dynamic Spatial-Temporal Trend Transformer

The Dynamic Spatial-Temporal Trend Transformer learns adaptive spatial and temporal representations, then uses Cross Spatial-Temporal Attention to generate dynamic trends that capture their joint evolution.

  • Adaptive Embedding: The model embeds historical traffic, temporal identities, and adaptive node representations to learn node-specific spatio-temporal patterns.Historical traffic is mapped separately across temporal and spatial dimensions, while day-of-week, timestamp, and adaptive node embeddings are concatenated.
  • Spatial and Temporal Encoding: Parallel spatial and temporal encoders use self-attention to capture relations among nodes and across time steps.The spatial encoder models node connections at different times, while the temporal encoder models temporal relations across spatial nodes.
  • Cross Spatial-Temporal Attention: Cross Spatial-Temporal Attention fuses spatial queries with temporal keys and values to capture the joint evolution of spatial and temporal features.The mechanism produces time-indexed feature-weight graphs that reflect correlations between temporal steps and spatial information.
  • Dynamic Trend Representation: The resulting spatio-temporal representation contains dynamic trends together with historical traffic and temporal identity information.Temporal attention output is retained through a residual connection, and the final representation is denoted H^(st).

4.3. Multi-view Graph Fusion Module

The Multi-view Graph Fusion Module combines dynamic trend information with compressed forward and backward static graph representations to model time-varying connectivity.

  • Motivation: The module combines spatio-temporal trends with static connectivity because node relationships can change in importance across time.Its overall design is presented as the Multi-view Graph Fusion Module.
  • Static Graph Encoding: Forward and backward adjacency matrices are normalized and projected into lower-dimensional representations to reduce sparsity and redundancy.The two directions represent distinct transfer relationships in the road network.
  • Multi-view Fusion: The projected graph views are broadcast and concatenated with dynamic trends before residual MLP encoding fuses spatial information from multiple viewpoints.The fused hidden representation has dimensionality N×(d_a+2×d_n), and a linear layer produces a preliminary connectivity representation.
  • Augmented Residual Attention: Enhanced residual attention aligns static graph features with dynamic trends and adjusts feature weights associated with temporal identities.Reinforced residuals increase information density to mitigate sparsity effects from the static graph structure.

4.4. Output and Loss

The model applies a linear transformation to the fused representation to predict the next P time steps and optimizes those predictions with MAE loss.

  • Prediction: A linear transformation maps the Multi-view Graph Fusion Module output to predictions for the next P time steps.The prediction tensor is denoted Y-hat ∈ R^(T×N×C_out).
  • Loss: Training minimizes Mean Absolute Error between the predictions and ground-truth traffic values.The output dimension is C_out, with P predicted steps and N nodes.

5. Experiments

The experiments compare DTRformer with established traffic-prediction baselines on four public datasets using standardized data splits, training settings, and evaluation procedures.

  • Datasets: Experiments compare DTRformer with baselines on four publicly available traffic datasets from California’s Caltrans performance measurement system.The datasets contain tens of thousands of timesteps and hundreds of sensors, with distance-based road-network adjacency matrices.
  • Experimental Settings: Approximately 60% of each dataset is used for training, 20% for validation, and 20% for testing after Z-Score normalization.The weighted neighbor matrix is constructed following DCRNN.
  • Baselines: The benchmark includes traditional forecasting methods, recurrent and convolutional graph models, dynamic-graph methods, and attention-based spatio-temporal models.Baselines include HI, DLinear, VAR, SVR, ARIMA, LSTM, Graph WaveNet, DCRNN, AGCRN, MTGNN, DGCRN, D2STGNN, STNorm, GMAN, PDFormer, STAEformer, and STID.
  • Implementation: The proposed implementation uses PyTorch 1.9.1, an NVIDIA 4090 GPU, Adam optimization, and an initial learning rate of 0.001.The embedding dimension is 24, the predefined-graph dimension is 100, and the number of attention heads is 4.
  • Evaluation: The evaluation reports MAE, RMSE, and MAPE, with test performance measured after model selection on the validation set.The experiments use T=12 observed samples in the metric definitions.

5.4. Experimental Results

Across four datasets, DTRformer generally achieves the strongest predictive performance, while ablations and efficiency comparisons examine its components, robustness, and computational trade-offs.

  • Overall Performance: DTRformer achieves the best results on most metrics across PEMS03, PEMS04, PEMS07, and PEMS08.The reported comparison describes the model as superior to the evaluated baselines.
  • NSE Analysis: NSE exceeds 0.96 on all four datasets, outperforming the other comparable models.The reported result is presented as evidence of strong predictive performance across datasets.
  • Taylor Diagram Analysis: DTRformer has the highest correlation coefficient and a standard deviation closest to observations across all four Taylor diagrams.The Taylor diagrams use angular position for correlation and radial distance for simulated standard deviation.
  • Multi-Horizon Prediction: At 15-, 30-, and 60-minute horizons, DTRformer performs best at almost all evaluated time points against four selected baselines.The comparison uses DCRNN, Graph WaveNet, D2STGNN, and STAEformer.
  • Ablation Study: The ablation study evaluates six model variants to assess adaptive embedding, the trend transformer, predefined graphs, graph fusion, and augmented residual attention.The study reports that dynamic embedding is indispensable, while augmented residual information helps fusion of multiple graph types converge better.
  • Prediction Visualization: The visualization shows accurate trend tracking for selected sensors, although random noise causes some local prediction details to remain inaccurate.For sensor 111, the model captures trend changes while avoiding overfitting the large noise signal.
  • Node Correlation: The node-correlation heatmap indicates that most nodes contribute little, while a small number of key nodes substantially influence predictions.Brighter colors represent stronger influence and stronger relationships to final prediction results.
  • Efficiency: DTRformer is faster than dynamic-graph models in inference and more efficient than STAEformer while incorporating graph embedding.Among attention-based models, only PDFormer has faster inference in the reported comparison.

6. Conclusion

DTRformer combines dynamic spatio-temporal trend extraction with predefined graph information to capture multi-view traffic-network dynamics. Experiments on four real-world datasets show that it outperforms existing methods, while MGFM further improves performance by reducing redundancy and integrating dynamic and static information.

  • 6. Conclusion: DTRformer combines DST2former for intricate spatio-temporal correlations with MGFM for fusing dynamic trends and predefined graph information.Together, the modules target multi-view dynamic information in traffic networks.
  • 6. Conclusion: Across four real-world traffic datasets, DTRformer significantly outperforms existing methods in traffic prediction.The experiments are described as demonstrating efficacy and robustness.
  • 6. Conclusion: MGFM further amplifies performance by reducing redundancy and integrating dynamic and static information.This complements the dynamic trend extraction performed by DST2former.
  • 6. Conclusion: The proposed model advances the state of the art while offering a robust solution for capturing multi-view dynamic information in traffic networks.This conclusion is based on the reported experimental findings.
Loading 2501.10796v1…