PaperlayerView exact revision on arXiv

Source-linked AI summary

Spatial-Temporal Graph ODE Networks for Traffic Flow Forecasting

Zheng Fang, Qingqing Long, Guojie Song, Kunqing Xie

arXiv:2106.12931v1cs.LG

TL;DR

Traffic forecasting must capture complex, long-range spatial-temporal dependencies that conventional shallow and separately processed models do not fully represent. STGODE uses tensor-based continuous graph dynamics with semantic adjacency, and experiments report superior performance over existing baselines across real-world traffic datasets. The paper also identifies stability constraints on parameter matrices for its continuous formulation.

  • Problem

    Traffic forecasting is challenging because long-range spatial-temporal dependencies are complex, while existing shallow GNNs and separate spatial-temporal modules have limited representation ability.

  • Method

    STGODE uses a tensor-based ODE to couple spatial and temporal features, incorporates semantic adjacency, and uses a restart distribution to alleviate over-smoothing.

  • Results

    STGODE outperforms existing baseline models across the evaluated real-world traffic datasets and metrics.

  • Takeaways & Limitations

    Continuous tensor-based graph dynamics and semantic neighbors support deeper modeling of wider-range spatial-temporal dependencies in traffic forecasting.

  • Takeaways & Limitations

    The continuous formulation constrains U and W to diagonalizable matrices with eigenvalues less than 1 to prevent divergent integrations.

Abstract

from arXiv · show

Spatial-temporal forecasting has attracted tremendous attention in a wide range of applications, and traffic flow prediction is a canonical and typical example. The complex and long-range spatial-temporal correlations of traffic flow bring it to a most intractable challenge. Existing works typically utilize shallow graph convolution networks (GNNs) and temporal extracting modules to model spatial and temporal dependencies respectively. However, the representation ability of such models is limited due to: (1) shallow GNNs are incapable to capture long-range spatial correlations, (2) only spatial connections are considered and a mass of semantic connections are ignored, which are of great importance for a comprehensive understanding of traffic networks. To this end, we propose Spatial-Temporal Graph Ordinary Differential Equation Networks (STGODE). Specifically, we capture spatial-temporal dynamics through a tensor-based ordinary differential equation (ODE), as a result, deeper networks can be constructed and spatial-temporal features are utilized synchronously. To understand the network more comprehensively, semantical adjacency matrix is considered in our model, and a well-design temporal dialated convolution structure is used to capture long term temporal dependencies. We evaluate our model on multiple real-world traffic datasets and superior performance is achieved over state-of-the-art baselines.

1 INTRODUCTION

Traffic flow forecasting is difficult because traffic networks contain long-range spatial dependencies, diverse temporal patterns, and complex interactions between them. STGODE addresses these limitations with continuous tensor-based modeling, semantic connections, and deeper spatial-temporal representations.

  • Traffic flow forecasting supports intelligent transportation systems by predicting future traffic conditions from historical traffic and road-network information.
  • Traffic networks combine nearby and distant spatial dependencies with diverse temporal patterns, making their interactions difficult to forecast.
  • Existing methods often model spatial and temporal patterns separately, while deeper GNNs can suffer from over-smoothing and inferior performance.
  • STGODE constructs spatial and semantic adjacency matrices to represent geographical connectivity and traffic-flow similarity.
  • STGODE introduces a continuous tensor-based GNN representation to capture longer-range spatial-temporal correlations beyond conventional depth limitations.
  • Experiments on real-world traffic datasets show that STGODE outperforms existing baseline models.

2 RELATED WORK

Prior traffic forecasting methods use graph-based representations, but deep GNNs remain limited by over-smoothing. Continuous graph ODEs provide a route to deeper message passing and longer-range dependencies.

  • GNNs compute node representations by sampling and aggregating features from neighboring nodes.
  • Traffic forecasting methods use graph neural networks to model spatial structure in non-Euclidean sensor data.
  • Deep GNNs can produce similar node representations through over-smoothing, restricting depth and leaving long-range dependencies insufficiently modeled.
  • Neural ODEs parameterize hidden-state derivatives continuously, replacing explicitly specified sequences of discrete hidden layers.
  • Continuous graph models use restart distributions to alleviate over-smoothing and enable deeper graph representations.

3 PRELIMINARIES

The paper represents traffic observations on a graph with spatial and semantic adjacency matrices, then uses normalized graph operators and continuous tensor dynamics to process spatial-temporal data.

  • A traffic network is represented as G = (V, E, A), with nodes, edges, and adjacency matrix A; the model uses spatial and semantic adjacency matrices.
  • Traffic observations are organized across nodes, features, and time into tensors describing network conditions.
  • Traffic forecasting learns a mapping from observed traffic-network tensor X to future traffic conditions.
  • 3.2 Regularized adjacency matrix: An adjacency matrix is normalized using the degree matrix, producing an operator whose eigenvalues lie in [−1, 1].
  • 3.2 Regularized adjacency matrix: A regularized adjacency form adds self-loop-based stabilization through a hyperparameter α∈(0, 1).
  • The continuous-time model parameterizes hidden dynamics with a neural network and differentiates through an ODE solver.
  • 3.4 Tensor Calculation: Tensor-matrix multiplication applies a matrix along a selected tensor mode, and multiplications on the same or different modes obey stated composition properties.

4 MODEL

STGODE combines spatial and semantic adjacency with a tensor-based ODE and dilated temporal convolutions to model spatial-temporal dependencies synchronously and over greater ranges.

  • 4.2 Tensor-based Spatial-Temporal Graph ODE: A STGODE block places a tensor-based ODE solver between two temporal dilation convolution blocks with residual connections.The solver receives spatial and semantic adjacency information separately, while the TCNs capture long-term temporal dependencies.
  • 4 MODEL: The model uses two cascaded STGODE layers, each composed of parallel blocks, followed by max pooling and an output layer.Cascaded layers extract higher-order features, while the block-level ODE and TCN components provide spatial-temporal processing.
  • 4.1 Adjacency Matrix Construction: STGODE uses spatial and semantic adjacency matrices to characterize geographical connectivity and traffic-pattern similarity.Semantic similarity is computed with DTW, which emphasizes time-series shape similarity rather than point-wise similarity.
  • 4.2 Tensor-based Spatial-Temporal Graph ODE: The continuous ODE formulation is derived as a continuous counterpart of a residual graph update, motivated by the limitations of deep discrete GNNs.Ordinary GNNs can suffer over-smoothing as depth increases, restricting long-range dependency modeling.
  • 4.2 Tensor-based Spatial-Temporal Graph ODE: The tensor formulation jointly processes spatial, temporal, and feature modes through adjacency, temporal-transform, and feature-transform matrices.The hidden representation is a spatial-temporal tensor, enabling spatial and temporal information to be handled simultaneously.
  • 4.2 Tensor-based Spatial-Temporal Graph ODE: Restarting from the initial representation preserves information from lower-order neighborhoods instead of allowing repeated propagation to discard most information.The expansion aggregates information from all layers while retaining the initial features.

5 EXPERIMENTS

STGODE is evaluated on real-world PeMS traffic datasets containing sensor measurements collected at regular intervals, against statistical, convolutional, recurrent, and graph-based baselines.

  • 5.1 Datasets: The experiments use six real-world traffic datasets collected by California’s PeMS system from highway detectors.The datasets include PeMSD7(M), PeMSD7(L), PeMS03, PeMS04, PeMS07, and PeMS08.
  • 5.1 Datasets: PeMS traffic data are collected every 30 seconds and aggregated into 5-minute intervals, producing 288 daily time steps.The raw measurements include traffic flow, average speed, and average occupancy.
  • 5.1 Datasets: The datasets vary substantially in sensor count and collection period, including PeMSD7(M) with 288 sensors and PeMSD7(L) with 1,026 sensors.Other described datasets include PeMS03 with 358 sensors and PeMS08 with 170 sensors.
  • 5.2 Baselines: STGODE is compared with ARIMA, STGCN, DCRNN, GraphWaveNet, and other baseline models.The listed baselines include statistical, graph-convolutional, recurrent, and graph-based forecasting approaches.

5.3 Experimental Settings

The experiments use a 6:2:2 data split and forecast the next 60 minutes from one hour of historical traffic, with standardized training and evaluation settings.

  • 5.3 Experimental Settings: All datasets are divided into training, validation, and test sets using a 6:2:2 ratio.
  • 5.3 Experimental Settings: One hour of historical traffic data is used to predict conditions during the following 60 minutes.
  • 5.3 Experimental Settings: Each STGODE layer contains three STGODE blocks, and the TCN hidden dimensions are set to 64, 32, and 64.
  • 5.3 Experimental Settings: The model uses Adam with learning rate 0.01, batch size 32, and 200 training epochs.
  • 5.3 Experimental Settings: Performance is evaluated with RMSE, MAE, and MAPE.

5.4 Experimental Results and Analysis

STGODE outperforms the evaluated baselines across all traffic-flow metrics and datasets. The authors attribute this to jointly modeling long-range spatial-temporal dependencies, semantic neighbors, and long-term temporal dependencies.

  • STGODE achieves the best performance on all metrics across all evaluated datasets.The authors attribute the result to the model’s spatial-temporal dependency modeling.
  • A tensor-based ODE framework extracts longer-range spatial-temporal dependencies.
  • Semantic neighbors establish global and comprehensive spatial relationships.
  • Temporal dilated convolution networks with residual connections capture long-term temporal dependencies.

5.5 Case Study

In a two-node case study, STGODE predictions remain closer to the ground truth than STGCN and respond rapidly to abrupt changes while smoothing small oscillations.

  • STGODE predictions are remarkably closer to the ground truth than STGCN for the two selected road-network nodes.
  • STGODE smooths small oscillations but responds rapidly when abrupt changes arise.
  • Long-range geographical and semantic neighbors help STGODE capture real-time dynamics and filter invalid information.

5.6 Model Analysis

Model analyses examine STGODE’s components and depth robustness. The results favor the ODE, semantic-neighbor, tensor-based, and initial-state designs, while STGODE remains stable as depth increases.

  • Ablation Experiments: STGCN* performs much better than previous STGCN in the ablation comparison.The improvement is attributed to the novel temporal convolution and semantic neighbors.
  • Ablation Experiments: STGODE with only spatial neighbors performs poorly, reinforcing the importance of semantic neighbors.
  • Ablation Experiments: The matrix-based version underperforms the tensor-based version because it cannot model spatial-temporal dependencies simultaneously.
  • Ablation Experiments: Removing the initial state produces inferior results, demonstrating the importance of connecting the initial state.
  • Network Depth: As network depth increases, STGCN* performance drops dramatically while STGODE performance remains stable.The comparison uses STGODE solver input time length and STGCN* convolution-layer count as depth measures.

6 CONCLUSION

The paper presents STGODE as a tensor-based spatial-temporal forecasting model that connects continuous differential equations with road-network node representations. Experiments support its effectiveness for deeper networks and wider-range dependencies, with semantic neighbors further enhancing performance.

  • STGODE bridges continuous differential equations to road-network node representations for traffic forecasting.
  • The approach enables deeper networks and wider-range dependencies without being affected by over-smoothing.
  • Semantic neighbors substantially enhance the model’s performance.
  • Extensive experiments demonstrate STGODE’s effectiveness against many existing methods.

7 APPENDIX

The appendix proof uses eigenvalue decompositions of three operators and rewrites a tensor-valued integral accordingly. It then evaluates the transformed integral element-wise to obtain the stated result.

  • The proof assumes eigenvalue decompositions for ˆ𝐴−𝐼, 𝑈−𝐼, and 𝑊−𝐼.
  • The tensor integral applies matrix exponentials of these three operators to H0 over the integration variable 𝜏.
  • The proof changes variables using the eigenvector matrices 𝑃1, 𝑃2, and 𝑃3 and the corresponding diagonal exponentials.
  • The transformed integral is considered element-wise before the integration result is stated.
Loading 2106.12931v1…