Source-linked AI summary
Graph Neural Controlled Differential Equations for Traffic Forecasting
Jeongwhan Choi, Hwangyong Choi, Jeehyun Hwang, Noseong Park
TL;DR
Traffic forecasting requires predicting future values over graph-structured, time-varying data, while existing work commonly combines graph convolutions with recurrent networks. STG-NCDE addresses this setting with separate NCDEs for temporal and spatial processing, achieving the best overall accuracy across six datasets against 20 baselines. It also supports irregular traffic forecasting with missing observations.
Problem
Spatio-temporal forecasting must predict future node values from historical graph data, but combining NCDE temporal processing with graph convolutional spatial processing had not been studied.
Method
STG-NCDE combines two NCDEs, one for temporal processing and one for spatial processing, within a single graph-based forecasting framework.
Results
STG-NCDE achieves the best accuracy across 6 benchmark datasets and outperforms 20 baselines on the reported evaluation metrics.
Takeaways & Limitations
The framework supports irregular traffic forecasting in which some input observations are missing.
Takeaways & Limitations
The forecasting formulation assumes that the graph topology, including its nodes and edges, remains fixed over time.
Abstract
from arXiv · showhide
Traffic forecasting is one of the most popular spatio-temporal tasks in the field of machine learning. A prevalent approach in the field is to combine graph convolutional networks and recurrent neural networks for the spatio-temporal processing. There has been fierce competition and many novel methods have been proposed. In this paper, we present the method of spatio-temporal graph neural controlled differential equation (STG-NCDE). Neural controlled differential equations (NCDEs) are a breakthrough concept for processing sequential data. We extend the concept and design two NCDEs: one for the temporal processing and the other for the spatial processing. After that, we combine them into a single framework. We conduct experiments with 6 benchmark datasets and 20 baselines. STG-NCDE shows the best accuracy in all cases, outperforming all those 20 baselines by non-trivial margins.
Introduction
Traffic forecasting predicts future node-level values on a fixed graph from historical, potentially irregular observations. STG-NCDE combines NCDE-based temporal processing with graph-based spatial processing and evaluates this design across broad benchmarks.
- Task formulation: Spatio-temporal forecasting predicts future node values across S horizons from historical graph features on a fixed topology.The graph has fixed nodes and edges, while node input features vary over time.
- Motivation: NCDEs provide a continuous analogue to recurrent neural networks and control the hidden trajectory through a continuous input path.The path X is constructed from observations by interpolation, allowing NCDEs to process irregular time-series data.
- Motivation: The paper addresses the previously unstudied combination of NCDE temporal processing and graph convolutional spatial processing.This integration is presented as a unified framework for spatio-temporal forecasting.
- Task formulation: Unlike the original NCDE setting with one time series, this problem jointly models |V| correlated time-series patterns associated with graph neighbors.The difference between single-series and graph-structured processing is illustrated in Figures 1 and 2.
- Proposed method: STG-NCDE creates one continuous path per node, jointly applies spatial and temporal graph processing, and predicts outputs from each node’s final hidden vector.The predicted tensor has shape R^|V|×S×M.
- Evaluation: Experiments use 6 PeMS benchmark datasets and 20 baselines, with STG-NCDE outperforming all compared methods on three standard metrics.The paper also evaluates irregular traffic forecasting where observations can be missing.
Related Work and Preliminaries
The paper situates NCDEs among continuous differential-equation models and prior traffic-forecasting architectures. It motivates STG-NCDE as a continuous, graph-aware alternative to established combinations of graph and temporal networks.
- Differential-equation models: Neural ordinary differential equations continuously model residual neural networks and can be solved with numerical ODE methods such as Euler, RK4, and DOPRI.With explicit Euler, the NODE formulation reduces to a residual connection.
- Differential-equation models: NCDEs generalize recurrent neural networks continuously, using a Riemann–Stieltjes integral rather than the Riemann integral used by NODEs.The paper notes that NODE-solving techniques can also be applied to NCDEs under the stated formulation.
- Traffic forecasting: Prior traffic-forecasting methods combine graph convolutions with recurrent networks, temporal convolutions, attention, adaptive topology, or localized spatial-temporal modules.Examples include DCRNN, STGCN, GraphWaveNet, ASTGCN, STG2Seq, STSGCN, LSGCN, and AGCRN.
Proposed Method
STG-NCDE models traffic as a spatio-temporal graph using two coupled NCDEs: one processes temporal information and the other processes spatial dependencies. Continuous interpolation, adaptive graph structure, and joint training produce the final forecasts.
- Pre-processing: The method first interpolates each node’s discrete feature time-series with a natural cubic spline to create a continuous, twice-differentiable path.Twice differentiability supports gradient calculation for the proposed model.
- Architecture: STG-NCDE combines separate NCDEs for temporal and spatial processing into one spatio-temporal forecasting framework.The temporal hidden trajectory controls the spatial NCDE, which incorporates neighboring-node trajectories.
- Temporal processing: The temporal NCDE processes each node’s hidden trajectory from the continuous input path, while its CDE function can be implemented with fully connected layers.This design yields a continuous RNN without requiring an RNN as the CDE function.
- Spatial processing: The spatial NCDE uses a trainable node-embedding matrix to form a normalized adaptive adjacency matrix that mixes node representations.The spatial graph operation corresponds to a first-order Chebyshev expansion with normalized adaptive adjacency.
- Training: Initial hidden states are generated with fully connected layers, and the model jointly trains these layers, the CDE functions, node embeddings, and output layer.The final node representations produce forecasts through the output layer and are optimized with L1 loss plus L2 weight decay.
- Training: The NCDE layers are well-posed when their CDE functions satisfy Lipschitz continuity, which the authors argue is fulfilled by their neural components.The paper links this well-posedness to stable training in practice.
Experiments
Experiments evaluate STG-NCDE on six real-world traffic datasets under the standard 12-to-12 forecasting setup and compare it with 20 baselines. STG-NCDE achieves the best reported accuracy across datasets, horizons, and standard metrics, while retaining performance under irregular observations.
- Datasets: The evaluation uses six PeMS traffic datasets, collected every 30 seconds, containing vehicle-count or velocity values.The datasets include PeMSD7(M), PeMSD7(L), PeMS03, PeMS04, PeMS07, and PeMS08.
- Experimental Settings: The benchmark uses 12 historical graph snapshots to predict the next 12 horizons, with a 6:2:2 train-validation-test split.Each consecutive dataset time-point is separated by five minutes, with S = 12 and M = 1.
- Overall Results: STG-NCDE has the best average accuracy across all six datasets and outperforms the 20 baseline methods on MAE, RMSE, and MAPE.The paper reports that all existing methods have worse errors in all three metrics; STGCN’s MAE is 17.0% worse than STG-NCDE’s.
- Ablation Study: Combining spatial and temporal processing yields better accuracy than either component alone.On PeMSD3, the spatial-only ablation has RMSE 15.92 versus 20.44 for temporal-only, while the combined model outperforms both.
- Additional Studies: The training loss stabilizes after the second epoch, and MAE and MAPE stabilize after node embedding size C = 7, with best accuracy at C = 10.These findings come from the PeMSD7 training-curve and sensitivity analyses.
- Horizon-wise Results: STG-NCDE produces smaller errors than other baselines at every forecasting horizon.The reported horizon-wise errors increase with the forecasting horizon while preserving STG-NCDE’s advantage.
- Irregular Forecasting: STG-NCDE remains applicable to irregular traffic forecasting without model-design changes when 10% to 50% of sensing values are randomly dropped.Its performance is not significantly degraded relative to the complete-data results, whereas the listed baselines cannot perform irregular forecasting.
Conclusions
The paper presents a spatio-temporal NCDE model with separate temporal and spatial NCDEs for traffic forecasting. Across experiments, it achieves the best overall accuracy and supports forecasting with missing input observations.
- Conclusions: STG-NCDE uses two NCDEs: one for temporal processing and one for spatial processing.The spatial NCDE provides an NCDE-based interpretation of graph convolutional networks.
- Conclusions: Across 6 datasets and 20 baselines, STG-NCDE achieves the best overall accuracy.
- Conclusions: STG-NCDE performs irregular traffic forecasting when some input observations are missing.The paper identifies this as a practical setting not actively considered by existing methods.
Best Hyperparameters
The appendix reports dataset-specific hyperparameter configurations for reproducibility. The listed settings vary across datasets, including the number of K, node embedding size C, hidden-vector dimensionality, learning rate, and weight decay.
- Best Hyperparameters: The paper provides best hyperparameter configurations for each dataset to support reproducibility.
- Best Hyperparameters: For PeMSD3, K is 1, C is 2, hidden-vector dimensionality is 64, learning rate is 1 × 10−3, and weight decay is 1 × 10−3.
Irregular Traffic Forecasting
Additional irregular traffic forecasting results are reported for datasets not included in the main paper. These results are provided in Tables 7 to 10.
- Irregular Traffic Forecasting: Tables 7 to 10 report irregular traffic forecasting results for the remaining datasets.These datasets are not reported in the main paper.
Sensitivity Analysis
The sensitivity analysis examines how node embedding size affects MAE and MAPE in the remaining datasets. Figure 6 presents these metric values while varying C.
- Sensitivity Analysis: Figure 6 shows MAE and MAPE as the node embedding size C varies in the remaining datasets.
Prediction Error at Each Horizon
Figure 7 reports prediction error at each forecasting horizon for the remaining datasets not reported in the main paper.
- Figure 7 reports prediction error at each horizon.
- The figure covers the remaining datasets omitted from the main paper’s reported results.
- These results extend the horizon-wise error analysis beyond the datasets presented in the main paper.
Traffic Forecasting Visualization
The paper visualizes forecasting outcomes against ground truth and Z-GCNETs, alongside sensitivity, horizon-error, and irregular-dataset forecasting-error analyses.
- Forecasting visualization: The method’s forecasting outcomes are visualized alongside ground truth and Z-GCNETs results in Fig. 8.
- Forecasting visualization: Fig. 8 includes visualizations for Nodes 12, 99, 108, and 141 in PeMSD3 over time.
- Irregular-dataset results: Tables 7–10 report forecasting error on irregular PeMSD3, PeMSD7, PeMSD7(M), and PeMSD7(L), respectively.
- Diagnostic analyses: Figures 6 and 7 cover sensitivity to C and prediction error at each horizon, respectively.