Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs

Ming Jin; Yu Zheng; Yuan-Fang Li; Siheng Chen; Bin Yang; Shirui Pan

Source-linked AI summary

Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs

Ming Jin, Yu Zheng, Yuan-Fang Li, Siheng Chen, Bin Yang, Shirui Pan

arXiv:2202.08408v2Feb 17, 2022cs.LG

TL;DR

Multivariate time series forecasting is hindered by discontinuous architectures, high complexity, and dependence on predefined graph structures. MTGODE models dynamic graphs with coupled continuous spatial and temporal neural ODEs, and experiments report superior performance across five benchmark datasets. Its scope includes assumptions about sparse, unidirectional learned graphs and documented limitations of discrete temporal formulations.

Problem
Existing forecasting methods struggle with discontinuous latent dynamics, high computational and memory overhead, and unknown graph structures in multivariate time series.
Method
MTGODE represents multivariate series as dynamic graphs and unifies graph propagation and temporal aggregation through coupled continuous ODE mechanisms with learned graph structures.
Results
MTGODE achieves the best performance on three conventional time series datasets and performs best across all datasets in the ablation study.
Takeaways & Limitations
The method learns continuous spatial-temporal dynamics without relying on predefined graph structures, while supporting deeper propagation and fine-grained temporal information aggregation.
Takeaways & Limitations
The learned graph is sparse and assumed to be unidirectional because changes in one time series are considered likely to lead unidirectionally to fluctuations in others.

Abstract

from arXiv · show

Multivariate time series forecasting has long received significant attention in real-world applications, such as energy consumption and traffic prediction. While recent methods demonstrate good forecasting abilities, they have three fundamental limitations. (i) Discrete neural architectures: Interlacing individually parameterized spatial and temporal blocks to encode rich underlying patterns leads to discontinuous latent state trajectories and higher forecasting numerical errors. (ii) High complexity: Discrete approaches complicate models with dedicated designs and redundant parameters, leading to higher computational and memory overheads. (iii) Reliance on graph priors: Relying on predefined static graph structures limits their effectiveness and practicability in real-world applications. In this paper, we address all the above limitations by proposing a continuous model to forecast $\textbf{M}$ultivariate $\textbf{T}$ime series with dynamic $\textbf{G}$raph neural $\textbf{O}$rdinary $\textbf{D}$ifferential $\textbf{E}$quations ($\texttt{MTGODE}$). Specifically, we first abstract multivariate time series into dynamic graphs with time-evolving node features and unknown graph structures. Then, we design and solve a neural ODE to complement missing graph topologies and unify both spatial and temporal message passing, allowing deeper graph propagation and fine-grained temporal information aggregation to characterize stable and precise latent spatial-temporal dynamics. Our experiments demonstrate the superiorities of $\texttt{MTGODE}$ from various perspectives on five time series benchmark datasets.

1 INTRODUCTION

MTGODE addresses discontinuous architectures, high complexity, and reliance on graph priors by modeling fully continuous latent spatial-temporal dynamics with dynamic graph learning and coupled ODE mechanisms.

Multivariate time series forecasting depends on modeling spatial-temporal correlations among mutually influenced variables, but effectively and efficiently capturing these dependencies remains challenging.
Existing methods use discrete or partially discrete architectures that create discontinuous latent trajectories, shallow graph propagation, and limited access to farther-neighbor correlations.
Stacking separately parameterized spatial and temporal modules introduces dedicated designs and redundant parameters, increasing computational and memory overheads.
Many graph-based forecasting models require predefined graph structures, although such interconnections are typically unknown in real-world applications.
MTGODE represents series as dynamic graphs with time-evolving node features and learns missing interdependencies through continuous graph propagation and graph structure learning.
Coupled spatial and temporal ODEs unify message passing into fully continuous latent dynamics, while experiments on five benchmark datasets report effectiveness and efficiency.

2 RELATED WORK

Related work spans statistical and deep learning forecasting, graph neural networks, and neural ODEs; MTGODE combines dynamic graph learning with continuous spatial and temporal mechanisms to address their remaining gaps.

Multivariate Time Series Forecasting: Statistical forecasting includes autoregressive and vector autoregressive models, while deep learning methods model nonlinear temporal and spatial patterns.
Graph Neural Networks: GNN-based forecasting models represent variables as graph nodes and use graph structures to model spatial and temporal interdependencies.
Graph Neural Networks: Prior approaches combine graph propagation with recurrent, convolutional, or attention mechanisms, but many depend on predefined graph structures or discrete modules.
Neural Ordinary Differential Equations: MTGODE couples continuous temporal aggregation with simplified continuous graph propagation and dynamic graph structure learning to model arbitrary multivariate time series.
Neural Ordinary Differential Equations: Neural ODEs approximate continuous latent-state evolution, but STGODE applies continuity only to graph propagation over predefined static graphs, leaving temporal aggregation discrete.

3 PROBLEM DEFINITION

The paper formulates multivariate time series as data from multiple variables, features, and time steps, then learns spatial-temporal representations for single-step or multi-step forecasting.

A multivariate time series X contains N variables, D feature dimensions, and S total training time steps.
Each variable contributes a feature-by-time series, while each time step contains observations across all variables and features.
Given T historical observations, the spatial-temporal encoder maps them to an N × D′ representation for downstream tasks.
The framework supports single-step forecasting of one future observation and multi-step forecasting across horizon H.

4 METHODOLOGY

MTGODE encodes multivariate time series as dynamic graphs and models spatial-temporal dynamics through coupled continuous graph propagation and temporal aggregation ODEs. The design learns graph structures without priors, reduces numerical and architectural limitations of discrete propagation, and supports forecasting through learned representations.

Overall Framework: MTGODE maps multivariate time series into latent space, dynamically learns graph structure, couples continuous temporal aggregation with continuous graph propagation, and decodes learned representations for forecasting.The framework comprises data preprocessing, continuous spatial-temporal encoding, and downstream task decoding.
Continuous Graph Propagation: Graph propagation starts from latent states and generalizes discrete K-hop propagation into a continuous process governed by an ODE.The continuous formulation replaces discrete propagation steps with a time variable and ODE solver.
Continuous Graph Propagation: MTGODE disentangles graph propagation depth from integration time, enabling fine-grained and long-range spatial dependencies while avoiding the discrete formulation’s over-smoothing and numerical-error problems.The discrete formulation enforces Δt_cgp = 1, tying propagation depth to integration time; MTGODE separates these quantities.
Dynamic Graph Structure Learning: Dynamic graph structures are directly optimized with the full model, producing sparse, unidirectional connections intended to describe stable interdependencies between variables.The graph is learned without predefined adjacency priors; sparsity is used to reduce computational cost.
Continuous Temporal Aggregation: The model replaces discrete temporal convolutions with a temporal neural ODE that uses one parameter set to characterize fine-grained temporal dynamics and reduce parameter and memory overheads.The continuous formulation lets Δt_cta approach zero and addresses the individually parameterized layers and large step size of the discrete formulation.
Continuous Graph Propagation: For fixed integration time T_cgp, learned spatial representations converge as K = T_cgp/Δt_cgp approaches infinity, while spatial numerical errors approach zero.These are stated theoretical properties of MTGODE’s continuous graph propagation.
Efficiency: Compared with discrete counterparts, MTGODE eliminates redundant residual and skip connections, reducing model complexity with time complexity O(LNRD′2).Here, L denotes the number of layers in discrete models.

5 EXPERIMENTAL STUDY

MTGODE is evaluated on five benchmark datasets against representative forecasting baselines, with experiments examining forecasting accuracy, continuous mechanisms, sensitivity, and efficiency. It generally outperforms competing methods while reducing the computational and memory costs associated with discrete architectures.

Datasets and setup: Experiments cover five datasets: Electricity, Solar-Energy, Traffic, Metr-La, and Pems-Bay, including settings with and without predefined graph structures.The evaluation uses both single-step and multi-step forecasting protocols.
Overall comparisons: MTGODE achieves the best performance on three time series datasets, including comparisons with HyDCNN and STG-NCDE.This result supports its effectiveness in multivariate time series forecasting.
Overall comparisons: MTGODE significantly surpasses MTGNN in most cases, especially at long-term forecasting horizons 6 and 12.The authors associate this advantage with capturing long-range and fine-grained spatial-temporal dependencies.
Overall comparisons: In multi-step traffic forecasting, MTGODE consistently outperforms MTGNN and STG-NCDE under similar graph-learning schemas.It also performs better than STGODE, which the authors attribute to continuous temporal dynamics and a more expressive graph module free from graph priors.
Overall comparisons: MTGODE significantly surpasses DCRNN, STGCN, and Graph WaveNet without graph priors, while matching or outperforming MRA-BCGN and GMAN.This comparison demonstrates competitiveness across predefined-graph and graph-free baselines.
Ablation and mechanism studies: Increasing temporal aggregation depth reduces MTGODE’s numerical errors without the parameter growth observed in its discrete variant.The continuous model therefore supports deeper aggregation for more stable and accurate temporal dynamics.
Efficiency: MTGODE has lower computational and memory overheads than its discrete variant and MTGNN, especially at larger model depths.The efficiency comparison uses multiply-accumulate operations and GPU memory.

6 CONCLUSION

The paper addresses multivariate time series forecasting by combining neural ODEs with dynamic graph structure learning. MTGODE models continuous latent spatial-temporal dynamics efficiently without relying on graph priors.

Conclusion: MTGODE uses neural ordinary differential equations and dynamic graph structure learning to model continuous latent spatial-temporal dynamics.The approach couples continuous graph propagation with temporal aggregation.
Conclusion: The method learns expressive representations efficiently without relying on predefined graph structures.The conclusion reports better potential for forecasting arbitrary multivariate time series.

APPENDIX A PROOF OF PROPERTY 1

The proof models the spatial propagation ODE as graph heat diffusion governed by the normalized graph Laplacian. Its heat-kernel eigendecomposition explains why unbounded discrete propagation causes over-smoothing, while decoupling propagation depth from terminal time preserves convergence.

Property 1: The spatial ODE is a graph heat diffusion process governed by the normalized graph Laplacian L.The heat kernel e^(-tL) provides its closed-form solution.
Property 1: The normalized Laplacian is symmetric positive semidefinite when the adjacency matrix is symmetrically normalized.This permits an eigendecomposition with nonnegative eigenvalues.
Property 1: The heat kernel decomposes into eigenvalue factors e^(-tλ_i), whose nonzero-eigenvalue components vanish as t approaches infinity.This spectral behavior drives the limiting representation.
Property 1: For A ≠ I_N, increasing discrete propagation depth without bound inevitably produces over-smoothing as the eigenvalues are zeroed.The proof identifies this as the limiting behavior when K = T approaches infinity.
Property 1: The spatial ODE decouples propagation depth K from terminal time T_cgp through K = T_cgp/Δt_cgp.This allows deeper propagation without requiring T_cgp to approach infinity, ensuring convergence of learned spatial representations.

APPENDIX B PROOF OF PROPERTY 2

The proof compares the analytical spatial-ODE solution with its Euler approximation and characterizes their numerical error. For fixed terminal time, increasing propagation depth drives the error toward zero, unlike unbounded integration time in the discrete formulation.

Property 2: The proposed spatial ODE has a closed-form graph-heat-diffusion solution for a fixed terminal integration time T_cgp.This solution is expanded through a Taylor series before numerical approximation.
Property 2: An Euler solver approximates the closed-form solution after K propagation steps.The proof defines numerical error by comparing the analytical and solved solutions.
Property 2: For fixed T_cgp, the numerical error approaches zero as propagation depth K approaches infinity.The stated bound gives E(K) for fixed terminal time tending toward zero with increasing K.
Property 2: Letting integration time grow with propagation depth instead leads to the over-smoothing behavior identified for the discrete propagation process.The proof contrasts fixed-terminal-time refinement with K = T approaching infinity.