Source-linked AI summary
Graph WaveNet for Deep Spatial-Temporal Graph Modeling
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Chengqi Zhang
TL;DR
Fixed graphs may miss or misrepresent node dependencies, while standard temporal models struggle with long-range sequences. Graph WaveNet learns spatial dependencies adaptively and uses dilated convolutions for temporal modeling, achieving state-of-the-art results on METR-LA and PEMS-BAY.
Problem
Existing methods rely on graph structures that may misrepresent or omit genuine dependencies and struggle to learn long-range temporal dependencies.
Method
Graph WaveNet jointly learns a self-adaptive adjacency matrix from data and uses stacked dilated temporal convolutions to model spatial-temporal dependencies.
Results
Graph WaveNet achieves state-of-the-art results on the METR-LA and PEMS-BAY traffic datasets, outperforming reported temporal and spatial-temporal baselines.
Takeaways & Limitations
Graph WaveNet supports spatial-temporal modeling when dependency structures are unknown and need to be discovered from data.
Abstract
from arXiv · showhide
Spatial-temporal graph modeling is an important task to analyze the spatial relations and temporal trends of components in a system. Existing approaches mostly capture the spatial dependency on a fixed graph structure, assuming that the underlying relation between entities is pre-determined. However, the explicit graph structure (relation) does not necessarily reflect the true dependency and genuine relation may be missing due to the incomplete connections in the data. Furthermore, existing methods are ineffective to capture the temporal trends as the RNNs or CNNs employed in these methods cannot capture long-range temporal sequences. To overcome these limitations, we propose in this paper a novel graph neural network architecture, Graph WaveNet, for spatial-temporal graph modeling. By developing a novel adaptive dependency matrix and learn it through node embedding, our model can precisely capture the hidden spatial dependency in the data. With a stacked dilated 1D convolution component whose receptive field grows exponentially as the number of layers increases, Graph WaveNet is able to handle very long sequences. These two components are integrated seamlessly in a unified framework and the whole framework is learned in an end-to-end manner. Experimental results on two public traffic network datasets, METR-LA and PEMS-BAY, demonstrate the superior performance of our algorithm.
1 Introduction
The introduction frames spatial-temporal graph modeling as learning dynamic node features from graph-structured dependencies, while identifying incomplete or inaccurate graphs and weak long-range temporal modeling as key limitations. Graph WaveNet addresses these issues with a learned self-adaptive adjacency matrix and dilated causal convolutions in an end-to-end CNN-based framework.
- Background: Spatial-temporal graph modeling predicts dynamic node-level inputs using inter-dependencies among connected nodes and supports applications including traffic forecasting, taxi demand prediction, and human action recognition.The task models each node’s dynamic features given the graph structure.
- Limitations: Existing approaches assume observed graph connections accurately represent genuine dependencies, although dependencies may exist without links or links may lack meaningful dependency.Attention can adjust weights between connected nodes but does not address missing connections.
- Limitations: RNN-based methods struggle with long-range temporal dependencies because iterative propagation is time-consuming and vulnerable to gradient explosion or vanishing.CNN-based approaches offer parallel computing, stable gradients, and low memory requirements, but the passage indicates limitations remain for long-range modeling.
- Graph WaveNet: Graph WaveNet learns a self-adaptive adjacency matrix from data through end-to-end supervised training to preserve hidden spatial dependencies and uncover unseen graph structures.Experiments validate improvements when spatial dependencies exist but are not provided in advance.
- Graph WaveNet: Graph WaveNet combines graph convolution with stacked dilated causal convolution to capture spatial-temporal dependencies simultaneously at multiple temporal granularities.The framework is evaluated on traffic datasets, achieving state-of-the-art results with low computation costs.
2 Related Works
Prior work uses graph convolution networks broadly for graph-structured learning, with spectral- and spatial-based approaches forming two mainstreams. Spatial-temporal graph networks primarily follow RNN-based or CNN-based directions, including graph convolution, diffusion convolution, and attention mechanisms.
- Graph convolution networks support learning on graph-structured data and applications including node embedding, classification, link prediction, and clustering.
- Two mainstream graph convolution approaches are spectral-based methods, which smooth node inputs using graph spectral filters, and spatial-based methods.
- Most spatial-temporal graph networks follow RNN-based or CNN-based approaches to model spatial-temporal dependencies.
- RNN-based methods have used graph convolution to filter inputs and hidden states before recurrent processing, while later work adopted diffusion convolution and attention mechanisms.
3 Methodology
Graph WaveNet models spatial-temporal dependencies by combining graph convolution with gated dilated temporal convolution. Its self-adaptive adjacency matrix learns hidden spatial dependencies, while the architecture directly produces multi-step forecasts from historical graph signals.
- Problem Definition: The task is to learn a function that forecasts future graph signals from S historical graph-signal steps on a graph G.Graph signals are dynamic feature matrices X(t) ∈ R^N×D, with historical and forecast sequences represented over S and T steps.
- Graph Convolution: Graph WaveNet learns a self-adaptive adjacency matrix end-to-end without prior knowledge, allowing it to discover hidden spatial dependencies.Two learnable node-embedding dictionaries generate dependency weights, which are processed with ReLU and SoftMax to form a normalized hidden transition matrix.
- Temporal Convolution: The gated temporal convolution uses dilated causal convolutions whose exponentially growing receptive field captures long-range temporal dependencies non-recursively.The gating operation combines an activation branch with a sigmoid-controlled branch to regulate information passed through the temporal layer.
- Framework Architecture: Graph WaveNet stacks spatial-temporal layers, each combining a graph convolution layer with a gated temporal convolution layer, followed by an output layer.The framework stacks multiple layers to model spatial dependencies at different temporal levels.
- Framework Architecture: Unlike recursive multi-step approaches, Graph WaveNet outputs X̂(t+1):(t+T) as a whole to avoid training–inference inconsistency.Its receptive field is designed to match the input sequence length so the final temporal dimension equals one.
4 Experiments
Experiments on METR-LA and PEMS-BAY show that Graph WaveNet achieves superior traffic-prediction performance, including stronger long-horizon predictions and more stable curves than WaveNet. Ablations support adaptive adjacency, while computation results show efficient inference and faster training than DCRNN.
- Prediction performance: Graph WaveNet outperforms temporal and spatial-temporal baselines, with larger improvement over GGRU at the 60-minute horizon than at 15 minutes.It surpasses ARIMA, FC-LSTM, WaveNet, STGCN, DCRNN, and GGRU, while its 15-minute gain over GGRU is small.
- Prediction performance: Graph WaveNet generates more stable 60-minute predictions than WaveNet, avoiding a sharp spike that deviates far from real values.Its prediction curve remains between the real-value curves throughout the illustrated test snapshot.
- Adjacency ablation: The forward-backward-adaptive adjacency configuration achieves the best results, while adaptive-only performance is nearly the same as forward-only.Across 12 prediction horizons, adaptive-only also performs better than forward-only on mean MAE and remains effective without a graph structure.
- Computation cost: Graph WaveNet runs five times faster than DCRNN and is the most efficient model at inference, although STGCN trains two times faster.Graph WaveNet generates 12 predictions in one inference run, unlike DCRNN and STGCN, which condition predictions on previous outputs.
5 Conclusion
The paper presents a spatial-temporal graph model that combines graph convolution with dilated causal convolution and automatically learns hidden spatial dependencies from data.
- 5 Conclusion: The model captures spatial-temporal dependencies by combining graph convolution with dilated causal convolution.This combination is presented as efficient and effective for spatial-temporal graph modeling.
- 5 Conclusion: The proposed method automatically learns hidden spatial dependencies from data.It targets systems whose dependency structure is unknown but must be discovered.