Source-linked AI summary
Fully-Connected Spatial-Temporal Graph for Multivariate Time-Series Data
Yucheng Wang, Yuecong Xu, Jianfei Yang, Min Wu, Xiaoli Li, Lihua Xie, Zhenghua Chen
TL;DR
Existing MTS methods often separate spatial and temporal modelling and overlook correlations between different sensors at different timestamps. FC-STGNN builds a decay-refined fully connected graph and applies moving-pooling graph convolution; across multiple datasets, it reports improvements over competing methods.
Problem
Existing MTS approaches overlook correlations between different sensors at different timestamps, limiting comprehensive modelling of spatial-temporal dependencies.
Method
FC-STGNN constructs a fully connected graph across timestamps with temporal-distance decay and uses moving-pooling graph convolution to capture local ST dependencies.
Results
FC-STGNN improves over second-best methods by 7.6% and 3.4% in FD001 and FD003 RMSE, and by 1.02% and 1.56% in UCI-HAR and ISRUC-S3 accuracy.
Takeaways & Limitations
The experiments support FC-STGNN’s effectiveness for modelling comprehensive and complex ST dependencies in MTS data.
Takeaways & Limitations
Directly applying existing attributed-graph methods is challenging because MTS sensors are unattributed nodes with only time-series signals.
Abstract
from arXiv · showhide
Multivariate Time-Series (MTS) data is crucial in various application fields. With its sequential and multi-source (multiple sensors) properties, MTS data inherently exhibits Spatial-Temporal (ST) dependencies, involving temporal correlations between timestamps and spatial correlations between sensors in each timestamp. To effectively leverage this information, Graph Neural Network-based methods (GNNs) have been widely adopted. However, existing approaches separately capture spatial dependency and temporal dependency and fail to capture the correlations between Different sEnsors at Different Timestamps (DEDT). Overlooking such correlations hinders the comprehensive modelling of ST dependencies within MTS data, thus restricting existing GNNs from learning effective representations. To address this limitation, we propose a novel method called Fully-Connected Spatial-Temporal Graph Neural Network (FC-STGNN), including two key components namely FC graph construction and FC graph convolution. For graph construction, we design a decay graph to connect sensors across all timestamps based on their temporal distances, enabling us to fully model the ST dependencies by considering the correlations between DEDT. Further, we devise FC graph convolution with a moving-pooling GNN layer to effectively capture the ST dependencies for learning effective representations. Extensive experiments show the effectiveness of FC-STGNN on multiple MTS datasets compared to SOTA methods. The code is available at https://github.com/Frank-Wang-oss/FCSTGNN.
Introduction
Existing MTS graph approaches separately model spatial and temporal dependencies, while FC-STGNN connects sensors across timestamps and uses moving-pooling graph convolution to capture broader and local ST dependencies.
- FC graph convolution: Its moving-pooling GNN layer uses sliding windows to capture local ST dependencies and temporal pooling to extract high-level features.This design avoids directly convolving over all sensors and patches while preserving local temporal patterns.
- Motivation: Existing approaches overlook correlations between different sensors at different timestamps, limiting comprehensive ST-dependency modelling.The omitted DEDT correlations are a central motivation for FC-STGNN.
- FC graph construction: FC-STGNN constructs a fully connected ST graph across timestamps using a temporal distance-based decay matrix.The decay matrix refines cross-timestamp connections while modelling comprehensive ST dependencies.
- Evaluation: Extensive experiments show FC-STGNN effectively models and captures complex ST dependencies in MTS data.The contribution statement reports effectiveness across the conducted experiments without specifying a single aggregate metric.
Related Work
Traditional MTS methods focus on temporal correlations, while GNN-based approaches add spatial modelling but generally do not explicitly capture cross-sensor, cross-timestamp correlations. FC-STGNN addresses this limitation with a framework designed for comprehensive ST dependency modelling.
- Conventional methods: Traditional MTS methods primarily use temporal encoders to capture correlations between timestamps.Examples include 1D-CNN, LSTM, and Transformer-based encoders.
- Conventional methods: These methods overlook spatial dependencies arising from signals collected by multiple sensors.Such spatial correlations can matter because readings from different sensors may be related.
- GNN-based methods: GNN-based MTS methods combine graph neural networks with temporal encoders to capture spatial and temporal dependencies separately.The related-work discussion identifies this combination as a common approach.
- Remaining limitation: Existing approaches do not explicitly consider correlations between different sensors at different timestamps, limiting comprehensive ST modelling.Methods developed for attributed graphs are also challenging to apply directly because MTS sensors are unattributed nodes with time-series signals.
- Proposed direction: FC-STGNN is introduced to address this limitation and enhance representation learning for MTS data.The framework targets comprehensive ST-dependency modelling rather than only separate spatial or temporal processing.
Methodology
FC-STGNN segments multivariate signals into patches, encodes sensor features, and constructs a fully connected graph across sensors and timestamps. A decay-weighted graph is processed with moving-window message passing and temporal pooling to capture comprehensive and local spatial-temporal dependencies.
- Graph Construction: The method segments each sensor’s signal into patches, encodes sensor-level features, and adds positional information across patches.Patch size determines the number of segmented timestamps, while positional encoding preserves relative patch directionality.
- Graph Construction: FC graph construction connects sensors across all patches using feature similarity, thereby modeling temporal, spatial, and different-sensor/different-timestamp dependencies.Similarity is computed with a learned dot-product transformation and represented in the graph adjacency matrix.
- Decay Matrix: A temporal-distance decay matrix strengthens correlations between temporally close sensors and weakens correlations between distant timestamps.The decay-weighted edge is defined by multiplying similarity with δ^(t-r).
- FC Graph Convolution: Moving-pooling graph convolution applies an MPNN within sliding windows to capture local spatial-temporal dependencies without convolving over the entire graph.A window of size M moves across patches with stride s, and message passing propagates neighbor information before feature updating.
- FC Graph Convolution: Temporal average pooling converts updated sensor features within each window into high-level features for downstream representation learning.Features from the windows are stacked, concatenated across parallel layers, and passed to an output layer.
Experimental Results
The experiments evaluate FC-STGNN on remaining useful life prediction, human activity recognition, and sleep stage classification using established datasets and metrics. Results are averaged across ten runs, with implementation details reported for reproducibility.
- Datasets: Experiments cover RUL prediction on C-MAPSS, HAR on UCI-HAR, and SSC on ISRUC-S3.C-MAPSS uses predefined train-test splits, while HAR and ISRUC-S3 use random splits.
- Evaluation: RUL prediction uses RMSE and Score, whereas HAR and SSC use Accuracy and Macro-averaged F1-Score.Lower values indicate better RUL performance; larger values indicate better HAR and SSC performance.
- Evaluation: All experiments are repeated ten times and averaged to reduce the effect of random initialization.Models are implemented in PyTorch 1.9 and trained for 40 epochs with ADAM at a 1e-3 learning rate.
Comparisons with State-of-the-Art
FC-STGNN is compared with conventional, Transformer-based, and GNN-based methods across C-MAPSS, UCI-HAR, and ISRUC-S3, showing large improvements in most cases.
- Comparison setup: FC-STGNN is evaluated against conventional, Transformer-based, and GNN-based methods using re-implemented original configurations.The comparison includes AConvLSTM, DAGN, InFormer, AutoFormer, GCN, HAGCN, HierCorrPool, and MAGNN.
- C-MAPSS: 7.6% and 3.4% RMSE improvements over the second-best results are reported for FD001 and FD003 of C-MAPSS, respectively.
- UCI-HAR and ISRUC-S3: 1.02% and 1.56% accuracy improvements over second-best methods are reported for UCI-HAR and ISRUC-S3, respectively.
Ablation Study
Ablation results indicate that fully connected graph construction, moving-window convolution, temporal pooling, and decay weighting each contribute to performance.
- Ablation design: The ablation study removes fully connected graph construction and convolution, moving-window processing, temporal pooling, and decay weighting in separate variants.
- Graph construction: 7.6% improvement over the w/o FC GC2 variant shows the benefit of the complete method on FD001 RMSE.
- Graph construction: Adding FC graph construction reduces the performance gap to 4.5% even without modelling local spatial-temporal dependencies.
- Moving-window convolution: Moving-window processing further narrows the gap to 3.4% by capturing local spatial-temporal dependencies.
- Temporal pooling: Temporal pooling enhances performance by extracting high-level sensor features and eliminating redundant features.
- Decay weighting: Removing the decay matrix decreases performance by 4.3%, while the observations hold across other sub-datasets.
Sensitivity Analysis
Sensitivity analyses examine parallel layers, patch sizes, moving windows, and decay rates, identifying settings that balance dependency coverage, local patterns, and distortion.
- No. of Parallel Layers: Two parallel layers outperform one, while additional layers can overfit and make further depth unnecessary.
- Patch Size Analysis: Smaller patch sizes work better for short C-MAPSS samples, whereas larger ISRUC-S3 samples perform better with patch sizes around 75.FD001 reaches optimal RMSE at patch size 6, while ISRUC-S3 improves around 75.
- Moving Window Size Analysis: Window size M = 2 and stride s = 1 are reported as optimal for capturing local spatial-temporal dependencies.M = 1 does not consider correlations between different sensors at different timestamps, while overly large windows can reduce performance.
- Decay Rate Analysis: Larger decay rates such as δ = 0.7 and δ = 0.9 perform better, whereas δ = 0.1 significantly distorts different-sensor, different-timestamp correlations.δ = 1 represents the variant without the decay matrix.
Model Complexity
The paper evaluates model complexity using FLOPs, trainable weights, training time, and inference time across RUL prediction and SSC scenarios. FC-STGNN shows reasonable complexity compared with SOTA approaches and requires the fewest FLOPs and trainable weights.
- The comparison measures FLOPs, trainable weights, training time, and inference time across RUL prediction and SSC scenarios.All methods are evaluated on the same computation platform, with training measured until convergence and inference measured per sample.
- FC-STGNN exhibits reasonable model complexity compared with four highly competitive approaches.
- FC-STGNN requires the fewest FLOPs and trainable weights among the compared methods.
Conclusion
FC-STGNN models comprehensive spatial-temporal dependencies by connecting sensors across timestamps and applying moving-pooling graph convolution. Experiments evaluate its capacity to model these dependencies in multivariate time-series data.
- FC-STGNN connects sensors across all timestamps to model correlations between different sensors at different timestamps.Its FC graph construction additionally considers DEDT correlations to represent comprehensive spatial-temporal dependencies.
- Its moving-pooling GNN uses a moving window and temporal pooling to capture local spatial-temporal dependencies and learn high-level features.
- Extensive experiments evaluate FC-STGNN’s capacity to model comprehensive spatial-temporal dependencies within multivariate time-series data.