PaperlayerView exact revision on arXiv

Source-linked AI summary

Fully-Connected Spatial-Temporal Graph for Multivariate Time-Series Data

Yucheng Wang, Yuecong Xu, Jianfei Yang, Min Wu, Xiaoli Li, Lihua Xie, Zhenghua Chen

arXiv:2309.05305v3cs.LG

TL;DR

Existing MTS methods often separate spatial and temporal modelling and overlook correlations between different sensors at different timestamps. FC-STGNN builds a decay-refined fully connected graph and applies moving-pooling graph convolution; across multiple datasets, it reports improvements over competing methods.

  • Problem

    Existing MTS approaches overlook correlations between different sensors at different timestamps, limiting comprehensive modelling of spatial-temporal dependencies.

    Source: Related Work · pp. 2–3

  • Method

    FC-STGNN constructs a fully connected graph across timestamps with temporal-distance decay and uses moving-pooling graph convolution to capture local ST dependencies.

    Source: Introduction · p. 2, Graph Construction · p. 4, FC Graph Convolution · p. 4

  • Results

    FC-STGNN improves over second-best methods by 7.6% and 3.4% in FD001 and FD003 RMSE, and by 1.02% and 1.56% in UCI-HAR and ISRUC-S3 accuracy.

    Source: Comparisons with State-of-the-Art · p. 6

  • Takeaways & Limitations

    The experiments support FC-STGNN’s effectiveness for modelling comprehensive and complex ST dependencies in MTS data.

    Source: Introduction · p. 2, Conclusion · p. 8

    Directly applying existing attributed-graph methods is challenging because MTS sensors are unattributed nodes with only time-series signals.

    Source: Related Work · p. 3

Abstract

from arXiv · show

Multivariate Time-Series (MTS) data is crucial in various application fields. With its sequential and multi-source (multiple sensors) properties, MTS data inherently exhibits Spatial-Temporal (ST) dependencies, involving temporal correlations between timestamps and spatial correlations between sensors in each timestamp. To effectively leverage this information, Graph Neural Network-based methods (GNNs) have been widely adopted. However, existing approaches separately capture spatial dependency and temporal dependency and fail to capture the correlations between Different sEnsors at Different Timestamps (DEDT). Overlooking such correlations hinders the comprehensive modelling of ST dependencies within MTS data, thus restricting existing GNNs from learning effective representations. To address this limitation, we propose a novel method called Fully-Connected Spatial-Temporal Graph Neural Network (FC-STGNN), including two key components namely FC graph construction and FC graph convolution. For graph construction, we design a decay graph to connect sensors across all timestamps based on their temporal distances, enabling us to fully model the ST dependencies by considering the correlations between DEDT. Further, we devise FC graph convolution with a moving-pooling GNN layer to effectively capture the ST dependencies for learning effective representations. Extensive experiments show the effectiveness of FC-STGNN on multiple MTS datasets compared to SOTA methods. The code is available at https://github.com/Frank-Wang-oss/FCSTGNN.

Introduction

Existing MTS graph approaches separately model spatial and temporal dependencies, while FC-STGNN connects sensors across timestamps and uses moving-pooling graph convolution to capture broader and local ST dependencies.

FC graph convolution

  • Its moving-pooling GNN layer uses sliding windows to capture local ST dependencies and temporal pooling to extract high-level features.This design avoids directly convolving over all sensors and patches while preserving local temporal patterns.Source: Introduction · p. 2

Motivation

  • Existing approaches overlook correlations between different sensors at different timestamps, limiting comprehensive ST-dependency modelling.The omitted DEDT correlations are a central motivation for FC-STGNN.Source: Introduction · p. 2

FC graph construction

  • FC-STGNN constructs a fully connected ST graph across timestamps using a temporal distance-based decay matrix.The decay matrix refines cross-timestamp connections while modelling comprehensive ST dependencies.Source: Introduction · p. 2

Evaluation

  • Extensive experiments show FC-STGNN effectively models and captures complex ST dependencies in MTS data.The contribution statement reports effectiveness across the conducted experiments without specifying a single aggregate metric.Source: Introduction · p. 2

Related Work

Traditional MTS methods focus on temporal correlations, while GNN-based approaches add spatial modelling but generally do not explicitly capture cross-sensor, cross-timestamp correlations. FC-STGNN addresses this limitation with a framework designed for comprehensive ST dependency modelling.

Conventional methods

  • Traditional MTS methods primarily use temporal encoders to capture correlations between timestamps.Examples include 1D-CNN, LSTM, and Transformer-based encoders.Source: Related Work · p. 2
  • These methods overlook spatial dependencies arising from signals collected by multiple sensors.Such spatial correlations can matter because readings from different sensors may be related.Source: Related Work · p. 2

GNN-based methods

  • GNN-based MTS methods combine graph neural networks with temporal encoders to capture spatial and temporal dependencies separately.The related-work discussion identifies this combination as a common approach.Source: Related Work · p. 2

Remaining limitation

  • Existing approaches do not explicitly consider correlations between different sensors at different timestamps, limiting comprehensive ST modelling.Methods developed for attributed graphs are also challenging to apply directly because MTS sensors are unattributed nodes with time-series signals.Source: Related Work · p. 3

Proposed direction

  • FC-STGNN is introduced to address this limitation and enhance representation learning for MTS data.The framework targets comprehensive ST-dependency modelling rather than only separate spatial or temporal processing.Source: Related Work · p. 3

Methodology

FC-STGNN segments multivariate signals into patches, encodes sensor features, and constructs a fully connected graph across sensors and timestamps. A decay-weighted graph is processed with moving-window message passing and temporal pooling to capture comprehensive and local spatial-temporal dependencies.

Graph Construction

  • The method segments each sensor’s signal into patches, encodes sensor-level features, and adds positional information across patches.Patch size determines the number of segmented timestamps, while positional encoding preserves relative patch directionality.Source: Overall Structure · p. 3, Graph Construction · p. 4
  • FC graph construction connects sensors across all patches using feature similarity, thereby modeling temporal, spatial, and different-sensor/different-timestamp dependencies.Similarity is computed with a learned dot-product transformation and represented in the graph adjacency matrix.Source: Graph Construction · p. 4

Decay Matrix

  • A temporal-distance decay matrix strengthens correlations between temporally close sensors and weakens correlations between distant timestamps.The decay-weighted edge is defined by multiplying similarity with δ^(t-r).Source: Decay Matrix · p. 4

FC Graph Convolution

  • Moving-pooling graph convolution applies an MPNN within sliding windows to capture local spatial-temporal dependencies without convolving over the entire graph.A window of size M moves across patches with stride s, and message passing propagates neighbor information before feature updating.Source: FC Graph Convolution · pp. 4–5
  • Temporal average pooling converts updated sensor features within each window into high-level features for downstream representation learning.Features from the windows are stacked, concatenated across parallel layers, and passed to an output layer.Source: Overall Structure · p. 4, FC Graph Convolution · p. 5

Experimental Results

The experiments evaluate FC-STGNN on remaining useful life prediction, human activity recognition, and sleep stage classification using established datasets and metrics. Results are averaged across ten runs, with implementation details reported for reproducibility.

Datasets

  • Experiments cover RUL prediction on C-MAPSS, HAR on UCI-HAR, and SSC on ISRUC-S3.C-MAPSS uses predefined train-test splits, while HAR and ISRUC-S3 use random splits.Source: Experimental Results · p. 5

Evaluation

  • RUL prediction uses RMSE and Score, whereas HAR and SSC use Accuracy and Macro-averaged F1-Score.Lower values indicate better RUL performance; larger values indicate better HAR and SSC performance.Source: Experimental Results · p. 5
  • All experiments are repeated ten times and averaged to reduce the effect of random initialization.Models are implemented in PyTorch 1.9 and trained for 40 epochs with ADAM at a 1e-3 learning rate.Source: Experimental Results · p. 5

Comparisons with State-of-the-Art

FC-STGNN is compared with conventional, Transformer-based, and GNN-based methods across C-MAPSS, UCI-HAR, and ISRUC-S3, showing large improvements in most cases.

Comparison setup

  • FC-STGNN is evaluated against conventional, Transformer-based, and GNN-based methods using re-implemented original configurations.The comparison includes AConvLSTM, DAGN, InFormer, AutoFormer, GCN, HAGCN, HierCorrPool, and MAGNN.Source: Comparisons with State-of-the-Art · p. 5

C-MAPSS

UCI-HAR and ISRUC-S3

Ablation Study

Ablation results indicate that fully connected graph construction, moving-window convolution, temporal pooling, and decay weighting each contribute to performance.

Ablation design

  • The ablation study removes fully connected graph construction and convolution, moving-window processing, temporal pooling, and decay weighting in separate variants.Source: Ablation Study · p. 6

Graph construction

  • 7.6% improvement over the w/o FC GC2 variant shows the benefit of the complete method on FD001 RMSE.Source: Ablation Study · p. 6
  • Adding FC graph construction reduces the performance gap to 4.5% even without modelling local spatial-temporal dependencies.Source: Ablation Study · p. 6

Moving-window convolution

  • Moving-window processing further narrows the gap to 3.4% by capturing local spatial-temporal dependencies.Source: Ablation Study · p. 6

Temporal pooling

  • Temporal pooling enhances performance by extracting high-level sensor features and eliminating redundant features.Source: Ablation Study · p. 6

Decay weighting

  • Removing the decay matrix decreases performance by 4.3%, while the observations hold across other sub-datasets.Source: Ablation Study · p. 6

Sensitivity Analysis

Sensitivity analyses examine parallel layers, patch sizes, moving windows, and decay rates, identifying settings that balance dependency coverage, local patterns, and distortion.

No. of Parallel Layers

Patch Size Analysis

  • Smaller patch sizes work better for short C-MAPSS samples, whereas larger ISRUC-S3 samples perform better with patch sizes around 75.FD001 reaches optimal RMSE at patch size 6, while ISRUC-S3 improves around 75.Source: Patch Size Analysis · p. 7

Moving Window Size Analysis

  • Window size M = 2 and stride s = 1 are reported as optimal for capturing local spatial-temporal dependencies.M = 1 does not consider correlations between different sensors at different timestamps, while overly large windows can reduce performance.Source: Moving Window Size Analysis · p. 8

Decay Rate Analysis

  • Larger decay rates such as δ = 0.7 and δ = 0.9 perform better, whereas δ = 0.1 significantly distorts different-sensor, different-timestamp correlations.δ = 1 represents the variant without the decay matrix.Source: Decay Rate Analysis · p. 8

Model Complexity

The paper evaluates model complexity using FLOPs, trainable weights, training time, and inference time across RUL prediction and SSC scenarios. FC-STGNN shows reasonable complexity compared with SOTA approaches and requires the fewest FLOPs and trainable weights.

  • The comparison measures FLOPs, trainable weights, training time, and inference time across RUL prediction and SSC scenarios.All methods are evaluated on the same computation platform, with training measured until convergence and inference measured per sample.Source: Model Complexity · p. 8
  • FC-STGNN exhibits reasonable model complexity compared with four highly competitive approaches.Source: Model Complexity · p. 8
  • FC-STGNN requires the fewest FLOPs and trainable weights among the compared methods.Source: Model Complexity · p. 8

Conclusion

FC-STGNN models comprehensive spatial-temporal dependencies by connecting sensors across timestamps and applying moving-pooling graph convolution. Experiments evaluate its capacity to model these dependencies in multivariate time-series data.

  • FC-STGNN connects sensors across all timestamps to model correlations between different sensors at different timestamps.Its FC graph construction additionally considers DEDT correlations to represent comprehensive spatial-temporal dependencies.Source: Conclusion · p. 8
  • Its moving-pooling GNN uses a moving window and temporal pooling to capture local spatial-temporal dependencies and learn high-level features.Source: Conclusion · p. 8
  • Extensive experiments evaluate FC-STGNN’s capacity to model comprehensive spatial-temporal dependencies within multivariate time-series data.Source: Conclusion · p. 8

Related papers

Loading 2309.05305v3…