PaperlayerView exact revision on arXiv

Source-linked AI summary

MST-GAT: A Multimodal Spatial-Temporal Graph Attention Network for Time Series Anomaly Detection

Chaoyue Ding, Shiliang Sun, Jing Zhao

arXiv:2310.11169v1cs.LGcs.AI

TL;DR

MTS anomaly detection requires modeling complex spatial and temporal dependencies across multimodal sensors, which prior methods do not explicitly capture. MST-GAT combines multimodal graph attention, temporal convolution, and jointly optimized reconstruction-prediction modules; on benchmark datasets, it outperforms state-of-the-art baselines and provides interpretable anomaly localization.

  • Problem

    MTS anomaly detection must address complex spatial and temporal dependencies, including correlations between time series from different modalities, that previous methods fail to capture explicitly.

  • Method

    MST-GAT combines multimodal graph attention with intra-modal and inter-modal attention, temporal convolution, and jointly optimized reconstruction and prediction modules.

  • Results

    MST-GAT outperforms state-of-the-art baselines on benchmark datasets and provides interpretable results by locating anomalous univariate time series.

  • Takeaways & Limitations

    The method explicitly models multimodal spatial-temporal dependencies and supports anomaly interpretation through reconstruction and prediction results.

  • Takeaways & Limitations

    The paper identifies the failure of previous methods to explicitly capture correlations among multimodal time series as the relevant scope boundary motivating MST-GAT.

Abstract

from arXiv · show

Multimodal time series (MTS) anomaly detection is crucial for maintaining the safety and stability of working devices (e.g., water treatment system and spacecraft), whose data are characterized by multivariate time series with diverse modalities. Although recent deep learning methods show great potential in anomaly detection, they do not explicitly capture spatial-temporal relationships between univariate time series of different modalities, resulting in more false negatives and false positives. In this paper, we propose a multimodal spatial-temporal graph attention network (MST-GAT) to tackle this problem. MST-GAT first employs a multimodal graph attention network (M-GAT) and a temporal convolution network to capture the spatial-temporal correlation in multimodal time series. Specifically, M-GAT uses a multi-head attention module and two relational attention modules (i.e., intra- and inter-modal attention) to model modal correlations explicitly. Furthermore, MST-GAT optimizes the reconstruction and prediction modules simultaneously. Experimental results on four multimodal benchmarks demonstrate that MST-GAT outperforms the state-of-the-art baselines. Further analysis indicates that MST-GAT strengthens the interpretability of detected anomalies by locating the most anomalous univariate time series.

1. Introduction

MTS anomaly detection is difficult because multimodal sensors exhibit complex spatial and temporal dependencies, while existing methods often fail to model multimodal correlations explicitly. MST-GAT addresses this with graph attention, temporal convolution, joint reconstruction-prediction optimization, and anomaly localization.

  • Methods that combine univariate detection results can perform poorly because individual series do not reflect the entity’s overall state.
  • Multimodal time series contain complex spatial dependencies, including topology and modal correlation, alongside temporal dependencies such as periods and trends.
  • Earlier temporal methods capture dynamic changes but ignore spatial dependence, while CNN-based approaches can underperform on graph data with complex topology.
  • The method interprets anomalies by locating the univariate time series most likely to cause each anomaly, using reconstruction and prediction results.The paper reports that these interpretations are consistent with human intuition.
  • MST-GAT uses M-GAT with multi-head, intra-modal, and inter-modal attention to explicitly capture spatial dependencies between multimodal time series.
  • A temporal convolution network captures temporal dependencies, while jointly optimized reconstruction and prediction modules produce anomaly explanations.
  • MST-GAT achieves the highest F1-score, all above 0.60, and the best AUC, all above 0.92, outperforming strong baselines on benchmark datasets.Ablation studies support the effectiveness of its different modules.

2. Related Works

Related work spans reconstruction- and prediction-based anomaly detection, graph neural networks, and multimodal machine learning, while motivating explicit multimodal spatial-temporal modeling.

  • Time Series Anomaly Detection: Existing deep-learning anomaly detectors follow reconstruction-based or prediction-based paradigms.Reconstruction methods learn the distribution of an entire time series, whereas prediction methods use residuals from forecasting the next timestamp.
  • Time Series Anomaly Detection: Reconstruction and prediction methods are complementary, motivating a joint network that integrates both paradigms.
  • Research Gap: Existing methods do not explicitly capture relationships between multimodal data, leaving multimodal correlation insufficiently modeled.
  • Research Gap: MST-GAT addresses this gap by using multimodal graph attention to construct spatial-temporal dependencies within multimodal time series.
  • Graph Neural Networks: Graph attention networks use attention mechanisms to assign aggregation weights to neighboring nodes and extract spatial features.
  • Multimodal Machine Learning: Multimodal machine learning integrates complementary information across modalities, but fusing it into a unified representation remains a primary challenge.

3. Methodology

MST-GAT combines multimodal graph attention with temporal convolution to capture spatial and temporal dependencies in multimodal time series.

  • MST-GAT uses a multimodal graph attention network and a temporal convolution network to capture spatial and temporal dependencies.

3.1. Problem Definition

The problem concerns timestamp-level anomaly detection in multimodal time series composed of multiple univariate streams, under an anomaly-free training assumption.

  • Multimodal time series contain multiple modalities belonging to the same entity, with each modality potentially containing one or more time series.
  • The model performs timestamp-level anomaly detection and assumes that training data contain no anomalies.
  • Each sensor stream is treated as a univariate time series, and M-GAT with temporal convolution explores multimodal correlations and spatial-temporal dependencies.
  • During training, a joint optimization network uses reconstruction probabilities and prediction values; during inference, these outputs contribute to anomaly scores.
  • The detection output is a binary timestamp sequence in which 1 indicates that the corresponding observation is anomalous.

3.2. Overview of MST-GAT

MST-GAT represents sensors as graph nodes, learns spatial relationships, models intra- and inter-modal dependencies, and jointly optimizes reconstruction and prediction for interpretable anomaly scores.

  • MST-GAT treats each sensor as a graph node and relationships between sensors as edges.
  • Graph Structure Learning: Graph structure learning uses time-series embeddings to learn spatial relationships among the streams.
  • Multimodal Graph Attention Network (M-GAT): M-GAT explicitly captures intra- and inter-modal relations using multi-head and relational attention modules.
  • Temporal Convolution Network: Temporal convolution operates along the time axis to capture temporal dependencies in the time series.
  • Joint Optimization and Anomaly Score: MST-GAT jointly optimizes reconstruction and prediction targets, then identifies anomalies using anomaly scores.
  • Joint Optimization and Anomaly Score: The resulting anomaly scores are further used to interpret detected anomalies.

3.3. Graph Structure Learning

MST-GAT learns a flexible graph for multimodal time series by representing each univariate series with an embedding and connecting similar series. The resulting sparse graph encodes dependencies among series and supports spatial attention.

  • Graph representation: The graph contains one node per univariate time series, with edges indicating dependencies between different series.Neighboring nodes of i are those satisfying A_ij > 0.
  • Embedding-based graph construction: Time series embeddings characterize each univariate series and provide a flexible representation for potential cross-modal correlations.The embedding v_i belongs to R^d and is used to construct the graph.
  • Adjacency matrix: The adjacency matrix assigns A_ij = 1 when node j belongs to node i’s selected neighbor set and otherwise assigns zero.The candidate set contains all N nodes, while TopK retains the largest cosine similarities.
  • Graph construction: Cosine similarity between embedding vectors selects the top K candidate nodes for each series, producing a sparse direct graph.The parameter K controls graph sparsity.

3.4. M-GAT in Spatial Dimension

M-GAT models spatial relationships among multimodal time series through multi-head attention and separate intra- and inter-modal relational attention. These modules aggregate neighboring representations while distinguishing modality-independent, same-modality, and cross-modality relations.

  • M-GAT input: M-GAT initializes node representations from fixed-length sliding-window inputs and processes test windows in the same way.The input at time t is denoted X-hat, while the training window length is w.
  • Attention architecture: The architecture combines multi-head attention with intra-modal and inter-modal attention modules.The three modules are shown in Figure 2.
  • Relational attention: Intra-modal and inter-modal attention separately capture correlations between series from the same modality and from different modalities.Their neighbor sets are defined by modality membership and are used to model multimodal dependencies explicitly.
  • Multi-head attention: Multi-head attention aggregates neighbor representations to model modality-independent spatial relationships among multimodal time series.Each node representation is updated using attention scores from multiple heads.
  • Motivation: Unlike prior GAT approaches based only on adjacency, M-GAT gives different relations distinct influences through additional relational attention modules.The design addresses the omission of multimodal dependencies in previous methods.
  • Sparse relational graphs: When relational neighbor sets exceed K, TopK selects the K largest cosine-similarity values before attention aggregation.This applies separately to intra-modal and inter-modal neighbor sets.

3.5. Convolution in Temporal Dimension

MST-GAT combines graph attention for spatial structure with temporal convolution for dynamics across adjacent time slices. The temporal convolution operates on the graph-level representation produced by M-GAT.

  • Spatial-temporal integration: M-GAT captures neighbor information in the spatial dimension, while the temporal convolution network applies standard convolution across time.The temporal network receives the graph-level representation from the final M-GAT layer.
  • Temporal convolution: The temporal-level representation is computed through convolution with kernel size Φ followed by a ReLU activation.The convolution operation is denoted by ∗.
  • Temporal dynamics: The temporal convolution updates node features by incorporating information from adjacent time slices.This mechanism is used to capture temporal dynamics.

3.6. Joint Optimization and Anomaly Score

MST-GAT jointly trains reconstruction and prediction modules from temporal features, then combines their outputs into timestamp-level anomaly scores. Reconstruction models the time-series distribution, while prediction forecasts the next observation.

  • Joint optimization: The reconstruction and prediction modules both receive the temporal convolution output and are optimized as two complementary tasks.The reconstruction module learns the whole-series distribution, whereas prediction targets the next timestamp.
  • Joint optimization: The joint loss is a weighted combination of reconstruction loss L_rec and prediction loss L_pred, controlled by γ_1.γ_1 balances the two optimization objectives.
  • Reconstruction: A variational autoencoder reconstructs the graph-temporal representation through a latent variable and estimates reconstruction probabilities for each univariate series.The reconstruction probability is denoted p_ψ(X_t), with p_i corresponding to series i.
  • Prediction: An MLP prediction module forecasts each series at the next timestamp and uses the forecast error to define prediction loss.The forecast is x-hat_i,t+1 and the ground truth is x_i,t+1.
  • Detection rule: POT selects the anomaly threshold on the validation set, labeling timestamps above it abnormal and the remainder normal.The overall training and inference process is summarized in Algorithm 1.

4. Experiments

Experiments on four benchmark datasets compare MST-GAT with monomodal and multimodal baselines, assess its components, and examine its interpretability. MST-GAT consistently performs strongly, models multimodal dependencies, and identifies anomalous sensors.

  • Experimental setup: Experiments evaluate MST-GAT on four public benchmark datasets against eight popular monomodal and multimodal anomaly detection methods.The evaluation includes ablation studies and an interpretability case study.
  • Results and analysis: MST-GAT consistently outperforms existing baselines on four benchmarks in terms of F1-score.It substantially exceeds baselines on datasets containing more complex anomalies.
  • Results and analysis: Time-series embeddings show high similarity within modalities and reflect both intra-modal and inter-modal correlations.For example, modalities C and T have similarities in [0.29, 0.50], while P and T have similarities in [0.01, 0.37].
  • Ablation study: Removing each component reduces performance; temporal convolution and explicit intra- and inter-modal attention contribute to anomaly-detection performance.The no-attention variant performs worst, while removing temporal convolution also degrades performance.

5. Conclusions

The paper concludes that MST-GAT combines multimodal graph attention, temporal convolution, and jointly trained reconstruction and prediction modules for multimodal time-series anomaly detection. Benchmark results report stronger performance and interpretable anomaly localization, while future work targets unaligned data and deployment efficiency.

  • Conclusion: MST-GAT captures spatial correlations and temporal dependence among multimodal time series using multimodal graph attention and temporal convolution.The model is proposed for multimodal time-series anomaly detection.
  • Conclusion: Jointly trained reconstruction and prediction modules support MST-GAT's anomaly-detection framework.The paper also describes an interpretation approach based on reconstruction probability and prediction value.
  • Conclusion: Experiments on benchmark datasets show that MST-GAT outperforms state-of-the-art baselines and provides interpretable results.The reported interpretation identifies anomalous univariate time series.
  • Future work: Future work includes extending MST-GAT to unaligned multimodal time series and reducing memory footprint and execution time for real-world deployment.The paper gives sensors with different sampling rates as an example of unaligned data.
Loading 2310.11169v1…