MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks

Dan Li; Dacheng Chen; Lei Shi; Baihong Jin; Jonathan Goh; See-Kiong Ng

Source-linked AI summary

MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks

Dan Li, Dacheng Chen, Lei Shi, Baihong Jin, Jonathan Goh, See-Kiong Ng

arXiv:1901.04997v1Jan 15, 2019cs.LGstat.ML

TL;DR

Multivariate CPS streams are difficult to monitor because conventional methods are inadequate for dynamic systems, labels are scarce, and existing unsupervised methods may miss cross-variable dependencies. MAD-GAN addresses this gap with an LSTM-RNN GAN that models variables jointly and combines discriminator and generator information through DR-Score. On SWaT and WADI, it showed superior performance over existing unsupervised detection methods, while remaining limited by subsequence-length and GAN-stability concerns.

Problem
Dynamic CPS data lack labeled anomalies, and existing methods do not fully exploit dependencies among multiple time-series variables.
Method
MAD-GAN trains LSTM-RNN generator and discriminator models jointly on multivariate time series and combines discrimination with reconstruction in DR-Score.
Results
MAD-GAN showed superior performance over existing unsupervised detection methods on the SWaT and WADI cyber-attack datasets.
Takeaways & Limitations
Using both GAN components enables unsupervised anomaly detection that represents multivariate correlations and normal-data variability.
Takeaways & Limitations
The paper identifies subsequence-length selection and potential GAN model instability as issues requiring further investigation.

Abstract

from arXiv · show

The prevalence of networked sensors and actuators in many real-world systems such as smart buildings, factories, power plants, and data centers generate substantial amounts of multivariate time series data for these systems. The rich sensor data can be continuously monitored for intrusion events through anomaly detection. However, conventional threshold-based anomaly detection methods are inadequate due to the dynamic complexities of these systems, while supervised machine learning methods are unable to exploit the large amounts of data due to the lack of labeled data. On the other hand, current unsupervised machine learning approaches have not fully exploited the spatial-temporal correlation and other dependencies amongst the multiple variables (sensors/actuators) in the system for detecting anomalies. In this work, we propose an unsupervised multivariate anomaly detection method based on Generative Adversarial Networks (GANs). Instead of treating each data stream independently, our proposed MAD-GAN framework considers the entire variable set concurrently to capture the latent interactions amongst the variables. We also fully exploit both the generator and discriminator produced by the GAN, using a novel anomaly score called DR-score to detect anomalies by discrimination and reconstruction. We have tested our proposed MAD-GAN using two recent datasets collected from real-world CPS: the Secure Water Treatment (SWaT) and the Water Distribution (WADI) datasets. Our experimental results showed that the proposed MAD-GAN is effective in reporting anomalies caused by various cyber-intrusions compared in these complex real-world systems.

1 Introduction

Modern cyber-physical systems generate large multivariate sensor streams, but conventional and existing learning-based methods do not fully address their dynamic, unlabeled, and correlated nature. MAD-GAN is proposed as an unsupervised approach that models multivariate interactions using GANs.

Motivation: Networked sensors and actuators in CPSs generate substantial multivariate time series data for continuous anomaly monitoring.These systems include smart buildings, factories, power plants, and data centres.
Limitations of existing methods: Traditional SPC techniques cannot adequately handle multivariate streams from increasingly dynamic and complex CPSs.The passage identifies CUSUM, EWMA, and Shewhart charts as conventional examples.
Motivation: Supervised anomaly detection is constrained by the lack of labeled anomaly data, motivating predominantly unsupervised approaches.The related-work passage explicitly identifies missing labels as a central constraint.
Research gap: GAN applications to time-series data remained limited compared with their established use in image processing.The paper describes only a few preliminary works generating continuous-valued sequences.
Contribution: MAD-GAN models complex multivariate correlations and uses both the GAN-trained generator and discriminator for anomaly detection.The proposed strategy is framed as unsupervised multivariate anomaly detection with GANs.

2 Related Works

Prior unsupervised anomaly-detection methods span linear, distance-based, probabilistic, density-estimation, and deep-learning approaches. The paper positions MAD-GAN as a deep-learning method designed for multivariate time-series dependencies and cyber-attack detection.

Method categories: Unsupervised anomaly detection methods are organized into linear, distance-based, probabilistic or density-estimation, and deep-learning categories.This taxonomy reflects the lack of labeled anomaly data for supervised training.
Linear methods: PCA reduces dimensionality while preserving significant variability from correlated process measurements.PCA is presented as a popular linear model-based approach.
Linear methods: PCA and PLS are limited because they are effective only for highly correlated data and require multivariate Gaussian distributions.These constraints are stated for the linear model-based methods discussed.
Probabilistic and density-estimation methods: Probabilistic and density-estimation methods can model variable correlations but do not capture temporal correlation across time steps.The paper therefore states that they do not work well for multivariate time-series data.
Deep-learning methods: Deep-learning methods such as AE, DAGMM, and LSTM Encoder-Decoder have reported promising multivariate anomaly-detection performance.MAD-GAN extends this line of unsupervised deep-learning work using GANs.
MAD-GAN: MAD-GAN combines LSTM-RNN-based GAN modeling with discrimination and reconstruction residuals to detect cyber-attack anomalies.The contribution summary states that it outperformed existing methods on two CPS datasets.

3 Anomaly Detection with Generative Adversarial Training

MAD-GAN trains recurrent GAN components on multivariate subsequences and detects anomalies by combining discriminator outputs with generator-based reconstruction information. The framework uses sliding windows, latent-space reconstruction, and a combined DR-Score for test labeling.

Architecture: The generator and discriminator are LSTM-RNNs trained adversarially, with the generator producing sequences from random latent inputs and the discriminator distinguishing fake from real normal sequences.Both networks are updated iteratively in the GAN framework.
Architecture: MAD-GAN treats the full variable set concurrently to capture latent interactions rather than modeling each data stream independently.Multivariate sequences are divided into subsequences with a sliding window before discrimination.
Data preparation: Sliding-window subsequences use window sizes sw = 30 × i for i = 1, 2, ..., 10 to represent system status at different resolutions.The paper describes this as an empirical search for an appropriate subsequence length.
Detection: The anomaly-detection pipeline applies trained generator and discriminator models to test subsequences using a combined Discrimination and Reconstruction Anomaly Score.The discriminator provides direct detection while the generator represents normal-data variability through reconstruction.
Reconstruction: Latent-space reconstruction starts from random samples, generates reconstructions, and updates latent samples using gradients from the error between test data and generated sequences.After sufficient iterations, the resulting latent samples are recorded as mappings for the test data.
Detection: The combined anomaly losses are mapped from subsequences back to the original time series to compute the DR-Score.The resulting score is used to label test subsequences as anomalous when it exceeds a predefined threshold.

4 CPSs and Cyber-attacks

The paper evaluates MAD-GAN on SWaT and WADI, realistic water-system testbeds exposed to varied cyber-attacks. Their multivariate dynamics and delayed, cross-sensor effects motivate modeling correlations across the system.

SWaT and WADI systems: SWaT is a small-scale operational water-treatment testbed modeled on a modern city water-treatment plant.Its six subprocesses include raw-water supply, pretreatment, ultrafiltration backwash, dechlorination, reverse osmosis, and a final process.
SWaT and WADI systems: WADI extends SWaT into a large-area water-treatment, storage, and distribution network with three control processes.It includes raw-water intake and storage, demand-based distribution to reservoir and consumer tanks, and water recycling.
Cyber-attacks: WADI supports simulated cyber and physical attacks, including PLC-network attacks, water leakage, and malicious chemical injections.The collection covered 16 continuous operating days: 14 normal days and 2 attack days.
Cyber-attacks: The datasets contain attacks launched through remote SCADA access, with 36 attacks inserted into SWaT and 15 into WADI.The assumed attacker has general knowledge of system operation and seeks to manipulate normal plant behavior.
Cyber-attacks: Attacks vary in duration and stabilization time, can affect other sensors or actuators after delays, and may produce similar responses in related sensors.For example, attacks on SWaT sensor LIT101 caused abnormal spikes in both LIT101 and LIT301.
Detection motivation: These cross-variable and delayed effects support multivariate anomaly detection that uses correlations among sensors and actuators to recognize attack-related system changes.The paper argues that aggregate changes across subprocesses can help identify attacks.

5.1 Data Preparation and System Architecture

The study prepares long multivariate streams as sliding-window subsequences and uses recurrent GAN components to model them. SWaT and WADI provide high-dimensional normal and attack data for evaluation.

Data preparation: SWaT contains 51 variables measured over 11 days, while WADI contains 103 variables collected under normal and attack conditions.The supplied passage reports 496,800 normal and 449,919 attack samples for SWaT, plus 789,371 normal samples for WADI.
Data preparation: The anomaly-detection process converts long multivariate streams into smaller subsequences using sliding windows.For SWaT, window sizes were sw = 30 × i for i = 1, 2, ..., 10, with shift length ss = 10.
System architecture: The generator uses a three-layer LSTM with 100 hidden units, while the discriminator uses a one-layer LSTM with 100 hidden units.The latent-space dimension was set to 15 after testing different dimensions.

5.2 Evaluation Metrics

MAD-GAN is evaluated with precision, recall, and F1 scores. Because missing attacks is prioritized over tolerating some false alarms, recall is the principal metric.

Metrics: Precision, recall, and F1 scores are used as the standard metrics for evaluating MAD-GAN’s anomaly-detection performance.The metrics are defined using true positives, false positives, true negatives, and false negatives.
Metric priority: Recall is the main evaluation metric because the application prioritizes detecting all cyber-attacks, even while tolerating a few false alarms.False positives remain relevant but are considered less important unless they become excessive.

5.3 Results

MAD-GAN was evaluated on SWaT and WADI using multivariate generation, anomaly-detection metrics, PCA resolutions, and varying subsequence lengths. It generated realistic samples and generally achieved strong detection, while performance depended on dataset balance, feature selection, and sequence length.

Multivariate Generation: After sufficient training iterations, MAD-GAN generated realistic multivariate samples resembling the CPS training-data distribution.Early outputs were random, whereas later outputs closely matched the original samples.
Multivariate Generation: MMD values converged to small values after 30–50 iterations for both datasets, with multivariate generation converging faster than univariate generation.Early multivariate MMD values were also lower than univariate values.
Anomaly Detection Performance: On WADI, MAD-GAN’s best-F1 recall was 3.02% lower than AE’s, but its best-recall result outperformed other methods by 65.64–94.36%.Its precision was poor in the best-recall case, but recall was near 100%, which the paper considers acceptable for intrusion detection.
Sequence Length and Stability: Longer subsequences increased false positives and slowed the LSTM-RNN when sw exceeded 200, while relatively small windows such as sw = 30 were considered safe choices for these CPS datasets.Recall fluctuated without convergence across tested lengths but remained roughly within 50–100% for both datasets.
Dimension Reduction: Using only SWaT’s first principal component yielded recall below 75%, whereas adding components raised recall above 90% and five components achieved recall above 95%.The first five components explained more than 99.5% of variance, and suitable feature combinations helped reduce false positives.

6 Conclusions

The paper presents MAD-GAN for multivariate anomaly detection in CPS time series and evaluates it on SWaT and WADI. The framework uses both GAN components and outperformed existing unsupervised detection methods, while leaving subsequence selection and model stability for further study.

MAD-GAN trains LSTM-RNNs on multivariate CPS time series and uses both discriminator and generator outputs through the DR-Score.The DR-Score detects anomalies using discrimination and reconstruction.
MAD-GAN was evaluated on the SWaT and WADI cyber-attack CPS datasets.
MAD-GAN showed superior performance over existing unsupervised detection methods, including a GAN-based approach.
The authors identify optimal subsequence length and GAN model instability as issues requiring further investigation.They also propose future work on feature selection, latent and principal-component dimensions, and detection-model stability.