PaperlayerView exact revision on arXiv

Source-linked AI summary

Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series

Dan Li, Dacheng Chen, Jonathan Goh, See-kiong Ng

arXiv:1809.04758v3cs.LGstat.ML

TL;DR

CPS anomaly detection lacks sufficient labelled anomalies and must capture nonlinear dependencies across interconnected sensor and actuator streams. The paper proposes GAN-AD, an unsupervised LSTM-RNN GAN combining reconstruction residuals with discriminator outputs, and reports improved multivariate detection on the six-stage SWaT system.

  • Problem

    CPS anomaly detection is challenged by scarce labelled anomalies, nonlinear multivariate correlations, and interactions across sensor and actuator streams.

  • Method

    GAN-AD trains LSTM-RNN generator and discriminator networks on multivariate normal sequences and combines reconstruction residuals with discriminator loss for anomaly detection.

  • Results

    GAN-AD improves multivariate detection over SPE by about 3%–12%, with 50%–60% higher precision and 5%–40% higher recall.

  • Takeaways & Limitations

    The method detects cyber-attack anomalies in a complex six-stage SWaT system using both trained GAN components.

Abstract

from arXiv · show

Today's Cyber-Physical Systems (CPSs) are large, complex, and affixed with networked sensors and actuators that are targets for cyber-attacks. Conventional detection techniques are unable to deal with the increasingly dynamic and complex nature of the CPSs. On the other hand, the networked sensors and actuators generate large amounts of data streams that can be continuously monitored for intrusion events. Unsupervised machine learning techniques can be used to model the system behaviour and classify deviant behaviours as possible attacks. In this work, we proposed a novel Generative Adversarial Networks-based Anomaly Detection (GAN-AD) method for such complex networked CPSs. We used LSTM-RNN in our GAN to capture the distribution of the multivariate time series of the sensors and actuators under normal working conditions of a CPS. Instead of treating each sensor's and actuator's time series independently, we model the time series of multiple sensors and actuators in the CPS concurrently to take into account of potential latent interactions between them. To exploit both the generator and the discriminator of our GAN, we deployed the GAN-trained discriminator together with the residuals between generator-reconstructed data and the actual samples to detect possible anomalies in the complex CPS. We used our GAN-AD to distinguish abnormal attacked situations from normal working conditions for a complex six-stage Secure Water Treatment (SWaT) system. Experimental results showed that the proposed strategy is effective in identifying anomalies caused by various attacks with high detection rate and low false positive rate as compared to existing methods.

I. INTRODUCTION

CPS anomaly detection is difficult because attacks affect complex, interconnected systems and existing methods struggle with nonlinear multivariate dynamics, scarce labels, and indirect effects. The paper proposes GAN-AD, an unsupervised LSTM-RNN GAN that uses both reconstruction residuals and discriminator outputs to detect attacks.

  • Motivation: CPSs connect networked sensors and actuators in mission-critical systems, making cyber-attacks a major security concern.Examples include water treatment, power, oil, gas, and autonomous-vehicle systems.
  • Limitations of existing methods: Existing detection techniques struggle with increasingly dynamic and complex CPSs, motivating supervised and unsupervised machine-learning approaches.Traditional SPC methods such as CUSUM, EWMA, and Shewhart charts are among the conventional approaches discussed.
  • Limitations of existing methods: Supervised methods often lack labelled anomaly classes, while unsupervised linear methods may miss nonlinear hidden correlations in multivariate CPS time series.Anomalies are typically rare, and simple comparisons with fixed normal ranges may also miss indirect attacks.
  • Proposed method: GAN-AD models nonlinear correlations among multiple sensor and actuator streams using an unsupervised GAN framework with LSTM-RNN generator and discriminator networks.The method adapts GAN-based generation to multivariate time series rather than treating each stream independently.
  • Proposed method: GAN-AD combines generator reconstruction residuals with discriminator-based discrimination loss to identify anomalous sequential CPS data.Testing samples are mapped into latent space for reconstruction, while the trained discriminator evaluates the real-time series.
  • Evaluation: The method evaluates cyber-attacks in a complex six-stage Secure Water Treatment system.The SWaT system is used as the paper’s CPS testbed.

III. ANOMALY DETECTION WITH GENERATIVE ADVERSARIAL TRAINING

The method replaces both GAN components with LSTM-RNNs to model CPS time series through adversarial training. It then uses generator reconstruction and discriminator classification to derive anomaly evidence.

  • A. GAN with LSTM-RNN: Both the GAN generator and discriminator are implemented as LSTM-RNNs to learn complex CPS time series.The model follows the regular GAN structure while representing temporal dependencies with recurrent networks.
  • A. GAN with LSTM-RNN: The generator maps random latent-space samples to generated sequences, while the discriminator distinguishes real training sequences from generated ones.The discriminator minimizes average negative cross entropy between predictions and sequence labels.
  • A. GAN with LSTM-RNN: Adversarial optimization jointly updates generator and discriminator parameters so the generator confuses the discriminator.The two networks are trained as a two-player minimax game.
  • Anomaly detection procedure: The anomaly-detection algorithm generates latent-space samples, evaluates real and generated sequences, and records current network parameters during training.These operations form the training procedure for the LSTM-RNN GAN.
  • Anomaly detection procedure: For testing, the method reconstructs samples through the generator, computes residuals against observed data, evaluates the discriminator, and combines the results into an anomaly score.The algorithm records residual and discrimination outputs before calculating the final score.

B. GAN-based Anomaly Score

The GAN-based anomaly score combines discriminator-based classification with residuals between testing samples and generator reconstructions. The generator maps latent representations to normal-distribution-like data, while iterative latent-space mapping supports residual calculation.

  • GAN-based Anomaly Score: The anomaly score uses both discriminator classification and residuals between testing samples and generator-reconstructed samples.The discriminator classifies real-time series, while residual loss compares testing data with reconstructions obtained through latent-space mapping.
  • Anomaly Detection with Discrimination: The trained discriminator directly detects anomalies by distinguishing fake samples from real samples with high sensitivity.
  • Anomaly Detection with Residuals: The generator represents the normal data distribution by mapping latent-space inputs to realistic samples.Similarity between a testing sample and its generator reconstruction indicates how closely it follows the learned distribution.
  • Anomaly Detection with Residuals: Latent representations are iteratively updated from reconstruction-error gradients until a sufficiently small error identifies the testing sample’s corresponding mapping.The procedure begins with a random latent set and generates reconstructed raw samples before updating the latent inputs.
  • Optimization: The strategy is summarized as mini-batch stochastic optimization using Adam and gradient descent to update model parameters.

C. Anomaly Detection Framework

The framework trains GAN-AD on normal multivariate time-series windows, then analyzes testing data for anomalous slots. PCA reduces high-dimensional inputs before anomaly scores and variable-level labels are produced.

  • Anomaly Detection Framework: GAN-AD trains on normal time-series data and analyzes real-time testing data to detect anomalous slots.The model generates fake samples that look real and applies the trained model to testing time-series data.
  • Dimensionality Reduction: PCA projects high-dimensional CPS measurements into a lower-dimensional principal-component space before GAN processing.This projection addresses the higher computational cost of applying LSTM-RNNs to inputs such as the 51-variable SWaT data.
  • Dimensionality Reduction: The original dimension m denotes the number of system variables, while n denotes the number of retained principal components.
  • Anomaly Labels: The framework assigns labels indicating whether each testing variable at each time step is attacked.
  • Anomaly Decision: An anomaly is detected when the cross-entropy error in the anomaly score exceeds a threshold τ.

A. Water Treatment System

SWaT is a six-process operational water-treatment test bed used to study cyber-attacks and system responses. The dataset includes attacks on sensors and actuators and supports related detection research.

  • Water Treatment System: SWaT is an operational water-treatment test bed representing a small-scale modern plant for cyber-security research.Its physical process and control system were designed to closely resemble real systems in the field.
  • Water Treatment System: The water-purification process comprises six subprocesses, P1 through P6, spanning raw-water supply, pretreatment, filtration, dechlorination, and reverse osmosis.The supplied passage describes the first five processes; P6 is named as the final subprocess in the sequence.
  • Cyber-Attacks: A total of 36 cyber-attacks were launched during the 2016 SWaT data collection process.
  • Cyber-Attacks: The attacks targeted both sensors, including water-level and flow-rate devices, and actuators, including valves and pumps.
  • Related Work: Prior SWaT studies included physical-invariant detection, RNN prediction with CUSUM, and Kalman-filter modeling on single variables.These examples represent related work addressing selected attacks or system dynamics.

C. SWaT Dataset

The 2016 SWaT collection spans 11 days, with attacks inserted during the final four days. Its attack responses vary in duration, stabilization time, delay, and cross-sensor effects, motivating multivariate modeling.

  • SWaT Dataset: The 2016 SWaT data collection lasted 11 days, with the system operated continuously for 24 hours per day.
  • Attack Timing: Cyber-attacks were inserted during the final four days and lasted from a few minutes to an hour.Attacks could be launched consecutively or after the system returned to its normal operating state.
  • Attack Responses: Attack effects and system stabilization durations vary across scenarios, with stronger dynamic effects requiring more stabilization time.Some attacks do not take effect immediately, and simpler flow-rate changes stabilize more quickly.
  • Cross-Variable Effects: An attack on one sensor or actuator may affect other sensors or actuators after a time delay.
  • Cross-Variable Effects: Attacks on similar sensors can produce similar responses, such as abnormal spikes in both LIT101 and LIT301 after an LIT101 attack.The example does not affect the cited flow-rate sensor and power-meter readings, supporting multivariate rather than independent modeling.

V. EXPERIMENTS

Experiments evaluate GAN-AD’s sample generation and anomaly-detection setup on SWaT data, including multivariate generation, MMD convergence, and standard classification metrics.

  • Dataset and setup: The 2016 SWaT dataset contains 51 measured variables over 11 days, including normal-operation and cyber-attack data.Normal data comprise 496,800 samples from the first 7 days, while attack data comprise 449,919 samples.
  • Dataset and setup: The experiments use two-minute windows, with T=120 and different shift lengths for normal and attack data.The normal-data shift is SLnor=10, whereas the attack-data shift is SLatt=120.
  • GAN configuration: The generator uses a depth-3 LSTM with 100 hidden units, while the discriminator uses a depth-1 LSTM with 100 hidden units.The latent-space dimension is set to 15 after testing different dimensions.
  • Sample generation: After sufficient training iterations, GAN-generated samples become realistic and match the distributions of the CPS sensors and actuators.Early generated samples are clearly different from training data, whereas later samples closely reproduce the original distribution.
  • Sample generation: MMD evaluates whether GAN-generated data have learned the training-data distributions, with values plotted across training iterations for univariate and multivariate generation.For both cases, MMD tends to converge to small values after 20–30 iterations; multivariate samples converge faster and initially have lower MMD.
  • Anomaly detection evaluation: GAN-AD anomaly detection is evaluated using Accuracy, Precision, Recall, F1 score, and False Positive Rate.Accuracy is defined as (TP + TN) / (TP + TN + FP + FN), with TP, FP, TN, and FN corresponding to anomaly and normal classifications.

E. Anomaly Detection Results

Anomaly-detection performance is evaluated for univariate and multivariate time series using established comparison methods and selected SWaT variables.

  • Comparison design: The univariate evaluation compares GAN-AD with the CUSUM approach used in prior work.
  • Comparison design: The multivariate evaluation compares GAN-AD with PCA-based unsupervised detection using Squared Predicted Error residual distances.
  • Evaluation data: The reported discussion uses nine sensors and actuators from different SWaT processes, all serving as attack points for single-point attacks.
  • Evaluation data: To assess detection across multiple sequences, all measured variables are fed uniformly into GAN-AD, with the 51-dimensional data projected to a lower-dimensional space using PCA.PCA-based SPE values are also evaluated as a multivariate anomaly-detection baseline.

1) Univariate:

Univariate detection often performed poorly, while GAN-AD achieved strong results for selected sensors and actuators. In multivariate detection, GAN-AD improved substantially over SPE, with a trade-off between higher recall and false-positive rates when retaining more principal components.

  • 2) Multivariate:: Univariate detection generally had low precision and recall and high false positive rates compared with multivariate detection.The comparison supports multivariate detection for complex CPSs with interconnected sensors and actuators.
  • 1) Univariate:: Near or above 80% accuracy was achieved by both CUSUM and GAN-AD for water-level sensors LIT-101, LIT-301, and LIT-401.Their performance was attributed to attacks on other sensors or actuators affecting water levels indirectly.
  • 1) Univariate:: 86.03% accuracy for GAN-AD on AIT-504 exceeded its 60.22% accuracy on AIT-202.CUSUM showed the same ordering, with 70.97% on AIT-504 versus 55.67% on AIT-202.
  • 1) Univariate:: 80.72% −87.68% accuracy was achieved by GAN-AD for actuators P-101 and MV-303 despite their binary on/off states.The frequency of 0 and 1 states over time remained useful for anomaly detection.
  • 1) Univariate:: GAN-AD outperformed CUSUM on FIT-401, whose roughly binary but high-frequency flow pattern CUSUM could not capture.CUSUM recognized all testing samples as anomalies, producing 100% recall and false positive rate, whereas GAN-AD achieved acceptable accuracy.
  • 2) Multivariate:: 3% −12% improvement was obtained by GAN-AD over SPE in multivariate detection, with 50%-60% higher precision and 5%-40% higher recall.Using PC=5 increased recall relative to PC=1, while GAN-AD with PC=1 achieved the best false positive rate and PC=5 introduced more noisy information.

VI. CONCLUSIONS

The paper proposes GAN-AD, an unsupervised method that uses both generator and discriminator networks to detect anomalies in multivariate CPS data. Evaluation on SWaT showed that GAN-AD outperformed existing unsupervised detection methods, while future work targets broader applications and improved modeling of sensor relationships.

  • GAN-AD simultaneously models multivariate CPS data distributions under normal operating conditions and detects cyber-attack anomalies in an unsupervised fashion.The method directly uses both the discriminator and generator trained on multivariate time series.
  • GAN-AD outperformed existing unsupervised detection methods on a complex Secure Water Treatment Testbed dataset.The evaluation used data from the SWaT testbed.
  • Future work will extend GAN-AD to predictive maintenance and fault diagnosis and improve its representation of relationships among networked sensors and components.Planned methodological improvements include a multi-GAN framework, feature selection, and principled choices of latent and principal-component dimensions.
Loading 1809.04758v3…