PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning-Based Anomaly Detection in Cyber-Physical Systems: Progress and Opportunities

Yuan Luo, Ya Xiao, Long Cheng, Guojun Peng, Danfeng Daphne Yao

arXiv:2003.13213v2cs.CReess.SP

TL;DR

CPS anomaly detection must address complex systems, sophisticated attacks, and the limitations of conventional methods. This paper systematically surveys DLAD methods through a taxonomy, domain analysis, experiments, and discussion of limitations. It synthesizes current progress while identifying deficiencies and directions for improving future CPS anomaly detection.

  • Problem

    CPS complexity and sophisticated attacks challenge conventional anomaly detection, which faces growing data volumes, domain-specific knowledge requirements, or large labeled-data requirements.

  • Method

    The paper proposes a taxonomy, reviews DLAD methods across CPS domains, experimentally examines neural models and workflows, and discusses limitations and research directions.

  • Results

    The survey systematizes state-of-the-art DLAD methods, characterizes prevailing approaches and domain-specific designs, and reports their limitations and open problems.

  • Takeaways & Limitations

    The taxonomy, experiments, and findings provide guidance for evaluating and improving DLAD methods and motivate future CPS anomaly-detection research.

  • Takeaways & Limitations

    DLAD methods may lack interpretability and may fail to detect long-duration collective anomalies.

Abstract

from arXiv · show

Anomaly detection is crucial to ensure the security of cyber-physical systems (CPS). However, due to the increasing complexity of CPSs and more sophisticated attacks, conventional anomaly detection methods, which face the growing volume of data and need domain-specific knowledge, cannot be directly applied to address these challenges. To this end, deep learning-based anomaly detection (DLAD) methods have been proposed. In this paper, we review state-of-the-art DLAD methods in CPSs. We propose a taxonomy in terms of the type of anomalies, strategies, implementation, and evaluation metrics to understand the essential properties of current methods. Further, we utilize this taxonomy to identify and highlight new characteristics and designs in each CPS domain. Also, we discuss the limitations and open problems of these methods. Moreover, to give users insights into choosing proper DLAD methods in practice, we experimentally explore the characteristics of typical neural models, the workflow of DLAD methods, and the running performance of DL models. Finally, we discuss the deficiencies of DL approaches, our findings, and possible directions to improve DLAD methods and motivate future research.

1 INTRODUCTION

CPSs face increasingly sophisticated threats and conventional anomaly detection methods struggle with data volume and domain-specific knowledge requirements. The paper surveys DLAD methods, organizes them systematically, and studies their limitations and future directions.

  • CPS complexity and sophisticated attacks can produce catastrophic consequences, making CPS security essential.
  • Conventional anomaly detection methods often require expert knowledge or assumptions about normal-data distributions, while machine learning typically requires abundant labeled data.
  • DLAD methods have been explored across neural architectures, threats, and CPS domains, but prior studies lack a unified presentation.
  • The paper proposes a taxonomy covering anomaly types, detection strategies, implementation, and evaluation metrics, and categorizes peer-reviewed CPS research accordingly.
  • The study experimentally examines neural-model characteristics, DLAD workflow, and model running performance, then discusses deficiencies and research directions.

2 BACKGROUND

The paper defines CPS components, threats, and the typical DLAD workflow, while positioning its survey as distinct from earlier work. DLAD models process CPS data into anomaly scores for online decisions.

  • A CPS connects physical components, sensors, actuators, control systems, SCADA, and level 0 and level 1 communications.
  • CPS threats include compromised sensor or actuator values, manipulated communication traffic, and compromised control systems.
  • Existing DLAD studies commonly target sensors, actuators, communications, and control systems rather than direct physical-space measurements.
  • Typical DLAD workflows collect and preprocess CPS data, train and test neural models, compute anomaly scores, and apply trained models online.Common inputs include sensor and actuator data, network traffic, and control-system logs; anomaly scores include prediction errors, reconstruction errors, and predicted labels.
  • The survey distinguishes itself from earlier surveys by focusing on deep learning-based anomaly detection in CPS.

3 TAXONOMY

The taxonomy classifies CPS DLAD methods by anomaly type, detection strategy, and implementation or evaluation. It covers attacks and faults, data and model choices, deployment settings, and evaluation practices.

  • The taxonomy has three aspects: anomaly type, detection strategy, and implementation and evaluation metrics.
  • 3.1 Type of anomalies: Anomalies are broadly categorized as attacks or faults affecting CPS components, communications, and control systems.
  • 3.1 Type of anomalies: Attack targets include network communication, sensor and actuator data, and control systems, with examples including DoS, packet injection, malware, and manipulated signals.
  • 3.1 Type of anomalies: Faults commonly arise in sensor and control-system layers because CPS devices are complex, heterogeneous, and exposed to harsh conditions.
  • 3.2 Detection strategies: DLAD detection strategies select input data, neural-network designs, and anomaly scores according to the targeted anomaly.Input categories include sensor and actuator data, network traffic, system calls and logs, and preprocessed time-series data; scores include prediction and reconstruction errors.
  • 3.3 Implementation and evaluation metrics: Implementation and evaluation cover real-world systems, testbeds, and manually created or simulated anomalies, with precision, recall, and F1 score commonly used for evaluation.Testbeds are scaled-down but functional environments, and anomalous data may be injected as point, statistical, or simulated attacks.

DoS

The listed work summarizes research on anomaly detection in industrial control systems, but the supplied passage does not explain the DoS-specific methods or findings.

  • The table lists multiple studies under industrial control systems, but the supplied entries do not provide interpretable DoS-specific results.

4 REVIEW OF DEEP LEARNING-BASED ANOMALY DETECTION METHODS

The review organizes CPS anomaly-detection research across four major system domains and general time-series methods using a common taxonomy.

  • Current research focuses mainly on industrial control systems, smart grids, intelligent transportation systems, and aerial systems.
  • The review also examines general-purpose methods for analyzing time-series data.
  • Existing work is summarized under a taxonomy whose metrics are listed as columns in Table 2.

4.1 DLAD methods in ICSs

ICS DLAD methods combine neural detectors with preprocessing, filtering, feature representations, temporal-spatial modeling, GANs, and ensembles. The review highlights efficiency gains alongside unresolved issues involving feature selection, false positives, input design, generalization, and computational cost.

  • Filtering before DLAD: A lightweight conventional filter can precede DLAD to reduce the data requiring analysis in resource-constrained ICS environments.The reviewed framework first filters data, then applies an LSTM detector to packet signatures.
  • Filtering before DLAD: The LSTM detector predicts each next packet signature and checks it against a database of normal signatures.
  • Feature representation: Deep feature representations use raw data, processed data, or learned embeddings to integrate features and reduce feature-space dimensionality.The review notes that rules for selecting feature counts and learning parameters remain unclear, and some features may miss essential data characteristics.
  • Temporal and spatial relationships: Models such as LSTM and CNN address temporal behavior and sensor correlations, but multi-dimensional ICS data still requires temporal, spatial, and logical context modeling.CNN inputs remain manually designed, and short time series may require padding to match the detection window.
  • GAN-based methods: GAN-based detection combines reconstruction and discrimination errors to detect false control signals from sensor and actuator values.The reviewed framework uses LSTM-based generator and discriminator models and outperforms listed baselines in precision, recall, and F1.
  • Ensemble learning: Ensembling statistical and deep-learning models can improve reliability, but seasonal assumptions and preprocessing complexity limit MBPF's applicability.MBPF combines TBATS with a multi-branch deep network after seasonality evaluation and outlier elimination.

4.2 DLAD methods in smart grid

Smart-grid DLAD research focuses mainly on stealthy false data injection attacks, using autoencoder, RNN, filtering, joint detection, and prediction-based strategies. Key limitations include computational demands, reliance on synthetic data, limited feature interpretability, and changing market context.

  • Threats and characteristics: False data injection attacks dominate smart-grid DLAD research because they stealthily manipulate measurements to compromise state estimation.Few studies address faults, control systems, or system logs and traces.
  • Detection strategies: Autoencoders and RNN variations are nearly equally adopted, with reconstruction and prediction errors serving as anomaly scores.Simulations, especially IEEE X-bus power systems, are commonly used for evaluation.
  • Representative techniques: Deep-learning filters can remove false data before bad-data detectors, while joint autoencoder-detector frameworks flag attacks only when both components agree.One joint framework uses 108 PMU-derived features and a reconstruction-error threshold.
  • Representative techniques: Prediction-based DBN models estimate electric-load intervals as normal state-variable ranges, enabling localization of attacks that create abnormal states.The model uses generator and market time-series information as input.
  • Limitations and open problems: Smart-grid DLAD must reduce massive PMU-data processing, validate performance beyond synthetic data, integrate interpretable multisource features, and adapt to changing market information.Market-aware models require frequent updates to capture altered economic conditions.

4.3 DLAD methods in ITSs

ITS DLAD methods increasingly incorporate environmental context and computational adaptations alongside vehicle and CAN-bus data. Context improves recall, but false positives, limited ECU correlation, and attack–fault ambiguity constrain practical deployment.

  • Representative techniques: Contextual information from the surrounding environment complements vehicle sensor data in ITS anomaly detection.Relevant sources include cameras, radars, and speed sensors.
  • Representative techniques: Adding contextual information improves recall but leaves precision unchanged, indicating that more anomalies are identified without reducing normal-data misclassification.The advanced model outperforms the basic model specifically on recall.
  • Representative techniques: Mobile edge devices are proposed to support low-latency processing of CAN-bus traffic and vehicle control commands.Short message delays can cause severe consequences when users respond.
  • Representative techniques: The CNN-KF model achieves an F1 score about 2% higher than CNN, favoring CNN-KF when computing resources are sufficient.CNN remains an option when vehicle resources are limited, while the method does not distinguish attacks from faults.
  • Limitations: Basic smart-car DL models report false-positive rates of 2% to 10%, and current component-level detection does not exploit correlations among ECUs.These limitations hinder practical deployment and motivate cross-ECU modeling.

4.4 DLAD methods in aerial systems

Aerial-system DLAD addresses faults and attacks using sensor, communication, kernel-event, and system-log data, with prediction or reconstruction errors thresholded for detection. Open issues include delayed and noisy thresholding, sampling losses, inadequate collective-anomaly coverage, and limited realism or deployment evaluation.

  • Threats and characteristics: Aerial DLAD covers point and contextual faults in aircraft and spacecraft, plus malicious code, eavesdropping, and spoofing in UAVs and aircraft.Some UAV studies augment conventional sensor and network inputs with kernel events and logs.
  • Detection strategies: Prediction and reconstruction errors require thresholds, which are often selected empirically by experts.Automatic thresholding aims to remove this expert-knowledge requirement.
  • Representative techniques: An automatic dynamic-threshold method smooths prediction errors with an exponentially weighted moving average before calculating the threshold.The approach is designed to automate threshold generation without expert knowledge.
  • Limitations: Large time windows can delay detection, while false positives remain high with telemetry-scale data despite automatic threshold tuning.Delayed detection may allow attacks or faults to cause catastrophic damage before alarms are produced.
  • Limitations: Archive sampling reduces data volume while preserving characteristics, but can lose early rows and miss collective anomalies.Different telemetry-channel sampling rates contribute to potential data loss.
  • Limitations and open problems: Aerial DLAD evaluations remain limited by desktop-only runtime measurement, stochastic-noise sensitivity, manually simulated simple anomalies, and unstudied sophisticated attacks.Reported recall can exceed precision by about 5%-20%, and replay attacks are among the missing scenarios.
  • Limitations and open problems: Across DLAD methods, recurring gaps include unclear input design, unexplained false positives, synthetic anomalies, inadequate feature methodology, and missing computational evaluation.These five categories summarize the section’s major limitations.

5 EXPLORATION OF DEEP LEARNING-BASED ANOMALY DETECTION MODELS

The experiments illustrate a typical DLAD workflow using LSTM models for temporal dependencies and CNN-based autoencoders for sensor correlations. Prediction and reconstruction errors, respectively, serve as anomaly scores, while model depth affects accuracy, training time, and size.

  • The experiments demonstrate the workflow of identifying anomalies, designing inputs and neural models, selecting anomaly scores, and evaluating performance.The experiments are intended to show model usage, the typical DLAD process, and running performance rather than introduce sophisticated methods.
  • 5.1 LSTM-based models: LSTM-based models predict future sensor values from past time windows to capture temporal dependencies in CPS time-series data.The implementation uses a 60-value past window, step size 1, and a single future-value prediction.
  • 5.1 LSTM-based models: Prediction errors between actual and predicted values are used as anomaly scores, with threshold exceedance indicating an anomaly.Models are trained on normal data by minimizing MSE; attack periods produce larger prediction errors than normal periods.
  • 5.1 LSTM-based models: Adding LSTM layers slightly reduces losses, but the 2-layer model performs best on unseen data and deeper models require longer training.The results indicate that simply increasing depth does not necessarily improve performance, and basic-model performance is not stable.
  • 5.2 CNN-based Autoencoders: CNN-based autoencoders model correlations among multivariate sensors by reconstructing a sensor-correlation matrix.The matrix is formed from sensor time windows and passed through convolutional, pooling, and upsampling layers.
  • 5.2 CNN-based Autoencoders: Reconstruction errors identify anomalies and reveal differing relationships among sensors, while additional CNN layers generally reduce validation errors.False data injection produces larger reconstruction errors than normal periods; the experiments also report increasing model size with more layers and faster CNN training than LSTM training.

6 LIMITATIONS OF DEEP LEARNING-BASED ANOMALY DETECTION METHODS

The paper identifies practical limitations of DLAD methods involving interpretability, computational and maintenance costs, data quality, long-duration anomalies, and attacks against the models themselves. These constraints are especially relevant when CPSs require real-time responses and operate with changing environments or limited resources.

  • DLAD results can lack interpretability because users may not understand the model’s decision process or what a reported anomaly represents.The paper defines interpretability as a detection-result cause that users can understand.
  • DLAD deployment can be computationally expensive because CPS devices are resource-constrained, models grow in scale, and updates require recurring computation.Costs also include hyperparameter tuning and maintaining models as sensor and traffic data continue to be generated.
  • High-quality training data are difficult to obtain because CPS environments change, labels are often absent, and anomalous cases are commonly synthetic.Synthetic anomalies may not occur in the real world or represent real threat characteristics, limiting deployment relevance and cross-method comparison.
  • DLAD methods may fail to detect collective anomalies that persist for long periods after detecting sudden changes at the beginning of an attack.In the described long-duration false data injection attack, period A contains sudden changes, whereas period B contains anomalies with no or small changes.
  • DLAD models are vulnerable to poisoning attacks that inject false training samples and can shift the learned decision boundary away from the true CPS characteristics.The paper also identifies adversarial examples as a separate threat to DLAD models.
  • Running-performance evaluation is largely absent from prior studies, despite CPS applications requiring immediate responses and sometimes having limited computing power.Startup time and prediction time are identified as relevant measurements.

7 TAKEAWAYS & CONCLUSION

The survey identifies recurring DLAD designs and limitations across CPS domains, then proposes research directions for more practical anomaly detection. Its experiments and synthesis aim to help address deployment issues and prioritize future work.

  • Our findings: Most studies lack explicit threat models detailing the attack or fault types they target.The paper notes that prevalent anomalies differ across CPS domains, with false data injection attacks especially common in smart-grid studies.
  • Our findings: Sensor time-series data are the dominant source, while network traffic is the second most-used source.Sensors are widespread, reflect CPS status, and can accumulate in quantities suitable for deep learning.
  • Our findings: RNNs, especially LSTMs, and autoencoders are the most common DLAD architectures, with hybrid models capturing temporal and contextual relations.RNNs model temporal dependencies, while autoencoders support unsupervised learning when labeled data are scarce.
  • Our findings: DLAD methods commonly use reconstruction or prediction errors, while implementation strategies vary by CPS domain.ICS studies often use functional testbeds such as SWaT, smart-grid studies frequently use IEEE X-bus simulations, and ITS studies apply real-world datasets.
  • Our findings: Current evaluation lacks specialized metrics for DLAD, and training time, updating frequency, computing overhead, and storage overhead remain insufficiently assessed.The F1 score balances false positives and false negatives, but does not capture these operational costs.
  • Improving deep anomaly detection methods: Future work should develop realistic benchmarks, improve real-time execution, locate anomalous devices or root causes, and adapt models to CPS-specific data and threats.The paper specifically calls for labeled real-world anomalous data, broader implementation evaluation, finer-grained diagnosis, and compatible architectures for different CPSs.
Loading 2003.13213v2…