Source-linked AI summary
A Survey on Anomaly Detection for Technical Systems using LSTM Networks
Benjamin Lindemann, Benjamin Maschler, Nada Sahlab, Michael Weyrich
TL;DR
Anomaly detection must handle complex, dynamic system behavior when anomaly causes are often unknown and conventional statistical methods are limited. This survey synthesizes LSTM-based approaches and emerging graph-based and transfer-learning methods, finding varied architectures capable of detecting complex anomalies while identifying challenges in graph data and benchmark availability.
Problem
Existing anomaly-detection surveys give limited attention to current neural-network and LSTM developments, despite anomalies affecting technical-system performance and stability.
Method
The paper surveys LSTM-based anomaly-detection approaches and emerging graph-based and transfer-learning methods, evaluating them by application, data, anomaly type, and other criteria.
Results
The surveyed LSTM architectures can precisely detect varying complex anomalies, while transfer learning can combine datasets across systems and scenarios for mutual benefit.
Takeaways & Limitations
The survey provides categorized state-of-the-art examples and analyses for readers entering anomaly detection and highlights graph-based and transfer-learning directions for practical systems.
Takeaways & Limitations
Graph-based methods face complex input-structure and context-profiling challenges, and the lack of public benchmark datasets hinders direct comparisons.
Abstract
from arXiv · showhide
Anomalies represent deviations from the intended system operation and can lead to decreased efficiency as well as partial or complete system failure. As the causes of anomalies are often unknown due to complex system dynamics, efficient anomaly detection is necessary. Conventional detection approaches rely on statistical and time-invariant methods that fail to address the complex and dynamic nature of anomalies. With advances in artificial intelligence and increasing importance for anomaly detection and prevention in various domains, artificial neural network approaches enable the detection of more complex anomaly types while considering temporal and contextual characteristics. In this article, a survey on state-of-the-art anomaly detection using deep neural and especially long short-term memory networks is conducted. The investigated approaches are evaluated based on the application scenario, data and anomaly types as well as further metrics. To highlight the potential of upcoming anomaly detection techniques, graph-based and transfer learning approaches are also included in the survey, enabling the analysis of heterogeneous data as well as compensating for its shortage and improving the handling of dynamic processes.
1. INTRODUCTION
The survey addresses a gap in existing anomaly-detection reviews by centering LSTM-based approaches and evaluating them across application-oriented criteria. It also considers emerging graph-based and transfer-learning methods.
- Existing surveys marginally consider neural-network approaches and do not centrally classify or analyze LSTM architectures for anomaly detection.
- The article surveys promising LSTM-based anomaly-detection approaches, with additional attention to graph-based and transfer-learning approaches.
- Approaches are evaluated using application-oriented criteria including temporal-anomaly detection, accuracy, and addressed use cases.
- The paper is organized around anomaly types and temporal context, LSTM architectures, emerging learning trends, survey results, and conclusions.
2. ANOMALY CLASSIFICATION
Anomalies are deviations from normal system behavior and may affect performance, stability, and output quality. The survey distinguishes point, collective, and contextual anomalies, emphasizing the temporal context required for contextual detection.
- Anomalies are irregularities or deviations from normal system behavior that can drive inefficiencies, instability, and system errors.
- Point anomalies are statistically defined outliers identified through probability densities and selected percentile thresholds.
- Collective anomalies arise when individually tolerable data vectors form an irregular group through their internal sequence structure.
- Contextual anomalies are individually or collectively tolerable observations that become irregular relative to surrounding data.
- For multivariate time series, contextual anomalies depend on short- and long-term surrounding structures and can be detected with sliding windows and dynamic thresholds.
- Contextual detection requires defining a shared context and identifying anomalous points within it using contextual and behavioral attributes.
3. ANOMALY DETECTION WITH LSTM NETWORKS
LSTM anomaly-detection systems use gated memory to model temporal dependencies, including short- and long-term behavior. The survey covers regular, encoder-decoder, and hybrid architectures for unsupervised and complex anomaly-detection tasks.
- Regular LSTM architectures: LSTM cells use gating and a cell state to control memory and learn long-term dependencies that conventional recurrent networks struggle to retain.
- Regular LSTM architectures: LSTM networks are suited to contextual anomalies because they learn temporal relations in stationary and non-stationary multivariate systems.
- Encoder-decoder-based approaches: Encoder-decoder architectures support unsupervised detection by learning compressed representations and reconstructing inputs without requiring labeled anomaly data.
- Encoder-decoder-based approaches: Combining LSTM and autoencoder components models short- and long-term temporal features for detecting complex time-variant anomalies.
- Encoder-decoder-based approaches: Seq2Seq approaches detect anomalies through unusual propagated cell states, encoder-decoder copying vectors, or modeled and predicted attributes.
- Hybrid approaches: Hybrid architectures combine LSTM prediction with a second network that detects deviations, dividing prediction and detection tasks across components.
- Hybrid approaches: CNN-LSTM combinations support multidimensional anomaly detection by correlating spatial, temporal, and application-specific dimensions.
4. RECENT TRENDS IN LEARNING-BASED ANOMALY DETECTION
Recent approaches extend learning-based anomaly detection with graph representations and transfer learning. Graphs add contextual structure for dynamic anomaly analysis, while transfer learning addresses limited training data and supports changing processes.
- Graph-based approaches: Graph-based approaches represent correlations and interdependencies between datapoints, supporting analysis of contextual and collective anomalies.Graphs can model heterogeneous relationships rather than treating datapoints independently.
- Graph-based approaches: Graph-based detection can be used before LSTM processing: identified anomalous clusters or extracted graph features can become LSTM inputs for modeling dynamic process behavior.The surveyed workflow may construct and version graphs over time, analyze anomalous clusters, and pass resulting features to LSTM networks.
- Graph-based approaches: Dynamic-graph anomaly detection must account for structural, content, and temporal features, including both short-term and long-term node patterns.Deep-learning methods such as graph embeddings and GCNs extract structural and content features, while recurrent units capture temporal information.
- Transfer learning approaches: Transfer learning targets the shortage of large, diverse training datasets by reusing knowledge from models trained on previous tasks.The survey distinguishes parameter transfer from relational knowledge transfer, with both intended to reduce target-task training-data requirements.
- Transfer learning approaches: A denoising autoencoder was pre-trained on a large source dataset and fine-tuned for anomaly detection across aluminum extrusion machines, but no algorithmic reference values enabled performance evaluation.This example illustrates parameter transfer without a directly comparable detection benchmark.
- Transfer learning approaches: Continual learning combines knowledge transfer across tasks with a stacked LSTM and enhanced loss function to retain prior capabilities during product changes.A comparison reported significant improvements for online elastic weight consolidation over other continual-learning approaches and a conventional baseline.
5. DISCUSSION
The survey organizes LSTM anomaly-detection approaches and finds distinct strengths across regular, encoder-decoder, hybrid, graph-based, and transfer-learning methods. It also identifies practical gaps, including missing implementations, difficult graph construction, limited benchmarks, and absent combined graph-transfer studies.
- The survey divides LSTM anomaly-detection approaches into regular, encoder-decoder, and hybrid architectures within five categories.
- Regular LSTM precisely detect collective and contextual anomalies, while encoder-decoder architectures improve detection in high-dimensional data spaces.Contractive and variational autoencoders support specific detection purposes.
- Graph-based approaches unify heterogeneous data sources, enabling analysis of anomaly causes and propagation within contextual frames.Physical-process examples can combine process, system, and environmental attributes in one graph.
- Graph-based methods remain constrained by complex graph construction, context-profiling, and scarce public benchmark datasets that impede direct comparisons.
- Transfer learning addresses insufficiently large and diverse datasets by sequentially transferring knowledge across tasks, states, systems, and scenarios.The survey reports growing industrial implementations and a promise of mutual benefit across systems and scenarios.
- No publicized results yet combine transfer learning with graph-based learning, despite the authors’ view that such combinations are needed for industrial challenges.Adaptive autonomous systems make distinguishing anomalies from newly learned normal behavior difficult.
6. CONCLUSION
The conclusion surveys LSTM-based anomaly detection for technical systems alongside graph-based and transfer-learning developments. It finds that multiple LSTM architectures can detect complex anomalies and recommends extending research toward interacting-system networks and combined approaches.
- The study investigates LSTM approaches for temporal anomaly detection in time-series data and surveys graph-based and transfer-learning advances for real-life applicability.
- Different LSTM architectures are capable of precisely detecting varying complex anomalies, including collective and contextual anomalies.The paper provides state-of-the-art examples and analyses for readers entering anomaly detection.
- Future research should address anomalies in networks of interacting systems and examine LSTM combinations with graph-based and transfer-learning techniques.The proposed directions target contextual-anomaly characterization and transfer of detected anomaly characteristics and knowledge.