Source-linked AI summary
Deep Learning for Anomaly Detection: A Survey
Raghavendra Chalapathy, Sanjay Chawla
TL;DR
Anomaly detection seeks to identify data instances that differ from others, including instances generated by previously unknown mechanisms. This survey organizes deep learning-based methods by assumptions and approaches, reviews their applications and effectiveness, and discusses strengths, limitations, computational complexity, and open challenges.
Problem
Anomaly detection must identify anomalous instances in diverse real-world data, where anomalies may indicate errors, malicious actions, failures, fraud, or previously unknown processes.
Method
The survey categorizes deep anomaly detection methods by their assumptions and approaches, reviews variants and applications, and assesses their strengths, weaknesses, effectiveness, and computational complexity.
Results
The survey finds that supervised methods are generally more accurate, semi-supervised GANs show promise with few labels, and unsupervised methods avoid annotation costs while learning data characteristics.
Takeaways & Limitations
The methods' assumptions about normal and anomalous data can guide model selection and effectiveness assessment for a particular dataset or application domain.
Takeaways & Limitations
Unsupervised methods often require priors about anomaly distributions, making them less robust to noisy data, while supervised and semi-supervised methods can have expensive training times.
Abstract
from arXiv · showhide
Anomaly detection is an important problem that has been well-studied within diverse research areas and application domains. The aim of this survey is two-fold, firstly we present a structured and comprehensive overview of research methods in deep learning-based anomaly detection. Furthermore, we review the adoption of these methods for anomaly across various application domains and assess their effectiveness. We have grouped state-of-the-art research techniques into different categories based on the underlying assumptions and approach adopted. Within each category we outline the basic anomaly detection technique, along with its variants and present key assumptions, to differentiate between normal and anomalous behavior. For each category, we present we also present the advantages and limitations and discuss the computational complexity of the techniques in real application domains. Finally, we outline open issues in research and challenges faced while adopting these techniques.
1 Introduction
Anomaly detection identifies data instances that differ from the rest, while deep learning learns hierarchical representations and can outperform traditional machine learning as data scale increases.
- Anomaly detection aims to identify instances that are dissimilar to the rest of a dataset.Such instances may result from errors or indicate a previously unknown generating process.
- Deep learning learns data representations as nested hierarchies of concepts across neural-network layers.It is presented as a flexible subset of machine learning with strong performance across application domains.
- Deep learning outperforms traditional machine learning as the scale of data increases.
2 What are anomalies?
Anomalies are observations or groups that depart from predominant data patterns, often revealing information relevant to decision-making. They may arise from malicious actions, system failures, or fraud.
- Anomalies are also called abnormalities, deviants, or outliers and lie away from regions containing most observations.In the illustrated dataset, N1 and N2 are normal regions, while O3, O1, and O2 are anomalous.
- Anomalies can arise from malicious actions, system failures, or intentional fraud.
- Anomaly detection is an essential step in decision-making systems because anomalies can reveal valuable information about data.
3 What are novelties?
Novelty detection identifies new or previously unobserved patterns and incorporates them into the regular data model rather than automatically treating them as anomalies.
- Novelty detection identifies novel or unobserved patterns in data.
- Detected novelties are applied to the regular data model instead of being treated as anomalous points.
- A novelty score and decision threshold can distinguish previously unseen points from anomalies or outliers.Points that significantly deviate from the threshold may be considered anomalous.
4 Motivation and Challenges: Deep anomaly detection (DAD) techniques
Deep anomaly detection is motivated by the difficulty traditional methods face with complex, large-scale data and manual feature design. The survey organizes related methods and compares them with prior domain-focused reviews.
- Traditional outlier-detection algorithms are sub-optimal on image and sequence datasets because they fail to capture complex data structures.
- Traditional methods become difficult to scale when datasets grow to gigabytes.
- Deep anomaly detection learns hierarchical discriminative features automatically, reducing the need to develop manual features.
5 Related Work
The survey addresses the shortage of broad comparative analyses of deep learning architectures for anomaly detection by organizing research methods and applications across domains.
- Existing deep anomaly detection surveys generally concentrate on a particular application domain or research area.
- The survey covers deep learning-based anomaly detection methods across multiple application domains.It positions this coverage as a broader review than domain-specific surveys.
- The review responds to a lack of comprehensive comparisons identifying architectures suited to particular datasets and application domains.
6 Our Contributions
The paper provides a structured survey of deep anomaly detection research, organizing state-of-the-art techniques while comparing their assumptions, practical properties, and computational demands.
- The survey presents a comprehensive outline of state-of-the-art deep anomaly detection research and real-world applications.
- It organizes techniques into a schema based on training objectives and introduces hybrid models and one-class neural networks as additional categories.
- For each category, the survey discusses adopted assumptions, techniques, challenges, advantages, disadvantages, and computational complexity.
7 Organization
The paper organizes anomaly detection by problem aspects, application domains, and technique categories, with categories determined by training objectives and label availability.
- Section 8 identifies aspects that determine anomaly detection problem formulations and introduces contextual and collective anomaly models.
- Section 9 describes application domains where deep learning-based anomaly detection has been applied.
- Table 2 links input-data types with corresponding deep anomaly detection architectures, including CNNs, LSTMs, and autoencoders.
- Techniques are categorized as supervised, unsupervised, hybrid, or one-class neural network methods based on training objectives and label availability.
- The survey discusses computational complexity during both training and testing for each technique category.
8 Different aspects of deep learning-based anomaly detection.
Deep anomaly detection is shaped by data characteristics, label availability, anomaly type, model objective, and output representation. The survey distinguishes supervised, semi-supervised, unsupervised, hybrid, and one-class approaches and covers point, contextual, and collective anomalies.
- Data characteristics: Deep anomaly detection architecture primarily depends on whether input data are sequential or non-sequential, and on their dimensionality.
- Label availability: Supervised methods train binary or multi-class classifiers using labels for both normal and anomalous instances.
- Label availability: Semi-supervised methods exploit labels for normal data because anomalous labels are harder to obtain.
- Label availability: Autoencoders trained on normal samples produce low reconstruction errors for normal instances and higher errors for unusual events.
- Label availability: Unsupervised methods detect outliers from intrinsic data properties and are used to automatically label unlabelled samples.
- Training objective: The survey introduces deep hybrid models and one-class neural networks as categories based on training objectives.
- Training objective: Deep hybrid models use neural networks, mainly autoencoders, as feature extractors before applying traditional detectors such as one-class SVM.Some variants jointly train the feature extractor with an OC-SVM or SVDD objective.
- Training objective: One-class neural networks learn representations driven by a one-class objective that creates a tight envelope around normal data.
9 Applications of Deep Anomaly Detection
The survey reviews deep anomaly detection across intrusion detection, fraud, medical and bio-informatics, and time-series and video applications. It organizes each domain around anomaly notions, data characteristics, detection challenges, and deep techniques.
- Intrusion detection: Intrusion detection covers host and network systems, with deep methods evaluated on system-call or network-traffic data.NIDS data has high volume, velocity, variety, and temporal structure.
- Medical and bio-informatics: Deep learning is applied in medical and bio-informatics anomaly detection to identify rare events in medical images and clinical EEG records.The stated applications include diagnosis and preventive treatment for medical conditions.
- Time series: For univariate time series, deep architectures learn hierarchical features that can improve outlier detection, while the survey presents benchmark tools and datasets.The survey also lists representative architectures in Table 16.
- Time series: Multivariate time-series anomaly detection remains challenging, but RNN- and LSTM-based methods are reported to detect interpretable anomalies and support fault-isolation diagnostics.The survey also describes generic and attention-based frameworks for multivariate data.
- Video surveillance: Video surveillance is modeled as anomaly detection because large amounts of unlabeled video are available, although real-life anomalies lack an explicit definition.That definitional issue is identified as a significant challenge for method performance.
10 Deep Anomaly Detection (DAD) Models
The survey categorizes deep anomaly detection models by label availability and training objective, comparing their assumptions, architectures, complexity, advantages, and limitations. It covers supervised, semi-supervised, hybrid, one-class, unsupervised, transfer-learning, zero-shot, and reinforcement-learning approaches.
- Supervised deep anomaly detection: Supervised DAD learns a separating boundary from annotated normal and anomalous instances, with fast testing and higher reported accuracy than semi-supervised and unsupervised methods.Its main constraints are the need for accurate labels and difficulty separating classes in highly complex, nonlinear feature spaces.
- Semi-supervised deep anomaly detection: Semi-supervised DAD learns a boundary around a single labeled class, and labeled data can substantially improve performance over unsupervised techniques.GANs trained semi-supervised have shown promise even with very few labeled examples, but learned hierarchical features may overfit rare anomalies.
- Deep hybrid models: Hybrid DAD models use deep networks, often autoencoders, to extract features before traditional detectors such as one-class SVM, combining the complexity of both components.Their assumptions emphasize robust hidden representations and a separate anomaly detector for complex, high-dimensional spaces.
- One-class neural networks: One-class neural networks jointly learn progressively richer representations and a one-class hyperplane or hypersphere to separate normal data from outliers.Reported experiments show comparable or better performance than existing methods on complex datasets, with reasonable training and testing time.
- Unsupervised deep anomaly detection: Unsupervised DAD learns inherent data characteristics without annotated training data, but performance is challenged by complex high-dimensional spaces, noise, corruption, and autoencoder compression choices.These methods are described as cost effective because they do not require annotated data, while often being less accurate than supervised or semi-supervised techniques.
- Emerging approaches: Transfer learning addresses insufficient training data by transferring knowledge from a source domain to a target domain, while zero-shot learning recognizes objects absent from the training set using descriptions or attributes.The survey identifies transferability and obtaining zero-shot metadata as important challenges, alongside the need for further investigation of deep reinforcement-learning anomaly detection.
11 Deep neural network architectures for locating anomalies
The survey links deep neural architectures to the structure of anomaly-detection inputs, including spatial, temporal, sequential, visual, and probabilistic data. It describes how architectures learn representations or identify anomalies through reconstruction and distribution modeling.
- Deep Belief Networks: Deep Belief Networks use stacked Restricted Boltzmann Machines as encoder-decoder models, but may produce high reconstruction error for anomalous samples while scaling to big data.The survey also associates DBNs with improved interpretability.
- Spatio Temporal Networks: Spatio Temporal Networks combine CNNs and LSTMs to extract spatial and temporal features from data with both structural and sequential relationships.CNNs learn spatial aspects, while LSTMs model correlations between nearby time points.
- Sum-Product Networks: Sum-Product Networks represent variables as leaves with sum and product internal nodes, enabling fast exact probabilistic inference across many layers.They are presented as more traceable than graphical models over high-treewidth structures without approximate inference.
- Word embeddings: Word2vec models produce embeddings that capture sequential relationships, and these embeddings are reported to improve performance in several deep-learning architectures.The survey connects them to sentence and time-sequence data.
- Generative models: Generative models learn data distributions to generate varied new points, with VAEs, GANs, and adversarial autoencoders highlighted as common approaches.Adversarial autoencoders impose an arbitrary prior on the latent code while learning the input distribution.
- CNNs and recurrent architectures: CNNs extract complex hidden features from high-dimensional structured data, whereas RNNs, LSTMs, and GRUs target temporal features and preserve information across time steps.The survey notes that LSTMs were introduced to address RNN difficulty capturing context as sequences lengthen.
- Autoencoders: Autoencoders reconstruct inputs through hidden representations; when trained mainly on normal data, anomalous samples tend to yield larger residual errors and are flagged as outliers.Architecture choice depends on the data, with convolutional networks preferred for image datasets and recurrent models for sequential data.
12 Relative Strengths and Weakness : Deep Anomaly Detection Methods
The survey compares deep anomaly detection categories by their strengths, weaknesses, assumptions, and practical computational costs. It emphasizes choosing methods according to the anomaly-detection context and dataset.
- Technique suitability depends on the specific anomaly-detection problem context because deep anomaly detection methods have distinct strengths and weaknesses.The survey analyzes relative strengths and weaknesses across technique categories rather than identifying one universally best method.
- Figure 13 presents autoencoder architectures including AE, LSTM, SDAE, DAE, GRU, CNN, CNN-LSTM-AE, and CAE for anomaly detection.
- Supervised and semi-supervised methods require expensive training, although testing is usually fast with a pretrained model.
- Unsupervised methods are widely used because acquiring labels is costly and time-consuming, but many assume priors about anomaly distributions and are less robust to noisy data.
- Hybrid models combine deep feature extraction with classical anomaly detectors but cannot influence hidden-layer representation learning through the final detection objective.
13 Conclusion
The survey organizes deep learning methods for anomaly detection across application domains, explains their assumptions and trade-offs, and discusses how to assess suitability for particular datasets. It concludes that the area remains active and should be updated as more sophisticated techniques emerge.
- The survey reviews deep learning-based anomaly detection methods and their applications across various domains.
- For each technique category, it describes assumptions about normal and anomalous data alongside strengths and weaknesses.
- These assumptions can guide assessment of a deep learning model’s effectiveness for a particular domain or dataset.
- Deep learning-based anomaly detection remains an active research area, motivating future updates as more sophisticated techniques are proposed.