A Survey on Explainable Anomaly Detection

Zhong Li; Yuxuan Zhu; Matthijs van Leeuwen

Source-linked AI summary

A Survey on Explainable Anomaly Detection

Zhong Li, Yuxuan Zhu, Matthijs van Leeuwen

arXiv:2210.06959v2Oct 13, 2022cs.LG

TL;DR

Anomaly detection research has largely prioritized accuracy over explainability, even as explanations became an ethical and regulatory requirement for high-stakes applications. This paper surveys explainable anomaly detection techniques, proposes a six-criterion taxonomy, and reviews more than 150 papers.

Problem
Anomaly detection research has mainly focused on detection accuracy while largely overlooking explainability, although explaining high-stakes AI decisions has become an ethical and regulatory requirement.
Method
The paper conducts a structured survey of explainable anomaly detection techniques and proposes a taxonomy based on six main criteria for categorizing them.
Results
The survey reviewed more than 150 papers and categorized explainable anomaly detection techniques using the proposed taxonomy.
Takeaways & Limitations
The taxonomy supports categorizing explainable anomaly detection techniques according to the aspects that characterize them.
Takeaways & Limitations
The survey identifies research challenges as opportunities for future work, while reviewed subspace-based methods commonly incur high computational costs.

Abstract

from arXiv · show

In the past two decades, most research on anomaly detection has focused on improving the accuracy of the detection, while largely ignoring the explainability of the corresponding methods and thus leaving the explanation of outcomes to practitioners. As anomaly detection algorithms are increasingly used in safety-critical domains, providing explanations for the high-stakes decisions made in those domains has become an ethical and regulatory requirement. Therefore, this work provides a comprehensive and structured survey on state-of-the-art explainable anomaly detection techniques. We propose a taxonomy based on the main aspects that characterize each explainable anomaly detection technique, aiming to help practitioners and researchers find the explainable anomaly detection method that best suits their needs.

1 INTRODUCTION

Anomaly detection research has largely prioritized accuracy, while explainability has become increasingly important for safety-critical applications and remains insufficiently covered by existing surveys. This paper addresses the gap through a comprehensive survey, refined taxonomy, and structured review of explainable anomaly detection techniques.

Background: An anomaly is an object notably different from the majority of remaining objects, with related terms including outlier and novelty.The paper uses “anomaly” consistently, although research often treats “outlier” as a synonym.
Motivation: Existing surveys of explainable anomaly detection are few and may be outdated, incomplete, or based on coarse taxonomies.The authors report finding only a handful of such surveys and identify limitations in their coverage and categorization.
Motivation: Explainability matters because anomaly detection is increasingly used in safety-critical domains, where explanations for AI decisions have become an ethical and regulatory requirement.The motivation connects model adoption concerns with the need to explain high-stakes outcomes to stakeholders.
Contributions: The paper conducts a comprehensive and structured survey of state-of-the-art explainable anomaly detection techniques and proposes a refined taxonomy.The taxonomy is intended to characterize the increasingly rich set of techniques in the literature.
Research questions and methodology: The survey organizes the literature around questions concerning XAD definitions, technique characteristics, anomaly interpretations, differences, challenges, and opportunities.It uses a comparative and iterative procedure with three cycles, including database selection, literature screening, and reference-based inspection.
Organization and aim: The paper surveys existing techniques according to the proposed taxonomy, then discusses open challenges and related opportunities.Its stated aim is to help practitioners and researchers identify methods suited to their needs while presenting methods’ pros and cons.

2 THE NEED FOR EXPLAINABLE ANOMALY DETECTION

Explainable anomaly detection matters because anomaly analysis involves distinct roles and definitions, and explanations may not faithfully reflect what detectors capture. The survey frames XAD as extracting relevant knowledge about anomalous decisions while considering human understanding and practical properties.

Roles and definitions: Anomaly analysis involves stakeholders, an anomaly detection method, and an anomaly explanation method, whose definitions of anomalies may differ.The paper distinguishes the Oracle-, Detection-, and Explanation-Definitions.
Roles and definitions: The Oracle-Definition is application-specific and context-dependent, while detectors operationalize anomalies through criteria such as distance, local density, or isolation.Examples include far k-nearest neighbours, low local density, and easily isolated objects.
Why explanations matter: A mismatch between the Oracle-Definition and Detection-Definition may cause detectors to miss relevant anomalies or flag uninteresting ones.Explanation methods can also diverge from detector decisions when they do not reflect the underlying decision-making process.
What XAD explains: XAD extracts relevant knowledge from anomaly detection models about data or learned relationships to provide insight into why specific objects are detected as anomalies.The survey treats XAD as encompassing both model-based and explanation-focused cases.
Scope and approaches: The survey considers intrinsically interpretable detectors and post-hoc techniques for non-interpretable models, while noting that human-centered questions are outside its scope.It includes both cases but leaves questions about human understanding and interaction to future work.

3 A TAXONOMY OF EXPLAINABLE ANOMALY DETECTION METHODS

The survey reviews prior anomaly-explanation categorizations and proposes a structured six-criterion taxonomy for characterizing explainable anomaly detection (XAD) methods. The criteria cover pipeline timing, explanation scope, model dependence, explanation perspective, explanation-generation techniques, and applicable data types.

3.1 Related Work: Prior categorizations include feature-selection versus score-and-search approaches, three anomaly-explanation categories, and four feature- or comparison-oriented groups.The survey describes these existing categorizations as insufficient for characterizing the increasingly rich XAD field.
3.2 Proposed Taxonomy: The proposed taxonomy uses six criteria to characterize each XAD method and help users identify methods satisfying particular requirements.The six dimensions are presented together in the survey’s overall taxonomy.
3.2 Proposed Taxonomy: The first criteria distinguish pre-model, in-model, and post-model techniques; global versus local explanations; and model-agnostic versus model-specific approaches.Global explanations concern complete model logic, whereas local explanations address a specific anomalous object or decision.
3.2 Proposed Taxonomy: Explanation perspectives include feature-based, sample-based, and combined feature-and-sample methods, with feature-based methods also covering patterns in sequential data.Sample-based methods compare anomalies directly with normal or contextual objects, while feature-based methods identify important features or values.
3.2 Proposed Taxonomy: The taxonomy also records explanation-generation techniques and whether methods apply to static or streaming tabular, sequential, image, text, video, or graph data.The six criteria are intended to provide a detailed characterization of existing methods.

4 LITERATURE REVIEW ON PRE-MODEL TECHNIQUES

Pre-model XAD techniques improve interpretability before anomaly detection through feature selection or human-understandable feature representations. The review covers their benefits, limitations, representative methods, and applicability across anomaly-detection models and data types.

4.1 Feature Selection For Anomaly Detection: Pre-model techniques operate before anomaly detection and include feature selection and feature representation methods.Feature selection retains a subset of original features, while representation methods learn high-level human-understandable features.
4.1 Feature Selection For Anomaly Detection: Feature selection retains important original features, improves interpretability, and can alleviate the curse of dimensionality in high-dimensional data.Unlike feature projection, it does not replace original features with combinations that are no longer interpretable.
4.1 Feature Selection For Anomaly Detection: Some categorical-data methods assume anomalies violate normal feature dependencies or that rare instances are strongly similar, assumptions that may not hold broadly.The survey notes that anomalies are often isolated and distinct, and that the dependency-based definition is not applicable to many benchmark detectors.
4.1 Feature Selection For Anomaly Detection: Filter-based feature selection is independent of subsequent detectors, so it may select suboptimal or irrelevant features for anomaly detection.This limitation follows because these methods select feature subsets regardless of the later anomaly-detection method.
4.1 Feature Selection For Anomaly Detection: PCA and ICA can reduce feature dimensionality and promote interpretability in subsequent detection, but their components combine original features and lose interpretability.The reviewed PIT-system approach combines dimensionality reduction with later anomaly detection.
4.2 Feature Representation For Anomaly Detection: High-level human-understandable representations can reduce the complexity of subsequent anomaly-detection models and improve their interpretability.Examples include symbolic time-series representations and constrained neural representations of characteristic normal shapes.
4.3 Summary: Most pre-model XAD techniques are model-agnostic, whereas the reference-based gait framework is the exception identified in Table 1.Pre-model techniques can generally be applied to subsequent anomaly-detection methods, although complete decoupling is not always possible.

5 LITERATURE REVIEW ON IN-MODEL TECHNIQUES

In-model techniques provide explanations through their intrinsic structures, including transparent models, feature subsets, and other comprehensible anomaly-scoring mechanisms. The survey organizes these methods into three high-level groups and highlights that transparency alone does not guarantee explainability when models become overly complex.

Overview: Inherently explainable models expose learned relationships or comprehensible anomaly-score calculations to help end-users understand decisions.The survey includes transparent supervised models, feature-subset methods, and miscellaneous unsupervised methods.
Limitations: Transparency is insufficient for explainability when a model becomes exceedingly complex and no longer remains human-understandable.The survey therefore treats model complexity as a boundary on the explanatory value of transparent structures.
Transparent Models: Rule-based and decision-tree methods explain anomalies through frequent patterns, violated constraints, branching conditions, or categorization rules.Examples include transaction patterns, process-trace rules, time-series decision trees, and tree-based autoencoders.
Transparent Models: Regression and other interpretable probabilistic models explain anomalies through prediction deviations, feature differences, or model outputs.LOESS can generate natural-language explanations from neighboring instances and associated feature differences.
Feature Subset Based Models: Feature-subset and pattern-based methods identify original features, patterns, or connections responsible for anomalies.Sparse PCA links transformed features to a few original features, while extracted patterns can serve as human-comprehensible explanations.
Taxonomy: The survey groups in-model techniques into transparent supervised models, feature-subset methods, and miscellaneous unsupervised models.The three groups organize the diverse in-model explainable anomaly detection literature.

6 LITERATURE REVIEW ON SHALLOW POST-MODEL TECHNIQUES

Shallow post-model techniques explain anomalies after detection or by inspecting model inputs and outputs, without interfering with the detection process. The survey covers subspace, surrogate, contextual, pattern, and visualization-based explanations, while noting limits in explanation scope and coverage.

Overview: Shallow post-model methods inspect a completed detection process or an opaque model and operate without interfering with anomaly detection.They may correlate model inputs with outputs or use an additional explainer.
Subspace based methods: Subspace-based methods find feature subspaces where an anomaly differs most, often returning a low-dimensional or well-separating subset as explanation.Methods use anomaly scores, separability, feature selection, or density-based measures to identify explanatory subspaces.
Other techniques: Surrogate and visualization methods use alternative models, feature importance, pattern comparison, or visual displays to make anomaly decisions more interpretable.Surrogates may be transparent rules or trees, while SHAP and visualization support shallow anomaly explanation.
Contextual explanations: Contextual methods explain anomalies relative to global or local reference groups defined by feature subsets, values, or clustered instances.They can report the most abnormal subspace, context-defining conditions, and additional properties distinguishing the anomaly.
Subspace based methods: Dimensionality-unbiased scoring and beam search support outlying-aspect explanations, while SiNNE addresses bias in anomaly scoring.The surveyed work includes Z-score, isolation path score, and minimal subspace explanations.
Summary: Shallow post-model methods generally provide local, feature-based explanations, and the survey identifies a lack of sample-based explanation methods.The survey also reports that explanations can be more reliable when models generate feature importance.

7 LITERATURE REVIEW ON DEEP POST-MODEL TECHNIQUES

Deep post-model techniques address the poor interpretability of accurate neural anomaly detectors using reconstruction errors, model-specific internal analyses, or post-hoc explainers. The survey finds that these methods are often faithful but usually model-specific, local, feature-based, and focused on sequential data.

Overview: Deep anomaly detectors often achieve high accuracy on complex data but are criticized for poor interpretability, motivating post-hoc XAI techniques.The survey discusses autoencoders, LSTMs, CNNs, GANs, and other neural networks.
AutoEncoders: Autoencoder-based methods explain anomalies through reconstruction errors, feature contributions, latent representations, or gradient-based analyses.Examples include SHAP, kernel SHAP, sparse optimization, gradient fingerprinting, and random-permutation feature importance.
Post-hoc explainers: Shapley-value and related explainers can identify contributing features, anomalous windows, latent-space associations, and pixel-wise outlyingness.The reviewed methods apply SHAP variants, GradientExplainer, Integrated Gradients, and SmoothGrad across several neural architectures and data types.
Limitations: Directly estimating feature contributions from deviating latent distributions or reconstruction errors can lead to high false positives and/or negatives.One reviewed approach instead estimates contributing features through an approximative probabilistic model of the true latent distribution.
Summary: Most deep post-model techniques are model-specific, whereas shallow post-model techniques are typically model-agnostic.Internal neural-network analyses limit generalization, although the resulting explanations are usually more faithful to the detector.
Summary: Model-agnostic explainers such as SHAP, LIME, and some rule learners are more likely to suffer from poor fidelity than internal analyses.The survey contrasts broader applicability with faithfulness to the original anomaly detector.
Summary: Most deep post-model techniques target sequential data such as time series and system logs, and nearly all provide only feature-based explanations.Only a small set of cited methods also produces sample-based explanations.

8 CONCLUSION AND FUTURE OPPORTUNITIES

The survey reviews more than 150 explainable anomaly detection papers and organizes the field through a taxonomy of six criteria, four high-level categories, and twelve fine-grained categories. It identifies open challenges involving definitions, evaluation, knowledge integration, adversarial robustness, and scalability.

Conclusion: More than 150 papers were reviewed to characterize existing explainable anomaly detection techniques.
Conclusion: The proposed taxonomy uses six criteria and organizes the review into four high-level and twelve fine-grained categories.The categories correspond to sections S4–7 and their subsections.
Research challenges: Non-uniform anomaly and XAD definitions hamper communication, related-work discovery, method comparison, and objective evaluation.The paper links this lack of uniformity to communication barriers across fields and the re-invention of methods.
Research challenges: XAD evaluation remains unresolved because explainability methods developed for classification or clustering do not transfer straightforwardly to anomaly detection, and post-hoc fidelity can be undermined by definition inconsistencies.Inconsistency among Oracle-, Detection-, and Explanation-Definitions can produce irrelevant anomalies or explanations with poor fidelity.
Future opportunities: The survey highlights prior-knowledge integration as a promising direction for improving interpretability in anomaly detection.Prior knowledge may take the form of equations, simulations, logic rules, knowledge graphs, or human feedback and can be integrated into machine-learning pipelines.
Future opportunities: Future XAD research should address adversarial manipulation and computational cost while preserving explanation fidelity and enabling near-real-time interpretation.The paper specifically notes manipulation risks in repurposed XAI methods and high costs for subspace anomaly detection and Shapley-value methods.