PaperlayerView exact revision on arXiv

Source-linked AI summary

Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunities

Subash Neupane, Jesse Ables, William Anderson, Sudip Mittal, Shahram Rahimi, Ioana Banicescu, Maria Seale

arXiv:2207.06236v1cs.CRcs.AI

TL;DR

AI-based intrusion detectors can be difficult for cybersecurity analysts to understand and use. This survey reviews explainable IDS approaches, compares white-box and black-box tradeoffs, and identifies definitional, stakeholder, and evaluation challenges.

  • Problem

    AI-based IDS can lack understandable reasoning, while explainability in intrusion detection lacks a consensus definition and evaluation metrics.

  • Method

    The survey synthesizes white-box and black-box XAI approaches for IDS and proposes a human-in-the-loop architecture and taxonomy.

  • Results

    White-box models generally provide more detailed explanations, whereas black-box models generally achieve higher prediction performance.

  • Takeaways & Limitations

    Designing X-IDS requires security-specific explainability definitions, stakeholder-tailored explanations, and metrics for evaluating explanations.

  • Takeaways & Limitations

    Explanations may be misleading or incorrectly interpreted, and adversaries may attack explanation mechanisms or their training data.

Abstract

from arXiv · show

The application of Artificial Intelligence (AI) and Machine Learning (ML) to cybersecurity challenges has gained traction in industry and academia, partially as a result of widespread malware attacks on critical systems such as cloud infrastructures and government institutions. Intrusion Detection Systems (IDS), using some forms of AI, have received widespread adoption due to their ability to handle vast amounts of data with a high prediction accuracy. These systems are hosted in the organizational Cyber Security Operation Center (CSoC) as a defense tool to monitor and detect malicious network flow that would otherwise impact the Confidentiality, Integrity, and Availability (CIA). CSoC analysts rely on these systems to make decisions about the detected threats. However, IDSs designed using Deep Learning (DL) techniques are often treated as black box models and do not provide a justification for their predictions. This creates a barrier for CSoC analysts, as they are unable to improve their decisions based on the model's predictions. One solution to this problem is to design explainable IDS (X-IDS). This survey reviews the state-of-the-art in explainable AI (XAI) for IDS, its current challenges, and discusses how these challenges span to the design of an X-IDS. In particular, we discuss black box and white box approaches comprehensively. We also present the tradeoff between these approaches in terms of their performance and ability to produce explanations. Furthermore, we propose a generic architecture that considers human-in-the-loop which can be used as a guideline when designing an X-IDS. Research recommendations are given from three critical viewpoints: the need to define explainability for IDS, the need to create explanations tailored to various stakeholders, and the need to design metrics to evaluate explanations.

I. INTRODUCTION & MOTIVATION

AI/ML-based IDSs are increasingly used to detect cybersecurity threats, but opaque DL models often provide accurate predictions without understandable justifications, undermining trust and analyst decision-making. The survey addresses this gap by reviewing XAI and X-IDS methods, comparing black-box and white-box approaches, proposing a user-centric architecture, and identifying research priorities.

  • IDSs monitor and analyze computer-system or network events for potential security problems that could breach Confidentiality, Integrity, or Availability.
  • AI/ML techniques are widely used to develop IDSs because they can achieve high detection rates and support efficient, accurate, and extendable systems.
  • Opaque DL models can achieve impressive prediction accuracies but lack clear justifications because their nested, nonlinear structures obscure which data influence decisions.
  • This lack of transparency creates trust and operational challenges for cybersecurity professionals who rely on understandable IDS predictions, especially when zero-day attacks are misclassified.
  • The survey reviews XAI and X-IDS research, develops a taxonomy for defining explainability, compares black-box and white-box approaches, and proposes a user-centric generic architecture.
  • It also identifies challenges in designing X-IDS and provides research recommendations for mitigating them in future work.

II. EXPLAINABLE ARTIFICIAL INTELLIGENCE (XAI) · A. NOTIONS OF EXPLAINABILITY · 1) Local explainability

The paper frames XAI around human-understandable explanations, while emphasizing unresolved definitional, stakeholder, and evaluation challenges. It organizes explainability by concepts, stakeholder tasks, evaluation metrics, scope, and model dependency, including local methods that justify individual predictions.

  • II. EXPLAINABLE ARTIFICIAL INTELLIGENCE (XAI): XAI remains difficult to define because the literature uses “explainability” and “interpretability” interchangeably, despite their distinct meanings.Interpretability concerns conveying meaning in human-comprehensible terms without additional explanations.
  • II. EXPLAINABLE ARTIFICIAL INTELLIGENCE (XAI): Understandability is central to XAI, but whether an explanation is clear or simple depends on the system’s audience.The paper therefore treats user characteristics as tightly coupled with explanation quality.
  • II. EXPLAINABLE ARTIFICIAL INTELLIGENCE (XAI): XAI lacks universal, objective criteria because researchers have not agreed on explainability, stakeholder-specific tasks, or metrics for validating explanations.The survey proposes a taxonomy to formalize these concepts, stakeholder perspectives, and evaluation techniques.
  • A. NOTIONS OF EXPLAINABILITY: Explainability taxonomies classify methods by attributes and theoretical approaches, including explanation scope, algorithm methodology, usage, and model dependency.Prior work also organizes black-box explainers by problem nature, explainer type, processed model, and supported data.
  • 1) Local explainability: Local explainability generates a unique explanation or justification for a single model prediction or decision.It addresses the specific outcome rather than the model globally.
  • 1) Local explainability: Common local explanation methods include LIME, Anchors, and LOCO, with LIME using an interpretable surrogate model to approximate the original model locally.LIME evaluates explanation fidelity through a loss function while penalizing surrogate-model complexity.
  • 1) Local explainability: SHAP provides a game-theoretic local explanation by calculating each feature’s significance in an individual prediction.Its Shapley values indicate how features contribute to the model output, and the approach is equivalent among LIME, DeepLIFT, and LRP.

2) Global explainability · 3) Model-specific interpretability · 4) Model-agnostic interpretability

Global explainability describes a model’s reasoning across possible outcomes, while model-specific methods constrain interpretation to certain model classes. Model-agnostic methods decouple explanations from the underlying model and support local or global interpretations through four broad technique types.

  • 2) Global explainability: Global explainability clarifies a model’s decision-making process across possible outcomes and attributes patterns across varied input data.It provides a whole-model view rather than explaining only one prediction.
  • 2) Global explainability: SP-LIME extends LIME with a submodular pick algorithm to reveal global correlations and provide a non-redundant decision boundary from individual instances.The method derives global understanding by selecting representative data instances.
  • 2) Global explainability: Concept Activation Vectors interpret neural-network internal states through human-friendly concepts.CAVs are presented as another global explainability method.
  • 2) Global explainability: GIRP constructs a global interpretation tree from local explanations across various machine-learning models.The method is described as Global Interpretation via Recursive Partitioning.
  • 3) Model-specific interpretability: Model-specific interpretability applies only to limited model classes that provide a required interpretation type.This restriction can reduce the available choice of more accurate and representative models.
  • 4) Model-agnostic interpretability: Model-agnostic methods are modular because their explanatory modules are independent of the models they interpret.They can interpret artificial neural networks and produce either local or global explanations.
  • 4) Model-agnostic interpretability: Model-agnostic interpretability techniques are broadly categorized as visualization, knowledge extraction, influence methods, and example-based explanations.These four categories summarize techniques identified across reviewed papers.

B. FORMALIZING EXPLAINABILITY TASKS FROM THE USER PERSPECTIVES · C. MEASURES FOR EVALUATING EXPLAINABILITY TECHNIQUES · III. SURVEY & TAXONOMY

The survey frames explainability as user-dependent and human-centered, then reviews evaluation frameworks and develops an X-IDS taxonomy organized around techniques, applicability, strengths, weaknesses, and challenges. It also situates IDS by intrusion-detection function, deployment location, and detection technique.

  • B. FORMALIZING EXPLAINABILITY TASKS FROM THE USER PERSPECTIVES: Explainability requires human-comprehensible models, with explanation content depending on whether the recipient is a regular user, expert user, or external entity.The cited framework identifies three targets of explanation.
  • B. FORMALIZING EXPLAINABILITY TASKS FROM THE USER PERSPECTIVES: Keeping humans in the loop shapes explainability through human-like explanations that simulate cognition and human-friendly explanations centered on people.These perspectives emphasize both how explanations are produced and whom they serve.
  • C. MEASURES FOR EVALUATING EXPLAINABILITY TECHNIQUES: Explanation evaluation remains limited, and proposed methods include application-grounded, human-grounded, and functionally grounded approaches.These classes assess interpretability through different relationships between explanations, users, and system functions.
  • C. MEASURES FOR EVALUATING EXPLAINABILITY TECHNIQUES: Deep-network explanations can be evaluated by processing, representation, and explanation-producing criteria, including alignment with user expectations.Additional criteria include model completeness, bias detection, substitute-task completeness, and human evaluation.
  • C. MEASURES FOR EVALUATING EXPLAINABILITY TECHNIQUES: The survey next describes its approach and develops an X-IDS taxonomy grounded in current literature.A psychological model of explanation from the Florida Institute for Human and Machine Cognition is also identified as a potential benchmark.
  • III. SURVEY & TAXONOMY: An IDS comprises tools, methods, and resources that help CSoC analysts identify, assess, and report intrusions, and is classified as host-based or network-based.Intrusion detection is presented as part of a broader protection system rather than a stand-alone measure.
  • III. SURVEY & TAXONOMY: IDS detection techniques are signature-based, anomaly-based, or hybrid; signature matching cannot detect zero-day, metamorphic, or polymorphic threats, whereas anomaly detection identifies deviations from expected behavior.The taxonomy emphasizes XAI techniques’ intrusion-detection relevance, applicability, strengths, weaknesses, challenges, and assumptions.

A. SALIENT FEATURES OF WHITE BOX TECHNIQUES · B. SALIENT FEATURES OF BLACK BOX TECHNIQUES

The paper classifies X-IDS explainability into intrinsically transparent white box techniques and opaque black box techniques. White box methods prioritize understandable decisions, while black box explanations interpret predictions through feature attribution, perturbation, decomposition, or hybrid constructions.

  • A. SALIENT FEATURES OF WHITE BOX TECHNIQUES: White box models produce results that experts can readily understand, although they are generally less efficacious than black box counterparts.Their transparency is built into the model rather than added after prediction.
  • A. SALIENT FEATURES OF WHITE BOX TECHNIQUES: Regression models draw on mature statistical foundations and are used in early modeling stages, complex pipelines, and domains requiring scrutiny and transparency.They are also highly computationally efficient, enabling rapid construction.
  • A. SALIENT FEATURES OF WHITE BOX TECHNIQUES: Rule-based approaches learn human-like decision rules, supporting practical anomaly explanations and transferring mined rules from AIDS into SIDS for zero-day identification.This connects explainability with multi-stage IDS operation.
  • A. SALIENT FEATURES OF WHITE BOX TECHNIQUES: White box X-IDS techniques comprise regression, rule-based, clustering, and statistical or probabilistic methods, whose explainability is intrinsic to their decision processes.The taxonomy contrasts these with black box feature, perturbation, decomposition, and hybrid approaches.
  • A. SALIENT FEATURES OF WHITE BOX TECHNIQUES: Clustering aggregates similar data objects using supervised or unsupervised learning and measures such as Euclidean, Manhattan, Cosine, Pearson, or graph-based similarity.Statistical and probabilistic methods include moment-based approaches, statistical ensembles, Markov Models, and Bayesian Networks, but have declined alongside black box methods.
  • B. SALIENT FEATURES OF BLACK BOX TECHNIQUES: Black box models are opaque and difficult to inspect or evaluate, creating a need to explore their decision processes in safety-, privacy-, and fairness-sensitive applications.These models compose nearly all state-of-the-art systems, but no singular solution currently addresses their opacity.
  • B. SALIENT FEATURES OF BLACK BOX TECHNIQUES: Feature-based explanations attribute predictions to input features, whereas perturbation-based explanations examine output changes after altering inputs across tabular, image, or text data.Examples include PDP, ALE, ICE, saliency maps, RISE, and LIME.
  • B. SALIENT FEATURES OF BLACK BOX TECHNIQUES: Decomposition-based explanations allocate importance by decomposing model signals such as gradients, while hybrid approaches combine white box and black box components in variable IDS pipelines.Hybrid constructions offer the greatest variability in explanation methodology, location, and application.

IV. APPROACHES TO EXPLAINABLE IDS (X-IDS) · A. BLACK BOX X-IDS MODELS · 1) Feature based approaches

The survey organizes explainable intrusion-detection methods into black-box and white-box approaches, further categorizing black-box X-IDS by how explanations are generated. Feature-based approaches explain predictions through feature influence, including SHAP, domain-knowledge mappings, and feature-set comparisons.

  • IV. APPROACHES TO EXPLAINABLE IDS (X-IDS): The survey distinguishes black-box and white-box XAI approaches, then categorizes black-box X-IDS as feature-based, perturbation-based, decomposition-based, or hybrid.These categories are based on how explanations are generated.
  • A. BLACK BOX X-IDS MODELS: Black-box predictors are uninterpretable models that require techniques to extract explanations from their internal logic or outputs.The survey defines black-box models as internally unknown or not human-interpretable.
  • 1) Feature based approaches: Feature explanations describe how input features influence predictions, using methods such as PDP, ALE, H-statistic, and SHAP.Feature engineering and feature selection make feature-based explanations natural extensions of existing machine-learning pipelines.
  • 1) Feature based approaches: A SHAP-based framework combines local and global explanations for binary and multi-class neural-network IDS classifiers.Local explanations select an attack and randomly sample 100 occurrences before generating SHAP explanations.
  • 1) Feature based approaches: A domain-knowledge framework maps ranked features to attack types and CIA principles, adding aggregate C, I, and A scoring features.Examples associate DoS with availability and Heartbleed or PortScan with confidentiality.
  • 1) Feature based approaches: 99.68% F1-score was achieved by Random Forest using the full 78-feature CICIDS dataset, while the domain-infused dataset performed similarly and sometimes identified otherwise missed attacks.The evaluator compared 78-feature, 50-attribute, 22-feature domain-infused, and three-feature CIA configurations across six algorithms.
  • 1) Feature based approaches: NetFlow features produced minor performance improvements, substantially lower false-positive rates in many tests, and slightly faster predictions than CICFlowMeter features.The comparison used Random Forests and Deep Feed Forward algorithms across three datasets.
  • 1) Feature based approaches: An auto-encoder combined with SHAP explains anomalies by linking input features to high reconstruction scores.Samples with higher reconstruction scores are considered anomalous, and features are divided according to their contribution to that score.

2) Perturbation based approaches · 3) Decomposition Based Approaches

Perturbation-based X-IDS methods explain predictions by modifying inputs, including through counterfactuals that identify minimal changes, misclassification weaknesses, and actionable alternative classifications. Decomposition-based methods assign relevance through backward propagation or gradient decomposition, supporting textual and visual explanations while also assessing model behavior and performance.

  • 2) Perturbation based approaches: Perturbation-based approaches modify, remove, or add input features to observe changes in predictions and are model agnostic.Their explanations rely on feature inclusion, removal, or modification.
  • 2) Perturbation based approaches: Counterfactual techniques explain misclassifications and adversarial weaknesses by making minimal changes that alter an input’s predicted label.The method locates a similar point x′ in feature space with an opposite predicted label and extrapolates nearby counterfactual points.
  • 2) Perturbation based approaches: 93% and 95% accuracy were achieved by the tested linear classifier and multi-layer perceptron, respectively, while t-SNE visualized misclassified and corrected samples.The projections were nearly identical, helping reveal features associated with misclassification.
  • 2) Perturbation based approaches: A counterfactual X-IDS framework seeks understandable and actionable explanations by answering why one classification occurred instead of another and enabling classification changes.Understandability concerns explaining an instance, whereas actionability provides advice for changing its classification.
  • 3) Decomposition Based Approaches: Decomposition-based approaches create relevance scores by propagating information backward from model outputs or decomposing model gradients.In LRP, relevance scores identify activated neurons that affect predictions and ultimately estimate input importance.
  • 3) Decomposition Based Approaches: An LRP-based explainable DNN provides prediction confidence, textual reasons, and online anomaly explanations that experts can evaluate offline.The authors associate these explanations with reducing DNN opaqueness and enhancing human trust.
  • 3) Decomposition Based Approaches: GRACE uses Grad-CAM heatmaps to explain CNN decisions after encoding 1D textual feature vectors as 2D images.It was evaluated using F1-Score (F1), Accuracy (A), and Computational complexity (T) across KDD-CUP-99, NSL-KDDCUP99, and UNSW-NB15.
  • 3) Decomposition Based Approaches: On NSL-KDD, Triplet methods achieved 86.6% A and 87.0% F1, compared with 85.7% A and 86.8% F1 for GRACE.Despite this exception, GRACE generally outperformed competitors, and the authors argued that its explanations could support more robust intrusion detection models.

4) Hybrid Approaches · B. WHITE BOX X-IDS MODELS

The surveyed hybrid and white-box X-IDS approaches seek to combine accurate detection with interpretable explanations, either through post-hoc mechanisms or transparent internal decision processes. White-box models avoid additional explanatory models for experts, while explanations for non-experts may still require visualizations.

  • 4) Hybrid Approaches: Szczepanski et al. combine a black-box predictor with a white-box explainer guided by fidelity, understandability, sufficiency, low construction overhead, and efficiency.The framework aims to remain reliable, understandable, flexible, and accurate.
  • 4) Hybrid Approaches: The hybrid explainer selects a centroid cluster close to the predicted sample that matches the ANN’s prediction, then visualizes that cluster as the explanation.The explainer may fail to return a valid tree, and its model should be trained on a feature-rich, diverse dataset.
  • 4) Hybrid Approaches: Pang et al. develop a Few-Shot Anomaly Detection framework that explains predictions through probability-based scoring and images highlighting anomalous areas.The approach addresses the difficulty of distinguishing anomalies when IDS or anomaly-detection models are trained mainly on unsupervised normal data.
  • 4) Hybrid Approaches: DevNet outperforms five other models on 7 out of 9 image datasets and achieves AUC scores between 80% and 98%.It explains detections by displaying anomalous regions using black-white and overlaid images.
  • B. WHITE BOX X-IDS MODELS: White-box models expose their internal logic and programming steps, enabling expert users to interpret decisions without additional models.For non-expert users, post-hoc explanations such as visualizations may still be needed, and interpretability can involve a performance cost.
  • B. WHITE BOX X-IDS MODELS: The survey focuses on commonly used white-box intrusion-detection approaches and summarizes their research, challenges, and contributions in Table 2.These approaches are selected using the survey’s Section III overview and Figure 2 taxonomy.

1) Regression · 2) Decision Tree and Rule Based

Regression models provide intrinsically interpretable relationships but can be sensitive to outliers, while decision-tree and rule-based IDSs explain predictions through ranked features, logical rules, and diagnosis justifications. The reviewed approaches include an explainable HPC-based double-regression framework, interpretable decision trees, hybrid expert-and-learned rules, and Snort’s alert-generating rule system.

  • 1) Regression: Linear and logistic regression are intrinsically interpretable because their transparent structure relates inputs to outputs through coefficients.Linear regression is described as establishing a best-fit line, with coefficients representing feature contributions.
  • 1) Regression: Regression-based IDSs can perform promisingly, but linear regression performance is susceptible to outliers.LDA and logistic regression have been deployed in anomaly-based IDSs, while outlier sensitivity is identified as a limitation of linear regression.
  • 1) Regression: The HPCDR framework introduces explainability for hardware-performance-counter IDSs by studying microarchitectural and malware attacks.The framework examines five microarchitectural datasets and two malware datasets, including Rowhammer, Spectre, Meltdown, Bashlite, and PNScan.
  • 1) Regression: Ridge regression replaces Shapley values in HPCDR to reduce computational overhead while generating interpretable results from perturbed model outputs.Random forest, decision tree, and neural-network classifiers are evaluated before their outputs are passed to ridge-regression stages using hardware performance counters.
  • 2) Decision Tree and Rule Based: Decision trees handle nonlinear input-output relationships and are most often constructed top-down using divide-and-conquer.The top-down approach begins with a root node and is contrasted with bottom-up construction.
  • 2) Decision Tree and Rule Based: Rule-based models are transparent because IF-THEN rules, including rules extracted from decision trees, explain their predictions.Relational statements are combined into logical implications that represent the model’s knowledge.
  • 2) Decision Tree and Rule Based: An ID3 decision-tree IDS on KDD ranks feature relevance, produces explainable rules, and reaches accuracy comparable to state-of-the-art.The algorithm was selected because it mimics a human-based decision strategy.
  • 2) Decision Tree and Rule Based: Hybrid and rule-based IDSs improve transparency through genetic-algorithm rule extraction, expert-written rules, dynamic decision-tree knowledge, and diagnosis justifications.The hybrid system presents each prediction as an intuitive tree-like suggested rule, while Snort uses rules to identify malicious packets, generate alerts, and log packets.

3) Statistical and Probabilistic Methods · 4) Clustering · V. DESIGNING AN EXPLAINABLE IDS (X-IDS)

The paper surveys statistical, probabilistic, and clustering methods for intrusion detection before proposing a user-centric, human-in-the-loop architecture for explainable IDS. It highlights the need to connect ML/DL predictions to their reasoning and organizes X-IDS design into pre-modeling, modeling, and post-modeling explainability phases.

  • 3) Statistical and Probabilistic Methods: Statistical and probabilistic IDS use moments such as means, standard deviations, and correlations to identify events outside predefined intervals.The surveyed approaches include univariate, multivariate, time-series, parametric, and non-parametric models.
  • 3) Statistical and Probabilistic Methods: Existing statistical and probabilistic IDS include mean-and-standard-deviation models, multivariate models, and Markov-process-based systems.
  • 4) Clustering: Explainable clustering summarizes behavioral patterns within clusters, helping users understand their underlying commonalities.Clustering classifies samples according to similarity and is widely used for unsupervised machine learning.
  • 4) Clustering: SOMs reduce high-dimensional training data to a low-dimensional representation while preserving important topological and metric relationships.They use an input layer for high-dimensional data and an output layer that generates a nonlinear mapping.
  • 4) Clustering: A model-specific SOM explanation technique combines winner-take-all training with histograms, t-SNE, heat maps, and U-Matrix visualizations to generate local and global CPS-security explanations.
  • V. DESIGNING AN EXPLAINABLE IDS (X-IDS): AI-based IDS can produce high false-positive and false-negative rates, while ML/DL systems commonly report only benign or malicious classifications without showing how inputs led to outputs.This disconnect leaves cybersecurity specialists reviewing IDS results outside the intrusion-detection process.
  • V. DESIGNING AN EXPLAINABLE IDS (X-IDS): The proposed user-centric X-IDS architecture follows DARPA guidance and contains pre-modeling, modeling, and post-modeling explainability phases for developing explanations for stakeholders.The architecture is intended as guidance for designing X-IDS across varied scenarios and applications, with human-in-the-loop methods and visualizations supporting cybersecurity decision-making.

A. PRE-MODELING PHASE … 3) Feature Engineering

The pre-modeling phase converts raw network flows into high-quality datasets through benchmarking, preprocessing, exploratory analysis, visualization, and feature engineering. The survey emphasizes dataset representativeness and dimensionality reduction as important considerations for deployable X-IDS.

  • A. PRE-MODELING PHASE: The pre-modeling phase takes raw network flows as input and produces a high-quality dataset for subsequent X-IDS development.It covers benchmark datasets and common preprocessing techniques.
  • 1) Datasets: Public datasets support X-IDS training and benchmarking, but representative labelled cybersecurity datasets remain difficult to obtain.Many public datasets use unprocessed flows from packet captures and emulated environments to address privacy concerns.
  • 1) Datasets: NSL-KDD enables comparison with prior work, CICIDS2017 provides 3 million samples for scalability testing, and UGR supports evaluation of long-term trends.NSL-KDD is relatively small, CICIDS2017 includes newer attacks and flows, and UGR is a multi-terabyte dataset collected over 5 months.
  • 1) Datasets: Public benchmark datasets are not suitable for deployable systems, so CSoC users should evaluate X-IDS on organizationally representative datasets.The recommendation addresses the gap between benchmark performance and deployment conditions.
  • 2) Exploratory Data Analysis (EDA) and Data Visualization:: Exploratory Data Analysis helps characterize dataset features before data are forwarded to feature engineering.Heat maps, network diagrams, bar charts, and correlation matrices can visualize feature-space characteristics.
  • 3) Feature Engineering: IDS preprocessing commonly normalizes numerical features and One-Hot Encodes categorical features, potentially creating a computationally expensive feature space.The literature discusses Feature Selection and Feature Extraction as two dimensionality-reduction approaches.
  • 3) Feature Engineering: Feature selection reduces dimensionality by retaining an untransformed subset of features through filters, wrappers, or embedded/hybrid methods.Scikit-Learn libraries have also been used for feature selection in published IDS research.
  • 3) Feature Engineering: Feature extraction transforms original features while retaining most defining attributes; Principal Component Analysis is the most common technique and is unsupervised.PCA does not require class knowledge to identify features.

B. MODELING PHASE … 2) Users

The proposed X-IDS architecture models high-quality data, explains and evaluates predictions, then presents results through an interface that supports stakeholder decisions and feedback-driven refinement. It balances black-box accuracy with white-box interpretability while requiring stakeholder-specific interfaces and evaluation practices.

  • B. MODELING PHASE: The modeling phase feeds a high-quality preprocessed dataset into an ML/DL model, passes predictions to an explainer, and evaluates the resulting explanations.This process produces explanations intended to help users understand the reasons behind predictions.
  • 1) AI Model: Black-box models prioritize prediction accuracy, whereas white-box models are favored for interpretability, creating a central trade-off in X-IDS design.The passage states that high prediction accuracy is required in intrusion detection systems.
  • 1) AI Model: Common black-box choices include SVM, CNN, RF, MLP, LSTM, and GAN, providing multiple effective algorithmic options for intrusion detection.LSTM is identified as an RNN variant, and GANs are described as increasingly popular.
  • 1) AI Model: Explainer modules approximate predictions produced by the selected AI model using either white-box or black-box algorithms.This approximation connects model outputs with explanation generation.
  • 2) Explainer Module and Evaluation: LIME, SHAP, and LRP enable rapid testing across algorithms and datasets, but SHAP may be too time-consuming for real-time X-IDS operation.The passage identifies SHAP’s lack of real-time execution as a limitation of relying solely on out-of-the-box explainers.
  • 2) Explainer Module and Evaluation: Because no standard explanation metrics exist, application-grounded, human-grounded, and function-grounded evaluations can provide baseline assessment approaches.The paper notes that prior authors have evaluated explanations in various ways.
  • C. POST MODELING EXPLAINABILITY PHASE: The post-modeling explainability phase renders evaluated AI recommendations, decisions, or actions in a graphical explanation interface that users employ for informed decisions.The phase comprises two components: the explanation interface and users.
  • 1) Explanation Interface / 2) Users: Custom dashboards should support understanding, while developer-, defense-practitioner-, and investor-specific explainability requirements require tailored interfaces and feedback-driven model updates.Unclear explanations, unhelpful outputs, incorrect predictions, and incorrect explanations can prompt revised explainers, new data, or different preprocessing.

VI. RESEARCH CHALLENGES & RECOMMENDATIONS … C. EVALUATION METRICS

The paper identifies three central X-IDS challenges: defining explainability, tailoring explanations to stakeholders, and evaluating explanation quality. It recommends security-specific definitions, stakeholder-aware explanations, and customized evaluation metrics.

  • VI. RESEARCH CHALLENGES & RECOMMENDATIONS: X-IDS research remains in its infancy, with development hindered by unclear explainability notions, stakeholder needs, and a lack of formal evaluation metrics.Existing research primarily focuses on explaining individual IDS-dataset samples.
  • A. DEFINING EXPLAINABILITY FOR INTRUSION DETECTION: Researchers need a common IDS-specific definition of explainability, potentially building on DARPA’s foundational XAI definition and incorporating Confidentiality, Integrity, and Availability principles.The paper argues that security-domain elements are necessary for an X-IDS definition.
  • A. DEFINING EXPLAINABILITY FOR INTRUSION DETECTION: Key definitional questions concern what explainability means for intrusion detection, how to create IDS explanations, and which audiences those explanations serve.Researchers should also reassess how explanations benefit CIA and how X-IDS algorithms should be categorized.
  • A. DEFINING EXPLAINABILITY FOR INTRUSION DETECTION: Current X-IDS work is narrow because it explains individual dataset samples without considering audience type when constructing explanations.This limitation reinforces the need for broader, stakeholder-aware X-IDS research.
  • B. DEFINING TASKS AND STAKEHOLDERS: After defining explainability, X-IDS designers should tailor explanations to stakeholders, including developers, defense practitioners, and investors.These stakeholder categories require different degrees of explanation and visualization.
  • C. EVALUATION METRICS: Evaluating explainer-module outputs is a major challenge because each stakeholder category may require customized metrics, while no consensus currently exists.The paper recommends considering previously proposed explanation-evaluation metrics as potential foundations.

D. ADVERSERIAL AI … VII. CONCLUSION

The paper identifies adversarial manipulation, misleading explanations, and scalability constraints as major X-IDS challenges, then summarizes explainability approaches, architecture, and research priorities. It recommends black-box X-IDS designs where high precision and avoidance of false positives are critical.

  • D. ADVERSERIAL AI: Attackers may target X-IDS classifiers or explanations by adding, deleting, or modifying explanatory content to evade detection.Adversarial AI includes malicious use of AI, attacks that cause classifier misclassification, and explanation manipulation as a new attack surface.
  • E. MISLEADING/INCORRECT EXPLANATIONS: Explanations can mislead users even without attacks, causing incorrect interpretation when the model itself is correct.The explainer may need modification to prevent user error in such situations.
  • E. MISLEADING/INCORRECT EXPLANATIONS: Incorrect explanations can negatively affect CSoCs, requiring analysts to critically assess prediction reasoning and methods to audit prior explanation errors.Ideally, an X-IDS should audit itself and generate explanations for the audit.
  • F. SCALABILITY AND PERFORMANCE: Explanations should not needlessly slow IDS operation, so X-IDSs must balance explaining every sample against strategically selecting samples.The paper calls for comprehensive CPU, RAM, and disk-usage analysis of current and future explainers.
  • VII. CONCLUSION: AI-based IDS research has achieved unprecedented prediction accuracy, supporting widespread industry adoption as cyber networks and critical infrastructures face growing attack exposure.The conclusion frames IDS as critical defense tools for CSoCs, cyber-physical systems, and critical infrastructure.
  • VII. CONCLUSION: The survey highlights unresolved explainability challenges: no consensus definition, insufficient user-centered formalization, and no established explanation-evaluation metrics.It proposes a taxonomy focused on relevance and applicability to intrusion detection.
  • VII. CONCLUSION: White-box models are inherently interpretable, whereas black-box models require post-hoc methods such as LIME and SHAP; the paper recommends black-box X-IDS designs for precision-critical IDS.The recommendation reflects the need to prevent attacks and avoid false positives.
  • VII. CONCLUSION: The paper proposes a generic three-layer X-IDS architecture based on DARPA’s XAI architecture and offers research recommendations for future X-IDS development.The architecture is intended to support varied scenarios and applications without dependence on a particular specification or technology.
Loading 2207.06236v1…