PaperlayerView exact revision on arXiv

Source-linked AI summary

Explainable Artificial Intelligence for Industrial Cybersecurity: A Review of Methods, Operational Integration, and Research Challenges

Amr S. Mohamed, Charlotte Fritz, Ahmad Mohammad Saber, Yiqun Ma, Ratinder Kaur, Mohammed Al-Darwbi, Daniela Friedrich, Deepa Kundur

arXiv:2609.00171v1cs.CReess.SY

TL;DR

Industrial cybersecurity increasingly uses AI, but opaque decisions pose challenges for trust, safety, and regulatory compliance. This review examines XAI methods for industrial SOC workflows and finds explainability important for analyst decision-making, incident investigation, and governance requirements.

  • Problem

    Opaque AI decisions challenge operational trust, safety, and regulatory compliance in industrial cybersecurity, where incorrect decisions can have serious consequences.

    Source: §VII · p. 26

  • Method

    The paper reviews XAI methods and visualizations for industrial cybersecurity, examining their suitability and operational deployment requirements.

    Source: §VII · p. 26

  • Results

    The review analyzes the strengths, limitations, and suitability of XAI methods and visualizations for industrial cybersecurity applications.

    Source: §VII · p. 26

  • Takeaways & Limitations

    The review identifies explainability as important for analyst decision-making, incident investigation, and regulatory and governance requirements for trustworthy AI systems.

    Source: §VII · p. 26

Abstract

from arXiv · show

The increasing digitalization of industrial infrastructure and the convergence of information technology (IT) and operational technology (OT) have expanded the cyberattack surface of industrial systems. To address the growing complexity of cyber threats, artificial intelligence (AI) and machine learning (ML) techniques are increasingly deployed within industrial cybersecurity operations, particularly in Security Operations Centers (SOCs). While these approaches improve anomaly detection, threat analysis, and automated response, their opaque decision-making presents challenges for operational trust, regulatory compliance, and incident response. EXplainable Artificial Intelligence (XAI) has emerged as a promising paradigm to improve the transparency and interpretability of AI-driven cybersecurity systems and decisions. This paper provides a comprehensive review of XAI techniques in industrial cybersecurity, focusing on industrial SOC environments and operational security workflows. We examine the role of AI in industrial SOC workflows, the types of operational data leveraged in industrial environments, and the benefits and limitations of AI-based threat detection. We then review major families of XAI approaches, including feature attribution methods, surrogate models, rule-based explanations, and visualization techniques, and analyze their applicability to industrial use cases. We further discuss the operational, regulatory, and safety requirements that distinguish industrial systems from traditional IT environments. Key challenges are examined, including limited labeled datasets, model reliability, explainability-performance tradeoffs, and the integration of XAI tools into SOC workflows. Finally, we identify open research directions and opportunities for developing trustworthy, operationally viable, and domain-specific XAI-enabled cybersecurity solutions for industrial environments.

I. INTRODUCTION · II. THE ROLE OF THE INDUSTRIAL SECURITY OPERATIONS CENTER

Industrial digitalization and IT/OT convergence expand the attack surface, while black-box AI and industrial safety constraints make explainability essential. The paper reviews XAI for industrial cybersecurity through the operational lens of industrial SOCs.

I. INTRODUCTION

  • Industry 4.0, IIoT, cloud, edge, and wireless connectivity improve visibility and efficiency but increase industrial system complexity and expand the cyberattack surface.Industrial environments combine physical processes, OT, communications, computing platforms, and human operators; ICSs include PLCs, HMIs, SCADA, and DCS, while ICPSs and IIoT connect operational systems and data across infrastructures.Source: §I · p. 1
  • Rule-based systems detect known threats but depend on manually crafted signatures and can miss novel attacks, motivating ML adoption for anomaly detection and security decision support.ML learns patterns from operational and security data to support large-scale monitoring, threat detection, analyst workload reduction, and routine task automation.Source: §I · p. 2
  • Because opaque ML decisions must be understandable, trustworthy, auditable, and actionable in safety-critical, regulated settings, XAI aims to make model behavior and security decisions transparent.Existing surveys largely address traditional IT, whereas industrial constraints, safety requirements, heterogeneous data, regulation, and real-time SOC decision-making shape industrial XAI needs.Source: §I · p. 2
  • The paper reviews industrial-cybersecurity XAI literature, datasets, models, methods, visualization, evaluation, operational suitability, SOC workflows, stakeholder needs, and industrial safety, regulatory, auditability, and accountability requirements.It also identifies research gaps and directions for trustworthy, operationally viable, domain-specific industrial XAI systems.Source: §I · p. 2
  • The review identifies two main research streams—network-traffic intrusion detection and process/OT-data anomaly detection—reflecting a persistent IT/OT divide.The authors describe the review as the first focused specifically on XAI for security in industrial contexts.Source: §I · p. 2

II. THE ROLE OF THE INDUSTRIAL SECURITY OPERATIONS CENTER

  • The industrial SOC provides a functional lens for examining security workflows even where responsibilities are distributed across engineering, IT, and operations teams rather than housed in a dedicated center.The section uses this perspective to consider typical responsibilities and challenges and where AI and XAI could augment operations.Source: §II · pp. 2–3
  • Unlike IT SOCs organized around confidentiality, integrity, and availability, OT prioritizes availability and safety, and incidents can cause equipment failure, downtime, environmental damage, or threats to life.Digitalization, remote access, and IT/OT convergence erode ICS isolation and broaden threats, making specialized OT SOC capabilities critical.Source: §II · p. 3

A. The Typical Workflow of an Industrial SOC · B. Challenges in Industrial SOCs

An OT SOC workflow spans preparation, real-time operations, and post-incident activities, while industrial SOC challenges include complex, high-stakes threats, collaboration, and regulatory verification. The passages also identify AI/XAI benefits and explanation methods relevant to SOC operations.

A. The Typical Workflow of an Industrial SOC

  • OT SOCs organize work into preparation, real-time operations, and post-incident phases, with preparation including asset inventory, behavioral baselining, and OT-specific data integration.The preparation-phase description is truncated after these examples.Source: A. The Typical Workflow of an… · p. 3

B. Challenges in Industrial SOCs

A. Deploying XAI Methods in Industrial SOCs … 1) Operational Constraints:

Industrial SOC security must preserve availability, safety, and uninterrupted processes, constraining updates, automated response, and active monitoring. The research agenda identifies data availability and industrial-specific XAI design, evaluation, and integration needs.

A. Deploying XAI Methods in Industrial SOCs

B. Emerging Research Needs for Industrial SOC-specific XAI

  • Research needs include integrating XAI into the AI design lifecycle, validating industrial-security-specific evaluation metrics, and aligning explanations with human reasoning and stakeholder needs.Source: B. Emerging Research Needs for… · p. 4
  • The research agenda identifies heavy reliance on IT/IoT datasets and realistic data generation as concerns for industrial SOC-specific XAI.Source: B. Emerging Research Needs for… · p. 4
  • Other directions include sequential and interactive XAI, physics-informed process simulations, knowledge-graph contextualization, and operational integration of LLMs and industrial foundation models.Source: B. Emerging Research Needs for… · p. 4

B. Challenges in Industrial SOCs

1) Operational Constraints:

  • Shutdowns and reboots require significant lead time and coordination, delaying patches and leaving OT systems exposed to known vulnerabilities for extended periods.Source: 1) Operational Constraints · p. 5
  • During incidents, OT may need to remain operational with manual operation available; automated responses require scrutiny to avoid interruptions, production loss, equipment damage, or regulatory violations.Source: 1) Operational Constraints · p. 5
  • OT security monitoring must be passive because active tools such as antivirus software and network scanners can degrade performance or behave unpredictably, limiting or precluding their use.Source: 1) Operational Constraints · pp. 5–6

2) Limited Visibility: … III. AI IN INDUSTRIAL SECURITY OPERATIONS CENTERS

Industrial SOC security is constrained by limited OT visibility, high-impact and evolving threats, alert overload, IT–OT coordination needs, and expanding compliance demands. These pressures motivate AI and XAI for scalable monitoring, threat detection, and decision-making.

2) Limited Visibility:

  • OT visibility—the ability to identify, monitor, and collect data on assets, traffic, and events—is limited by heterogeneous legacy devices, proprietary protocols, and missing logging,,.Resource-constrained PLCs and RTUs may lack capacity for complex detection models, while vendor-specific protocols impede standard monitoring.Source: 2) Limited Visibility · p. 6

3) High-Stakes Threat Landscape:

  • Industrial cyberattacks threaten human safety, the environment, and economic assets; NERC CIP violations can incur fines up to one million U.S. dollars per day.Equipment damage, criminal liability, and reputational harm are also potential consequences; threats come from adversaries including organized crime, terrorist groups, and nation-states,.Source: 3) High-Stakes Threat Landscape · p. 6

4) Complex Threats:

  • APTs, zero-day exploits, polymorphic malware, and insider threats challenge conventional detection, while manually maintained rules miss novel attacks despite detecting known CVEs through IoCs,,.Rules and heuristics require substantial ongoing manual effort as threats evolve.Source: 4) Complex Threats · p. 6

5) Data and Alert Overload:

  • SOC analysts must validate continuous alerts, assess true positives, scope and prioritize incidents, group related events, and correlate threat intelligence, a repetitive and resource-intensive workflow with many false positives.The volume and false-positive burden make alert handling time- and resource-consuming.Source: 5) Data and Alert Overload · p. 6

6) Lack of Interdisciplinary Collaboration:

  • Bridging OT engineers and IT analysts is essential for security and continuity; nearly 30% of surveyed organizations had merged SOCs, amid an acute industrial cybersecurity skills shortage,.Effective coordination requires specialized knowledge of OT systems and cybersecurity, and qualified analysts are difficult and costly to hire and retain.Source: 6) Lack of Interdisciplinary… · p. 6

7) Verifying Regulatory Compliance:

  • NERC CIP provisions call for monitoring connections, devices, and communications, while EU NIS2 requires monitoring and analysis of cyber threats, vulnerabilities, and incidents, including near-real-time monitoring.Industrial cybersecurity regulations also entail compliance costs and potential severe penalties for non-compliance.Source: 7) Verifying Regulatory Compliance · pp. 6–7, 3) High-Stakes Threat Landscape · p. 6
  • Industrial interconnectedness, operational complexity, and security-data volume motivate AI and XAI for scalable monitoring, threat detection, and decision-making in industrial SOCs.Traditional manual and rule-based approaches alone are increasingly difficult to use to secure modern industrial environments.Source: 7) Verifying Regulatory Compliance · p. 7

A. Benefits of AI in Industrial SOCs … 1) Erroneous Responses and Cybersecurity Risks:

AI can strengthen industrial SOC visibility, monitoring, threat analysis, analyst capacity, and compliance, but erroneous AI decisions and adversarial use can create operational and safety risks.

1) Improved Situational Awareness:

2) Real-Time Monitoring and Analysis:

  • AI supports scalable, real-time monitoring by automating telemetry processing and event correlation across complex IT/OT infrastructures under timing, reliability, and safety constraints.Source: 2) Real-Time Monitoring and Analysis · p. 7

3) Transformed Understanding of Complex Threats:

  • AI can learn normal industrial behavior and identify deviations, malware, suspicious trends, policy rules, and subtle patterns across heterogeneous data streams, including threats without predefined signatures.Source: 3) Transformed Understanding of… · p. 7

4) Reduced Data and Alert Overload:

  • AI can process large, diverse data streams beyond manual analyst capacity and prioritize alerts by context, trends, and risk scoring to help reduce false positives and detection times,,.SOC teams face burnout and alert fatigue, and excessive alerts can consume resources enough to impede effective security.Source: 4) Reduced Data and Alert Overload · p. 7

5) AI as an Interdisciplinary Collaborator:

  • AI can automate routine work and help less experienced staff manage complex operations, shifting analysts toward AI-triage oversight, threat hunting, and investigation validation,.The passage presents both reduced dependence on Tier 1 analysts and their evolution toward supervising triage as possible workforce changes.Source: 5) AI as an Interdisciplinary… · p. 7

6) Automated Regulatory Compliance:

  • AI can support continuous monitoring and compliance by automating log analysis, audit preparation, and network baselining across OT environments.A 2025 survey reported that 98% of surveyed security leaders currently leverage AI to support compliance; NIS2 encourages AI use for detecting and preventing cyberattacks.Source: 6) Automated Regulatory Compliance · pp. 7–8

1) Erroneous Responses and Cybersecurity Risks:

  • Poorly designed AI can produce false positives and erroneous decisions that divert attention from genuine threats or disrupt legitimate engineering and safety procedures.Early adoption can also bring operational delays or safety hazards; Gartner predicted up to 85% of AI projects may deliver erroneous outcomes due to bias,.Source: 1) Erroneous Responses and… · p. 8
  • AI models and data require protection because leaks could give adversaries ways to disrupt operations, while threat actors also use AI for phishing, deepfakes, malware, and DDoS attacks,,.These risks reflect AI’s dual use as both a defensive tool and an enabler of advanced threats.Source: 1) Erroneous Responses and… · p. 8

2) Talent Shortage: … A. Background on XAI Methods

Cybersecurity staffing and specialized AI expertise, along with opaque decisions in high-stakes industrial settings, constrain AI adoption; XAI seeks to make AI decisions understandable and support trustworthy, auditable security operations.

2) Talent Shortage:

  • A global cybersecurity workforce shortage and limited specialized AI expertise impede AI adoption, while ICS/OT deployment also demands cross-domain knowledge amid staffing and investment constraints.Integration requires knowledge of control systems, operational constraints, and AI model behavior; engineering and OT security teams may face underinvestment, limited staffing, or competing priorities.Source: 2) Talent Shortage · p. 8

3) Lack of Transparency:

  • In safety- and reliability-critical industrial settings, opaque AI decisions hinder operators’ ability to understand, trust, and act on them; XAI offers human-interpretable justifications for responsible, auditable use.Source: 3) Lack of Transparency · p. 8

IV. EXPLAINABLE AI (XAI) IN INDUSTRIAL SOCS

  • XAI objectives include justifying AI decisions, revealing learned patterns, exposing biases and errors to improve systems, and supporting accountability and maintenance,.Source: §IV · p. 8

A. Background on XAI Methods

  • For industrial security, explainability clarifies AI behavior through intrinsic transparency or external tools; explanations are contextual and may target data, model structure, or predictions.Transparent models support ante-hoc explanations, whereas post-hoc methods explain opaque models after training.Source: A. Background on XAI Methods · pp. 8–9
  • Explanations may be global, local, or cohort-level, and associative, contrastive, or causal; associative and contrastive methods are useful when domain knowledge is limited.Associative methods include SHAP, LIME, ALE, PDP, and saliency maps; contrastive methods include anchors and counterfactual reasoning, while causal approaches can embed physical dynamics and domain constraints.Source: A. Background on XAI Methods · pp. 9–10
  • Surrogate models approximate opaque models and can run alongside high-performing black boxes to retain accuracy while enabling interpretable oversight, validation, and compliance.Explanations may be visual, textual, or statistical; dimensionality reduction can project high-dimensional explanations into 2D or 3D spaces.Source: A. Background on XAI Methods · p. 10
  • Industrial XAI must generate explanations rapidly and tailor them to users’ expertise, cognitive load, and decision context, with qualities such as actionability, transparency, accountability, and debuggability.Relevant stakeholders include developers, engineers, analysts, auditors, and regulators who use outputs for oversight, validation, and operational decisions.Source: A. Background on XAI Methods · pp. 10–11
  • Proposed objective metrics for comparing XAI methods include fidelity, stability, comprehensibility, and robustness.These metrics address the subjective nature of usability criteria and are relevant to the surveyed literature.Source: A. Background on XAI Methods · p. 11, Table III · p. 11

B. Benefits of XAI in Industrial SOCs · 1) XAI for Error Detection in AI Model Decisions: · 2) Support for Existing and Future Talent:

XAI can help identify and correct unreliable AI decisions while giving industrial SOC analysts context to interpret alerts and make timely, informed decisions.

1) XAI for Error Detection in AI Model Decisions:

  • Industrial AI errors can arise from bias –, operational drift, adversarial manipulation,, and noisy or incomplete data, with potential safety or operational consequences.These risks matter in real-time monitoring, anomaly detection, and automated response.Source: 1) XAI for Error Detection in AI Model… · p. 11
  • By surfacing model logic and feature contributions, XAI can help stakeholders detect flawed decisions, identify bias, improve data quality, and retrain models.Without transparency, systematically flawed decisions can be difficult to detect.Source: 1) XAI for Error Detection in AI Model… · p. 11
  • XAI can reveal why false positives disrupt operations or false negatives let critical threats go undetected, supporting refinement of detection strategies.Source: 1) XAI for Error Detection in AI Model… · p. 11

2) Support for Existing and Future Talent:

  • High alert volumes, limited staffing, and alert fatigue leave analysts manually reconstructing attack activity and correlating data, which can delay response.XAI tailored to SOCs can provide contextual insights for timely, informed, and confident decisions.Source: 2) Support for Existing and Future… · pp. 11–12
  • Contextualized alerts explain why an alert matters, its severity, and possible actions, supporting triage, prioritization, investigation, escalation, and post-incident review.Structured explanations can also support incident documentation aligned with NERC CIP and NIS2.Source: 2) Support for Existing and Future… · p. 11
  • Contextualized explanations can improve users’ objective understanding and satisfaction, drawing on domain knowledge such as Knowledge Graphs,, or relevant training examples,,.Contextualization is also recognized as a core component of effective explanation.Source: 2) Support for Existing and Future… · p. 11
  • RNNs can ground security-event explanations in the MITRE ATT&CK Framework, while attack graphs visualize event sequences by attack phase and provide a common taxonomy for threat intelligence.Source: 2) Support for Existing and Future… · pp. 11–12

3) Increased Trust, Adoption, and Human-AI Collaboration: … A. Datasets

Explainability is presented as a way to support calibrated trust and human-AI collaboration in industrial SOCs, while regulatory expectations and dataset limitations shape what trustworthy, actionable XAI can provide.

3) Increased Trust, Adoption, and Human-AI Collaboration:

  • Industrial SOC personnel rely on AI recommendations but face limited model explanations and distrust; explainability can support calibrated trust and better decisions,,,,,,,.Low trust can lead to disuse, whereas excessive trust in flawed systems can cause misuse and safety breaches; explanations let analysts interrogate, validate, or challenge outputs and support feedback loops.Source: 3) Increased Trust, Adoption, and… · p. 12

4) Improved Accountability and Automated Regulatory Compliance:

  • OECD, NIST, EU, and IEC frameworks emphasize transparency, accountability, risk management, or traceability, while industrial AI classification can be difficult,,,,.The EU AI Act was stated to become fully enforceable by August 2026; classification requires regulatory knowledge, and it determines which obligations apply.Source: 4) Improved Accountability and… · pp. 12–13
  • 99% of AI governance respondents implement human-monitored validation, and humans verify 69% of AI-powered decisions; standalone high-risk AI deployment remains very low,.The EU AI Act specifies transparency, human oversight, and instructions for high-risk systems; cybersecurity-only systems are excluded because they generally assist rather than replace human decision-making,,,.Source: 4) Improved Accountability and… · pp. 12–13
  • As security automation through SOAR expands, reduced human oversight brings explainability, transparency, and accountability obligations; XAI can provide audit trails and decision rationales.A Siemens survey of 36 experts and decision-makers suggested XAI could become a dedicated component of certification, evaluated alongside performance.Source: 4) Improved Accountability and… · p. 13

V. OVERVIEW OF XAI DATASETS AND METHODS

  • Cybersecurity datasets are scarce because legacy equipment and proprietary protocols limit visibility and interoperability, incidents are infrequent, and privacy and disclosure concerns impede sharing, –.Organizations may avoid publicly disclosing incidents because of reputational, goodwill, or brand concerns.Source: §V · p. 13

A. Datasets

  • Most surveyed studies use IDS datasets with labeled benign and malicious traffic; NSL-KDD and UNSW-NB15 provide attack categories and packet-, flow-, and statistical features,.Synthetic datasets captured and analyzed with tools such as TCPDump, Wireshark, Snort, or Zeek can simplify labeling and support supervised classifiers used for XAI experiments.Source: A. Datasets · p. 13
  • Many IDS benchmarks lack OT protocols, asset context, and process variables, so faithful explanations may still be less actionable for industrial defense [44].Such explanations may not map directly to process behavior, safety implications, or engineering response decisions.Source: A. Datasets · pp. 13–14
  • OT anomaly-detection studies often use LSTM autoencoders trained on normal time series, detecting anomalies through reconstruction error; these datasets remain scarce and lack a recognized benchmark [100], [101].The models learn to reconstruct normal patterns and signal anomalies when reconstruction fails.Source: A. Datasets · p. 14

B. Methods · 1) SHapley Additive EXplanations (SHAP):

SHAP is the most widely applied XAI method in the surveyed industrial cybersecurity literature, providing feature-attribution explanations and visualizations used across supervised detection and unsupervised anomaly analysis. The review relates these capabilities to industrial SOC workflows, including alert triage and anomaly localization.

B. Methods

  • The review connects XAI methods to industrial SOC needs such as alert triage, anomaly localization, model validation, analyst trust, and post-incident documentation.SHAP visualizations can help analysts identify network, device, or process variables contributing to alerts for triage, escalation, and root-cause analysis.Source: B. Methods · p. 14, 1) SHapley Additive EXplanations (SHAP) · p. 15

1) SHapley Additive EXplanations (SHAP):

  • SHAP was the most widely applied surveyed method, assigning each feature a Shapley value based on its average marginal contribution across feature subsets.Its cooperative-game-theory basis gives equal attribution to features with equal influence.Source: 1) SHapley Additive EXplanations (SHAP) · p. 14
  • FastSHAP and TreeSHAP approximate costly exact Shapley computations, with the cited passage describing faster explanations that can support real-time use.TreeSHAP is specified for tree-based models.Source: 1) SHapley Additive EXplanations (SHAP) · p. 14
  • Force and waterfall plots explain individual predictions from a baseline, decision plots extend this view across predictions, and bar and beeswarm plots summarize global feature importance.Bar plots rank mean absolute SHAP values but omit direction; beeswarm plots show value distributions, direction, and feature values.Source: 1) SHapley Additive EXplanations (SHAP) · pp. 14–15
  • Supervised IDS studies applied SHAP to ensembles and other classifiers across tabular datasets, using beeswarm, bar, force, and waterfall plots,,,,,.The cited examples include RF, XGBoost, LGBM, DT, SVM, Logistic Regression, DNN, and boosting models.Source: 1) SHapley Additive EXplanations (SHAP) · p. 15
  • An interpretable IDS paired a lightweight RF and SHAP-based Credibility Assessment Module with a secondary Transformer packet-inspection module to mitigate adversarial risks.The authors hypothesized that distinct feature spaces and architectures would reduce adversarial transferability; they tested the system on CSE-CIC IDS 2018 and CIC-IoT 2023.Source: 1) SHapley Additive EXplanations (SHAP) · p. 15
  • SHAP explained deep-learning IDS models on tabular and sequential data, including CNNs, and recurrent models,,,.The recurrent examples used LSTM or GRU architectures to capture temporal dependencies in network traffic or sensor data.Source: 1) SHapley Additive EXplanations (SHAP) · p. 15
  • For unsupervised anomaly detection, SHAP interpreted autoencoder-based DDoS predictions, and localized anomalies in HAI time-series and gas-pipeline SCADA data,.The HAI work used custom heatmaps to show prediction error and SHAP contributions across time steps; pipeline explanations included network features and OT signals such as pump status and pipe pressure.Source: 1) SHapley Additive EXplanations (SHAP) · p. 15

2) Local Interpretable Model-Agnostic Explanations (LIME): … 5) Rule-based Explanations:

The reviewed methods range from local explanations and feature-effect visualizations to global importance measures and readable rules. Their operational value depends on balancing insight and transparency with stability, accuracy, and the complexity of industrial security data.

2) Local Interpretable Model-Agnostic Explanations (LIME):

  • LIME approximates an individual black-box prediction with an interpretable surrogate fitted to perturbed samples, but sampling variance can make explanations unstable in high-stakes SOC settings.Its feature-weight bar charts show the leading features and whether they support or oppose the predicted class.Source: 2) Local Interpretable Model-Agnostic… · p. 16
  • SP-LIME aggregates explanations for diverse representative instances to approximate global patterns, while a gas-pipeline study used it to derive generalized attack rules from local explanations.The pipeline evaluation covered seven attack types; combining AI models produced robust detection, with LIME translating predictions into actionable attack signatures.Source: 2) Local Interpretable Model-Agnostic… · p. 16
  • White-box models offered more stable explanations but lower accuracy; SHAP outperformed LIME across four and six evaluation metrics, while LIME was more efficient for local explanations [127, 120, 112].The comparisons also found LIME limited in descriptive accuracy, sparsity, and completeness, while SHAP was more time-efficient for global explanations.Source: 2) Local Interpretable Model-Agnostic… · pp. 16–17
  • Combining SHAP and LIME insights with recursive feature elimination raised model accuracy from 95.59% to 97.02% in a 6G IoT security study.Other reviewed work applied both explainers to deep-learning and hybrid intrusion-detection models across IoT and network datasets.Source: 2) Local Interpretable Model-Agnostic… · p. 16

3) Feature Influence Visualization Techniques:

  • PDP shows average feature effects but can mislead with correlated features, ICE reveals instance-level heterogeneity but may be noisy, and ALE estimates local effects more robustly under correlation.These two-dimensional plots relate feature values to predicted likelihood and can help analysts interpret the effects of sensor readings, control signals, or traffic features.Source: 3) Feature Influence Visualization… · p. 17

4) Permutation Feature Importance:

  • Permutation Feature Importance estimates global feature reliance by shuffling each feature and measuring the resulting performance drop, supporting audits for meaningful signals versus dataset artifacts.Repeated shuffles provide mean and standard-deviation estimates; Keshk et al. and Gummadi applied PFI in IDS frameworks.Source: 4) Permutation Feature Importance · p. 17

5) Rule-based Explanations:

  • Decision trees expose auditable feature-split rules, but can overfit and vary with training data; ensembles improve robustness yet obscure decision logic across many trees.Tree-derived feature importance aggregates and normalizes impurity reduction across splits, while readable rules can support SOC detection and audit documentation.Source: 5) Rule-based Explanations · p. 17
  • RuleFit extracts ensemble-tree rules as binary features, then uses an L1-regularized sparse linear model to select and weight interpretable rules and original features.Rule importance can be shown with coefficient bar plots, and individual predictions decomposed into rule contributions.Source: 5) Rule-based Explanations · pp. 17–18

6) Combinations: · 7) Custom Methods: · C. Research Landscape

Studies combine global and local XAI methods, while custom frameworks such as TRUST offer statistical explanations; surveyed research is concentrated in supervised IDS studies and feature-attribution methods. The section also classifies explanations by the questions they answer.

6) Combinations:

  • Gummadi et al. combined global SHAP, LOCO, PFI, and ALE with local CEM and LIME to analyze feature importance.LOCO measures the performance drop from removing features; CEM identifies minimal changes that alter a prediction or preserve it.Source: 6) Combinations · p. 18
  • DeepLIFT outperformed LIME, SHAP, and Integrated Gradients across faithfulness, complexity, robustness, and reliability in a study of Suricata alert prioritization [169].Analyst-identified features aligned with those highlighted by the methods, but the evaluation used one organization’s IT dataset and binary classification.Source: 6) Combinations · p. 18

7) Custom Methods:

  • TRUST selects informative latent factors with mutual information, fits class-specific multimodal Gaussian distributions, and reported 25-fold faster processing than LIME.Its latent factors are combinations of input features and may be difficult for non-technical users to interpret; its Gaussian and independent-sampling assumptions may not hold in practice.Source: 7) Custom Methods · p. 18

C. Research Landscape

  • Most surveyed studies used supervised tabular ML on IDS datasets; sequential models were less common, and unsupervised methods appeared in only a few anomaly-detection studies.Benchmark availability, including NSL-KDD, UNSW-NB15, and CIC-IDS, helped enable ML application and comparison.Source: C. Research Landscape · pp. 18–19
  • SHAP was the most widely used XAI method, with bar plots common globally and force or waterfall plots used locally; LIME bar plots also appeared frequently.The surveyed trend favors associative, feature-attribution explanations.Source: C. Research Landscape · p. 19
  • The taxonomy groups explanations about how a model predicts, why it made a particular prediction, and why it rejected an alternative.These categories describe the prediction process, rationale, and rejected alternatives.Source: C. Research Landscape · p. 19
  • Other taxonomy categories address hypothetical input changes, minimal changes for a different prediction, conditions preserving the current prediction, and model performance.These are labeled “What if?”, “How to be that?”, “How to still be this?”, and “Performance.”Source: C. Research Landscape · p. 19

VI. OPERATIONAL INTEGRATION, SECURITY CONSIDERATIONS, AND RESEARCH DIRECTIONS … 2) Security Considerations:

The section maps XAI to industrial SOC workflows and examines how explanation limitations and security risks complicate deployment. It identifies a need for systematic evaluation and secure practices tailored to industrial SOCs.

A. Deploying XAI Methods in Industrial SOCs

  • Surveyed studies often omit which industrial stakeholders or SOC tasks their explanations support, limiting operational applicability in specialized industrial workflows.The section presents its XAI-to-SOC-task mapping as non-exhaustive and informed by the surveyed literature and its interpretation of industrial SOC workflows.Source: 1) Proposed Approach · p. 19

1) Proposed Approach:

  • During preparation, models and explainers are fitted; in operations, XAI supports alert triage, correlation, prioritization, validity assessment, and escalation documentation.Analysts can use model confidence to prioritize alerts and explanations to assess contributing features and possible false positives.Source: 1) Proposed Approach · p. 19
  • SHAP or LIME feature contributions and benign counterfactuals can help analysts prioritize validation and escalation based on feature impact and distance from benign behavior.A distant benign counterfactual may indicate a highly anomalous event, while a close one may warrant scrutiny as possible borderline behavior.Source: 1) Proposed Approach · p. 19
  • XAI insights can support rule updates, incident investigations, cross-functional communication, and compliance documentation while reducing cognitive load across SOC tiers.Source: 1) Proposed Approach · p. 20

2) Security Considerations:

  • Misinterpreted, inconsistent, or poorly scoped explanations can distract analysts, waste investigation time, justify incorrect predictions, or lead to flawed security rules.Treating local explanations as global can cause faulty generalizations; SOC personnel need to understand each method’s scope, assumptions, and limitations.Source: 2) Security Considerations · pp. 20–21
  • LIME may produce divergent explanations because local sampling assumes local linearity, while correlated features can make simplified explanations from methods such as PDP, PFI, SHAP, and LOCO misleading.Decision-tree explanations may also change substantially with minor training-data perturbations; simplified explanations can trade fidelity for interpretability.Source: 2) Security Considerations · pp. 20–21
  • Computationally intensive explanations such as SHAP can delay incident response, compete with monitoring resources, and create performance bottlenecks or denial-of-service risks.Source: 2) Security Considerations · p. 21
  • Exposed XAI outputs can help adversaries mislead analysts, justify false predictions, infer decision boundaries, evade detection, or leak sensitive operational information.The literature has a gap in systematic evaluation of XAI methods’ scope, assumptions, and vulnerabilities, as well as secure deployment practices for industrial SOCs.Source: 2) Security Considerations · p. 21

B. Emerging Research Needs for Industrial SOC-specific XAI … 5) Stakeholder-Driven XAI Design:

The section calls for XAI designed and evaluated around industrial SOC stakeholders, workflows, and operational needs. It highlights early design integration, domain-specific metrics, rigorous user validation, and greater alignment with human reasoning as research priorities.

1) Integrating XAI into the AI Design Lifecycle:

  • Specify explainability early: it shapes model selection, architecture, and evaluation, while iterative feedback between XAI and data curation can improve data integrity, predictive performance, and actionable explanations.Influential data points identified by XAI can be checked for data quality, bias, and labeling errors.Source: 1) Integrating XAI into the AI Design… · pp. 21–22

2) Industrial Security-specific XAI Evaluation Metrics:

  • Only a few surveyed studies quantitatively assessed XAI, while inconsistent criteria, thresholds, and terminology leave evaluation dependent on relative comparisons or subjective user studies.The cited examples of quantitative assessment are Kalakoti et al., and Arreche et al.,; relative comparisons include SHAP versus LIME under identical settings.Source: 2) Industrial Security-specific XAI… · p. 22
  • Rather than universal metrics, define industrial-security-specific explainability metrics, their definitions, and absolute thresholds for operational fitness, in alignment with Pietil¨a et al..Relative comparisons and human-centered validation provide insights but introduce subjectivity and hinder automated evaluation,, [178].Source: 2) Industrial Security-specific XAI… · p. 22

3) Validation for Building Trust in Industrial XAI:

  • Only Lundberg et al. studied presentation quality, with 7 of 30 participants being cybersecurity professionals; reported trust gains were not statistically significant, motivating larger, rigorous studies.This includes assessing visualization-based XAI in high-stakes industrial security.Source: 3) Validation for Building Trust in… · p. 22
  • Validate explanations with actual stakeholders in their work contexts: expertise, knowledge, cognitive capacity, alertness, and time shape usability, and trust can fluctuate or erode after erroneous decisions.Application-level testing should involve SOC analysts across tiers, managers, auditors, IT teams, process engineers, and MLOps engineers; interviews and questionnaires require careful design and dedicated resources.Source: 3) Validation for Building Trust in… · pp. 22–23

4) Alignment of XAI Approaches with Human Reasoning:

  • Only a few surveyed works used contrastive XAI—CEM, counterfactual explanations, and anchors —despite evidence that people naturally use contrastive reasoning,,.The authors identify applying contrastive methods to industrial security as a research gap.Source: 4) Alignment of XAI Approaches with… · p. 23

5) Stakeholder-Driven XAI Design:

  • Most surveyed papers used general-purpose XAI methods and visualizations without stakeholder tailoring, while AI and ML ranked lowest in user satisfaction among evaluated security tools.Personalization and intuitive interaction are reported to improve trust, satisfaction, and adoption,.Source: 5) Stakeholder-Driven XAI Design · p. 23
  • Design explanations to address stakeholders’ workflow knowledge gaps and meet usability criteria, then specialize methods and visualizations to their data, workflows, documentation, communication, and compliance needs,.The cited criteria include coherence, novelty, recency, and fidelity to model reasoning.Source: 5) Stakeholder-Driven XAI Design · p. 23

6) Sequential XAI Techniques for Industrial Environments : … 9) Contextualized Explanations Supported by Knowledge Graphs:

The sections identify gaps in static explanations for temporal industrial data and non-interactive XAI, then describe interactive, physics-informed, and knowledge-graph approaches for making explanations more context-aware and operationally grounded.

6) Sequential XAI Techniques for Industrial Environments :

  • Sequential industrial XAI remains largely limited to static feature importance, although incidents may emerge across time steps and require explanations of influential steps, transitions, and feature evolution.Future methods should attribute importance to temporal context so explanations reflect how anomalies manifest and propagate.Source: 6) Sequential XAI Techniques for… · p. 23

7) Engaging and Interactive XAI Systems:

  • Most reviewed XAI methods provide fixed explanations, while question-asking can deepen understanding and connect explanations to users’ prior knowledge –.Static XAI does not support further user interaction or iterative exploration.Source: 7) Engaging and Interactive XAI Systems · p. 23
  • Studies report improved user comprehension, trust, and collaboration with dialogue or flexible explanation exploration,,, while users ask questions, seek visualization clarification, and generate alternatives.Rohlfing et al. argue for co-constructing explanations to establish common ground and tailor them to users’ needs.Source: 7) Engaging and Interactive XAI Systems · pp. 23–24
  • Conversational agents using question answering over Knowledge Graphs and Large Language Models – could use SOC, model, XAI, visualization, and data context to answer stakeholder questions.This supports dynamic conversations with context-aware, user-tailored explanations.Source: 7) Engaging and Interactive XAI Systems · p. 24

8) Physics-Informed Machine Learning for Process Simulations:

  • Physics-informed machine learning integrates domain-specific physical laws into model training, combining data-driven patterns with physically consistent behavior and predictions aligned with domain expertise [194],.In industrial security, it can ground anomaly detection in physical-law violations, attack classification in physics-based propagation, and explanations in specific physical mechanisms.Source: 8) Physics-Informed Machine Learning… · p. 24

9) Contextualized Explanations Supported by Knowledge Graphs:

  • Knowledge Graphs combined with Graph Neural Networks encode entities and relationships such as devices, sensors, faults, events, causality, and connectivity, learning embeddings of system roles and dependencies.Graphs can be constructed from expert knowledge, structured databases, or unstructured text.Source: 9) Contextualized Explanations… · p. 24
  • Graph-based detection can identify deviations from expected structure and support explanations tracing causal chains and root causes; Knowledge Graphs can link OT failures to attack vectors and contextualize IT attack classifications.Graph relationships can yield human-readable narratives grounded in system topology and operational logic.Source: 9) Contextualized Explanations… · p. 24
  • Retrieval-Augmented Generation lets LLM chat agents query Knowledge Graph facts and relationships for more factually grounded, context-relevant conversational explanations [198],, while Knowledge Graph–PIML hybrids combine semantic context with physical consistency –.The hybrid can support localized predictions and symbolic inference alongside numerical simulation.Source: 9) Contextualized Explanations… · p. 24

10) Operational Integration of Large Language Models (LLMs) and Industrial Foundation Models (FMs): … VII. CONCLUSION

The review examines how XAI can make AI-driven industrial cybersecurity more transparent and operationally trustworthy, while identifying dataset realism, deployment, and evaluation gaps. It also describes complementary roles for LLMs and industrial foundation models alongside traditional ML/DL.

10) Operational Integration of Large Language Models (LLMs) and Industrial Foundation Models (FMs):

  • Traditional ML/DL process numerical data for deterministic, time-critical tasks, while LLMs enrich SOC pipelines with summaries, recommendations, and contextual analysis of text and sequential data.LLMs complement rather than replace purpose-built models where strict latency and domain constraints apply; retrieval, fine-tuning, and human feedback can align them with business needs.Source: 10) Operational Integration of Large… · pp. 24–25
  • Industrial FMs adapted with parameter-efficient fine-tuning can reduce data and compute needs; IBM researchers reported halved deployment time and improved forecasting accuracy.The adaptation freezes most pretrained weights and updates small adapter modules or a few layers.Source: 10) Operational Integration of Large… · p. 25

1) Heavy Reliance on IT/IoT Datasets:

  • Public benchmark IDS datasets enable reproducibility and comparisons, but their IT/IoT focus has steered industrial XAI research toward network traffic rather than cross-level industrial attacks.The review calls for XAI that traces attacks across IT and OT and correlates indicators of compromise across both domains.Source: 1) Heavy Reliance on IT/IoT Datasets · p. 25
  • Industrial networks segment OT from enterprise IT, so attacks may move laterally across layers; BlackEnergy 3 pivoted from enterprise computers to control equipment in the 2015 Ukraine grid attack.The Purdue architecture uses a level 3.5 demilitarized zone as a buffer between OT and IT networks.Source: 1) Heavy Reliance on IT/IoT Datasets · p. 25

2) Realistic Data Generation Solutions:

  • Synthetic and small-scope datasets may miss realistic behavior, device and protocol diversity, and adversary tactics, while large-scale real-world evaluations remain rare.These limitations can impair generalization and leave model performance across equipment and operational contexts largely untested.Source: 2) Realistic Data Generation Solutions · pp. 25–26
  • Digital twins can generate synchronized cyber-physical data across ICS layers and support safe attack injection, labeled-data creation, and human-in-the-loop XAI experiments,,,.Unlike high-fidelity simulation testbeds, twins support bidirectional communication and continuous updates from live operational data.Source: 2) Realistic Data Generation Solutions · p. 26
  • ICS honeypots capture authentic traces from real adversaries, and combining them with digital twins can improve data fidelity and attack realism,,,.Reinforcement-learning agents can also generate increasingly complex, adaptive attack strategies in simulation and digital-twin testbeds –.Source: 2) Realistic Data Generation Solutions · p. 26

VII. CONCLUSION

  • The review positions XAI as a way to improve transparency and interpretability in industrial cybersecurity, while highlighting dataset scarcity, accuracy–interpretability tradeoffs, SOC integration, and evaluation standardization as challenges.It emphasizes explanations that are meaningful, reliable, and actionable for human operators in safety-critical settings.Source: §VII · p. 26

Related papers

Loading 2609.00171v1…