Source-linked AI summary
Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications
Yu-Liang Chou, Catarina Moreira, Peter Bruza, Chun Ouyang, Joaquim Jorge
TL;DR
Model-agnostic counterfactual explanations aim to make opaque AI systems understandable, but their causal grounding remains unclear. This systematic review develops a taxonomy and finds that current approaches rely on spurious correlations rather than cause-effect relationships, limiting causal understanding for users.
Problem
Method
The paper systematically reviews counterfactual and causability research, surveying causality-based approaches to extract knowledge gaps and construct a taxonomy.
Source: §3 · p. 10, §10 · p. 46Results
Current model-agnostic counterfactual approaches lack a formal causal theory and generate explanations based on spurious correlations rather than cause-effect relationships.
Source: §6.8 · p. 38, §10 · p. 47Takeaways & Limitations
Without causal grounding, these systems cannot promote causal understanding to users and risk producing biased, sub-optimal, or erroneous explanations.
Source: §10 · p. 47The reviewed systems are limited by their lack of a formal and structured theory of causality, preventing them from promoting causal understanding to users.
Source: §10 · p. 47
Abstract
from arXiv · showhide
There has been a growing interest in model-agnostic methods that can make deep learning models more transparent and explainable to a user. Some researchers recently argued that for a machine to achieve a certain degree of human-level explainability, this machine needs to provide human causally understandable explanations, also known as causability. A specific class of algorithms that have the potential to provide causability are counterfactuals. This paper presents an in-depth systematic review of the diverse existing body of literature on counterfactuals and causability for explainable artificial intelligence. We performed an LDA topic modelling analysis under a PRISMA framework to find the most relevant literature articles. This analysis resulted in a novel taxonomy that considers the grounding theories of the surveyed algorithms, together with their underlying properties and applications in real-world data. This research suggests that current model-agnostic counterfactual algorithms for explainable AI are not grounded on a causal theoretical formalism and, consequently, cannot promote causability to a human decision-maker. Our findings suggest that the explanations derived from major algorithms in the literature provide spurious correlations rather than cause/effects relationships, leading to sub-optimal, erroneous or even biased explanations. This paper also advances the literature with new directions and challenges on promoting causability in model-agnostic approaches for explainable artificial intelligence.
1. Introduction
The introduction motivates explainability and causability for opaque AI decisions, particularly in high-stakes settings, emphasizing causal distinctions between causes and correlations. It frames counterfactuals as “what-if” explanations and outlines a systematic review of their theories, algorithms, applications, taxonomy, and analyses.
1. Introduction
- AI systems’ opaque internal mechanisms create a black-box problem, leaving people to accept machine-generated decisions without understanding them, including in high-stakes contexts.Deep learning has matched or exceeded human performance in tasks including medical decision-making [1].Source: §1 · p. 2
1.1. The need for Explainability
- XAI seeks human-understandable explanations of model predictions, driven by audit requirements and evidence that predictive systems can encode racial, gender, and other spurious biases [5, 7, 10, 11, 14, 15, 16, 17].Examples include healthcare algorithms denying minority groups extra care and medical imaging models relying on technicians’ metal tokens [10].Source: §1.1 · p. 2
1.2. From Explainability to the need of Causability
- Model-agnostic post-hoc methods build local approximations of predictive black boxes using feature correlations rather than true underlying mechanisms, and the inability to disentangle correlation from causation can produce sub-optimal or erroneous explanations.Source: §1.2 · p. 4
1.3. Counterfactuals as Means to Achieve Causability
- Causability concerns whether explanations achieve a specified level of causal understanding for human experts, while causal explanations should identify why features contributed to a prediction [22, 47, 48].The introduction links causal understanding to cause-effect relationships and argues that explanations should acknowledge the causes of the effects being explained [51].Source: §1.2 · p. 4, §1.3 · p. 4, Figure 1 · p. 5
- Counterfactuals are conditional assertions with false antecedents that point out which changes a user needs to make to achieve a predictive outcome.Source: §1.3 · p. 5
1.4. Contributions
- The paper argues that failing to disentangle correlation from causation can produce sub-optimal or erroneous explanations, making causability necessary for human-understandable XAI.Its systematic review critically examines counterfactual and causability literature through theory, algorithms, and applications, including potential effects on biased predictions.Source: §1.4 · p. 6
1.5. Paper Organisation
- The paper introduces a taxonomy for model-agnostic counterfactuals, analyzes algorithms and their underlying theories, discusses applications and causability, presents causability characteristics, answers research questions, and concludes.Source: §1.5 · pp. 6–7
2. A General Overview of Current Model-Agnostic Approaches in XAI
The section distinguishes inherently interpretable models from model-agnostic post-hoc approaches and focuses on counterfactuals because they can flexibly explain any pre-existing trained model. It reviews LIME and SHAP as prominent model-agnostic methods before extending the analysis to counterfactual and causability approaches.
2. A General Overview of Current Model-Agnostic Approaches in XAI
- Model-agnostic (post-hoc) methods include partial dependency plots and surrogate models; partial dependency plots compute the marginal effect of one or two features on predictions, and post-hoc methods can be applied to any predictive model.Source: §2 · pp. 7–8
2.1. LIME - Local Interpretable Model-Agnostic Explanations
- LIME approximates any classifier with a locally faithful interpretable model by perturbing samples near a local decision boundary and learning feature importance weights.The local model may be linear regression or a decision tree, with feature weights based on similarity between the original instance and perturbed predictions.Source: §2.1 · p. 8
2.2. Approaches Based on LIME
- LIME-based approaches generate image saliency maps and expose uncertainty from sampling randomness, sampling proximity, and variation across explained data points.LIME can also debug black-box models affected by biases such as data leakage, while Anchor [76] adds IF-THEN rules and coverage boundaries.Source: §2.2 · pp. 8–9
2.3. SHAP - SHapley Additive exPlanations
- SHAP [37] maps feature values to players in a coalitional game and fairly allocates prediction gains using Shapley values and four fairness properties.The properties are additivity, symmetry, efficiency, and dummy, with non-contributing features assigned a Shapley value of zero.Source: §2.3 · pp. 9–10
2.4. Approaches Based on SHAP
- SHAP-based studies found transparent credit-scoring results and captured dispersion, nonlinearity, structural breaks, and complex nonlinear joint feature impacts.Shapley flow further uses causal graphs to compute value flows describing black-box mechanisms, motivating the review’s subsequent focus on counterfactual and causability approaches.Source: §2.4 · p. 10
3. Systematic Literature Review Towards Counterfactuals and Causability in XAI
This section systematically reviews counterfactuals and causability in XAI using PRISMA, database searches, LDA topic modelling, and keyword co-occurrence analysis to examine relevant theories, algorithms, applications, and knowledge gaps. It identifies limited attention to causal relations and causability, highlights medical decision-making as a motivation for counterfactual research, and notes methodological scope limitations.
3. Systematic Literature Review Towards Counterfactuals and Causability in XAI
- The review investigates theories, algorithms, conditions, and challenges for counterfactual approaches that could promote causability and human-understandable explanations in XAI.Its purpose is to identify knowledge gaps by extracting, systematising, and analysing existing causality- and counterfactual-based approaches.Source: §3 · p. 10, §3.1 · p. 11
3.1. Research Questions
- The study follows PRISMA to define research questions, search and select literature, extract and analyse publication data, and identify review biases and limitations.The research questions address theoretical approaches, counterfactual algorithms, necessary and sufficient conditions for causability, and future challenges.Source: §3 · pp. 10–11, §3.1 · p. 11
3.2. Search Process
- Searches across IEEE Xplore, Scopus, and Web of Science used an explainability- and causality-focused query, reducing initial records from 6878, 116, and 126 to 4712, 709, and 124.LDA topic modelling was then applied to refine the search results and ensure alignment with the review’s scope.Source: §3.2 · pp. 11–12
3.3. Topic Modelling
- After filtering for “causal” and “counterfactual” terms, the final document set included 125 IEEE Xplore and 632 Scopus documents.The LDA model contained four topics.Source: §3.3 · pp. 12–13
3.4. Word Co-Occurrence Analysis
- Keyword co-occurrence analysis shows XAI research shifting from machine-centric systems toward human-centric interpretability, while counterfactual research gained substantial interest from 2019 onward.The observed shift coincided with growing explainability needs associated with autonomous systems and GDPR-era privacy and security requirements.Source: §3.4 · pp. 14–15
- The co-occurrence analysis links counterfactual XAI literature to medical decision-making, including patient treatment, experts, and diagnosis.Related clinical research supports using counterfactuals and causality to interpret predictive systems.Source: §3.4 · p. 15
- Causal-relations research in explainable AI is scarce, causability literature is recent, and the analysis identifies growing opportunities for counterfactual research.The review frames these gaps as motivating new approaches underpinned by causal reasoning.Source: §3.4 · p. 16
3.5. Inclusion and exclusion criteria
- The review selected and excluded papers using predefined eligibility criteria summarized in Table 1.These criteria were intended to identify literature addressing the proposed research questions.Source: §3.5 · p. 16, Table 1 · p. 17
3.6. Risk of bias
- Restricting searches to three databases, omitting reference extraction, and limiting keywords may have caused relevant articles to be missed.The authors identify cognitive bias and the resulting search incompleteness as risks to the review.Source: §3.6 · p. 17
4. Counterfactual Approaches Explainable AI: Properties for Good Counterfactuals
This section reviews counterfactual approaches for explainable AI by examining distance functions and key properties of good counterfactuals, including proximity, plausibility, sparsity, diversity, and feasibility. It emphasizes balancing proximity with decision certainty and practical, interpretable changes while recognizing trade-offs among candidate counterfactuals.
4. Counterfactual Approaches Explainable AI: Properties for Good Counterfactuals
- Counterfactuals explain how changing an instance’s features can flip its prediction to a desired outcome, making them example-based approaches for XAI.Different candidates may be preferred for minimal Euclidean distance, feasible paths, or better-defined decision-boundary regions.Source: §4 · p. 17, Figure 8 · p. 18
4.1. The Importance of Distance Functions in Counterfactual Approaches for XAI
- Most reviewed XAI counterfactual methods use L0-, L1-, L2-, or L∞-norms, whose differing properties determine sparsity, differentiability, and the scale of feature changes.L0 counts changed features but is nondifferentiable; L1 promotes sparse solutions; L2 is differentiable but less sparse; L∞ limits maximum feature change and generally yields less sparse solutions.Source: §4.1 · pp. 18–19
4.2. Properties to Generate Good Counterfactuals
- Good counterfactuals should be proximal, plausible, and sparse: they should remain close to the input, change legitimate mutable features, and alter as few features as possible.Changing immutable features such as gender or race can encode bias, whereas changing mutable features such as income is considered appropriate.Source: §4.2 · pp. 20–21
- Diversity complements proximity by generating multiple distinct counterfactual explanations, while predictive multiplicity challenges the assumption that one sparse recommendation is always superior.The supplied discussion introduces feasibility as a property responding to concerns about counterfactual recommendations, without specifying its later criteria.Source: §4.2 · p. 21
- The closest counterfactual may lie on an uncertain decision boundary, producing biased explanations; γ is preferable because it lies in a well-defined region and requires the shortest path to x [56].This supports human-interpretable explanations with the fewest possible feature changes.Source: §4.2 · p. 22
- Plausibility requires legitimate counterfactuals that do not change immutable features such as gender or race.Source: §4.2 · p. 22
- Feasibility excludes paradoxical or impractical recommendations, such as telling low-skilled mortgage applicants to double their salary before increasing their skills [56].Source: §4.2 · p. 22
- Satisfying feasibility automatically guarantees plausibility and promotes more interpretable counterfactual explanations.Source: §4.2 · p. 22
5. Counterfactual Approaches in Explainable AI: The Theory
The review analyzes 26 model-agnostic counterfactual algorithms and organizes them by theoretical grounding into a taxonomy of distinct algorithmic categories. These categories differ in their mechanisms and in how readily they satisfy properties such as plausibility, feasibility, diversity, and causality.
Taxonomy
- The review analyzes 26 algorithms and develops a taxonomy based on the “master theoretical algorithm” underlying each approach.The taxonomy was created from recurring theoretical backgrounds among model-agnostic counterfactual methods.Source: §5 · p. 22
- The proposed taxonomy lists instance-, constraint-, genetic-, regression-, game theory-, case-based reasoning-, network-, and probabilistic-centric approaches.Figure 10 presents the taxonomy and the main algorithms assigned to each category.Source: §5 · p. 22, Figure 10 · p. 23
Instance-Centric
- Instance-centric methods seek nearby counterfactuals through distances, novel losses, and optimization, but are more susceptible to violating plausibility, feasibility, and diversity.Some methods add loss-function mechanisms to mitigate these limitations.Source: §5 · p. 22
Constraint-Centric
- Constraint-centric methods formulate counterfactual generation as constraint satisfaction, offering general strategies that can readily satisfy properties such as diversity.Satisfiability modulo theory solvers are among the strategies used.Source: §5 · p. 23
Genetic-Centric
- Genetic-centric methods use crossover and mutation to search for counterfactuals and often satisfy diversity, plausibility, and feasibility.Their optimization process is based on genetic algorithms.Source: §5 · p. 23
Regression- and Game Theory-Centric
- Regression- and game-theory-centric methods extend LIME- or SHAP-like procedures, but their counterfactuals have difficulty satisfying plausibility and diversity.Regression approaches use feature weights after permutation, whereas game-theory approaches use Shapley values.Source: §5 · p. 23
Case-Based Reasoning
- Case-based reasoning retrieves and adapts stored counterfactuals, enabling it to satisfy plausibility, feasibility, and diversity.The search retrieves the closest counterfactuals to a given query from a database of good explanations.Source: §5 · p. 24
Probabilistic-Centric
- Probabilistic-centric methods model counterfactual generation probabilistically, and probabilistic graphical models have potential to satisfy Pearl’s causality framework.These approaches include random walks, Markov sampling, variational autoencoders, and probabilistic graphical models.Source: §5 · p. 24
6. Counterfactual Approaches to Explainable AI: The Algorithms
The review identified 18 model-agnostic XAI counterfactual algorithms and analyzed them according to their properties, applications, supporting data structures, theoretical backgrounds, and public availability.
- The review identified and analyzed 18 model-agnostic XAI counterfactual algorithms in depth.The algorithms were classified according to the different properties presented in Section 4.Source: §6 · p. 24
- The algorithms were classified by application to classification or regression problems and by their supporting data structures.Source: §6 · p. 24
- Table 2 classifies the collected algorithms by properties, theoretical backgrounds, applications, and public availability, with each algorithm analyzed according to its theoretical grounding.Source: §6 · p. 24
master algorithm. · 6.1. Instance-Centric Algorithms
This section examines instance-centric counterfactual algorithms that seek nearby desirable observations through proximity-based optimization. It covers variations in loss, distance, and optimization functions, including prototypes, feature weighting, feasibility constraints, diversity, and sparsity to improve interpretability and practicality.
6.1. Instance-Centric Algorithms
- WatcherCF finds a counterfactual x′ close to x_i while balancing prediction distance against feature distance through λ, using MAD-normalized L1 distance for sparsity.Higher λ prioritizes closeness to the desired outcome y′, and Adam minimizes the loss.Source: §6.1 · pp. 24–25
- Prototype Counterfactuals [100] add prototype guidance, autoencoder regularization, elastic-net terms, and L_proto to produce interpretable, in-distribution counterfactuals and accelerate search.Encoder or k-d-tree prototypes significantly speed up counterfactual generation; FISTA with L1 regularization optimizes the perturbation.Source: §6.1 · p. 25
- Weighted Counterfactuals extend WatcherCF with positive and weighted counterfactuals, using global feature importance or nearest-neighbour weights to improve interpretability.Feature-importance weights produced more compact counterfactuals and more human-understandable interpretable features than nearest-neighbour weights.Source: §6.1 · p. 26
- FACE constructs feasible counterfactuals through density-weighted shortest paths, classifier-confidence constraints, and Dijkstra’s algorithm over existing training datapoints.Experiments found that the third weight equation combined with ϵ-graphs generated the most feasible counterfactuals, while feasibility avoids impractical or offensive recommendations.Source: §6.1 · pp. 27–28
- DiCE extends and improves WatcherCF by generating sets of diverse counterfactual explanations for the same instance, emphasizing diversity, proximity, and sparsity.Its formulation combines a hinge-loss component, a distance-function component, and a DPP diversity factor.Source: §6.1 · p. 28
- TRUCE seeks minimal prediction-changing modifications through inverse classification, with Growing Spheres identifying close differently classified neighbours under sparsity constraints.The sparsity constraints define the algorithm’s notion of closeness.Source: §6.1 · p. 28
- The method defines a cost c(x,e) for moving from observation x to an enemy e classified in a desirable class.For binary classification, a desirable feature is found when the observation is classified into the other class.Source: §6.1 · p. 29
- Its distance function combines weighted L2 and L0 norms to balance proximity with human-interpretable, sparse feature changes.The L2-norm measures proximity between x and e, whereas the L0-norm supports a weighted average intended to preserve interpretability.Source: §6.1 · p. 29
- A greedy growing-sphere optimizer searches directions toward the decision boundary, minimizing the L2 distance and the minimum feature change.The algorithm applies the growing sphere procedure to find the closest feature in all possible directions.Source: §6.1 · p. 29
- Later work distinguishes unjustified counterfactuals produced by post-hoc model artifacts from justified explanations aligned with ground-truth data.Unjustified explanations do not represent ground truth, whereas justified explanations follow the ground-truth data.Source: §6.1 · p. 29
6.2. Constraint-Centric Approaches
Constraint-centric counterfactual methods encode plausibility, diversity, coherence, or feature-change costs through optimization and satisfiability constraints. Reported results show favorable distance and plausibility for MACE, while LVP-FI outperforms the other generalized inverse classification methods.
Model-Agnostic Counterfactual Explanations for Consequential Decisions (MACE)
- MACE maps counterfactual search to satisfiability modulo theory, encoding predictive, distance, and constraint functions as logical formulae.It supports any Lp-norm distance and any number of constraints, with plausibility and diversity among the considered constraints.Source: §6.2 · pp. 29–30
- MACE [93] achieved high plausibility and generated counterfactuals at more favorable distances than existing optimization-based approaches.Source: §6.2 · p. 30
Coherent Counterfactuals
- Coherent Counterfactuals uses mixed-polytope constraints and integer programming to efficiently generate coherent and diverse counterfactuals for mixed data.The approach first seeks minimum-change counterfactuals using the L1-norm, then maps solutions back into the original space.Source: §6.2 · p. 30
Generalized Inverse Classification
- LVP-FI outperformed HC+LS, GA, GA+LS, and LVP-BI in generalized inverse classification, while LVP-BI was comparable to GA and GA+LS.The five methods were evaluated by the average likelihood of test instances conforming to a non-ideal class under varying budget constraints.Source: §6.2 · pp. 30–31
6.3. Genetic-Centric Approaches · 6.4. Regression-Centric Approaches
Genetic-centric approaches use evolutionary search to generate diverse, constrained, robust, or fairness-aware counterfactuals, while regression-centric approaches derive counterfactual explanations from local surrogate models and feature rankings. In five case studies, CLEAR achieved significantly higher fidelity than LIME, whereas LIME-C was reported as stable, effective, and relatively fast.
6.3. Genetic-Centric Approaches
- LORE generates balanced synthetic neighborhoods with a genetic algorithm, fits an interpretable decision tree, and extracts counterfactual decision rules.Its distance combines matching for categorical features with L2 normalization for continuous features.Source: §6.3 · p. 31
- CERTIFAI uses a custom genetic algorithm to assess model robustness with CERScore and fairness across linear or nonlinear models and mixed tabular or image inputs without model approximations.It minimizes MAD-normalized L1 distance while imposing continuous, categorical, and immutable-feature constraints.Source: §6.3 · p. 31
- Genetic-centric methods generally optimize counterfactual distance while enforcing feasibility, including mixed-type and immutable-feature constraints, and can preserve diversity across candidate explanations.CERTIFAI varies immutable-feature values to assess fairness, while MOCE explicitly maintains feature-space diversity.Source: §6.3 · p. 32
- MOCE converts counterfactual search into multi-objective genetic optimization, producing diverse explanations that trade off prediction proximity, instance similarity, sparsity, and training-data plausibility.NSGA-II evaluates counterfactuals using four objective values and favors solutions with lower objective values.Source: §6.3 · p. 32
6.4. Regression-Centric Approaches
- CLEAR regressions achieved significantly higher fidelity than LIME in five case studies.CLEAR generates boundary counterfactuals, fits local regressions with interaction terms, and iterates using counterfactual fidelity error.Source: §6.4 · p. 33
- LIME-C combines additive feature attribution with counterfactual generation, relying on sufficiently accurate feature rankings and using SEDC to produce ranked counterfactuals.The method was reported as stable and effective across data and models, including large instances requiring many feature removals, while computing counterfactuals relatively fast.Source: §6.4 · pp. 33–34
- SEDC produces minimal word-removal counterfactuals for document classification by ranking words through β_jX_ij and selecting the smallest set that changes the predicted class [95].Cosine similarity measures proximity between the original and counterfactual documents.Source: §6.4 · p. 34
6.5. Game Theory Centric Approaches · 6.6. Case-Based Reasoning Approaches · 6.7. Probabilistic-Centric Approaches
This section surveys game-theoretic, case-based, and probabilistic-centric approaches for constructing counterfactual explanations, including methods using Shapley values, retrieved cases, and probabilistic models. It also presents recourse as a model-agnostic approach based on Pearl’s causal probabilistic framework.
6.5. Game Theory Centric Approaches
- SHAP-C combines Kernel SHAP with SEDC, ranks features by their SHAP values, and perturbs the most important features until a minimum set flips the datapoint’s prediction.Source: §6.5 · p. 34
- SHAP-CC [110] produces “Why P not Q” contrastive-counterfactual explanations by using positive and negative Shapley values to answer why the predicted class occurs and why the desired class does not.The approach uses P-contrast questions and presents the explanation in natural language.Source: §6.5 · p. 35
6.6. Case-Based Reasoning Approaches
- CBR for good counterfactuals stores sparse, plausible, diverse, and feasible examples, then retrieves, reuses, and revises a nearby explanation case until the prediction changes.Its explanatory coverage and counterfactual potential are properties intended to promote good counterfactuals.Source: §6.6 · p. 35
6.7. Probabilistic-Centric Approaches
- PRINCE [111] searches a heterogeneous user-item network for behavior subsets that cause recommendation changes, using Personalized PageRank and reverse local push for polynomial-time computation.In Amazon and Goodreads experiments, simpler heuristics failed to find the best explanations, whereas PRINCE guaranteed optimality and outperformed baselines in interpretability user studies.Source: §6.7 · p. 36
- ABELE uses adversarial autoencoders to generate training-like image counterfactuals and outperformed state-of-the-art methods such as LIME in coherence, stability, and fidelity.It builds a latent neighborhood, learns local decision and counterfactual rules, selects and decodes exemplars, and extracts saliency maps.Source: §6.7 · p. 37
- CRUDS uses conditional subspace variational autoencoders to alter classification-relevant latent features, and its counterfactuals preserved true covariate dependencies on all but one evaluated dataset.The evaluation covered seven synthetic and three real datasets.Source: §6.7 · pp. 37–38
- Recourse is the only model-agnostic algorithm described as using Pearl’s causal probabilistic framework and structural causal models to generate counterfactuals through do-calculus interventions.Recourse focuses on actionable changes by optimizing a personalised cost function rather than merely minimizing distance.Source: §6.7 · p. 38
- Karimi et al. [117] proposed two probabilistic approaches that relax strong structural causal model assumptions: Bayesian averaging with additive Gaussian noise and assumption-free average recourse effects.These approaches address the difficulty of extracting a structured causal model from observed data, since relying on an imperfect partial model can produce incorrect counterfactuals.Source: §6.7 · p. 38
- Recourse produced more reliable synthetic-data recommendations under a partial structural causal model than other non-probabilistic approaches.The experiments evaluated the methods on synthetic data.Source: §6.7 · p. 38
6.8. Summary
The review finds that most model-agnostic counterfactual XAI methods lack a formal causal foundation and therefore rely on spurious correlations, risking sub-optimal, erroneous, or biased explanations. It also reports that causal information improves user decisions only when users lack prior task knowledge.
Review scope
- The review systematically analyzed counterfactual algorithms according to their underlying theories, algorithms, applications, and properties, as summarized in Table 2.Table 2 classifies collected model-agnostic counterfactual algorithms by theoretical backgrounds, properties, and applications.Source: §6.8 · pp. 38–39, Table 2 · p. 39
User studies
- Causal information improved users’ decisions when they lacked prior knowledge or beliefs about the task, but did not improve decision quality for users with existing domain knowledge.Zheng et al. [119] reported mixed results across these user conditions.Source: §6.8 · p. 38
Findings
- Most counterfactual XAI approaches lack Pearl’s formal causal framework and rely on spurious correlations rather than cause-effect relationships, producing potentially sub-optimal, erroneous, or biased explanations.This inability to disentangle correlation from causation limits the causal quality of explanations delivered to decision-makers.Source: §6.8 · pp. 38–39
7. Counterfactual Approaches to Explainable AI: Applications
Counterfactual explanations are applied across medical decision support, loan credit evaluation, and interactive visual analytics, with tools enabling users to explore model decisions and actionable changes. These applications address the need for causal understanding, although learning causal relationships from observational medical data remains difficult.
Medical decision support
- Counterfactual explanations are especially needed in medical decision support, where structured causal models could support experts amid small labelled datasets and heterogeneous data integration challenges.However, learning causal relationships from observational data is very difficult.Source: §7 · pp. 39–40
Loan credit evaluation
- Loan credit evaluation uses model-agnostic counterfactual explainers to produce compact, intelligible explanations from feature-importance weights for end users.Related finance applications explored algorithmic explanations through model-agnostic counterfactual explanations.Source: §7 · p. 40
Interactive visual analytics
- The What-IF tool lets practitioners probe, visualize, and analyze machine-learning systems while exploring decision boundaries and how data changes affect predictions.It is designed for investigation with minimal coding.Source: §7 · p. 40
- ViCE generates instance-centric counterfactuals for home-equity credit decisions, helping users contextualize models and interactively explore data and predictions.The tool highlights counterfactual explanations in an interactive visual analytics interface.Source: §7 · p. 40
- DECE generates instance- and subgroup-level counterfactuals, allowing users to seek more actionable alternatives and supporting decision-exploration and instance-explanation tasks.Its user interaction distinguishes it from ViCE by enabling counterfactual refinement to suit user needs.Source: §7 · p. 40
8. Towards Causability: Opportunities for Research
The review finds causability research scarce and argues that XAI systems must move beyond loss minimization toward causal, counterfactual, human-centric, interactive, and semantically grounded explanations.
8. Towards Causability: Opportunities for Research
- Only one recent article proposed an explanation framework, FATE, based on causability; it focuses on human interaction and examines explainability’s effects in terms of user trust, without formal causal theory.Source: §8 · pp. 40–41
8.1. The Main Characteristics of a Causability System •
- Explanations should adapt to users through human-in-the-loop interaction, contextual knowledge, and domain-specific information, enabling a symbiotic human–counterfactual generation process.Domain experts could interact with interfaces by asking what-if questions to inspect explanatory factors and construct mental representations.Source: §8 · p. 41, §8.1 · p. 42
8. Towards Causability: Opportunities for Research
- Causability cannot be reduced to minimizing a loss function; explanations must be human-centric and generate human-understandable mental representations.The review identifies human understandability as requiring more than a simplistic objective optimization process.Source: §8 · p. 41
8.1. The Main Characteristics of a Causability System •
- Current model-agnostic XAI algorithms lack formal causal foundations, so causability systems should expose cause/effect relationships through causal explanations.Causal information may improve decisions for users without prior task knowledge but not for users with existing domain knowledge, and evidence remains mixed.Source: §8.1 · pp. 41–42
- Causability explanations should be counterfactual and grounded in formal causal theory, while also satisfying diversity, feasibility, and plausibility.Counterfactual reasoning supports causal judgments and helps users understand alternative events.Source: §8.1 · p. 42
- Causability frameworks should support causal inference queries and semantic annotations using contextual knowledge, story models, and narratives to produce believable, human-understandable explanations.Interactive queries can increase transparency, while semantic narratives help convert sub-symbolic information into explanations and promote causability [159].Source: §8.1 · pp. 42–43
9. Answers to Research Questions
This section addresses research questions by examining the theoretical grounding and causability of counterfactual XAI algorithms, alongside the development of accessible, causal, and interactive explanation systems for diverse users.
9.1. RQ1 & RQ2: What are the main theoretical approaches and algorithms for counterfactuals in XAI?
- The review classifies counterfactual algorithms into instance-, constraint-, genetic-, regression-, game-theory-, case-based-, and probabilistic-centric approaches with differing abilities to satisfy plausibility and diversity.Constraint-centric, genetic-centric, case-based, and probabilistic approaches tend to support plausibility and diversity, whereas instance-, regression-, and game-theory-centric approaches face greater difficulties.Source: §9.1 · p. 43
- Most model-agnostic counterfactual algorithms lack causal theoretical grounding, producing spurious correlations rather than causal relationships and potentially sub-optimal, erroneous, or biased explanations.Recourse [117] is the closest identified approach to causal counterfactuals, but extracting structured causal models from observational data remains challenging.Source: §9.1 · p. 44
9.2. RQ3: What are the su fficient and necessary conditions for a system to promote causability (Applications)?
- Causability systems should combine formal causal theories, counterfactual explanations, human-centric adaptation, user interaction, and semantic annotations containing contextual and domain knowledge.Explanations should be specific to users’ needs, and interaction can improve understanding and interpretability of an XAI system.Source: §9.2 · p. 45
9.3. RQ4: What are the pressing challenges and research opportunities in XAI systems that promote Causability?
- Key research opportunities include causal XAI despite difficult observational-data discovery, standardized evaluation protocols across objective, functional, and user levels, and intelligent interfaces for interactive explanations.Graphical interfaces can support visual exploration, probabilistic inference, and pattern recognition, while user-centric evaluation should consider understanding, persuasive power, and task performance.Source: §9.3 · pp. 45–46
- Intelligent Exploration Interfaces aim to make model explanations accessible to non-expert users.Source: §9.3 · p. 46
- Causality-based predictive and interactive models are described as inherently interpretable and self-contained.Source: §9.3 · p. 46
- Outcome Explorer is a causality-guided interactive interface enabling experts and non-experts to acquire a comprehensive understanding of models.Source: §9.3 · p. 46
10. Conclusion
The review proposes a six-part taxonomy of model-agnostic counterfactual algorithms and finds that they lack formal causal grounding, limiting their ability to provide causally understandable explanations. It identifies causal inference, counterfactuals, probabilistic graphical models, and multimodal interaction as directions for causable XAI and next-generation intelligent user interfaces.
Taxonomy
- The review introduces a taxonomy of model-agnostic counterfactuals spanning six approaches: instance-, constraint-, genetic-, regression-, game-theory-, and case-based-reasoning-centric methods, plus probabilistic-centric methods.These categories extend the literature’s organization of model-agnostic counterfactual algorithms.Source: §10 · pp. 46–47
Causal grounding
- Model-agnostic counterfactuals lack the formal, structured causal theory proposed by, so they cannot reliably promote causal understanding to users.Their explanations identify feature relationships through correlation rather than causation.Source: §10 · p. 47
Limitations
- Because current systems rely on correlations, their explanations risk being biased, sub-optimal, or erroneous.The limitation follows from deriving relationships between features through correlation rather than causation.Source: §10 · p. 47
Future directions
- The review calls for causable XAI grounded in causal inference, counterfactual reasoning, and probabilistic graphical models.It frames causal models as a growing research area offering opportunities for XAI approaches based on probabilistic theories of causality and graphical models.Source: §10 · p. 47
Intelligent user interfaces
- XAI grounded in robust causal models could become a cornerstone of next-generation intelligent user interfaces through storytelling, argumentation, and multimodal interaction.The proposed combination aims to support persuasive engines and the rhetoric of causability.Source: §10 · p. 47
Related papers
This paper cites
- A Unified Approach to Interpreting Model Predictions2017
- Towards A Rigorous Science of Interpretable Machine Learning2017
- Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI2019
- Explaining Explanations: An Overview of Interpretability of Machine Learning2018
- Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes2017
- Bias and Debias in Recommender System: A Survey and Future Directions2020