Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review

1 INTRODUCTION

• Human-understandable explanations can help people understand decisions, challenge unfair treatment, act toward a desired outcome, and support transparency, bias checks, and legal compliance.They can also help developers identify and fix bugs and performance issues.Source: §1 · p. 1
• Unlike explanations that answer why a prediction was made, CFEs suggest feature changes that could move a person toward a desired outcome, providing algorithmic recourse.For a denied loan applicant, the feedback could suggest changes to help cross the decision boundary and obtain a loan.Source: §1 · p. 2
• The review examines more than 350 recent CFE algorithms using shared parameters and desiderata to compare their assumptions and approaches.It categorizes the papers to make their advantages and disadvantages easier to compare.Source: §1 · p. 2
• CFEs can work with black-box models without model disclosure or restrictions on model complexity, and they need not approximate the underlying model to provide feedback.The review notes that CFEs’ intuitive nature also makes them amenable to legal frameworks.Source: §1 · p. 2
• The review’s categorization helps researchers and developers choose an algorithm suited to their assumptions and desired generation speed and quality.It also provides an introduction to counterfactual explanations for beginners.Source: §1 · p. 2

2.1 Social Implications of Machine Learning

• Fairness and explainability are broad ways to address machine learning’s social implications, including bias tied to protected features such as race, sex, and religion.Fairness research includes multiple demographic- and individual-capacity-based notions; prior work reviews these definitions and research categories, while explainable machine learning has also drawn interest in healthcare.Source: §2.1 · p. 2

2.2 Explainability in Machine Learning

• Classifier failures involving tank backgrounds and snow revealed that models can rely on unintended cues, and explainability techniques helped uncover these issues.The tank classifier performed well on training and test data but poorly on the battlefield; a husky was classified as a wolf because of snow in the background.Source: §2.2 · p. 2
• Explainability distinguishes global model explanations from explanations of specific outcomes; this work focuses on counterfactual explanations as an example-based outcome approach.Model explanations may use trees, rule sets, or model-agnostic methods, while outcome explanations include feature attribution, model simplification, and example-based approaches.Source: §2.2 · p. 2
• Counterfactual explanations apply to supervised learning when a datapoint lacks the desired prediction, and most prior work has studied classification with discrete labels.The passage defines the learned function as mapping feature inputs to labels for unseen datapoints.Source: §2.2 · pp. 2–3

2.3 History of Counterfactual Explanations

• Counterfactual explanations build on philosophical, psychological, and social-science work, and user studies found preferences for them over case-based reasoning [33].Psychology research links counterfactuals to causal reasoning; Fernández-Loría et al. [98] describe cases where feature-importance methods miss model behavior that CFEs capture, and Asher et al. argue CFEs are epistemically accessible.Source: §2.3 · p. 3

3.1 An Example

• A CFE tells a denied applicant what to change to reach the desired decision, unlike an explanation of why the original decision occurred.The future outcome is guaranteed only if the classifier does not change before the applicant reapplies.Source: §3.1 · p. 3

3.2 Desiderata and Major Themes of Research

• A CFE must reach the desired class while minimizing change, respecting immutable features and user preferences, and favoring sparse edits that are easier to understand.The optimization balances validity against distance from the original input; distance measures can include L1/L2, quadratic, CDF-based, or user-perceived feature costs.Source: §3.2 · pp. 3–4
• Realistic CFEs should stay near training data and preserve observed correlations, because real-world changes to one feature can affect others.Figure 1 contrasts the shorter red path outside the data manifold with the longer green path that follows it.Source: Figure 1 · p. 3, §3.2 · p. 4
• Algorithms also vary in delivery: amortized methods avoid per-input optimization, while black-box access and model agnosticity support use across models.Mahajan et al. [210] and Verma et al. propose amortized approaches; Dandl et al. [67] and Verma et al. propose black-box approaches.Source: §3.2 · p. 4

3.3 Relationship to other related terms

• The review uses CFEs and recourse interchangeably, while inverse classification shares their goal; pertinent negatives connect contrastive explanations, but adversarial examples remain distinct.Ustun et al. note that recourse accounts for actionability, a distinction reinforced by causal approaches [169]; adversarial methods often seek imperceptible changes rather than CFE sparsity, plausibility, and actionability.Source: §3.3 · pp. 4–5

4.1 Properties of counterfactual algorithms

• Algorithms differ in model access: some require model internals, gradient-based methods need differentiable models, and black-box methods use gradient-free optimization or heuristics.The review also describes methods that alter training or combine classification and counterfactual generation in a learned architecture.Source: §4.1 · p. 5
• Model agnosticism depends on the approach: black-box methods can work with any underlying model, while gradient-based, solver-based, and model-specific methods have restrictions.Gradient methods require differentiable models, solver methods require linear or piece-wise linear models, and some approaches work only for particular model types.Source: §4.1 · p. 5
• Most methods generate one counterfactual per input, while some produce several per input and amortized methods avoid separate optimization across inputs.The review distinguishes amortized inference across multiple inputs from generating multiple counterfactuals for a single input.Source: §4.1 · pp. 5–6
• Counterfactual attributes cover sparsity, data-manifold adherence, and causal relations, addressed through varied constraints, data-distribution models, and causal graphs.Some causal approaches require a complete causal graph, while Duong et al., Mahajan et al. [210], Verma et al., and Yang et al. can use partial graphs.Source: §4.1 · p. 6
• Optimization-problem attributes include feature preferences and categorical distance, with methods varying in how they handle actionable features and categorical variables.Actionable features are a subset of mutable features; categorical handling includes filtering, one-hot encoding, separate distance functions, and methods that naturally handle categories.Source: §4.1 · p. 6

5.1 Commonly used datasets for evaluation

• Evaluation uses image and tabular datasets, while excluding paper-specific synthetic datasets that may be unavailable.Examples include MNIST for images and COMPAS and MIMIC-IV for tabular data.Source: §5.1 · pp. 6–7

5.2 Metrics for evaluation of counterfactual generation algorithms

• Validity, proximity, sparsity, and generation time assess whether counterfactuals reach the target, remain close to the input, change few features, and can be produced efficiently.Proximity commonly uses L1, L2, or Mahalanobis distance; lower average distance is preferred. No consensus hard cap exists for sparsity, though Keane and Smyth use at most two feature changes.Source: §5.2 · p. 8
• Diversity measures whether multiple recommendations offer different options, while training-data closeness and causal constraint satisfaction assess realism and feasibility.Diversity can be encouraged through optimization objectives, hard constraints, or mutual-information minimization; closeness is measured using methods such as nearest-neighbor distance, local outlier factor, VAE reconstruction error, KDE, or MMD.Source: §5.2 · p. 9
• Autoencoder-specific IM1 and IM2 assess class-manifold interpretability, while Label Variation Score and Oracle Score target out-of-distribution CFEs that conventional metrics may miss.Lower IM1 and IM2 indicate greater interpretability; Oracle Score checks validity with a second classifier. Some reviewed papers report only example CFEs rather than these metrics.Source: §5.2 · p. 9

5.3 Other works

• Related work proposes desirable CFE properties, critiques unjustified explanations and hidden assumptions, and develops causal approaches that encode feature relations or preferences.Sokol and Flach discuss desirable properties; Laugel et al. [190] [192] address properties and training-data grounding; Downs et al. use CSVAE, while Crupi et al. propose a causality technique usable with other approaches.Source: §5.3 · pp. 9–10
• Other approaches target specific models, multi-agent interests, or global counterfactuals, including Bayesian-network methods, a Stackelberg-game formulation, and rule-list explanations.Examples include Bayesian-network CFE methods, a top-k joint-utility approach, and AReS for global rule-list CFEs.Source: §5.3 · p. 10
• Later work modifies earlier generation methods—for example, RL and genetic-algorithm variants—and benchmarks 10 approaches across 22 datasets and nine metrics.Extensible toolboxes allow approaches to be plugged in and compared.Source: §5.3 · p. 10
• Additional work addresses real-world feature constraints, action costs, prediction-error bounds, inverse optimization, predictive multiplicity, multiclass CFEs, and brute-force generation.Examples include disentangling actions from feature changes, bounding CFE cost under predictive multiplicity, and hierarchical multiclass decomposition.Source: §5.3 · p. 10

• The section directs readers to work on counterfactual explanations for agent actions, such as reinforcement learning or planning [39, 237].Source: §6 · p. 10
• A survey addresses counterfactual explanations for graph neural networks.Source: §6 · p. 10
• The section points to work on functional data and behavioral data.Source: §6 · p. 10

Anomaly and data-drift detection

• CFEs explain data drift, time-series and autoencoder-detected anomalies, and feature changes that could increase a classifier’s confidence [140, 290, 135, 15].Ravi et al. survey explainability techniques for convolutional autoencoders used in image anomaly detection.Source: §7 · pp. 10–11

Training dataset debugging

• CFEs are proposed to debug ML models and datasets by diagnosing behavior, adjusting decision boundaries with synthetic data, finding mislabeled data, and locating bugs or influential training examples.Source: §7 · p. 11

Data augmentation

• CFEs augment training data for market-volatility prediction, class imbalance, tabular robustness, and crop-growth prediction under climate change.Source: §7 · p. 11

Drug designing

• Multi-agent reinforcement learning with CFEs identifies changes to drug and protein molecules that increase their affinity.Source: §7 · p. 11

ML model bias detection

• CFEs are also proposed for ML model bias detection [94].Source: §7 · p. 11

Various applications

• Other applications span employment advice, social-network community detection, dimensionality reduction, student support, testing, mental-health reasoning, recommender evaluation, constraint solving, entity resolution, model comparison, and LLM evaluation.Mazzine et al. address job-seeker advice and bias detection; Sadler et al. community detection; Fujiwara et al. [108] interactive dimensionality reduction; Tsiakmaki and Ragos student performance; Cong et al. [63] Kolmogorov-Smirnov test failures; Marchezini et al. mental health; Yao et al. recommender explanations; Gupta et al. unsolvable constraint problems; Teofili et al. entity resolution; Artelt et al. [21] differences between models; Frohberg and Binder [107] introduce CRASS for LLM reasoning and language understanding.Source: §7 · p. 11

Real-world deployment

• Hired is reported as a real-world CFE deployment: a GAN-based approach suggests changes to salary expectations, experience, and skills to improve candidates’ approval prospects.Source: §7 · p. 11

Research Challenge 1. Unify counterfactual explanations with traditional “explainable AI.”

• Counterfactual explanations offer actionable feedback but do not identify the principal reasons for an original decision, motivating efforts to combine them with feature-attribution explanations.Progress includes methods based on necessity and sufficiency, feature importance, and attribution scores, as well as using local decision trees or attribution methods to generate counterfactuals [126],,,,,, [10],,,.Source: §8 · p. 11

Research Challenge 2. Provide counterfactual explanations as discrete and sequential steps of actions.

• Because real-world actions are discrete and often sequential, counterfactuals should give ordered steps rather than only a modified target state.Proposed approaches include program synthesis, mixed-integer programming, and reinforcement learning; a user study found a preference for action sequences over a single-step goal,, [250],,.Source: §8 · pp. 11–12

Research Challenges 3–4. Extend counterfactual explanations beyond classification; provide them as an interactive service.

• Counterfactual methods are expanding beyond classification to regression, graph neural networks, and time-series models, while interactive services let users refine or explore explanations.Examples include Bayesian optimization for regression, reinforcement learning for graph neural networks, case-based reasoning for time-series classification [74], and interactive CFE interfaces or sequences [141],, [9],.Source: §8 · p. 12

Research Challenges 5–8. Incomplete causal graphs; missing feature values; scalability; classifier bias.

• Further challenges include handling incomplete causal graphs and missing feature values, improving scalability, and accounting for classifier bias across demographic groups.Some methods support partial causal graphs [210],,,,; amortized inference generates counterfactuals for new inputs after training [210],,,,,. Fairness work includes subgroup recourse rules and regularizers intended to support equal recourse,,,,.Source: §8 · p. 12

Research Challenges 9–11. Robustness; dynamics; applicant preferences.

• Counterfactual explanations should remain robust to model retraining, slight input changes between individuals, and minor changes to the suggested counterfactual; they should also account for data drift and applicant-specific preferences.Source: §8 · pp. 12–13

Research Challenges 12–15. What must not change; model privacy; fairwashing; engineered features.

• Explanations should disclose what must not change to avoid harmful outcomes, protect model privacy, and guard against explanations being used to obscure bias.CFE queries can enable model extraction, including a reported high-fidelity surrogate trained with as few as 1,000 queries [7]; whether fair-recourse methods can prevent fairwashing remains open [5], [6].Source: §8 · p. 13

Research Challenges 15–18. Engineered features; categorical features; user studies; visualization interfaces.

• Open design and evaluation needs include interpretable counterfactuals for engineered features, appropriate handling of categorical features, user studies, and visualization interfaces.Categorical-feature methods vary and no approach is clearly established; user studies report preferences for concrete, consistent, relevant explanations and specific directives, while several works develop graphical CFE interfaces,,,,,,,.Source: §8 · pp. 13–14

Research Challenges 19–21. Multi-agent recourses; non-manipulative improvement; regulatory ties.

• Further challenges are to generate recourses that account for multiple agents’ interests, discourage manipulative feature changes, and strengthen ties between machine-learning and regulatory communities.Proposals include distinguishing improvement, manipulative, and immutable features, using causality to prevent gamification, and adapting regulation and policy to explainable ML decisions.Source: §8 · p. 14

9 CONCLUSIONS

• The review covers more than 350 papers proposing algorithmic solutions for finding counterfactual explanations to machine-learning decisions.It evaluates the papers using the same rubric to clarify the advantages and disadvantages of different approaches.Source: §9 · p. 14
• The shared rubric helps organizations choose algorithms suited to their application constraints by making approaches’ advantages and disadvantages easier to understand.Source: §9 · p. 14
• The review identifies gaps for researchers seeking open problems and offers an introduction to counterfactual explanations with resources for further exploration.Source: §9 · p. 14

A FULL TABLE

• Table 1 presents selected critical columns from the initially larger paper categorization; the full table is available here.The initial categorization used more columns and a much larger table.Source: A FULL TABLE · p. 23

B.1 How we collected the paper to review?

• The authors collected more than 350 papers for the review.The collection procedure is described as the exact process used to arrive at this set.Source: §B.1 · p. 23
• For the first survey version, they began with papers recommended by others [210] [250] and snowballed their references.Source: §B.1 · p. 23
• For the updated version, they collected papers citing Wachter et al.’s first ML counterfactual-explanations paper and the first survey version.Source: §B.1 · p. 23
• They searched Semantic Scholar and Google Scholar for “counterfactual explanations”, “recourse”, and “inverse classification”.Source: §B.1 · p. 23
• The search focused on papers from the preceding five years, a timeframe anchored to the 2017 publication that began ML counterfactual-explanation discussions for tabular data.Source: §B.1 · p. 23
• The collection included papers published before 31 May 2022 at conferences including KDD, IJCAI, FAccT, AAAI, WWW, NeurIPS, and WHI, as well as papers uploaded to arXiv.Source: §B.1 · p. 23

B.2 Scope of the review

• The review restricts its coverage to recent work on counterfactual explanations in machine-learning classification, inspired by FATE and legal explainability requirements.It does not claim the underlying idea is new: related work appeared in data mining [98], causal inference, software engineering, and earlier social sciences, philosophy, and psychology.Source: §B.2 · p. 23
• Most reviewed papers cite a 2017 paper as the seminal work that started discussion of counterfactual explanations.The review distinguishes that recent machine-learning discussion from earlier exploration of similar ideas in other fields.Source: §B.2 · p. 23

C BURGEONING LEGAL FRAMEWORKS AROUND EXPLANATIONS IN AI

• The GDPR, enforced from May 25, 2018, provides a right to information about automated systems’ existence, logic, and envisaged consequences, as well as a right not to be subject to automated decision-making.The closeness of these provisions to a “right to explanation” is debatable, but the Article 29 Working Party interpreted the GDPR as requiring explanations of specific decisions, making counterfactual explanations apt.Source: C BURGEONING LEGAL FRAMEWORKS AROUND… · p. 23
• In the US, ECOA and FCRA require creditors to give reasons for adverse actions such as loan rejection.They generally identify principal reasons by comparing an applicant’s features with population averages.Source: C BURGEONING LEGAL FRAMEWORKS AROUND… · p. 23
• UK and French government reports, US DARPA’s XAI program, and EU initiatives have also addressed AI explainability.DARPA launched XAI in 2016 to encourage work on explainable models, the psychological requirements of explanations, and explanation interfaces.Source: C BURGEONING LEGAL FRAMEWORKS AROUND… · p. 23
• The US Blueprint for an AI Bill of Rights outlines five principles, including explanations for automated decisions and discussion of human alternatives.Its other principles concern safe and effective systems, algorithmic discrimination protections, and data privacy.Source: C BURGEONING LEGAL FRAMEWORKS AROUND… · p. 23
• It remains unclear whether or how proposed explainable-machine-learning techniques can meet legal requirements, motivating collaboration among researchers, regulators, legal communities, and consumer watchdog groups.The paper presents such collaboration as a way to help ensure trustworthy AI.Source: C BURGEONING LEGAL FRAMEWORKS AROUND… · p. 23