PaperlayerView exact revision on arXiv

Source-linked AI summary

Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties

Lisa Schut, Oscar Key, Rory McGrath, Luca Costabello, Bogdan Sacaleanu, Medb Corcoran, Yarin Gal

arXiv:2103.08951v1cs.LGstat.AP

TL;DR

Counterfactual explanations can help users understand why classifiers make particular predictions, but their similarity to adversarial examples makes interpretability an important distinction. The paper uses classifier predictive uncertainty to generate realistic, unambiguous explanations without an auxiliary model, and reports that its method matches or exceeds existing methods on realism while generating more interpretable explanations. The approach requires a classifier that provides uncertainty estimates, which may increase computational cost.

  • Problem

    Counterfactual explanations seek nearby inputs with desired classifications, but interpretability distinguishes them from adversarial examples and existing definitions vary.

    Source: §1 · p. 1

  • Method

    The method uses classifier predictive uncertainty to generate interpretable counterfactuals, targeting realism and unambiguity without an auxiliary generative model.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    The method matches or exceeds existing methods on realism and generates more interpretable counterfactual explanations, according to the reported evaluation.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The method offers a fast way to generate realistic, unambiguous, and minimal counterfactual explanations without relying on an auxiliary generative model.

    Source: §6 · p. 9

    The method requires a classifier that estimates epistemic and aleatoric uncertainty, which may increase computational cost.

    Source: §4.1 · p. 7

Abstract

from arXiv · show

Counterfactual explanations (CEs) are a practical tool for demonstrating why machine learning classifiers make particular decisions. For CEs to be useful, it is important that they are easy for users to interpret. Existing methods for generating interpretable CEs rely on auxiliary generative models, which may not be suitable for complex datasets, and incur engineering overhead. We introduce a simple and fast method for generating interpretable CEs in a white-box setting without an auxiliary model, by using the predictive uncertainty of the classifier. Our experiments show that our proposed algorithm generates more interpretable CEs, according to IM1 scores, than existing methods. Additionally, our approach allows us to estimate the uncertainty of a CE, which may be important in safety-critical applications, such as those in the medical domain.

1 INTRODUCTION

Counterfactual explanations can help users understand decisions and identify changes that would alter them, but they must be interpretable rather than merely class-changing. This paper defines interpretability through realism and unambiguity, and proposes generating such explanations by minimizing classifier uncertainty without an auxiliary generative model.

1 INTRODUCTION

  • Counterfactual explanations seek a nearby input with a desired classification, making them conceptually similar to adversarial examples, but interpretability distinguishes them.The paper notes that interpretability itself has varying definitions in existing literature.Source: §1 · p. 1
  • The paper defines an interpretable counterfactual as realistic—a likely scenario—and unambiguous—not a pathological borderline case.Figure 1 contrasts an unrealistic example, an ambiguous example, and a realistic, unambiguous output from the proposed algorithm.Source: §1 · p. 1, Figure 1 · p. 2
  • Existing approaches often focus on realism without ambiguity and may rely on auxiliary generative models that fit some datasets poorly and add engineering overhead.The paper identifies these models as a potential bottleneck for complex datasets.Source: §1 · p. 2
  • The proposed approach uses epistemic and aleatoric predictive uncertainty to capture realism and ambiguity, respectively.The authors associate low epistemic uncertainty with greater likelihood under the data distribution and low aleatoric uncertainty with less ambiguity.Source: §1 · p. 2
  • The method uses a classifier that estimates both uncertainty types and does not require an auxiliary generative model.The authors say existing neural classifiers can be extended to represent uncertainty, for example with Monte Carlo dropout (Gal and Ghahramani, 2016).Source: §1 · p. 2
  • The method generates more interpretable counterfactual explanations than existing methods, according to the paper’s qualitative and quantitative evaluation.The authors report this result despite not requiring an auxiliary model.Source: §1 · p. 2

2 CE DESIDERATA

The paper sets out a framework of counterfactual-explanation desiderata, while acknowledging that the list is subjective and incomplete. It emphasizes realism and unambiguity as distinguishing qualities, alongside minimality, actionability, and speed.

2 CE DESIDERATA

  • A counterfactual explanation is the alternative input x′, while the counterfactual perturbation is the change Δ from original input x.The relation is expressed as x′ = x + Δ.Source: §2 · p. 2
  • The paper’s desiderata are a considered framework, not a complete or objective list of every property an explanation might need.The authors note that some desiderata are omitted and refer readers to Lipton (2018) for broader discussion.Source: §2 · p. 2

Minimal Perturbation

  • Minimal perturbations keep the alternative instance similar to the original, producing concise explanations and avoiding information overload.When two perturbations reach the desired outcome, the paper favors the more concise one.Source: §2 · pp. 2–3

Realistic Explanation

  • A realistic explanation represents a possible world and a scenario the user can understand.The paper also notes that feature values should be realistic in combination, such as a plausible bedroom-to-bathroom ratio.Source: §2 · p. 3

Unambiguous Explanation

  • Unambiguous explanations avoid borderline cases that resemble multiple classifications, supporting explanations that people can understand and learn from.The paper illustrates ambiguity with an input that could be read as either a 4 or a 9.Source: §2 · p. 3

Realistic or Actionable Perturbation

  • A realistic counterfactual may still fail to offer recourse if the user cannot apply its suggested change, such as relocating a house.The paper distinguishes actionable perturbations from merely realistic explanations.Source: §2 · p. 3

Run Time of the Algorithm

  • Interactive recourse settings make runtime important because generation often involves non-convex optimization and repeated evaluations of potentially expensive models.The paper notes that other computational properties, including memory use, also matter.Source: §2 · p. 3

2 CE DESIDERATA

  • The method explicitly targets realism and unambiguity through its loss function and addresses the remaining desiderata implicitly through its optimization procedure.The authors identify realism and unambiguity as particularly important because they distinguish counterfactual explanations from adversarial examples.Source: §2 · p. 3

3 METHODOLOGY

The methodology uses a classifier’s predictive uncertainty to guide interpretable counterfactual explanations, then implements that objective with a fast greedy algorithm. It treats realism and unambiguity as distinct targets and describes optional adversarial training and feature constraints.

3 METHODOLOGY

  • The post-hoc method generates minimal counterfactuals by implicitly minimizing epistemic and aleatoric uncertainty through a fast, greedy search.It is applied to trained classifiers and does not require an auxiliary generative model.Source: §3 · p. 3

3.1 Uncertainty as a Proxy for Realism and Unambiguity

  • For realism, the method seeks counterfactuals likely under the target class’s training distribution, using low epistemic uncertainty as a proxy; low aleatoric uncertainty targets unambiguity.The target-class distribution matters: a counterfactual should look realistic for its proposed class.Source: §3.1 · p. 4

3.2 Uncertainty in Practice

  • Predictive entropy captures both uncertainty types, and minimizing cross-entropy also minimizes predictive entropy, allowing the method to drop its separate uncertainty penalty.The simplification avoids minimax optimization over λ and tuning Λ.Source: §3.2 · pp. 4–5

3.3 A Greedy Algorithm for Generating Minimal CEs

  • The greedy algorithm adapts JSMA by changing the feature with the largest gradient of the target-class loss, using δ-sized updates as a heuristic for minimizing L0 distance.It stops at high target-class confidence or the iteration limit; users can lock features they know are immutable, though this assumes that knowledge.Source: §3.3 · p. 5

3.4 Adversarial Training

  • The method requires uncertainty estimates and gradients; adversarial training is an optional way to improve realism, with gradients reported as more aligned with human perception.The proposed rationale includes improved uncertainty estimates and more robust, less noise-focused features.Source: §3.4 · pp. 5–6, Figure 2 · p. 6

4 RELATED WORK

The related work covers counterfactual-generation objectives, their connection to adversarial examples, and approaches to evaluating realism and minimality. The paper contrasts auxiliary-model methods with its uncertainty-based approach and explains its metric choices.

4.1 Generating CEs

  • Prior counterfactual methods use autoencoders to encourage manifold realism or class prototypes, while another approach searches latent space through a generative model.These approaches add auxiliary models; the prototype objective also requires multiple scaled terms, and generative-model outputs may inherit model pathologies such as blur.Source: §4.1 · p. 6
  • The proposed approach avoids auxiliary-model overhead and minimax optimization or extra hyperparameters, but requires a classifier that estimates uncertainty, which can increase computation.The experimental ensemble is more expensive to train and evaluate than a single model.Source: §4.1 · pp. 6–7

4.2 Adversarial examples

  • Counterfactual and adversarial examples both seek minimal classification-changing perturbations; interpretability distinguishes the paper’s counterfactual goal.Adversarial examples need not be interpretable.Source: §4.2 · p. 7

4.3 Evaluation metrics

  • The evaluation reports L1 distance for minimality and uses IM1 to evaluate realism, omitting IM2 because it fails a sanity check on out-of-distribution data.IM1 and IM2 are autoencoder-reconstruction-based metrics; IM2 scores were not significantly different for junk and in-distribution data.Source: §4.3 · p. 7
  • Human evaluation provides ground-truth interpretability judgments but is not automated or easy to scale for frequent hyperparameter evaluation.The paper therefore does not use this approach.Source: §4.3 · p. 7
  • The vulnerability metric counts justified counterfactuals, but its neighborhood-based algorithm does not scale to high dimensions and its link to human interpretability is unclear.The paper does not use this approach.Source: §4.3 · p. 7

5 EMPIRICAL ANALYSIS

Across MNIST, Breast Cancer Wisconsin, and Boston Housing, the evaluation compares the proposed method with VLK and JSMA on realism and perturbation size, then ablates adversarial training and ensemble size on MNIST.

5.1 Datasets

  • The evaluation covers MNIST images and two tabular datasets: Breast Cancer Wisconsin and Boston Housing.The classifiers’ reported accuracies are 98.5% on MNIST, 96.9% on Breast Cancer validation data, and 86.3% on Boston Housing validation data.Source: §5.1 · p. 8

5.2 Compared Methods

  • The benchmarks include Van Looveren and Klaise (2019) as a state-of-the-art CE approach and JSMA as a baseline for interpretability and minimal perturbations.The authors use JSMA to assess whether their method generates more interpretable CEs and to check that JSMA efficiently produces minimal perturbations.Source: §5.2 · p. 8

5.3 Results

  • The method achieves lower IM1 scores than Van Looveren and Klaise (2019), indicating more realistic CEs without an auxiliary generative model.Table 1 reports realism using IM1, for which lower is better.Source: §5.3 · p. 8, Table 1 · p. 8
  • Against JSMA, the method produces larger perturbations but better IM1 scores, trading perturbation size for realism.Figure 3 shows qualitative examples; the passage describes the proposed method’s CEs as more realistic.Source: §5.3 · p. 8, Figure 3 · p. 9
  • The proposed method’s MNIST counterfactual strokes are less smooth than real images, although they capture high-level changes such as adding a stroke to turn a 4 into a 9.The authors attribute the lack of smoothness to iterative single-pixel changes and say stylistic properties matter if using the method as a generative model.Source: §5.3 · p. 9

5.4 Ablation Study

  • On MNIST, interpretability initially improves with more ensemble components but saturates after 10; adversarial training improves interpretability while reducing sparsity.Figure 4 reports IM1 and mean L1 distance, with shaded 95% confidence intervals estimated over 10 seeds.Source: §5.4 · p. 9, Figure 4 · p. 9

6 CONCLUSION

The conclusion presents a fast method for generating realistic, unambiguous, and minimal counterfactual explanations without an auxiliary generative model. Empirically, it matches or exceeds existing methods on realism, while adapting it to black-box models is left for future work.

6 CONCLUSION

  • The authors introduce a fast method for generating realistic, unambiguous, and minimal counterfactual explanations.They report that the method does not rely on an auxiliary generative model, reducing engineering overhead.Source: §6 · p. 9
  • The method matches or exceeds existing methods with respect to the realism of generated counterfactual explanations.Source: §6 · p. 9
  • Adapting the proposed method to work with black-box models is identified as future work.Source: §6 · p. 9

A DERIVATIONS

The derivation relates minimizing targeted cross-entropy to minimizing predictive entropy. The accompanying algorithm iteratively changes salient input features until the target-confidence threshold or iteration limit is reached.

A DERIVATIONS

  • For a classification model, minimizing the targeted cross-entropy objective yields an input that also minimizes predictive entropy.The derivation considers a single input and target class, assumes positive class probabilities, and states that the proof extends to multiple observations.Source: A DERIVATIONS · p. 12

B COUNTERFACTUAL EXPLANATION GENERATION ALGORITHM

  • The algorithm takes an input, target class, model ensemble, confidence threshold, perturbation size, and iteration limits, and returns a counterfactual.Source: B COUNTERFACTUAL EXPLANATION… · p. 13
  • It repeatedly updates the most salient eligible feature by δ, clips the result to the permitted range, and stops at the target-confidence threshold or maximum iterations.A feature can be changed no more than the specified maximum number of times.Source: B COUNTERFACTUAL EXPLANATION… · p. 13

C ADVERSARIAL TRAINING

This section motivates adversarial training as a possible route to more interpretable explanations. It describes training with adversarial examples, reported links to sparse and stable explanations, and a connection to uncertainty estimation.

C ADVERSARIAL TRAINING

D EVALUATION

The evaluation finds a significant in-/out-of-distribution difference for IM1, but not IM2, which the authors omit after it fails a sanity check. The MNIST experiments describe the evaluation setup and reproducibility details.

D EVALUATION

  • The evaluation omits IM2 after it fails a sanity check comparing in- and out-of-distribution images on MNIST.MNIST was used because it made visual verification easier.Source: D EVALUATION · p. 13
  • IM1 scores differ significantly between in- and out-of-distribution data, while IM2 scores do not at the 5% significance level.Figure 5 presents the IM1 and IM2 comparisons; IM2 is omitted from the evaluation after failing a sanity check.Source: D EVALUATION · pp. 13–14, Figure 5 · p. 14

E EXPERIMENT SETUP

  • Experiments were implemented in PyTorch, with repository instructions for reproducing results and generating example counterfactuals.The repository also includes environment setup and dependency versions.Source: E EXPERIMENT SETUP · p. 14

E.1 MNIST

  • MNIST classifiers used a 50-model ensemble, and target classes were randomly selected from class-specific eligible sets to avoid difficult transformations.For example, the procedure avoids transformations such as a 6 into a 2.Source: §E.1 · p. 14
  • For MNIST, each method-dataset pair used 100 test points across 10 random seeds, reporting mean IM1 and L1 distance.The reported aggregates are the mean and standard deviation of the per-seed means.Source: §E.1 · p. 15

E.2 Wisconsin Breast Cancer Dataset and Boston Housing Dataset

The tabular experiments describe their data split and classifier setup, while ensemble models outperform a single classifier by accuracy across confidence levels. Ensembling with adversarial training separates in- from out-of-distribution predictive entropy, and reproducing a prior method's reported scores remained difficult.

E.2 Wisconsin Breast Cancer Dataset and Boston Housing Dataset

  • The tabular experiments split each dataset 70/10/20 for training, validation, and testing, generating counterfactuals on the test set.The classifier uses an ensemble of 20 models.Source: §E.2 · p. 15
  • The authors' IM1 scores for Van Looveren and Klaise (2019) were worse than those in that paper and were not directly comparable because the test-set sizes differed.Repeating the experiment with 19 test points also did not reproduce the original results.Source: §E.2 · pp. 16–17

F.1.1 Calibration

  • Ensembles have higher accuracy than a single classifier as confidence varies, with Figure 6 showing ensemble sizes from 5 to 50.The confidence calibration experiment combines MNIST and FashionMNIST in varying proportions.Source: §F.1.1 · p. 17, Figure 6 · p. 17

F.1.2 Out-of-Distribution Uncertainty

  • A single model does not distinguish in- from out-of-distribution inputs, whereas ensembling with adversarial training yields clearly separated predictive entropy.The comparison uses breast-cancer test samples and tabular data sampled from a normal distribution.Source: §F.1.2 · p. 17, Figure 7 · p. 18

F.2 Qualitative Analysis

The qualitative examples show counterfactuals capturing high-level image changes, while the feature analysis illustrates pixel edits and reports that interpreting tabular feature patterns requires expert knowledge.

F.2 Qualitative Analysis

  • For MNIST, the method proposes adding a middle stroke and removing parts of a 0 to form a 3; both changes are needed for a realistic result.The examples show that high-level changes can suffice for explanatory purposes, although stylistic properties are not captured.Source: §F.2 · p. 18, Figure 8 · p. 19
  • FashionMNIST examples depict high-level changes such as adding a split to a dress or a zipper to a coat.Figure 9 shows dress-to-pants and pullover-to-coat examples.Source: §F.2 · p. 19, Figure 9 · p. 20

F.3 Feature Analysis

  • For a 0-to-6 counterfactual, the illustrated edits remove pixels at the top right and add a white diagonal stroke; the proposed changes appear realistic.Figures 10 and 11 show average inputs, counterfactuals, and changes.Source: §F.3 · p. 20, Figure 10 · p. 20, Figure 11 · p. 21
  • Interpreting the most frequently changed cell-nuclei features and average perturbation sizes requires expert knowledge.The authors identify this interpretation as future work.Source: §F.3 · p. 20, Figure 12 · p. 21, Figure 13 · p. 21
Loading 2103.08951v1…