Source-linked AI summary
Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations
Ramaravind Kommiya Mothilal, Amit Sharma, Chenhao Tan
TL;DR
Counterfactual explanations aim to help people act on machine-learning predictions, but useful explanations must be both feasible for the user and diverse in the alternatives they present. The paper proposes a framework to generate and evaluate such sets, and reports diverse counterfactuals that approximate local decision boundaries across four datasets, while identifying remaining causal and behavioral limitations.
Problem
Counterfactual explanations should help users identify actionable changes, but generated alternatives must balance diversity, proximity, and real-world feasibility.
Method
The paper generates sets of diverse counterfactuals using optimization with proximity, user-defined constraints, and causal-feasibility filtering, and evaluates them with quantitative metrics.
Results
Across four datasets, the method produces substantially more diverse counterfactuals than prior approaches and approximates local decision boundaries at least as well as LIME.
Takeaways & Limitations
Diverse and feasible counterfactual sets can provide users with multiple ways to change an outcome while retaining direct correspondence to the underlying model.
Takeaways & Limitations
The method assumes gradient knowledge, incorporates causal knowledge only through post-hoc filtering, and leaves behavioral validation for future work.
Abstract
from arXiv · showhide
Post-hoc explanations of machine learning models are crucial for people to understand and act on algorithmic predictions. An intriguing class of explanations is through counterfactuals, hypothetical examples that show people how to obtain a different prediction. We posit that effective counterfactual explanations should satisfy two properties: feasibility of the counterfactual actions given user context and constraints, and diversity among the counterfactuals presented. To this end, we propose a framework for generating and evaluating a diverse set of counterfactual explanations based on determinantal point processes. To evaluate the actionability of counterfactuals, we provide metrics that enable comparison of counterfactual-based methods to other local explanation methods. We further address necessary tradeoffs and point to causal implications in optimizing for counterfactuals. Our experiments on four real-world datasets show that our framework can generate a set of counterfactuals that are diverse and well approximate local decision boundaries, outperforming prior approaches to generating diverse counterfactuals. We provide an implementation of the framework at https://github.com/microsoft/DiCE.
1 INTRODUCTION
The paper argues that useful counterfactual explanations should show actionable alternative profiles, balancing diversity with proximity and causal feasibility. It proposes a framework and evaluation metrics, demonstrating strong diversity and local-model approximation across four datasets.
- Motivation: Counterfactual explanations show feature changes that could produce a different model outcome, making them more actionable than reasons that merely describe rejection.They provide direct what-if outputs from the underlying model.
- Motivation: Actionable counterfactual sets should balance diverse suggested changes, proximity to the original input, and causal constraints on feasible actions.Examples should account for relative ease of change and avoid implausible modifications such as changing race or lowering education.
- Framework: The proposed framework generates any number of diverse counterfactuals for differentiable classifiers while supporting custom feature weights and user-defined constraints.Its optimization explicitly addresses diversity–proximity and continuous–categorical feature tradeoffs.
- Evaluation: The paper introduces quantitative metrics for validity, diversity, proximity, and approximate behavioral understanding, while acknowledging that behavioral experiments remain necessary.The metrics support parameter tuning and comparison with local explanation methods such as LIME.
- Results: Across COMPAS, Adult-Income, German-Credit, and Lending Club, the method produces substantially more diverse counterfactuals than prior methods and locally approximates models comparably to LIME.It obtains higher F1 scores than LIME in most configurations, especially on Adult-Income and COMPAS.
- Results: Generated counterfactuals can reveal model biases, including cases where suggested changes involve sensitive attributes such as race or gender.The paper presents this as a consequence observed during qualitative inspection.
2 BACKGROUND & RELATED WORK
The background contrasts proxy-based local explanations with example-based counterfactuals. Proxy methods improve interpretability by approximating complex models, whereas counterfactuals present alternative perturbations that would change the prediction.
- Feature importance: Local explanation methods such as LIME fit simpler models around individual predictions to estimate feature importance for complex classifiers.Related approaches also use decision trees or unified feature-importance frameworks.
- Feature importance: Proxy-based explanations trade exact faithfulness to the original model against human interpretability because surrogate models can approximate the true decision surface.The paper characterizes this as a limitation of explanations sourced from simpler surrogates.
- Visualization: Visual explanations are common in computer vision but can be difficult to interpret for nonvisual decisions such as recidivism prediction and loan approvals.This motivates considering explanation formats beyond visual feature highlighting.
- Examples: Example-based approaches explain predictions through representative or alternative instances, including prototypes, criticisms, and counterfactual perturbations.Counterfactuals are framed as alternative inputs that would have changed the model’s prediction.
- Counterfactuals: Counterfactual objectives combine a term that pushes an example toward a different prediction with a term that keeps it close to the original instance.This formulation motivates extending counterfactual generation with diversity.
- Diversity: The paper extends counterfactual explanations from single alternatives toward diverse sets, drawing on evidence that diverse information items benefit search and recommendation users.Russell’s earlier approach targeted diversity for linear classifiers using integer programming.
3 COUNTERFACTUAL GENERATION ENGINE
The counterfactual generation engine seeks actionable sets by combining validity, proximity, diversity, sparsity, and user-defined feasibility constraints. It optimizes these objectives jointly for differentiable binary classifiers while handling practical feature-scaling and categorical-feature issues.
- The engine generates k counterfactuals that change the model’s binary decision while assuming a differentiable, static model.The original instance and counterfactuals are d-dimensional.
- Actionable counterfactual sets combine feasible individual examples with diversity that offers users multiple ways to change the outcome.Feasibility incorporates proximity and user-defined constraints, while diversity provides alternative changes.
- Diversity and Feasibility Constraints: DPP diversity uses the determinant of a kernel matrix to favor sets of counterfactuals that differ from one another.The kernel uses pairwise distances between counterfactual examples.
- Diversity and Feasibility Constraints: Proximity favors counterfactuals close to the original input, while sparsity favors changing fewer features and user constraints restrict feasible manipulations.Constraints can specify feature ranges or which variables may change.
- Optimization: The combined loss balances prediction validity, proximity, and DPP diversity through hyperparameters λ1 and λ2.The prediction term minimizes the distance between the model’s prediction for each counterfactual and the desired outcome.
- Optimization: Gradient descent optimizes randomly initialized counterfactuals for up to 5,000 steps or until convergence and validity.The objective is non-convex, so achieving the desired class for every counterfactual may not be possible.
4 EVALUATING COUNTERFACTUALS
The paper evaluates counterfactual sets using metrics for validity, proximity, diversity, and local decision-boundary understanding, while recognizing tradeoffs and the need for human evaluation.
- Limitations: Evaluation cannot replace behavioral experiments, and diversity and proximity cannot both be maximized, so metric priorities depend on the application domain.The authors frame their metrics as tools for fine-tuning counterfactual-generation parameters and motivating human studies.
- Validity, Proximity, and Diversity: Validity measures the fraction of unique generated examples that produce a different outcome from the original input.
- Validity, Proximity, and Diversity: Proximity is the mean feature-wise distance between each counterfactual and the original input, while sparsity counts changed features.Evaluation uses original feature scales for continuous-distance interpretability, unlike generation.
- Validity, Proximity, and Diversity: Diversity measures mean pairwise distances between counterfactual examples, with separate metrics for categorical and continuous features.A sparsity-based diversity metric additionally measures the fraction of features differing between pairs.
- Understanding the Local Decision Boundary: A secondary 1-nearest-neighbor classifier approximates user understanding by classifying points according to their proximity to the original input or counterfactual examples.Its accuracy is evaluated against the original model within local spheres sampled at radiuses of 0.5, 1, and 2 MAD.
- Datasets and Baselines: The four evaluation datasets are COMPAS, Adult-Income, German-Credit, and LendingClub, with differing mixtures of continuous and categorical features.Models use 80%-20% train-test splits, cross-validation, one-hot encoding for categorical features, and continuous-feature scaling.
- Datasets and Baselines: DiverseCF is compared with SingleCF, MixedIntegerCF, RandomInitCF, and related sparsity-corrected variants.DiverseCF optimizes both diversity and proximity, whereas MixedIntegerCF applies only to linear models.
5 EXPERIMENT RESULTS
DiverseCF produces more valid and diverse counterfactual sets than the evaluated baselines, while exposing a diversity–proximity tradeoff. Its counterfactuals also approximate local decision boundaries effectively, but causal constraints remain necessary for feasibility.
- Quantitative evaluation: DiverseCF generates nearly 100% valid counterfactuals across all four datasets, while baseline uniqueness declines as the requested set size increases.On COMPAS at k = 10, most counterfactuals from RandomInitCF are duplicate or invalid.
- Quantitative evaluation: DiverseCF produces more diverse counterfactuals than baseline methods across continuous and categorical features, reaching eleven times baseline continuous diversity on LendingClub at k = 10.It also changes more continuous features than the baselines in most settings, despite not being explicitly optimized for that metric.
- Quantitative evaluation: DiverseCF trades lower proximity for greater diversity, although categorical proximity differs by at most approximately 30% from baselines and DiverseCF-Sparse improves proximity and sparsity.The results show that users can fine-tune the method toward greater proximity when desired.
- Explaining linear ML models: For linear models, DiverseCF finds 100% valid counterfactuals and consistently higher diversity than MixedIntegerCF, while retaining or improving proximity and sparsity on Adult-Income and LendingClub.The comparison omits COMPAS results for k > 4 because MixedIntegerCF could not find more than four counterfactuals for any input.
- Approximating local decision boundaries: With only 2–11 training examples, DiverseCF-based 1-NN classifiers obtain higher F1 than LIME in most configurations, including 0.44 versus 0.19 on Adult-Income at k = 4 and a 0.5MAD threshold.LendingClub is an exception at 0.5 MAD and k = 10, although DiverseCF precision remains 0.61 versus 0.19 for LIME.
- Causal constraints: Causal post-hoc filtering is needed because more than one-third of education-changing counterfactuals are infeasible, rising to as high as 50% for highly educated people.The paper identifies education–age constraints and non-decreasing education as examples, while leaving causal constraints during generation for future work.
7 CONCLUDING DISCUSSION
The framework generates many unique, valid, and diverse counterfactuals, while approximating local decision boundaries at least as well as methods such as LIME.
- DiverseCF generates a high number of unique, valid, and diverse counterfactuals for a given input across machine learning models.
- DiverseCF can handle high values of k for which NoDiversityCF and RandomInitCF cannot find k unique and valid counterfactuals.
- Performance improves quickly as k increases, suggesting that two counterfactuals may suffice for a 1-NN classifier to capture the local data distribution in these datasets.
- DiverseCF examples approximate the local decision boundary at least as well as local explanation methods such as LIME.
6 CAUSAL FEASIBILITY OF CF EXAMPLES
Independently varying features can produce infeasible counterfactuals because features may be causally associated. Future work targets causal generation, black-box models, user constraints, behavioral evaluation, and choice overload.
- Varying each feature independently can produce infeasible counterfactuals when features are causally associated.
- The method assumes knowledge of the machine-learning model's gradient, motivating methods for fully black-box models.
- Future work includes incorporating causal knowledge during counterfactual generation rather than applying it only as a post-hoc filtering step.
- The authors identify user preferences, interactive constraint selection, behavioral experiments, and cognitive cost as open areas for improving counterfactual explanations.
A SUPPLEMENTARY MATERIALS
The supplementary materials describe implementation and reproducibility resources, including an open-source implementation and a comparison of DiverseCF with MixedIntegerCF.
- The open-source DiCE implementation can generate counterfactual examples for other datasets or machine-learning models.
- Figure 5 compares DiverseCF with MixedIntegerCF for explaining a linear machine-learning model on German-Credit.
A.1 Building ML Models
The study trains one machine-learning model per dataset using established benchmarks and selects single-hidden-layer neural networks for their generalization ability.
- Each dataset uses a machine-learning model with accuracy comparable to previously established benchmarks.
- The models are trained with the Adam optimizer in TensorFlow, with hyperparameters tuned from previous analyses.
- A single-hidden-layer neural network gives the best generalization ability across all datasets in the authors' analyses.
A.2 Explaining linear ML models: German-Credit
For German-Credit, DiverseCF consistently produces more diverse counterfactuals than MixedIntegerCF. Figure 5 compares the methods across validity, diversity, proximity, and sparsity for linear models.
- DiverseCF consistently generates more diverse counterfactuals than MixedIntegerCF on German-Credit.
- Table 4 is identified as a dataset description table.
- Figure 5 compares DiverseCF and MixedIntegerCF using %Valid CFs, diversity, proximity, and sparsity.The comparison concerns linear machine-learning models for German-Credit.