PaperlayerView exact revision on arXiv

Source-linked AI summary

Counterfactual Shapley Additive Explanations

Emanuele Albini, Jason Long, Danial Dervovic, Daniele Magazzeni

arXiv:2110.14270v4cs.LGcs.AI

TL;DR

Feature attributions do not clearly specify how to handle missing features or guide users toward recourse. The paper proposes CF-SHAP, which uses counterfactuals in the SHAP background distribution and adds feature-change direction guidance. It reports superior counterfactual-ability to existing methods on three public datasets using tree ensembles.

  • Problem

    In actionable recourse, it is unclear how feature attributions should be used, including how to choose their background dataset and guide desirable feature changes.

    Source: §1 · p. 1, §3.1 · p. 3

  • Method

    CF-SHAP uses a set of counterfactuals as the SHAP background distribution and enriches attributions with feature-change direction guidance.

    Source: §1 · p. 2, §7 · p. 10

  • Results

    CF-SHAP using 100-nearest neighbours has the best counterfactual-ability on three public datasets and beats the baselines on all three, with improvements greater than 0%.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    Counterfactual backgrounds can yield feature attributions that better advise useful input changes, with direction information needed to express the changes.

    Source: §7 · p. 10

    The paper identifies alternative action and cost functions, evaluation on other models, and links to causality as future directions.

    Source: §7 · p. 10

Abstract

from arXiv · show

Feature attributions are a common paradigm for model explanations due to their simplicity in assigning a single numeric score for each input feature to a model. In the actionable recourse setting, wherein the goal of the explanations is to improve outcomes for model consumers, it is often unclear how feature attributions should be correctly used. With this work, we aim to strengthen and clarify the link between actionable recourse and feature attributions. Concretely, we propose a variant of SHAP, Counterfactual SHAP (CF-SHAP), that incorporates counterfactual information to produce a background dataset for use within the marginal (a.k.a. interventional) Shapley value framework. We motivate the need within the actionable recourse setting for careful consideration of background datasets when using Shapley values for feature attributions with numerous synthetic examples. Moreover, we demonstrate the efficacy of CF-SHAP by proposing and justifying a quantitative score for feature attributions, counterfactual-ability, showing that as measured by this metric, CF-SHAP is superior to existing methods when evaluated on public datasets using tree ensembles.

1 INTRODUCTION

The introduction frames a gap between feature attributions and actionable recourse: SHAP does not specify how to handle missing features or guide users’ behavior, while counterfactual methods have their own limits. The paper proposes CF-SHAP and a counterfactual-ability evaluation framework, reporting the strongest counterfactual-ability on three public datasets for CF-SHAP with 100-nearest neighbours.

1 INTRODUCTION

  • SHAP estimates a missing feature’s contribution using a background distribution, but does not explicitly guide users on desirable changes.The introduction identifies the choice of background distribution as an issue and notes the lack of behavioral guidance.Source: §1 · p. 1
  • Counterfactual explanations seek a nearby, plausible input with a different outcome, but often offer only one option and leave users facing many explanations.The introduction notes that a single counterfactual may limit users’ ability to choose suitable changes and that numerous explanations can impose cognitive load.Source: §1 · pp. 1–2
  • CF-SHAP combines Shapley-based feature attribution with counterfactual explanations to retain a simple attribution format while adding actionability.The paper also proposes a counterfactual-ability framework and uses counterfactuals as the Shapley background distribution, with guidance on feature-change direction.Source: §1 · p. 2
  • On three publicly available datasets, CF-SHAP using 100-nearest neighbours as a simple counterfactual generation technique is shown to have the best counterfactual-ability among benchmarked baseline feature attribution techniques.Source: §1 · p. 2
  • The paper focuses on tree-based models, for which interventional Shapley values can be computed exactly and efficiently using a proposed algorithm.The introduction motivates this scope by citing the adoption and performance of tree ensembles for tabular classification and regression.Source: §1 · p. 2

2 BACKGROUND

The background establishes a binary-classification setting in which outcome 1 is treated as adverse. It introduces Shapley values as feature attributions computed with a background distribution, then describes counterfactuals as nearby, plausible inputs with different outcomes.

2 BACKGROUND

  • The paper considers a trained binary classifier and, unless otherwise specified, treats outcome 1 as adverse with decision threshold 0.The model maps real-valued inputs to real-valued outputs, which are thresholded to produce binary predictions.Source: §2 · p. 2

2.1 Shapley values

  • Shapley values fairly allocate a model output among features, with SHAP simulating missing features through a marginal expectation over background distribution D.The paper denotes the resulting vector for an input x by ϕ.Source: §2.1 · p. 2

2.2 Counterfactual Explanations

  • A counterfactual is a plausible input close to the original that produces a different prediction; the paper’s figure contrasts input-invariant backgrounds with counterfactual backgrounds.For the paper’s tree-based setting, the considered counterfactual methods must generate multiple points and not require a differentiable model; the figure states that the counterfactual background yields more actionable guidance.Source: §2.2 · p. 2, Figure 1 · p. 3

3 COUNTERFACTUAL SHAP

The section argues that Shapley explanations need input-relevant background data and directional guidance to support counterfactual interpretation. CF-SHAP uses counterfactuals as background data and pairs the resulting Shapley values with derived trends.

3.1 Choice of the background distribution

  • Counterfactual backgrounds compare an input with similar, more reachable cases and make the contrast more relevant to users than input-invariant backgrounds may be.The authors note that explanations based on the training set may not advise users how to change features or identify which features led to an adverse outcome.Source: §3.1 · pp. 3–4
  • CF-SHAP computes Shapley values using a counterfactual distribution that depends on the explained input.The counterfactual technique used to obtain that distribution is specified as part of its definition.Source: §3.1 · p. 4

3.2 Guidance on the direction of the change

  • Shapley values identify influential features but do not indicate whether a user should increase or decrease them to change an adverse prediction.Derived trends address this directional gap using the feature's value relative to the counterfactual distribution: they indicate increasing or decreasing the feature to move toward that distribution.Source: §3.2 · p. 4
  • Derived trends can distinguish required changes for inputs with similar Shapley values, while Shapley values alone do not provide that direction.Figure 2 illustrates derived trends providing guidance on change direction that Shapley values alone cannot.Source: Figure 2 · p. 4, §3.2 · p. 4

3.3 Counterfactual SHAP

  • A CF-SHAP explanation is the tuple of counterfactual Shapley values and derived trends, and it retains Shapley's additivity, missingness, and consistency properties.The explanation is defined with respect to a counterfactual distribution for the input.Source: §3.3 · p. 5

4 COUNTERFACTUAL-ABILITY

The section defines counterfactual-ability as a way to assess how feature attributions may help users reverse adverse outcomes. It sets up a flexible action-and-cost framework, then instantiates it with assumptions about user behavior and feature changes.

4 COUNTERFACTUAL-ABILITY

  • The framework aims to evaluate how explanations may help users reverse adverse decisions, while acknowledging that user behavior can vary substantially by context.The paper does not claim to resolve the challenge of predicting how users engage with explanations.Source: §4 · p. 5

4.1 General Evaluation Framework

  • Counterfactual-ability measures the negative cost of reaching the minimum-cost counterfactual within the action subset induced by an explanation.The action function describes possible user changes, while the cost function measures their difficulty.Source: §4.1 · p. 5
  • Because finding the minimum-cost counterfactual can be intractable for users, the evaluation restricts action subsets to half-lines originating at the query point.This restriction follows an assumption that users apportion effort across features independently of total effort.Source: §4.1 · pp. 5–6

4.2 Instances of the Evaluation Framework

  • Proportional recourse changes features in proportion to their Shapley values, whereas random recourse uses a random utility vector over the considered features.The paper motivates random recourse as requiring fewer assumptions about user preferences and as relevant when users receive only a top-k list.Source: §4.2 · pp. 6–7
  • The instances use trend-aware changes to adverse, top-k features and measure recourse cost by their quantile shift.Quantiles are computed from training-data marginal feature distributions; the feature changes are treated as interventions.Source: §4.2 · pp. 6–7
  • The proposed action and cost instances assume feature changes are interventions; accounting for causal structure is left as future work.The authors note that causal-structure-aware changes could be used where a full causal understanding of the input space is possible.Source: §4.2 · p. 7

5 EXPERIMENTS

CF-SHAP outperforms the SHAP baselines on counterfactual-ability and plausibility across the tested datasets, with results robust to several action and cost choices. The experiments use counterfactuals from nearest neighbors as the background distribution and assess explanation runtime.

5 EXPERIMENTS

  • CF-SHAP uses the 100 nearest training points with a different prediction as its counterfactual background, keeping those counterfactuals on-manifold.The neighbors use Manhattan distance in quantile space; K=100 matches SHAP's default background sample count for comparison.Source: §5 · pp. 7–8
  • Experiments use TreeSHAP with XGBoost on HELOC, Lending Club, and Wine Quality, comparing CF-SHAP against SHAP with training, predicted-class, and label-based backgrounds.Counterfactual-ability is assessed over up to 4,000 rejected test samples per dataset.Source: §5 · pp. 7–8
  • CF-SHAP beats all three baselines on counterfactual-ability across all three datasets, with improvement tending to rise as users can change more top-ranked features.The comparison uses random recourse and an L1 cost in Figure 4.Source: §5 · p. 8, Figure 4 · p. 8
  • Results are robust across the tested action functions and L1/L2 costs, while the best K-NN setting varies by dataset from K=10 to K=100.The appendix reports the alternative action- and cost-function results and hyperparameter experiments.Source: §5 · p. 8
  • CF-SHAP beats the baselines in plausibility on all datasets and is reported as highly robust to plausibility normalization, action function, and cost function.The reported Figure 5 setting uses random recourse and total quantile shift cost.Source: §5 · p. 9, Figure 5 · p. 9
  • CF-SHAP has execution time similar to the baselines, with average explanation times as low as 620 μs for Wine Quality, 646 μs for HELOC, and 2646 μs for Lending Club.The authors report that K-NN adds no significant overhead; TreeSHAP complexity scales linearly with background-dataset size.Source: §5 · p. 9

6 RELATED WORK

Related work connects feature attributions and counterfactuals through different approaches, while this paper uses counterfactual generation to improve Shapley-based attributions. Prior work also differs in how it represents feature importance and evaluates it in counterfactual terms.

6 RELATED WORK

  • A Bayesian decision-theory approach [69] optimizes the Shapley background distribution for expected user reward but does not provide concrete selection or update guidance, or a practical action interpretation.Source: §6 · p. 9
  • Prior approaches [42] and derive attributions from diverse counterfactuals using feature-change frequency, which can assign importance to features correlated with those the model uses.The authors note that such behavior may be useful in some settings, such as medical science, but less suited to credit scoring.Source: §6 · pp. 9–10
  • An adversarial-perturbation approach focuses on changing the class but does not directly enforce other constraints, such as plausibility, which this paper addresses through a counterfactual-based background distribution.Source: §6 · p. 10
  • Work [16] [49] studies the complementary direction: using feature attributions to guide counterfactual search, rather than using counterfactual generation to empower attributions.Source: §6 · p. 10
  • Few prior metrics evaluate attributions in counterfactual terms; the fidelity error in [70] changes one feature at a time, whereas this paper's parameter k allows multiple features to change.Source: §6 · p. 10

7 CONCLUSION AND FUTURE WORK

CF-SHAP uses counterfactuals as the Shapley background and adds trends to describe change direction; on three public datasets, its nearest-neighbour variants outperform existing feature-attribution methods. The authors identify user-preference-aware actions, other models, and connections to causality as future directions.

7 CONCLUSION AND FUTURE WORK

  • On 3 publicly available datasets, CF-SHAP with simpler nearest-neighbour counterfactual techniques performs better than existing feature-attribution methods.The authors also propose evaluating counterfactual-ability and plausibility using action and cost functions.Source: §7 · p. 10
  • Counterfactual feature attributions need derived trends to describe the direction of change, alongside Shapley values.The authors report that counterfactuals as the background distribution can better advise useful changes to overcome an adverse outcome.Source: §7 · p. 10
  • Future work could couple possible actions with probabilities to account for users’ preferences for certain actions.The authors also propose grounding action and cost functions in findings about how users interpret attributions and change their behaviour.Source: §7 · p. 10
  • Testing on models such as neural networks, exploring model-agnostic counterfactual methods, and studying connections to causality remain future directions.The authors mention conditional backgrounds and counterfactual methods that explicitly use causality as possible approaches.Source: §7 · p. 10
  • The authors highlight a gap in efficient generation of diverse counterfactual explanations for widely adopted non-differentiable models such as decision-tree ensembles.They also point to the importance of diverse counterfactual generation techniques advocated by practitioners [43] [53] [57].Source: §7 · p. 10

A.1 Datasets and Models

The experiments use three publicly available datasets and XGBoost models. Data splits differ for the loan dataset, and model parameters are selected by cross-validation with a variance penalty.

A.1 Datasets and Models

  • Experiments used 3 publicly available datasets: HELOC, WINE, and LC.Table 2 describes the datasets.Source: §A.1 · p. 13
  • HELOC and WINE used stratified random 70/30 train/test splits, while LC used a non-random 70/30 split by loan issuance date.Source: §A.1 · p. 13
  • An XGBoost model was trained for each dataset, with parameters tuned by 2,000 Bayesian-optimization iterations to maximize average validation ROC-AUC over 5-fold cross-validation.The optimization penalized high model variance using validation and training ROC-AUC values.Source: §A.1 · p. 13
  • Decision thresholds were selected from the ROC-AUC curve by maximizing the sum of false-positive rate and true-positive rate.Table 2 reports the thresholds and model performance.Source: §A.1 · p. 13

A.2 Technical setup

The appendix records the experimental environment and implementation choices, including interventional TreeSHAP and a 10,000-sample cap for nearest-neighbour selection. It also points to the CF-SHAP algorithm procedure.

A.2 Technical setup

  • Experiments ran on an AWS c6i.8xlarge virtual machine with 32 vCPUs and 64 GB RAM, using Ubuntu 20.04, Python 3.6.13, shap 0.39.0, sklearn 0.24.2, and xgboost 1.3.3.Source: §A.2 · p. 13

A.3 Source Code

  • The paper states that experiment-reproduction code will be made available and computes SHAP values with TreeSHAP using the interventional, non-conditional setting.The implementation uses the model output and the shap package’s TreeExplainer.Source: §A.3.1 · p. 13

A.3.2 K-Nearest Neighbours.

  • K-nearest neighbours were selected from a random sample of 10,000 training examples to make results indifferent to dataset size.Source: §A.3.2 · p. 13

B COUNTERFACTUAL SHAP ALGORITHM: PROPERTIES AND EXECUTION TIME

B.1 Properties of Counterfactual SHAP

CF-SHAP retains SHAP’s additivity, local accuracy, missingness, and consistency properties, while its counterfactual backgrounds make linearity across models typically fail. Counterfactual outputs can approximate the decision threshold, and the described K-NN implementation has linear runtime in background size.

B.1 Properties of Counterfactual SHAP

  • CF-SHAP retains SHAP’s local accuracy, missingness, and consistency properties, and is additive: the model output equals the baseline plus the feature attributions.The paper attributes these properties to their holding independently of the background distribution.Source: §B.1 · p. 13
  • A counterfactual background’s average model output should approximate threshold t, but this depends on the model and counterfactual distribution.For continuous models with counterfactuals on the decision boundary, the average equals t; this is not theoretically guaranteed for tree-based models. Projected K-nearest neighbours produce an average close to the threshold, with performance very similar to the unprojected variant (Figure 6).Source: §B.1 · p. 13, Figure 6 · p. 14
  • Shapley-value linearity describes summing attributions across models with a fixed background; it is distinct from CF-SHAP’s additivity property.The paper notes that linearity is useful for rapidly calculating ensemble SHAP values with TreeSHAP [37].Source: §B.1 · p. 13
  • Because CF-SHAP backgrounds are tailored to each query and model, its attribution values for a summed model will typically not equal the sum of the separate models’ values.Counterfactuals for one model may not be counterfactuals for another model on the same dataset.Source: §B.1 · p. 14

B.2 Computational complexity

  • Total computation time sums counterfactual generation, TreeSHAP, and trend computation; for K-NN, each component scales linearly with background size n.The trends are computed as a simple average.Source: §B.2 · p. 15

B.3 Execution Time: Experiments

The experiments measure the time to generate a single explanation and report execution-time findings for CF-SHAP and other explanation techniques.

B.3 Execution Time: Experiments

  • Execution time is measured per explanation by running each algorithm across dataset samples for over 0.1 seconds, then dividing elapsed time by explanations computed.They repeat the procedure 10 times to account for OS-scheduler error and use the minimum time because the methods are deterministic.Source: §B.3 · p. 15
  • CF-SHAP 100-NN takes only 4.3% more execution time than SHAP D-PRED (n=100), while (Tree-)SHAP runtime scales linearly with background-set size.The authors note that methods using the full training set or variants have considerably larger runtimes; random samples or k-means medoids can approximate those distributions.Source: Table 4 · p. 15, §B.3 · p. 15
  • For the main experiments, K is set to 100 to match SHAP’s default random sample of 100 background points.The execution-time table reports average time for one sample’s explanation; n denotes the number of points in the random background sample.Source: Table 4 · p. 15, §B.3 · p. 16

D ROBUSTNESS TO DIFFERENT SETTINGS

The appendix tests CF-SHAP’s robustness to changes in action and cost assumptions and to the number of nearest neighbors. Its reported improvements are robust across alternative assumptions, while neighbor-count results vary by dataset.

D ROBUSTNESS TO DIFFERENT SETTINGS

  • The robustness experiments repeat the Section 5 evaluations under alternative action-function and cost-function definitions.The main-text assumptions were random recourse and total quantile-shift cost using L1.Source: D ROBUSTNESS TO DIFFERENT SETTINGS · p. 16
  • The K-sensitivity experiment compares CF-SHAP with SHAP D-PRED over 4,000 explanations, varying the number of neighbors from 1 to 1,000.The authors chose SHAP D-PRED as the toughest baseline to beat.Source: D ROBUSTNESS TO DIFFERENT SETTINGS · p. 16
  • The best K ranges from 10 to 100 depending on the dataset; increasing K above 100 does not improve counterfactual-ability.Decreasing K below 10 also does not improve it, and using a single counterfactual causes a sharp fall.Source: D ROBUSTNESS TO DIFFERENT SETTINGS · p. 16
  • CF-SHAP’s counterfactual-ability and plausibility results are highly robust across alternative action and cost functions, and it performs better than baselines under those assumptions.The alternatives include random recourse with L2 quantile-shift cost and proportional recourse with L1 or L2 cost.Source: D ROBUSTNESS TO DIFFERENT SETTINGS · p. 16, Figure 8 · p. 17, Figure 9 · p. 17

Related papers

Loading 2110.14270v4…