PaperlayerView exact revision on arXiv

Source-linked AI summary

Explaining individual predictions when features are dependent: More accurate approximations to Shapley values

Kjersti Aas, Martin Jullum, Anders Løland

arXiv:1903.10464v3stat.MLcs.LGstat.ME

TL;DR

Complex models require interpretable explanations of individual predictions, but Kernel SHAP can be inaccurate when features are dependent because it assumes independence. The paper extends Kernel SHAP with dependence-aware expectation estimates and aggregates contributions from dependent features. Across linear and nonlinear simulations, the proposed methods more accurately approximate true Shapley values than original Kernel SHAP, while the best method depends on the feature distribution and conditioning regime.

  • Problem

    Kernel SHAP assumes feature independence, although dependent features are common and can produce very wrong explanations, even for simple linear models.

  • Method

    The paper extends Kernel SHAP by estimating conditional expectations under feature dependence and aggregates Shapley values for dependent features using hierarchical clustering.

  • Results

    Across simulations with linear and nonlinear models, Gaussian and non-Gaussian distributions, and dimensions 3 and 10, the proposed methods more accurately approximated true Shapley values than original Kernel SHAP.

  • Takeaways & Limitations

    The best dependence-aware method depends on the underlying feature distribution and the number of features being conditioned on.

  • Takeaways & Limitations

    The proposed non-parametric approach performs poorly when the dependence structure and marginal distributions are far from Gaussian, and kernel density estimation suffers from the curse of dimensionality.

Abstract

from arXiv · show

Explaining complex or seemingly simple machine learning models is an important practical problem. We want to explain individual predictions from a complex machine learning model by learning simple, interpretable explanations. Shapley values is a game theoretic concept that can be used for this purpose. The Shapley value framework has a series of desirable theoretical properties, and can in principle handle any predictive model. Kernel SHAP is a computationally efficient approximation to Shapley values in higher dimensions. Like several other existing methods, this approach assumes that the features are independent, which may give very wrong explanations. This is the case even if a simple linear model is used for predictions. In this paper, we extend the Kernel SHAP method to handle dependent features. We provide several examples of linear and non-linear models with various degrees of feature dependence, where our method gives more accurate approximations to the true Shapley values. We also propose a method for aggregating individual Shapley values, such that the prediction can be explained by groups of dependent variables.

1. Introduction

The paper motivates local explanations for complex machine-learning predictions and focuses on Shapley values as a principled framework. It identifies feature dependence as a major problem for Kernel SHAP and proposes an extension that handles dependence and groups dependent contributions.

  • Local explanations describe how input features influence a specific model prediction, unlike global explanations of overall feature influence.
  • Shapley values distribute the difference between an individual prediction and the global average prediction among features.
  • Shapley values have also been used to measure global feature importance, including settings with independent and dependent features.
  • Kernel SHAP offers a computationally efficient approximation but assumes feature independence, which can produce very wrong explanations when features are dependent.
  • The paper extends Kernel SHAP to dependent features and proposes clustering Shapley values for dependent variables to improve presentation.

2. The exact Shapley value and the Kernel SHAP approximation

This section presents Shapley values as coalition-based feature contributions and explains Kernel SHAP as a weighted-least-squares approximation. Exact computation becomes intractable with many features, while estimating conditional model outputs is the key methodological challenge.

  • The Shapley value averages each player's marginal contribution across all coalitions in which that player can join.
  • In prediction explanations, Shapley values decompose the difference between a specific prediction and the global average prediction.
  • The contribution function v(S) is defined using the model's expected output conditional on the observed values of features in subset S.
  • For independent features in a linear regression model, the Shapley values take a simple form.
  • With M features, exact Shapley computation involves 2^M subsets and becomes computationally intractable beyond a few features.
  • The paper's main methodological contribution is developing and comparing methods for estimating the conditional expectations required when computing Shapley values.
  • Kernel SHAP approximates Shapley values through weighted least squares and estimates the contribution function for sampled feature combinations.

3. Incorporating dependence into the Kernel SHAP method

The paper incorporates feature dependence into Kernel SHAP by estimating conditional feature distributions instead of sampling missing features independently. It develops Gaussian, Gaussian-copula, empirical, and combined approaches, with method choice depending partly on distributional form and conditioning dimension.

  • Motivation: Kernel SHAP can fail when highly dependent features are treated as independent, because its integral approximation replaces the conditional distribution with a marginal distribution.The independence assumption is used specifically when approximating the conditional expectation underlying coalition values.
  • Approach: The proposed methods estimate p(x_¯S|x_S = x*) directly and sample from that conditional distribution rather than sampling missing features independently.The paper proposes four approaches: Gaussian, Gaussian copula, empirical conditional, and a combination of empirical and parametric methods.
  • Gaussian approach: Under a multivariate Gaussian assumption, the conditional distribution remains multivariate Gaussian, with parameters estimated from the training-data mean and covariance matrix.Samples from this conditional distribution are then used to approximate the Kernel SHAP integral.
  • Gaussian copula: A Gaussian copula preserves empirical marginal distributions while modeling dependence through a Gaussian copula, enabling conditional sampling when features are not close to multivariate Gaussian.The approach transforms empirical margins to Gaussian variables, samples conditionally, and transforms them back to the original scale.
  • Empirical conditional distribution: The empirical approach uses weighted training observations to approximate the conditional distribution, with distances based only on features in S and bandwidth σ controlling smoothing.Its weighting scheme is connected to the Nadaraya-Watson estimator, and AICc can select σ, although selecting it separately for every subset and observation is computationally intensive.
  • Empirical conditional distribution: The empirical conditional method performs well when dim(x_S) is small, whereas multivariate Gaussian and Gaussian-copula methods outperform it when conditioning on more features.This motivates combining the empirical method for low-dimensional conditioning sets with a parametric method for higher-dimensional ones.

4. Experiments

The experiments evaluate dependent-feature Shapley approximations across dimensions, feature distributions, predictive models, and sampling methods. Across three-dimensional settings, dependence-aware methods generally outperform original Kernel SHAP, with the best method varying by feature distribution.

  • Experimental design: Exact Shapley values are computed in simulated low- and moderate-dimensional settings because exact evaluation requires O(2^M) integrals.The experiments use M = 3 and M = 10, with varied feature distributions, sampling models, and predictive functions.
  • Experimental design: The experiments compare original Kernel SHAP with Gaussian, copula, empirical, combined, and TreeSHAP variants under controlled sampling budgets.Methods use K = 1000 training-set samples per feature combination and test observation; TreeSHAP is included for piecewise constant models.
  • Three-dimensional results: In three-dimensional Gaussian-feature experiments, original Kernel SHAP works well only under independence and is outperformed by all alternatives once ρ exceeds 0.05.The Gaussian conditional model generally performs best, while AICc-based σ selection outperforms a fixed σ = 0.1 in this setting.
  • Three-dimensional results: For skewed, heavy-tailed, and bimodal features, dependence-aware methods outperform original Kernel SHAP, with preferred methods depending on the distribution.The fixed empirical σ = 0.1 method is uniformly best among empirical alternatives for GH features, while empirical methods are strongest for widely separated Gaussian-mixture modes.
  • Three-dimensional results: Piecewise constant-model results largely reproduce the linear-model pattern: original Kernel SHAP is outperformed, while Gaussian or empirical methods lead for matching feature distributions.TreeSHAP is only slightly better than original Kernel SHAP in experiments D and E and worse in experiment F.
  • Conclusions: Overall, the three-dimensional experiments show that accounting for feature dependence is important, and the approximate-AICc empirical method is recommended over the slower exact version.The recommended approximate version has fairly similar, and sometimes better, results than the exact version.

4.3. Dimension 10.

The dimension-10 simulations extend earlier experiments to larger feature sets and compare proposed dependence-aware approximations with original Kernel SHAP and TreeSHAP. Results broadly resemble the three-dimensional experiments, with performance depending on model, distribution, and dependence strength.

  • Experimental setup: The experiments cover linear and piecewise constant models with Gaussian features, plus a piecewise constant model with generalized hyperbolic features.The dimension-10 settings use both Gaussian and GH-distributed feature constructions.
  • Results: The Gaussian-feature results are mostly consistent with the corresponding three-dimensional experiments.The paper reports this pattern for both the linear and piecewise constant settings.
  • Evaluation: The ten-dimensional results use Monte Carlo integration because exact Shapley values are not computationally practical in this setting.Figures 8 and 9 and Table 1 report the simulation results.
  • Results: For the linear model, the proposed method is slightly better than original Kernel SHAP at small and medium dependence but worse at high dependence.This dependence-strength pattern is reported for the dimension-10 linear experiment.
  • Results: For skewed and heavy-tailed features, empirical-0.1+Gaussian has the best MAE values and skill scores, slightly outperforming empirical-0.1+copula.The same passage reports that the suggested approaches outperform original Kernel SHAP and TreeSHAP in this experiment.

4.4. Real data example.

The real-data example applies the methods to mortgage-default prediction from transaction-derived features. It compares dependence-aware Shapley approximations with original Kernel SHAP for an XGBoost classifier, where true Shapley values are unavailable.

  • Data and model: The data contain 28 features extracted from six 365-day consumer transaction time series.Features summarize balances, transaction counts, transfers, and account totals using means, extrema, standard deviations, and scaled standard deviations.
  • Data and model: The training and test sets contain 12,696 and 1,921 observations, respectively, and the 50-tree XGBoost model achieves a test AUC of 0.88.The model uses default parameter settings.
  • Comparison: The real-data comparison evaluates empirical-0.1+Gaussian against original Kernel SHAP.The combined method is selected based on the preceding skewed, heavy-tailed simulation example.
  • Comparison: Because the true Shapley values are unknown, the real-data analysis compares the estimated explanations rather than measuring error against ground truth.Figure 12 displays Shapley values from the two methods for four features.

5. Clustering of Shapley values corresponding to feature dependence

The paper aggregates Shapley values for dependent features to make individual explanations easier to interpret. It uses dependence-based hierarchical clustering and presents summed contributions for feature groups.

  • Motivation: Grouping dependent features can reveal important combined contributions that may be missed when ranking individual Shapley values.A correlated group may enter the top-ten explanation even when none of its individual features does.
  • Clustering workflow: The proposed workflow clusters features using training or sufficiently large test data, then computes group-level Shapley contributions.The clustering is intended to remain consistent across individuals rather than changing for every prediction.
  • Dependence measure: Kendall’s τ is preferred to Pearson correlation because Shapley values are not restricted to linear dependence.The paper describes rank correlation as more robust and less assumption-heavy for this purpose.
  • Clustering method: The dissimilarity is D_j,k = 1 − |τ_j,k|, so perfectly correlated features have distance 0 and independent features have distance 1.Hierarchical agglomerative clustering with complete linkage is then applied to this dissimilarity matrix.
  • Real-data illustration: Using α = 0.1 in the Kelley-Gardner-Sutcliffe penalty function, the real-data features are divided into twelve groups.Figure 13 presents the combined Shapley values for these groups for two example predictions.

6. Summary and discussion

The paper modifies Kernel SHAP to account for dependent features and evaluates four conditional-distribution approaches. Simulations show more accurate Shapley approximations than original Kernel SHAP, while clustering aggregates dependent-feature contributions for clearer presentation.

  • Problem: Original Kernel SHAP assumes independence between feature subsets, which can produce very wrong explanations when features are highly dependent.This assumption replaces the relevant conditional distribution with a marginal distribution.
  • Method: The paper proposes four dependence-aware approaches based on multivariate Gaussian, Gaussian-copula, empirical, and combined conditional-distribution estimates.The combined approaches pair the empirical method with either the Gaussian or Gaussian-copula approach.
  • Results: Across dimensions 3 and 10, linear and nonlinear models, and Gaussian and non-Gaussian distributions, the proposed methods more accurately approximate true Shapley values than original Kernel SHAP.For nonlinear models, the methods also clearly outperform TreeSHAP in the reported experiments.
  • Results: The combined approach is regarded as the most promising because the empirical method performs better with few conditioned features, whereas Gaussian and copula methods perform better with more.The combined method is also applied to a 28-variable mortgage-default case where true Shapley values are unavailable.
  • Aggregation: Hierarchical clustering sums individual Shapley values within feature groups and presents these group totals instead of separate feature contributions.This aggregation is intended to improve interpretation when features are dependent.
  • Limitation: The main limitation is computational time, especially the AICc bandwidth calculation in the non-parametric approach.With a fixed bandwidth, the combined approach has the same computational time as original Kernel SHAP.
  • Scope: Categorical tabular data pose a greater challenge for the parametric approaches than for the proposed non-parametric approach.The paper discusses numerical conversion, one-hot encoding, mixed-type distances, and entity embeddings as possible treatments.

Appendix A. Shapley properties in the prediction explanation setting

In prediction explanations, Shapley values decompose the deviation from the global mean while satisfying efficiency, symmetry, dummy-player, and linearity properties.

  • The baseline φ0 = E[f(x)] measures the portion of a prediction attributable to the global average rather than features.
  • Efficiency: Efficiency requires feature Shapley values to sum to the difference between the prediction and the global average.
  • Symmetry: Symmetry assigns equal values to features that contribute equally in every combination with other features.
  • Dummy player: The dummy-player property assigns zero to a feature that never changes the prediction.
  • Linearity: Linearity makes a feature’s Shapley value equal to the sum of its values across additive prediction functions or their linear combinations.

Appendix B. Shapley values when the model is linear

The paper derives Shapley values for linear regression models, covering both independent and dependent feature settings.

  • The section proves an explicit Shapley-value formula for linear regression with independent features and derives the contribution function for dependent features.

B.1. Linear model and independent features.

For linear models with independent features, the contribution function simplifies under independence, making each feature’s marginal contribution independent of the coalition.

  • When the predictive model is linear and features are independent, the Shapley values take a simple form.
  • The derivation’s transition to the integral expression follows from the linear-model and independent-features assumptions.
  • After computing v(S), the contribution for v(S ∪ {j}) is obtained directly.
  • Because v(S ∪ {j}) − v(S) is independent of S, the Shapley formula simplifies across coalitions.
  • The dependent-feature case instead derives v(S) using the conditional distribution of unobserved features.

B.2. Linear model and dependent features.

For linear models with dependent features, computing the contribution function requires conditional expectations over unobserved features, and obtaining them analytically is generally difficult.

  • With dependent features, the contribution function v(S) is derived using the conditional distribution of unobserved features given the observed coalition.
  • Analytical computation can avoid time-consuming simulations when E[x_¬S | x_S = x* _S] is available.
  • Obtaining the required conditional expectation analytically is not straightforward in general.

Appendix C. Generalized Hyperbolic Distribution

This appendix presents the Generalized Hyperbolic distribution, including its density, moments, and conditional distribution under partitioning. It also specifies the parameterization used for conditional distributions.

  • The appendix introduces the density of a d-dimensional Generalized Hyperbolic random vector X.
  • The squared Mahalanobis distance δ(x, µ, Σ) measures the distance between x and µ, while Kλ denotes a modified Bessel function of the third kind.
  • The mean and covariance of X are E(X) = µ + E(W)β and Var(X) = E(W)Σ + Var(W)ββT.
  • When X is partitioned into X1 and X2, the conditional distribution X2 given X1 = x1 is also Generalized Hyperbolic distributed.
  • The conditional distribution uses the GH* notation because it adopts a slightly different Generalized Hyperbolic parameterization.
Loading 1903.10464v3…