PaperlayerView exact revision on arXiv

Source-linked AI summary

CXPlain: Causal Explanations for Model Interpretation under Uncertainty

Patrick Schwab, Walter Karlen

arXiv:1910.12336v1cs.LGstat.ML

TL;DR

Fast, accurate, and uncertainty-aware feature-importance estimation remains challenging for diverse, high-dimensional machine-learning models. CXPlain treats explanation as causal learning by training a separate model to explain a target model, then uses bootstrap ensembles for uncertainty. Experiments report higher accuracy and faster evaluation than existing model-agnostic methods, with uncertainty estimates correlated with held-out estimation accuracy.

  • Problem

    Fast and accurate feature-importance estimation and uncertainty quantification remain challenging across diverse machine-learning models and high-dimensional data.

    Source: §1 · p. 1

  • Method

    CXPlain trains a separate explanation model with an adapted causal objective to estimate feature importance for any predictive model, using bootstrap resampling for uncertainty estimates.

    Source: §1 · p. 2, §3 · p. 3, §6 · p. 8

  • Results

    CXPlain was significantly more accurate than leading model-agnostic methods on MNIST and ImageNet, faster than LIME and SHAP, and produced uncertainty estimates correlated with held-out accuracy.

    Source: §5 · pp. 7–8, §6 · p. 8

  • Takeaways & Limitations

    CXPlain provides fast post-training explanations and uncertainty estimates that can help assess the reliability of feature-importance scores.

    Source: §1 · p. 2, §5 · p. 7, §6 · p. 8

    CXPlain requires a separate training stage, and its causal associations concern marginal predictive-performance improvements rather than real-world causality between inputs and outputs.

    Source: §5 · p. 8

Abstract

from arXiv · show

Feature importance estimates that inform users about the degree to which given inputs influence the output of a predictive model are crucial for understanding, validating, and interpreting machine-learning models. However, providing fast and accurate estimates of feature importance for high-dimensional data, and quantifying the uncertainty of such estimates remain open challenges. Here, we frame the task of providing explanations for the decisions of machine-learning models as a causal learning task, and train causal explanation (CXPlain) models that learn to estimate to what degree certain inputs cause outputs in another machine-learning model. CXPlain can, once trained, be used to explain the target model in little time, and enables the quantification of the uncertainty associated with its feature importance estimates via bootstrap ensembling. We present experiments that demonstrate that CXPlain is significantly more accurate and faster than existing model-agnostic methods for estimating feature importance. In addition, we confirm that the uncertainty estimates provided by CXPlain ensembles are strongly correlated with their ability to accurately estimate feature importance on held-out data.

1 Introduction

Feature-importance explanations help users understand and validate model decisions, but fast, accurate, and uncertainty-aware attribution remains difficult for complex and high-dimensional models. CXPlain addresses this challenge with causal explanation models and reports improved accuracy, speed, and uncertainty estimation.

  • Feature-importance estimates help users understand, validate, and interpret predictive-model decisions.Source: §1 · p. 1
  • Fast and accurate attribution is difficult because machine-learning models use varied structures, algorithms, and decision functions, while existing estimates often carry substantial uncertainty.Source: §1 · p. 1
  • CXPlain trains a supervised explanation model with a causal objective to estimate feature importance for another model without requiring changes to the predictive model.Source: §1 · p. 1
  • CXPlain combines bootstrap-based uncertainty estimation with experiments showing greater accuracy and faster evaluation than existing model-agnostic methods.Source: §1 · p. 2

2 Related Work

Existing explanation methods differ in model compatibility and reliability, and uncertainty in their attributions can make explanations difficult to trust. CXPlain is presented as a causal, model-agnostic explanation model designed to address these limitations while enabling uncertainty estimation and fast evaluation.

  • Existing feature-importance methods include gradient-based, sensitivity-based, feature-removal, and mimic-model approaches.Source: §2 · p. 2
  • Explanation reliability is understudied, although attribution methods can be sensitive to input shifts and produce substantially uncertain estimates.Source: §2 · pp. 2–3
  • Table 1 compares CXPlain with representative feature-importance estimation methods.Source: Table 1 · p. 2
  • CXPlain learns to explain any machine-learning model with a causal objective, without retraining, adapting, or requiring detailed knowledge of the explained model.Source: §2 · p. 3

3 Methodology

CXPlain reframes feature-importance estimation as supervised causal explanation: a separate model learns to explain any predictive model without retraining or adapting it. Its objective derives relative feature contributions from prediction errors, while bootstrap ensembles quantify uncertainty.

Causal Explanations (CXPlain)

  • CXPlain trains a separate explanation model to estimate the importance scores of a target predictive model from input features.The framework leaves the predictive model unchanged and learns explanations using supervised training.Source: §3 · p. 3, Figure 1 · p. 3

Causal Objective

  • CXPlain computes predictive errors with and without each feature by comparing target-model outputs for the full and feature-masked inputs.Masking can use zeroes, mean replacement, or more distribution-aware schemes depending on the input data.Source: §3 · p. 4
  • The causal objective measures each feature’s contribution through the decrease in predictive error associated with adding that feature to the available information.This adapts a Granger-style causal definition to feature-importance estimation.Source: §3 · pp. 3–4
  • The resulting feature contributions are normalised into relative importance scores between 0 and 1 that sum to 1.These scores form the target importance distribution used to train the explanation model.Source: §3 · p. 4
  • Training minimises the Kullback–Leibler divergence between target importance distributions and the explanation model’s predicted importance distributions.Precomputing target importances for each training sample requires N(p + 1) evaluations of the predictive model.Source: §3 · p. 4

Uncertainty of Importance Estimates

  • Bootstrap ensembles quantify uncertainty by using confidence-interval width for each feature’s importance estimate.Each ensemble member is trained on a bootstrap-resampled training set, and confidence-interval quantiles provide lower and upper bounds.Source: §3 · pp. 4–5

4 Experiments

The experiments compare CXPlain with existing methods on feature-importance accuracy, computation time, and uncertainty calibration across image benchmarks. They evaluate masked-image behavior, attribution maps, runtime, and correlations between uncertainty and held-out ranking errors.

Experimental design

  • The experiments compare CXPlain with state-of-the-art feature-importance methods on MNIST and ImageNet image-classification benchmarks.They focus mainly on image classification and neural-network models to visualize and quantify estimation performance.Source: §4 · p. 5, §4.1 · p. 5

Feature-importance evaluation

  • CXPlain (U-net) attribution maps are reported as more semantically focused on salient image regions than SHAP and LIME for the Gorilla-versus-Zebra benchmark.For MNIST, accurate masks should make an 8-versus-3 image more closely resemble a 3 after the pixels distinguishing an 8 are removed.Source: Figure 5 · p. 6, Figure 4 · p. 6

Experimental design

  • Top-importance pixel masks are evaluated by removing 10% of MNIST pixels and 30% of ImageNet pixels, then measuring the resulting confidence change.The experiments also plot importance assignments and masked images for qualitative assessment.Source: §4.1 · p. 6

Uncertainty evaluation

  • Uncertainty calibration is assessed by correlating bootstrap uncertainty ui with per-feature rank error REi on held-out MNIST images.The uncertainty is the width between the 95th and 5th percentile bootstrap confidence limits for importance estimates among the top 2.5% of pixels by ∆log-odds.Source: §4.2 · pp. 6–7

Computational performance

  • Compute time is compared in log2 seconds for N = 100 test images on identical hardware, with lower values indicating faster estimation.Figures 6 and 7 report significance using *** for p < 0.001 under the MWW test.Source: §4.1 · p. 6, Figure 6 · p. 7, Figure 7 · p. 7

5 Results and Discussion

CXPlain matched or exceeded competing feature-importance methods while evaluating faster, and its bootstrap uncertainty estimates tracked explanation errors. The authors also note that CXPlain requires one-off training and that its causal interpretation is limited to predictive-model performance, not real-world causality.

Predictive Performance

  • CXPlain was competitive with DeepSHAP on MNIST and outperformed LIME on ImageNet.On MNIST, the U-net variant was competitive with DeepSHAP; on ImageNet, it significantly outperformed LIME.Source: §5 · p. 7

Computational Performance

  • CXPlain computed feature-importance estimates significantly faster than LIME and SHAP on MNIST and ImageNet.The speed gap was considerably larger on ImageNet because higher-dimensional images made LIME and SHAP’s many model evaluations slower.Source: §5 · p. 7

Quality of Uncertainty Estimates

  • M = 5 bootstrap models produced uncertainty estimates significantly correlated with feature-importance accuracy on N = 100 unseen images.Increasing the ensemble from M = 5 to M = 100 significantly increased this correlation, while visual uncertainty patterns resembled rank errors.Source: §5 · pp. 7–8, Figure 8 · p. 8, Figure 9 · p. 8

Limitations

  • CXPlain’s one-off training requirement is a limitation, although the cost typically amortises through faster evaluation.The authors also restrict its causal interpretation to marginal improvements in the predictive model’s performance.Source: §5 · p. 8
  • CXPlain’s reported associations do not indicate real-world causal relationships between a model’s inputs and outputs.Its causal objective concerns the degree to which input features improve the explained predictive model’s performance.Source: §5 · p. 8

6 Conclusion

The conclusion presents CXPlain as a model-agnostic explanation model that produces fast feature-importance estimates and bootstrap uncertainty estimates. Experiments on MNIST and ImageNet found higher accuracy than existing model-agnostic methods and much faster evaluation.

  • CXPlain trains a separate explanation model with a causal objective to estimate feature importance for a target predictive model.Source: §6 · p. 8
  • CXPlain is compatible with any machine-learning model, produces importance estimates quickly after training, and supports bootstrap uncertainty estimates.Source: §6 · p. 8
  • CXPlain was significantly more accurate than existing model-agnostic methods on MNIST and ImageNet while providing importance estimates orders of magnitude faster.Source: §6 · p. 8

Supplementary Material for: "CXPlain: Causal Explanations for Model

The supplementary material identifies the paper as “CXPlain: Causal Explanations for Model Interpretation under Uncertainty” and lists Patrick Schwab and Walter Karlen as authors from ETH Zurich.

A Hyperparameters

The experiments used Python and TensorFlow, standardized hardware, image benchmarks with ResNet targets, and a Twitter sentiment benchmark to test CXPlain across modalities and model types.

  • Experiments used Python, TensorFlow, standardized compute hardware, and 10000 perturbed samples per image for LIME and SHAP.Source: A Hyperparameters · p. 12

MNIST Benchmark

ImageNet and Twitter Benchmarks

  • The ImageNet target was a binary ResNet-32 classifier with 96.73% test accuracy, while Twitter experiments used 100000 balanced short messages.Source: A Hyperparameters · p. 13

B Training Bootstrap Ensembles of Causal Explanation Models

The bootstrap ensemble trains multiple causal explanation models on resampled training data, producing an ensemble for estimating feature importance.

C Computing Infrastructure

All experiments used the same specified computing hardware, including an Intel processor, an Nvidia GPU, and 32 GB of RAM.

D Qualitative Samples for the MNIST and ImageNet Benchmarks

Additional qualitative samples are presented for the MNIST and ImageNet benchmarks to illustrate feature-importance outputs.

MNIST

ImageNet

E Qualitative Samples for the Twitter Sentiment Analysis Benchmark

Supplementary materials show qualitative feature-importance examples for Twitter sentiment analysis, including semantically meaningful word-level assignments.

Qualitative Twitter examples

  • CXPlain (MLP) assigns high importance to words with positive or negative connotations in short Twitter messages.The explanations were subjectively judged to appear semantically meaningful.Source: E Qualitative Samples for the Twitter… · p. 14

Supplementary configurations

Cross-method comparisons

Qualitative Twitter examples

  • Table S5 presents examples of short messages and the importances assigned by CXPlain (MLP).Source: Table S5 · p. 18

Table S5 encoding

  • Table S5 uses deeper colors to indicate higher importance scores.Source: Table S5 · p. 18
  • All messages shown in Table S5 are labeled as positive in sentiment.Source: Table S5 · p. 18

Related papers

Loading 1910.12336v1…