Source-linked AI summary
Distribution-Based Feature Attribution for Explaining the Predictions of Any Classifier
Xinpeng Li, Kai Ming Ting
TL;DR
Feature attribution lacked a formal standard for ensuring that explanations reflect the data distribution underlying a classifier. This paper defines distribution-supported attribution and introduces DFAX, a model-agnostic method that directly leverages that distribution; extensive experiments report superior effectiveness and efficiency over state-of-the-art baselines.
Problem
Feature attribution lacked a formal problem definition specifying that explanations must be supported by the underlying probability distribution represented by the dataset.
Method
DFAX is a model-agnostic feature-attribution method that directly leverages the underlying data distribution through probability or density estimation.
Results
Extensive quantitative and qualitative experiments demonstrate superior effectiveness and efficiency for DFAX compared with state-of-the-art model-agnostic methods.
Takeaways & Limitations
DFAX provides a fast, effective distributional approach to feature attribution that operates directly from the probability distribution represented by the data.
Takeaways & Limitations
Existing methods may fail the distribution-support criterion by creating synthetic instances, while compliant methods may use only part of the available dataset or fail to fully exploit X.
Abstract
from arXiv · showhide
The proliferation of complex, black-box AI models has intensified the need for techniques that can explain their decisions. Feature attribution methods have become a popular solution for providing post-hoc explanations, yet the field has historically lacked a formal problem definition. This paper addresses this gap by introducing a formal definition for the problem of feature attribution, which stipulates that explanations be supported by an underlying probability distribution represented by the given dataset. Our analysis reveals that many existing model-agnostic methods fail to meet this criterion, while even those that do often possess other limitations. To overcome these challenges, we propose Distributional Feature Attribution eXplanations (DFAX), a novel, model-agnostic method for feature attribution. DFAX is the first feature attribution method to explain classifier predictions directly based on the data distribution. We show through extensive experiments that DFAX is more effective and efficient than state-of-the-art baselines.
Introduction
Black-box AI models have driven demand for explainable AI, but feature attribution lacked a formal problem definition. The paper introduces a distribution-based criterion, proposes DFAX, and reports superior effectiveness and efficiency against model-agnostic baselines.
- Black-box models cannot explain their own decisions, motivating explainable AI techniques that help users understand and trust them.
- Feature attribution provides post-hoc explanations by assigning contribution scores that quantify input-feature importance for a model output.
- Model-agnostic methods treat models as black boxes and use input-output behavior, but the field lacked a formal feature-attribution problem definition.
- The paper introduces a formal definition and analyzes existing methods to identify limitations affecting performance or practical applicability.
- DFAX is a model-agnostic explainer designed to comply with the definition while directly leveraging the underlying data distribution.
- Extensive quantitative and qualitative experiments demonstrate DFAX’s superior effectiveness and efficiency compared with state-of-the-art model-agnostic methods.
Related Work
Prior model-agnostic attribution methods mainly use local surrogate models or feature perturbations, while DFAX relies on distribution estimation. Kernel density estimation and approximations provide the computational basis for efficient density queries.
- Local approximation methods: Local approximation methods define a neighborhood around an instance, fit a simple surrogate model, and derive attributions from that surrogate.
- Local approximation methods: LIME generates synthetic neighboring points, while DLIME uses hierarchical clustering on training data to define a neighborhood.
- Perturbation-based methods: Perturbation-based methods alter feature values and measure the resulting degradation in classifier performance.
- Perturbation-based methods: SHAP estimates feature importance with Shapley values, whereas PFI measures expected performance loss after permuting a feature across the dataset.
- Kernel density estimation: Kernel density estimation is a non-parametric approach for estimating point density or probability in a dataset, with Gaussian and Isolation kernels yielding different estimators.
- Kernel density estimation: Approximating the kernel with a finite-dimensional feature map enables subsequent probability or density estimation in O(1) time after one kernel mean-map computation.
Problem Definition
The paper defines feature attribution as distribution-supported scoring of each feature’s influence on a classifier prediction. The explanatory model must be built from the unmodified dataset representing the operational distribution, excluding synthetic or out-of-distribution instances.
- The definition assumes the target instance, dataset X, and training dataset D are i.i.d. samples from the same underlying distribution P.
- Feature attribution assigns each feature a score quantifying its influence on a target instance’s classifier prediction.
- The explanatory model must be built directly from dataset X, which represents the underlying probability distribution P.
- Any modification of X that changes its distribution invalidates the attribution because synthetic or out-of-distribution instances describe behavior outside the classifier’s relevant operational distribution.
Analyses of Existing Methods
The analysis uses Definition 1 to assess model-agnostic attribution methods, finding trade-offs between distributional compliance, data usage, and computational feasibility.
- Definition 1 supplies a criterion for assessing whether feature attribution methods are supported by the classifier’s underlying data distribution.
- Local approximation methods: DLIME satisfies Definition 1 by using unmodified X, but fits a surrogate model on only a local subset of the dataset.This local focus is identified as an inherent limitation of local approximation methods.
- Local approximation methods: LIME shares the local-focus limitation and violates Definition 1 because random perturbations generate synthetic neighborhoods independent of distribution P.
- Perturbation-based methods: Shapley regression values comply with Definition 1 because they use X in each feature-subset subspace, but retraining across all subsets is computationally infeasible.The method estimates classifier probabilities after withholding features outside subset S.
- Perturbation-based methods: Shapley sampling values make SHAP tractable by sampling subsets and avoiding repeated retraining, but modify X with feature values from the target instance.These modifications create synthetic instances that are not i.i.d. samples from P, so the explanation is not supported by the training distribution.
- Existing methods can therefore misuse X by creating synthetic instances or fail to fully use the dataset, conflicting with the goal of explaining model behavior on its operational distribution.
Proposed Method: DFAX
DFAX explains classifier predictions from the dataset’s conditional feature distributions rather than synthetic perturbations or a separately fitted surrogate. It uses the unmodified dataset globally and can pre-compute kernel information for acceleration.
- DFAX is motivated by a distributional approach intended to satisfy Definition 1 while overcoming limitations identified in existing methods.
- DFAX estimates conditional probabilities for class subsets using points in X whose predicted classes belong to those subsets.
- For feature s, DFAX scores a target instance by the difference between its conditional probability under the target class and under all other classes.The probability is estimated with a KDE in the one-dimensional subspace defined by s.
- DFAX measures whether the target feature value characterizes data points from the target class without simultaneously characterizing points from other classes.The paper presents this as a distributional feature-attribution principle.
- DFAX satisfies Definition 1 by estimating conditional probabilities from unmodified X and defers probability computation until a target instance is provided.
- DFAX uses global information from the entire dataset and can accelerate computation by pre-computing the kernel mean map when a finite-dimensional feature map is available or approximable.
- DFAX decouples attribution from the classifier by operating on X and pre-computed predictions, avoiding further classifier queries.The same decoupling allows predictions to be replaced with groundtruth labels for explaining the data’s inherent class structure.
Experiments
The experiments evaluate DFAX against model-agnostic baselines across diverse datasets using deletion and insertion scores. DFAX generally achieves the strongest performance, supported by distributional information and full use of the dataset.
- Experimental Setup: The study evaluates DFAX on ten real-world datasets spanning tabular, text, and image modalities, using diverse classifiers.Each dataset uses 100 randomly selected testing samples, with the remainder forming the training set.
- Experimental Setup: DFAX is compared with LINEX, SLISE, SHAP, MAPLE, DLIME, and a random sanity-check baseline.The baselines represent local-approximation and perturbation-based families.
- Evaluation Metrics: Deletion and insertion scores measure attribution quality by progressively masking or reintroducing important features and tracking classifier probabilities.The scores use random values drawn from a standard normal distribution when features are masked.
- Quantitative Results: DFAXG and DFAXS rank first or second on nine of ten datasets, with the Diabetes dataset as the only exception.DFAXG generally performs better than DFAXS, especially on insertion scores.
- Quantitative Results: SHAP and MAPLE have average insertion scores below the random baseline, while DFAX outperforms the strongest baseline methods by a large margin.The authors attribute DFAX’s performance to distributional support and its use of global information from the entire dataset.
Qualitative Evaluation
Qualitative evaluations examine attribution quality in sentiment classification and spatial transcriptomics. DFAX selects more semantically plausible words and preserves tissue predictions more accurately than DLIME.
- RottenTomatoes: In RottenTomatoes sentiment examples, DFAX identifies sentiment-indicative words such as “compelling” and “bad,” whereas DLIME selects less reasonable alternatives.DFAX highlights “fascinating” and “moving” for positive sentiment and “dull” and “too” for negative sentiment.
- HER2st: The HER2st evaluation masks the 157 non-salient genes and tests whether predictions remain consistent with the original 314-gene predictions.The protocol assumes effective attribution methods preserve the most critical genes.
- HER2st: DFAX preserves HER2st tissue predictions with 95.64% accuracy, compared with 79.51% for DLIME.DFAX produces six cancer-cell misclassifications, compared with 44 for DLIME.
- HER2st: Figure 2 visualizes original-cell predictions and predictions using the 157 salient genes selected by DFAX or DLIME.The comparison uses spatial coordinates from the HER2st dataset.
Runtime Comparison
The runtime comparison measures the time required to generate attributions for one target instance across three representative datasets. DFAX is the fastest method in all three cases and is often faster by orders of magnitude.
- Runtime Evaluation: The runtime experiment compares attribution-generation time across three datasets varying in size, feature count, and number of classes.Runtime is measured in seconds for a single target instance.
- Runtime Results: DFAX achieves the shortest runtime on all three datasets and is often faster than other baselines by orders of magnitude.The authors present this result as evidence of DFAX’s efficiency for large-scale real-world applications.
Conclusions
DFAX is presented as a fast, effective feature-attribution explainer grounded directly in the data distribution, enabled by a formal problem definition. Experiments report quantitative and qualitative superiority over state-of-the-art methods, while future work includes axiomatic analysis and feature-group attribution.
- Conclusions: DFAX operates directly from a data distribution through probability or density estimation, addressing the need for distribution-supported explanations.The authors describe it as the first fast and effective explainer with this distributional perspective.
- Conclusions: Extensive experiments report quantitative and qualitative superiority over state-of-the-art methods, with runtime comparisons confirming DFAX’s efficiency.
- Conclusions: The formal definition establishes a guideline for evaluating and designing feature-attribution explanation methods.
- Conclusions: Future work will investigate DFAX’s axiomatic properties and extend it to feature-group attribution.
A Hyperparameter Specifications
The appendix specifies hyperparameter-selection procedures and final settings for DFAX variants and baselines across datasets. It also documents the qualitative-comparison visualization used for MNIST and FMNIST.
- A Hyperparameter Specifications: Hyperparameters were selected by grid search using average deletion and insertion scores on a 100-instance validation set.The validation instances were randomly sampled from the training set.
- A Hyperparameter Specifications: DFAXG uses γ as its sole hyperparameter, with values searched from 1e-4 through 1e4 across all ten datasets.The passage lists dataset-specific final γ values, including 1e-1 for MNIST and 1e-3 for FMNIST.
- A Hyperparameter Specifications: The appendix uses Figure 4 to visualize the 100 most salient pixels for MNIST and FMNIST images.
- A Hyperparameter Specifications: DLIME’s single hyperparameter is the number of agglomerative-clustering clusters, searched from 2 through m on all ten datasets.
B Additional Qualitative Evaluation
The qualitative evaluation compares the 100 most salient pixels identified by DFAX and baselines on MNIST and FMNIST. DFAX highlights digit- and object-relevant structure, whereas the baselines produce weaker or less semantically coherent attributions.
- B Additional Qualitative Evaluation: The comparison identifies the 100 most salient pixels for each image and visualizes original images alongside each method’s selected pixels.
- B Additional Qualitative Evaluation: On FMNIST, DFAX highlights semantically meaningful features such as T-shirt shoulders, trouser waists, pullover outlines, and boot structure.
- B Additional Qualitative Evaluation: GroupSHAP and GPFI produce weakly structured or uninformative attributions, while DLIME performs poorly on MNIST and remains less semantically relevant on FMNIST.
- B Additional Qualitative Evaluation: The visualizations support DFAX’s superior performance over the baselines, including methods designed for feature-group attribution.