PaperlayerView exact revision on arXiv

Source-linked AI summary

A Consistent and Efficient Evaluation Strategy for Attribution Methods

Yao Rong, Tobias Leemann, Vadim Borisov, Gjergji Kasneci, Enkelejda Kasneci

arXiv:2202.00449v2cs.CVcs.LG

TL;DR

Pixel-perturbation evaluations of attribution methods can yield inconsistent rankings and require costly retraining. The paper analyzes information leakage and introduces ROAD, which improves consistency while avoiding retraining and reducing computational costs by up to 99%.

  • Problem

    Pixel-perturbation evaluation strategies can produce inconsistent attribution-method rankings and require prohibitively expensive retraining.

  • Method

    The paper uses an information-theoretic analysis to identify mask-based confounding and develops debiased evaluation through Noisy Linear Imputation and ROAD.

  • Results

    ROAD achieves consistent evaluation with no retraining and saves 99% of computational costs compared with retraining-based evaluation.

  • Takeaways & Limitations

    ROAD provides a more consistent and efficient framework for evaluating attribution explanations without the retraining step.

  • Takeaways & Limitations

    Some inconsistencies remain, and perfect agreement may not be desirable because attribution methods can differ in the characteristics they identify.

Abstract

from arXiv · show

With a variety of local feature attribution methods being proposed in recent years, follow-up work suggested several evaluation strategies. To assess the attribution quality across different attribution techniques, the most popular among these evaluation strategies in the image domain use pixel perturbations. However, recent advances discovered that different evaluation strategies produce conflicting rankings of attribution methods and can be prohibitively expensive to compute. In this work, we present an information-theoretic analysis of evaluation strategies based on pixel perturbations. Our findings reveal that the results are strongly affected by information leakage through the shape of the removed pixels as opposed to their actual values. Using our theoretical insights, we propose a novel evaluation framework termed Remove and Debias (ROAD) which offers two contributions: First, it mitigates the impact of the confounders, which entails higher consistency among evaluation strategies. Second, ROAD does not require the computationally expensive retraining step and saves up to 99% in computational costs compared to the state-of-the-art. We release our source code at https://github.com/tleemann/road_evaluation.

1. Introduction

Existing pixel-perturbation evaluations can rank attribution methods inconsistently and require expensive retraining. The paper analyzes these confounders and proposes debiased strategies that improve consistency and efficiency.

  • Evaluation setting: Feature-attribution evaluations use perturbations to test whether changing important pixels reduces predictive quality while changing unimportant pixels has little effect.These measures target the fidelity of explanations to the underlying model.
  • Motivation: Evaluation outcomes can change sharply with perturbation functions and removal orders, producing contradictory rankings of attribution methods.Most Relevant First and Least Relevant First can favor different methods.
  • Contributions: Information-theoretic analysis reveals that perturbation-based evaluation results can be significantly confounded.The analysis examines mechanisms underlying evaluation strategies based on perturbation.
  • Contributions: Noisy Linear Imputation significantly decreases sensitivity to hyperparameters such as removal order.The strategy is proposed to compensate for the identified confounder while remaining efficient and effective.
  • Contributions: 99% computational-cost savings are achieved by ROAD compared with retraining-based evaluation strategies such as ROAR.ROAD is presented as an objective and efficient evaluation framework that avoids retraining.

2. Related Work

Related work spans many attribution techniques and explanation desiderata, while functional-grounded evaluation often perturbs important or unimportant image features. Prior studies observed sensitivity to evaluation hyperparameters, but a sound theoretical explanation remained limited.

  • Attribution methods: The literature contains numerous attribution methods that assign importance scores to input features.Examples include Integrated Gradients, SmoothGrad, and other feature-attribution approaches.
  • Evaluation metrics: Functional-grounded metrics evaluate explanations without requiring human-generated ground truth, which can be difficult or impossible to obtain.A common idea is to change the image region judged most important and assess the resulting prediction.
  • Open problem: Earlier studies reported that changing attribution-evaluation hyperparameters can produce varying results.This observation was made for image and tabular attribution settings, while a sound theoretical explanation was still missing.
  • Analytical framing: The analytical model separates information in remaining feature values from information contained in the binary removal mask.The imputed input determines the evaluation outcome after masking and imputation.

3. Preliminaries

This section formalizes pixel-perturbation evaluation strategies, including feature removal, imputation, and retraining, and introduces information-theoretic quantities used to analyze them.

  • Pixel-perturbation strategies: Pixel-perturbation strategies successively replace selected features with imputed values, using MoRF or LeRF removal orders.MoRF removes the most important features first, whereas LeRF removes the least important features first.
  • Pixel-perturbation strategies: In the MoRF setting, the top-k features selected by an attribution explanation are removed successively as k increases from 0 to d.The selected features are represented by a binary mask indicating which dimensions belong to the top-k set.
  • Retraining evaluation: The ROAR approach evaluates modified samples with a newly trained classifier after feature removal.The imputation operator restores retained features to their original positions and fills removed positions with a chosen value.
  • Retraining evaluation: Under zero imputation, the removed top-k features are discarded, and a faster accuracy decline indicates that more influential features were successfully removed.The resulting input is represented as (1 −M) ⊙x.
  • Information-theoretic preliminaries: For the two-class problem, mutual information imposes upper and lower bounds on obtainable optimal classification accuracy rather than determining it as a function.These bounds allow mutual information to serve as a surrogate for obtainable accuracy in the perturbation analysis.
  • Information-theoretic preliminaries: Mutual information measures information shared between variables; here, I(x; C) denotes information shared between the complete feature vector and the class variable.The section also introduces entropy, joint entropy, and conditional entropy as supporting information-theoretic quantities.

4. Analysis

The analysis shows that pixel-removal evaluations can be confounded because binary masks leak class information, inflating evaluation outcomes beyond the information in retained features. It formalizes this leakage through mutual-information terms and motivates minimally revealing imputation to separate mask and feature effects.

  • Accuracy and Mutual Information: Classification accuracy is bounded by mutual information, making mutual information a surrogate for attainable accuracy in retraining-based pixel-removal evaluations.The relevant quantity is the information remaining in the least important features after removal.
  • Information Decomposition: The evaluation outcome decomposes into Feature Information, Mask Information, and a Mitigator term that captures their overlap.Feature Information is the class information in the retained features, while Mask Information captures class-discriminative information in the binary mask.
  • Class Information Leakage: Class Information Leakage occurs when the binary mask carries class information that increases the evaluation outcome beyond the selected features.This can make an attribution method appear worse in MoRF because removing features does not reduce accuracy as expected.
  • Invertible Imputation: Invertible imputation allows the mask to be reconstructed from the imputed image, causing the Mitigator to vanish and leaving mask leakage uncompensated.Replacing removed pixels with a reserved absence value is an example of invertible imputation.
  • Minimally Revealing Imputation: Minimally Revealing Imputation seeks conditional independence between the imputed features and mask so the Mitigator can compensate for Mask Information.The goal is to make the mask difficult to infer from the imputed image while preserving the class-dependent information needed for evaluation.
  • Empirical Leakage: On CIFAR-10, classifiers using only binary masks reached almost 80% accuracy versus 85% with full images, showing that mask shape alone carries substantial class information.The masks were computed for variants of Integrated Gradients and Guided Backpropagation.

5. Debiasing Evaluation Strategies for Local Attribution Methods

The analysis identifies class information leakage through perturbation masks as a major confounder and introduces Noisy Linear Imputation to reduce it. The strategy approximates removed pixels from neighboring values while adding noise, making imputations harder to distinguish from originals.

  • Class Information Leakage: Fixed-value imputation leaks class information through the binary mask, substantially influencing evaluation outcomes.A mask-only classifier reaches almost 80% accuracy for IG-SG, while an imputation predictor nearly perfectly detects fixed-value imputations.
  • Noisy Linear Imputation: Noisy Linear Imputation is designed to approach Minimally Revealing Imputation while remaining efficient for repeated image processing.It solves a sparse system based on neighboring pixels and adds small random noise to prevent the linear dependency from being learned.
  • Noisy Linear Imputation: The method is motivated by the high correlation among image pixels, allowing each removed pixel to be approximated using weighted neighboring pixels.When removed pixels are adjacent, the equations form a connected but sparse system that can still be solved efficiently.
  • Noisy Linear Imputation: The proposed strategy reduces leakage by producing imputations that are substantially harder for a predictor to distinguish from original pixels.The imputation predictor places Noisy Linear Imputation closer to the optimal minimally revealing strategy than fixed-value imputation.

6. Experiments

Experiments show that Noisy Linear Imputation improves consistency across removal orders and makes retraining largely unnecessary. The resulting ROAD framework preserves rankings while sharply reducing computational requirements, although some inconsistencies remain.

  • 6.1. Consistency under Removal Orders: Noisy Linear Imputation yields more consistent attribution rankings between MoRF and LeRF than fixed-value imputation.With fixed values, IG-SG is best in LeRF but worst in MoRF; the inconsistency vanishes with Noisy Linear Imputation.
  • 6.2. Efficiency: 0.84 in MoRF and 0.94 in LeRF are the rank correlations between retrained and non-retrained evaluation with Noisy Linear Imputation.The authors attribute the reduced difference to the smaller distribution shift produced by nearly minimally revealing imputations.
  • 6.2. Efficiency: ROAD uses Noisy Linear Imputation without retraining and requires only 0.7% of ROAR’s computational resources.The framework is defined as removal with Noisy Linear Imputation and no retraining.
  • 6.2. Efficiency: Under ROAD, IG-SG, GB-SQ, GB-Var, and IG perform best in MoRF, while these methods have higher accuracies in LeRF.The reported MoRF preference corresponds to lower accuracies early in the removal curve.
  • 6.2. Efficiency: Some inconsistencies remain because different evaluation strategies may capture different attribution characteristics, making perfect agreement potentially undesirable.The paper specifically reports that GB and GB-Var perform badly in both MoRF and LeRF.

7. Conclusion and Outlook

The paper concludes that ROAD provides an efficient evaluation approach for global fidelity while addressing class information leakage. It reports broad accessibility and identifies more sophisticated imputations and additional metrics as future directions.

  • Conclusion: ROAD measures global fidelity among attribution explanations while circumventing the Class Information Leakage issue.The conclusion presents these as central advantages of the framework.
  • Conclusion: 99% runtime reduction relative to ROAR is reported for ROAD, improving accessibility through lower resource requirements.The conclusion characterizes ROAD as open-source and readily implementable in practical use cases.
  • Outlook: Future work will investigate more sophisticated imputation models and evaluation metrics beyond fidelity.

A.2. Reproduction of the proof of the relation between mutual and accuracy in the binary case

The appendix reproduces bounds connecting mutual information with Bayesian classifier accuracy in the binary case. It assumes equal class priors and discrete features, and derives the result using conditional entropy and convexity arguments.

  • A.2. Proof Setting: The proof considers binary class labels with equal priors and discrete feature variables.The assumptions are P(C = 0) = P(C = 1) = 1/2 and discrete features x.
  • A.2. Proof: Mutual information is expressed as a weighted mean of a function of conditional Bayesian classifier accuracies.The weights are p(s), shorthand for P(x = s), and the function uses binary entropy.
  • A.2. Proof: The derivation uses binary entropy to relate conditional entropy and classifier accuracy.The appendix states H(C|s) = H2(acc(C|s)) and uses monotonicity of the inverse binary entropy.
  • A.2. Proof: Jensen’s inequality supplies the convexity step used to derive the accuracy bounds from mutual information.The appendix notes that 1−H2(·) is convex and the probabilities p(s) are convex multipliers.

A.3. Analysis of the LeRF Ordering

The LeRF analysis evaluates removing the least important features while retaining the highly important ones. Its information-theoretic formulation shows that mask information can inflate evaluation outcomes, enabling spurious explanation maps to score well.

  • LeRF evaluation: LeRF removes the k least important features per instance, retaining the top-k features through an imputation operator.The resulting classifier should improve quickly as important features are added, but change little when low-importance features are inserted.
  • Information-theoretic analysis: The LeRF quantity of interest is the class information contained in the imputed version of the selected highly important features.The analysis seeks to maximize this class information to obtain a good evaluation score.
  • Information-theoretic analysis: Mask information contributes positively to the evaluation goal, so the shape of removed pixels can affect scores independently of feature importance.This effect is especially problematic when the imputation operator reveals which pixels were removed.
  • Spurious explanations: A spurious LeRF map that removes pixels from the top for one class and the bottom for another lets a retrained classifier infer the class from mask shape alone.Such a map can achieve the best possible accuracy without providing a reasonable attribution of feature importance.

B. GAN Imputation

The paper compares fixed-value, Noisy Linear, and GAN imputation as ways to conceal which pixels were removed. GAN imputation can produce natural-looking samples and approach minimally revealing imputation, but its computational and artifact-related drawbacks motivate Noisy Linear Imputation.

  • GAN imputation: GAIN is used as a GAN-based imputation operator, with hyperparameters selected for CIFAR-10.The search includes the reconstruction-loss weight α and hint rate.
  • GAN imputation: GAN imputation produces more natural-looking samples than fixed-value and Noisy Linear imputation, making the mask harder to infer.A trained imputation predictor verifies that GAN imputation is closest to the minimally revealing condition.
  • Limitations: GAN imputation may introduce features absent from the original image and performs poorly when too many pixels are removed.The paper notes artifacts at high missingness percentages, which can distort evaluation results.
  • Recommendation: GAIN training is expensive and requires hyperparameter tuning, so the paper recommends Noisy Linear Imputation for the evaluation framework.The recommendation reflects the desired combination of few hyperparameters and computational efficiency.

C.1. Implementation Details

The experiments evaluate attribution methods on CIFAR-10 and Food-101 using trained ResNet models, repeated runs, multiple imputation settings, and retraining/no-retraining variants. Noisy Linear Imputation is used to assess consistency across evaluation strategies.

  • Datasets and models: CIFAR-10 experiments use a vanilla ResNet-18 trained for 40 epochs, achieving 84.5% test accuracy.The model uses SGD with an initial learning rate of 0.01 and one learning-rate reduction schedule.
  • Evaluation protocol: Results for CIFAR-10 are averaged over N = 5 runs, with standard errors reported for the experiments.The standard deviations for the evaluated explanations are generally below one percent.
  • Evaluation protocol: The study compares twelve evaluation strategies spanning retraining/no-retraining, MoRF/LeRF, and fixed-value, Noisy Linear, and GAN imputation.Rank correlations are computed across the strategies using rankings from five consecutive runs as independent observations.
  • Consistency analysis: Using Noisy Linear Imputation increases consistency between MoRF and LeRF rankings with and without retraining for IG- and GB-family methods.The qualitative comparisons are reported for CIFAR-10 across the corresponding imputation and attribution-method variants.
  • Datasets and models: Food-101 experiments use a vanilla ResNet-50 trained for 40 epochs, achieving 81.67% test accuracy.The learning rate starts at 0.01 and is reduced by a factor of 0.1 every 10 epochs.

D.2. Correlation Analysis

Correlation analyses compare retraining and no-retraining strategies across imputation operators, perturbation orders, attribution families, and datasets. Noisy Linear Imputation generally improves consistency in retraining comparisons, while Food-101 results are noisier and show a weaker no-retraining gain between MoRF and LeRF.

  • Food-101 correlation analysis: Food-101 rank correlations compare eight evaluation strategies combining retraining/no-retraining, MoRF/LeRF, and three imputation operators.The comparisons cover fixed, Noisy Linear, and GAN imputation for the evaluated attribution methods.
  • Food-101 correlation analysis: Retraining and no-retraining methods remain highly consistent on Food-101, supporting the reported efficiency gains on the larger dataset.This agreement is reported across the evaluated strategy combinations.
  • Food-101 correlation analysis: Food-101 shows improved MoRF/LeRF consistency with retraining but a slight decrease with no-retraining, and its results are generally noisier than CIFAR-10.Small correlation differences can change rankings, particularly in the no-retraining setup.
  • CIFAR-10 correlation analysis: Noisy Linear and GAN imputation fairly increase MoRF/LeRF consistency over fixed imputation in retraining, but consistency decreases in no-retraining.This pattern is reported in the full CIFAR-10 correlation results.
Loading 2202.00449v2…