PaperlayerView exact revision on arXiv

Source-linked AI summary

For Those Who Believe in Faithfulness: Optimizing the Area Under Insertion and Deletion Curves for Ranking Relative Feature Importance

Bjørn Leth Møller, Bulat Ibragimov, Christian Igel

arXiv:2610.09844v1cs.LGcs.CV

TL;DR

Attribution faithfulness is commonly evaluated with insertion and deletion curves, but existing heuristics had not directly optimized their area. The paper derives a top-k-based objective and approximate gradient, then incorporates them into NEM as Ra-NEM. It reports state-of-the-art faithfulness and robustness, with low latency for real-time processing.

  • Problem

    Attribution maps are commonly evaluated for faithfulness with insertion and deletion curves, but prior heuristics had not directly optimized their area.

    Source: §1 · p. 1

  • Method

    The paper derives a Monte Carlo objective for insertion/deletion AUC using variable-k top-k selection and integrates it into NEM as Ra-NEM.

    Source: §1 · p. 1, §5 · p. 7

  • Results

    Ra-NEM achieved state-of-the-art performance in faithfulness and robustness across different performance metrics.

    Source: §6 · p. 9

  • Takeaways & Limitations

    Ra-NEM’s low latency allows real-time processing, including applications that require numerous explanations.

    Source: §5 · p. 7, §6 · p. 9

    Ra-NEM produces an ordinal feature ranking, so attribution values do not represent the classifier’s actual reduction; the study focuses on ordinal faithfulness metrics.

    Source: §5.7 · p. 8

Abstract

from arXiv · show

The adoption of machine learning for socially relevant tasks requires effective explainable artificial intelligence (XAI) methods to better understand the behavior of machine learning models. Attribution methods are a popular XAI approach in which input-output relationships are characterized by heat maps that reflect the relative importance of input features for a particular prediction. The quality of such maps is often assessed by measuring faithfulness based on the area under insertion and deletion curves, which measures changes in the model output as features are added and removed. In this study, we derive an objective function from this notion of faithfulness and a way to approximate its gradient. We establish the connection between insertion curves and top-$k$ feature selection, which leads to a loss function measuring the quality of attributions. Randomization of the loss allows us to efficiently approximate its gradient. To show the effectiveness of the general approach, we combine the loss function with the neural explanation mask framework. The resulting method, termed Ra-NEM, can be used with any differentiable model without affecting the model's performance. Experiments demonstrate that Ra-NEM provides accurate attributions robustly and efficiently. Compared to other algorithms, the attributions have not only higher faithfulness but also perform well in terms of other XAI metrics. The high inference speed of Ra-NEM makes the method suitable for online applications. The code is available online: https://github.com/baerminator/Ra_Nem

1 INTRODUCTION

Attribution maps help explain model predictions, and faithfulness is commonly assessed with insertion and deletion curves. The introduction proposes directly optimizing these curves to generate high-quality attributions.

1 INTRODUCTION

  • Faithfulness is commonly assessed with insertion and deletion curves, but prior attribution heuristics had not directly optimized their area under the curve.Higher insertion-curve AUC and lower deletion-curve AUC indicate better explanations.Source: §1 · p. 1
  • Experiments report that the objective alone produces faithful attributions that score highly on Average Sensitivity and Minimal Deletion Set and pass the MPRT.Ra-NEM is also evaluated across deep-learning architectures for image classification against state-of-the-art attribution methods.Source: §1 · p. 1
  • The study links variable-k top-k feature selection to insertion and deletion curves, derives Monte Carlo sample complexity, and proposes differentiable masking for the NEM framework.The resulting method, Ranking-NEM (Ra-NEM), produces sparse faithful explanations with low latency.Source: §1 · p. 1

2 RELATED WORK

The related-work section surveys attribution methods and the Neural Explanation Mask framework. It contrasts feature-ranking approaches with set-of-feature explanations and describes how NEM generates attributions.

2.1 Heuristics for solving the attribution problem

  • Occlusion-based methods measure output changes after perturbing inputs, while gradient-based methods use input gradients; the former are generally more faithful but slower.Examples include RISE and Smooth Pixel Mask for occlusion methods, and Saliency and Integrated Gradients for gradient methods.Source: §2.1 · p. 2
  • Additive explanations rank individual features, whereas set-of-feature explanations identify a sufficient subset without ranking every feature, complicating insertion/deletion comparisons.The study follows prior evaluation strategies to compare these different explanation types.Source: §2.1 · p. 2

2.2 Neural Explanation Masks

  • NEM trains an explanation module on unlabeled data alongside a frozen model, then generates explanations by evaluating the module at inference time.Previous NEM methods produce sets of important features rather than ranking features by relative importance, so they cannot reliably identify the top k for a chosen k.Source: §2.2 · p. 2

3 METHODOLOGY

The methodology connects insertion and deletion faithfulness to top-k feature selection, derives a sampled objective and differentiable approximation, and integrates them into Ra-NEM.

3.1 Approximating the insertion curve via top-k feature selection

  • Attributions are learned by optimizing insertion and/or deletion AUC, which scores model outputs after features are added or removed under a masking operator.The perturbation operator and output-quality measure determine how these curves are computed.Source: §3.1 · p. 3, §3.1.1 · p. 3

3.1.1 Insertion and deletion curves.

  • Under monotonicity, the ranking that maximizes insertion AUC yields the optimal feature subset at every subset size.The paper notes that additive feature attribution implies the more restrictive monotonicity assumption.Source: §3.1.1 · p. 3

3.1.2 Monte Carlo approximation.

  • Uniformly sampled feature indices provide a Monte Carlo AUC estimate whose expectation equals the insertion AUC.Sampling makes evaluation practical when computing the curve at all feature counts is infeasible.Source: §3.1.2 · p. 3

3.1.3 Bounds on sample complexity.

  • The paper bounds Monte Carlo AUC error for sampled indices and also derives a variance-aware bound, treating curve values as a finite population sampled without replacement.The theorem defines the population using insertion-curve values for the input and attribution.Source: §3.1.3 · p. 4

3.2 A differentiable loss in terms of top-k feature selection

  • The loss replaces direct AUC optimization with uniformly sampled top-k selections that maximize the model-quality change, using Gumbel perturbations for differentiable selection.The Gumbel top-k trick samples ordered features without replacement and is efficient for large k.Source: §3.2 · p. 4

3.3 Ra-NEM

  • Ra-NEM trains NEM’s explanation module end-to-end with sampled binary masks, unlike prior continuous-mask variants, making it applicable to discrete-symbol sequences.The experiments described here use the loss within NEM without additional loss terms.Source: §3.3 · pp. 4–5

4 EXPERIMENTS & RESULTS

The experiments evaluate explanation methods across four ImageNet classifiers using faithfulness, robustness, complexity, and inference-time measures. Ra-NEM ranks first overall in robustness, is among the fastest methods, and performs best on the Minimal Deletion Set metric; results also examine perturbation and sample-count effects.

4.1 Evaluated architectures and dataset

  • The evaluation covers ResNet50, ConvNeXt, VGG16, and ViT, using 1,000 ImageNet validation images and separate non-overlapping samples to train NEMs.The explained models were pretrained on ImageNet’s training split; NEM training used 10,000 images.Source: §4.1 · p. 5

4.2 Baseline methods

  • The comparison includes gradient- and occlusion-based attribution methods, adds FORgrad, and evaluates Ra-NEM against NEMt using identical masking-network architectures.IBA was not evaluated on ViT, and Smooth Pixel Mask used a 10% area constraint.Source: §4.2 · p. 5, §4.3 · p. 5, §4.4 · p. 5

4.4 NEM training and post processing

  • Ra-NEM and NEMt use identical U-Net decoder masking networks trained for 10 epochs; Ra-NEM samples six k values per input and optimizes six loss terms.The NEMt loss uses λ = 1/50.Source: §4.4 · p. 5

4.5 Performance metrics

  • Faithfulness is measured by insertion-AUC minus deletion-AUC, with evaluation also covering Minimal Deletion Set, robustness, randomization, and inference time.The study tests two perturbation choices because insertion/deletion scores can depend on that choice.Source: §4.5 · pp. 5–6

4.6 Results

  • Ra-NEM ranks first overall in robustness, is the most efficient of the two fastest methods, and achieves the best Minimal Deletion Set score among high-performing methods.The Randomization score is 0.051, and Table 2 reports the Minimal Deletion Set comparison.Source: §4.6 · p. 6, Table 2 · p. 6
  • On ResNet50, FORgrad improves gradient-based faithfulness at a complexity cost, but Ra-NEM still outperforms the FORgrad-adjusted methods.The comparison is shown in Table 3.Source: Table 3 · p. 6, §4.6 · p. 6
  • Faithfulness scores are sensitive to perturbation choice, although method rankings appear stable across models; increasing sampled k can improve faithfulness while increasing training time.Table 4 reports perturbation effects for ConvNeXt, and Table 11 reports sample-count effects for Ra-NEM on ResNet50.Source: §4.6 · pp. 6–7, Table 4 · p. 7
  • For the illustrated ResNet50 dog image, Ra-NEM’s ranking aligns more closely with the central object than RISE and Grad-CAM++, which show a circular bias.The paper also provides examples of Ra-NEM performing relatively poorly in supplementary section F.Source: §4.6 · p. 6, Figure 2 · p. 7

5 DISCUSSION

Ra-NEM is trained by optimizing an estimate of insertion/deletion-curve AUC using top-k feature selection with uniformly sampled k. The discussion reports strong faithfulness, robustness, and speed while noting assumptions and limits on comparisons.

5.1 Faithfulness

  • Ra-NEM had the highest faithfulness on three of four models, leading ConvNeXt by 0.095 over NEMt but trailing RISE by 0.139 on VGG16.The authors attribute its strong average performance partly to other methods varying across models.Source: §5.1 · p. 7

5.2 Randomization and robustness

  • Ra-NEM was the most robust studied method to input noise by Average Sensitivity and passed the randomization check with an MPRT score of 0.051.The authors interpret the near-zero score as evidence that the attribution depends on model mechanics rather than a shortcut such as object segmentation.Source: §5.2 · p. 8

5.3 Time

  • Ra-NEM was the fastest method across models, closely followed by NEMt.The NEM approaches require forward passes through the explained model and masking network, whereas Saliency also requires a backward pass through the explained model.Source: §5.3 · p. 8

5.4 Ra-NEM and previous NEM methods

  • Compared with earlier NEM methods [10],, Ra-NEM removes the need to specify a complexity–accuracy trade-off and has no loss-function tuning parameters.Its binary masking also facilitates adaptation to modalities such as text or graphs, where partial occlusion is not well defined.Source: §5.4 · p. 8

5.5 Using the insertion/deletion curve as a target

  • Using both insertion and deletion curves, three training perturbation schemes, and random k avoids fixing the number of perturbation steps in advance.The authors frame curve optimization as a generalization of balancing explanation complexity and accuracy,,.Source: §5.5 · p. 8

5.6 Monotonicity

  • The deletion-curve approximation is proved under monotonicity, but Ra-NEM does not require that assumption in practice.The authors say the optimization still selects features that approximate the original input in model prediction for arbitrary subset sizes.Source: §5.6 · p. 8

5.7 Fairness of comparison

  • Comparisons may be biased because Smooth Pixel Mask and NEMt produce set-of-feature explanations rather than rankings of individual features.The evaluation follows earlier work,,; whether the results generalize to other domains remains for future work.Source: §5.7 · p. 8
  • Ra-NEM's feature values encode ordinal ranking, not the magnitude of classifier-output reduction.The study focuses on faithfulness metrics that test ordinal relationships, while some criteria, [37] assume a linear relationship between classifier change and attribution.Source: §5.7 · p. 8

6 CONCLUSIONS

The conclusion presents a differentiable objective for directly optimizing insertion/deletion AUCs and applies it in Ra-NEM. The authors report strong faithfulness and robustness alongside low latency, while identifying wider input modalities as a future direction.

6 CONCLUSIONS

  • The paper derives Monte Carlo sample complexity for AUC approximation and an efficient differentiable objective for direct AUC optimization.The authors say these general findings can improve existing XAI methods.Source: §6 · pp. 8–9
  • Ra-NEM adds the objective to NEM, uses top-k feature selection, and achieves state-of-the-art faithfulness and robustness across different performance metrics.The conclusion describes its attributions as more accurate.Source: §6 · p. 9
  • Ra-NEM's low latency supports real-time processing and resource-efficient applications such as online video analysis; extending NEM to more input modalities is a future direction.Source: §6 · p. 9

APPENDIX A DETAILED RESULTS

Appendix A identifies the main-results tables as model-specific comparisons of XAI methods. Their captions specify metrics and statistical testing, but the supplied passages do not report table cell values.

APPENDIX A DETAILED RESULTS

APPENDIX B TRAINING NEM MODELS

Appendix B examines Ra-NEM training costs and variability, including the effects of sample count and perturbation choice. The reported results describe a faithfulness–training-time trade-off and fairly stable performance across runs and perturbations.

APPENDIX B TRAINING NEM MODELS

  • Ra-NEM takes longer to train than NEMt because it uses additional samples to increase training stability.Table 9 reports training time for different explained models after 10 epochs on 10,000 ImageNet validation images.Source: Appendix B · p. 11, Table 9 · p. 11
  • Across five ResNet50 runs with identical hyperparameters, faithfulness was fairly stable, with a maximum difference of 0.021.Source: Appendix B · p. 11, Table 10 · p. 11
  • For ResNet50, increasing the number of training samples can improve faithfulness but also increase training time.The authors report that sample count controls this trade-off.Source: Appendix B · pp. 11–12, Table 11 · p. 12
  • For ConvNeXt, downstream faithfulness was fairly robust to the choice of training perturbation.Source: Appendix B · p. 12, Table 12 · p. 12

APPENDIX C PSEUDOCODE FOR THE OMEGA OPERATOR

Appendix C describes the omega operator used to select and mask features for insertion and deletion. It combines noisy top-k selection with a differentiable approximation before replacing masked input features.

APPENDIX C PSEUDOCODE FOR THE OMEGA OPERATOR

  • The operator treats attributions as logits and uses Gumbel top-k sampling to select features without replacement.It normalizes attributions with logsumexp before selection.Source: Appendix C · p. 12
  • A sigmoid soft-threshold guides optimization before hard thresholding keeps the selected top-k features, with the hard step ignored in gradient computation.The straight-through operator is used for that gradient computation.Source: Appendix C · p. 12
  • The resulting mask is applied to the input, and masked-out features are replaced according to the chosen operator.Zero replacement yields deletion; blur(x) replacement yields insertion.Source: Appendix C · p. 12

APPENDIX D REPRODUCIBILITY CHECK LIST

Appendix D lists selected experimental hyperparameters, including temperature, sample count, sampling strategy, imputation, and explanation post-processing.

APPENDIX D REPRODUCIBILITY CHECK LIST

  • The evaluation uses a fixed temperature of 1 and a sample count k of 6.The appendix also specifies sampling one value from each of six uniform bins over [1, d], where d = 224 × 224 spatial dimensions.Source: Appendix D · p. 12
  • The sampling strategy divides [1, d] into k uniform bins and samples one value from each bin.Here d = 224 × 224 is the number of spatial dimensions.Source: Appendix D · p. 12
  • The listed noise imputation value is ImageNet-scaled normal noise, and explanation post-processing uses min–max normalization.Source: Appendix D · p. 12

APPENDIX E EXAMPLE IMAGES FOR ALL MODELS

Appendix E provides additional randomly selected example images for all models. One ResNet50 example illustrates how Ra-NEM’s finer-grained attribution differs from other occlusion-based methods.

APPENDIX E EXAMPLE IMAGES FOR ALL MODELS

  • The appendix presents additional randomly selected example images for all models.Source: Appendix E · p. 12
  • In a ResNet50 example, all compared occlusion-based methods highlight the eye, while Ra-NEM identifies a smaller necessary region.Ra-NEM marks a small region of the eye and its outline.Source: Figure 3 · p. 13
  • Ra-NEM’s finer granularity more precisely reveals specific image features than the other occlusion-based methods in the example.Source: Figure 3 · p. 13

APPENDIX F EXAMPLE IMAGES OF POOR RA-NEM PERFOR-

Appendix F presents examples selected for Ra-NEM’s relatively poor faithfulness performance compared with other methods. The examples show cases where its attributions miss distinctions between image elements or assign importance to background or incorrect objects.

APPENDIX F EXAMPLE IMAGES OF POOR RA-NEM PERFOR-

  • The examples were selected where Ra-NEM’s faithfulness scores were lowest relative to the average faithfulness of the other methods.They illustrate poor-performance cases, not how often those cases occur.Source: Appendix F · p. 13
  • A ViT example attributes scattered explanations to Grad-CAM++ and RISE, possibly reflecting smoothing combined with ViT processing.The passage suggests that ViT’s lower spatial-cohesion bias could make imposed spatial bias adverse.Source: Figure 11 · p. 17
  • The ViT examples also contrast Ra-NEM’s interpretability-faithfulness balance with Integrated Gradients’ limited interpretability despite high faithfulness.Another caption notes that NEMt occasionally produces very large ViT attributions.Source: Figure 11 · p. 17, Figure 13 · p. 18
  • For the shown ViT examples, Ra-NEM does not properly distinguish between image elements or objects.The captions describe these failures in two examples.Source: Figure 16 · p. 19, Figure 17 · p. 20
  • In the VGG16 examples, Ra-NEM switches foreground for background.This occurs in both captions.Source: Figure 18 · p. 20, Figure 19 · p. 21
  • In the ResNet50 examples, Ra-NEM assigns the greatest importance to the wrong object.Both captions describe focus on an incorrect object.Source: Figure 20 · p. 21, Figure 21 · p. 22
Loading 2610.09844v1…