PaperlayerView exact revision on arXiv

Source-linked AI summary

Guided Integrated Gradients: An Adaptive Path Method for Removing Noise

Andrei Kapishnikov, Subhashini Venugopalan, Besim Avci, Ben Wedin, Michael Terry, Tolga Bolukbasi

arXiv:2106.09788v1cs.CVcs.LG

TL;DR

Integrated Gradients can produce noisy, spurious pixel attributions because gradients accumulate along its path, motivating a model-conditioned alternative. The paper introduces Adaptive Path Methods and Guided IG, which adapts the path using model gradients and generally improves attribution quality while reducing noise.

  • Problem

    Integrated Gradients often produces noisy attributions on irrelevant pixels in vision models, with gradient accumulation along the integration path identified as one source.

  • Method

    The paper generalizes path methods through Adaptive Path Methods and proposes Guided IG, which adapts the attribution path using the input, baseline, and model.

  • Results

    Guided IG outperforms other tested attribution methods quantitatively and reduces noise in final explanations across the reported experiments.

  • Takeaways & Limitations

    Guided IG provides a model-conditioned alternative to straight-line Integrated Gradients paths for producing less noisy image explanations.

  • Takeaways & Limitations

    Guided IG is one APM instance evaluated on visual models and datasets, and other APM variants or modalities may be better suited elsewhere.

Abstract

from arXiv · show

Integrated Gradients (IG) is a commonly used feature attribution method for deep neural networks. While IG has many desirable properties, the method often produces spurious/noisy pixel attributions in regions that are not related to the predicted class when applied to visual models. While this has been previously noted, most existing solutions are aimed at addressing the symptoms by explicitly reducing the noise in the resulting attributions. In this work, we show that one of the causes of the problem is the accumulation of noise along the IG path. To minimize the effect of this source of noise, we propose adapting the attribution path itself -- conditioning the path not just on the image but also on the model being explained. We introduce Adaptive Path Methods (APMs) as a generalization of path methods, and Guided IG as a specific instance of an APM. Empirically, Guided IG creates saliency maps better aligned with the model's prediction and the input image that is being explained. We show through qualitative and quantitative experiments that Guided IG outperforms other, related methods in nearly every experiment.

1. Introduction

The paper studies noise in Integrated Gradients attributions and proposes adapting the integration path to the model and input. Guided IG, an adaptive-path instance, reduces attribution noise and outperforms tested alternatives.

  • Motivation: Attribution noise in vision models can arise from accumulating gradients along the integration path, including on irrelevant pixels.The authors distinguish this source from explanations involving curvature, Riemann-sum approximation, or baseline choice.
  • Related approaches: Existing methods mainly reduce attribution noise by manipulating inputs, baselines, network activations, or aggregation procedures.Examples include SmoothGrad, XRAI, alternative baselines, BlurIG, and softplus-smoothed networks.
  • Adaptive Path Methods: Adaptive Path Methods condition the attribution path on the input, baseline, and model instead of relying only on a model-agnostic path.The proposed intuition is that model-conditioned paths can avoid anomalous regions with irregular gradients.
  • Guided IG: Guided IG selects unfinished features with the smallest absolute partial derivatives and moves only that subset toward the input at each step.Features leave the candidate set once their values match the explained input.
  • Results: Guided IG is reported to outperform other attribution methods quantitatively while reducing noise in final explanations.The paper presents Guided IG as one instance of the broader Adaptive Path Methods framework.

2. Related Work

The related work spans black-box perturbation, back-propagation, intermediate-layer visualization, and hybrid explanation methods. This paper extends path-based Integrated Gradients to address noise in pixel attributions.

  • Method categories: Explanation methods include black-box perturbation, back-propagation, intermediate-layer visualization, and combinations of these approaches.The paper focuses on perturbation- and back-propagation-based methods.
  • Black-box methods: Black-box methods perturb or modify inputs and observe model outputs, typically requiring several model evaluations for one input.Examples use segmentation masks, smooth masks, noise, blurring, or optimization over model outputs.
  • Back-propagation methods: Back-propagation methods use model gradients with respect to inputs or intermediate representations to produce pixel- or region-level saliency maps.The paper specifically builds on path-based Integrated Gradients.

3. High Gradient Impact on IG Attribution

The paper attributes noisy Integrated Gradients maps to high, correlated gradients encountered along model-agnostic paths. Straight-line paths can therefore accumulate irrelevant gradient contributions.

  • Integrated Gradients: Integrated Gradients assigns each pixel an attribution by integrating model-output gradients along a path from baseline to input.The path is parameterized by images γ(α), with α ranging from 0 to 1.
  • Gradient impact: On the straight-line path, gradient magnitudes can remain high even when the directional derivative toward the input is near zero.The area under the directional-derivative curve equals total attribution, while unrelated gradient magnitude can remain large.
  • Noise mechanism: High-norm gradients from irrelevant input dimensions can dominate the gradient map, and correlated gradient directions may prevent cancellation during integration.This can give spurious pixels non-zero attributions despite not contributing to the model output.
  • Model-agnostic paths: Straight paths may pass through high-gradient regions whose gradients point away from the integration direction.A low cosine similarity between the gradient and path direction indicates this mismatch.
  • Model-agnostic paths: Nearby points with very low or very high prediction scores can imply high gradients along the straight path even when those points are not on the path.The figure illustrates this local-geometry explanation using points surrounding the path.
  • Gradient correlation: The paper examines gradient cosine similarity across steps of Integrated Gradients and Guided IG paths to characterize gradient correlation.The comparison uses reference steps α=0.1, 0.5, and 0.9.

4. Adaptive Paths and Guided IG

Adaptive Path Methods generalize path methods by allowing the attribution path to depend on the model, input, and baseline. Guided IG greedily avoids high-gradient directions while retaining path-method axiomatic properties, with anchored variants controlling deviation from the straight-line path.

  • Adaptive Path Methods: Adaptive Path Methods generalize path methods by making the attribution path depend on the model function as well as the input and baseline.They address limitations of model-agnostic paths such as straight-line integration paths.
  • Guided IG: The optimal noise-minimizing path is infeasible to compute because it requires the network's prediction surface throughout the input space, motivating Guided IG's greedy approximation.The objective seeks to avoid high-gradient directions associated with attribution accumulation unrelated to the input.
  • Guided IG: Guided IG selects the lowest-absolute-gradient, not-yet-matched features and moves only that subset toward the input at each step.The path begins at the baseline and ends at the explained input; the paper uses a 10% subset in its efficient approximation.
  • Properties and efficiency: Guided IG has the same L1 path length as IG, while its L2 path length can be larger but is bounded by √N times the IG path length.The method has the same asymptotic time complexity as IG, but dynamic path computation prevents parallelization for a single input.
  • Bounded Guided IG: Anchored Guided IG divides the straight-line baseline-to-input path into K + 1 segments and forces the adaptive path through K anchor locations.More anchors make the result closer to IG, while zero anchors yields the unbounded variant.
  • Axiomatic Properties: Guided IG satisfies Completeness, Sensitivity(a), Sensitivity(b), Symmetry-Preserving, and Implementation Invariance properties described in the paper.Implementation Invariance follows because Guided IG relies on function gradients rather than network internals.

5. Experiments and Results

The experiments evaluate Guided IG on closed paths and image-attribution benchmarks, finding lower closed-path error and stronger performance than tested alternatives. Qualitative results show attributions concentrated around predicted objects with less irrelevant noise.

  • Evaluation setup: Guided IG is evaluated on closed paths before benchmark experiments on natural and medical image datasets.The evaluation uses common attribution metrics and compares multiple models and methods.
  • Closed-path evaluation: The closed-path test estimates ground-truth attribution as zero for every feature, because reverse paths cancel individual attributions.Random paths A → B → C → A are decomposed into segments and evaluated with attribution methods.
  • Closed-path evaluation: Guided IG produces lower mean squared error than IG on 10,000 sampled paths across 200 ImageNet validation images.The error is averaged across images, pixels, and channels.
  • Benchmark results: Guided IG outperforms other tested methods, while adding anchors such as GIG(20) yields substantial performance improvement.The comparison uses AUC-based attribution metrics, including AUC-ROC and SIC AUC.
  • Benchmark results: On diabetic-retinopathy data, smoothing is not a good strategy because sparser attributions may be preferred; XRAI combined with Guided IG performs best on most models.The paper also reports baseline and anchor effects across ImageNet, Open Images, and medical images.
  • Qualitative results: Guided IG generally clusters attributions around the predicted-class object with comparatively less noise elsewhere than IG.The qualitative comparison uses Inception v2 and includes retina-image examples.

6. Discussion

The discussion attributes Guided IG’s gains to adapting the path to avoid high-gradient information and reports good performance across baseline choices. It also identifies anchor-count variation and modality coverage as boundaries for interpretation.

  • Interpretation: Adapting the path to avoid high-gradient information allows Guided IG to perform better than other tested methods.The experiments also show good performance irrespective of baseline choice.
  • Anchors: Using 20 anchor points may be a reasonable default, although performance varies with the number of anchors.Alternative path-bounding strategies remain a future direction.
  • Scope: Guided IG is one APM instance, and other adaptive path methods may suit particular tasks, domains, or models better.The evaluation covers visual models and datasets, while suitability for text or graph models remains open.

7. Conclusion

The paper introduces Adaptive Path Methods as alternatives to straight-line Integrated Gradients and presents Guided IG as a method for reducing path-accumulated attribution noise. Guided IG improves image-model attribution metrics, while other modalities remain future work.

  • Contribution: Adaptive Path Methods generalize Integrated Gradients by replacing straight-line paths with paths adapted to the explained model and input.Guided IG is the paper’s specific adaptive-path technique.
  • Contribution: Guided IG adapts the integration path to avoid introducing attribution noise while optionally minimizing path length toward a straight path.The method is evaluated on image models using common attribution metrics.
  • Conclusion: Guided IG achieves improved results on common attribution metrics for image models.The conclusion identifies text and graph models as opportunities for future investigation.

A.1. Lemma 1

The lemma establishes the symmetry condition used in the paper’s attribution analysis: symmetric variables receive equal attributions when their values and path behavior remain symmetric.

  • Proof strategy: The proof reduces symmetry preservation to equality of partial derivatives for symmetric variables when their values are equal.The lemma establishes this derivative equality from the function’s symmetry.
  • Definition: An attribution method is symmetry preserving when symmetric variables with identical input and baseline values receive identical attributions.This is the operational definition used for the lemma.
  • Path condition: If symmetric variables have equal values at every point on the integration path, their attributions are equal.The path itself must preserve equality between the symmetric variables.
  • Guided IG: For equal symmetric feature values, Guided IG selects both features together or excludes both from each step’s feature subset.The selection follows equal partial derivatives and the argmin rule.

A.4. Theorem 1

Theorem 1 establishes that Guided IG preserves symmetry by maintaining equal values for symmetric variables throughout its integration path. The section also specifies the inputs and update structure used to compute unbounded Guided IG.

  • Theorem 1: Guided IG is symmetry preserving.The theorem states this property explicitly.
  • Theorem 1: A path method preserves symmetry when symmetric variables have equal values at every point along the integration path.This criterion is given through Lemma 2.
  • Theorem 1: Equal starting values and equal rates of change whenever the variables are equal are sufficient to maintain equality along the path.The proof separates these into two sufficiency requirements.
  • Theorem 1: For symmetric variables equal at the input and baseline, Guided IG proves starting-point equality and equal rates of change, completing the symmetry-preservation proof.The rate-of-change argument considers whether both variables belong to the selected feature set.
  • Algorithm: Unbounded Guided IG takes an example, baseline, step count, gradient function, and target feature-change fraction as inputs.The algorithm specifies T > 0 and p ∈ (0, 1].
  • Algorithm: At each step, the algorithm assigns the p fraction of features with the lowest absolute gradients to the selected set and updates their attributions.The update uses the difference between the current feature value and a temporary value.

C. Example Results

The example results compare Guided IG and IG directly and compare both with GradCAM using image-level attribution examples and AUC localization outcomes. The figures show that the preferred method varies across examples.

  • Comparing IG and Guided IG: Figure 9 presents selected OpenImages V5 examples comparing feature attributions from Integrated Gradients and Guided IG.The examples use a black baseline for both methods.
  • Comparisons with GradCAM: Figure 10 shows examples where GradCAM outperforms IG and Guided IG on AUC localization metrics.The examples use OpenImages V5 images and segmentations with an InceptionV2 model trained on ImageNet.
  • Comparisons with GradCAM: Figure 11 shows examples where IG outperforms GradCAM on AUC localization metrics.For gradient-based methods, black- and white-baseline attributions were calculated separately and averaged.
Loading 2106.09788v1…