On the (In)fidelity and Sensitivity for Explanations

• The paper frames quantitative objective measures as a complement to subjective evaluation, which includes qualitative displays, human satisfaction, and model understanding.It distinguishes objective measures from axiom- or property-based tests of explanation mechanisms.Source: §1 · p. 1
• Infidelity measures the expected difference between an explanation’s input-perturbation dot product and the corresponding output perturbation, across varied perturbation distributions.The distributions can perturb toward a baseline or multiple reference points, or add small Gaussian noise for robustness to input or reference-point mis-specification.Source: §1 · p. 2
• The infidelity-minimizing explanation can be viewed as a novel combination of Smooth-Grad [40] and Integrated Gradients [43], with a kernel specified by the perturbations.The paper also introduces perturbations yielding novel explanations and reports qualitative improvement in human experiments.Source: §1 · p. 2
• The proposed robust sensitivity measure captures explanation changes under insignificant input perturbations, and its unconstrained minimum is a trivial constant explanation.The paper therefore considers lowering sensitivity in a restrained way, using fidelity to quantify the tradeoff.Source: §1 · p. 2
• Kernel smoothing can lower an explanation’s sensitivity while also lowering its infidelity; experiments report qualitative improvements in most cases.The meta-algorithm includes Smooth-Grad [40]; an appendix alternative uses adversarial training but requires modifying the predictor.Source: §1 · p. 2

2.1 Defining the infidelity measure

• Infidelity uses expected squared error between an attribution’s predicted change and the model’s change under real-valued random perturbations; the perturbation distribution specifies which changes matter.Examples include differences from fixed, noisy, or multiple baselines, and perturbations affecting a subset of features. The default norm in experiments is ℓ2.Source: §2.1 · p. 3, §2 · p. 3

2.2 Explanations with least Infidelity

• When the perturbation-dependent matrix is invertible, the optimal explanation is a specially smoothed completeness-satisfying explanation, such as Integrated Gradients; for deterministic perturbations, optimality is equivalent to completeness.The smoothing uses a special kernel, and a pseudoinverse or small diagonal adjustment can address computational instability or non-invertibility.Source: §2.2 · pp. 3–4

2.3 Many Recent Explanations Optimize Infidelity

• Under baseline-difference perturbations, optimal explanations satisfy completeness (also met by IG, DeepLift, and LRP); coordinate perturbations recover gradients in the small-step limit and occlusion-1, while binary coalition perturbations recover Shapley values.For deterministic baseline perturbations, the completeness relation concerns the explanation multiplied elementwise by the perturbation and the corresponding output difference.Source: §2.3 · p. 4

2.4 Some Novel Explanations with New Perturbations

• Noisy-baseline perturbations yield an explanation satisfying a robust variant of completeness, while square-patch perturbations for images improve computational complexity and capture spatial relationships.The noisy baseline is Gaussian around a clean baseline; square patches have a predefined length, and more complex image masks can also be used.Source: §2.4 · p. 4

2.5 Local and Global Explanations

• Infidelity explanations are naturally local; when a perturbation is a subset of the input-minus-baseline difference, multiplying the explanation by that difference gives a global explanation, and the two should be compared separately.Here, local attributions express sensitivity to feature changes, while global attributions express function-value changes.Source: §2.5 · p. 5

3 Objective Measure: Explanation Sensitivity

• Explanation sensitivity can be framed by input gradients, summarized with a norm, or bounded uniformly in a neighborhood; bounded gradients imply local Lipschitz continuity.Max-sensitivity is introduced as a related measure of the maximum explanation change under small input perturbations.Source: §3 · p. 5
• Max-sensitivity measures the maximum change in an explanation for input perturbations within radius r, and can be robustly estimated by Monte Carlo sampling.Unlike local Lipschitz continuity, it remains finite when the explanation score is bounded, including cases where local Lipschitz continuity may be unbounded, such as ReLU-based gradient explanations.Source: §3 · p. 5
• Minimizing max-sensitivity produces a constant explanation, which may be nonsensical and shows that sensitivity alone is only a partial measure of explanation quality.Natural explanations may have sensitivity because the model or the explanation method is sensitive, so reducing it can risk degrading fidelity.Source: §3 · p. 5
• Appropriately reducing an explanation’s sensitivity can also lower its infidelity, rather than degrading fidelity toward a vacuous constant explanation.The paper presents this as a key contribution and develops the relationship between sensitivity and infidelity in the next section.Source: §3 · p. 6

4 Reducing Sensitivity and Infidelity by Smoothing Explanations

• The section motivates smoothing by showing that explanation sensitivity much larger than function sensitivity around x implies large infidelity at some nearby point.It proposes appropriately lowering the sensitivity of a given explanation to reduce both its sensitivity and infidelity.Source: §4 · p. 6
• Kernel smoothing defines Φ_k(f,x) as a kernel-weighted average of Φ(f,z); with a Gaussian kernel, it becomes Smooth-Grad [40].Source: §4 · p. 6
• Theorem 4.1 bounds smoothed SENSMAX by the original explanation's sensitivity averaged under the kernel, which can be much lower than worst-case sensitivity when sensitivity is concentrated in some directions.The theorem's bound is the kernel-weighted average of the original SENSMAX around x.Source: §4 · p. 6
• Theorem 4.2 shows that smoothing reduces infidelity when its stated C1/C2 condition holds; experiments validate that smoothed explanations can be both less sensitive and more faithful.The section notes that the averaged infidelity is usually close to the original explanation's infidelity.Source: §4 · p. 6
• Appendix results show that adversarial training also produces gradient explanations that are less sensitive and more faithful.Source: §4 · p. 6

Setup

• Experiments use randomly selected MNIST, CIFAR-10, and ImageNet images, restricting local explanation comparisons to MNIST and estimating measures by Monte Carlo sampling.Infidelity uses noisy-baseline perturbations for local explanations and square removal for global explanations; the setup denotes vanilla gradient, integrated gradient, Guided Back-Propagation, and KernelSHAP as Grad, IG, GBP, and SHAP, respectively.Source: §5 · pp. 6–7

Explanation Sensitivity and Infidelity

• Noisy Baseline (NB) and Square achieve the lowest infidelity under their respective perturbations; Smooth-Grad improves both sensitivity and infidelity for all base explanations across datasets.Table 1 covers local explanations on MNIST and global explanations on MNIST, CIFAR-10, and ImageNet. SHAP has the lowest infidelity under the SHAP-defined perturbation.Source: Table 1 · p. 7, §5 · p. 7

Visualization

• NB and Square produce cleaner maps that highlight classified objects; on MNIST, NB also reveals regions whose brightening would increase the prediction score.Square alone highlights the whole bannister in one ImageNet example, while NB shows the digits and a score-increasing region above a 6.Source: Figure 1 · p. 7, Figure 2 · p. 7, §5 · p. 7

Sanity Check

• Explanations without absolute values pass the randomization sanity check; with absolute values, correlations are high, but Square has the lowest correlation and most distinct visualizations.The comparison used pretrained and randomized ResNet-50 models, with the final fully connected layer randomized.Source: Figure 3 · p. 8, §5 · p. 8

Human Evaluation

• In a 16-user study, infidelity generally aligned with human accuracy at identifying whether the image or caption drove model predictions.The study used two models with test accuracy above 0.95 and four explanations; OPT had the best infidelity score by construction.Source: §5 · p. 8

6 Related Work

• Prior work proposes objective evaluations for saliency explanations; perturbation-based attributions measure prediction differences after perturbing feature sets, while other approaches assess explanation continuity.Source: Table 3 · p. 9
• Sample-based methods attribute decisions to previously observed samples [21, 45, 17], whereas concept-based methods use high-level human concepts [18, 14, 6]; attribution explanations are generally applicable to a wide range of tasks and easy to understand.Perturbation-based attributions measure prediction differences after perturbing features; [47] uses grey-patch occlusions, while [49, 7] add a generative model, similar in spirit to counterfactual visual explanations [15]. Gradient methods [5, 38, 47, 41, 35] include variants [3, 4, 37, 43], and feature-importance averaging has cooperative-game roots [11, 25].Source: Table 3 · p. 9
• Prior work studies explanation sensitivity and reliability; this paper reports that fidelity-optimal explanations pass parameter-randomization sanity checks and SmoothGrad lowers sensitivity and infidelity.Kindermans et al. [19] and SmoothGrad [40] address sensitivity; Ribeiro et al. [32], extended by [46, 30], use local linear models. Adebayo et al. [1] find some saliency methods insensitive to parameter randomization, while Ghorbani et al. [13] and Zhang et al. [48] show perceptually indistinguishable images can substantially change saliency.Source: Table 3 · p. 9

7 Conclusion

• The paper proposes infidelity and sensitivity metrics, unifies many existing explanations as optimizing infidelity, and characterizes its optimizer as a perturbation-kernel combination of two methods.It also proposes two perturbations with their respective optimal explanations.Source: §7 · p. 9
• The paper shows theoretically and empirically that appropriate smoothing can improve both sensitivity and infidelity, and reports alignment between infidelity and human evaluation when explanation ground truth is available.Source: §7 · p. 9

A A Case study of infidelity and sensitivity for gradient explanations

• Gradient explanations are examined because gradients are computationally inexpensive, with the case study motivating analysis of their sensitivity, faithfulness, and possible effects of adversarial training.Source: A A Case study of infidelity and… · p. 13

A.1 Sensitivity bound for gradient in Neural Networks

• For SoftPlus neural networks with zero biases, the gradient explanation's sensitivity is upper-bounded under ℓ2 distance.SoftPlus is presented as a differentiable close approximation of ReLU.Source: §A.1 · p. 13

A.2 Infidelity and Sensitivity upper bound for Gradient

• For bounded-Hessian functions around an input, gradient infidelity can be upper-bounded; gradients are fully faithful for models with no curvature.Source: §A.2 · p. 13
• A lower Hessian upper bound leads to improved infidelity and sensitivity for gradient explanations.Source: §A.2 · p. 13

A.3 Optimizing Explanation Infidelity and Gradient Sensitivity by Adversarial Training

• A Hessian regularizer may be expensive, while L2 weight regularization can lower gradient sensitivity; gradient-norm penalties also involve Hessians and large training time [33].Source: §A.3 · p. 14
• Adversarial training minimizes expected worst-case classification loss over bounded perturbations, including perturbations that change a point’s label.The adversarial loss takes the supremum of the classification loss over perturbations of norm at most ϵ.Source: §A.3 · p. 14
• Optimizing adversarial risk effectively penalizes gradient and Hessian norms, suggesting models with small, smooth gradients that are robust to perturbations.The section says gradient explanations of adversarially trained models are naturally robust to perturbations.Source: §A.3 · p. 14
• The experiments use Projected Gradient Descent [26] to train an adversarially robust network.Source: §A.3 · p. 14

A.4 Proof of Theorem A.1

• The proof bounds logistic adversarial loss by clean loss plus an additional term u(x).Source: §A.4 · pp. 14–15

A.5 A toy example

• In the toy example, nearby inputs have different raw gradients, while gradient-SG is close to (0.5, 0.5), more faithful to output and less sensitive across inputs.At (20, 11.9), the gradient is (1, 0); at (20, 12.1), it is (0, 1).Source: §A.5 · p. 15

B.1 Detailed Experiment Settings

• MNIST uses a trained CNN with over 99% test accuracy, while CIFAR-10 uses a baseline Wide-ResNet with 94% test accuracy.For MNIST and CIFAR-10, the Square perturbation sizes range from 1 x 1 to 10 x 10 and 1 x 1 to 15 x 15, respectively; ImageNet uses 10 x 10 to 30 x 30.Source: §B.1 · pp. 15–16
• Experiments compare Grad, IG, GBP, and SmoothGrad variants alongside NB and Square, reporting average infidelity and sensitivity over 50 instances.Explanations are normalized to unit norm for sensitivity and optimally scaled for infidelity; 1,000 points evaluate infidelity and 50 evaluate sensitivity.Source: §B.1 · p. 16
• The baseline image is set to zero for all explanations.Source: §B.1 · p. 16
• KernelSHAP is not reported for ImageNet because it does not scale to the data and produces noise even with 2 x 2 superpixels.The passage notes that more scalable SHAP methods do not yet scale to ImageNet images.Source: §B.1 · p. 16

B.2 Implementation Tricks

• For inputs with zero-valued features, the perturbation matrix may be non-invertible; the authors derive an alternative optimal-solution form that multiplies by the image.They note that this issue always occurs for MNIST’s black-background images.Source: §B.2 · p. 16
• Selecting z so the matrix is invertible lets the authors use this alternative form when calculating SHAP and Square.The passage contrasts this approach with workarounds such as a pseudo-inverse or adding a small identity matrix.Source: §B.2 · pp. 16–17

B.4 Infidelity and Sensitivity for Robust Network

• The robust MNIST model yields lower sensitivity and infidelity for all explanations than the regularly trained model.The comparison uses the model trained in [26].Source: Table 4 · p. 17, §B.4 · p. 17

B.2 Implementation Tricks

• For evaluating global explanations under square perturbations, the authors use binary perturbations, yielding the original infidelity form.Source: §B.2 · p. 17

B.3 Sensitivity and Infidelity for Varying Smoothing Strength

• As Smooth-Grad’s smoothing radius increases from 0.1 to 2.0, infidelity first decreases and then gradually increases, while sensitivity decreases.The experiment is on MNIST; the saliency maps are normalized to zero mean and unit variance before visualization.Source: §B.3 · p. 17, Figure 6 · p. 17
• With an appropriate amount of smoothing, infidelity and sensitivity can improve simultaneously, although the least-sensitive explanation is not faithful.This conclusion is drawn from the varying-radius Smooth-Grad experiment on MNIST.Source: §B.3 · p. 17

B.5 Additional Viusalization Examples

• Additional visualization results are shown in Figures 8, 7, and 9.Source: §B.5 · p. 17

C A connection between Explanation and Model Sensitivity and Infidelity

• Robust fidelity measures maximum infidelity after a small perturbation to the input, without changing the infidelity-optimal explanation.The robust measure evaluates discrepancies between the explanation and model response after perturbing x; the optimal explanation remains the same as for ordinary infidelity.Source: C A connection between Explanation and… · p. 18
• Robust infidelity is lower-bounded by A(x) − B(x) − B(x − I), relating explanation sensitivity to model sensitivity.A(x) tracks explanation sensitivity and B(x) tracks model sensitivity; the bound makes high explanation sensitivity relative to model sensitivity undesirable.Source: C A connection between Explanation and… · p. 18

D.1 Proof of Proposition 2.1

• The proof concludes that the optimal explanation is unique under the stated condition.The supplied proof excerpt does not preserve the condition's full mathematical expression.Source: §D.1 · p. 18

D.2 Proof of Proposition 2.3

• When infinitesimal coordinate perturbations yield zero infidelity, the explanation equals the input gradient, ∇_x f(x).The proof takes I = εe_i with ε approaching zero.Source: §D.2 · p. 19

D.3 Proof of Propostion 2.4

• When coordinate deletion perturbations yield zero infidelity, the explanation is occlusion-1.The proof uses I = e_i dx and relates x_iΦ_i to the change in f when coordinate i is set to zero.Source: §D.3 · p. 19

D.4 Proof of Proposition 2.5

• The proof shows the transformed attribution equals the Shapley value for the original function.The excerpt equates the Shapley value for the transformed function with that for f(h_x(·)).Source: §D.4 · p. 19

D.5 Proof of Theorem C.1

• The proof approximates A(x) by E[||I||] times SENSMAX(Φ, f, x, r).This connects the bound's explanation-sensitivity term to the SENSMAX measure.Source: §D.5 · p. 20

D.6 Proof of Theorem 4.1

• The D.6 proof excerpt links SENSMAX(Φ, f, z, r) with the kernel k(x, z) dz.The supplied passage is a formula fragment and does not include the full expression.Source: §D.6 · p. 20

D.7 Proof of Theorem 4.2

• The proof introduces T1 and T2 as shorthand for the quantities being compared, then reorders their discrepancy into a form used in the bound.The shorthand is T1(I, Φ, x) and T2(I, f, x).Source: §D.7 · pp. 20–21
• By Cauchy–Schwarz and (6), the proof bounds the integrated squared discrepancy between T1 and T2.It then uses (18) in (17) to obtain the bound when C1 < 1.Source: §D.7 · p. 21
• Because the derived bound holds for any u, the proof obtains (16), then completes the theorem using (7) and (16).The text attributes the first final inequality to (7) and the second to (16).Source: §D.7 · p. 21