PaperlayerView exact revision on arXiv

Source-linked AI summary

Use HiResCAM instead of Grad-CAM for faithful explanations of convolutional neural networks

Rachel Lea Draelos, Lawrence Carin

arXiv:2011.08891v4eess.IVcs.CVcs.LG

TL;DR

Grad-CAM can highlight locations the model did not use, undermining its reliability as an explanation method. The paper proposes HiResCAM, proves its faithfulness for CNNs ending in one fully connected layer, and finds that Grad-CAM’s attention expansion benefits weakly supervised segmentation.

  • Problem

    Grad-CAM’s gradient averaging can produce misleading explanations by highlighting locations the model did not use, a concern for understanding model behavior.

  • Method

    HiResCAM is a class-specific CAM-family method whose element-wise computation identifies locations increasing the class score and generalizes CAM.

  • Results

    HiResCAM faithfully reflects model computations, whereas Grad-CAM produces misleading explanations but often expands attention and achieves superior weakly supervised segmentation performance.

  • Takeaways & Limitations

    HiResCAM provides faithful, class-specific explanations for supported CNN architectures, while Grad-CAM’s attention expansion may remain useful for downstream segmentation tasks.

  • Takeaways & Limitations

    The faithfulness guarantee requires applying HiResCAM at the last convolutional layer of CNNs ending in one fully connected layer; arbitrary architectures lack this guarantee.

Abstract

from arXiv · show

Explanation methods facilitate the development of models that learn meaningful concepts and avoid exploiting spurious correlations. We illustrate a previously unrecognized limitation of the popular neural network explanation method Grad-CAM: as a side effect of the gradient averaging step, Grad-CAM sometimes highlights locations the model did not actually use. To solve this problem, we propose HiResCAM, a novel class-specific explanation method that is guaranteed to highlight only the locations the model used to make each prediction. We prove that HiResCAM is a generalization of CAM and explore the relationships between HiResCAM and other gradient-based explanation methods. Experiments on PASCAL VOC 2012, including crowd-sourced evaluations, illustrate that while HiResCAM's explanations faithfully reflect the model, Grad-CAM often expands the attention to create bigger and smoother visualizations. Overall, this work advances convolutional neural network explanation approaches and may aid in the development of trustworthy models for sensitive applications.

1 Introduction

Gradient-based visual explanations are computationally efficient and class-specific methods such as CAM and Grad-CAM are widely used, including in sensitive applications. The paper identifies a limitation of Grad-CAM’s gradient averaging that can produce misleading explanations.

  • Gradient-based visual explanation methods for CNNs are popular because they are computationally efficient.
  • CAM and Grad-CAM provide class-specific explanations and have been deployed in sensitive settings including medical imaging.
  • Grad-CAM’s gradient averaging step is not guaranteed to reflect the locations used for prediction and can produce misleading explanations.
  • HiResCAM is proposed as a faithful CAM-family explanation method to address this limitation.

Grad-CAM HiResCAM

The paper compares Grad-CAM and HiResCAM explanations for identical models, images, and classes. HiResCAM faithfully reflects model-used locations, whereas Grad-CAM can produce larger and smoother visualizations that differ from the model’s calculations.

  • HiResCAM explanations are often more focal than Grad-CAM explanations, but can sometimes highlight more image regions.
  • Figure 1 shows cases where HiResCAM highlights the correct object while Grad-CAM does not, and cases where their highlighted regions differ in extent.
  • HiResCAM is proved faithful to model-used locations, while Grad-CAM explanations can fail to reflect the model’s computations.
  • Experiments and crowd-sourced assessment show that Grad-CAM explanations are often bigger and rounder than faithful HiResCAM explanations.

2 Related Work

Related work includes input-level and output-level gradient-based explanation methods. CAM and Grad-CAM are class-specific, less noisy, and widely used, but the paper argues that Grad-CAM can highlight locations the model did not use.

  • 2.1 Input-level approaches: Input-level methods compute gradients with respect to the input image, but shattered gradients can give their explanations a white-noise appearance and reduce class specificity in practice.
  • 2.1 Input-level approaches: Gradient × Input multiplies the input image element-wise by the class-score gradient, while ϵ-LRP is equivalent to a modified form of this method.
  • 2.2 Output-level approaches: CAM produces class-specific explanations by weighting feature maps with final-layer class weights before global average pooling.
  • 2.2 Output-level approaches: Grad-CAM generalizes CAM by averaging gradients over spatial dimensions to obtain feature-map importance weights.
  • 2.2 Output-level approaches: CAM and Grad-CAM are widely used in weakly supervised localization and model explanation, including sensitive medical applications.
  • 2.2 Output-level approaches: The paper identifies Grad-CAM as unreliable because it sometimes highlights locations the model did not use, motivating HiResCAM.

3 Methods

The paper formulates class-specific output-level CNN explanations and introduces HiResCAM to preserve spatially varying gradient effects that Grad-CAM averages away. HiResCAM computes element-wise gradient–feature-map products before summing over features and can operate across CNN feature-map dimensionalities.

  • 3.1 Problem setup: Class-specific visual explanation seeks an attention map whose higher values identify input regions that increase a class score.
  • 3.1 Problem setup: Output-level methods compute explanations from convolutional feature maps, then upsample and superimpose them on the input image.
  • 3.2 Motivation: HiResCAM is an output-level, gradient-based method that produces a separate spatial attention map for every class.
  • 3.2 Motivation: Grad-CAM averages gradients across spatial dimensions, limiting how precisely its visualization depicts locations used for prediction.
  • 3.2 Motivation: HiResCAM preserves element-level rescaling and sign changes that Grad-CAM blurs across each feature map.
  • 3.4 HiResCAM formulation: HiResCAM first computes the class-score gradient with respect to feature maps, matching Grad-CAM’s first step.
  • 3.4 HiResCAM formulation: It then element-wise multiplies gradients and feature maps before summing over the feature dimension.
  • 3.4 HiResCAM formulation: HiResCAM supports feature maps with any number of spatial dimensions and can be applied at any convolutional layer.

3.5 CNNs ending in one fully connected layer

For CNNs ending in one fully connected layer, HiResCAM reflects the class-score calculation and is guaranteed to highlight locations that increase the score. Grad-CAM can instead highlight locations the model did not use because its weights average across spatial positions.

  • HiResCAM applied at the last convolutional layer highlights all image locations that increase the class score.This guarantee applies to CNNs ending in one fully connected layer.
  • The architecture consists of convolutional feature maps followed by one final fully connected layer producing class scores.
  • A worked example shows HiResCAM directly reflects the class-score calculation whereas Grad-CAM does not.Integer inputs are used only for simplicity; actual weights and activations are not integers.
  • HiResCAM uses spatially aligned products of final-layer weights and feature-map activations to preserve the input-specific class-score calculation.The spatial dimensions of the weights are inferred to match the feature maps.
  • Grad-CAM averages fully connected-layer weights for each feature map, preventing its explanation from reflecting the class-score calculation.

3.6 CNNs ending in global average pooling then one fully-connected layer: the CAM architecture

For CAM-architecture CNNs, HiResCAM reduces to CAM after normalization while remaining applicable to a broader class of architectures. CAM, HiResCAM, and last-layer Grad-CAM therefore produce identical explanations in this architecture.

  • CAM architecture consists of convolutional feature maps, global average pooling, and one final fully connected layer.
  • HiResCAM is proven to be a generalization of CAM for this architecture.
  • HiResCAM and CAM differ only by a constant factor of 1/(D1D2), which disappears during normalization.
  • CAM, last-convolutional-layer HiResCAM, and last-convolutional-layer Grad-CAM produce identical explanations for CAM-architecture CNNs.Examples include ResNets, GoogLeNet, and DenseNets.
  • HiResCAM is broader than CAM because it also applies to CNNs outside the CAM architecture.

3.7 All other CNNs

Outside the one-fully-connected-layer setting, HiResCAM and Grad-CAM can be computed but lack a proof that they highlight only relevant locations. The paper recommends CAM or HiResCAM only within their supported architectural scopes.

  • For CNNs ending in multiple fully connected layers or other architectures, HiResCAM and Grad-CAM are computable but not provably faithful.
  • No gradient-based explanation method is known to guarantee consistently class-specific relevant locations for every arbitrary CNN architecture or layer.
  • For sensitive applications, the paper recommends CAM with CAM-architecture CNNs or HiResCAM with CNNs ending in one fully connected layer.
  • HiResCAM is related to Gradient × Input but avoids raw-input multiplication and can be integrated into training without extra backward passes when used as recommended.

4 Model explanation is not weakly-supervised segmentation

Model explanation asks which locations a classifier used for a prediction, whereas weakly-supervised segmentation aims to cover all pixels belonging to an object. Consequently, segmentation-oriented expansion and explanation faithfulness are different objectives.

  • Model explanation: Model explanation should identify locations used for a particular prediction, including relevant background when the model relies on it.For example, explanations should highlight tracks used to identify a train or water used to identify a boat.
  • Weakly-supervised segmentation: Weakly-supervised segmentation instead seeks to identify all object pixels from whole-image labels.Because classifiers often focus on discriminative object parts, WSS methods expand attention beyond those parts.
  • Different goals: CAM- and Grad-CAM-based WSS methods therefore pursue a fundamentally different goal from the explanation methods they leverage.WSS methods may deliberately expand classifier attention to cover more of an object.
  • Evaluation: IoU should not evaluate explanation correctness because models may use background or correlated objects rather than the relevant object.Using object segmentation as the reference assumes the classifier always relied on that object, contrary to the motivation for explanation methods.
  • HiResCAM’s objective: HiResCAM is intended to faithfully represent locations used for prediction, even when those locations lie outside the object of interest.Its purpose is accurate explanation rather than expanding attention or maximizing WSS performance.

5 Experiments

Experiments compare HiResCAM and Grad-CAM on natural-image explanations, WSS utility, human perception, intermediate calculations, and medical-image visualizations. HiResCAM exactly reflects used locations, while Grad-CAM often expands attention and can perform better for WSS.

  • Datasets: The study evaluates natural-image explanations on PASCAL VOC 2012 and provides a qualitative medical-imaging comparison on RAD-ChestCT.PASCAL VOC includes 1,449 validation images and 2,148 segmentation maps; RAD-ChestCT contains 36,316 scans with 83 abnormalities.
  • Explanation correctness: For CNNs ending in one fully connected layer, HiResCAM’s mean L2 distance to the locations used by the model is always 0, whereas Grad-CAM’s is always nonzero.The result directly tests whether each explanation matches the model’s actual computational locations.
  • Weakly-supervised segmentation: Grad-CAM outperforms HiResCAM on weakly-supervised segmentation, potentially because its explanations expand beyond the regions used by the model.The paper explicitly treats WSS performance as distinct from explanation correctness.
  • Model behavior: HiResCAM IoU reveals that the evaluated models use some background and correlated objects when predicting classes.Because HiResCAM reflects model computations, this IoU is used to assess model behavior rather than explanation-method correctness.
  • Human evaluation: In AMT evaluations, workers typically judged Grad-CAM explanations bigger, smoother or rounder, and more focused on the relevant object.The evaluation used 250 image-class pairs, five workers per question, and 3,750 total judgments.
  • Mechanism: Intermediate calculations suggest that Grad-CAM expands attention by over-emphasizing top feature maps.Figure 4 also presents paired explanations for 72 randomly selected image-class pairs using the same inputs and classes.
  • Medical imaging: Medical-image visualizations show that Grad-CAM can falsely suggest attention to the wrong anatomical structure, while HiResCAM indicates use of the lung fields for lung findings.The methods use exactly the same model and input CT volume; only the explanation method differs.

6 Conclusion

The paper presents HiResCAM as a faithful generalization of CAM for model understanding, while identifying Grad-CAM’s attention expansion as useful for downstream segmentation. Grad-CAM can mislead explanations despite that utility.

  • Conclusion: HiResCAM is guaranteed to highlight locations used by any CNN ending in one fully connected layer.The paper characterizes it as a new generalization of CAM.
  • Conclusion: Grad-CAM can create misleading explanations because its attention expands beyond the model’s calculations.The expansion may nevertheless improve weakly-supervised segmentation performance.
  • Conclusion: Figure 8 shows Grad-CAM creating the incorrect impression that AxialNet focused on the wrong anatomical structure, whereas HiResCAM identifies the lung fields for lung findings.The paired explanations use the same model and CT volume and differ only in explanation method.

7 Appendix

The appendix details the experimental models and the step-by-step construction of Grad-CAM and HiResCAM visualizations. It also explains feature selection, CAM rendering, and why the final-layer bias does not affect these visualizations.

  • Models: The appendix describes ResNet-34v and DenseNet-121v, both initialized with ImageNet-pretrained convolutional layers and ending with custom convolutional and fully connected components.These model details support the natural-image experiments reported in the main paper.
  • Feature selection: Grad-CAM selects top features using the highest average α_m times the activation map, while HiResCAM uses the highest average gradient times activation map.The two methods therefore rank feature dimensions using different quantities.
  • Figure notation: Feature columns marked g, h, or gh indicate whether features rank among Grad-CAM’s, HiResCAM’s, or both methods’ top features.A negative parenthetical marker denotes a negative α_m value for Grad-CAM.
  • Visualization pipeline: The CAM row progresses from the raw feature sum through rescaling and upsampling to the explanation overlaid on the input image.All 512 features contribute to the raw CAM, including features omitted from the displayed step-by-step figure.
  • Bias term: The final fully connected layer’s bias disappears from the gradient calculation and therefore does not alter CAM or HiResCAM visualizations.The bias is input-independent, so it cannot contribute input-specific information to the explanation.
Loading 2011.08891v4…