Source-linked AI summary
Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs
Ruigang Fu, Qingyong Hu, Xiaohu Dong, Yulan Guo, Yinghui Gao, Biao Li
TL;DR
CNNs achieve strong vision performance but remain difficult to interpret, and existing CAM methods lack clear theoretical support. This paper introduces Sensitivity and Conservation axioms and proposes XGrad-CAM, which enhances Grad-CAM’s sensitivity, conservation, and visualization performance.
Problem
Existing CAM methods provide reasonable visualizations but lack clear and sufficient theoretical support for interpreting CNN decisions.
Method
XGrad-CAM derives feature-map weights as a gradient-weighted average under Sensitivity and Conservation axioms while retaining a linear combination of feature maps.
Results
XGrad-CAM enhances Grad-CAM in sensitivity and conservation and significantly improves visualization performance compared with Grad-CAM.
Takeaways & Limitations
XGrad-CAM offers mathematically explained visualization applicable to arbitrary classification CNNs for highlighting objects of interest.
Takeaways & Limitations
Gradient-based CAM methods generally violate continuity because of shattered gradients and implementation invariance because they are layer sensitive.
Abstract
from arXiv · showhide
To have a better understanding and usage of Convolution Neural Networks (CNNs), the visualization and interpretation of CNNs has attracted increasing attention in recent years. In particular, several Class Activation Mapping (CAM) methods have been proposed to discover the connection between CNN's decision and image regions. In spite of the reasonable visualization, lack of clear and sufficient theoretical support is the main limitation of these methods. In this paper, we introduce two axioms -- Conservation and Sensitivity -- to the visualization paradigm of the CAM methods. Meanwhile, a dedicated Axiom-based Grad-CAM (XGrad-CAM) is proposed to satisfy these axioms as much as possible. Experiments demonstrate that XGrad-CAM is an enhanced version of Grad-CAM in terms of conservation and sensitivity. It is able to achieve better visualization performance than Grad-CAM, while also be class-discriminative and easy-to-implement compared with Grad-CAM++ and Ablation-CAM. The code is available at https://github.com/Fu0511/XGrad-CAM.
1 Introduction
CNNs achieve strong performance but remain difficult to interpret, motivating CAM methods that explain predictions through image regions. XGrad-CAM introduces Sensitivity and Conservation axioms, retains class-discriminative visualization, and reports improved visualization performance over existing CAM methods.
- Motivation: CNNs perform strongly across vision tasks but are often criticized as black boxes with unexplained parameters.Interpretability is especially desirable for applications such as medical diagnosis and autonomous driving.
- Motivation: Grad-CAM highlights important image regions using deep feature maps, weighting each map by the average of its gradients.The paper identifies the lack of clear theoretical support for this weighting choice as a limitation of CAM methods.
- Method: XGrad-CAM defines feature-map weights as weighted gradient averages obtained by solving an optimization problem under Sensitivity and Conservation constraints.It remains a linear combination of feature maps.
- Method: XGrad-CAM is applicable to arbitrary classification CNNs and highlights objects of interest with class-discriminative visualizations.The paper also combines it with Guided Backprop to produce Guided XGrad-CAM, which provides more object details than XGrad-CAM.
- Results: XGrad-CAM is reported as an enhanced version of Grad-CAM in both sensitivity and conservation.The conclusion also reports significantly improved visualization performance compared with Grad-CAM.
- Results: XGrad-CAM achieves better visualization performance than Grad-CAM, Grad-CAM++, and Ablation-CAM when considering class discriminability, efficiency, and localization capability.The comparison is based on extensive experiments involving these recent CAM methods.
2 Related Work
CNN decision-visualization methods include perturbation-, propagation-, and activation-based approaches, with CAM methods using deep feature maps to localize influential regions. The paper frames Sensitivity and Conservation as requirements for relating explanations to output changes and model scores, while noting additional limitations of gradient-based CAM methods.
- Visualization Methods: CNN decision-visualization methods are categorized as perturbation-based, propagation-based, or activation-based.These methods aim to highlight image regions responsible for a CNN’s decision.
- Perturbation-Based Methods: Perturbation-based methods generate heatmaps from class-score changes after occluding, removing, masking, or altering image regions, but they are inefficient.The approach provides evidence for and against a classification.
- Propagation-Based Methods: Propagation-based methods use gradients and can be fast, but vanilla-gradient saliency maps are usually noisy.Guided Backprop, Layerwise Relevance Propagation, and DeepTaylor modify backpropagation to obtain better heatmaps.
- Activation-Based Methods: Activation-based CAM methods visualize CNN decisions by linearly combining deep feature maps, although original CAM is restricted to GAP-CNNs.Grad-CAM determines feature-map weights from average gradients, while Ablation-CAM avoids gradients but requires hundreds of forward propagations per image.
- Axioms: Visualization axioms include continuity, implementation invariance, Sensitivity, and Conservation.The paper studies Sensitivity and Conservation for CAM visualization.
- Axioms: Sensitivity links each explanation response to the model-output change caused by replacing its corresponding input feature with a baseline.The formal relation is Ri(x;m) = f(x;m) − f(x\xi;m).
- Axioms: Conservation requires the sum of explanation responses to match the model output.The paper motivates it as ensuring that the class score is mainly accounted for by feature maps rather than unexplained factors.
- Axioms: Gradient-based CAM methods can violate continuity because of shattered gradients and implementation invariance because they are layer sensitive.These limitations are stated in the paper’s axiomatic analysis.
3 Approach
XGrad-CAM derives feature-map weights from Sensitivity and Conservation axioms, then produces class-specific maps through rectification and upsampling. It approximates the axiom-based optimization in deep target layers and generalizes CAM beyond GAP-CNNs.
- CAM formulation: CAM methods linearly combine target-layer feature maps, with feature-map weights determining each map’s contribution to class c.The resulting map is ReLU-rectified and upsampled to the input size to highlight class-relevant regions.
- Axioms: Sensitivity requires each feature-map importance to equal the class-score change caused by removing that feature map.The removed map is replaced by zero when computing the changed score.
- Axioms: Conservation requires CAM responses to redistribute the class score across the feature maps rather than leave unexplained contributions.This complements Sensitivity by constraining the aggregate explanation.
- XGrad-CAM derivation: XGrad-CAM determines linear-combination weights by approximately minimizing violations of Sensitivity and Conservation.The paper normalizes the axiom-related terms because they lack a direct relationship with the optimization variables.
- XGrad-CAM derivation: In deep target layers, the normalized axiom-related terms are empirically small across several CNN models, supporting the approximation used for XGrad-CAM.The observation comes from 1000 input images and is illustrated for AlexNet, VGG-16, VGG-19, Inception_V3, and ResNet-101.
- XGrad-CAM output: XGrad-CAM equals CAM for GAP-CNNs while extending to arbitrary CNN models; Guided XGrad-CAM additionally multiplies its upsampled map element-wise by Guided Backprop.The resulting XGrad-CAM map is rectified and upsampled before identifying regions responsible for class c.
4 Experiments and Results
The experiments compare CAM methods on class discrimination, localization, efficiency, and compliance with Sensitivity and Conservation axioms. XGrad-CAM improves Grad-CAM’s localization and axiomatic behavior while balancing discriminability and efficiency.
- Experimental Setup: The experiments evaluate CAM methods using class-discriminability, localization capability, and axiomatic analysis.The comparison includes Grad-CAM, Grad-CAM++, Ablation-CAM, and XGrad-CAM.
- Perturbation Analysis: XGrad-CAM achieves a confidence-drop score of 0.491 versus 0.469 for Grad-CAM in perturbation analysis.Ablation-CAM performs similarly to XGrad-CAM but requires about 40 times more computation because it runs hundreds of forward propagations per image.
- Perturbation Analysis: Grad-CAM++ achieves the best perturbation-analysis performance, but its class-discriminability is lost on images containing multiple object classes.Class-discriminability is not reflected by confidence drop on ILSVRC-12 because those images usually contain a single object class.
- Perturbation Analysis: XGrad-CAM covers a more complete area of the object than Grad-CAM in visual examples and is presented as a practical balance of discrimination, efficiency, and localization.The paper characterizes Grad-CAM++ as not class-discriminative, Ablation-CAM as time-consuming, and Grad-CAM as insufficient for localization.
- Axiom Analysis: Grad-CAM++ breaks the Sensitivity and Conservation axioms, while lower axiom-analysis values indicate better agreement with those axioms.The authors argue that considering these axioms is important when designing visualization methods.
5 Conclusion
The paper presents XGrad-CAM as an axiom-motivated visualization method with mathematical explanations and reports improved visualization properties over Grad-CAM and related methods.
- XGrad-CAM is motivated by the Sensitivity and Conservation axioms.The axioms provide the basis for the method's derivation.
- A clear mathematical explanation is provided for CAM visualization methods.The paper also explains why Grad-CAM and Ablation-CAM can be effective from the axioms' perspective.
- XGrad-CAM enhances Grad-CAM in sensitivity and conservation while significantly improving visualization performance.
A Proof
The proof establishes a class-score decomposition for arbitrary layers in ReLU-CNNs and analyzes the associated bias terms empirically across layers and feature maps.
- For any class and arbitrary layer in a ReLU-CNN, the class score equals gradient×feature contributions plus a bias term.The result is established using mathematical induction across layers.
- The proof assumes a ReLU-CNN whose only nonlinearity is ReLU rectification.Bias terms corresponding to units output by ReLU or pooling layers are zero.
- For 1000 VGG-16 input images, the bias term is rather large in shallow layers.The class of interest is the top-1 predicted class.
- In the last spatial layer, the bias magnitude is small relative to score changes for most feature maps.Exceptions usually occur for unimportant feature maps whose removal causes only a tiny score change.
C CAM, Grad-CAM, Ablation-CAM and XGrad-CAM on GAP-CNNs
For GAP-CNNs, CAM, Grad-CAM, Ablation-CAM, and XGrad-CAM produce equivalent visualizations on the last spatial layer because their feature-map weights coincide up to an irrelevant constant.
- GAP-CNNs use fully convolutional layers, global average pooling, and a linear classifier with softmax.The class score is a weighted sum of pooled feature-map activations.
- The class-specific classifier weight connects each feature map to the target class, with bc serving as a bias.
- In GAP-CNNs, Grad-CAM, Ablation-CAM, and XGrad-CAM have exactly the same feature-map weights.
- CAM's weights differ only by a constant Z, which does not affect visualization.Therefore, all four methods achieve the same performance on the last spatial layers of GAP-CNNs.
D Additional Visualization Results
Additional visualizations examine guided CAM behavior and class discrimination, with qualitative results showing Grad-CAM++ can highlight regions belonging to irrelevant classes.
- Guided versions are used to evaluate class-discriminability because they have the same class-discriminability as the original methods.The section compares XGrad-CAM, Guided Backprop, and Guided XGrad-CAM.
- On VOC 2007 validation images containing multiple object classes, Grad-CAM++ highlights regions of irrelevant classes.The comparison is qualitative and concerns class-discriminability.
- Grad-CAM++ is reported as not class-discriminative compared with the other three CAM methods.