PaperlayerView exact revision on arXiv

Source-linked AI summary

Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks

Aditya Chattopadhyay, Anirban Sarkar, Prantik Howlader, Vineeth N Balasubramanian

arXiv:1710.11063v3cs.CV

TL;DR

CNNs achieve strong results but remain difficult to explain because their internal decisions are poorly understood. The paper proposes Grad-CAM++, a generalized gradient-based visualization method, and reports improved localization, faithfulness, and human trust across several tasks, including knowledge distillation. The authors also identify limitations in prior Grad-CAM visualizations, particularly for multiple instances and complete object coverage.

  • Problem

    Deep CNNs are perceived as black boxes because their internal functioning and decisions are difficult to explain to human users.

  • Method

    Grad-CAM++ uses weighted positive partial derivatives of final convolutional feature maps to generate class-specific visual explanations.

  • Results

    Grad-CAM++ provides improved visual explanations across localization, human-trust, faithfulness, captioning, action recognition, and knowledge-distillation evaluations.

  • Takeaways & Limitations

    The method offers a generalized approach for explaining CNN decisions across image and video tasks while addressing Grad-CAM’s localization shortcomings.

  • Takeaways & Limitations

    Grad-CAM can fail to localize multiple same-class objects and may capture only parts of a single object.

Abstract

from arXiv · show

Over the last decade, Convolutional Neural Network (CNN) models have been highly successful in solving complex vision problems. However, these deep models are perceived as "black box" methods considering the lack of understanding of their internal functioning. There has been a significant recent interest in developing explainable deep learning models, and this paper is an effort in this direction. Building on a recently proposed method called Grad-CAM, we propose a generalized method called Grad-CAM++ that can provide better visual explanations of CNN model predictions, in terms of better object localization as well as explaining occurrences of multiple object instances in a single image, when compared to state-of-the-art. We provide a mathematical derivation for the proposed method, which uses a weighted combination of the positive partial derivatives of the last convolutional layer feature maps with respect to a specific class score as weights to generate a visual explanation for the corresponding class label. Our extensive experiments and evaluations, both subjective and objective, on standard datasets showed that Grad-CAM++ provides promising human-interpretable visual explanations for a given CNN architecture across multiple tasks including classification, image caption generation and 3D action recognition; as well as in new settings such as knowledge distillation.

1 INTRODUCTION

The paper introduces Grad-CAM++ to improve visual explanations of CNN decisions, addressing Grad-CAM’s weaknesses in localizing multiple or complete objects. It evaluates the method across recognition, captioning, action recognition, human trust, faithfulness, and knowledge distillation.

  • Motivation: Grad-CAM explanations can perform poorly when localizing multiple instances of the same class and may fail to capture an entire object.These limitations motivate a more general visualization approach.
  • Method: Grad-CAM++ introduces pixel-wise gradient weighting for positions in the final convolutional feature map.The method derives closed-form weights and higher-order derivatives for softmax and exponential outputs.
  • Evaluation: Human studies report greater trust in the underlying model for Grad-CAM++ visualizations than for Grad-CAM visualizations.
  • Evaluation: Grad-CAM++ improves upon Grad-CAM in weakly supervised localization of object classes.The paper supports this conclusion through visual examples and objective evaluations.
  • Broader applications: The method extends visual explanations beyond recognition to image captioning, 3D action recognition, and knowledge distillation.In a constrained teacher-student setting, an explanation-inspired loss produced promising student results.

2 RELATED WORK

Prior CNN-interpretability methods include deconvolution, guided backpropagation, CAM, Grad-CAM, and model-agnostic local explanations. The paper positions Grad-CAM++ as a generalization addressing Grad-CAM’s incomplete localization and difficulty with multiple instances.

  • Existing CNN explanations: Deconvolution and guided backpropagation visualize image regions associated with higher-layer activations, while LIME locally approximates complex decisions with simpler interpretable models.These methods represent different strategies for probing or explaining deep models.
  • CAM and Grad-CAM: CAM uses weighted last-convolutional-layer activation maps but requires a global-average-pooling penultimate layer and post-training linear classifiers for each class.These requirements limit CAM’s applicability to architectures with a GAP penultimate layer.
  • CAM and Grad-CAM: Grad-CAM produces class-specific visual explanations for differentiable deep CNNs without retraining or architectural modification, and Guided Grad-CAM adds fine-grained detail by fusing its map with Guided Backpropagation.The Grad-CAM map is upsampled and combined with pixel-space Guided Backpropagation visualizations.
  • Grad-CAM limitations: Grad-CAM struggles to localize multiple occurrences of one class and often highlights only parts of a single object rather than the complete object.The paper links these shortcomings to Grad-CAM’s unweighted averaging of partial derivatives.
  • Grad-CAM++ motivation: Grad-CAM++ generalizes Grad-CAM to address these localization issues, with closed-form derivations and experiments designed for objective and subjective evaluation.The proposed method is presented as a more general explanation algorithm for a given CNN architecture.

3 GRAD-CAM++: PROPOSED METHODOLOGY

Grad-CAM++ replaces Grad-CAM’s unweighted gradient aggregation with pixel-wise weighting of positive gradients, producing class-specific saliency maps from final convolutional feature maps. The formulation supports smooth class scores, closed-form derivative expressions, single-pass computation, and Guided Grad-CAM++ visualizations.

  • Intuition and formulation: Grad-CAM’s unweighted averaging can cause feature maps with smaller spatial footprints to fade, weakening localization when objects or object parts activate different maps.This provides the motivation for replacing unweighted averaging with a weighted average of pixel-wise gradients.
  • Intuition and formulation: Grad-CAM++ assigns pixel-wise importance to final convolutional feature-map locations based on their contribution to a selected class score.The method is motivated by feature-map pixels that contribute to object presence and uses weighting coefficients for each spatial position.
  • Weighting design: Grad-CAM++ uses positive gradients to emphasize visual features that increase the output neuron’s activation, with closed-form solutions for the pixel-wise weighting coefficients.The paper derives higher-order derivative expressions for exponential and softmax output functions.
  • Weighting design: When α^kc_ij = 1/Z, Grad-CAM++ reduces to Grad-CAM, making it a generalized formulation of the earlier method.The paper explicitly characterizes Grad-CAM++ as a generalized version of Grad-CAM under this special case.
  • Empirical intuition: The paper’s visual examples compare CAM, Grad-CAM, and Grad-CAM++, illustrating Grad-CAM++’s better coverage of relevant regions than the unweighted Grad-CAM map.The examples address multiple occurrences and object localization, while the method overview presents the three computation expressions.
  • Computational properties: The formulation permits smooth class scores including exponential and softmax functions, and the gradient weights can be computed with a single backward pass.For linear or ReLU final-layer activations, higher-order derivative computation becomes simpler; softmax remains computable in one backward pass.
  • Saliency-map construction: The Grad-CAM++ saliency map is a ReLU-applied linear combination of forward activation maps, and Guided Grad-CAM++ additionally fuses its upsampled map with Guided Backpropagation.The fused visualization is produced at image resolution through pointwise multiplication.

4 EXPERIMENTS AND RESULTS

The experiments evaluate explanation faithfulness, human trust, and object localization by comparing Grad-CAM++ with Grad-CAM across recognition and localization settings. Grad-CAM++ performs better on objective explanation metrics, receives higher human preference, and improves localization, especially at higher thresholds.

  • Experimental design: Grad-CAM++ was evaluated against Grad-CAM using objective faithfulness metrics, human studies, and object-localization experiments across ImageNet and PASCAL VOC datasets.Experiments used an off-the-shelf VGG-16 model, with additional results for AlexNet and ResNet-50.
  • Objective evaluation: The objective evaluation measured average confidence drop, percentage confidence increase, and Win %, with Grad-CAM++ outperforming Grad-CAM on all three metrics.The metrics assess whether explanation regions align with the model’s class prediction.
  • Human trust: The human study found that Grad-CAM++ visualizations instilled greater trust in the underlying model than Grad-CAM visualizations.Thirteen subjects compared paired explanation maps without knowing which method generated each map.
  • Object localization: For PASCAL VOC 2012 localization, Grad-CAM++ improved over Grad-CAM on the IoU-based localization metric, with the improvement increasing at larger threshold values δ.The evaluation used the same threshold for both methods, and Figure 6 illustrates improved localization examples.

5 LEARNING FROM EXPLANATIONS: KNOWLEDGE DISTILLATION

The paper transfers teacher-network knowledge through Grad-CAM++ explanations, augmenting student training with an interpretability loss alongside classification loss. Experiments report improved explanation-based distillation, including for substantially smaller students.

  • Loss formulation: The student loss combines standard cross-entropy with an interpretability loss derived from teacher explanations generated by Grad-CAM++.The interpretability term encourages the student to learn image regions relevant to the teacher’s decision.
  • CIFAR-10 experiment: Knowledge distillation using teacher explanations improved student training relative to standard approaches, and Grad-CAM++ outperformed Grad-CAM for explanation-based distillation.The teacher’s error rate was 5.8% in the CIFAR-10 experiment.

6 EXPLANATIONS FOR IMAGE CAPTIONING AND 3D ACTION RECOGNITION

Grad-CAM++ extends visual explanations to CNN-based image captioning and 3D action recognition. Across these tasks, it produced more complete caption explanations and better video-task performance than Grad-CAM.

  • Task scope: Grad-CAM++ can explain decisions from models that use a CNN as an integral module, including image-captioning and video-recognition systems.The paper presents visual explanations for both image captions and 3D action recognition.
  • Image Captioning: In Flickr30k image captioning, Grad-CAM++ produced more complete heatmaps than Grad-CAM for predicted captions.For one caption, Grad-CAM++ highlighted both the girl and plant, whereas Grad-CAM highlighted only the girl.
  • Image Captioning: Across five captions for the same image, Grad-CAM++ provided more complete explanations corresponding to the captions than Grad-CAM.The captions described variations of a boy riding or racing a bicycle.
  • 3D Action Recognition: Grad-CAM++ performed better than Grad-CAM on all evaluated metrics for 3D action recognition using 3k Sports-1M videos.The experiment used a C3D model and 16-frame video windows; qualitative results subsampled each video to six frames.

7 DISCUSSION

The discussion examines why Grad-CAM++ uses positive gradients and whether its apparent gains merely reflect larger explanation regions. The reported analyses support positive-gradient weighting and comparable or better faithfulness across spatial extents.

  • Why only Positive Gradients in Grad-CAM++?: Grad-CAM++⊥ removes the restriction to positive gradients by incorporating all gradients when weighting activation maps.The modified method retains the Grad-CAM++ framework but drops the ReLU operation.
  • Why only Positive Gradients in Grad-CAM++?: Grad-CAM++⊥ performed poorly compared with Grad-CAM, supporting the claim that positive gradients are critical for determining activation-map importance.The comparison used objective evaluation on the Pascal VOC 2007 validation set.
  • Faithfulness and explanation size: The occlusion analysis varies threshold θ to measure the trade-off between explanation-region area and relative class confidence after occlusion.The analysis compares Grad-CAM and Grad-CAM++ using ROC curves.
  • Faithfulness and explanation size: At each quantile θ, Grad-CAM++ highlighted regions that were at least as faithful as Grad-CAM, regardless of spatial extent.Faithfulness was assessed through the model’s relative confidence after occluding explanation regions.

8 CONCLUSION

The paper concludes that Grad-CAM++ generalizes gradient-based visual explanations for CNNs, addressing Grad-CAM’s localization shortcomings and extending across tasks and knowledge distillation.

  • Grad-CAM++ provides a generalized approach for visual explanations of CNN-based architectures.The method is presented as a simple and effective generalization of earlier gradient-based explanation methods.
  • The method addresses Grad-CAM’s shortcomings, especially poor object localization and multiple occurrences of a class in one image.
  • Grad-CAM++ was evaluated objectively for model faithfulness and subjectively for human trust using standard CNN models and ImageNet and Pascal VOC datasets.
  • The approach also showed superior performance on image caption generation and video understanding for action recognition.
  • Using explanations for knowledge distillation enabled a student network to learn better representations than the original teacher network, with lower test error.

SUPPLEMENTARY MATERIALS

The supplementary materials extend the experiments beyond VGG-16 to AlexNet and ResNet-50, covering objective evaluations and additional image-captioning results.

  • Experiments use AlexNet and ResNet-50 to test whether the reported results depend on CNN architecture.The last convolutional feature maps serve as the activation maps for both architectures.
  • The supplementary object-recognition analysis compares Grad-CAM and Grad-CAM++ objectively on AlexNet and ResNet-50.
  • Additional qualitative results examine visual explanations for image captioning experiments.

A.1 Evaluation of Object Recognition Performance with Other Architectures

Across AlexNet and ResNet-50, supplementary objective evaluations follow the main-text trend and further support improved Grad-CAM++ performance.

  • Qualitative results are provided for AlexNet and ResNet-50 in Figures 11 and 12.
  • The Average drop % metric tracks the generalization ranking of the networks across ImageNet.The reported ranges are 28–31% for ResNet-50, 36–47% for VGG-16, and 62–83% for AlexNet.

A.2 Additional Results on Image Captioning

Additional image-captioning examples compare Grad-CAM and Grad-CAM++ across caption variations and architectures, with Grad-CAM++ providing more complete explanations.

  • The supplementary tables report objective explanation evaluations on ImageNet and Pascal VOC for ResNet-50 and AlexNet.
  • The additional captioning experiments consider images paired with five different captions.
  • Grad-CAM++ provides more complete visual explanations across the reported captioning images.
  • Figures 11 and 12 compare Grad-CAM and Grad-CAM++ explanations for AlexNet and ResNet-50 decisions.Their panels illustrate poor class localization and multiple occurrences of the same class.
  • Figures 13–15 show visual explanations for images generating five alternative captions involving children, women, and a priest.
Loading 1710.11063v3…