Source-linked AI summary
Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Models
Daniel Omeiza, Skyler Speakman, Celia Cintas, Komminist Weldermariam
TL;DR
Deep CNNs are difficult to explain, while existing visualization methods have limitations in architecture applicability and object localization. The paper introduces Smooth Grad-CAM++, which averages gradients from perturbed images before applying Grad-CAM++, and reports improved visual maps, localization, and multiple-object capture.
Problem
Existing CNN visualization methods can underperform for multiple same-class objects, fail across all CNN architectures, or incompletely capture single objects.
Method
Smooth Grad-CAM++ averages gradients from many small image perturbations and applies them within the Grad-CAM++ algorithm, with selectable layers, feature maps, and neurons.
Results
Smooth Grad-CAM++ produces improved visual maps, object localization, and maps for multiple occurrences of the same class compared with existing methods.
Takeaways & Limitations
The technique provides inference-level visual explanations for CNN layers, feature maps, and individual neurons, supporting insight into what CNN models learn.
Abstract
from arXiv · showhide
Gaining insight into how deep convolutional neural network models perform image classification and how to explain their outputs have been a concern to computer vision researchers and decision makers. These deep models are often referred to as black box due to low comprehension of their internal workings. As an effort to developing explainable deep learning models, several methods have been proposed such as finding gradients of class output with respect to input image (sensitivity maps), class activation map (CAM), and Gradient based Class Activation Maps (Grad-CAM). These methods under perform when localizing multiple occurrences of the same class and do not work for all CNNs. In addition, Grad-CAM does not capture the entire object in completeness when used on single object images, this affect performance on recognition tasks. With the intention to create an enhanced visual explanation in terms of visual sharpness, object localization and explaining multiple occurrences of objects in a single image, we present Smooth Grad-CAM++ \footnote{Simple demo: http://35.238.22.135:5000/}, a technique that combines methods from two other recent techniques---SMOOTHGRAD and Grad-CAM++. Our Smooth Grad-CAM++ technique provides the capability of either visualizing a layer, subset of feature maps, or subset of neurons within a feature map at each instance at the inference level (model prediction process). After experimenting with few images, Smooth Grad-CAM++ produced more visually sharp maps with better localization of objects in the given input images when compared with other methods.
1 Introduction
Deep CNNs achieve strong results across computer-vision tasks but remain difficult to explain, motivating visualization methods for understanding image-classification decisions. Smooth Grad-CAM++ combines gradient smoothening with Grad-CAM++ to improve visual appeal, localization, and class-object capturing.
- Deep learning models perform well across tasks including image classification, object detection, semantic segmentation, image captioning, and visual question answering.
- Conventional intelligent models often lack the ability to explain their decisions to human users, especially in risk-averse domains.
- Saliency methods identify image regions or pixels that particularly influence a model’s final classification.
- Smooth Grad-CAM++ introduces gradient smoothening into Grad-CAM++ to visualize layers, feature-map subsets, and neuron subsets with improved localization and class-object capturing.It averages gradient matrices generated from multiple noised versions of the sample image.
2 Background
CNN visualization research progressed from deconvolution and guided backpropagation to CAM, Grad-CAM, Grad-CAM++, and SmoothGrad. These methods differ in architecture applicability, localization formulation, and whether averaging gradients improves map quality.
- Deconvnet and guided backpropagation visualize image regions or features associated with highly activated neurons across CNN layers.
- CAM provides localization for CNNs with global average pooling but is limited to modified architectures without fully connected layers.
- Grad-CAM generalizes CAM to any CNN-based architecture and requires no retraining.
- Grad-CAM computes class-score gradients with respect to the last convolutional feature map, globally averages them into weights, and combines feature maps into a heatmap.
- Grad-CAM++ weights visual explanations using positive partial derivatives of last-convolutional-layer feature maps with respect to a target class score.
- SmoothGrad sharpens gradient-based sensitivity maps by averaging maps from random samples near an input, and Smooth Grad-CAM++ applies this smoothing to Grad-CAM++ gradients.The noise is Gaussian with standard deviation σ, and the resulting gradients are used in the Grad-CAM++ algorithm.
3 Method
Smooth Grad-CAM++ averages gradients from noised inputs and applies them within Grad-CAM++ to generate saliency maps at selected layers, feature maps, or neurons. Its API supports configurable noise, feature-map selection, neuron regions, and coordinate subsets.
- 3.2 Gradients Averaging: Smooth Grad-CAM++ averages first-, second-, and third-order partial derivatives across noised inputs before computing Grad-CAM++ weights.The noised samples are generated by adding Gaussian noise, and the averaged derivatives are substituted into the Grad-CAM++ computation.
- 3.2 Gradients Averaging: The method substitutes the averaged Grad-CAM++ weights into the weighted feature-map computation to produce the final class-discriminative saliency map.The resulting map can be plotted with matplotlib or another image plotting library.
- 3.5 Choosing Feature Maps: The figures specify saliency visualizations for selected feature maps and neurons using nsample=5 and σ = 0.3.The examples include feature maps 3, 10, and 32, and neuron coordinates in feature maps 3 and 10.
- 3.4 Choosing Layer: The API allows visualization of one convolutional layer per instance, and the paper experiments with the last convolutional layer of a pre-trained VGG-16 model.Any learned deep CNN model can be selected for visualization.
- 3.5 Choosing Feature Maps: The filter parameter selects feature-map indices in the chosen convolutional layer, generating one map for each selected feature map.Feature-map indices correspond to the k values used in the equations.
- 3.6 Choosing Neurons: Smooth Grad-CAM++ can visualize subsets of neurons within a feature map, either across coordinate regions or at individually specified coordinates.With region=true, coordinates define bounds; with region=false, only the listed coordinates are visualized while other activations are clipped at zero.
4 Results
Smooth Grad-CAM++ produced clearer visual explanations, localized objects effectively, and captured larger portions of objects in the tested images.
- Smooth Grad-CAM++ produced clearer explanations of features learned by the model.
- Smooth Grad-CAM++ highlighted a larger portion of the water-bird’s legs.
- Smooth Grad-CAM++ captured more of the class object in the dog image and provided good localization.
- The technique visualized three selected feature maps and provided neuron-level saliency maps for specified feature maps.The selected feature maps were 10, 32, and 3.
5 Conclusion
The paper proposes Smooth Grad-CAM++ to improve visual explanations of CNNs at inference time. It reports improved maps, object localization, and handling of multiple same-class occurrences, while supporting visualization of selected layers, feature maps, and neurons.
- Smooth Grad-CAM++ averages gradients from perturbed images before applying the generalized Grad-CAM algorithm.
- The method improves generated visual maps compared with existing methods and performs well in object localization and multiple same-class object occurrences.
- Smooth Grad-CAM++ creates maps for specific layers, feature-map subsets, and neurons of interest.