Source-linked AI summary
SmoothGrad: removing noise by adding noise
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, Martin Wattenberg
TL;DR
Deep-network explanations often rely on gradient-based sensitivity maps, yet these maps can be visually noisy and difficult to interpret. The paper introduces SmoothGrad, which averages maps from noisy image perturbations, and reports more visually coherent maps, while identifying unresolved questions about its mechanism, evaluation, and generality.
Problem
Gradient-based sensitivity maps can be visually noisy, making it difficult to determine whether highlighted pixels reflect meaningful network behavior.
Method
SmoothGrad averages sensitivity maps computed from multiple noisy versions of an input, while training with noise provides a complementary smoothing technique.
Results
Experiments suggest SmoothGrad produces visually more coherent sensitivity maps than unsmoothed gradients and can align better with meaningful features.
Takeaways & Limitations
Gradient-based sensitivity maps can be sharpened by averaging maps from perturbed inputs, with further enhancement from training on randomly perturbed data.
Takeaways & Limitations
The proposed explanation for the sharpening effect is only a plausibility argument, and other causes such as differential noise effects across textures remain possible.
Abstract
from arXiv · showhide
Explaining the output of a deep network remains a challenge. In the case of an image classifier, one type of explanation is to identify pixels that strongly influence the final decision. A starting point for this strategy is the gradient of the class score function with respect to the input image. This gradient can be interpreted as a sensitivity map, and there are several techniques that elaborate on this basic idea. This paper makes two contributions: it introduces SmoothGrad, a simple method that can help visually sharpen gradient-based sensitivity maps, and it discusses lessons in the visualization of these maps. We publish the code for our experiments and a website with our results.
1. Introduction
Interpreting deep networks is difficult, and image explanations often use pixel-level influence maps to illuminate classification decisions. The paper introduces SmoothGrad, compares it with gradient-based methods, and discusses visualization practices.
- Interpretability matters for understanding complex models and for building applications in domains such as health care and education.
- Image-classification explanations commonly identify influential image regions using occlusion techniques or gradient-based pixel importance values.
- Sensitivity maps can highlight meaningful regions but often appear visually noisy, leaving unclear whether the noise reflects network behavior or superficial factors.
- SmoothGrad averages sensitivity maps from noisy versions of an image and can be combined with other sensitivity-map algorithms.
- The paper compares SmoothGrad with gradient-based methods, offers empirical evidence and a conjecture about its operation, and discusses visualization enhancements.
2. Gradients as sensitivity maps
Gradient sensitivity maps assign pixel influence through derivatives of class scores, but raw gradients can be noisy because derivatives may fluctuate sharply at small scales. SmoothGrad approximates Gaussian smoothing by averaging gradients over noisy nearby inputs.
- Image classifiers choose the class with the highest activation score among the classes in C.
- For piecewise-differentiable class scores, a sensitivity map is formed by differentiating the class score with respect to the input image.
- The gradient estimates how much a tiny pixel change would alter a class score, so large values may indicate key image regions.
- Raw gradient maps are typically visually noisy and correlate only roughly with regions humans consider meaningful.
- Prior methods address limitations of local gradients by estimating global pixel importance or modifying backpropagation to emphasize positive contributions.
- SmoothGrad treats noisy maps as potentially arising from sharp small-scale derivative fluctuations and averages sensitivity maps sampled around the input with Gaussian noise.The method uses n samples and Gaussian noise with standard deviation σ.
3. Experiments
Experiments evaluate SmoothGrad on image-classification models and examine both sensitivity-map generation and visualization. The results suggest smoother maps align more closely with meaningful features, while visualization choices substantially affect impressions.
- SmoothGrad’s estimated smoothed gradient produced visually more coherent sensitivity maps than the unsmoothed gradient, aligning better with meaningful features to human observers.
- Experiments used an Inception v3 model trained on ILSVRC-2013 and a convolutional MNIST model based on a TensorFlow tutorial.
- Sensitivity maps are commonly shown as heatmaps, but channel-to-color mappings can substantially change the resulting visual impression.
Absolute value of gradients
Gradient-based sensitivity maps can contain signed values and a few extreme pixels, making visualization choices consequential. Taking absolute values and capping outliers are separate choices that affect how maps are displayed.
- Signed values: Sensitivity maps often contain signed values, so visualizations must choose whether to distinguish positive and negative values or show absolute values.The useful choice depends on dataset characteristics; for MNIST, positive gradients indicate positive class signal because digits share a color.
- Outlying values: A few pixels can have gradients much higher than average, potentially distorting color scales.Such outlying values can make maps appear almost entirely black without post-processing.
- Visualization setup: Figure 3 visualizes averaged sensitivity maps after adding Gaussian noise to input pixels across 50 samples at varying normalized noise levels.The noise level is σ/(xmax −xmin), with five ImageNet gazelle images arranged by rows and noise levels by columns.
- Outlying values: Capping extreme gradient values at the 99th percentile produces more visually coherent sensitivity maps.The paper reports the 99th percentile as sufficient for this purpose.
Multiplying maps with the input images
Multiplying gradient-based maps by input images can make visualizations sharper and simpler, but it can also obscure features represented by zero-valued pixels. The paper therefore reports results both with and without this multiplication.
- Visual effects: Multiplying gradient-based values by the input pixels tends to produce visually simpler and sharper sensitivity maps.However, some apparent sharpness may come from edges already present in the input image rather than the sensitivity map.
- Limitation: Input multiplication can prevent zero-valued pixels from appearing on the sensitivity map.A black ball encoded as zero may therefore fail to be highlighted even when correctly classified against a white background.
- Interpretation: In a linear system y = Wx, xiwi represents the contribution of xi to the final score y.This provides a rationale for interpreting gradient–input products as feature contributions.
- Comparison: The paper shows results both with and without image multiplication in Fig. 5.
3.2. Effect of noise level and sample size
SmoothGrad is controlled by the noise level and the number of noisy samples averaged together. These two hyper-parameters determine the perturbation scale and averaging extent.
- Hyper-parameters: SmoothGrad has two hyper-parameters: σ, the Gaussian perturbation noise level, and n, the number of samples averaged.
- Noise level: σ controls the standard deviation of the Gaussian perturbations added to the input.
- Sample size: n controls how many perturbed sensitivity maps contribute to the average.
Noise, σ
The paper examines how noise level and sample size affect SmoothGrad visualizations. Moderate noise can balance sharpness and image structure, while increasing sample size smooths estimates with diminishing visual returns.
- Noise level: 10%-20% noise seems to balance sensitivity-map sharpness with preservation of the original image structure.The paper notes that the ideal noise level depends on the input, despite generally good results for Inception.
- Sample size: Increasing sample size makes the estimated gradient smoother, with little apparent visualization change for n > 50.
3.3. Qualitative comparison to baseline methods
The paper evaluates sensitivity maps qualitatively through visual coherence and discriminativity. SMOOTHGRAD generally produces more coherent and discriminative maps, while Guided BackProp can be sharper but fails in some settings.
- Evaluation criteria: Because sensitivity maps lack ground truth for quantitative evaluation, the paper assesses visual coherence and discriminativity qualitatively.Visual coherence concerns object-focused highlights; discriminativity concerns distinguishing between multiple objects and classes.
- Visual coherence: Among 200 randomly inspected images, SMOOTHGRAD consistently produced more visually coherent maps than Integrated Gradients and vanilla gradient.The comparison used side-by-side visual inspection against three gradient-based methods.
- Visual coherence: Guided BackProp produced the sharpest maps but was prone to failure, especially for images with uniform backgrounds.SMOOTHGRAD had its highest observed impact when objects were surrounded by uniform background color.
- Discriminativity: SMOOTHGRAD qualitatively showed better discriminativity than the other methods in images containing at least two objects from different classes.The comparison visualized differences between class-specific sensitivity maps using a diverging color map.
3.4. Combining SmoothGrad with other methods
SMOOTHGRAD can augment other gradient-based sensitivity-map methods by averaging maps from noisy images. This augmentation improved the visual coherence of Integrated Gradients and Guided BackProp.
- Core procedure: SMOOTHGRAD averages vanilla sensitivity maps computed from n noisy versions of an image.The same smoothing procedure can be applied to any gradient-based method.
- Combination results: Applying SMOOTHGRAD to Integrated Gradients and Guided BackProp improved the visual coherence of both methods' sensitivity maps.The paper presents these augmented results in Fig. 7.
- Additional analysis: The authors provide a web page containing sensitivity maps for 200+ images and four different methods for further analysis.The resource supports inspection of the compared visualizations.
3.5. Adding noise during training
The paper also examines noise added during training as a way to sharpen sensitivity maps. Training-time and evaluation-time noise each have denoising effects, and using both produces the most visually coherent maps.
- Training-time noise: Adding noise to training samples improved the sharpness of the network's sensitivity maps.This adapts a well-known regularization technique to improve map legibility.
- Observed effects: Training-time noise provided a denoising effect on sensitivity maps in both MNIST and Inception experiments.The effects of adding noise during training and evaluation are shown in Figs. 8 and 9.
- Combined noise: Using training-time and evaluation-time noise together produced the most visually coherent map among the four tested combinations.The paper describes the effects of the two techniques as additive.
4. Conclusion and future work
The experiments suggest that averaging gradients over noisy image perturbations sharpens sensitivity maps, with additional improvement from noise-perturbed training. Future work includes explaining the effect, developing better comparison metrics, and testing broader settings.
- Conclusion: Averaging maps from many small image perturbations smooths gradient-based sensitivity maps, while noise-perturbed training can enhance that effect.The paper presents these as two forms of smoothing supported by experiments.
- Future work: The conjecture that noisy sensitivity maps arise from noisy gradients remains supported by plausibility arguments and limited empirical evidence rather than definitive theory.The authors note that differential effects of random noise on textures could have other explanations.
- Future work: More direct approaches could train smoother class-score functions using penalties on partial derivatives or neighboring-pixel derivative differences.These proposals aim to create smoother or more spatially coherent maps.
- Future work: Future work should develop better metrics for spatial coherence and discriminativity when comparing sensitivity maps.Suggested directions include image-segmentation databases and systematic discriminativity measurements.
- Future work: The generalization of these de-noising techniques to other network architectures and tasks remains an open question.