Source-linked AI summary
Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers
Alexander Binder, Grégoire Montavon, Sebastian Bach, Klaus-Robert Müller, Wojciech Samek
TL;DR
LRP explains neural-network image classifications by assigning relevance to input pixels, but standard rules do not cover product-type nonlinearities such as local renormalization. The paper extends LRP using first-order Taylor expansions and evaluates it on CIFAR-10, Imagenet, and MIT Places, where it clearly outperforms original LRP.
Problem
Standard LRP does not cover product-type nonlinearities, including local renormalization layers in convolutional neural networks.
Method
The paper extends LRP to local renormalization layers using first-order Taylor expansions of fully nonlinear neurons.
Results
Taylor-based treatment improves heatmap AUC on CIFAR-10 and clearly outperforms original LRP across CIFAR-10, Imagenet, and MIT Places.
Takeaways & Limitations
Taylor expansion provides an effective treatment for local renormalization layers in LRP heatmap generation.
Abstract
from arXiv · showhide
Layer-wise relevance propagation is a framework which allows to decompose the prediction of a deep neural network computed over a sample, e.g. an image, down to relevance scores for the single input dimensions of the sample such as subpixels of an image. While this approach can be applied directly to generalized linear mappings, product type non-linearities are not covered. This paper proposes an approach to extend layer-wise relevance propagation to neural networks with local renormalization layers, which is a very common product-type non-linearity in convolutional neural networks. We evaluate the proposed method for local renormalization layers on the CIFAR-10, Imagenet and MIT Places datasets.
1 Introduction
Neural networks perform strongly in image classification, but identifying which image regions drive their decisions remains an open question. This paper extends Layer-wise Relevance Propagation to nonlinearities such as local renormalization layers.
- Motivation: Neural networks have achieved excellent performance in large-scale image-classification competitions.Their training can use millions of samples through GPU parallelization.
- Motivation: Understanding which image regions matter for a classification decision remains an open question for neural networks and other nonlinear models.
- Prior work: Layer-wise Relevance Propagation was proposed to explain which pixels are relevant to an image-classification decision.The framework was also applied to bag-of-words models and Fisher vectors in subsequent work.
- Contribution: The paper extends LRP to neural networks with nonlinearities beyond the commonly used formulation, including local renormalization layers.The proposed approach is based on first- or higher-order Taylor expansion.
2 Layer-wise Relevance Propagation for Neural Networks
LRP propagates a classifier’s output relevance backward through network layers to assign relevance scores to input pixels. Its redistribution rules use weighted activations, stabilization, and separate treatment of positive and negative contributions.
- Purpose: LRP assigns each input pixel a relevance score indicating evidence for or against the presence of a class.Positive scores represent evidence for the class, while negative scores represent evidence against it.
- Propagation: LRP decomposes a neuron’s relevance into messages sent to its input neurons, then sums incoming messages to obtain each lower-layer relevance.This recursively propagates relevance from the output neuron toward the input layer.
- Interpretation: The resulting input-layer relevances form pixel-wise heatmaps that can be evaluated against classification behavior and human intuition.
- Redistribution rules: The ϵ-rule uses a small stabilizer term to avoid numerical degeneration when a neuron’s activation is close to zero.
- Redistribution rules: The β-rule treats positive and negative weighted activations separately, with β controlling how much inhibition enters relevance redistribution.A larger β, such as β = 1, produces sharper heatmaps.
- Interpretation: LRP’s meaningful pixel decomposition derives contributions from weighted activations rather than gradients used during backpropagation training.
3 Extending LRP to local renormalization layers
The paper extends LRP to fully nonlinear neurons, including local renormalization layers that standard LRP cannot handle exactly, by deriving relevance weights from Taylor expansions. For local renormalization, the method analyzes expansion points and yields relevance weights with specified sign, magnitude, and limiting-behavior properties.
- General Taylor-based extension: The proposed strategy applies first-order Taylor expansion to fully nonlinear neurons whose pooling and activation fall outside the standard LRP structure.The approach extends Taylor-based relevance decomposition beyond ReLU neurons and their local linearity.
- Relevance decomposition: The Taylor expansion decomposes a nonlinear neuron activation onto its input neurons by assigning sum elements to incoming neurons and redistributing the zero-order term.The resulting decomposition supplies the relevance-propagation weighting needed for nonlinear activations.
- Local renormalization: Local renormalization normalizes one neuron using its surrounding neurons and forms a product-type nonlinearity that standard LRP cannot tackle exactly.The interaction is modeled as a network layer with a dedicated activation function.
- Expansion-point choice: The expansion point can be the actual layer input z1 or a single-neuron input z2 where only the normalized neuron fires.At z2, derivatives for the other variables are zero, motivating a different relevance redistribution behavior.
- Weighting properties: The proposed weighting preserves the normalized neuron's relevance sign, permits sign reversal for suppressing neighbors, and makes their absolute relevance proportional to squared input.As c →0 and b →0, local renormalization approaches the identity and the approximation recovers the identity; treating normalization as constant provides a baseline that propagates relevance only to the normalized neuron.
4 Experiments
The experiments evaluate pixel-wise relevance using perturbation-based AUC measures across CIFAR-10, Imagenet, and MIT Places, comparing LRN treatments and LRP parameter settings.
- Evaluation protocol: Pixel relevance is evaluated by sequentially replacing image pixels with random RGB values and measuring the resulting prediction-score curve.A strong relevance measure should produce faster score decay when higher-scoring pixels are perturbed first.
- CIFAR-10: Table 1 compares identity and first-order Taylor treatments of local renormalization layers for two pixel-score computation approaches on CIFAR-10.The table caption specifies that lower scores are better.
- Evaluation protocol: Lower AUC indicates better identification of pixel relevance because the classification score declines more rapidly when highly scored pixels are perturbed first.The experiments compare highest-score-first, random-order, and lowest-score-first perturbation sequences.
- Cross-dataset evaluation: First-order Taylor treatment improves heatmap AUC in all tested CIFAR-10 cases, supporting its effectiveness for nonlinear neuron layers.The same experiment is extended to Imagenet and MIT Places using 5,000 images from each unlabeled test set.
- Cross-dataset evaluation: With Taylor treatment on Imagenet and MIT Places, the AUC ordering is ϵ = 1, ϵ = 0.01, ϵ = 100, β = 1, β = 0, from lowest to highest.Using identity preserves this ordering except that ϵ = 100 and ϵ = 0.01 are swapped.
- Qualitative analysis: Taylor expansion with ϵ = 1 combines high pixel selectivity and low noise, matching its measured superiority in the quantitative experiments.The results also report that Taylor treatment further improves AUC scores, while other parameter choices are on par or slightly worse.
5 Conclusion
The paper extends LRP to product-type nonlinearities using first-order Taylor expansions and evaluates the approach on three datasets.
- 5 Conclusion: The proposed first-order Taylor extension handles product-type nonlinearities occurring in local renormalization layers of deep convolutional networks.The method is evaluated on three popular datasets and is reported to clearly outperform original LRP.