Source-linked AI summary
Towards Best Practice in Explaining Neural Network Decisions with LRP
Maximilian Kohlbrenner, Alexander Bauer, Shinichi Nakajima, Alexander Binder, Wojciech Samek, Sebastian Lapuschkin
TL;DR
Neural networks are powerful but often opaque, motivating a quantitative test of whether recent layer-dependent LRP practice improves explanations. The paper evaluates composite LRP against earlier approaches in visual object recognition and reports better object localization and class discrimination, including stronger performance for smaller objects.
Problem
Recent layer-dependent LRP methods were supported mainly by human observation, leaving their benefits insufficiently quantified.
Method
The paper compares composite, layer-dependent LRP with uniform-rule LRP and other attribution methods on ImageNet and PascalVOC using object localization and class-discrimination evaluations.
Results
LRPCMP consistently outperforms other methods on large datasets, with the clearest gains for smaller bounding boxes; LRPCMP :α2+♭ performs best overall.
Takeaways & Limitations
Layer-dependent LRP produces more representative attribution maps while improving object localization and class discrimination and addressing gradient shattering.
Abstract
from arXiv · showhide
Within the last decade, neural network based predictors have demonstrated impressive - and at times super-human - capabilities. This performance is often paid for with an intransparent prediction process and thus has sparked numerous contributions in the novel field of explainable artificial intelligence (XAI). In this paper, we focus on a popular and widely used method of XAI, the Layer-wise Relevance Propagation (LRP). Since its initial proposition LRP has evolved as a method, and a best practice for applying the method has tacitly emerged, based however on humanly observed evidence alone. In this paper we investigate - and for the first time quantify - the effect of this current best practice on feedforward neural networks in a visual object detection setting. The results verify that the layer-dependent approach to LRP applied in recent literature better represents the model's reasoning, and at the same time increases the object localization and class discriminativity of LRP.
I. INTRODUCTION
Deep neural networks achieve strong performance but are often treated as black-box predictors, motivating XAI methods such as LRP. This paper quantitatively evaluates whether layer-specific LRP improves explanations over earlier uniform applications.
- Motivation: Neural networks are widely used as high-performing predictors, but their decision processes are often difficult to interpret.This motivates neural-network interpretability as an active research area.
- Emerging practice: LRP explanations maintain a strong connection to predictor outputs, and recent work increasingly assigns different decomposition rules to different network layers.This layer-dependent practice emerged from qualitative observations of more robust attribution maps.
- Study aim: The paper tests whether layer-specific LRP improves on earlier approaches by comparing attribution methods on ImageNet and PascalVOC images with ground-truth object localizations.The experiments use different neural-network models and measure object localization through relevance attribution.
- Research gap: The study addresses the need to quantify benefits that recent layer-dependent LRP literature had previously supported mainly through human observation.The evaluation focuses on precise localization of ground-truth objects and comparison across XAI methods.
A. Layer-wise Relevance Propagation
LRP propagates output relevance backward through network layers using decomposition rules that distribute relevance according to layer activations and mappings. Different rules address different numerical or representational issues, while uniform application can produce suboptimal explanations in deeper networks.
- LRP rules: LRPz proportionally decomposes upper-layer relevance to lower-layer neurons using localized preactivations and conserves total relevance between layers.The rule distributes relevance according to each localized preactivation relative to its layer aggregation.
- LRP rules: LRPε adds a signed small constant to the denominator to prevent division by zero and reduce the influence of weak or noisy mappings.This modifies the basic proportional decomposition rule.
- LRP rules: LRPαβ separately decomposes activatory and inhibitory mappings, with α weighting activations and β implicitly satisfying α+β = 1 for layerwise conservation.The commonly used α = 1 setting can be derived from Deep Taylor Decomposition.
- LRP rules: The LRP♭ rule uniformly spreads a neuron's relevance across its inputs and propagates higher-level concepts through receptive fields without further transformation.It is intended for lower convolutional or input layers and is unsuitable for fully connected layers.
- Limitations of uniform rules: Uniform single-rule LRP can yield suboptimal explanations, while network-wide LRPz and LRPε can exhibit gradient shattering in deeper models.LRPαβ is more robust against gradient shattering but can lack class or object discriminativity and assumes strictly positive layer activations.
B. A Current Best Practice for LRP
Recent LRP practice uses a composite, layer-dependent strategy that assigns different decomposition rules to different parts of a neural network. The paper summarizes this configuration as LRPCMP.
- Composite strategy: LRPCMP applies purpose-specific LRP rules to different network regions rather than decomposing the entire network with one uniform rule.The strategy combines rules to improve robustness against gradient shattering while preserving object discriminativity.
- Rule assignment: The composite pattern commonly uses LRPε or LRPz near the output, LRPαβ in convolutional layers, and LRP♭ near the input.The separate treatment of positive and negative mappings complements localized convolutional feature activations.
A. Motivation
The paper motivates quantitative evaluation of attribution quality by asking whether explanations identify object regions rather than contextual features. PascalVOC and ImageNet bounding boxes provide the experimental basis for this assessment.
- Evaluation setting: PascalVOC and ImageNet provide object bounding-box annotations that support quantitative evaluation of attribution localization.These datasets create an experimental setting for testing whether relevance scores concentrate on ground-truth objects.
- Occlusion test: Figure 2 compares prediction changes after occluding pixels inside versus outside class-specific bounding boxes on PascalVOC and ImageNet.Lower prediction-change values indicate a stronger reaction, with shaded regions representing standard deviation.
- Evaluation goal: A useful attribution method should mark image regions belonging to the object and disregard visual features unrelated to the object itself.The paper treats this as a measurable form of object understanding in visual recognition.
B. Verifying Object-centricity During Prediction
The study assumes the models primarily base predictions on target objects despite contextual biases, then tests this assumption by comparing object and background occlusion. It evaluates attribution localization using unweighted and object-size-weighted relevance ratios.
- The datasets contain contextual biases, so the experiments assume models predominantly base decisions on target objects rather than image context.
- Object and class-specific background regions are separately occluded, and the change in ground-truth output is measured for each image.The occluded regions are replaced with mean color values.
- Object occlusion consistently causes a sharper decrease in the target-class output, especially for smaller objects.This supports the assumption that predictions mainly rely on the object itself.
- Attribution quality is measured by the fraction of total positive relevance inside the object bounding box, using µ and size-weighted µw.Rin denotes positive relevance inside the box, Rtot total positive image relevance, and the weighted variant incorporates object and image sizes.
- Higher µ and µw values indicate that more relevance is assigned to the object rather than the background, with µw emphasizing smaller objects.The size factor increases as bounding boxes become smaller.
IV. EXPERIMENTS AND RESULTS
Experiments evaluate multiple LRP variants and complementary attribution methods on ImageNet and PVOC 2007 using pretrained neural networks and ground-truth class labels. The study focuses on comparing LRP configurations while keeping the two software packages' LRP functionality aligned.
- Experiments use ImageNet and PVOC 2007 because both provide large numbers of ground-truth object bounding boxes.
- PVOC evaluation covers approximately 10,000 samples and approximately 15,000 class-by-sample measurements with a model achieving 72.12 mean AP.
- ImageNet evaluation uses 50,000 validation samples with bounding boxes and a pretrained VGG-16 model achieving 90.1% top-5 accuracy.
- Attributions are computed with Keras/TensorFlow iNNvestigate for VGG-16 and the Caffe-based LRP Toolbox for PVOC and CaffeNet.
- Both packages support the same LRP functionality, keeping the study focused on effects of LRP variants rather than package differences.
- The comparison includes LRPz, LRPαβ, several LRPCMP parameterizations, Guided Backprop, and ImageNet-only Pattern Attribution.
B. Qualitative Observations
Qualitative attribution maps show that layer-specific LRP variants produce more intuitive, object-focused explanations than uniformly applied rules. The placement of decomposition rules changes attribution scale, semantics, localization, and class discrimination.
- Uniform-rule maps fail to show class discriminativeness and often highlight similar image regions across target classes.
- LRPz produces highly complex attribution structures associated with gradient shattering, making the maps difficult to use for further analyses.
- Layer-specific LRPCMP maps better match intuitive object-based relevance and shift from local features toward coarse localization as rule placement changes.
- Applying LRPαβ in upper layers loses object localization, whereas applying it in lower layers avoids gradient-shattering issues.
- The Fig. 1(g) configuration resembles CAM in the fully connected portion while retaining LRP's ability to distribute negative relevance.
- The VGG-16 model distinguishes different object types in the same image despite being trained with one object category per sample.
C. Quantitative Results
Quantitative evaluation measures how much relevance falls inside object bounding boxes, with special attention to small objects. LRPCMP consistently performs best, particularly when boxes are small, while methods converge as boxes cover most of the image.
- The evaluation plots average in-total relevance ratio µ against bounding-box size for PVOC 2007 and ImageNet, where larger values indicate more precise object attribution.
- The analysis reports µ and µw overall and also averages restricted to objects covering at most 25% or 50% of the image.
- The uniform whole-image attribution baseline is outperformed by all methods.
- LRPz performs worse on ImageNet than PVOC, which the authors associate with the deeper ImageNet model and stronger gradient shattering.
- LRPCMP consistently outperforms other methods, with the strongest advantage for smaller bounding boxes under µw.
- When bounding boxes cover more than 50% of the image, all methods converge toward perfect performance.
- LRPCMP:α2+♭ achieves the best results, while the composite strategy is more effective overall than fine-tuning decomposition-rule parameters.
D. Conclusion
The study introduces LRPCMP, a composite LRP strategy using multiple layer-specific decomposition rules, and compares it with earlier uniform-rule approaches. Results show more representative attribution maps, improved localization and class discrimination, resistance to gradient shattering, and precise negative relevance attribution with one modified backward pass.
- D. Conclusion: LRPCMP combines multiple purpose-specific decomposition rules across model layers and is compared with earlier uniformly applied LRP and other attribution methods.The strategy is evaluated qualitatively and quantitatively on two widely used computer vision datasets.
- D. Conclusion: LRPCMP produces measurably more representative attribution maps and improves object localization and class discrimination.These properties are reported as improvements over previous LRP approaches and other uniformly applied methods.
- D. Conclusion: LRPCMP provides a solution against gradient shattering affecting previous attribution approaches.The paper reports this benefit alongside sustained object discriminativity.
- D. Conclusion: LRPCMP precisely attributes negative relevance scores to class-contradicting features while requiring only one modified backward pass.The method uses established tools from the LRP framework.
APPENDIX
The appendix provides additional illustrative attribution examples for different input images and object classes using a pretrained VGG-16 model. The examples compare attributions for paired object classes across animals and accessories.
- APPENDIX: The appendix examples use different input images and object classes, extending the illustrative comparisons presented earlier.The text describes Figure 4 as providing further examples similar to Figure 1.
- APPENDIX: Figure 4 shows different attributions for paired output classes using the pretrained VGG-16 model.The pairs include Bernese Mountain Dog/French Bulldog, Persian Cat/Siamese Cat, Zebra/African Elephant, and Sunglasses/Windsor Tie.