PaperlayerView exact revision on arXiv

Source-linked AI summary

Explaining NonLinear Classification Decisions with Deep Taylor Decomposition

Grégoire Montavon, Sebastian Bach, Alexander Binder, Wojciech Samek, Klaus-Robert Müller

arXiv:1512.02479v1cs.LGstat.ML

TL;DR

The paper addresses the lack of transparency in nonlinear classifiers by introducing deep Taylor decomposition, which decomposes a classification decision into input-element contributions. It produces pixel-wise relevance heatmaps through relevance propagation, with experiments on MNIST and ILSVRC and reported stability across architectures and datasets.

  • Problem

    Deep neural networks perform strongly on challenging tasks but lack transparency, making their classification reasoning difficult to interpret.

  • Method

    Deep Taylor decomposition explains multilayer neural-network decisions by decomposing neuron contributions and redistributing them backward to input pixels.

  • Results

    The method produces heatmaps that assign relevance across detected objects, while remaining stable across architectures and datasets without hyperparameter tuning.

  • Takeaways & Limitations

    The approach provides pixel-level explanations for individual image-classification predictions and can be applied to existing pretrained networks without retraining.

  • Takeaways & Limitations

    Nearest-root Taylor decomposition can be time-consuming and may be unsolvable because its minimization problem can be non-convex.

Abstract

from arXiv · show

Nonlinear methods such as Deep Neural Networks (DNNs) are the gold standard for various challenging machine learning problems, e.g., image classification, natural language processing or human action recognition. Although these methods perform impressively well, they have a significant disadvantage, the lack of transparency, limiting the interpretability of the solution and thus the scope of application in practice. Especially DNNs act as black boxes due to their multilayer nonlinear structure. In this paper we introduce a novel methodology for interpreting generic multilayer neural networks by decomposing the network classification decision into contributions of its input elements. Although our focus is on image classification, the method is applicable to a broad set of input data, learning tasks and network architectures. Our method is based on deep Taylor decomposition and efficiently utilizes the structure of the network by backpropagating the explanations from the output to the input layer. We evaluate the proposed method empirically on the MNIST and ILSVRC data sets.

I. INTRODUCTION

Deep neural networks achieve strong performance but remain difficult to interpret, motivating methods that explain individual classification decisions through input-level relevance. This paper introduces deep Taylor decomposition as a principled framework for producing such explanations.

  • I. INTRODUCTION: Deep neural networks deliver impressive performance but lack transparency because their multilayer nonlinear structure obscures their reasoning.This limits experts’ ability to verify, interpret, and understand individual decisions.
  • I. INTRODUCTION: Interpretable classifiers should identify which input structures, such as image pixels, support a particular prediction.Such information can help assess prediction quality, verify classifier reasoning, and identify regions for further analysis.
  • I. INTRODUCTION: Deep Taylor decomposition replaces one analytically intractable Taylor decomposition with simpler tractable decompositions performed separately for each neuron.The resulting relevance redistribution proceeds from hidden-neuron contributions to a pixel-space heatmap.
  • I. INTRODUCTION: Deep Taylor decomposition yields rules similar to the αβ-rule and ϵ-rule, while providing specific instantiations of their previously heuristic hyperparameters.The paper does not perform a broader empirical comparison with several other recently proposed methods.
  • I. INTRODUCTION: The framework focuses on explaining individual data-point predictions rather than providing only a general analysis of the trained model.This distinguishes prediction-level relevance from broader analyses of feature spaces, transformations, or neuron roles.
  • I. INTRODUCTION: Existing alternatives include feature-visualization propagation, spatial response maps, and attention models, with some requiring specialized and more complex architectures.Gradient-based sensitivity analysis examines function variations but does not by itself provide a full explanation of why a data point received its prediction.

A. Taylor Decomposition

Taylor decomposition explains a classification decision by redistributing a function’s value onto input pixels around a root point. Deep Taylor decomposition reduces the computational burden by exploiting neural-network structure, while sensitivity analysis provides only relative local contributions.

  • Taylor decomposition: Taylor decomposition assigns pixel relevances from a first-order expansion of the classification function around a root point ex where f(ex)=0.The resulting heatmap is the element-wise product of the gradient at ex and the difference x−ex.
  • Taylor decomposition: In the building example, a blurred root image removes the detected object, so multiplying sensitivities by x−ex highlights pixels contributing to the building detection.For multichannel images, relevance is summed across color channels to produce one pixel heatmap.
  • Taylor decomposition: A suitable root point removes the detected object while minimally deviating from the original input, enabling an explanation less sensitive to surrounding content.The nearest root satisfies f(ex)=0 and lies near x under a chosen distance metric.
  • Taylor decomposition: Finding the nearest root can require iterative minimization, which is expensive for costly functions and may be unsolvable for non-convex objectives.This motivates alternatives that avoid the high computational requirement of standard Taylor decomposition.
  • Taylor decomposition: Sensitivity analysis uses a single gradient evaluation, but measures only a local effect and does not provide a full explanation of the classification decision.Because zero-order relevance is not redistributed, only relative contributions between R_p values are meaningful.

C. Deep Taylor Decomposition

Deep Taylor decomposition explains a neural-network score by recursively decomposing relevance across adjacent layers, ultimately assigning contributions to input variables. It exploits the network’s factorization into simpler subfunctions and uses root-point Taylor decompositions to propagate relevance.

  • C. Deep Taylor Decomposition: The method treats the network as hierarchically composed feature extractors whose increasingly complex representations support layer-wise relevance redistribution.Low-level pixel features are combined into mid-level features and then into a final nonlinear score.
  • C. Deep Taylor Decomposition: Relevance is propagated from the output score through hidden neurons to input variables by repeatedly decomposing each neuron’s relevance.Different root points may be selected for decomposing different hidden-neuron relevances.
  • C. Deep Taylor Decomposition: Deep Taylor decomposition applies Taylor decomposition separately to mappings from lower-layer neuron activities to higher-layer relevance.This replaces one analytically intractable decomposition with multiple simpler decompositions, one for each neuron.
  • C. Deep Taylor Decomposition: Local conservation and positivity imply global layer-wise relevance conservation, positivity, and consistency when they hold at every decomposition step.The resulting relevance scores satisfy the chain from the output relevance to the sum of pixel relevances.
  • C. Deep Taylor Decomposition: The one-layer detection-pooling example uses a constrained network with nonpositive biases, guaranteeing a root point at the origin for Taylor decomposition.The network maps input neurons to detection neurons and then pools them into an output score.

A. Unconstrained Input Space and the w2-Rule

For an unconstrained input space, the nearest-root construction yields the w2-rule, which redistributes relevance using squared weights and pooling across neurons. The rule is theoretically consistent for the considered one-layer function class.

  • A. Unconstrained Input Space and the w2-Rule: For unrestricted real-valued inputs, the nearest root is found using the intersection of the neuron’s zero plane and its maximum-descent line.The root is chosen nearest to the actual data point in Euclidean distance.
  • A. Unconstrained Input Space and the w2-Rule: The resulting w2-rule redistributes relevance according to squared weight magnitudes and pools contributions across all detection neurons.The same rule also applies when the neuron relevance is zero, in which case no relevance is propagated.
  • A. Unconstrained Input Space and the w2-Rule: The deep Taylor decomposition with the w2-rule is consistent for every function in the one-layer network class G.Its data-dependent upper-layer relevances produce an individual explanation for each data point.

B. Constrained Input Space and the z-Rules

When inputs are constrained, nearest-root search may be difficult or leave the admissible domain, motivating z-rules that restrict roots to feasible segments. These rules provide domain-specific relevance propagation with consistency guarantees under stated conditions.

  • B. Constrained Input Space and the z-Rules: With constrained inputs, the Euclidean nearest root may lie outside the admissible domain, so root search can require a restricted feasible subset.The general constrained search problem may otherwise be difficult.
  • B. Constrained Input Space and the z-Rules: For positive input spaces, the z+-rule uses positive weight contributions and is applied to higher layers with positive neuron activations.The rule corresponds to the αβ-rule with α = 1 and β = 0.
  • B. Constrained Input Space and the z-Rules: The deep Taylor decomposition with the z+-rule is consistent under its stated conditions, as is the zB-rule for inputs in the bounded domain B.The summarized properties include applicability, conservation, positivity, consistency, uniqueness, and computation speed.
  • B. Constrained Input Space and the z-Rules: The zB-rule incorporates input lower and upper bounds when restricting root search to box-constrained pixel domains.Its additive terms enforce positivity rather than merely preventing a zero denominator.

C. Experiment

Experiments compare relevance heatmaps and consistency properties on MNIST-style detection networks while extending the method to deeper architectures through relevance models. The zB-rule produces complete, well-aligned heatmaps, and deep Taylor maintains theoretical consistency.

  • C. Experiment: MNIST experiments train a detection-pooling network to identify target digits appearing beside distractor digits, with outputs set to 0 or 100.The task is formulated as regression using mean-square error.
  • C. Experiment: Deep Taylor heatmaps assign most relevance to target-digit pixels, while the zB-rule produces complete and well-aligned maps despite some blurring.Sensitivity maps are unbalanced and border-heavy, nearest-root maps are incomplete, and the w2-rule maps are blurred and misaligned.
  • C. Experiment: The quantitative analysis tests conservation by comparing total output relevance with summed pixel relevances and tests positivity through relevance histograms.Scatter points on the diagonal indicate conservation, while histogram colors distinguish positive and negative assignments.
  • C. Experiment: Sensitivity analysis does not redistribute relevance conceptually, nearest-root Taylor dissipates much relevance, and deep Taylor ensures full consistency.The deep Taylor result follows the theoretical predictions of Propositions 1 and 3.
  • C. Experiment: The zB-rule spreads relevance across more pixels than nearest-root methods, as indicated by a shorter relevance-histogram tail.This provides a concrete distinction between the constrained-domain rule and nearest-root propagation.
  • C. Experiment: For deeper networks, relevance models make the mapping from lower-layer activations to upper-layer relevance explicit and can incorporate bottom-up and top-down information.Their propagation is approximately conservative, with error determined by the difference between predicted and true relevance.

B. Training-Free Relevance Model

The training-free relevance model scales deep Taylor decomposition to large networks without training a separate relevance model for each neuron. It decomposes relevance layer by layer using a mostly z+-rule propagation scheme.

  • B. Training-Free Relevance Model: The training-free method can be repeated through successive layers until the input layer without retraining the original network.All layers use the z+-rule except the first, where alternatives such as the w2-rule or zB-rule may be applied.
  • B. Training-Free Relevance Model: The resulting relevance expression combines linear pooling, a positive Lp/L1 pooling ratio, and a weighted sum of higher-level relevances.The weighted higher-level term measures sensitivity to neuron activation.
  • B. Training-Free Relevance Model: The relevance model approximates pooling and top-down contextualization factors as constants, yielding a structure similar to the earlier detection-pooling network.The pooling ratio and top-down term are treated as weakly dependent on lower-layer activations.
  • B. Training-Free Relevance Model: The method redistributes relevance in proportion to detection-layer activations under the corresponding relevance model.This follows the Taylor decomposition of the relevance model.
  • B. Training-Free Relevance Model: The summarized comparison identifies trade-offs involving consistency, uniqueness, training requirements, and computation speed across heatmapping methods.The supplied comparison text also notes fitting-error and nonconvexity caveats for some approaches.

C. Experiment on MNIST

Experiments extend the training-free decomposition from MNIST to large ImageNet networks and examine whether its heatmaps preserve relevance and identify image structures. The method produces positive, largely conservative explanations, while difficult backgrounds can still attract relevance.

  • C. Experiment on MNIST: The trained min-max relevance model and training-free method produce heatmaps of similar quality on the two-layer MNIST network.The network contains two nonlinear detection-pooling layers and uses a learned min-max model for the first layer.
  • C. Experiment on MNIST: The MNIST training-free deep Taylor heatmaps are fully conservative and positive, whereas the trained min-max model produces near-conservative heatmaps.Standard Taylor decomposition is positive but dissipates relevance.
  • D. Experiment on ILSVRC: On large ILSVRC networks, deep Taylor heatmaps cover whole detected objects and usually suppress distracting image structure.Examples include identifying object parts, multiple object instances, and ignoring backgrounds such as grass or wood patterns.
  • D. Experiment on ILSVRC: CaffeNet and GoogleNet produce highly similar heatmaps, while GoogleNet’s greater accuracy is associated with better localization of relevant image regions.The networks are kept unchanged during the training-free explanation procedure.
  • D. Experiment on ILSVRC: Complex or overlapping backgrounds can confuse both the network and heatmapping procedure, causing substantial relevance to be assigned to the background.The supplied examples include overlapping sheep and a scooter against a high-contrast urban scene.
  • D. Experiment on ILSVRC: The volcano example shows interpretable explanations at both global and local scales, including relevance on both sides of the object’s edge.The zoom also exposes different first-layer convolution strides: 4 for CaffeNet and 2 for GoogleNet.

I. DERIVATIONS OF PROPAGATION RULES

The derivations obtain propagation rules by selecting root points of neuron relevance functions along prescribed search directions. This turns the generic Taylor decomposition into explicit relevance redistribution formulas.

  • I. DERIVATIONS OF PROPAGATION RULES: Each propagation rule corresponds to a different choice of root point for a neuron’s relevance function.The derivation uses the network’s detection-layer relevance as its starting point.
  • I. DERIVATIONS OF PROPAGATION RULES: For neurons in C1, the data point itself is the nearest root of the relevance function.This case is treated separately from neurons in C2.
  • I. DERIVATIONS OF PROPAGATION RULES: For neurons in C2, the nearest root is found by intersecting the search direction with the relevant hyperplane.The derivation solves for the intersection parameter using the neuron’s affine equation.
  • I. DERIVATIONS OF PROPAGATION RULES: The general derivation proceeds by defining a search line, verifying a root within the input domain, and substituting the direction into the generic propagation equation.This three-step procedure generates the relevance rule for each chosen search direction.
  • I. DERIVATIONS OF PROPAGATION RULES: The w2-rule searches for the nearest root along the gradient direction of the neuron relevance function.The search line lies in the input domain and contains a root obtained from the relevance value and weights.

B. z-Rule

The z-rule selects a root on the segment between the input and a designated boundary or zero point. Variants constrain the search according to the input domain and activation signs.

  • B. z-Rule: The z-rule chooses the nearest root on the segment connecting the zero point to the input.The segment contains a root at its first endpoint because the bias is negative by design.
  • B. z-Rule: The propagation formulas are obtained by inserting each rule’s search direction into the generic Taylor-based relevance equation.The z-rule derivation defines weighted activations before performing this substitution.
  • B. z-Rule: The z+-rule selects a root on a segment determined by inputs whose weighted contributions are negative and the current input.Its search direction is defined by separating negative and nonnegative weighted activations.
  • B. z-Rule: The zB-rule selects a root on a segment connecting the input to a boundary point defined by lower and upper input bounds.The illustrated example uses l1 = −1, h1 = 1, l2 = −1, and h2 = 1.

II. ALGORITHMS FOR PROPAGATION RULES

The propagation rules express relevance redistribution through matrix operations, using weights, activations, upper-layer relevance, and—in bounded cases—lower and upper activation bounds.

  • Matrix-based implementation: The rules use matrix multiplications and element-wise operations, making them implementable with numerical libraries such as Matlab or Python/Numpy.The implementation relies on matrix multiplication, element-wise multiplication, and element-wise division.
  • w2-rule: For the squared-weight rule, normalized squared weights can be precomputed and reused across multiple heatmap computations.The normalization uses V ← W ⊙ W and N ← V ⊘( · V).
  • Rule inputs: The propagation inputs include a weight matrix W, input activations X, and an upper-layer relevance vector R.The bounded rule additionally uses lower bounds L and upper bounds H.
  • Sign-constrained rules: Positive-part and negative-part operators separate positive and negative matrix contributions in the corresponding propagation rules.The matrices U and V can be precomputed and reused for multiple heatmap computations.
  • zB-rule: The bounded propagation rule redistributes relevance using weighted activations adjusted by lower and upper bounds.Its inputs and output explicitly incorporate W, X, R, L, and H.

III. PROOFS OF PROPOSITIONS

The propositions establish that several deep Taylor propagation rules produce heatmaps that are both conservative and positive, hence consistent under the paper’s definitions.

  • Definitions: A conservative heatmapping preserves total relevance when assigned relevance is summed in pixel space.This is the paper’s Definition 1 for relevance conservation.
  • Definitions: A positive heatmapping has values that are all greater than or equal to zero.This is the paper’s Definition 2.
  • Definitions: A consistent heatmapping is both conservative and positive.Definition 3 combines the two required properties.
  • w2-rule: The deep Taylor decomposition with the w2-rule is consistent for all functions in the specified function class.Proposition 1 establishes consistency through conservation and positivity.
  • z+-rule: The deep Taylor decomposition with the z+-rule is consistent in the sense of the paper’s consistency definition.The proof handles the zero-contribution case by showing that no relevance needs redistribution.
  • zB-rule: The deep Taylor decomposition with the zB-rule is consistent for all functions and data points in the specified bounded set.The proof uses bounded inputs and shows that nonpositive contributions do not require relevance redistribution.

IV. EMPIRICAL COMPARISON WITH LRP

The empirical comparison examines deep Taylor and LRP heatmaps across MNIST and large convolutional networks. Deep Taylor is less sensitive to model and dataset choice, while LRP can be sharper but parameter-sensitive.

  • Comparison setup: The experiments compare deep Taylor heatmaps with layer-wise relevance propagation rules, including the αβ-rule and the ϵ-stabilized rule.The LRP hyperparameters are typically selected to obtain the desired heatmap quality.
  • Comparison setup: The comparison covers one-layer and two-layer MNIST networks, plus BVLC CaffeNet and GoogleNet.Deep Taylor uses a min-max relevance model for the two-layer MNIST case and a training-free relevance model for the larger networks.
  • Robustness across settings: Deep Taylor heatmap quality is less influenced by model and dataset choice than LRP with a fixed parameter set.Deep Taylor heatmaps look similar across the evaluated settings.
  • Heatmap properties: LRP heatmaps are sharper and less affected by convolutional stride artefacts than Taylor-based heatmaps.LRP also tends to assign little evidence to irrelevant image regions.
  • Parameter sensitivity: LRP parameter choices produce different artefacts, including overly sparse heatmaps, negative relevance, and residual relevance on irrelevant MNIST digits.The example α = 2, β = 1 performs well for CaffeNet but behaves differently for GoogleNet and MNIST.
Loading 1512.02479v1…