PaperlayerView exact revision on arXiv

Source-linked AI summary

B-cos Networks: Alignment is All We Need for Interpretability

Moritz Böhle, Mario Fritz, Bernt Schiele

arXiv:2205.10268v1cs.CVstat.ML

TL;DR

DNN explanations can be faithful summaries yet remain difficult for people to interpret, while improving visual quality can come at a cost to model faithfulness. This paper replaces linear transforms with B-cos transforms that induce faithful linear summaries and promote weight-input alignment. The authors report interpretable explanations and similar classification performance across adapted architectures, with computational overhead and untested architecture types as limitations.

  • Problem

    DNN explanations must faithfully summarize model computations while remaining interpretable, but accurate linear summaries may not be intuitive and visual-quality gains have sometimes come at a cost to faithfulness.

    Source: §1 · p. 1

  • Method

    The paper replaces linear transforms with input-dependent B-cos transforms that induce a faithful linear summary and promote weight-input alignment during optimization.

    Source: §1 · p. 1

  • Results

    The authors report that B-cos explanations are highly interpretable and outperform other explanation methods across tested architectures, while maintaining similar performance or incurring only minor classification-accuracy drops.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    B-cos transforms can be used as drop-in replacements in several common DNN architectures, and the reported results indicate that strong performance and interpretability need not be at odds.

    Source: §1 · p. 2, §6 · p. 8

    Normalizing weights and computing down-scaling factors increased training and inference time by up to 60%, and integration beyond CNNs for image classification, such as vision transformers, was not investigated.

    Source: §5.2.2 · p. 8

Abstract

from arXiv · show

We present a new direction for increasing the interpretability of deep neural networks (DNNs) by promoting weight-input alignment during training. For this, we propose to replace the linear transforms in DNNs by our B-cos transform. As we show, a sequence (network) of such transforms induces a single linear transform that faithfully summarises the full model computations. Moreover, the B-cos transform introduces alignment pressure on the weights during optimisation. As a result, those induced linear transforms become highly interpretable and align with task-relevant features. Importantly, the B-cos transform is designed to be compatible with existing architectures and we show that it can easily be integrated into common models such as VGGs, ResNets, InceptionNets, and DenseNets, whilst maintaining similar performance on ImageNet. The resulting explanations are of high visual quality and perform well under quantitative metrics for interpretability. Code available at https://www.github.com/moboehle/B-cos.

1. Introduction

The paper addresses the difficulty of producing explanations that are both faithful to DNN computations and interpretable to people. It proposes B-cos transforms to make networks inherently explainable, with induced linear summaries that reflect learned, task-relevant patterns.

Motivation and approach

  • B-cos transforms are proposed as drop-in replacements for linear transforms, so networks can provide faithful explanations designed for human interpretation.The paper reports integrating them into VGG, ResNet, InceptionNet, and DenseNet architectures.Source: §1 · p. 1

Faithful explanations

  • Sequences of B-cos transforms induce a single input-dependent linear transform that faithfully summarizes the full model computation.The induced transform reflects learned weights that best align with a given input.Source: §1 · p. 1

Alignment and interpretability

  • The transform’s alignment pressure promotes weights that align with task-relevant input patterns, making the resulting summaries interpretable.The explanations are intended to highlight task-relevant patterns in the input.Source: §1 · pp. 1–2

Reported contributions

  • The authors report that B-cos architectures maintain similar performance while producing explanations that outperform other methods on tested architectures under quantitative and qualitative evaluation.They also report competitive CIFAR-10 performance for a plain B-cos convolutional network.Source: §1 · p. 2

2. Related work

Prior work includes post-hoc explanations and models designed to be inherently interpretable. The paper positions B-cos networks as combining a faithful linear summary with architectural compatibility across existing DNNs.

Post-hoc explanations

  • Post-hoc explanations can be perturbation-based, activation-based, or backpropagation-based; B-cos explanations are model-inherent.The networks are optimised to be explainable under the backpropagation-based approach used to obtain their explanations.Source: §2 · p. 2

Inherently interpretable models

  • Like BagNets and CoDA Nets, B-cos networks admit a faithful single-linear-transform summary; like CoDA Nets, they use structurally induced alignment pressure.The paper distinguishes its approach by designing it for compatibility with existing neural network architectures.Source: §2 · p. 2

Weight-input alignment

  • In B-cos networks, increased alignment comes from architectural constraints, rather than loss-based model regularisation, that require weight-input alignment to solve the optimisation task.Source: §2 · p. 2

Non-linear transforms

  • A transform equivalent to B-cos with B=2 was evaluated in for performance; this paper introduces it to increase interpretability and tests scaling to large-scale classification.The related transforms in emphasise cosine similarity between weights and inputs.Source: §2 · p. 2

3. B-cos neural networks

The section introduces B-cos transforms as replacements for linear units and explains how they produce faithful linear explanations while encouraging weight-input alignment. It then describes simple and advanced network constructions, including MaxOut.

3.1. The B-cos transform

  • B-cos modifies linear transforms to promote weight-input alignment; B controls the transform’s response to alignment, and B=1 is equivalent to a linear transform with normalized weights.The transform bounds neuron output by the input norm, with equality only when weights and input are aligned; larger B suppresses poorly aligned responses.Source: §3.1 · pp. 2–3

3.2. Simple (convolutional) B-cos networks

  • B-cos replaces each layer’s dot products and can be applied to convolutional kernels; for B>1, its layers are nonlinear, so separate activation functions are not required.The transform is used as a drop-in replacement, and the approach also transfers to architectures with skip connections.Source: §3.2 · pp. 3–4

3.1. The B-cos transform

  • For the two classification problems illustrated, optimal weights differ at B=1 but converge to the same optimum for higher B.The figure contrasts this with a linear classifier, whose weights represent the decision boundary and differ across the two tasks.Source: Figure 2 · p. 3, §3.1 · p. 3

3.2.1 Computing explanations for B-cos networks

  • A B-cos network’s input-dependent linear transforms can be composed into one transform that faithfully summarizes its computations and supports explanations of activations.The resulting explanations can be visualized as a transform row or as input-dimension contribution maps.Source: §3.2.1 · p. 4

3.2.2 Optimising B-cos networks for classification

  • B-cos outputs are bounded, and maximizing them requires units to align with their inputs; BCE training therefore promotes alignment with class-specific patterns.Increasing B suppresses poorly aligned weights further, increasing alignment pressure.Source: §3.2.2 · p. 4

3.2.3 MaxOut to increase modelling capacity

  • MaxOut forwards the larger activation from two B-cos transforms per neuron, preserving alignment pressure while increasing modelling capacity.The authors report that MaxOut networks were easier to optimise than networks using ReLU, potentially because of the dying-neuron problem.Source: §3.2.3 · p. 4

3.3. Advanced B-cos networks

  • To adapt conventional architectures, the authors replace convolutional and fully connected layers with B-cos versions using two MaxOut units, and remove other nonlinearities and batch normalization.These changes are intended to maintain alignment pressure and allow a single-linear-transform summary.Source: §3.3 · pp. 4–5

4. Experimental setting

The experiments assess B-cos networks on CIFAR-10 and ImageNet and evaluate their explanations with a grid pointing game. The setup also specifies image encoding and visualisation procedures.

Datasets and models

  • The study evaluates B-cos network accuracy and explanations on CIFAR-10 and ImageNet, using a nine-layer fully convolutional model for CIFAR-10 and four established architectures for ImageNet.The ImageNet models are VGG-11, ResNet-34, InceptionNet v3, and DenseNet-121; the CIFAR-10 model is tested with and without MaxOut.Source: §4 · p. 5

Image encoding

  • Images are encoded as [r, g, b, 1−r, 1−g, 1−b], which reduces bias toward bright regions and makes colors with the same original angle unambiguous under the new encoding.The paper says this encoding permits decoding the first-layer linear transformation into colors using pixel angles.Source: §4 · p. 5

Explanation evaluation

  • The grid pointing game measures how much positive attribution each explanation assigns to the correct location for each class logit in synthetic multi-class image grids.The authors construct 500 grids from confidently and correctly classified images and compare model-inherent maps with post-hoc methods.Source: §4 · p. 5

Visualisation details

  • For visualizing neuron transforms, the authors select positively contributing pixels, normalize paired color-channel weights, and scale opacity by the 99.5th-percentile weight norm.The normalization preserves each paired channel angle while placing values in the allowed color range.Source: §4 · p. 5

5. Results

B-cos networks retain competitive classification performance while providing model-inherent explanations that score highly on localisation. Increasing B improves weight alignment and interpretability, and neuron explanations reveal concepts across layers, including responses to watermarks.

5.1. Simple B-cos networks

  • Higher B increases weight alignment and explanation interpretability, while also decreasing accuracy and improving localisation.The visual comparison covers horse and car classes; localisation gains are described as significant.Source: Figure 5 · p. 6, Figure 4 · p. 6, §5.1 · p. 6
  • The plain B=2 CIFAR-10 network is competitive, and adding two MaxOut units per neuron yields 93.0%, on par with a ResNet-56.The strongest B=1.25 model reaches 93.8%, similar to the strongest reported ResNet result of 93.6%.Source: §5.1 · p. 6, Table 1 · p. 6

5.2. Advanced B-cos networks

  • B-cos VGG-11 outperforms its conventional counterpart, while DenseNet-121 loses 1.1 p.p. in Top-1 accuracy (74.4% vs. 73.3%).The authors report competitive results despite removing batch norm, ReLU, and MaxPool; a longer DenseNet-121 training schedule closes the accuracy gap with its pretrained counterpart.Source: §5.2 · p. 6, Table 2 · p. 6
  • Model-inherent contribution maps outperform evaluated post-hoc explanations and achieve close to optimal localisation scores across converted B-cos architectures.The comparison includes gradient-based and perturbation-based attribution methods.Source: Figure 6 · p. 6, §5.2 · p. 7

5.2.1 Qualitative evaluation of explanations

  • Class-logit linear transforms align with class-discriminant patterns and resemble the objects in input images.The paper attributes this alignment to pressure during optimisation.Source: §5.2.1 · p. 7
  • Neuron explanations shift from low-level concepts in early layers to more complex concepts later, with some layer-87 neurons specific to wheels, faces, or eyes.Some neurons respond preferentially to watermarks, which the authors note may be informative for classification despite lacking semantic meaning.Source: §5.2.1 · pp. 7–8, Figure 8 · p. 7, Figure 10 · p. 8
  • For uncertain predictions, differences between class contribution maps show image features that support each of the two most likely classes.The comparison is presented as a Δ-Explanation.Source: Figure 9 · p. 7, §5.2.1 · p. 8

5.2.2 Limitations

  • Weight normalisation and an additional down-scaling factor increase training and inference time by up to 60% versus same-size baselines.The authors expect optimised implementation to reduce this overhead.Source: §5.2.2 · p. 8

6. Conclusion

The conclusion presents B-cos transforms as a way to improve DNN interpretability through weight-input alignment while preserving faithful model explanations. The authors report that the transforms can replace conventional linear transforms with only minor classification-accuracy drops.

6. Conclusion

  • B-cos transforms increase weight-input alignment during optimisation and can improve interpretability with only minor drops in classification accuracy.The authors describe the transforms as drop-in replacements for linear transforms in conventional DNNs.Source: §6 · p. 8
  • Structurally constraining networks to use alignment allows explanations that faithfully reflect the underlying model.The authors frame this as evidence that strong performance and interpretability need not be at odds.Source: §6 · p. 8
  • The authors argue that interpretable DNN decisions are an essential ingredient for building trust, particularly in safety-critical situations.Source: §6 · p. 8

Supplementary Material

The supplement adds qualitative and quantitative results, implementation details, and derivations. Its qualitative material clarifies that B-cos linear mappings are the actual input-to-output transforms used by the model, not merely visualisations.

Supplementary Material

  • The supplement presents additional model-inherent explanations for DenseNet-121 class logits and intermediate neurons, other B-cos networks, and comparisons with post-hoc attribution methods.Source: Supplementary Material · p. 11
  • Additional quantitative material covers localisation results for two B-cos networks and pretrained conventional DNNs, ImageNet results across model variants, and watermark-neuron predictive power.Source: Supplementary Material · p. 11
  • The supplement also details architectures, training, interpretability evaluation, derivations, and image encoding for visualising linear transforms.Source: Supplementary Material · p. 11

A. Additional qualitative examples

  • B-cos mappings W1→L(x) are the actual linear transformations applied to inputs to compute outputs, providing an accurate summary of model computations.The supplement illustrates this with class logits computed by scalar products and class probabilities obtained by applying sigmoid; probabilities do not sum to one because the models use BCE loss.Source: A. Additional qualitative examples · p. 12, Figure A1 · p. 12

A.1. Additional explanations for class logits [DenseNet-121]

The additional examples show B-cos explanations for DenseNet-121 class logits and intermediate neurons, as well as intermediate neurons in ResNet-34, InceptionNet, and VGG-11. The explanations align with discriminative features, which may include contextual background, and intermediate neurons represent specific concepts.

A.1. Additional explanations for class logits [DenseNet-121]

  • Model-inherent explanations are reported as higher in visual quality than post-hoc comparisons while accurately summarising model computations for the displayed classes.GradCam highlights similar regions and LIME also produces colour explanations, but both are post-hoc approximations.Source: §A.1 · p. 12, Figure A2 · p. 12
  • DenseNet-121 class-logit mappings align with class-discriminative features, sometimes including background context because the model was not trained to highlight only class objects.Examples include mappings focused on specific image regions as well as context features associated with classes.Source: §A.1 · p. 13, Figure A3 · p. 14, Figure A4 · p. 15

A.2. Additional explanations for intermediate neurons [DenseNet-121]

  • Among 16 additional highly activating layer-87 neurons, the explanations indicate specific concepts such as faces, snouts, water, and grass.The examples are drawn from the 20 most highly activating neurons.Source: §A.2 · p. 16, Figure A5 · p. 16

A.3. Explanations for other B-cos networks

  • Intermediate-neuron explanations for ResNet-34, InceptionNet, and VGG-11 show neuron complexity tending to increase across layers, as in DenseNet-121.Source: §A.3 · p. 17, Figure A6 · p. 17

B.1. Localisation scores.

The localisation results report that B-cos networks are best explained by their model-inherent linear transforms among the tested methods. Comparisons with pretrained conventional models are used to estimate the interpretability gain from replacing their linear transforms with B-cos transforms.

B.1. Localisation scores.

  • The model-inherent linear transforms provide the best explanation for the B-cos network among the tested methods in the grid pointing game.Source: §B.1 · p. 18, Figure B1 · p. 18
  • No tested method explains the baseline models better than model-inherent linear transforms explain the respective B-cos networks.The authors use this comparison to estimate the interpretability gain associated with replacing conventional linear transforms by B-cos transforms.Source: §B.1 · p. 18, Figure B2 · p. 18
  • LIME and GradCam can score well on localisation, but LIME has high variance and requires many forward passes, while GradCam offers low-resolution explanations of only the classification head.B-cos explanations are described as high-resolution, coloured, and covering the entire model, and can be extracted in one forward and backward pass.Source: §B.1 · p. 18

B.2. Impact of model size on performance

B-cos DenseNet-121 models retain competitive ImageNet accuracy, with accuracy improving as model size increases. The appendix also examines watermark-responsive neurons and records CIFAR-10 and ImageNet implementation choices.

B.2. Impact of model size on performance

  • B-cos DenseNet-121 accuracy improves with model size; a growth-factor-22 model with two MaxOut units is similar in size to the pretrained baseline and has a 1.2-point accuracy drop.Increasing model size via MaxOut is more computationally efficient than increasing growth factor with a single unit, because feature-channel count stays unchanged.Source: Figure B3 · p. 19, §B.2 · p. 19
  • 74.4 →72.6 (∆=1.8) is the Top-1 accuracy drop for a B-cos DenseNet-121 without MaxOut versus the conventional DenseNet-121.The no-MaxOut model uses no non-linearity other than B-cos.Source: Table B1 · p. 19, §B.2 · p. 19

B.3. Class-discriminative information content in watermark neurons

  • Neuron 341, which consistently responds to image text, activates on a non-uniform class distribution among its 500 highest-activation images.The authors interpret the skew as indicating that watermarks are informative for classification.Source: Figure B4 · p. 19, §B.3 · p. 19

C. Implementation details

  • The implementation appendix includes pseudocode for B-cos convolution and describes training and post-hoc attribution implementations.The algorithm is identified as B-cos-Conv2d.Source: C. Implementation details · p. 20

C.1.1 CIFAR10

  • The CIFAR-10 experiments use a nine-layer convolutional architecture, and the appendix details signal scaling and training settings.The reported architecture uses mostly 3 × 3 kernels and output channels from 64 to 256, ending with 10 channels.Source: §C.1.1 · p. 20

C.1.2 ImageNet

  • For ImageNet, layer-output scaling and output temperature are network-dependent; their values were experimentally chosen for accuracy and stable training.The scaling factor s is 100 for DenseNets and ResNets, 200 for InceptionNet, and 1000 for VGG; the appendix says automatic temperature selection remains future work.Source: §C.1.2 · p. 20

C.2. Attribution methods

The appendix lists the post-hoc attribution methods compared with the model-inherent explanation and describes their evaluation on a grid pointing game.

C.2. Attribution methods

  • The comparison includes vanilla gradients, Input×Gradient, Integrated Gradients, DeepLIFT, GradCam, LIME, and RISE.The model-inherent explanation is the linear transform W1→L(x).Source: §C.2 · p. 20
  • Integrated Gradients uses 50 integration steps; RISE uses 500 masks, and LIME uses 500 samples with its stated default kernel size and feature count.RISE’s defaults are s = 8 and p = 0.1; LIME’s are k = 4 and n = 5.Source: §C.2 · pp. 20–21

C.2.1 Localisation metric

  • The grid pointing game evaluation uses 500 3 × 3 images assembled by sampling classes and selecting high-confidence images not already used.Images are sorted by classification confidence for each class before sampling.Source: §C.2.1 · p. 21

D. Additional derivations and discussions

The appendix expresses B-cos units as rescaled linear transforms and explains how an augmented pixel encoding supports visualization by alignment angle.

D.1. On the B-cos transform in matrix form

  • A B-cos unit computes a linear transform rescaled by the absolute cosine similarity raised to B−1.The cosine is between the input and normalized weights; the sign is retained in the linear form.Source: §D.1 · p. 21
  • Parallel B-cos units can be expressed in matrix form, with the normalized weights of individual units forming the rows.The elementwise cosine and absolute-value operations apply across the units.Source: §D.1 · p. 21

D.2. On the relevance of image encoding for the visualisations

  • Encoding pixels as [r, g, b, 1−r, 1−g, 1−b] gives dark and bright pixels equal signal to weight and makes color recoverable from pixel-vector direction.Under the conventional RGB encoding, a black pixel has zero input and cannot contribute to the linear output; the added channels also distinguish colors that share an RGB-vector angle.Source: §D.2 · pp. 21–22

Related papers

Loading 2205.10268v1…