PaperlayerView exact revision on arXiv

Source-linked AI summary

This looks more like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagation

Srishti Gautam, Marina M. -C. Höhne, Stine Hansen, Robert Jenssen, Michael Kampffmeyer

arXiv:2108.12204v1cs.LG

TL;DR

Self-explaining ProtoPNet can provide coarse, spatially imprecise explanations, complicating artifact analysis. The paper introduces PRP for more precise model-aware explanations and uses them in multi-view clustering; it reports that this clustering is more efficient than a single-view approach.

  • Problem

    The effectiveness of self-explaining approaches has a research gap in quantitative analysis of their explanations, while ProtoPNet’s explanations are coarse and spatially imprecise.

    Source: §I · p. 2

  • Method

    The paper introduces PRP for more precise model-aware prototypical explanations and uses multiple prototype views in clustering to filter artifact-containing data.

    Source: §I · p. 2

  • Results

    Multi-view clustering using explanations from multiple prototypes is more efficient than the single-view LRP-based SpRAy approach [7].

    Source: §I · p. 2

  • Takeaways & Limitations

    The authors report that artifact information is entangled across ProtoPNet prototypes, making prototype pruning unreasonable, and instead use clustering to identify and remove artifact-containing data.

    Source: §I · p. 2, §VI · p. 14

Abstract

from arXiv · show

Current machine learning models have shown high efficiency in solving a wide variety of real-world problems. However, their black box character poses a major challenge for the understanding and traceability of the underlying decision-making strategies. As a remedy, many post-hoc explanation and self-explanatory methods have been developed to interpret the models' behavior. These methods, in addition, enable the identification of artifacts that can be learned by the model as class-relevant features. In this work, we provide a detailed case study of the self-explaining network, ProtoPNet, in the presence of a spectrum of artifacts. Accordingly, we identify the main drawbacks of ProtoPNet, especially, its coarse and spatially imprecise explanations. We address these limitations by introducing Prototypical Relevance Propagation (PRP), a novel method for generating more precise model-aware explanations. Furthermore, in order to obtain a clean dataset, we propose to use multi-view clustering strategies for segregating the artifact images using the PRP explanations, thereby suppressing the potential artifact learning in the models.

I. INTRODUCTION

The introduction motivates evaluating self-explaining models’ explanations and presents ProtoPNet as a case study. It proposes PRP and multi-view clustering to address imprecise explanations and artifact learning.

I. INTRODUCTION

  • PRP backpropagates prototype relevances to produce more precise, model-aware explanations than ProtoPNet’s coarse, spatially imprecise maps.The authors attribute ProtoPNet’s imprecision to model-agnostic upsampling.Source: §I · p. 2
  • Multi-view clustering uses explanations from multiple prototypes to suppress artifacts from data more efficiently than the single-view LRP-based SpRAy approach [7].The approach aims to preserve ProtoPNet’s “this-looks-like-that” explanations while filtering artifact-containing data.Source: §I · p. 2
  • The paper addresses a research gap in quantitative analysis of self-explaining approaches’ explanations, despite extensive study of post-hoc explainability, [9].The benefit of post-hoc methods has also been questioned [5].Source: §I · p. 2
  • ProtoPNet offers case-based reasoning by linking decision features to similar training-image features through prototypes.Its prototype layer can be inserted into arbitrary CNN architectures, and its test-image regions can be compared with prototype-image regions.Source: §I · pp. 1–2
  • The authors report that artifact features in ProtoPNet cannot feasibly be eradicated, because artifact information is entangled across prototypes.They propose filtering artifact-containing data with multi-view clustering instead of pruning artifact prototypes.Source: §I · p. 2

II. RELATED WORK

Related work distinguishes post-hoc explanation approaches from self-explaining networks and describes ProtoPNet’s prototype-based explanations. It also reviews artifact types that can affect model predictions.

A. Explainability methods

  • Model-aware methods account for internal model structure and tend to produce more precise explanations; LRP propagates output relevance back through layers to input features.Model-agnostic methods such as LIME and SHAP treat models as black boxes [13], [14].Source: A. Explainability methods · p. 2
  • ProtoPNet compares test-image regions with prototype training-image regions, whereas SENN identifies important training images and CBC reasons over class components [6],,.ProtoPNet can learn prototypes of varying sizes and be incorporated into different network architectures.Source: A. Explainability methods · pp. 2–3
  • XProtoNet adds an occurrence module for dynamic-sized prototype features, but model-agnostic upsampling can still produce misleading visualizations.XProtoNet was introduced for automated diagnosis in chest radiography.Source: A. Explainability methods · p. 3

B. Artifacts

  • Clever Hans artifacts are unintentional learned correlations, while backdoors are maliciously added artifacts intended to provoke erroneous predictions.The paper focuses on suppressing both types.Source: B. Artifacts · p. 3

1) Clever Hans:

Clever Hans artifacts are spurious training-data cues that models may use, while backdoor attacks intentionally associate data and labels to induce incorrect predictions. The section describes ProtoPNet, its explanation limitations, and PRP-based multi-view clustering for artifact suppression.

1) Clever Hans:

  • Clever Hans artifacts are spurious correlations in training data that a model may use, while backdoor attacks deliberately modify data and labels to induce incorrect predictions.A watermark in horse images is an artifact example; a post-it added to stop signs and labeled as speed signs illustrates a backdoor attack.Source: 1) Clever Hans · p. 3, 2) Backdoor attacks · p. 3

C. Multi-view Clustering

  • PRP maps from class prototypes serve as multiple views of an image for multi-view clustering intended to suppress artifacts in the data.The approach treats prototype explanations as distinct views that can reflect artifact and non-artifact features.Source: C. Multi-view Clustering · p. 3

A. ProtoPNet

  • ProtoPNet assigns each class a fixed set of prototypes, whose patch similarities are max-pooled and whose prototypes are projected onto same-class training patches.Prototype visualizations are produced by upsampling similarity maps to input-image size.Source: A. ProtoPNet · p. 4

B. Evaluation of ProtoPNet’s explanations

  • PRP identifies the watermark text as relevant to the horse prediction, whereas ProtoPNet’s explanation spreads relevance broadly and cannot distinguish text from grass.The example illustrates how PRP provides a more fine-grained prototype explanation.Source: B. Evaluation of ProtoPNet’s… · p. 4, Figure 1 · p. 4
  • ProtoPNet explanations are coarse because low-resolution activation maps are bilinearly upsampled without accounting for large effective receptive fields.The naive upsampling fails to truthfully localize relevance to the correct input area.Source: B. Evaluation of ProtoPNet’s… · pp. 4–5

C. Case Study: Clever Hans artifact detection with ProtoPNet

In the CH-100 experiment, ProtoPNet’s upsampled heatmaps did not clearly localize the inserted artifact. Removing prototypes 6 and 8 reduced artifact-test accuracy, but retraining showed artifact information remained in the other prototypes.

IV. PROTOTYPICAL RELEVANCE PROPAGATION AND ENHANCED SUPPRESSION OF ARTIFACTS

Prototypical Relevance Propagation (PRP) replaces ProtoPNet’s upsampled visualizations with model-aware prototype explanations. The section describes propagating prototype similarity relevance back to input features and using the resulting maps as multiple views for clustering.

IV. PROTOTYPICAL RELEVANCE PROPAGATION AND ENHANCED SUPPRESSION OF ARTIFACTS

  • PRP addresses ProtoPNet’s low-resolution and spatially imprecise visualizations while retaining prototype-based self-explanation.Source: §IV · p. 6

A. Prototypical Relevance Propagation (PRP)

B. Multi-view Clustering

The paper treats each prototype explanation as a view of the same image and applies multi-view clustering to PRP maps to group artifact and non-artifact images. It compares spectral and deep-learning clustering approaches.

  • PRP explanations serve as multiple views of each image, which the paper clusters into artifact and non-artifact groups.Source: B. Multi-view Clustering · p. 7
  • The comparison includes two-step spectral methods Co-Reg and WMSC, a one-step rank-constrained method MCGC, and deep clustering methods.Source: B. Multi-view Clustering · pp. 7–8
  • Deep methods encode each view separately, learn fusion weights, and form cluster assignments from the fused representation; CoMVC adds selective contrastive alignment to SiMVC.Source: B. Multi-view Clustering · p. 8

A. Dataset

The study uses LISA traffic-sign data with synthetic Clever Hans and backdoor artifacts, alongside PASCAL VOC horse images, to examine ProtoPNet explanations. PRP maps are more spatially precise and more faithful in relevance-ordering tests than ProtoPNet heatmaps.

A. Dataset

  • The experiments use LISA traffic-sign data and PASCAL VOC 2007 horse images, whose watermark is naturally present in some horse images.LISA’s 47 classes are grouped into five high-level classes; the person class is removed from PASCAL VOC to reduce duplicate images across classes.Source: A. Dataset · p. 8, B. PRP maps vs ProtoPNet heatmaps · p. 8

1) Clever Hans:

  • Clever Hans training sets place a yellow post-it artifact in 100%, 50%, or 20% of LISA stop-sign images, while BD-15 labels 15% of artifact-bearing stop signs as speed-limit signs.The experiments also compare artifact tests, with artifacts in all stop-sign test images, against clean tests, with artifacts in none.Source: 1) Clever Hans · p. 8, 2) Backdoor · p. 8

B. PRP maps vs ProtoPNet heatmaps

  • On CH-50 images, ProtoPNet heatmaps highlight broad, imprecise regions, whereas PRP maps pinpoint the artifact across the stop-sign prototypes.PRP also highlights disjoint regions: prototype 5 marks both the artifact and “ST” on the sign, while prototypes 6 and 8 are later found to capture no significant features.Source: B. PRP maps vs ProtoPNet heatmaps · pp. 8–9, Figure 6 · p. 9
  • PRP maps are higher-resolution and more variable than ProtoPNet heatmaps, and can show disjoint relevant regions that naive spatial upsampling misses.For example, PRP marks both the artifact and sign text as relevant for prototype 5.Source: B. PRP maps vs ProtoPNet heatmaps · p. 9
  • For both clean and artifact test images, PRP identifies more relevant pixels by prototype similarity, while the test also reveals ineffective prototypes 6 and 8.The relevance-ordering test starts from random images and gradually adds pixels sorted by explanation relevance; these prototypes also respond strongly to random noise.Source: B. PRP maps vs ProtoPNet heatmaps · pp. 9–10, Figure 9 · p. 10
  • PRP maps produce a steeper class-score increase than ProtoPNet heatmaps as the most relevant pixels are added, for both clean and artifact images.Figure 10 reports the relevance-ordering test averaged over the same images with and without artifacts.Source: B. PRP maps vs ProtoPNet heatmaps · p. 10, Figure 10 · p. 10

C. Assessing the network behavior with PRP maps

PRP provides finer-grained explanations while retaining ProtoPNet’s prototype-based “this-looks-like-that” reasoning. The maps expose artifact and non-artifact features in stop-sign and horse prototypes.

C. Assessing the network behavior with PRP maps

D. Multi-View Clustering for suppressing artifacts

Because artifact information is entangled across ProtoPNet prototypes, the authors propose clustering PRP maps as multiple views to find artifact-bearing training samples. They report that this approach captures information better than SpRAy, especially with multiple views.

D. Multi-View Clustering for suppressing artifacts

  • The authors propose removing detected artifact-bearing training samples and retraining, because PRP shows artifact features are incorporated across almost all prototypes.They argue that pruning artifact prototypes is not feasible when artifact information is entangled throughout the network.Source: D. Multi-View Clustering for… · pp. 11–12
  • PRP maps for a class’s prototypes serve as multiple views of each image, which multi-view clustering uses to separate artifact-bearing from clean training images.The proposed clustering assigns images to two groups: artifact and clean.Source: D. Multi-View Clustering for… · p. 12
  • Across Clever Hans and backdoor settings, the authors compare multi-view clustering on PRP maps with SpRAy and report better information capture, especially in the multiple-view setting.Table II covers accuracy and F1 scores across data scenarios and compares clustering on PRP and LRP maps.Source: D. Multi-View Clustering for… · p. 12, Table II · p. 12

1) Clever Hans type artifacts in 50% training data:

On CH-50, multi-view clustering of PRP maps separates artifact and clean images effectively, while SpRAy fails on the same artifact-clustering task. A single prototype’s PRP map is insufficient for SpRAy to distinguish the groups.

1) Clever Hans type artifacts in 50% training data:

  • The comparison assumes artifact presence can be recognized in PRP maps and clusters downsampled maps into two groups.The evaluated methods include SpRAy, SiMVC, CoMVC, Co-Reg, WMSC, and MCGC; maps are downsampled to 80x80.Source: 1) Clever Hans type artifacts in 50%… · p. 12
  • Co-Reg and WMSC each achieve above 94% accuracy, while SiMVC and CoMVC reach almost 100% when CH-50 artifact and clean classes are balanced.The reported comparison concerns separating artifact-bearing from clean images using multi-view clustering on PRP maps.Source: 1) Clever Hans type artifacts in 50%… · p. 12
  • SpRAy fails to cluster CH-50 artifacts using either LRP maps or concatenated PRP maps.Table II reports accuracy and F1 scores for the clustering methods and SpRAy comparisons.Source: 1) Clever Hans type artifacts in 50%… · p. 12
  • 53.80% accuracy and 0.68 F1-score are the best single-prototype SpRAy results, from prototype 7; combined SpRAy-PRP yields 59.12% accuracy and 0.68 F1-score.The authors conclude that no individual prototype lets SpRAy separate artifact from clean data.Source: Table III · p. 12, 1) Clever Hans type artifacts in 50%… · p. 13

2) Clever Hans type artifact in 20% training data:

The experiments show that multi-view clustering can separate artifact and clean images in tested settings, while ProtoPNet visualizations can understate how broadly a backdoor affects its prototypes. The authors conclude that PRP explanations and clustering help expose and remove artifact features.

2) Clever Hans type artifact in 20% training data:

  • On CH-20, stop-sign accuracy was 99.7% on artifact test data and 95.8% on clean test data, indicating continued Clever Hans effects.This scenario evaluates clustering when fewer artifacts occur in training data.Source: 2) Clever Hans type artifact in 20%… · p. 13
  • 97.99% accuracy was achieved by SiMVC on CH-20, while SpRAy produced F1-scores of 0.04 and 0.08 on LRP and PRP maps.The authors interpret the low SpRAy scores as clustering almost all images into one cluster.Source: 2) Clever Hans type artifact in 20%… · p. 13

3) Backdoor type artifact in 15% training data:

  • With the artifact in all stop-sign test images, most were classified as speed limits and only 1% were classified correctly.The experiment used the BD-15 dataset; Figure 15 shows speed-limit prototypes and their heatmaps.Source: 3) Backdoor type artifact in 15%… · p. 13
  • Removing the backdoor prototype raised stop-sign accuracy from 1.0% to 6.5%, but retraining the last layer lowered it to 2.5%.PRP showed that almost all prototypes were activated by the artifact, despite referring to speed-limit signs.Source: 3) Backdoor type artifact in 15%… · p. 13
  • 99.42% accuracy and 0.98 F1-score were achieved by Co-Reg in the backdoor setting; SpRAy-PRP scored 0.02 F1, while SpRAy-LRP scored 0.91 F1.SiMVC and CoMVC scored 0.60 and 0.57 F1, respectively.Source: 3) Backdoor type artifact in 15%… · p. 13

VI. CONCLUSION

  • The authors conclude that PRP gives more precise, higher-resolution prototypical explanations that help uncover spurious features for identification and removal through multi-view clustering.They argue that evaluating self-explaining models' explanation quality is important.Source: §VI · p. 14

VII. APPENDIX

The appendix describes ProtoPNet's cost function, the relevance-propagation steps used for SpRAy-LRP, and class combinations for the LISA five-class datasets.

A. ProtoPNet: Cost function

  • ProtoPNet's overall cost function uses cross-entropy, cluster, and separation losses.The notation defines N as the number of training images, yi as image i's true label, Pyi as prototypes for that class, and ez as convolutional patches matching prototype size.Source: A. ProtoPNet: Cost function · p. 15

B. SpRAy-LRP

  • For SpRAy-LRP, class-score relevances are backpropagated to the similarity-score layer using LRPCMP and LRPϵ; PRP rules are then used for the rest of the network.Relevances for all prototypes are combined to obtain relevance at the CONV layer.Source: B. SpRAy-LRP · p. 15

C. LISA 5 class dataset

  • The LISA five-class dataset table lists class combinations for CH-100, CH-50, CH-20, and BD-15.Source: Table V · p. 15

Related papers

Loading 2108.12204v1…