PaperlayerView exact revision on arXiv

Source-linked AI summary

Interpretable Image Recognition with Hierarchical Prototypes

Peter Hase, Chaofan Chen, Oscar Li, Cynthia Rudin

arXiv:1906.10651v2cs.CVcs.LGeess.IVstat.ML

TL;DR

Visual prototype methods had not used taxonomic class organization to explain predictions at multiple levels. This paper introduces hierarchical prototypes for classification across a taxonomy, including unseen classes at an appropriate broader level. On two ImageNet-subset tasks, the model performs approximately as well as its black-box counterpart while producing interpretable predictions.

  • Problem

    Prototype-based vision methods had not incorporated taxonomic class organization, limiting explanations to the predicted class rather than its broader taxonomic levels.

    Source: §1 · p. 1

  • Method

    The paper combines interpretable feature prototypes, hierarchical class organization, and novel-class detection to classify images at multiple taxonomic levels.

    Source: §1 · p. 2

  • Results

    The model performs approximately as well as its counterpart black-box model on familiar-class and previously unseen-class tasks while producing interpretable predictions.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Hierarchical prototypes support interpretable predictions for unseen classes at the broader taxonomy level to which they belong.

    Source: §1 · p. 2

    Meaningful features may not be captured by existing quantitative interpretability metrics, so domain experts must check feature relevance for their applications.

    Source: Ensuring Interpretability · p. 3

Abstract

from arXiv · show

Vision models are interpretable when they classify objects on the basis of features that a person can directly understand. Recently, methods relying on visual feature prototypes have been developed for this purpose. However, in contrast to how humans categorize objects, these approaches have not yet made use of any taxonomical organization of class labels. With such an approach, for instance, we may see why a chimpanzee is classified as a chimpanzee, but not why it was considered to be a primate or even an animal. In this work we introduce a model that uses hierarchically organized prototypes to classify objects at every level in a predefined taxonomy. Hence, we may find distinct explanations for the prediction an image receives at each level of the taxonomy. The hierarchical prototypes enable the model to perform another important task: interpretably classifying images from previously unseen classes at the level of the taxonomy to which they correctly relate, e.g. classifying a hand gun as a weapon, when the only weapons in the training data are rifles. With a subset of ImageNet, we test our model against its counterpart black-box model on two tasks: 1) classification of data from familiar classes, and 2) classification of data from previously unseen classes at the appropriate level in the taxonomy. We find that our model performs approximately as well as its counterpart black-box model while allowing for each classification to be interpreted.

1 Introduction

The paper argues that interpretable vision should combine taxonomic predictions at multiple levels with explanations tailored to each level. It introduces hierarchical prototypes for familiar and previously unseen classes, reporting performance approximately as strong as a counterpart black-box model while keeping predictions interpretable.

1 Introduction

  • Vision models should use human-interpretable features, predict at each level of a known taxonomy, and detect unseen subclasses within known coarse classes.The paper frames these functions as paralleling human visual judgment.Source: §1 · p. 2
  • Taxonomic predictions let users trade specificity for accuracy, including making a coarse decision when sibling-class distinctions do not change the appropriate policy response.The paper gives the example of recognizing a gun when the distinction between pistol and assault rifle is uncertain.Source: §1 · p. 2
  • Explanations can differ by taxonomic level, showing distinct evidence for a broad class and its finer subclasses, and revealing how specifically a prediction went wrong.The paper contrasts wheels as evidence for an automobile with sirens as evidence for an ambulance.Source: §1 · p. 2
  • The approach aims to recognize unseen classes as members of familiar broader classes, rather than forcing them into a known fine-grained class.The paper motivates this because deployed models will encounter images from novel classes.Source: §1 · p. 2
  • The model performs approximately as well as its counterpart black-box model on familiar-class and previously unseen-class classification while producing interpretable predictions.The unseen-class task assigns images to the appropriate level in the taxonomy.Source: §1 · p. 2

2 Related Work

Related work covers interpretable vision, hierarchical classification, and novel-class detection, but these approaches had not been successfully combined. The paper combines hierarchical class-part prototypes with a detector for unseen subclasses within known coarse classes.

2 Related Work

  • Prior vision research had developed hierarchical classification, interpretable modeling, and novel-class detection, but had not successfully combined them.Existing explanation approaches included post-hoc methods and methods that localized image regions.Source: §2 · p. 2, Interpretable Models · p. 2

Interpretable Models.

  • Prototype-based interpretable models include approaches by Branson et al. (2014), Li et al. (2018), and Chen et al. (2018), with differing ways of showing or visualizing features.The paper says its heat-map method shows local information that Branson et al. (2014) do not demonstrate, while Li et al. (2018)'s decoder does not work for complex naturalistic images.Source: Interpretable Models · p. 2
  • Unlike flat prototypes particular to dataset classes, this method learns prototypes for both leaf classes and parent groups, enabling explanations at multiple taxonomy levels.This supports distinctions such as why a panda is an animal rather than a vehicle, and why it differs from a lion.Source: Interpretable Models · p. 2

Hierarchical Classification.

  • The model uses a supervised CNN that branches before classification, producing representations unique to each subclassification task.Other approaches construct taxonomy predictions from a single network output.Source: Hierarchical Classification · p. 3
  • Its latent-space prototypes represent image parts, unlike prior CNN-based hierarchical methods, which did not use prototypes for directly interpretable classifications.The paper contrasts these with a Bayesian graphical model using pixel-space prototypes that each represent an entire class (Salakhutdinov, Tenenbaum, and Torralba 2012).Source: Hierarchical Classification · p. 3
  • The paper adapts novel-class detection methods to identify instances inside a known coarse class but outside its known subclasses.The cited review describes methods that use model logits, including supervised approaches that require withheld out-of-distribution data (Shafaei, Schmidt, and Little 2018).Source: Hierarchical Classification · p. 3

3 Problem Descriptions

The section frames interpretable recognition as explaining predictions with meaningful features and classifying along a taxonomy, including when an image belongs to an unfamiliar fine-grained class. It formalizes hierarchical classification and novel-class detection within that taxonomy.

Ensuring Interpretability.

  • The model follows a prototype approach because its interpretable features are image parts drawn from concrete training instances, rather than whole class examples.The passage also describes this as case-based reasoning.Source: Ensuring Interpretability · p. 3
  • Interpretability metrics may depend on densely labeled datasets that miss relevant features, so domain experts must check feature meaningfulness for their applications.For ImageNet-like domains, the passage says laypeople can check whether features are meaningful.Source: Ensuring Interpretability · p. 3
  • Hierarchical classification predicts an image’s label at each taxonomy level, representing a path from coarse to fine classes while assigning zero probability to physically impossible paths.Branches need not have the same depth.Source: Ensuring Interpretability · pp. 3–4
  • Novel-class detection estimates whether an image belongs to a known child class conditional on membership in its parent, generalizing out-of-distribution detection at the root.Here, “novel class” includes classes entirely new to the taxonomy and classes unseen at a fine level but contained within a known broader class.Source: Ensuring Interpretability · p. 4

4 Model

HPnet extends prototype-based recognition to a predefined class taxonomy, using parent-specific prototype layers to produce interpretable predictions at different levels. Its prototypes capture local image evidence and encode information conditional on the parent class.

Image Recognition Model

  • HPnet generalizes Pnet to a predefined taxonomy, reducing to Pnet when the taxonomy has only one level of fine-grained classes.The model architecture is shown in Figure 4.Source: Image Recognition Model · p. 4, Figure 4 · p. 5
  • A CNN maps an image to patch vectors; parent-specific prototype layers compute similarity features, which fully connected layers turn into distributions over each parent’s child classes.The prototype similarity score is higher when a patch vector is close to a prototype.Source: Image Recognition Model · p. 4
  • Projecting prototypes onto training patches gives them visually inspectable receptive fields, and their upsampled activation maps show the image regions that activate them.Viewing these heat maps alongside highly activated prototypes helps interpret a classification.Source: Image Recognition Model · p. 5
  • Prototypes encode conditional information: ambulance prototypes distinguish ambulances from other vehicles, while vehicle prototypes distinguish vehicles from animals..Source: Image Recognition Model · p. 5
  • The model allocates eight prototypes per child class in the experiments, though equivalent prototype vectors can leave a child with fewer unique prototypes.Allocation is even across child classes within each parent’s prototype set.Source: Image Recognition Model · p. 5
  • Fine-grained class probabilities can be obtained by multiplying the probabilities along each path through the taxonomy.The model computes this for every path down the taxonomy.Source: Image Recognition Model · p. 5

Training Algorithm

Training alternates optimization with prototype projection, while an objective combines prediction accuracy, class-specific clustering and separation, and regularization. The phases initialize and optimize model layers, project prototypes onto eligible training patches, and optimize the fully connected layers.

Training Algorithm.

  • The model alternates layer optimization with projection because gradient descent alone could not preserve the constraint that each prototype equal a latent data patch.The training phases’ epoch counts are listed in Table 3 in Appendix A.Source: Training Algorithm · p. 5, Objective Function · p. 6
  • VGG-16 is trained from random initialization rather than ImageNet pretraining to avoid confounding novel-class tests that also use ImageNet data.The Pnet and HPnet convolutional layers are initialized from the most accurate trained VGG-16 model.Source: Training Algorithm · p. 5

Objective Function.

  • The objective sums cross-entropy, clustering, separation, and regularization terms across prototype layers; clustering and separation adapt Chen et al. (2018) to each layer.Cross-entropy encourages accurate predictions, while the clustering and separation costs encourage class-specific prototype structure.Source: Objective Function · p. 6
  • The regularizer applies l2 weights to a class’s own prototype similarities and l1 weights to other classes’ similarities, encouraging sparse reliance on non-class prototypes.The passage says this simplifies interpretation.Source: Objective Function · p. 6

Convolutional Layers and Prototypes.

  • In the first optimization phase, stochastic gradient descent updates the feature extractor and prototypes while keeping each classification layer fixed.The initialization sets same-class connections to 1 and other-class connections to −.5, so same-class activations add evidence and other-class activations diminish it.Source: Convolutional Layers and Prototypes · p. 6

Optimization of All Layers.

  • A subsequent phase optimizes all network layers together, including each prototype layer’s fully connected weights, which become sparse during this phase..Source: Optimization of All Layers · p. 6

Projection of Prototypes.

  • Every five epochs, prototypes are projected onto their closest training patches from their pre-allocated class; multiple prototypes may project to the same patch.Projection provides a direct correspondence between prototypes and training-data latent representations.Source: Projection of Prototypes · p. 6

Convex Optimization of Fully Connected Layers.

Novel Class Detection Model

Novel-class detection is fit separately at each taxonomy parent, distinguishing familiar children from not-yet-seen children and supporting predictions such as “novel vehicle”.

Novel Class Detection Model

  • A separate detector is fit for each parent node to distinguish its known children from children absent during recognition-model training.The vehicle example uses familiar vehicles {ambulance, sports car, pickup} and novel vehicles {cab, forklift, tractor, mountain bike}.Source: Novel Class Detection Model · p. 6
  • The authors test ScoreSVM, PbThreshold, and logistic regression as classifiers for novel-class detection [Shafaei, Schmidt, and Little (2018)].The classifiers are tested in the parent-specific detection setup.Source: Novel Class Detection Model · pp. 6–7
  • Combining parent-level novelty detection with image recognition supports nuanced predictions such as “novel vehicle”.The prediction uses the novel-vehicle detector's probability and the image-recognition model's probability.Source: Novel Class Detection Model · p. 7

5 Experiments

Experiments assess HPnet on an ImageNet subset through classification accuracy, prototype interpretation, and novel-class detection. HPnet has similar average accuracy to black-box and flat counterparts, and novel data can be classified coarsely well above chance.

5 Experiments

  • The recognition model uses 15 ImageNet classes, with 50 of each class’s 1,300 training images held out for validation; 15 additional classes are reserved for novel-class detection.The novel classes fall under the same coarse classes, with four kinds of vehicles and no scuba divers.Source: §5 · p. 7

Data Augmentation.

  • CEDA adds an equal number of random-noise images to each batch and encourages uniform predicted distributions for those images.The reported clustering comparison may be confounded because the two models use VGG-16 initializations with different accuracy.Source: Data Augmentation · p. 7, Table 1 · p. 7

Interpreting the Recognition Model’s Latent Space

  • A vehicle prototype activates wheels and wheel wells across pickups and sports cars, illustrating a feature shared across vehicle classes.The interpretation uses test images whose patch vectors are nearest to the prototype.Source: Interpreting the Recognition Model’s… · p. 7
  • The latent-space clustering metric averages the share of each prototype’s five nearest neighbors that belong to its corresponding class.Table 1 reports that prototypes tend to be surrounded by patch vectors from images of the correct classes; initialization differences may explain the change between models.Source: Interpreting the Recognition Model’s… · pp. 7–8, Table 1 · p. 7

Classifying a Test Image

  • In the forklift example, learned concepts such as wheels support classifying it as a vehicle, while the detector identifies it as a novel vehicle.It is novel relative to the familiar vehicle classes {ambulance, pickup, sports car}.Source: Classifying a Test Image · p. 8, Figure 6 · p. 8

Image Classification Accuracy

  • HPnet averages similar accuracy to black-box and flat counterparts, with a 2.32% fine-grained accuracy drop versus pretrained VGG-16.Table 2 reports fine-grained in-distribution, coarse-grained in-distribution, and coarse-grained novel-data accuracies averaged across five models per method.Source: Image Classification Accuracy · p. 8, Table 2 · p. 8
  • Novel data can be classified correctly at the coarse level far above the approximately 17% chance rate.The passage does not provide the observed accuracy value for this comparison.Source: Image Classification Accuracy · p. 8

Novel Class Detection

  • 73.72% accuracy on average over five recognition models for logistic regression on HPnet + CEDA; no detection method was clearly superior.The study tests ScoreSVM, a maximum-probability threshold, and logistic regression, each with and without CEDA.Source: Novel Class Detection · p. 8

6 Conclusion

The conclusion presents a model that classifies at each taxonomy level, detects novel fine-grained classes, and uses features tailored to parent nodes. The authors report that its ImageNet application demonstrates viability and richer explanations.

6 Conclusion

  • The model classifies at every semantic-taxonomy level, detects novel fine-grained classes, and uses interpretable features tailored to each parent node.Source: §6 · pp. 8–9
  • The authors describe it as the first vision model to combine all three tasks synchronously, and the first CNN-based model to organize prototypes hierarchically and detect novel classes across taxonomy levels.Source: §6 · p. 9
  • The ImageNet application demonstrates the method’s viability, while the authors report richer explanations and dramatically improved interpretability.Source: §6 · p. 9

Coarse Predictions from Flat Models

The section compares two ways to derive coarse-grained predictions from flat models for the C-Novel metric. The authors use summed fine-class probabilities because that approach performs better empirically.

Coarse Predictions from Flat Models

  • Flat-model coarse predictions can be evaluated by checking whether the fine-grained prediction falls within the true coarse class or by aggregating fine-class probabilities by coarse class.Source: Coarse Predictions from Flat Models · p. 9
  • With probability aggregation, the predicted coarse class is the one with the highest summed probability across its fine-grained classes.For example, vehicle probability sums ambulance, pickup, and sports-car probabilities.Source: Coarse Predictions from Flat Models · p. 9
  • The authors report that probability aggregation outperforms checking the fine-grained prediction, so they present accuracies computed with aggregation.Source: Coarse Predictions from Flat Models · p. 9

Testing Procedure for Novel Class Detection

Novel-class detection accuracy is averaged across five image-recognition models and, within each model, across leave-one-class-out tests for each parent node.

Testing Procedure for Novel Class Detection

  • For each recognition model, test accuracy for a parent node averages leave-one-class-out tests, with each novel class held out in turn.For the vehicle detector, cab, forklift, tractor, and mountain bike each serve once as the test novel class.Source: Testing Procedure for Novel Class… · p. 9
  • Detection accuracy averages results across five image-recognition models, with training images from ImageNet’s training partition and testing images from its testing partition.Source: Testing Procedure for Novel Class… · p. 9
  • Each test set is balanced between familiar and novel classes, and hyperparameters are tuned using 50 held-out training images.Source: Testing Procedure for Novel Class… · p. 9

Tables and Figures

The figures illustrate prototype activations and example classifications, while the tables describe clustering quality, test accuracies, and novel-class detection accuracy. The reported test-accuracy comparison says HPnet performs similarly to VGG-16 and Pnet.

Tables and Figures

  • Figure 7 depicts the novel-data taxonomy, whose data are not seen by the image-recognition model during training.Source: Figure 7 · p. 9
  • Table 3 reports epochs per training phase; projection occurs every five epochs during the convolutional-layer and all-layer phases, followed by convex optimization.The final convex-optimization phase lasts 10 epochs rather than 2.Source: Table 3 · p. 9
  • Table 4 reports latent-space clustering quality as the proportion of prototype nearest neighbors belonging to the same class, averaged across five models with 95% confidence intervals.Source: Table 4 · p. 9
  • Figure 8 illustrates an assault rifle classified as a weapon; the top four prototypes account for 75% of the weapon logit magnitude, and the model’s weapon accuracy is 83%.Source: Figure 8 · p. 10
  • Figure 9 illustrates a revolver classified as an everyday object, alongside 53% accuracy for classifying novel revolvers as weapons.The caption says inspecting prototype nearest neighbors could help diagnose the error and inform data collection or augmentation.Source: Figure 9 · p. 11
  • Figures 10–12 show weapon and ambulance prototypes whose nearest neighbors suggest trigger-related, color-on-white, text, or dark-lines-on-light patterns.Source: Figure 10 · p. 12, Figure 11 · p. 12, Figure 12 · p. 13
  • HPnet performs similarly to its black-box counterpart VGG-16 and flat counterpart Pnet on fine in-distribution, coarse in-distribution, and coarse novel-data accuracies.Table 5 reports averages across five trained models per method with 95% confidence intervals.Source: Table 5 · p. 14
  • Novel-class detection accuracies are averaged across five models per method and across parent nodes, with familiar and novel test data balanced.Source: Table 6 · p. 14
Loading 1906.10651v2…