Source-linked AI summary
Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions
Oscar Li, Hao Liu, Chaofan Chen, Cynthia Rudin
TL;DR
The paper tackles the difficulty of interpreting nonlinear neural networks and their history of being optimized primarily for accuracy. It introduces an autoencoder-based prototype network that compares inputs with learned latent-space prototypes and visualizes them through decoding. The resulting explanations are built into the model and remain loyal to its computations, while the interpretability terms tend to act as regularizers that improve robustness to overfitting in the reported experiments.
Problem
Nonlinear neural networks are difficult to understand, while posthoc explanations can require separate modeling and may not reflect the network’s actual reasoning.
Method
The paper combines an autoencoder and prototype layer, comparing encoded inputs with learned prototypes in latent space and visualizing prototypes through the decoder.
Results
The learned network naturally provides explanations for each prediction that are loyal to what the network computes.
Takeaways & Limitations
The prototypes provide insight into the network’s inner workings, class relationships, and important aspects of the latent space while partially tracing classification paths.
Abstract
from arXiv · showhide
Deep neural networks are widely used for classification. These deep models often suffer from a lack of interpretability -- they are particularly difficult to understand because of their non-linear nature. As a result, neural networks are often treated as "black box" models, and in the past, have been trained purely to optimize the accuracy of predictions. In this work, we create a novel network architecture for deep learning that naturally explains its own reasoning for each prediction. This architecture contains an autoencoder and a special prototype layer, where each unit of that layer stores a weight vector that resembles an encoded training input. The encoder of the autoencoder allows us to do comparisons within the latent space, while the decoder allows us to visualize the learned prototypes. The training objective has four terms: an accuracy term, a term that encourages every prototype to be similar to at least one encoded input, a term that encourages every encoded input to be close to at least one prototype, and a term that encourages faithful reconstruction by the autoencoder. The distances computed in the prototype layer are used as part of the classification process. Since the prototypes are learned during training, the learned network naturally comes with explanations for each prediction, and the explanations are loyal to what the network actually computes.
1 Introduction
The paper addresses the interpretability limits of nonlinear neural networks by building explanations directly into a prototype-based architecture. Its latent-space prototypes connect deep learning with case-based reasoning while remaining tied to the network’s computations.
- Motivation: Nonlinear neural networks are difficult for humans to understand, motivating efforts to make their predictions more transparent.The paper links this difficulty to concerns about trusting model predictions and detecting incorrect inputs or decisions.
- Contribution: The proposed architecture builds its own explanations into the model rather than explaining a previously trained black box.This design is presented as aligned with prototype classification and case-based reasoning, while contrasting with posthoc analysis.
- Motivation: Posthoc interpretability can require separate modeling and may produce multiple convincing explanations that do not reflect the network’s actual reasoning.The paper also notes that posthoc explanations may not be readily interpretable to humans.
- Prototype classification: The network classifies observations by proximity to prototype observations, allowing predictions to be related to particular training examples and, when uncertain, prototypes from different classes.The paper illustrates this with ambiguous handwritten digits such as 4 and 9.
- Latent-space reasoning: Unlike some prototype methods, the paper’s prototypes resemble training observations and are compared with inputs in an adaptive latent space.The autoencoder creates the latent space, enabling a flexible dissimilarity measure rather than relying on pixel-space L2 distance.
2 Methodology
The model combines an autoencoder with a prototype classification network that compares encoded inputs to learned prototypes and uses those distances for prediction. Its objective jointly balances classification accuracy, reconstruction fidelity, and prototype interpretability.
- Architecture: The architecture uses an encoder f and decoder g to reduce input dimensionality, reconstruct inputs, and provide latent features for a prototype classifier h.The classifier outputs a probability distribution over K classes from the encoded input.
- Architecture: Each prototype unit computes the squared L2 distance between an encoded input z and a learned prototype vector.These distances form the prototype layer’s representation for subsequent classification.
- Architecture: The prototype distances are transformed by a weighted fully connected layer and softmax normalization into class probabilities.The weight matrix W has dimensions K × m, and the softmax output represents the probability distribution over classes.
- Prediction: The classification process is distance-based in a learned low-dimensional feature space, with learned weights allowing all prototypes to contribute to each class prediction.With one prototype per class and W = −I_K×K, the nearest latent-space prototype determines the predicted class; more generally, W is learned when m differs from K.
- Interpretability: The architecture supports interpretation by decoding prototype vectors into visual examples and using learned connection strengths to indicate class representativeness.Because prototypes and encoded inputs share a latent space, the model can visualize prototype learning without posthoc analysis.
3 Case Study 1: Handwritten Digits
The MNIST case study evaluates the interpretable network’s accuracy, reconstruction quality, and prototype-based explanations. Its learned prototypes visualize digit variation and support prediction through latent-space distances and class-specific weights.
- Architecture and training: The model uses a convolutional autoencoder to encode 28×28 grayscale digits, classify latent representations through prototypes, and decode prototypes for visualization.The encoder and decoder use four convolutional layers each; the final decoder layer uses a sigmoid output for pixel values in (0, 1).
- Accuracy: After 1500 epochs, the model reached 99.53% training accuracy and 99.22% test accuracy on MNIST.Training used 55,000 training examples, 5,000 validation examples, and 10,000 test examples, with elastic deformation augmentation.
- Accuracy: 99.24% and 99.23% test accuracies from two ablated networks were comparable to the interpretable model’s 99.22%, indicating no observed predictive-accuracy penalty from the architecture.The ablations replaced the prototype layer or removed the decoder and changed the nonlinearity to ReLU.
- Visualization: The autoencoder achieved an average squared L2 reconstruction error of 4.22, supporting faithful decoding of prototype vectors into pixel space.Random training images and their reconstructions are shown in Figure 2.
- Visualization: Decoded prototypes are sharp-looking digit images, and the 15 prototypes capture variation in handwriting, including different loop shapes for 6s and curvatures for 3s.Figure 3 visualizes the learned prototypes in pixel space.
- Prediction explanations: For a test image of 6, the three smallest latent-space distances corresponded to decoded 6 prototypes, with one prototype substantially closer than the other two.The distance vector is multiplied by a learnable weight matrix before the softmax layer, so prototypes can influence classes differently.
4 Case Study 2: Cars
The car case study classifies rendered vehicles by viewing angle using decoded prototypes, while retaining comparable accuracy to a network without interpretable components. The R1 and R2 terms make prototypes realistic, representative, and distinct.
- The dataset contains 64 × 64 × 3 rendered car images across 11 azimuth-angle classes, split into 1,859 training and 154 test images.
- The model uses eleven prototypes, and decoded prototypes converge to a shared average car color because color is not important for angle prediction.
- The interpretable network does not sacrifice much accuracy compared with a model whose decoder and prototype layer are replaced by a fully connected layer.
- Removing both R1 and R2 produces decoded prototypes unlike real images, while removing only R1 likewise prevents realistic prototypes.
- Removing only R2 yields prototypes that incompletely represent the input space, with some prototypes becoming similar to one another.
- R1 aligns prototypes with training examples, whereas R2 encourages coverage of latent space and distinct prototypes, keeping explanations faithful to nearby classification evidence.
5 Case Study 3: Fashion MNIST
The Fashion-MNIST case study applies the same prototype-based model to clothing classification. It achieves accuracy comparable to similar standard convolutional networks, while prototypes emphasize class contours over fine-grained details.
- Fashion-MNIST contains 60,000 training examples and 10,000 test examples of 28×28 grayscale clothing images spanning 10 classes.
- 89.95% testing accuracy is comparable to standard convolutional networks with similar architecture complexity, reported at 87.6-92.5%.
- The learned prototypes include at least one representative for each class but contain fewer details than the original images.
- Prototype contours retain information the model considers more useful for classification than fine-grained details such as stripes, collars, and texture.
6 Discussion and Conclusion
The paper combines deep learning with case-based reasoning so predictions can be traced through learned prototypes. The authors report that the interpretability terms may also improve robustness to overfitting, while acknowledging that the model does not fully solve accountability and transparency problems.
- The model combines deep learning with case-based reasoning, using prototypes to provide insight into class relationships and important aspects of latent space.
- The prototypes allow partial tracing of the classification path for a new observation, rather than providing a full solution to accountability and transparency.
- The authors observed that adding R1 and R2 tended to act as regularization and help make the network robust to overfitting.
- The supplementary material and code are made available through the project repository.