Source-linked AI summary
Distilling a Neural Network Into a Soft Decision Tree
Nicholas Frosst, Geoffrey Hinton
TL;DR
Deep neural networks perform well but are difficult to explain because their distributed representations and learned regularities obscure individual classification decisions. The paper distills a trained neural network into a soft decision tree using informative targets, yielding a more explainable model that generally outperforms a tree trained directly on data while remaining below the neural network’s performance.
Problem
Deep neural networks’ distributed representations and weak statistical regularities make particular classification decisions difficult to explain, while ordinary decision trees generally have weaker generalization and can overfit.
Method
The paper trains a soft decision tree using a neural network’s predictions as soft targets, transferring knowledge through hierarchical decisions and leaf class distributions.
Results
The distilled soft decision tree generalizes better than one trained directly on the data but performs worse than the neural network providing its soft targets.
Takeaways & Limitations
A soft decision tree provides directly explainable classification decisions while retaining some benefits of deep neural networks.
Takeaways & Limitations
Soft decision trees begin to overfit at fewer total parameters than multi-layer neural networks because lower nodes receive only a small fraction of the training data.
Abstract
from arXiv · showhide
Deep neural networks have proved to be a very effective way to perform classification tasks. They excel when the input data is high dimensional, the relationship between the input and the output is complicated, and the number of labeled training examples is large. But it is hard to explain why a learned network makes a particular classification decision on a particular test case. This is due to their reliance on distributed hierarchical representations. If we could take the knowledge acquired by the neural net and express the same knowledge in a model that relies on hierarchical decisions instead, explaining a particular decision would be much easier. We describe a way of using a trained neural net to create a type of soft decision tree that generalizes better than one learned directly from the training data.
1 Introduction
The paper addresses the interpretability–generalization tension by distilling a trained neural network into a soft decision tree that mimics its input-output function. The resulting tree uses hierarchical decisions for explanations while retaining some neural-network generalization benefits.
- Motivation: Distributed representations make deep neural-network decisions difficult to explain, especially for intermediate hidden layers and combinations of active features.Weak and potentially spurious statistical regularities further complicate interpreting individual hidden units.
- Motivation: Decision trees offer short, input-based explanations, but their lower nodes often overfit because they receive only a small fraction of the training data.Avoiding this overfitting can require a training set exponentially larger than the tree depth.
- Approach: The proposed approach trains a decision tree to mimic a deep neural network’s discovered input-output function, rather than trying to interpret the network’s hidden units.Unlabelled or synthetically generated data can expand the labelled training set used for the tree.
- Approach: Without unlabelled data, distillation and soft decisions can transfer some of the neural network’s generalization abilities to the decision tree.The paper uses a soft decision tree specifically to support this transfer.
- Outcome: At test time, the decision tree may be slightly less accurate than the neural network but is often faster and directly explainable.Its decisions can be engaged with through the tree’s hierarchical decision structure.
2 The Hierarchical Mixture of Bigots
The model is a soft binary decision tree whose learned filters assign inputs probabilistically through a hierarchy to static class distributions at the leaves. It can prioritize a single most-probable path for concise explanations or average all leaves for slightly better accuracy but much greater explanation complexity.
- Tree structure: Soft binary decision trees use learned filters and biases at inner nodes, learned class distributions at leaves, and mini-batch gradient descent to train all parameters simultaneously.Their decision boundaries need not align with input-vector axes.
- Tree structure: At each inner node, the sigmoid of the filtered input determines the probability of taking the right branch.The input is x, and σ is the sigmoid logistic function.
- Tree structure: The model acts as a hierarchical mixture in which filters assign examples to leaves by path probability and each leaf supplies a static distribution over output classes.The leaf distributions do not inspect the data after training.
- Tree structure: An inverse temperature β sharpens the node decisions by changing the branch probability to pi(x) = σ(β(xwi + bi)).This is introduced to avoid very soft decisions in the tree.
- Prediction and explanation: The model can select the leaf with greatest path probability for concise explanations or average all leaf distributions for marginally better accuracy.The averaging approach has exponentially greater explanation complexity, so the paper uses the maximum-path leaf output.
- Training: Training minimizes path-probability-weighted cross entropy between leaf distributions and the target distribution.For an input x, P_ℓ(x) denotes the probability of arriving at leaf ℓ.
3 Regularizers
The tree uses a cross-entropy penalty to encourage balanced branching, with depth-dependent adjustments to account for changing data distribution and classification scope.
- Regularization penalty: The penalty is the cross entropy between the desired 0.5, 0.5 branch distribution and each node’s actual average distribution α, (1 −α).For node i, α is computed from the path probabilities of examples reaching that node.
- Regularization penalty: The penalty prevents poor solutions where internal nodes assign nearly all probability to one subtree and logistic gradients become nearly zero.
- Depth dependence: Penalty strength decays exponentially with node depth, proportional to 2^-d, because equal splits become less appropriate for nodes specializing in fewer classes.The authors report better test accuracy with this depth-dependent penalty.
- Running averages: An exponentially decaying running average with a depth-proportional time window improves estimation of subtree-use probabilities for lower nodes.Lower nodes see an exponentially smaller fraction of each training batch.
- Visualization: The depth-4 MNIST visualization shows learned filters at inner nodes and class-distribution visualizations at leaves.One internal filter distinguishes between potential classifications 3 and 8.
4 MNIST Results
On MNIST, directly trained soft decision trees underperform the neural network, while distillation with soft targets substantially improves tree accuracy.
- Overfitting: The tree’s lower nodes receive only a small fraction of the training data, causing soft decision trees to overfit at fewer total parameters than multi-layer neural networks.
- Direct training: 99.21% test accuracy was achieved by the convolutional neural net, compared with at most 94.45% for a depth-8 soft decision tree trained on true targets.
- Distillation: 96.76% test accuracy was achieved by the soft decision tree trained with targets combining true labels and neural-network predictions.This result was about halfway between the neural net and the directly trained soft decision tree.
5 Explaining how a soft decision tree makes a classification
The soft decision tree is designed to make classifications explainable through hierarchical decisions rather than neural-network-style hierarchical features. A classification can be traced through the learned filters along its root-to-leaf path.
- Decision explanation: A classification can be explained by examining all learned filters along the path between the root and the classification’s leaf node.
- Decision explanation: The model relies on hierarchical decisions instead of hierarchical features, which are difficult to engage with beyond the first one or two neural-network layers.
- Related explanations: Gradient-based neural-network explanations may identify a single input point that excites a neuron without representing the entire manifold of inputs producing that activation.
- Related explanations: Local explainable models can approximate a neural network’s behavior by sampling inputs around an area of interest and fitting a model to its outputs.
6 Other Data Sets and Results
Distillation improved soft decision tree accuracy on both spatial Connect4 and non-spatial Letter data, while learned filters also exposed meaningful game subtypes.
- Connect4: 80.60% Connect4 accuracy after distillation exceeded 78.63% without distillation and prior gradient-descent trees at comparable depths.The distilled tree used depth 8; prior trees reached 76.50% at depth 8 and 77.45% at depth 20.
- Connect4: The Connect4 tree’s learned filters separated games according to whether pieces occupied board edges or the center.The first two layers were visualized to inspect these learned filters.
- Letter: 81.0% Letter accuracy after distillation exceeded 78.0% for a depth-9 tree trained on raw data.The neural-net ensemble supplying the soft targets had 95.9% test accuracy.
7 Conclusion
The paper proposes soft decision trees trained with neural-network predictions as informative targets to make model behavior more explicable. The trees generalize better than directly trained trees but remain less accurate than the neural networks providing their targets.
- Conclusion: The proposed model is an explicable soft decision tree trained by stochastic gradient descent using neural-network predictions as informative targets.Learned filters make hierarchical decisions and select a static class-probability distribution as output.
- Conclusion: Soft decision trees generalize better than trees trained directly on the data but perform worse than the neural networks supplying their soft targets.The method therefore retains some neural-network benefits while producing decisions that can be explained.