Improving the Interpretability of Deep Neural Networks with Knowledge Distillation

I. INTRODUCTION

• DNN hidden representations remain black boxes, making it difficult for users to understand predictions and for designers to refine models, particularly in high-stakes domains.The examples include clinical decision support, disaster response, and recidivism prediction.Source: §I · p. 1
• Decision trees are valued for interpretability in healthcare but can overfit and perform poorly on large, heterogeneous EHR datasets.The introduction describes the goal as finding a balance between interpretability and performance.Source: §I · p. 1
• Visualization-based interpretation faces concerns because semantic information may reside in higher-layer spaces rather than individual units, challenging activation-maximization approaches.The introduction presents this as a reason to seek alternative interpretation methods.Source: §I · p. 1
• Many prior distillation works target smaller neural networks for resource-constrained settings rather than interpretation through non-neural models.The cited settings include edge computing and on-the-fly training.Source: §I · p. 2
• Distilling DNNs into vanilla decision trees through multi-output regression improves accuracy by about 1% to 5%, addressing the interpretability–performance trade-off.The authors describe this as an initial step using matching logits.Source: §I · p. 2

II. RELATED WORK

• The paper distinguishes its vanilla decision-tree student from the soft decision tree in [15], which mimics a trained DNN’s input-output function.The soft tree is modeled as a hierarchical mixture of experts and trained with gradient descent.Source: §II · p. 2
• Prior healthcare pipelines distill DNNs into gradient boosting trees using soft predictions, whereas this work trains a decision tree by matching logits.The authors also note that gradient boosting trees rely on post-hoc partial-dependence determinations.Source: §II · p. 2
• Another DNN-to-tree approach compares GA2Ms and gradient boosting trees but is limited to binary classification, unlike this paper’s multi-class aim.The authors describe that work’s results as inconclusive and unpublished.Source: §II · p. 2
• Tree regularization favors neural networks whose decision boundaries can be approximated by small trees, while this paper uses post-hoc interpretation.The cited work measures human simulatability by average decision-path length.Source: §II · p. 2
• A separate approach combines distillation with dimension reduction for visualization and uses Naive Bayes as its student classifier.It visualizes data points assigned similar probability vectors by a deep classifier.Source: §II · p. 2

III. METHODOLOGY

• The methodology investigates distilling deep models into non-neural models, focusing on CNNs and decision trees.The paper proceeds to describe CNNs, decision trees, knowledge distillation, and its own method.Source: §III · p. 3

A. Convolutional Neural Networks

• CNNs process grid-structured data such as images through convolutional and pooling layers followed by fully connected layers, with nonlinear activations such as ReLU.The paper relates the architecture to a visual-cortex hierarchy.Source: A. Convolutional Neural Networks · p. 3
• Convolution applies shared filters and biases across an image’s width and height, while ReLU applies a nonsaturating nonlinearity to feature-map units.The paper reports that ReLU can speed gradient-descent training relative to saturating nonlinearities such as tanh.Source: A. Convolutional Neural Networks · p. 3
• Pooling downsamples feature maps by summarizing nearby outputs, reducing representation dimensions and helping make features invariant to small input shifts and distortions.Max pooling returns the maximum value in a rectangular region.Source: A. Convolutional Neural Networks · p. 3

B. CART for Regression

• CART supports numeric regression targets and greedily builds a tree through top-down recursive partitioning, making it suitable for the paper’s multi-output regression formulation.The paper focuses its CART description on regression tasks.Source: B. CART for Regression · p. 3
• Each CART node splits the training samples by an attribute and threshold, dividing the feature space into two child nodes.The split is represented by an attribute and a threshold at the node.Source: B. CART for Regression · p. 3
• For regression, CART measures node impurity with mean squared error over the target values in each child node.The child-node sample count is denoted by Mn.Source: B. CART for Regression · p. 3
• The algorithm selects split parameters by minimizing its objective and recursively splits until maximum depth, purity, or a minimum-sample stopping condition is reached.The stated stopping conditions also include a node containing one sample.Source: B. CART for Regression · p. 3

C. Matching Logits

• Soft targets reveal probabilities for all classes, whereas hard targets retain only the predicted label.The passage notes that soft targets can boost distillation results significantly.Source: C. Matching Logits · p. 4
• Matching logits uses scores before softmax because small probabilities can vanish to zero when cross-entropy is applied to student predictions.This avoids the information loss associated with passing logits into probability space.Source: C. Matching Logits · p. 4
• In the high-temperature limit, matching logits is a special case of soft-target distillation when student logits are separately zero-meaned for each training instance.The equivalence is established through gradient descent on the cross-entropy function.Source: C. Matching Logits · p. 4

D. Distilling CNN into Decision Trees

• The trained CNN produces logits Z from the original features X, which CART uses as its training targets alongside X.The framework passage describes the CNN architecture used for MNIST as two convolutional and two pooling layers followed by two fully connected layers.Source: D. Distilling CNN into Decision Trees · p. 4
• The multiclass task is reformulated as multi-output regression, with decision trees handling multiple target outputs simultaneously.The authors anticipate this algorithm-adaptation approach to outperform transforming targets into independent single-output problems because that transformation ignores target dependencies.Source: D. Distilling CNN into Decision Trees · p. 4
• A softmax layer converts CART's numerical test outputs into class probabilities for final predictions.The passage introduces the CART test result as k and the resulting class probabilities as the final output.Source: D. Distilling CNN into Decision Trees · pp. 4–5

IV. EXPERIMENTS

• The experiments use two datasets, with teacher models implemented on TensorFlow to support scalability to big datasets.Source: §IV · p. 5

A. Datasets

• MNIST contains handwritten digits from 0 to 9, represented as 28 × 28 grayscale images with 784 flattened features.Source: A. Datasets · p. 5
• Connect-4 records two players’ positions on a seven-column, six-row grid, and its classes indicate the outcome for the first player.Source: A. Datasets · p. 5
• The experiments select MNIST, a deep-learning benchmark, and Connect-4 from the UCI repository.Source: A. Datasets · p. 5

B. Experimental Setup

• The MNIST training model is a deep CNN with two convolutional layers followed by two fully connected layers, ending in a 10-node output layer whose nodes represent digits 0–9.Its pooling stages produce 32 feature maps of size 14 × 14 and 64 of size 7 × 7. The first fully connected layer has 1,024 hidden nodes, followed immediately by dropout at 0.5.Source: B. Experimental Setup · p. 5
• The Connect-4 teacher is an MLP with hidden-layer widths 256, 128, and 128, and a three-node output for win, loss, or draw.Dropout is set to 0.8 after each hidden layer, and the training loss includes an L2 penalty of 0.0001.Source: B. Experimental Setup · p. 5, Table III · p. 5
• The Connect-4 test sample contains 10,000 instances sampled to preserve the original win (65.83%), loss (24.62%), and draw (9.55%) distribution.The MLP also uses an L2 penalty of 0.0001 in its training loss.Source: B. Experimental Setup · p. 5

C. Experimental Results

• 1% to 5% accuracy improvement is reported for distillation, and students outperform vanilla trees at the same tree depth on MNIST and Connect-4.The reported comparisons are presented in the MNIST and Connect-4 test-accuracy tables.Source: Table IV · p. 6, Table V · p. 6, D. Discussion · pp. 6–7
• The authors limit tree depth to 10 to keep the resulting models interpretable to human readers.They state that trees deeper than 10 become extremely hard for human cognition to comprehend.Source: C. Experimental Results · p. 6

D. Discussion

• The distillation effect still relies largely on the student’s own generalization ability, and the authors report that students remain far from matching teacher results.They also report that training a good teacher helped boost distillation results.Source: D. Discussion · p. 6
• Without a specified maximum depth, MNIST student accuracy was 88.28% versus 87.4% for gini trees; Connect-4 student accuracy was 79.06% versus 77.57%.For Connect-4, the teacher accuracy was 83.22%; the authors say unconstrained vanilla trees can be much deeper and less interpretable than student models.Source: D. Discussion · p. 6

• The distilled student decision trees significantly improve accuracy over vanilla decision trees on MNIST and Connect-4.The approach uses matching logits to learn from the dark knowledge in DNN soft predictions.Source: §V · p. 7
• The authors identify alternative methods for solving multi-output regression as a way to better exploit knowledge distillation.The current approach uses algorithm adaptation.Source: §V · p. 7
• Future work could add a softmax temperature and train with both soft targets and true labels.Source: §V · p. 7
• The authors also propose designing new inherently interpretable models that could match non-interpretable models’ performance.Source: §V · p. 7