Source-linked AI summary
Beyond Sparsity: Tree Regularization of Deep Models for Interpretability
Mike Wu, Michael C. Hughes, Sonali Parbhoo, Maurizio Zazzi, Volker Roth, Finale Doshi-Velez
TL;DR
Deep models are difficult to interpret, especially when users need to simulate predictions in decision-critical settings. The paper trains differentiable models with tree regularization so their predictions are approximated by small decision trees, finding accurate models in simpler, approximately human-simulatable regimes across speech and medical tasks.
Problem
Deep models’ predictions are difficult to interpret, motivating models that preserve accuracy while enabling human users to simulate their calculations.
Method
Tree regularization penalizes models whose thresholded predictions require long average decision paths in an approximating binary decision tree.
Results
Across HIV treatment, sepsis treatment, and speech processing, tree-regularized models improve prediction accuracy in the regime of simpler, approximately human-simulatable models.
Takeaways & Limitations
Tree regularization offers a way to move beyond sparsity toward deep models that humans can more easily simulate and trust.
Takeaways & Limitations
The approach uses a nondifferentiable tree-complexity objective, and future work must address settings with inputs that are not inherently interpretable.
Abstract
from arXiv · showhide
The lack of interpretability remains a key barrier to the adoption of deep models in many applications. In this work, we explicitly regularize deep models so human users might step through the process behind their predictions in little time. Specifically, we train deep time-series models so their class-probability predictions have high accuracy while being closely modeled by decision trees with few nodes. Using intuitive toy examples as well as medical tasks for treating sepsis and HIV, we demonstrate that this new tree regularization yields models that are easier for humans to simulate than simpler L1 or L2 penalties without sacrificing predictive power.
1 Introduction
The paper targets human-simulability: enabling users to step through predictions while retaining deep-model accuracy. It introduces tree regularization to favor models approximated by small decision trees, including time-series models and hybrid GRU-HMMs.
- Motivation: Human-simulability requires users to step through every calculation behind a prediction in reasonable time.Small decision trees are presented as easy to simulate, whereas even modest deep models can have too many parameters and connections.
- Motivation: Clinical and other decision-critical domains need simulatability for auditing perturbed inputs, checking substeps, and identifying systemic bias.
- Related work: Existing interpretation methods mainly explain already-trained models rather than optimizing the models themselves for interpretability.
- Contributions: Tree regularization favors decision boundaries that can be approximated by small decision trees, penalizing models requiring many calculations to simulate.
- Contributions: Tree-regularized GRUs reach a high-accuracy, low-complexity sweet spot unavailable to L1 or L2 regularization at any strength.Hybrid GRU-HMMs further improve prediction by modeling residuals of interpretable discrete HMMs.
2 Background and Notation
The paper formulates supervised binary prediction for fixed examples and variable-length sequences, using neural networks and GRUs to produce timestep-level predictions. Training combines logistic loss with a regularization term, while GRUs use gated state transitions and sigmoid output probabilities.
- Notation: Each supervised example has input features and binary target outputs; time-series examples contain variable-length timestep sequences.
- Simple neural networks: An MLP maps inputs and parameters to predictions, and training minimizes an objective over the labeled dataset.
- Simple neural networks: For binary targets, logistic loss is combined with a regularization term that may be L1, L2, or the proposed regularization.
- Recurrent neural networks: An RNN transforms each input timestep and previous hidden state into a hidden-state sequence used to predict timestep targets.
- Gated recurrent units: GRUs use reset gates to forget past states and update gates to retain the previous state or use a candidate state.
- Gated recurrent units: The binary-label probability at timestep t is obtained by applying a sigmoid transformation to the state, with output weights w as layer parameters.
3 Tree Regularization for Deep Models
Tree regularization measures neural-model complexity through the average path length of a decision tree reproducing the model’s binary predictions. Because this cost is nondifferentiable, a differentiable surrogate estimates it during training, with augmentation and recent-epoch sampling addressing limited and shifting training examples.
- Tree regularization: Tree regularization fits a binary decision tree to the network’s thresholded predictions and penalizes its average decision-path length on a reference dataset.Average path length counts the boolean decision calculations needed for an average prediction, linking the penalty to human simulation.
- Tree regularization: The average-path-length cost uses TRAINTREE to reproduce network predictions and PATHLENGTH to count the nodes required for each input’s output.The cost is defined algorithmically as the average path length over the reference examples.
- Differentiable surrogate: The exact tree-based cost is nondifferentiable with respect to neural parameters, unlike standard L1 and L2 norms.This prevents direct use of ordinary gradient descent on the exact cost.
- Differentiable surrogate: A standalone MLP surrogate maps candidate parameter vectors W to estimated average path lengths and is trained by minimizing squared error.The surrogate provides a differentiable approximation that can be optimized alongside the target network.
- Surrogate training: Early surrogate training has few labeled parameter examples, so the method augments them with randomly sampled weights and unregularized-training restarts.Each target-network iteration contributes only one parameter-vector example, creating the initial data shortage.
- Surrogate training: Using only examples from the past E epochs keeps surrogate training relevant as model parameters move away from their initial values.The fixed epoch window also speeds training, with E chosen empirically.
4 Tree-Regularized MLPs: A Demonstration
A deliberately overexpressive MLP on a noisy 2D classification task tests tree regularization against L1 and L2 penalties. The results show that tree regularization can combine low path-based complexity with strong prediction performance, while the baselines cannot do so across regularization strengths.
- Experimental setup: The 2D Parabola experiment uses an overexpressive three-layer MLP, noisy boundary labels, and L1 and L2 weight penalties as baselines.The dataset contains 500 uniformly sampled points, with 10% boundary-region labels flipped and 30% held out for testing.
- Results: The fitness plots place each trained model by average-path-length complexity on the x-axis and AUC prediction performance on the y-axis.Models are trained across many regularization strengths to compare the resulting complexity–performance trade-off.
- Results: Simple L1 or L2 regularization produces no models with both small node count and good predictions at any tested regularization strength.Large L1 or L2 strengths yield overly simple linear boundaries with poor accuracy.
- Results: Tree regularization directly produces simple tree-like boundaries at high regularization strengths while retaining good predictions.Decision-tree preferences for axis-aligned splits make these functions require few nodes.
- Results: Tree-regularized axis-aligned functions use very few nodes and are more effective than the L1 and L2 counterparts in this demonstration.The L1 boundary is sharper, whereas the L2 boundary is rounder.
5 Tree-Regularized Time-Series Models
Tree regularization trains time-series deep models whose predictions are accurate yet closely approximated by small decision trees. Across synthetic and medical tasks, it improves the accuracy–complexity tradeoff, supports clinically interpretable proxies, and can be combined with HMMs for further gains.
- Approach: Tree regularization applies a decision-tree path-length penalty to deep models while allowing their decision-boundary complexity to be measured under any regularization.Algorithm 1 computes average decision-tree path length from a fixed model’s parameters, enabling comparison with L1 and L2.
- Results: Across tasks, tree regularization achieves higher AUC at low path lengths than standalone trees and L1- or L2-regularized deep models.In the signal-and-noise task, tree regularization reaches AUC near 0.9 at average path length 10, whereas L1 and L2 require path lengths above 25.
- Results: HIV adherence shows AUC gains of 0.03–0.05 at path lengths 19–25, while all methods perform poorly at smaller path lengths.The passage identifies the task as difficult and connects improved adherence prediction with subsequent HIV therapy decisions.
- Interpretability: Decision-tree proxies are small enough for manual simulation, with path length ≤25, and their synthetic-task logic matches relevant parts of the true labeling process.The simplest synthetic tree checks whether the first state and first output dimension are active.
- Interpretability: Sepsis proxies exposed clinically sensible breathing-related features for ventilation and counter-intuitive mortality patterns that prompted hypotheses about uncaptured variables.An ICU clinician found the trees useful for judging whether the deep model could be trusted; such reasoning was not possible from simple sensitivity analyses.
- Faithfulness: Tree proxies agree with their corresponding deep time-series models on about 85–90% of test examples, supporting trust in their simulated paths for most cases.Fidelity is defined as the percentage of test examples where tree and deep-model predictions agree.
- Runtime: Tree regularization costs less than twice the per-epoch runtime of L2 in the reported TIMIT comparison: 3977 seconds versus 2116 seconds.The authors expect parallelizing the smaller surrogate model could reduce per-epoch time further.
- Hybrid models: A residual GRU-HMM improves HMM predictions at small path lengths; on mechanical ventilation, it reaches AUC 0.88 at path length 28 versus 60 for a same-state GRU.The authors suggest jointly trained deep residual models may provide better interpretability.
6 Discussion and Conclusion
The paper introduces tree regularization to make complex differentiable models more human-simulatable, and reports accuracy gains for simpler models across three real-world domains. It identifies broader applicability and limitations for future work.
- Tree regularization encourages complex decision boundaries to be approximated by human-simulatable functions.This is intended to help domain experts understand and approximately compute complex model behavior.
- Future work should improve learned-model stability and address settings where inputs are not inherently interpretable, such as image pixels.
- Across HIV treatment, sepsis treatment, and human speech processing, tree-regularized models improve prediction accuracy in the simpler-model regime.
A Details for Decision-Tree Training
The appendix describes how decision trees and differentiable surrogate models are trained to estimate and regularize deep-model complexity. Post-pruning simplifies trees, while augmentation, retraining, and windowing improve surrogate tracking.
- Training decision trees with post-pruning: Decision trees are trained with scikit-learn using information gain and Gini impurity, then post-pruned on validation data.
- Training decision trees with post-pruning: Each leaf is constrained to contain at least five training examples, and leaf-size hyperparameters affect tree usefulness.
- Training decision trees with post-pruning: The pruning heuristic removes leaf nodes when validation squared error improves, eliminating subtrees that fail to generalize.
- Surrogate training: Surrogate predictors track true average path length during target-model training, but excessive capacity can overfit and degrade performance.An MLP with 25 hidden nodes plus weight augmentation, random restarts, fixed windows, and retraining improves prediction accuracy.
- Surrogate training: Weight augmentation expands sparse training data by sampling weight vectors, while retraining on recent epochs keeps the surrogate aligned with changing parameters.
B Experimental Protocol
The experiments evaluate GRU, HMM, and GRU-HMM models across synthetic datasets and specified training configurations. Protocol details define surrogate construction, data generation, model sizes, and preprocessing.
- Model and surrogate protocol: Decision trees use model inputs and output predictions, while GRU-HMM trees use only the GRU output node; surrogates are one-hidden-layer MLPs with 25 nodes.
- Dataset and training protocol: The appendix records dataset summaries and training parameters in Table B.1.
- Synthetic datasets: The synthetic parabola dataset samples 200 points uniformly in the unit square and flips 10% of points near the decision boundary.
- Signal-and-noise HMM: The signal-and-noise HMM outputs an emitted-state one-hot vector concatenated with seven binary input features, requiring state 1 and feature 1 to output 1.
- Training configurations: Synthetic experiments vary GRU nodes, HMM states, and GRU-HMM configurations across the listed model-size grids.
C Extended Results
Extended experiments provide additional visualizations and test AUC tables across synthetic and real datasets. Larger models with regularization generally perform best, while Figure C.1 displays performance–complexity trade-offs for GRU variants.
- Extended visualizations: Supplementary materials expand fitness traces and tree visualizations for signal-to-noise HMM, Sepsis, and TIMIT, and add output dimensions for Sepsis and HIV.
- Extended quantitative results: Test AUC tables span synthetic and real datasets across GRU node counts, HMM state counts, and regularization strengths.
- Extended quantitative results: Larger models with regularization tended to perform best in the reported experiments.
- Performance–complexity trade-offs: Figure C.1 compares performance and complexity trade-offs for L1, L2, and Tree regularization on GRU and GRU-HMM models.The caption cautions that the plotted scales differ.
C.2 Signal-and-noise HMM: Tree Visualization
The figures examine how tree regularization changes decision-tree complexity and performance across synthetic, medical, and speech tasks. Increasing regularization can collapse trees toward simpler structures, while the figures compare this trade-off with L1 and L2 regularization.
- Signal-and-noise HMM: Increasing tree regularization on the signal-and-noise HMM dataset collapses the surrogate tree to a single node at high strengths.At one setting, the tree closely resembles the ground-truth data-generating function.
- Sepsis: The Sepsis figures compare performance and complexity trade-offs among L1, L2, and Tree regularization for GRU models.The evaluation includes multi-dimensional classification across five output dimensions.
- HIV: The HIV figures compare L1, L2, and Tree regularization for GRU models, with five outputs trained jointly.Table C.3 reports performance metrics for multi-dimensional classification on held-out HIV data.
- TIMIT: The TIMIT evaluation compares performance and complexity under L1, L2, and Tree regularization and includes example decision trees.Table C.4 reports performance metrics across models on held-out TIMIT data.
D GRU-HMM: Deep Residual Timeseries Model
GRU-HMM models use an HMM to represent probabilistic structure and a GRU to model residual target-prediction errors. Tree regularization constrains the GRU component, producing roughly one-percent performance gains over vanilla equivalents while preserving low-complexity trade-offs.
- HMM foundation: An HMM models latent-state sequences that jointly generate observed data and binary outcomes through transition and emission parameters.The latent states form a Markov chain over K discrete states.
- GRU-HMM design: GRU-HMM models use a GRU to capture residual errors from HMM belief-state predictions and tree-regularize the GRU outputs.The HMM component remains unconstrained because surrogate trees are trained only on the GRU output node.
- Empirical results: Approximately 1% better performance is reported for deep residual models than for vanilla equivalents across synthetic and real-world datasets, with roughly the same parameter count.The comparison covers the same suite of datasets used for the vanilla GRU and HMM models.
- Regularization trade-offs: Tree regularization dominates L1 and L2 in low-node-count regions, although the effect is less pronounced for GRU-HMM models than for GRU models.The comparisons use increasing λ values and performance-complexity trade-off curves.
- Training cost: Training GRU-TREE and GRU-HMM-TREE includes surrogate-training costs, but sparse retraining can amortize those costs so epoch times are approximately unchanged.Table E.1 measures epoch time as a single pass through all training data.
F Extended Stability Tests
The stability tests evaluate how consistently tree regularization produces similar surrogate trees across independent runs. Strong regularization yields reproducible structures, whereas weak regularization produces high variability in tree size and shape.
- Strong regularization: Seven of ten independent runs at λ = 1000.0 produced the same tree structure, while the remaining trees were similar and shared splits and features.The runs used random initializations on the signal-and-noise HMM dataset.
- Generalization of the stability pattern: Similar stability patterns were reported for real-world datasets beyond the signal-and-noise HMM experiment.The paper connects the synthetic stability test to broader dataset behavior without giving additional quantitative results.
- Weak regularization: At λ = 0.01, independently trained trees showed high variance in size and shape.The low-regularization examples are drawn from ten runs on the signal-and-noise HMM dataset.