PaperlayerView exact revision on arXiv

Source-linked AI summary

REALM: Regime-Switching, Explainable, and Activation-Induced Linear Models

Xiaoran Cheng, Sen Na, Jia Li

arXiv:2609.32141v1cs.LGcs.AIstat.MEstat.ML

TL;DR

Deep ReLU activation patterns can create too many highly localized regimes for interpretable local explanations. REALM clusters activations from a distilled student network, fits local linear experts, and explains assignments with a separate gate; experiments report competitive accuracy, while analysis characterizes a complexity–stability trade-off.

  • Problem

    Deep ReLU activation cells can grow rapidly and become nearly instance-specific, making their explanations highly localized and difficult to compare across instances.

    Source: §I · p. 1

  • Method

    REALM clusters binarized activations from a distilled wide, shallow student into regimes, fits local linear experts, and trains an explanatory gate to predict regime assignments.

    Source: Overview of REALM · p. 2, §VI · p. 12

  • Results

    REALM achieves competitive predictive accuracy against prior DNN-guided mixture surrogates and inherently interpretable models, while ablations show that local experts recover performance that within-regime constants cannot.

    Source: C. Ablation Studies · p. 10, §VI · p. 13

  • Takeaways & Limitations

    An idealized analysis characterizes a complexity–stability trade-off in the number of regimes: finer partitions may improve approximation while reducing regime-assignment stability.

    Source: §VI · p. 12

Abstract

from arXiv · show

Deep ReLU networks are piecewise-affine mappings that partition the input space into cells, each characterized by a distinct activation pattern. This structure motivates fitting a local linear model within each cell to preserve predictive accuracy while improving interpretability. The challenge is to identify regimes that are stable, data-adaptive, and easy to explain. We propose REALM, a mixture of linear models whose regimes are induced by neural activation patterns. Because the number of activation cells in a deep neural network (DNN) can grow rapidly with depth, we first distill a deep teacher into a wide, shallow student network (WSSN), then binarize and cluster its hidden-layer activations to define the regimes and fit a linear model within each regime. Since the regimes are discovered from internal structure, the router does not carry the predictive burden. To make regime assignment interpretable, we train a multiclass logistic regression, the explanatory gate, to reproduce the regime assignments. The two-level structure is interpretable at both stages in terms of raw tabular or learned convolutional features: the gate identifies features that determine regime assignments, while the linear models identify features that drive predictions within each regime. We analyze an idealized setting that illustrates a trade-off between partition complexity and stability: as the number of regimes grows, finer partitions can improve approximation but may reduce regime-assignment stability. Experiments on tabular and image datasets show that REALM achieves competitive predictive performance relative to other DNN-guided mixture surrogates and inherently interpretable models while producing stable regime-level explanations.

I. INTRODUCTION

Interpretable surrogates must balance faithful approximation of nonlinear DNN decisions with readable regime assignments. REALM uses a distilled ReLU student’s activation geometry to form shared regimes, then explains both regime assignment and within-regime predictions.

I. INTRODUCTION

  • A single global interpretable surrogate can struggle to capture a highly nonlinear DNN without sacrificing predictive accuracy, while jointly trained routers can absorb complex predictive logic.This can make it difficult to disentangle whether predictions come from the router, experts, or their interaction.Source: §I · p. 1

II. PRELIMINARIES: PIECEWISE-AFFINE STRUCTURE OF RELU DNNS

  • ReLU activation patterns define affine cells, but exact partitions can grow rapidly and become too instance-specific for comparable explanations, motivating shared regimes that merge cells.REALM’s cells are unions of activation cells on which the student remains approximately affine.Source: §I · pp. 1–2, §II · p. 3

I. INTRODUCTION

  • Unlike MLM and SEE-Net, which cluster continuous DNN representations layer by layer, REALM clusters binarized WSSN activation patterns into regimes that are unions of activation cells.The cited comparison also notes that SEE-Net transforms the DNN into a pre-interpretable architecture, which can add training cost.Source: §I · p. 2

Overview of REALM

  • REALM achieves competitive performance against SEE-Net across six datasets and against MLM and inherently interpretable models on tabular data, with stable regime-specific explanations.The paper benchmarks SEE-Net on tabular and image tasks, and MLM and other interpretable models on tabular tasks.Source: Overview of REALM · p. 2, Main Contributions · p. 2
  • REALM distills a trained DNN into a WSSN, clusters its activation patterns into regimes, fits local linear experts, and trains a separate gate to explain assignments.The student’s single hidden layer induces fewer activation cells than the deep teacher, and its lower-dimensional patterns stabilize distance-based clustering.Source: Main Contributions · p. 2, Overview of REALM · p. 2
  • REALM provides dual interpretability: local feature attributions explain predictions within regimes, while the explanatory gate attributes regime assignments to raw tabular or learned convolutional features.The idealized analysis identifies a complexity–stability trade-off: finer partitions can improve approximation but may reduce assignment stability.Source: Main Contributions · p. 2

III. REALM: REGIME-SWITCHING, EXPLAINABLE, AND ACTIVATION-INDUCED LINEAR MODELS

REALM first distills the teacher into a wide, shallow ReLU student, then clusters the student’s activation patterns and fits local linear experts. A separate logistic-regression gate explains regime assignments; for image data, these models use a fixed learned feature representation.

Stage 1: Teacher-to-Student Distillation

  • The three-stage pipeline distills a teacher into a WSSN, clusters its activation patterns into regimes, and fits local experts with a separate explanatory gate.The gate explains assignments; in the main formulation it does not determine routing.Source: §III · p. 3, Figure 1 · p. 4, Stage 3: Fixed-Router Mixture of… · p. 5
  • For image data, regime discovery, experts, and the explanatory gate operate on a fixed learned convolutional representation rather than raw pixels.The feature extractor is trained jointly with the WSSN during distillation and frozen afterwards.Source: Stage 1: Teacher-to-Student Distillation · p. 3

Stage 2: Soft Clustering for Latent Regime Discovery

  • After distillation, REALM freezes the WSSN and softly clusters its activation patterns, yielding a posterior that merges activation cells into approximately affine regimes.An inverse temperature sharpens assignments so predictions rely on fewer experts; the clustering summarizes the student’s organization rather than absorbing predictive burden into the router.Source: Stage 2: Soft Clustering for Latent… · pp. 3–4

Stage 3: Fixed-Router Mixture of Regularized Local Experts

  • With the clustering posterior fixed, REALM mixes regularized local linear experts and trains a separate multiclass logistic regression to predict hard regime labels.Expert coefficients explain predictions, while gate coefficients explain regime assignment; gate coefficients are interpreted as contrasts between regimes.Source: Stage 3: Fixed-Router Mixture of… · pp. 4–5
  • For tabular regimes below a minimum size, REALM adds teacher-labeled synthetic samples generated around continuous-variable regime statistics and using valid categorical encodings.Image datasets are not augmented because every regime exceeds the minimum size threshold.Source: Stage 3: Fixed-Router Mixture of… · p. 5
  • REALM-EG uses the explanatory gate as the router and performs competitively with REALM, with both assignment rules and within-regime predictions readable from linear coefficients.Its experts are refitted on samples assigned by the gate, and the gate posterior weights the experts during training and testing.Source: Stage 3: Fixed-Router Mixture of… · p. 5

IV. THEORETICAL ANALYSIS OF STRUCTURAL STABILITY AND SUPPORT RECOVERY

The theoretical analysis studies joint regime-assignment reliability and sparse support recovery under an idealized piecewise sparse linear model. Its assumptions include known regimes, exact within-regime linearity, sparse true coefficients, and additive Gaussian noise.

IV. THEORETICAL ANALYSIS OF STRUCTURAL STABILITY AND SUPPORT RECOVERY

  • The analysis assumes latent piecewise sparse linear data with known regimes, exact within-regime linearity, sparse true coefficients, a learned test-time selector, and additive Gaussian noise.These assumptions idealize REALM, which discovers regimes through activation-pattern clustering and is only approximately linear within each regime; fitted experts need not be sparse.Source: §IV · p. 5

A. Model Setup

The model setup treats the input as a signal in R^p and assumes data arise from a latent piecewise sparse linear model. Regime assignment uses a linear winner-take-all selector, with a margin measuring the winning regime’s advantage over its closest competitor.

A. Model Setup

  • The setup assumes a latent piecewise sparse linear model and a linear regime selector; its pairwise margin measures the winning regime’s score advantage over its closest competitor.The analysis considers stability at inputs with positive margin.Source: A. Model Setup · pp. 5–6

B. Stability of Regime Assignment

The section bounds how additive perturbations can change a regime assignment under a linear winner-take-all selector. Stability improves with larger margins and is weakened by noise and regime complexity.

B. Stability of Regime Assignment

  • A positive assignment margin lets the linear selector retain its regime under perturbations satisfying the stated deterministic condition.The selector is evaluated at a fixed input signal.Source: B. Stability of Regime Assignment · p. 6
  • The probabilistic misassignment bound depends on the assignment margin, perturbation noise, and the number of regimes.A larger margin promotes stability, higher noise erodes it, and the regime-complexity cost grows slowly with K.Source: B. Stability of Regime Assignment · p. 6
  • The probabilistic guarantee is nontrivial only when the margin exceeds an upper bound on expected maximum Gaussian fluctuation.Otherwise, the bound is vacuous.Source: B. Stability of Regime Assignment · p. 6

C. Joint Regime and Support Recovery with Learning Error

This section combines regime-selection error, perturbation-driven switching, and within-regime sparse support recovery in an end-to-end reliability bound. The result assumes oracle-trained experts and standard conditions for LASSO support recovery.

C. Joint Regime and Support Recovery with Learning Error

  • The support-recovery analysis conditions on training inputs and uses oracle-trained experts, isolating regime uncertainty rather than assignment errors’ effects on expert training.The oracle index sets are unavailable to the practical algorithm because the true selector is unknown.Source: C. Joint Regime and Support Recovery… · p. 6
  • Under standard within-regime design and signal conditions, LASSO recovers the true regime’s support with probability at least 1−c1 exp(−c2n_k*λ^2/σ^2).The guarantee applies under the stated penalty choice and for given training inputs.Source: C. Joint Regime and Support Recovery… · pp. 6–7
  • The end-to-end support-recovery guarantee combines learned-selector error, perturbation-driven switching, and within-regime LASSO failure.The learned-selector error is represented by η_n and added to the other error terms; the guarantee concerns oracle-trained experts.Source: C. Joint Regime and Support Recovery… · p. 7

D. Design Implications and Structural Trade-Offs

The section treats K as a design choice balancing approximation capacity against stability and support-recovery reliability. Larger K can refine partitions, but may weaken guarantees through switching instability and smaller per-regime samples.

D. Design Implications and Structural Trade-Offs

  • With margins, noise, and gating energy fixed, increasing K raises the noise-complexity radius at rate √logK and weakens the stability guarantee.Changing K in REALM may also alter margins and improve approximation, effects not captured by this fixed-regime analysis.Source: D. Design Implications and Structural… · p. 7
  • With balanced regimes, larger K raises the signal strength required by the support-recovery guarantee.The per-regime sample size decreases as K grows.Source: D. Design Implications and Structural… · p. 7
  • K trades representational capacity against statistical reliability: finer partitions can reduce approximation bias but may increase switching instability and estimation variability.Larger K also shrinks per-regime sample size and can make support-recovery conditions harder to satisfy.Source: D. Design Implications and Structural… · p. 7
  • Reliable regime learning and adequate margins motivate a moderate number of well-separated regimes to balance approximation accuracy and explanation reliability.The section examines this qualitative trade-off in an ablation on K.Source: D. Design Implications and Structural… · p. 7

V. EXPERIMENTS

The experiments evaluate REALM on synthetic data with known regimes and on six tabular and image datasets. They examine regime recovery, predictive performance, routing, and the interpretability of assignments and local prediction rules.

V. EXPERIMENTS

  • The evaluation spans three tabular and three image datasets, comparing REALM with DNN-guided mixture surrogates and other models; the main experiments route using WSSN-induced posteriors.The explanatory gate is evaluated as a routing variant separately from the main experiments.Source: §V · p. 7

A. Synthetic Data Study

  • REALM’s predicted regimes align most closely with the known synthetic geometry, compared with MLM and SEE-Net.With five regimes, REALM follows the trapezoidal-bowl boundaries more closely than the alternatives.Source: A. Synthetic Data Study · pp. 7–8, Figure 2 · p. 8

B. Real-World Data Study

  • REALM is the strongest interpretable model on all three tabular datasets, while strong black-box models such as MLP and XGBoost perform better.Near-one-hot routing lets most predictions be explained primarily by a single linear expert.Source: B. Real-World Data Study · p. 8, Table I · p. 9
  • REALM improves over SEE-Net (Intp) on all three image datasets and nearly matches the teacher CNN on MNIST.On Flowers, the distilled WSSN achieves a higher AUC than the teacher, and REALM retains part of that gain.Source: B. Real-World Data Study · p. 8, Table II · p. 9
  • The linear explanatory gate recovers assignments almost perfectly on Bike Sharing, Covertype, MNIST, and CIFAR-10, while Flowers’ Gate AUC is 0.974.California Housing is harder to explain with a linear assignment rule and uses the largest partition, K = 150.Source: B. Real-World Data Study · p. 8
  • Routing is sharp and regime sizes are balanced on MNIST; routing is less sharp on CIFAR-10 and Flowers, whose regime sizes remain relatively balanced.The routing and regime-size distributions are summarized over five independent runs.Source: B. Real-World Data Study · p. 8, Figure 3 · p. 9
  • The image analyses show that gate coefficients distinguish regime assignments, while local feature-map contributions can recur across images and concentrate in a few dominant filter groups.For example, the gate assigns opposite-signed coefficients to some pooled features across Flowers regimes, and filter groups recur across cars, frogs, and other images.Source: B. Real-World Data Study · pp. 8–10, Figure 4 · p. 10
  • Across MNIST, CIFAR-10, and Flowers, the learned image regimes organize examples by human-readable features such as writing style, pose, framing, and color.Flowers shows less pronounced visual differences between regimes than MNIST and CIFAR-10.Source: B. Real-World Data Study · pp. 8–9

C. Ablation Studies

REALM remains competitive across tabular and image datasets, and its ablations show that local experts—not regime assignments alone—supply important predictive power. The results also examine explanatory-gate routing and how the number of regimes affects performance.

C. Ablation Studies

  • Replacing local experts with majority votes reduces performance on all six datasets and increases cross-run test-metric variability on four, showing that regimes alone are insufficient for strong prediction.The findings indicate that linear experts recover predictive performance that within-regime constants cannot.Source: C. Ablation Studies · p. 10, Figure 7 · p. 12
  • REALM-EG performs comparably to REALM: its largest degradation is 1.6% on Flowers, it slightly improves on Bike Sharing and California Housing, and differences on three other datasets are negligible.REALM-EG uses the explanatory gate as the router and refits experts under its posterior.Source: C. Ablation Studies · p. 10, Figure 8 · p. 12
  • Performance improves monotonically with K on Covertype and California Housing, while Bike Sharing RMSE is lowest at K = 10 (mean 134.707, std 14.026) and rises thereafter.On the image datasets, CIFAR-10 AUC declines with K, especially beyond 50, Flowers has the largest decrease, and MNIST remains near-perfect.Source: C. Ablation Studies · pp. 10–12, Figure 9 · p. 13
  • For California Housing, the study chose K = 150 for better interpretability and likewise chose K = 30 for Covertype; Bike Sharing shows the ratio alone does not identify the best K.For Bike Sharing (n = 11,122, p = 19), RMSE is minimized at K = 10 (mean 134.707, std 14.026), where the ratio is 19.9, and increases thereafter; the ratio decreases to approximately 1.0 at K = 200.Source: C. Ablation Studies · p. 11

VI. CONCLUSION

  • The idealized analysis characterizes a trade-off in K: finer regimes can improve approximation while regime-assignment stability may decline.The conclusion notes that this complexity–stability trade-off is consistent with the ablation performance curves.Source: §VI · pp. 12–13
  • REALM attains competitive accuracy against SEE-Net across six datasets and against MLM and inherently interpretable models on tabular data, with stable regime-specific explanations.Ablations also show that the explanatory gate can route with little accuracy loss.Source: §VI · p. 13

I. DATASETS

The supplementary material describes six datasets, REALM’s training pipeline, and additional interpretability analyses. It also states assumptions and support-recovery results for an idealized within-regime regression setting.

I. DATASETS

  • The evaluation uses three tabular datasets—Bike Sharing, California Housing, and Covertype—and three image datasets—MNIST, Flowers, and CIFAR-10.The datasets cover rental demand, housing values, forest cover, handwritten digits, flowers, and object classes.Source: §I · p. 14

II. IMPLEMENTATION DETAILS

  • Training distills a DNN teacher into a WSSN, freezes the student while optimizing regime centroids, then fits local experts and the explanatory gate with routing posteriors fixed.Tabular inputs use a one-hidden-layer WSSN on original features; image models use dataset-specific convolutional feature extractors.Source: §II · p. 14

III. ADDITIONAL INTERPRETABILITY ANALYSES

  • Routing is sharp across the three tabular datasets and no regime dominates, while California Housing has many small regimes and a long right tail in regime size.These routing and regime-size patterns are shown in Fig. 10.Source: §III · p. 14, Figure 10 · p. 15
  • Local expert coefficients can change sign across regimes, while the explanatory gate shows greater feature sparsity on Bike Sharing and California Housing than on Covertype.In the largest regime, the gate retains an average of 54/54 Covertype features, 11.9/19 Bike Sharing features, and 1.4/8 California Housing features.Source: §III · p. 15, Table IV · p. 15, Figure 11 · p. 16
  • Image contribution maps associate MNIST channels with digit strokes and bends, while Flowers maps highlight dandelion seed-head structure and daisy petals and discs.For MNIST, Top-4 and all-channel sums coincide because the feature extractor has only four channels.Source: §III · pp. 15–16, Figure 12 · p. 17

IV. ASSUMPTIONS AND AUXILIARY RESULTS FOR SECTION IV-C OF THE MAIN PAPER

  • The support-recovery analysis assumes restricted eigenvalue, irrepresentable, and beta-min conditions, which respectively constrain design degeneracy, off-support dependence, and detectability of nonzero coefficients.The analysis conditions on training inputs.Source: §IV · p. 16
  • Under the stated assumptions and penalty choice, the LASSO support-recovery error in the true regime is bounded by an exponential term with exponent proportional to −n_k*λ²/σ².The bound is adapted from Wainwright and uses oracle training samples from the true regime.Source: §IV · pp. 16–17
  • The end-to-end support-recovery bound combines within-regime recovery with perturbation stability and the learned selector’s regime-identification accuracy.The selector assumption summarizes its generalization error through η_n, taken as given in this analysis.Source: §IV · p. 17

A. Proof of Proposition IV.2

The proof of Proposition IV.2 uses Gaussian concentration and a maximal inequality to establish perturbation bounds. The proof of Proposition IV.3 then separates support-recovery errors into selector mistakes, true regime switches, and LASSO failure in the correct regime.

A. Proof of Proposition IV.2

  • The proof bounds a perturbation-dependent linear, Lipschitz quantity with Gaussian concentration, handles negative thresholds by the trivial probability bound, and uses a maximal inequality for its expectation.Substituting these bounds yields the stated perturbation bound.Source: A. Proof of Proposition IV.2 · pp. 17–18

B. Proof of Proposition IV.3

  • The proof of Proposition IV.3 decomposes support mismatch into learned-selector error, a true regime switch under perturbation, or LASSO support-recovery failure in the true regime.The selector error is bounded by η_n, Proposition IV.2 bounds regime switching, and Theorem IV.1 bounds oracle LASSO failure.Source: B. Proof of Proposition IV.3 · p. 18

Related papers

Loading 2609.32141v1…