Detecting Statistical Interactions from Neural Network Weights

Michael Tsang; Dehua Cheng; Yan Liu

Source-linked AI summary

Detecting Statistical Interactions from Neural Network Weights

Michael Tsang, Dehua Cheng, Yan Liu

arXiv:1705.04977v4May 14, 2017stat.MLcs.LG

TL;DR

Neural networks can capture important statistical interactions but are difficult to interpret, creating a need for efficient interaction detection. The paper proposes NID, which decodes interactions from feedforward-network weights by exploiting nonlinear hidden-unit structure. NID detects general interactions without exponential candidate search and shows effectiveness on synthetic and real-world data, while higher-order interlinked interactions and correlated features remain limitations.

Problem
Neural networks are powerful but opaque, while their non-additive feature interactions matter for scientific discovery and critical predictions.
Method
NID detects variable-order, unrestricted-form interactions by interpreting feedforward neural-network weight matrices and common hidden-unit connections.
Results
NID demonstrates effective interaction detection on simulated and real-world datasets without searching an exponential interaction-candidate space.
Takeaways & Limitations
Learned neural-network weights can be decoded to identify general statistical interactions efficiently, including higher-order interactions.
Takeaways & Limitations
Higher-order NID can confuse interlinked low-order interactions with single higher-order interactions, and feature correlations can cause spurious or missed interactions.

Abstract

from arXiv · show

Interpreting neural networks is a crucial and challenging task in machine learning. In this paper, we develop a novel framework for detecting statistical interactions captured by a feedforward multilayer neural network by directly interpreting its learned weights. Depending on the desired interactions, our method can achieve significantly better or similar interaction detection performance compared to the state-of-the-art without searching an exponential solution space of possible interactions. We obtain this accuracy and efficiency by observing that interactions between input features are created by the non-additive effect of nonlinear activation functions, and that interacting paths are encoded in weight matrices. We demonstrate the performance of our method and the importance of discovered interactions via experimental results on both synthetic datasets and real-world application datasets.

1 INTRODUCTION

The paper addresses the need to interpret neural networks’ learned feature interactions, especially in high-stakes applications. It introduces NID, an efficient weight-based framework that detects interactions without exhaustively searching candidate combinations.

Motivation: Neural networks’ predictive power is accompanied by black-box behavior that complicates adoption in critical decision-making domains.The paper highlights healthcare and finance as settings where understanding predictions is paramount.
Prior work: Existing neural-network interpretation methods primarily explain individual feature importance, attention, or model-specific visualizations.These approaches include gradients, prediction decompositions, attention mechanisms, feature maps, and gates.
Motivation: Statistical interactions describe joint feature effects that are not additive and can support scientific discovery and hypothesis validation.Examples include identifying joint evidence for particles and understanding interactions in medical risk prediction.
Contribution: NID detects statistical interactions of any order or form by examining feedforward neural-network weight matrices.It avoids an exponential candidate search through a first-hidden-layer importance approximation and a 2D traversal of the input weight matrix.
Contribution: The framework provides theoretical justification for hidden-unit interactions and evaluates interaction rankings with a generalized additive model on simulated and real-world datasets.Experiments demonstrate NID’s effectiveness relative to state-of-the-art interaction-detection methods.

2 RELATED WORK AND NOTATIONS

Prior interaction-detection methods trade computational cost, prespecified interaction forms, or interpretability scope against flexibility. The paper positions NID as a fast alternative that extracts generalized non-additive interactions directly from neural-network weights.

2.1 INTERACTION DETECTION: Individual-testing methods become computationally expensive as interaction order increases, with four-way ANOVA and beyond rarely performed.Additive Groves avoids restricting interaction functional forms but still conducts individual interaction tests.
2.1 INTERACTION DETECTION: Lasso-based methods are fast but require prespecifying interaction terms, resulting in O(p^2) pairwise terms and O(2^p) higher-order terms.Their captured interaction forms are limited by the specified terms.
2.1 INTERACTION DETECTION: NID avoids individual tests, detects variable-order interactions without restricting functional form, and may reduce false discoveries from multiple testing.Its speed comes from not testing every interaction candidate separately.
2.2 INTERPRETABILITY: Unlike earlier interpretability methods, NID extracts generalized non-additive interactions between variables from neural-network weights.This extends interpretation beyond feature importance and attention-style explanations.
2.3 NOTATIONS: The notation defines an interaction as a feature subset I with |I| ≥2 and uses |W| for elementwise absolute weight values.Vectors, matrices, restricted feature subvectors, and neural-network layers are specified for the framework.
2.3 NOTATIONS: The network is represented as a directed acyclic graph whose vertices are input features and hidden units, with edges determined by nonzero weights.Figure 1 illustrates an interaction created when a hidden unit receives strongly weighted inputs from x1 and x3; outgoing paths also affect its strength.

3 FEATURE INTERACTIONS IN NEURAL NETWORKS

Statistical interactions arise from non-additive joint effects and are created at nonlinear hidden units. NID quantifies their strength using input weights and outgoing-path influence, focusing on the first hidden layer for efficient detection.

A statistical interaction exists when multiple features jointly influence the output non-additively.
Non-additive interactions can involve different orders, and higher-order interactions require their corresponding lower-order subsets.
Nonlinear hidden units create feature interactions, whose effects propagate through subsequent network layers to the output.
Interacting features must connect strongly to a common hidden unit, while non-overlapping paths combine without creating interactions.
Because nonzero weights alone imply too many candidates, NID characterizes relative interaction strength and quantifies interactions at the first hidden layer.
Hidden-unit influence incorporates outgoing paths through cumulative absolute-weight products, which upper-bound hidden-unit gradient magnitudes.

4 INTERACTION DETECTION

NID ranks interaction candidates from learned first-layer weights and selects a cutoff with an interaction-aware GAM. Its architecture separates main effects from interacting paths to reduce spurious interactions.

NID trains a regularized feedforward network, ranks interaction candidates from learned weights, and determines a top-K cutoff.
4.1 ARCHITECTURE: The MLP-M architecture adds univariate networks at the output, encouraging main effects to remain separate from interacting paths.
4.1 ARCHITECTURE: Sparsity regularization suppresses unimportant interacting paths and pushes main-effect modeling toward the univariate networks.
4.2 RANKING INTERACTIONS OF VARIABLE ORDER: The greedy ranking algorithm traverses the input-to-first-hidden-layer matrix and retains top-ranked candidates of every order without exhaustive enumeration.
4.2 RANKING INTERACTIONS OF VARIABLE ORDER: The greedy strategy can improve a higher-order interaction’s ranking over redundant subsets under a mild assumption.
4.3 CUTOFF ON INTERACTION RANKING: MLP-Cutoff gradually adds ranked interactions to a GAM until validation performance plateaus, reporting the retained interactions.
4.4 PAIRWISE INTERACTION DETECTION: A pairwise variant ranks all feature pairs by first-hidden-layer interaction strength for standard pairwise detection tasks.

5 EXPERIMENTS

Experiments on synthetic and real-world datasets evaluate NID’s interaction rankings, cutoff procedure, and comparison with baseline methods. NID identifies predictive interactions efficiently, while errors concentrate in challenging interlinked or correlated settings.

5.1 EXPERIMENTAL SETUP: NID uses the minimum generalized mean as its averaging function after comparing six candidates on ten synthetic functions over ten trials.The tested candidates were maximum, root mean square, arithmetic mean, geometric mean, harmonic mean, and minimum.
5.1 EXPERIMENTAL SETUP: Experiments cover synthetic functions with varying interaction order, strength, nonlinearity, and overlap, plus four real-world regression and classification datasets.The real-world datasets include common prediction tasks, scientific discovery, and very-high-order interaction detection.
5.2 PAIRWISE INTERACTION DETECTION: NID’s pairwise interaction strengths are evaluated against Two-Way ANOVA, HierLasso, RuleFit, and Additive Groves using AUC on synthetic functions.Ten trials of AG and NID were run per dataset, with the highest- and lowest-AUC trials removed; Table 2 reports the resulting AUCs.
5.2 PAIRWISE INTERACTION DETECTION: MLP-M generally achieves comparable or better AUC than MLP, except on F6, where spurious main effects interfere with the {8, 9, 10} interaction.MLP-M performs better on F2 and F4 but worse on F6 because the interaction can be approximated as independent parabolas.
5.2 PAIRWISE INTERACTION DETECTION: Heat maps usually place strong synthetic interaction strengths at ground-truth locations, while real-world maps reveal interpretable patterns such as the longitude–latitude interaction in California housing.F6 and F7 remain difficult cases, and high-strength interactions also appear in the bike sharing, Higgs boson, and letter datasets.
5.3 HIGHER-ORDER INTERACTION DETECTION: NID’s higher-order rankings find true synthetic interactions before cutoff, and MLP-Cutoff reaches MLP-M performance after adding selected interactions.The cutoff stops adding interactions once validation performance reaches or surpasses MLP-M; superset interactions can make subset interactions redundant.
5.3 HIGHER-ORDER INTERACTION DETECTION: A relatively small number of variable-order interactions is highly predictive, including a 16-way interaction in the letter dataset.For bike sharing and Higgs boson, relatively few superset interactions suffice because corresponding subset interactions become redundant.
5.4 LIMITATIONS: Higher-order detection struggles with interlinked pairwise variables and can produce spurious or missed interactions when features are correlated.The method may confuse interlinked pairwise interactions with single higher-order interactions, while feature correlations are also problematic for other interaction-detection methods.

6 CONCLUSION

The conclusion presents NID as a weight-interpretation framework that accurately detects general statistical interactions without exponential candidate search. It identifies common hidden units as the basis for decoding interactions and points to broader architectures as future work.

NID detects statistical interactions by interpreting the learned weights of a feedforward neural network.
The framework accurately detects general interaction types without searching an exponential solution space of interaction candidates.
Interactions between features must be modeled at common hidden units, which NID uses to decode network weights.
Future work will account for common units in intermediate hidden layers and extend weight interpretation to other deep neural architectures.

A PROOF AND DISCUSSION FOR PROPOSITION 2

The proposition links statistical interactions to shared hidden-unit ancestry in a graph defined by nonzero network weights. The discussion explains interaction strength for ReLU units while noting that common descendants can sometimes produce canceled interactions.

The network graph contains vertices for input features and hidden units, with edges corresponding to nonzero weight-matrix entries.
For any interaction, some hidden-unit vertex has all interacting input features among its input-layer ancestors.
The proof groups final-layer hidden units by the first feature missing from their input ancestry to derive a contradiction.
A common descendant usually creates an interaction, but later layers can negate earlier interactions in exact cancellation cases.
For the bivariate ReLU function max{α1x1 + α2x2, 0}, interaction strength is quantified by the cross-term coefficient of the best quadratic approximation.
The interaction coefficient |β5| scales with c^-1 when the approximation region expands to (−c, c) × (−c, c), c > 1.

C PROOF FOR LEMMA 3

The lemma establishes a Lipschitz-based bound for neural-network output sensitivity under a 1-Lipschitz activation function. Its proof handles ReLU by approximation with differentiable 1-Lipschitz functions and bounds derivatives through hidden layers.

Lemma 3 assumes that the activation function φ(·) is 1-Lipschitz to estimate the network output’s sensitivity.
For nondifferentiable activations such as ReLU, the proof uses differentiable 1-Lipschitz approximations converging to the original activation.
The proof differentiates the final output with respect to a unit at hidden layer ℓ.
The argument concludes by establishing an inequality through separate rewritings of its left- and right-hand sides.

D PROOF FOR THEOREM 4

Theorem 4 gives a ranking condition for higher-order interactions proposed by Algorithm 1 with μ(·)=min(·). Under its assumptions, either a subset interaction ranks below the full interaction or a subset is not proposed.

Theorem 4 considers a d-way interaction I, d ≥ 3, and its d subset interactions under Algorithm 1 with μ(·)=min(·).
If a hidden unit proposes a subset interaction, the theorem assumes it also proposes the full interaction at that unit.
Under the theorem’s weight condition, either a proposed subset ranks lower than I or at least one subset is not proposed.
The theorem’s proof proceeds by contradiction from the assumption that every subset is proposed and ranks at least as highly as I.
Figure 9 presents ROC curves for NID-MLP-M corresponding to Table 2.

F LARGE p EXPERIMENT

NID remains accurate in a large-pairwise-interaction setting, detecting 950 ground-truth interactions with an AUC of 0.984 from a deep MLP-M network.

F LARGE p EXPERIMENT: The ROC curve for the large-p experiment reports the method’s interaction-detection performance.Figure 10 is specifically the ROC curve for the large-p experiment.
F LARGE p EXPERIMENT: The experiment used p = 1000 features, n = 1e4 samples, and K = 5 rank-one components for the interaction-weight matrix.The feature matrix was standard normal, noise variance was 0.1, and the sparse vectors had 2% nonzero density.
F LARGE p EXPERIMENT: MLP-M used five hidden layers with sizes 500, 400, 300, 200, and 100 before NID extracted interactions.The network used the same hyperparameters as before but a larger main-network architecture.
F LARGE p EXPERIMENT: 0.984 pairwise interaction strength AUC was obtained on 950 ground-truth pairwise interactions.The AUC was measured in the same way as in Table 2, with the corresponding ROC curve shown in Figure 10.

G COMPARING REGULARIZATION METHODS

The study compares L1, L2, group lasso, and sparse group lasso regularization, emphasizing input-weight groups because first-layer connections are central to NID.

G COMPARING REGULARIZATION METHODS: NID performance was compared across L1, L2, group lasso, and sparse group lasso regularization.The comparison measured average pairwise interaction strength AUC over 10 trials for each synthetic test function.
G COMPARING REGULARIZATION METHODS: Group lasso was applied specifically to groups in the input weight matrix because first-hidden-layer connections are especially important for this approach.The regularization comparison therefore targets both ordinary weight sparsity and structured input-level sparsity.
G COMPARING REGULARIZATION METHODS: Networks using group or sparse group lasso on input weights applied L1 regularization to all remaining weights.Regularizer strengths were increased until validation performance worsened from underfitting, using dataset sizes of 1e5.
G COMPARING REGULARIZATION METHODS: Table 4 reports average AUC of NID’s pairwise interaction strengths for different regularizers on the synthetic-function test suite.The table summarizes performance across the regularization conditions rather than a single experiment.

H COMPARISONS WITH LOGISTIC REGRESSION BASELINES

NID is compared with logistic-regression interaction baselines and examined through synthetic and real-world visualizations. Under relaxed hierarchical conditions, NID has fair precision but generally low recall, while its visualized interactions capture nonlinear joint effects.

H COMPARISONS WITH LOGISTIC REGRESSION BASELINES: NID was compared with FHIM and Shooter on binary-class synthetic datasets for pairwise and higher-order multiplicative interactions.FHIM identifies pairwise multiplicative interactions, while Shooter uses tree-structured feature expansion for pairwise and higher-order interactions.
H COMPARISONS WITH LOGISTIC REGRESSION BASELINES: The comparison generated interaction orders through degree 5, because MLP-M validation performance degraded quickly beyond that dataset setting.The data used a 5% sparsity factor and no added noise.
H COMPARISONS WITH LOGISTIC REGRESSION BASELINES: Shooter performs near perfectly, whereas NID obtains fair precision but generally low recall under the relaxed hierarchical experimental setting.NID scores were averaged over 10 trials, while the Shooter implementation was deterministic.
H COMPARISONS WITH LOGISTIC REGRESSION BASELINES: NID’s hierarchical correspondence across interaction orders causes it to miss many true interactions when the generated interactions lack strict hierarchical correspondence.The identified interactions across levels were always subsets or supersets of another predicted interaction.
J INTERACTION VISUALIZATION: For F6, MLP-M represented the {8,9,10} interaction as spurious main effects with parabolas in its univariate networks.This behavior supported the paper’s hypothesis about how the network models that interaction under the experimental conditions.
J INTERACTION VISUALIZATION: The California housing longitude-latitude visualization requires joint information from both features rather than an additive representation.Its highly interacting structure corresponds to the high rank of the {1, 2} interaction in the NID experiments.