PaperlayerView exact revision on arXiv

Source-linked AI summary

Hide&Seek: Learning to Explain in an End-to-End Differentiable Network

Tal Ellinson, Hadi Mohasel Afshar, Sally Cripps

arXiv:2608.16689v1stat.MLcs.LG

TL;DR

Existing instance-wise feature-selection methods face information leakage and nondifferentiable discrete sampling. Hide&Seek jointly learns selection and prediction with differentiable feature replacement, and performs strongly across experiments while remaining fast to train.

  • Problem

    Existing instance-wise feature-selection methods face information leakage and nondifferentiable discrete sampling, limiting reliable and efficient joint selection and prediction.

  • Method

    Hide&Seek jointly trains a selector and predictor using continuous masks, distributional replacement values, and parsimony-weight annealing.

  • Results

    Across synthetic, semi-synthetic, image, and genetic-data experiments, Hide&Seek achieves strong instance-wise feature-selection performance and remains fast to train.

  • Takeaways & Limitations

    Distributional feature replacement preserves joint selector–predictor training benefits without introducing an artificial information channel across the evaluated settings.

  • Takeaways & Limitations

    Fixed-value ablation models can misidentify switch features as unimportant at rates reaching 50% under the analyzed ill-posed loss proxy.

Abstract

from arXiv · show

Instance-wise feature selection is a valuable tool for interpreting labeled data and the predictions of black-box models. In contrast to global feature selection techniques, instance-wise methods dynamically identify important features for each instance. A growing number of methods learn a selector, which identifies important features, and a predictor, which uses these to make predictions. However, these pioneering methods face challenges including information leakage and lack of differentiability, which can slow training. In this paper, we present Hide&Seek, an end-to-end differentiable model for instance-wise feature selection. We jointly learn feature selection and prediction under a single objective without information leakage. Hide&Seek outperforms existing state-of-the-art models across a range of experiments and is fast to train. We achieve this by reformulating feature removal as a differentiable operation where instead of discretely removing features, we replace a proportion of each feature. Training is further stabilized via a parsimony-weight annealing framework.

1. Introduction and Context

Feature selection supports interpretability, transparency, scientific discovery, and model performance, motivating instance-wise approaches that tailor important features to each input. Hide&Seek addresses prior methods’ non-differentiability and information leakage through differentiable feature replacement, random-distribution ablation, and parsimony-weight annealing.

  • Motivation: Feature selection identifies informative variables, helping explain black-box predictions while simplifying models and reducing overfitting.It is used to interpret labeled datasets and support scientific discovery and transparency.
  • Motivation: Global feature selection estimates importance across datasets, whereas instance-wise methods identify important features dynamically for each input.The introduction motivates instance-wise selection because diagnostic pathways can differ between patients.
  • Related Work: Selector–predictor methods such as L2X, INVASE, and REAL-x learn feature inclusion probabilities, but discrete sampling creates differentiability challenges.L2X uses a Gumbel–Softmax relaxation, which requires deciding the number of important features in advance.
  • Limitations: Information leakage occurs when predictors exploit fixed ablation values for unselected features as switch signals during joint training.REAL-x avoids leakage by decoupling predictor and selector training, but this disjoint training limits joint optimization.
  • Contributions: Hide&Seek jointly trains a fast, end-to-end differentiable selector and predictor by replacing feature proportions rather than discretely removing subsets.It prevents leakage with random draws from the feature distribution and stabilizes training through parsimony-weight annealing.

2. Problem Formulation

The problem is to identify a minimal, instance-specific feature subset that preserves label-prediction information while avoiding the non-differentiability and leakage introduced by fixed-value ablation. Hide&Seek addresses this by using continuous masks, distribution-based stochastic replacement, and a sparsity-regularized predictive objective.

  • 2. Problem Formulation: Instance-wise feature selection seeks a minimal subset S ⊆ D for each input x whose selected vector x_S supports prediction of y.The feature space is d-dimensional, with D := {1, . . . , d}.
  • 2. Problem Formulation: Fixed-size approximations of x_S replace unselected features with uninformative values such as zeros or feature means.This permits modeling with a d-dimensional vector despite instance-dependent subset sizes.
  • 2. Problem Formulation: Fixed-value ablation creates discrete, non-differentiable selection and can leak information to a jointly trained selector and predictor.The formulation therefore reformulates both selection and replacement.
  • 2. Problem Formulation: Continuous masks m ∈ [0, 1]^d construct each feature as a linear combination of its original signal and a stochastic replacement value.A higher m_j indicates a higher likelihood that feature j belongs to S, replacing a proportion of each feature rather than whole features.
  • 2. Problem Formulation: Sampling replacements from the product of marginal feature distributions is efficient and harder to exploit than fixed-value ablation.Fixed-value leakage requires recognizing one ablated value, whereas marginal-sampling leakage requires learning the true joint distribution and detecting out-of-distribution replacements.
  • 2.1. Optimization: The selector is optimized through KL divergence, approximated by model-induced cross-entropy plus an l1 sparsity penalty weighted by λ.For classification, the negative log-likelihood becomes the standard cross-entropy loss.

3. Proposed Model

Hide&Seek jointly learns feature importance and prediction with two feed-forward modules, using continuous masks and marginal-noise replacement for end-to-end differentiability. A quadratic annealing schedule delays parsimony pressure, prioritizing prediction early and mask parsimony later.

  • Architecture: Hide takes input x and produces a continuous mask m ∈ [0, 1]^d representing the importance of d features, while Seek maps z to predicted output ŷ.The architecture consists of two feed-forward fully connected neural network modules: Hide and Seek.
  • Differentiable selection: Feature importance is learned without discrete feature sampling by replacing a proportion of each feature with marginal noise, enabling end-to-end differentiability and joint network training.This formulation avoids discretely retaining or replacing features.
  • Parsimony annealing: Excessive parsimony emphasis early in training can trap optimization in poor local minima, motivating an annealing schedule for the regularization weight.The joint objective minimizes cross-entropy while increasing parsimony.
  • Parsimony annealing: The regularization parameter λ_t grows quadratically toward fixed λ_max over T training epochs, progressively increasing the weight on the parsimony term.Here, λ_max is a fixed hyperparameter and t indexes training epochs.
  • Parsimony annealing: The schedule prioritizes prediction accuracy early and mask parsimony later, producing a lower final loss and strong feature-importance metrics.These effects are reported from the schedule illustrated in Figure 2 and accompanying experiments.

4. Experiments

Across synthetic and correlated-feature experiments, Hide&Seek achieved strong instance-wise feature selection while avoiding information leakage. It also maintained high switch-feature accuracy under multicollinearity and used substantially fewer parameters than competing selector–predictor models.

  • Synthetic data: Hide&Seek outperformed the other models on instance-wise feature-selection metrics in six synthetic datasets.The datasets included global and switch-feature settings, evaluated using True Positive Rate and False Discovery Rate.
  • Synthetic data: 17× to 30× fewer parameters were used by Hide&Seek than REAL-x, L2X, and INVASE, respectively, while achieving higher predictive performance than other selector–predictor models.The reported comparison is 17× fewer parameters than REAL-x and L2X, and 30× fewer than INVASE.
  • Switch analysis: 100% median switch accuracy was achieved by Hide&Seek across Syn4–6, significantly exceeding INVASE (≈51%) and L2X (≈58%).REAL-x also achieved perfect switch accuracy, attributed to disjoint predictor training.
  • Multicollinearity: Hide&Seek consistently outperformed the other models for instance-wise feature selection across pairwise correlations ρ from 0 to 0.9.The correlated setting used multivariate-normal features with constant off-diagonal correlation ρ.
  • Multicollinearity: Greater than 99.4% average switch accuracy was maintained by Hide&Seek across Syn4–6 for every tested ρ, indicating no leakage under high pairwise correlations.Switch accuracy was averaged across Syn4–6.

5. Conclusion

Hide&Seek is an end-to-end differentiable instance-wise feature-selection method that jointly trains a selector and predictor while avoiding information leakage through distributional replacement. The conclusion reports theoretical leakage analysis, robustness under continuous masks, and strong, fast performance across diverse datasets.

  • Method: Hide&Seek jointly trains a selector and predictor with continuous masks, replacing unselected features using draws from feature distributions rather than fixed ablations.This prevents the ablation pattern from becoming an information source and enables efficient joint optimization.
  • Information leakage: Theoretical analysis shows the usual ablation-model objective can be an ill-posed proxy for the intended expected KL objective, causing important features to be omitted without reducing predictive performance.For switch features, this produces a lower bound on achievable misidentification.
  • Information leakage: Under the idealized construction, information leakage is theoretically impossible when replacement values are conditioned on selected features and the mask is binary.Empirically, the method remains robust to leakage with continuous masks and marginal-distribution replacement values.
  • Empirical results: Across synthetic, credit-default, MNIST, and genetic microarray experiments, Hide&Seek achieves strong instance-wise feature-selection performance while remaining fast to train.The results support distributional replacement as a way to retain joint selector–predictor training without creating an artificial information channel.

Impact Statement

The paper advances Explainable AI by improving black-box model interpretability for transparency and informed decision-making, while emphasizing validation because misleading explanations can introduce risk. It also proves that a seemingly natural prior XAI loss is ill-posed because it can cause unintended information leakage.

  • Impact Statement: Improved interpretability of black-box models can support transparency and informed decision-making, but incorrect explanations can introduce risk.The paper emphasizes the need for careful validation of explanations.
  • Impact Statement: A seemingly natural loss function used in prior XAI work is ill-posed because it can lead to unintended information leakage.This finding highlights the broader challenge of designing neural-network objectives that do not encourage unintended behavior.

A. On Feature Ablation and Information Leakage … A.3. Preventing Information Leakage

The section formalizes instance-wise feature selection and shows that conventional ablation can create ill-posed objectives through information leakage. It motivates replacement-based masking, while discussing related leakage modes and conditional sampling intended to preserve the original feature distribution.

  • A. On Feature Ablation and Information Leakage: Instance-wise feature selection jointly learns a selector S(x) identifying important feature indices and a predictor distribution conditioned on the selected values.The selector maps each instance to a feature subset, and the selector–predictor pair minimizes an intended loss.
  • A. On Feature Ablation and Information Leakage: Ablation methods replace unselected features with fixed symbols to convert conditioning on arbitrary feature subsets into fixed-size inputs.The replacement symbol is typically outside the original feature support, although zero and user-defined values are also used.
  • A. On Feature Ablation and Information Leakage: Conventional ablation can make the proxy loss ill-posed because selector–predictor pairs exploit the ablation mask, allowing information to leak between them.The theorem states that proxy-loss minimizers need not minimize the intended loss and may achieve lower proxy loss by exploiting masking patterns.
  • A. On Feature Ablation and Information Leakage: P*(Xi ∈ Ai,max) is a lower bound on the achievable rate at which Xi is ablated and misidentified as unimportant when Y depends on Xi through a partition.Ai,max is the most probable partition cell; the corollary is existential and does not imply every optimization run discovers the leakage strategy.
  • A.1. Comparison to Jethani et al. (2021): Jethani et al. identified leakage in jointly trained selector–predictor networks, including label encoding through binary ablation masks in classification.The paper extends the analysis to partition-dependent features and to both regression and classification without restricting the number of labels.
  • A.2. Sampling ˆx Using Model-X Knockoffs: Hide&Seek samples replacement values from the product of feature marginals, and reported experiments demonstrate absence of leakage in this setting.The appendix also reports that marginal sampling outperforms Model-X knockoff sampling in the referenced experiments.
  • A.3. Preventing Information Leakage: Leakage is prevented when the modified predictor input has the original distribution; ideally, unselected replacements satisfy ˆx ¯S ∼ p(X ¯S | xS).For binary masks, p(Z)=p(XS)p(ˆX ¯S | XS) equals p(X) exactly when the replacement conditional matches p(X ¯S | XS).

A.4. Correlated Synthetic Data · B. Model infrastructure · B.1. Hardware

The correlated-data experiment found that Hide&Seek could not learn that marginally sampled switch-feature replacements were out-of-distribution. The appendix also specifies model implementations, training settings, and experimental hardware.

  • A.4. Correlated Synthetic Data: Hide&Seek’s Seek module failed to learn that marginally sampled replacement values for the switch feature were out-of-distribution.This result held despite marginal sampling according to eq. (7).
  • A.4. Correlated Synthetic Data: Table 7 reports switch accuracy for identifying X11 as important across ρ and models, averaging results across Syn4, Syn5, and Syn6.Each value is the mean switch accuracy across the three datasets for one run, with expected variability shown by Figure 3’s boxplots.
  • B. Model infrastructure: Hide&Seek uses two fully connected feed-forward ReLU networks, with sigmoid masking, softmax prediction, two 32-dimensional hidden layers, and 500 unbatched epochs.The model uses Adam with learning rate = 0.001.
  • B. Model infrastructure: INVASE uses 100-dimensional selector layers, 200-dimensional predictor layers, 10,000 training epochs, batch size 1,000, and λ = 0.1 for synthetic data.The implementation uses the INVASE repository code.
  • B. Model infrastructure: REAL-x uses 100-dimensional selector layers, 200-dimensional predictor layers, 500 training epochs, batch size 1,000, and λ = 0.15 for synthetic data.The implementation uses the REAL-x repository code.
  • B. Model infrastructure: L2X has two networks with two hidden layers each, using 100 dimensions per layer in the first network and 200 dimensions per layer in the second.The implementation follows the L2X repository code.
  • B.1. Hardware: The experiments used an AMD EPYC 9354P CPU, 192GB ECC DDR5-RAM, two 1.92TB NVMe SSD drives, and 2x NVIDIA L4 GPUs.The CPU has 32 cores and 256MB L3 cache; each GPU has 7,680 cores, 240 Tensor Cores, and 24GB memory.

C. Metric calculations … D.2. Predictive Performance

The paper defines dataset-level TPR, FDR, and F1 by averaging instance-wise metrics, then reports synthetic-data analyses showing masks aligned with generating rules and Hide&Seek’s stronger AUROC despite fewer parameters. REAL-x underperforms, plausibly because its disjoint training uses Bernoulli(0.5) masks that can create out-of-distribution inputs.

  • C. Metric calculations: TPR, FDR, and F1 are computed for each input instance, averaged across the dataset, and reported as medians over 20 seeded runs when specified.Full Section 4.1 distributions are shown as boxplots.
  • C. Metric calculations: TPR is true positives divided by true positives plus false negatives, while FDR is false positives divided by true positives plus false positives.Recall equals TPR, and precision equals 1 − FDR.
  • D. Further analyses: Section D analyzes the synthetic-data experiment in Section 4.1 through further analyses and experiments.These analyses include mask distributions and predictive performance.
  • D.1. Mask Distributions: Learned mask distributions closely align with the data-generating rules across the six synthetic experiments.X11 is the switch-feature used in Syn4–6.
  • D.2. Predictive Performance: Hide&Seek consistently achieves higher AUROC than INVASE, L2X, and REAL-x despite having significantly fewer parameters.The comparison concerns the models used in Section 4.1.
  • D.2. Predictive Performance: REAL-x’s poorer predictive performance is likely caused by out-of-distribution draws when predicting on highly parsimonious data.Its predictor is trained disjointly on feature masks drawn from Bernoulli(0.5).

D.3. λ Sensitivity … D.6. Syn3 - Specification vs. Implementation

Across sensitivity, feature-count, runtime, and Syn3 specification analyses, Hide&Seek remains robust while using a compact jointly trained model, and the experiments explicitly document tuning and implementation choices affecting comparability.

  • D.3. λ Sensitivity: Hide&Seek remains robust as λ varies, preserving percentage significance near ground truth while maintaining AUROC values mostly above 0.8.The paper attributes this robustness to annealing the parsimony weight toward the end of training.
  • D.3. λ Sensitivity: Hide&Seek’s AUROC strongly correlates with IWFS F1, initially improving as increasing parsimony removes unimportant features.The passage describes AUROC as a tuning metric whose relationship with IWFS F1 is evaluated post-hoc.
  • D.3. λ Sensitivity: INVASE’s original λ = 0.1 setting yields high IWFS F1, whereas selecting λ by highest AUROC would have produced lower IWFS results.The alternative AUROC-selected values reported are λ = 0.95 or λ = 0.45.
  • D.4. Results for k = 3 and k = 4: For SHAP, LIME, L2X, LASSO, and RForest, k is set from the dataset’s target number of important features, with additional k = 3 and k = 4 results shown for Syn4–5.The main-text settings are k = 2 for Syn1, k = 4 for Syn2–3, and k = 5 for Syn4–6; Syn4–5 can have only 3 important features when X11 < 0.
  • D.5. Run Time Analysis: Hide&Seek uses ≈3k parameters, compared with ≈100k for INVASE and ≈57k for REAL-x.The passage also notes that Hide&Seek does not use batching.
  • D.5. Run Time Analysis: INVASE trains substantially longer because of its REINFORCE architecture, while REAL-x requires separate selector and predictor training despite using differentiable REBAR gradients.Runtime comparisons include training, prediction, and feature attribution on 10,000 samples using identical hardware.
  • D.6. Syn3 - Specification vs. Implementation: The Syn3 specification differs between prior papers and their linked code, so the experiments use the previous code’s data-generating model for comparability.The discrepancy is noted for work by Yoon et al. (2018) and Chen et al. (2018).

D.7. Parsimony-Weight Annealing Analysis · E. Further experiments

The parsimony-weight annealing analysis finds stable performance for q ∈ {2, 3, 4, 5}, with q = 2 prioritizing prediction early and parsimony later. Further experiments extend evaluation to 1,000,000 samples, 100 features, and California Housing data.

  • D.7. Parsimony-Weight Annealing Analysis: q ∈ {2, 3, 4, 5} yields stable TPR, FDR, F1, and AUROC results, whereas q ∈ {0.5, 1} performs poorly.Table 11 averages these metrics across six synthetic datasets and 20 seeds with λmax = 0.3.
  • D.7. Parsimony-Weight Annealing Analysis: q = 2 allows the model to emphasize prediction accuracy early in training and parsimony later.This schedule is selected for the experiments described in the paper.
  • D.7. Parsimony-Weight Annealing Analysis: Hide&Seek uses an annealing schedule for the parsimony weight λt over training epochs t.The schedule is analyzed through the parameter q and λmax.
  • D.7. Parsimony-Weight Annealing Analysis: Figure 11 evaluates q ∈ {0, 1, 2, 3} across different λmax values using cross-entropy, regularized parsimony, and combined-loss curves.The metrics are calculated on a hold-out validation set using the Syn4 data.
  • D.7. Parsimony-Weight Annealing Analysis: When λmax is large enough to impose parsimony, q = 2 provides the best results and a lower final combined loss than q = 3.The analysis also reports q = 2 as more stable than q = 3.
  • E. Further experiments: The further-experiments section trains models on 1,000,000 samples, evaluates performance with 100 features, and uses California Housing data.These are the three experiment settings listed for the section.

E.1. Training on 1,000,000 Samples … F. Extra detail

Across additional experiments, Hide&Seek remains competitive or superior as training data and feature dimensionality increase, while Hide&Seekens further improves performance through ensembling and column subsampling. The paper also evaluates instance-wise feature selection on correlated California Housing features and provides further detail on non-synthetic experiments.

  • E.1. Training on 1,000,000 Samples: With 1,000,000 training samples, Hide&Seek still outperforms other models, although L2X improves when trained on more data.REAL-x required its parsimony regularizer λ to be tuned down as training samples increased from N = 10, 000 to N = 1, 000, 000.
  • E.2. Training on 100 Features and Ensembling: The high-dimensional experiment increases synthetic features from 11 to 100, adding 89 independent noise signals while preserving the earlier feature relationship.The target remains the same function of the original 11 features.
  • E.2. Training on 100 Features and Ensembling: Hide&Seekens ensembles 10 independent Hide&Seek models trained on random 90% feature subsets, averaging masks before applying the standard > 0.5 selection threshold.This extension leverages the base model’s fast training and column subsampling.
  • E.2. Training on 100 Features and Ensembling: Hide&Seek is competitive in the 100-feature setting, while Hide&Seekens outperforms all other models and matches or exceeds ideal INVASE.The results support applying Hide&Seek to large datasets and using ensembling and column subsampling to improve performance.
  • E.3. California Housing: The California Housing experiment evaluates feature selection against correlated real-world features in a semi-synthetic setting with a three-partition Longitude switch feature.The dataset contains 20,640 block groups from the US 1990 Census, with housing variables used to construct the experiment.
  • E.3. California Housing: Ground-truth importance is defined by sampling Y from P(Y = 1|U) = 1 1+eU, with U determined by geographic Longitude partitions and housing-feature functions.The reported partitions include formulas involving Average Rooms, Average Bedrooms, Median Income, and Median House Age.
  • E.3. California Housing: Models are evaluated on recovering ground-truth housing attributes and Longitude after a 12,828/1,604/1,604 train-validation-test split and λ tuning across a specified grid.Features are standardized using training data, and results are reported in Table 14.
  • F. Extra detail: The extra-detail section supplies further information about the paper’s non-synthetic data experiments.It is explicitly presented as additional detail on those experiments.

F.1. Credit Default Data Detail · F.2. MNIST Detail · F.3. Breast Cancer Subtype Classification Detail

The appendix details the Credit Default, MNIST, and breast-cancer-subtype experiments, covering input features, preprocessing and hyperparameter choices, synthetic-data construction, and gene-importance analysis. These details clarify how each experiment was configured and evaluated.

  • F.1. Credit Default Data Detail: The Credit Default experiment used 23 features, with next-month default payment as the raw binary target.The listed inputs include demographic, credit-limit, bill-amount, and payment-history variables; the target was not used in Syn4–6.
  • F.2. MNIST Detail: The MNIST experiments tuned λ for Hide&Seek, INVASE, and REAL-x over {0.05, 0.1, 0.2, 0.3, 0.4, 0.5}.The setting with the highest prediction accuracy was selected for each method.
  • F.2. MNIST Detail: For Hide&Seek, zero-mean, unit-variance scaling produced higher prediction accuracy than global [0, 1] rescaling for every tested λ value.Feature-wise standardization used the training-set mean and variance.
  • F.3. Breast Cancer Subtype Classification Detail: The breast-cancer-subtype experiment analyzed 100 genes, matching the experiment of Covert et al. (2021).Table 15 lists the genes used, while Table 6 reports the top 10 genes identified by each model.
  • F.1. Credit Default Data Detail: Eleven of the 23 Credit Default features were used as {X1, . . . , X11} to generate Syn4–6.The selected features are identified by the black box in the correlation matrix.
  • F.3. Breast Cancer Subtype Classification Detail: Detailed gene-importance results are reported as mean ± standard error.Importance is measured by mean mask size for Hide&Seek, INVASE, and REAL-x; mean importance scores for SHAP and LIME; and mean selection frequency for L2X.
Loading 2608.16689v1…