PaperlayerView exact revision on arXiv

Source-linked AI summary

Regional Explanations via Causal Sufficiency and Necessity

Xuexin Chen, Peng Liang, Zijian Li, Zhiyong Lin, Ruichu Cai

arXiv:2609.18049v1cs.LG

TL;DR

Existing explainability methods often focus on features, counterfactual changes, or rules, while region-level accounts of when and only when a prediction behavior arises remain less explored. SNRE learns input-output region pairs using a region-level PNS objective and stochastic interventions. The paper reports strong sufficiency-necessity performance, robust explanations, and practical utility in model analysis.

  • Problem

    Region-level characterization of when and only when a prediction behavior arises remains less explored than feature-, counterfactual-, and rule-based explanations.

    Source: §I · p. 1

  • Method

    SNRE optimizes a region-level PNS objective using complementary stochastic interventions, interpretable algebraic regions, and a learnable soft feature mask.

    Source: §I · p. 2

  • Results

    SNRE achieves the highest PNS on all four datasets, with a better balance of sufficiency and necessity on German Credit than methods with stronger one-sided scores.

    Source: B. Quantitative Evaluation of Learned… · p. 8

  • Takeaways & Limitations

    Learned region pairs provide explicit structural summaries of model behavior that can support analyses of regional changes after fine-tuning and informative data selection.

    Source: §I · p. 2, C. Extracting Sufficient-and-Necessary… · p. 7

Abstract

from arXiv · show

Model explainability is essential for understanding and trusting machine learning models. Existing explainable AI methods often explain predictions through feature importance, counterfactual explanations, or rules. However, a region-level characterization of when and only when a prediction behavior arises remains less explored. This paper proposes Causal Sufficient and Necessary Regional Explanations (SNRE), a framework that learns an input region $A$ and output region $B$ such that membership in $A$ is both sufficient and necessary for the model output to fall in $B$. Motivated by the classical Probability of Necessity and Sufficiency (PNS), we formulate a region-level PNS measure through stochastic interventions and derive a differentiable finite-sample estimator for optimization. SNRE parameterizes the input-output region pair with explicit and interpretable algebraic region families, together with a learnable feature mask, balancing expressiveness and interpretability. Experiments demonstrate that SNRE learns region pairs with strong sufficiency-necessity performance, robust explanation behavior, and practical utility for model analysis.

I. INTRODUCTION

The paper asks whether explanations can identify input and output regions linked by a sufficient-and-necessary relation, extending beyond explanations that identify features, counterfactual changes, or rules. It positions SNRE as learning such region pairs and situates them against regional and PNS-based explanation work.

A. Regional Explanations

  • Conventional rule-based explanations define input-side regions associated with target predictions, but do not specify where the same prediction becomes unlikely.SNRE instead directly learns continuous input-output region pairs under sufficiency-necessity principles.Source: §I · p. 1, A. Regional Explanations · p. 2

I. INTRODUCTION

  • SNRE learns input-output region pairs so inputs inside A induce outputs in B while inputs outside A prevent outputs in B.The framework optimizes a region-level PNS objective using interpretable algebraic regions and a learnable soft feature mask.Source: §I · p. 2

B. PNS-based Explanations

  • Existing PNS-based explanation methods evaluate features, tokens, subgraphs, attributes, or masks rather than learning continuous regions in both input and output spaces.The paper identifies region-level PNS as a way to address this distinction.Source: B. PNS-based Explanations · p. 2, §I · p. 2

III. BACKGROUND AND PROBLEM FORMULATION

A. Causal Sufficiency and Necessity

  • At the region level, A is sufficient and necessary for B when placing inputs inside A induces B and placing them outside A prevents B.The paper distinguishes this relation from sufficiency without necessity and necessity without sufficiency.Source: A. Causal Sufficiency and Necessity · p. 3

B. Problem Formulation

The formulation seeks an explicit input region A and output region B whose membership events have a strong sufficiency-necessity relation for a fixed trained predictor. It motivates region-level PNS through stochastic interventions because hard interventions cannot contrast general regions with their complements.

B. Problem Formulation

  • The goal is to estimate input and output regions whose membership events have a strong PNS-based relation for a fixed, differentiable trained predictor.The setting excludes stochastic prediction models such as Bayesian neural networks with sampled parameters.Source: B. Problem Formulation · p. 3
  • The target is an explicit, geometrically interpretable region pair, not recovery of the full implicit decision region of a neural predictor.The exact decision region may be highly nonlinear and difficult to interpret.Source: B. Problem Formulation · p. 3

IV. REGION-LEVEL PNS VIA STOCHASTIC INTERVENTIONS

  • The paper motivates a region-level sufficiency-necessity measure by extending the semantics of classical PNS through stochastic interventions.The goal is a computable and optimizable formulation for finite data.Source: §IV · p. 3, B. Problem Formulation · p. 3

A. Motivating Region-level PNS from Classical PNS

  • Classical PNS concerns binary causes and outcomes, using potential outcomes under hard interventions do(X = x).It quantifies whether a cause is both sufficient and necessary for an outcome.Source: A. Motivating Region-level PNS from… · p. 3
  • Classical PNS contrasts interventions that satisfy a condition with interventions that violate it; for binary inputs these are do(X=1) and do(X=0).The region-level analogue must contrast X∈A with X/∈A.Source: A. Motivating Region-level PNS from… · p. 3
  • A hard intervention fixes one input value, so it cannot represent typical model behavior over all inputs inside or outside a region A.Region events require contrasting distributions over X∈A and X∈¯A.Source: A. Motivating Region-level PNS from… · p. 3, Figure 2 · p. 4

B. Stochastic Interventions for Regional Events

Complementary stochastic interventions sample from the input region and its complement, making regional sufficiency and necessity assessable under the same domain. The resulting region-level PNS has a finite-sample estimator whose reliability improves with sample size in the illustrated continuous-region setting.

B. Stochastic Interventions for Regional Events

  • The two intervention regimes condition the input distribution on A and its complement, then independently apply hard interventions to sampled input values.Their supports partition the reference domain, providing non-degenerate coverage of both events.Source: B. Stochastic Interventions for… · p. 4

C. Region-Level PNS and Estimation

  • Region-level PNS measures the sufficiency-necessity strength between input event X ∈ A and output event Y ∈ B under complementary interventions.The input domain is the support of P(X), and input and output events are evaluated together across the two intervention policies.Source: C. Region-Level PNS and Estimation · p. 4
  • The finite-sample estimator converges to PNS when both A and its complement have nonzero probability mass; implementation requires empirical samples in each.This prevents trivial degeneration of the estimate.Source: C. Region-Level PNS and Estimation · p. 4
  • As m increases, estimated PNS peaks near the reference region A⋆ and decays as A moves away; at m = 1, incorrect regions can appear spuriously strong.The figure fixes B⋆ while shifting A; larger samples better capture typical outputs under the conditional distributions on A and its complement.Source: C. Region-Level PNS and Estimation · p. 4, Figure 3 · p. 4

V. SNRE: SUFFICIENT AND NECESSARY REGIONAL EXPLANATIONS

SNRE jointly learns input and output regions for a fixed predictor by optimizing a differentiable relaxation of region-level PNS, with a soft mask selecting input dimensions. It uses explicit region families intended to balance expressiveness and interpretability.

V. SNRE: SUFFICIENT AND NECESSARY REGIONAL EXPLANATIONS

  • SNRE optimizes a differentiable region-level PNS objective to learn input and output regions for a fixed predictor, using sample-based marginalization and parallel weighted aggregation modules.A learnable soft feature mask selects input dimensions involved in the relation.Source: §V · p. 5

A. Region Parameterization and Feature Mask

1) Input region parameterization:

  • The input region is bounded by a quadratic hypersurface, whose algebraic form supports varied shapes and analytic reasoning about geometric properties.The paper names ellipsoids, hyperboloids, and paraboloids as representable shapes.Source: 1) Input region parameterization · p. 5

2) Output region parameterization:

  • For classification outputs, SNRE uses a compact, lower-complexity output-region family; higher-dimensional outputs may instead use a quadratic hypersurface.The output region may be fixed or partially learned when the target event is specified in advance.Source: 2) Output region parameterization · p. 6

3) Feature selection via a learnable mask:

SNRE uses a learnable mask to select dimensions defining the input region, while treating the remaining dimensions as background variables for marginalization. It optimizes a differentiable PNS estimate using sigmoid-softened event memberships and weighted empirical averages.

3) Feature selection via a learnable mask:

  • The learnable mask gates input dimensions in the quadratic region function, and dimensions with near-zero mask values are effectively removed from the region definition.The square-root gating avoids overly suppressing the quadratic term and supports more stable gradients; the mask is thresholded at inference to select features.Source: 3) Feature selection via a learnable… · p. 6
  • At inference, thresholding the trained mask yields a binary feature subset; dimensions outside that subset are treated as background variables and marginalized.The mask remains continuous during training for gradient-based optimization.Source: 3) Feature selection via a learnable… · p. 6

B. Differentiable PNS Objective and Optimization

  • Sigmoid relaxations with temperature τ provide smooth approximations to input- and output-region indicators and their complementary events.These soft memberships are substituted into the Monte Carlo PNS estimator to form a differentiable objective.Source: B. Differentiable PNS Objective and… · p. 6
  • Mixed intervention inputs take masked dimensions mainly from one training sample and unmasked dimensions from another, averaging over the dataset for marginalization.Normalized weights approximate conditional expectations under the in-region and out-of-region intervention policies by empirical averages.Source: B. Differentiable PNS Objective and… · p. 6

C. Extracting Sufficient-and-Necessary Regional Explanations

A learned region pair summarizes when inputs in a selected-feature region map into an output region and when inputs outside map outside it. Experiments use fixed predictors and compare SNRE with rule-based and PNS-based methods across real and synthetic datasets.

VI. EXPERIMENTS

  • The experiments use datasets for binary-classification tasks, and Fig. 5 depicts a toy joint-learning example for an input-output region pair.The figure shows input-region snapshots and trajectories for region-level PNS and the output-region parameter c.Source: Table I · p. 7, Figure 5 · p. 7

C. Extracting Sufficient-and-Necessary Regional Explanations

  • The learned pair (A∗, B∗) describes high-probability mapping into B∗ for inputs in A∗ and outside B∗ for inputs outside A∗, with respect to selected features.The feature mask is thresholded after optimization to define the selected subset.Source: C. Extracting Sufficient-and-Necessary… · pp. 6–7

VI. EXPERIMENTS

  • Experiments cover four real-world tabular datasets and two synthetic datasets, asking whether learned pairs retain sufficiency and necessity on unseen samples and outperform existing methods.The evaluation also examines design choices, robustness, efficiency, and practical utility.Source: §VI · p. 7
  • SNRE and baselines explain a fixed predictor on shared train, validation, and test splits; each experiment is repeated five times and reports the mean.The predictor is trained first and kept fixed during explanation learning.Source: §VI · p. 7
  • The comparison includes 17 baselines: 11 rule-based and 6 PNS-based methods; selected features from PNS methods are converted into input boxes for comparison.The box bounds use training-sample feature means and standard deviations among samples satisfying the fixed output event, with ρ selected on validation data.Source: §VI · p. 7

A. Toy Example: Visualizing Joint Learning of Input-Output Region Pairs

In the toy example, joint optimization tightens the input boundary around positive samples while raising the output threshold. On four test datasets, SNRE achieves the highest PNS, and the experiments examine region shapes and feature-mask strategies.

A. Toy Example: Visualizing Joint Learning of Input-Output Region Pairs

  • In the toy example, optimization tightens the quadratic input boundary around the positive inner disk as the output interval shifts toward high-confidence predictions.The lower bound of the output interval rises from about 0.5 toward 1.0, while the PNS objective approaches 1.Source: A. Toy Example: Visualizing Joint… · pp. 7–8

B. Quantitative Evaluation of Learned Sufficient-and-necessary Region Pairs

1) Input region parameterization:

  • Quadratic regions achieve the best or near-best PNS on both datasets, with the strongest balance of PS and PN on German Credit.On Phishing Websites, the quadratic parameterization leads simultaneously in PS, PN, and PNS.Source: 1) Input region parameterization · p. 8, Figure 6 · p. 9
  • For rotated elliptical data, quadratic boundaries capture rotation and anisotropy, unlike rigid linear, box, and sphere shapes, while retaining explicit geometric semantics.The comparison also describes the MLP boundary as a comparable fit but implicit; quadratic hypersurfaces trade expressive power against interpretability.Source: 1) Input region parameterization · pp. 8–9, Figure 7 · p. 9

2) Feature Mask:

  • The learned mask maintains high PNS as noise-feature ratios rise, while No-mask and Random-k fluctuate and decline.The comparison covers two datasets and evaluates PNS stability under varying noise-feature ratios.Source: 2) Feature Mask · p. 9, Figure 8 · p. 9

D. Robustness analysis of SNRE explanations

SNRE remains stable under noisy training inputs across four datasets, and on Phishing Websites it achieves PNS 0.948 with moderate model size and fast inference.

D. Robustness analysis of SNRE explanations

  • Across four datasets, SNRE’s input and output regions and feature masks remain stable overall when Gaussian noise is added to training inputs.On Mushroom, both IoU measures remain almost 1.0 and Mask Hamming stays close to 0; the other datasets show moderate fluctuations but consistently high IoU and small mask discrepancy.Source: D. Robustness analysis of SNRE… · pp. 9–10

E. Model Size and Efficiency

  • PNS 0.948 is SNRE’s best result on Phishing Websites, alongside 9,452 parameters and 0.0582 ms inference per sample.It uses fewer parameters than RRL (87,509) and is faster at inference than both RRL and DRNet, though it is neither smallest nor fastest overall.Source: Table III · p. 10, E. Model Size and Efficiency · p. 10

F. Case Study: Diagnosing Fine-Tuning-Induced Regional Prediction Drift

The case studies use SNRE regions to assess fine-tuning drift and guide active learning; the conclusion reports strong sufficiency-necessity quality and robust explanations across real-world and synthetic datasets.

G. Case Study: Active Learning via SNRE-Guided Sampling

  • SNRE-guided sampling achieved consistently higher, more stable test AUC and accuracy with faster convergence than random or uncertainty sampling.The comparison is shown across active-learning rounds on German Credit; the authors also report larger fluctuations for the alternatives.Source: Figure 11 · p. 11, G. Case Study: Active Learning via… · p. 11
  • SNRE-guided sampling selected a coherent applicant subgroup with older ages, longer loan durations and residence, and more single males than the unlabeled pool.The authors report that this subgroup was not stably covered by the learned input region and that labeling it encouraged recalibration and expansion of the explanatory region.Source: Figure 12 · p. 11, G. Case Study: Active Learning via… · p. 11

F. Case Study: Diagnosing Fine-Tuning-Induced Regional Prediction Drift

  • 0.733 retention for Conservative fine-tuning exceeded Aggressive (0.586) and Medium (0.573), matching the visually closest base-model boundary.The retention score averages overlap of the input region, feature mask, and output region before and after fine-tuning.Source: F. Case Study: Diagnosing… · pp. 10–11

VII. CONCLUSION

  • Across real-world and synthetic datasets, SNRE achieved strong sufficiency-necessity quality and robust explanation behavior, while case studies illustrated the practicality of learned region pairs.Source: §VII · pp. 11–12

APPENDIX A MONTE CARLO ESTIMATION OF CONTINUOUS PNS

The appendix derives a Monte Carlo estimator for continuous-region PNS under stochastic interventions, then uses softening and mixing operations to obtain a differentiable objective.

  • With a deterministic predictor and input variables without parents, each intervention determines the output, and independently sampled intervention policies yield a factorized joint probability.Source: Appendix A · p. 13
  • The Monte Carlo estimator averages over background samples while drawing intervened variables from empirical distributions inside and outside the input region.The in-region and complement distributions are defined conditionally on membership in A and its complement.Source: Appendix A · p. 13
  • Substituting softening relaxations and a mixing operation into the hard empirical estimator yields the differentiable objective used for optimization.Source: Appendix A · p. 13

APPENDIX C DETAILS ON EXPERIMENTS

The appendix describes the datasets and baseline families used in the experiments and gives the empirical-objective derivation, including its convergence justification.

A. Datasets

  • By the law of large numbers, each empirical average converges almost surely to its corresponding conditional probability.Source: A. Datasets  · p. 13

APPENDIX B DERIVATION OF THE EMPIRICAL AND DIFFERENTIABLE

  • The empirical objective derives from stochastic-intervention PNS after decomposing inputs into intervened variables I and background variables Z to be marginalized.Source: Appendix B · p. 13

A. Datasets

  • Evaluation covers four real-world tabular datasets and two synthetic datasets with labels determined by credit, phishing, mushroom, delinquency, or constructed geometric and cluster structure.Source: Appendix B · p. 13

1) Rule-based explanation baselines:

  • Rule-based baselines include decision sets, Bayesian and falling rule lists, trees, rule learners, and other rule-list methods.The passage describes IDS, SBRL, FRL, DT, FLDT, RRL, CORELS, SamRuLe, MDL-RL, DRNet, and IMLI.Source: 1) Rule-based explanation baselines · p. 14

2) PNS-based explanation baselines:

  • PNS-based baselines estimate or optimize necessity-and-sufficiency effects through features, masks, counterfactuals, or perturbations.The listed methods are FANS, LEWIS, LENS, NSE4T, CF2, and NSEG.Source: 2) PNS-based explanation baselines · p. 14

C. Evaluation Metrics

The section defines empirical sufficiency and necessity scores using a hard feature mask, then specifies the predictor and SNRE training setup.

1) Sufficiency and necessity scores:

  • The empirical sufficiency and necessity scores use a hard feature mask to combine selected features from samples inside or outside A with unselected features marginalized across evaluation samples.Necessity tests whether forcing selected features outside A makes the output fall outside B.Source: 1) Sufficiency and necessity scores · p. 14

D. Implementation

  • Predictors are three-layer MLPs trained with Adam and early stopping; regional experiments use hidden dimension 64, batch size 1024, a 60%/20%/20% split, and SNRE learning rate 0.008.Predictor training has a maximum of 200 epochs and patience 20; SNRE early stopping uses patience 25, with hyperparameters selected on the validation set.Source: D. Implementation · p. 14

Related papers

Loading 2609.18049v1…