Sequential Conditional Independence Testing with Machine Learning Models

1 INTRODUCTION

• Conditional independence testing asks whether X provides information about Y beyond Z, helping identify important features without mistaking correlated inputs for disease-related signals.The important features form the Markov blanket when no input is an exact function of the others.Source: §1 · p. 1
• Model-X shifts the modeling burden from the output-input relationship to the dependence structure among inputs, which may be easier to model.Prior work in interpretable machine learning also supports dependence-based methods such as Conditional Feature Importance over Leave-One-Covariate-Out approaches.Source: §1 · p. 1
• Model-agnostic conditional-independence methods can use black-box predictors as surrogates for the data-generating distribution.This is particularly relevant as machine learning achieves predictive performance on complex data.Source: §1 · pp. 1–2
• E-values and e-processes support flexible, sequential evidence monitoring when data collection continues or combines evidence across studies.An e-process can help determine whether to collect more data or whether sufficient evidence has already been obtained.Source: §1 · p. 2
• The paper compares original-model loss with loss after replacing a feature by a conditionally independent counterpart, and studies guarantees when the model-X assumption is violated.The approach builds on sequential feature-importance testing with black-box machine-learning models.Source: §1 · p. 2

2 PROBLEM STATEMENT

• The model-X null tests X ⊥ Y | Z while fixing the conditional distribution K(X | Z).The kernel is part of the null hypothesis, not an estimated feature of the alternative.Source: §2 · p. 2
• Introducing a dummy variable X̃ drawn under the same kernel makes the null equivalent to conditional exchangeability of X̃ and X given (Y, Z).The dummy is conditionally independent of X and Y given Z.Source: §2 · p. 2
• The CRT replaces observed X with draws from K(X | Z), compares the resulting importance statistics with the observed statistic, and yields p-values.Its performance depends strongly on the choice of importance statistic T.Source: §2 · p. 2
• Sequential model-X e-values are nonnegative random variables with null expectation at most 1, and GRO e-values maximize expected log e-value under a point alternative.The cited model-X e-value is GRO against an alternative with density fY|X,Z.Source: §2 · p. 2
• The GRO alternative density fY|X,Z is generally unknown and difficult to estimate or connect to prediction-oriented machine-learning models.This leaves the practical application of that GRO construction unclear.Source: §2 · p. 2
• The eCRT compares predictive loss on observed and conditionally resampled features using an antisymmetric e-value based on the two statistics.Under the null, the statistics are exchangeable; the construction uses an antisymmetric function and requires the resulting e-value to remain positive.Source: §2 · pp. 2–3
• The eCRT tests a larger exchangeability null than model-X conditional independence, so it remains valid but may discard information.Model-X implies exchangeability of the paired statistics, but the converse does not hold.Source: §2 · pp. 2–3
• Because K is usually estimated in practice, the resulting e-values are approximate and can inflate type-I error.The passage motivates tighter inflation bounds and says the paper uses triple-robustness results to improve them.Source: §2 · p. 3

3 ANALYSIS UNDER THE ALTERNATIVE

• Enlarging the null or pushing forward the distribution can reduce approximation and estimation errors, but introduces a null-enlargement error into e-power.The analysis separates these sources of loss when assessing an estimated e-variable.Source: §3 · p. 3, Figure 1 · p. 3

3.1 Model-X vs Antisymmetric e-variables

• The authors propose ML-based exponential e-variables and show antisymmetric alternatives can outperform estimated GRO e-variables because of approximation and estimation errors.Theorem 3.3 allows more powerful antisymmetric e-variables when the alternative density is not of the exponentiated-loss form.Source: §3.1 · p. 3, §3.2 · p. 5

3.1.1 Exponential e-value

• The exponential class uses exponentiated losses, normalized under the null, as a machine-learning-based approximation to the original GRO e-variable.The construction follows the model-X density e-value by using pseudoprobabilities and a variance-like parameter.Source: §3.1.1 · p. 4

3.1.2 Coin-betting e-value

• The coin-betting adaptation applies to bounded losses; its stated optimality results do not directly transfer to the model-X null setting.The analysis assumes a fixed predictor, which the passage says can be replaced by a sequential one.Source: §3.1.2 · p. 4, §3.1.3 · p. 4, §3.1 · p. 4

3.1.3 Comparison of both e-values

• Neither e-variable uniformly dominates: coin betting can be more powerful in some settings, while distillation’s null enlargement can be offset by reduced approximation error.The comparison also notes that using more complex learners may reduce approximation error while making estimation error more significant.Source: §3.1.3 · pp. 4–5

3.2 Optimal antisymmetric e-variable

• For exchangeability e-values based on (T, eT), an antisymmetric form 1 + g(T, eT) can dominate any such e-value, and the GRO has a closed form.The GRO expression is characterized against an alternative with density fT,eT.Source: §3.2 · p. 5

3.3 Intermediate null hypotheses

• The push-forward model-X null tests the distilled statistics (T, eT); its GRO has more e-power than e-variables using that information distillation, but still relies on a hard-to-estimate conditional density.The push-forward can reduce approximation and estimation error at the cost of null enlargement.Source: §3.3 · pp. 5–6
• B-exchangeability compares T with B resampled statistics, but its (B + 1)-dimensional density must be estimated and therefore suffers much higher estimation error.The passage states that it may have more e-power than any derandomized e-variable.Source: §3.3 · p. 6

4 ANALYSIS UNDER THE NULL

• Estimated conditional distributions affect both model-X and antisymmetric e-variables, so the resulting processes have approximate rather than exact guarantees.The paper contrasts its structure-aware bounds with worst-case bounds over all statistics.Source: §4 · p. 6, §4.1 · p. 6

4.1 Double Robustness

• Under the conditional-independence null, the model-loss statistic becomes less sensitive to conditional-distribution estimation error as the predictor approaches the X-independent Bayes predictor.The paper attributes this double robustness to both convergence of the estimated conditional distribution and convergence of the fitted model toward the null regression function.Source: §4 · p. 6, §4.1 · pp. 6–7
• The bounds cover Lipschitz, uniform, and L2 convergence assumptions, with more regular convergence assumptions yielding less restrictive distributional-distance bounds.The L2 case specifies an explicit convergence rate.Source: §4.1 · p. 7
• The discrepancy between distributions is bounded by a product of conditional-distribution estimation error and the fitted model’s remaining dependence on X.The two factors are an and bn, respectively.Source: §4.1 · p. 7

4.2 Type-I error inflation

• Type-I error inflation is controlled by cumulative approximation-event failure probabilities and cumulative e-process inflation.The model’s convergence assumptions need only hold with high probability, rather than exactly.Source: §4.2 · pp. 7–8
• In a Gaussian linear setting with quadratic loss, the bound decays as n^-2 rather than the n^-1/2 captured by prior total-variation bounds, and incorporating bλ yields n^-5/2.The n^-5/2 rate incorporates the convergence rate of bλ.Source: §4.2 · p. 8

5 EXPERIMENTS

• For a small HIV drug-resistance effect, the exponential e-process did not reject, while most antisymmetric e-processes rejected and outperformed it.For an unimportant mutation none rejected; in the large-signal regime all e-processes rejected.Source: §5 · p. 8
• For a large HIV drug-resistance effect, the exponential e-process had the slowest growth rate, indicating non-negligible approximation error.The experiment compared exponential and antisymmetric e-values across three mutations.Source: §5 · p. 8
• The reproduced simulation and triple-robustness experiment found converging estimates, with the e-process staying almost constant and not rejecting under the null.The authors also report further simulations of derandomization.Source: §5 · p. 8

6 DISCUSSION

• The exponential e-value provides a natural GRO estimate, but density misspecification creates approximation error and estimation within that class can make it worse than antisymmetric e-values.The antisymmetric class uses the larger exchangeability null, introducing null-enlargement error.Source: §6 · p. 8
• Intermediate null hypotheses could reduce null-enlargement error by better exploiting the structure of the model-X conditional-independence null.The error decomposition helps compare the two e-value classes and motivates this exploration.Source: §6 · p. 8
• A sequential mixture could shift weight from coin-betting toward a density e-value as more data improve its estimate, but effective implementation remains future work.The discussion suggests data-driven, dynamic weighting over time.Source: §6 · p. 8
• The proposed type-I error analysis uses approximate e-values and statistic structure to exploit double robustness when the model-X assumption is challenged.The authors describe this as providing a more reliable tool for sequential testing with machine learning.Source: §6 · p. 8

AI use statement

• The authors report using generative AI to assist with proof writing, method implementation, and translation, and say they reviewed the assisted work.They state that AI was not used to develop theoretical models, formulate mathematical claims, or interpret results.Source: AI use statement · p. 9

CHECKLIST

• The checklist marks full assumptions, proofs, and explanations for theoretical claims as included, and reproducibility materials for main experiments as included.It also marks experimental statistics and error bars as included.Source: CHECKLIST · p. 11

A ALGORITHMS

• The algorithms accommodate any machine-learning model that can be retrained from scratch or updated online, and classification or regression tasks with a performance loss.The 2-Exchangeability GRO e-value can use any kernel density estimate.Source: A ALGORITHMS · p. 12

A.1 Exponential e-process

• The exponential e-process updates its model sequentially and allows optional stopping, with 1/α as a standard stopping threshold.The model may be retrained on the growing dataset or updated using stochastic gradient descent.Source: §A.1 · p. 12

A.2 Standard coin-betting and coin-betting with unbounded losses

• The coin-betting procedure replaces the exponential procedure’s update step; clipping the loss difference can keep the e-value nonnegative when losses are unbounded.The paper also describes learning the bet fraction and loss-control parameter prequentially from past data.Source: §A.2 · p. 13

A.3 2-Exchangeability GRO

• The 2-Exchangeability GRO procedure estimates a density over importance statistics, but its current optimization omits derandomization and loses power relative to other antisymmetric e-values.The authors suggest adaptive weighting of recent observations or prequentially optimizing a parametric density model as future options.Source: §A.3 · p. 14

B INTERMEDIATE NULL HYPOTHESES

• Table 1 orders nulls by progressively retaining less information, which enlarges the null, and lists each null’s GRO e-variable.Source: Table 1 · p. 15, B INTERMEDIATE NULL HYPOTHESES · p. 15
• PFX-CI uses an importance statistic T and its conditionally independent counterpart to define a push-forward model-X null.Its e-power is bounded by that of X-CI, with the gap reflecting coarsening under the data-processing inequality.Source: B INTERMEDIATE NULL HYPOTHESES · p. 16
• The conditional exchangeability null replaces perfect model-X knowledge with B kernel samples, and its randomization error decreases to zero as B grows.The PFX-CI GRO e-variable is recovered as a limit of GRO e-variables for these nulls.Source: B INTERMEDIATE NULL HYPOTHESES · pp. 16–17
• B-exchangeability tests marginal exchangeability of importance statistics, treating (Y, Z) as latent, and has no greater e-power than conditional exchangeability.Source: B INTERMEDIATE NULL HYPOTHESES · p. 17
• Exchangeability-null GRO e-variables generally cannot reach PFX-CI e-power, even with infinitely many derandomization samples.The power gap can be strict when φ is not injective.Source: B INTERMEDIATE NULL HYPOTHESES · p. 18
• Choosing s trades easier estimation of fT|s(Y,Z) against e-power loss; if φ is a function of s, perfect estimation beats any exchangeability e-variable.The denominator can be known exactly or estimated by sampling from KZ.Source: B INTERMEDIATE NULL HYPOTHESES · p. 18
• Derandomization can break the GRO guarantee, so the number of derandomization samples must be included when defining intermediate nulls.With derandomization, antisymmetric e-variables can exceed the e-power of Eexch and Esym.Source: B INTERMEDIATE NULL HYPOTHESES · pp. 18–19

C FROM APPROXIMATE E-VARIABLES TO APPROXIMATE TYPE-I ERROR

• Approximate e-processes incur type-I error inflation through failed approximation events and cumulative multiplicative inflation.The bound accounts for the probability that any approximation event fails and for the process's cumulative inflation.Source: C FROM APPROXIMATE E-VARIABLES TO… · p. 20

D.1 σ(Y, Z)-measurability of λ and η

• For the stated σ(Y, Z)-measurable choices of λ and η, the sequential threshold 1/α has valid α-level type-I error control.Source: §D.1 · p. 20

D.2 Optimal e-values for the 0-1 loss

• Under 0–1 loss, the optimally chosen coin-betting and model-X e-values coincide.The result applies at their respective optimal (Y, Z)-dependent choices.Source: §D.2 · pp. 20–22

D.5.2 Proof of GRO property for Table 1

• The GRO e-variables for the null hypotheses in Table 1 are the variables provided there.The proof identifies them as likelihood ratios against null distributions, after verifying the e-variable property.Source: §D.5.2 · p. 25

D.5.3 Derandomization monotonically increases e-power

• With conditionally independent, identically distributed exchangeability draws, derandomization makes e-power monotonically increase with B.Source: §D.5.3 · p. 26

D.5.4 Proof for the limiting GRO e-variable for conditional exchangeability

• As B tends to infinity, the conditional-exchangeability GRO e-variable converges almost surely under the alternative to the conditional likelihood ratio.The argument assumes conditional independence of T and the exchangeability draws given (Y, Z), with the draws conditionally identically distributed.Source: §D.5.4 · p. 26

D.5.5 E-power sequence of conditional exchangeability e-variables

• The conditional-exchangeability e-power sequence is weakly increasing in B and converges to the conditional likelihood ratio when its expectation is finite.Source: §D.5.5 · pp. 26–27
• Uniform integrability in B follows because the A_B terms are averages of identically distributed integrable variables, while the single T-dependent term is integrable under f by assumption.The conditional i.i.d. structure gives the A_B terms a common law; the proof checks integrability of the T-dependent term separately.Source: §D.5.5 · pp. 27–28

D.5.6 Limiting GRO e-variable for exchangeability

• The limiting exchangeability e-variable is a conditional likelihood ratio of T and eT given φ(Y, Z), rather than the full pair (Y, Z).The proposition establishes an almost-sure limit under its stated regularity and density assumptions.Source: §D.5.6 · p. 28
• The exchangeability GRO limit requires regularity assumptions chosen to simplify the convergence proof, which the authors say may be weakened.The limiting expression conditions on φ(Y, Z), which encodes information about realized (Y, Z) available from an infinite sample of eT values.Source: §D.5.6 · p. 28

D.6.1 From approximate e-variables to approximate type-I error (Proposition C.3)

• Approximate e-processes yield type-I error control by bounding the chance of process failure with a union bound and applying Ville’s inequality to a nonnegative supermartingale.The proof first constructs the supermartingale and then controls the probability that any approximate-process condition fails.Source: §D.6.1 · pp. 30–31

D.6.2 Double robustness

• The double-robustness proof treats Lipschitz, uniform, and L2 convergence separately, invoking Wasserstein, total-variation, and chi-squared distance arguments, respectively.The supplied proof passages identify the distance arguments used for these convergence cases.Source: §D.6.2 · pp. 31–32

D.6.3 Coin-betting type-I error inflation

• For coin-betting, the conditional expectation error between the true and estimated kernels bounds one-step inflation, allowing an approximate e-process bound to be applied.The proof isolates a reference loss term that is identical under both kernels and cancels from their difference.Source: §D.6.3 · pp. 32–33

E.1 Estimating the conditional distribution

• For discrete X, conditional class probabilities can be modeled and sampled; for continuous X, the appendix describes Gaussian sampling or regression with permuted residuals.It also points to conditional-sampling approaches based on foundation models.Source: §E.1 · p. 33

E.2 Coin-betting can beat Exponential setting (Figure 2)

• When noise is large and the effect size is small, coin-betting can outperform the exponential e-value; at B = 1.5, its prequential evidence grows exponentially faster.The comparison optimizes the betting parameters over a grid for the e-power analysis; the sequential experiment updates them prequentially.Source: §E.2 · p. 34

E.3 E-values considered in the experiments

• Except for sign, hyperparameters are learned prequentially and optimized for e-power; the sign procedure averages over a grid of λ values instead.The sign-prequential variant sequentially optimizes λ.Source: §E.3 · p. 34

E.4 Simulation setting: e-power of antisymmetric vs exponential e-variables

• At β = 0, neither e-process rejects; for small signals, the antisymmetric sign e-process outperforms the exponential, while for larger signals the exponential accumulates evidence fastest.Figure 4 reports averages over 20 random seeds and batch sizes 5, 10, and 20.Source: Figure 4 · p. 35, §E.4 · p. 35
• The Figure 4 simulation uses a learned model-X kernel and Lasso predictor, with kernel and betting parameters tuned on 200 pre-training datapoints.The study uses batches of 5, 10, and 20 and 20 random seeds; the true conditional distribution is known but not supplied to the e-processes.Source: §E.4 · p. 35
• The simulated GRO exchangeability and symmetry e-variables do not exhibit their expected dominance because they are approximated by kernel density estimation and derandomized with B = 5 samples.The authors note that derandomization means their growth-rate optimality no longer necessarily holds.Source: §E.4 · p. 35

E.5 Real data: HIV dataset (Figure 3)

• For mutation 48M, most antisymmetric e-variables reject, while the exponential e-variable fails to reject and the exchangeability GRO does not dominate the other antisymmetric methods.The authors attribute the exponential result to approximation error and the GRO comparison to derandomization and estimation error.Source: §E.5 · p. 36
• For mutation 12S, which has no proven effect on drug resistance, none of the e-processes collects evidence against the null.Source: §E.5 · p. 36
• For mutation 47V, all considered methods reach evidence of at least 1/α and reject the null; the mutation has a reported major effect.The exponential e-variable collects the least evidence, which the authors attribute to model misspecification despite the large effect.Source: §E.5 · p. 36

E.6 Triple robustness for safety guarantees

• The experiment processes 20,000 generated data points, using 200 triplets for pre-training and a two-hidden-layer, 50-node-per-layer MLP.The MLP is trained with mean squared error, which is also used as the statistic for sequential bets; a Gaussian sampler generates X given Z.Source: §E.6 · p. 36
• The average betting martingale is almost constant and does not reject the null, while the GRAPA-selected betting parameter converges to zero.The experiment tracks kernel Wasserstein distance and predictor risk difference as the other two quantities associated with the triple-robustness demonstration.Source: §E.6 · pp. 36–37

• The comparison uses a linear Gaussian setting with explicit relevant densities, isolating derandomization from learning the correct density function.Source: §E.7 · p. 37
• The antisymmetric comparator uses a clipped difference between observed and resampled squared-residual losses, with betting parameters λ and M.Source: §E.7 · p. 37
• With exact densities, Eexch is exactly growth-rate optimal for testing Pexch with a single draw of eX, eliminating estimation error.Source: §E.7 · p. 37
• Table 3 compares e-power before and after B=20 derandomization, while Figure 6 plots e-power against the number of derandomization samples.The figure distinguishes re-tuned parameters with full lines from frozen parameters with dotted lines.Source: Table 3 · p. 39, Figure 6 · p. 40
• At B=1, Eexch retains its GRO property, but derandomization removes it; antisymmetric e-power gains mainly come from adapting betting parameters.As B grows, more aggressive betting—and therefore a larger optimal λ—is desirable, while the GRO e-variable has no free parameters to adapt.Source: §E.7 · pp. 39–40