Doubly Robust Deep Kernel Tests for Structured Outcomes

1 Introduction

• For images, text, and graphs, a mean outcome may be undefined or poorly aligned with data structure, while coordinate-wise image averaging ignores spatial structure.The paper frames the target as comparing interventional distributions directly on structured outcome spaces.Source: §1 · p. 1
• A fixed kernel may poorly fit the outcome geometry and yield low test power, motivating learned representations optimised directly for a test-power criterion.Deep kernels use neural networks to represent outcomes before comparing distributions.Source: §1 · p. 2
• The proposed estimator combines treatment-assignment and conditional-outcome models, remaining consistent when either model is correctly specified.Either approach alone is sensitive to model misspecification.Source: §1 · p. 2
• DR-KTE-Deep combines doubly robust interventional embeddings with deep-kernel learning and testing, and the paper evaluates it on image, text, and graph outcomes.The stated contributions include a uniform finite-sample bound for its learning criterion and conditional asymptotic validity of the held-out test after nuisance estimation and kernel selection.Source: §1 · p. 2

2 Related Work

• Quantile treatment effects and counterfactual distribution or density methods target distributional changes, but are typically developed for scalar or finite-dimensional Euclidean outcomes.The paper instead compares structured-outcome distributions without reducing them to low-dimensional summaries.Source: §2 · p. 2
• Prior causal kernel tests use prescribed outcome kernels or bandwidth aggregation, while other causal kernel methods estimate functions rather than compare interventional distributions.The cited test methods include Fawkes et al. (2024), Martinez-Taboada et al. (2023), and Zenati et al. (2025b); function-estimation approaches include Singh et al. (2024) and Kim & Wang (2026).Source: §2 · pp. 2–3
• Power-oriented kernel learning includes kernel-mixture and parameter optimisation, characteristic deep kernels, and approaches learning causal witnesses or outcome features.DR-KTE-Deep instead learns the outcome kernel for an omnibus test of marginal interventional distributions, estimating its criterion from observational data with causal nuisance models.Source: §2 · p. 3

3 Doubly Robust Kernel Test

• Identification requires unconfounded treatment assignment given observed covariates and positivity for treatments assigned by either intervention.The setup also assumes consistency and a structural response, with interventions changing treatment assignment while preserving the conditional outcome response.Source: §3 · pp. 3–4
• The method represents each interventional outcome distribution as an RKHS mean embedding and compares the embeddings using their squared distance, Dθ.For a characteristic kernel, Dθ equals zero if and only if the interventional distributions are equal.Source: §3 · p. 4
• The pseudo-feature combines a conditional-outcome embedding with propensity-weighted outcome residuals, and its expectation equals the embedding contrast if either nuisance model is correct.The equality holds if the logging-policy density or the conditional outcome embedding is correctly specified on the support of |r|.Source: §3 · pp. 4–5
• The test estimates the squared embedding discrepancy with an inner product of independent contrast estimates and uses a studentised cross-U statistic.For a fixed kernel, under the stated moment, projected non-degeneracy, and nuisance-bias conditions, the statistic is asymptotically standard normal under the null; large positive values provide evidence against it.Source: §3 · p. 5

4 Learning Deep Kernels

• The criterion Jθ = Dθ/sθ governs growth of the studentised test statistic under fixed alternatives, motivating its use for kernel selection.The empirical regularised criterion stabilises selection when the estimated variance is close to zero; the final test statistic remains unregularised.Source: §4 · pp. 5–6
• A learned neural-feature Gaussian kernel adapts outcome geometry, while a characteristic-kernel component preserves sensitivity to arbitrary distributional differences.The resulting kernel is characteristic for every θ, so its discrepancy is zero exactly when the interventional distributions are equal.Source: §4 · pp. 6–7
• With conditional moment, non-degeneracy, and nuisance-bias conditions, the held-out test has asymptotic level α after kernel learning and selection.Validity is conditional on the full learning procedure and does not require optimised parameters to converge to a deterministic kernel.Source: §4 · p. 7
• The uniform learning-criterion bound does not guarantee that training attains a global maximiser.The bound is stated under boundedness, Lipschitz, overlap, and measurability conditions on a compact finite-dimensional parameter class.Source: §4 · p. 7
• The uniform bound separates sampling error from nuisance-estimation error, retaining a doubly robust product remainder for the MMD numerator.Estimating the variance adds nuisance-error terms; for fixed class constants and confidence level, the sampling term is O(√(log m/m)), faster than the cited O(m^-1/3) rates.Source: §4 · pp. 7–8

Experimental Design

• The evaluation spans mean-preserving medical-image perturbations, generated text in three scenarios, and simulated code-repair and contact-network graphs.Image perturbations represent localized lesions, scattered deposits, or morphological changes.Source: §5 · pp. 8–10

Image Outcomes

• On medical-image constructions, DR-KTE-Deep achieves substantially higher estimated power than the baselines, while DR-KTE-Opt only slightly improves on the fixed kernel.Under H0, rejection rates remain close to α = 0.05, with slight inflation for OCT-DME.Source: §5 · p. 8, Table 1 · p. 9
• For OCT-Drusen, deep-kernel power grows with sample size and effect strength, while Gaussian baselines remain near the nominal rejection rate across the sample-size sweep.Figure 3 varies N at δ = 0.4 and varies δ at N = 1500.Source: Figure 3 · p. 9, §5 · p. 9

Text Outcomes

• On text outcomes, DR-KTE-Deep has the highest estimated power, with substantial gains on Oncology Notes and Clinical Notes and smaller improvements on Movie Reviews.Average H0 rejection rates are close to α = 0.05; power remains higher across sample sizes, with larger gains as δ increases.Source: §5 · p. 9, Table 2 · p. 10, Figure 4 · p. 10

Graph Outcomes

• On both graph tasks, DR-KTE-Deep achieves higher estimated average power, with smaller improvements on Contact Networks; H0 rejection rates range from 0.048 to 0.052.The Gaussian baselines already have high power on both constructions.Source: Table 3 · p. 10, §5 · p. 10

Discussion

• The method learns a deep outcome kernel by maximising an estimated power criterion, with a uniform finite-sample bound for the regularised criterion and conditional asymptotic null validity.The test uses separate samples to learn the kernel and compute the test statistic.Source: §6 · pp. 10–11
• DR-KTE-Deep achieves substantially higher empirical power than fixed and bandwidth-optimised baselines while maintaining Type-I error close to nominal.The learned kernel adapts to the geometry of the outcome space.Source: §6 · p. 11
• The stated scope is two interventions fixed in advance; reusing observations for learning and testing and extending to multiple candidate interventions are left for future work.The paper notes that reuse could further improve power while maintaining Type-I error control.Source: §6 · p. 11

AI Use Statement

• Synthetic text was generated with language models; generative AI also assisted coding, proof strategies, and proofreading, with authors checking the text and verifying code.The authors state they take responsibility for the final content, including AI-assisted artifacts.Source: AI Use Statement · p. 12

Appendix

• The appendix covers notation, theoretical results and proofs, experimental procedures, and additional experiments and ablations.Section C describes training, testing, data constructions, architectures, and hyperparameters.Source: Appendix · p. 18

Data, interventions and samples

• An observation is a covariate-treatment-outcome triplet, and the notation distinguishes nuisance-fitting, validation, and test samples.The dataset is written as the collection of observed triplets.Source: Data, interventions and samples · p. 18

Kernels and interventional mean embeddings

• Outcome kernels map outcomes to RKHS features; their embedding contrast has squared MMD D_θ, with H0 equivalent to D_θ = 0 for characteristic kernels.The notation also defines the intervention-specific mean embeddings and conditional outcome mean embedding.Source: Kernels and interventional mean… · p. 18

Nuisance estimation, kernel learning and testing

• The test uses a doubly robust pseudo-feature whose oracle expectation is the embedding contrast, then forms a normalized statistic and a normal-tail p-value.The fitted pseudo-feature uses estimated logging-policy and conditional-embedding functions.Source: Nuisance estimation, kernel learning… · pp. 18–19

Notation for the theoretical analysis

• The theoretical notation tracks nuisance errors and kernel-class sampling complexity; under the null, the product of weighted nuisance errors bounds selected-kernel score bias.The comparison class also restricts projected variance to be bounded below.Source: Notation for the theoretical analysis · p. 19

B Theoretical Results and Proofs

• The text says it proves the results of Sections 3 and 4, and defines OP(1) as quantities bounded in probability and oP(1) as quantities converging to zero in probability.Source: B Theoretical Results and Proofs · p. 19

Identification

• A measurable response kernel Q is both the observed conditional law of Y given (A, X) and a conditional law of Y^a given X for each fixed action a.The result relies on consistency and conditional independence of treatment and the structural variable given covariates.Source: §B.1 · pp. 19–20

Soft interventions

• Under a soft intervention p, the outcome law is obtained by integrating Q(B | a, x) against p(a | x)ρ(da)P_X(dx).The intervention action is drawn conditionally on X, independently of U given X.Source: §B.1 · pp. 19–20

Identification condition

• The soft-intervention expression is determined by the observational distribution when p assigns no mass where the logging policy π0 is zero, up to null sets.The paper assumes this support condition for both target policies.Source: §B.1 · p. 20

Moment conditions

• The observed-law first-moment condition is needed for the observational conditional mean because target moments may not control actions outside target support.Uniformly bounded kernels satisfy the stated moment conditions.Source: §B.1 · p. 21

B.2 Observed-Data Representation and Double Robustness

• The observed-data representation is doubly robust: its bias is zero if either the propensity nuisance or the outcome-mean nuisance is correct wherever |r| > 0.The result applies to continuous-action soft interventions, but not fixed-value hard interventions; atomic interventions require localization and explicit control of smoothing bias (Colangelo & Lee, 2020; Zenati et al., 2025a; Bhattacharjee et al., 2025).Source: §B.2 · pp. 21–22

B.3 Fixed-Alternative Growth Rate

• For fixed θ with true nuisance functions, the proof establishes convergence of effect and variance estimates to ∆θ(π, π′) and Dθ for the fixed-alternative growth-rate result.It identifies the target variance quantity as Dθ = ⟨∆θ(π, π′), ∆θ(π, π′)⟩.Source: §B.3 · p. 22

B.4 Characteristic Ensemble Safeguard

• The ensemble kernel is characteristic when every base kernel is characteristic and the characteristic-kernel mixture weight has a positive lower bound.The lower bound is strictly positive for unequal distributions and zero when the distributions are equal.Source: §B.4 · p. 23

B.5 Memberwise Ensemble Aggregation

• With positive denominators, the aggregate regularized efficacy is at least the minimum member efficacy; conditional independence also yields variance reduction, equal to M^-1 times member variance for identically distributed members.The efficacy bound requires square-integrable member projections; the variance claim conditions on the learning and validation data.Source: §B.5 · p. 23

B.6 Held-Out Null Validity

• Under held-out regularity and a product nuisance-error rate of oP(m^-1/2), the fitted statistic is asymptotically normal under H0 with asymptotic level α, including for a fixed kernel.The sufficient rate is aπ,test bµ,test = oP(m^-1/2).Source: §B.6 · pp. 24–26

B.7 Uniform Control of the Learning Criterion

• The learning-bound setup conditions on nuisance fitting, uses independent selection halves, and assumes a compact parameter set with uniform kernel bounds and regularity.The assumptions make the empirical and fitted-population criteria continuous and their supremum difference measurable.Source: §B.7 · pp. 27–28

B.7.1 Kernel-Class Regularity

• The kernel-class regularity conditions require bounded oracle pseudo-features and Lipschitz variation of induced observed-data kernels over compact Θ.The induced kernels are jointly measurable and uniformly bounded under these conditions.Source: §B.7.1 · p. 28
• Under boundedness and Lipschitz conditions, the deep-kernel ensemble and induced oracle and fitted observed-data kernels satisfy regularity conditions for uniform bounds.The conditions include a compact parameter set, a uniformly bounded diagonal, and uniform parameter-Lipschitzness for the kernel family.Source: §B.7.1 · pp. 28–30

B.7.2 Oracle Concentration and Adaptation

• Uniform oracle concentration supports variance-stable adaptation when Θ_s is nonempty and r_s,m(δ) ≤ s^2/2.The theorem also gives a stated specialization for fixed B_h, L_h, and s.Source: §B.7.2 · pp. 31, 33

B.7.3 Nuisance Perturbation

• The fitted-score error has an exact three-term decomposition, and conditional-embedding error has no linear contribution to variance perturbation, whereas propensity error need not cancel.If the fitted propensity equals the true propensity on the support of |r|, the population bias term B_θ is zero.Source: §B.7.3 · pp. 34–35

B.7.4 Uniform Criterion Bound and Consequences

• Conditioning on nuisance-fitting data, a covering argument combines oracle concentration and nuisance-perturbation bounds for the fitted criterion, with cyclic-fold control obtained by a union bound.No independence among the three rotation events is required.Source: §B.7.4 · p. 36
• For fixed class constants, the variance-stability margin holds with probability tending to one, supporting approximation of the unregularized criterion; required learner rates are assumed, not proved.The approximation result is stated for a variance-stable class.Source: §B.7.4 · p. 37

B.8 Null Second-Order Structure

• Under the null, the embedding contrast’s second-order expansion induces a one-degenerate kernel for the quadratic form.The conditional expectation of this kernel given either observation is zero almost surely.Source: §B.8 · p. 38

B.9 Held-Out Fixed-Alternative Scaling

• On a fixed alternative, the fitted-score numerator and variance differ from their target quantities by oP(1), and division is stable when the selected kernel lies in Θs.The argument assumes independent held-out test halves and a conditionally integrable fitted score; the zero-denominator event has probability tending to zero.Source: §B.9 · pp. 39–40

C Experimental Details

• The experimental-details section outlines the training and testing procedure, experiment construction, and neural-network architectures and optimisation hyperparameters.Source: C Experimental Details · p. 40

C.1 Training and Testing Procedure

• Each ensemble member optimises a learning criterion across three fold rotations, selects its highest-validation-statistic EMA checkpoint, and contributes to an averaged kernel before nuisance refitting and held-out testing.The test uses 200 fresh test datasets per repetition; results average 20 outer repetitions at α = 0.05.Source: §C.1 · pp. 40–41

C.2 Image Outcomes

• Image experiments apply semi-synthetic perturbations to MedMNIST images, using localised lesions, scattered deposits, or morphological changes to construct treatment-dependent outcomes.Breast-Tumor combines localised and morphological perturbations; Derma-Melanoma distorts and desaturates a mole.Source: §C.2 · pp. 41–42

C.3 Text Outcomes

• Text outcomes are generated by discretising treatment-dependent pseudo-outcomes into tones that condition language-model generation of potential outcome texts.The constructions include oncology notes, movie reviews, and clinical notes.Source: §C.3 · pp. 42–43

C.4 Graph Outcomes

• Graph outcomes include transmission trees from contact networks and LLM repair attempts represented as abstract syntax trees; tree encodings can discard information from the original graph or program.The labelled-tree encoding is injective, but the WL subtree kernel is not generally characteristic and lacks the same omnibus guarantees as Gaussian kernel tests.Source: §C.4 · pp. 43–44

C.5 Architectures and Hyperparameters

• Feature extractors vary by outcome: fully connected networks for scalar, Causal-Blob, and embedded-text inputs; transformers for token sequences; convolutional networks for images; and message-passing networks for graphs.The architectures use four layers for the fully connected and image networks, two transformer layers, and three message-passing layers.Source: §C.5 · p. 44, Figure 5 · p. 44, Figure 6 · p. 45

D.1 Additional Results on Image Outcomes

• On OCT-DME, deep-kernel power is near one across the sample-size sweep, while Gaussian baselines have low power; on Derma-Melanoma, increasing N raises deep-kernel power more than baseline power.Figures 9 and 10 show power trends as sample size or effect strength varies.Source: §D.1 · p. 45, Figure 9 · p. 45, Figure 10 · p. 46
• Type-I error ranges from 0.045 to 0.054, and DR-KTE-Deep has higher estimated power than baselines in every MNIST, RetinaMNIST, and BUSI setting, with its largest improvement on RetinaMNIST Scenario III.Table 6 reports the comparisons at N = 1500 and N = 3000.Source: §D.1 · p. 46, Table 6 · p. 47
• For Fake-MNIST, DR-KTE-Deep exceeds all baselines in power at both sample sizes, DR-KTE-Opt slightly improves on DR-KTE, and Type-I error ranges from 0.050 to 0.054.Table 7 reports N = 1500 and N = 3000.Source: Table 7 · p. 48, §D.1 · p. 48

D.2 Causal-Blob

• On Causal-Blob, DR-KTE-Deep has the highest power at both sample sizes, with its largest gain over DR-KTE-Opt at N = 3000, while Type-I error remains near nominal for all three tests.The alternative distributions share a mean but differ in higher-order moments.Source: §D.2 · p. 48, Table 8 · p. 49

D.3 Scalar Outcomes

• For scalar outcomes, power is close to one in Scenarios II and IV, while Scenario III is more challenging and improves substantially with sample size; DR-KTE-Opt has the highest estimated power in Scenario III.Kernel-test Type-I error is close to nominal at both sample sizes.Source: Table 9 · p. 49, §D.3 · p. 49

D.4 Estimator Comparison

• DR-KTE-Deep has higher estimated power than IPW-Cross and KPT in all three alternative scenarios, with the largest gains in Scenario III; its null rejection rate is near α = 0.05 while both comparators have higher Type-I error.Table 10 reports the estimator comparison at N = 1600.Source: Table 10 · p. 50, §D.4 · p. 50

• Using the fitted outcome embedding improves power on Causal-Blob and scalar Scenarios II and III, with the largest gain in Scenario III, while both estimators have near-nominal Type-I error and power close to one in Scenario IV.Table 11 reports means and standard errors at N = 1500.Source: §D.5 · p. 50, Table 11 · p. 51

• DR-KTE-Deep has the highest power on OCT-Drusen and Fake-MNIST under H1, while Type-I error remains near nominal for all compared methods.Table 12 reports power under H1 and Type-I error under H0 at N = 1500.Source: Table 12 · p. 51, §D.6 · p. 51

D.7 Text Representation Ablation

• All three methods achieve high, similar power with text embeddings; Type-I error is near nominal for Oncology Notes and Clinical Notes but conservative on Movie Reviews.The representation uses Qwen3-Embedding-0.6B vectors with a Gaussian kernel.Source: §D.7 · p. 51, Table 13 · p. 52

D.8 Effect of Ensemble Size

• On Causal-Blob, average power rises from 0.350 to 0.394 as ensemble size increases, but variability does not decrease consistently.On OCT-Drusen, power rises slightly and standard deviation is lower at M = 5 than at M = 1.Source: §D.8 · p. 52, Table 14 · p. 52, Figure 13 · p. 53

D.9 Weisfeiler-Lehman Outcome Kernel

• With Weisfeiler-Lehman kernels, learning the outcome kernel raises power from 0.743 to 0.964 on LLM Code Repairs; Type-I error remains near nominal for both methods.The two variants have similar power on Contact Networks, with a slightly higher average for the learned kernel.Source: §D.9 · p. 53, Table 15 · p. 53

D.10 Linear Outcome Kernel

• For mean-matched distributional effects, DR-KTE-Deep has high power while DR-ATE rejects near the nominal rate; with mean-shift alternatives, DR-KTE-Deep has higher rejection rates across the remaining experiments.The comparisons include medical-image, scalar, text, and graph outcomes.Source: §D.10 · p. 53, Table 16 · p. 54, Table 17 · p. 54

D.11 Scaling with Outcome Dimension

• As outcome dimension grows, DR-KTE-Deep has substantially higher power through d = 8, but all four methods are near their null rejection rates at d = 16 and d = 32.Type-I error is above nominal for all four methods.Source: §D.11 · p. 55, Figure 14 · p. 56

D.12 Effect of Joint Maximisation

• Joint maximisation generally yields similar or lower estimated power than DR-KTE-Deep while improving on DR-KTE; OCT-Drusen is the exception, with higher power under joint maximisation.Type-I error remains close to nominal, and joint selection uses one ensemble checkpoint rather than each member’s best validation checkpoint.Source: §D.12 · p. 56, Table 22 · p. 57