PaperlayerView exact revision on arXiv

Source-linked AI summary

Two-Sample Testing via Path-based Inference

Eshant English, Wei-Cheng Lai, Yanfeng Yang, Kenji Fukumizu, Taiji Suzuki, Christoph Lippert

arXiv:2610.05684v1stat.MLcs.AIcs.LG

TL;DR

Two-sample testing asks whether two finite datasets came from the same distribution, but existing methods can struggle with structured, high-dimensional data and low sample counts. The paper tests symmetry between the two halves of a Gaussian-bottleneck path using held-out regression risks and permutation calibration. The tests improve power over the strongest baseline by up to 33 percentage points at an equal total sample budget.

  • Problem

    Two-sample testing must distinguish whether finite datasets came from the same distribution, while existing methods can struggle with structured, high-dimensional data in low-sample settings.

    Source: §1 · p. 4

  • Method

    The method connects both distributions to a shared Gaussian bottleneck and compares matched fields using held-out regression-risk contrasts along the path.

    Source: §3 · p. 7, §3.4 · p. 11

  • Results

    The tests are valid in finite samples for any trained networks and improve power over the strongest baseline by up to 33 percentage points at an equal total sample budget.

    Source: §1 · p. 5, §3.6 · p. 17, §7 · p. 38

  • Takeaways & Limitations

    Generative dynamics can serve as objects of statistical inference, not only as tools for generation.

    Source: §7 · p. 38

    The approach requires splitting each dataset into training and testing sets to learn the fields; testing symmetry on generated midpoint samples is not calibrated unless modeling error is negligible.

    Source: §7 · p. 38

Abstract

from arXiv · show

Modern deep generative models are primarily studied for their ability to generate realistic samples, yet the generative dynamics they learn can also serve as objects of statistical inference. We develop this idea for two-sample testing, the problem of deciding whether the same distribution generated two finite datasets. Using stochastic interpolants, we connect both distributions to a shared Gaussian bottleneck, so that each half of the resulting path is a Gaussian channel acting on a single population. We prove that the null hypothesis holds if and only if the population denoiser, or equivalently, the velocity fields of the two halves, coincide at any single noise level, which amounts to a reflection symmetry of the path about the bottleneck. Deviations from this symmetry yield a continuum of two-sample witnesses, which we estimate via held-out regression risks on learned denoisers and velocities and aggregate along the path; under an information-theoretic weighting, the aggregated discrepancy equals the Jeffreys divergence between the noise-smoothed distributions. Calibrating the resulting statistics by permutation yields tests that are valid in finite samples for any trained networks and consistent when the fields are learned accurately. On a synthetic benchmark and three image benchmarks, the proposed tests improve power over the strongest baseline by up to 33 percentage points at an equal total sample budget, with the best choice of regression representation and path weighting depending on the data modality. These results show that generative paths provide a principled representation for statistical testing, extending stochastic-interpolant models beyond generation.

1 Introduction

Although generative models are widely used for synthesis, their dynamics remain relatively unexplored as tools for statistical inference. This section motivates testing the full path between distributions and presents a Gaussian-bottleneck symmetry test with finite-sample calibration and benchmark gains.

1 Introduction

  • Dynamical-transport models learn a field that generates a continuous process between distributions and have supported applications in vision, audio, and computational biology.The paper positions inference as an underexplored use of these generative dynamics.Source: §1 · p. 3
  • The paper treats the full deterministic path as a statistical object, with the shared Gaussian-bottleneck path symmetric about its midpoint under the null.This exact symmetry is presented as the path-based counterpart to the intuition that identical distributions should produce no transport.Source: §1 · p. 3
  • Paths can differ substantially even when endpoint maps look similar, and endpoint similarity can reflect modeling bias rather than distributional closeness.Path-based inference therefore uses the process connecting the datasets rather than only its endpoints.Source: §1 · pp. 3–4
  • Two-sample methods still struggle with structured, high-dimensional data, especially in low-sample settings, and may also need to identify observations driving a rejection.The paper notes applications including distribution-shift detection, generated-data checking, and demographic-group comparisons.Source: §1 · p. 4
  • The proposed tests improve power over the strongest baseline by up to 33 percentage points at an equal total sample budget.The paper also develops held-out risk statistics, relates an information-theoretic weighting to Jeffreys divergence, and proves finite-sample validity under permutation calibration.Source: §1 · p. 5

2 Preliminaries

The preliminaries define stochastic interpolants as paths built from endpoint samples and Gaussian latent noise, with deterministic probability-flow dynamics and regression objectives. They then formalize two-sample testing as a finite-data test of equality between unknown distributions.

2.1 Stochastic Interpolants

  • The interpolant x_t = α_t x_0 + β_t x_1 + γ_t z joins endpoint samples, while the boundary conditions recover x_0 and x_1 and latent noise smooths intermediate distributions.Setting γ_t to zero recovers deterministic interpolants underlying flow matching and rectified flow.Source: §2.1 · p. 5
  • The probability-flow ODE transports the interpolant marginals deterministically, mapping each initial condition to a unique trajectory.The marginal velocity is learned through a regression objective, and network objectives are estimated by Monte Carlo sampling.Source: §2.1 · pp. 5–6
  • Denoisers are conditional expectations of the endpoints and latent given the current state, and the velocity is a linear combination of these denoisers.For Gaussian latent noise, the latent denoiser also determines the score of the interpolant density.Source: §2.1 · p. 6

2.2 Two-Sample Testing

  • Two-sample testing asks whether two finite datasets drawn from unknown densities satisfy H0: ρ0 = ρ1 or the alternative H1: ρ0 ≠ ρ1.A test is a binary decision rule that may use internal randomness, including data splitting and permutations.Source: §2.2 · p. 7
  • The goal is to control the type I error at a prescribed level while maximizing power against alternatives.The validity guarantee is non-asymptotic at finite sample sizes, whereas power is analyzed as sample sizes grow.Source: §2.2 · p. 7

3 Path-Based Two-Sample Testing

The method routes each distribution through a shared Gaussian bottleneck and compares the regression fields on the two mirrored path halves. Their denoiser, velocity, and score contrasts encode the same population difference up to known positive scaling.

3 Path-Based Two-Sample Testing

  • Under the null, matched fields on the two path halves coincide; under the alternative, their differences provide witnesses indexed by position along the path.The construction compares fields along the path rather than comparing only the endpoint distributions.Source: §3 · p. 7

3.1 A Gaussian-Bottleneck Interpolant

  • Each half of the interpolant is a Gaussian channel acting on one population, and both meet at a shared Gaussian midpoint independent of the coupling.The channel law depends on the marginal distribution alone, and the midpoint density is the common pivot.Source: §3.1 · p. 8
  • The mirrored schedule creates two outward paths from the pivot, with channel slices recovering the data distributions as the radius approaches the endpoint.The construction uses the same latent coefficient on both halves.Source: §3.1 · p. 8

3.2 Denoiser and Velocity Fields

  • Denoiser, velocity, and score contrasts encode the same population difference up to known positive scalar functions of the path position.The common affine term in the velocity representation cancels when taking the contrast between channels.Source: §3.2 · p. 10

3.3 Path Invariance Characterises the Null

The null is equivalent to agreement between the two path halves’ regression fields at a matched noise level, and held-out risk contrasts turn their disagreement into two-sample witnesses.

3.3 Path Invariance Characterises the Null

  • At any fixed matched radius, the null is equivalent to equality of the two channels’ denoisers or velocities; under the alternative, their fields differ at every matched radius.The corresponding interpolant is reflection-symmetric about the bottleneck under the null.Source: §3.3 · pp. 10–11

3.4 Held-Out Risk Witnesses

  • The two risk contrasts provide scalar held-out witnesses by comparing regression fields from both channels on observations from each dataset.The witness does not receive the observation’s group label.Source: §3.4 · p. 11
  • With population fields, the held-out risk contrasts vanish if and only if the null holds.The risk identities follow from orthogonality of regression residuals.Source: §3.4 · p. 12
  • For denoising, the risk contrast is the squared-error change from adding the other channel’s denoiser disagreement to the matching channel’s residual.The residual is orthogonal to functions of the corrupted observation, which removes the cross term in expectation.Source: §3.4 · p. 12

3.5 Path Aggregation

Path aggregation combines local contrasts across a truncated range of radii. With Jeffreys weights, the population target equals the Jeffreys divergence between the noise-smoothed channels over that range.

3.5 Path Aggregation

  • The local contrasts are aggregated over Iτ = [τ, 1/2 − τ] using a positive weight; uniform weighting assigns equal measure to retained radii.The Jeffreys scheme uses information-theoretic weights.Source: §3.5 · pp. 12–13
  • The empirical statistic estimates the path target using differences of held-out group means, with Monte Carlo sampling along the path.The stated result uses held-out subsets from both datasets.Source: §3.5 · p. 13
  • The population target of the aggregated statistic is positive exactly when the null fails, under the stated sampling and integrability conditions.The sampling density must be strictly positive on Iτ.Source: §3.5 · p. 13
  • Jeffreys-weighted aggregation targets the Jeffreys divergence between the two channels, truncated to the retained interval of radii.As the truncation approaches the bottleneck, the smoothed-channel divergence converges to the Jeffreys divergence between the original populations.Source: §3.5 · pp. 14–15

3.6 Held-Out Permutation Calibration

Held-out permutation calibration gives finite-sample validity without requiring accurate learned fields. Consistency under alternatives requires suitable field-error control and increasing held-out sample sizes.

3.6 Held-Out Permutation Calibration

  • The test statistic compares held-out group means of scores computed from fields trained on a separate data split.The pooled held-out scores are randomly relabelled while preserving group sizes to form permutation statistics.Source: §3.6 · pp. 15–16
  • Under the null, permutation calibration controls the rejection probability at level κ for any fitted fields, without moment assumptions or field-accuracy requirements.The stated validity conditions include independent held-out data, Monte Carlo draws, and permutations.Source: §3.6 · pp. 16–17

3.7 Consistency

  • Condition (a) makes precise the role of cross errors: they measure how well each learned field extrapolates to the other channel, where the witness also evaluates it.Near s = 1/2, the channels separate and cross errors are hardest to control; this motivates truncation τ and uniform weighting.Source: §3.7 · pp. 17, 19
  • Power can persist when each learned model fits its own population better than the other, a weaker requirement than either model being consistent.The risk contrast identity applies to learned fields.Source: §3.7 · p. 17
  • Under the consistency proposition’s conditions, the test is consistent when the alternative holds and integrated learned-field errors vanish as held-out sample sizes grow.The conditions also require positive sampling density on Iτ and finite population targets.Source: §3.7 · pp. 18–20

3.8 Relationship between the Four Variants

The four PBI variants pair denoising or velocity regression with uniform or Jeffreys path weighting. Their population targets and finite-sample behavior depend differently on those choices.

3.8 Relationship between the Four Variants

  • The method has four variants: denoising or velocity regression, each combined with uniform or Jeffreys path weighting.The variants share the same procedure, with the representation and weight varied.Source: §3.8 · p. 20, Table 1 · p. 21
  • The two Jeffreys-weighted variants share the same population target: the truncated Jeffreys divergence.Finite-sample behavior can still differ because denoiser and velocity regression have different noise and approximation properties.Source: §3.8 · p. 20, Table 1 · p. 21
  • Uniform weighting is an alternative when Jeffreys weights amplify the difficult low-noise region near s = 1/2.Uniform variants omit the information-theoretic reweighting.Source: §3.8 · p. 20

3.9 Monte Carlo Approximation over the Path

The path integral is estimated by Monte Carlo using held-out observations, sampled noise levels, and Gaussian perturbations. Multiple corrupted copies of each observation are averaged as one statistical unit before testing.

3.9 Monte Carlo Approximation over the Path

  • For each held-out observation, draw K noise levels from a density positive on the integration interval and independent Gaussian noise, then form corrupted inputs and regression targets.The estimate uses importance weights w/μ so nonuniform time sampling leaves the population target unchanged.Source: §3.9 · pp. 21–22
  • Fit each population’s field on its own training split, without validation or early stopping, and return the network after the fixed iteration budget.The field estimates either the denoiser or outward velocity, depending on the selected representation.Source: §3.9 · p. 22
  • Average the K corrupted-copy contributions for each observation before testing; increasing K reduces only Monte Carlo variance.Uniform time sampling uses constant μ, while the K copies remain one statistical unit.Source: §3.9 · pp. 22–23

4 Experiments

The experiments compare four path-based inference variants with established two-sample tests under an equal total observation budget. Across the synthetic and image benchmarks, power and the preferred representation and path weighting vary by dataset; the PBI variants also remain near nominal Type-I error, while architecture comparisons are descriptive.

4.1 High-Dimensional Gaussian Mixtures

  • On HDGM, PBI-Velocity J gains 22.8 pp over C2ST-L at n = 100, while PBI-Velocity U gains 13.2 pp over AutoTST at n = 200.Jeffreys weighting raises denoiser power at each sample size, and PBI-Denoiser J exceeds the strongest baseline throughout.Source: §4.1 · pp. 24–25, Table 2 · p. 25

4.2 MNIST

  • On MNIST, all four PBI variants exceed AutoTST from n = 150 onward; PBI-Velocity U gains 25.2 pp at n = 200.Uniform weighting is more effective than Jeffreys weighting, particularly for the velocity representation.Source: Table 4 · p. 26, §4.2 · p. 26

4.3 OrganAMNIST

  • On OrganAMNIST, every PBI variant exceeds AutoTST at each sample size; PBI-Denoiser J gains 33.2 pp at n = 200, and all variants reach 1.000 power at n = 350 and 420.The lower-sample regime favors the denoiser, while both parameterisations saturate once n ≥350.Source: §4.3 · p. 27, Table 6 · p. 28

4.4 Fashion-MNIST

  • On Fashion-MNIST, PBI-Denoiser U is the only variant to beat MMDAgg at every sample size, with a 23.2 pp gain at n = 100.Both velocity variants trail MMDAgg across sample sizes and show greater seed-to-seed variability; the benchmark favors denoiser representation with uniform weighting.Source: §4.4 · pp. 27–28, Table 8 · p. 29

4.5 Type-I Error

  • All four PBI variants stay close to the nominal 0.05 Type-I error level across all four benchmarks and sample sizes.The reported deviations are comparable to variation across training seeds.Source: §4.5 · p. 28, Figure 3 · p. 30

4.6 Samples from the Fitted Models

  • Figures 4–6 compare held-out observations with samples from separately trained denoiser and velocity models at the smallest image-benchmark sample sizes.The samples are visibly noisy, although the corresponding tests attain the reported powers at those sample sizes.Source: §4.6 · p. 29, Figure 4 · p. 31, Figure 5 · p. 32

4.7 Ablation on Model Architecture

  • In the architecture ablation, uniform weighting beats Jeffreys weighting for every matched setting, but the preferred architecture varies; these comparisons are descriptive, not significance tests.On MNIST, Separate models reach 0.820 for PBI-Denoiser U; on Fashion-MNIST, Restricted Common Contrast reaches 0.952 and 0.916 for PBI-Velocity U and J, respectively.Source: §4.7 · pp. 31–34

5 Observation-Level Asymmetry under Corruption

Observation-level asymmetry scores concentrate on corrupted observations in a controlled mixture, while localization is weaker for corruption overlapping an existing mode. The scores support distribution-level testing and ranking, but not per-observation error guarantees.

Experimental setup

  • The two-dimensional Figure 1 setup uses an equal-weight three-Gaussian reference; each held-out sample contains 500 observations per group, about 50 Dte1 observations are corrupted, and labels are evaluation-only.Source: §5 · p. 34

Score construction

  • For each held-out observation, the asymmetry score contrasts the two channel denoisers’ reconstruction performance, using exact or fitted posterior-mean denoisers and uniform or Jeffreys weights.Fitted Gaussian-mixture fields can miss the corruption when the selected mixture does not resolve it.Source: §5 · p. 34

Results

  • Corrupted observations score 20–40 times higher; new-cluster and shifted-mode corruptions reach AUROC 0.98–1.00, while deformed-mode tails alone are localized (P@k ≈0.4).The deformed mode shares its center with an uncorrupted mode, making nearby corrupted observations individually indistinguishable; the global test is also less powerful for this corruption.Source: §5 · p. 35

Geometric comparison

  • The geometric mirror mismatch orders corruptions similarly but is less discriminative than the regression-risk score and requires two ODE solves per observation.Under the null, reflection symmetry implies the mirror endpoint equals the original observation.Source: §5 · pp. 35–36

Scope

  • The scores rank observations but provide no per-observation error control; guarantees for individual flags require an additional multiple-testing or conformal procedure.Source: §5 · p. 36

6 Related Work

Prior work uses learned statistics, score comparisons, and generative paths for inference-related tasks, but the paper distinguishes its path-based test by its witness construction and finite-sample permutation calibration. Its closest comparison is midpoint inference from flow models, which uses a different field and training setup.

Two-sample tests with learned statistics

  • Unlike learned discriminative witnesses in sample-splitting tests, this method obtains a witness from two generative arcs fitted to single populations and tests its held-out mean difference [Kübler et al., 2022b].The observation-level scores also localize discrepancies in data space, complementing interpretable test locations [Chwialkowski et al., 2015; Jitkrittum et al., 2016].Source: §6 · p. 36

Scores, denoisers and divergences

  • Theorem 7 identifies the Jeffreys-weighted statistic’s population target through the symmetrized mismatched-estimation identity for Gaussian-smoothed distributions [Guo et al., 2004; Verdú, 2010].Source: §6 · pp. 36–37

Generative models as tools for inference

Midpoint inference from flow models

  • Unlike midpoint zero-flow tests, the Gaussian-bottleneck test compares fields from the two path halves across noise levels, enabling single-population diffusion training and finite-sample permutation calibration [Wang et al., 2026].Source: §6 · p. 37

7 Discussion and Limitations

The discussion positions learned transport dynamics as inference objects and reports gains over the strongest baseline, while noting costs and limits tied to field learning and variant choice.

7 Discussion and Limitations

  • Across four benchmarks, the tests improve on the strongest baseline by up to 33 percentage points at an equal total sample budget.The discussion also reports that the tests match or exceed the strongest baseline at the smallest sample size of every image benchmark.Source: §7 · pp. 38–39
  • Learning the fields requires splitting each dataset into training and testing sets, while midpoint checks on generated samples are calibrated only when modeling error is negligible.The authors also report that learned-path inferences agree across training and held-out data, but not with newly sampled data from the same model.Source: §7 · p. 38
  • Power depends on field accuracy and representational capacity; shared misspecified fits can leave the test powerless even when H1 holds.The reported experiments use off-the-shelf architectures and settings without hyperparameter search, and the authors note that better-trained models should yield more power.Source: §7 · p. 39
  • No representation-and-weighting combination dominates across benchmarks, so variants must be fixed in advance or combined through valid aggregation.Aggregated tests and pretrained-model use remain unevaluated future work.Source: §7 · p. 39

Appendix A. Experimental Details and Notations

The appendix collects notation, model backbones, optimization budgets, and implementation choices for the baselines.

Appendix A. Experimental Details and Notations

  • Table 13 summarizes the notation used throughout the paper.The appendix identifies Table 13 as the notation summary.Source: Table 13 · p. 40, Appendix A · p. 40
  • Vector-valued HDGM observations use an FC-MLP with four 128-wide SiLU hidden layers, while image observations use a three-level U-Net.The U-Net uses c = 16 for MNIST and Fashion-MNIST and c = 32 for OrganAMNIST; experiments use separate networks for ρ0 and ρ1.Source: Appendix A · p. 40
  • Recorded optimization steps vary by benchmark and sample size: HDGM uses 1,300, 1,300, and 2,600 steps for n = 500, 1,000, and 2,000, respectively.MNIST and OrganAMNIST use 1,600 steps at each sample size, while Fashion-MNIST uses 2,500.Source: Appendix A · p. 40
  • Baseline implementations use official MMDAgg and AutoTST code, alongside adapted MMD-D, MMD-O, C2ST-L, and C2ST-S implementations.C2ST-S uses hard predictions and C2ST-L probabilities; MMD-O optimizes a Gaussian-kernel bandwidth, while MMD-D learns neural features and a deep kernel.Source: Appendix A · p. 41

Appendix B. Per-Benchmark Power and Type-I Error Curves

The appendix separates per-benchmark power and empirical Type-I error curves for HDGM and three image datasets, with numerical power values provided in Section 4 tables.

Appendix B. Per-Benchmark Power and Type-I Error Curves

  • The appendix presents separate curves for each benchmark, with power in Figures 8, 10, 12, and 14 and Type-I error in Figures 9, 11, 13, and 15.Numerical power values are in the tables of Section 4.Source: Appendix B · p. 41
  • Power curves show all ten methods individually for HDGM, MNIST, OrganAMNIST, and Fashion-MNIST as per-population sample size increases.The captions give κ = 0.05 and note that condensed versions appear in Figure 2.Source: Figure 8 · p. 42, Figure 10 · p. 44, Figure 12 · p. 46
  • Empirical Type-I error curves for all four benchmarks use samples drawn from ρ0 and show the PBI variants close to nominal κ = 0.05.The captions state that this calibration is guaranteed by Theorem 9.Source: Figure 9 · p. 43, Figure 11 · p. 45, Figure 13 · p. 47

Related papers

Loading 2610.05684v1…