Change Detection in Probability Flow ODE: Online Testing in Diffusion Latent Spaces

Artem Kraevskiy; Artem Prokhorov

Source-linked AI summary

Change Detection in Probability Flow ODE: Online Testing in Diffusion Latent Spaces

Artem Kraevskiy, Artem Prokhorov

arXiv:2608.22807v2Aug 24, 2026cs.LGstat.ML

TL;DR

The paper addresses sequential change-point detection when pre- and post-change conditional densities are unknown and lack closed forms. It maps observations through a frozen conditional PF-ODE encoder, tests Gaussian latent windows with MMD, and applies a calibrated Shiryaev-Roberts procedure. The method detects synthetic shifts within 15-20 windows and detects a real MNIST 0 →1 change in four windows, while requiring a well-converged null encoder.

Problem
Unknown conditional densities make likelihood-ratio change-point detection intractable, while high-dimensional density estimation suffers from the curse of dimensionality.
Method
A frozen conditional PF-ODE encoder maps pre-change data to N(0, I), after which MMD and a Tartakovsky-Spivak mixture Shiryaev-Roberts detector operate on latent windows.
Results
The method detected all three synthetic shifts within 15-20 windows and detected a 0 →1 MNIST digit-class change in four windows.
Takeaways & Limitations
The method supports sequential detection for unknown high-dimensional conditional distributions and can detect covariance and higher-order structural shifts without parametric regime assumptions.
Takeaways & Limitations
The denoiser must converge to a genuinely clean null with Z ∼ N(0, I), which the authors identify as the binding practical constraint.

Abstract

from arXiv · show

A rapidly growing range of sequential data tasks, such as identifying trend reversals in financial markets, auto-segmenting video and audio recordings, detecting changes in movement direction from motion sensors cannot be fully addressed without detection of distributional shifts in time-ordered data. We consider a sequential change-point detection problem where the conditional density switches at an unknown time, yet neither the pre- nor post-change distribution admits a closed-form. Classical likelihood-ratio statistics are inapplicable in this settings. A conditional diffusion model, trained on pre-change-point data with a frozen context encoder, defines a deterministic bijection via the probability flow ODE. Pre-change observations are mapped onto standard Gaussian latent variables. Post-change observations, processed through the same frozen map, deviate from this reference. We employ the Maximum Mean Discrepancy as the test statistic, derive closed-form expressions for its components under the Gaussian null, and establish its asymptotic distribution as a degenerate U-statistic. Afterwards we apply an online detection procedure of Shiryaev--Roberts to the resulting statistic with exact threshold calibration. The method detects arbitrary distributional shifts, including covariance rotations and higher-order structural breaks, without parametric assumptions on either regime.

1 Problem Formulation

The paper formulates sequential change-point detection when conditional densities switch at an unknown time but neither regime has a known analytic form. It addresses likelihood-ratio intractability and high-dimensional distributional complexity by encoding observations into a Gaussian latent space.

An unobservable geometrically distributed change-point switches the conditional density from p0(· | h) to p1(· | h).
Classical Shiryaev-Roberts detection requires a conditional log-likelihood ratio that is intractable when p0 and p1 are unknown.
High-dimensional density estimation is further hindered by the curse of dimensionality and domain-specific features such as heavy tails, tail dependence, and image sparsity.
A trained conditional PF-ODE model deterministically maps a complex unknown multivariate distribution to N(0, I) and back, using a frozen history context.
The paper derives MMD against a Gaussian alternative, applies Shiryaev-Roberts threshold selection, and evaluates the procedure on synthetic and real-world data.

2 Maximum Mean Discrepancy

The MMD framework compares encoded observations with the exact Gaussian reference using kernel mean embeddings. Under the null, the unbiased statistic is degenerate with a weighted generalised-χ2 limit; under alternatives, it has ordinary √w-rate Gaussian behavior.

MMD is the RKHS distance between the kernel mean embeddings of two distributions, and characteristic kernels make it zero exactly when the distributions match.
The method estimates MMD2 between a window of encoded observations and N(0, I) by combining empirical kernel expectations with the exact Gaussian reference.
The RBF kernel is characteristic on R^d for every σ > 0, allowing detection of arbitrary distributional differences in the encoded space.
Only the pairwise Term A costs O(w^2d), while Terms B and C cost O(wd) and O(1), respectively.
Under H0, the unbiased MMD estimator is a first-order degenerate U-statistic, so the usual √w term vanishes and normalization by w yields a weighted sum of centred χ2 variables.
Under H1, degeneracy generally breaks down and the estimator has √w-rate asymptotic normality, while its mean grows linearly in w against the null’s Op(1) behavior.

3 Shiryaev-Roberts Statistic for MMD-Based Change-Point Detection

The method converts windowed MMD statistics into an online Shiryaev-Roberts detector using a mixture likelihood ratio and calibrated thresholds. It supports both Bayes-risk optimization and false-alarm-constrained asymptotic minimax calibration.

The classical Shiryaev-Roberts statistic recursively aggregates likelihood-ratio evidence for a change occurring at any past time.
Non-overlapping encoded windows produce i.i.d. statistics within each regime, with a finite-sample null density estimated by Monte Carlo under N(0, I).
The detector replaces the unavailable likelihood ratio with a Tartakovsky-Spivak mixture over the unknown signal strength δ2.
3.2 The Shiryaev-Roberts recursion for mixture MMD likelihood ratio.: The hyperparameter α corresponds to a minimum detectable effect through Eπ[δ2] = 1/α, while plug-in calibration provides an alternative selection strategy.
Poorly chosen mixture hyperparameters slow detection but do not affect the false-alarm rate governed by threshold calibration.
3.4 Threshold calibration: Bayes-risk calibration yields a threshold exactly optimal for a specified delay–false-alarm cost trade-off.
3.4 Threshold calibration: ARL-constrained calibration achieves the desired false-alarm rate and is asymptotically minimax-optimal as γ →∞.
The complete online procedure combines the frozen PF-ODE encoder, closed-form RBF MMD2, mixture likelihood ratio, and calibrated SR threshold.

4 Experiments on synthetic data

Synthetic experiments evaluate three two-dimensional distributional shifts using trained conditional diffusion detectors and calibrated online monitoring. The method detects all tested shifts within a narrow delay band despite substantially different signal strengths.

The experiments use three synthetic p0 → p1 pairs in R2, including a thin-ring redistribution, a 90° Gaussian-mixture rotation, and a four-cluster collapse.
Each detector is trained with a GRU history encoder and conditional denoiser, then run on a fresh length-200 series with a change at τ = 100.
Figure 1 tracks example pre- and post-change draws, rolling MMD2, and the Shiryaev-Roberts statistic against its calibrated threshold.
The sliding-window visualization uses autocorrelated statistics rather than the formally specified independent non-overlapping windows, while calibration remains internally consistent.
All three delays fall within 15-20 steps despite an almost fourfold spread in estimated signal strength ˆδ2.
The SR statistic reacts exponentially fast after evidence accumulates, so signal strength mainly determines when escalation begins rather than its subsequent rate.

5 A Real-Data Illustration: MNIST Digit 0 →1

The unchanged detector was applied to MNIST 0→1 images in a 256-dimensional observation space, where it detected the real-data change after four windows. The result suggests detection speed tracked latent signal strength rather than observation dimension.

Setup: MNIST digits were down-sampled to 16×16, with digit “0” as p0 and digit “1” as p1, using the same conditional architecture with an added MLP frame embedding.The experiment kept the detector unchanged while adapting the history encoder and denoiser to images.
Setup: The MNIST configuration used w = 20 and σ = 16, with a threshold calibrated for PFA ≤5% over a 105-window horizon.Calibration followed Approach B, matching the synthetic experiments.
Detection result: The detector fired at ˆτ = 66 for a change at τ = 62, producing a delay of four windows.The test series had length 125 and used a genuine change-point at τ = 62.
Detection result: The estimated signal strength was ˆδ2 ≈ 0.160, slightly better than the strongest synthetic pair, despite the higher-dimensional observation space.The reported result is consistent with detection speed tracking ˆδ2 rather than d at fixed w.
Interpretation: A well-separated 256-dimensional digit shift can be easier to detect than a two-dimensional shift because the encoder can preserve signal without ˆδ2 shrinking with d.The passage distinguishes density-estimation difficulty from the signal carried by a well-trained encoder.

6 Comparison with other methods

The comparison evaluates the PF-ODE–MMD–Shiryaev–Roberts pipeline against three baselines under a shared false-alarm budget. It remains well calibrated on synthetic shifts but fails to hold the budget on MNIST because the recovered latent null is not cleanly Gaussian.

Baselines: The study compares the proposed PF-ODE encoder and MMD statistic with raw-observation MMD, a PF-ODE latent mean-embedding statistic, and Hotelling’s T 2 with CUSUM.All methods use only the burn-in segment as known-p0 data, without oracle access to the true generating distribution.
Evaluation: The evaluation uses 1,000 independent Monte Carlo trials per dataset and reports false-alarm probability, miss probability, and mean detection delay at PFA ≤5%.Only the proposed method and method B have thresholds provably calibrated to that budget.
Synthetic comparison: On the three synthetic pairs, the proposed method observed false-alarm rates of 1.8–2.9%, while baselines reached 24–80%.The proposed method was the only one that stayed near the 5% design budget.
MNIST comparison: MNIST was the only exception: the proposed method substantially overshot the 5% budget because recovered latent codes had mean per-dimension variance 0.845 rather than 1.EDICT produced only a slight improvement and did not resolve the calibration problem.

7 Conclusion and Future Directions

The paper combines an injective conditional PF-ODE encoder, MMD-based scalar window statistics, and calibrated mixture Shiryaev–Roberts detection for unknown conditional densities. It reports fast detection across synthetic and MNIST shifts while identifying encoder convergence, kernel choice, and repeated monitoring as important boundaries.

Contribution: The method addresses sequential change detection when neither pre- nor post-change conditional density has a closed-form.Its construction uses a conditional PF-ODE model, MMD against a Gaussian reference, and a mixture Shiryaev–Roberts decision rule.
Method: The injective PF-ODE encoder maps the unknown data distribution to N(0, I), while MMD2 reduces each window to a scalar that is degenerate under H0 and √w-Gaussian under H1.The Shiryaev–Roberts mixture and Pollak calibration turn these statistics into a sequential decision rule.
Results: Across three synthetic shifts, the method detected all changes within 15–20 windows despite an almost fourfold spread in signal strength ˆδ2.On unchanged MNIST data, the 0→1 change was detected in four windows.
Scope: The method is intended for sequential data with unknown high-dimensional conditional distributions, including financial, audiovisual, motion-sensor, and self-driving telemetry applications.The stated scope includes settings where heavy tails or high dimensionality make closed-form modeling impractical.
Limitations: A fixed-bandwidth RBF kernel may limit detectability, and extending the single-change-point algorithm to formally guaranteed continuous monitoring remains open.The paper suggests adaptive or learned kernels and restart-based monitoring as future directions.

B Derivation of Terms B and C for the RBF Kernel

The paper derives Gaussian closed forms for the RBF-kernel MMD components and centers the estimator to obtain an unbiased, first-order degenerate U-statistic under the null. Its rescaled statistic converges to a generalized χ2 limit, while alternatives restore √w-rate normal behavior.

Gaussian kernel expectations: The RBF kernel factorizes across independent Gaussian coordinates, reducing multidimensional expectations to products of one-dimensional Gaussian integrals.The derivation uses Y − Y′ ∼ N(0, 2I) and completes the square for the cross term.
Gaussian kernel expectations: The one-dimensional Gaussian integral yields the closed form used to evaluate the RBF-kernel MMD terms under Q = N(0, I).The paper applies the resulting coordinatewise expression to the multidimensional kernel expectations.
Centered MMD estimator: Removing diagonal terms eliminates the O(1/w) bias of the diagonal-included V-statistic, producing an unbiased degree-2 U-statistic.The unbiased construction uses a kernel centered with respect to Q.
Null asymptotics: Under H0, the centered kernel is first-order degenerate, so the usual √w Gaussian term vanishes and the correct normalization is w.The statistic has exactly zero mean under H0 for every window size w.
Null asymptotics: The rescaled null statistic converges to a generalized χ2 distribution, an infinite weighted sum of centered χ2(1) variables without a closed-form CDF.For the RBF kernel under Gaussian P0, eigenvalues decay geometrically and the limit is dominated by the first few eigenfunctions.

F Numerical Stability of the Shiryaev–Roberts Recursion

The raw Shiryaev–Roberts recursion is numerically fragile because estimated null-density tails can create unbounded likelihood ratios and sustained evidence causes geometric overflow. A log-scale reparameterization, together with likelihood-ratio capping, preserves threshold decisions while stabilizing computation; calibration remains numerically determined because the relevant operator has no elementary solution.

Failure modes: Estimated null-density tails can make a single-window likelihood ratio unbounded and uninformative rather than merely large.When g0 falls at or below machine epsilon, dividing the mixture density by g0 produces the instability.
Failure modes: Under sustained alternatives, log R_t grows linearly while R_t grows geometrically and can overflow double precision after a few dozen strongly informative windows.The overflow threshold is approximately R_t > e^709, within the horizons used in the experiments.
Stabilization: Capping each per-window log-likelihood ratio at L = 15 limits its contribution to approximately e^15 ≈ 3.3 × 10^6 and removes tail-driven artefactual jumps.The cap leaves the recursion unchanged when |log Λ^π_t| < L, including the null regime and most of the alternative regime.
Stabilization: The exact transformation M_t = log(1 + R_t) yields an overflow-safe recursion and converts threshold crossing into M_t ≥ log(1 + A).The equivalent [0,1)-scaled chain Π_n = 1 − e^−M_n and its calibration carry over unchanged.
Calibration: The SR process can be viewed as a discrete-time Markov chain whose threshold rule solves an optimal-stopping problem, but its value function and threshold require numerical value iteration.The one-step operator integrates over a likelihood-ratio distribution involving the non-closed-form null density g0.
Calibration: No elementary solution is available for the Bayes-risk system because the operator depends on a likelihood-ratio density formed from a closed-form mixture and non-closed-form g0.Approach A numerically traces the delay/false-alarm frontier as the cost c varies.
Calibration: Approach B calibrates a threshold to a target average run length by numerically estimating mean first-passage times under the no-change law.The resulting rule is asymptotically minimax-optimal as γ → ∞, while moderate γ can exhibit non-negligible overshoot.

H The Full Detection Procedure

Algorithm 1 combines the frozen PF-ODE encoder, closed-form RBF MMD statistic, mixture likelihood ratio, and calibrated Shiryaev–Roberts threshold into an offline-plus-online detector. It is designed for a single changepoint, while multiple regime changes are left as an unanalyzed restart extension.

Procedure: Algorithm 1 combines the frozen PF-ODE encoder, closed-form RBF MMD2, mixture likelihood ratio, and calibrated SR threshold.The procedure has a one-time offline stage followed by a per-window online loop.
Offline stage: Offline processing trains the conditional model on burn-in data, freezes the context, tabulates g0 by Monte Carlo, and calibrates A for target average run length γ.The algorithm initializes the SR statistic at R0 = 0 before monitoring begins.
Online stage: Online processing reads each window, maps its observations through the frozen encoder, and updates the detector until an alarm occurs.Each latent representation is computed as Z_i = Φ(X_i; h_fixed).
Design choices: Freezing h_fixed makes Φ(·; h_fixed) a fixed bijection, while deterministic PF-ODE encoding avoids the non-injective stochastic-sampler alternative.The PF-ODE model and encoder are trained jointly using the conditional Stein score.
Scope and cost: The i.i.d. window-based null construction does not apply unchanged to overlapping sliding windows, and the paper calibrates thresholds under that sliding construction.The near-minimax delay guarantee proved for non-overlapping windows is not guaranteed to transfer unchanged.
Scope and cost: PF-ODE encoding is the dominant computational cost, while downstream MMD and SR operations use closed-form arithmetic on w values.This separates the expensive representation step from the online scalar-statistic updates.
Scope and cost: Algorithm 1 targets a single changepoint; continuous monitoring across multiple regime changes is proposed as a restart extension but is not analyzed.The suggested restart re-estimates h_fixed from fresh burn-in data and resets R to zero.

I Training the PF-ODE Encoder: Denoising Score Matching and the Stein Identity

The paper trains a conditional PF-ODE encoder by estimating the noised conditional Stein score with denoising score matching, yielding an injective map from complex pre-change densities to a Gaussian latent reference. This avoids requiring closed-form densities or Jacobian-trace computation, but the downstream guarantees depend on score-training convergence.

PF-ODE role: An injective PF-ODE flow is sufficient for the change-detection argument, while the learned conditional Stein score is the paper’s sole machine-learning-dependent quantity.The remaining MMD and Shiryaev–Roberts components are closed-form or Monte-Carlo-tabulated probability theory built on the frozen map.
PF-ODE role: The diffusion interpolates the unknown conditional pre-change density toward N(0, I), and its deterministic probability-flow ODE preserves these marginals.The PF-ODE map is formed by solving the deterministic ODE from τ = 0 to τ = T.
PF-ODE role: Lipschitz drift in x makes the PF-ODE flow map Φ(·; h) injective, whereas the noised density’s score remains unavailable in closed form before the Gaussian endpoint.Diffusion smooths the intractable density but does not make it analytically tractable at intermediate times.
Stein identity: Stein’s identity and integration by parts replace direct access to ∇x log p with an objective requiring only samples from the unknown density.The implicit score-matching objective still requires a full Jacobian trace, which is infeasible at d = 256.
Stein identity: Denoising score matching instead trains against the known Gaussian conditional-transition score, using noise prediction and ordinary backpropagation without computing a Jacobian trace.The conditional and marginal score objectives share the same minimizer because the marginal score is the posterior mean of the conditional score.
Training condition: The context encoder and denoiser are trained jointly so the learned history representation supports conditional score estimation across diffusion times.All downstream guarantees are conditional on this training objective converging; an undertrained network produced latent variance 0.845 rather than 1.

J Model Architectures Used in the Experiments

The experiments use a single-layer GRU history encoder and pre-norm residual denoisers, with architecture-specific conditioning for 2D pairs and MNIST. The denoisers predict v rather than ε to improve inversion conditioning.

History encoder: Both experiments use a single-layer GRU whose hidden state provides the causal, one-step-shifted history representation.For MNIST, each 16 × 16 frame is first embedded into 64 dimensions by a 256 → 256 → 64 MLP.
Denoiser: Both denoisers are pre-norm residual stacks that predict v = µtε − σtx0 instead of ε directly.This permits recovery of x0 and ε without dividing by µt.
2D pairs: The 2D-pair model conditions each residual block through a context-projected FiLM scale-and-shift pair.The conditioning is applied after each block’s linear-SiLU transform.
MNIST: The MNIST model concatenates the input, diffusion-step embedding, and history state before projecting them into the residual stack.The illustration uses classifier-free context dropout with probability 0.10, whereas the corrected Table 1 recipe removes it.

K Baseline Detectors and Equal-Footing Evaluation

The evaluation compares the proposed detector with raw-space MMD, latent mean-shift detection, and Hotelling’s T^2 plus CUSUM under a shared burn-in and false-alarm budget. Thresholds are calibrated once and then applied to independent test series.

Equal-footing evaluation: All baselines use only the monitored series’ burn-in segment for reference pools, covariance estimates, or null calibration, matching the information available to the proposed method.An idealised p1 pilot pool is used identically for the proposed method and Baselines A and B only to set detection speed.
Training protocols: The corrected MNIST recipe used for Table 1 increases capacity and training from the illustration recipe, including 512 hidden units, 8 blocks, 120,000 steps, and 999 DDIM steps.The illustration instead uses width 256, 4 blocks, 6,000 steps, and 100 DDIM steps.
Baseline A: Baseline A applies raw two-sample MMD between the current window and the burn-in reference pool, then self-normalises the statistic online.Its raw-space null law lacks a known closed form, so a trailing buffer of 60 recent values supplies the centering and scaling.
Baseline B: Baseline B retains the PF-ODE encoder but replaces MMD^2 with the squared norm of the latent window mean.Because the synthetic changes preserve approximately unchanged means, this detector is expected and found to be nearly blind to them.
Baseline C: Baseline C combines diagonal-covariance Hotelling’s T^2 with CUSUM using an asymptotic χ2_d null calibration.Its calibration degrades badly for non-Gaussian two-component mixtures at the evaluated window length.
Equal-footing evaluation: Each detector is calibrated once at a shared false-alarm budget PFA ≤ 5%, then evaluated on 1,000 independent test series with uniformly sampled post-burn-in changepoints.The fixed-threshold protocol mirrors deployment in which a threshold is set once and reused across monitoring episodes.