PaperlayerView exact revision on arXiv

Source-linked AI summary

Robustness of Diffusion Models under Distribution Shift

Wei Luo, Neil K. Chada, Shijie Zhang, Lu Yu

arXiv:2609.27546v1math.STcs.LGmath.PRstat.MLstat.TH

TL;DR

Existing diffusion theory largely gives guarantees when training and deployment distributions coincide, motivating analysis of score estimation under Wasserstein shifts. The paper develops an OU-based minimax theory that separates estimation and shift costs, with corresponding results for low-dimensional structure and positive-time reverse sampling.

  • Problem

    Most existing theoretical guarantees evaluate diffusion scores and sampling under a fixed data distribution, while deployment may use a different distribution.

    Source: §1 · p. 3

  • Method

    The paper analyzes worst-case score estimation for deployment laws in Wasserstein neighborhoods of a reference law under the Ornstein–Uhlenbeck diffusion.

    Source: §1 · pp. 3–4

  • Results

    For OU diffusion, robust error decomposes into statistical estimation and a quadratic Wasserstein shift cost with minimax-optimal dependence; the statistical term adapts to subspace dimension, and positive-time reverse sampling has matching KL guarantees.

    Source: §1 · p. 4, §7 · p. 13

  • Takeaways & Limitations

    More data and structural assumptions can reduce the statistical difficulty, but cannot remove intrinsic uncertainty caused by distribution shift.

    Source: §7 · p. 13

    The analysis focuses on the OU process; extending it to other diffusion dynamics and noise schedules remains open.

    Source: §7 · p. 13

Abstract

from arXiv · show

Score-based diffusion models are increasingly considered in settings where the underlying data distribution may differ from the training distribution, yet existing theoretical guarantees largely focus on the no-shift setting. In this work, we study robust score estimation under Wasserstein perturbations of a reference distribution. For the Ornstein--Uhlenbeck diffusion, we show that robust estimation decomposes into two fundamental components: the statistical cost of learning the reference distribution and the intrinsic cost of distribution shift. The latter scales quadratically with the Wasserstein radius, and this dependence is minimax optimal. We construct an explicit finite-sample estimator achieving the resulting robust minimax rate without knowing the shift radius. When the reference distribution lies on an unknown low-dimensional subspace, the statistical term adapts to the intrinsic dimension while the shift cost remains unchanged. Finally, we show that the same decomposition governs positive-time reverse sampling and obtain matching minimax guarantees in KL divergence. Together, these results characterize how finite data, intrinsic dimension, and distribution shift affect the robustness of score-based diffusion models.

1 Introduction

Score-based diffusion models rely on estimated noisy-distribution scores, but theory has largely studied fixed training and evaluation distributions despite deployment shifts. This section frames robust score estimation under Wasserstein shifts and summarizes results separating statistical estimation error from irreducible shift cost.

1 Introduction

  • Existing score-estimation and sampling guarantees largely evaluate models on the same fixed distribution used for training, unlike deployment across changing data sources.The introduction motivates robust analysis by noting shifts from domain changes, temporal drift, sampling bias, and variations in data collection.Source: §1 · p. 3
  • Because scores depend on local density variation, Wasserstein proximity alone does not ensure score proximity, and forward Gaussian smoothing weakens at early diffusion times.The paper identifies propagation of initial-distribution perturbations to scores along the diffusion trajectory as a key step in a robustness theory.Source: §1 · p. 3
  • The robust risk evaluates a score estimator learned from reference samples on shifted noisy marginals against their shifted scores, capturing both estimation and deployment uncertainty.The setting considers times t in [t0, T], with t0 > 0 to avoid the singular zero-noise regime.Source: §1 · pp. 3–4
  • For the Ornstein–Uhlenbeck process, robust error separates reference-distribution estimation cost from a Wasserstein-radius-squared shift cost, and a matching lower bound makes that dependence minimax optimal.The paper presents this as the first minimax theory for robust score-based diffusion models under distribution shift.Source: §1 · p. 4
  • An explicit finite-sample estimator achieves the optimal robust rate under compact support and fixed dimension without requiring knowledge of the shift radius.Source: §1 · p. 4
  • For an unknown k-dimensional linear subspace in ambient dimension d, the statistical term depends on k while the shift cost is unchanged; positive-time reverse sampling has the same decomposition with matching KL lower bounds.The paper concludes that more data and structural assumptions reduce statistical difficulty but cannot remove intrinsic shift uncertainty in score estimation or generation.Source: §1 · pp. 4–5

2 Setting and Problem Formulation

The section formulates robust score estimation by learning from a reference law and evaluating over Wasserstein-near deployment laws. It specifies the OU forward and reverse processes, then defines the worst-case shifted score risk.

2 Setting and Problem Formulation

  • The framework distinguishes p0, the reference distribution used to learn the score, from q0, a possible shifted deployment distribution.Source: §2 · p. 5

2.1 Diffusion Dynamics and Score Approximation

  • The forward process is the Ornstein–Uhlenbeck diffusion, whose transition kernel is Gaussian and whose semigroup maps an initial law to its noisy marginal.Source: §2.1 · p. 6
  • The reverse process uses the noisy marginal score; since that score is unknown, estimation is studied over t in [t0, T] with t0 > 0.Under mild conditions, reversing from pT reproduces the time-reversed forward process and generates samples from p0.Source: §2.1 · p. 6

2.2 Wasserstein Distribution Shifts

  • Deployment perturbations lie in a Wasserstein neighborhood of p0, and Gaussian OU smoothing gives the resulting noisy marginals well-defined scores for t > 0.Neither initial law is required to have a density.Source: §2.2 · p. 6
  • Under shift, an estimator learned from p0 is evaluated on qt and compared with the shifted score, rather than evaluated against the reference marginal score.Source: §2.2 · p. 6

2.3 Shifted Score Risk

  • The shifted score loss is defined for each possible deployment law q0 in the Wasserstein neighborhood and its corresponding noisy marginal qt.Source: §2.3 · p. 6
  • The robust shifted score risk takes the worst case over deployment laws and averages over reference training samples and estimator randomness.With finite reference samples, it captures both the statistical difficulty of learning p0 and uncertainty from distribution shift; its dependence on ε, t0, and sample size is studied.Source: §2.3 · p. 7

3 The Intrinsic Cost of Distribution Shift

The OU comparison principle separates robust score risk into reference-estimation error and a shift cost quadratic in Wasserstein distance; a matching lower bound makes the shift scaling unavoidable.

3.1 A Robust Comparison Principle

  • Propagating one estimated initial law through the OU semigroup generates the full score trajectory, allowing shifted risk to be controlled by divergence at one positive diffusion time.The comparison uses the same OU path for the estimated and data distributions.Source: §3 · p. 7, §3.1 · p. 7
  • The comparison principle splits robust risk into a term for shifted-versus-reference distributions and a term for estimating the reference marginal, with the shift term of order ε².It reduces finite-sample analysis to controlling the order-two Rényi divergence at positive time t0.Source: §3.1 · p. 7

3.2 Minimax Optimality of the Quadratic Shift Cost

  • Every estimator incurs shift error of order ε²/[t0(T − t0)] in the stated regime, matching the upper bound and making the quadratic shift cost minimax optimal.The lower bound assumes 0 < t0 ≤ 1/(16π²), 2t0 ≤ T, and 0 < ε ≤ √t0.Source: §3.2 · p. 8

4 Finite-Sample Robust Score Estimation

The section constructs a finite-sample Gaussian-blanket estimator and establishes its robust rate under compact support, then shows that the rate is minimax optimal and does not require knowing the shift radius.

4.1 The Gaussian-Blanket Estimator

  • Gaussian blanket regularization supplies a deterministic Gaussian lower envelope, addressing the denominator-control problem that positivity of the smoothed empirical measure alone does not resolve.The finite-sample construction assumes supp(p0) ⊆ B(0,D); the oracle inequality and quadratic shift dependence extend to P2(R^d).Source: §4.1 · pp. 8–9

4.2 Finite-Sample Upper Bound

  • For fixed d and D, the robust rate scales as 1/[n(T − t0)t0^(d/2)] for estimation and ε²/[t0(T − t0)] for shift.The Gaussian-blanket estimator does not depend on ε, and the shift coefficient is independent of dimension, sample size, and support radius.Source: §4.2 · p. 9

4.3 Minimax Optimality

  • The upper bound is sharp up to constants in both its finite-sample and distribution-shift terms, so the estimator attains the robust minimax rate in the stated regime.The lower-bound argument constructs a single family whose score separation persists over a nontrivial time interval.Source: §4.3 · p. 10
  • The same estimator is minimax optimal simultaneously for all 0 ≤ ε ≤ √t0, without radius-dependent tuning.Source: §4.3 · p. 10

5 Adaptation to Intrinsic Dimension

When the reference distribution lies on an unknown linear subspace, the estimator adapts its statistical cost to intrinsic dimension k, while the distribution-shift cost remains unchanged; the resulting rate is minimax optimal.

5 Adaptation to Intrinsic Dimension

  • The estimator uses an empirical subspace and adapts without exact recovery, controlling the source mass missed by that span.Source: §5 · p. 11
  • For k ≥ 2 and τ0 ≤ 1, the statistical rate replaces ambient-dimension dependence with intrinsic-dimension dependence, while the shift term remains unchanged.The shift distribution may leave the source subspace.Source: §5 · pp. 11–12

5.1 Minimax Optimality

  • The intrinsic-dimensional robust rate is minimax optimal, with constants depending only on k and D and not on ambient dimension d.Source: §5.1 · p. 12

6 From Robust Estimation to Robust Generation

The section carries the robust score-estimation decomposition into positive-time reverse sampling: generated-distribution KL has statistical and quadratic shift costs, both minimax optimal. With low-dimensional source structure, only the statistical term adapts to intrinsic dimension.

6.1 Minimax Optimal Reverse Sampling

  • Self-consistent terminal initialization makes the approximate reverse process recover the estimated OU trajectory at its output time.The approximate process replaces the true score with the estimated score.Source: §6.1 · p. 12
  • Positive-time reverse-sampling KL retains statistical and quadratic distribution-shift terms, and both components are minimax optimal.The sampler is evaluated at t0 > 0, avoiding the potentially singular shifted initial law.Source: §6.1 · pp. 12–13

6.2 Intrinsic-Dimensional Reverse Sampling

  • For intrinsic-dimensional reverse sampling, the KL decomposition replaces ambient dimension with intrinsic dimension while leaving the shift cost unchanged.The estimator uses the intrinsic-dimensional score construction with self-consistent initialization.Source: §6.2 · p. 13

7 Discussion

The discussion identifies the theory’s scope and open extensions. Appendix A supports the upper-bound argument with OU information estimates, a fixed-time Gaussian-channel bound, a relative de Bruijn identity, and a three-density KL inequality.

7 Discussion

  • The theory establishes robust score estimation and generation under OU diffusion, with quadratic minimax-optimal shift cost and corresponding positive-time reverse-sampling guarantees.It also constructs a finite-sample estimator and extends results to low-dimensional source structure.Source: §7 · p. 13
  • Open directions include general diffusion dynamics and noise schedules, nonlinear intrinsic structure, practical training guarantees, and discretized samplers.The current intrinsic-dimensional results rely on linear subspace structure.Source: §7 · p. 13

A Proofs for Robust Score Upper Bounds

  • The upper-bound proof plan combines two OU information estimates with the robust oracle bound and Gaussian-blanket bounds; statistical and shift lower bounds are proved separately.The lower-bound proofs are in Appendices B and C, respectively.Source: A Proofs for Robust Score Upper Bounds · p. 18

A.1 Fixed-Time OU Gaussian-Channel Bound

  • The fixed-time OU Gaussian-channel bound is a special case of a reverse transport inequality; its proof uses coupling, mixture convexity, and data processing.The authors include a direct proof to make the OU normalization explicit.Source: §A.1 · pp. 18–19

A.2 Relative de Bruijn Identity

  • The OU relative de Bruijn identity gives local absolute continuity of KL(qt∥pt) and an almost-everywhere differential identity for positive times.It also gives an integral relation over any finite positive-time interval.Source: §A.2 · p. 19

A.3 Proof of Proposition 3.1

  • The Proposition 3.1 proof combines the fixed-time Gaussian-channel estimate and relative de Bruijn identity with a three-density KL inequality.The inequality applies to strictly positive densities under finite-divergence conditions.Source: §A.3 · p. 21

Proof of Proposition 3.1

  • To prove Proposition 3.1, the argument applies the three-density inequality at t0, uses the OU trajectory identity, then takes the supremum over shifted laws.The resulting bound is independent of the particular shifted initial law.Source: Proof of Proposition 3.1 · p. 21

A.4 Proof of Proposition 4.1

This part develops the arguments for the estimator guarantee and statistical lower bounds in Proposition 4.1, including Gaussian domination, add-one analysis, and a hard family with separated heat scores. It controls error terms over a heat-time window and integrates the score separation to obtain the scale needed for Fano’s method.

A.4 Proof of Proposition 4.1

  • The Proposition 4.1 proof combines Gaussian blanket domination with a dominated-kernel add-one bound, applying the latter to the Gaussian kernels and regularized density.The domination bound is controlled by Mϑ,t0 for t ≥ t0.Source: §A.4 · p. 22, Proof of Proposition 4.1 · pp. 22–23

Proof of Proposition 4.1

  • Lemma A.4 bounds the Gaussian kernel relative to φvt by Mϑ,t0 for t ≥ t0 and y ∈ B(0, D), supplying the domination condition used in the add-one argument.The proof also verifies finiteness of the quantity required by Proposition 3.1 for training samples in B(0, D).Source: §A.4 · p. 22, Proof of Proposition 4.1 · pp. 23–24

A.5 Proof of Theorem 4.2

  • The proof specializes the bound to rate (4.5) and gives its order comparison when τ0 ≤ 1.The comparison uses n+1 ≍ n and τ0 = e^t0 − 1 ≍ t0 in that regime.Source: §A.5 · p. 25

B Proofs for Statistical Lower Bounds

  • The lower-bound proof adapts a hard-family construction from Dou et al. [2024] and uses one family that remains score-separated across [τ0, 2τ0].The same family supports the integrated score lower bound and the nominal positive-time KL lower bound.Source: B Proofs for Statistical Lower Bounds · p. 25

B.1 Reduction to the Integrated Heat-Score Problem

  • Because p0 belongs to its own Wasserstein uncertainty set, the robust integrated-score risk is at least the nominal risk, reducing the lower bound to nominal estimation.The reduction applies to every estimator, including any internal randomization.Source: §B.1 · p. 25

B.2 Construction of a Compactly Supported Hard Family

  • The compactly supported hard family uses disjoint bump cells of width ρ = K0√τ0, with m of order τ0^-d/2 and amplitude η² = κ(1 ∧ m/n).Its densities are supported in Q ⊂ B(0, D) and remain between 3f◦/4 and 5f◦/4 on Q.Source: §B.2 · pp. 26–27

B.3 Uniform Score Separation over the Heat-Time Window

  • Uniform score separation holds for every pair in the constructed family throughout [τ0, 2τ0], under the stated bounds on τ0 and η.The proof uses interior density and score envelopes and persistence of gradient energy to establish this window-wide separation.Source: §B.3 · pp. 28–30
  • The positive term is at least (4A1/25)η^2Hρ^(d−2), and subtracting the controlled error terms proves the score-separation claim.The error bounds use the stated constraints and the definition of η0.Source: §B.3 · p. 31
  • Under Lemma B.6’s assumptions, integrating over [τ0, 2τ0] and using the Hamming and scale conditions yields integrated packing separation of at least c_intη^2.The proof uses d_Ham(b, b′) ≥ m/8 and mρ^d ≥ 2^−dℓ^d.Source: §B.3 · p. 31

B.4 Fano Reduction and Proof of Theorem 4.3

The proof of the statistical lower bound reduces estimation to decoding among separated alternatives. A related fixed-time argument shows that density separation remains visible after heat-scale smoothing.

B.4 Fano Reduction and Proof of Theorem 4.3

  • A separated finite family, nearest-neighbor decoding, and Fano’s inequality establish the integrated score-estimation lower bound of Theorem 4.3.The argument also allows randomized estimators and restricts the risk to a heat-time interval.Source: §B.4 · pp. 31–33

B.5 Fixed-Time Density Separation at the Heat Scale

  • At the single heat time τ0, L2 separation persists after smoothing, yielding fixed-time Hellinger separation that also holds for the corresponding OU marginals.Unlike the integrated lower bound, this argument does not require persistence over a time interval.Source: §B.5 · pp. 33–34

B.6 Fixed-Time KL Lower Bound and Nominal Minimax Rate

The appendix establishes nominal positive-time KL lower bounds and proves shift-related score lower bounds using oscillatory perturbations, then extends the construction to fixed higher dimensions. Its proof separates Wasserstein admissibility, score separation, and persistence under OU smoothing.

B.6 Fixed-Time KL Lower Bound and Nominal Minimax Rate

  • The nominal positive-time KL minimax result is supported by a fixed-time statistical KL lower bound and an upper bound from the paper’s earlier risk result.The lower-bound argument uses a finite family and Fano’s calculation.Source: §B.6 · pp. 35–36

C.1 One-Dimensional Construction and Lower-Bound Statement

  • The one-dimensional construction uses compactly supported oscillatory perturbations whose smoothed densities remain positive, making their scores well defined.The alternatives share support [−1, 1].Source: §C.1 · p. 37

C.2 Auxiliary Estimates

  • The proof organizes its lower bound around Wasserstein admissibility, two-point score separation, and persistence of perturbation energy over an early-time window.The Wasserstein estimate verifies admissibility; persistence controls the energy over time.Source: §C.2 · p. 37

C.2.1 Wasserstein Size of the Perturbation

  • The oscillatory alternatives satisfy a Wasserstein-size bound, verified by interpolating densities and applying the Benamou–Brenier formula.The interpolation’s flux vanishes at the support boundary.Source: §C.2.1 · pp. 37–38

C.2.2 Two-Point Score Separation

C.2.3 Persistence of One Oscillatory Mode

  • An oscillatory mode remains detectable during the early-time window when its frequency is below the critical scale.The persistence estimate supplies the lower bound used in the construction.Source: §C.2.3 · pp. 39, 41

C.3 Proof of Proposition C.1

  • The one-dimensional proof selects a frequency compatible with the amplitude constraint and uses the resulting two alternatives to establish the score-risk lower bound.The alternatives are chosen to satisfy the Wasserstein constraint.Source: §C.3 · pp. 41–42

C.4 Proofs of Theorems 3.2 and 4.4

  • The higher-dimensional shift lower bound embeds the one-dimensional alternatives so score separation occurs only in the first coordinate.The product construction reduces the lower bound to the one-dimensional two-point separation.Source: §C.4 · pp. 43–44

C.5 Proof of Theorem 6.1

The proof of the positive-time KL shift result combines nominal estimation error with an intrinsic shift cost. It establishes the lower bound using two shifted Gaussian targets that cannot be distinguished from reference samples, and then proves an upper bound with the same additive structure.

C.5 Proof of Theorem 6.1

  • The robust positive-time KL risk has matching upper and lower bounds proportional to the sum of a statistical term A and a shift term B.The upper bound is obtained with the self-consistent Gaussian-blanket output.Source: §C.5 · pp. 44–45
  • The shift-cost lower bound uses two targets shifted in opposite directions from a point-mass reference, so reference samples are identical under both alternatives.After OU evolution the targets are Gaussians with opposite means.Source: §C.5 · pp. 44–45
  • The proof converts separation between the two target distributions into a lower bound on worst-case KL risk using Pinsker’s inequality and total variation.It averages the two risks before taking the infimum over estimators.Source: §C.5 · p. 45

D.1 Missed Energy of the Empirical Span

  • For adaptation to intrinsic dimension, the empirical span can miss source energy, but the lemma bounds that energy without a lower bound on nonzero eigenvalues.The stated condition is that the second-moment matrix has rank at most k.Source: §D.1 · pp. 46–47

D.2 An Orthogonal OU Comparison

  • The orthogonal OU comparison separates error within the empirical span from error in its orthogonal complement.The projected source is supported on the span, while the target need not be.Source: §D.2 · p. 47

D.3 Proof of Theorem 5.1

The proof of the intrinsic-dimension upper bound estimates from a data-derived span and separates in-span estimation error from orthogonal error. It also transfers the lower bound from dimension k to an unknown k-dimensional subspace.

D.3 Proof of Theorem 5.1

  • The pilot sample’s empirical span has expected missed source energy at most kD^2/(n1 + k).This bound is used in averaging the upper-bound argument over the pilot sample.Source: §D.3 · p. 49
  • Theorem 5.1’s bound is uniform over target laws in the Wasserstein class, after controlling the span-based estimator and taking the supremum over that class.The proof’s final expectation bound is independent of q0 in the class.Source: §D.3 · p. 49

D.4 Intrinsic-Dimensional Score Minimax Rate

  • The unknown-subspace lower bound transfers any ambient estimator into a latent k-dimensional estimator with no greater risk.The construction adds independent Gaussian noise in the orthogonal directions.Source: §D.4 · pp. 50–51
  • The resulting minimax lower bound is the k-dimensional bound, with constants c0 and ck,D.The transfer uses embedded latent source and target laws.Source: §D.4 · p. 51

E Proofs for Robust Reverse Sampling

This section proves guarantees for robust reverse sampling, including KL comparisons, Gaussian-initialization bounds, and intrinsic-dimensional upper and minimax guarantees. It also develops ReLU constructions for approximating Gaussian-blanket scores and scalar arithmetic operations, controlling approximation error with bounded width and complexity tied to accuracy and approximation range.

E.1 KL Comparison with General Initialization

  • Proposition E.2 supplies a KL comparison for the OU-compatible score estimator with general, possibly data-dependent terminal initialization.Its proof invokes the conditional score-to-KL bridge and establishes the reverse equation's unique non-explosive solution.Source: §E.1 · pp. 51–52

E.2 Standard Gaussian Initialization

  • Under Assumption 1, the excerpts identify a standard Gaussian terminal-mismatch lemma and standard Gaussian initialization corollary; the latter sets νT=γd for τ0≤1 and T≥log 2.Source: §E.2 · p. 53

E.3 Intrinsic-Dimensional KL Upper Bound

  • Theorem E.5 is stated under Theorem 5.1’s assumptions and construction, for k ≥2 and τ0 ≤1; the cited proof says uniqueness in law gives the strong solution the same marginals.Source: §E.3 · p. 54

F.1 Uniform Approximation Guarantee

  • The score correction admits a uniform ReLU approximation with explicit complexity bounds and a time-integrated compatibility-defect guarantee, uniformly over samples in the support ball.The known Gaussian-blanket score is retained exactly while the sample-dependent correction is approximated.Source: F Neural Approximation of the… · p. 55, §F.1 · p. 56

F.2 Ratio Representation and Localization

  • A ratio representation bounds the denominator below by one, while the score correction becomes uniformly small outside a sample-independent ball.These properties enable spatial localization of the network approximation.Source: §F.2 · pp. 56–57

F.3 Elementary ReLU Arithmetic

  • The construction uses constant-width ReLU arithmetic modules for multiplication, exponentiation, reciprocals, and logarithms, with depth and sparsity bounds; clipping is exact and cannot increase distance to the clipped interval.The module bounds depend logarithmically on their range and target accuracy for multiplication, and quadratically in that logarithmic factor for exponentiation and reciprocals.Source: §F.3 · p. 58
  • A clipped ReLU square network approximates u^2 on [0, 1] within xi, and exact ReLU absolute values support the related arithmetic constructions.The square network has depth and sparsity O(1 + log xi^-1); parallel copies give the stated error for the two-square construction.Source: §F.3 · p. 59
  • Repeated squaring approximates e^-z on [0, B] with error at most xi, output in [0, 1], and logarithmic depth and sparsity in B and xi^-1.The construction approximates e^-z by (1 - z/N)^N, then implements the power through repeated squaring.Source: §F.3 · pp. 59–60
  • A doubling-stage geometric-series network approximates the reciprocal on [1, B] with error less than xi and bounded width.Its output is clipped to [0, 1], and only a fixed number of scalar values is carried between stages.Source: §F.3 · pp. 60–61
  • The logarithm construction controls total approximation error by xi, uses bounded width, and sets its truncation length m at most Cκ(1 + log κ + log xi^-1).The proof bounds the arithmetic error and adds the truncated-series error.Source: §F.3 · p. 61

F.4 Proof of the Uniform Approximation Guarantee

The appendix constructs a uniform ReLU approximation to the explicit score correction, then transfers the robust score guarantee to that approximation under a bound on its error.

F.4 Proof of the Uniform Approximation Guarantee

  • The construction implements the explicit score correction through bounded time-coefficient and ratio approximations, followed by an assembled numerator and reciprocal-denominator computation.The ratio modules are evaluated in parallel; clipping extends the approximation from a cube to the whole space.Source: §F.4 · pp. 61, 63, 64, 65
  • The quotient comparison uses a correction bounded by KQ, preventing denominator error from introducing an additional factor of n.The reciprocal is applied to a denominator bounded below by 1.Source: §F.4 · p. 64
  • The approximation has controlled network width, depth, and sparsity, with O(n + d) parallel branches and fixed serial stages.The construction also establishes measurability through continuity in the sample and inputs.Source: §F.4 · p. 65

F.5 Transfer to Robust Score Risk

  • The ReLU score approximation preserves the Gaussian-blanket robustness rate when its squared approximation error is bounded by the combined statistical and distribution-shift terms.The neural score retains the Gaussian term exactly and approximates only the sample-dependent correction.Source: §F.5 · p. 67
  • The neural result establishes constructive realizability, not recovery by a particular score-matching training procedure, and its constants may deteriorate as t0 approaches zero or T grows without bound.The approximation bounds are stated for fixed 0 < t0 < T < ∞.Source: §F.5 · p. 67

G Numerical Illustration

A controlled full-rank three-dimensional experiment illustrates the theory: increasing source sample size reduces finite-sample error, while shift effects dominate at larger radii. The ε=0 experiment separately illustrates reference-domain sample-size decay.

G Numerical Illustration

  • The experiment uses a full-rank three-dimensional example to evaluate both time-averaged score risk and positive-time KL error under distribution shift.The source distribution has four noncoplanar support points and the study evaluates twelve translation directions.Source: G Numerical Illustration · pp. 67–68
  • In the controlled three-dimensional experiment, larger source samples move time-averaged score risk and positive-time KL error toward known-source references, while shift effects dominate at larger radii.Observed errors remain below the corresponding theoretical upper bounds, and the shift behavior is consistent with quadratic dependence on ε.Source: G Numerical Illustration · p. 69, Figure 1 · p. 70
  • At ε = 0, reference-domain score risk shows approximately n^-1 decay across seven source-sample sizes.This is a finite-sample illustration only; it does not probe the worst-case τ0^-d/2 dependence or the full minimax rate.Source: G Numerical Illustration · p. 69, Figure 2 · p. 70
  • Increasing endpoint quadrature order from 32 to 48 changes mean score risks by at most 0.11% over the examined configurations.This serves as a numerical stability check.Source: G Numerical Illustration · p. 69

Related papers

Loading 2609.27546v1…