Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data

1 Introduction

• Prior flow-matching guarantees leave generalization theory incomplete: some assume oracle velocity or score estimates, while others depend on ambient dimension or restrictive regularity conditions.The paper notes that ambient-dimensional rates can become uninformative in the high-dimensional regimes where flow matching is used.Source: §1 · pp. 2–3
• Existing low-dimensional flow-matching analyses can require restrictive manifold conditions or provide only in-expectation W1/W2 guarantees; this work targets arbitrary p ≥ 1 without manifold-reach or global velocity-Lipschitz assumptions.The introduction characterizes its guarantees as non-asymptotic and high-probability.Source: §1 · pp. 3–4

Summary of Contributions

• For every p ≥ 1, the paper derives non-asymptotic, high-probability Wasserstein-p guarantees for flow-matching models trained from i.i.d. samples.The bounds apply at arbitrarily high confidence and include prescriptions for choosing hyperparameters based on sample size and the population distribution.Source: §1 · pp. 3–4
• The resulting rates depend on the target’s intrinsic dimension rather than ambient dimension, without density, smoothness, or Lipschitz assumptions on the data distribution or velocity-field class.The paper presents this as a theoretical explanation for flow matching’s effectiveness on structured high-dimensional data.Source: §1 · p. 4
• For targets on regular sets and 1 ≤ p < d~ / 2, the bound gives O(n^-1/d) for every d > d~, matching minimax rates up to arbitrarily small exponent slack.The stated examples include compact manifolds, compact convex sets, and self-similar sets.Source: §1 · p. 4

2.1 Notations and Definitions

• Section 2.1 introduces notation for neural networks, Wasserstein-p distance, metric balls, and covering numbers used in the analysis.The network class is specified through bounds on depth, weights, biases, and Lipschitz constant.Source: §2.1 · pp. 4–5

2.2 Flow-matching

• The setup learns a time-dependent velocity field that transports an easy-to-sample reference distribution to the data distribution along a probability path.The paper uses a linear interpolant between independent samples from the reference and target distributions.Source: §2.2 · p. 5
• Flow matching trains a neural network to approximate the target velocity field by minimizing a time-integrated squared error, with a conditional objective that can be estimated from data.In practice, the integral is numerically approximated and the learned field is used to generate samples.Source: §2.2 · p. 6
• Analyzing the deterministic sampler is difficult because trajectory perturbation bounds typically require a globally Lipschitz learned field and can grow exponentially with its Lipschitz constant.The paper notes that this constant may grow near terminal time, weakening Wasserstein-distance control.Source: §2.2 · p. 6

2.3 Intrinsic Data Dimension

• Upper Minkowski dimension describes support complexity through metric entropy without requiring smoothness or manifold structure, whereas support-only dimension can ignore how probability mass is distributed.The paper discusses Wasserstein dimension as a notion sensitive to mass distribution.Source: §2.3 · pp. 7–8

3 A Proof of Concept Experiment

• The experiment uses images generated with only 10 or 100 active coordinates in a 128-dimensional BigGAN latent space, creating data with different intrinsic dimensions.All remaining latent coordinates are fixed to zero.Source: §3 · p. 8
• Models are evaluated by simulating the learned probability-flow ODE for 100 steps and computing FID against an equally sized held-out set, averaged over 10 repetitions.The training-set sizes range from 1,000 to 10,000.Source: §3 · p. 9
• As training samples increase from 1,000 to 10,000, the intrinsic-dimension-10 model consistently achieves substantially lower FID than the intrinsic-dimension-100 model.The gap persists at large sample sizes and widens slightly as more data become available.Source: §3 · p. 9
• The reported comparison is consistent with the paper’s prediction that intrinsic rather than ambient pixel dimension governs flow-matching sample complexity and achievable error rates.The authors state that empirical probability-flow ODE simulations confirm intrinsic-dimensional error decay under standard deterministic sampling.Source: §3 · pp. 8–9

4.1 Main Result and Inference

• Under its design choices, the main theorem gives high-probability Wasserstein-p bounds for every p ≥ 1, with the sample-size exponent governed by the target’s upper p-Wasserstein dimension.The result requires d > d*_p(Pdata) and suitable choices of the velocity-field class, step size, and early-stopping time.Source: §4.1 · p. 10
• For regular supports and 1 ≤ p < d~ / 2, the bound applies with every d > d~ and approaches the minimax distribution-estimation rate as d decreases to d~.The stated supports include compact manifolds, compact convex sets, and self-similar sets.Source: §4.1 · p. 10
• The theorem requires no density, smoothness, or Lipschitzness condition on the target distribution or velocity class, but the prescribed architecture still depends on ambient dimension.The authors clarify that the claim concerns the exponent of n; architecture dependence on D can be more than polynomial.Source: §4.1 · p. 11
• The early-stopping offset balances truncation with sampling error and prevents the target velocity from blowing up near terminal time; the discretization step is chosen to match sampling error.The Euler–Maruyama error is stated as O(δ^-(10κ1+2κ2)h).Source: §4.1 · p. 11

Error Decomposition

• The proof decomposes error into statistical and numerical components, with the numerical term measuring the discrepancy from replacing continuous dynamics by piecewise-constant Euler–Maruyama updates.The stated discretization error decreases as the uniform step size h tends to zero.Source: Error Decomposition · pp. 11–12
• The proof controls network approximation of the target velocity and, for the Monte Carlo estimator, also accounts for generalization error in estimating the objective’s expectation.The analysis then uses a KL bound via truncation.Source: Error Decomposition · pp. 12–13

5 Conclusion and Discussions

• For every p ≥ 1 and d above the target’s Wasserstein-p dimension, the bound’s exponent is governed by intrinsic dimension up to arbitrarily small slack.The guarantee uses appropriately chosen network architecture, early stopping, discretization, and truncation.Source: §5 · p. 13
• The analysis removes the usual uniform-Lipschitz assumption on the learned velocity field by establishing regularity of the optimal field instead.The prescribed network sizes are polynomial in n for each fixed ambient dimension D.Source: §5 · p. 13
• For intrinsically low-dimensional distributions, including compact smooth manifolds, the rates avoid the ambient-dimensional curse and approach corresponding minimax rates.The authors present these guarantees as a theoretical basis for flow matching’s observed effectiveness on high-dimensional data with low-dimensional structure.Source: §5 · p. 13
• The authors prescribe early stopping δ = Θ(n^−1/(dκ3)) and truncation M = Θ(n^(2p/d)) to control velocity-field behavior and transport-tail costs.Early stopping keeps the learned field close to the ground truth while avoiding blow-up as t approaches 1.Source: §5 · p. 13
• Brownian increments give Euler–Maruyama error O(δ^−(10κ1+2κ2)h), motivating a step size that places this error on the velocity-estimation-error scale.The passage attributes the h rather than h² increment scale to the Brownian component of the sampler.Source: §5 · p. 13
• The guarantees are optimization-agnostic because training error for nonconvex, time-dependent velocity networks remains hard to control.The approximation analysis uses only Lipschitz regularity of the optimal field; exploiting its Gaussian-mixture structure or extra smoothness may reduce network size, while guarantees for practical U-Nets or transformers remain unavailable.Source: §5 · p. 14

A.1 Existence and Uniqueness of the True Vector Velocity Field (Proof of Theorem 4)

• The true velocity field generates a unique solution for every initial point, and pushing the initial law through this flow yields P_t.The resulting densities satisfy the continuity equation in the weak sense.Source: §A.1 · p. 22

A.2 Equivalence to the Conditional Flow-matching Objective

• The squared-velocity expression separates learned-to-true error, true-to-interpolant error, and their cross term.Source: §A.2 · p. 25

A.3 Equivalence of the probability-flow ODE and the SDE family

• The score ∇log ϱ_t is globally Lipschitz in x, uniformly over the stated time interval.Its Lipschitz constant is O(Dδ^-4κ1).Source: §A.3 · pp. 25–26
• Under the stated assumptions, the SDE has a pathwise unique strong solution whose law equals the interpolant marginal ϱ_t throughout [0, 1−δ].Source: §A.3 · pp. 26–27

B.1 Error Decomposition

• The target process and Euler–Maruyama process share the diffusion coefficient and initial law, differing only through their drifts.Source: §B.1 · p. 27
• Girsanov’s theorem is used to relate the path-law discrepancy to drift differences, with the resulting bound evaluated at the terminal marginals.Source: §B.1 · p. 28
• The drift contribution is split into estimator error and time-discretization terms by adding and subtracting the true velocity at grid points.Source: §B.1 · p. 28

B.2 Discretization Error

• The diffusion coefficient cannot be tuned to remove the O(h) discretization term.The σ factors cancel in the bound, leaving the contribution uniform in σ.Source: §B.2 · p. 30
• The discretization contribution to KL(P_1−δ∥P̂_1−δ(v̂)) is of order D^3δ^-c1h, uniformly in σ.Source: §B.2 · p. 30

B.3 Approximation Result

• A ReLU network approximates v⋆ at the selected time points within ε on [−R,R]^D, with bounded spatial support and a stated Lipschitz bound.Source: §B.3 · pp. 30, 32

C.1 Proof of Theorem 6

• For d>d_p^*(P_data), the proof bounds the remaining error terms by O(n^-1/d) and obtains the result with probability at least 1−ξ.Source: §C.1 · pp. 33–34
• Conditioning on X_tk removes the cross term, so minimizing the empirical objective also minimizes the corresponding left-hand expression over the velocity class.Source: §C.1 · p. 33

C.2 Supporting Lemmata

• Uniform empirical control is obtained using an ℓ∞ cover of the velocity class and a union bound over time indices and cover elements.Source: §C.2 · p. 37

D.1 Results for the Velocity Field

• The velocity field is expressed as the conditional expectation of the interpolant velocity given Xt = x, using the conditional mean m(x, t).The passage also defines m(x, t) and Σ(x, t) as the conditional mean and variance of X1 given Xt = x.Source: §D.1 · pp. 38–39
• For t ∈ [0, 1 − δ], ∥∂t v⋆(x, t)∥ ≲ δ^−κ1(1 + ∥x∥).The bound holds for x ∈ R^D.Source: §D.1 · p. 41
• The time derivative of v⋆ is decomposed into terms involving x, the score g(x), and the temporal derivative of the score.The displayed expression is ∂t v⋆(x, t) = u̇t x + q̇t g(x) + qt ∂tg(x).Source: §D.1 · p. 42

D.2 Results for the Wasserstein Distance

• M^−1/(2p) bounds Wp(P1−δ, TM(P1−δ)).The bound is stated for the Wasserstein-p distance between P1−δ and its image under TM.Source: §D.2 · p. 45

• A ReLU network architecture approximates any 1-Lipschitz function on [0, 1]^D with f(0) = 0 to uniform error ε.The architecture’s stated depth is L = O(log(1/ε) + D), with B = O(1).Source: E Supporting Results from the Literature · p. 45
• A cited auxiliary result defines a ReLU network class by bounds on weight count, layer count, and maximum absolute weight.The passage attributes the lemma to Nakada and Imaizumi (2020).Source: E Supporting Results from the Literature · p. 45