Probability flow ODEs in score-based and reflected diffusion models

1 Probability-flow ODEs in diffusion models

• A continuity-equation solution does not guarantee that the PF-ODE generates a regular Lagrangian flow from the data time.The PDE describes Eulerian density evolution, while a deterministic sampler requires a flow of individual samples; the distinction is especially relevant for data supported on a lower-dimensional manifold.Source: §1 · pp. 1–2
• Under the manifold hypothesis, positive-time diffusion regularizes singular data, but the PF-ODE may become singular as time approaches zero, collapsing normal directions.The paper gives an explicit example of this endpoint behavior.Source: §1 · p. 2
• For learned scores, deterministic sampler stability depends on velocity-field regularity, divergence, growth, and compression, not only on score-matching error.Source: §1 · p. 2
• For reflected models, confinement follows from tangency of the probability-flow velocity rather than from imposing a zero-normal-score condition.Source: §1 · p. 2
• The paper treats early stopping, velocity regularization, divergence control, low-density validation, and boundary constraints as structural parts of deterministic sampling.It illustrates these principles with numerical experiments focused on endpoint conditioning, the DSM–velocity-error gap, and Jacobian regularization.Source: §1 · p. 2

2.1 Marginal matching requires a Lagrangian flow

• A deterministic sampler needs a flow map whose trajectories solve the velocity ODE and transport p0 to pt; a continuity equation alone does not supply that map.Source: §2.1 · p. 2
• Local BV regularity of the drift and score, linear growth, and a one-sided bound on negative divergence suffice for a unique regular Lagrangian flow, without a globally Lipschitz score.Source: §2.1 · p. 2
• Under the manifold hypothesis, the PF-ODE is well posed on [δ, T], but reducing δ improves endpoint fidelity while worsening conditioning and leaves pδ → µ0 to a limiting argument or separate denoising step.Source: §2.1 · p. 2

2.2 Reverse sampling is stronger than forward marginal matching

• Reverse sampling needs control of both divergence signs; with both bounds, the reverse flow transports pT to p0, or pδ after early stopping, and supports inversion and change-of-variables likelihoods.Source: §2.2 · p. 3
• Forward marginal matching, reverse generation, and exact likelihood evaluation are distinct specifications, so divergence control should match the intended downstream operation.Source: §2.2 · p. 3

3 Singularity at the data endpoint under the manifold hypothesis

• For every t > 0, the sphere-smoothed law has a smooth, strictly positive density, although the endpoint flow is singular.The result concerns data exactly supported on a lower-dimensional manifold, not distributions that retain a positive ambient density.Source: §3 · pp. 3–4
• In the higher-codimension example, no ambient PF-ODE trajectory can start at t = 0 from points with nonzero normal coordinate.Those points have full ambient Lebesgue measure.Source: §3 · p. 3
• The obstruction is to a uniformly regular, invertible ambient flow, not to deterministic transport defined only µ0-a.s. on the manifold.The ambient reference measure is d-dimensional Lebesgue measure, whereas µ0 is concentrated on a Lebesgue-null set.Source: §3 · p. 4
• The sphere example shows that favorable distributional convergence does not guarantee a regular, invertible ambient PF-ODE as noise vanishes.The exact score still forces collapse of normal directions.Source: §3 · p. 4
• Stopping at δ > 0 and handling the passage from pδ to the data separately is a natural strategy when stable transport or likelihood evaluation is required.The flow's conditioning deteriorates near zero even for a perfect score, as normal Jacobian factors and ambient volume elements collapse.Source: §3 · pp. 4–5

4 Learned scores: training error versus sampler stability

• On [δ, T], trajectory stability is controlled by unweighted velocity error under local Sobolev regularity and a uniform compression bound.For Sobolev velocities, the measure of initial points whose trajectories differ by more than η has a logarithmic modulus in the velocity error.Source: §4 · p. 5
• Low-density regions receive little weight in score matching but are amplified when converting the loss to the velocity norm that controls deterministic trajectories.The conversion factor can grow sharply as δ decreases or the spatial ball grows.Source: §4 · p. 5
• Stable learned flows require control of spatial regularity, divergence, and growth beyond density-weighted score matching alone.Suggested mechanisms include spectral normalization, Jacobian penalties, divergence regularization, and uniform network-growth control.Source: §4 · p. 5

5 Reflected diffusion models: deterministic confinement without reflection

• On every [δ, T], reflected VE/VP models in smooth bounded domains admit a confined regular Lagrangian PF-ODE flow with the reflected diffusion’s marginals for arbitrary initial laws.This holds under positive-time regularization and drift–score regularity.Source: §5 · p. 6

5.1 The boundary condition acts on the full velocity

• Boundary-aware scores should enforce the co-normal relation for the full velocity; setting the normal score to zero is inconsistent for VP dynamics with nonzero normal drift.For reflected VE dynamics with zero drift, the condition reduces to Neumann.Source: §5.1 · p. 6
• In the regular regime, confinement follows from velocity tangency, so the deterministic PF-ODE needs neither reflection events nor projection steps.The reverse SDE still retains reflection.Source: §5.1 · p. 6

5.2 Endpoint and boundary failure modes

• For compactly supported data, reflected Brownian motion can have a score behaving like 1/t near the endpoint, leaving the PF-ODE ill-posed from time zero despite positive-time well-posedness.The flow is well posed on every [δ, T].Source: §5.2 · p. 6
• When density degenerates at the boundary, no-flux can coexist with boundary current and confined trajectories yet still produce unbounded compression and no regular Lagrangian flow.Uniformly elliptic reflected VE/VP models avoid this mechanism after positive-time regularization.Source: §5.2 · p. 6
• Nonconvex or nonsmooth boundaries can degrade near-wall second-derivative estimates, making domain geometry part of reflected-score conditioning.Smooth convex domains are favorable for Neumann/co-normal regularity.Source: §5.2 · p. 6

6 Implications: Design principles for diffusion samplers

• Treat the learned probability-flow velocity as the object requiring regularity and conditioning controls, including Jacobian, divergence, growth, and compression.For invertible generation and encoding, divergence control should cover both signs; reflected models require a co-normal boundary condition.Source: §6 · p. 6, §6.1 · p. 7

6.1 Regularize the quantity that is actually integrated

• The 90th percentile of unweighted velocity error fell from 0.431 without a Jacobian penalty to 0.0825, 0.0351, and 0.00678 as λJ increased to 10^-3, 10^-2, and 10^-1.A spatially uniform Jacobian penalty can suppress poorly identified modes in low-density regions.Source: §6.1 · p. 7

6.2 Use density-stratified validation

• Validation should stratify score and velocity/Jacobian errors by density and test round trips, perturbation sensitivity, and volume distortion along trajectories.For constrained models, also evaluate tangency and compression by distance to the boundary.Source: §6.2 · pp. 7–8
• With identical DSM error, the low-density tail perturbation produced 12.3× larger unweighted velocity error, 152× larger excess Jacobian penalty, and 15.9× larger mean endpoint trajectory error.The comparison used a uniform validation grid.Source: §6.2 · p. 7, Figure 1 · p. 7

6.3 Choose schedules together with the endpoint cutoff

• Choose the noise schedule and endpoint cutoff together: more rapid noise regularization can improve conditioning at a positive cutoff, while reducing δ can erase that benefit.For reflected models, boundary regularity matters: constants can deteriorate at corners or nonconvex singularities.Source: §6.3 · p. 8
• At δ = 10^-2, the residual normal scale is 0.1, inverse sensitivity is 10, and for codimension 8 the normal volume factor is 10^-8.Improving endpoint fidelity by reducing δ can therefore worsen ambient-map invertibility, even at a positive cutoff.Source: §6.3 · p. 8

• The appendix summarizes results from [7] [8] and refers to Ambrosio for mathematical definitions.Source: A Some mathematical results on ODE… · p. 10

A.1 PF-ODE in the whole space

• Local BV regularity, integrable growth, a one-sided divergence bound, and corresponding score regularity suffice for a unique regular Lagrangian flow transporting p0 to pt.An integrable bound on the opposite divergence sign also gives a time-reversed flow that is the almost-everywhere inverse.Source: §A.1 · p. 10
• For compactly supported initial laws in linear VP/VE diffusions, Gaussian regularization supplies the required score bounds on [δ, T], but worst-case VE bounds grow polynomially in δ^-1.Source: §A.1 · p. 10

A.2 Learned-score stability

• With fixed regularity and compression constants, learned-flow trajectories converge in measure as the unweighted velocity error Ev tends to zero.The paper relates Ev to denoising score-matching loss.Source: §A.2 · p. 10

A.3 PF-ODE in a bounded domain

• If the density is positive up to the boundary and the velocity has the required BV regularity and one-sided divergence bounds, the no-flux condition yields a flow that stays in Ω and transports pδ to pt.Source: §A.3 · p. 10
• For C2,α domains and sufficiently regular drift, the conditions hold for every δ > 0 and arbitrary initial law, but the constants degenerate as δ ↓0.The early-stopped transport statement applies to reflected VE and VP diffusions used in constrained score-based modeling.Source: §A.3 · p. 10

• Report the data-endpoint truncation δ and terminal denoising procedure, then measure Jacobian and divergence norms over time and density quantiles.Source: B Practical checklist for a PF-ODE… · p. 10
• Test trajectory and likelihood sensitivity to solver choice and score perturbations; when using invertibility, test compression in both directions.Source: B Practical checklist for a PF-ODE… · p. 11
• For reflected models, evaluate vθ · ν on boundary samples using the drift-dependent co-normal target, and stratify score error by density level.Source: B Practical checklist for a PF-ODE… · p. 11

Numerical experiments

• The experiments use exact reference scores to distinguish score approximation, flow conditioning, and numerical integration error.Source: §B.1 · p. 11

Jacobian regularization of a poorly identified tail mode

• The 90th percentile of Ev falls from 0.4308 without regularization to 0.03510 at λJ = 10^-2 and 0.006779 at λJ = 10^-1.The tail basis is rarely sampled, so its coefficient is poorly identified without regularization; population DSM error also decreases.Source: §B.1 · p. 11, Table 2 · p. 11

Equal DSM error, different velocity and trajectory errors

• 12.3-fold larger Ev, 151.7-fold larger Jerr, and 15.9-fold larger mean grid trajectory error occur for the tail perturbation despite equal DSM error.The comparison uses matched population DSM error; trajectory errors are evaluated at t = δ and the grid mean is not an expectation under pT.Source: §B.1 · p. 11, Table 3 · p. 12

Endpoint fidelity versus conditioning

• At δ = 10^-2, normal RMS per coordinate is 0.1 and inverse normal sensitivity is 10; normal volume factors are 10^-8 for c = 8 and 10^-32 for c = 32.For the scalar equation, RK45 function evaluations rise from 98 at cutoff 10^-1 to 470 at 10^-5.Source: §B.1 · p. 12, Figure 2 · p. 12