PaperlayerView exact revision on arXiv

Source-linked AI summary

Stochastic Interpolants: A Unifying Framework for Flows and Diffusions

Michael S. Albergo, Nicholas M. Boffi, Eric Vanden-Eijnden

arXiv:2303.08797v4cs.LGcond-mat.dis-nnmath.PR

TL;DR

Generative modeling seeks ways to transport between arbitrary densities exactly in finite time. This paper introduces stochastic interpolants and derives deterministic and stochastic models from their density equations; numerical comparisons report better empirical performance for sampling with positive diffusion.

  • Problem

    Existing methods leave open how to bridge arbitrary densities, rather than requiring a Gaussian base, exactly over a finite time interval.

    Source: §1.1 · p. 3

  • Method

    Stochastic interpolants combine endpoint samples with a latent variable to construct continuous-time paths that bridge the two densities.

    Source: §1.1 · p. 4

  • Results

    The interpolant density satisfies a transport equation and tunable forward and backward Fokker–Planck equations, yielding ODE and SDE models with matching time-dependent densities.

    Source: §1.1 · p. 4

  • Takeaways & Limitations

    The framework offers a practical route to solving the Schrödinger bridge problem with stochastic interpolants.

    Source: §3.4 · p. 30

    Numerical investigation of this Schrödinger bridge formulation is left for future work.

    Source: §3.4 · p. 30

Abstract

from arXiv · show

A class of generative models that unifies flow-based and diffusion-based methods is introduced. These models extend the framework proposed in Albergo and Vanden-Eijnden (2023), enabling the use of a broad class of continuous-time stochastic processes called stochastic interpolants to bridge any two probability density functions exactly in finite time. These interpolants are built by combining data from the two prescribed densities with an additional latent variable that shapes the bridge in a flexible way. The time-dependent density function of the interpolant is shown to satisfy a transport equation as well as a family of forward and backward Fokker-Planck equations with tunable diffusion coefficient. Upon consideration of the time evolution of an individual sample, this viewpoint leads to both deterministic and stochastic generative models based on probability flow equations or stochastic differential equations with an adjustable level of noise. The drift coefficients entering these models are time-dependent velocity fields characterized as the unique minimizers of simple quadratic objective functions, one of which is a new objective for the score. We show that minimization of these quadratic objectives leads to control of the likelihood for generative models built upon stochastic dynamics, while likelihood control for deterministic dynamics is more stringent. We also construct estimators for the likelihood and the cross entropy of interpolant-based generative models, and we discuss connections with other methods such as score-based diffusion models, stochastic localization, probabilistic denoising, and rectifying flows. In addition, we demonstrate that stochastic interpolants recover the Schrödinger bridge between the two target densities when explicitly optimizing over the interpolant. Finally, algorithmic aspects are discussed and the approach is illustrated on numerical examples.

1 Introduction · 2 Stochastic interpolant framework

This section introduces stochastic interpolants for bridging endpoint densities and characterizes the resulting distributions through transport and Fokker–Planck equations. It also presents quadratic objectives for estimating the associated velocity and score, and outlines connections and extensions of the framework.

1.1 Background and motivation

  • Stochastic interpolants bridge arbitrary densities in finite time; their density obeys a transport equation and tunable-diffusion Fokker–Planck equations, enabling ODE and SDE samplers with the same time-dependent density.The ODE and SDE have different paths but share estimates for velocity and score; their drift coefficients are characterized by quadratic objectives estimated from endpoint and latent-variable data.Source: §1.1 · p. 4, Figure 1 · p. 5

1.2 Main contributions and organization

  • The framework proves density regularity and transport/Fokker–Planck equations, learns drift and score through quadratic objectives, and establishes likelihood control for stochastic but more stringent conditions for deterministic dynamics.It introduces a score objective and a denoiser objective; ODE drift regression also requires minimizing a Fisher divergence, while SDE diffusion can be tuned to maximize likelihood.Source: §1.2 · pp. 4–5
  • The paper also develops one-sided and mirror interpolants, linear-path variants, links to score-based diffusion, denoising and rectified flows, and algorithms for learning and sampling.It presents optimization over the interpolant as a route to the Schrödinger bridge, while fixed interpolants yield bias-free generative models.Source: §1.2 · pp. 5–6, §1.3 · p. 8

1.3 Related work

  • Unlike approaches tied to Gaussian endpoints or a particular sampling process, this framework decouples bridge construction from sampling and incorporates noise through a latent variable and tunable diffusion without bias.It also learns continuous-time transport drift without simulating the dynamics during training, while allowing generation through either deterministic or stochastic transport.Source: §1.3 · pp. 6–8

2.1 Definitions and assumptions

  • The interpolant combines an endpoint-constrained path I(t, x0, x1) with a latent term γ(t)z, using coupled endpoint samples and independent standard Gaussian noise.The path begins at x0 and ends at x1; γ vanishes at both endpoints and is positive in between, while the endpoint coupling may be chosen freely subject to the required marginals.Source: §2.1 · pp. 9–10
  • The zero-mean Gaussian latent process vanishes at both endpoints, and its single-time covariance determines the interpolant law; a Brownian bridge is one equivalent choice.When the covariance is γ(t)^2Id, the interpolant distribution matches the specified construction, and the stated definition is simpler for analysis and implementation.Source: §2.1 · pp. 11–12

2.2 Transport equations, score, and quadratic objectives

  • The interpolant has a positive, time-dependent density joining ρ0 to ρ1 and satisfying a transport equation with velocity b(t, x) = E[ẋt|xt = x].The transport equation can be evolved forward from ρ0 or backward from ρ1.Source: §2.2 · p. 13
  • The velocity b is the unique minimizer of a quadratic objective, while the density score is s(t, x) = −γ^-1(t)E[z|xt = x].The score result also characterizes the score as the unique minimizer of a quadratic objective.Source: §2.2 · pp. 13–14, In addition it satisfies · p. 14

In addition it satisfies

  • For any continuous nonnegative diffusion coefficient ϵ(t), the density satisfies forward and backward Fokker–Planck equations.The denoiser gives access to the score for t in (0, 1), and its equivalent objective avoids the factor γ^-1(t), which requires care near the endpoints.Source: In addition it satisfies · p. 14

1. The forward Fokker-Planck equation · 2. The backward Fokker-Planck equation

The forward and backward Fokker–Planck equations give well-posed evolutions that recover the opposite endpoint densities, and associated stochastic processes share the interpolant’s law.

1. The forward Fokker-Planck equation

  • Solved forward from t = 0 to t = 1, Equation (2.20) carries the initial density ρ0 to the terminal density ρ1.Source: §1 · p. 14

2. The backward Fokker-Planck equation

  • Solved backward from t = 1 to t = 0, Equation (2.22) carries the final density ρ1 to the initial density ρ0.Source: §2 · p. 15
  • With ρB(tB, x) = ρ(1 − tB, x), the backward equation becomes well-posed forward in reversed time, and the diffusion coefficient may be a positive semi-definite tensor.Source: §2 · p. 15
  • Removing the latent variable removes the guarantee of spatial regularity for b and s; including it is expected to improve dynamics and target regularity while broadening bridge flexibility.Source: §2 · p. 15
  • The forward and backward Fokker–Planck equations are more robust than the transport equation to approximation errors in velocity b and score s.Source: §2 · p. 15
  • Given samples (x0, x1) from ν, interpolant samples at any t enable empirical estimation of the objectives for velocity and score.Source: §2 · p. 16

2.3 Generative models

  • The generative-model corollary states that, at any t, the interpolant’s law coincides with the laws of processes associated with the transport equation and both Fokker–Planck equations.Source: §2.3 · pp. 16–17

1. The solutions of the probability flow associated with the transport equation (2.9) · 2. The solutions of the forward SDE associated with the FPE (2.20) · 3. The solutions of the backward SDE associated with the backward FPE (2.22)

The probability-flow ODE and the forward and backward SDEs provide ways to evolve the interpolant’s density, and their solution processes have the same marginal law at each time despite being different stochastic processes. These dynamics support generative sampling, while diffusion changes the conditions needed for likelihood control.

1. The solutions of the probability flow associated with the transport equation (2.9)

  • The probability-flow ODE can be solved forward from ρ0 or backward from ρ1, and its solution has marginal law ρ(t).The ODE is also used for sampling from a simple initial density to generate samples from a complex target density.Source: §1 · p. 17, §3 · pp. 17–18

2. The solutions of the forward SDE associated with the FPE (2.20)

  • The forward SDE is solved from its initial data, and its solution has marginal law ρ(t), like the probability-flow ODE and backward SDE.These processes have different sample paths, which can affect numerical integration accuracy and statistical-error propagation.Source: §2 · p. 17, §3 · p. 18

3. The solutions of the backward SDE associated with the backward FPE (2.22)

  • The backward SDE is solved from final data; reversed Itô calculus provides rules for working with it directly, and its solution has marginal law ρ(t).The ODE or reverse SDE can also be used to estimate the target density, given the initial density.Source: §3 · pp. 17–18

2.4 Likelihood control

  • With diffusion, drift and score losses control likelihood for learned Fokker–Planck models; for deterministic transport, matching the drift alone is insufficient in general.The diffusive term adds a negative KL-divergence term, removing the need for explicit Fisher-divergence control; the stated likelihood control applies to the losses (2.13) and (2.16), or (2.29) and (2.16), for stochastic models.Source: §2.4 · pp. 18–20

2.5 Density estimation and cross-entropy calculation

  • The transport-equation solution can be represented by the probability-flow ODE, while related forward and backward Fokker–Planck density representations use auxiliary SDEs.The section introduces these representations for density estimation and cross-entropy calculation.Source: §2.5 · pp. 20–21

1. The solution to the forward FPE · 2. The solution to the backward FPE

The sections describe forward and backward FPE solutions and use cross-entropy expressions to assess evolved densities. They also identify limitations in estimating these quantities when the score is approximate.

1. The solution to the forward FPE

  • In the forward-FPE solution, B denotes expectation along the path of Y^B.Source: §1 · p. 21

2. The solution to the backward FPE

  • The backward-FPE formulation uses expectations along forward and backward paths, including expectations conditioned on Y^B or Y^F.Source: §2 · pp. 21–22
  • Solving the forward FPE from its specified initial data yields an approximation at t=1, while solving the backward FPE from final data yields one at t=0.Even when the density at the opposite endpoint is known exactly, generating these endpoint samples still uses the corresponding forward or backward SDE with approximate drifts.Source: §2 · pp. 21–22
  • Cross-entropies of evolved endpoint densities relative to the target densities provide tests of sample quality for the ODE and forward or backward SDEs.Source: §2 · p. 22
  • Empirical expectations over endpoint data enable cross-validation of drift and score approximations and comparison of transport-equation and forward/backward-FPE density evolutions.Source: §2 · p. 23
  • Using Hutchinson trace estimates for divergences can bias log-expectation calculations; Jensen's inequality gives upper bounds, but these are generally not sharp.Source: §2 · p. 23
  • Approximation errors generally make the estimated score differ from the evolved densities' log-gradients, leaving extra terms unestimated; using the score as a proxy is uncontrolled.Source: §2 · p. 23

3 Instantiations and extensions

The section develops diffusive, one-sided, and mirror interpolants as specialized bridges, then shows how optimizing an interpolant can recover the Schrödinger bridge between target densities.

3.1 Diffusive interpolants

  • Brownian-bridge diffusive interpolants are continuous but nondifferentiable, yet share stochastic interpolants’ time marginals when γ(t) = 2a(t)t(1−t), yielding the same generative models in an easier-to-sample form.The corresponding representation avoids Itô calculus and allows direct sampling using samples from ρ0, ρ1, and a standard Gaussian.Source: §3.1 · p. 24
  • A regular diffusion can generate ρ1 from a point mass at x0, whereas a regular probability-flow ODE cannot map that point to a full target density.The theorem establishes a continuous drift for the diffusion, including at t = 0; the ODE limitation follows from uniqueness of its solutions.Source: §3.1 · pp. 25–26

3.2 One-sided interpolants for Gaussian ρ0

  • For a Gaussian base, one-sided interpolants use a target sample and Gaussian latent noise to bridge N(0, Id) to ρ1, providing an instantiation of score-based diffusion.Their construction uses a path J and noise scale α, and can share the general stochastic interpolant’s density under the stated identification.Source: §3.2 · p. 26

3.3 Mirror interpolants

  • Mirror interpolants bridge ρ1 to itself; with K(t, x1) = x1, the denoiser is the only quantity that needs to be learned.If ρ1 is available only through empirical samples, its functional form is generally unavailable; setting K to zero over an interior time interval provides a reference density and reduces the construction to two joined one-sided interpolants.Source: §3.3 · p. 28

3.4 Stochastic interpolants and Schr¨odinger bridges

  • Under a reversible map to a standard Gaussian, optimizing the interpolant recovers the Schrödinger-bridge density and velocity; as diffusion vanishes, the problem formally reduces to optimal transport.This result assumes the reversible map exists; in the zero-diffusion case, the required classical-solution assumption is more stringent, and numerical investigation is left for future work.Source: §3.4 · p. 30

4 Spatially linear interpolants

Spatially linear interpolants combine endpoint data with independent Gaussian latent noise, with endpoint conditions fixing the bridge. The section shows how latent noise and diffusion choices affect intermediate densities and sample trajectories.

4 Spatially linear interpolants

  • The bridge is linear in the endpoint samples and latent variable, with endpoint coefficients and positive interior noise set by stated boundary conditions.The variables (x0, x1) and z are independent, and z is standard Gaussian.Source: §4 · p. 30

4.1 Factorization of the velocity field

  • The velocity and score use three conditional expectations; their weighted sum reconstructs x, so any one can be derived from the other two to reduce computation.The three conditional expectations are also characterized as unique minimizers of quadratic objectives.Source: §4.1 · pp. 30–31

4.2 Some specific design choices

  • For zero-mean, identity-covariance endpoints, preserving identity covariance throughout requires α(t)^2 + β(t)^2 + γ(t)^2 = 1; similar-scale endpoints allow approximate preservation.The authors focus on choices satisfying this constraint, while noting the functions can be modified slightly without significantly affecting conclusions.Source: §4.2 · pp. 31–32
  • Gaussian encoding-decoding joins an interpolant from ρ0 to standard Gaussian noise with one from noise to ρ1, and requires γ(t) > 0 to avoid a midpoint collapse.Despite encoding ρ0 into noise on the first half, the probability-flow ODE defines one continuity equation mapping ρ0 to ρ1 across the full interval.Source: §4.2 · pp. 32, 34
  • Adding latent noise smooths intermediate densities and velocities, suppressing spurious modes; in the illustrated Gaussian-mixture example, encoding-decoding removes them altogether.The encoding-decoding choice passes through a standard Gaussian at t = 1/2.Source: §4.2 · pp. 32–33, Figure 4 · p. 33

4.3 Impact of the latent variable γ(t)z and the diffusion coefficient ϵ(t)

  • The latent variable smooths the density and velocity, whereas diffusion leaves the density unchanged and alters sampling stochasticity; nonzero diffusion also controls likelihood error when learned fields are approximate.The probability-flow ODE maps each initial point to a single endpoint, while forward-SDE endpoint spread depends on diffusion amplitude.Source: §4.3 · pp. 34–35
  • Among the discussed choices, only γ from (4.13) gives a nonvanishing endpoint score contribution to the velocity; the score enters through b for ODEs and explicitly for SDEs.The choice is appealing for its endpoint influence, but alternatives remain available.Source: §4.3 · p. 35

4.4 Spatially linear one-sided interpolants

  • For spatially linear one-sided interpolants, only one of the conditional expectations needs to be estimated; the resulting probability-flow ODE can be viewed as a denoiser-based generative model.The paper develops this connection further in Section 5.2.Source: §4.4 · p. 36

5 Connections with other methods

The section connects stochastic interpolants to score-based diffusion and stochastic localization, denoising, and rectified flows. It describes finite-time dynamics, iterative denoising, and rectification as links to these methods.

5.1 Score-based diffusion models and stochastic localization

  • Reparameterizing score-based diffusion as a one-sided linear stochastic interpolant yields nonsingular endpoint dynamics and unbiased finite-time generators without extra computation, estimating only one function.A finite-time score-based diffusion model can be biased because it starts its backward SDE from a Gaussian even when the process has not yet reached a Gaussian marginal. The backward SDE is also used in stochastic localization (Montanari, 2023).Source: §5.1 · pp. 37–38

5.2 Denoising methods

  • Repeated denoising updates form a consistent integration scheme for the interpolant’s probability-flow ODE, while a single conditional-mean estimate is not itself a sample.The conditional expectation of the clean data is used in denoising methods (Kadkhodaie and Simoncelli, 2021) and is Stein’s unbiased risk estimator (SURE; Stein, 1981).Source: §5.2 · p. 39

5.3 Rectified flows

  • Under the invertibility assumption, rectification gives straight-line solutions with the same generative map and target endpoint, and can also yield a single-step map.For α(t) = 1 − t and β(t) = t, the rectified paths are straight lines between the starting point and its original flow endpoint.Source: §5.3 · p. 40
  • Straight-line solutions do not by themselves imply optimal transport; gradient-constrained infinite rectification computes Brenier’s polar decomposition and offers a stochastic-interpolant perspective on consistency models.Rectification without the gradient-field constraint leaves the map unchanged; the cited consistency-model discussion describes distilling a learned velocity field into a single-step map.Source: §5.3 · p. 41

6 Algorithmic aspects

Section 6 gives practical guidance for learning drift and denoiser models and sampling their ODEs or SDEs. It highlights endpoint stability and how sampling choices affect sample quality.

6.1 Learning

  • Learning choices yield exact generative models without numerical or statistical errors, but those errors can make their practical performance differ across applications.The section frames learning drift coefficients and sampling with ODEs or SDEs as complementary algorithmic tasks.Source: §6.1 · p. 41, §6 · p. 41
  • For deterministic generation, learning b directly typically requires only one model and is more efficient than learning v and s separately.The recommendation applies when deterministic sampling is desired, including to exploit adaptive integration or exact likelihood computation.Source: §6.1 · p. 42
  • Antithetic sampling keeps both conditional mean and variance finite at the endpoints, where the ordinary score-loss term has diverging variance.It was necessary for stable training of objectives involving γ^-1(t); learning the denoiser η_z is an alternative because its objective is well behaved on [0, 1].Source: §6.1 · pp. 42–43

6.2 Sampling

  • Algorithm 4 samples ODEs or SDEs by converting the denoiser to a score, adding ε(t)s to b for the forward drift, and integrating with a chosen timestepper.Setting ε(t) = 0 reduces the drift to b; to avoid endpoint instability when using the denoiser, ε(t) can vanish near the endpoints or integration can stop before t = 1 followed by denoising.Source: §6.2 · p. 44
  • For spatially-linear one-sided interpolants, the denoiser suffices to represent the probability-flow velocity, and a finite-limit drift enables ODE or SDE models on [0, 1].The construction uses only a score or denoiser and avoids endpoint singularity in the resulting model.Source: §6.2 · p. 45
  • SDE sampling is generically better than ODE sampling for the illustrated target, with the smallest gap for γ(t) = t(1 − t); higher noise requires smaller steps.The figure reports good SDE performance across noise levels.Source: Figure 7 · p. 46, §6.2 · p. 46

7 Numerical results

The numerical studies examine learned stochastic interpolants across 2D density estimation, 128D Gaussian mixtures, and Oxford-flower image generation. They compare deterministic and stochastic sampling and vary interpolant and drift-design choices.

7.1 Deterministic versus stochastic models: 2D

  • In 2D checkerboard density estimation, stochastic sampling empirically improves performance, while γ(t) from (4.13) gives the best probability flow and outperforms the original interpolant even at ϵ = 0.At large ϵ, SDE sampling with the same learned drift and score performs better than probability-flow sampling.Source: Figure 8 · p. 47, §7 · p. 47, §7.1 · p. 48

7.2 Deterministic versus stochastic models: 128D Gaussian mixtures

  • For 128D Gaussian mixtures, learning b and η with a suitable ϵ > 0 achieves the best performance among the four tested drift-coefficient combinations.Each combination has an optimal nonzero ϵ; Figure 12 reports KL divergence between target and model marginals.Source: §7.2 · pp. 50–51, Figure 12 · p. 50
  • Across the 128D mixture models, small ϵ over-concentrates density in modes, excessive ϵ over-spreads it, and learning b and η generally outperforms learning v and s.The exception is that learning η does not generally outperform s when ϵ is large enough for performance to degrade; the authors recommend denoiser learning with singularities treated appropriately.Source: Figure 10 · p. 49, §7.2 · pp. 50–51

7.3 Image generation

  • For Oxford-flower generation, increasing ϵ yields more diverse SDE-generated images from the same initial sample, and a generated example appears visually different from its five nearest training images.The image experiments illustrate scalability and flexibility, but the paper leaves standard benchmarks such as FID and broader dataset evaluation for future work.Source: §7.3 · p. 51, Figure 13 · p. 52

5 Nearest neighbors in training set

Nearest-neighbor comparisons show that a trained linear one-sided interpolant does not memorize the dataset. For the mirror interpolant, an SDE can resample an input flower into a proximal flower not seen in the dataset, whereas the ODE returns the input unchanged.

5 Nearest neighbors in training set

  • Visually distinct ℓ1 nearest neighbors show that a generated image from the trained linear one-sided interpolant is not memorized from the dataset.Figure 14 compares one generated image with its five nearest dataset neighbors.Source: Figure 14 · p. 52
  • With ϵ = 10, the mirror-interpolant SDE resamples an input flower to a proximal flower unseen in the dataset, while its ODE returns the input unchanged.The example uses the 128 × 128 Oxford flowers dataset.Source: §5 · p. 53, Figure 15 · p. 53

8 Conclusion

The stochastic interpolant method provides a general framework, mathematical theory, and efficient algorithms for deterministic and stochastic generative models that map between two densities exactly in finite time. Its design space connects to optimal transport and Schrödinger bridges and may support a broad range of future applications.

8 Conclusion

  • The framework provides mathematical theory and efficient algorithms for deterministic and stochastic generative models that map between two densities exactly in finite time.The method is intended as a general framework for generative models built on dynamical transport of measure.Source: §8 · p. 53
  • Design parameters shape the interpolant process, with specific instantiations including mirror and one-sided interpolants and connections to optimal transport and Schrödinger bridges.The conclusion highlights a broader space of possible designs beyond the instantiations detailed in the paper.Source: §8 · p. 53
  • Candidate application domains include image inpainting and super-resolution, spatiotemporal forecasting, molecular-configuration sampling, and machine learning-assisted Markov chain Monte-Carlo.These are presented as potential applications, including inverse problems and scientific problems.Source: §8 · p. 53

Appendix A. Bridging two Gaussian mixture densities · Appendix B. Proofs · B.1 Proof of Theorems 6, 7, and 8, and Corollary 10.

The appendices derive explicit interpolant quantities for Gaussian-mixture endpoints and supply proofs of core regularity, transport, and optimization results. They also characterize the Gaussian probability-flow equation and give the interpolant score as a conditional expectation.

Appendix A. Bridging two Gaussian mixture densities

  • For Gaussian-mixture endpoints, the interpolant density remains a Gaussian mixture, with its velocity and score given by explicit mixture expressions.The proposition also states that both fields grow at most linearly in x and are approximately linear where density modes remain well-separated.Source: Appendix A · pp. 54–55
  • For Gaussian endpoints, the interpolant mean is α(t)m0 + β(t)m1 and covariance is α(t)^2C0 + β(t)^2C1 + γ(t)^2Id.The associated probability-flow ODE is linear; it can be solved analytically when Ċ(t) and C(t) commute, including the example C0 = Id.Source: Appendix A · p. 55

Appendix B. Proofs

  • Appendix B provides details for proofs omitted from the main text and reproduces the original theorem statements alongside their proofs.Source: Appendix B · p. 56

B.1 Proof of Theorems 6, 7, and 8, and Corollary 10.

  • Theorem 6 establishes that the interpolant density is positive and regular, matches ρ0 and ρ1 at the endpoints, and satisfies the transport equation.The proof derives the density and score regularity and verifies transport through the conditional-mean velocity.Source: §B.1 · pp. 56–58
  • Theorem 7 identifies the velocity b as the unique minimizer of the stated quadratic objective over the specified continuous differentiable vector-field class.The proof uses positivity of the interpolant density to establish uniqueness.Source: §B.1 · pp. 59–60
  • Theorem 8 gives the interpolant score as s(t, x) = ∇log ρ(t, x) = −γ(t)^−1 E[z | x_t = x] for interior times.It states that this score belongs to C1([0, 1]; (Cp(Rd))d) for any p ∈ N.Source: §B.1 · p. 60

In addition it satisfies … B.2 Proof of Lemma 19

The sections establish a conditional-expectation characterization and quadratic-objective minimization for the score, then show that the interpolant density evolves under forward and backward Fokker–Planck equations between the endpoint densities. The appendix states reverse Itô calculus identities and derives the score-minimization objective.

In addition it satisfies

  • E[z | x_t=x] = -gamma(t)s(t,x), and the score is the unique minimizer of the quadratic objective L_s when the interpolant density is positive.The conditional-expectation identity is established where gamma(t) > 0.Source: In addition it satisfies · pp. 60–61

1. The forward Fokker-Planck equation

  • The forward Fokker–Planck equation is well-posed from t = 0 to t = 1 and carries initial density rho_0 to final density rho_1.Its drift is defined in the forward-equation formulation, and the result applies for continuous nonnegative diffusion coefficient epsilon.Source: §1 · p. 61, In addition it satisfies · p. 61

2. The backward Fokker-Planck equation

  • The backward Fokker–Planck equation is well-posed from t = 1 to t = 0 and carries final density rho_1 back to initial density rho_0.Its drift is defined in the backward-equation formulation.Source: §2 · p. 61
  • The forward and backward Fokker–Planck equations follow from the transport equation and the score identity, which allow conversion between the equations.Source: §2 · p. 61

B.2 Proof of Lemma 19

  • For solutions of the backward SDE, Lemma 53 states a backward Itô formula and backward Itô isometries under the specified test-function conditions.The formula applies to f in C1([0, 1]; C2_0(Rd)); the isometries apply to g in C0([0, 1]; (C0(Rd))d).Source: §B.2 · p. 62
  • The derivation of the score objective uses integration by parts, drops a term constant in the candidate score, and expresses the remaining terms as an expectation over x_t before time integration.Source: §B.2 · p. 63

B.3 Proofs of Lemmas 21 and 22, and Theorem 23. · B.4 Proofs of Lemma 25 and Theorem 26

These appendices derive terminal-density divergence bounds from transport and Fokker–Planck equations, and give characteristic-flow and forward/backward SDE constructions. Theorem 23 relates stochastic and deterministic model bounds to their corresponding objectives.

B.3 Proofs of Lemmas 21 and 22, and Theorem 23.

  • Lemma 55 derives a terminal KL-divergence bound for Fokker–Planck solutions with diffusion ϵ > 0.The proof rewrites the Fokker–Planck equations as score-dependent transport equations, applies Lemma 21, and uses Young’s inequality.Source: §B.3 · pp. 64–65
  • Theorem 23 bounds terminal-density divergence using the velocity and score objectives, and relates those bounds to the probability-flow velocity objective.Its setup defines the forward drift as b̂ + ϵŝ and the probability-flow velocity as b̂ + γγ̇ŝ.Source: §B.3 · pp. 65–66
  • Theorem 23’s proof uses the Fokker–Planck endpoint law and Lemma 22 to obtain its bound, then substitutes the objective definitions.The proof also rewrites the drift in terms of the probability-flow velocity and score field for its final bound.Source: §B.3 · p. 66

B.4 Proofs of Lemma 25 and Theorem 26

  • Lemma 56 expresses transport-equation solutions from an initial density through the ODE flow map.The result applies at any time t in [0, 1].Source: §B.4 · p. 66
  • Lemma 56 also gives the transport solution from a prescribed final density by integrating the flow backward.This representation holds at any time t in [0, 1].Source: §B.4 · p. 67
  • Theorem 26 sets forward and backward drifts to b̂ + ϵŝ and b̂ − ϵŝ, respectively, for SDEs run from the initial and final conditions.The backward terminal condition is specified as independent of its Brownian motion.Source: §B.4 · p. 67

1. The solution to the forward FPE · 2. The solution to the backward FPE · B.5 Proof of Theorem 31

The derivations use path-conditioned expectations and Itô identities to obtain FPE solutions. Theorem 31 shows that an initial stochastic interval yields a well-defined forward SDE and that diffusion can be discontinued afterward.

1. The solution to the forward FPE

  • For the forward-FPE solution, B denotes expectation along the path of Y^B, conditioned on its time-t value.The supplied passages introduce this conditional path expectation but do not include the resulting formula.Source: §1 · p. 67

2. The solution to the backward FPE

  • The backward-FPE derivation applies the backward Itô formula, rewrites the resulting equation as a total differential, and uses the boundary condition ρ̂(0)=ρ0.Conditioning on Y^B at t=1 and using the zero mean of the right-hand-side term yields the stated expectation.Source: §2 · p. 68
  • A second Itô calculation gives a total-differential identity with a Brownian increment, and the derivation also uses the endpoint condition ρ̂(1)=ρ1.The expectation is conditioned on Y^F, as indicated in the passage.Source: §2 · pp. 68–69

B.5 Proof of Theorem 31

  • Assuming I(t,x0,x1)=x0 on [0,δ] and a>0, the forward SDE is well-defined on that interval and its solutions have the law of x_t.The proof also establishes continuity of u_d in time with values in (C^p(R^d))^d.Source: §B.5 · pp. 69–70
  • On [0,δ], x_t has distribution N(x0, 2at(1−t)Id), and its score is given explicitly for t∈(0,δ].The score is used to express the Fokker–Planck drift, which remains nonsingular at t=0.Source: §B.5 · p. 70
  • It is enough that ∂tI(0,x0,x1)=0, and after the initial diffusive interval the process can switch to a probability-flow ODE.The proof notes this condition implies I(t,x0,x1)=x0+O(t^2); the diffusion coefficient can be set to zero for t∈(δ,1].Source: §B.5 · p. 70

B.6 Proof of Lemma 40 and Theorem 41. … C.1 Image Experiments

The appendix establishes the optimizer and reconstruction properties of stochastic interpolants, proves consistency and endpoint-distribution results, and specifies experimental implementations. It also describes the image-model architecture and a terminal-time denoising procedure.

B.6 Proof of Lemma 40 and Theorem 41.

  • Under Assumption 39, every optimizer of (3.38) yields an interpolant with density solving (3.35), and its control is u = ∇λ.The stated bridge uses γ(t)=0 at the endpoints, γ(t)>0 in the interior, and the regularity conditions in Theorem 41.Source: §B.6 · pp. 71–72
  • The constrained optimizer has the prescribed endpoint densities and satisfies the transport-diffusion equation, with its unique minimizer characterized by Euler–Lagrange equations.The constraint is ∂tρ + ∇·j = ϵ∆ρ with ρ(0)=ρ0 and ρ(1)=ρ1.Source: §B.6 · p. 71

B.7 Proof of Theorem 45

  • The integration scheme (5.14) is consistent for the probability-flow equation, and the result also holds on nonuniform time grids.The proof takes the limit as N and j grow with j/N approaching t.Source: §B.7 · p. 72

B.8 Proof of Theorem 47

  • Every solution of the probability-flow equation associated with (5.18) maps an initial sample from ρ0 to a final sample distributed as ρ1.Source: §B.8 · p. 73
  • Under Assumption 46, the velocity field reduces as stated in Theorem 47, and the reconstruction velocity is the unique minimizer of (5.19) and solves (5.21).Source: §B.8 · p. 73

Appendix C. Experimental Specifications

  • The Section 7.1 models use depth-4, width-512 feed-forward networks and train for 7,000 iterations with antithetic sampling and Adam.Batches contain 25 base draws, 400 target draws, and 100 time slices; the learning rate starts at .002 and halves every 1,500 iterations, and sampling uses a Heun-based integrator.Source: Appendix C · p. 73

C.1 Image Experiments

  • Image experiments use the Ho et al. (2020) U-Net for learning b, v, s, or η; for SDEs learning ηz, integrating slightly before tf=1.0 and then applying denoising gave the best results.Table 3 provides image-dataset architecture and training hyperparameters.Source: §C.1 · p. 73, Table 3 · p. 74

Related papers

Loading 2303.08797v4…