Flow Matching Guide and Code

Yaron Lipman; Marton Havasi; Peter Holderrieth; Neta Shaul; Matt Le; Brian Karrer; Ricky T. Q. Chen; David Lopez-Paz; Heli Ben-Hamu; Itai Gat

Source-linked AI summary

Flow Matching Guide and Code

Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky T. Q. Chen, David Lopez-Paz, Heli Ben-Hamu, Itai Gat

arXiv:2412.06264v1Dec 9, 2024cs.LG

TL;DR

Flow Matching has advanced generative modeling across several domains, and this guide aims to provide a self-contained resource on its foundations, design choices, and extensions. It reviews FM alongside an accompanying codebase and describes state-of-the-art applications across fields.

Problem
Flow Matching spans diverse applications, and the paper identifies a need for a self-contained reference detailing its design choices and community-developed extensions.
Source: §1 · p. 3
Method
The paper reviews Flow Matching’s design choices and extensions and provides an accompanying codebase.
Source: §1 · p. 3
Results
Flow Matching has pushed the state of the art in image, video, speech, audio, protein, and robotics generation.
Source: §1 · p. 3
Takeaways & Limitations
The manuscript and codebase are intended to help newcomers adopt and build upon Flow Matching.
Source: §1 · p. 3
The exact distribution sampled by classifier-free guidance is unknown, despite its popularity for training conditional models.
Source: §4.10.2 · p. 34

Abstract

from arXiv · show

Flow Matching (FM) is a recent framework for generative modeling that has achieved state-of-the-art performance across various domains, including image, video, audio, speech, and biological structures. This guide offers a comprehensive and self-contained review of FM, covering its mathematical foundations, design choices, and extensions. By also providing a PyTorch package featuring relevant examples (e.g., image and text generation), this work aims to serve as a resource for both novice and experienced researchers interested in understanding, applying and further developing FM.

1 Introduction

Flow Matching has advanced generative modeling across multiple domains, and this manuscript pairs a comprehensive review of its design choices and extensions with code intended to help newcomers apply and extend the framework. It presents Flow Matching as a two-step recipe that also generalizes to broader generative Markov processes.

1 Introduction

FM has pushed state-of-the-art generative modeling in images, videos, speech, audio, proteins, and robotics.The cited applications include images (Esser et al., 2024), videos (Polyak et al., 2024), speech (Le et al., 2024), audio (Vyas et al., 2023), proteins (Huguet et al., 2024), and robotics.Source: §1 · p. 3
The manuscript aims to be a self-contained FM reference and provide code that helps newcomers adopt and build on the framework.It covers design choices and extensions developed by the research community.Source: §1 · p. 3
FM learns a velocity field whose ODE flow transports samples from a source distribution to a target by choosing an interpolating probability path and training a neural network to implement it.Earlier continuous normalizing flows required simulation and differentiation during likelihood training, motivating simulation-free FM algorithms (Lipman et al., 2022; Liu et al., 2022; Albergo and Vanden-Eijnden, 2022).Source: §1 · p. 3
FM generalizes to discrete and manifold state spaces and, through Generator Matching, to CTMPs including flows, diffusions, and jump processes.Across these processes, the recipe is to choose a source-to-target path and train a generator that implements it, unifying existing models and enabling new generative models across modalities (Holderrieth et al., 2024).Source: §1 · p. 3
Diffusion models fit the FM perspective through forward-noising probability paths and score-based generator parameterization, while diffusion bridges extend the construction to arbitrary source-target couplings.Linear FM is described as a limiting case of bridge matching.Source: §1 · p. 4
The manuscript progresses from a PyTorch cheat-sheet and continuous flows to Euclidean and manifold FM, discrete and general Markov-process models, Generator Matching, and diffusion models.Generator Matching unifies models covered in earlier sections.Source: §1 · p. 4

2 Quick tour and key concepts

Flow Matching learns a neural velocity field that transports samples from a known source distribution to a target distribution along a designed probability path. The guide illustrates linear conditional paths, a tractable conditional regression objective, and a standalone PyTorch implementation.

2 Quick tour and key concepts

Flow Matching trains a neural velocity field to generate a probability path from source distribution p to target distribution q, then samples by integrating the field’s ODE from t = 0 to t = 1.Generation begins with a new sample from p; the learned flow maps it to a sample resembling q.Source: §2 · p. 4
A linear conditional optimal-transport path interpolates between source noise and each target example, and aggregating these conditional paths gives the probability path.The example uses a Gaussian source and defines Xt|1 = tx1 + (1 − t)X0.Source: §2 · p. 5
Conditioning the Flow Matching loss on a sampled target example makes the otherwise complicated velocity-field regression tractable, using the conditional path and its velocity.The simplest implementation regresses the predicted velocity toward X1 − X0 for Xt = (1 − t)X0 + tX1.Source: §2 · pp. 5–6
The guide provides a standalone pure PyTorch implementation of the quick tour and says more sophisticated variants and design choices are implemented in the accompanying flow_matching library.The standalone implementation is identified as Code 1.Source: §2 · p. 6

3 Flow models

This section introduces flow models as deterministic continuous-time processes that transform source densities into target densities, and explains their ODE-based sampling and likelihood computation. It also covers practical trace estimation and the simulation requirements of likelihood training, motivating Flow Matching’s simulation-free alternative.

3.1 Random vectors

Random vectors are introduced through continuous probability densities, with expectation defined as the least-squares constant approximation to the variable.The section also establishes notation for distributions and expectations.Source: §3.1 · p. 8

3.2 Conditional densities and expectations

Conditional densities describe distributions given another variable, while conditional expectation is the least-squares function of that variable and can denote either a function or a random variable.The text distinguishes E[X|Y = y] from E[X|Y].Source: §3.2 · pp. 8–9

3.3 Diffeomorphisms and push-forward maps

A diffeomorphism pushes a density forward using the change-of-variables formula, yielding a transformed density expressed through the inverse map's Jacobian determinant.The push-forward operator is denoted ♯.Source: §3.3 · p. 10

3.4 Flows as generative models

A flow model applies time-dependent diffeomorphisms to an initial random variable, making later states depend deterministically only on the current state.This deterministic dependence makes the process Markov.Source: §3.4 · p. 10

3.4.1 Equivalence between flows and velocity fields

A C^r velocity field defines a flow through an ODE, and conversely each C^1 flow determines a unique velocity field.For r ≥ 1, local existence and uniqueness hold; reaching t = 1 requires stronger assumptions, such as global Lipschitzness or integrability.Source: §3.4.1 · p. 11

3.4.2 Computing target samples from source samples

Euler integration approximates target samples with error vanishing as the step size shrinks, while the second-order midpoint method often performs better in practice.Sampling starts from X0 ∼ p and numerically solves the flow ODE.Source: §3.4.2 · pp. 11–12

3.5 Probability paths and the Continuity Equation

A locally Lipschitz integrable velocity field generates a probability path if and only if the pair satisfies the Continuity Equation.The interval is [0, 1), allowing target distributions with compact support when velocity may be undefined at t = 1.Source: §3.5 · p. 13

3.6 Instantaneous Change of Variables

Flows enable tractable exact likelihood computation log p1(x) for every x through Instantaneous Change of Variables.The paper identifies this as a consequence of the Continuity Equation.Source: §3.6 · p. 14
Log-likelihood changes along a flow trajectory according to an ODE, and estimating log p1(x) entails simulating the ODE backward from t = 1 to t = 0.The log-likelihood ODE is derived from the Continuity Equation and the flow ODE.Source: §3.6 · pp. 14–15
Because computing the divergence trace becomes harder as dimensionality grows, prior work uses Hutchinson’s unbiased trace estimator (Grathwohl et al., 2018).For a fixed random sample, the calculation can be performed with a single backward pass via a vector-Jacobian product.Source: §3.6 · p. 14

3.7 Training flow models with simulation

Likelihood-based training learns velocity-field parameters by maximizing training-data log-likelihood, but requires precise ODE simulations because only errorless solutions yield unbiased gradients.Flow Matching is presented as a simulation-free alternative that avoids solving ODEs during training.Source: §3.7 · p. 15

4 Flow Matching … 4.5 Flow Matching loss

Flow Matching trains a neural velocity field to generate a probability path from a known source distribution to the data distribution, then samples by solving an ODE. Its practical conditional loss avoids computing the intractable marginal velocity while preserving the marginal-flow learning objective.

4 Flow Matching

FM learns a neural velocity field that generates a prescribed probability path from source to target, then samples by solving an ODE.The framework identifies a source distribution and data target, specifies an interpolating path, and learns a velocity field to generate it.Source: §4 · p. 16

4.1 Data

Source samples are commonly drawn from an easily sampled known distribution, while target samples come from a finite dataset; their coupling may be independent or dependent.Examples include Gaussian noise paired independently with images, or low-resolution images paired with their high-resolution versions.Source: §4.1 · p. 16

4.2 Building probability paths

Conditioning path construction on target examples simplifies designing conditional paths, which can be aggregated into a marginal path interpolating from the source distribution to the target.Conditional paths must satisfy boundary conditions that yield the required marginal endpoints.Source: §4.2 · pp. 16–17

4.3 Deriving generating velocity fields

The marginal velocity generating the marginal path is the target-posterior-weighted average of conditional velocities, equivalently a least-squares approximation given the current sample.The weights are posterior probabilities of target samples given the current sample.Source: §4.3 · pp. 17–18

4.4 General conditioning and the Marginalization Trick

Under regularity and integrability assumptions, the Marginalization Trick shows that averaging conditional velocities yields a marginal velocity generating the marginal probability path.The result applies when conditioning on an arbitrary random variable, not only on target samples.Source: §4.4 · p. 18

4.5 Flow Matching loss

Conditional Flow Matching (CFM) has the same gradients as Flow Matching, and its minimizer is the marginal velocity, avoiding direct computation of that intractable target.The marginal target requires integrating over the training set, whereas CFM uses conditional velocities.Source: §4.5 · pp. 19–20
More generally, Bregman-divergence regression has a global minimum when the prediction equals the conditional expectation of the target given the input.This result provides a general basis for scalable Flow Matching losses.Source: §4.5 · pp. 20–21
Sampling time from a non-uniform distribution can outperform weighting the loss by that distribution in large-scale image generation, despite mathematical equivalence (Esser et al., 2024).The weighted objective is formed by sampling t from a probability density over [0, 1].Source: §4.5 · p. 21

4.6 Solving conditional generation with conditional flows · 4.7 Optimal Transport and linear conditional flow

Conditional flows reduce conditional path and velocity design to constructing a flow with suitable boundary conditions. The guide’s linear conditional flow yields straight, constant-velocity trajectories and useful kinetic-energy properties.

4.6 Solving conditional generation with conditional flows

Constructing a conditional flow ψt with the required boundary conditions defines both the conditional probability path and its velocity field by differentiation.The flow starts from the initial conditional distribution and concentrates its mass at x1 as t approaches 1.Source: §4.6 · pp. 21–22

4.6.1 The Conditional Flow Matching loss, revisited

The CFM loss minimizer has the form given by Liu et al. (2022), and the library’s ProbPath samples states and velocities for a Monte Carlo loss estimate.Source: §4.6.1 · p. 22

4.6.2 The Marginalization Trick for probability paths built from conditional flows

Under the stated regularity, positivity, support, and conditional-flow assumptions, the conditional path and velocity induce a marginal velocity generating a path interpolating p and q.Source: §4.6.2 · p. 23

4.6.3 Conditional flows with other conditions

Target-, source-, and two-sided conditioning yield the same marginal path and velocity when the conditional flow is a diffeomorphism in each endpoint.The cited options are Z = X1, Z = X0, and Z = (X0, X1).Source: §4.6.3 · p. 24
A smooth interpolant alone does not ensure its marginal velocity generates the marginal path; the text gives a counterexample and notes that extra conditions are needed.Source: §4.6.3 · p. 24

4.7 Optimal Transport and linear conditional flow

Straight OT trajectories have constant velocity, are easier to sample with ODE solvers, and reach the target in one Euler step.The velocity is ϕ(X0) − X0.Source: §4.7 · p. 25
The linear conditional flow minimizes a bound on kinetic energy, and its marginal velocity’s kinetic energy is no greater than that of the original coupling.Source: §4.7 · p. 26
When q is a point mass, the linear conditional flow is the OT displacement interpolant and analytically solves the dynamic OT problem.Source: §4.7 · p. 26

4.8 Affine conditional flows

The section develops affine conditional flows through scheduler choices, alternative velocity parameterizations, and post-training scheduler changes. It also explains Gaussian paths and their kinetic optimality.

4.8 Affine conditional flows

Affine conditional flows generalize linear Conditional-OT flows through schedulers whose α_t increases and σ_t decreases between specified endpoints.The endpoint constraints are α_0 = 0, σ_1 = 0, α_1 = 1, and σ_0 = 1.Source: §4.8 · p. 26
Under an independent coupling, bounded-support target, and smooth positive source density with finite second moments, the marginal velocity generates a path interpolating the source and target.A Gaussian source N(·|0, I) is given as an example.Source: §4.8 · pp. 26–27

4.8.1 Velocity parameterizations

Affine marginal velocity can be learned through x1-prediction or x0-prediction using Conditional Matching, whose gradients coincide with those of the Matching loss.Table 1 provides conversions between parameterizations.Source: §4.8.1 · pp. 27–28
The x1- and x0-prediction conversions have endpoint singularities that can cause practical problems, but endpoint velocity expressions can correct them in some cases.For an independent Gaussian source and EX1X1 = 0, the endpoint expressions are u0(x) = σ̇0x and u1(x) = α̇1x.Source: §4.8.1 · pp. 28–29

4.8.2 Post-training velocity scheduler change

All schedulers theoretically yield the same sampling at time t = 1, and a closed-form transformation adapts a trained velocity field to another scheduler.The transformation keeps the data coupling fixed; its stated potential benefit is improved sample efficiency and generation quality.Source: §4.8.2 · p. 29

4.8.3 Gaussian paths

VP and VE Gaussian paths subsume diffusion-model marginal probabilities, but do not exactly reproduce the source distribution at t = 0.For Gaussian paths, the score is proportional to x0-prediction and can be converted to other parameterizations.Source: §4.8.3 · pp. 30–31
A consequence of the conversion formulas is that, for Gaussian paths, the marginal velocity is a gradient and therefore kinetic-optimal for the fixed marginalized Gaussian probability path p_t(x).Source: §4.8.3 · p. 31

4.9 Data couplings · 4.10 Conditional generation and guidance

These sections explain how source–target couplings shape Flow Matching and how guidance conditions generation. They cover paired and multisample couplings, conditional models, and classifier-based guidance.

4.9 Data couplings

FM training assumes a coupling between source and target distributions that preserves their marginals, such as independent samples or paired samples from a dataset.The sections explore concrete couplings for training Flow Matching models.Source: §4.9 · p. 31

4.9.1 Paired data

For paired tasks, noisy reverse-dependency sampling forms source–target pairs; Liu et al. (2023); Albergo et al. (2024) report applications that outperform guided diffusion.Demonstrated applications include super-resolution, in-painting, and de-blurring; noise supports a source density and encourages diversity.Source: §4.9.1 · p. 32

4.9.2 Multisample couplings

For k > 1, multisample couplings reduce transport cost versus independent coupling and approach quadratic OT cost, inducing straighter trajectories as k → ∞ (Pooladian et al., 2023; Tong et al., 2023).The implicit joint preserves the marginals; k = 1 reduces to independent coupling.Source: §4.9.2 · p. 32

4.10 Conditional generation and guidance

Guidance controls generated samples using a signal, with applications including image-to-image translation and text-to-image generation; the setup assumes labeled target samples (x1, y).Source: §4.10 · p. 33

4.10.1 Conditional models

Conditional models learn q(x1|y) by aggregating conditional probability paths, with unchanged FM/CFM losses and one network over y; guidance is harder for complex, non-repeating signals such as captions.The conditional target distribution is q(x1|y).Source: §4.10.1 · p. 33

4.10.2 Classifier guidance and classifier-free guidance

Dhariwal and Nichol (2021) show classifier guidance outperforms the conditional model for both class and text conditioning; CFG’s exact sampling distribution is unknown, yet it was most popular at writing.The conditional and unconditional scores are learned simultaneously using the same model.Source: §4.10.2 · p. 34

5 Non-Euclidean Flow Matching

The section extends Flow Matching to Riemannian manifolds, where it learns flows through tangent-space velocity fields and offers simulation-free training when suitable conditional paths are available. It develops manifold probability paths, marginalization, and conditional flows, while noting limitations when geodesics or premetrics have singularities or require simulation.

5.1 Riemannian manifolds

The framework considers complete, connected, smooth Riemannian manifolds with a metric that defines inner products on tangent spaces.Riemannian manifolds generalize Euclidean notions of distance and angle and can represent data such as locations on Earth or protein backbones.Source: §5.1 · p. 35, §5 · p. 35

5.2 Probabilities, flows and velocities on manifolds

Riemannian Flow Matching learns a tangent-space velocity field that generates a target probability path and compares velocities using a Bregman divergence on each tangent plane.The path has source and target marginals p and q; the divergence can be defined using the Riemannian inner product and a strictly convex function on each tangent space.Source: §5.2 · p. 36

5.3 Probability paths on manifolds

Marginal probability paths on manifolds are constructed by integrating conditional paths against the target distribution and imposing endpoint boundary constraints.The conditional paths may satisfy the constraints through a conditional coupling π0|1(x0|x1) = π0,1(x0, x1)/q(x1).Source: §5.3 · p. 36

5.4 The Marginalization Trick for manifolds

Under its assumptions, averaging conditional manifold velocity fields yields a marginal velocity field that generates the marginal probability path.The theorem assumes smooth conditional paths and velocities, positivity of the marginal path, and either bounded support of q or a compact manifold.Source: §5.4 · pp. 36–37

5.5 Riemannian Flow Matching loss

The gradients of the Riemannian Flow Matching and Riemannian Conditional Flow Matching losses coincide.This establishes that conditional training preserves the gradient of the marginal Riemannian Flow Matching objective.Source: §5.5 · p. 37

5.6 Conditional flows through premetrics

On manifolds with closed-form exponential and logarithmic maps, geodesic conditional flows provide a simulation-free training recipe and recover Euclidean conditional OT flows.Chen and Lipman (2024) proposed geodesic conditional flows; with κ(t) = t, the construction reduces to the Euclidean conditional OT flow.Source: §5.6 · p. 38
Premetric conditional flows can avoid difficult geodesic computations or impose a desired bias, but require in-training simulation; their velocities can still be recovered accurately compared with approximated manifold scores.Chen and Lipman (2024) give spectral distances on general geometries as an example of alternative premetrics.Source: §5.6 · p. 38
Geodesic and premetric conditional flows can have singularities at critical points, but the problematic sets are usually of zero volume and are not reported to cause practical problems.An augmented scheduler can make geodesic flows globally smooth, while a relaxed non-degeneracy condition addresses zero-gradient points for premetric flows.Source: §5.6 · p. 39

6 Continuous Time Markov Chain Models

Continuous Time Markov Chains (CTMCs) provide an alternative generative-model foundation for discrete finite-state data and are building blocks for Discrete Flow Matching (DFM) (Campbell et al. (2024); Gat et al. (2024)). The section defines their state space, rate-based dynamics, marginal evolution, and probability-preserving velocities.

6 Continuous Time Markov Chain Models

CTMCs are presented as an alternative to flow for generating discrete, finite-state data and as building blocks for DFM.The guide positions this section as analogous to its treatment of flows as building blocks for Flow Matching.Source: §6 · p. 40

6.1 Discrete state spaces and random variables

The discrete state space is S = T^d, where T = [K] is a finite vocabulary and each state is a d-token tuple.A random variable X takes values in S and is described by a probability mass function p_X.Source: §6.1 · p. 40

6.2 The CTMC generative model

CTMCs model discrete data as time-dependent Markov chains whose transition probabilities are determined by rates between states.The transition kernel has first-order form p_{t+h|t}(y|x) = δ(y, x) + h u_t(y, x) + o(h); the initial distribution is p, and rate conditions ensure valid probabilities.Source: §6.2 · p. 40
Naive Euler simulation incurs o(h) update-probability errors, so the step h must be sufficiently small to keep updates valid.The section also presents an alternative Euler method intended to ensure valid PMFs for any h > 0 while maintaining the o(h) local error.Source: §6.2 · p. 41

6.3 Probability paths and Kolmogorov Equation

Continuous time-continuous rates yield a unique marginal solution to the Kolmogorov Equation, whose probability flux has a divergence structure.For CTMCs, the solution exists for all t ∈ [0, 1) without extra conditions; the flux is j_t(y, x) := u_t(y, x)p_t(x).Source: §6.3 · p. 41

6.3.1 Probability preserving velocities

Adding a rate-condition-compliant divergence-free velocity preserves the marginal probability during sampling.Such velocities solve the divergence-free velocity equation and leave the Kolmogorov Equation satisfied.Source: §6.3.1 · p. 42

7 Discrete Flow Matching

Discrete Flow Matching extends the Flow Matching recipe to discrete state spaces by learning a continuous-time Markov chain that follows a probability path from a source to a target distribution. The section develops marginal, factorized, and mixture paths, along with training losses and sampling methods, probability-preserving velocity variants, and PyTorch implementations.

7 Discrete Flow Matching

DFM learns CTMC velocities that generate a probability path from source PMF p to target PMF q, training them with a Bregman-divergence loss.This adapts the continuous Flow Matching blueprint to discrete state spaces.Source: §7 · p. 42

7.1 Data and coupling

Source and target samples may use independent or general couplings, including paired documents for translation and independent or special-token sources for text generation.These coupling choices determine how source and target samples are related.Source: §7.1 · p. 42

7.3 The Marginalization Trick

Conditioning probability paths and velocities on Z yields a marginal velocity that generates the marginal path under smoothness and positivity assumptions.Strict positivity can be ensured by mixing in an arbitrarily small amount of the uniform distribution.Source: §7.2 · p. 42, §7.3 · p. 43

7.4 Discrete Flow Matching loss

The Discrete and Conditional Discrete Flow Matching losses have identical gradients, and the conditional loss is minimized by the marginal velocity.The equivalence permits training from conditional velocities while targeting the marginal velocity.Source: §7.4 · pp. 43–44

7.5 Factorized paths and velocities

Factorized velocities restrict transitions to states differing in at most one token, reducing the model output dimension from K^d to d · K.The model predicts per-coordinate rates rather than rates for every possible state.Source: §7.5 · p. 44, Figure 14 · p. 44

7.5.1 Simulating CTMC with factorized velocities

Factorized CTMCs can be simulated coordinate-wise using independent transitions and the Euler method.Each coordinate’s transition probabilities are determined by its own factorized velocity.Source: §7.5.1 · p. 44

7.5.2 Building probability paths with factorized velocities

Factorized conditional paths produce factorized marginal velocities, enabling paths between arbitrary source and target PMFs through per-coordinate path and velocity construction.The velocity construction solves an underdetermined linear system with |T| unknowns rather than one over the full state space.Source: §7.5.2 · pp. 45–46

7.5.3 Conditional Discrete Flow Matching loss for factorized velocities

The factorized conditional loss uses a Bregman divergence over each coordinate’s rate set.This defines the conditional training objective for factorized velocities.Source: §7.5.3 · p. 46

7.5.4 Mixture paths

The generalized KL loss for mixture paths provides an evidence lower bound on target-distribution likelihood and is commonly used for evaluation.The likelihood refers to the model marginal generated at t = 1.Source: §7.5.4 · p. 48
Sampling proceeds coordinate-wise by drawing from the learned posterior and then taking an Euler step with the corresponding velocity.The posterior draw selects the next coordinate value; the Euler update determines whether to set it or keep its current value.Source: §7.5.4 · p. 49
Probability-preserving velocities can be difficult to find without learning additional quantities, but an iid source and independent coupling provide a closed-form case.Adding a divergence-free component to the velocity still generates the same marginal path.Source: §7.5.4 · p. 49
The authors provide a pure-PyTorch standalone implementation and a flow_matching library example for training discrete flow with arbitrary data coupling.Source: §7.5.4 · p. 50

8 Continuous Time Markov Process Models

This section unifies flow and discrete continuous-time Markov chain models in a generative framework for general state spaces, describing processes through generators and their evolution along probability paths via Kolmogorov equations. It derives adjoint forward equations under a condition linking criteria for probability paths and characterizes generators for discrete and Euclidean state spaces.

8.1 General state spaces and random variables

The framework allows general state spaces and probability measures, including cases where distributions lack a density with respect to a common reference measure.It encompasses spaces such as Euclidean, discrete, and Riemannian spaces, as well as products for joint multimodal generation.Source: §8.1 · p. 52

8.2 The CTMP generative model

A continuous-time Markov process has no memory, and its transition kernel together with its initial distribution uniquely determines the process.The generator provides a first-order approximation to the transition kernel and is easier to parameterize.Source: §8.2 · p. 53

8.2.2 Examples of CTMP models

Flows have deterministic transitions, diffusions have Brownian-driven transitions, and CTMCs are determined by rates, illustrating distinct Markov-process classes.The framework treats these processes as examples of the general Markov-process model.Source: §8.2 · p. 53

8.2.1 Formal definition of generator

Feller-process generators are defined as operators on test functions, with a one-to-one correspondence between a generator and a Feller process.The Feller conditions impose time continuity and preservation of functions that vanish at infinity.Source: §8.2.1 · p. 54

8.3 Probability paths and Kolmogorov Equation

Theorem 17 connects a probability path to its generator through the Kolmogorov forward equation (KFE), its adjoint form when densities exist, and generator-induced marginals.The KFE formulation also applies when probability distributions lack densities; equivalence between the KFE and adjoint KFE requires a differentiable density under the stated regularity conditions.Source: §8.3 · p. 58
Under general state-space settings, no known result establishes that condition (3) implies (1), so the authors assume it and note it holds for state spaces of interest in machine learning.The assumption is made under the stated theorem conditions.Source: §8.3 · p. 58

8.3.1 Examples of KFEs

The adjoint KFE recovers the continuity equation for flows, the Fokker–Planck equation for diffusions, the jump continuity equation, and the CTMC KFE.These examples connect adjoint generators to familiar equations for flows, diffusions, jumps, and discrete-state Markov chains.Source: §8.3.1 · pp. 59–60

8.4 Universal representation theorem

Under weak regularity assumptions, generators of Feller processes are characterized by a rate transition matrix for finite discrete spaces and velocity, diffusion, and jump components for Rd.The discrete representation corresponds to a CTMC; the Euclidean representation combines the listed generator components.Source: §8.4 · p. 61

9 Generator Matching

This section presents Generator Matching as a framework for generative modeling across arbitrary modalities and general Markov processes that unifies existing generative models. It covers how the framework combines models while preserving a target probability path and extends to multimodal generation by reusing models for individual modalities.

9 Generator Matching

GM supports arbitrary data modalities and general Markov processes, unifying diffusion models, discrete diffusion, and prior Flow Matching variants.The framework also enables new models, combinations of model classes, and models across modalities.Source: §9 · p. 62

9.1 Data and coupling

The source and target distributions may be arbitrary probability measures on a general state space, coupled independently or through any joint distribution with those marginals.Source: §9.1 · p. 62

9.2 General probability paths

A probability path can be built by sampling a latent state and then a conditional path, with the resulting marginal path constrained to begin at p and end at q.Source: §9.2 · p. 62

9.3 Parameterizing a generator via a neural network

A generator can be parameterized by applying a fixed linear operator to a learned function constrained to a convex set, specializing to natural parameterizations such as flow vector fields.Source: §9.3 · pp. 62–63

9.4 Marginal and conditional generators

The General Marginalization Trick constructs a marginal-path generator by averaging conditional generator parameters under the posterior distribution of the latent state given the current state.This yields the neural-network training target and encompasses earlier marginalization tricks as special cases.Source: §9.4 · p. 64

9.5 Generator Matching loss

The conditional Generator Matching loss is tractable when conditional parameters can be derived analytically, and its gradients match those of the marginal loss.Its minimizer is the marginal generator parameterization, and the stated properties require a Bregman divergence.Source: §9.5 · p. 65

9.6 Finding conditional generators as solutions to the KFE

Conditional generators are challenging to derive analytically in general, but examples include jump models for mixture paths on arbitrary state spaces and density paths under stated regularity assumptions.For density paths, a target-state-independent jump distribution yields a valid jump process; extending the model to multiple dimensions makes the distribution location-dependent and supports generation models.Source: §9.6 · pp. 66–68

9.7 Combining Models

GM can combine models for the same state space because the generator and the KFE are linear, allowing solutions to be combined as in linear algebra.Source: §9.7 · p. 68
Proposition 3 states that specified combined generators solve the KFE for a marginal probability path, defining generative models with that path.Source: §9.7 · p. 68
Markov superpositions combine jump and flow models into a piecewise-deterministic ODE-and-jump process; Holderrieth et al. (2024) report performance improvements from such examples.Source: §9.7 · pp. 68–69
Langevin dynamics are divergence-free and can be added with arbitrary βt ≥ 0 to any generative model, enabling stochastic sampling for diffusion models.Source: §9.7 · p. 69
A Metropolis-Hastings scheme is divergence-free and can be added to any generative model following the same probability path.Source: §9.7 · p. 69

9.8 Multimodal models

Factorized conditional probability paths enable multimodal models to recycle modality-specific models, including combinations of discrete and continuous Flow Matching.Holderrieth et al. (2024) provide a rigorous treatment, and Campbell et al. (2024) instantiate the construction for multimodal protein generation.Source: §9.8 · p. 70

10 Relation to Diffusion and other Denoising Models

The section frames diffusion as a special case of Flow Matching and Generator Matching, then extends the same perspective to denoising models in other state spaces.

10.1 Time convention

A first difference between denoising diffusion models and Flow Matching is time parameterization: diffusion time is inverted, denoted r and ranging from 0 to ∞, while Flow Matching uses t.The conventions are connected by a strictly monotonically decreasing mapping k:(0,1]→[0,+∞), with k(1)=0 and lim(t→0) k(t)=+∞.Source: §10.1 · p. 71

10.2 Forward process vs. probability paths

A forward diffusion SDE defines a valid Flow Matching conditional path, and affine drift yields Gaussian paths with analytically computable conditionals.The marginal approaches a Gaussian only approximately: no finite diffusion time gives an exact Gaussian marginal.Source: §10.2 · pp. 71–72

10.3 Training a diffusion model

Diffusion training is equivalent to Flow Matching with an independent-coupling Gaussian path from an affine-drift SDE and score parameterization.The x0-prediction training loss is Denoising Score Matching; score and x0 parameterizations are theoretically interchangeable but singular at t = 0.Source: §10.3 · p. 72

10.4 Sampling

Under the same Gaussian-path conditions, diffusion ODE sampling matches Flow Matching, while stochastic SDE sampling matches Generator Matching.The stochastic processes have the same marginals in theory, but estimation and simulation errors make the optimal noise level β_t unknown in practice.Source: §10.4 · p. 73

10.5 The role of time-reversal and the backward process

Time-reversing diffusion marginals, generating their probability path, and solving the corresponding Fokker-Planck equation are equivalent generative-modeling problems.A true time-reversal matches joint distributions across time, but the probability flow ODE only matches marginals; the latter is used for low-evaluation sampling and a true reversal may be suboptimal.Source: §10.5 · p. 74

10.6 Relation to Other Denoising Model

Denoising models in other state spaces can be viewed as Generator Matching models that noise data forward and then reverse that process.This informal characterization uses diffusion time, a forward-constructed probability path, and a time-reversal solution to the KFE, with possible exceptions.Source: §10.6 · p. 75

Appendix · A Additional proofs · A.1 Discrete Mass Conservation

The appendix establishes that the Kolmogorov equation with the stated rate conditions preserves PMFs, and characterizes when those dynamics are generated by a transition kernel. The proof uses conservation of total mass, nonnegativity, and uniqueness of solutions.

A.1 Discrete Mass Conservation

A solution starting from a PMF remains a PMF when its rates are continuous in time and satisfy the stated rate conditions.This is Lemma 1, for t ∈ [0, 1].Source: §A.1 · p. 81
Summing both sides of the Kolmogorov equation establishes the mass-conservation condition used in the PMF-preservation proof.The proof also requires nonnegativity, since a PMF must have nonnegative entries.Source: §A.1 · p. 81
Nonnegativity follows from an invariant-set argument: at a zero-probability boundary, the rate conditions make the relevant derivative nonnegative.The proof applies the convex invariant-set result to the PMF constraint and uses nonnegative off-diagonal rates.Source: §A.1 · p. 81
Theorem 13 equates satisfying the Kolmogorov equation and rate conditions with being generated by a transition kernel.The theorem states the equivalence for t ∈ [0, 1).Source: §A.1 · p. 81
If a transition kernel generates the marginals, taking the small-time limit yields the Kolmogorov equation and the rate conditions.A negative off-diagonal rate would make a transition probability negative for small h; a nonzero column sum would violate normalization.Source: §A.1 · pp. 81–82
Conversely, the equation and rate conditions yield a generating transition kernel by uniqueness, with the transition-kernel semigroup property.The proof constructs conditional transition laws from the Kolmogorov equation and uses Lemma 1 to show they are PMFs.Source: §A.1 · p. 82

A.2 Manifold Marginalization Trick

Under Assumption 2, a conditionally integrable velocity field that generates each conditional probability path also generates the marginal probability path. The proof establishes the marginal continuity equation and verifies the marginal field’s required regularity.

A.2 Manifold Marginalization Trick

Under Assumption 2, if the conditionally integrable field u_t(x|x_1) generates p_t(·|x_1), its marginal field u_t(·) generates p_t(·).This is the Manifold Marginalization Trick.Source: §A.2 · p. 82
The proof derives the marginal continuity equation by integrating the conditional equation and exchanging differentiation and integration under the stated regularity conditions.The exchange is justified by the Leibniz rule when the conditional path and field are C1 in t and x and q has bounded support, or when M is compact.Source: §A.2 · p. 82
The proof also verifies that the marginal field is integrable and locally Lipschitz, using the same arguments as in Theorem 3.It uses conditional generation and Theorem 9, along with strict positivity of p_t(x) and the marginal-velocity formula in (5.10).Source: §A.2 · p. 82

A.3 Regularity assumptions for KFE

Assumption 5 holds under relatively weak assumptions, but no known result gives regularity assumptions for the KFE across general state spaces and Markov processes. The section points to uniqueness results for specific settings and notes the assumption holds for state spaces of interest to machine-learning practitioners.

A.3 Regularity assumptions for KFE

Assumption 5 holds under relatively weak assumptions, but general regularity assumptions for KFE uniqueness across state spaces and Markov processes remain unstated.The assumption is presented explicitly, and the authors say it holds for any state space of interest to machine-learning practitioners.Source: §A.3 · p. 82
For flows in R^d and manifolds, uniqueness results include Villani et al. (2009), DiPerna and Lions (1989), and Ambrosio (2004).The cited flow result in Villani et al. is the mass conservation formula on page 15.Source: §A.3 · p. 83
For diffusion in R^d and manifolds, Villani et al. (2009) provide a diffusion theorem on page 16.Source: §A.3 · p. 83
For general Itô SDEs in R^d, cited uniqueness results include Figalli (2008), Kurtz (2011), and Bogachev et al. (2022).The cited results are Theorems 1.3 and 1.4 of Figalli and Corollary 1.3 of Kurtz.Source: §A.3 · p. 83
For discrete state spaces, the KFE is a linear ODE with a unique solution when its coefficients are continuous.The passage points to theorem 13 for this condition.Source: §A.3 · p. 83