PaperlayerView exact revision on arXiv

Source-linked AI summary

Generator Matching: Generative modeling with arbitrary Markov processes

Peter Holderrieth, Marton Havasi, Jason Yim, Neta Shaul, Itai Gat, Tommi Jaakkola, Brian Karrer, Ricky T. Q. Chen, Yaron Lipman

arXiv:2410.20587v3cs.LGcs.AI

TL;DR

Generative methods have been developed for different data types, while sharing an iterative Markov-process structure. Generator Matching learns generators to provide a common framework for these methods and their combinations; the paper reports competitive image and multimodal protein-generation results, including improvements from flow-jump superposition.

  • Problem

    Generative methods cover different data types, but share an iterative, step-wise Markov-process structure.

    Source: §1 · p. 1

  • Method

    Generator Matching learns generators for Markov processes, unifying existing methods and supporting model combinations across process classes.

    Source: §1 · pp. 1–2

  • Results

    GM achieves competitive results in image and multimodal protein generation, and flow-jump Markov superposition boosts both component models, with mixed sampling improving current flow-model SOTA.

    Source: §1 · p. 2, §9 · p. 10

  • Takeaways & Limitations

    The linearity of generators and the Kolmogorov Forward Equation supports combining generative models for the same state space.

    Source: §7.1 · p. 8

    The generator-process correspondence and the characterization rely on regularity assumptions.

    Source: §4 · p. 4

Abstract

from arXiv · show

We introduce Generator Matching, a modality-agnostic framework for generative modeling using arbitrary Markov processes. Generators characterize the infinitesimal evolution of a Markov process, which we leverage for generative modeling in a similar vein to flow matching: we construct conditional generators which generate single data points, then learn to approximate the marginal generator which generates the full data distribution. We show that Generator Matching unifies various generative modeling methods, including diffusion models, flow matching and discrete diffusion models. Furthermore, it expands the design space to new and unexplored Markov processes such as jump processes. Finally, Generator Matching enables the construction of superpositions of Markov generative models and enables the construction of multimodal models in a rigorous manner. We empirically validate our method on image and multimodal generation, e.g. showing that superposition with a jump process improves performance.

1 INTRODUCTION

Recent generative methods use iterative updates, including diffusion and flow approaches, and related methods have been developed for multiple data modalities. Generator Matching builds on their shared Markov-process structure to unify existing methods and support new models and combinations.

1 INTRODUCTION

  • Earlier work moved from single-pass generation to iterative diffusion and flow methods, with related constructions later extending to discrete and manifold data.The cited approaches include VAEs (Kingma, 2013), GANs, diffusion models (Song et al., 2020; Ho et al., 2020), and flow matching (Lipman et al., 2022; Liu et al., 2022; Albergo et al., 2023).Source: §1 · p. 1
  • Generator Matching learns generators for Markov processes, whose shared step-wise structure underlies the generative methods described here.The framework uses scalable training objectives and applies to arbitrary state spaces.Source: §1 · p. 1
  • GM unifies diffusion, flow matching, stochastic interpolants, and discrete diffusion across data modalities.It also supports new model classes and combinations across different Markov-process types.Source: §1 · p. 1
  • The paper identifies jump models and Markov superpositions as contributions, alongside multimodal model construction and competitive image and protein-generation results.The reported model combinations include ensembles and models built from single modalities.Source: §1 · p. 2

2 GENERATIVE MODELING VIA PROBABILITY PATHS

Generator Matching sets up generation through probability paths between a simple prior and the data distribution. A conditional path is sampled using a data point, and its marginal path is designed to have the prior and data distribution as endpoints.

2 GENERATIVE MODELING VIA PROBABILITY PATHS

  • The goal is to generate novel samples from a data distribution on a state space that may be Euclidean, discrete, manifold-valued, or multimodal.GM also applies to distributions without densities, including those with discrete support.Source: §2 · p. 2
  • A conditional probability path is a time-varying distribution given a data point, and sampling first selects that point before drawing from the path.This hierarchical construction induces the marginal probability path and makes training scalable.Source: §2 · p. 2
  • The path is usually chosen to begin at the simple prior and end at the data point, so the marginal path connects the prior to the data distribution.The stated design principle requires p_0 = p_simple and p_1 = p_data.Source: §2 · p. 2
  • Mixtures apply to arbitrary state spaces, while geometric averages are given for Euclidean spaces.For the geometric average, x_t = σ_t x_0 + α_t z, with functions subject to the stated endpoint conditions.Source: §2 · p. 3
  • The framework includes probability paths built from diffusion forward processes or interpolant functions, as well as conditioning on more general latent variables.The passage notes that diffusion literature reverses the time convention, with t = 0 corresponding to data.Source: §2 · p. 3

3 MARKOV PROCESSES

A Markov process predicts the future from the present without needing the earlier history. In Generator Matching, it supplies step-wise transitions whose marginals follow a specified path from the simple prior to the data distribution.

3 MARKOV PROCESSES

  • The Markov assumption says that, given the present state, earlier states do not affect the prediction of a future state.Formally, the conditional probability of a future event given the full history equals that given the most recent state.Source: §3 · p. 3
  • A Markov process is paired with a transition kernel and an initial distribution, with a one-to-one correspondence under the Markov assumption.The kernel assigns a probability distribution over the next state for each current state.Source: §3 · p. 3
  • GM seeks a process whose marginals follow a specified path, then generates data by repeatedly sampling its next state until time 1.The initial state is sampled from the simple prior, and the target marginal at time 1 is the data distribution.Source: §3 · p. 3
  • Because general transition kernels are difficult to parameterize with neural networks, small-step approximations motivate representing the process through a generator.The passage connects this approach to diffusion models’ use of simple parametric approximations for small time steps.Source: §3 · p. 3

4 GENERATORS

Generator Matching represents Markov processes through generators, which describe the first-order change of transition kernels and can be parameterized to define a process. The paper characterizes generator forms on discrete and Euclidean spaces, including jump components in Euclidean space.

4 GENERATORS

  • A generator is the first-order derivative of a transition kernel, and generators let models learn the infinitesimal evolution of Markov processes.The transition-kernel expansion is defined through its action on test functions.Source: §4 · pp. 3–4
  • Test functions probe probability distributions, and a family that characterizes distributions distinguishes them through their expectations.The passage gives infinitely differentiable compactly supported functions as an example for R^d.Source: §4 · pp. 3–4
  • Under mild regularity assumptions, a generator uniquely corresponds to a Markov process, motivating parameterization through a generator L_θ.Directly parameterizing a linear operator is difficult, so the paper also considers subclasses of processes.Source: §4 · pp. 4–5
  • For finite discrete spaces, generators are rate transition matrices; for R^d, they combine components including velocity, diffusion, and jump measures.The Euclidean representation is stated as a sum of generator components.Source: §4 · p. 5
  • The characterization exhaustively describes the design space for discrete spaces and R^d, where learning diffusion coefficients or jump models has been little explored.The passage contrasts these possibilities with prior work on flow components and fixed diffusion coefficients.Source: §4 · p. 5

5 KOLMOGOROV FORWARD EQUATION AND MARGINAL GENERATOR

The Kolmogorov Forward Equation (KFE) describes how a Markov process’s generator governs the evolution of its marginal probabilities. Generator Matching simplifies finding a generator for a marginal path by first solving the KFE for conditional paths, yielding distinct Markov processes with matching marginal paths.

5 KOLMOGOROV FORWARD EQUATION AND MARGINAL GENERATOR

  • A generator satisfying the KFE generates the specified probability path when initialized from its initial distribution.The KFE expresses the evolution of expectations of test functions as ∂t ⟨pt, f⟩ = ⟨pt, Ltf⟩.Source: §5 · p. 5
  • Solving the KFE for each conditional path simplifies construction of a process for the marginal path.The framework’s principle is to derive a conditional generator satisfying the KFE for pt(·|z).Source: §5 · p. 6
  • A jump model provides another solution for the CondOT geometric-average path, matching common flow models’ marginal path while producing different sample paths.Figure 2 depicts different individual trajectories but matching marginal histograms up to approximation error.Source: §5 · p. 6, Figure 2 · p. 7
  • A pure-diffusion SDE without drift is presented as a solution for the mixture path, with reflection at the data-support boundaries.Its diffusion coefficient is learned, and the model specifies how much noise to add to the current state.Source: §5 · p. 6

6 GENERATOR MATCHING

Generator Matching parameterizes a marginal generator and trains a neural network to approximate it. Conditional Generator Matching makes this tractable: for Bregman divergences, its gradients match those of the Generator Matching loss.

6 GENERATOR MATCHING

  • The network approximates a generator parameterization, such as flow velocity, diffusion coefficient, or jump rate.The target marginal parameterization is unknown, making the direct Generator Matching objective intractable.Source: §6 · p. 7
  • For any Bregman divergence, Generator Matching and Conditional Generator Matching have the same gradients, so minimizing the conditional loss also minimizes the Generator Matching loss.The passage states that this property requires D to be a Bregman divergence.Source: §6 · p. 7
  • Conditional Generator Matching provides a scalable training objective and supports a broad class of loss functions.The paper says it universally characterizes loss functions, including new loss functions for diffusion models; some losses can yield ELBO bounds.Source: §6 · p. 7

7 APPLICATIONS OF GENERATIVE MATCHING THEORY

Generator Matching’s generality supports new models and applications. The linearity of generators and the KFE also permits combining models that generate the same probability path.

7 APPLICATIONS OF GENERATIVE MATCHING THEORY

  • Generator Matching’s generality provides use cases beyond its unifying framework for existing generative models.The section introduces applications of the theory and states that it gives rise to new models.Source: §7 · p. 8

7.1 COMBINING MODELS

  • Because the generator and KFE are linear, generators solving the KFE for the same path can be combined to define a generative model with that marginal path.Source: §7.1 · p. 8

1. Markov superposition: α1 tLt +α2

The section introduces a Markov superposition involving weighted generators.

1. Markov superposition: α1 tLt +α2

  • The section names a combination of generators a Markov superposition.Source: §1 · p. 8

2. Divergence-free components: Lt + βtLdiv

The section combines a generator with a divergence-free component and describes forward- and backward-time generators for the same probability path.

2. Divergence-free components: Lt + βtLdiv

  • The construction adds a divergence-free component, Ldiv, to Lt with a nonnegative weight βt.The supplied equation fragments identify Lt and βtLdiv as the components.Source: §2 · p. 8
  • βt is constrained to be nonnegative.Source: §2 · p. 8
  • Lt solves the KFE for pt in forward time, while L̄t solves it in backward time.Source: §2 · p. 8

3. Predictor-corrector: α1 tLt + α2

The predictor-corrector construction combines Markov generative models, including different process classes. For multimodal generation, Generator Matching forms a product-space model from component paths and generators.

3. Predictor-corrector: α1 tLt + α2

  • Markov superpositions combine generative models from different classes, such as flow and jump models, using separate networks or one jointly trained network.Divergence-free components can be obtained with MCMC methods including Hamiltonian Monte Carlo and Langevin dynamics.Source: §3 · p. 8

7.2 MULTIMODAL AND HIGH-DIMENSIONAL GENERATIVE MODELING

  • For state spaces S1 and S2, Generator Matching constructs a joint multimodal model on their product space and can reduce a high-dimensional KFE problem to one dimension.Source: §7.2 · p. 8
  • The conditional generator is assembled component-wise from solutions on each state space; the marginal generator can depend on all dimensions while sampling updates components independently.The loss is the sum of the component-wise losses.Source: §7.2 · p. 8

8 RELATED WORK

Generator Matching recasts several existing generative methods within a framework based on Markov processes and extends the design space to additional models and combinations.

8 RELATED WORK

  • The overview also points to models on manifolds and multimodal state spaces beyond the Rd and discrete cases discussed in this selection.Source: §8 · p. 8
  • Denoising diffusion models, flow matching, rectified flows, and stochastic interpolants are described as instances or extensions of Generator Matching.In this perspective, denoising diffusion uses a flow model learned with the CGM loss, and a divergence-free Langevin component can be added for stochastic sampling.Source: §8 · p. 9
  • In discrete spaces, Generator Matching recovers continuous-time Markov-chain generative models, including discrete diffusion methods [Campbell et al., 2022; Santos et al., 2023; Gat et al., 2024].Autoregressive paths with fixed jump times can also recover common language-model training as an edge case.Source: §8 · p. 9
  • Benton et al. (2024) focus on recovering existing denoising diffusion models, while this work aims to characterize the broader design space of Markov generative models.The authors name jump models on Rd and Markov superpositions among the unexplored parts of that space.Source: §8 · p. 9

9 EXPERIMENTS

The experiments examine jump models, Markov superpositions, and multimodal generation. They report image-generation gains from combining flow and jump models and state-of-the-art protein results with increased diversity from adding jumps.

9 EXPERIMENTS

  • The experiments focus on jump models, combining model classes, and models for multiple modalities; additional small-scale studies examine toy data and new Bregman divergences.Source: §9 · pp. 9–10
  • Markov superposition of flow and jump models boosts the performance of each, while mixed sampling improves on the current flow-model SOTA.Mixed sampling uses a second-order sampler for flow and Euler sampling for jumps; Table 2 reports image-generation FID scores.Source: Table 2 · p. 9, §9 · p. 10
  • The protein experiment combines a flow on Rd, jumps on SO(3)d, and a discrete Markov model on {1, . . . , 20}d, using pre-trained MultiFlow without fine-tuning.The authors pseudo-marginalize conditional jumps by predicting x1 and taking a conditional step.Source: §9 · p. 10
  • Adding a jump model to MultiFlow yields state-of-the-art protein-generation performance and greatly increases the diversity metric.Table 3 reports protein-generation results; diversity is the share of unique proteins passing the designability check.Source: §9 · p. 10, Table 3 · p. 10

10 DISCUSSION

This section develops the theoretical and technical foundations of Generator Matching, covering its relation to density evolution and time-reversal, generator representations and results, and multimodal training objectives. It also examines Markov processes for specified probability paths, including diffusion, jump, and flow constructions, their discretization and performance, and related implementations.

10 DISCUSSION

  • Generator Matching unifies diffusion, flow matching, and discrete diffusion through linear generator and KFE structure, while enabling model superpositions and multimodal generation.The authors identify learning diffusion coefficients, exploring richer jump models, and developing better samplers or distillation as future directions.Source: §10 · p. 10

A.1 SETUP AND DEFINITIONS

  • The framework assumes a Polish state space and test functions that distinguish probability measures, while KFE sufficiency for general processes is assumed rather than established.The authors note that uniqueness results exist for specific spaces and process classes, but they know of no general regularity result; they say the assumption holds for state spaces of interest to practitioners.Source: §A.1 · p. 16, §A.2 · p. 18

A.3 ADOINT KFE

  • The adjoint KFE describes density evolution when densities exist and generalizes the continuity and Fokker-Planck equations.For flows, the adjoint generator yields the continuity equation; for diffusion, it yields the Fokker-Planck equation.Source: §A.3 · p. 19, §A.5.1 · p. 22, §A.5.2 · p. 23

A.4 TIME-REVERSAL

  • Generative modeling needs only a weak time-reversal matching marginals, not a strong reversal matching joint distributions; the probability flow ODE is an example.The authors restrict their framework to weak reversals because finding strong reversals adds mathematical complexity and may give suboptimal results.Source: §A.4 · p. 20

A.5.3 JUMPS; A.5.4 CONTINUOUS-TIME MARKOV CHAIN (CTMC)

  • Jump processes yield an adjoint KFE with an inflow term, while finite-state CTMCs use rate matrices and sample transitions from I + hQt.For CTMCs, the cited sampling matrix has columns summing to one.Source: §A.5.3 · p. 24, §A.5.4 · pp. 24–25

A.6 LINEAR PARAMETERIZATION OF GENERATORS

  • A linear parameterization represents generator families using a convex closed parameter set and a linear operator, with examples covering flows, diffusion, jumps, and continuous-time Markov chains.The representation maps a continuous function from the state space into the parameter set through the operator.Source: §A.6 · pp. 25–26

C.1 PROOF OF THEOREM 1; B SAMPLING WITH UNIVERSAL REPRESENTATION (ALG. 2)

  • The universal representation expresses generators through drift, diffusion, and jump components, and Algorithm 2 simulates the resulting process with o(h) generator error.The proof drops the dying term for processes running from t = 0 to t = 1 and, under a finite-discontinuity assumption, discards a compensation term.Source: B SAMPLING WITH UNIVERSAL… · p. 26, §C.1 · p. 27

C.1.1 TIME-INHOMOGENOUS CASE

  • A time-inhomogeneous process can be made time-homogeneous by augmenting its state with time, yielding a time-dependent spatial generator plus deterministic time evolution.The augmented process has time derivative one, zero time-related diffusion terms, and jump kernels supported at the current time.Source: §C.1.1 · pp. 27–28

C.2 PROOF OF PROPOSITION 1

  • The marginal generator constructed from conditional generators solves the KFE for the marginal probability path, including under a convex closed linear parameterization.Convexity and closedness ensure the marginal parameter remains in the parameter set.Source: §C.2 · pp. 28–29

C.3.2 BREGMAN DIVERGENCE AS SUFFICIENT CONDITION FOR PROPOSITION 2; C.3.3 BREGMAN DIVERGENCE AS NECESSARY CONDITION FOR PROPOSITION 2

  • For arbitrary data distributions, probability paths, and network parameterizations, matching the two gradients requires a Bregman divergence; for any Bregman divergence, the gradients agree.The two losses differ only by a constant in the model parameters when the divergence is Bregman.Source: §C.3.2 · p. 30, §C.3.3 · p. 32

C.4 PROOF OF PROPOSITION 3

  • A weighted combination of generators solving the same KFE remains a solution, but its time-derivative components must be combined, zeroed, or sign-flipped to preserve the KFE.These choices correspond to Markov superposition, divergence-free components, and predictor-corrector use cases.Source: §C.4 · p. 32

C.5 PROOF OF PROPOSITION 4

  • For multimodal models, factorized conditional paths yield a valid joint Markov process, and the marginal generator depends on each modality’s marginal posterior rather than cross-modal posterior dependencies.The generator can be parameterized with inputs from all modalities while scaling linearly with dimensionality; sampling updates each modality independently.Source: §C.5 · pp. 33–34

D.1 PATH ELBO LOSS - GENERAL CASE

  • The general path ELBO loss bounds the KL divergence between joint marginals over discretized process states using the data-processing inequality and KL chain rule.The derivation assumes transition kernels for a reference and parameterized Markov process and absolute continuity with respect to a reference measure.Source: §D.1 · pp. 36–37

D.2 ELBO LOSS FOR CONTINUOUS-TIME MARKOV CHAIN (CTMC)

  • For continuous-time Markov chains, the derived ELBO gives a way to train the rate kernel and recovers a previously known bound for continuous-space jump processes.The cited prior result is Opper & Sanguinetti (2007, equation (3)).Source: §D.2 · p. 38

E.1 OVERVIEW AND EMPIRICAL STUDY

  • On checkerboards, flow performs better with CondOT while jump performs better with mixture paths, showing no universally best path-model pairing.The study introduces models spanning mixture and CondOT paths with flow, diffusion, jump, and Markov superposition classes.Source: §E.1 · p. 40, E SYSTEMATIC STUDY OF PROBABILITY… · p. 40, Figure 5 · p. 41
  • Flow is more prone to discretization error than jump on mixture paths, while jump is more prone than flow on CondOT paths.The proposed explanation is that flow needs a high-Lipschitz vector field to transport mass on mixture paths, while many small jump movements challenge sampling on CondOT paths.Source: §E.1 · p. 40, Figure 7 · p. 43, Figure 8 · p. 44

E.2 DERIVATION OF FLOW MODEL FOR MIXTURE PATH

  • The mixture-path flow solution remains valid despite a discontinuous vector field with infinite Lipschitz constant.A cross-entropy classifier parameterization is mentioned as a future possibility, but is not considered in this work.Source: §E.2 · p. 45

E.3 DERIVATION OF DIFFUSION SOLUTION TO MIXTURE PATH

  • For the mixture probability path, the section seeks a forward-diffusion KFE solution represented by an SDE, with the noise parameter later taken toward zero.The passage establishes the derivation’s target and setup, not an empirical result.Source: §E.3 · p. 45
  • A reflected Brownian motion with a uniform initial distribution generates the conditional mixture path for a compact-support prior.The reflected SDE satisfies the required boundary condition; it can be simulated with o(h)-approximation error.Source: §E.3 · p. 47

E.4 DERIVATION OF JUMP MODEL FOR MIXTURE PATH

  • A jump process with intensity λt(x|z) = κ̇t/(1 − κt) and jumps to z satisfies the KFE for the mixture path.Source: §E.4 · pp. 47–48

E.5 DERIVATION OF FLOW SOLUTION TO CONDOT PATH

  • The vector field for the CondOT path satisfies the continuity equation, so its corresponding flow generates that path.Source: §E.5 · p. 49

E.6 DERIVATION OF DIFFUSION SOLUTION TO CONDOT PATH

  • A purely diffusion-based Markov process with zero drift cannot generate the CondOT probability path because no choice yields a non-negative diffusion coefficient.The contradiction holds for 0 < t < 1.Source: §E.6 · pp. 49, 51

E.7 DERIVATION OF JUMP SOLUTION TO CONDOT PATH

  • The one-dimensional CondOT path admits a jump process with a state-independent jump distribution and an intensity derived from the path’s jump continuity equation.Near t = 1, states close to the final mean z have p(x) < 0, while other states must jump.Source: §E.7 · pp. 54–55

F DETAILS FOR EUCLIDEAN JUMP MODEL

  • The scheduler is valid for deciding whether to jump during an interval [t, t + h), and its modification made a significant difference in image-generation results, with CIFAR-10 FID 12 vs. 4.5.Source: F DETAILS FOR EUCLIDEAN JUMP MODEL · p. 56

G.1 JUMP SOLUTION TO THE KFE

  • A state-independent jump distribution with non-negative intensity gives a general solution to the jump continuity equation on arbitrary state spaces.The construction also characterizes a minimal jump intensity.Source: §G.1 · pp. 58–59

G.2 PROBABILITY PATHS AND COMPUTING λt, Jt

  • For SO(3), a Fisher-von-Mises path and scheduler approximate the geodesic path while maintaining numerical stability; sampling uses plain Euler updates.The path has support over all of SO(3) for all times, and the sampling procedure assumes a unimodal conditional distribution when repurposing the FrameFlow model.Source: §G.2 · p. 59, §G.3 · p. 59

G.4 EXPERIMENT DETAILS

Generator Matching places diverse generative models within a framework of Markov-process generators, recovering flow, diffusion, discrete, and manifold models while also accommodating jump processes. The section relates these models through their generators and describes applications, limitations, and experimental results.

G.4 EXPERIMENT DETAILS

  • Jump modifications achieve the best results in all reported protein-generation categories; the jump-flow ensemble performs similarly to pure jumps.The experiments sample 400 proteins across lengths 70, 100, 200, and 300; diversity and novelty are evaluated after designability filtering.Source: §G.4 · p. 60, Table 5 · p. 61

H.1 FLOW MATCHING

  • Flow matching is a special case of Generator Matching, using a flow generator and mean-squared error for its conditional training loss.The framework generalizes flow matching principles to generators of Markov processes on arbitrary state spaces.Source: §H.1 · p. 61

H.2 DENOISING DIFFUSION MODELS

  • Denoising diffusion models are flow models with stochastic Langevin sampling and probability paths built from forward noising and time reversal.Any positive weighting of the divergence-free Langevin component gives a valid sampling procedure; discrete-time formulations incur loss error and use an approximate ELBO loss.Source: §H.2 · pp. 61, 63

H.3 STOCHASTIC INTERPOLANTS

  • Stochastic interpolants construct probability paths with a simulator, whose conditional expected velocity defines a flow generator for training.Separately learned scores can add Langevin sampling; the cited work states this permits control of KL divergence, unlike pure-flow bounds in general.Source: §H.3 · p. 63

H.4 DISCRETE SPACES VIA CONTINUOUS-TIME MARKOV CHAINS ("DISCRETE DIFFUSION")

  • On discrete spaces, Generator Matching recovers continuous-time Markov-chain models; factorized paths permit independent dimension-wise rates or learning posterior marginals.The posterior marginals can be trained with cross-entropy.Source: §H.4 · p. 64

H.5 GEOMETRIC DATA AND MANIFOLDS

  • Generator Matching recovers Riemannian flow matching on manifolds, while manifold diffusion training requires simulated noising and approximate score targets.The conditional solution to the KFE is not analytically known for the described noising process, even for geometries with analytic geodesics.Source: §H.5 · p. 65

H.6 MULTIMODAL SPACES

  • A factorized probability path combines discrete amino-acid, Euclidean translation, and SO(3) frame models into a multimodal protein generator.The component loss functions can be summed.Source: §H.6 · p. 66

H.7 JUMP MODELS ON NON-DISCRETE STATE SPACES

  • Non-discrete jump examples include PDMPs, Lévy-driven models, and trans-dimensional diffusion; unlike Yoon et al. (2023), GM can learn jump kernels or diffusion coefficients.The cited Lévy-process work learns only the flow component, leaving other process components as hyperparameters.Source: §H.7 · p. 66

I ADDITIONAL EXPERIMENTS ON LOSS FUNCTIONS

The CIFAR-10 flow-model experiments show that changing the Bregman divergence can improve training stability and final performance. The tested loss mixes MSE with another function, and the reported FID improves over MSE.

I ADDITIONAL EXPERIMENTS ON LOSS FUNCTIONS

Related papers

Loading 2410.20587v3…