Source-linked AI summary
An ODE-based Sampler for Discrete Diffusion Models
Samson Gourevitch, Yazid Janati, Eric Moulines, Eric P. Xing, Alain Durmus, Giacomo Zanella
TL;DR
The paper asks whether discrete diffusion can have a sampler that is deterministic after its initial condition, analogous to the probability-flow ODE for continuous diffusion. It introduces exponential residual flows, which preserve the prescribed marginals, and reports sampling quality comparable to ancestral updates at matched model evaluations, with the clearest few-step gains for uniform diffusion.
Problem
The paper asks whether a discrete diffusion sampler can be deterministic after initialization while preserving the process marginals; a deterministic map on a finite state space cannot split probability mass.
Source: §1 · p. 1Method
ERF augments the discrete state with exponential residuals that evolve deterministically and trigger token changes at zero, preserving the original CTMC marginals.
Source: §1 · p. 2, §3.1 · p. 5Results
Sampling quality is comparable to ancestral updates at matched model evaluations, and consistency distillation yields its clearest few-step gains for uniform diffusion.
Source: §6.1 · p. 9, §7 · p. 11Takeaways & Limitations
ERF provides deterministic transport between augmented states as a pathwise target for consistency distillation.
Source: §4 · p. 7
Abstract
from arXiv · showhide
The marginals of a continuous diffusion model can be reproduced by a deterministic ordinary differential equation (ODE), known as the probability flow ODE, which provides a foundation for accelerated sampling and consistency distillation. In this paper we address the question whether an analogous formulation exists for discrete diffusion models and answer it affirmatively by introducing exponential residual flows (ERFs). Our framework augments the discrete state with a continuous residual variable for each token and position. These residuals evolve deterministically according to piecewise-defined ODEs whose vector fields depend on the current discrete state and the forward and backward rate matrices of the prescribed diffusion model. The discrete state changes whenever a residual reaches zero. We prove that this construction preserves the marginals of the original diffusion. We derive a practical discretization of our sampler and specialize it to masked and uniform diffusion. The deterministic trajectories of ERFs support consistency distillation and yield empirical improvements over previous approaches across a variety of benchmarks including code and text generation tasks.
1 INTRODUCTION
Discrete diffusion models can update multiple positions in parallel, but their standard reverse samplers use stochastic transitions. The paper asks whether a discrete sampler can instead be deterministic after initialization and introduces exponential residual flows to do so while preserving the original marginals.
1 INTRODUCTION
- Standard reverse-process simulation is stochastic, using fresh randomness at jumps or stochastic transitions in ancestral sampling.Discrete diffusion models allow multiple positions to be updated in parallel, offering a path toward faster decoding and potentially more efficient steering and control.Source: §1 · p. 1
- A deterministic map on a finite state space cannot split probability mass, so a discrete sampler deterministic after initialization requires continuous auxiliary randomness at initialization.The introduction distinguishes ERF from prior auxiliary-randomness approaches that do not define a marginal-preserving deterministic flow for a discrete diffusion process.Source: §1 · p. 1
- ERF augments each token and position with an independently initialized exponential residual that evolves deterministically and triggers a token change when it reaches zero.After initialization, no additional randomness is required; the augmented dynamics preserve the original CTMC marginals at every time.Source: §1 · p. 2
2 BACKGROUND ON DISCRETE DIFFUSION MODELS
Discrete diffusion constructs a stochastic process that corrupts a target data distribution into a simple reference distribution, then learns an approximation to the reverse process. The reverse process transports samples from the reference distribution back toward the data distribution.
2 BACKGROUND ON DISCRETE DIFFUSION MODELS
- A forward process progressively corrupts target data distribution p0 until it approaches the reference distribution p1, and the generative model approximates the reverse process.The forward process is a continuous-time Markov chain started at p0.Source: §2 · p. 2
- Reversing the forward process in time yields a CTMC with the same marginals, transporting the source distribution p1 toward the target p0.The reverse state is denoted Xt, with Xt = Yt−.Source: §2 · p. 2
- A factorized learned model parameterizes reverse transitions using a neural denoiser that approximates the leave-one-out denoiser.The leave-one-out denoiser is defined through an expectation of the one-hot token at a position given the other positions and time.Source: §2 · pp. 2–3
- The forward and backward processes are described using rate matrices, which specify transition rates for the continuous-time dynamics.The passage introduces the forward rate matrix and the family of backward rate matrices.Source: §2 · p. 3
3 DETERMINISTIC SAMPLING VIA EXPONENTIAL RESIDUAL FLOWS
ERF replaces repeated randomness in reverse-CTMC simulation with deterministic residual dynamics after initialization while preserving the CTMC’s time marginals. The section also develops practical integration and rate choices for masked and uniform diffusion.
3.1 EXPONENTIAL RESIDUAL FLOW
- Initialized with exponential residuals, ERF evolves residuals deterministically between jumps; a residual reaching zero selects the next token change.Source: §3.1 · p. 5
- Under standard assumptions, ERF is almost surely well defined with finitely many jumps and preserves the reverse CTMC’s time marginals.Source: §3.1 · p. 5
3.2 INTEGRATING THE EXPONENTIAL RESIDUAL FLOW
- To address difficult reverse-rate integration, they discretize time and freeze the leave-one-out denoiser, making approximate-rate time integrals explicit for most models.Source: §3.2 · p. 6
- With an optimal leave-one-out predictor, this freezing strategy is exact for a single token or factorized distribution.Each interval is an exact ERF step for the factorized distribution and has finitely many jumps.Source: §3.2 · p. 6
- For UDM, net-flux rates have analytically invertible time integrals, eliminating jump-time bisection and increasing computational speed.Source: §3.2 · p. 7
4 APPLICATION TO DISTILLATION
The paper uses deterministic exponential-residual-flow (ERF) trajectories to train students by consistency distillation. Students learn to agree on clean-state predictions from two points on the same teacher trajectory.
4 APPLICATION TO DISTILLATION
- ERF makes the augmented state at an earlier time a deterministic function of the later state, enabling consistency targets for discrete diffusion.The state includes residuals; the student predicts the clean discrete state, not endpoint residuals.Source: §4 · p. 7
- Consistency distillation trains student predictions from two points on one ERF trajectory to agree on its clean discrete endpoint.The student output is relaxed to a factorized distribution over discrete states to make learning more tractable.Source: §4 · p. 7
- The training objective samples t uniformly from δ to 1 and sets s = t − δ, with a target network updated by EMA or refreshed between rounds.An exact minimizer requires a clean-endpoint boundary condition and nonzero probability of s = 0; the authors report that imposing it did not improve performance in practice.Source: §4 · pp. 7–8
- Refresh sampling is exact when the teacher transports the prescribed augmented marginals and the student learns its endpoint map, whereas bridge sampling is not generally exact but performs well empirically.Both procedures redraw exponential variables independently after each transition.Source: §4 · p. 8
5 RELATED WORK
The related work contrasts ERF with stochastic CTMC samplers and deterministic approaches that do not preserve the same combination of trajectory determinism and marginals. It also positions ERF as a way to make consistency distillation possible for discrete diffusion.
CTMC sampling and discretization
- ERF recycles residuals after one random initialization, giving deterministic trajectories while preserving exact one-time CTMC marginals, though its trajectory law generally differs.The discretization freezes only the leave-one-out denoiser over each step while retaining known time and state dependence of the rates.Source: §5 · p. 8
Deterministic sampling of discrete diffusion models
- Net-flux rates reduce jumps but remain stochastic, while reusing Gumbel noise across discretization steps can drift from the target distribution; ERF preserves one-time marginals.Source: §5 · p. 8
Distillation of discrete diffusion models
- Unlike prior trajectory-based distillation approaches with stochastic trajectories, conditioning on exponential residuals makes the teacher trajectory deterministic given (Xt, Et), enabling consistency distillation.The section contrasts SDTT’s aggregation of teacher probabilities across stochastic unmasking steps and DCD’s matching along projected Gaussian trajectories.Source: §5 · p. 8
6 EXPERIMENTS
Experiments compare ERF sampling and ERF consistency distillation (ERC) with ancestral samplers and distillation baselines across masked and uniform diffusion. ERF sampling has comparable quality at matched NFE, while ERC’s strongest gains are with uniform diffusion.
6 EXPERIMENTS
- The experiments first assess ERF against ancestral sampling at matched NFE, then test whether ERC can generate competitively with fewer NFEs than the teacher.Source: §6 · p. 8
- The study evaluates masked and uniform models on OWT and TinyGSM, and also tests the released DiffusionGemma model on MMLU-Pro.OWT generation is unconditional; TinyGSM models generate Python solutions, and evaluations use GPT-2 Large generative perplexity with token entropy or answer accuracy.Source: §6 · pp. 8–9
6.1 EVALUATION OF THE ERF SAMPLER
- At matched NFE, ERF and ancestral sampling have comparable quality across evaluated datasets and budgets, with similar TinyGSM accuracy and largely overlapping OWT entropy–perplexity curves.Source: Figure 2 · p. 9, §6.1 · p. 9
6.2 DISTILLATION
- UDM ERC doubles the strongest competing baseline’s TinyGSM accuracy at 2 and 4 NFEs, remains higher at 8 and 16, and approaches teacher accuracy by 32; on OWT it has lower perplexity at matched entropy across the frontier.MDM ERC approximately matches the second-best baseline on both datasets; on Sudoku, ERC gives modest but consistent gains in many evaluated settings.Source: Figure 3 · p. 9, Figure 4 · p. 10, §6.2 · p. 11
6.1 EVALUATION OF THE ERF SAMPLER
- At 48 NFEs and temperature 1, ERF Uniform and U-NetFlux are within 0.7 percentage points of the UDM ancestral sampler on MMLU-Pro.Table 1 reports DiffusionGemma results on 140 questions with three paired seeds and 95% question-cluster bootstrap intervals.Source: §6.1 · p. 9, Table 1 · p. 9
- At 8 NFEs, U-NetFlux averages about one jump per generated position versus seven for Uniform, with speedups of 2.83× on TinyGSM and 2.00× on OWT while maintaining similar sampling quality.U-NetFlux solves crossing times explicitly and requires fewer solves.Source: §6.1 · p. 10, Table 2 · p. 10
6.2 DISTILLATION
- Distillation comparisons use shared pretrained teachers and student initialization, with ERC training budgets of 100 GPU-hours on OWT and 50 on TinyGSM.ERC uses four training rounds, sweeps δ, and selects sampler and temperature settings using evaluation criteria; the ablations vary transport updates, gap, sampler, Gumbel inputs, and temperature placement.Source: §6.2 · pp. 10–11
7 CONCLUSION
The conclusion summarizes exponential residual flows as deterministic dynamics that preserve diffusion marginals and provide a pathwise target for consistency distillation, alongside reported sampling quality and consistency results. It also discusses when leave-one-out score freezing is exact and the finite-time explosion that can result from freezing scores while retaining the original forward rates.
7 CONCLUSION
- ERF achieves sampling quality comparable to ancestral updates at matched model evaluations, while consistency distillation yields its clearest few-step gains for uniform diffusion.The construction transports prescribed discrete-diffusion marginals using deterministic residual dynamics and recycles exponential variables drawn at each jump.Source: §7 · p. 11
C.3 EQUIVALENCE OF LEAVE-ONE-OUT-FROZEN GILLESPIE AND ANCESTRAL SAMPLING
- Leave-one-out-frozen Gillespie sampling is equivalent to standard ancestral sampling, whereas score or ordinary-denoiser freezing does not generally share this equivalence.The equivalence follows for the leave-one-out-frozen rates; the passage explicitly excludes the general case for score and ordinary-denoiser freezing.Source: §C.3 · p. 21
C.5.2 EXACTNESS OF LEAVE-ONE-OUT FREEZING UNDER FACTORIZATION
- With factorized data, leave-one-out freezing gives the exact reverse generator on every discretization interval, so exact integration introduces no discretization error.This exactness is conditional on the data distribution factorizing across positions.Source: §C.5.2 · p. 24
- Under factorized uniform diffusion, cached score and posterior values generally differ from exact reverse rates at earlier times, whereas denoiser freezing is exact for masked diffusion.At a masked position, the ordinary and leave-one-out posteriors both equal pℓ_0 under factorization.Source: §C.5.2 · p. 24
C.5.3 A COUNTEREXAMPLE FOR SCORE FREEZING
- With original forward rates, a one-position, two-token UDM under frozen reverse rates can undergo infinitely many jumps in finite time with positive probability.Successive two-jump cycles have geometrically decreasing durations; explosion can occur even when the cached score is population-optimal, while Gillespie sampling with the same bounded frozen reverse rates is nonexplosive.Source: §C.5.3 · p. 24
C.5.4 WHY THE EXPLOSION CANNOT OCCUR WITH LEAVE-ONE-OUT FREEZING
Leave-one-out freezing avoids explosion by making the frozen reverse generator an exact time reversal under a suitable law, while net-flux rates preserve marginal evolution with lower jump activity. The construction also supplies compatible forward rates for ERF residual evolution without extra model or density evaluations.
C.5.4 WHY THE EXPLOSION CANNOT OCCUR WITH LEAVE-ONE-OUT FREEZING
- Leave-one-out freezing makes the frozen reverse generator the time reversal of the original forward process under a different initial law, preventing infinitely many jumps.The nonexplosion argument also applies to other entry laws when the conditional residual distribution has a density for every state.Source: §C.5.4 · p. 25
D.1 NET FLUX CONSTRUCTION AND RELATION TO PRIOR WORK
- Net-flux rates preserve the same marginal evolution while having no larger total jump intensity than the conditional construction of Campbell et al. (2024).The positive-part projection preserves pairwise current and gives the minimum-activity generator with that current.Source: §D.1 · pp. 25–26
D.2 FORWARD NET-FLUX RATES AND PASSIVE RESIDUALS
- For strictly positive marginals and exact canonical rates, forward net-flux rates follow by taking positive parts of the canonical forward and reverse rates in opposite orders.No additional model or density evaluation is needed; these rates provide the passive speeds used by ERF.Source: §D.2 · p. 26
D.3 SCORE AND DENOISING-POSTERIOR FREEZING FOR UDM NET FLUX
For UDM net flux, score and denoising-posterior freezing yield explicit rate constructions, but score freezing can produce incompatible forward and reverse rates. Leave-one-out freezing instead preserves time-reversal balance and supports the augmented-marginal argument.
D.3 SCORE AND DENOISING-POSTERIOR FREEZING FOR UDM NET FLUX
- For UDM, score-frozen rates integrate directly, while posterior-frozen rates require splitting the integral at a sign crossing that occurs at most once.The constructions apply the positive-part projection to the frozen canonical rates; passive speeds sum the positive-part complements across possible token changes.Source: §D.3 · pp. 26–27
- In the binary score-freezing example, the projected reverse rate is 1 while the opposite forward rate is 0, preventing any positive candidate distribution from satisfying the balance.The text warns that score freezing can therefore create incompatible rates and infinitely many jumps.Source: §D.3 · p. 27
- Leave-one-out net-flux rates preserve time-reversal balance and the difference between forward fluxes, so the augmented-marginal and nonexplosion arguments apply.The compatible distribution is the product of forward marginals initialized by cached leave-one-out predictions.Source: §D.3 · p. 27
E.1 CONSISTENCY DISTILLATION TRAINING
The section describes round-based consistency distillation and multi-step sampling with ERF trajectories, then distinguishes when bridge sampling preserves the desired marginals. It also outlines implementations of D-MMD, DCD, and iDLM.
E.1 CONSISTENCY DISTILLATION TRAINING
- The target-only procedure refreshes the frozen target between rounds while the pretrained teacher generates every trajectory.Training samples paired times separated by gap δ and encourages student predictions from two points on the same trajectory to agree.Source: §E.1 · pp. 27, 29
- Joint refresh uses the frozen EMA student both as the consistency target and, after the first round, as the model supplying ERF rollout predictions.The network is reevaluated at each numerical substep, but not after every token jump within a substep.Source: §E.1 · p. 28
E.2 MULTI-STEP SAMPLING
- Multi-step sampling predicts a clean state at each grid time, then either re-noises it independently for refresh sampling or conditions the next state on the current state for bridge sampling.Both procedures initialize the discrete state from p1 and residuals from independent exponential variables.Source: §E.2 · p. 30
E.3 BRIDGE SAMPLING AND THE ERF COUPLING
- For masked diffusion, each position jumps at most once, so ERF preserves the reverse-chain law and bridge sampling is exact; for general ERF couplings, bridge sampling need not preserve the target marginal.Refresh sampling is always valid under the stated procedure because it samples a clean state from p0 and draws fresh independent residuals.Source: §E.3 · p. 30
E.4 D-MMD
- D-MMD uses sampled masked-diffusion states held fixed during differentiation, with generator and auxiliary updates differentiating only their respective explicit model probabilities.The displayed objectives are surrogate losses, not derivatives through the parameter-dependent sampling law.Source: §E.4 · pp. 30–31
E.5 DCD
- DCD implements its coupled forward samples directly in UDM state space by marginalizing the Gaussian construction and sampling the resulting transition.The transition is described as a maximal coupling of the two one-position UDM marginals conditional on the clean token.Source: §E.5 · p. 31
E.6 IDLM
- iDLM alternates student and auxiliary-denoiser updates, retaining a Gaussian relaxation for differentiable UDM training while evaluating discrete tokens.The UDM relaxation temperature is 10^-3, and the teacher remains the shared pretrained UDM checkpoint.Source: §E.6 · p. 32
F EXPERIMENTAL DETAILS
These experimental details specify the Sudoku setup and the protocols for sampling-quality, solver, substep, and runtime comparisons. The figures and table captions identify the reported comparisons but do not provide outcome values here.
F.1 SUDOKU
- Sudoku examples concatenate an 81-digit quiz, separator, 81-digit solution, and end token; exact-grid accuracy requires all 81 predicted positions to match.The quiz and separator remain fixed, and constraint violations are also computed.Source: §F.1 · p. 32
G.1 SAMPLING QUALITY ACROSS NFE BUDGETS
- Sampling quality is evaluated across eight NFE budgets, using GSM8K accuracy and OpenWebText token entropy and GPT-2 Large generative perplexity.The figures compare ERF with ancestral sampling using matched pretrained checkpoints within each diffusion family.Source: §G.1 · p. 34, Figure 5 · p. 34, Figure 6 · p. 35
G.2 EULER AND CATEGORICAL TAU-LEAPING
- ERF is compared with Euler and categorical τ-leaping for masked, uniform, and U-NetFlux rates at 8, 64, and 512 NFEs.The Euler baseline clips and renormalizes invalid categorical weights; categorical τ-leaping rejects multiple proposed jumps at a position.Source: §G.2 · p. 36, Figure 7 · p. 36, Figure 8 · p. 37
G.3 UNIFORM SUBSTEPS
- Bounded Uniform variants divide frozen-model intervals into 1–32 substeps and allow at most one jump per position per substep; the unbounded variant resolves every crossing.Substeps do not add model evaluations, and only completed settings appear in the TinyGSM and OpenWebText comparisons.Source: §G.3 · p. 38, Figure 10 · p. 39, Figure 11 · p. 40
G.4 RUNTIME AND JUMP-COUNT PROTOCOL
- Runtime profiles use three seeds, fixed batches, shared checkpoints, and synchronized timing intervals that include sampling but exclude checkpoint loading and metric computation.Jump counts come from a separate replay verified against the original run; Table 6 reports mean time, speedup, and jumps per generated position.Source: §G.4 · p. 41, Figure 12 · p. 41, Table 6 · p. 42
G.5 DISTILLATION EVALUATION
The section specifies evaluation choices for distillation, Sudoku, update policies, consistency gaps, sampling schemes, and Gumbel conditioning. The supplied passages primarily describe protocols and comparison grids rather than their outcomes.
G.5 DISTILLATION EVALUATION
- Main distillation evaluations use 50 GPU-hours per TinyGSM student and 100 per OpenWebText student, with batch size 128 and final seed-1 EMA checkpoints.ERC refreshes its target and rollout teacher at 25%, 50%, and 75% of the training budget.Source: §G.5 · p. 42
G.6 SUDOKU DISTILLATION
- Sudoku tables report exact-grid accuracy by difficulty and NFE, retaining each method’s highest two-seed mean across the specified samplers and temperatures.ERC is selected at 8 NFEs with refresh sampling, then held fixed across NFE budgets.Source: §G.6 · p. 43, Table 7 · p. 43, Table 8 · p. 43
G.7 TARGET AND TEACHER UPDATES
- Target-update comparisons vary frozen, EMA, periodic target refresh, and joint target-and-teacher refresh policies, under fixed or increasing training gaps.TinyGSM uses both inference samplers, while OpenWebText uses bridge sampling in the stated protocol.Source: §G.7 · p. 43, Figure 13 · p. 44, Figure 14 · p. 44
G.8 CONSISTENCY GAP
- Consistency-gap evaluations compare four δ values with target refresh each round, either keeping the pretrained transport denoiser fixed or refreshing it from the student EMA.OpenWebText scores average log-perplexity over shared entropy intervals and then across NFEs.Source: §G.8 · p. 47, Figure 19 · p. 47, Figure 20 · p. 47
G.9 BRIDGE AND REFRESH SAMPLING
- Bridge and refresh sampling are compared across methods, with denoiser temperature for MDM and either denoiser or leave-one-out temperature placement for UDM.TinyGSM selects a temperature per method by mean accuracy across six NFEs; OpenWebText panels sweep measured temperatures at fixed NFEs.Source: §G.9 · p. 50, Figure 25 · p. 51, Figure 26 · p. 51
G.10 GUMBEL CONDITIONING
- Gumbel conditioning is compared with and without Gumbel inputs after 10,000 updates, with target and ERF transport denoiser refreshed together.The matched OpenWebText comparison uses bridge sampling; no matched refresh comparison is available.Source: §G.10 · p. 58, Figure 37 · p. 58, Figure 38 · p. 58
H DIFFUSIONGEMMA EXPERIMENT
The passages give the DiffusionGemma evaluation setup, motivate an ERF residual velocity through a continuity equation, and outline assumptions and construction details for the sampler.
H DIFFUSIONGEMMA EXPERIMENT
- The DiffusionGemma experiment evaluates a 26B-parameter, 4B-active instruction-tuned checkpoint on 140 MMLU-Pro questions across three run seeds.Custom samplers use 48 NFEs, while the native procedure averages 11.1 forward evaluations, so timings are not compute-matched.Source: H DIFFUSIONGEMMA EXPERIMENT · p. 59
I.1 INTUITION ON THE ODES (9)
- The ODE motivation rewrites the Kolmogorov equation as a continuity equation on the extended state space, which suggests the imposed residual dynamics.The residual vector field depends on the discrete state and includes the forward exit rates.Source: §I.1 · pp. 59–60
I.2.1 NOTATION AND ASSUMPTIONS
- The formal setup considers time-inhomogeneous rate matrices on a token-sequence state space and imposes regularity and positive-marginal assumptions.The stated assumptions include no explosion and continuously differentiable marginals.Source: §I.2.1 · p. 60
I.2.2 ERF SAMPLER REVISITED
- The sampler initializes the state from p1 and residuals independently from Exp(1), then advances along the residual vector field until the next candidate jump time.The construction states that failures and infinite jump sequences have probability zero.Source: §I.2.2 · pp. 61–62
I.3 MAIN RESULT
This section establishes rigorous results for the residual-flow construction, including change-of-variables descriptions of residual distributions and almost-sure completion with finitely many jumps.
I.3 MAIN RESULT
- Almost surely, the sampler has finitely many jumps and is well defined on [0, 1]; its conclusion concerns deterministic times, not independence at jump times or Markovianity of X.The theorem states a deterministic-time distributional conclusion; the supplied passage does not assert that X is Markovian.Source: §I.3 · p. 62
I.4 PRELIMINARIES
- Forward and reverse off-diagonal rates and exit rates are continuous and bounded, and the residual regions are Borel, disjoint, and identify a unique first-hit reconstruction outside exceptional sets.The reconstruction result also covers active-boundary residuals when the corresponding reverse rate is positive.Source: §I.4 · pp. 62, 64, 65
I.5 CHANGE OF VARIABLES
- For deterministic u, the change-of-variables result represents the measure of segments with source times u < v < 1 that have no active hit down to u and end in a specified residual set.The first hit, when present, satisfies the stated hit condition almost surely, and the probability of a first hit at any fixed time is zero.Source: §I.5 · pp. 66, 68
- For the finite source measures, a next hit is almost surely unique with positive incoming speed, and fixed-time hits and omitted singular parts have measure zero.At a hit, only the firing residual is zero.Source: §I.5 · p. 68
I.6 INDUCTION ON THE NUMBER OF JUMPS
- For every finite recursion level, endpoint and next-hit formulas hold, and each hit almost surely has only its firing residual at zero with no atom at a prescribed time.These regularity claims do not assume nonexplosion.Source: §I.6 · p. 70
I.7 BOUNDS ON THE PARTIAL SUMS AND NUMBER OF JUMPS
- Almost surely, the construction has no failure and only finitely many hits on [0,1]; its full path is measurable at every fixed time, with zero probability of a hit at any deterministic time.Source: §I.7 · pp. 73–74
I.8 PROOF OF THEOREM I.1
The proof establishes almost-surely finite jumps and constructs the sampler’s distribution at deterministic times by summing over jump-count events. It then characterizes last-jump distributions, including a single-token specialization.
I.8 PROOF OF THEOREM I.1
- Almost surely, the sampler has finitely many jumps and is well defined on [0, 1].Source: §I.8 · pp. 74–75
- For each deterministic u<1, summing densities over jump-count events yields the state-residual density; at u=1, the result follows from initialization.Source: §I.8 · p. 74
- Integrating the product density over residual-coordinate boxes gives the corresponding distribution function.Source: §I.8 · p. 74
I.9 LAST-JUMP DISTRIBUTION
- After at least one jump, the joint measure of the last-jump time and passive residuals is characterized conditional on the current state and firing coordinate.Source: §I.9 · p. 75
- For L=1, the specialized last-jump expression is zero when r is outside Ru(j).Source: §I.9 · pp. 75–76
Related papers
This paper cites
- Score-Based Generative Modeling through Stochastic Differential Equations2020
- Simple and Effective Masked Diffusion Language Models2024
- A Continuous Time Framework for Discrete Denoising Models2022
- A modified Next Reaction Method for simulating chemical systems with time dependent propensities and delays2007
- Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulation2026
- Beyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMD2026