Discrete Diffusion Models: A Unified Framework from Tokenization to Generation

• Discrete diffusion enables parallel iterative refinement, global revision, and controllable editing, addressing autoregressive decoding’s O(L) sequentiality and irrevocable left-to-right commitments.These properties support global planning, long-range coordination, infilling, constrained editing, and controllable generation.Source: §1 · pp. 1–2
• Native discrete diffusion requires designing categorical corruption and reverse denoising processes because embedding-and-rounding methods create geometry mismatch, rounding errors, and representation collapse.Corruption choices determine token-space perturbation topology, denoising difficulty, generation order, likelihood computation, schedules, and interactions with vocabulary structure.Source: §1 · p. 2
• The paper makes tokenization—the construction of vocabulary, codebook, or domain alphabet—the primary organizing principle linking state-space structure to corruption-denoising dynamics and controllability.This lens covers subword granularity, multimodal codebook topology, and structural alphabets such as amino acids, nucleotides, atom types, and bond types.Source: §1 · pp. 2, 4
• It unifies discrete diffusion models by decomposing them into corruption operator, denoiser parameterization, training objective, and sampler, organizing major formulation families around this shared structure.The framework includes transition-matrix, absorbing-state masking, substitution and mixture corruption, continuous-time, and score/ratio-based approaches.Source: §1 · pp. 2, 4
• The framework is instantiated across text and code, tokenized multimodal generation, proteins, genomics, molecules, graphs, planning, agents, and tool-use systems, exposing shared patterns and domain-specific constraints.Applications include diffusion language models, quantized images, audio and video, scientific alphabets and graphs, and structured decision-making pipelines.Source: §1 · pp. 2–4
• The paper treats scaling, systems, and evaluation as core design dimensions and identifies open problems including in-context learning, KV-cache analogues, streaming generation, unified token spaces, and theory.These topics are framed as actionable research questions rather than afterthoughts.Source: §1 · p. 4

Survey gap

• Prior surveys cover text, multimodal, scientific, and industrial diffusion applications, but under-emphasize tokenization, domain-intrinsic discrete structures, and their coupling to model design.Existing reviews often treat tokenization as a local choice and review formulations, objectives, inference, systems, or evaluation separately rather than as one design space.Source: §2 · pp. 4–5

Framework contribution

• The paper makes tokenization, state-space factorization, and representation diagnostics primary organizing principles, rather than peripheral implementation details.It argues that consequential discrete-diffusion choices arise before and beyond the denoiser architecture itself.Source: §2 · p. 5

Cross-domain scope

• The framework unifies language, multimodal, protein, genomics, molecular, and graph settings while integrating scaling, systems, deployment, parallelism, caching, efficiency, controllability, calibration, and validity.This cross-domain view brings scientific discrete structures and operational concerns into one shared framework.Source: §2 · p. 5

Practical design reference

• The paper distills reusable checklists for selecting tokenization, corruption, objectives, reverse parameterizations, samplers, constraints, and evaluation protocols across domains.The intended reference compares decisions affecting semantics, validity, denoising, decoding, quality, efficiency, controllability, calibration, and structural correctness.Source: §2 · p. 5

Scope and positioning

• Relative to Yu et al. (2025), the paper prioritizes discrete state-space construction, adds scientific alphabets plus planning, agent, and tabular settings, and integrates scaling, systems, and evaluation standardization.Yu et al. (2025) provides complementary depth on multimodal quantization, which this paper treats more briefly.Source: §2 · p. 5

3.1 Generative Modeling: AR vs. Iterative Refinement

• AR models factorize sequences left-to-right and require O(L) serial decoding, while iterative refinement updates many positions jointly in T ≪ L steps.AR training uses teacher forcing and per-token cross-entropy, but committed tokens cannot be revised using later evidence; iterative refinement enables global revision with bidirectional context.Source: §3.1 · p. 8

3.2 Continuous Diffusion Primer

• Continuous diffusion’s embedding-and-rounding approach mismatches discrete vocabulary geometry, motivating diffusion defined natively through categorical corruption and reverse prediction.Continuous scores can point between token embeddings, causing rounding errors and embedding collapse; discrete diffusion instead uses masking, substitution, or structured transitions.Source: §3.2 · p. 9

3.3 Discrete Spaces, Markov Chains, and Notation

• Discrete diffusion represents sequences or structured objects over categorical alphabets, corrupts them through Markov transition matrices, and schedules the terminal state toward a known stationary prior.The framework supports token sequences, molecular graphs, and other indexed discrete components; cumulative transition matrices permit direct sampling of noisy states.Source: §3.3 · p. 9
• Reverse generation starts from the stationary distribution and iteratively denoises using full-sequence context, typically predicting clean-token distributions through the dominant x0-parameterization.The notation also covers alternatives such as direct xt−1 prediction, log-probability ratios, and score analogs, while conditioning can incorporate prompts, labels, constraints, or partial sequences.Source: §3.3 · pp. 10–11

4.1 What Is a Token? A Cross-Domain View

• Tokenization maps raw data to categorical vocabularies whose structure determines corruption topology: semantic tokens lack metrics, quantized codes inherit geometry, and scientific alphabets provide domain-grounded similarity and validity.Uniform, structured, or absorbing transition kernels therefore define qualitatively different denoising problems and generation behavior.Source: §4.1 · pp. 11–12

4.2 Text and Code Tokenization

• Text and code tokenization faces large-vocabulary costs and fragmentation trade-offs, while masking diffusion enables bidirectional refinement and constraint-aware correction for structurally sensitive programs.Subword vocabularies can cause combinatorial transition spaces and uneven semantic corruption; byte-level schemes lengthen sequences, whereas code generation can use analyzer or compiler feedback to enforce syntactic validity.Source: §4.2 · p. 14

4.3 Quantized Tokens for Images, Audio, and Video

• Quantized media tokenizers trade reconstruction fidelity, codebook utilization, sequence efficiency, and generative complexity across VQ, RVQ, and scalar quantization, while structured codebook geometry can improve diffusion conditioning.RVQ improves fidelity without increasing sequence length but complicates modeling; scalar quantization is stable yet geometrically inflexible, and semantically regular codebooks support non-uniform corruption toward nearby codes.Source: §4.3 · pp. 14–15

4.4 Natural Discrete Structures in Science

• Scientific representations use compact natural alphabets or constrained molecular graphs, where non-local dependencies, chemical validity, and discretization granularity determine the corruption process, denoising burden, and expressiveness.Masking supports joint reconstruction of correlated biological positions, while molecule representations range from fragment sequences to node-edge graphs with hard valence, connectivity, and stereochemical constraints.Source: §4.4 · pp. 15–16

4.5 Diagnostics and Metrics for Tokenizations

• Tokenization evaluation should combine established information, reconstruction, and codebook-utilization metrics with proposed geometric and diffusion-facing diagnostics, while recognizing that the latter remain largely unvalidated recommendations.Suggested probes include neighborhood consistency, domain-specific validity metrics, reconstruction-generation gaps, and denoising-loss curves across timesteps.Source: §4.5 · pp. 16, 18
• Schedule sweeps reveal how strongly tokenization design couples to linear, cosine, or log-linear noise schedules, with robust tokenizations simplifying development (Sahoo et al., 2024).Final sample quality across schedule families measures this coupling.Source: §4.5 · p. 19
• Optimal unmasking order depends on token-prediction conditional entropy, while random-order masking training prepares models for worst-case orderings (Kim et al., 2025d).Source: §4.5 · p. 19
• Geometric and diffusion-facing diagnostics remain underexplored, with no established model-free predictor of diffusion-ready tokenizers, motivating lightweight proxies and reporting metrics from all diagnostic families.Current prediction requires end-to-end generative-model training.Source: §4.5 · p. 19

5.1 Discrete-Time Markov Chains and Transition Matrices

• D3PM (Austin et al., 2021a) defines discrete diffusion through transition matrices, enabling closed-form reverse posteriors and the dominant predict-x0-then-posterior parameterization.The transition matrix specifies the corruption prior, while the model predicts x0 and analytically obtains reverse transitions from the posterior.Source: §5.1 · p. 19

5.2 Masking and Absorbing-State Diffusion

• Absorbing-state diffusion dominates practice because masking yields simple closed-form corruption, stable masked-language-modeling training, and compatibility with bidirectional Transformers.Its marginal is a mixture of clean and masked states, reducing training to predicting masked token identities from bidirectional context.Source: §5.2 · pp. 20–21, §5.1 · p. 20

5.3 Substitution Noise and Mixture Corruptions

• Uniform substitution creates label-noise denoising that becomes harder with vocabulary size, whereas hybrid corruption adds robustness to plausible-but-wrong tokens during iterative generation.Small substitution components can improve generation quality, but the optimal mixing ratio depends on domain and scale, and pure masking remains competitive at large scale.Source: §5.3 · pp. 21–22

5.4 Continuous-Time Discrete Diffusion

• Continuous-time CTMC formulations replace transition matrices with rate matrices, support flexible inference discretization, and simplify objective derivations while introducing matrix-exponential implementation costs.Specially structured rates can provide closed-form exponentials, and absorbing-state rates recover the simple masking mixture without computational overhead.Source: §5.4 · p. 22

5.5 Score/Ratio-Based Characterization of Reverse Transitions

• Ratio- and score-based methods characterize reverse transitions directly, and SEDD (Lou et al., 2024) reports tighter likelihood bounds than x0 prediction under uniform corruption.Predict-x0 and ratio parameterizations are expressively equivalent for exact reverse transitions but differ in inductive bias and optimization under finite capacity.Source: §5.5 · p. 23

5.6 A Unifying View

• All discrete diffusion models comprise a corruption operator, denoiser parameterization, training objective, and sampler, with formulations differing mainly in corruption and parameterization.Training objectives and samplers can often be mixed across families, while any corruption operator can in principle pair with any parameterization.Source: §5.6 · pp. 23–24
• The unified view treats masking, substitution, structured corruption, x0 prediction, logits, and ratios as interchangeable design components whose best pairings depend on domain, vocabulary size, and compute.It motivates systematic exploration of underused combinations while deferring detailed training-objective and inference-algorithm analysis to later sections.Source: §5.6 · p. 24, Table 5 · p. 24

6.2 Simplified Denoising Objectives

• Reweighted denoising is the dominant practical objective: supervision can target only corrupted tokens, use theoretically valid weights, or depend primarily on the number of masked tokens n rather than continuous time (Sahoo et al., 2024).Zhu et al. (2025b) add anti-uniform sharpening, while Zheng et al. (2025) derive natural 1/n weighting; explicit noise-level conditioning remains useful across heterogeneous corruption regimes.Source: §6.2 · pp. 26–27

6.3 Score- and Ratio-Matching Objectives

• Score- and ratio-matching connect reverse dynamics to continuous-time likelihood theory, but unconstrained scores can be unstable; denoising-compatible ratio matching makes the approach more scalable (Shi et al., 2024b; Haxholli et al., 2025).MD4 provides an exact score-to-denoising identity for masked diffusion, while CEDD implements ratio matching through denoising cross-entropy.Source: §6.3 · p. 27

6.4 Parameterizations of the Reverse Model

• Predicting clean tokens or categorical logits is generally preferred over direct reverse transitions, because the forward process can analytically reconstruct reverse steps and direct logits improve stability and generation quality (Gulrajani & Hashimoto, 2023).MDLM’s SUBS forbids remasking and can remove explicit time conditioning, while latent variables restore global dependence without abandoning factorized token prediction.Source: §6.4 · p. 28

6.5 Conditioning and Guidance During Training

• Discrete diffusion supports flexible control through token guidance, classifier guidance, and weakened-conditioning branches, often without dedicated conditional pretraining (Gulrajani & Hashimoto, 2023; Chen et al., 2023a).These mechanisms repeatedly modulate the reverse trajectory to enforce spans, lexical constraints, negation, semantics, syntax, or length.Source: §6.5 · p. 29

6.6 Practical Training Recipes

• Training recipes treat noise schedules, numerical precision, and compute allocation as design variables: schedule choice affects optimization variance and discretization quality, while diffusion models may be smaller and trained longer.MD4 reports schedule effects even when the continuous-time objective is schedule-shape invariant (Shi et al., 2024b). Zheng et al. (2025) recommend float64 for Gumbel-based categorical sampling, while Plaid suggests smaller models trained longer than autoregressive counterparts (Gulrajani & Hashimoto, 2023).Source: §6.6 · pp. 29–30

7.5 Acceleration: Distillation, Caching, and Parallelism

• Wall-clock speed is a systems property of sampler policy, model architecture, and execution substrate, so evaluations should report real calls, latency or throughput, hardware, memory overhead, and matched quality.Nominal claims such as “10x speedup” can reflect fewer rounds, accepted tokens, active tokens, batching, or hardware-specific throughput.Source: §7 · p. 30, §7.5 · p. 36

7.1 Ancestral Sampling and Step Schedules

• Sampler design improves discrete diffusion by changing reverse-time discretization, correction dynamics, numerical updates, or schedules, but fewer nominal steps do not necessarily mean fewer calls or lower latency.Effective comparisons should report NFE alongside transition changes, auxiliary correctors or validators, matched quality, and hardware conditions.Source: §7.1 · pp. 31–32

7.2 Confidence-Based Remasking (Iterative Parallel Decoding)

• Confidence-based remasking treats token order as an inference resource, using confidence, entropy, stability, independence, or revision policies to decide which positions advance, reopen, or remain active.Confidence can reduce rounds while missing informative medium-confidence positions, because scores may measure ease rather than future correctness or importance.Source: §7.2 · pp. 32–33

7.3 Semi-Autoregressive and Block Updates

• Block and semi-autoregressive decoding manage dependencies by interpolating between causal autoregression and parallel denoising, with adaptive regions, flexible lengths, and parallelism constrained by dependency error.Larger blocks expose more parallelism but increase within-block dependency burden, while overly broad windows or simultaneous dependent updates can degrade fluency.Source: §7.3 · pp. 33–34

7.4 Guidance and Constraints at Inference

• Inference-time guidance spans soft preference steering, reward and search, hard validity constraints, and posterior conditioning, each exploiting reverse-process freedom but requiring distinct evaluation metrics.Discrete guidance can alter unmasking rates and reverse dynamics, while constrained diffusion can achieve zero violations in several constrained generation settings.Source: §7.4 · pp. 34–35

7.5 Acceleration: Distillation, Caching, and Parallelism

• Acceleration combines step-reducing distillation with selective computation, cache reuse, suffix pruning, speculative verification, and dLLM-specific draft policies rather than relying on NFE reduction alone.A denoising step need not be a full-sequence Transformer call, and bidirectional cache methods increasingly use state-aware refresh policies paired with token acceptance.Source: §7.5 · pp. 35–36

8.1 Backbone Architectures

• Bidirectional Transformers support full-context denoising and symmetric positional offsets, enabling stable direct length extrapolation and lightweight Diffusion-aware NTK extension to 128K tokens.SSMs such as Mamba offer a complementary linear-time alternative for long-context diffusion inference, whose quadratic Transformer cost can bottleneck massive contexts.Source: §8.1 · p. 37

8.2 Scaling Laws and Compute-Optimality

• DLMs scale competitively at 8B under matched compute and improve downstream performance through 7B adaptation, but compute-optimal ratios, inference budgets, and corruption choices remain task- and regime-dependent.More denoising steps can improve complex reasoning, whereas excessive fixed budgets cause <eos> overflow; ASS reports reducing steps from O(L) to O(log L) under its decoding design.Source: §8.2 · p. 38

8.3 Training Pipeline: Pretraining, SFT, and Alignment

• DLM training combines AR-to-diffusion adaptation, diffusion-specific corruption and masking, blockwise SFT, and increasingly trajectory-level preference optimization to address denoising-path credit assignment and inference mismatch.Post-training methods include trajectory likelihood approximations, ratio-free objectives, tree-structured rollouts, and token-level safety alignment for arbitrary-order decoding.Source: §8.3 · pp. 39–40

8.4 Deployment and Product Constraints

• DDMs can exploit parallel hardware for high throughput, with Mercury reporting over 1,000 tokens per second, but multiple denoising steps create a significant TTFT constraint for interactive streaming.Reported throughput figures are self-reported and not directly comparable across differing hardware, batch sizes, serving stacks, and baselines.Source: §8.4 · p. 40
• Hybrid blockwise decoding balances parallel refinement with reduced diffusion cost, while bidirectional DDMs support editing, inpainting, self-correction, and reasoning-trace insertion beyond standalone generation.Blockwise SFT improves performance on complex reasoning tasks, and PRISM and IGPO use re-masking or inpainting for revision and sparse-reward exploration.Source: §8.4 · p. 40

9 Applications

• Discrete diffusion applications span engineered tokenizations and naturally discrete domains with uneven evidence maturity, from established text, code, and tokenized media to specialized or emerging scientific, planning, and tabular settings.Table 8 organizes domains by how their discrete state spaces arise and distinguishes well-established, active specialized, and comparatively emergent evidence.Source: §9 · pp. 40–41, Table 8 · p. 41

9.1 Language Modeling and Text Generation

• Text diffusion has progressed from embedding- and simplex-space models to masked diffusion language models and large-scale diffusion LLMs, supporting global revision, bidirectional conditioning, controllable refinement, and robustness to local errors.Its main limitations are multi-step latency, weaker streaming, difficulty matching autoregressive models in in-context learning and reasoning, and inconsistent evaluation.Source: §9.1 · pp. 41–42

9.2 Code: Infilling, Repair, and Structured Constraints

• Code diffusion uses masked infilling with bidirectional context, enabling flexible generation orders and non-causal reasoning for fill-in-the-middle completion, bug fixing, and repository editing.Systems combine code-subword masking, bidirectional denoisers, reweighted objectives, and samplers augmented with syntactic or test-based constraints.Source: §9.2 · p. 42
• Iterative code denoising can enforce syntax and functional validity through intermediate constraints, abstract-syntax-tree patterns, unit-test guidance, and dynamic length adjustment for localized edits in long files.These mechanisms support code synthesis, mathematical-equation discovery, accelerated unit-test generation, and variable-length repository-level modifications.Source: §9.2 · p. 42

9.3 Multimodal Foundation Models (Text + Vision/Audio)

• Diffusion multimodal language models replace autoregressive multimodal backbones with bidirectional masked denoising, unifying understanding and generation across text, images, video, and audio.They use iterative remasking for incremental refinement or, in some omni backbones, perform understanding and generation within a single denoising step.Source: §9.3 · p. 43
• Multimodal diffusion systems either concatenate discrete modality tokens for shared self-attention, retain continuous visual encoders with text-only diffusion, or use hybrids with block-wise denoising, context pruning, and flexible token budgets.These choices trade a straightforward unified denoiser against codebook, vocabulary, positional-encoding, alignment, and computation-design challenges.Source: §9.3 · p. 43
• Because modalities carry different information per token, multimodal models require modality-specific corruption schedules, including separate mask ratios, block-wise video masking, flow-based velocity schedules, or distinct stream timesteps.The masking distribution also shapes which visual features are learned, even under a purely generative objective.Source: §9.3 · pp. 43–44
• dMLLMs introduce cascading early errors, anchor-induced repetition, tokenizer-limited visual or auditory artifacts, and sensitivity to test-time computation allocation, making them complementary rather than direct replacements for autoregressive multimodal foundations.Remasking, verification, scheduling, repetition control, and tokenizer design therefore remain central tuning burdens.Source: §9.3 · p. 44

Shared pipeline

• The canonical pipeline tokenizes continuous data, denoises the resulting codes with discrete diffusion, and decodes them back into the original signal.VQ-Diffusion (Gu et al., 2022) and its improved version (Tang et al., 2022) established this categorical alternative to Gaussian diffusion, while later work added conditioning and guidance without changing the tokenize-then-denoise structure.Source: §9.4 · p. 44

Images and videos

• For images and video, parallel token denoising supports constrained editing, infilling, global composition, and temporally structured generation through remasking and block- or frame-level corruption.Representative systems include masked image generation (Chang et al., 2023; You et al., 2025b), video prediction and cinematic transitions (Lu et al., 2024; Wu et al., 2025d), and unified AR/discrete-diffusion video generation.Source: §9.4 · p. 44

Audio and speech

• For audio and speech, neural codecs such as residual vector quantization produce parallel token streams enabling generation, music synthesis, inpainting, and iterative refinement for recognition.This framework spans text-to-sound, music, expressive speech, and discriminative applications including ASR, while some work applies vector quantization directly in acoustic space.Source: §9.4 · p. 45

Strengths and limitations

• Tokenized diffusion excels at parallel decoding, remasking-based editing, and reuse of text or multimodal backbones, supporting inpainting, outpainting, and localized editing without retraining.These workflows re-mask and re-decode selected token regions under desired conditioning.Source: §9.4 · p. 45
• Codec fidelity limits output quality because discarded details cannot be recovered, while small codebooks can cause blocky images, tinny speech, and video flickering.Improved tokenizers and residual codecs mitigate artifacts, but do not remove the fundamental upper bound imposed by the codec.Source: §9.4 · p. 45
• Codebook topology also shapes denoising transitions: nonsmooth neighborhoods over code indices can produce unnatural signal changes, making tokenized diffusion complementary to continuous latent diffusion.It is most suitable when editing, infilling, discrete-modality integration, or highly parallel decoding outweighs the need for maximum perceptual fidelity.Source: §9.4 · p. 45

9.5 Proteins and Biological Sequences

• Protein diffusion models generate amino-acid sequences under stability, motif, and functional constraints, with DPLM and DPLM-2 extending transferable and multimodal sequence–structure modeling (Wang et al., 2024b; Wang et al., 2025k).Guided denoising combines sequence models with property predictors across antibody, peptide, membrane-protein, combinatorial, and function-preserving protein design (Gruver et al., 2023; Luo et al., 2022; Zhu et al., 2024a; Chen et al., 2025c; Wang et al., 2025j; Tang et al., 2025a; Wu et al., 2024; Yin et al., 2025; Goel et al., 2026; Baron et al., 2025).Source: §9.5 · pp. 45–46
• Protein generation increasingly conditions on three-dimensional structure through inverse folding, geometric constraints, conformational modeling, property optimization, and context-guided diffusion for distribution shift (Yi et al., 2025; Villegas-Morcillo et al., 2025; Bai et al., 2025; Gao et al., 2023; Lu et al., 2025b; Wang et al., 2024e; Klarner et al., 2024).Evaluation combines perplexity, recovery, motif accuracy, scTM, pLDDT, and, where available, wet-lab measurements of stability, binding, and function (Zhu et al., 2024b).Source: §9.5 · p. 46

9.6 Genomics (DNA/RNA) and Regulatory Sequences

• Nucleic-acid diffusion exploits global bidirectional refinement to coordinate regulatory grammar, splicing motifs, and secondary structure across long DNA and RNA sequences, including controllable DNA design via Dirichlet flow matching (Stärk et al., 2024).Applications include RNA structure prediction and inverse folding, joint nucleic-acid–protein sequence–structure generation, and multimodal single-cell and regulatory modeling (Wang et al., 2024f; Hou et al., 2025; Morehead et al., 2024; Wang et al., 2026a; Nair et al., 2025; Zou et al., 2024).Source: §9.6 · p. 46
• Because expression, accessibility, and structure measurements are noisy, nucleic-acid diffusion should treat conditioning signals as soft targets rather than hard constraints.Source: §9.6 · p. 46

9.7 Molecules and Graph Generation

• Molecular graph diffusion models categorical atom and bond attributes jointly, but validity—not denoiser capacity—is the binding constraint, making guidance, projection, and constrained decoding central to sampling (Vignac et al., 2023).Subsequent work varies corruption, sampling order, state-space structure, representations, stability, and systems efficiency, including higher-order graphs, sparse formulations, and quantization (Siraudin et al., 2025; Xu et al., 2024; Kim et al., 2025e; Boget et al., 2023; Kong et al., 2023; Zhao et al., 2024a; Li et al., 2025c; Wang et al., 2025o; Carballo-Castro et al., 2025; Huang & Birdal, 2025; Gailhard et al., 2025; Wen et al., 2024; Chen et al., 2023d; Qin et al., 2025; Li et al., 2024b; Wang et al., 2024c; Tang et al., 2024a; Boget, 2025; Jo et al., 2024; Liu et al., 2025g; Gong & Sun, 2025).Source: §9.7 · pp. 46–47
• Two-dimensional molecular generation must enforce valence, connectivity, atom–bond consistency, and property constraints, whereas three-dimensional design adds equivariance, target conditioning, steric interactions, and fixed-size limitations.Methods use constrained sampling, fragment or motif tokenization, linker design, conditional guidance, graph transformers, composable conditioning, language prompts, and plug-and-play projection (Peng et al., 2023; Guan et al., 2023; Gu et al., 2024a; Lee et al., 2025c; He et al., 2025e; Lee et al., 2025b; Levy & Rector-Brooks, 2023; Lee et al., 2025e; Igashov et al., 2024; Tseng et al., 2023; Huang et al., 2023; Liu et al., 2024a; Qiao et al., 2025a; Huang et al., 2024; Nguyen et al., 2024; Gong et al., 2024; Sharma et al., 2024a).Source: §9.7 · p. 47
• Discrete graph diffusion extends to retrosynthesis, spectrum-conditioned molecules, crystals, protein co-design, planning, architecture search, graph analysis, and drug design, while emerging work addresses fairness, watermarking, and backdoor attacks.These applications frame property-guided generation as search over discrete configurations under learned objectives (Current et al., 2025; Yadav et al., 2025; Qiao et al., 2025b; Bohde et al., 2025; Zeni et al., 2023; Kelvinius et al., 2025; Park et al., 2025; Das et al., 2025; Campbell et al., 2024; Asthana et al., 2024; Huang et al., 2025a; Shi et al., 2024a; Yin et al., 2024; Li et al., 2025i; Chen et al., 2023b; Özçelik et al., 2025; Liu et al., 2025g; Kose & Shen, 2024; Gheda et al., 2026; Wang et al., 2025e).Source: §9.7 · p. 47

9.8 Planning, Agents, and Decision Making

• Discrete diffusion supports planning, structured action generation, and constraint-aware decision making across language agents, embodied control, multimodal reasoning, and combinatorial optimization.These applications use iterative refinement in structured discrete spaces rather than a single left-to-right generation process.Source: §9.8 · pp. 47–48
• Global revision can let diffusion agents refine plans, reasoning traces, actions, and trajectories using later context, but effectiveness depends on objectives, noise schedules, remasking, guidance, and inference design.Without suitable design, models may retain local inconsistencies or converge slowly.Source: §9.8 · pp. 47–48
• Diffusion over offline-RL trajectories generates or guides state-action sequences toward higher-reward behavior, while tokenization and dynamics-feasibility constraints critically determine practical performance.Reward or value conditioning can produce trajectories unlikely under the behavior policy but associated with higher returns.Source: §9.8 · p. 48
• For tool use, diffusion jointly refines structured fields and integrates JSON or schema constraints through masking, projection, or guided sampling, but validators, execution feedback, and verification remain necessary.Diffusion alone does not guarantee correct tool arguments or outputs.Source: §9.8 · p. 48
• In combinatorial optimization, diffusion iteratively refines diverse candidate solutions using learned problem-specific priors, while repair, projection, or constraint-aware transitions help maintain feasibility.Applications include graph optimization, routing, scheduling, and structured design, with effectiveness depending on how constraints are encoded.Source: §9.8 · p. 48
• Across planning domains, discrete diffusion is complementary to autoregressive and classical methods because its promised global consistency and conditioning flexibility require careful tokenization, corruption, and inference strategies.Constraint handling remains central to whether iterative refinement produces valid decisions.Source: §9.8 · pp. 48–49

9.9 Tabular Data and Imputation

• Tabular diffusion combines categorical columns with numerical columns through discretization or hybrid continuous-discrete processes, including shared-schedule Gaussian and multinomial diffusion in TabDDPM (Kotelnikov et al., 2023).Later approaches include mixed-type multimodal diffusion in TabDiff (Shi et al., 2025a; 2024c), joint non-autoregressive generation in TabNAT (Zhang et al., 2025b), and masked tree-based prediction in Unmasking Trees (McCarter, 2025).Source: §9.9 · p. 49
• Missing-value imputation maps directly to masked diffusion: observed cells provide context, arbitrary missing subsets are filled during reversal, and evaluation jointly considers fidelity, privacy, and downstream utility.Because fidelity improvements can reduce privacy, evaluations should report all three criteria rather than a single aggregate score.Source: §9.9 · p. 49

Layout and document generation

• Layout diffusion jointly models categorical element types and quantized coordinates, while iterative remasking improves global alignment and local placement under layout-specific validity metrics.Research conditions on partial layouts or graphs, corrects sticking and pattern collapse, and specializes to floor plans, vector graphics, and landmark layouts.Source: §9.10 · p. 49

3D assets generation

• 3D generation discretizes shapes into primitives or VQ tokens, balancing geometric expressiveness against long sequences that motivate block-wise or absorbing diffusion schedules.Methods cover part-level shapes, large scenes, mixed continuous-discrete indoor modeling, part-by-part meshes, and 2D-to-3D alignment.Source: §9.10 · pp. 49–50

Sketches, fonts, and stylized media

• For sketches, fonts, calligraphy, and stylized media, discrete diffusion preserves parallel decoding and editability in compact code spaces that better enforce legibility and stroke order.Representative tasks include sketch-conditioned inpainting, glyph synthesis, font creation, calligraphy, and arbitrary style transfer.Source: §9.10 · p. 50

Recommendation and information retrieval

• Recommendation and retrieval systems reuse masked denoising for interaction sequences, generative recommendation, reranking, bundle generation, graph-based RAG retrieval, and graph-feature imputation.The section summarizes these directions briefly and refers planning-oriented applications to Section 9.8.Source: §9.10 · p. 50

Specialized scientific and structured domains

• Across biomedical, communications, knowledge, forecasting, vision-language, and scientific tasks, tokenizers and categorical denoisers enable parallel, editable, constraint-aware generation over domain-specific vocabularies.Examples include MRI super-resolution, quantized channel symbols, low-power spiking models, structured knowledge, time-series forecasting, captioning, segmentation, defect detection, lithography, and uncertainty-aware diffusion.Source: §9.10 · p. 50

A checklist for new case studies

• The proposed case-study checklist standardizes reporting of tokenization, corruption, objectives, architecture, sampling, constraints, evaluation, and failure modes for cross-domain comparison.The checklist is an authors’ recommendation rather than an established community standard, intended to clarify which design choices generalize across domains.Source: §9.10 · pp. 50–51

10.1 Likelihood, Perplexity, and Calibration

• Likelihood and perplexity are informative but not directly comparable with AR metrics because diffusion evaluations may use variational bounds or denoising surrogates sensitive to tokenization, normalization, and sampling.Evaluation should pair likelihood-style measures with generation quality, timestep-wise calibration, and sampler diagnostics; confidence errors can freeze incorrect tokens or waste refinement computation.Source: §10.1 · pp. 51–53

10.2 Quality Metrics Across Domains

• Task-specific and constraint-aware evaluation is preferred over single surface metrics, with separate tokenizer reconstruction, generation quality, and encode-generate-decode consistency checks across domains.Reference-overlap metrics can misjudge diverse dLLM outputs, while media, biological, molecular, graph, and optimization tasks require complementary structural, functional, validity, novelty, or distributional measures.Source: §10.2 · pp. 53–54
• Agent evaluation should prioritize execution and end-to-end success, while also reporting syntactic or schema validity and the currently under-standardized benefits of iterative self-correction.Pass@k, schema compliance, tool-loop success, correction frequency, and pass@k across denoising steps distinguish functional correctness from fluent but unusable outputs and reveal whether added compute improves correction.Source: §10.2 · pp. 54–55

10.3 Speed, Cost, and Efficiency Metrics

• Efficiency comparisons should report denoising steps, forward passes, latency, throughput, memory, hardware, and implementation conditions rather than relying on one speed number.Because quality changes with sampling budgets and system settings, the recommended comparison is a quality–latency frontier, extended to full planning-cycle cost for agents: Total cost ≈ N_denoise C_forward + N_tool C_tool + N_verify C_verify.Source: §10.3 · pp. 55–57
• End-to-end efficiency should pair decoding latency with pass rate, repair rate, validation cost, tool calls, verification failures, GPU time, and monetary cost.Parallel candidates can reduce wall-clock latency while increasing total compute, whereas sequential candidates increase both latency and compute.Source: §10.3 · p. 57
• Efficiency reports should include forward passes, latency, throughput, memory, hardware, batch and sequence settings, acceleration, and quality–latency frontiers rather than denoising steps alone.Agentic evaluation should measure the full planning cycle, including candidate generation, tool execution, verification, and successful task completion per unit cost.Source: §10.3 · p. 57

10.4 Editing, Infilling, and Conditional Fidelity

• Editing benchmarks measure infilling consistency, minimal-change editing, and constraint adherence, but discrete diffusion additionally requires out-of-span preservation rate alongside these measures.HumanEval-Infilling, SARI, COLLIE, and EdiText (Lee et al., 2025a) exemplify existing and diffusion-specific evaluation directions.Source: §10.4 · p. 58
• Conditional generation should jointly report conditioning fidelity and unconditional quality, while treating guidance weight as a first-class variable because early high guidance can degrade quality whereas late guidance improves fidelity.Fidelity should use a held-out attribute classifier rather than the conditioning model to avoid circular evaluation (Koh et al., 2025).Source: §10.4 · pp. 58–59

11.1 Diffusion as Global Discrete Refinement vs. Autoregressive Generation

• Diffusion updates many positions using full-sequence context, enabling coarse-to-fine planning and revision of earlier content that autoregressive decoding cannot perform without explicit revision machinery (Ye et al., 2025a; Liu et al., 2025e).This reframes reverse generation as global discrete refinement rather than left-to-right commitment.Source: §11.1 · p. 59
• Revision can improve coherence and constraint satisfaction through remasking, but only when decoding policies budget steps for correction rather than greedily freezing uncertain tokens.The text presents revision as a potential advantage of the sampler, not a guaranteed property of diffusion models.Source: §11.1 · p. 59
• The reverse chain is proposed as a stochastic optimizer over discrete configurations, with learned proposals and guidance supporting soft constraints while trading exact optimality guarantees for amortized speed.This is explicitly an interpretive perspective connecting diffusion to combinatorial search and offline black-box optimization.Source: §11.1 · p. 59
• Autoregressive models remain preferable for streaming and incremental interaction, so hybrids and task-dependent matching are favored over a winner-take-all choice between paradigms.Relevant task factors include latency, revision needs, constraint density, and streaming requirements.Source: §11.1 · pp. 59–60

11.2 Trustworthiness: Hallucination, Uncertainty, and Safety Constraints

• Reduced hallucination remains unestablished, whereas native confidence signals could enable abstention, clarification, retrieval, or verification if calibrated into actionable uncertainty estimates.Bidirectional context and revision are plausible benefits, but parallel decoding may produce mutually inconsistent tokens.Source: §11.2 · p. 60
• Guidance and hard per-step constraints can steer safe structured outputs, but diffusion-specific jailbreak, privacy, and adversarial vulnerabilities mean safety differences from AR models remain open.The cited work covers jailbreak and red-teaming, intrinsic safety, membership inference, privacy, and adversarial prompting (Zhang et al., 2025g; Nöther et al., 2025; Wen et al., 2025c; Li et al., 2025j; Shi et al., 2026; Yamabe & Sakuma, 2025; Pang & Wang, 2025; Wei et al., 2024; Wang et al., 2025d).Source: §11.2 · p. 60

12.1 Scaling Laws, Compute Trade-offs, and Benchmark Standardization

• Scaling laws remain poorly understood, requiring standardized independent ablations of model size, denoising steps, noise schedules, tokenization, and training tokens across domains and scales.The field must test whether diffusion scaling matches autoregressive models and whether the compute-optimal frontier favoring smaller models trained longer generalizes.Source: §12.1 · p. 60
• Benchmark standards should report steps, latency, hardware, decoding, batch and prompt lengths, and tokenization, with reproducible suites spanning text, graphs, and biological sequences.Because diffusion exposes a quality-compute frontier rather than a single operating point, standardized reporting is necessary for interpretable comparisons and testable scaling claims.Source: §12.1 · p. 61

12.2 Closing the Gap with AR: In-Context Learning, Caching, and Streaming

• Closing diffusion’s gaps with autoregressive models requires stronger in-context learning plus efficient caching and streaming without sacrificing bidirectional refinement.Promising directions include multi-pass prompting, retrieval-augmented denoising, prompt-protecting conditioning, hybrid decoding, block-wise or delayed KV reuse, speculative refinement, and confidence-gated freezing.Source: §12.2 · p. 61

12.3 Unified Token Spaces and Cross-Modal/Scientific Integration

• Unified token spaces could support cross-modal transfer and joint scientific sequence-structure modeling, while requiring coordinated vocabulary granularity, codebook topology, corruption schedules, and process-level physical constraints.Examples include text, image, audio, and video tokens; amino-acid sequences with structural descriptors; and molecular graphs with 3D conformer tokens.Source: §12.3 · p. 61

12.4 Theory: Expressivity, Convergence, and Identifiability

• Theory still needs unified accounts of schedule, transition design, and parameterization effects on sampling error, alongside limits on expressivity and systematic identifiability analyses.Existing results include guarantees and error bounds for discrete samplers (Zhang et al., 2025i; Chen & Ying, 2025; Liang et al., 2025a; Wakasugi & Suzuki, 2025; Wan et al., 2025; Su et al., 2025), parallel-decoding and information-theoretic analyses (Jiang et al., 2025a; Foresti et al., 2026; Bai, 2025; Chen et al., 2026), and identifiability work (Liang et al., 2025b; Ye et al., 2025f).Source: §12.4 · pp. 61–62

• The survey unifies discrete diffusion around state-space construction and reviews formulation components, objectives, inference, scaling, systems, and applications across diverse domains.Its scope spans text and code, multimodal generation, proteins, genomics, molecules, graphs, planning and agents, and tabular data.Source: §13 · p. 62
• Effective discrete diffusion designs co-adapt state-space design, corruption, learning objectives, and sampling to data structure and task demands, enabling parallel refinement, infilling, editing, and constraint-aware generation.These capabilities follow from the models’ bidirectional, iterative nature and are difficult to obtain from purely left-to-right models.Source: §13 · p. 62
• Hybrid systems combining autoregressive planning or backbones with diffusion refinement or editing may dominate practical deployments, while scaling, in-context learning, caching, unified token spaces, and theory remain open problems.The survey presents these problems as concrete and tractable determinants of how far discrete diffusion can advance as a framework for controllable discrete generation and structured discovery.Source: §13 · p. 62

Dual use in scientific generation

• Scientific generators can lower barriers to designing biological and chemical sequences, while hard validity and property constraints can exclude hazardous regions alongside access controls, filters, and intended-use documentation.The survey avoids operational detail on harmful applications.Source: Broader Impact · p. 62

Safety of diffusion language models

• Diffusion language models require diffusion-specific safety evaluation because arbitrary-order decoding may enlarge jailbreak exposure, while watermarking and membership-inference defenses may not transfer unchanged from autoregressive models.Recommended reporting includes order-varying jailbreak attacks, training-regime-appropriate privacy tests, and whether watermarking is order-agnostic.Source: Broader Impact · pp. 62–63

Synthetic-data privacy

• Synthetic-table fidelity can leak information about real records, creating tension among fidelity, privacy, and utility; evaluations should report membership-inference or closest-record analyses alongside fidelity and utility.Optimizing fidelity alone can degrade privacy, so privacy costs should remain visible rather than hidden by an aggregate score.Source: Broader Impact · p. 63

Accessibility and concentration of capability

• Broader, better-documented access to discrete diffusion methods may improve reproducibility and scrutiny, but also expands access to the dual-use concerns the survey identifies.The survey emphasizes design principles and published citations rather than step-by-step misuse guidance.Source: Broader Impact · pp. 62–63