PaperlayerView exact revision on arXiv

Source-linked AI summary

Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported

Hasan Amin, Ming Yin, Rajiv Khanna

arXiv:2609.33947v1cs.CLcs.LG

TL;DR

Few-step diffusion language model quality can falter despite better denoiser predictions, and per-output metrics do not capture whether outputs repeat. The paper tunes a sampler, introduces group-level quality and diversity evaluation, and analyzes parallel sampling; its results show sharper sampling can improve judged generation without retraining.

  • Problem

    DLMs promise parallel generation, but high-quality samples can require many refinement steps, and denoiser prediction quality does not necessarily indicate sample quality at a fixed budget.

    Source: §1 · p. 1

  • Method

    The paper sharpens an unchanged MDLM sampler, introduces group-level quality and diversity scores, and analyzes prediction–generation error under parallel unmasking.

    Source: §1 · pp. 2–3, §3.2 · p. 5

  • Results

    At 64 steps, MDLM♯ reaches gPPL 41 versus 107 for the standard sampler at 1,024 steps, while GROUPEVAL finds higher quality and diversity despite DiDi-Instruct’s 1.5–4.7× perplexity lead.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    The deployed checkpoint-plus-sampler generator should be tuned and evaluated jointly for output quality and range, rather than judged by training loss or single-output scores alone.

    Source: §5 · p. 9

    The central audit concerns unconditional generation with small masked DLMs, and sampler tuning is limited to MDLM and DiDi-Instruct in cross-model comparisons.

    Source: §5 · p. 9

Abstract

from arXiv · show

Diffusion language models (DLMs) promise fast parallel generation, yet high-quality samples often require large number of refinement steps, which diminishes their advantage in practice. This has led to massive interest in and rapid development of new methods for effective few-step generation. We show that much of the supposed quality gap at few steps can instead arise from a suboptimally configured sampler. Modest sampler sharpening, without any model retraining, enables a couple years old masked DLM to rival supposedly far improved successors. This differently sampled DLM in fact achieves lower generative perplexity in just 16 steps than what its standard sampler obtains with 1024, while improving both judged quality and semantic diversity. We further show that conventional per-output metrics can fundamentally obscure these gains, since any optimal trade-off between two such metrics can be attained by a generator supported on at most two outputs. We subsequently introduce GroupEval, which separately evaluates quality and across-output semantic diversity, and offers fresh insights including uncovering how 1.5-4.7x perplexity gains of a distilled model yield no corresponding quality gain. Finally, we explain why sharpening helps: parallel unmasking destroys dependencies among simultaneously generated tokens, creating a gap between prediction and generation. We prove that pervasive temperature choice of one is generically suboptimal under parallel sampling even for an exact denoiser, and that worse predictions can yield better samples. Through these results, we argue for a broader evaluation principle of treating the deployed generator as the object of comparison, benchmarking it against tuned baselines, and assessing quality and diversity jointly and with more human-aligned measures.

1 INTRODUCTION

The paper argues that few-step diffusion language model performance depends on the deployed sampler, not just denoiser predictions. It combines sampler tuning, group-level evaluation, and analysis of the prediction–generation gap to reassess few-step generation.

1 INTRODUCTION

  • DLM training evaluates a denoiser, whereas deployment uses a finite-step generator, so better predictions need not produce better samples at a fixed sampling budget.The paper studies this distinction in the original masked diffusion language model.Source: §1 · p. 1
  • At 64 steps, unchanged MDLM with temperature .9 and fp32 reaches gPPL 41 versus 107 for its standard sampler at 1,024 steps.The sharpened baseline also beats distilled DUO’s best reported result at any budget.Source: §1 · p. 1
  • At standard sampling, a checkpoint trained 44× longer or a model with 5× more parameters becomes a worse 2–8-step generator despite better held-out posterior loss.The paper says these comparisons reverse as sampler sharpening increases.Source: §1 · p. 2
  • Per-output metrics cannot detect output collapse: every optimal trade-off between two such scores can be attained by a generator supported on at most two outputs.GROUPEVAL separately measures quality and semantic diversity, and favors MDLM♯ over DiDi-Instruct despite DiDi’s 1.5–4.7× perplexity lead.Source: §1 · pp. 2–3
  • Parallel unmasking loses dependencies among simultaneously revealed tokens, so even an exact denoiser is generally best sampled at a temperature other than one.The paper’s finite-step analysis also shows that worse predictions can yield better few-step samples.Source: §1 · pp. 2–3

2 THE FEW-STEP GAP IS LARGELY A SAMPLING ARTIFACT

The paper finds that sampler configuration can dominate few-step MDLM results: modest sharpening improves gPPL and changes comparisons with larger or longer-trained models. These effects arise without changing the checkpoint, schedule, or algorithm.

2 THE FEW-STEP GAP IS LARGELY A SAMPLING ARTIFACT

  • An MDLM generator depends on its checkpoint, number of sampling steps, temperature, and categorical precision; each update reveals masked positions and draws their tokens from the denoiser.The fp32 random numbers are too coarse to reach the extreme tail, making rare tokens less likely; precision also affects reveal decisions.Source: §2 · p. 3
  • The sampler setting MDLM♯ was selected once from a temperature–precision sweep, then held fixed across budgets, judges, and models without changing weights, schedule, or algorithm.The setting was chosen as the sharpest that kept token entropy at least 5 at most budgets.Source: §2 · p. 3
  • At 64 steps, MDLM♯ reaches gPPL 41.3 versus 107.1 for the standard sampler at 1,024 steps; temperature alone reaches 53.1.At 1,024 steps, adding fp32 yields a 4.9× gPPL reduction; the table reports GPT-2 gPPL with unigram entropy in brackets.Source: §2 · p. 3, Table 1 · p. 3
  • At 64 steps, MDLM♯ scores GPT-2-Large gPPL 44.5, below distilled DUO’s best result at any budget, and from 128 steps matches or beats DiDi-Instruct.The cross-model comparison evaluates other models at their standard sampler.Source: §2 · p. 4

2.1 LESS IS MORE, UNTIL THE SAMPLER IS FIXED

  • At standard sampling, the longer-trained checkpoint is 1.40× worse in gPPL at four steps, and the 170M MDLM-Small is worse than the 34M MDLM-Tiny at 2–8 steps.Both inversions shrink with sharpening and disappear at temperature .8; from 64 steps, the larger or longer-trained model is 1.4–1.9× better in gPPL.Source: §2.1 · p. 4, Figure 2 · p. 4

3 ARE THE GAINS REAL? EVALUATE GROUPS, NOT SAMPLES

The paper argues that per-output scores miss whether a generator repeats itself, and introduces GROUPEVAL to measure quality and semantic diversity across output groups. Its results show that sharper MDLM sampling improves both measures, while perplexity advantages do not reliably imply quality advantages.

3.1 PER-SAMPLE METRICS REWARD COLLAPSE

  • Every optimal trade-off between two per-output scores can be attained by a generator supported on at most two outputs, which can conceal repetition despite favorable scores.For groups of eight, such a generator’s GED is capped at 3.3, whereas the studied MDLM samplers score 6–9.Source: §3 · p. 4, §3.1 · p. 5

3.2 GROUPEVAL: QUALITY PER OUTPUT, MODES PER GROUP

  • GROUPEVAL scores each output’s rubric-based quality as GEQ and group-level semantic diversity as validity-weighted normalized mode entropy, GED.Its fixed group protocol lets the two scores distinguish good but similar outputs from varied but weak ones.Source: §3.2 · p. 5

3.3 SHARPER SAMPLING IMPROVES QUALITY AND DIVERSITY TOGETHER

  • Under the default judge, MDLM♯ raises GEQ and GED at every budget while token entropy falls by .44–.62 nats.At 16 steps, invalid outputs fall from 13.3% to 6.6%, and the effective number of modes among valid outputs per group of eight rises from 4.73 to 5.74; all five judges give MDLM♯ higher GEQ at all four budgets.Source: §3.3 · p. 5, Table 2 · p. 6

3.4 A PERPLEXITY LEAD IS NOT NECESSARILY A QUALITY LEAD

  • Likelihood evaluators favor DiDi-Instruct over MDLM by 1.5–4.7×, yet GROUPEVAL gives MDLM higher GEQ in seven of eight comparisons and higher GED in all eight.At 16 steps under the sharpened sampler, GED is 6.6 for MDLM versus 2.0 for DiDi-Instruct.Source: §3.4 · p. 6
  • Sharpening roughly halves DiDi-Instruct’s gPPL but yields no judge-robust quality gain; conditional GEQ also separates correct from incorrect solutions on GSM8K and HumanEval.The conditional checks report AUROC .91 on GSM8K and .96 on HumanEval.Source: §3.4 · p. 6

4 WHY A PERFECT DENOISER NEEDS SHARPENING

Parallel sampling can make a perfect denoiser produce outputs that are too random, so the temperature minimizing prediction loss need not minimize output error. The section explains this mismatch through reveal-path mixing, proves temperature one is generically suboptimal for parallel sampling, and shows sharpening can repair samples even as posterior loss worsens.

4.1 TRAINING INTEGRATES, SAMPLING SUMS

  • Training integrates prediction losses across masking levels, whereas a few-step sampler sums over reveal rounds and independently draws tokens revealed together.The output averages reveal-path components: same-round draws lose token dependence, while sequential draws preserve it.Source: §4 · p. 6, §4.1 · p. 7
  • The sampler’s error decomposes into posterior error, the parallelism cost, and a temperature-dependent credit for mixing reveal paths.The parallelism cost depends only on the data and grid; it remains even for a perfect model, while posterior error vanishes for the Bayes denoiser at λ = 1.Source: §4.1 · pp. 7–8

4.2 TEMPERATURE ONE IS ALMOST NEVER RIGHT FOR PARALLEL SAMPLING

  • For parallel sampling with at least two rounds, λ = 1 is stationary only for a measure-zero set of full-support targets; otherwise some temperature gives a better output law.When each round reveals at most one coordinate, the exact Bayes denoiser at λ = 1 instead reproduces the target law.Source: §4.2 · p. 8

4.3 EXACT REPAIR: A WORSE PREDICTOR CAN GENERATE BETTER

  • In the two-bit repair example, λ⋆≈2.59 restores the target agreement probability, although the denoiser’s posterior loss increases.For the finite-repair regime, a less accurate predictor sampled at λ = 1 can have strictly lower output error than the Bayes denoiser; the sharpening benefit fades as rounds become sequential.Source: Figure 3 · p. 7, §4.3 · pp. 8–9

4.4 IN TRAINED MODELS, SHARPENING HURTS PREDICTION AND HELPS GENERATION

  • Sharpening from τ = 1 to .9 raises schedule-matched posterior loss by .036–.068 nats per token while gPPL falls 2.6–3.8× across the measured checkpoints and budgets.All 15 paired intervals for the loss increase are above zero; the posterior-optimal grid temperature is 1 or 1.05.Source: §4.4 · p. 9

5 DISCUSSION AND CONCLUSION

The discussion argues that progress in few-step generation should be measured on deployed generators, with tuned sampler baselines and joint assessment of quality and diversity. The paper’s conclusions are scoped by its experiments and by limitations of both the theory and GroupEval validation.

5 DISCUSSION AND CONCLUSION

  • Sharpening can compensate for dependence lost in same-round draws, while better schedules, joint draws, and revision address that loss differently.The paper links the mechanism to the finding that better predictions need not produce better generators.Source: §5 · p. 9
  • GEQ and GED confirm MDLM’s sharpening gain, but show no judge-robust gain from sharpening DiDi-Instruct and no judged quality lead behind its perplexity advantage.These conclusions apply to the outputs studied.Source: §5 · p. 9
  • The audit centers on unconditional generation with small masked DLMs; sampler tuning was limited to MDLM and DiDi-Instruct, and the theory assumes exact, non-adaptive reveal sampling.The authors also identify comprehensive human validation of GroupEval’s quality scores and mode assignments as useful further work.Source: §5 · p. 9
  • A deployed generator—not its checkpoint or single-output scores—is the comparison target, so sampler tuning and joint quality-and-diversity evaluation can change apparent progress.The authors recommend tuning temperature and categorical precision for both new methods and baselines under the same protocol.Source: §5 · p. 9

REPRODUCIBILITY STATEMENT

The paper identifies where detailed proofs and implementation materials can be found.

REPRODUCIBILITY STATEMENT

  • Detailed proofs appear in Sections B.1 and C, and the authors make relevant code, including precise prompts, available online.The cited repository is https://github.com/shasanamin/more-effective-dlms.Source: REPRODUCIBILITY STATEMENT · p. 10

ETHICS STATEMENT

The ethics statement describes risks of web-trained text generation and the scope and limitations of the automated evaluation.

ETHICS STATEMENT

  • GroupEval measures rubric-based perceived quality and local semantic diversity, not factuality, safety, or demographic fairness, and judge biases or hosted-model changes may affect scores.The work evaluates text generation and does not involve a new human-subject study.Source: ETHICS STATEMENT · p. 10

AI USE STATEMENT

The appendix documents the sampling and evaluation setup, then audits sampler choices across MDLM-Small checkpoints and budgets, MDLM-Tiny, and broader configuration grids. The authors also disclose that AI tools assisted with literature retrieval, theoretical-framework refinement, and presentation figures.

AI USE STATEMENT

  • AI tools assisted with literature retrieval, refinement of the theoretical framework, and creation of presentation figures; the authors remain responsible for the final text and artifacts.Source: AI USE STATEMENT · p. 10

A.1 EXPERIMENTAL SETUP

  • Experiments vary temperature and categorical-kernel precision while keeping weights, time grids, and reveal rules fixed, without retraining, fine-tuning, or distillation.The standard sampler is (τ=1, fp64); the sharpened setting is (τ=0.9, fp32), an empirical guard-constrained operating point rather than a universal recommendation or calibration optimum.Source: §A.1 · pp. 15–16

A.2 THE FOUR-CORNER AUDIT HOLDS ACROSS CHECKPOINTS AND BUDGETS

  • At 1,024 updates, (τ = .9, fp32) improves public-checkpoint gPPL by 4.9× while passing the H ≥ 5 guard, and is best among the four corners for every M ≥2.Across the listed checkpoint–budget cells, the standard sampler has the largest gPPL.Source: Table 3 · p. 16

A.3 MDLM-TINY: THE SAME EFFECT AT SMALLER SCALE

A.4 FULL SAMPLING-CONFIGURATION GRIDS

  • The full grids report gPPL with within-sequence entropy for standard and sharpened settings across update budgets, with cells below H=5 marked.The grids cover MDLM-Tiny and MDLM-Small checkpoints, as well as unguided and reward-guided DiDi-Instruct.Source: §A.4 · p. 17, Table 4 · p. 17, Table 5 · p. 18

A.5 THE SHARPENED SAMPLER IS ALSO MEMORY AND COMPUTE EFFICIENT

The appendix reports the sharpened sampler’s resource use, cross-model diagnostics, posterior-loss comparison, and evaluation framework. It also describes where the sharpening result does not extend and what GroupEval’s diversity score can and cannot establish.

A.5 THE SHARPENED SAMPLER IS ALSO MEMORY AND COMPUTE EFFICIENT

  • At M = 1024, MDLM♯ uses 40.0% less peak memory and achieves 17.2% higher throughput than the standard sampler.The H100 comparison uses batch size 8 and includes the extra log-softmax path for τ ≠ 1.Source: §A.5 · p. 20

A.6 THE UNCHANGED BASELINE RIVALS ITS DISTILLED SUCCESSORS

  • At M = 1024, MDLM♯ has H = 4.98, just below the guard; sharpened DiDi-Instruct also crosses slightly at M ≥512, while a τ = .95 variant passes throughout.The entropy crossings are diagnostics, not certificates of across-sample diversity or distributional correctness.Source: §A.6 · p. 20

A.7 SHARPENING WORSENS THE SCHEDULE-MATCHED POSTERIOR LOSS

  • Sharpening’s improved outputs are not explained by lower schedule-matched posterior loss: at τ = .9, the reported early/late gPPL reversal occurs despite lower loss for the late checkpoint.The late checkpoint’s profile mean is lower by 0.674 nats (95% CI 0.629–0.722).Source: §A.7 · p. 21

A.8 LARGE DIFFUSION LMS ALREADY SIT AT THEIR SHARP LIMIT

  • Greedy decoding is best for both large diffusion models on GSM8K and for Dream on HumanEval; both models fail at sufficiently high temperature.The authors caution that sharpening positive-temperature baselines does not establish that instruction-tuned defaults need changing.Source: §A.8 · p. 22, Figure 7 · p. 22

B.1 PROOF OF PROPOSITION 1

  • With two per-output scores, every Pareto-optimal value can be attained by a generator supported on at most two outputs.The proof maps generators to a convex polygon of score pairs; its boundary lies on vertices or edges.Source: §B.1 · p. 23

B.2 ELEMENTARY PROPERTIES AND LIMITS OF GED

  • A group of eight samples from a two-output generator has GEDg ≤10/3, while local GED saturation does not establish recovery of the data distribution.Small groups can reach the GED ceiling while observing a vanishing fraction of the total modes.Source: §B.1 · p. 23, §B.2 · p. 23

B.3.1 RUBRIC AND DETERMINISTIC SCORES

  • GroupEval scores individual quality and local across-output semantic diversity separately, using a fixed rubric and common protocol for interpretable comparisons.GED values from different group sizes are not interchangeable because group size changes collision probability and log n normalization.Source: §B.3.1 · p. 24

B.3.2 OUTPUTS, JUDGES, AND UNCERTAINTY

  • Across ten sampler–budget cells, pairwise judge correlations are .90–.95 for GEQ but .48–.90 for GED, indicating closer agreement on quality rankings than diversity rankings.The correlations include variation across budgets; within-budget contrasts test the sharpening gain directly.Source: §B.3.2 · p. 25

B.4.1 MDLM-SMALL: FULL JUDGE-SEPARATED RESULTS

Across the judge-separated studies, MDLM♯ improves judged quality robustly, while DiDi-Instruct’s likelihood advantage does not produce a judge-robust quality gain. On the same outputs, likelihood evaluators favor DiDi but judged quality and diversity generally favor MDLM, with the likelihood conclusion dependent on aggregation.

B.4.1 MDLM-SMALL: FULL JUDGE-SEPARATED RESULTS

  • At M ≥16, all twenty quality contrasts favor MDLM♯, thirteen conditional 95% intervals exclude zero, and diversity rises in sixteen of twenty contrasts despite lower unigram entropy in every pair.Source: §B.4.1 · p. 26

B.4.2 DIDI-INSTRUCT: A JUDGE-SENSITIVE BOUNDARY CASE

  • DiDi-Instruct’s sharpening gain is not judge-robust: GPT-OSS intervals include zero at every budget, Luna favors the standard sampler at all four primary budgets, and Terra is mixed.Luna’s intervals are below zero at M ∈{16, 1024}; its hybrid assignments and GED ceilings also limit interpretation of diversity.Source: §B.4.2 · p. 27, Table 12 · p. 27

B.4.3 EXACT-OUTPUT CROSS-GENERATOR COMPARISON

  • At M ≥16, both likelihood evaluators favor DiDi in all eight comparisons, while MDLM has higher GEQ in seven and tied GEQ in one, and higher GED in all eight.The evaluators’ cell rankings closely agree (Spearman ρ = .982); these outputs were rescored under matched conditions.Source: §B.4.3 · p. 27, Table 13 · p. 28
  • The likelihood-versus-quality disagreement is specific to the declared token-pooled metric: equal-output macro-gPPL on first documents can reverse a cross-model sign, and within-generator correlations weaken when generators are pooled.The reported GEQ/log-gPPL correlations are −.57 and −.74 for MDLM, and −.93 for DiDi; pooled correlations are −.20 and −.18.Source: §B.4.3 · p. 28

B.5 TASK-CONDITIONED CRITERION CHECK

The task-conditioned check finds that cGEQ tracks automatic-success ordering on GSM8K and HumanEval, but it is criterion-related evidence rather than independent human-preference validation. The finite-step analysis separates denoiser error from parallelization cost and shows that parallel sampling can add randomness even with a perfect denoiser, while grid refinement reduces the parallelization term.

B.5 TASK-CONDITIONED CRITERION CHECK

  • cGEQ matches automatic-success ordering on both tasks, with pooled AUROC .91 on GSM8K and .96 on HumanEval, but the evidence is not independent human-preference validation.Functional execution remains authoritative for HumanEval.Source: §B.5 · p. 28, Table 14 · p. 28
  • Human calibration of quality and validation of mode assignments remain open; small groups can miss modes, and repeated judging does not replace independent generation.Multiple judges test sensitivity, not human alignment.Source: §B.5 · p. 28

C.2.2 THE FINITE-STEP DECOMPOSITION

  • The finite-step decomposition separates the parallelization cost, determined by data and grid, from posterior error, which depends on the denoiser or temperature.Posterior error vanishes for the Bayes denoiser at λ = 1; it is what training and held-out loss measure.Source: §C.2.2 · p. 30

C.2.1 THE ENTROPY AREA IDENTITY AND THE TRAINING OBJECTIVE

  • Training integrates the entropy profile, while the finite-step sampler uses a left Riemann sum; its expected cross-entropy is JN(λ), decomposing into parallelization and posterior-error terms.The population objective’s entropy-area identity does not assert that a clipped implementation logs omitted endpoints.Source: §C.2.1 · pp. 29–30, §C.2.2 · p. 30

C.2.3 CONSEQUENCES FOR PARALLEL SAMPLING

  • Refining a grid by inserting a point never increases the parallelization term BN, but this does not establish a decrease in denoiser error or final output divergence.The result follows from h being nonincreasing.Source: §C.2.3 · pp. 31–32
  • For two tokens, parallel sampling with a Bayes denoiser raises output entropy by at least c I(X1; X2), the shared-round probability times their mutual information.The paper does not establish that this bound extends to L ≥3 under independent reveal rounds.Source: §C.2.3 · p. 32
  • For L = N = 1024 with equal reveal masses, the expected empty and singleton block counts are 376.53 and 376.89, and the token-sharing probability is .63194.These describe sampling geometry, not a lower bound on output error or actual model calls.Source: §C.2.3 · p. 32

C.3 PROOF OF THEOREM 2: THE BAYES TEMPERATURE IS GENERICALLY SUBOPTIMAL

The proof shows that parallel sampling can make temperature one suboptimal even for an exact Bayes denoiser. Analyticity and an explicit target witness establish generic nonstationarity, while enumeration checks the result on small random targets.

C.3 PROOF OF THEOREM 2: THE BAYES TEMPERATURE IS GENERICALLY SUBOPTIMAL

  • When every round reveals at most one coordinate, Bayes conditionals recover the target distribution for every reveal ordering, including autoregressive decoding.Thus temperature one attains zero output KL in this setting.Source: §C.3 · p. 32
  • For parallel grids, an analytic output-KL derivative and one explicit full-support target prove temperature one is nonstationary almost everywhere.The witness uses a correlated pair with agreement probability s and otherwise uniform independent coordinates; sharpening strictly improves its Bayes sampler.Source: §C.3 · pp. 32–33
  • Sharpening is the improving direction when reveal-path weights favor paths with higher conditional entropy, while temperature one is stationary when the derivative terms cancel.The random-target verifier found sharpening improved 279 of 280 targets; the median output-KL reduction at the optimum was 5.6%, for small random targets only.Source: §C.3 · p. 33

C.4 PROOF OF PROPOSITION 2: EXACT REPAIR AND INVERTED RANKINGS

This proof analyzes a two-bit collision example, where sharpening can exactly repair output error despite increasing posterior loss, and extends the argument to inverted predictor rankings. It then describes measurement and implementation details and relates sampler tuning to few-step comparisons.

C.4 PROOF OF PROPOSITION 2: EXACT REPAIR AND INVERTED RANKINGS

  • In the finite-repair regime, sharpening to λ∗ > 1 exactly recovers the target output law, although posterior loss is strictly positive; a worse predictor can outperform Bayes at fixed temperature one.For an equal N-round grid, the output-optimal temperature tends to the Bayes value as collisions vanish; this is a property of the stated family, not a prescribed language-model schedule.Source: §C.4 · p. 34

C.5 POSTERIOR-LOSS MEASUREMENT AND IMPLEMENTATION

  • The forced-mask estimator is unbiased, but the proof does not establish a universal variance advantage over alternative estimators.Its target coordinate is always masked, so no inverse factor 1/(1 − a) is needed.Source: §C.5.1 · p. 35

C.5.1 THE FORCED-MASK ESTIMATOR AND TEMPERATURE DERIVATIVES

  • For an exact Bayes denoiser, λ = 1 minimizes posterior loss; a finite contrast in posterior loss does not establish the derivative at one.A negative derivative supports sufficiently small increases in inverse temperature, not arbitrary finite sharpening.Source: §C.5.1 · p. 35

C.5.2 SCHEDULES, PRECISION, AND VERIFICATION

  • The idealized clipped schedule maps a uniform time grid to M ordinary reveal masses and a terminal mass, with N = M + 1.For L = 1024, matching pre-cleanup laws bounds the output-TV difference from terminal cleanup by approximately .01019; no numerical transition-kernel perturbation bound is estimated.Source: §C.5.2 · pp. 35–36

D EXTENDED RELATED WORK AND IMPLICATIONS FOR SAMPLER DESIGN

  • The discussion frames sampler design around how much few-step performance is attainable from the original checkpoint, which sampling changes help, and how to measure the result.The audit asks how much the original checkpoint can do when its sampler is tuned.Source: D EXTENDED RELATED WORK AND… · p. 36, §D.1 · p. 36

D.1 STRONGER BASELINES AND THE FEW-STEP GAP

  • The unchanged public MDLM-Small sampler at τ = .9 and fp32 reaches gPPL 41.3 at 64 steps, versus 107.1 at 1,024 standard steps, and matches or beats DiDi-Instruct from 128 steps.The baseline correction also exceeds the standard sampler at 1,024 steps on GEQ at 256 steps under every judge.Source: §D.1 · p. 36
  • The sampler audit also exposes an inversion between predictor accuracy and few-step generation quality.The cited related evidence connects this correction to prior findings that masked-model gaps can reflect parameterization and sampling design.Source: §D.1 · p. 36

D.2 SAMPLER DESIGN AND THE PRICE OF PARALLELISM

The discussion separates sampler mechanisms and argues for comparisons that control baseline sharpness and evaluate the deployed generator. GroupEval evaluates quality and diversity separately, revealing that perplexity gains do not always correspond to judged quality gains.

D.2 SAMPLER DESIGN AND THE PRICE OF PARALLELISM

  • Sampler mechanisms target different parts of the output-error decomposition: schedules affect the grid term, temperature trades posterior error against path credit, and joint draws address same-round dependence.State-dependent reveal laws and revision samplers fall outside the stated accounting.Source: §D.2 · p. 37
  • Comparisons should tune temperature and precision for both baseline and proposed sampler, report steps separately from network calls, and evaluate checkpoints with their samplers.The discussion also recommends group-level evaluation alongside gPPL and unigram entropy.Source: §D.2 · p. 37
  • For the tested 8B diffusion LMs decoded greedily, positive temperatures do not help.The analysis does not attribute a particular adaptive sampler’s gain to temperature under greedy decoding.Source: §D.2 · p. 37

D.3 EVALUATING QUALITY AND DIVERSITY ACROSS OUTPUTS

  • GroupEval uses blinded group-level mode partitioning, validity-weighted entropy, and a separate quality score for unconditional generation.Its released rubric, grouping rules, and scorer support a fixed, reusable protocol.Source: §D.3 · p. 38
  • Sharpening DiDi-Instruct roughly halves its gPPL but yields no judge-robust quality gain, while GroupEval keeps quality and diversity as separate requirements.Human calibration and validation of mode assignments remain open.Source: §D.3 · p. 38

Related papers

Loading 2609.33947v1…