PaperlayerView exact revision on arXiv

Source-linked AI summary

PIG: Parallel Inconsistency Guidance for Accelerating Diffusion Language Models

Quentin Fruytier, Anton Xue, Sanjay Shakkottai, Sujay Sanghavi

arXiv:2610.17375v1cs.CL

TL;DR

Parallel decoding can make jointly committed tokens inconsistent because they cannot condition on one another, while sequential decoding sacrifices parallelism. PIG predicts this disagreement from a single model pass to guide commits. Across two DLMs and several math, code, and multiple-choice benchmarks, it extends the accuracy–TPF frontier over confidence-based decoding.

  • Problem

    Parallel commits can produce jointly inconsistent tokens because positions committed together cannot condition on one another, while sequential decoding sacrifices the parallelism that makes DLMs attractive.

    Source: §1 · p. 2

  • Method

    PIG uses a lightweight adapter to predict sequential–parallel disagreement from the current DLM forward pass and guides parallel commit-set selection with that estimate.

    Source: §1 · pp. 2–3

  • Results

    Across two DLMs and math, code, and MMLU tasks, PIG extends the accuracy–TPF frontier; PIG.INT reaches up to 1.7× higher tokens per forward at matched accuracy on LLaDA.

    Source: §1 · p. 2

  • Takeaways & Limitations

    PIG directly targets disagreement with sequential decoding, and the authors conclude that this quantity can be predicted cheaply from a single model pass.

    Source: §6 · p. 11

    PIG uses greedy sequential decoding as its reference, and its predictor still depends on training-side data quality.

    Source: §6 · p. 11

Abstract

from arXiv · show

Diffusion language models accelerate generation by decoding multiple masked positions per forward pass. However, accuracy is known to drop when parallel decoding trajectories deviate from their sequential counterparts, which are slow and expensive "oracles" that adhere to the joint distribution of language. In this paper, we propose Parallel Inconsistency Guidance (PIG), a new paradigm that uses a trained auxiliary model to directly estimate the inconsistency between parallel and sequential generation from a single model forward pass. During inference, masked positions are traversed by decreasing confidence, and tokens are added to the parallel decode set one-at-a-time until the total estimated inconsistency between the parallel and sequential generation exceeds a threshold. Surprisingly, it suffices to train a lightweight ~2M parameter auxiliary model on a small mixture drawn from OpenCoder, GSM8K-train, MATH-train, and MBPP-train. For LLaDA-8B Instruct and NemotronDiffusion-3B Instruct, PIG extends the accuracy-TPF frontier on GSM8K, MATH500, MBPP, HumanEval, and MMLU, when compared to the industry standard Fast-dLLM and measures favorably on LLaDA-8B Instruct against DAPD, KLASS, EB-sampler, and other learned unmasking policies.

1 Introduction

Parallel decoding saves model evaluations but can make jointly committed tokens inconsistent because they cannot condition on one another. PIG predicts disagreement with sequential decoding from a single forward pass, then uses it to guide parallel commits.

1 Introduction

  • Masked diffusion models predict many positions in one evaluation, enabling flexible reveal orders and fewer forward passes, while existing methods select positions using confidence or dependence.The paper distinguishes its trajectory-preservation objective from the common objective of selecting positions with high conditional independence.Source: §1 · p. 2
  • Parallel commits can be jointly inconsistent because each masked position is predicted from the same canvas and cannot condition on the other simultaneous commits.Sequential decoding updates context after each commit, but sacrifices the parallelism that makes DLMs attractive; prior approaches use confidence or dependence signals.Source: §1 · p. 2
  • PIG frames parallel decoding as preserving the one-token-at-a-time trajectory, selecting as many positions as possible while keeping sequential–parallel disagreement low.PIG.INT controls inconsistency within a round, while PIG.EXT controls inconsistency avoidable by deferring positions.Source: §1 · pp. 2–3
  • A lightweight adapter to the frozen DLM predicts how committing one candidate would change other positions’ predictions using information from the current forward pass, avoiding sequential replay.The approximately 2M-parameter adapter estimates finite-difference structure, reaching r ≈0.8 per edge and r ≈0.9 on aggregate decoder statistics.Source: §1 · pp. 2–3
  • Up to 1.7× higher tokens per forward (TPF) at matched accuracy on LLaDA and 1.4× over Nemotron’s native confidence decoder; up to 28 additional accuracy points at the fastest confidence settings.PIG.INT reaches up to 4.4× higher TPF than dependency-guided decoders and compares favorably with learned unmasking policies.Source: §1 · pp. 2–3
  • Across LLaDA and Nemotron on GSM8K, MATH-500, MBPP, HumanEval, and MMLU, PIG improves the accuracy–TPF frontier over confidence decoding, with largest gains at aggressive parallelism.The approach also transfers across the two DLM architectures despite their different inference structures.Source: §1 · p. 3

2 Related Work

Prior parallel decoding methods primarily use prediction confidence or dependencies among masked positions to decide which tokens to commit together. Other work changes the generation protocol through blockwise, alternative-order, or hybrid decoding; PIG is presented as complementary to these approaches.

Confidence-based parallel decoding

  • Confidence-based methods use thresholds, top-k selection, entropy, stability, or learned policies; PIG instead scores proposed commit sets by predicted disagreement with sequential decoding.Fast-dLLM commits positions above a confidence threshold, as does Nemotron’s native diffusion decoder.Source: §2 · p. 3

Dependency-aware parallel decoding

  • Dependency-aware methods estimate interactions from attention, clustering, or learned conditional influence, whereas PIG uses interactions only insofar as they predict sequential–parallel disagreement.The cited theoretical work relates parallelization error to distributional dependence, including conditional total correlation and inference scheduling.Source: §2 · p. 3

Blockwise and hybrid decoding

  • Blockwise, alternative-order, and hybrid autoregressive–diffusion protocols alter generation structure; PIG is complementary and chooses which currently available positions to commit together.Block Diffusion denoises within autoregressively generated blocks, while Nemotron also supports diffusion proposals with autoregressive verification.Source: §2 · pp. 3–4

3 Background

A masked diffusion language model predicts a vocabulary distribution at every masked position in one forward pass. A decoding round selects positions, assigns tokens, and commits them; sequential and parallel decoding differ in whether later predictions condition on earlier commits.

3 Background

  • A masked diffusion language model produces a vocabulary distribution at every masked position in a single forward pass, unlike an autoregressive model that predicts only the next token.The canvas is a partially masked token sequence, and its masked positions form the set eligible for decoding.Source: §3 · p. 4
  • A decoding round evaluates the canvas, selects a subset of masked positions, assigns each a sampled or argmax token, and updates the canvas with those commits.This work focuses on greedy argmax decoding.Source: §3 · p. 4
  • Sequential decoding conditions later predictions on earlier commits and requires |S| evaluations, while parallel decoding commits all selected argmax predictions from one evaluation; their disagreement is measured by Hamming distance.The selected positions are treated as an ordered sequence for the sequential reference.Source: §3 · p. 4

4 Methodology

PIG treats disagreement between parallel and sequential decoding as regret and selects parallel commit sets under regret constraints. Its two algorithms control regret within a round or the cost of waiting, using a lightweight predictor derived from one DLM pass.

4.1 Parallel decoding as regret-constrained subset selection

  • PIG.INT scans positions in decreasing confidence order and commits the longest prefix whose predicted single-round regret stays within budget φ.The exact regret objective is impractical because it requires combinatorial search and sequential replay; PIG approximates it from confidence and estimated perturbations.Source: Figure 2 · p. 5, §4.1 · pp. 5–6

4.2 Look-ahead decoding as cross-round regret

  • PIG.EXT estimates whether delaying commits would reduce regret on the next candidate window, closing the round when the predicted value of waiting exceeds θ.The look-ahead uses a window of 16 positions.Source: Figure 3 · p. 6, §4.2 · p. 6

4.3 A lightweight adapter enables one-pass regret prediction

  • A lightweight adapter predicts directed entropy finite differences from the current DLM pass, supporting both regret estimators without additional backbone evaluations at inference.The adapter has approximately 2M parameters, is fitted once per backbone on a mixture of 3,721 examples, and costs less than 0.1% of a backbone forward pass in MACs.Source: Figure 4 · p. 7, §4.3 · pp. 7–8

5 Experiments

Across the tested datasets and block sizes, PIG improves the accuracy–TPF frontier over confidence-based decoding, with the largest gains as decoding becomes more parallel. Experiments compare two models and four math and code benchmarks, with additional comparisons and audits extending the evaluation.

5 Experiments

  • Experiments evaluate LLaDA-8B-Instruct and Nemotron-Labs-Diffusion-3B on GSM8K, MATH-500, MBPP, and HumanEval, using each model’s confidence decoder as the primary baseline.LLaDA uses a 256-token canvas and benchmark-specific few-shot settings; Nemotron uses a 2048-token canvas and zero-shot evaluation.Source: §5 · p. 8

PIG improves the accuracy–TPF frontier

  • PIG consistently improves the accuracy–TPF frontier over Fast-dLLM, with the largest gains as decoding becomes more parallel.PIG.INT performs best at the slower end of the frontier, while PIG.EXT performs best at the faster end; the reported gains are strongest at medium block sizes.Source: PIG improves the accuracy–TPF frontier · p. 9
  • At matched accuracy, PIG.INT reaches 1.5 to 1.8× the TPF of DAPD-Direct on three benchmarks and 0.8× on MBPP, while remaining at equal or higher accuracy; against reported EB-Sampler and KLASS points, it reaches 2.1 to 4.4× TPF.The EB-Sampler and KLASS comparisons use published operating points with different prompts and graders.Source: PIG improves the accuracy–TPF frontier · pp. 9–10
  • Additional experiments report that dynamic blocks extend PIG.INT’s reach, MMLU results are consistent with the main findings, and live audits show PIG.INT regret below φ while PIG.EXT overpredicts at large θ.Dynamic blocks extend PIG.INT from 9.6 to 12.0 TPF on GSM8K and from 7.6 to 9.7 TPF on MATH-500.Source: PIG improves the accuracy–TPF frontier · p. 10

6 Discussion, Future Work, and Conclusion

PIG treats disagreement with sequential decoding as a direct target for parallel commits, and the authors report improved accuracy–compute frontiers over confidence-based decoding. They identify limits in the reference policy, training-data dependence, and choosing between PIG variants.

6 Discussion, Future Work, and Conclusion

  • PIG directly targets disagreement with sequential decoding, and the authors suggest richer predictors could further improve performance.They describe this as a complementary design principle for parallel decoders.Source: §6 · p. 11
  • The reference trajectory is greedy sequential decoding, and the predictor and calibration still depend on training-side data quality.Other sequential policies could yield different regret notions and preferred parallel sets; transfer across tasks does not remove dependence on training data.Source: §6 · p. 11
  • The authors lack a principled rule for choosing PIG.INT versus PIG.EXT on a new model or task.Future work proposes a unified decoder that decides when to control current-round regret and when to use PIG.EXT’s look-ahead criterion, and suggests jointly learning the estimator and selection policy.Source: §6 · p. 11
  • Across two DLM families and standard math and code benchmarks, PIG.INT and PIG.EXT improve the accuracy–compute frontier over confidence-based parallel decoding.The conclusion describes selecting which masked positions can safely be committed together by predicting sequential–parallel regret from a single model pass.Source: §6 · p. 11

A Theoretical Motivation and Analysis

The theoretical analysis formulates decode-order choice through expected loss and its deployable entropy form, then approximates interactions with a first-order graph representation. Exact ordering on that graph is hard, motivating confidence-based ordering with PIG controlling admission.

A Theoretical Motivation and Analysis

  • The analysis derives per-pass disagreement and per-accelerated-token regret guarantees for PIG.INT and PIG.EXT, respectively, under one calibration assumption on the inconsistency estimator.It also motivates the one-pass representation and explains why PIG fixes token order by confidence and spends its budget on admission.Source: A Theoretical Motivation and Analysis · p. 16

A.1 The decode-ordering objective and its entropy form

  • Training averages over reveal orders, while decoding executes one; for a fixed order, cumulative conditional errors determine its expected teacher-forced loss.Order selection at a fixed model is combinatorial because each token’s term depends on its full revealed set.Source: §A.1 · p. 16
  • At deployment, PIG uses the model’s expected per-step loss, or entropy, as a substitute for teacher-forced readouts and measures the resulting decode entropy.The entropy objective selects the trajectory the model itself prices cheapest.Source: §A.1 · p. 17

A.2 The M¨obius expansion and the graph representation

  • The Möbius expansion decomposes each token’s context-dependent readout into subset interactions; truncating at singletons yields a first-order approximation.Its first-order coefficients are finite differences: the gain in a token’s readout from revealing another token given the existing context.Source: §A.2 · p. 17
  • At each round, the adapter predicts directed entropy finite differences from one pass; PIG.INT aggregates their magnitudes, while PIG.EXT uses their signs to predict next-round entropies.Higher-order interactions are not visible in that one-pass representation, and calibration maps the proxy to a mismatch probability.Source: §A.2 · pp. 17–18

A.3 The first-order problem is a linear ordering problem, and it is NP hard

  • Maximizing the first-order objective is a linear ordering problem that is NP-hard, while omitted higher-order interactions make even an exact solution optimize a partly speculative objective.The paper therefore keeps the confidence order and uses PIG’s control to decide how many tokens to admit in a parallel pass.Source: §A.3 · p. 18

A.4 Guarantees for PIG.INT and PIG.EXT

The guarantees formalize parallel–sequential disagreement against sequential references anchored to the parallel decoder’s reached canvas. PIG.INT bounds within-round inconsistency, while PIG.EXT bounds the additional inconsistency associated with deferring commits.

A.4 Guarantees for PIG.INT and PIG.EXT

  • The reference is re-anchored each round to the parallel decoder’s reached canvas, so total inconsistency counts fresh round disagreements rather than whole-output Hamming distance.Both parallel and sequential trajectories condition on the previous round’s canvas, including any earlier divergence.Source: §A.4 · p. 19
  • With a calibrated estimator, PIG.INT bounds expected whole-decode piece-wise inconsistency by φ · NFE + ϵ̄ · N.The per-round admission rule enforces a φ budget; summing round bounds yields the whole-decode guarantee.Source: §A.4 · p. 20
  • When ϵ̄ ≈ 0, φ is an exchange rate of at most φ expected disagreements per forward pass, uniformly over the decode.The total number of forward evaluations is inversely related to φ.Source: §A.4 · p. 20
  • For PIG.EXT, widening a round instead of granting the window another forward pass incurs a bounded marginal inconsistency under calibrated current and deferred estimates.The lemma’s stated cost is bounded by |N(S)| (2ϵ̄ + δ̄).Source: §A.4 · p. 21
  • PIG.EXT bounds whole-decode piece-wise deferred inconsistency using a cascade floor and the θ budget, with calibrated-estimator error terms.Its guarantee assumes nonnegative per-token deferral terms and uses an (ϵ̄, δ̄)-calibrated estimator.Source: §A.4 · pp. 21–22

B The Entropy Finite Differences Estimator: Implementation and Training

The estimator reconstructs directed entropy changes between masked positions from frozen-backbone features, enabling decoder statistics from one existing forward pass. Its training uses sampled exact labels from training-split states, with a stated boundary at fully masked initial canvases.

B.1 The finite-difference estimator

  • The adapter predicts directed entropy finite differences, whose aggregate coupling and local-curvature statistics are computed without extra model evaluations.A frozen-model forward pass with one source committed measures an exact finite-difference column; the estimator predicts the matrix from the decoder’s existing pass.Source: §B.1 · p. 23
  • The adapter has about 2.2M parameters and combines per-position backbone features with transformer layers and a directed pairwise readout.The frozen backbone supplies hidden states, token ids, attention maps, and per-position scalars; separate receiver and source maps make the estimate directed.Source: §B.1 · p. 23
  • At L = 256, the estimator adds 12.8 ms per decoding round, below the 15 ms per-round budget and less than the replaced U-Net’s 17.9 ms.The reported production-decoder measurement on a shared H100 is an upper bound because the node was not reserved.Source: §B.1 · p. 24
  • Training uses 34,142 harvested states from 3,721 training-side problems across GSM8K, MATH, OpenCoder, and MBPP, with about 111M labelled directed edges.States are snapshotted every 24 commits under single-token and budgeted-set decoding policies; no benchmark test set enters the harvest.Source: §B.1 · p. 24
  • The fully masked initial canvas is absent from training harvests, so the estimator extrapolates to t = 0 through its time conditioning.Held-out evaluation includes t = 0 states, where the estimator tracks gold local curvature; adding such training snapshots is noted as an unmade change.Source: §B.1 · p. 24
  • The shipped LLaDA checkpoint uses Huber regression on labelled edges only and reaches validation per-edge correlation 0.827 at epoch 7.The loss supervises only planted-source columns; the Nemotron twin uses the same recipe with model-specific changes and reaches validation edge correlation 0.794.Source: §B.1 · p. 25

B.2 The calibrated mismatch table

The calibrated mismatch tables convert estimated coupling and confidence or entropy into empirical mismatch rates for decoding. The accompanying algorithms use these offline-fitted objects in alternative selection rules, while the appendix points readers to further analyses and complete operating-point grids.

B.2 The calibrated mismatch table

  • The tables are empirical mismatch-rate lookups indexed by aggregate coupling and confidence for PIG.INT, or entropy for PIG.EXT.Cells with fewer than 50 commits fall back to the pooled rate of their row; each backbone uses its own table because coupling scales and mismatch levels differ.Source: §B.2 · pp. 25–26
  • In the decision region ν ≥ 0.6, GSM8K-train and OpenCoder fits differ by at most 0.024 on LLaDA and 0.028 on Nemotron, motivating the pooled table.Regime transfer from fitted canvas or window geometries to other reported block geometries is assumed rather than measured.Source: §B.2 · p. 26
  • The entropy-indexed table supports PIG.EXT lookahead by pricing candidates at current and predicted post-commit entropy; LLaDA uses a separate probe of 395,854 commits.Nemotron’s entropy table uses the same 601,730 commits as its confidence table; the LLaDA confidence probe did not record entropy.Source: §B.2 · pp. 26–27, Figure 10 · p. 32

C Algorithms

  • The algorithms share an outer loop and one-pass quantities, but PIG.INT bounds within-round regret while PIG.EXT bounds avoidable cross-round regret.Both are alternatives, not stages; their adapter and calibration tables are fitted offline once per backbone.Source: C Algorithms · pp. 33–34

D Additional Experiments and Analyses

  • The appendix lists additional analyses—including dynamic blocks, MMLU transfer, regret-estimate audits, and comparisons with dependency-guided and learned policies—and locates full grids in Section E.The appendix states that Section E gives the protocol for the appendix lanes.Source: D Additional Experiments and Analyses · p. 37

D.1 Dynamic block decoding unlocks additional parallelism

Dynamic blocks expand the candidate pool to address candidate starvation, improving the accuracy–TPF frontier, especially for PIG. PIG’s decoding width also varies with information curvature, taking larger steps in flatter regions and smaller steps as curvature increases.

D.1 Dynamic block decoding unlocks additional parallelism

  • Dynamic blocks maintain B masked candidates when possible, addressing candidate starvation by expanding eligibility while leaving each decoder’s selection rule unchanged.Dynamic blocks also improve Fast-dLLM, while larger pools particularly help PIG exploit additional low-regret candidates.Source: §D.1 · pp. 37–38
  • On GSM8K, dynamic blocks extend PIG.INT from 9.6 to 12.0 TPF, and on MATH-500 from 7.6 to 9.7 TPF; at 10 and 9 TPF, accuracy is 0.71 and 0.28.At those throughputs, PIG.INT is 17 points above its fixed-block extrapolation and 15–16 points above dynamic-block Fast-dLLM; PIG.EXT gains 3–5 points at every throughput listed.Source: Figure 11 · p. 37, §D.1 · p. 38, Table 3 · p. 38

D.2 PIG adapts to the information curvature of decoding

  • PIG.INT and PIG.EXT vary their parallel set size in line with maintaining constant curvature, taking larger steps in flatter regions and smaller steps as curvature rises.PIG.EXT’s cross-round regret directly measures how another model evaluation would change upcoming candidates; confidence-based decoding tracks curvature less closely.Source: §D.2 · pp. 38, 40, Figure 12 · p. 39

D.3 Domain transfer: PIG carries over to MMLU without refitting

On MMLU, PIG’s frontier retains its shape without refitting the estimator or tables trained on mathematics and code. The evidence is limited to one backbone and a 500-question sample, with all lanes below 4 TPF.

D.3 Domain transfer: PIG carries over to MMLU without refitting

  • The MMLU test uses 500 questions stratified across all 57 subjects, and reuses the estimator checkpoint and tables fitted on mathematics and code without refitting or retuning.Nemotron-3B-Instruct is evaluated zero-shot with fixed blocks of 16, 32, and 64 tokens.Source: §D.3 · p. 40
  • At BL=32, PIG.EXT reaches 0.584 at 3.09 TPF, faster and more accurate than Fast-dLLM’s fastest point; at BL=64 it matches Fast-dLLM’s fastest accuracy at 1.4× its throughput.At BL=32 and 64, PIG.INT is more accurate than Fast-dLLM at matched TPF on every cell within Fast-dLLM’s range; at BL=16, the lanes stay within each other’s intervals.Source: §D.3 · p. 41, Figure 13 · p. 41
  • The transfer evidence covers one benchmark on one backbone at n = 500, with Wilson half-widths near four points; every lane stays below 4 TPF.Within that scope, the three decoders preserve their math-and-code ordering without refitting.Source: §D.3 · pp. 41–42

D.4 The adapter is a calibrated predictor of regret

Audits on decoder trajectories find that PIG.INT’s predicted regret tracks realized regret and stays below its budget, while PIG.EXT’s cross-round estimate is generally conservative but underpredicts a low-probability tail. These findings are conditional on the audited settings and, for PIG.INT, on admitted tokens.

D.4 The adapter is a calibrated predictor of regret

  • For PIG.INT, the audit replays admitted tokens from each round’s pre-commit canvas and compares realized disagreement with the table value at the same confidence and coupling features.The decode is unchanged by the audit, and replay forwards are excluded from NFE; the audited population is 754,594 admitted tokens, not refused candidates.Source: §D.4 · p. 42
  • Across four LLaDA benchmarks, PIG.INT’s realized regret is 0.13–0.56 φ against predicted 0.41–0.66 φ on cells with at least 1,000 audited rounds, and the maximum realized regret is 0.91 φ.Realized over predicted regret is 0.7–1.1 on those cells, with the slowest MBPP cell an exception at 0.29.Source: Figure 14 · p. 42, §D.4 · p. 43
  • For PIG.EXT, the deferred estimate underpredicts 1–7% of window-token changes where it predicts less than 1%, while the current-round estimate remains accurate in that tail.Across the audited points, realized cross-round regret is 0.73–1.09 at the floored point, 0.85–1.57 at θ = 0.5, and 0.99–2.00 at θ = 1.5.Source: §D.4 · pp. 43–44, Figure 15 · p. 43
  • The PIG.EXT audit covers 32 problems per benchmark, with every sixth round audited; under the confidence floor, the audited window is N(S), not the rule’s nearly empty above-floor pool.The deferred estimate is too optimistic by a few percent in a tail, while other errors are in the safe direction.Source: §D.4 · p. 44

D.5 Comparison with dependency-guided decoders

At matched block geometries, PIG.INT and PIG.EXT compare favorably with dependency-guided and learned decoders on specified operating points, though some external comparisons are not paired. The appendix also records the frontier grids and operating-point tables behind the analyses.

D.5 Comparison with dependency-guided decoders

  • At BL=64, DAPD-Direct is faster than DAPD-Staged, while PIG.INT and PIG.EXT pass through or above DAPD-Direct on the math panels; DAPD’s reported advantage is confined to the single-block setting.The comparison uses the authors’ code reproduction under this protocol; the authors attribute a substantial part of the discrepancy from reported DAPD values to prompt protocol, which reshapes its attention graph.Source: §D.5 · pp. 45–46, Figure 16 · p. 47

D.6 A learned unmasking policy at matched block geometry

  • At BL=32, the published learned policy sits on the paper’s lanes in the reported 2–4.5 TPF band, while PIG’s 5–14 TPF frontier range is reported for that policy only at its α=10 end.The comparison matches block geometry, canvas, and NFE accounting, but differs in prompt and grader.Source: §D.6 · p. 47, Figure 17 · p. 48

D.7 One number per comparison: matched-accuracy throughput and accuracy at a fixed throughput

  • At 10 TPF on LLaDA, PIG.INT is the most accurate lane on every benchmark, 4 to 19 points above Fast-dLLM; at 15 TPF, PIG.EXT leads on GSM8K and MBPP by 10 to 14 points.On Nemotron code, PIG.EXT retains 36.9 against 9.1 points on MBPP at 5 TPF, and 44.5 against 19.5 on HumanEval at 4.9 TPF.Source: §D.7 · pp. 48–49

E Complete Frontier Grids and Tables

  • The appendix provides the complete frontier grids for LLaDA and Nemotron and tables listing every operating point with Wilson 95% intervals across four benchmarks per model.It also includes LLaDA strict-extraction grids alongside flexible-extraction grids.Source: E Complete Frontier Grids and Tables · p. 50

E.1 LLaDA-8B-Instruct

The LLaDA-8B-Instruct evaluation compares parallel decoding regimes on four math and code benchmarks using a protocol based on Fast-dLLM’s published setup.

E.1 LLaDA-8B-Instruct

  • The evaluation covers GSM8K, MATH-500, MBPP, and HumanEval with task-specific prompts and graders, including flexible and strict extraction for GSM8K and MATH-500.It uses 5/4/3/0 shots respectively, a 256-token canvas, greedy commits, and no KV cache; every accuracy includes a Wilson 95% interval.Source: §E.1 · p. 50
  • Five geometries are tested: fixed 32- and 64-token blocks, dynamic 32- and 64-candidate windows, and a free canvas.PIG.EXT uses confidence floors at the two slowest settings, then a plain θ ladder from 0.5.Source: §E.1 · p. 50
  • Fast-dLLM’s confidence-threshold rule is reproduced on each panel with matching eligible positions, prompts, canvas, and grader, while its threshold sweeps from 0.95 to 0.1.No KV cache is used, so TPF counts forwards over the full canvas for every method.Source: §E.1 · p. 51
  • Under strict extraction, GSM8K accuracy falls for every lane because the instruct model omits the #### format in roughly a third of generations, while lane ordering remains the same.Flexible extraction is therefore the main-text metric.Source: §E.1 · p. 51

E.2 Nemotron-Labs-Diffusion-3B

The Nemotron-Diffusion-3B evaluation uses the model’s zero-shot setup across benchmark tasks and several decoding geometries. Its appendix reports complete frontier grids and operating-point tables, with lane ordering consistent across the tested block sizes.

E.2 Nemotron-Labs-Diffusion-3B

  • Nemotron is evaluated zero-shot on GSM8K, MATH-500, MBPP, and HumanEval with a 2,048-token canvas, task-specific prompts, and the model’s chat template.TPF is mean generated tokens per forward and is not comparable across models because the canvases differ.Source: §E.2 · p. 51
  • At block size 32, three geometries are tested: fixed blocks, sliding windows, and commit bands nested in windows twice as wide; the baseline is the model’s confidence-threshold decoder swept from 0.9 to 0.3.Figures 21 and 22 repeat the grid at block sizes 16 and 64; the baseline there uses the same geometry and end-of-sequence guard.Source: §E.2 · p. 51, Figure 20 · p. 55, Figure 21 · p. 56
  • The lane ordering is the same at block sizes 16, 32, and 64; MATH-500 was not run at block size 64.At block size 16, PIG.EXT has no confidence-floored cells.Source: §E.2 · p. 52, Figure 20 · p. 55, Figure 21 · p. 56

E.3 Frontier tables

  • The appendix tables list every operating point with Wilson 95% intervals, including points cropped from the frontier figures.Tables 8–11 cover Nemotron’s GSM8K, MATH-500, MBPP, and HumanEval grids; PIG.EXT uses the composite operating rule.Source: §E.3 · p. 58, Table 8 · p. 63, Table 9 · p. 64

Related papers

Loading 2610.17375v1…