PaperlayerView exact revision on arXiv

Source-linked AI summary

S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation

Ligong Han, Hao Wang, Han Gao, Kai Xu, Akash Srivastava

arXiv:2603.25702v2cs.CL

TL;DR

Few-step block-diffusion decoding struggles to preserve quality when confidence thresholds are aggressive, while existing sequence-level corrections add training or test-time cost. S2D2 uses the same pretrained model to draft in parallel and verify autoregressively, routing verification selectively; across three block-diffusion families, it improves the accuracy-speed tradeoff over strong confidence-threshold baselines.

  • Problem

    Few-step block-diffusion decoding can weaken sequence-level dependencies, while prior sequence-level solutions add training or inference overhead.

  • Method

    S2D2 uses standard block-diffusion decoding as a drafter and block-size-1 decoding of the same model as a selectively routed autoregressive verifier.

  • Results

    Across three mainstream block-diffusion families, S2D2 improves the accuracy-speed tradeoff and often achieves higher accuracy with lower latency than dynamic confidence-threshold baselines.

  • Takeaways & Limitations

    S2D2 provides a plug-and-play, training-free inference-time acceleration strategy for existing block-diffusion models.

  • Takeaways & Limitations

    S2D2 is not identical to block-size-1 decoding because drafting and cache updates need not be fully causal under the original block-diffusion attention pattern.

Abstract

from arXiv · show

Block-diffusion language models offer a promising path toward faster-than-autoregressive generation by combining block-wise autoregressive decoding with within-block parallel denoising. However, in the few-step regime needed for practical acceleration, standard confidence-thresholded decoding is often brittle: aggressive thresholds hurt quality, while conservative thresholds require unnecessary denoising steps. Existing approaches that address this issue either require additional training or incur extra test-time compute. We present S2D2, a training-free self-speculative decoding framework for block-diffusion language models. Our key observation is that a block-diffusion model becomes autoregressive when the block size is reduced to one, allowing the same pretrained model to act as both drafter and verifier. S2D2 inserts a speculative verification step into standard block-diffusion decoding and uses lightweight routing policies to decide when verification is worth its cost. This yields a hybrid decoding trajectory in which diffusion proposes tokens in parallel, while the autoregressive mode acts as a local sequence-level critic. Across three mainstream block-diffusion families, S2D2 consistently improves the accuracy-speed tradeoff over strong confidence-thresholding baselines. On SDAR, we observe up to $4.7\times$ speedup over autoregressive decoding, and up to $1.57\times$ over a tuned dynamic decoding baseline while improving accuracy by up to $4.5$ points. On LLaDA2.1-Mini, S2D2 remains complementary to built-in self-correction, including a conservative setting where it is $4.4\times$ faster than the static baseline with slightly higher accuracy.

1 Introduction

S2D2 addresses brittle few-step block-diffusion decoding by using the same pretrained model for parallel drafting and autoregressive verification, without extra training. Across block-diffusion families, it improves the accuracy-speed tradeoff over dynamic confidence-threshold baselines.

  • Few-step block-diffusion decoding weakens sequence-level dependencies and can accumulate errors as denoising steps decrease.
  • S2D2 reuses block-size-1 decoding of the same model as an autoregressive verifier for standard block-diffusion drafts.This enables self-speculation without distillation, auxiliary models, or architectural changes.
  • Lightweight routing policies invoke verification only when its expected benefit justifies an extra forward pass.
  • Verification acts as a local sequence-level critic rather than exactly reproducing block-size-1 autoregressive decoding.
  • S2D2 often improves accuracy and speed over strong dynamic confidence-threshold baselines across three mainstream block-diffusion families.

2 Related Work

Prior diffusion-language-model acceleration methods combine block-wise autoregression, parallel diffusion, or speculative verification in different ways. S2D2 focuses on training-free inference-time acceleration by reusing an existing model’s autoregressive mode for verification.

  • Block-diffusion hybrids combine block-wise autoregressive generation, within-block diffusion, and KV caching to make few-step decoding practical.
  • Earlier speculative methods use weakened same-model drafting, arbitrary-subset verification, or hierarchical batching over multiple prefix states.
  • S2D2 reuses the existing block-size-1 autoregressive mode for single-pass verification without additional training.
  • LLaDA2.1 supports token editing through an “unmask early, correct later” strategy but does not perform verifier-based sequence-level acceptance.
  • S2D2 differs from EDLM and density-ratio methods by avoiding additional modeling or extra multi-sample inference.

3 Background

Block-wise autoregressive diffusion decodes masked blocks sequentially while reusing finalized-block KV caches, and within each block applies diffusion-style masked updates. Practical LLaDA decoding instead uses confidence-based proposal acceptance, while speculative decoding verifies drafted tokens autoregressively.

  • Block-wise autoregressive diffusion initializes a masked block, decodes it conditionally, and reuses finalized blocks’ KV cache.
  • At diffusion step t, M_t denotes positions that remain masked in the current block.
  • Masked absorbing-state diffusion defines reverse transitions from timestep t to an earlier timestep s.
  • LLaDA’s few-step implementation proposes tokens and confidences from logits, then accepts masked positions using a fixed schedule or dynamic threshold.
  • Autoregressive speculative decoding lets a drafter propose multiple tokens and a verifier scan them left-to-right using acceptance probabilities based on draft and verifier probabilities.
  • Rejected drafted tokens are resampled from the residual distribution, preserving the target autoregressive distribution while often accepting multiple tokens per verifier pass.

4 Method

S2D2 reuses a block-diffusion model as both parallel drafter and block-size-1 autoregressive verifier, applying selective speculative correction within a hybrid decoding trajectory. Lightweight routing determines when verification is worthwhile, while acceptance and residual resampling use verifier probabilities to correct draft tokens.

  • Self-Speculative Decoding: S2D2 uses standard block-diffusion decoding as the drafter and block-size-1 decoding of the same model as an autoregressive verifier.This avoids auxiliary models, retraining, distillation, and architectural changes.
  • Self-Verification: At each denoising step, S2D2 optionally verifies the first contiguous masked span using left-to-right speculative acceptance with probability min(1, q_i/p_i).The first rejection triggers residual resampling and terminates that speculative segment.
  • Verifier Masks: For position-aligned models, the 2L trick computes verifier confidences for all drafted positions in one forward pass, while right-shifted models use the standard causal mask.The verifier conditions on drafted tokens to the left while keeping each evaluated position masked.
  • Scope and Interpretation: S2D2 verifies only the first contiguous masked span and is not equivalent to globally autoregressive decoding because drafting and cache updates remain generally block-diffusion-based.The resulting trajectory is hybrid, and verification may stop at the first rejection or be skipped by routing.
  • Verification Routing: Lightweight routing invokes verification only when its expected accepted-token gain can offset one extra verifier forward pass.Policies include minimum-span, score-threshold, hysteresis, and contextual-bandit routing; scores can use static or confidence-aware costs.
  • AR-Guided Correction: Unlike EDLM’s global multi-sample reweighting, S2D2 performs online local correction through speculative acceptance and residual resampling, invoking verification selectively for acceleration.Lower-residual-energy draft tokens are more likely to be accepted, while higher-energy mismatches receive residual correction.

5 Experiments

Experiments across SDAR, Fast-dLLM v2, and LLaDA2.1 evaluate S2D2’s accuracy–speed tradeoffs against diffusion, autoregressive, and dynamic decoding baselines. S2D2 generally improves the frontier, including large-block settings and LLaDA’s conservative self-correction regime.

  • Experimental setup: S2D2 evaluates five models from three block-diffusion families on reasoning, code-generation, and instruction-following benchmarks.The evaluated families are SDAR, Fast-dLLM v2, and LLaDA2.1; tasks include GSM8K, MBPP, HumanEval, and IFEval.
  • SDAR: 4.7× speedup over AR and 1.57× over dynamic decoding accompany a 4.5-point average-accuracy gain for SDAR-1.7B config-B.The reported accuracies are 52.9 versus 48.4, with speedups measured relative to AR and dynamic decoding respectively.
  • Fast-dLLM v2: 1.07× faster than dynamic decoding and a +4.5-point average-accuracy gain are reported for Fast-dLLM v2 config-C at SB = 32.S2D2 recovers much of the large-sub-block accuracy drop while adding speedup.
  • LLaDA2.1: 4.4× faster than the static baseline with slightly higher accuracy, S2D2 remains complementary to LLaDA2.1-Mini’s built-in self-correction.In the conservative setting, S2D2 reaches 2.2× speedup and 79.3% average accuracy versus 1.7× and 78.7% for diffusion.
  • Analysis: AR-ness and confidence trajectories are task-dependent: SDAR is more AR-like on MBPP, while LLaDA 2.1 is more AR-like on GSM8K.SDAR confidence typically rises during decoding, whereas LLaDA confidence often starts high and drops near the end.
  • Analysis: S2D2 generally maintains a better accuracy–speed frontier and flatter inference-time scaling than standard diffusion, especially at larger SDAR block sizes.Figure 3 varies block sizes, denoising steps, schedules, and mask-span settings.

6 Conclusion

The conclusion presents S2D2 as a training-free, plug-and-play hybrid that reuses one block-diffusion model for parallel drafting and autoregressive verification. Across block-diffusion families, it improves the accuracy–speed tradeoff and provides a local sequence-level correction mechanism.

  • Conclusion: S2D2 reuses one pretrained model as both a standard block-diffusion drafter and a block-size-1 autoregressive verifier.The design is training-free and plug-and-play across multiple block-diffusion families.
  • Conclusion: S2D2 often delivers higher accuracy and lower latency than strong dynamic confidence-thresholding baselines across multiple block-diffusion families.The conclusion characterizes speculative verification as a local sequence-level correction mechanism.
  • Conclusion: The analysis frames speculative verification as a stochastic, greedy form of autoregressive energy correction.This perspective is offered as motivation for further training-free inference-time methods for diffusion language models.

A.1 Algorithm details

The decoding pipeline uses a shared block-wise autoregressive outer loop, with standard diffusion or S2D2 providing the block sampler. S2D2 adds verifier-based acceptance and routing while retaining block-level cache reuse.

  • Standard diffusion: Algorithm 2 implements dynamic confidence-thresholded diffusion decoding, with static decoding recovered by setting τ = 1.Each diffusion step performs a draft forward pass, samples logits, and selects tokens using confidence.
  • S2D2 sampler: Algorithm 3 adds verifier-based acceptance to the same block framework while preserving the shared outer loop.The S2D2 sampler performs draft inference, optionally invokes verification, then returns to diffusion when verification is skipped or rejected.
  • Routing: Algorithm 4 routes verification using minimum-span, score-threshold, or hysteresis policies.These policies decide whether the extra verifier forward pass is worthwhile.
  • Shared decoding loop: Algorithm 1 initializes masked blocks, samples them, appends generated tokens, updates KV caches, and repeats until generation finishes.The loop receives the prompt, model, block size, and block sampler, then returns the generated sequence.

A.2 From MDLM posterior sampling to LLaDA confidence-based decoding

The appendix connects discrete-diffusion posterior sampling with confidence-based token visibility and presents S2D2’s verifier-aware block sampler. The sampler verifies a contiguous masked span and accepts or resamples tokens using verifier-to-draft score ratios.

  • Posterior sampling: MDLM’s reverse transition for absorbing-state discrete diffusion is introduced under the SUBS parameterization.The subsequent interpretation treats each masked position as deciding whether it remains masked or becomes visible.
  • Posterior sampling: The reverse process can be viewed as first sampling visibility and then sampling an unmasked token from the model prediction.Exact reverse sampling therefore factorizes across masked positions.
  • S2D2 sampling: S2D2’s block sampler invokes DOVERIFY before verifier scoring and token acceptance.When verification is not invoked, it falls back to confidence-thresholded token selection.
  • Speculative verification: Verifier acceptance proceeds left-to-right through the first contiguous masked span using min(1, q_i/p_i), with residual resampling after rejection.Here p_i denotes draft scores and q_i verifier scores as used by the acceptance ratio.
  • Attention mask: The optional autoregressive attention mask applies causal structure to the committed prefix while retaining separate block dimensions.The four dimensions are j × j, j × (B − j), (B − j) × j, and (B − j) × (B − j).

Hysteresis Policy

Hysteresis routing maintains an ON/OFF verification state, switching according to a score and separate activation and deactivation thresholds. Confidence-based decoding instead selects positions for unmasking using fixed schedules or dynamic thresholds.

  • Hysteresis Policy: Hysteresis routing turns verification off when its score falls below τoff and back on when the score reaches τon.The policy preserves the current state unless one of these threshold conditions triggers a switch.
  • Confidence-Based Decoding: LLaDA decoding selects positions to unmask after a draft forward pass using token confidence.Selection can follow either a fixed schedule or a dynamic threshold.
  • Confidence-Based Decoding: Confidence-based selection replaces independent Bernoulli reveal decisions with confidence-prioritized selection among positions.This introduces dependence among positions through competitive reveal decisions.

A.3 Connection to residual energy

S2D2 shares an AR-versus-diffusion discrepancy with residual-energy approaches but applies it online through speculative acceptance and residual resampling. Its broader experiments show accuracy–efficiency gains, while estimator choice and verifier reliability define important practical boundaries.

  • Connection to residual energy: The verifier-to-draft discrepancy is expressed through draft probability pi and verifier probability qi for each drafted token.These probabilities define the local comparison used by speculative decoding.
  • Connection to residual energy: S2D2 uses the AR-versus-diffusion discrepancy online through speculative acceptance and residual resampling, unlike EDLM’s global importance reweighting.EDLM fits an AR-parameterized residual model, whereas S2D2 reuses the block-size-1 AR mode as a verifier.
  • Connection to residual energy: S2D2’s stochastic greedy interpretation accepts lower-residual-energy drafted tokens more readily and rejects higher-energy mismatches for correction.The interpretation is heuristic rather than exact energy minimization.
  • Results: S2D2 improves the accuracy–speed frontier across block-diffusion families, with large-batch SDAR throughput gains more modest because the 2L verifier increases workload.The MBPP accuracy improvement remains large across batch sizes.
  • Boundary Conditions: A weak block-size-1 AR mode can reduce accuracy through residual resampling, so S2D2 should be used cautiously when that conditional is unreliable.Allowing diffusion to re-denoise rejected positions largely recovers the diffusion result in the cited D2F LLaDA variant.
  • Estimator Ablation: Entropy-based acceptance estimation yields higher downstream accuracy than margin-based estimation despite lower prefix-length prediction accuracy.The main experiments therefore use the soft entropy-based estimator.
  • Ratio Tempering: A ratio-tempering value of γ = 1.25 can slightly improve accuracy in some settings but typically causes minor speed degradation.The default γ = 1 is retained as the overall tradeoff.
Loading 2603.25702v2…