PaperlayerView exact revision on arXiv

Source-linked AI summary

DEER: Draft with Diffusion, Verify with Autoregressive Models

Zicong Cheng, Guo-Wei Yang, Jia Li, Zhijie Deng, Meng-Hao Guo, Shi-Min Hu

arXiv:2512.15176v1cs.LGcs.AI

TL;DR

Autoregressive drafters constrain speculative decoding through sequential generation and accumulating uncertainty, while naïvely using diffusion drafters creates distribution mismatch. DEER aligns a discrete diffusion drafter to an autoregressive target and drafts blocks in one step; it reports up to 32-token acceptance lengths and a 5.54× HumanEval speedup versus EAGLE-3’s 2.41×.

  • Problem

    Autoregressive speculative drafters decode sequentially and accumulate uncertainty from unverified tokens, limiting accepted draft lengths and speedups.

    Source: §1 · p. 1, §3 · p. 3

  • Method

    DEER uses a discrete-space dLLM as its drafter, aligns it to the target AR model in two stages, and generates draft blocks in one step.

    Source: §1 · pp. 1–2

  • Results

    Up to 32-token acceptance lengths and a 5.54× HumanEval speedup for Qwen3-30B-A3B, versus EAGLE-3’s 2.41× speedup.

    Source: §1 · p. 2

  • Takeaways & Limitations

    DEER demonstrates dLLMs as a practical, highly parallelizable alternative for efficient decoding, with longer accepted blocks and speedups across multiple code-generation benchmarks and model scales.

    Source: §5 · p. 9

Abstract

from arXiv · show

Efficiency, as a critical practical challenge for LLM-driven agentic and reasoning systems, is increasingly constrained by the inherent latency of autoregressive (AR) decoding. Speculative decoding mitigates this cost through a draft-verify scheme, yet existing approaches rely on AR draft models (a.k.a., drafters), which introduce two fundamental issues: (1) step-wise uncertainty accumulation leads to a progressive collapse of trust between the target model and the drafter, and (2) inherently sequential decoding of AR drafters. Together, these factors cause limited speedups. In this paper, we show that a diffusion large language model (dLLM) drafters can naturally overcome these issues through its fundamentally different probabilistic modeling and efficient parallel decoding strategy. Building on this insight, we introduce DEER, an efficient speculative decoding framework that drafts with diffusion and verifies with AR models. To enable high-quality drafting, DEER employs a two-stage training pipeline to align the dLLM-based drafters with the target AR model, and further adopts single-step decoding to generate long draft segments. Experiments show DEER reaches draft acceptance lengths of up to 32 tokens, far surpassing the 10 tokens achieved by EAGLE-3. Moreover, on HumanEval with Qwen3-30B-A3B, DEER attains a 5.54x speedup, while EAGLE-3 achieves only 2.41x. Code, model, demo, etc, will be available at https://czc726.github.io/DEER/

1. Introduction

Autoregressive drafters limit speculative-decoding speedups through sequential generation and accumulating uncertainty, while naïvely applying diffusion drafters creates a distribution mismatch. DEER uses a discrete diffusion drafter, a two-stage alignment pipeline, and one-step block drafting; the introduction reports longer accepted drafts and faster decoding than EAGLE-3.

1. Introduction

  • AR drafters accumulate uncertainty through left-to-right generation and must decode sequentially, limiting accepted drafts and attainable speedups.Early discrepancies can be recursively amplified when drafts condition on unverified outputs.Source: §1 · p. 1
  • Up to 32-token acceptance lengths, versus 10 typical, and 5.54× HumanEval speedup, versus EAGLE-3’s 2.41×, headline DEER’s reported gains.The HumanEval speedup comparison is for Qwen3-30B-A3B.Source: §1 · p. 2
  • DEER uses a discrete-space dLLM as its sole drafter, with one-step block drafting to avoid left-to-right dependency.Naïve dLLM drafting is misaligned because standard dLLMs are trained for global generation rather than prefix-conditioned continuation.Source: §1 · p. 2
  • Reliable block regeneration is the dLLM’s ability to repeatedly accept partially masked suffixes and regenerate them coherently.The introduction presents this as an emergent capability of the training pipeline.Source: §1 · p. 2
  • The two-stage alignment pipeline uses AR-style continuation distillation followed by exponentially weighted suffix masking to improve prefix-conditioned continuation fidelity.Stage 1 aligns to the target AR distribution; Stage 2 refines token-level stability near the verification boundary.Source: §1 · p. 2

2. Related Work

Related work contrasts DEER with sequential AR and heuristic drafters, other diffusion-based drafting approaches, and hybrid or self-drafting designs. It also situates DEER within the development of diffusion language models that generate multiple tokens in parallel.

2.1. Speculative Decoding

  • AR and tree-based methods draft left-to-right, while n-gram and retrieval methods lack global context; DEER instead drafts full blocks with diffusion [Leviathan et al., 2023; Cai et al., 2024; Ankner et al., 2024; Li et al., 2024a;b; 2025b; Fu et al., 2024; Li et al., 2025a].The section says early uncertainty can corrupt sequential draft chains and that heuristic methods can lose acceptance on complex sequences.Source: §2.1 · p. 2

2.2. Self-Drafting and Hybrid Architectures

2.3. Diffusion language models

  • Unlike hybrid AR-diffusion training, which can be expensive and conflict with the base objective [Liu et al., 2025], DEER uses a dedicated dLLM; diffusion LMs can generate multiple tokens in parallel [Austin et al., 2021a; Li et al., 2022; Nie et al., 2025; Ye et al., 2025].The section says DEER’s modular design avoids retraining the target LLM and preserves its original capabilities.Source: §2.2 · p. 3, §2.3 · p. 3

3. DEER

DEER replaces the autoregressive drafter with a diffusion model, aligning it for prefix-conditioned continuation before blockwise draft-and-verify decoding. Its training stages address continuation mismatch and improve accuracy near the verification boundary.

3.1. Diffusion-to-AR Alignment

  • A two-stage Diffusion-to-AR Alignment pipeline adapts pretrained dLLMs to prefix-conditioned continuation, which they do not inherently model.Directly using a pretrained dLLM as a drafter causes distribution mismatch and unstable suffix predictions.Source: §3 · p. 3, §3.1 · p. 3, §3.1.1 · p. 3

3.1.1. Stage I: AR-style Distillation

  • Stage I trains on randomly truncated teacher answers with masked suffixes and a SEP boundary, aligning dLLM continuation behavior with the AR teacher.The dLLM denoises the masked continuation, learning to predict the future from an observed prefix.Source: §3.1.1 · p. 3

3.1.2. Stage II: Scribe Refinement

  • Stage II masks only the last R tokens and exponentially weights tokens nearer the prefix to refine accuracy where speculative acceptance is most sensitive.This masked-span curriculum concentrates model capacity near the region where verification first interacts with the draft, improving block acceptance reliability.Source: §3.1.2 · pp. 3–4

3.2. Inference

  • DEER uses a diffusion drafter to propose full token blocks, while an autoregressive model verifies or corrects them token by token.The dLLM's block-wise masked conditioning makes proposals independent of previously drafted tokens, avoiding the error accumulation that degrades AR drafts.Source: §3.2 · p. 4
  • AR drafts accumulate uncertainty as each token conditions on unverified predecessors, whereas dLLM block proposals do not amplify their own earlier mistakes.The inference loop accepts each draft token with a computed probability or replaces it with an AR sample, appending either choice to the prefix.Source: §3.2 · pp. 4–5

4. Experiment

Experiments evaluate DEER's decoding efficiency, acceptance patterns, training refinement, scaling, generative behavior, and mathematical reasoning. DEER outperforms comparison methods across code-generation settings and also accelerates mathematical benchmarks, including with a partially converged dLLM drafter.

4.1. Experimental Settings

  • The evaluation covers four code-generation benchmarks and three math-reasoning benchmarks, using Open-dLLM for code and a diffusion-converted Qwen2.5-0.5B-Instruct drafter for math.Comparisons include Medusa, Hydra, and EAGLE-3; reported efficiency measures are speedup ratio and average acceptance length.Source: §4.1 · pp. 5–6

4.2.1. Overall Efficiency

  • 5.03 accepted tokens for DEER versus 3.05 for EAGLE-3 on Qwen3-30B-A3B at temperature 0; on HumanEval, acceptance is 6.58 versus 3.21 and DEER reaches up to 2× EAGLE-3's speedup.Across model families, DEER improves average acceptance length by 50–120% relative to EAGLE-3, depending on model scale and task.Source: §4.2.1 · pp. 6–7

4.2.2. Acceptance Distribution Mechanism

  • 32 tokens is DEER's maximum accepted draft length, versus EAGLE-3's 7–8; drafts longer than 8 tokens exceed 15% probability across tested models.The long-draft probability rises from 15.8% for Qwen3-4B to 16.6% for Qwen3-30B-A3B.Source: §4.2.2 · p. 7, Table 4 · p. 7

4.3.1. Impact of Stage II

  • Stage II raises average accepted-token length on HumanEval from 5.38 to 6.58 and on LiveCodeBench from 3.87 to 5.03.The largest gains are 1.20 tokens on HumanEval and 1.16 on LiveCodeBench; the reported gap grows with benchmark difficulty.Source: §4.3.1 · p. 7

4.3.2. Hyperparameter Sensitivity

  • With α=1.01, Stage II optimization is stable; α=1.02 produces noisier curves with slight upward drift, while α=1.05 leads to early divergence.The authors report a relatively narrow stability window and improved suffix alignment within the stable regime, without extra data or extended training.Source: §4.3.2 · pp. 7–8, Figure 6 · p. 8

4.4. Batch Inference Scalability (RQ3)

  • 159.87 tokens/s for DEER versus 38.35 tokens/s for the autoregressive baseline at batch size 8, in runs without KV cache.DEER yields substantial speedups across tested batch sizes; modest gaps at smaller batch sizes are attributed mainly to fixed speculative-decoding overheads.Source: Table 5 · p. 7, §4.4 · p. 8

4.5. Generative Capabilities (RQ4)

  • DEER-trained dLLMs can regenerate and extend incomplete code blocks from a local prefix without a full-sentence prompt or architectural modifications.The illustrated tokens are produced at different denoising iterations.Source: Figure 5 · p. 7, §4.5 · p. 8

4.6. Performance on Mathematical Reasoning (RQ5)

  • Across three math datasets, DEER averages 2.12× speedup versus EAGLE-3's 1.91×, despite using a partially converged dLLM drafter.The draft model was trained for 40 epochs on UltraChat; standalone generations were not yet semantically reliable.Source: §4.6 · pp. 8–9, Table 6 · p. 8

5. Conclusion

DEER uses a discrete diffusion language model as its sole drafter, with a two-stage alignment pipeline for prefix-conditioned continuation. Across multiple code-generation benchmarks and model scales, it yields longer accepted blocks and consistent speedups, including without KV caching.

  • DEER uses a discrete dLLM as its sole drafter, avoiding the left-to-right uncertainty accumulation of autoregressive drafters.Source: §5 · p. 9
  • Its Diffusion-to-AR Alignment pipeline combines AR-style distillation with lightweight refinement near the prefix boundary.Source: §5 · p. 9
  • Across multiple code-generation benchmarks and model scales, DEER yields longer accepted blocks and consistent speedups, even without KV caching.The conclusion presents dLLMs as a practical, highly parallelizable alternative for efficient LLM decoding.Source: §5 · p. 9

A. Details of Experimental Settings

Experiments use eight 80 GB NVIDIA A100 GPUs. Code and math tasks use separate 0.5B-parameter diffusion drafters and training configurations.

  • Experiments were conducted on eight NVIDIA A100 GPUs, each with 80 GB of memory.Source: A. Details of Experimental Settings · p. 12
  • For code tasks, a modified open-dCoder checkpoint provides a 0.5B-parameter diffusion drafter trained for one epoch in each alignment stage.Stage I uses AdamW at 1 × 10^-4; Stage II uses AdamW at 5 × 10^-5 on 100k examples.Source: A. Details of Experimental Settings · p. 12
  • For math tasks, a 0.5B-parameter drafter initialized from Qwen2.5-0.5B is converted to a dLLM and trained on UltraChat for 40 epochs.Stage I uses AdamW at 1 × 10^-4 for five epochs, followed by one refinement epoch at the same learning rate.Source: A. Details of Experimental Settings · p. 12

B. More Sensitivity Analysis and Experiment

DEER's acceptance-length distribution shows a renewed rise in probability mass for long accepted blocks, beyond its approximately exponential decay at shorter lengths.

B.1. Details of Accept lenth

  • Beyond 30 tokens, acceptance probability rises again toward the maximum range, a pattern the authors call the long-block resurgence effect.Below 30 tokens, the distribution decays approximately exponentially, consistent with most speculative decoding methods.Source: §B.1 · p. 12
  • The authors argue that long-block resurgence supports their uncertainty-accumulation motivation: later draft tokens are less exposed to left-to-right error propagation when not conditioned on earlier draft tokens.Source: §B.1 · p. 12
  • Figure 7 plots token counts on the x-axis and frequency on a log-scale y-axis for Qwen3-8B, Qwen3-14B, and Qwen3-30B-A3B.Source: Figure 7 · p. 12

B.2. Sensitivity Analysis of block size

Average acceptance length increases with larger block sizes, with slower growth at higher sizes; larger Qwen3 backbones also show higher acceptance.

B.2. Sensitivity Analysis of block size

  • Qwen3-14B improves acceptance length by 5%–14% over Qwen3-8B, and Qwen3-30B-A3B delivers further gains of 14%–33% across block sizes 4 to 32.The authors attribute the gains to stronger output determinism in larger base models.Source: §B.2 · p. 12
  • Average acceptance length increases with block size, while growth gradually slows, indicating a trade-off with computation overhead.Source: §B.2 · p. 12
  • Figure 8 plots block size on the x-axis and average acceptance length on the y-axis for three Qwen3 models.Source: Figure 8 · p. 13
  • Table 7 compares drafter sizes between EAGLE-3 and DEER under matched target models; DEER uses a 0.5B-parameter discrete diffusion drafter for all targets.Source: Table 7 · p. 13, §B.2 · p. 13

C. Additional Analysis of Drafter Size

DEER and EAGLE-3 use drafters of comparable scale, yet DEER achieves longer average acceptance lengths and higher maximum accepted block sizes under matched-capacity regimes. The authors attribute these gains primarily to diffusion drafting and the alignment pipeline, rather than drafter size alone.

C. Additional Analysis of Drafter Size

  • DEER and EAGLE-3 drafters are of the same order of magnitude—hundreds of millions of parameters—and remain well below their target model sizes.The comparison covers matched target backbones.Source: C. Additional Analysis of Drafter Size · p. 13
  • For the 8B backbone, DEER uses a 470M drafter versus EAGLE-3’s 400M; for 14B, EAGLE-3 uses 610M versus DEER’s 470M.For the 30B backbone, DEER uses 470M versus EAGLE-3’s 140M.Source: C. Additional Analysis of Drafter Size · p. 13
  • Under matched-capacity regimes, DEER achieves longer average acceptance lengths and higher maximum accepted block sizes, which the authors attribute primarily to diffusion drafting and alignment.The authors state that gains do not primarily come from simply scaling up the drafter.Source: C. Additional Analysis of Drafter Size · p. 13

D. Training Cost Comparison

The section compares approximate fine-tuning costs for Medusa, Hydra, EAGLE-3, and DEER on Qwen3-8B, and attempts the same configurations on Qwen3-14B. In the reported setup, Medusa and Hydra run out of memory at 14B, so only EAGLE-3 and DEER have reported 14B training times.

D. Training Cost Comparison

  • On Qwen3-14B, Medusa and Hydra run out of memory under their default publicly released configurations in the authors’ setup, even after batch-size and sequence-length tuning.The section excludes them from 14B-scale efficiency comparisons.Source: D. Training Cost Comparison · pp. 13–14
  • On Qwen3-8B, all four methods train within a comparable range of GPU hours.The table reports approximate fine-tuning times for speculative decoding drafters on Qwen3 backbones.Source: Table 8 · p. 14
  • The comparison reports Qwen3-14B training times only for EAGLE-3 and DEER, while Medusa and Hydra are omitted after their out-of-memory errors.The measurements are approximate wall-clock GPU hours under the authors’ training setup and official or default configurations.Source: D. Training Cost Comparison · p. 14

E. Correctness Proof of the DEER Speculative Decoding Algorithm

The proof establishes that DEER’s draft–verify procedure preserves the target autoregressive model’s output distribution. It first proves a one-token result, applies it to DEER’s drafter, and extends it to finite output sequences; the section also notes a current batched-inference limitation tied to diffusion-model KV caching.

E. Correctness Proof of the DEER Speculative Decoding Algorithm

2. Define the acceptance probability

  • The one-step procedure accepts a draft token with its acceptance probability; otherwise it samples a replacement from the residual distribution, yielding the target token distribution.The final token has exactly the target distribution at that position.Source: §2 · p. 15

E.1. A One-Step Draft–Then–Verify Lemma

  • For any proposal distribution satisfying the support condition, the one-step draft–then–verify procedure is lossless.The stated condition is that the proposal never assigns zero probability where the target is positive.Source: §E.1 · p. 14, §2 · p. 15

E.2. Instantiating the Proposal with the DEER Drafter

  • For DEER’s drafter, training ensures the support condition holds, so the instantiated draft–verify step gives a final-token conditional distribution matching the AR model.The acceptance rule is the same form as Eq. (5) in the main text.Source: §E.2 · p. 16

E.3. Sequence-Level Losslessness of DEER

  • For any finite sequence, DEER produces the same joint output distribution as direct autoregressive sampling from the target model.The proof extends the one-step result by induction over token positions.Source: §E.3 · p. 16

F. KV cache of DLLM

  • For batch sizes greater than one, the implementation cannot yet integrate with vLLM or SGLang because mainstream inference frameworks lack mature dLLM KV-cache support.The authors expect future integration of dLLM KV-cache techniques to enable batched inference in these frameworks.Source: F. KV cache of DLLM · p. 17
Loading 2512.15176v1…