PaperlayerView exact revision on arXiv

Source-linked AI summary

d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation

Yu-Yang Qian, Junda Su, Lanxiang Hu, Peiyuan Zhang, Zhijie Deng, Peng Zhao, Hao Zhang

arXiv:2601.07568v3cs.LGcs.AI

TL;DR

dLLMs offer parallel and random-order generation, but their accuracy–parallelism trade-off makes it difficult to obtain both speed and quality, while existing evaluation metrics treat these dimensions separately. d3LLM combines pseudo-trajectory distillation with entropy-based multi-block decoding and KV-cache refresh, and introduces AUP to evaluate the joint trade-off. It achieves the highest AUP on 9 of 10 tasks, with speedups of 3.6×–5× over AR models and 10× over vanilla LLaDA/Dream with negligible accuracy degradation.

  • Problem

    Existing dLLM methods typically target either efficiency or performance, while single metrics fail to capture how accuracy changes as parallelism increases.

  • Method

    d3LLM uses teacher decoding trajectories and curriculum learning for training, entropy-based multi-block decoding with KV-cache refresh for inference, and AUP for joint evaluation.

  • Results

    d3LLM achieves the highest AUP on 9 of 10 tasks, 3.6×–5× speedup over Qwen-2.5-7B-it, and 10× speedup over vanilla LLaDA/Dream with negligible accuracy degradation.

  • Takeaways & Limitations

    The framework balances accuracy and parallelism across open-source foundation dLLMs, including coding generation with performance comparable to AR coders and 8× speedup over vanilla Dream-Coder.

  • Takeaways & Limitations

    AUP cannot serve as a training objective because its runtime parallelism statistics and parallelism–accuracy pair set are nondifferentiable with respect to model parameters.

Abstract

from arXiv · show

Diffusion large language models (dLLMs) offer capabilities beyond those of autoregressive (AR) LLMs, such as parallel decoding and random-order generation. However, realizing these benefits in practice is non-trivial, as dLLMs inherently face an accuracy-parallelism trade-off. Despite increasing interest, existing methods typically focus on only one-side of the coin, targeting either efficiency or accuracy. To address this limitation, we propose d3LLM (Pseudo-Distilled Diffusion Large Language Model), striking a balance between accuracy and parallelism: (i) during training, we introduce pseudo-trajectory distillation to teach the model which tokens can be decoded confidently at early steps, thereby improving parallelism; (ii) during inference, we employ entropy-based multi-block decoding with a KV-cache refresh mechanism to achieve high parallelism while maintaining accuracy. To better evaluate dLLMs, we also introduce AUP (Accuracy Under Parallelism), a new metric that jointly measures accuracy and parallelism. Experiments demonstrate that our d3LLM achieves up to 10$\times$ speedup over vanilla LLaDA/Dream, and 5$\times$ speedup over AR models without much accuracy drop. Our code is available at https://github.com/hao-ai-lab/d3LLM.

1. Introduction

dLLMs provide parallel decoding and other capabilities, but existing approaches face a fundamental accuracy–parallelism trade-off. d3LLM addresses this trade-off through trajectory-informed training and confidence-based multi-block inference, achieving substantial speedups with negligible accuracy degradation.

  • dLLMs use bidirectional attention to enable parallel decoding, error correction, and random-order generation beyond autoregressive models.
  • Open-source dLLMs such as LLaDA and Dream achieve around 20 tokens per second, substantially below some closed-source diffusion models.
  • Existing methods typically improve either efficiency or performance, because greater parallelism generally causes lower accuracy and vice versa.
  • Standard random masking gives training no guidance about which tokens can be decoded early, while single-block inference limits parallelism.
  • d3LLM uses pseudo-trajectory distillation and curriculum learning during training, then entropy-based multi-block decoding with periodic KV-cache refresh during inference.
  • d3LLM achieves the highest AUP on 9 of 10 tasks, 3.6×–5× speedup over Qwen-2.5-7B-it, 10× speedup over vanilla LLaDA/Dream, and 8× speedup over vanilla Dream-Coder.

2. Problem Formulation

The paper introduces AUP to evaluate how well dLLMs preserve accuracy as parallelism increases, rather than relying on separate efficiency or accuracy metrics. AUP weights the accuracy–parallelism curve to favor high-accuracy operation and excludes points below a defined degradation threshold.

  • Single metrics such as tokens per second, tokens per forward, or accuracy can miss the trade-off between dLLM efficiency and generation quality.
  • An accuracy–parallelism curve is obtained by sweeping a method’s speed–quality control knob to produce multiple parallelism–accuracy pairs.
  • Plain AUC can reward high speed even after accuracy collapses, so the desired metric should prioritize high-accuracy regimes before rewarding additional parallelism.
  • AUP is a weighted area under the accuracy–parallelism curve, with weights penalizing accuracy drops relative to the task’s best achievable accuracy.
  • AUP uses parallelism measured by TPF and accuracy in percentage, retaining only points with yi ≥ ymin, where ymin = y1 − 5.
  • AUP increases substantially when parallelism rises without accuracy loss but only modestly when parallelism gains sacrifice accuracy.
  • The penalty parameter α controls sensitivity to accuracy degradation; the paper sets α = 3 as the default.
  • AUP is an evaluation metric rather than a training objective because runtime TPF and the pair set underlying AUP are nondifferentiable with respect to model parameters.

3. d3LLM: Balance Accuracy and Parallelism

d3LLM combines pseudo-trajectory distillation during training with entropy-based multi-block decoding and KV-refresh during inference to balance accuracy and parallelism. Its training schedules improve decoding efficiency, while its inference mechanisms increase throughput while preserving generation quality.

  • Training: Pseudo-trajectory distillation uses the teacher dLLM’s decoding order as intermediate supervision, with curriculum learning to improve parallelism while preserving accuracy.The curriculum progressively increases training difficulty.
  • Training: 18% TPF improvement is achieved over random masking by training on noisy sequences that align student unmasking with the teacher’s decoding order.The student predicts ground-truth labels for masked tokens using cross-entropy loss.
  • Training: Approximately 12% higher TPF results from increasing the mask ratio from 0.0 to 0.8 instead of using a fixed mask ratio.Without curriculum noise, distillation becomes unstable and accuracy degradation is more likely.
  • Training: An additional 8% TPF improvement comes from progressively increasing the decoding window from 16 to 32 during training.The progressive window adapts the model to increasingly larger context spans.
  • Inference: Entropy-based multi-block decoding processes current and future blocks in parallel, prioritizing lower-entropy predictions and transitioning blocks through staged activation states.Blocks are conservatively decoded after 10% preceding-block completion and more aggressively activated after 95% completion.
  • Inference: Approximately 35% higher TPS in long-context scenarios results from combining KV-caching with periodic full-forward KV-refreshes before stabilizing blocks.A short delay before caching completed-block states helps maintain cache reliability.

4. Experiments

Experiments evaluate d3LLM across LLaDA- and Dream-based models, diverse benchmarks, AUP, throughput, and component ablations. d3LLM consistently balances accuracy and parallelism, achieving strong AUP and substantial speedups with negligible degradation.

  • Results on LLaDA-based Models: d3LLM-LLaDA achieves the highest AUP scores across all five benchmark tasks, including 637.7 on GSM8K-CoT and 107.6 on MATH.It outperforms dParallel, which scores 358.1 on GSM8K-CoT and 64.5 on MATH.
  • Results on LLaDA-based Models: d3LLM-LLaDA dominates the upper-right region of the MATH accuracy–parallelism curve and achieves the largest radar-chart coverage area across five tasks.The Dream-based variant likewise achieves the largest overall coverage area among Dream-based methods.
  • Results on Dream-based Models: d3LLM-Dream achieves the highest AUP scores on 4 out of 5 tasks, including 391.3 on GSM8K-CoT and 141.4 on MBPP-Instruct.Fast-dLLM-v2 instead leads on MATH with an AUP score of 126.7.
  • Wall-Clock Speed Comparison: 288.9 TPS on H100 and 183.3 TPS on A100 give d3LLM-LLaDA 5.0× and 3.6× speedups over Qwen-2.5-7B-it, respectively.It also reaches 10.3× the throughput of vanilla LLaDA on H100; d3LLM-Dream reaches 8.5× vanilla Dream throughput there.
  • Ablation Study: The ablation reaches 9.11 TPF and 73.1% accuracy in the full model, after pseudo-trajectory distillation and curriculum strategies improve parallelism and recover accuracy.Pseudo-trajectory distillation raises TPF from 6.41 to 7.55, curriculum noise raises it to 8.46, and curriculum windows yield the full model.
  • Hyperparameter Analysis: Curriculum noise improves TPF from 7.49 to 9.11 with 73.1% accuracy, while the curriculum window strategy yields a 19.0% improvement in AUP score.The reported schedules balance parallelism and accuracy more effectively than fixed settings.

5. Conclusion

The paper identifies an accuracy–parallelism trade-off in dLLMs and proposes d3LLM to balance both objectives. Its framework combines pseudo-trajectory distillation, entropy-based multi-block decoding with KV-cache refresh, and the AUP metric.

  • d3LLM addresses the fundamental accuracy–parallelism trade-off observed in diffusion LLMs.
  • Pseudo-trajectory distillation teaches the model which tokens can be decoded confidently early by leveraging the teacher’s unmasking order.
  • Entropy-based multi-block decoding with KV-cache refresh enables high parallelism while maintaining accuracy.
  • AUP jointly measures accuracy and parallelism for evaluating diffusion LLMs.

Impact Statement

The paper reports that d3LLM targets more efficient diffusion-LLM inference without much accuracy degradation. Supplementary experiments compare LLaDA-, Dream-, and Coder-based variants using accuracy–parallelism curves and AUP scores.

  • The work aims to improve diffusion-LLM inference efficiency without much accuracy degradation.
  • Supplementary experiments: LLaDA-based evaluations compare d3LLM-LLaDA with vanilla LLaDA, Fast-dLLM-LLaDA, D2F, and dParallel-LLaDA.
  • Supplementary experiments: The supplementary results include accuracy–parallelism curves and AUP comparisons for LLaDA-, Dream-, and Coder-based methods.
  • Supplementary experiments: Dream-based evaluations compare d3LLM-Dream with vanilla Dream, Fast-dLLM-Dream, Fast-dLLM-v2-7B, and dParallel-Dream.

A.4. Validation of the Hyperparameter in AUP Metric

The AUP sensitivity analysis varies α, which controls the penalty for accuracy degradation, and examines its effect on comparative method scores. d3LLM remains competitive across α settings, while hardware-independent AUP is distinguished from hardware-sensitive TPS.

  • α controls AUP’s penalty for accuracy degradation, with larger values making performance drops more costly.
  • The analysis compares AUP sensitivity across LLaDA-based and Dream-based methods on GSM8K-CoT using varying α values.
  • When α increases, methods with larger accuracy drops experience sharper AUP decreases, while d3LLM maintains competitive scores across settings.
  • AUP is hardware-independent because it uses Tokens Per Forward rather than hardware-sensitive Tokens Per Second.
  • Compared with SDTT, d3LLM substantially outperforms both SDTT variants in AUP on GSM8K-CoT.

A.6. Analysis of ymin in AUP

The analysis tests whether AUP comparisons depend on the minimum accuracy threshold. Varying the threshold tolerance changes absolute scores but preserves method rankings.

  • The minimum accuracy threshold is defined as y_min = y_1 − Δy, where y_1 is the highest achieved accuracy.
  • Across Δy values from 1 to 10, absolute AUP scores change but the relative ranking of LLaDA-based methods remains entirely consistent.
  • The analysis reports AUP, Accuracy, and Tokens Per Forward as evaluation quantities for comparing d3LLM with contenders.

A.7. Compare with Speculative Decoding Method

The comparison covers speculative decoding and dLLM baselines, while also examining optimized serving throughput. EAGLE-3 leads AUP, whereas d3LLM demonstrates substantial throughput on SGLang across GPU platforms.

  • Comparison with Speculative Decoding: EAGLE-3 attains the highest overall AUP score because speculative decoding verifies drafted tokens and avoids accuracy degradation under strong parallelism.The evaluation does not constrain total FLOPs.
  • System-Level Throughput: d3LLM-LLaDA reaches 1310 TPS on B200, 551 TPS on H100, and 251 TPS on A100 with SGLang.The reported evaluation uses GSM8K-CoT, zero-shot prompting, and TP=1.
  • dLLM Baselines: The comparison includes vanilla LLaDA, vanilla Dream, Fast-dLLM, Fast-dLLM-v2, dParallel, D2F, and SDTT as dLLM baselines.These methods span training-free acceleration, block diffusion, distillation, hybrid decoding, and reduced-step inference approaches.
  • System-Level Throughput: SGLang evaluates throughput using TPS on B200, H100, and A100 GPUs, alongside TPF, accuracy, and speedup relative to Qwen-2.5-7B-Instruct.These measurements provide a system-level comparison across hardware platforms.

A.10. Details of Datasets

The evaluation uses five benchmarks spanning mathematical reasoning, code generation, and longer-context reasoning. Training and inference settings specify distillation data, LoRA optimization, decoding controls, and block sizes.

  • Datasets: The five benchmarks are GSM8K-CoT, HumanEval, MBPP, MATH, and Long-GSM8K.They cover grade-school and advanced mathematics, Python code generation, and longer-context reasoning.
  • Datasets: GSM8K-CoT evaluates multi-step grade-school mathematics with chain-of-thought generation before the final answer.The setting is zero-shot in the stated benchmark description.
  • Datasets: HumanEval contains 164 handwritten Python problems whose unit tests assess functional correctness.MBPP adds around 1,000 crowd-sourced entry-level programming tasks with automated tests.
  • Datasets: Long-GSM8K contains 8.5K diverse grade-school word problems and uses 5-shot prompting with approximately 1000-token prompts.The benchmark requires multi-step arithmetic reasoning under longer context windows.
  • Training Settings: Training uses teacher block diffusion models with block size 32 and the dParallel distillation dataset, totaling approximately 122k Dream samples and 92k LLaDA samples.The data comes from PRM12K, AceCode, GSM8K training data, and Numina-Math.
  • Implementation Settings: LoRA targets all linear layers, while inference uses greedy decoding, block size 32, entropy thresholds of 0.4–0.5, and decoded-token threshold 0.95.Maximum generation length is 256 tokens for most tasks and 512 for Dream-Coder.

A.12. Further Improvements of d3LLM

The paper identifies several extensions beyond its current algorithmic distillation and decoding framework. These include combining d3LLM with speculative decoding, applying it to stronger dLLMs, and adding other optimization techniques.

  • Combining with Speculative Decoding: Speculative decoding could combine a smaller dLLM draft model with verification by a larger model.The proposed combination may further improve parallelism while preserving accuracy.
  • Applying to Stronger Foundation dLLMs: Applying d3LLM to stronger foundation dLLMs is proposed because newer converted models outperform the LLaDA and Dream models used here.The paper describes d3LLM as largely model-agnostic and potentially plug-and-play.
  • Applying to Stronger Foundation dLLMs: Potential future enhancements include reinforcement learning, continuous learning, and quantization.These techniques are presented as ways to further enhance d3LLM effectiveness.
  • Summary and Future Work: The work primarily studies algorithmic distillation and decoding recipes, while SGLang system optimizations substantially amplify the reported gains.Further improvements to diffusion language-model performance and efficiency are left for future work.

C. Limitation

The main evaluation focuses on LLaDA and Dream, although the proposed distillation and inference approach is described as model-agnostic for newer dLLMs.

  • Evaluation Scope: The evaluation primarily covers LLaDA and Dream despite the emergence of potentially stronger dLLMs such as SDAR, ReFusion, and LLaDA 2.0.The paper presents model-agnosticism as enabling future application to these newer models.
Loading 2601.07568v3…