PaperlayerView exact revision on arXiv

Source-linked AI summary

Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing

Xu Wang, Chenkai Xu, Yijie Jin, Jiachun Jin, Hao Zhang, Zhijie Deng

arXiv:2508.09192v1cs.LGcs.AI

TL;DR

Open-source dLLMs had not surpassed similarly sized AR LLMs in inference speed, despite their potential for parallel generation. The paper introduces D2F, which combines KV-cache-compatible block-wise generation with inter-block parallel decoding, and reports faster-than-AR inference alongside large speedups over vanilla dLLMs.

  • Problem

    Existing open-source dLLMs had not achieved superior inference speed over similarly sized AR LLMs, despite their potential to generate multiple tokens in parallel.

  • Method

    D2F trains dLLMs to predict later token blocks from partially predicted preceding blocks, combining block-wise causal KV caching with pipelined inter-block parallel decoding.

  • Results

    D2F achieves up to 2.5× faster inference than LLaMA3 and Qwen2.5 and more than 50× acceleration over vanilla LLaDA and Dream while maintaining comparable average benchmark performance.

  • Takeaways & Limitations

    D2F establishes an open-source dLLM approach that supports faster-than-AR inference while retaining the parallel decoding advantages of diffusion models.

Abstract

from arXiv · show

Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to autoregressive (AR) LLMs for text generation, with the potential to decode multiple tokens in a single iteration. However, none of the existing open-source dLLMs have achieved superior inference speed over AR LLMs of similar size. This paper breaks this barrier based on a simple and effective strategy named discrete diffusion forcing (D2F). D2F equips dLLMs with two key capabilities: (1) block-wise autoregressive generation to enable KV cache utilization; (2) prediction of following tokens without requiring completion of prior blocks for inter-block parallel decoding. In this way, the vanilla dLLMs are refurbished into an AR-diffusion hybrid paradigm for efficient inference. D2F can be implemented with an asymmetric distillation process based on pre-trained dLLMs. We further propose a pipelined parallel decoding algorithm, which enables a trade-off between efficiency and efficacy. Empirically, D2F dLLMs achieve more than $\mathbf{2.5\times}$ inference speed than LLaMA3 and Qwen2.5 on GSM8K. Compared to vanilla dLLMs like LLaDA and Dream, the acceleration can be more than $\mathbf{50\times}$ while maintaining comparable output quality. The code is available at https://github.com/zhijie-group/Discrete-Diffusion-Forcing.

1 INTRODUCTION

D2F addresses the open-source dLLM speed gap by combining block-wise KV-cache-compatible generation with inter-block parallel decoding. This hybrid design yields faster-than-AR inference while preserving comparable benchmark performance.

  • Motivation: Open-source dLLMs had not demonstrated faster inference than similarly sized AR LLMs, despite their potential for parallel token generation.Existing approaches pursued KV-cache strategies or parallel sampling, but block-wise sequential generation could eliminate inter-block parallelism.
  • Discrete diffusion forcing: D2F extends diffusion forcing to discrete text, allowing subsequent blocks to be predicted before preceding blocks are fully denoised.The resulting paradigm combines block-wise sequential generation with inter-block parallelism instead of requiring complete preceding blocks.
  • Method: D2F dLLMs denoise token blocks with increasing mask ratios, use block-wise causal attention for accurate KV caching, and learn through asymmetric distillation.Inference uses a pipelined parallel decoding algorithm that trades off efficiency and performance.
  • Results: More than 50× acceleration was achieved over LLaDA-Instruct-8B and Dream-Base-7B without compromising average performance across mathematical and programming benchmarks.The evaluated benchmarks included GSM8K, MATH, HumanEval, and MBPP.
  • Results: Up to 2.5× faster inference than prevalent AR LLMs such as LLaMA3 and Qwen2.5 establishes D2F as a faster-than-AR open-source dLLM approach.The contribution is presented as the first faster-than-AR dLLMs.

2 RELATED WORK

Related work addresses dLLM speed through caching, sampling optimization, and hybrid architectures. D2F's pipelined decoding combines parallel block processing with dynamic block activation.

  • Diffusion language models: dLLMs seek to reduce AR decoding latency by iteratively denoising masked sequences rather than generating tokens sequentially.This parallel denoising paradigm motivates diffusion-based language generation.
  • Acceleration methods: Existing dLLM acceleration methods use approximate caching schemes or sampling optimizations to reduce redundant computation and decoding steps.Bidirectional attention makes standard KV caching incompatible, while confidence-aware strategies reduce iterations.
  • Pipelined decoding: D2F dynamically adds blocks when the latest block exceeds a completion threshold, while fully activated blocks decode multiple tokens more aggressively.The illustrated threshold is τadd = 1/6.
  • AR-diffusion hybrids: AR-diffusion hybrids apply autoregressive dependencies across units while retaining diffusion denoising within units, a design previously explored in video generation.This architecture separates temporal dependency modeling from within-unit denoising.

3 PRELIMINARY: DIFFUSION LARGE LANGUAGE MODELS (DLLMS)

Masked dLLMs corrupt text into progressively noisier sequences and learn to reverse that process. Their practical speed is limited by bidirectional-attention caching conflicts and the difficulty of parallelizing dependent tokens.

  • Forward process: Masked diffusion progressively replaces tokens in an L-token sequence with [MASK] across a continuous time schedule t ∈ [0, 1].At the endpoint, the sequence is fully masked.
  • Reverse process: dLLMs learn pθ(Y 0|Y t) to denoise a fully masked sequence into language samples while predicting all masked tokens simultaneously at each step.This simultaneous prediction provides the theoretical basis for parallel generation.
  • Inference bottlenecks: Bidirectional attention conflicts with standard KV caching, causing redundant computation across denoising steps.This is a central practical inference bottleneck for masked dLLMs.
  • Inference bottlenecks: Conditional independence assumptions make interdependent-token generation difficult, requiring more iterative steps for high-quality outputs.Prior methods do not simultaneously provide precise KV caching and efficient parallel decoding.
  • D2F setup: D2F training divides sequences into blocks, applies a monotonically increasing noise schedule, and updates a D2F model through asymmetric distillation from a pretrained dLLM.The supplied training procedure samples blocks and noise levels before predicting block distributions.

4 METHOD

D2F refurbishes dLLMs into an AR-diffusion hybrid by combining block-wise causal generation, inter-block parallel decoding, asymmetric distillation, and pipelined inference.

  • Discrete Diffusion Forcing: D2F provides block-level AR generation for standard KV-cache reuse and inter-block parallel decoding from incomplete predecessors.These capabilities reduce redundant computation while increasing the number of tokens decoded per inference step.
  • Discrete Diffusion Forcing: D2F partitions sequences into equal-sized blocks and applies monotonically increasing noise levels so earlier blocks are more complete than later blocks.The reverse process can finalize preceding blocks while advancing denoising in subsequent blocks.
  • Discrete Diffusion Forcing: Block-wise causal attention preserves exact KV reuse across completed blocks while retaining bidirectional attention within each block.This attention structure supports cached computation without requiring fully denoised preceding blocks before later-block decoding advances.
  • Asymmetric Distillation: D2F is distilled from a pre-trained bidirectional dLLM using an asymmetric KL-divergence objective under the student's causally restricted view.The teacher uses a global view of noisy blocks, while the student learns to reproduce its mask predictions with block-wise causal attention.
  • Pipelined Parallel Decoding: Algorithm 2 updates the KV cache after blocks are completed during pipelined D2F inference.The procedure appends blocks according to decoding progress and confidence thresholds while operating on cached forward passes.
  • Pipelined Parallel Decoding: Pipelined parallel decoding maintains a sliding window, dynamically appends masked blocks, and uses dual-state activation to balance efficiency against decoding quality.New blocks begin semi-activated and become fully activated after their predecessors accumulate sufficient decoding progress.

5 EXPERIMENTS

D2F is evaluated on mathematical reasoning and code-generation benchmarks against accelerated dLLMs and comparable AR LLMs. It improves inference speed while preserving performance, with results depending on pipeline and block-size settings.

  • Experimental Setup: D2F is evaluated on GSM8K, GSM8K-CoT, MATH, HumanEval, and MBPP using LLaDA-Instruct-8B and Dream-Base-7B.Comparisons include Fast-dLLM, dLLM-Cache, LLaMA3-Instruct-8B, and Qwen2.5-Base-7B.
  • Main Results: 119.9 tokens/s on GSM8K gives D2F-Dream-Base-7B a 2.5× speedup over LLaMA3-Instruct-8B and 2.3× over Qwen2.5-Base-7B.The maximum generation length is 512 for all methods.
  • Main Results: 52.9× speedup on LLaDA-Instruct-8B preserves comparable performance, while D2F reaches 90.2 tokens/s on MATH, 2.1× Fast-dLLM’s Dual-Cache.The LLaDA comparison reports 47.6 versus 0.9 tokens/s and scores of 38.0 versus 39.0.
  • Main Results: 91.2 tokens/s on GSM8K-CoT yields a 9.6× speedup over Dream-Base-7B and a 1.8× speedup over Fast-dLLM, with slight performance improvement.A unified maximum length of 256 is used for this comparison because the base model struggles with the stop token.
  • Throughput-Performance Trade-off: 150.9 tokens/sec with a score of 71.2 on GSM8K gives D2F 3.1× LLaMA3’s throughput while exceeding its score of 70.1.Dream-Base-7B instead drops from 71.4 to 42.8 when sampling steps decrease from 512 to 128.
  • Ablations and Analysis: Increasing inference block size reduces throughput and first improves then worsens performance; block size 48 peaks at 77.5 versus 75.9 for size 16.The ablation tests integer multiples of the training block size, which is 16.
  • Ablations and Analysis: With τact = 0.85, the dual-state setting τadd = 0.7 raises score from 72.6 to 74.2 and throughput from 136.8 to 139.0 TPS.Further reducing τadd improves score while marginally reducing throughput.
  • Ablations and Analysis: On MBPP, structured D2F noise scheduling improves score by 5.0 points and throughput by 24.0 TPS over independent random schedules.The comparison uses τact = 0.95, maximum length 512, block size 32, τconf = 0.9, and τadd = 0.1.

6 CONCLUSION

The paper introduces D2F as a training paradigm that conditions next-block prediction on partially predicted preceding blocks. This supports KV caching and parallel generation, enabling faster-than-AR dLLM inference.

  • 6 CONCLUSION: D2F conditions next-block prediction on partially predicted tokens from previous blocks, supporting KV cache and parallel generation.The paper reports D2F as the first dLLM to support faster-than-AR inference.

A.1 ABLATION STUDIES ON D2F COMPONENTS

The ablation separates D2F’s cache and parallel-decoding contributions. KV caching provides substantial speedup, while the pipelined parallel decoder adds further acceleration.

  • D2F Components: Cache-only enables standard KV caching but generates blocks serially, whereas Cache + Para combines caching with pipelined parallel decoding.The two configurations isolate cache use from asynchronous block generation.
  • D2F Components: Adding the parallel pipeline increases GSM8K speedup for LLaDA from 2.4x to 7.3x beyond the cache-only configuration.The result highlights the contribution of asynchronous generation.

A.2 CONTROL EXPERIMENT FOR DATA CONTRIBUTION

The control experiment separates D2F’s methodology from the contribution of distillation data by fine-tuning Dream-Base on the same dataset with identical LoRA parameters. Direct fine-tuning yields minor task-score gains but significantly reduces inference speed, supporting D2F as the key source of acceleration.

  • Dream-Base* was created by directly fine-tuning Dream-Base on the same distillation dataset used for D2F.The control model used LoRA with exactly the same parameters as D2F training.
  • Dream-Base* shows minor score improvements on some tasks but suffers a significant inference-speed drop.The speed degradation is attributed to computational overhead from the added LoRA layers.
  • The control experiment identifies D2F methodology, rather than the training data, as the key driver of the exceptional inference acceleration.

A.3 HYPERPARAMETER DETAILS

The experiments specify separate sampling configurations for baseline and D2F models, including generation limits, block size, and pipelined-decoding thresholds. Several tables document ablations, baseline-relative performance ratios, and fairness-oriented hyperparameter choices.

  • Sampling hyperparameters: Table 8 specifies maximum generation length, inference block size, and pipelined-decoding thresholds for D2F models.The listed thresholds include token addition, block activation, and token confirmation thresholds.
  • Sampling hyperparameters: Baseline hyperparameters are adopted from prior work to ensure fair comparison with D2F configurations.
  • Ablation and control tables: Tables 5, 6, and 7 report ablations and baseline-relative performance ratios for D2F-LLaDA, Dream-Base*, and D2F-Dream.Table 6 compares Dream-Base with its directly fine-tuned version, Dream-Base*; Tables 5 and 7 study proposed-method variants.
  • Sampling hyperparameters: For D2F, the length parameter sets only a maximum generation limit and does not affect the sampling distribution because generation is block-wise.For Dream-based models, D2F’s maximum length matches the baseline because the base model often fails to generate proper termination tokens.
Loading 2508.09192v1…