PaperlayerView exact revision on arXiv

Source-linked AI summary

Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles

Qingyan Wei, Yaojie Zhang, Zhiyuan Liu, Puyu Zeng, Yuxuan Wang, Biqing Qi, Dongrui Liu, Linfeng Zhang

arXiv:2506.10848v3cs.CLcs.AIcs.LG

TL;DR

Existing dLLM sampling strategies can be static, limiting flexibility in how many tokens are sampled and where. SlowFast Sampling alternates exploratory and accelerated decoding using certainty, convergence, and positional principles; experiments report speedups up to 15.63×, or 34.22× with dLLM-Cache on GPQA, while maintaining generation quality.

  • Problem

    Existing confidence-based and semi-autoregressive dLLM samplers can have static speed and accuracy drops at high parallelism, motivating more flexible token selection.

    Source: §1 · p. 1

  • Method

    SlowFast Sampling uses exploratory decoding to locate emerging certainty and convergence, then parallel-decodes high-certainty tokens, guided by certainty, convergence, and positional principles.

    Source: §1 · p. 2

  • Results

    SlowFast Sampling achieves up to 15.63× speedup, or up to 34.22× with dLLM-Cache on GPQA, while maintaining generation quality.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    Experiments indicate that dLLMs with suitable sampling and optimization can outperform autoregressive LLMs in throughput.

    Source: §4.3 · p. 9, §5 · p. 9

Abstract

from arXiv · show

Diffusion-based language models (dLLMs) have emerged as a promising alternative to traditional autoregressive LLMs by enabling parallel token generation and significantly reducing inference latency. However, existing sampling strategies for dLLMs, such as confidence-based or semi-autoregressive decoding, often suffer from static behavior, leading to suboptimal efficiency and limited flexibility. In this paper, we propose SlowFast Sampling, a novel dynamic sampling strategy that adaptively alternates between exploratory and accelerated decoding stages. Our method is guided by three golden principles: certainty principle, convergence principle, and positional principle, which govern when and where tokens can be confidently and efficiently decoded. We further integrate our strategy with dLLM-Cache to reduce redundant computation. Extensive experiments across benchmarks and models show that SlowFast Sampling achieves up to 15.63$\times$ speedup on LLaDA with minimal accuracy drop, and up to 34.22$\times$ when combined with caching. Notably, our approach outperforms strong autoregressive baselines like LLaMA3 8B in throughput, demonstrating that well-designed sampling can unlock the full potential of dLLMs for fast and high-quality generation.

1 INTRODUCTION

The introduction frames dLLM sampling as a flexibility problem: existing approaches are static, while parallel token generation offers a potential latency advantage. It presents SlowFast Sampling as a dynamic alternative guided by three principles.

1 INTRODUCTION

  • Existing confidence-based and semi-autoregressive samplers can lose accuracy at high parallelism and maintain a static sampling speed.The paper motivates a method that can adapt both how many tokens are sampled and where they occur in the sequence.Source: §1 · p. 1
  • The three principles use token confidence, confidence convergence, and positional clustering to guide token selection, refinement, and caching.Higher-confidence tokens are more likely to be decoded correctly early; convergence signals minimal need for further refinement, while clustered positions can support caching.Source: §1 · p. 2
  • SlowFast Sampling alternates an exploratory stage that locates emerging certainty and convergence with an accelerated stage that parallel-decodes high-certainty tokens.The two-stage strategy is designed to reduce effort on parts already determined.Source: §1 · p. 2
  • SlowFast Sampling achieves up to 15.63× speedup, or up to 34.22× with dLLM-Cache on GPQA, while maintaining generation quality.The paper describes its speed-quality trade-off as superior to baseline and simpler sampling methods.Source: §1 · p. 2

2 RELATED WORK

Related work describes diffusion language models and two main acceleration directions: caching and sampling changes. The supplied GPQA comparison reports higher throughput with SlowFast Sampling and further gains with caching.

2.1 DIFFUSION MODELS FOR LANGUAGE

  • On GPQA with LLaDA, SlowFast Sampling achieves 15.63× speedup; with dLLM-Cache, throughput reaches 54.75 tokens/sec and up to 34.22× speedup, with minor accuracy drops.The comparison is for 8-shot, length 1024 generation against vanilla decoding.Source: Figure 2 · p. 3
  • Masked diffusion models iteratively predict masked text tokens from context; LLaDA and Dream are examples of dLLMs with bidirectional architectures.The passage notes that both demonstrate performance comparable to similarly sized autoregressive models.Source: §2.1 · p. 3

2.2 ACCELERATION METHODS FOR DIFFUSION-BASED LLMS

  • Acceleration research spans feature and KV caching, confidence-based token selection, and semi-autoregressive decoding in fixed blocks.The related-work discussion also describes dynamic cache eviction, delayed KV caching, and exact simulation approaches for masked diffusion models.Source: §2.2 · p. 3

3 METHODOLOGY

The methodology frames dLLM inference as iterative denoising and uses certainty, convergence, and positional patterns to adapt decoding. SlowFast alternates cautious exploration with accelerated decoding of stable spans, while caching predictions beyond them.

3.1 PRELIMINARY

  • dLLM inference iteratively denoises masked sequences across N steps; LLaDA uses random or low-confidence remasking, or left-to-right blocks denoised iteratively.The sampling strategy affects generation quality and latency.Source: §3.1 · pp. 3–4

3.2 THREE GOLDEN PRINCIPLES OF DLLMS

  • The certainty principle treats high-confidence predictions as more likely to be correct and remain unchanged, making them candidates for early acceptance or less frequent resampling.The paper links this principle to reducing redundant computation on already determined sequence parts.Source: §3.2 · pp. 4–5
  • The convergence principle uses stable token identities and confidence over recent steps as a signal that tokens are less likely to change and need less re-evaluation.The paper describes confidence and predicted identity fluctuating before settling during iterative refinement.Source: §3.2 · p. 5
  • The positional principle observes that high-confidence, early-converging tokens often form contiguous blocks or localized regions rather than appearing randomly across the sequence.These patterns motivate adapting which sequence locations receive decoding effort.Source: §3.2 · p. 5

3.3 SLOWFAST SAMPLING

  • Within the exploratory window, cautious decoding selects the highest-confidence top-k tokens and predicts how far confidence extends into the sequence.If stability is not reached within Kmax exploratory steps, the method uses a conservative horizon or defaults to cautious full-sequence decoding.Source: §3.3 · pp. 5–6
  • SlowFast alternates cautious exploration with accelerated decoding: it identifies a stable region, then decodes that span in parallel while caching predictions outside it.Exploration tracks candidate horizons and tests whether their variance across a recent window falls below a stability threshold.Source: §3.3 · pp. 5–6, Figure 3 · p. 6
  • In the accelerated span, high-confidence masked tokens are unmasked in parallel; if too few qualify, top-k refinement provides a fallback and the next cycle starts at the span endpoint.Low-confidence predictions outside the span can be cached and reused while those positions remain outside an active decoding span.Source: §3.3 · pp. 6–7

4 EXPERIMENT

Experiments evaluate SlowFast Sampling on representative diffusion language models, comparing its speed and generation quality across sampling strategies and with caching. Ablations examine the decoding phases and stability-check settings, while a further comparison tests throughput against an autoregressive model.

4.1 EXPERIMENT SETTINGS

  • Experiments evaluate LLaDA 8B and Dream 7B on benchmarks using Tokens Per Second for speed and task-specific metrics such as GSM8K accuracy for quality.Experiments were conducted on NVIDIA RTX 4090 GPUs.Source: §4.1 · p. 7

4.2 MAIN RESULTS

  • Combining SlowFast Sampling with dLLM-Cache delivers higher throughput while maintaining comparable model performance for LLaDA 8B and Dream 7B.The comparison covers the models with and without the combined method.Source: Table 2 · p. 8, §4.2 · p. 8
  • SlowFast Sampling achieves higher inference efficiency with competitive generation quality across the compared sampling strategies and improves efficiency with lossless acceleration in most cases.The strategy comparison covers diffusion, semi-autoregressive, autoregressive, and Fast-dLLM sampling; the main-results passage reports lossless acceleration in most cases.Source: §4.2 · pp. 7–8, Table 3 · p. 8

4.3 ABLATION STUDY

  • 54.75 vs. 33.79 TPS: LLaDA Base with SlowFast Sampling and dLLM-Cache exceeds LLaMA3 8B throughput while maintaining comparable accuracy.Table 4 reports a throughput advantage of up to +20.96 TPS.Source: Table 4 · p. 8, §4.3 · p. 9
  • In the sentence-generation case study, slow phases emit a few reliable tokens and fast phases decode longer high-confidence spans.The figure tracks the confidence map across alternating phases, with blue tokens marking those generated in the current step.Source: §4.3 · pp. 8–9, Figure 4 · p. 9
  • The stability check uses Kmax = 8, Whist = 2, and stable variance threshold σ² = 1.0 to balance exploration, quality, and speed.The authors describe Kmax = 8 as sufficient exploration, Whist = 2 as an effective balance, and the threshold as restricting acceleration to stable regions.Source: §4.3 · p. 9, Figure 5 · p. 9

5 CONCLUSION

The conclusion presents SlowFast Sampling as a two-stage dynamic decoding approach guided by token certainty, convergence, and positional bias. Experiments report faster inference with strong generation quality, including throughput gains over autoregressive LLMs.

5 CONCLUSION

  • SlowFast Sampling uses a two-stage pipeline that adaptively balances exploration and efficient parallel decoding, guided by token certainty, convergence, and positional bias.The conclusion characterizes the approach as dynamic and principled.Source: §5 · p. 9
  • Experiments report speedups up to 15.63×, or up to 34.22× with dLLM-Cache on GPQA, while maintaining strong generation quality.Source: §5 · p. 9
  • The conclusion reports that SlowFast Sampling outperforms autoregressive LLMs in throughput and frames the work as a step toward practical dLLMs.Source: §5 · p. 9

REPRODUCIBILITY STATEMENT

The reproducibility statement describes where implementation and evaluation details are reported and notes that code and experiment scripts are provided in supplementary materials.

REPRODUCIBILITY STATEMENT

ETHICS STATEMENT

The ethics statement says the work uses publicly available benchmarks and does not involve human subjects, personal or sensitive data, or applications directly impacting individuals.

ETHICS STATEMENT

  • The research does not involve human subjects, personal or sensitive data, or applications directly impacting individuals.Source: ETHICS STATEMENT · p. 10
  • Experiments use publicly available benchmarks, which the statement associates with fairness, reproducibility, and transparency.Source: ETHICS STATEMENT · p. 10
  • The authors report no known potential misuse beyond standard language-model concerns such as bias or downstream misuse, and encourage responsible use under the ICLR Code of Ethics.Source: ETHICS STATEMENT · p. 10

A.1 LLM USAGE STATEMENT.

The authors state that an LLM was used only to polish writing and improve clarity, not to conduct or generate the research.

A.1 LLM USAGE STATEMENT.

  • The LLM assisted only with writing clarity; the authors say it was not involved in ideation, experimental design, data analysis, or scientific-content generation.The authors take responsibility for the paper’s content.Source: §A.1 · p. 12

A.2 COMPATIBILITY WITH ADVANCED SAMPLING METHODS.

The threshold analysis describes how SlowFast selects exploratory candidate regions and balances accuracy against speed during accelerated decoding.

A.2 COMPATIBILITY WITH ADVANCED SAMPLING METHODS.

  • τmin_conf defines the candidate region in exploration; τhigh_conf controls the speed-quality trade-off in accelerated decoding.The figure caption identifies these thresholds’ respective roles in exploratory range and fast decoding.Source: §A.2 · p. 12, Figure 6 · p. 12
  • On GSM8K, τmin_conf = 0.1 is described as optimal, while τhigh_conf = 0.85 gives a near-peak score without a drastic speed reduction.Lower τmin_conf values yield unstable regions, while higher values create overly conservative, inefficient regions; increasing τhigh_conf favors accuracy at the cost of TPS.Source: §A.2 · p. 12

A.3 EXPERIMENTAL DETAILS

Across the supplied appendix sections, SlowFast’s trade-off improves over baselines, its unified settings remain robust across tasks and a model variant, and its dynamic decoding handles separated stable regions.

A.3 EXPERIMENTAL DETAILS

  • Inference settings were standardized to each model’s official recommendations, including steps, block length, generation length, and few-shot examples.Tables 5 and 6 summarize settings for LLaDA and Dream, respectively.Source: §A.3 · p. 12, Table 5 · p. 12, Table 6 · p. 12

A.4 ABLATION STUDY ON HYPERPARAMETERS AND ROBUSTNESS.

  • A unified hyperparameter set maintains near-optimal performance on MATH, MBPP, and HumanEval and transfers to Dream 7B on GSM8K.The authors report that this stability means extensive tuning is not required for new tasks or model variants.Source: §A.4 · p. 13, Figure 7 · p. 13, Figure 8 · p. 13

A.5 PERFORMANCE-EFFICIENCY TRADE-OFF ACROSS STRATEGIES

  • Figure 9 shows SlowFast shifts the performance-efficiency trade-off toward higher TPS and accuracy than the baseline in both evaluation setups.The setups are 5-shot with 256 tokens and 8-shot with 1024 tokens.Source: Figure 9 · p. 13, §A.5 · p. 13

A.6 CASE STUDIES ON POSITIONAL PRINCIPLE AND NON-CONTIGUOUS DECODING.

  • Figure 10 shows SlowFast can jump over a generally small low-confidence gap between stable blocks while maintaining acceleration without sacrificing accuracy.The case study concerns minor positional uncertainty.Source: §A.6 · p. 13, Figure 10 · p. 14
  • In a manually constructed extreme case, SlowFast identifies a second high-confidence block after an unstable region and resumes acceleration.The figure demonstrates handling of widely separated, non-contiguous stable blocks.Source: §A.6 · p. 14, Figure 11 · p. 14

A.7 ALGORITHM: SLOWFAST SAMPLING FOR DLLMS

  • The algorithm explores step by step, checks endpoint-prediction variance for stability, then parallel-decodes high-confidence tokens and caches out-of-span features.The stability check uses a history window; the fast stage proceeds only in a reliably stable region.Source: §A.7 · p. 14
Loading 2506.10848v3…