Fast-dLLM v2: Efficient Block-Diffusion LLM

• AR decoding generates tokens sequentially, limiting parallelism and inference efficiency.Source: Figure 1 · p. 1, §1 · p. 2
• Diffusion LLMs enable joint token or block prediction but often struggle with KV caching, latency, and flexible generation lengths.Source: §1 · p. 2
• Block diffusion combines intra-block diffusion with autoregressive conditioning across blocks, supporting flexible lengths and KV caching, but had limited large-scale validation.Source: §1 · p. 2
• Fast-dLLM v2 adapts pretrained AR models using blockwise attention and complementary masking, requiring ∼1B fine-tuning tokens versus ∼500B for Dream.Source: §1 · p. 2
• 2.5× speedup over standard AR decoding is achieved while maintaining comparable generation quality across large-scale, diverse-task experiments.Source: §1 · p. 2

• Discrete diffusion evolved from Markov-chain formulations to masked diffusion models that replace tokens with [MASK] according to a mask ratio.Source: §2.1 · pp. 2–3
• Blockwise diffusion methods interpolate between denoising diffusion and autoregressive generation by applying diffusion within blocks under a left-to-right structure.Source: §2.2 · p. 3
• Diffusion LLM acceleration primarily uses caching mechanisms and advanced decoding strategies to reduce bidirectional-attention costs.Source: §2.3 · p. 3
• DualCache reuses prefix and suffix KV activations, while other methods delay, adapt, sparsify, or window cache reuse.Source: §2.3 · p. 3

3.1. Preliminary

• Autoregressive models predict each token from its preceding clean context, whereas diffusion models predict original tokens from independently masked sequences.Source: §3.1 · p. 3
• Training decodes sequences block by block with partial masking and applies loss only to predictions of masked tokens.Source: Figure 2 · p. 4, §3.1 · p. 4

3.2. Adaptation to Block Diffusion LLM

• Sequences are padded to block-size multiples, packed into fixed-length contexts, and split into aligned non-overlapping blocks.Source: §3.2 · p. 4
• Complementary mask views duplicate each sample with masks m and 1 − m so tokens receive supervision in masked and unmasked contexts.Source: §3.2 · p. 4
• Shifted labels use the preceding position’s logit to predict masked tokens, preserving AR-like temporal representations during intra-block diffusion.Source: §3.2 · p. 4
• Inference decodes one block at a time, reuses decoded blocks as clean prefix context, and refines the current block in parallel.Source: Figure 3 · p. 5, §3.3 · p. 5
• A hybrid attention mask processes corrupted and clean sequences together while supporting block parallelism and causal dependencies between blocks.Source: §3.2 · p. 5
• Confidence-aware decoding finalizes high-confidence tokens in parallel while retaining uncertain positions for later refinement.Source: §3.3 · p. 5

Main Results

• Fast-dLLM v2 reaches average scores of 45.0 at 1.5B and 60.3 at 7B, surpassing the listed baselines at both scales.At 7B, it exceeds Qwen2.5-7B-Nemo-FT (59.6) and Dream (57.6).Source: §4.2 · p. 6
• Both model sizes perform on par with or better than counterparts trained with standard next-token prediction on identical data and training steps.The comparison uses original Qwen-2.5 models tuned with next-token prediction under matched training conditions.Source: §4.1 · p. 6, §4.2 · p. 7

Performance and Speed

• At threshold 0.9, GSM8K throughput rises from 39.1 to 101.7 tokens/s, yielding a 2.6× speedup with only a marginal accuracy drop.Threshold 1.0 recovers the standard non-parallel decoding process.Source: Figure 4 · p. 6, §4.2 · p. 7
• On GSM8K, diffusion generation reaches up to 1.5× higher throughput than autoregressive generation on A100 and 1.8× on H100 at batch size 64.The comparison uses threshold 0.9 and sub-block cache across varying batch sizes.Source: §4.2 · p. 7, Figure 5 · p. 8

Ablation Study

• The full training recipe with padding and complementary masking improves average accuracy by +3.7 points over naive token shifting.Padding prevents cross-sample leakage, while complementary masking provides supervision for all input tokens.Source: §4.3 · pp. 7–8
• Sub-block size 8 gives the highest average accuracy across tasks, whereas mismatched inference block sizes substantially degrade performance.For example, GSM8K drops from 62.0 to 58.5 under a mismatched block size; larger sub-blocks increase throughput but slightly reduce accuracy.Source: §4.3 · p. 8
• Sub-block caching provides substantial speedups in compute-bound settings without observable accuracy changes.Its gains are negligible at small batch sizes but substantial at practical compute-bound batch sizes such as 32.Source: §4.3 · p. 8

5. Conclusion

• Fast-dLLM v2 integrates blockwise diffusion, complementary masking, hierarchical caching, and parallel decoding for efficient diffusion-style generation.The framework includes block-level context reuse and sub-block refinement caching.Source: §5 · pp. 8–9
• Sub-block caching preserves accuracy and provides substantial speedup in compute-bound settings such as batch size 32.Its gains are negligible at small batch sizes, where memory bandwidth is underutilized.Source: Figure 6 · p. 9
• Larger sub-block sizes increase decoding throughput by reducing sequential forward passes and increasing intra-step parallelism, but may slightly reduce accuracy.The reported optimal sub-block size is 8, while mismatched training and inference sizes reduce performance.Source: Table 3 · p. 9, Table 4 · p. 9
• 2.5× speedup over standard AR decoding is achieved without loss of generation quality on large-scale Qwen2.5-Instruct models.Experiments cover the 1.5B and 7B model sizes.Source: §5 · p. 9

A.1. Training Setup

• Pretrained Qwen2.5-Instruct models with 1.5B and 7B parameters are fine-tuned under the block-wise diffusion framework.Experiments generally use context length 2048 and batch size 256 on 64 NVIDIA A100 GPUs.Source: §A.1 · p. 13
• The fine-tuning data comes from a high-quality LLaMA-Nemotron instruction-following subset spanning broad domains.Sequences are block-wise packed, and padding tokens are excluded from loss and gradient updates.Source: §A.1 · p. 13
• The 1.5B model uses 6,000 steps at 2 × 10^-5, while the 7B model uses 2,500 steps at 1 × 10^-5.Both settings use AdamW and linear warmup for the first 500 steps.Source: §A.1 · p. 13
• Training processes approximately 3.15 billion tokens for the 1.5B model and 1.31 billion tokens for the 7B model.These totals follow from the stated step counts and 524,288 tokens per step.Source: §A.1 · p. 13
• All experiments use block size 32 with right-padded, block-aligned sequences to support consistent batch construction.The alignment is designed to utilize model context efficiently under hardware constraints.Source: §A.1 · p. 13

A.2. Attention Mask Design

• Training concatenates the noised sequence x_t and clean sequence x_0 into a single 2L-token input processed with a hybrid attention mask.The mask is defined over the combined sequence and supports simultaneous noised and clean representations.Source: §A.2 · p. 13, Figure 7 · p. 13
• The block-diagonal mask provides bidirectional self-attention among tokens within each noised block for within-block refinement.This enables tokens in the same block to attend to one another during diffusion training.Source: §A.2 · p. 14
• The offset block-causal mask lets noised tokens attend to clean tokens in previous blocks, preserving inter-block causal conditioning.It connects the current noised block with its clean causal context.Source: Figure 7 · p. 13, §A.2 · p. 14
• The block-causal mask lets clean tokens attend to all previous and current block positions, supporting autoregressive-like progression.This retains left-to-right structure for the clean sequence.Source: §A.2 · p. 14
• During inference, previously generated blocks are cached, while only the current noised block is refined with bidirectional within-block attention.The current block attends causally to the unmasked tokens in prior blocks, enabling KV-cache reuse and reduced memory footprint.Source: §A.2 · p. 14

A.3. Details on Training Objective

• The training objective minimizes cross-entropy only over masked tokens.The loss predicts masked positions from their available contextual representations.Source: §A.3 · p. 14
• The objective omits the usual normalization by the number of masked tokens intentionally because complementary masks are used.Each sample pairs masks m and 1 − m at times t and 1 − t.Source: §A.3 · p. 14
• Complementary masking guarantees that the total number of tokens contributing to each sample’s loss equals the full sequence length L.The paired masks cover the sequence across the two training views.Source: §A.3 · p. 14

Evaluation setup

• Evaluations cover reasoning, knowledge, and code-generation benchmarks using greedy decoding, with GPQA as the exception under 5-shot prompting.All other tasks use zero-shot settings.Source: §A.4 · p. 15
• LM-Eval evaluates non-code tasks, while EvalPlus calculates reliable pass rates for HumanEval and MBPP.Source: §A.4 · p. 15

Inference configuration

• Block size = 32, sub-block size = 8, and parallel decoding is disabled with threshold = 1.Source: §A.4 · p. 15
• The fixed configuration matches training and inference setups to facilitate evaluation of Fast-dLLM v2’s block-wise diffusion capability.Source: §A.4 · p. 15

Case-study design

• Fast-dLLM v2 is examined in representative single-turn and multi-turn dialogue scenarios presented in Tables 5 and 6.Source: B. Case Study · p. 15, Table 5 · p. 16, Table 6 · p. 17

Single-turn Dialogue Scenarios

• In the single-turn math example, the model factors a rational function and solves for undefined x values to determine vertical asymptotes.Source: B. Case Study · p. 15
• In the single-turn code example, the model generates a recursive Python implementation of Tower of Hanoi with an appropriate explanation.Source: B. Case Study · p. 15

Multi-turn Dialogue Scenarios

• In multi-turn dialogue, Fast-dLLM v2 retains context and builds on previous turns for temporal reasoning and step-by-step arithmetic problem solving.The arithmetic example includes intermediate calculations and reports the final result in hours and minutes.Source: B. Case Study · p. 15

Overall findings

• Together, the cases demonstrate mathematical reasoning, code generation, temporal understanding, and contextual coherence across both dialogue settings.Source: B. Case Study · p. 15