Source-linked AI summary
DFlow: Enabling Verifier Information Flow in Block Diffusion Speculative Decoding
Yaojie Zhang, Linfeng Zhang, Bin Cui, Xupeng Miao
TL;DR
Block diffusion speculative decoding drafts future tokens in parallel, but existing methods discard verifier representations at rejected positions, losing information that could guide later drafting. DFlow relays those representations across rounds and improves average acceptance length over DFlash across Qwen3 model scales.
Problem
Block diffusion methods discard verifier representations for rejected suffix positions, preventing that information from carrying into subsequent drafting rounds.
Source: §1 · pp. 1–2Method
DFlow relays rejected-suffix verifier representations to unresolved positions in later drafting rounds and trains this reuse with multi-round self-conditioning.
Source: §1 · p. 2, §4.2 · p. 6Results
Under greedy decoding, DFlow improves average acceptance length over DFlash by 10.4%, 13.2%, and 13.4% on Qwen3-1.7B, Qwen3-4B, and Qwen3-8B, respectively.
Source: §1 · p. 2Takeaways & Limitations
Across Qwen3 models and diverse benchmarks, verifier information beyond the rejection boundary can be reused to improve subsequent drafting and decoding efficiency.
Source: §7 · p. 9The approach rests on the hypothesis that verifier representations at rejected positions can provide useful guidance for subsequent drafting.
Source: §4.1 · p. 4
Abstract
from arXiv · showhide
Block diffusion speculative decoding improves LLM inference efficiency by proposing a block of future tokens in parallel and verifying them with a single forward pass through the target model. However, existing methods retain only the accepted prefix and discard the rejected suffix, preventing the computation spent on these positions from benefiting subsequent drafting rounds and forcing the drafter to repeatedly reconstruct representations for future tokens from scratch. We observe that rejection only determines whether a proposed token can be committed, while the verifier representations at rejected positions can still provide useful information for subsequent predictions. Based on this observation, we propose DFlow, a simple yet effective framework that enables verifier information to flow across drafting rounds. DFlow reuses the hidden states produced by the target verifier for the rejected suffix to guide subsequent drafting without additional target computation. To effectively learn this information flow across drafting rounds, we introduce a self-condition train strategy that feeds verifier representations from earlier predictions back into subsequent predictions. Experiments on Qwen3 models across diverse benchmarks demonstrate that DFlow consistently improves draft quality and acceptance length over DFlash.
1 INTRODUCTION
Block diffusion speculative decoding drafts future-token blocks in parallel, but discarded verifier representations at rejected positions create an information discontinuity between rounds. DFlow reuses those representations to guide subsequent drafting and improves acceptance length over DFlash.
1 INTRODUCTION
- Block diffusion speculative decoding predicts future-token blocks in parallel, but existing methods discard rejected suffixes and their intermediate representations between drafting rounds.Overlapping future positions are then reset to mask embeddings and reconstructed from scratch.Source: §1 · p. 1
- Rejection determines whether a proposed token can be committed, but the verifier representation at that position may still contain useful continuation information.The rejected suffix has already been processed by the target model before verification is finalized.Source: §1 · p. 2
- DFlow carries verifier hidden states from rejected suffix positions forward to guide subsequent drafting, without an additional target-model forward pass.Multi-Round Self-Conditioned Training unfolds consecutive drafting and verification rounds to teach this information reuse.Source: §1 · p. 2
- Under greedy decoding, DFlow improves average acceptance length over DFlash by 10.4%, 13.2%, and 13.4% on Qwen3-1.7B, Qwen3-4B, and Qwen3-8B, respectively.Figure 1 compares methods on Qwen3-1.7B under greedy decoding.Source: §1 · p. 2, Figure 1 · p. 2
2 RELATED WORK
Related work reduces speculative-decoding overhead through parallel block drafting and improves draft quality through richer representations or dependency modeling. DFlow instead reuses verifier representations beyond the rejection boundary to directly improve later drafts.
2 RELATED WORK
- DFlash reduces sequential drafting overhead by generating an entire draft block in one draft-model forward pass, making draft quality a key bottleneck to further speedup.DFlare combines target-layer features for different draft depths, while Domino adds a lightweight causal head to refine parallel predictions.Source: §2 · p. 3
- Other block-diffusion methods strengthen draft quality through dependency modeling, including DSpark, DFly, and DFlash 2’s lightweight dynamic convolutions.These approaches retain parallel drafting while refining within-block dependencies.Source: §2 · p. 3
- Unlike prior reuse mechanisms that precompute or selectively perform later operations, DFlow propagates verifier representations beyond the rejection boundary to improve subsequent draft predictions directly.Earlier approaches reused draft predictions, routing decisions, hidden states, attention patterns, or KV importance for later computation.Source: §2 · p. 3
3 PRELIMINARIES
DFlash drafts future tokens in parallel using target-model context, then verifies proposals by exact matching. Only the accepted prefix is committed; rejected positions that overlap the next drafting window are discarded and redrafted from scratch.
3 PRELIMINARIES
- DFlash drafts K future tokens in parallel, conditioning its lightweight drafter on a fused representation of hidden states from selected target layers.Those hidden states are concatenated and projected, then injected into each draft layer’s key and value projections.Source: §3 · p. 3
- Future positions start from mask embeddings and are jointly predicted, including positions that may overlap with positions processed in the preceding round.The committed context supplies target information, but overlapping future positions are reconstructed from mask embeddings.Source: §3 · p. 3
- Exact-match verification checks draft proposals from left to right; a_r counts consecutive proposals matching target predictions before the first mismatch.The verified block is partitioned at that rejection point.Source: §3 · p. 4
- The matching proposals form the accepted prefix, while the remaining rejected suffix is discarded after verification, even when it overlaps the next drafting window.Those future positions are consequently drafted from scratch again in the next round.Source: §3 · p. 4
4 DFLOW
DFlow carries verifier representations from rejected suffixes into later drafting rounds, addressing the loss of information across the rejection boundary. It aligns and conditions those representations for reuse, then trains the drafter with multi-round self-conditioning.
4 DFLOW
- DFlow reuses rejected-suffix verifier representations at corresponding positions in later drafts, avoiding an additional target forward pass.The representations are transformed into the drafter’s hidden space and aligned with the tokens they predict.Source: §4 · p. 4, §4.2 · pp. 4–5
4.1 INFORMATION DISCONTINUITY ACROSS DRAFTING ROUNDS
- Rejection governs token commitment, but the verifier representations for rejected positions may still guide subsequent drafting.Those representations capture contextual information from verification that would otherwise be discarded.Source: §4.1 · p. 4
4.2 VERIFIER INFORMATION RELAY
- DFlow projects selected target-layer states into the drafter’s hidden space and shifts them so verifier and drafter representations correspond to the same predicted token.The reusable suffix states are jointly projected with target features required by DFlash, adding only a small number of positions.Source: §4.2 · p. 5
- A lightweight MLP uses the target correction and first rejected proposal to set channel-wise scaling and correction for each relayed representation.This conditions the relay on the semantic change at the rejection boundary.Source: §4.2 · p. 5
4.3 MULTI-ROUND SELF-CONDITIONED TRAINING
- Multi-Round Self-Conditioned Training unfolds consecutive draft-and-verify rounds, relaying rejected-suffix states so later rounds learn prediction with verifier information.The initial round uses mask embeddings; subsequent reject-conditioned rounds share drafter parameters, with k = 2 such rounds used in practice.Source: §4.3 · p. 6
- Training transitions locate the next anchor by matching proposals to the target trajectory, while relayed states remain tied to actual proposals; losses across rounds jointly train the shared drafter.The target stays frozen, and verifier representations and transitions are detached from the computation graph.Source: §4.3 · pp. 6–7
- Table 1 reports average acceptance length across evaluated benchmarks in low-concurrency settings for greedy decoding and sampling.The caption specifies T = 0 for greedy decoding and T = 1 for sampling.Source: Table 1 · p. 7
5 EXPERIMENTS
DFlow improves acceptance length over DFlash across Qwen3 scales and decoding settings, while also increasing throughput under concurrent serving. Ablations attribute the primary gain to verifier information relay, with only 1.2% runtime overhead.
5.1 EXPERIMENTAL SETUP
- Evaluation covers Qwen3-1.7B, Qwen3-4B, and Qwen3-8B on mathematical reasoning, code generation, and open-ended dialogue benchmarks.Training uses 80K ShareGPT samples; multi-round self-conditioned training uses k = 2 reject-conditioned rounds after initial mask prediction.Source: §5.1 · p. 7
5.2 MAIN RESULTS
- Acceptance length rises over DFlash across all three Qwen3 scales under greedy and sampling decoding.Greedy: 3.56→3.93, 3.71→4.20, and 3.66→4.15; sampling: 2.99→3.26, 3.15→3.54, and 3.10→3.45 for Qwen3-1.7B, Qwen3-4B, and Qwen3-8B, respectively.Source: §5.2 · p. 8
- DFlow achieves higher throughput than DFlash across Qwen3-4B and Qwen3-8B and concurrency levels from 1 to 32.Table 2 reports absolute TPS for baseline rows and TPS with speedup subscripts for other entries.Source: §5.2 · p. 8, Table 2 · p. 8
5.3 ABLATION STUDIES
- Verifier relay raises acceptance length from 3.66 to 3.93, while multi-round mask training alone raises it from 3.56 to 3.67.The relay-disabled result closely matches the multi-round mask-training result, indicating verifier reuse is the primary source of DFlow’s gain.Source: §5.3 · pp. 8–9
- Verifier information relay adds 0.51 ms of latency, or 1.2% overhead over DFlash.The added computation mainly processes reusable rejected positions and applies the lightweight relay module; draft execution and target verification remain dominant.Source: Figure 3 · p. 8, §5.3 · p. 9
6 COMPATIBILITY WITH DOMINO
DFlow combines with Domino by relaying verifier information across drafting rounds, while online rollout aligns the rejected suffix used for relay with inference. On Qwen3-1.7B at T = 0, the combined method reaches an average acceptance length of 4.50, above Domino’s 4.02.
6 COMPATIBILITY WITH DOMINO
- DFlow propagates verifier information across drafting rounds, whereas Domino progressively refines predictions across positions within each round.The methods address distinct dimensions of drafting and can be combined.Source: §6 · p. 9
- On Qwen3-1.7B at T = 0, DFlow with Domino reaches average acceptance length 4.50, compared with 4.02 for Domino alone.Without online rollout, average acceptance length drops to 2.22.Source: §6 · p. 9
- Online rollout determines the rejected suffix and verifier representations used for relay, while teacher forcing remains the objective for optimizing Domino.Teacher-forced later-position predictions are more accurate and can make the rejected suffix overly optimistic relative to inference.Source: §6 · p. 9
7 CONCLUSION
The conclusion presents DFlow as a way to reuse verifier representations from rejected draft positions across drafting rounds. Qwen3 experiments report consistent gains over DFlash and compatibility with Domino.
7 CONCLUSION
- DFlow propagates verifier representations from the rejected suffix to subsequent unresolved positions and uses Multi-Round Self-Conditioned Training to learn that reuse without extra target forward passes.Source: §7 · p. 9
- Experiments on Qwen3 models across diverse benchmarks show consistent gains over DFlash in acceptance length and decoding efficiency.Source: §7 · p. 9
- The conclusion reports that DFlow remains compatible with stronger drafting methods such as Domino.Source: §7 · p. 9
A.1 IMPLEMENTATION DETAILS
The implementation details specify an EAGLE-3 checkpoint, training configuration, and a shared generation limit for evaluation.
A.1 IMPLEMENTATION DETAILS
- For EAGLE-3, the experiments use the released model checkpoint provided by AngelSlim2.Source: §A.1 · p. 13
- DFlash and DFlow are trained on ShareGPT for 6 epochs with a learning rate of 6 × 10−4.Source: §A.1 · p. 13
- All methods use a maximum of 2048 newly generated tokens for evaluation.Source: §A.1 · p. 13