Source-linked AI summary
Ceiling-Clipped Acceptance Histograms Indicate Stranded Speed-up in Block-Diffusion Speculative Decoding
Ephrem Wu
TL;DR
The paper asks how to detect and recover acceptance stranded by block-diffusion drafters whose trained horizon clips otherwise acceptable cycles. It uses ceiling-bin histograms to select expansion targets and DBloom to post-train longer blocks, improving committed length across evaluated settings and matching JetSpec comparisons. The main scope boundaries are inference-time widening without retraining and evaluation based on EOS-terminated responses.
Problem
Mean committed length hides whether cycles are limited by rejection or by the drafter’s block boundary, while the ceiling bin reveals the latter.
Method
DBloom reads the original acceptance histogram, then post-trains a B16 drafter at B24 with a short curriculum emphasizing newly exposed positions.
Results
DBloom raises high-ceiling committed length by a median of +0.8 tokens, up to +1.1, and shows no statistically significant loss to JetSpec through 256 nodes.
Takeaways & Limitations
The ceiling bin can rank benchmarks for expected expansion gains before training compute is spent, while effective expansion requires retraining the larger horizon.
Takeaways & Limitations
Naive inference-time widening reduced speed-up in every measured cell, and metrics exclude responses that fail to emit EOS before the generation cap.
Abstract
from arXiv · showhide
Speculative decoding speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, preserving the target's output distribution. High-acceptance block-diffusion drafters such as DFlash and DFlare fill an entire block in one parallel pass. In many cycles, the target accepts the whole block, so the drafter exhausts its trained block horizon before verification fails. We call this unrealized acceptance stranded speed-up. A mean committed length, per prompt or per cycle, hides it, whereas the acceptance histogram exposes it as a spike in the ceiling bin, the fraction of cycles that accept the entire block. We recommend the histogram as a preflight check before spending training compute. Naively widening the block at inference does not recover the speed-up, because once the block outgrows its training size, the drafter's bidirectional attention shifts its distribution even at early positions and erodes front-of-block verification. Instead, we post-train the drafter on a longer block with a short curriculum that emphasizes the newly exposed positions, a method we call DBloom. Expanding the pretrained DFlash and DFlare drafters from block size 16 to 24 across Qwen3-8B and Qwen3-4B targets raises the per-prompt committed length on the high-ceiling benchmarks by a median of +0.8 tokens (up to +1.1). Once continuation fine-tuning precedes expansion, the increase reaches 1.37 tokens. The same expansion also lifts committed length on all seven benchmarks for Gemma-4-12B-IT, a different model family, by a median of +0.41 tokens (Arm A), and the full continuation-then-expand pipeline (Arm B) adds +0.29 to +0.98 tokens over the same B16 drafter. In a prompt-matched comparison against JetSpec, a contemporary tree-based drafter not used in our design, DBloom commits more tokens on every benchmark at tree budgets up to 64 nodes.
1 Introduction
DBloom identifies stranded speed-up from ceiling-clipped acceptance histograms and expands DFlash and DFlare block horizons through limited post-training. Across evaluated targets, expansion improves committed length, while the ceiling-bin signal indicates where gains are expected.
- Speculative decoding preserves the target distribution while drafting several tokens for verification in one target pass.
- A ceiling-bin spike reveals cycles where the target accepts the entire offered block, exposing acceptance hidden by mean committed length.
- Naive inference-time widening does not recover stranded acceptance; larger-block retraining is required.
- Spearman ρ = +0.88 to +0.93 links ceiling-bin mass with acceptance gains from expanding B16 to B24.
- A short horizon-weighted curriculum using about 30K expansion prompts raises high-ceiling committed length by a median of +0.8 tokens, up to +1.1.The expansion corpus is 1.0% of the DFlare budget and 2.0% of the DFlash budget.
- At linear budget 24, DBloom-DFlare shows no statistically significant committed-length loss to JetSpec through its 256-node maximum.
2 Background and related work
Block-diffusion drafters provide parallel alternatives to sequential autoregressive drafters, while tree methods increase acceptance by spending more draft budget. DBloom instead changes the trained block horizon and is evaluated against these complementary approaches.
- EAGLE and EAGLE-3 improve acceptance over original speculative decoders but retain sequential dependence across draft depth.
- DFlash and DFlare are high-acceptance block-diffusion drafters evaluated as DBloom targets, with DFlash weaker than DFlare in the evaluated Qwen3 settings.
- Table 1 reports native B16, naive B24, and trained Arm-A B24 means, with trained B24 above native B16 and native B16 above naive B24 on every benchmark and in the mean.
- A block-8 DFlash drafter fully accepts the block 35.7% of the time on MATH-500, while larger inference blocks require separately trained drafters.
- JetSpec increases committed length by growing a candidate tree to as many as 256 nodes, whereas DBloom retrains a linear block horizon of 24.
3 Why naive block expansion rarely increases speed-up
Inference-time block widening fails because a B16 drafter’s distribution shifts when run at B24, eroding early verification. The resulting speed-up loss contrasts with trained expansion, which is needed to recover acceptance.
- In every measured cell, increasing block size at inference without retraining reduced speed-up.
- On DFlare Qwen3-4B MATH-500, naive B24 reduces speed-up from 6.5× to 3.0×, a 54% drop.
- Naive expansion raises committed length on no cell; DFlare Qwen3-4B MATH-500 falls from 8.8 to 4.1 tokens.
- Running a B16 drafter at B24 shifts its proposal distribution even at early positions, so front-of-block verification erodes and reliable gains require retraining.
4 Method
DBloom uses the original drafter’s acceptance histogram to select expansion opportunities, then post-trains a B16 drafter at B24 with emphasis on newly exposed positions. Two training arms differ only in whether same-horizon continuation precedes expansion.
- DBloom inspects the original acceptance histogram in one evaluation pass before choosing where to expand the drafter.
- 4.1 The acceptance histogram and its ceiling bin: The histogram records per-cycle accepted lengths, with the ceiling bin n = B−1 representing cycles that accept all offered draft tokens.
- 4.1 The acceptance histogram and its ceiling bin: A large ceiling-bin mass indicates block-limited acceptance and identifies domains where expansion can convert clipped cycles into longer accepted prefixes.
- 4.2 Curriculum post-training and the two arms: DBloom initializes from B16 weights, fine-tunes at B24, and learns eight new tail positions while preserving a competent front.
- 4.2 Curriculum post-training and the two arms: A flat-step horizon weighting assigns weight 1 to old positions and a 3× boost to eight new tail positions.
- 4.2 Curriculum post-training and the two arms: Arm A expands the original B16 drafter directly, whereas Arm B applies same-block continuation before the identical B24 expansion.
5 Experimental setup
The experiments evaluate DBloom across a controlled grid of block-diffusion drafters and targets, using consistent greedy decoding and prompt-level committed-length metrics. They compare two training arms, assess expansion costs and controls, and restrict acceptance statistics to EOS-terminated responses.
- Evaluation design: DBloom is evaluated with DFlash and DFlare paired separately with Qwen3-8B and Qwen3-4B targets.The controlled two-by-two grid tests whether findings persist across drafter architectures and target scales.
- Evaluation design: JetSpec provides an out-of-design comparison on Qwen3-8B across its tree-node budgets.DBloom was designed from DFlash and DFlare acceptance histograms, so JetSpec tests transfer beyond those design inputs.
- Evaluation protocol: The evaluation uses greedy decoding at temperature zero, a 16,384-token generation cap, and full benchmark test sets.Benchmarks include GSM8K, MATH-500, AIME21-26, HumanEval, MBPP, LiveCodeBench, and MT-Bench.
- Metric conventions: τ is the prompt-mean committed length, whereas E[n]+1 is the cycle-mean committed length, and both count accepted draft tokens plus one bonus token.The metrics differ in statistical weighting and are reported separately with prompt-level bootstrap confidence intervals.
- Training arms: The two arms start from the same B16 drafter and share the B24 expansion; Arm B additionally applies same-block B16 continuation fine-tuning.The shared expansion isolates the effect of Arm B’s continuation step.
- Metric conventions: Acceptance statistics are computed only on EOS-terminated responses because capped repeated-phrase runaways can inflate near-ceiling acceptance.The most-filtered cell retains 111 of 179 AIME21-26 responses, while speed-up remains a direct timing measurement.
- Training data: The B24 expansion uses 30K prompts from high-ceiling domains, while short-form code is excluded because its low ceiling does not improve with expansion.The expansion corpus is self-distilled, decontaminated with 32-gram matching, and shared by DFlash and DFlare for each target.
- Training cost and controls: Block expansion costs about 1.0%–2.0% of the corresponding full training budgets, and B24 expansion outperforms same-corpus B16 post-training controls.The control adds +0.05 to +0.21 tokens at B16 versus +0.26 to +1.02 for full B24 expansion in the cited Qwen3-8B DFlare setting.
6 Results
Across the evaluation grid, the block-ceiling fraction predicts where B24 expansion recovers committed length, with gains concentrated on high-ceiling benchmarks and transfer to Gemma. DBloom also matches JetSpec in committed length under the tested comparison.
- 6 Results: The ceiling fraction orders benchmarks by expected expansion gain across 35 drafter-benchmark pairs, with high correlations for both arms.The association is descriptive and in-sample across four Qwen cells and one Gemma-4-12B-IT cell.
- 6 Results: The DBloom-DFlare Arm-B B24 drafter clears both B16 baselines on high-ceiling benchmarks while remaining near baseline on low-ceiling benchmarks.The same qualitative pattern appears at both Qwen3-8B and Qwen3-4B scales.
- 6 Results: +0.8 tokens median incremental gain, up to +1.1, is achieved by B24 expansion across 24 high-ceiling comparisons.End-to-end gains over the original B16 have a median of +1.0 tokens, up to +1.7, and all 24 comparisons remain significantly positive after Holm correction.
- 6 Results: Low-ceiling benchmarks provide little expansion headroom, and Arm-B gains there come mostly from same-block continuation rather than B24 expansion.For DFlash Qwen3-4B on MBPP, the expansion change is +0.07 [-0.01, +0.14] (ns), while continuation supplies +0.80 and expansion adds +0.21 on DFlash Qwen3-8B.
- 6 Results: +0.41 tokens median Arm-A gain is obtained on all seven Gemma-4-12B-IT benchmarks, with gains ranging from +0.09 to +0.73.Every interval is above zero, and the ceiling indicator remains applicable within the new model-family cell.
- 6 Results: +0.29 to +0.98 tokens are added by the Gemma Arm-B continuation-then-expand pipeline over the DFlare-Gemma B16 drafter.The within-cell ceiling-indicator correlation is ρ = +0.64, while the combined Arm-B correlation remains ρ = +0.93 over 35 points.
- 6.3 A linear budget of 24 keeps pace with a 256-node tree in committed length: DBloom-DFlare Arm B shows no statistically significant committed-length loss to JetSpec at tree budgets up to 256 nodes.At the largest budget, DBloom is ahead on five of seven benchmarks and has no significant loss on any.
7 Analysis and limitations
The loss-weighting ablation finds negligible movement across six profiles, while the JetSpec comparison supports DBloom’s committed-length parity and the study’s scope remains deliberately narrow.
- 7.1 Loss weighting does not move committed length: 6.75 to 6.79 mean E[n]+1 and 4.88 to 4.91× speed-up span all six loss-weighting profiles.The ablation varies only per-position weighting on Qwen3-8B at B24, with the corpus and other settings fixed.
- 7.1 Loss weighting does not move committed length: The flat tail-position weighting is retained because fancier profiles produce no gain at this scale.The recipe applies a 3× boost to newly exposed tail positions; the ablation was run only on Qwen3-8B.
- 7.1 Loss weighting does not move committed length: The JetSpec forest plot compares paired DBloom-minus-JetSpec committed-length differences across tree budgets, with 95% bootstrap intervals around Arm A and Arm B means.Intervals entirely right of zero indicate significant wins; intervals straddling zero are inconclusive.
- 7.2 Scope: The study covers Qwen3-8B and 4B with two drafter architectures plus one Gemma model-family check, so universal transfer is not established.Prompt-level bootstraps and three Qwen3-8B DFlare Arm-A seeds support the reported comparisons, but the grid remains limited.
8 Conclusion
DBloom expands pretrained block-diffusion drafters through fine-tuning on a longer horizon, targeting newly exposed tail positions while preserving learned behavior on existing offsets. The expansion recipe is shared across DFlash and DFlare and uses a short post-training stage.
- Conclusion: The expansion stage is always a fine-tune initialized from B16 weights rather than a B24 model trained from scratch.The recipe is identical across DFlash, DFlare, and the evaluated targets except for the initialization checkpoint.
- Conclusion: The objective uses target-generated hard labels and a two-level horizon-weighted loss that gives the new tail positions a 3× weight.Training uses self-distilled target responses and a flat weighting scheme over old versus newly added offsets.
- Conclusion: Expansion optimization uses AdamW with gradient clipping and a cosine schedule, while the continuation stage uses a separate higher learning rate.Arm A and Arm B step 2 use 1 × 10^-5 for four epochs; Arm B step 1 uses 5 × 10^-5 for two epochs.
B Loss-weighting ablation, per benchmark
The six loss-weighting profiles produce nearly identical committed lengths and speed-ups, despite substantially different shapes across in-block positions. The final flat-step profile is therefore selected for simplicity rather than superior measured performance.
- Per-benchmark ablation: All six weighting arms remain in the same narrow band on every benchmark, so weighting does not change committed length at this corpus scale.Each arm changes only per-position weighting while holding the corpus and all other settings fixed.
- Per-benchmark ablation: The flat 3× boost assigns weight 1.0 to old offsets and 3× to the eight newly added B24 tail positions.The six profiles differ only in how they shape weights across the block; the flat step is the final recipe.
- Per-benchmark ablation: Figure 4 plots weighting profiles against in-block offset k, with the boundary at k = 16 marking the start of the new tail.Despite their different shapes, none changes committed length in Table 5.
C Data-versus-expansion control
A control separating data quantity from block expansion shows that the wider block, rather than merely post-training on 30K prompts, drives the committed-length gain.
- Data-versus-expansion control: The data-only effect is far smaller than the expansion effect on every benchmark.The comparison uses paired prompt-mean committed length changes over a shared EOS-terminated survivor set.
- Data-versus-expansion control: The B16+30K control keeps block size 16, whereas Arm-A expansion post-trains on the same corpus at B24.Both conditions use the original B16 DFlare drafter as their starting point, isolating the effect of expansion.
D Seed robustness
Committed-length summaries depend on how cycles are weighted: prompt-weighted τ and cycle-weighted E[n]+1 can diverge when prompt generation length correlates with acceptance. This distinction affects throughput interpretation and can make τ overstate or understate realized speed-up.
- Committed-length statistics: Each cycle accepts a prefix of n draft tokens and commits one bonus token, producing n+1 tokens.τ and E[n]+1 both include the bonus token but weight cycles differently.
- Committed-length statistics: τ averages prompt-level committed lengths equally, whereas E[n]+1 averages the same quantities weighted by each prompt’s cycle count.The two statistics are equal only when generation length and acceptance are uncorrelated.
- Committed-length statistics: When long generations draft poorly, E[n]+1 falls below τ because their many low-acceptance cycles dominate the cycle-weighted mean.The reverse occurs when long generations draft well, causing E[n]+1 to exceed τ.
- Committed-length statistics: τ can remain 5.0 while E[n]+1 ranges from 3.2 to 6.8, depending on whether the many-cycle prompt drafts poorly or well.The illustration holds per-prompt behavior fixed while changing which prompt contributes more cycles.
- Committed-length statistics: On MATH-500, E[n]+1 = 9.07 is 0.38 tokens below τ = 9.45 for the DBloom-DFlash Qwen3-4B Arm-B B24 drafter.Reporting only τ would overstate throughput in this negative-covariance case.
- Committed-length statistics: Across published target-drafter pairs, E[n]+1 tracks measured decode-only speed-up closely when target, drafter, and block size are fixed.The paired runs share per-cycle overhead and are evaluated on their own EOS-terminated responses.
F Why naive block expansion lowers per-position acceptance, and by how much the accepted length falls
Naively widening a block changes the drafter’s proposal distribution even at early positions, eroding front-of-block acceptance. Accepted length therefore reflects a trade-off between newly reachable survival and losses on previously reachable positions.
- Bidirectional attention makes widening the block shift proposals at every position, including early positions trained under the smaller horizon.New key columns and query rows alter early positions’ attention mixtures and proposed tokens.
- Per-position acceptance empirically falls after naive expansion, although the distribution shift can preserve or correct individual greedy matches.At already accepted positions, a shift can only preserve or break the match; at rejected positions, it can leave rejection or correct it.
- Accepted length equals uncapped gain G from newly reachable positions minus front erosion L on old positions, so it rises only when G > L.The gain is bounded by survival at the old boundary, making the surviving ceiling central to the expansion outcome.
- Naive expansion raises accepted length on no EOS-terminated cell; DFlare Qwen3-8B AIME21-26 changes by −0.01, while 4B cases fall by approximately the full front erosion.On 8B drafters, boundary survival around 9–16% nearly offsets erosion; on 4B drafters, boundary survival is approximately zero.
- A heuristic geometric-survival model requires both a large surviving ceiling S and a capable drafter β for expansion to increase accepted length.The model is explicitly heuristic because it assumes homogeneous, independent survival at newly exposed positions.
G EOS termination and its effect on committed length
Committed-length measurements are sensitive to EOS termination and capped runaways, which can make atypical responses inflate reported acceptance. The section also reports positive expansion gains and matched-prompt JetSpec comparisons under specified evaluation protocols.
- EOS termination: Capped runaways can inflate committed length because they accumulate many decode cycles, especially when repetitive text is easy for the drafter to predict.Cycle-mean E[n]+1 is more affected than prompt-mean τ because one runaway contributes many cycles but only one prompt.
- EOS termination: Runaways span both high and low acceptance: repeated lines average 14.9 accepted tokens, whereas changing integer lines average 8.0.Neither type is treated as representative of a completed response.
- Expansion gains: Across 24 high-ceiling comparisons, B24 expansion has median Δτ +0.8 over the expanded drafter and +1.0 end to end; all comparisons are positive.Across all seven benchmarks, the median is +0.6, with every Δτ positive and 53 of 56 significant after Holm correction.
- Evaluation scope: Comparisons using a 16,384-token cap are not directly protocol-matched to source results using shorter caps, which especially affects long-generation math.A separate Gemma4 DFlare-versus-DFlash reference also differs in training budget and data.
- JetSpec comparison: Arm B shows no significant committed-length loss against JetSpec at any tested tree budget, while Arm A has one significant loss at MBPP with 256 nodes.At 256 nodes, AIME21-26 and MT-Bench are inconclusive for Arm B rather than demonstrated equal.