PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Parallel Blindness: Information Floors and Model Gaps in Block Drafting

Xinwei Qiang, Xiang Fang, Chang Chen, Yue Guan, Yufei Ding

arXiv:2608.27339v1cs.LGcs.CLcs.IT

TL;DR

Block drafters must propose future tokens before earlier target tokens are realised, so rejection mixes missing path information with imperfect modelling of observable information. The paper separates these losses using an information floor estimated from target rollouts and a model gap above that floor. Across domains and targets, the all-parallel floor reaches 0.286 at the final Qwen3-4B slot, one realised token removes 86–100% of it, and current drafters retain substantial model gaps.

  • Problem

    Accepted length cannot distinguish rejection forced by missing earlier target realisations from rejection caused by imperfect use of available information.

  • Method

    The paper estimates an information floor as the best shared proposal under a specified information constraint and defines the excess observed rejection as the model gap.

  • Results

    Across four domains and targets, the Qwen3-4B all-parallel floor reaches 0.286 at the final slot, one realised token removes 86–100% of it, and model gaps remain large for DFlash and DSpark.

  • Takeaways & Limitations

    The results separate the value of short-range conditioning from proposal quality: one preceding token removes most information loss, while substantial rejection remains above the floor.

  • Takeaways & Limitations

    The measurements average over target rollouts, whereas serving reaches later positions only on paths whose earlier draft tokens were accepted.

Abstract

from arXiv · show

Block drafters propose several tokens in one forward pass, before earlier target tokens are realised. Their rejection mixes two losses: missing within-block path information and imperfect modelling of observable information. Accepted length cannot distinguish them. We separate the two with an information floor, the minimum expected rejection at a specified conditioning order; rejection above this floor is the model gap. Estimating both from target rollouts across four domains, four open-weight targets, and a frontier API target yields three findings. First, the all-parallel floor reaches $0.286$ at the final slot on Qwen3-4B, limiting even the best proposal to $71\%$ per-slot acceptance. Second, one realised token removes $86$--$100\%$ of this floor, a locality also recovered by an independent mutual-information analysis. Third, current drafters remain far above their floors: the final-slot model gap accounts for $43$--$64\%$ of DFlash rejection and $85$--$92\%$ of DSpark's oracle-conditioned rejection. These findings separate the value of short-range conditioning from proposal quality.

1 INTRODUCTION

Block drafting rejection combines an information floor caused by missing within-block target realisations with a model gap caused by imperfect use of observable information. Measuring both from target rollouts shows substantial parallel-blindness costs, strong one-token locality, and large remaining model gaps.

  • Motivation: Accepted length cannot distinguish rejection caused by missing earlier target realisations from rejection caused by imperfect drafter modelling.The paper frames separating these losses as necessary for identifying where further progress can come from.
  • Method: The information floor is the rejection of the best proposal subject to the same information constraint as the drafter.The model gap is the difference between observed rejection and this floor.
  • Method: Target rollouts estimate the floor by exposing alternative continuation paths and finding the best shared proposal, while drafter evaluation on the same prompts estimates its model gap.The procedure is repeated across prompts, domains, and target models.
  • Findings: 0.286 is the Qwen3-4B all-parallel floor at the seventh draft position, limiting maximum per-slot acceptance to 71%.Open-ended chat has higher floors than constrained arithmetic or code across all four targets.
  • Findings: 86–100% of the floor disappears when each position observes only the immediately preceding realised target token, leaving at most 0.041 rejection.At the final draft position, the preceding within-block trajectories have median effective support under two trajectories.
  • Findings: 43–64% of final-position DFlash rejection on three larger targets lies in the model gap, while DSpark’s oracle-conditioned gap accounts for 89–100%.For DFlash on Qwen3-4B, the model gap accounts for 55–67% of per-slot rejection at affected positions.

2 MEASURING INFORMATION FLOORS AND MODEL GAPS

The paper decomposes per-slot rejection into an information floor imposed by available within-block conditioning and a model gap above that floor. It estimates both from target rollouts while distinguishing free-rollout measurement from serving risk.

  • Risk measures: Per-slot rejection risk is the expected rejection over target trajectories, and rejection probability equals total variation distance between target and proposal distributions.Accepted length instead retains joint path dependence through cumulative accept factors and survival weights.
  • Conditioning order: At position k, order-m conditioning exposes the context and the most recent m realised within-block tokens, with order 0 seeing only the context.Order 1 corresponds to a proposal conditioned on the immediately preceding realised token; the full prefix eliminates the within-block information restriction.
  • Information floors: The information floor is the lowest per-slot risk allowed by a specified conditioning constraint, independent of any particular drafter.It is obtained by minimizing expected distance to target conditionals separately for each available information state.
  • Floors and gaps: The model gap is the drafter’s rejection above the information floor: G_k = R_k − T^(m)_k.More conditioning can only lower the floor, so the decomposition compares a drafter with the best proposal under the same information constraint.
  • Estimation: The floor and gap are estimated from target free rollouts by collecting path-conditioned target distributions, solving for the best shared proposal, and evaluating the drafter on the same trajectories.Prompt-level aggregation uses Hájek weights and bootstrap uncertainty intervals; serving risk is reweighted separately by survival weights.

3 HOW MUCH DOES PARALLEL BLINDNESS COST?

The all-parallel information floor grows with draft depth and is higher for open-ended domains, while path branching explains substantial context-to-context variation. A small amount of realised-token conditioning removes most of this cost.

  • Floor magnitude: 0.286 is the order-0 floor at slot 6 on Qwen3-4B, limiting the best all-parallel proposal to about 71% per-slot acceptance there.The floor rises from 0.078 at slot 1, while slot 0 has an exactly zero floor.
  • Domain variation: Open-ended domains consistently have higher floors than constrained arithmetic and code domains across the reported slots.The paper relates this pattern to a wider range of plausible continuations in open-ended prompts.
  • Context heterogeneity: Floor cost is highly uneven across anchors: the top decile contributes 60% at slot 1 and 28% at slot 6.At slot 6, the mean floor is 0.285 versus a median of 0.181, and one quarter of anchors remain below 0.01.
  • Path branching: At slot 6, effective support correlates +0.90 with the floor, ranging from a median of 1.8 trajectories to 18.9 at the ninetieth percentile.The same relationship holds within domains, with correlations from +0.85 to +0.92.
  • Path-conditioned modes: At slot 6, two prototypes remove 43% of the floor and four remove 71%, indicating that path-conditioned target distributions often cluster around a few modes.This oracle-routed analysis measures clusterability rather than deployable drafter performance.

4 HOW MUCH CONDITIONING IS NEEDED?

The order-0 information floor captures rejection caused by missing within-block history, while order-1 conditioning tests how much the immediately preceding realised token removes that cost. Across complementary acceptance and information-theoretic analyses, one token recovers nearly all missing path information.

  • Order-1 conditioning: 86–100% of the order-0 floor is removed by conditioning on the immediately preceding realised token, leaving an order-1 floor of at most 0.041.The comparison uses the same free rollouts and exact total variation; at slot 1, the order-1 floor is exactly zero.
  • Validation: An independent implementation differs by only 0.0032 at slot 6, supporting the robustness of the order-1 floor estimate.The implementation uses different engines, sampled paths, vocabulary coverage, and conditional estimators without shared estimator code.
  • Order-1 conditioning: 94.0% of missing path information is recovered by one realised token, rising to 99.1% with two and 99.8% with four tokens.Across nontrivial slots, one token recovers 92.2–95.3%, while two recover 98.8–99.4%.
  • Why one token suffices: At the median anchor, six-token prefixes have effective support of only 1.8 trajectories, so the preceding token usually identifies the active continuation branch.The measured locality is attributed to limited branching in target continuations.
  • Decomposition: Figure 2 decomposes DFlash risk into T (0) and G, while DSpark’s oracle-conditioned risk is T (1) plus Gpost, with Eexp added for self-conditioning.These components distinguish information floors from model and exposure terms across draft positions.

5 HOW FAR ARE DEPLOYED DRAFTERS FROM THEIR FLOORS?

The deployed drafters remain substantially above the rejection floors imposed by their available information. DFlash’s excess rejection is dominated by its model gap, and DSpark’s oracle-conditioned risk is likewise mostly modelling error rather than irreducible information loss.

  • DFlash: 55–67% of DFlash’s per-slot rejection at slots 1–6 is model gap, exceeding the information floor throughout the block.At slot 0, the floor is zero and the entire rejection risk of 0.136 is model gap.
  • Scope of decomposition: The DSpark decomposition applies for k ≥1; at slot 0, no predecessor exists and the order-0 floor is zero.Finite-sample unmatched predecessor groups cover 98–99% of path mass, limiting reported floor and gap changes to at most 0.02.
  • DSpark: DSpark’s order-1 model gap accounts for 89–100% of oracle-conditioned risk across the block.The oracle supplies the target’s realised predecessor, matching the information available to the order-1 floor.
  • DSpark: At slot 6, DSpark’s oracle-conditioned risk is 0.367 versus an order-1 floor of 0.041, with a 0.214 exposure penalty when using its own predecessor.The exposure penalty measures sensitivity to predecessor errors and is not itself a serving loss.

6 HOW DO FLOORS AND GAPS VARY ACROSS TARGETS AND SCALE?

The floor–gap separation persists across larger open-weight targets and a frontier API target. Across targets, deployed drafters remain well above their floors, while one realised token continues to remove most of the all-parallel floor.

  • Open-weight scaling: 43%–64% of DFlash’s final-slot risk is model gap across Qwen3-4B, Qwen3-8B, Qwen3-14B, and Gemma-4-12B.The repeated pipeline uses the same four domains and 384 anchors for all four targets.
  • Open-weight scaling: 85%–92% of DSpark’s oracle-conditioned final-slot risk is model gap, while its order-1 floor remains below 0.061 across targets.The oracle-conditioned order-1 risk exceeds the order-0 floor on every target despite observing the realised predecessor.
  • Frontier target: One realised token removes 86–90% of the DeepSeek-V4-Pro all-parallel floor across slots, with open-ended domains again showing higher floors.The frontier measurement uses 120 API anchors.
  • Frontier target: Published serving results imply max_k R_k ≳ 0.107 on batch-1 coding and ≳ 0.233 on roleplay, but these bounds use different operating laws from the API floors.The direct frontier gap estimate therefore comes from the slot-0 identity rather than deeper-slot comparisons.

7 HOW DOES PER-SLOT RISK TRANSLATE TO SERVING PERFORMANCE?

Free-rollout per-slot risk overstates the risk encountered during serving because only surviving paths reach deeper slots. Survival weighting, path dependence, and single-slot oracle values therefore change which positions offer the greatest practical improvement.

  • Serving-weighted risk: Serving reweighting lowers final-slot risk from 0.635 to 0.211 for DFlash and from 0.366 to 0.158 for DSpark.Across slots 1–6, the corrections average 0.261 for DFlash and 0.124 for DSpark, growing monotonically with depth.
  • Serving-weighted risk: The serving-weighted risk is nearly flat across slots 1–6 because paths reaching deep slots are systematically easier for the drafter.This contrasts with a free-rollout risk that nearly triples across the block.
  • Path dependence: Joint survival at the final slot is 12.3× the independent estimate for DFlash and 2.63× for DSpark.Accounting for this dependence raises accepted length from 3.40 to 4.57 for DFlash and from 4.39 to 5.23 for DSpark.
  • Intervention value: A prefix-specific best response upper-bounds the accepted-length gain from any deployable same-information change to one slot with other slots fixed.The oracle optimises one slot separately for every prefix while retaining the released proposal elsewhere.
  • Intervention value: From slot 0 to slot 6, model gap grows by 2.6× while single-slot serving value falls by 3.9×.The opposing trends identify different priorities for reducing modelling error versus improving serving performance.

8 DISCUSSION AND RELATED WORK

The paper connects short-range causal refinement to related drafting methods and grounds its decomposition in target-rollout and information-theoretic measurements. These comparisons emphasize that missing path information and model quality are distinct sources of rejection.

  • Related methods: Blockwise methods predict several future tokens in parallel, whereas sequential and feature-autoregressive methods alter the drafting pipeline with causal refinement.The paper situates its floor–gap analysis alongside classical speculative decoding, blockwise methods, sequential heads, and feature-autoregressive drafters.
  • Related methods: One-token locality aligns with existing systems: DSpark finds little difference between Markov and RNN heads, while DeLS-Spec reports accepted lengths of 6.28 and 6.35.Domino and xPress likewise add lightweight causal refinement to parallel drafts, targeting the measured model gap.
  • Related methods: DFlash2’s final-position top-16 recall is 87.8%, while PCTree raises mean accepted length from 10.225 to 11.156 at fixed checkpoint and tree budget.These results are presented as consistent with few-mode continuation structure and predecessor-based mode selection.
  • Implications: DFlash improves accepted length with deeper drafters and target features without changing the product factorisation, placing those gains above an unchanged floor.Its decaying position weights also agree with the paper’s 3.9× first-to-last serving-value ratio.
  • Information-theoretic reading: The mutual-information identity equates expected blind-versus-full log-loss difference with I(Yk; Z<k | X).The blind observer marginalises the unknown prefix, while the full observer conditions on the realised within-block prefix.
  • Measurement scope: The estimator’s identity remains valid under truncation when the trajectory and scoring conditionals use the same truncated autoregressive law, but not as mutual information of the raw target.This distinction limits how the diagnostic should be interpreted.
  • Experimental scope: The study uses Qwen3-4B as its primary target with block size γ = 7, evaluates released DFlash and DSpark checkpoints, and repeats target-side measurements on larger targets and a frontier API.The four domains are gsm8k, mbpp, alpaca, and arena-hard; arena-hard has substantially longer contexts than the other domains.
  • Measurement design: Anchors are prompt–position pairs grouped by six context-length buckets and three response positions, with H´ajek aggregation preserving the intended block-weighted estimand.The independent recovery diagnostic uses a separate eligibility-based sampling design and fresh paths.

C.3 FINITE-SAMPLE SENSITIVITY

Finite-sample checks find small sensitivity to path count, solver fitting, and recovery thresholds, while truncation changes the floor and risk differently. The reported decompositions therefore retain explicit numerical and sampling qualifications.

  • Path-count sensitivity: Both floors move by at most 0.0022 when the path count changes fourfold from M = 256 to M = 1024.At M = 1024, order-1 importance-sampling effective sample sizes range from 693 to 924 across slots.
  • Solver sensitivity: Split-half scoring changes T (0) by at most 0.006 at any slot and by 0.003 after pooling, with no consistent sign.The order-0 effective sample size is exactly M because it uses uniform path weights.
  • Recovery sensitivity: Increasing M from 64 to 512 moves pooled ρ1 from 93.5% to 93.9%, while ρ2 and ρ4 vary by at most 0.5 percentage points.Varying the effective-sample-size threshold changes ρ1 by at most 1.2 percentage points.
  • Vocabulary truncation: Top-K truncation bounds floor error by the expected target mass outside the returned support, and top-20 versus top-256 calibration differs by at most 1.7 × 10^-5.This calibration supports the frontier measurement despite the API’s top-20 cap.
  • Training-law sensitivity: Under the drafter training law, slot-6 truncation reduces the information floor by 42% and drafter risk by 11%, increasing G/R to 67%.The model gap remains dominant under this sampling law; the two proposal-warping implementations differ in risk by at most 0.008.
  • Uncertainty and resolution: At slot 6, Qwen3-4B’s 95% prompt-bootstrap interval for G/R is [49.5, 60.7]%, with corresponding intervals reported for three larger targets.The primary numerical floor diagnostics treat approximately 10^-3 as estimator resolution because the order-1 slot-1 floor is zero by identity.

D.1 CONTINUATION CONCENTRATION

Continuation distributions usually concentrate around a few target paths, and effective support strongly tracks the all-parallel floor. Oracle-routed and conditioning analyses therefore examine how a small number of modes and short prefixes structure the remaining rejection.

  • Continuation concentration: The median collision-equivalent effective support is 1.8 trajectories, while the ninetieth percentile is 18.9 across target rollouts.Both values are far below M = 1024, the no-collision value.
  • Continuation concentration: At slot 6, log effective support correlates +0.90 with T (0), remaining high within each domain at +0.845 to +0.920.On the raw scale, the corresponding overall correlation is +0.53, ranging from +0.53 to +0.65 within domains.
  • Oracle-routed K-median: The K-median diagnostic lets an oracle route realised paths to K proposal distributions, asking whether target conditionals cluster around recurring modes.At K = 1, the objective exactly equals T (0); a concrete multi-candidate drafter would additionally need routing and verification rules.
  • Oracle-routed K-median: Each returned K-median solution is feasible, so its objective upper-bounds the global optimum and its reported reduction is a lower bound.Agreement across restarts does not certify global optimality; the weighted mean restart spread is at most 0.034, though a thin tail reaches 0.48.
  • Locality of path information: The ρ1 estimates remain 93.9%, 95.1%, and 94.0% as the retained eligible-cell population changes from 2234 to 3234 to all 3490 cells.This sensitivity analysis supports stable locality under substantially different retention thresholds.
  • Locality of path information: One realised token recovers 92.2–95.3% of missing path information at every measured slot, and two tokens recover at least 98.8%.Pooled recovery is 94.0% for one token, 99.1% for two, and 99.8% for four.
  • Floor–gap decomposition: The decomposition compares DFlash with the order-0 floor and separates DSpark’s order-1 model gap from exposure caused by its own predecessor.Components are aggregated before rounding, so displayed sums can differ by 0.0001.
  • Floor–gap decomposition: The split-half predecessor groups cover 0.98–0.99 of path mass, so unmatched mass can change either component by at most 0.02.The displayed rows therefore form an approximate finite-sample decomposition.

F.1 SERVING-RISK REWEIGHTING

The analysis separates free-rollout risk, serving-weighted risk, and oracle improvement within each drafter’s information class. It also quantifies dependence among accept factors and distinguishes architectural limits from model-specific headroom.

  • Serving-risk reweighting: Serving risk reweights free-rollout paths by their probability of reaching each slot, so rarely reached anchors contribute proportionally less.The free and serving estimates use the same paths, enabling paired comparisons within each anchor.
  • Serving-risk reweighting: Slot 0 is omitted because its free and serving risks are identical when W−1 = 1.
  • Dependence among accept factors: 4.574 and 5.232 are joint-survival accepted lengths for DFlash and DSpark, versus 3.397 and 4.386 from multiplying marginal acceptance rates.Both calculations use the same recorded paths; the difference retains dependence among accept factors.
  • Frontier-scale serving bounds: For official DSpark, accepted length about 5.0 at γ = 7 implies r ≳0.107, while 2.67 for roleplay at γ = 6 implies r ≳0.233.These bounds come from numerically inverting the survival inequality.
  • Oracle optimisation: The prefix-specific oracle can choose separate actions by measured prefix and predecessor cell, providing an upper bound on improving one slot within the information class.Cross-fitting estimates this gain using held-out paths.
  • Oracle optimisation: 2.20× for DFlash and 1.98× for DSpark are sweep gains relative to isolated gains, reflecting interactions through later-slot reach and earlier continuation value.The sweep does not estimate trainable headroom or the joint optimum.
  • Architectural slack: The information floor covers all mappings with the available information, while architectural slack is the cost of restricting proposals to a fixed architecture class.Realised training gains may be smaller than the best improvement available within that class.

G.2 ARCHITECTURAL SLACK CAN BE LARGE

A parity construction shows that architectural restrictions can create substantial risk even when the information floor is zero. The example separates representational limits from missing information and notes that exact-computation separations do not quantify expected TV magnitude.

  • Parity construction: 1/2 is the maximum expected probability assigned to the correct token for proposal classes independent of the parity label.The constant proposal assigning 1/2 to each token attains equality.
  • Parity construction: The constant proposal has risk and model gap both equal to 1/2 although the information floor is zero.
  • Formal architectural limits: Parity is excluded from AC0 but included in TC0, establishing possible architectural slack in the cited formal models.These separations do not determine the expected total-variation magnitude under the measured law.
  • Barycentre complexity: The acceptance-optimal blind proposal is a TV barycentre over the continuation family, so representing each conditional does not imply representing the aggregate.
Loading 2608.27339v1…