PaperlayerView exact revision on arXiv

Source-linked AI summary

Denoising Blocks, Not Tokens: Efficient Compressed Continuous Diffusion with Branching Token Realization

Xinsong Feng, Peng Du, Zhizhuo Yang, Daniel M. Bikel, Jiayun Wang, Haipeng Chen

arXiv:2610.09311v1cs.LGcs.AI

TL;DR

Token-level diffusion still processes a state for every output token at each denoising step, limiting sequence-length savings. BLD compresses tokens into block latents, generates them in conditioned groups, and decodes spans through parallel local branches; it improves efficiency against the reported baselines, while long-range coherence remains challenging.

  • Problem

    Token-level DLMs process a state sequence as long as the output at every denoising step, limiting sequence-length savings despite parallel generation.

    Source: §1 · p. 1

  • Method

    BLD compresses token spans into block latents, generates latents in groups conditioned on previously generated latents, and decodes spans through parallel local autoregressive branches.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    On the same GPU, BLD reduces generation FLOPs by more than 80× and raises throughput by more than 6× versus similarly sized ELF-L; versus AR, throughput is more than 6× higher and latency more than 4× lower.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Compressed latent diffusion can substantially improve long-sequence generation efficiency, although long-range coherence remains challenging.

    Source: §5 · p. 9

    BLD performs worse than the strongest token-level continuous DLMs on MAUVE and Gen-PPL, and evaluation covers only unconditional generation on OpenWebText.

    Source: §5 · p. 9

Abstract

from arXiv · show

Diffusion language models (DLMs) generate text through iterative parallel refinement, offering the potential for higher throughput than autoregressive (AR) decoding. However, most DLMs still maintain one generative state per token, so every denoising step processes a state sequence as long as the output sequence, limiting the throughput gains from parallel generation. Continuous DLMs provide an additional degree of freedom: a single continuous state can represent multiple tokens, allowing diffusion to operate on a much shorter latent sequence. We introduce \emph{Branching Latent Diffusion (BLD)}, which exploits this flexibility by compressing a 1024-token sequence into only 64 block latents, a $16\times$ reduction. BLD combines latent compression with \emph{branching token realization}, where each latent is decoded by a local AR branch and all branches run in parallel. Because strong compression makes joint latent generation difficult, BLD generates the latents in groups, conditioning each group on previously generated latents. In end-to-end evaluation on the same GPU, BLD reduces generation FLOPs by more than $80\times$ and increases throughput by more than $6\times$ relative to the similarly sized ELF-L baseline. Compared with the AR baseline, BLD achieves more than $6\times$ higher throughput and more than $4\times$ lower latency. Despite the compression, BLD maintains competitive local fluency and diversity, although long-range coherence remains challenging. Overall, BLD shows that moving diffusion from token-level states to compressed latent sequences can substantially improve the efficiency of long-sequence generation.

1 INTRODUCTION

Token-level diffusion still processes a state for every output token, while continuous states can represent multiple tokens. BLD uses compressed block latents, grouped generation, and parallel token branches to improve generation efficiency, though long-range coherence remains challenging.

1 INTRODUCTION

  • Token-level DLMs process the full output-length sequence at every denoising step, while strong compression makes joint latent generation harder and can leave long-range coherence challenging.Continuous states can represent multiple tokens, but fully parallel diffusion over strongly compressed latents preserves local fluency while struggling with long-range coherence.Source: §1 · pp. 1–2
  • BLD compresses token sequences into block latents and decodes each latent’s token span with a local autoregressive branch, with all branches running in parallel.The denoiser operates on K latent states, while token-decoding depth falls from L to approximately L/K.Source: §1 · p. 2
  • To make strongly compressed latent generation more tractable, BLD denoises groups jointly while conditioning each group on previously generated latents.This replaces a difficult joint generation problem with smaller conditional diffusion problems while retaining within-group parallelism.Source: §1 · p. 2
  • On the same GPU, BLD reduces generation FLOPs by more than 80× and raises throughput by more than 6× versus similarly sized ELF-L; versus AR, throughput is more than 6× higher and latency more than 4× lower.Single-sequence latency is slightly higher than ELF-L because grouped sampling adds sequential depth at the latent level.Source: §1 · p. 2

2 BRANCHING LATENT DIFFUSION

BLD compresses token sequences into block latents, generates those latents in conditioned groups, and realizes token spans with parallel local autoregressive branches. Its design combines global modeling in a shorter latent sequence with within-span token generation and staged training.

2 BRANCHING LATENT DIFFUSION

  • BLD combines token-to-block fusion, grouped continuous diffusion over block latents, and branching token realization to model globally in compressed space and decode short spans.Latent groups are generated across groups, then the complete latent sequence is expanded into token spans decoded in parallel.Source: §2 · p. 2, Figure 1 · p. 3

2.1 TOKEN-TO-BLOCK FUSION

  • 16× compression maps 1,024 tokens to 64 block latents; dynamic chunking sets contiguous spans and attention pooling aggregates their contextualized token representations.Each latent corresponds to a contiguous token span and is designed to capture local content and global context for consistency across blocks.Source: §2.1 · p. 3

2.2 GROUPED LATENT FLOW PRIOR

  • Fully parallel latent diffusion can lose sequence-specific content under strong compression; grouped denoising generates each G-block group jointly while conditioning on a previously generated clean prefix.Group size trades off parallelism and modeling difficulty; the main model uses G = 8.Source: §2.2 · p. 4

2.3 BRANCHING TOKEN REALIZATION

  • After latent generation, K local autoregressive branches decode their associated spans in parallel, reducing token-decoding depth to max_i m_i, approximately L/K for balanced blocks.Each branch cross-attends to the latent sequence; branches do not condition on tokens generated by other branches.Source: §2.3 · p. 5

2.4 TRAINING

  • Training separates block autoencoding, decoder adaptation to prior-generated latents, and grouped-prior learning; contiguous latent masking encourages cross-block information sharing during autoencoding.The frozen encoder and decoder support teacher-forced parallel training of groups that are generated autoregressively at inference.Source: §2.4 · pp. 5–6

3 EXPERIMENTS

The experiments evaluate BLD’s generation quality, computational cost, and runtime against diffusion and autoregressive baselines, then examine group size and decoder adaptation. BLD combines large FLOP and throughput gains with competitive quality, while group size and decoder adaptation affect the quality–efficiency trade-off.

3.1 EXPERIMENTAL SETUP

  • The evaluation uses 1024-token sequences compressed into 64 latents; comparisons include continuous and discrete DLMs and an AR baseline, with runtime measured on the same GPU.ELF quality comparisons are indicative rather than fully controlled because tokenizer and training budget differ; AR uses the same data and token budget as BLD.Source: §3.1 · p. 6

3.2 MAIN RESULTS

  • BLD (G = 8) uses 1.1 TFLOPs per sequence versus ELF-L’s 96.4 (88× fewer) and reaches 9.4 sequences/s versus ELF-L’s 1.4 (6.7×) on H100.Throughput is 6.3× AR; batch-1 latency is 2.67 seconds versus 2.25 for ELF-L and 12.57 for AR.Source: Table 1 · p. 7, Table 2 · p. 7, §3.2 · pp. 7–8
  • At G = 8, BLD achieves Gen-PPL 26.6 and entropy 5.17; its MAUVE of 0.82 is below ELF-M, ELF-L, and AR at 0.93, 0.91, and 0.92.The fully parallel variant has lower Gen-PPL (24.6) but also lower entropy (5.08) and MAUVE (0.77).Source: §3.2 · p. 7, Table 1 · p. 7

3.3 EFFECT OF GROUP SIZE

  • At G = 8, latency is roughly halved versus G = 4 and computation falls from 1.31 to 1.10 TFLOPs, while Gen-PPL rises from 24.7 to 26.6.Increasing G further reduces latency and computation but raises Gen-PPL and lowers entropy; G = 1 has the worst Gen-PPL despite its largest clean prefix.Source: §3.3 · p. 8, Table 3 · p. 8
  • Target-specific prefix conditioning at G = 8 raises distinctness from approximately 0.22 to 0.33 and reduces hard-loop rate from 14.5% to 4.3%, without increasing inference cost.The main model uses target-specific conditioning.Source: §3.3 · p. 8

3.4 EFFECT OF DECODER ADAPTATION

  • With the fixed G = 8 prior, decoder adaptation on G = K reduces Gen-PPL from 127 to 26.6 and raises MAUVE from 0.67 to 0.82.Further adaptation on G = 8 slightly improves reconstruction NLL but does not improve generation quality.Source: §3.4 · p. 9, Table 4 · p. 9

4 RELATED WORK

Prior language-generation work includes token-level continuous and discrete diffusion, compressed latent diffusion, and block- or patch-level modeling. BLD combines compressed continuous latent denoising with branching token realization.

Continuous diffusion language models

Discrete and block diffusion language models

Latent and compressed diffusion for language

  • Cosmos studies compressed, noise-robust representations, with strongest results on shorter sequences and more moderate compression; BLD targets 16× compression over 1024 tokens (Meshchaninov et al., 2025).Source: §4 · p. 9

Block- and patch-level language modeling

5 CONCLUSION

BLD moves diffusion to compressed block latents and uses grouped generation with parallel local AR branches. The paper reports efficiency gains while identifying quality, evaluation, and long-range-coherence limitations.

5 CONCLUSION

  • BLD reduces generation FLOPs and improves end-to-end batch throughput over size-matched token-level continuous DLM and AR baselines, but long-range coherence remains challenging.It also reduces single-sequence latency relative to the AR baseline.Source: §5 · p. 9
  • BLD performs worse than the strongest token-level continuous DLMs on MAUVE and Gen-PPL, and evaluation is limited to unconditional OpenWebText generation.Comparisons are not training-matched, latency gains do not establish a single-sequence advantage over all baselines, and only one model scale, sequence length, and compression ratio are studied.Source: §5 · pp. 9–10

A.1 UNDERSTANDING COMPRESSED LATENT DIFFUSION

  • Better reconstruction does not consistently yield better generation; representation quality must be assessed alongside the difficulty of modeling the latent distribution.Source: §A.1 · p. 14
  • Under squared-error denoising, conditional-mean predictions can lie between valid latent states and decode poorly, motivating more informative conditioning.Source: §A.1 · p. 14
  • At G = 8, target-specific access to clean context substantially improves grouped generation over a shared prefix summary.The findings motivate position-specific access to the committed prefix.Source: §A.1 · p. 14

A.2 RECONSTRUCTABILITY AND GENERATABILITY

Under compression, reconstruction quality and generatability diverge: better reconstructions can accompany harder prior generation. Fixed PCA can improve generation at a distortion cost, while simple pooling becomes unreliable at higher compression.

A.2 RECONSTRUCTABILITY AND GENERATABILITY

  • At two tokens per latent, clean reconstruction stays essentially unchanged while midpoint recovery worsens; source recovery is not unconditional generation quality.Source: §A.2 · p. 14, Table 5 · p. 14
  • At eight and sixteen tokens per latent, simple pooling no longer reconstructs text reliably, confounding representation loss with denoising difficulty.Two- and four-token settings separate the effects more clearly.Source: §A.2 · p. 14
  • At rank 192, oracle PPL rises from 30.56 to 55.66 while generated PPL falls from 554.19 to 393.98; distinctness rises from 0.466 to 0.476.The PCA rate scan uses a frozen block-latent autoencoder and fresh priors.Source: §A.2 · p. 15, Table 6 · p. 15
  • Rank 192 offers a better balance between representation distortion and latent generation error, while rank 128 discards too much information for useful reconstruction.An ambient-dimensional control suggests the improvement primarily comes from changing retained information, not reducing denoiser input dimensionality.Source: §A.2 · p. 15
  • Learned and stochastic bottlenecks improve reconstruction but produce worse generations than fixed PCA representations.For the learned 192-dimensional code, oracle PPL improves across 5k, 10k, and 15k steps while generated PPL rises from 2201.84 to 3165.75.Source: §A.2 · p. 15
  • Nominal dimensionality and local decoder robustness alone do not measure generative difficulty; the trade-off includes both reconstruction distortion and how reliably the prior generates encoded information.Source: §A.2 · pp. 14–15

A.3 CONDITIONAL AMBIGUITY UNDER COMPRESSION

The probes examine how compressed latents behave when conditions are ambiguous and whether further denoising recovers the source document. Midpoint interpolation can decode incoherently even when whole-block mixing remains usable.

A.3 CONDITIONAL AMBIGUITY UNDER COMPRESSION

  • Midpoint interpolation between document latents decodes into incoherent text, while small perturbations toward another document are largely harmless.The probe uses 64 held-out documents with the encoder and decoder fixed; at a = 0.5, PPL is close to the prior’s free-running value of 682.Source: §A.3 · pp. 15–16
  • Whole-block transplants decode donor blocks at 99.9–100% exact match while untouched blocks remain at 99.94%.The authors associate interpolation failure more closely with intermediate states within individual blocks than with mixing blocks from different documents.Source: §A.3 · p. 16
  • Ambiguous midpoint conditions produce intermediate predictions that the decoder interprets poorly, whereas conditions identifying a realization support accurate predictions.The memorization diagnostic distinguishes midpoint behavior from near-exact recovery of memorized plans, which does not generalize to unseen documents.Source: §A.3 · p. 16
  • Additional denoising can sharpen predictions without improving source accuracy, and at low noise it can reduce both source fidelity and generation quality.For unconditional generation, recovering the particular source document is unnecessary; sharper predictions and source recovery are distinct outcomes.Source: §A.3 · p. 16
  • Sampler changes can move generation along a quality–diversity frontier but do not add observed context.Table 8 reports a development-stage, 256-token-window diagnostic comparing a block-autoregressive latent prior with a fully parallel diffusion control, not full-document quality.Source: §A.3 · p. 16, Table 8 · p. 17

A.4 THE ROLE OF CONDITIONAL STRUCTURE

These experiments distinguish the availability of clean context from how each target accesses and uses it. Clean-prefix conditioning improves cross-block structure, while conditioning architecture and group size affect the remaining failures.

A.4 THE ROLE OF CONDITIONAL STRUCTURE

  • Clean-prefix block-autoregressive generation produces stronger cross-block dependence and richer document-specific content than fully parallel generation over uncertain blocks.Fully parallel generated latents retain substantially less cross-block structure than real latent sequences, despite cross-block attention.Source: §A.4 · p. 17
  • The block-autoregressive advantage in Table 8 is a short-window diagnostic, not a full-length quality claim.Inspected 1024-token samples show coherence deteriorating after roughly token 512, with tails entering name, citation, or byline patterns; this is partly traced to training-corpus packing without random-offset cropping.Source: §A.4 · p. 17
  • Group size trades off parallelism against unresolved structure: larger groups reduce sequential stages but require jointly recovering more uncertain latent blocks.Clean prefixes provide conditioning information, but the architecture determines how that information reaches each target; access and effective use remain distinct challenges.Source: §A.4 · pp. 17–18
  • A shared prefix representation becomes unstable at G = 8 and G = 16, where generation often collapses near group boundaries into repeated or low-diversity patterns.The training loss changes only modestly despite these failures, which are poorly reflected by the flow objective.Source: §A.4 · p. 17
  • At G = 8, target-specific queries raise distinctness from approximately 0.22 to 0.33 and lower hard-loop rate from approximately 14.5% to 4.3%.The queries occupy each target block’s position and attend to the complete clean prefix; the latent representation, decoder, group size, and training budget remain unchanged.Source: §A.4 · p. 17
  • Full-prefix cross-attention and joint block-causal processing improve prefix-information retention, but can yield copied names, phrase permutations, repeated fragments, and incoherent tails.The joint Transformer produces cleaner unconditional documents than the cross-attention head, yet still shows register transitions and incoherent tails.Source: §A.4 · p. 18

A.5 EXPOSURE AND REMAINING FAILURE MODES

The section investigates exposure mismatch and remaining failure modes in conditional latent generation, then specifies the grouped sampling procedure. Real prefixes do not prevent initial content loss, and rollout training improves some robustness without eliminating repetition.

A.5 EXPOSURE AND REMAINING FAILURE MODES

  • With 32 or 48 real blocks as prefix, the G = 8 prior often retains register and domain for two or three groups but loses a named subject in the first generated group.In eight inspected continuations, rare price-list and chat-log formats also collapse immediately; generated-prefix exposure contributes to later drift, while real prefixes do not resolve initial document-specific content loss.Source: §A.5 · p. 18
  • Anti-repetition objectives evaluated only on real prefixes may have little direct signal about repeated states that arise during generation.Training with model-generated context directly probes this mismatch but does not by itself establish a remedy.Source: §A.5 · p. 18
  • At G = 4, one-step prefix rollout delays list and byline patterns, but six of eight inspected documents still enter one; at G = 1 fluency worsens, and at G = 8 or 16 all inspected documents still loop or list.The one-step replacement does not reproduce all errors encountered with fully sampled prefixes; the conditional weakness is also visible at the first sampled block after a real prefix.Source: §A.5 · p. 18
  • Across two seeds, R3 improves PPL by about four points near entropy 5.41 and about 1.3 points at matched distinctness, but its loop rate remains comparable to control.Rollout training improves conditional robustness in this fully parallel recipe without resolving its repetition attractor; Table 9 uses 256-token PPL windows and loop rates from 50 inspected full texts.Source: §A.5 · p. 18, Table 9 · p. 19
  • Decoder adaptation tolerates some latent errors but does not reliably repair missing or inconsistent document-level information.Different decoder variants show similar semantic failures from the same degraded latent sequence, so cross-block semantic consistency depends strongly on the generated latents.Source: §A.5 · p. 19

B.1 GROUPED SAMPLING ALGORITHM

  • Each group is generated conditioned on the committed latent prefix, with target-specific conditioning states computed once per group and reused through denoising.The flow head reuses the backbone-computed states during sampling.Source: §B.1 · p. 19
  • For the reported G = 8 sampler, T = 50, γ = 1, σ = 0.7, ϵt = 0.05, and the final integration time is tT = 1.The stochastic update uses an SDE-style re-noising heuristic; its (1 − α)σ noise coefficient is not chosen to preserve the marginal variance of the linear interpolation.Source: §B.1 · p. 19

B.2 GROUPED LATENT PRIOR TRAINING

Grouped-prior training preserves the conditional factorization used during generation while allowing conditioning computation and group objectives to be parallelized during optimization.

B.2 GROUPED LATENT PRIOR TRAINING

  • Training uses clean encoder latents as teacher-forced context, so conditioning states can be computed in one block-causal backbone pass.The flow objectives for different groups are then evaluated in parallel.Source: §B.2 · p. 19
  • Queries for a group can access clean blocks before that group but cannot access its target or future blocks.The attention mask preserves the conditional factorization used during generation.Source: §B.2 · p. 19
  • Groups need to be generated sequentially only at inference, when sampled groups replace the real teacher-forced prefixes.Optimization does not process the groups sequentially.Source: §B.2 · p. 19

C IMPLEMENTATION DETAILS

The implementation pairs a separately developed fully parallel latent prior with grouped-prior training and reports runtime measurements under a specified single-GPU protocol. The comparison between prior designs is not controlled to isolate grouping alone.

C.1 FULLY PARALLEL LATENT PRIOR

  • The fully parallel reference jointly denoises 64 block latents with a 684M-parameter bidirectional Transformer and shares its frozen fusion encoder and adapted decoder with grouped models.The reference uses 24 layers, 24 heads, and hidden size 1536.Source: §C.1 · p. 19
  • An approximately 100M-parameter fully parallel prior collapsed onto a few repeated topics, so the paper reports its strongest developed fully parallel prior instead.That reference was tuned longer and used a different training and sampling recipe than grouped priors.Source: §C.1 · p. 21
  • Training mixes independent and shared block-noise levels, sometimes keeps clean context blocks, and fine-tunes with model-predicted context and a jointly trained branching decoder.Sampling uses a 50-step SDE with staggered noise schedules, with earlier-finished blocks providing context.Source: §C.1 · p. 21
  • The Table 1 comparison evaluates the strongest developed configurations, not grouping alone; grouped priors use neither per-block noise nor rollout training and process approximately 10.5B tokens.The fully parallel reference processes approximately 2B tokens in its first stage, with its fine-tuning token budget still unconfirmed; the adapted decoder is shared.Source: §C.1 · p. 21

C.2 INFERENCE STACK FOR RUNTIME MEASUREMENTS

  • Runtime measurements use one NVIDIA H100 GPU, median batch-1 latency and batch-32 throughput, synchronized from initial noise or first token through output token IDs.Measurements include sampling and decoding but exclude model loading and detokenization.Source: §C.2 · p. 21
  • BLD samples latent groups sequentially and decodes all 64 branches in parallel; ELF processes the full 1024-position sequence at every denoising step, while AR uses a key/value cache.BLD uses 50 SDE steps per group; MDLM and BD3-LM use 1023 token-reveal calls for a 1024-token document, with a cache for BD3-LM but not MDLM.Source: §C.2 · pp. 21–22

D.1 BLOCK AND BRANCH LENGTHS

The section examines how block boundaries and generated branch lengths behave, including a paired chunking diagnostic and selected unconditional samples. Inference boundaries are nearly uniform, while dynamic chunking outperforms uniform chunking in the reported reduced setup.

D.1 BLOCK AND BRANCH LENGTHS

  • 99.3% of internal boundaries on full-length documents fall one token before their grid point, typically producing 15-token first blocks, 16-token interior blocks, and 17-token last blocks.Boundary scores favor positions t ≡15 (mod 16); shorter documents may have shorter spans when narrowed windows contain no preferred position.Source: Table 11 · p. 22, §D.1 · p. 23
  • In a paired reduced-stack setup, uniform chunking reconstructs real latents more accurately but yields a broader latent spectrum and worse generated text than dynamic chunking.This supports a training-time contribution from dynamic chunking, but does not identify its mechanism or measure its effect on the final G = 8 prior; the separate mean-pooling probe left k99 essentially unchanged at 392 versus 393 and did not evaluate generation.Source: §D.1 · p. 23, Table 12 · p. 23
  • The token-decoding depth has a median of 16 for both G = 8 and G = K, with 95th percentiles of 22 and 17, respectively, and an observed maximum of 24 for both.The priors generate 971 and 1021 tokens per document on average, respectively; longer branches raise decoding depth for some documents.Source: Table 11 · p. 22, §D.1 · p. 23

D.2 SELECTED UNCONDITIONAL SAMPLES

  • Two contiguous unconditional output passages are shown, one from BLD with G = 8 and one with G = K.The excerpts illustrate samples rather than establishing a comparative quality result.Source: §D.2 · p. 23

D.3 COMPARISON WITH COSMOS

The Cosmos comparison reports generation cost and quality measurements, but its differing evaluation lengths preclude a controlled quality ranking. The authors also note an unresolved perplexity discrepancy.

D.3 COMPARISON WITH COSMOS

  • Cosmos quality scores are not directly comparable with the approximately 1024-token baseline and BLD scores because Cosmos is evaluated on prefixes of at most 128 or 256 GPT-2 tokens.At those lengths, 5.1% and 9.0% of Cosmos outputs remain shorter than the respective prefix limits.Source: §D.3 · p. 24, Table 13 · p. 24
  • 26.7 TFLOPs per full generated document is approximately 24× the cost of BLD with G = 8.The reported Cosmos perplexity and diversity under the regex-tokenization protocol are 98.9 and 0.346, compared with reported values of 55.0 and 0.319; the perplexity discrepancy remains unresolved.Source: §D.3 · p. 24, Table 13 · p. 24
Loading 2610.09311v1…