PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking the Roles of Diffusion and Decoding: Predictive-State Flows for Parallel Language Generation

Jinjin He, Xinsong Feng, Lucheng Fu, Sinan Wang, Bo Zhu, Jiayun Wang

arXiv:2610.12898v1cs.CLcs.LG

TL;DR

Continuous diffusion language models often assign the prior the fine-grained token choices, making the joint continuous latent distribution difficult to model. PSF instead generates predictive states with a rectified-flow prior and decodes blocks in parallel; on unconditional OpenWebText generation, it reports lower perplexity and higher MAUVE than ELF-B at substantially lower FLOPs. These experiments are limited to models up to 677M parameters, a fixed 1024-token canvas, and unconditional OpenWebText generation.

  • Problem

    In continuous diffusion language models, latents are often mapped nearly deterministically to tokens, leaving the prior to model fine-grained token choices and their dependencies in continuous space.

    Source: §1 · pp. 1–2

  • Method

    PSF uses a rectified-flow prior to generate predictive states summarizing preceding context, then a block-causal decoder generates token realizations in parallel with autoregression within blocks.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    PSF achieves generative perplexity 22.8 at entropy 5.16 using 0.31 TFLOPs per 1024-token sequence, versus 24.0 and 7.2 TFLOPs for ELF-B, with higher MAUVE.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    Continuous language-model latents need not preserve every output detail; predictive states illustrate designing latents around how generative roles are divided between prior and decoder.

    Source: §5 · p. 9

    Experiments use at most 677M generation parameters, a fixed 1024-token canvas, and unconditional OpenWebText generation; consistency across many blocks in longer documents remains untested.

    Source: §5 · p. 9

Abstract

from arXiv · show

In continuous diffusion language models (DLMs), latents generated by the diffusion prior are mapped almost deterministically to tokens, either by a decoder or through a token embedding matrix. Unlike in image generation, where visual details can vary continuously, the diffusion prior must capture fine-grained discrete token choices in continuous space, which can make the joint latent distribution difficult to model. We argue that generative roles should instead be divided: the diffusion prior should generate high-level predictive information, while the decoder should resolve the remaining token-level choices. Based on this view, we introduce Predictive-State Flows (PSF), which partitions a sequence into blocks and associates each block with a predictive state computed solely from the preceding context. A rectified-flow prior generates the sequence of predictive states, after which a block-causal decoder generates all blocks in parallel, autoregressively only within each block. This design reduces diffusion from L token-level states to K << L predictive states and reduces token-level sequential depth to L/K. On unconditional OpenWebText generation, a 197M-parameter model with K=8 achieves a generative perplexity of 22.8 at entropy 5.16 using 0.31 TFLOPs per 1024-token document, versus 24.0 and 7.2 TFLOPs for ELF-B with 32 sampling steps. The corresponding measured operating points yield MAUVE@512 of 0.93 and 0.73, respectively. Controlled comparisons further show that content states, which additionally encode their own blocks, yield 8-16% higher generative perplexity than predictive states, and that replacing sampled predictive-state trajectories with a shared mean lowers MAUVE@512 by 0.11. These results suggest that latent design for continuous DLMs should consider not only how compactly text is represented, but also how generative roles are divided between the prior and the decoder.

1 INTRODUCTION

Continuous diffusion language models often leave the prior responsible for fine-grained token choices, raising the question of which information the prior should generate and which decisions the decoder should resolve. PSF assigns predictive information to a short prior-generated state trajectory and local token realization to a block decoder.

1 INTRODUCTION

  • Reconstruction-based text latents tend to preserve fine-grained discrete token distinctions, making their joint distribution difficult for a continuous diffusion prior to learn.The paper contrasts this with image-generation latents, which can discard visually negligible detail and represent remaining variation continuously.Source: §1 · pp. 1–2
  • A 197M-parameter PSF model with K = 8 achieves generative perplexity 22.8 at entropy 5.16 using 0.31 TFLOPs per 1024-token sequence, versus 24.0 and 7.2 TFLOPs for ELF-B.The nucleus-sampling comparison reports generative perplexity of 19.4 for predictive states versus 21.2 when the prior additionally generates block-realization information.Source: §1 · pp. 2–3
  • Reducing K from 64 to 8 improves perplexity from 39.2 to 22.8 while lowering diffusion-prior computational cost, but lower FLOPs do not directly ensure wall-clock speedups.Measured latency and throughput vary with block length and batch size.Source: §1 · p. 3
  • PSF assigns high-level predictive information to the prior and leaves token-level realization to the decoder, using a rectified-flow trajectory and parallel block decoding.A one-token landing target coordinates adjacent blocks, while autoregression remains within each block.Source: §1 · pp. 2–3

2 PREDICTIVE-STATE FLOW

Predictive-State Flow generates a sequence of predictive states with a rectified-flow prior, then decodes the corresponding blocks in parallel. Its design shifts each state to summarize only preceding context, couples neighboring blocks with landing targets, and trains and samples the prior and decoder in stages.

2 PREDICTIVE-STATE FLOW

  • PSF uses a rectified-flow prior to generate predictive states, while a predictive block decoder generates corresponding token realizations in parallel and uses one-token landing targets to couple adjacent blocks.The decoder autoregresses only within each block.Source: §2 · p. 3

2.1 PREDICTIVE STATES

  • Each predictive state summarizes only the context preceding its block; the prior models this document plan, while the decoder resolves each block's token-level realization.The learned s_bos state represents the empty prefix.Source: §2.1 · p. 3

2.2 PARALLEL BLOCK DECODER WITH LANDING TARGETS

  • Because independently decoded blocks can be inconsistent at boundaries, block k conditions from its second decoding step on the first token of block k + 1.The final block receives a learned no-target embedding; after the first decoding step, the branches proceed in parallel for the remaining steps.Source: §2.2 · pp. 3–4

2.3 FLOW PRIOR OVER PREDICTIVE STATES

  • The rectified-flow prior models joint predictive-state trajectories, using noisy trajectories and clean-trajectory prediction before integrating the induced velocity field at generation time.Sampling starts from Gaussian noise and uses 50 Euler steps.Source: §2.3 · p. 4

2.4 TRAINING

  • Training proceeds through reconstruction, decoder fine-tuning, and flow-prior training, including adaptation to rollout states, realization codes, and different block lengths.For rollout adaptation, re-encoded states replace original states independently per block with probability 0.5, while token targets remain unchanged.Source: §2.4 · pp. 4–5

2.5 GENERATION AND REFINEMENT

  • Generation samples a predictive-state trajectory, then decodes all blocks in parallel over L/K autoregressive steps, with first-token landing targets coupling neighboring branches.Optional refinement re-encodes the generated document and runs another parallel decode.Source: §2.5 · p. 5

3 EXPERIMENTS

Experiments evaluate unconditional OpenWebText generation at matched entropy, comparing quality, efficiency, and design choices. Results vary with block length and state/decoder choices, while sampled plans improve distributional similarity.

3.1 MAIN RESULTS

  • At entropy 5.16, K=8 states-only with a 4-gram ban reaches Gen. PPL 19.0 versus 24.0 for 32-step ELF-B; plain-sampler MAUVE@512 is 0.93 versus 0.73.PSF-L with the ban reaches 16.4, near the unbanned same-data AR’s 16.0.Source: §3.1 · pp. 6–7

3.2 THE LANDING TARGET

  • At K=64, the landing target chiefly lowers seam perplexity: without it, the first position costs 5.5×–33× as much, while positions 4–13 remain about 30.Removing the target costs 14% with 63 seams but 3–6% with 7 seams at entropy 5.16.Source: §3.2 · p. 7, Table 2

3.3 EFFICIENCY AND BLOCK LENGTH

  • At K=8, PSF uses 23× fewer FLOPs than 32-step ELF-B; at batch 256, K=16 reaches 1.9× its throughput.At batch 32, the states-only sampler is 6.7× faster than the same-data AR.Source: §3.3 · p. 7
  • Reducing K from 64 to 8 lowers Gen. PPL from 39.2 to 22.8 and prior cost from 631 to 138 GFLOPs, but decoder held-out cross-entropy rises from 2.63 to 2.97.The authors use K=8, accepting greater serial depth; the block-length comparison does not isolate block length at equal training budget.Source: §3.3 · pp. 7–8, Figure 4 · p. 8

3.4 PREDICTIVE STATES AND LOCAL REALIZATION

  • At matched entropy and K=8, content states that encode their own block are 8–16% worse in Gen. PPL than predictive states across sampling settings.The gap persists when the prior is bypassed, suggesting the decoder’s use of the representation also matters.Source: §3.4 · p. 8, Table 3 · p. 8
  • Generated realization codes lower Gen. PPL at tested K≥16 settings but raise it at K=8; the final design generates codes for K≥16 and uses states-only generation at K=8.Zeroing codes raises perplexity by 5–38% at K≥16 but lowers it by 6% (plain) and 9% (nucleus) at K=8.Source: §3.4 · p. 8
  • At K=8, replacing sampled plans with their position-wise mean lowers MAUVE@512 from 0.866 to 0.754 and raises lexical topic-switch rate from 12–14% to 30–47%.The intervention also changes length and entropy, and re-encoded outputs are closer to their own sampled plan than another document’s in 98% of cases.Source: §3.4 · p. 8
  • The best configuration still has a higher lexical topic-switch rate than real text: 9.6% versus 3.6%.Topic-persistence and LLM-judged coherence are evaluated separately.Source: §3.4 · p. 8

4 BACKGROUND AND RELATED WORK

Prior text-generation approaches either diffuse token sequences, decode content latents autoregressively, or combine diffusion and autoregression across blocks. PSF instead generates predictive states jointly and realizes tokens within blocks.

Diffusion and flow models for text

  • Discrete DLMs reverse multinomial or masking corruption, while continuous DLMs denoise token embeddings; these models denoise all L tokens at every step, unlike PSF’s K ≪ L latents.ELF keeps the sequence in a frozen T5 embedding space and reports Gen. PPL 24 on OpenWebText in 32 steps (Hu et al., 2026).Source: §4 · pp. 8–9

Block-wise and semi-autoregressive generation

  • BD3-LM, SSD-LM, and AR-Diffusion diffuse within blocks and autoregress across them; PSF reverses that arrangement, with diffusion across blocks and autoregression within.Blockwise parallel and speculative decoding propose tokens for verification by an autoregressive model, whereas PSF uses no separate target model (Stern et al., 2018; Cai et al., 2024; Leviathan et al., 2023).Source: §4 · p. 9

Planning and hierarchical generation

  • Unlike latent diffusion approaches that encode content to be generated, PSF jointly generates continuous predictive states that exclude their own blocks and couples adjacent blocks with a one-token landing target.Plan-and-Write uses keyword storylines (Yao et al., 2019), Skeleton-of-Thought expands textual outlines (Ning et al., 2024), and the Latent Transformer generates shorter discrete latents autoregressively (Kaiser et al., 2018).Source: §4 · p. 9

5 CONCLUSION

PSF separates predictive-state generation from token realization, reporting computational and modeling benefits while leaving longer-range coherence and broader scaling unresolved. The appendix records the architecture, training, evaluation, and baseline details behind these results.

5 CONCLUSION

  • With K=8, PSF uses 23× fewer FLOPs than 32-step ELF-B while achieving lower generative perplexity, higher MAUVE, and higher batched throughput.The conclusion describes this operating point for a 1024-token document.Source: §5 · p. 9
  • Predictive states outperform content states that also encode their blocks, while increasing block length improves PSF even as it degrades block diffusion.The conclusion presents these as consistent comparative findings.Source: §5 · p. 9
  • Experiments are limited to at most 677M generation parameters, 1024-token canvases, and unconditional OpenWebText; long-document block consistency remains untested.Topic persistence is already the weakest aspect of the samples.Source: §5 · p. 9

APPENDIX

  • The appendix states that diffusion and flow refer to the same rectified-flow prior and ODE sampling process, and records reproducibility details.Those details cover architecture, training schedules, evaluation, and sampling points.Source: APPENDIX · p. 13, A EXPERIMENTAL DETAILS · p. 14

A.1 ARCHITECTURE AND INFORMATION FLOW

  • The appendix documents information flow: predictive states derive from preceding blocks, the prior jointly generates the trajectory, and decoder branches read cumulative state memory.The diagrams depict the states-only path; realization-code inputs are described separately.Source: §A.1 · p. 14, Figure 5 · p. 14, Figure 6 · p. 14

A.2 ARCHITECTURES, TRAINING SCHEDULE AND OPTIMIZER

  • The appendix points to network specifications, model-specific schedules, shared optimizer and prior settings, and staged training details.PSF adds landing targets after base training, whereas PSF-L uses them from the start; both freeze the encoder trunk after the base stage.Source: §A.2 · pp. 14–15

A.3 EVALUATION PROTOCOL

  • Evaluation defines generative perplexity, entropy, repetition, MAUVE, interpolation to entropy 5.16, and a loose likelihood-bound diagnostic.MAUVE uses 512 judge tokens by default; results from different windows are not directly comparable.Source: §A.3 · p. 17

A.4 BASELINES

  • Baselines are sampled with released checkpoints and native samplers, then rescored under a common protocol; published BD3-LM and MDLM values may use different text processing.The appendix reports a Cosmos checkpoint separately because its tokenizer, preprocessing, and native length differ from the 1024-token setup.Source: §A.4 · pp. 17–18

A.5 ELF-B STEP COUNT AND SAMPLER SETTINGS

ELF-B’s operating point changes with step count and sampler settings: modest tuning at 32 steps yields little improvement, while higher-step settings trade perplexity against repetition. At matched entropy, plain PSF trails some 48–64-step ELF-B settings.

A.5 ELF-B STEP COUNT AND SAMPLER SETTINGS

  • At 32 steps, tested noise-scale and guidance changes improve ELF-B by at most about 2%, comparable to sampling variation.The best reported setting is 23.46 at entropy 5.152 with guidance 4.Source: §A.5 · pp. 18–19, Table 8 · p. 19
  • At 64 steps, ELF-B with γ=1.5 reaches 17.57 at entropy 5.067 with 13.1% looped documents, while γ=2 reaches 15.5 at entropy 5.02 with 19% looped documents.The released 64-step default has a loop rate of 14.6%.Source: §A.5 · p. 19
  • At matched entropy, plain PSF is 0.2–3.5% behind ELF-B’s 48–64-step points with γ ≤1.5.The compared ELF-B points are at entropy 5.07–5.10.Source: §A.5 · p. 19

A.6 A SAME-DATA AUTOREGRESSIVE BASELINE

The same-data autoregressive baseline provides a reference with matched preprocessing, parameter count, and approximately matched training-token budget. At entropy 5.16, PSF variants remain behind the baseline’s nucleus-sampling result, while PSF-L with states-only sampling and a 4-gram ban comes within 3%.

A.6 A SAME-DATA AUTOREGRESSIVE BASELINE

  • The same-data AR matches PSF’s 197M generation parameters and uses 50B non-padding training tokens, within 7% of PSF’s 54B.It uses the block decoder’s pre-tokenized OpenWebText and a 1024-token canvas.Source: §A.6 · p. 19
  • At entropy 5.16, the same-data AR reaches 16.0 on the nucleus sweep, while PSF-L with states-only sampling and a 4-gram ban reaches 16.4.PSF-L uses 178 rather than 1024 dependent steps in this comparison.Source: §A.6 · p. 19
  • At entropy 5.16, PSF at K=8 is 43% behind the AR’s nucleus-sweep result with the plain sampler and 21% behind with states-only nucleus sampling.The AR reaches 18.6 on the temperature sweep and 16.0 on the nucleus sweep.Source: §A.6 · p. 19

A.7 SAMPLING SETTINGS AND THE SWEEPS BEHIND EVERY INTERPOLATED NUMBER

The appendix documents how sampling settings, interpolation brackets, and seed checks underlie reported comparisons. It also limits the headline comparisons to sharpened samplers near entropy 5.0–5.2.

A.7 SAMPLING SETTINGS AND THE SWEEPS BEHIND EVERY INTERPOLATED NUMBER

  • Plain sampling uses guidance 1 and settings nearest entropy 5.16, while states-only rows use temperature 1 and specified nucleus and ban choices.PSF-L’s unbanned states-only sweep stops at 5.144, so it does not bracket 5.16 and is not extrapolated.Source: §A.7 · p. 20
  • The full sweep data retain 77 settings, while Figure 3(b) extends temperature sweeps and Figure 3(a) includes ELF-B and BD3-LM baseline curves.The newly sampled anchors may not coincide with original sweeps because batch and random-number layouts differ.Source: §A.7 · pp. 20, 22
  • Figure 9 plots Gen. PPL and loop rate across 36 sampling settings with loop rate at most 20%, omitting native t=p=1 settings.Temperature sweeps fix nucleus at 0.95; nucleus sweeps fix temperature at 1.Source: Figure 9 · p. 21
  • At entropy 5.16, interpolation gives plain-sampler perplexities of 39.2, 32.2, 24.9, 22.8, 20.3, and 18.4 across the listed PSF and PSF-L block-count settings.States-only brackets give 19.4 for PSF, 19.0 for PSF with the ban, and 16.4 for PSF-L with the ban.Source: §A.7 · p. 21
  • Figure 2 selects the lowest-perplexity supported interpolation per model size and block count, while its PSF-L throughput panel times nucleus 0.5 with the ban.For plain PSF at K=8, the tighter bracket gives 22.5, while the tables retain 22.8 from a wider bracket.Source: §A.7 · p. 21
  • Native t=1, p=1 settings have entropy 5.6–5.7 and perplexity 82–122; the headline comparisons instead concern sharpened samplers in the 5.0–5.2 entropy band.At held-out entropy 5.41, PSF is about 45 and PSF-L about 37, versus 16.9 for held-out text.Source: §A.7 · pp. 21–22

A.8 SEEDS AND VARIANCE

  • Multiple-seed checks measure variation at fixed trained checkpoints, not uncertainty across training runs; differences around 1–2% are comparable to sampling variation.For the reported checks, standard deviations are 0.04–0.4 perplexity points and about 0.005 nats of entropy for the paper’s settings.Source: §A.8 · p. 22, Table 10 · p. 22

B ADDITIONAL ABLATIONS

  • The ablations generally use PSF with K=8, the plain sampler, prior codes, and generative perplexity at entropy 5.16.Unless otherwise specified, each setting uses 1000 documents and the GPT-2 Large judge.Source: B ADDITIONAL ABLATIONS · p. 23

B.1 THE LANDING TARGET AND THE TRAINING RECIPE

The landing target is a central part of the training recipe: removing it from the final model costs 13–15%. At K=8, switching it off at inference also worsens perplexity, while boundary positions remain substantially harder to predict than mid-block tokens.

B.1 THE LANDING TARGET AND THE TRAINING RECIPE

  • Removing the landing target from the final model costs 13–15%, supporting its retention in the full training recipe.The recipe ablations also show interacting effects: self-realized prefixes alone worsen true-plan decode, while the source label restores it and brings sampled-plan perplexity close to true-plan decoding.Source: §B.1 · p. 23, Table 11 · p. 23
  • The inference target comparison is sensitive to its evaluation setup: without the target, K=64 documents are shorter, while K=8 lengths are nearly equal.Table 2 compares positions at a fixed setting, avoiding the entropy-matching effect described for the K=64 comparison.Source: §B.1 · p. 23
  • At K=8, switching off the target gives perplexity 23.3 against 22.5 with the nearest brackets as plotted (+3.3%).Retraining gives like-for-like gaps of +6.2% with the plain sampler and +4.7% with the nucleus sampler.Source: §B.1 · p. 23
  • At K=8, first-token boundary perplexity is 332 against 18 mid-block, and last-position perplexity rises to 200.At K=16, first-token perplexity is 145 against 17 mid-block; halving K makes boundaries less frequent.Source: §B.1 · p. 23
  • At K=8, zeroing the lookahead code gives perplexity 22.4 against 22.8, while setting the source label to “original state” gives 24.3 against 24.7 at entropy 5.191.Loop rates rise to 9.3% and 9.9%, respectively, against 6.9% and 9%.Source: §B.1 · pp. 23–24

B.2 REALIZATION CODES, THE STATES-ONLY PRIOR AND CONTENT STATES

The section tests whether realization codes and predictive versus content states affect generation and plan use. Results show trade-offs across perplexity, looping, and distributional behavior, with the controlled content-state comparison favoring predictive states.

B.2 REALIZATION CODES, THE STATES-ONLY PRIOR AND CONTENT STATES

  • At K=8, zero codes are 6% (plain) to 9% (nucleus) better in perplexity at matched entropy, but have higher loop rates; fresh N(0, I) codes are 13% better and raise looping to 19.3%.The code effect reverses at shorter blocks: zeroing codes costs 38%, 22%, and 5% at K=64, 32, and 16.Source: §B.2 · p. 25
  • Under teacher forcing, posterior-mean codes lower decoder NLL by 0.94, 0.91, 0.77, and 0.53 nats per token at K=64, 32, 16, and 8.In generation, codes reduce entropy along with perplexity at a fixed temperature, so the benefit need not persist at matched entropy.Source: §B.2 · p. 25
  • At K=8, zero codes lower perplexity by 6.3% with plain sampling and 9.7% with nucleus sampling at matched entropy, but MAUVE@512 falls by 0.052 and 0.051.The lexical topic-switch rate rises from 6.9% to 11.8% and from 6.8% to 13.1%, respectively.Source: §B.2 · p. 25
  • Retraining without the code channel reaches the zero-code generation numbers, but true-plan decode is 18% worse than for the code-trained decoder with zero codes (21.4 against 18.1 at entropy 5.13).The states-only prior uses zero code targets and reproduces the zero-code numbers for both sizes without changing the architecture.Source: §B.2 · p. 25
  • Content states have 8–16% higher perplexity across sampling settings and 12% higher true-plan perplexity than predictive states.Both variants inherit a decoder trained on predictive states, so the comparison tests the indexing choice under shared adaptation and an initialization that may favor predictive states.Source: §B.2 · p. 25
  • Replacing sampled plans with position-wise mean plans raises fixed-setting perplexity by 7.0–8.7% for PSF, while mean-plan documents are 11–13% shorter and switch topics more often.Matched-entropy differences range from −16% to +22% across models and scoring windows, making them difficult to interpret as a plan effect alone.Source: §B.2 · p. 25
  • Re-encoded outputs are closer to their own plan than another sample’s in 98% of documents, with mean cosine 0.40–0.41 versus 0.13–0.14 for blocks 2–8.Block 2 is an exception, with mean cosine 0.11, as for a random plan.Source: §B.2 · p. 26

B.3 BLOCK LENGTH, SCALE AND TRAINING BUDGET

The block-length and training comparisons favor K=8 for both model sizes, while the study also tests prior-step counts and optional refinement. Runtime and refinement results qualify how those settings translate to speed and generation quality.

B.3 BLOCK LENGTH, SCALE AND TRAINING BUDGET

  • Each halving of K lowers perplexity along the training chain, while prior cost falls from 631 to 138 GFLOPs and block-decoder cost stays near 172.For PSF-L, generative perplexity is 16%, 18%, and 19% lower at K=32, 16, and 8, respectively; Table 13 follows the training chain and does not isolate block length at equal training budget.Source: §B.3 · p. 26, Table 13 · p. 26
  • Direct fine-tuning at K=8 reaches perplexity 22.9 after 20k decoder steps, close to the chain’s 22.8, whereas the 6k-step run reaches 24.5 against 22.7 ± 0.4.The initial gap closes with additional decoder and prior training; the 20k-step route took 37 hours on eight H100s.Source: §B.3 · p. 26
  • Continuing PSF-L’s K=8 block decoder for 6k and 12k more steps improves perplexity from 18.4 to 18.2 and 18.0, respectively.At K=64, additional self-realized-prefix decoder training together with prior training brings PSF-L to 36.4, 7% ahead of PSF.Source: §B.3 · p. 26

B.4 PRIOR STEPS AND REFINEMENT

  • At K=8, rereading headline values at 32 or 64 prior steps changes them by about 1%, with no consistent direction; shorter-block models are more sensitive.At K≥16, 32 steps cost 1–3.4% (6.5% for PSF-L K=32), while 64 steps gain at most 1% (3.1% for PSF-L K=32).Source: §B.4 · p. 27
  • At batch 32 and 32 prior steps, PSF has lower perplexity than ELF-B at entropy 5.16, but ELF-B is faster at 20.9 versus 25.9 ms.At 64 steps PSF is faster, 28.2 versus 40.9 ms; the paper retains 50 steps as a common setting.Source: §B.4 · p. 27
  • One refinement round is 4–10% worse at every tested block length and size, while two rounds are 15% worse; confidence-ordered partial commit brings perplexity within ±2.5% of one-shot.The tested refinement settings show no consistent improvement, so the paper uses one-shot generation.Source: §B.4 · p. 27

C GPU TIME AND THROUGHPUT

The evaluation reports that throughput depends on block length, batch size, and sampler, and it checks the quality comparisons under alternate judges and length-controlled protocols. These checks preserve some comparisons while narrowing or delimiting others.

C GPU TIME AND THROUGHPUT

  • At batch 256, plain K=8 reaches 62.9 documents per second (1.26× ELF-B), while the evaluated states-only K=8 sampler is near parity at 48.9 against 49.8.At batch 32, K≥16 reaches 60.4–70.2 documents per second against ELF-B’s 47.8; K=8 is slower.Source: C GPU TIME AND THROUGHPUT · pp. 27–28, Table 14 · p. 28

D ADDITIONAL EVALUATIONS

  • The throughput comparison depends on batch size and sampler, and the section’s evaluations assess distributional similarity, diversity, repetition, lexical persistence, coherence, and prompted continuation.The additional evaluations use 1000 documents and GPT-2 Large unless otherwise specified, with tables indicating whether scores are measured or entropy-matched.Source: C GPU TIME AND THROUGHPUT · p. 28, D ADDITIONAL EVALUATIONS · pp. 28–29

D.1 OTHER JUDGES AND OTHER ANCHOR ENTROPIES

  • The broader ordering is more stable than close comparisons under Qwen judges, but the plain K=8 margin over ELF-B shrinks from about 5% to 3%.PSF-L states-only rows move ahead of the same-data AR under both Qwen judges, while K=64 falls behind BD3-LM block 4.Source: §D.1 · p. 29
  • The advantage over ELF-M and ELF-L is specific to the 5.0–5.2 nat band; at higher-entropy operating points, those larger ELF models overtake the plain samplers.For example, ELF-M reaches 21.65 at entropy 5.184, below PSF’s 22.75 at 5.164.Source: §D.1 · p. 29

D.2 DOCUMENT LENGTH, MAUVE AND A LENGTH-CONTROLLED WINDOW

  • On the first 512 GPT-2 Large tokens, plain K=8 and states-only samplers are 21% and 34% ahead of ELF-B, while the gap to BD3-LM narrows to 7–23%.The analysis uses a common 512-token window because output lengths differ across systems; restricting to documents longer than 512 retains only 68% of BD3-LM’s documents.Source: §D.2 · p. 29, Table 15 · p. 30
  • At K≤16, plain samplers score MAUVE 0.90–0.93, while states-only scores are 0.85–0.89 for PSF and 0.88–0.91 for PSF-L.The plain ranges numerically match or exceed evaluated diffusion baselines and GPT-2 small, while remaining below the same-data AR’s 0.93–0.95.Source: §D.2 · pp. 29–30

D.3 DIVERSITY AND OVERLAP WITH THE TRAINING SET

Across the appendix evaluations, generated documents remain less diverse than real text, and judges prefer real text to every tested generator. Sampling controls can eliminate the measured loop pattern or improve lexical persistence, while prompted continuation shows limited lexical relevance even with added sampling interventions.

D.3 DIVERSITY AND OVERLAP WITH THE TRAINING SET

  • The position diagnostic showed no late increase in judge negative log-likelihood: the second-half-to-second-bin ratio was 0.82–0.91 for these models, versus 0.85 for held-out text and 0.90 for ELF-B.This local predictability check does not establish document coherence.Source: §D.3 · pp. 30–31
  • Every generator was less diverse than real text; states-only rows had Self-BLEU 0.36–0.40 and distinct-4 0.74–0.75, while their indexed 13-gram overlap was 0.5–1.0%.The indexed overlap remained below held-out text's 2.7%; this statistic measures overlap with the indexed training subset, not text quality.Source: §D.3 · p. 30, Table 16 · p. 31

D.4 REPETITION AND THE 4-GRAM BAN

  • The ban eliminated the measured loop criterion, defined as an 8-gram occurring at least three times in the first 256 T5 tokens, but broader lexical and discourse consistency required separate checks.At fixed temperature, the ban raised entropy by 0.05–0.10 nats; matched-entropy perplexity changes were −2% to +1% for n ≥4 and −4.5% for n = 3.Source: §D.4 · p. 31
  • At entropy 5.16, the 4-gram ban gave 22.7 versus 22.8 for PSF K = 8 and 19.0 versus 19.4 with its states-only prior, with no banned run triggering the loop criterion.Applying the ban to autoregressive baselines preserved their advantage over the banned states-only PSF rows.Source: §D.4 · p. 31, Table 17 · p. 32

D.5 TOPIC PERSISTENCE AND LEXICAL COUPLING

  • With b = 3, the code-augmented K = 8 model reached J1 = 15.6±0.4 and a 1.6% switch rate, but cost 10–14% perplexity at equal entropy; states-only coupling recovered less persistence at a 6% cost.The lexical bonus directly rewards word reuse, so improved overlap alone does not establish discourse coherence.Source: §D.5 · p. 32, Figure 11 · p. 33

D.6 LLM-JUDGED COHERENCE

  • Both judges preferred real text to every evaluated generator, and direct comparisons found the small states-only configuration lost to ELF-B, BD3-LM, and the same-data AR in each tested window.The 32B judge's whole-document plain-sampler result was inconclusive: only 23 pairs were order-consistent.Source: Table 18 · p. 33, §D.6 · p. 33

D.7 PROMPTED CONTINUATION WITHOUT RETRAINING

  • Adding a 19M-parameter prompt-attention path left unconditional perplexity similar at 22.4 and entropy 5.16, while raw prompt overlap was 3.2% versus 3.0% with the prompt withheld.Under teacher forcing with true predictive states, extra prompt tokens lowered continuation negative log-likelihood by only 0.2%.Source: §D.7 · p. 34
  • With r = 4 resampling passes and b = 3, prompted continuation relevance rose to 4.4–5.2% above chance, compared with 6.8% for real continuations and 6.5% for the same-data AR.The measured oracle-plan rows reached 4.5–4.7%, but used different samplers and entropies and do not establish an upper bound on prompt use.Source: Figure 12 · p. 34, §D.7 · p. 34

E SELECTED SAMPLES

The selected samples are presented as model-and-sampler-labeled document excerpts, with unigram entropy and GPT-2 Large perplexity reported for each. Their displayed scores include a range of entropy and perplexity pairs.

E SELECTED SAMPLES

  • Each selected-sample box names the model and sampler, reports unigram entropy in nats and GPT-2 Large perplexity, and shows up to 1300 characters of unedited text.Scores use up to the first 1024 judge tokens of each document.Source: E SELECTED SAMPLES · p. 34
  • Other displayed samples include entropy 5.23 and generative perplexity 14.96, and entropy 5.16 and generative perplexity 9.91.Source: E SELECTED SAMPLES · p. 35
  • One displayed sample has entropy 5.35 and generative perplexity 19.61.Source: E SELECTED SAMPLES · p. 36

Related papers

Loading 2610.12898v1…