PaperlayerView exact revision on arXiv

Source-linked AI summary

Speculative Coupled Decoding for Training-Free Lossless Acceleration of Autoregressive Visual Generation

Junhyuk So, Hyunho Kook, Chaeyeon Jang, Eunhyeok Park

arXiv:2510.24211v2cs.CV

TL;DR

Autoregressive visual generation is slow because it requires sequential token-by-token computation, while existing speculative approaches can be limited by draft-model training, speed, or quality trade-offs. SCD couples draft sampling between SJD iterations to stabilize decoding while preserving losslessness. The paper reports speedups of up to 4.2× for image generation and 13.6× for video generation over standard AR decoding, without quality degradation or additional training.

  • Problem

    Autoregressive generation requires one forward pass per token, making sequences with thousands of tokens—such as high-resolution images and videos—especially slow.

    Source: §1 · p. 1

  • Method

    SCD replaces independent SJD draft sampling with coupling across consecutive iterations to increase identical-token sampling and stabilize the drafting trajectory while preserving losslessness.

    Source: §1 · p. 2, §4 · p. 5

  • Results

    Up to 4.2× speedup for image generation and 13.6× for video generation, without quality degradation or additional training.

    Source: §6 · p. 8

  • Takeaways & Limitations

    SCD improves visual autoregressive decoding speed while retaining quality and avoiding the need for additional training.

    Source: §6 · p. 8

    Large windows or batches can make parallel-operation overhead significant and prevent practical speedups, especially with limited parallel-computation capacity.

    Source: D. Limitations · p. 15

Abstract

from arXiv · show

Autoregressive (AR) modeling has recently emerged as a promising new paradigm in visual generation, but its practical adoption is severely constrained by the slow inference speed of per-token generation, which often requires thousands of steps to produce a single sample. While several Speculative Decoding (SD)-based methods have been proposed to solve this problem by generating multiple tokens in a single forward step, they suffer from limited speedup, degraded quality, or require the training of a draft model. To solve these problems, we propose a new training-free, lossless SD framework, Speculative Coupled Decoding (SCD), by extending the recently proposed Speculative Jacobi Decoding (SJD). While SJD shows strong potential for accelerating AR generation by combining Jacobi iteration and SD, we found that its acceptance rate is still significantly limited due to the instability arising from the independent sampling process used during draft token generation. To overcome this, we introduce an information-theoretic approach, Coupling, which stabilizes the drafting trajectory of SJD by maximizing the probability of sampling identical draft tokens across consecutive iterations, significantly enhancing the acceptance rate while preserving its lossless property. Remarkably, this method requires only a single-line modification to the existing algorithm with almost zero overhead, yet achieves substantial performance gains, delivering up to a 4.2x speedup in image generation and 13.6x speedup in video generation compared to standard AR decoding, without any degradation or the need for additional training. The source code is available at https://github.com/junhyukso/SCD

1. Introduction

Autoregressive generation is costly for high-dimensional visual data, while existing speculative-decoding approaches face speed, quality, or draft-training trade-offs. SCD couples successive SJD draft samples to improve stability and acceptance while preserving losslessness.

1. Introduction

  • AR generation requires one forward pass per token, making high-resolution images and videos—often represented by thousands of tokens—particularly slow.The paper identifies this latency as a barrier to real-world deployment of multimodal AR models at scale.Source: §1 · p. 1
  • Speculative decoding drafts multiple candidate tokens for parallel verification and can preserve the target distribution, but vision-generation speedups are limited and separate draft models require training.The paper also notes that some SD approaches degrade quality.Source: §1 · p. 2
  • SJD reuses its previous verification-step distribution as the next draft, avoiding a separately trained draft model, but reaches only about 2× speedup for image generation.The introduction contrasts this with text-generation SD methods achieving over 4× acceleration.Source: §1 · p. 2
  • SCD couples draft sampling across consecutive Jacobi iterations, improving stability and acceptance while retaining lossless decoding without additional training.The paper reports speedups of approximately 3.8× for image generation and 10× for video generation over standard AR decoding.Source: §1 · p. 2

2. Preliminaries

The preliminaries establish notation and describe speculative decoding as a parallel draft-and-verify procedure whose acceptance behavior affects speed while preserving the target distribution. SJD applies this framework without a separate draft model by reusing its own prior verification output.

2. Preliminaries

  • The notation uses X_t^i for the token at sequence position i in Jacobi iteration t and p_t^i(·) for that position’s token distribution; distributions are assumed to share support V.Subscripts and superscripts may be omitted when the intended sequence or distribution collection is clear.Source: §2 · p. 2

2.1. Speculative Decoding

  • Speculative decoding drafts multiple tokens, evaluates them with the target model in parallel, and verifies them sequentially until rejection; accepted tokens can be emitted with fewer sequential evaluations while preserving the target distribution.The per-token acceptance probability min{1, p(x)/q(x)} is identified as a key factor in overall speedup.Source: §2.1 · pp. 2–3
  • Standard speculative decoding uses a faster draft distribution to propose tokens for target-model verification, but separate draft training, inter-model communication, and limited non-text speedups constrain its use beyond text generation.The paper frames SJD as a training-free response to these drawbacks.Source: §2.1 · pp. 2–3

2.2. Speculative Jacobi Decoding

  • SJD uses the distribution from its previous verification step as the next draft, eliminating a separate draft model while retaining lossless sampling through the verification mechanism.The passage reports approximately 2× speedup in AR image generation.Source: §2.2 · p. 3

3. Motivation and Analysis

The analysis attributes SJD’s weak acceptance and convergence to instability in independently sampled drafts, then motivates coupling consecutive draft samples to stabilize token trajectories.

3. Motivation and Analysis

  • Higher similarity between consecutive drafts tends to yield higher speedup, and changed-token counts strongly correlate with SJD’s NFE.The analysis links similar contexts to more similar output distributions, lower total variation, and higher acceptance under a mild Lipschitzness assumption.Source: §3 · p. 3
  • Approximately 94% (60 of 64) of draft tokens change between iterations, limiting acceptance and complicating convergence.The analysis connects these changes to SJD’s acceptance rate and behavior over consecutive iterations.Source: §3 · p. 4
  • Standard SJD’s mean acceptance rate stays low and fluctuates with high variance, without improving after an initial jump.Representative tokens also fluctuate without a consistent upward trend.Source: §3 · p. 4, Figure 4 · p. 5

3.1. Analysis

  • Visual-model distributions are flat, and their Rényi-2 entropy constrains collision probability even when consecutive distributions are similar.The analysis identifies the gap between probability-space proximity and realized-token similarity as a factor limiting SJD speedup.Source: §3.1 · p. 4
  • Coupling raises collision probability to its maximal-coupling upper bound and yields steadier, upward-trending acceptance trajectories.The reported SCD trajectories show less fluctuation and a general upward trend, unlike standard SJD.Source: §3.1 · p. 5

4. Methods

SCD couples draft sampling across consecutive Jacobi iterations to raise the probability of identical tokens while preserving the target distribution. Maximal and Gumbel couplings offer different ways to stabilize drafts, with sampling overhead under 5% per NFE step.

4. Methods

  • Coupling consecutive draft distributions targets higher collision probability, while marginalization preserves each distribution and keeps the final output matched to the target model.The method reframes its objective as maximizing coupling cost, the probability of sampling identical variables.Source: §4 · p. 5

4.3. Implementation Details

  • SCD changes only the drafting sampler; sampling operations, including couplings, account for under 5% of latency per NFE step, and post-processing preserves exactness.The paper defines p(t) after logit post-processing, including top-k and CFG.Source: §4.1 · p. 5, §4.3 · p. 7

4.1. Maximal Coupling

  • Maximal coupling reaches the upper-bound cost of 1 − TV distance, raising SJD’s collision probability and acceptance rate while reducing NFEs.The paper reports higher context similarity and a higher overall acceptance rate than standard SJD.Source: §4.1 · pp. 5–6

4.2. Gumbel Coupling

  • Gumbel coupling shares a noise vector and guarantees a lower bound on coupling cost, supporting long-range stabilization and sometimes better NFEs than maximal coupling.The paper says it can help when draft prediction is relatively easy, where keeping early draft tokens unchanged may be beneficial.Source: §4.2 · p. 6

5. Experimental Results

The experiments report faster image and video generation without quality loss, and analyze how coupling strength, coupler choice, and guidance scale affect decoding. The evaluation spans multiple visual AR models and datasets.

5. Experimental Results

  • The evaluation covers Lumina-mGPT, Janus-Pro, Lumina-mGPT-2, and Cosmos-1-AR, using image and video datasets and quality and speed metrics.The reported measures include FID, IS, CLIP score, NFEs, and latency on one NVIDIA A100.Source: §5 · p. 7
  • 4.2× image and 13.6× video acceleration over Vanilla AR are reported without quality loss; image results also retain identical FID, IS, and CLIP scores.The video comparison is on Cosmos-1-AR; its results are presented in Table 4.Source: §5 · pp. 7–8, Table 4 · p. 8
  • On Janus-Pro, SCD accelerates SJD by up to 2.1× and reaches 3.7× final step compression.Table 3 reports Janus-Pro (7B) results on MS-COCO.Source: §5 · p. 7, Table 3 · p. 7

5.1. Further Analysis

  • As α increases toward 1, mean Hamming distance and NFE decrease monotonically, with improvement continuing to α ≈1.α = 0 recovers vanilla SJD; α = 1 corresponds to SCD.Source: §5.1 · p. 8
  • πMC has lower one-step Hamming distance, but πGS has better long-range stability and can reduce NFEs when draft prediction is relatively easy.The paper identifies video with high temporal similarity and low-resolution Janus image generation as such settings, where retaining early draft tokens can improve convergence.Source: §5.1 · p. 8, Figure 7 · p. 8
  • As CFG scale λ increases, speedup slightly decreases, while SCD continues to outperform SJD across the practical scale range.The paper describes the practical range as approximately 3–5.Source: §5.1 · p. 8, Figure 6 · p. 8
  • Qualitative comparisons show outputs visually indistinguishable from AR while achieving approximately 4× acceleration.The paper points to Figure 2 for the visual comparison.Source: §5.1 · p. 8

6. Conclusion

The conclusion attributes SJD's limited acceptance rate to unstable draft contexts and presents coupling as a simple way to stabilize its trajectory. The reported image and video speedups come without quality degradation or additional training.

6. Conclusion

  • SCD replaces independent draft sampling with coupling to stabilize SJD's Jacobi trajectory, reporting up to 4.2× image and 13.6× video speedup without quality degradation or additional training.The identified bottleneck is draft-context instability limiting SJD's acceptance rate.Source: §6 · p. 8

Impact Statement

The impact statement reports no significant societal concerns to highlight. The included proof passages state an acceptance-rate identity and a constraint on independent-sampling collision probability.

Impact Statement

B.1. Proof of Proposition 2.1

  • The proof states that the acceptance rate is 1 − DTV(p, q).Source: §B.1 · p. 11

B.2. Proof of Proposition 3.3

  • Independent collision probability is exponentially restricted by the distributions' Rényi-2 entropies, regardless of how close the distributions are.Source: §B.2 · p. 12

B.3. Proof of Theorem 4.2

The appendix argues that coupling preserves the draft token's required marginal distribution, supporting the output-distribution guarantee. It also describes coupling validity, a Gumbel-Max construction, and a mixture implementation for α-coupling, alongside related work on multimodal and parallel decoding.

B.3. Proof of Theorem 4.2

  • Using a coupled pair preserves the required marginal distribution for each draft token, so the final output distribution matches the base model.Source: §B.3 · p. 12

B.4. Proof of Theorem 4.4

  • The MRS process satisfies the definition of Coupling, as established by its joint distribution and marginal property.Source: §B.4 · p. 12

B.5. Proof of Theorem 4.5

  • The πGS coupling is valid by the Gumbel-Max trick, which yields samples from the input categorical distribution.Source: §B.5 · p. 13

B.6. Proof of α-Coupling

  • α-coupling linearly interpolates two valid couplings and can be implemented by choosing between them with a Bernoulli draw.Source: §B.6 · pp. 13–14

C. Related Works

  • Related work covers unified multimodal and autoregressive vision models, speculative decoding, and fixed-point methods for accelerating sequential computation.The paper frames its method as accelerating convergence of Jacobi iteration for sequential sampling in a discrete probabilistic setting.Source: C. Related Works · pp. 14–15

D. Limitations

SCD relies on parallel computation, which can limit practical speedups when parallel-operation overhead grows or parallel hardware is constrained.

D. Limitations

  • Large windows or batch sizes can make parallel-operation overhead significant.The authors identify this as a potential obstacle to practical speedups.Source: D. Limitations · p. 15
  • The method may fail to achieve practical speedups when the parallel-computation unit is limited.Source: D. Limitations · p. 15
  • The authors describe this structural limitation as shared by all SD methods and predict it will diminish as hardware advances.Source: D. Limitations · p. 15

E. Experimental Details

The experiments specify image and video evaluation settings and implementation details, and report qualitative similarity alongside large NFE reductions.

E.1. Image Generation

  • Lumina-mGPT experiments use a 7B model at 768×768, with temperature 1, Top-K 2000, and guidance scale 3.0.Quality is evaluated on 5,000 MS-COCO 2017 prompts using FID, IS, and CLIP-Score.Source: §E.1 · p. 15
  • Janus-Pro experiments use a 7B model at 384×384, generating 24×24 image tokens with temperature 1, Top-K 1000, and guidance scale 5.0.The evaluation reports mean FID, IS, and CLIP score over three generated images per MS-COCO validation prompt.Source: §E.1 · p. 15

E.2. Video Generation

  • Video evaluation uses 150 real-estate-10k clips, conditioning on 9 frames and generating 24 future frames per clip.Decoding uses top-p 0.8 and temperature 1.0; speed is measured by NFE and wall-clock latency on one RTX6000ADA, with quality measured by FVD.Source: §E.2 · p. 15

F. Algorithms

  • The maximal- and Gumbel-coupling implementations are computationally almost identical to SJD, with almost zero computation and memory overhead.Source: F. Algorithms · p. 15
  • The SCD pseudocode separates parallel drafting, parallel evaluation, and verification within a window.The algorithm excerpts identify these stages and the return of the generated sequence.Source: F. Algorithms · p. 16

G. More Visualization

  • Visualization prompts span humans, animals, landscapes, close-ups, fantasy, paintings, and physical phenomena such as reflections and waves.Prompts also include descriptors such as “8K” and “sharp focus” to encourage detailed, realistic images.Source: G. More Visualization · p. 16
  • More than 4× NFE reduction in image generation and 13× in video generation accompanied outputs closely resembling vanilla AR.The visualization discussion also reports diverse image categories, including reflections and waves, under both coupling variants.Source: G. More Visualization · p. 16

H. More Dataset

On Parti-Prompt, the reported acceleration is comparable to MS-COCO results, supporting generalization across the two evaluated prompt datasets.

H. More Dataset

  • Approximately 4× acceleration on Parti-Prompt is comparable to MS-COCO results, which the authors cite as evidence of generalization.The evaluation uses 1,600 distinct real-world prompts and CLIP score for generation quality.Source: H. More Dataset · p. 17

I. Comparison with Training-based SD

In a comparison with training-based speculative decoders, the authors report that SCD outperforms the baselines while remaining training-free and lossless.

I. Comparison with Training-based SD

  • SCD reportedly surpasses EAGLE and LANTERN by a large margin while retaining training-free and lossless constraints.The comparison uses LlamaGen XL (Stage 2) with 500 MS-COCO prompts; Table 7 presents the comparison.Source: §I · p. 17, Table 7 · p. 21

Related papers

Loading 2510.24211v2…