PaperlayerView exact revision on arXiv

Source-linked AI summary

Block Cascading: Training Free Acceleration of Block-Causal Video Models

Hmrishav Bandyopadhyay, Nikhil Pinnaparaju, Rahim Entezari, Jim Scott, Yi-Zhe Song, Varun Jampani

arXiv:2511.20426v1cs.CVcs.AI

TL;DR

Strict sequential dependencies in block-causal video generation constrain parallelism amid a speed-quality trade-off. Block Cascading starts future blocks from partially denoised predecessor context, overlapping generation across GPUs without fine-tuning. It reports average ∼2× speed-up with 5 GPUs while maintaining quality comparable to baselines, reaching 30 FPS for 1.3B models and 12.5 FPS for 14B models.

  • Problem

    Strict causal dependencies require each block to wait for its predecessor to finish denoising, while 1.3B models reach only 16 FPS and 14B models 4.5 FPS, forcing a choice between responsiveness and quality.

    Source: §1 · p. 1

  • Method

    Block Cascading is a training-free strategy that starts future blocks from partially denoised predecessor context and overlaps denoising across blocks.

    Source: §1 · p. 2

  • Results

    With 5 GPUs, Block Cascading averages ∼2× speed-up across settings while maintaining quality comparable to baselines, reaching 30 FPS for 1.3B models and 12.5 FPS for 14B models.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The approach boosts streaming inference speed and supports downstream applications such as interactive and controllable video generation.

    Source: §6 · p. 8

    The approach has sub-linear multi-GPU scaling and is a poor alternative for large video batches, which are better parallelized with distributed sampling.

    Source: §5 · p. 8

Abstract

from arXiv · show

Block-causal video generation faces a stark speed-quality trade-off: small 1.3B models manage only 16 FPS while large 14B models crawl at 4.5 FPS, forcing users to choose between responsiveness and quality. Block Cascading significantly mitigates this trade-off through training-free parallelization. Our key insight: future video blocks do not need fully denoised current blocks to begin generation. By starting block generation with partially denoised context from predecessors, we transform sequential pipelines into parallel cascades where multiple blocks denoise simultaneously. With 5 GPUs exploiting temporal parallelism, we achieve ~2x acceleration across all model scales: 1.3B models accelerate from 16 to 30 FPS, 14B models from 4.5 to 12.5 FPS. Beyond inference speed, Block Cascading eliminates overhead from KV-recaching (of ~200ms) during context switches for interactive generation. Extensive evaluations validated against multiple block-causal pipelines demonstrate no significant loss in generation quality when switching from block-causal to Block Cascading pipelines for inference. Project Page: https://hmrishavbandy.github.io/block_cascading_page/

1. Introduction

Block Cascading challenges the strict sequential scheduling behind block-causal video generation: partially denoised predecessor blocks can provide enough context for future blocks to start early. The resulting parallel cascade accelerates generation and supports interactive prompt switching without fine-tuning.

1. Introduction

  • Small 1.3B models reach only 16 FPS while 14B models run at 4.5 FPS on H100 GPUs, creating a speed-quality trade-off that limits interactive use of large models.The paper attributes the trade-off partly to rigid scheduling with strict causal dependencies.Source: §1 · p. 1
  • Future blocks can begin denoising from partially denoised predecessor outputs, which can yield visual quality comparable to conditioning on fully denoised blocks.The paper gives t=750 as an example intermediate timestep.Source: §1 · pp. 1–2
  • Block Cascading overlaps denoising across blocks, so multiple blocks progress simultaneously instead of waiting for each predecessor to finish.Future blocks are partially generated by the time the current block is decoded and viewed.Source: §1 · p. 2
  • Pre-generating future blocks allows prompt switches by changing the text prompt, avoiding KV-recaching latency spikes and enabling gradual context adaptation without abrupt scene changes.The introduction describes KV-recaching overhead as 200ms+ in block-causal pipelines.Source: §1 · p. 2
  • Five-GPU scaling is sub-linear: a 14B model reaches 2.79× FPS, with VAE decoding and cross-GPU KV communication contributing overhead.The paper notes that VAE decoding can be moved to another GPU to overlap with next-block denoising.Source: §1 · p. 2
  • With 5 GPUs, Block Cascading averages ∼2× speed-up across settings, reaching 30 FPS for 1.3B models and 12.5 FPS for 14B models while maintaining quality comparable to baselines.The introduction also reports 10% speed-up on a single GPU.Source: §1 · p. 2

2. Related works

Related work addresses video-generation speed through timestep distillation and autoregressive pipelines, while KV caching reduces repeated attention computation. Block Cascading builds on these approaches by relaxing causal constraints to process future blocks in parallel.

2.1. Timestep Distillation

  • Timestep distillation compresses a diffusion ODE trajectory by training a few-step student against a multi-step teacher, with methods including progressive distillation, DMD, and SiD.The paper uses timestep-distilled models for few-step Block Cascading inference.Source: §2.1 · p. 2

2.2. Auto-Regressive Video Generation

  • Recent video-generation work uses autoregressive pipelines, either distilled from bidirectional models or trained from scratch, and Block Cascading relaxes their causality restrictions by processing future blocks in parallel.Bidirectional diffusion models can produce high-quality video but are slow and have quadratic attention scaling with generated-frame count.Source: §2.2 · pp. 2–3

2.3. KV Caching

  • KV caching retains previous-frame key-value features to avoid recomputing context, but attention scales quadratically with cache size and cache eviction can cause drifting.Block Cascading shares KV features across parallel blocks, reducing reliance on an external KV cache during inference.Source: §2.3 · p. 3

3. Methodology

Block Cascading uses partially denoised blocks as context so future blocks can start before their predecessors finish, then batches dependent blocks for parallel denoising. It extends this cascade with bidirectional attention for adaptation and interactive prompt switching.

3.1. Background and Overview

  • Block-causal pipelines roll out temporally grouped blocks in sequence, using clean past KV features to condition future blocks.CausVid and Self-Forcing use KV caching to avoid recomputing past features, while LongLive adds KV-recaching for new prompts.Source: §3.1 · p. 3

3.2. Noisy Caching

  • Block Cascading starts future blocks with noisy context from unfinished predecessors, replacing strict sequential generation with overlapping block denoising.Future blocks can use partially denoised outputs; the pipeline speculatively rolls forward with noisy cached features.Source: §3.1 · p. 3, §3.2 · p. 4

3.3. Block Cascading

  • Mini-batches denoise blocks individually and in parallel, sharing attention across previous and current blocks rather than jointly denoising stitched frames.The multi-GPU pipeline shares KV features through a global pool; a fixed window size bounds the blocks batched together.Source: §3.2 · p. 4, §3.3 · p. 4

3.4. Training Free Adaptation

  • Bidirectional attention across the cascade smooths feature inconsistencies from context mismatch and can sometimes exceed the block-causal baseline in quality.Naive causal denoising can cause frame-jump artifacts, particularly in early frames with insufficient clean-latent context.Source: §3.4 · p. 5

3.5. Interactive Control

  • During prompt switches, new text context is injected into future blocks at their current noise levels, avoiding KV-recaching and its FPS drop.Block-causal pipelines recache past KV with new prompts, which can cause 200ms+ latency spikes; the cascade blends context across denoising stages.Source: §3.5 · p. 5, Figure 3 · p. 5

4. Experiments

The experiments compare training-free Block Cascading against original inference pipelines across short, long, and interactive video generation. They assess streaming speed, cascading configurations, and video quality through quantitative analysis and user studies.

4. Experiments

  • The experiments adapt Self-Forcing, LongLive, and Krea without training, evaluating short, long, and interactive generation across model capacities.They use a seven-block window, with a one-block static sink for LongLive except during interactive generation.Source: §4 · pp. 5–6

4.1. FPS Analysis

  • Streaming FPS averages blocks 8 and 9 of a 13-block generation with a seven-block attention window.This metric represents steady-state performance while new blocks are generated and decoded; instantaneous FPS varies over generation.Source: §4.1 · p. 7

4.2. Ablative Studies

  • Causal P4 yields the highest FPS (+2.9 single-GPU, +14.5 multi-GPU), but can artifact; bidirectional P4 offers high quality with marginally different multi-GPU FPS.The user study finds better performance for bidirectional P4 and causal P2/P3 than causal P4.Source: §4.2 · pp. 7–8

4.3. User Study

  • Users had difficulty distinguishing cascaded from original Self-Forcing and LongLive videos, and preferred cascaded results for interactive LongLive and short Krea videos.The study reviewed videos with the same seed and prompt for comparisons; preference may reflect additional bidirectional-attention context.Source: §4.3 · p. 8

4.4. Quantitative Analysis

  • ∼2× average FPS boost across five GPUs accompanies no significant quality drop versus original pipelines.The reported VBench scores closely follow the originals; Table 1 reports competitor video-quality scores and FPS.Source: §4.4 · p. 8, Table 1 · p. 8

5. Limitations

The reported limitations concern window-size compatibility and sub-linear multi-GPU scaling. Smaller cascades can accommodate strict small-window models, at some cost to speed.

5. Limitations

  • A pretraining window-size constraint can limit Block Cascading when full parallelism is desired.With a seven-block inference window and a checkpoint trained with a four-block window, some samples showed slightly higher drifting.Source: §5 · p. 8
  • Smaller-window models can use P2 or P3, but with reduced inference speed.Reducing the degree of parallelism is the stated adaptation for strict small-window models.Source: §5 · p. 8
  • Multi-GPU scaling is sub-linear, making Block Cascading a poor alternative for large video batches.Distributed sampling is identified as the better way to parallelize large batches.Source: §5 · p. 8

6. Conclusion

Block Cascading parallelizes block-causal inference without retraining or fine-tuning, and user studies report no video-quality degradation. The conclusion reports streaming speeds of 30 FPS for 1.3B models and 12.5 FPS for 14B models using multiple GPUs.

6. Conclusion

  • Block Cascading parallelizes inference in pretrained block-causal pipelines without retraining or fine-tuning.Source: §6 · p. 8
  • User studies demonstrate no video-quality degradation when Block Cascading is used during inference in pretrained causal pipelines.Source: §6 · p. 8
  • Multiple GPUs support streaming speeds of 30 FPS for 1.3B video models and 12.5 FPS for 14B models.The conclusion links these speeds to downstream applications such as interactive and controllable video generation.Source: §6 · pp. 8–9

Related papers

Loading 2511.20426v1…