PaperlayerView exact revision on arXiv

Source-linked AI summary

Rolling Forcing: Autoregressive Long Video Diffusion in Real Time

Kunhao Liu, Wenbo Hu, Jiale Xu, Ying Shan, Shijian Lu

arXiv:2509.25161v1cs.CV

TL;DR

Streaming video generation needs low-latency, temporally coherent rollouts over long horizons, but existing approaches suffer from error accumulation and train–test exposure bias. Rolling Forcing combines rolling-window joint denoising, attention-sink global context, and self-generated-history training. It achieves real-time multi-minute streaming on a single GPU with substantially reduced error accumulation.

  • Problem

    Interactive applications require low-latency, temporally coherent long-video streams, while existing causal generation accumulates errors over long horizons and training can suffer exposure bias.

  • Method

    Rolling Forcing jointly denoises multiple frames in rolling windows, anchors global context with initial-frame key–value states, and trains over non-overlapping windows conditioned on self-generated histories.

  • Results

    Rolling Forcing achieves real-time streaming generation of multi-minute videos on a single GPU with substantially reduced error accumulation and state-of-the-art temporal coherence and visual fidelity.

  • Takeaways & Limitations

    The framework supports real-time long-horizon streaming while reducing drift through local mutual refinement and persistent global context.

  • Takeaways & Limitations

    Using only short-term history causes gradual drift in long-range properties such as exposure, color tone, and white balance.

Abstract

from arXiv · show

Streaming video generation, as one fundamental component in interactive world models and neural game engines, aims to generate high-quality, low-latency, and temporally coherent long video streams. However, most existing work suffers from severe error accumulation that often significantly degrades the generated stream videos over long horizons. We design Rolling Forcing, a novel video generation technique that enables streaming long videos with minimal error accumulation. Rolling Forcing comes with three novel designs. First, instead of iteratively sampling individual frames, which accelerates error propagation, we design a joint denoising scheme that simultaneously denoises multiple frames with progressively increasing noise levels. This design relaxes the strict causality across adjacent frames, effectively suppressing error growth. Second, we introduce the attention sink mechanism into the long-horizon stream video generation task, which allows the model to keep key value states of initial frames as a global context anchor and thereby enhances long-term global consistency. Third, we design an efficient training algorithm that enables few-step distillation over largely extended denoising windows. This algorithm operates on non-overlapping windows and mitigates exposure bias conditioned on self-generated histories. Extensive experiments show that Rolling Forcing enables real-time streaming generation of multi-minute videos on a single GPU, with substantially reduced error accumulation.

1 INTRODUCTION

Streaming video generation must deliver frames online with low latency while preserving quality and temporal coherence over long horizons, but strict autoregressive generation can accumulate errors. Rolling Forcing addresses this with joint denoising, persistent global context, and self-generated-history training.

  • Motivation: Streaming generation serves interactive applications by producing frames sequentially for immediate display or downstream consumption.World models, neural game engines, and immersive XR environments require low latency alongside long-horizon coherence.
  • Motivation: Strictly causal frame prediction lets predecessor errors compound, causing long-horizon drift and quality degradation.Existing causal generators inherit each frame’s imperfections from prior frames.
  • Rolling Forcing: Rolling-window joint denoising processes multiple consecutive frames with bidirectional attention and progressively increasing noise levels.Mutual refinement corrects local errors before frames are finalized, suppressing long-horizon drift.
  • Rolling Forcing: Attention sink caching preserves initial-frame key–value states as a global context anchor for stronger long-term consistency.Recent clean frames are also cached to reduce latency and maintain temporal consistency.
  • Rolling Forcing: Non-overlapping-window training enables few-step distillation over extended denoising windows while conditioning on self-generated histories to mitigate exposure bias.The training windows collectively cover all video frames.

2 RELATED WORK

Prior work spans bidirectional diffusion models and autoregressive video generators, while newer streaming methods combine causal rollout with denoising diffusion. Concurrent StreamDiT instead modifies the architecture and requires extensive additional pretraining.

  • Bidirectional Video Generation Models: Bidirectional video generation has progressed from pixel- and latent-space diffusion to Space–Time U-Nets and DiT-based architectures.Large industrial investments have supported increasingly capable video diffusion models.
  • Autoregressive Video Generation Models: Autoregressive video models support extended rollouts by generating spatiotemporal tokens sequentially, with recent work combining autoregression and denoising diffusion.These approaches extend generation beyond the short clips commonly targeted by bidirectional models.
  • Concurrent and Closed-Source Work: StreamDiT uses FIFO-style denoising, micro-steps, and window attention, requiring extensive additional pretraining with large-scale data and computation.Rolling Forcing keeps the pretrained model architecture unchanged.

3 METHODS

Rolling Forcing combines autoregressive video diffusion with rolling-window denoising, cached temporal and global context, and efficient self-generated-history training. Its design relaxes strict frame-level causality while controlling computation and exposure bias.

  • 3.1 PRELIMINARIES: EXPOSURE BIAS IN AUTOREGRESSIVE VIDEO DIFFUSION MODELS: Autoregressive video diffusion factorizes a video distribution frame-by-frame, modeling each conditional distribution by progressively denoising noise while conditioning on prior frames.Consecutive frame chunks may also be generated instead of individual frames.
  • 3.1 PRELIMINARIES: EXPOSURE BIAS IN AUTOREGRESSIVE VIDEO DIFFUSION MODELS: Training on clean or independently noised ground-truth histories creates exposure bias because inference conditions on the model’s own predictions.The required prediction–ground-truth pairs conditioned on self-generated histories are unavailable for the denoising loss.
  • 3.1 PRELIMINARIES: EXPOSURE BIAS IN AUTOREGRESSIVE VIDEO DIFFUSION MODELS: Self Forcing uses DMD distillation and self-generated histories to mitigate exposure bias, but severe error accumulation remains beyond the trained temporal window.DMD compares smoothed data and generator output distributions across sampled noise timesteps without requiring ground-truth video data.
  • 3.2 AUTOREGRESSIVE VIDEO GENERATION VIA ROLLING DIFFUSION WINDOW: Rolling Forcing jointly denoises T consecutive frames with progressively higher noise levels, bidirectional attention, and aligned rolling windows instead of strictly causal single-frame denoising.The window length is Lwin = T, and alignment allows the window to roll forward indefinitely.
  • 3.3 TEMPORAL AND GLOBAL HISTORY CONTEXT: Recent-frame KV states provide temporal context, while cached initial-frame KV states provide a global anchor for long-term consistency; history-cache attention remains causal.The total attention window is matched to the bidirectional teacher through Ltem + Lglo + Lwin = Lbidirectional.
  • 3.4 ROLLING FORCING POST-TRAINING: Training selects non-overlapping windows for gradient computation, reducing memory usage while supervising the predicted clean video.The enlarged attention window makes DMD training T times more computationally complex and potentially causes out-of-memory errors even on 80G GPUs.

4 EXPERIMENTS

Rolling Forcing is evaluated against comparable autoregressive video models using standardized quality, drift, and real-time metrics, plus qualitative comparisons and ablations. It preserves quality over long rollouts, achieves real-time generation, and benefits from mixed training and attention sink designs.

  • Implementation Details: Evaluation uses VBench quality metrics on 200 MovieGen prompts with videos standardized to 30s, 16 fps, and 832 × 480 resolution.The evaluation covers temporal flickering, subject and background consistency, motion smoothness, aesthetic quality, and imaging quality.
  • Implementation Details: ∆Quality_Drift measures the absolute imaging-quality difference between each video’s first and last 5 seconds, directly reflecting error accumulation.Real-time performance is measured using throughput and stable-speed latency rather than first-frame latency.
  • Comparisons: Rolling Forcing achieves the highest overall quality scores and a substantially lower ∆Quality_Drift than comparable open-source autoregressive models.The comparison includes SkyReels-V2, MAGI-1, CausVid, and Self Forcing among relevant baselines.
  • Comparisons: Over 2 minutes of autoregressive generation, Rolling Forcing preserves high-fidelity, consistent video quality while compared models show color shifts, artifacts, and unnatural motion.Qualitative comparisons are presented in Figure 4.
  • Comparisons: Rolling Forcing achieves real-time generation with sub-second latency, marginally faster than Self Forcing and CausVid.This result supports its suitability for long-horizon video streaming applications.
  • Ablation Studies: Removing Self Forcing training causes substantial degradation in consistency and overall quality, primarily through unnatural camera motion.The mixed training strategy is evaluated in Table 2.
  • Ablation Studies: Removing the attention sink produces noticeable drift in generated videos, demonstrating the contribution of the global context frame.The attention-sink ablation is illustrated in Figure 5.

5 CONCLUSION

Rolling Forcing targets real-time, long-horizon video generation by reducing error accumulation while preserving sub-second latency. Its combined denoising, attention-sink, and training designs improve coherence and fidelity over multi-minute streams.

  • 5 CONCLUSION: Rolling-window joint denoising enables mutual refinement across consecutive frames and reduces long-term drift.Frames are denoised together rather than independently.
  • 5 CONCLUSION: Attention sinks anchor initial frames as persistent context, enhancing global consistency over long horizons.The mechanism preserves initial-frame context during streaming generation.
  • 5 CONCLUSION: Extensive experiments show state-of-the-art temporal coherence and visual fidelity with significantly better quality and efficiency than prior streaming approaches.The evaluation covers multi-minute streaming sequences.

A ADDITIONAL IMPLEMENTATION DETAILS

The implementation uses persistent global context with a rolling temporal cache and a few-step flow-matching diffusion process. Training extends generated sequences while using re-encoding and self-generated histories to support the distilled model.

  • KV Cache: The KV cache uses Ltem = 3, Lglo = 3, and Lwin = 15 latent frames.The configuration distinguishes temporal, global, and denoising-window sizes.
  • KV Cache: Inference persists global-context KV states while discarding obsolete temporal frames to maintain constant memory usage.Recent frames preserve short-term consistency, while initial frames remain the global anchor.
  • Inference: Rolling Forcing inference initializes KV states, iterates over video frames, and applies RoPE to selected cached states.The algorithm maintains model outputs and a KV cache throughout generation.
  • Training: Training randomly samples 21–27 latent frames, computes DMD loss on the last 21, and re-encodes a boundary frame before concatenation.The re-encoding addresses the distinct statistics of the first VAE-encoded frame.
  • Noise schedule and model parameterization: The method uses flow matching with a 5-step schedule [1000, 800, 600, 400, 200] and data prediction through Gθ.The parameterization keeps cskip = cin = cout = 1 and cnoise(t) = t.

B VBENCH SCORES ACROSS ALL DIMENSIONS

The paper evaluates Rolling Forcing across VBench’s quality and semantic dimensions, using standardized evaluation over 946 prompts. Reported gains are strongest in frame-wise fidelity and extend to semantic scores against distilled baselines.

  • Tables: Table 3 presents full quality evaluation on VBench, while Table 4 presents full semantic evaluation.The provided captions identify the table topics but not their detailed layouts.
  • VBench evaluation: Rolling Forcing is evaluated on all 946 VBench prompts across all 16 reported metrics.The values use the official standardized evaluation scripts.
  • Reported outcomes: Rolling Forcing shows substantial overall-quality improvements, particularly in frame-wise fidelity, and outperforms distilled baselines on semantic scores.The supplied passages report the comparison qualitatively without individual table values.

C INTERACTIVE VIDEO STREAMING

Rolling Forcing supports interactive video streaming by allowing users to change prompts during generation. New prompts steer subsequent video content through updated cross-attention.

  • Interactive video streaming: Users can modify prompts during generation to steer the streaming video content.The capability is demonstrated in Figure 6.
  • Interactive video streaming: The implementation discards the previous text-prompt cross-attention cache before applying the new prompts.This updates prompt conditioning during ongoing generation.
  • Interactive video streaming: Prompt changes provide interactive control without requiring a new video-generation sequence.The passage describes prompt replacement during streaming rather than restarting generation.

D DYNAMIC ROPE

The paper evaluates alternative RoPE placements for global context frames and adopts the placement immediately preceding the temporal context, which yields consistent videos with minimal artifacts. Other placements produce jumping, flickering, static, or unnatural-motion artifacts.

  • D DYNAMIC ROPE: The adopted design places global context indices immediately before the temporal context, at i − L_tem − L_glo:i − L_tem − 1.The denoising window indices are i:i + T−1, while temporal context frames occupy i − L_tem:i−1.
  • D DYNAMIC ROPE: Option 2 produces strong jumping artifacts because relative positions exceed the trained offset range.Figure 7 illustrates the abrupt reset-like artifact associated with this placement.
  • D DYNAMIC ROPE: Option 3 introduces flickering because the model confuses global and temporal contexts.
  • D DYNAMIC ROPE: Option 4 collapses into static outputs because generated frames are forced to replicate the global context.
  • D DYNAMIC ROPE: Option 5 induces unnatural motion because the model converges toward the misplaced global anchor.Only option 1 yields consistent videos with minimal artifacts among the evaluated placements.

E LIMITATIONS

Rolling Forcing suppresses long-horizon error accumulation but remains limited by discarded mid-sequence memory, costly training, and possible latency in interactive applications.

  • E LIMITATIONS: Frames leaving the temporal context are discarded, so the model retains no memory of middle-sequence content.The paper identifies more advanced memory mechanisms as a promising direction.
  • E LIMITATIONS: The enlarged attention window and DMD loss substantially increase GPU memory usage during training, limiting scalability to higher-capacity models.The paper calls for more efficient training or distillation strategies to reduce these costs.
  • E LIMITATIONS: The rolling diffusion window may increase interactive latency because future frames are partially pre-generated before the current frame is finalized.The paper suggests mixed inference strategies that switch between frame-by-frame and rolling denoising.

F BROADER SOCIETAL IMPACT

Real-time long-horizon text-to-video generation could broaden interactive media, live storytelling, and educational tools, while also increasing misuse and extended-bias risks.

  • F BROADER SOCIETAL IMPACT: Real-time long-duration video synthesis could broaden access to interactive media, live storytelling, and educational tools.
  • F BROADER SOCIETAL IMPACT: Realistic live-stream generation heightens risks of misleading streams and harmful biases amplified over extended outputs.The paper encourages future research on safeguards including content filtering.
Loading 2509.25161v1…