PaperlayerView exact revision on arXiv

Source-linked AI summary

Long-Horizon Streaming Video Generation via Hybrid Attention with Decoupled Distillation

Ruibin Li, Tao Yang, Fangzhou Ai, Tianhe Wu, Shilei Wen, Bingyue Peng, Lei Zhang

arXiv:2604.10103v2cs.CV

TL;DR

Streaming video generation loses distant context under sliding-window attention, while long-form inference remains computationally costly. Hybrid Forcing combines linear temporal history retention, sparse local attention, and staged distillation. It achieves state-of-the-art performance on short- and long-horizon benchmarks, including real-time generation at 29.5 FPS on one H100 GPU.

  • Problem

    Sliding-window attention evicts distant history during streaming, while long-form video generation remains computationally costly and slow.

    Source: §1 · p. 2

  • Method

    Hybrid Forcing combines compact linear temporal history aggregation, block-sparse sliding-window attention, and decoupled distillation that begins with dense attention.

    Source: §1 · p. 3

  • Results

    Hybrid Forcing achieves state-of-the-art performance on short- and long-horizon benchmarks and reaches 29.5 FPS on one H100 GPU without quantization or compression.

    Source: §1 · p. 4

  • Takeaways & Limitations

    In 126-latent-frame VBench-Long evaluation, Hybrid Forcing maintains stable performance and achieves improved Quality compared with its short-form results.

    Source: §4.2 · p. 11

Abstract

from arXiv · show

Streaming video generation (SVG) distills a pretrained bidirectional video diffusion model into an autoregressive model equipped with sliding window attention (SWA). However, SWA inevitably loses distant history during long video generation, and its computational overhead remains a critical challenge to real-time deployment. In this work, we propose Hybrid Forcing, which jointly optimizes temporal information retention and computational efficiency through a hybrid attention design. First, we introduce lightweight linear temporal attention to preserve long-range dependencies beyond the sliding window. In particular, we maintain a compact key-value state to incrementally absorb evicted tokens, retaining temporal context with negligible memory and computational overhead. Second, we incorporate block-sparse attention into the local sliding window to reduce redundant computation within short-range modeling, reallocating computational capacity toward more critical dependencies. Finally, we introduce a decoupled distillation strategy tailored to the hybrid attention design. A few-step initial distillation is performed under dense attention, then the distillation of our proposed linear temporal and block-sparse attention is activated for streaming modeling, ensuring stable optimization. Extensive experiments on both short- and long-form video generation benchmarks demonstrate that Hybrid Forcing consistently achieves state-of-the-art performance. Notably, our model achieves real-time, unbounded 832x480 video generation at 29.5 FPS on a single NVIDIA H100 GPU without quantization or model compression. The source code and trained models are available at https://github.com/leeruibin/hybrid-forcing.

1 Introduction

Long-form streaming video generation is constrained by lost temporal context and costly inference. Hybrid Forcing combines long-term history retention with efficient local attention and a staged training strategy.

1 Introduction

  • As sliding-window attention evicts earlier frames, streaming models lose distant context, allowing earlier errors to accumulate and destabilize long-form generation.Source: §1 · p. 2
  • Retaining the first frame can reduce early drift, but may diminish motion diversity and leaves informative intermediate frames underused.Source: §1 · p. 2
  • Hybrid Forcing pairs a compact linear temporal state that summarizes evicted tokens with block-sparse local attention and a decoupled distillation strategy.The initial DMD phase uses dense attention before hybrid attention is activated for streaming training.Source: §1 · p. 3
  • A consistent temporal RoPE constraint and teacher-guided regression address temporal position mismatch and long-horizon mode collapse.The RoPE index is capped during training and inference, while teacher rollouts from identical initial noise supervise the student.Source: §1 · pp. 3–4
  • 29.5 FPS on a single H100 GPU without quantization or model compression; Hybrid Forcing achieves state-of-the-art performance on short- and long-horizon benchmarks.Source: §1 · p. 4

2 Related Work

Prior long-duration video-generation work includes diffusion-based extensions and causal autoregressive approaches. Sparse attention has reduced computation in video generation, but its effectiveness and integration in streaming settings remain unresolved.

2 Related Work

  • NUWA-XL, LaVie, and FreeLong extend video generation to longer durations, but incur substantial computation and degraded quality over extended videos.Source: §2 · p. 4
  • Causal autoregressive approaches include CausVid, Self-Forcing [15], Rolling Forcing [25], LongLive [50], Relic [14], and MAGI-1.Source: §2 · p. 4
  • Sparse Video Generation [47] and AdaSpa [56] exploit attention sparsity to reduce computation, while VSA [58] and SLA [56] introduce adapted or learnable sparse-attention approaches.Source: §2 · pp. 4–5
  • Despite efficiency gains, autoregressive models using sliding-window attention can miss long-term context, and sparse methods' effectiveness in streaming video generation remains unproven.Some sparse approaches may also encounter kernel errors when applied to streaming attention.Source: §2 · pp. 4–5

3 Methods

The methods address streaming video’s lost long-range context and costly local attention with hybrid attention, capped relative temporal indexing, and staged training. They also add regularization and decouple semantic representation learning from temporal modeling.

3.1 Preliminary

  • Streaming video generation uses a few-step student distilled from a pretrained bidirectional model, but sliding-window attention loses long-range context and quadratic within-window computation impedes real-time deployment.The student is initialized from ODE trajectories and then distilled with DMD.Source: §3.1 · pp. 5–6

3.2 Hybrid Attention

  • Hybrid attention sums linear history attention over evicted frames with block-sparse local attention, retaining global context while reducing redundant short-range computation.The linear state updates online at eviction, with constant memory independent of video length; reducing the local window from 21 to 9 can further lower latency.Source: §3.2 · pp. 6–7

3.3 Relative RoPE

  • Relative RoPE caps temporal indices at the pretrained model’s maximum and uses relative offsets, while long-horizon distillation rollouts reduce train–test discrepancy and improve extended-generation coherence and stability.The cap is applied during both training and inference.Source: §3.3 · p. 8

3.4 Regularization Loss

  • A regularization term supervises the student’s first generation step against the frozen teacher’s full rollout from the same initial noise, while later steps retain the standard DMD objective.The supervision is applied in latent space; the regularizer is added when t = 1.Source: §3.4 · p. 8

3.5 Decoupled Distillation

  • Decoupled distillation first uses DMD under dense attention to form information-rich representations, then activates linear temporal and block-sparse attention to learn structured long-range dependencies.The staged strategy is intended to avoid premature restriction of information flow and stabilizes optimization.Source: §3.5 · p. 9

4 Experiment

Hybrid Forcing combines efficient local attention with long-range history retention and reports strong short- and long-duration results, including stable generation beyond 10 minutes. Ablations attribute gains to hybrid attention, decoupled distillation, and stability measures.

4.1 Experiment Setting

  • The setup uses 30-second, 16-FPS long-form comparisons and measures throughput on a single NVIDIA H100 GPU.Training enables hybrid attention after 1,000 initial steps, uses a sliding window of 9, and applies top-20% block sparsity.Source: §4.1 · pp. 9–10

4.2 Results

  • Hybrid Forcing achieves the best performance across VBench Total, Quality, and Semantic scores, with throughput 35% above the second-best streaming baseline.It also surpasses bidirectional Wan2.1 in overall performance with approximately 30× higher throughput under identical hardware settings.Source: §4.2 · pp. 10–11, Table 1 · p. 10
  • Dynamic performance exceeds the second-best method by 19.3%, while Hybrid Forcing achieves the best overall performance and highest throughput among few-step autoregressive baselines in long-duration evaluation.The evaluation extends generation to 126 latent frames; the authors also report stable performance and improved Quality relative to short-form results.Source: §4.2 · p. 11, Table 2 · p. 11
  • Hybrid Forcing maintains temporal stability over unbounded horizons, with qualitative generation demonstrated for videos longer than 10 minutes.In 30-second comparisons, the method reproduces the full “house melt” process with high motion dynamics and visual fidelity.Source: §4.2 · pp. 11–12, Figure 4 · p. 13

4.3 Ablation Study

  • Hybrid attention alone raises throughput from 21.8 to 29.5 and the total score from 80.25 to 81.56 versus the native short-window baseline.The Dynamic metric rises from 53 to 67; the ablation uses a fixed local window of 9 and retains 3 sink frames.Source: §4.3 · p. 12, Table 3 · p. 13
  • Decoupled distillation further raises the total score to 83.49, with Dynamic at 74 and Semantic at 78.78.The first DMD-only phase uses dense attention to form information-rich representations before later distillation learns history compression and sparsification.Source: §4.3 · p. 13
  • Constraining temporal RoPE extrapolation mitigates positional distribution shift, while trajectory-level regularization stabilizes motion and reduces long-horizon drift.Without regularization, the ablation shows gradual camera bias and occasional mode collapse; Fig. 5 also reports persistent camera drift and character duplication when the loss is removed.Source: §4.3 · pp. 13–14, Figure 5 · p. 14

5 Conclusion

Hybrid Forcing combines long-history aggregation, efficient local refinement, and decoupled distillation to support stable long-horizon streaming. Its reported throughput is 29.5 FPS on one H100 GPU, with deployment bounded by the teacher model and hardware setting.

  • The framework integrates linear temporal attention, block-sparse sliding-window attention, decoupled distillation, a relative RoPE constraint, and teacher-guided regularization for long-horizon streaming.Source: §5 · p. 14
  • The student’s performance is bounded by the provided Self-Forcing ODE model and teacher generation quality, and real-time throughput may not extend to low-end GPUs.Source: §5 · p. 14

A Algorithm details

The appendix points readers to three algorithms for the paper’s training, inference, and hybrid-attention procedures.

  • Algorithm 1, Algorithm 2, and Algorithm 3 provide details for training, inference, and hybrid attention, respectively.Source: A Algorithm details · p. 19

B Template for VBench prompts enhancements

The VBench prompt-enhancement appendix describes the model and template used to expand evaluation prompts while preserving their intent.

C Detailed evaluation results on the short VBench benchmark and VBench-Long Benchmark

Hybrid Forcing performs strongly across short- and long-duration VBench evaluations, with especially clear advantages on several metrics in the 30-second setting. It also leads in total score and throughput in a separate 30-second comparison.

Detailed evaluation results on the short VBench benchmark and VBench-Long Benchmark

  • In Fig. 6, Hybrid Forcing consistently outperforms competing representative few-step autoregressive models across all 16 VBench metrics on VBench and VBench-Long, with particularly notable improvements in semantic-related metrics.Source: C Detailed evaluation results on the… · p. 20
  • Hybrid Forcing ranks first or second on most 30-second VBench-Long metrics, with pronounced advantages in aesthetic quality, dynamic degree, and spatial relationship.In the 5-second setting, it leads on dynamic degree, appearance style, color, spatial relationship, and overall consistency while remaining comparable on other metrics.Source: C Detailed evaluation results on the… · p. 20
  • Hybrid Forcing achieves the best overall Total Score and highest throughput among compared methods in the 30-second setting.Fig. 7 compares total score and throughput across seven methods.Source: C Detailed evaluation results on the… · p. 20

D More qualitative comparisons under the 5-second and 30-second inference setting

The qualitative comparisons describe stronger motion dynamics for Hybrid Forcing in 5-second videos and comparable or better quality with higher throughput in 30-second videos. Several competing methods show reduced motion or visual degradation over longer durations.

D More qualitative comparisons under the 5-second and 30-second inference setting

  • Hybrid Forcing produces natural motion patterns with stronger visual fidelity and temporal consistency in the 5-second comparisons.LongLive and MemFlow show limited motion dynamics, while object motion ranges evolve more slowly than with other methods.Source: D More qualitative comparisons under… · pp. 20–22
  • ×1.42, ×1.57, and ×1.86 higher throughput than LongLive, MemFlow, and Rolling Forcing, respectively, accompanies comparable or better visual quality in 30-second videos.Hybrid Forcing achieves the highest generation throughput among the compared methods.Source: D More qualitative comparisons under… · p. 22
  • CausVid, Self-Forcing, and InfRoPE show visual degradation after approximately 10 seconds in 30-second videos, with severe structural artifacts in some cases.The turtle example shows its tail gradually deforming and merging visually with its head.Source: D More qualitative comparisons under… · p. 22

E Cases of 10-minute videos

Examples show character identity and background appearance remaining largely consistent through 10 minutes of continuous generation. The paper also reports 29.5 FPS and identifies latency-sensitive applications where long-duration consistency and high inference speed matter.

E Cases of 10-minute videos

  • Character identity and background appearance remain largely consistent throughout 10 minutes of continuous generation.Fig. 13 presents single-prompt examples of 10-minute videos.Source: E Cases of 10-minute videos · p. 23, Figure 13 · p. 28
  • 29.5 FPS is reported alongside suitability for latency-sensitive applications requiring long-duration consistency and high inference speed.Examples include real-time livestreaming, virtual avatar systems, and interactive world models.Source: E Cases of 10-minute videos · p. 23
  • Fig. 7 compares total score and throughput across seven methods on 30-second video inference, and its caption reports superior quality and efficiency for Hybrid Forcing.Fig. 6 presents all 16 VBench and VBench-Long metrics for 5-second and 30-second videos.Source: Figure 7 · p. 24, Figure 6 · p. 24

Related papers

Loading 2604.10103v2…