Source-linked AI summary
Self-Forcing++: Towards Minute-Scale High-Quality Video Generation
Justin Cui, Jie Wu, Ming Li, Tao Yang, Xiaojie Li, Rui Wang, Andrew Bai, Yuanhao Ban, Cho-Jui Hsieh
TL;DR
Long-video autoregressive generation suffers because short-horizon teachers provide limited temporal and error-recovery supervision. Self-Forcing++ distills teacher guidance on self-generated long rollouts, achieving substantially longer videos with strong fidelity and consistency, including multi-minute generation when computation is scaled.
Problem
Short-horizon teachers limit autoregressive students’ training, causing quality degradation from compounding errors when generating beyond five seconds.
Method
Self-Forcing++ re-injects noise into self-generated degraded long rollouts and applies teacher-guided distribution-matching distillation with rolling KV caching and windowed sampling.
Results
Self-Forcing++ extends generation to 100 seconds and, with scaled computation, reaches 4 minutes and 15 seconds while maintaining high visual quality and fidelity.
Takeaways & Limitations
The approach extends high-quality, temporally coherent video generation without long-video teachers, long-video retraining, or overlapping-frame reuse.
Takeaways & Limitations
The method trains more slowly than teacher-forcing and lacks long-term memory, which can cause content divergence in regions occluded for extended periods.
Abstract
from arXiv · showhide
Diffusion models have revolutionized image and video generation, achieving unprecedented visual quality. However, their reliance on transformer architectures incurs prohibitively high computational costs, particularly when extending generation to long videos. Recent work has explored autoregressive formulations for long video generation, typically by distilling from short-horizon bidirectional teachers. Nevertheless, given that teacher models cannot synthesize long videos, the extrapolation of student models beyond their training horizon often leads to pronounced quality degradation, arising from the compounding of errors within the continuous latent space. In this paper, we propose a simple yet effective approach to mitigate quality degradation in long-horizon video generation without requiring supervision from long-video teachers or retraining on long video datasets. Our approach centers on exploiting the rich knowledge of teacher models to provide guidance for the student model through sampled segments drawn from self-generated long videos. Our method maintains temporal consistency while scaling video length by up to 20x beyond teacher's capability, avoiding common issues such as over-exposure and error-accumulation without recomputing overlapping frames like previous methods. When scaling up the computation, our method shows the capability of generating videos up to 4 minutes and 15 seconds, equivalent to 99.9% of the maximum span supported by our base model's position embedding and more than 50x longer than that of our baseline model. Experiments on standard benchmarks and our proposed improved benchmark demonstrate that our approach substantially outperforms baseline methods in both fidelity and consistency. Our long-horizon videos demo can be found at https://self-forcing-plus-plus.github.io/
1 Introduction
Long-video generation is limited by short teacher horizons and training–inference mismatches that cause error accumulation. Self-Forcing++ trains students on their own degraded long rollouts, enabling substantially longer, high-quality generation without overlapping-frame reuse.
- Most state-of-the-art video models generate only 5–10-second clips because Diffusion Transformers are costly and non-streaming.
- Training autoregressive students on clips up to the teacher’s 5-second horizon leaves them unprepared for compounding errors during longer rollouts.Quality can deteriorate beyond the training horizon into static or stalled content.
- Self-Forcing++ generates beyond the teacher’s horizon and corrects the student using its own error-accumulated long trajectories.The method leverages the short-video teacher’s knowledge to correct degraded student rollouts.
- Distribution-matching distillation on re-noised degraded rollouts is combined with long-horizon rolling KV caching and windowed sampling.
- 100 seconds marks a 20× increase over the baseline while maintaining high visual quality and avoiding overlapping-frame reuse.
- With increased computation, Self-Forcing++ generates videos up to 4 minutes and 15 seconds, reaching 99.9% of positional-embedding capacity and 50× the baseline length.
2 Related Work
Related work spans diffusion-transformer video models, techniques for extending video duration, autoregressive generation, and reinforcement-learning approaches for generative models.
- Video Diffusion Models: Diffusion Transformers replaced convolutional hierarchies with transformer blocks to capture global spatio-temporal dependencies and scale with data and computation.
- Long Video Generation: Most DiT-based models remain limited to 5–10-second videos because of their substantial training and inference costs.
- Long Video Generation: RIFLEx extends generation length training-free by revisiting positional encoding and avoiding encodings that induce repetitive motion.
- Long Video Generation: Autoregressive approaches include Nova, Pyramid-Flow, SkyReels-V2, and MAGI-1, which reformulate or structure video synthesis for longer generation.
- Reinforcement Learning: Reinforcement learning has been applied to generative models through image and video reward functions such as ImageReward, Pick-a-Pic, HPS V2, VideoReward, and VisionReward.
3 Method
Self-Forcing++ extends student-teacher distillation to long-horizon video by training on sampled windows from self-generated rollouts, while preserving temporal context through backward noise initialization and rolling KV caching. It further uses optical-flow-guided optimization and an improved evaluation protocol to address long-term smoothness and unreliable conventional metrics.
- 3.2 Extend training beyond teacher’s limit: Backward noise initialization reintroduces noise into denoised latent vectors so teacher-student alignment is evaluated on temporally consistent trajectories.The resulting state retains temporal dependencies from preceding generated frames and follows the prescribed noise schedule.
- 3.2 Extend training beyond teacher’s limit: Self-Forcing++ trains the student on contiguous windows sampled from self-generated rollouts much longer than the teacher’s horizon.The student rolls out N frames, samples a window of length T, and matches the teacher distribution within that window.
- 3.2 Extend training beyond teacher’s limit: Rolling KV-cache training removes the fixed-cache versus rolling-cache mismatch that contributes to error accumulation and temporal flickering during long-video inference.Unlike CausVid, the approach avoids recomputing overlapping frames.
- 3.3 Improving Long-Term Smoothness via GRPO: GRPO uses relative optical-flow magnitude between consecutive frames as a proxy for motion continuity when long-sequence generation loses long-term memory.The method applies GRPO within the autoregressive video-generation framework to guide temporally smooth outputs.
- 3.4 New metrics for long videos evaluation: The evaluation protocol uses Gemini-2.5-Pro to score over-exposure and error accumulation as visual stability on a 0–100 scale.This supplements VBench, which the authors report can overrate degraded and over-exposed frames.
4 Experiments
Experiments evaluate short- and long-horizon video generation against multiple baselines, including ablations of attention-window length, GRPO, and training-budget scaling. Self-Forcing++ remains competitive on short clips and shows increasingly strong fidelity, consistency, and duration performance as the horizon and training budget increase.
- Evaluation settings: Evaluations cover 5-second videos using 946 prompts across 16 VBench dimensions and 50/75/100-second extensions using 128 MovieGen prompts.The long-horizon setting uses VBench Long and the proposed improved benchmark.
- Short-horizon results: 80.37 semantic score and 83.11 total score on 5-second clips surpass the remaining baselines while remaining comparable to Self-Forcing.The model was not specifically trained for the initial 5 seconds.
- Long-horizon results: 26.04 text alignment and 54.12 dynamic degree at 100 seconds outperform CausVid by 6.67% and 56.4%, and Self-Forcing by 18.36% and 104.9%, respectively.The comparison is reported for the text alignment and dynamic degree metrics.
- Long-horizon results: Long-video baselines degrade through motion collapse, exposure instability, or noise collapse, whereas the proposed method sustains coherent motion, stable brightness, and visual quality.CausVid trends toward over-exposure, Self-Forcing progressively darkens, and some diffusion-forcing methods recover sporadically with low fidelity.
- Attention-window ablation: A 9-latent-frame attention window raises visual stability from 40.12 to 52.50 but increases inconsistency by reducing available context.The ablation measures visual stability on 50-second videos.
- GRPO ablation: GRPO suppresses optical-flow spikes associated with abrupt transitions, improving long-range consistency and perceptual quality.The comparison computes variance with window size 8 and attributes the spikes to rolling-window inference.
- Training-budget scaling: 25× training budget produces a 255-second video with negligible quality loss, while 20× yields stable high-fidelity videos lasting over 50 seconds.At 4× the model maintains semantic coherence longer, and at 8× it begins generating detailed backgrounds and more accurate subjects.
5 Conclusion
Self-Forcing++ mitigates error accumulation in autoregressive long-video generation by using a short-video teacher to guide the student on self-generated long rollouts. It extends generation beyond four minutes while maintaining high fidelity and introduces Visual Stability for long-video evaluation.
- The method mitigates error accumulation in autoregressive long-video generation.
- Self-Forcing++ uses a short-video teacher to guide the student on its own self-generated long rollouts, avoiding long-video supervision.
- Over 4 minutes of video generation represents a 50× improvement over the baseline while maintaining high fidelity.
- The paper proposes Visual Stability to address critical biases in existing long-video evaluation benchmarks.
6 Limitations and Further work
The method remains constrained by slower self-rollout training and the underlying model’s limited capacity. It also lacks long-term memory, which can cause content divergence in regions occluded for extended periods.
- Self-Forcing++ trains more slowly than teacher-forcing because self-rollout training is costly.
- Without long-term memory, content can diverge in regions that remain occluded for extended periods.
- Future work targets parallelized self-rollout training, latent-vector fidelity control, KV-cache quantization or normalization, and long-term memory mechanisms.
7 Discussion
Self-Forcing++ is part of a group of autoregressive methods that generate videos lasting several minutes. Related approaches differ in their noise schedules, attention mechanisms, context handling, and KV-cache strategies.
- Rolling Forcing applies progressively varied noise levels across video frames and uses attention sink frames to balance short- and long-term consistency.
- Rolling Forcing improves training efficiency by sampling non-overlapping frames.
- LongLive uses KV re-caching for prompt switching, clean contexts, and attention sink frames to mitigate error accumulation.
- Rolling Forcing, LongLive, and Self-Forcing++ can generate high-quality videos up to several minutes long.
8 Appendix
The appendix reports additional evaluation data, implementation details, and background on distillation and autoregressive conversion. It also examines noise injection as a way to simulate accumulated errors in long sequences.
- The appendix provides full VBench dimension-level results beyond the aggregated metrics reported in the main tables.
- Table 4 compares models across multiple quality metrics on 50s, 75s, and 100s videos, including VBench Long aggregates.
- The model uses Wan2.1-T2V-1.3B, sampled 16K ODE training trajectories, and filtered LLM-extended VidProM data.
- Training uses batch size 8, specified generator and critic learning rates, a generator-to-critic update ratio of 5, and a 21-frame rolling KV-cache window.
- MAGI-1 and SkyReels-V2 use variable noisy-context injection but are insufficient to mitigate error accumulation during long-video rollouts.
- DMD distills the bidirectional video model into a few-step model before causal attention converts it into an autoregressive generator.
8.4 Improving Long-Term Smoothness via GRPO
This section describes GRPO-based modeling and evaluates temporal repetition in long-video generation. The method uses importance-weighted, clipped policy terms and group-relative advantages, while KV-cache-only generation shows stronger resistance to repetition.
- GRPO formulation: GRPO uses importance weights comparing current and old policies, with clipping controlled by ε.The importance weight is ρt,i = πθ(at,i | st,i) / πθold(at,i | st,i).
- GRPO formulation: The advantage Ai is computed across a group of G generated outputs.The section introduces group-based advantage computation for the GRPO objective.
- Video generation formulation: Gaussian noise is injected at each non-terminal step to formulate the probability of the final autoregressively generated video.The formulation conditions on previously generated samples and uses T non-terminal sampling steps across N autoregressive steps.
- NoRepeat evaluation: KV-cache-only methods such as Self-Forcing and Self-Forcing++ resist repeated temporal patterns without recomputation or overlapping frames.NOVA, MAGI-1, and CausVid are reported as more susceptible to temporal repetition in long videos.
8.5 Evaluation with gemini-2.5-pro and manually verification
The evaluation uses Gemini-2.5-Pro and manual verification to assess long-video quality and exposure stability. Self-Forcing++ sustains long-term visual stability, while baselines exhibit method-specific degradation such as over-exposure, darkening, stagnation, or structural collapse.
- Gemini-2.5-Pro evaluation: 50-second videos generated by the method are evaluated with Gemini-2.5-Pro and compared against CausVid and Self-Forcing.The evaluation includes representative results and whole-video ratings for specified prompts.
- Evaluation results: Baselines display distinct long-video failures: CausVid over-exposes and loses motion, Self-Forcing darkens and stagnates, and MAGI-1 undergoes over-exposure and structural collapse.SkyReels-V2 generally preserves structure but shows moderate to severe over-exposure.
- Manual verification: Manual verification compares averaged annotations from two authors on 20 randomly sampled MovieGen videos with Gemini-2.5-Pro scores.For 50-second sequences, the reported Spearman rank correlation reached 100% before the passage truncates.
- Evaluation results: Self-Forcing++ achieves sustained long-term visual stability, ranking above diffusion-forcing methods, CausVid, Self-Forcing, and NOVA.SkyReels-V2 and MAGI-1 rank next, while Self-Forcing and NOVA degrade to comparable low levels.
- Exposure evaluation: The exposure rubric ranges from catastrophic exposure failure to well-exposed video, with intermediate scores describing limited or minor flaws.Gemini-2.5-Pro is instructed to explain observed exposure characteristics before assigning a score.
8.6 Discussion of Diffusion Forcing vs Autoregressive
The discussion contrasts diffusion forcing with clean-context autoregressive generation for long videos. Diffusion forcing offers stronger long-term memory but incurs training instability from many possible noise-level combinations.
- Comparison: Both diffusion-forcing methods and clean-context autoregressive models can generate long videos.The comparison includes SkyReels, MAGI, Self-Forcing, and Self-Forcing++.
- Diffusion forcing: Diffusion forcing retains many frames at different noise levels, which naturally provides better long-term memory.Different noise levels are applied to frames in the current stage.
- Diffusion forcing: Diffusion forcing trades long-term memory for training instability because the number of noise-level combinations can become extremely large.