Source-linked AI summary
OPSD-V: On-Policy Self-Distillation for Post-Training Few-Step Autoregressive Video Generators
Hongyu Liu, Chun Wang, Feng Gao, Xuanhua He, Yue Ma, Ziyu Wan, Yong Zhang, Xiaoming Wei, Qifeng Chen
TL;DR
Few-step AR video generators remain vulnerable to long-horizon degradation because existing teacher supervision is limited by short-clip temporal range. OPSD-V uses real long-video context to create cleaner teacher caches while supervising the student on its own rollout, improving video quality and motion dynamics without changing inference cost or path.
Problem
Existing few-step AR video generators lack teacher supervision that directly covers truly long autoregressive trajectories, limiting long-horizon generation.
Method
OPSD-V post-trains an on-policy student on its own inference-time rollout states while using real long-video context to construct cleaner teacher caches.
Results
OPSD-V improves video quality and motion dynamics on Self-Forcing and LongLive while preserving the same model scale and inference cost.
Takeaways & Limitations
Real long-video context can provide dense denoising-level correction under self-generated KV-cache states without changing the original few-step inference path.
Takeaways & Limitations
The authors identify more effective teacher-cache policies, adaptive supervision schedules, and stronger context-enhanced teachers as directions for further improvement.
Abstract
from arXiv · showhide
We propose OPSD-V, an on-policy self-distillation paradigm for post-training few-step autoregressive (AR) video diffusion models. Existing few-step AR video generators can produce long videos with low latency, but still suffer from error accumulation and weakened motion dynamics during long autoregressive rollout. OPSD-V reduces long-horizon degradation while preserving the original few-step inference path. The key idea is to introduce real long-video data as temporal context during training and use it to provide dense trajectory-level supervision. Specifically, the student follows the exact inference-time rollout, generating each chunk conditioned on its own previously generated KV cache. In parallel, the teacher is evaluated at the same student-visited denoising states, but uses a cleaner AR-consistent temporal cache in which older history can be replaced by real-video context. This provides dense denoising-level corrective targets under on-policy AR cache dynamics, without changing the sampler, number of denoising steps, or inference-time cache mechanism. We apply OPSD-V to representative few-step AR video models, including Self-Forcing and LongLive. Experiments show consistent improvements in visual quality, motion dynamics, and VBenchLong scores. A user study with 10 participants comparing 20 video pairs shows that OPSD-V is preferred over the base models in 66.0% of overall-preference judgments (82.5% excluding ties).
1 Introduction
Existing few-step AR video generators face long-horizon degradation because short-clip teacher supervision does not directly cover long autoregressive trajectories. OPSD-V uses real long-video context to construct cleaner teacher targets on the student’s own rollout while preserving the efficient inference path.
- Motivation: Few-step AR generators still suffer long-horizon error accumulation and weakened motion dynamics despite low-latency generation.Their DMD-style supervision is tied to the temporal range and quality of finite short-clip teachers.
- Core idea: OPSD-V treats real long videos as privileged temporal context rather than direct output targets, avoiding semantic and temporal mismatch with student rollouts.The student remains fully on-policy while the teacher uses richer long-video context.
- Method: The teacher replaces older generated cache history with corresponding real-video chunks while retaining the most recent generated chunk, preserving autoregressive continuation from the student state.Dense velocity matching then operates along the fixed few-step denoising trajectory without changing the low-latency sampling path.
- Diagnostic evidence: A test-time cache intervention using real older history visibly stabilizes LongLive rollouts without updating parameters or changing the denoising sampler.The observation identifies degradation in generated KV-cache history as a key bottleneck.
- Evaluation: OPSD-V is evaluated on Self-Forcing and LongLive using LoRA-based continued training on 3,800 customized long videos of approximately one minute each.Self-Forcing is equipped with the same attention-sink mechanism used for long-video evaluation, while LongLive provides a streaming long-video backbone.
2 Related Work
Prior work combines autoregressive video generation with few-step distillation and studies on-policy self-distillation in language models. OPSD-V differs by applying cache-aware, dense denoising supervision to an already efficient few-step AR video model.
- Video diffusion models: Video diffusion and flow-matching systems have improved text-to-video fidelity, motion quality, prompt following, and temporal coherence.Representative systems include CogVideoX, HunyuanVideo, Wan, and LongCat-Video.
- AR video generation: AR video models generate frames or chunks sequentially using previous content and transformer KV caches, with forcing-based methods providing training supervision.This line of work includes teacher-forcing and diffusion-forcing approaches.
- Distinction: OPSD-V focuses on post-training an existing efficient few-step AR video model rather than developing a new causal generator.Its supervision is applied densely on the student’s own inference-time cache states.
- On-policy self-distillation: On-policy self-distillation trains models on trajectories they actually sample while obtaining stronger teacher distributions from richer context.The same model can act as student and teacher under different contexts.
3 Preliminaries
Few-step AR video generation factorizes videos into sequential latent chunks, using fixed-step denoising and KV caches to carry historical context. On-policy self-distillation matches student and stronger teacher distributions along student-sampled trajectories.
- AR formulation: A causal AR video diffusion model generates a video as a sequence of latent chunks conditioned on a prompt or other conditioning signal.The video distribution is factorized over sequential chunks.
- KV-cache dynamics: Historical context is represented by a transformer KV cache that is updated by appending entries computed from each generated clean chunk.Future chunks therefore depend recursively on previously generated chunks through the evolving cache.
- Few-step denoising: Each chunk is generated from Gaussian noise through a fixed small number of denoising steps, with a velocity predictor specifying the solver’s denoising direction.The solver transitions the noisy latent through the timestep schedule.
- Few-step distillation: DMD compresses a slow diffusion teacher into a one-step or few-step generator by matching student and teacher or data distributions.CausVid and Self-Forcing apply related distillation ideas to causal video generation.
- On-policy self-distillation: OPSD matches a student distribution conditioned on its sampled prefix with a teacher distribution using additional privileged context.The matching occurs along the student-sampled trajectory.
- Diffusion extension: D-OPSD preserves few-step inference behavior by supervising a student along its own denoising trajectory while conditioning the teacher on richer paired context.OPSD-V extends this principle to few-step autoregressive video generation.
4 Method
OPSD-V post-trains few-step causal AR video generators by keeping the student on its inference-time rollout while giving an EMA teacher cleaner, AR-consistent temporal context. Dense velocity supervision is applied at the student’s visited denoising states without changing the sampler or inference cache mechanism.
- Student on-policy rollout: OPSD-V uses a shared real-video prefix to initialize both branches, while the student then follows the fixed few-step inference-time rollout with its generated chunks written into the KV cache.The shared prefix anchors the scene but is excluded from generation and loss computation.
- Student on-policy rollout: The student records its noisy latent states and denoising timesteps, then the teacher is evaluated at those same states rather than sampling an independent denoising trajectory.Stop-gradient through rollout transitions prevents later losses from backpropagating through the full autoregressive history.
- AR-consistent teacher cache: For each supervised chunk, the teacher replaces older generated history with corresponding real-video chunks while retaining the latest student-generated chunk.This cache policy reduces accumulated teacher-context error while preserving continuation from the student’s current autoregressive state.
- Dense denoising-level objective: The training objective matches student and teacher velocity predictions at the same student-visited noisy latents and fixed denoising timesteps.The teacher prediction is stop-gradient, and supervision is defined over supervised chunk-step pairs.
- Training procedure: The implementation uses a four-step sampler, supervises all four denoising steps, and excludes the first M = 7 chunks as rollout warm-up.The warm-up corresponds to the Wan-based models’ local seven-chunk training window, or 3×7 = 21 latent frames.
- Training procedure: Memory-efficient training backpropagates each normalized loss term immediately, accumulates gradients without updating the model, and releases the current activation graph.This avoids retaining graphs across the complete long-video rollout.
5 Experiments
OPSD-V post-training improves long-horizon video generation for Self-Forcing and LongLive under unchanged four-step inference and KV-cache settings. Experiments show stronger visual quality and motion preservation, while ablations support velocity matching and on-policy student states.
- Experimental Setup: OPSD-V evaluates Self-Forcing and LongLive using a customized dataset of 3,800 approximately one-minute videos and LoRA-based continued training.Both models use the Wan2.1-T2V-1.3B backbone.
- Experimental Setup: OPSD-V preserves the original four-step sampler, inference-time KV-cache mechanism, and low-latency path during evaluation.The student follows the inference-time rollout, while the teacher uses real-video older history and the latest student-generated chunk.
- Qualitative Comparison: Across LongLive and Self-Forcing examples, OPSD-V better preserves scene structure and motion dynamics while reducing blur, artifacts, and late-stage drift.The comparison uses identical prompts, seeds, samplers, and attention-sink settings.
- Quantitative Comparison: OPSD-V improves Quality Score and Dynamic Degree on both backbones without increasing inference cost, while Semantic Score remains comparable with a slight decrease.Results are averaged over MovieGenBench and MeiBench using VBenchLong.
- User Study: User preference favors OPSD-V in 66.0% of overall judgments, rising to 82.5% when ties are excluded.The study includes 10 participants comparing 20 paired videos each across LongLive and Self-Forcing.
- Ablation Study: Velocity matching preserves later high-frequency structure better than clean-latent matching, and teacher-trajectory supervision fails because it mismatches student-visited states.The final design therefore evaluates teacher predictions on the student’s on-policy denoising trajectory.
6 Analysis and Future Work
OPSD-V uses real long videos as privileged temporal context rather than direct targets, allowing on-policy corrective supervision while preserving the student’s inference-time rollout. The authors identify scaling data and exploring alternative cache and loss designs as future directions.
- Real long videos provide richer temporal structure and cleaner long-range dynamics than short-clip teacher distributions.
- OPSD-V keeps the student on its own rollout while using cleaner real-video cache states for dense denoising-level corrective supervision.
- Direct teacher forcing would break the inference-time rollout distribution, while reconstruction targets would not correct generated-cache states.
- Future work: Performance may benefit from increasing the amount, diversity, and quality of training videos and training compute.
- Future work: Alternative teacher-cache policies, adaptive supervision schedules, and stronger context-enhanced teachers may further improve long-horizon generation.
7 Conclusion
OPSD-V is a cache-aware on-policy self-distillation framework for few-step autoregressive video generation. It supervises student rollout states with real long-video context, improving video quality and motion dynamics without changing the original inference path or inference cost.
- OPSD-V supervises the student on its own inference-time rollout states while using real long-video context to construct cleaner teacher caches.
- The framework provides dense denoising-level correction under self-generated KV-cache states without changing the original few-step inference path.
- Experiments on Self-Forcing and LongLive improve video quality and motion dynamics while preserving the same model scale and inference cost.