Source-linked AI summary
Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout
Zhuoran Zhao, Shengju Qian, Tongtong Liang, Xianghao Kong, Songchun Zhang, Junchao Huang, Guian Fang, Xin Wang, Pan Hui, Anyi Rao
TL;DR
AR video diffusion distillation can yield over-saturated and over-smoothed videos because DMD’s reverse-KL objective seeks limited teacher modes and rollout errors accumulate. Mask Forcing uses dual-noise masking to diversify self-rollouts and provide cleaner denoising context. Across multiple distillation methods and settings, it improves visual quality and convergence without real video data or additional post-training.
Problem
Self-rollout DMD can produce over-saturated, over-smoothed videos because reverse-KL mode seeking concentrates the student on limited teacher modes and intermediate predictions accumulate errors.
Method
Mask Forcing injects lower-noise signals into rollout inputs through random spatial and temporal masks, diversifying student trajectories and guiding noisier tokens with cleaner context.
Results
Mask Forcing improves visual quality across multiple AR video distillation methods in chunk-wise and frame-wise settings, with faster convergence and no real video data or additional post-training.
Takeaways & Limitations
Perturbing self-rollout trajectories lets DMD receive learning signals beyond the modes already covered by the student while reducing rollout error accumulation.
Abstract
from arXiv · showhide
Autoregressive (AR) video diffusion models have shown great potential in real-time video generation. Recent methods distill pretrained bidirectional video diffusion models into causal AR students through Distribution Matching Distillation (DMD), but the generated videos often suffer from over-saturation and over-smoothing issues, resulting in limited visual quality and realism. The key contributing factor is the mode-seeking behavior of the reverse KL objective in DMD, which can cause the student distribution to collapse onto only a few modes of the teacher distribution. To address this, we propose Mask Forcing, a Dual-Noise Masking Rollout strategy that perturbs the AR student self-rollout to mitigate mode collapse induced by reverse-KL mode seeking. The core idea is to inject cleaner signals into noisy rollout inputs via random masks along spatial and temporal axes during the self-rollout process of AR diffusion distillation. Such perturbations encourage the student rollouts to explore more regions of the teacher distribution, allowing DMD to provide learning signals beyond the modes already covered by the student. Moreover, the cleaner tokens act as denoising guidance for other noisier tokens, improving the intermediate rollout predictions and reducing error accumulation. Extensive experiments demonstrate that our method improves multiple AR video diffusion distillation methods with higher visual quality efficiently, without incorporating real video data or additional post-training stages.
1 INTRODUCTION
AR video diffusion enables real-time generation, but self-rollout DMD can produce over-saturated, over-smoothed videos because reverse-KL mode seeking and rollout error accumulation limit quality. Mask Forcing perturbs rollouts with dual-noise masking, broadening exploration and using cleaner tokens to improve denoising, with experiments showing better quality and faster convergence without real video data or extra post-training.
- AR diffusion models target real-time video generation, addressing the latency limits of bidirectional models with multiple denoising timesteps.
- Self-rollout combined with DMD often produces over-saturated and over-smoothed videos with limited visual quality and realism.
- Reverse-KL mode seeking can collapse the student onto high-probability teacher regions, while unsupervised intermediate rollouts accumulate errors.
- Mask Forcing injects lower-noise tokens through random spatial and temporal masks, diversifying rollout trajectories and providing cleaner denoising context.
- Experiments across chunk-wise and frame-wise settings improve visual quality and accelerate convergence across multiple AR video distillation methods without real video data or additional post-training.
2 RELATED WORK
Prior work addresses causal video diffusion through Teacher Forcing, Diffusion Forcing, and distribution-matching or consistency objectives, but each leaves alignment, diversity, or quality trade-offs. Mask Forcing instead retains DMD and broadens the student rollouts through dual-noise masking, while masked modeling provides related context for recovering masked inputs.
- Causal training paradigms: Teacher Forcing suffers train-test gap and exposure bias, while Diffusion Forcing reduces distribution shift but still fails to align training with inference.
- Mode seeking and covering: DMD’s reverse-KL objective can improve sample fidelity but concentrate the student distribution on a limited subset of teacher modes.
- Mode seeking and covering: Forward-divergence consistency objectives cover more teacher modes but may average across modes and reduce sample quality.
- Mask Forcing: Mask Forcing retains DMD and perturbs student rollouts with dual-noise masking to broaden the distribution exposed to DMD.
- Masked modeling: Masked modeling masks part of an input and trains a model to recover it from visible context, supporting representation and generative learning.
3 METHOD
Mask Forcing modifies self-rollout DMD training with dual-noise masking to broaden teacher-mode coverage and provide cleaner denoising context. The method targets reverse-KL mode collapse and error accumulation in autoregressive video diffusion rollouts.
- 3.1 PRELIMINARIES: AR video generation factorizes videos into chunks and generates each chunk conditionally through diffusion denoising using preceding chunks as context.The preceding context is stored in a key-value cache.
- 3.1 PRELIMINARIES: Reverse-KL DMD can concentrate the student on limited teacher modes, while completed-rollout supervision leaves intermediate prediction errors unsupervised and able to accumulate.These limitations are associated with over-saturation, over-smoothing, and reduced realism.
- 3.2.2 SELF-ROLLOUT WITH DUAL-NOISE MASKING: Mask Forcing injects lower-noise signals into noisy rollout inputs through random masks applied within and across video chunks.The base denoising schedule remains, while an additional lower-noise timestep supplies cleaner tokens.
- 3.2.2 SELF-ROLLOUT WITH DUAL-NOISE MASKING: At each denoising step, masked positions use cleaner re-noised tokens while unmasked positions retain the original noisier tokens, creating a dual-noise input.The student is conditioned on the scheduled timestep, and the perturbation replaces standard re-noising during self-rollout.
- 3.2.2 SELF-ROLLOUT WITH DUAL-NOISE MASKING: Cleaner tokens provide denoising context for noisier tokens, while masking perturbs trajectories so DMD evaluates a broader region of the sample space.The resulting rollout distribution can mix trajectory-conditioned student distributions and collectively cover more teacher-supported regions.
4 EXPERIMENTS
Experiments evaluate Mask Forcing across multiple AR video distillation baselines, generation settings, benchmarks, qualitative comparisons, ablations, convergence analyses, and human preferences. The method improves visual quality and distributional alignment, while mask and timestep choices reveal a quality–motion trade-off.
- Comparisons with Baselines: Mask Forcing improves baseline methods on visual quality and semantic score under both chunk-wise and frame-wise settings, while also improving motion metrics overall.On VBench, it surpasses the base methods across reported metrics in both settings.
- Comparisons with Baselines: Qualitative comparisons show fewer over-saturation and over-smoothing artifacts, greater realism, and richer high-frequency details with Mask Forcing.These improvements are reported for both standard comparisons and single-prompt long-video generation.
- Ablation Studies: Increasing the timestep window improves HPSv3 from 9.16 at ∆=50 to 9.98 at ∆=450, but excessive windows can limit motion diversity.The adopted ∆=250 reaches HPSv3 9.84 while maintaining Dynamic Degree 82.
- Ablation Studies: Mask Forcing accelerates convergence for all evaluated baselines in both visual quality and distributional alignment.Convergence is assessed with HPSv3 and MMD in CLIP and V-JEPA2 feature spaces.
- Human Evaluation: Human evaluations favor Mask Forcing over Self Forcing, Causal Forcing, and LongLive, with preference rates of 80%, 79%, and 83%, respectively, and 72% for long-video comparison against LongLive.The study uses pairwise judgments from 24 users assessing visual quality and realism.
5 CONCLUSION
Mask Forcing uses dual-noise masking during AR video diffusion distillation to broaden rollout exploration, improve denoising guidance, and reduce error accumulation. Experiments show improved visual quality and convergence speed across multiple baselines.
- Mask Forcing injects lower-noise signals into noisy rollout inputs through random spatial and temporal masks.This dual-noise masking strategy diversifies student trajectories during self-rollout.
- The perturbations encourage coverage of more teacher-distribution regions, allowing DMD to provide learning signals beyond already-covered student modes.Cleaner tokens also guide denoising for noisier tokens, improving intermediate rollout predictions and mitigating error accumulation.
- Mask Forcing improves visual quality and convergence speed across multiple AR video distillation baselines.The method is evaluated without real video data or additional post-training stages.
6 APPENDIX
The appendix formalizes Mask Forcing’s masked-rollout distribution and shows how trajectory-dependent conditional outputs can reduce reverse-KL relative to their average. Experiments report improved quality, diversity, human preference, and convergence across multiple autoregressive video distillation settings.
- Theoretical analysis: Mask Forcing marginalizes conditional student rollout distributions over masking trajectories, yielding a fake marginal score for DMD optimization.The score-difference update operates on the marginal distribution because the fake-score model is not conditioned on the masking trajectory.
- Theoretical analysis: The reverse-KL decomposition separates average conditional reverse KL from a mutual-information term measuring dependence on the masking trajectory.The mutual-information term is nonnegative and vanishes when masking trajectories do not change the conditional output distribution.
- Implication for mode coverage: Distinct masking trajectories can produce a marginal mixture with strictly lower reverse KL than the average conditional reverse KL, improving mode coverage under limited capacity.The analysis connects this reduction to covering multiple teacher-supported regions without requiring every conditional rollout to cover them individually.
- Comparisons and quality: Mask Forcing outperforms Causal-rCM and DistillAlign in visual quality, motion dynamics, and semantic alignment, while receiving 83% and 77% preference over the two methods.Qualitative comparisons also report richer details and greater realism.
- Convergence and long videos: Mask Forcing achieves higher HPSv3 and lower CMMD in fewer LongLive training steps and maintains higher CLIP and HPSv3 scores during long-video generation.Both methods show score declines over 0–30 seconds, which the authors attribute likely to autoregressive self-rollout error accumulation.
- Diversity: Mask Forcing improves CLIP diversity across all three baselines and substantially improves DINOv3 diversity for Causal Forcing and LongLive, though Self Forcing’s DINOv3 diversity decreases slightly.The reported results associate these diversity gains with improvements in visual quality, semantic alignment, distributional alignment, and human preference.