PaperlayerView exact revision on arXiv

Source-linked AI summary

Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation

Hongzhou Zhu, Min Zhao, Guande He, Hang Su, Chongxuan Li, Jun Zhu

arXiv:2602.02214v5cs.CV

TL;DR

Existing few-step autoregressive video distillation does not theoretically bridge the architectural gap between bidirectional teachers and causal students. Causal Forcing uses an autoregressive teacher for ODE initialization before DMD, outperforming all baselines and Self Forcing by 19.3% in Dynamic Degree, 8.7% in VisionReward, and 16.7% in Instruction Following.

  • Problem

    Existing distillation methods face an architectural gap when converting bidirectional video diffusion models into causal autoregressive students, violating the frame-level injectivity required for ODE distillation.

  • Method

    Causal Forcing first trains an autoregressive diffusion model via teacher forcing, uses it for ODE initialization, and then applies DMD.

  • Results

    19.3% higher Dynamic Degree, 8.7% higher VisionReward, and 16.7% higher Instruction Following than Self Forcing, while outperforming all baselines across metrics.

  • Takeaways & Limitations

    The analysis identifies frame-level injectivity and autoregressive ODE initialization as central to bridging the architectural gap in autoregressive diffusion distillation.

  • Takeaways & Limitations

    The work is scoped to converting bidirectional diffusion models into few-step autoregressive generators, where the architectural gap remains a central challenge.

Abstract

from arXiv · show

To achieve real-time interactive video generation, current methods distill pretrained bidirectional video diffusion models into few-step autoregressive (AR) models, facing an architectural gap when full attention is replaced by causal attention. However, existing approaches do not bridge this gap theoretically. They initialize the AR student via ODE distillation, which requires frame-level injectivity, where each noisy frame must map to a unique clean frame under the PF-ODE of an AR teacher. Distilling an AR student from a bidirectional teacher violates this condition, preventing recovery of the teacher's flow map and instead inducing a conditional-expectation solution, which degrades performance. To address this issue, we propose Causal Forcing, which uses an autoregressive teacher for ODE initialization to bridge the architectural gap, and then applies the same DMD procedure as in Self Forcing. Empirical results show that our method outperforms all baselines across all metrics, surpassing the SOTA Self Forcing by 19.3\% in Dynamic Degree, 8.7\% in VisionReward, and 16.7\% in Instruction Following. Project page: \href{https://thu-ml.github.io/CausalForcing.github.io/}{https://thu-ml.github.io/CausalForcing.github.io/}; the code: \href{https://github.com/thu-ml/Causal-Forcing}{https://github.com/thu-ml/Causal-Forcing}.

1. Introduction

Causal Forcing attributes autoregressive video diffusion degradation to an unresolved architectural gap and violated frame-level injectivity during ODE initialization. It bridges the gap with an autoregressive teacher before applying DMD, outperforming all evaluated baselines across metrics.

  • Motivation: Recent autoregressive video diffusion models support real-time and interactive applications through frame-level autoregression with diffusion within each frame.Applications include world modeling, game simulation, and embodied intelligence.
  • Problem: Existing methods distill bidirectional video diffusion models into few-step autoregressive students using ODE initialization followed by DMD, but face a fundamental architectural gap.The gap is not resolved by the later DMD stage and must be addressed during ODE initialization.
  • Problem: Distilling an autoregressive student directly from a bidirectional teacher violates frame-level injectivity, allowing one noisy frame to map to multiple clean frames and producing blurred, inconsistent videos.Frame-level injectivity requires each noisy frame to correspond uniquely to a clean frame under the autoregressive teacher’s PF-ODE.
  • Method: Causal Forcing bridges the architectural gap by initializing ODE distillation with an autoregressive teacher, then applying causal ODE distillation and DMD.The autoregressive teacher is trained using teacher forcing, which the paper reports as superior to diffusion forcing theoretically and empirically.
  • Results: Causal Forcing consistently outperforms all evaluated baselines across every metric, with gains in dynamic degree, visual quality, and instruction-following capability.The evaluations compare against various baseline models under comprehensive testing.

2. Background

The background contrasts full-sequence and autoregressive video diffusion for interactive generation, reviews teacher-forcing and diffusion-forcing training, and introduces ODE and score-based distillation for few-step models.

  • Interactive video diffusion: Full-sequence diffusion generates all frames simultaneously, whereas autoregressive diffusion generates frames sequentially so users can steer subsequent content.Autoregressive models target an autoregressive factorization of N-frame video distributions.
  • Interactive video diffusion: Teacher forcing conditions on clean past frames, while diffusion forcing adds independent noise and conditions on a noisy prefix.Teacher forcing can concatenate clean and noisy videos with a causal attention mask; diffusion forcing lets the current noisy frame attend directly to the noisy prefix.
  • Few-step diffusion distillation: ODE distillation trains a student flow map to regress each noisy state to the clean endpoint on the teacher’s PF-ODE trajectory.The flow map is Gθ:(xt,t) 7→ x0, and recent interactive-video methods use direct regression as a simplified consistency-distillation variant.
  • Few-step diffusion distillation: Distribution Matching Distillation minimizes the divergence between student and data distributions by matching score predictions from frozen real and trainable fake diffusion models.The student generates ˜x, which is independently noised to induce pθ,t, while sreal and sfake predict scores under noisy data and student distributions.

3. Method

Causal Forcing addresses autoregressive video distillation’s architectural gap by showing that ODE initialization requires frame-level injectivity and by using an autoregressive teacher. It combines teacher-forced autoregressive diffusion training, causal ODE distillation, and asymmetric DMD to recover the correct flow map and outperform Self Forcing.

  • Injectivity Analysis: ODE distillation for an autoregressive student requires each noisy frame to map injectively to a unique clean frame under an autoregressive teacher’s PF-ODE.Bidirectional teachers are injective at the video level but can map the same noisy frame to multiple clean frames because they denoise using all frames.
  • Diagnosing Self Forcing: Self Forcing’s DMD stage cannot resolve the architectural gap when initialization retains the mismatch between bidirectional and autoregressive models.Removing only the sampling-step gap leaves performance significantly degraded, indicating that the subsequent DMD stage is insufficient.
  • Injectivity Analysis: When frame-level injectivity fails, the autoregressive student cannot recover the teacher’s flow map and instead learns a conditional expectation, producing blurry videos.The bidirectional teacher’s use of future-frame information creates information loss for the causal student and causes distribution mismatch.
  • Causal Forcing: Causal Forcing uses an autoregressive diffusion teacher to satisfy frame-level injectivity and consists of teacher-forced training, causal ODE distillation, and asymmetric DMD.Teacher forcing aligns training with the causal inference process, while causal ODE initialization supplies the appropriate starting point for DMD.
  • Causal Consistency Distillation: Causal consistency distillation likewise leverages a native autoregressive teacher’s frame-level injectivity, whereas asymmetric consistency distillation violates injectivity and collapses.The causal consistency framework substantially outperforms asymmetric consistency distillation and can potentially initialize DMD.

4. Experiments

Experiments evaluate Causal Forcing against bidirectional, autoregressive, and distilled autoregressive video models using VBench, VisionReward, and Instruction Following. The method consistently outperforms all baselines across metrics, while ablations support teacher forcing and causal ODE initialization.

  • Evaluation: Evaluation uses VBench for primary assessment, VisionReward for visual quality, and Instruction Following for instruction adherence.VisionReward correlates with human judgments, and higher scores are better even when values are negative.
  • Performance comparison: Causal Forcing consistently outperforms all baselines across metrics, achieving the best dynamic degree, visual quality, and instruction following ability.Compared baselines include bidirectional models, autoregressive diffusion models, and distilled autoregressive models.
  • Performance comparison: 2079% higher throughput is achieved than comparable-scale Wan2.1 while matching its performance and surpassing it in some respects.The comparison also reports substantially faster inference than Wan2.1.
  • Ablation studies: 111.2% VisionReward improvement favors teacher forcing over diffusion forcing across all metrics during autoregressive diffusion training.Diffusion forcing produces higher dynamic degree, but the passage attributes this largely to collapse that pathologically inflates the motion metric.
  • Ablation studies: Causal ODE initialization substantially outperforms Self Forcing’s ODE initialization in ODE-initialized DMD.The ablation compares autoregressive diffusion training, score distillation, and consistency distillation under chunk-wise and selected frame-wise settings.

5. Further Discussion

Causal Forcing extends to long-video generation but faces degradation from a training–inference gap when extrapolated beyond its 5-second training context. It also differs from GAN-based autoregressive distillation through its theoretical analysis, DMD-based algorithm, and temporal KV-cache architecture.

  • Long video generation: Causal Forcing is trained on 5s videos with at most 5s attention, so directly extrapolating it to longer videos creates a training-inference gap and degrades performance.Long-video adaptation methods such as training-based LongLive are described as orthogonal solutions.
  • Distinction from other AR distillation paradigms: Unlike GAN-based APT2, Causal Forcing follows the asymmetric DMD paradigm of CausVid and Self Forcing while using a bidirectional diffusion model as the DMD teacher.APT2 instead adopts GAN-based autoregressive distillation and uses teacher-forcing CD for initialization.
  • Distinction from other AR distillation paradigms: Causal Forcing theoretically shows that forward-KL-based distillation, including ODE/CD, should use an autoregressive teacher to train an autoregressive student.The passage contrasts this analysis with APT2, which provides no theoretical analysis.
  • Distinction from other AR distillation paradigms: Causal Forcing differs architecturally from APT2 by supporting temporal KV cache inference.The cited passage presents this as part of Causal Forcing’s algorithmic and architectural distinction from APT2.

6. Conclusion · A. Extended Related Work

The paper identifies frame-level injectivity as essential for ODE initialization and introduces Causal Forcing, which uses an autoregressive teacher before DMD. Extended related work situates interactive autoregressive diffusion models within video generation, world modeling, entertainment, and long-video generation.

  • 6. Conclusion: Existing autoregressive video diffusion distillation methods violate the frame-level injectivity requirement needed to bridge the architectural gap during ODE initialization.The analysis identifies architectural-gap bridging as essential and frames frame-level injectivity as a fundamental requirement.
  • 6. Conclusion: Causal Forcing trains an autoregressive diffusion model with teacher forcing, then uses it as the ODE-distillation teacher before the subsequent DMD stage.This ordering is designed to address the limitations identified in existing distillation methods.
  • 6. Conclusion: Causal Forcing consistently outperforms all baselines across all metrics, demonstrating the effectiveness of its proposed distillation procedure.The conclusion reports consistent gains across every evaluated metric without specifying numerical values in the supplied passage.
  • A. Extended Related Work: Diffusion models and scalable diffusion transformers have motivated extensive research on video generation and large-scale DiT-based video models.The related work cites numerous diffusion-based and DiT-based video-generation efforts.
  • A. Extended Related Work: Recent autoregressive diffusion video models combine frame-wise autoregressive generation with diffusion applied within each frame to support interactive video generation.Examples include Pyramid Flow, MAGI-1, and SkyReels-v2.
  • A. Extended Related Work: Real-time interactive video generation supports video world modeling, enabling users to explore simulated environments and take actions within realistic world simulations.The passage identifies HY-WorldPlay, RELIC, Hunyuan-GameCraft-2, and Yume-1.5 as examples.
  • A. Extended Related Work: Interactive autoregressive diffusion models also support entertainment and media applications, while related systems demonstrate strong long-video generation capabilities.Examples include Knot Forcing, Live avatar, Motionstream, Rolling Forcing, LongLive, Self-Forcing++, and Dee.

B. Proofs of Propositions

The proofs assume standard regularity conditions ensuring that all expectations are finite and well-defined and all involved probability density functions are integrable.

  • Assumptions: The proofs assume finite, well-defined expectations and integrable probability density functions, as standard mild conditions in diffusion modeling.These assumptions follow the regularity conditions described by Song et al. (2020).

B.1. The Flaw of Self Forcing’s ODE Distillation

The section proves that PF-ODE mappings can be non-injective within chunks, so chunk-level regression cannot recover the original chunk-wise data distribution. Instead, squared-loss training converges to a conditional mean, producing a distribution mismatch.

  • Lemma B.1: Lemma B.1 establishes chunk-wise PF-ODE non-injectivity when the clean output varies with coordinates omitted from the input chunk.The proof constructs distinct omitted-coordinate values that share the same chunk input but map to different clean chunk outputs.
  • Lemma B.1: The resulting conditional variance is positive with nonzero probability, showing that omitted coordinates leave multiple possible clean chunk outputs.The argument states P(Var(ϕ(xt, t)u | xu_t, t) > 0) > 0.
  • Proposition B.2: Under squared-loss chunk-level regression, the optimal model is the conditional mean rather than a sample from the chunk-wise data distribution.This is the central conclusion of Proposition B.2.
  • Proposition B.2: Because bidirectional DiT diffusion models use nonconstant attention, the lemma’s non-injectivity condition applies and the learned output distribution differs from the target.The distribution mismatch follows from positive conditional variance and the regression projection argument.

B.2. Distribution Mismatch in Autoregressive Diffusion Forcing · C. More Discussion of Our Method

The section formalizes distribution mismatch in autoregressive diffusion forcing under three assumptions and proves that diffusion-trained conditional distributions differ from the desired data conditionals. It derives this mismatch through the Markov structure induced by independent noising and notes that recent strategies perform on par with teacher forcing in the evaluated setting.

  • B.2. Distribution Mismatch in Autoregressive Diffusion Forcing: The analysis introduces three regularity assumptions before proving Proposition 3.4.These assumptions concern optimal diffusion-training conditionals, dependence between X and Y, and a positive non-degenerate posterior kernel.
  • B.2. Distribution Mismatch in Autoregressive Diffusion Forcing: (A1) assumes the diffusion-trained model yields the optimal conditional distribution pDF(X | Z) = pdata(X | Z).The passage identifies this as a standard assumption under the diffusion training objective.
  • B.2. Distribution Mismatch in Autoregressive Diffusion Forcing: (A2) assumes X and Y are dependent, reflecting that frames within a video are closely related rather than independent.Formally, pdata(X | Y = y) is not pdata(Y)-a.e. constant.
  • B.2. Distribution Mismatch in Autoregressive Diffusion Forcing: (A3) assumes pdata(Y | Z = y) is positive and non-degenerate on the support of pdata(Y).In particular, its density is positive for every y in that support.
  • B.2. Distribution Mismatch in Autoregressive Diffusion Forcing: The proof shows that querying the diffusion-trained model at a clean prefix yields pdata(X | Z = y), not generally pdata(X | Y = y).By (A1), pDF(X | y) = pDF(X | Z = y) = pdata(X | Z = y), and the argument establishes a nonzero divergence from pdata(X | y).
  • B.2. Distribution Mismatch in Autoregressive Diffusion Forcing: The contradiction argument uses the Markov chain X → Y → Z and the tower property to show that equality would force pdata(X | Y = y) to be almost everywhere constant.The fixed-point property of the conditional-expectation operator under (A3) contradicts the dependence assumption (A2).
  • B.2. Distribution Mismatch in Autoregressive Diffusion Forcing: Recent autoregressive diffusion training strategies perform on par with teacher forcing in the reported setting.This comparison is stated in the caption of Table 3.

C.1. Further Remarks on Autoregressive Diffusion Training Strategies

Diffusion forcing is suboptimal for autoregressive diffusion training but remains useful for bidirectional video continuation. Experiments with PFVG, BAgger, and Resampling Forcing found no significant improvement over teacher forcing in the 5s setting.

  • Diffusion Forcing: Diffusion forcing is suboptimal for autoregressive diffusion training but remains useful for bidirectional video continuation.It was originally introduced to train bidirectional diffusion models for long-video continuation.
  • Diffusion Forcing: At inference, continuation concatenates a clean prefix with noise, matching diffusion forcing’s training setup.The clean prefix consists of the tail frames of the given video.
  • Alternative Training Strategies: PFVG, BAgger, and Resampling Forcing provide no significant improvement over teacher forcing.These alternatives were evaluated as recent autoregressive diffusion training strategies.
  • Alternative Training Strategies: Their limited gains in the 5s setting are understandable because most were designed for long-video training and generation.The paper leaves deeper investigation of these strategies for future work.

C.2. Multi-Step Autoregressive Diffusion as Initialization for Asymmetric DMD · C.3. Causal ODE Distillation from Bidirectional Initial Model

Multi-step autoregressive diffusion improves asymmetric DMD initialization but leaves a few-step causal mismatch, which causal ODE distillation resolves more effectively. Evidence further indicates that paired-data construction, rather than student initialization, primarily explains the ODE-distillation performance gap.

  • C.2. Multi-Step Autoregressive Diffusion as Initialization for Asymmetric DMD: Teacher Forcing-trained multi-step autoregressive diffusion substantially improves asymmetric DMD dynamics and visual quality over Self Forcing’s ODE initialization.These gains are illustrated by Fig. 8’s middle-versus-left comparison.
  • C.2. Multi-Step Autoregressive Diffusion as Initialization for Asymmetric DMD: Multi-step autoregressive diffusion narrows the bidirectional-to-causal gap under 50-step sampling but does not fully resolve it in the few-step regime.At low step counts, autoregressive conditioning depends on preceding frames whose quality degrades, creating an additional conditional-context mismatch.
  • C.2. Multi-Step Autoregressive Diffusion as Initialization for Asymmetric DMD: Causal ODE distillation converts the autoregressive model into a few-step model with stronger temporal consistency, making it a more suitable DMD initialization.Replacing multi-step initialization with causal ODE initialization produces clear gains in Fig. 8.
  • C.2. Multi-Step Autoregressive Diffusion as Initialization for Asymmetric DMD: Causal ODE initialization achieves the best overall quality and VisionReward score among the DMD initialization strategies.Teacher Forcing-trained autoregressive diffusion improves substantially over Self Forcing’s ODE initialization but remains inferior to causal ODE initialization.
  • C.2. Multi-Step Autoregressive Diffusion as Initialization for Asymmetric DMD: Under 4-step generation, autoregressive diffusion shows inter-frame abrupt changes, whereas the causal ODE-distilled model remains more stable.This comparison occurs before the DMD stage and indicates suboptimal causality for the autoregressive diffusion model at 4 steps.
  • C.3. Causal ODE Distillation from Bidirectional Initial Model: Causal ODE distillation pairs data generated by an autoregressive diffusion model with a student initialized from that same autoregressive model.Self Forcing instead uses bidirectionally generated paired data and a bidirectional initial model.
  • C.3. Causal ODE Distillation from Bidirectional Initial Model: With autoregressive paired data, initializing the student from a bidirectional model yields comparable quality to autoregressive initialization and remains much better than Self Forcing’s ODE distillation.This indicates that the ODE-distillation performance gap is not primarily caused by student initialization.

D. More Implementation Details

The implementation constructs causal training data from a teacher-forcing autoregressive diffusion model, uses shared four-step inference, and evaluates performance on dynamic, visual-quality, and instruction-following criteria. Causal consistency distillation achieves better quality and stability than asymmetric consistency distillation, supporting the need for a native causal teacher.

  • Training details: The method trains on about 3K bidirectional-model samples, first fits an autoregressive teacher for 2K steps, then generates a 3K-sample causal ODE dataset.The initial dataset uses VidProM prompts and Wan-generated videos.
  • Optimization and inference: All training uses batch size 64 and Adam with learning rate 2 × 10^-6, β1 = 0, and β2 = 0.999; inference uses four shared timesteps.The four timesteps are 1, 0.9375, 0.8333, and 0.625.
  • Causal consistency distillation: Causal consistency distillation uses LCM with 48 discretized timesteps, UniPC, EMA rate 0.99, 3K training steps, and the same four-step inference schedule.The method uses an x0-prediction form for Gθ, which satisfies the required boundary conditions without an additional wrapped-network design.
  • Causal consistency distillation: Causal consistency distillation produces better-quality, more stable results than asymmetric consistency distillation, which appears blurry and exhibits abrupt artifacts.The comparison highlights the necessity of a native causal teacher for autoregressive consistency distillation.
  • Evaluation details: Evaluation uses 100 prompts with rich action sequences and dynamics, covering Dynamic Degree, VisionReward, and Instruction Following.Dynamic Degree uses VBench’s custom-input evaluation on the 100-prompt motion set, while VisionReward measures overall visual quality.
Loading 2602.02214v5…