PaperlayerView exact revision on arXiv

Source-linked AI summary

RepForcing: Representation-Space Initialization for Autoregressive Video Distillation

Siming Fu, Zheming Fu, Lianrui Mu, Haoyang Huang, Chaojie Gu, Ruizhe He

arXiv:2610.20111v1cs.CV

TL;DR

Few-step autoregressive video distillation seeks lower sampling cost, but causally compatible initialization can still leave detail unresolved. RepForcing replaces intermediate-teacher trajectory regression with teacher-forced representation-space supervision, and its four-step chunk-wise model improves both reported metrics over Causal Forcing after DMD.

  • Problem

    Causal ODE initialization provides causally compatible targets, but under few-step sampling it can still leave fine detail unresolved.

    Source: §1 · p. 2

  • Method

    RepForcing compares teacher-forced clean predictions and training targets through normalized features of a frozen video diffusion transformer, replacing intermediate-teacher training and causal ODE distillation before asymmetric DMD.

    Source: §1 · p. 2

  • Results

    84.66 VBench Total and 10.074 VisionReward exceed the reported Causal Forcing results by 0.62 points and 59.25%, respectively, for four-step chunk-wise generation.

    Source: §5.2 · p. 8

  • Takeaways & Limitations

    At four sampling steps, RepForcing better preserves generation quality than intermediate-teacher-based initialization before DMD.

    Source: §1 · p. 2

Abstract

from arXiv · show

Distilling pretrained video diffusion models into few-step autoregressive (AR) generators typically combines student initialization with distribution matching distillation (DMD). An intermediate AR teacher can provide causally compatible targets for ODE initialization. However, causally compatible targets alone do not ensure high-quality generation under a short sampling schedule. We analyze the posterior-mean predictions favored by direct latent regression and empirically find that causal ODE initialization alleviates blur but still leaves fine detail unresolved. To address this limitation, we propose Representation Forcing, which learns few-step causal initialization through representation-space supervision rather than intermediate-teacher trajectory regression. The student is trained with teacher forcing by comparing clean predictions and training targets through normalized features of a frozen video diffusion transformer. This phase reuses the synthetic videos generated by a bidirectional model for AR teacher training and replaces both teacher training and causal ODE distillation, without generating additional causal ODE trajectories. The subsequent asymmetric DMD procedure remains unchanged. Empirical results show that, at four sampling steps, RepForcing better preserves generation quality than intermediate-teacher-based initialization before DMD. After DMD, our four-step chunk-wise model improves VBench Total from 84.04 to 84.66 (+0.62 points) and VisionReward from 6.326 to 10.074 (+59.25%) over the reported causal-ODE-based baseline.

1 INTRODUCTION

Few-step autoregressive video generation aims to reduce repeated-denoising cost, but causal compatibility alone does not ensure detail at short sampling schedules. RepForcing uses representation-space supervision for initialization and improves reported four-step results after DMD.

1 INTRODUCTION

  • Reducing repeated denoising matters for real-time video generation, while bidirectional teachers can violate the frame-level injectivity requirement for causal ODE distillation.Intermediate AR teachers provide causally compatible targets, but require two training stages before DMD.Source: §1 · p. 2
  • Direct latent regression favors posterior-mean predictions that can blur texture or blend structures, and causal ODE initialization alleviates blur but leaves fine detail unresolved.The stated issue concerns individual clean predictions, not the output distribution of a complete multi-step sampler.Source: §1 · p. 2
  • RepForcing compares teacher-forced clean predictions and training targets through normalized features of a frozen video diffusion transformer.It reuses synthetic videos generated for intermediate AR teacher training and requires no additional causal ODE trajectory pairs.Source: §1 · p. 2
  • A single teacher-forced RepForcing phase replaces intermediate-teacher training and causal ODE distillation before the same asymmetric DMD refinement.The student retains its causal architecture and uses its own generated prefixes during subsequent DMD.Source: §1 · p. 2
  • After DMD, four-step chunk-wise RepForcing improves VBench Total by 0.62 points and VisionReward by 59.25% over the reported intermediate-teacher baselines.The reported comparison evaluates the final generator; before DMD, RepForcing also better preserves generation quality at four steps.Source: §1 · p. 2

2 RELATED WORK

Related work covers autoregressive video generation, few-step initialization for AR distillation, and learned representations as training signals. RepForcing differs by changing the regression metric while retaining the generator architecture and VAE.

Autoregressive Video Generation

  • Autoregressive video diffusion generates blocks conditioned on prior content for streaming, while related work addresses the training–inference history gap and longer generation horizons.Self Forcing trains on self-rollouts with KV caching, and AAPT uses adversarial post-training for single-step streaming (Huang et al., 2025; Lin et al., 2026).Source: §2 · p. 3

Few-Step Initialization for AR Distillation

  • Prior few-step AR initialization uses an intermediate multi-step teacher, whereas RepForcing learns directly from shared bidirectional-model-generated training videos by changing the regression metric.Causal Forcing uses a teacher-forced AR model; Causal Forcing++ and Causal-rCM substitute consistency approaches for trajectory distillation (Zhu et al., 2026; Zhao et al., 2026; Zheng et al., 2026).Source: §2 · p. 3

Learned Representations as Training Signals

3 PRELIMINARIES

The preliminaries establish the synthetic video targets, flow-matching setup, and teacher-forced AR initialization pipeline. They distinguish direct clean predictions from teacher-generated ODE endpoints.

3 PRELIMINARIES

  • The reused first-stage dataset contains approximately 3K videos synthesized by bidirectional Wan from VidProM prompts, with “clean” and “ground truth” referring to the unnoised synthetic targets.The video latents are split into frame- or chunk-wise blocks and paired with text conditions.Source: §3 · p. 3

3.1 FLOW MATCHING

  • Flow matching noises a clean latent as x_t = α_t x_0 + σ_t ε and trains a velocity network; sampling integrates a probability-flow ODE whose endpoint need not equal a single clean prediction.Figure 2 compares initialization strategies across 4, 8, and 12 sampling steps; at four steps, intermediate-teacher methods show blur or structural artifacts while RepForcing maintains coherent structure.Source: §3.1 · p. 3, Figure 2 · p. 4

3.2 AUTOREGRESSIVE DIFFUSION AND ODE INITIALIZATION

  • Teacher forcing uses clean prefixes and independently noised blocks, while free AR sampling uses generated prefixes.A block-causal mask enables one-pass training.Source: §3.2 · p. 4
  • Causal Forcing trains and freezes an AR teacher, then samples few-step trajectories whose deterministic endpoints supply regression pairs rather than the clip targets.Only the prefix comes from the training clip; the endpoint is generated by the teacher for the causal input.Source: §3.2 · p. 4

4 METHOD

The analysis finds that direct latent regression favors conditional-mean predictions and that causal ODE initialization still leaves fine detail unresolved under four-step sampling. RepForcing replaces intermediate-teacher initialization with representation-space teacher forcing and retains asymmetric DMD.

4.1 ANALYSIS: SUBOPTIMALITY OF INITIALIZATION WITH AN INTERMEDIATE TEACHER

  • Latent ℓ2 regression favors the conditional mean of clean continuations, while teacher-forced AR outputs are blurred under four-step sampling.The proposition concerns direct clean predictions, not the complete multi-step sampler's output distribution.Source: §4.1 · p. 5
  • Causal ODE initialization alleviates blur but still degrades visual quality at four steps, despite coherent results at eight or twelve steps.Deterministic targets remove direct-regression target ambiguity but do not ensure high quality in the few-step regime.Source: §4.1 · p. 5

4.2 REPRESENTATION FORCING

  • RepForcing replaces intermediate AR teacher training and causal ODE distillation with representation-space teacher forcing on the same synthetic videos, without generating new causal ODE trajectories.The synthetic videos provide clean targets and prefixes.Source: §4.2 · p. 6
  • At four steps, RepForcing better preserves generation quality than intermediate-teacher-based initialization before subsequent refinement.The representation-space initialization is followed by asymmetric DMD.Source: §4.2 · p. 6
  • A shared frozen video DiT extracts token features from clean predictions and targets, and token-wise cosine supervision updates the causal student through the extractor.Target features are detached; no latent MSE or adversarial term is added.Source: §4.2 · p. 6
  • Extractor selection favors whole-video features that capture motion and inter-frame relations, read at an intermediate depth rather than shallow or deepest layers.The paper expects shallow features to inherit latent averaging and deepest features to become invariant to fine appearance.Source: §4.2 · p. 6
  • After representation-space initialization, asymmetric DMD uses few-step autoregressive rollouts conditioned on student-generated prefixes, and the feature extractor is no longer used.This refinement stage is retained unchanged.Source: §4.2 · p. 6

5 EXPERIMENTS

The experiments assess whether RepForcing’s initialization and final generator preserve quality in few-step AR generation. Results include chunk-wise and frame-wise comparisons, qualitative samples, and matched representation and extraction-depth ablations.

5.1 SETUP

  • The evaluation reports initialization quality before DMD, final quality after DMD, and results under several sampling settings.The evaluation follows Causal Forcing’s protocol: 81-frame videos at 832 × 480, with six higher-is-better metrics.Source: §5 · p. 6, §5.1 · p. 7

5.2 RESULTS

  • 84.66 VBench Total and 10.074 VisionReward under four-step chunk-wise generation exceed reported Causal Forcing results by 0.62 points and 59.25%, respectively.Table 1 compares distilled methods within chunk-wise or frame-wise settings; its baseline results come from specified sources rather than controlled retraining.Source: Table 1 · p. 8, §5.2 · p. 8
  • RepForcing attains the highest VBench Total, Quality, VisionReward, and Instruction Following scores among listed distilled methods in both chunk-wise and frame-wise groups.In frame-wise generation, it reaches 84.38 VBench Total and 10.598 VisionReward; Semantic remains below Causal Forcing in both settings, and chunk-wise Dynamic Degree is 69 versus 71 for Causal-rCM.Source: §5.2 · p. 8, Table 1 · p. 8
  • Figure 4’s chunk-wise samples show RepForcing retaining local details, including fur, facial features, clothing folds, blossoms, and rock textures.Body and hand poses change across sampled frames.Source: §5.2 · p. 8, Table 1 · p. 8

5.3 ABLATION STUDIES

  • Wan2.1-14B features achieve the highest Total, Quality, Semantic, VisionReward, and Instruction Following scores among tested extractors.Compared with InternVid, they improve VBench Total by 1.58 points and VisionReward from 7.653 to 10.074; DINOv2 has the highest Dynamic Degree.Source: §5.3 · p. 9, Table 2 · p. 9
  • Extraction at 1/5 network depth achieves the highest Total, Quality, Semantic, VisionReward, and Instruction Following scores among tested depths.The tested fractions map to blocks 5, 8, 20, and 30; 1/2 and 3/4 yield no further improvement on those metrics, while 3/4 has the highest Dynamic Degree.Source: §5.3 · p. 9, Table 2 · p. 9

6 CONCLUSION

The conclusion presents RepForcing as a representation-supervised alternative to intermediate-teacher initialization and reports improved few-step results. Additional visualizations compare frame-wise samples across methods.

6 CONCLUSION

  • RepForcing supervises clean predictions through normalized features of a frozen video diffusion transformer, replacing intermediate-teacher training and causal ODE distillation with one teacher-forced phase.It reuses the same synthetic training videos and is followed by asymmetric DMD; no additional causal ODE trajectories are generated.Source: §6 · p. 9
  • Before DMD, RepForcing better preserves generation quality than intermediate-teacher-based initialization at four sampling steps.After DMD, it achieves higher VBench Total and VisionReward than reported Causal Forcing baselines in both chunk-wise and frame-wise generation.Source: §6 · p. 9

A MORE VISUALIZATION

  • Figure 5 compares three sampled frames per prompt from Causal Forcing, Causal Forcing++, and RepForcing across four frame-wise examples.The examples depict human activities and animal behavior.Source: A MORE VISUALIZATION · p. 13, Figure 5 · p. 13

B.1 TRAINING AND EVALUATION PROTOCOL

The protocol distinguishes offline synthetic training samples from DMD self-rollouts and specifies the frozen feature extractor settings. Frame-wise visual comparisons provide examples of appearance fidelity across sampled frames.

B.1 TRAINING AND EVALUATION PROTOCOL

  • RepForcing reuses approximately 3K clean synthetic videos as targets and teacher-forcing prefixes, but not Causal Forcing’s causal ODE trajectory pairs.These samples are generated by bidirectional Wan from VidProM prompts; they are not real captured videos or the student’s online DMD self-rollouts.Source: §B.1 · p. 13

B.2 FEATURE EXTRACTOR SETTINGS

  • Stage 1 extracts features from the frozen Wan2.1-14B DiT’s eighth block, using native bidirectional attention over the full video.Predicted and target latents are processed without VAE decoding or added noise, with τ = 0; the DMD teacher instead scores noisy videos using the full network.Source: §B.2 · pp. 13–14

B.3 QUALITATIVE COMPARISON OF FRAME-WISE GENERATION

  • In frame-wise samples, RepForcing renders details such as orange segments, snow texture, feather texture, and nest material while subjects change pose.The examples also retain identifiable clothing, appearance, and scene elements across displayed frames.Source: §B.3 · p. 14

C.1 PROOF OF PROPOSITION 1

The proof uses conditional expectation to identify the fixed-time latent-regression minimizer. Residual conditional uncertainty then yields a strict second-moment inequality for direct predictions, not for a complete sampler.

C.1 PROOF OF PROPOSITION 1

  • Conditional expectation gives an orthogonal risk decomposition, making m(U) the unique minimizer up to almost-sure equality.The argument assumes square-integrable predictions and finite second moment.Source: §C.1 · p. 14
  • Residual conditional covariance contributes positive trace when the posterior is non-point-mass on a set of positive probability.Positive Gaussian likelihood preserves non-point-mass uncertainty from the conditional prior.Source: §C.1 · p. 14
  • The posterior-mean prediction has a strictly smaller second moment than Y, so it cannot share Y’s distribution.This conclusion concerns direct regression at fixed t, not the output distribution of a complete multi-step ODE sampler.Source: §C.1 · p. 14

C.2 CONDITIONAL INTERPRETATION OF REPRESENTATION REGRESSION

Representation-space regression favors realizable video features closest to the conditional feature mean, which need not correspond to the latent-coordinate mean. The appendix also supplies prompts for the qualitative examples in Figures 1, 2, 4, and 5.

C.2 CONDITIONAL INTERPRETATION OF REPRESENTATION REGRESSION

  • The optimal prediction has realizable features closest to the conditional feature mean, which need not correspond to the coordinate-wise mean of target latents.The feature mean itself need not be realizable, and the optimal video may not be unique.Source: §C.2 · pp. 14–15
  • This characterizes the loss preference, not the blockwise optimum of the actual causal student, whose predictions have distinct inputs.Source: §C.2 · p. 15

D PROMPTS

  • Complete text prompts are provided for the qualitative examples in Figures 1, 2, 4, and 5.Within each figure, prompts are listed left to right and then top to bottom across example groups.Source: D PROMPTS · pp. 15–16

Related papers

Loading 2610.20111v1…