PaperlayerView exact revision on arXiv

Source-linked AI summary

History Forcing: Distilling History-Dependent Continuations for Long-Horizon Video World Models

Hangyi Kuang, Chengzhi Wang, Yefei He, Weijie Wang, Yuke Zhao, Zhuokun Chen, Zisu Huang, Yifan Yang, Xiaoxuan He, Feng Chen, Bohan Zhuang

arXiv:2611.02782v1cs.CV

TL;DR

Long-horizon autoregressive video generation needs historical context, but existing distillation methods do not explicitly strengthen historical constraints independently of generative supervision. History Forcing adds a history-derived residual to distillation targets. It reports 85.07 VBench quality for 15s generation and a 79.82 VBench-Long aggregate for 60s generation, outperforming compared 1.3B models.

  • Problem

    Existing distillation methods do not explicitly strengthen historical constraints independently of generative supervision, despite the need for history when subjects or scenes reappear after leaving view.

    Source: §1 · p. 2

  • Method

    History Forcing compares teacher predictions with and without informative history and weights and adds the resulting residual to the score or velocity target for distillation.

    Source: §1 · p. 2

  • Results

    85.07 VBench quality for 15s generation and 79.82 VBench-Long aggregate for 60s generation; History Forcing outperforms compared 1.3B models.

    Source: §1 · p. 3

  • Takeaways & Limitations

    History Forcing reduces appearance and layout drift in long videos and better preserves object-state changes across out-of-sight intervals, producing more consistent states on camera return.

    Source: §6 · p. 11

    Experiments use 1.3B teachers and students for long-video generation and a 5B world-model backbone; supervision remains constrained by the underlying models’ ability to capture temporal dynamics.

    Source: §6 · p. 11

Abstract

from arXiv · show

Few-step autoregressive video diffusion enables efficient generation, yet maintaining consistency over long rollouts remains challenging, especially when subjects or scenes reappear after going out of sight. Despite progress in fidelity and rollout robustness, existing distillation methods do not fully exploit richer historical information to constrain the student's current prediction. To address this gap, we introduce \textbf{History Forcing}, a unified on-policy distillation framework that strengthens historical supervision for few-step autoregressive video generators. The framework uses the residual between teacher predictions with and without informative historical context to decouple this supervision. Such a historical context can be obtained using earlier generated chunks with less accumulated drift or a fixed-view teacher rollout that keeps the scene visible while the student looks away. The residual is then weighted and added to the score or velocity prediction used for distillation, forming a corrected target. By aligning the student with this corrected target through distillation, History Forcing incorporates long-range historical constraints into autoregressive generation. Our method achieves state-of-the-art performance among 1.3B models, with a VBench quality score of 85.07 for 15s video generation and a VBench-Long aggregate score of 79.82 for 60s generation. For out-of-sight world modeling, evaluations further demonstrate more plausible scene evolution and more consistent states when the camera returns, with Evolution Quality improving from 2.35 to 2.70 on PlayWorld and Object Consistency from 58.43 to 62.18 on MemoBench.

1 Introduction

Long-horizon autoregressive video generation needs useful historical context, but existing distillation targets do not explicitly strengthen historical constraints independently of generative supervision. History Forcing adds a history-derived residual to distillation targets, with history sources tailored to long-video generation and out-of-sight world modeling.

1 Introduction

  • Long-range history matters because subjects or scenes may reappear after leaving view, when recent frames no longer contain information about them.Insufficient historical information in later predictions can contribute to gradual subject and scene drift.Source: §1 · p. 2
  • Existing distillation targets lack an explicit mechanism to strengthen historical constraints independently of the teacher’s overall generative supervision.Matching an AR teacher’s full conditional distribution couples temporal consistency and visual quality, so its visual-quality weaknesses may constrain the student.Source: §1 · p. 2
  • History Forcing compares teacher predictions with and without informative history, then weights and adds their residual to the score or velocity target used for distillation.This isolates and strengthens historical supervision within a unified on-policy distillation framework.Source: §1 · pp. 2–3
  • For long-video generation, earlier student-generated chunks provide subject and scene information less affected by accumulated drift, while fixed-camera teacher rollouts supply history for out-of-sight evolution.The latter guides the student's continuation when the camera revisits the region.Source: §1 · pp. 2–3
  • 85.07 VBench quality for 15s generation and 79.82 VBench-Long aggregate for 60s generation; with 1.3B teachers, History Forcing outperforms compared 1.3B models.Student training rollouts are at most 15s; the framework also improves scene evolution and revisit consistency on PlayWorld and MemoBench.Source: §1 · p. 3

2 Related Work

Related work addresses long-horizon video generation, few-step causal diffusion distillation, and historical conditioning in video world models. These approaches improve access to context and alignment, while the cited discussion identifies limits in explicitly representing historical contributions to distillation targets and in continuation across long observation gaps.

Long-horizon autoregressive video diffusion

  • Long-horizon video methods use extended self-rollouts, bounded contexts, rolling denoising, error-aware training, memory, or history-aware supervision; History Forcing instead embeds historical supervision into on-policy distillation targets.Training-free approaches also extend pretrained models through diagonal denoising, temporal feature manipulation, and cache compression.Source: §2 · p. 3

Few-step causal video diffusion distillation

  • Few-step causal distillation methods improve sampling efficiency and teacher–student alignment, but the contribution of historical context remains implicit in their targets.Examples include CausVid and Self Forcing with DMD, Context Forcing with longer history, and OPSD-V with real-video history.Source: §2 · p. 3

History mechanisms in video world models

  • World-model memory and history-aware training improve scene consistency, yet coherent continuation across long observation gaps remains challenging.Methods include spatial-temporal memory, geometry-indexed view retrieval, and fixed-view virtual monitors.Source: §2 · p. 3

3 Preliminaries

The preliminaries describe diffusion in latent space, sequential chunk generation, and distribution-matching distillation. Figure 3 previews how History Forcing uses a history contrast to modify distillation targets.

Diffusion models

  • A diffusion model perturbs clean latent x0 into xt = αt x0 + σtϵ and learns to reverse the noise process; under linear flow matching, the predicted velocity is vt = ϵ − x0.A predicted velocity gives the clean estimate ˆx0 = xt − σtˆvt.Source: §3 · p. 3

Distribution matching distillation

  • Figure 3 shows History Forcing comparing AR-teacher predictions with and without added history, then applying the contrast to DMD's reference score or the self-distillation velocity target.The predictions concern the same noisy student chunk.Source: Figure 3 · p. 4

Autoregressive video diffusion

  • Autoregressive video diffusion generates video sequentially in chunks, with each inference chunk iteratively denoised while conditioned on previously generated chunks.The model operates on latent feature maps encoded from RGB frames and decodes denoised latents into video frames.Source: §3 · p. 4

Distribution matching distillation

  • Distribution Matching Distillation trains a few-step generator to match a reference distribution, using a gradient approximation based on reference and fake scores.The reference score comes from a pretrained diffusion model, while an online denoiser estimates the fake score from student-generated samples.Source: §3 · p. 4

4 Method

History Forcing compares teacher predictions with and without additional history, then uses their residual to strengthen historical supervision while retaining the reference model’s role. The method applies this correction to long-horizon consistency and out-of-sight scene evolution.

4 Method

  • History Forcing adds a weighted residual between AR teacher predictions with and without additional history to the score or velocity target used for distillation.The shared condition and noisy chunk are held fixed across the teacher predictions.Source: §4 · p. 4, §4.1 · p. 5

4.1 History-Reweighted Distribution

  • The history-reweighted target adjusts a reference distribution according to how additional history changes the AR teacher’s likelihood for the noisy chunk.The target assumes a finite, positive normalization function; the resulting posterior-gradient direction guides the reference score toward the additional history.Source: §4.1 · p. 5

4.2 Distillation for Long-Horizon Consistency

  • For long-horizon consistency, earlier chunks from the student’s rollout supply less-drifted subject and scene information while the bidirectional reference score retains visual-quality supervision.The two AR teacher contexts share recent frames, and the objective is optimized on student-generated inference-time rollouts.Source: §4.2 · pp. 5–6

4.3 Refinement for Out-of-Sight Evolution

  • Earlier student-rollout chunks constrain revisited appearance and scene structure, but provide limited guidance about changes that occur while a region is outside the camera’s view.The method applies supervision to chunks generated during revisits.Source: §4.3 · p. 6
  • For out-of-sight evolution, a fixed-camera teacher rollout supplies historical frames from before each chunk while the student follows its prescribed camera trajectory.On-policy self-distillation uses an EMA teacher’s history-guided velocity target along the student’s sampling trajectory, avoiding an auxiliary fake-score model.Source: §4.3 · p. 6

5 Experiments

Experiments compare History Forcing across long-video generation, out-of-sight world modeling, and ablations of its historical supervision. Results report gains in quality and consistency, with qualitative comparisons and modest training overhead.

5.2 Quantitative Comparison

  • 85.07 is the VBench quality score for 15s generation, and History Forcing achieves the highest quality among the compared methods.Adding AR-teacher history raises the score from 84.44 to 84.82; fine-tuning the teacher on 10s videos further raises it to 85.07.Source: §5.2 · p. 8
  • 79.82 is History Forcing’s VBench-Long aggregate score among compared 1.3B students for 60s generation, 0.60 points above Rolling Forcing.Student training rollouts are at most 15s; the score approaches the 5B LongLive-2.0 model’s 79.90.Source: §5.2 · p. 8
  • On PlayWorld, History Forcing leads across all four metrics, while on MemoBench Real it improves all nine baseline metrics and beats OPSD-V on eight.Table 3 compares PlayWorld Out-of-sight and MemoBench Real; PlayWorld metrics use a 1–5 rating scale.Source: §5.2 · p. 8, Table 3 · p. 8

5.3 Ablation Studies

  • For long-video generation, τ = 750 yields the highest subject consistency (92.58), while λ = 1 yields the highest quality score among tested history weights.Increasing λ from 0.5 to 2 improves subject and background consistency but reduces dynamic degree; lowering the timestep threshold concentrates supervision at lower noise levels.Source: §5.3 · p. 9
  • In world-model self-distillation, supervising all four denoising steps outperforms supervising only the last two or three, and λ = 3 is favored across both benchmarks.For revisits, recent student observations provide little information about the unseen region; λ > 1 amplifies the history-induced prediction change.Source: §5.3 · p. 9
  • Sharing the latest six student-generated latent frames lets the residual isolate earlier history; removing them lowers quality from 84.82 to 81.91.Replacing the oldest three history frames with initial sink frames also lowers quality, to 83.81.Source: §5.3 · p. 9
  • Jointly normalizing the history term with the DMD gradient raises quality from 84.09 to 84.82 and dynamic degree from 59.72 to 78.06.The alternative yields higher subject and background consistency, so the comparison favors joint normalization for balancing consistency and motion dynamics.Source: §5.3 · p. 9
  • Restricting historical supervision to one sampled chunk adds 2.31% training time per step and 1.99% peak allocated memory relative to DMD.Qualitatively, History Forcing maintains body structure, clothing, and setting in a 60s rollout and preserves a glass shape and surrounding pattern across revisits.Source: §5.3 · p. 10, §5.4 · p. 10, Figure 5 · p. 10

6 Conclusion

History Forcing decouples historical supervision by comparing teacher predictions with and without informative history. The experiments report reduced long-video appearance and layout drift and more consistent object states after out-of-sight intervals.

6 Conclusion

  • History Forcing compares teacher predictions with and without informative history to decouple long-range historical constraints from visual-quality supervision.For out-of-sight evolution, fixed-view teacher history guides the OPSD velocity target.Source: §6 · p. 11
  • Experiments report reduced appearance and layout drift in long videos and better preservation of object-state changes across out-of-sight intervals.The reported outcome is more consistent states when the camera returns.Source: §6 · p. 11
  • Long-video generation experiments use 1.3B teachers and students, while world modeling uses a 5B backbone.Historical supervision remains constrained by the underlying models’ ability to capture temporal dynamics; limited scene evolution in fixed-view rollouts can weaken out-of-sight state-change supervision.Source: §6 · p. 11

Appendix

The appendix contains additional implementation, inference, evaluation, and qualitative-comparison details.

Appendix

  • Appendix A provides additional training, inference, and evaluation details, while Appendix B provides more qualitative comparisons and examples.Source: Appendix · p. 16

A Additional Implementation Details

The implementation uses shared optimizer, hardware, sampling, and video-generation settings across both applications.

A Additional Implementation Details

  • Both applications use AdamW, train on eight NVIDIA A100 GPUs, and sample with four SDE steps; autoregressive chunks contain three latent frames.The stated settings include betas (0, 0.999), weight decay 0.01, maximum gradient norm 10, seed 42, and 16 FPS generation.Source: A Additional Implementation Details · p. 16

A.1 Long-Video Generation

The appendix specifies the training, inference, and evaluation settings for long-video generation.

A.1 Long-Video Generation

  • The student learning rate is 2 × 10−6, the fake-score model learning rate is 4 × 10−7, and the teachers remain frozen during distillation.The fake-score model updates every iteration, while the student updates once every 5 iterations.Source: §A.1 · p. 16
  • For the 10s AR teacher, the fine-tuning checkpoint is iteration 2500, with attention windows of 21 and 42 latent frames for the 5s and 10s variants.The AR teacher predictions use separate KV caches with matched temporal positions for the current chunk.Source: §A.1 · p. 16
  • Inference uses student EMA weights at 832×480 resolution, with a 21-frame KV window and 3 sink frames.These settings apply to all student variants in both training and inference.Source: §A.1 · p. 16
  • The 15s VBench evaluation uses 5 videos per Wan2.1-augmented prompt, while 60s VBench-Long uses 1 video per each of 1,003 original MovieGenBench prompts.Each 60s rollout contains 1,005 frames.Source: §A.1 · p. 16

A.2 Out-of-sight World Modeling

The appendix describes camera trajectories, teacher-rollout history construction, inference settings, and evaluation measures for out-of-sight world modeling.

A.2 Out-of-sight World Modeling

  • Training camera trajectories rotate or translate left or right before returning to the initial viewpoint.The student and fixed-view teacher rollouts are generated online with shared initial noise and SDE transition noise.Source: §A.2 · p. 16
  • The fixed-view teacher uses a 12-frame KV window with 3 sink frames, while history-conditioned inputs include up to 4 earlier teacher-rollout latent frames.Both inputs retain 1 sink frame for the initial image, and shared recent context is reduced to 4 student-generated latent frames.Source: §A.2 · p. 16
  • Inference evaluates fine-tuned student weights at 1280 × 704 resolution, using a 12-frame KV window with 3 sink frames and 1 video per input.Source: §A.2 · p. 16
  • PlayWorld evaluation uses matched initial images, prompts, camera trajectories, and random seeds, with Evolution Quality and Subject Identity rated on 1–5 scales before instruction gating.MemoBench Real automated metrics use the official MemoBench implementation.Source: §A.2 · pp. 16–17

B Additional Qualitative Analysis

Qualitative comparisons cover out-of-sight world modeling and 15s and 60s video generation, with examples of preserved objects, subjects, and scene layouts.

Out-of-sight world modeling

  • History Forcing preserves objects and plausible scene evolution upon revisit in examples of sand filling, tomatoes, coffee pouring, and solution pouring.The baseline spills sand outside the jar; History Forcing keeps sand predominantly inside, tomatoes in the bowl, and liquids in their containers.Source: B Additional Qualitative Analysis · p. 17, Figure 6 · p. 18

15-second video generation

60-second video generation

Out-of-sight world modeling

  • Some appearance and scene drift remains, and temporal errors in fixed-camera teacher rollouts can reduce historical-supervision accuracy and contribute to revisit inconsistencies.Source: B Additional Qualitative Analysis · p. 17

Related papers

Loading 2611.02782v1…