PaperlayerView exact revision on arXiv

Source-linked AI summary

LongTake: Learning to Sustain Dynamics in Long-Horizon Video Generation

Byoungwoo Park, Jaemoo Choi, Juho Lee, Yongxin Chen

arXiv:2609.38562v1cs.CV

TL;DR

Short-video supervision provides little direct guidance for later stages of evolving scenes, where long-horizon AR generation must sustain dynamics and visual quality. LongTake trains with long real-video prefixes and uses that model to initialize self-rollout DMD, with Hybrid DMD extending teacher supervision to later frames. Both variants lie on the dynamic-degree/aesthetic-quality Pareto front at 30s and 60s, and Hybrid DMD attains the highest dynamic degree among evaluated methods.

  • Problem

    Short-clip training provides no direct supervision for effects unfolding over tens of seconds, although long-horizon generation requires coherent scene evolution and sustained dynamics.

    Source: §1 · pp. 1–2, §4.1 · p. 4

  • Method

    LongTake trains an AR model to predict later frames from long ground-truth video prefixes, then uses it to initialize self-rollout DMD and supervise later frames in Hybrid DMD.

    Source: §1 · p. 2, §4.4 · p. 7

  • Results

    On 30s and 60s rollouts, both LongTake variants lie on the dynamic-degree/aesthetic-quality Pareto front above 90 dynamic degree; Hybrid DMD has the highest dynamic degree among evaluated methods.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    Long-Horizon TF provides direct initialization for self-rollout DMD without a separate few-step initialization stage.

    Source: §1 · p. 2

    Evaluation is limited to a 1.3B backbone and rollouts up to 60s, and Hybrid DMD depends on the predictive quality of its Long-Horizon AR teacher.

    Source: §6 · p. 9

Abstract

from arXiv · show

World models, game simulators, and long-take video creation require coherent scene evolution and sustained dynamics over extended durations. Autoregressive (AR) video diffusion provides a natural framework for long-horizon generation, yet extended rollouts often become near-static or lose visual quality. We hypothesize that these failures reflect the limited guidance provided by short-video supervision on how ongoing scene dynamics develops over longer durations. This motivates us to introduce LongTake, a two-stage training pipeline built around Long-Horizon Teacher Forcing (TF) on curated real long videos. Long-Horizon TF trains the AR model to predict later frames conditioned on long ground-truth video prefixes, extending direct supervision beyond the short training horizon. This supervision is designed to help the model sustain dynamics and preserve visual quality during long-horizon generation. Our central finding is that this training stage strengthens direct initialization for distribution matching distillation (DMD) under student self-rollout, without the intermediate few-step distillation stage used in standard pipelines. Under the same five-second DMD training setup, our initialization yields substantially higher dynamic degree than short horizon TF initialization on 30-second rollouts at comparable aesthetic quality, and surpasses the evaluated baselines in both measures. Hybrid DMD further reuses this teacher to extend supervision to later frames of the self-rollout while retaining bidirectional joint supervision over the initial window. On long-horizon self-rollouts, LongTake lies on the Pareto front of dynamic degree and aesthetic quality, and Hybrid DMD attains the highest dynamic degree among evaluated methods at both 30s and 60s.

1 INTRODUCTION

Long-horizon AR video generation needs to sustain scene dynamics and visual quality, but short-clip training provides no direct supervision for later stages of an evolving scene. LongTake uses Long-Horizon TF on real long videos to initialize self-rollout DMD, with both variants on the dynamic-degree/aesthetic-quality Pareto front at 30s and 60s.

1 INTRODUCTION

  • Long-horizon AR rollouts can lose visual quality or settle into near-static or repetitive states, while consistency metrics can favor low-dynamic videos after requested motion fades.Short-clip training provides no direct supervision for effects unfolding over tens of seconds; the motivating question is whether real long-video supervision can sustain coherent dynamics while preserving visual quality.Source: §1 · pp. 1–2
  • Long-Horizon TF trains on curated real long videos to predict later frames from long ground-truth video prefixes beyond the short training horizon.This stage directly initializes self-rollout DMD without the separate few-step initialization stage.Source: §1 · p. 2
  • Under the same 5s self-rollout DMD setup, Long-Horizon TF initialization yields stronger long-horizon dynamics at comparable visual quality to short-horizon TF initialization.It also surpasses the three-stage pipeline in both measures.Source: §1 · p. 2
  • Hybrid DMD reuses the Long-Horizon TF model to supervise later frames of the student self-rollout and attains the highest dynamic degree.It retains joint supervision over the initial window from the frozen bidirectional teacher.Source: §1 · p. 2
  • On 30s and 60s rollouts, LongTake lies on the dynamic-degree/aesthetic-quality Pareto front with dynamic degree above 90, while baselines with higher aesthetic quality stay at or below 58.Hybrid DMD attains the highest dynamic degree.Source: §1 · p. 2

2 RELATED WORK

Prior work studies long-video supervision and extended-scene dependencies, while AR distillation pipelines commonly use a separate initialization stage before self-rollout DMD. LongTake builds on direct TF initialization by training with long-video prefixes and extends teacher supervision to later student-rollout chunks.

2 RELATED WORK

  • LVD-2M and Presto emphasize dynamic long-take videos, while Long Context Tuning learns dependencies across extended scenes; LongTake studies curated long-video supervision for direct DMD initialization.Causal Forcing and Causal Forcing++ use separate ODE or consistency-distillation stages before self-rollout DMD, while SGF evaluates direct initialization from a TF-trained model.Source: §2 · p. 3
  • Self-Forcing++ applies DMD to short windows from extended student rollouts, while Context Forcing and Context-Matched Distillation use long-context teachers or student-generated prefixes.Hybrid DMD scores each next chunk under its student-generated prefix using the Long-Horizon AR teacher, while retaining bidirectional joint supervision over the initial window.Source: §2 · p. 3

3 PRELIMINARIES

The preliminaries describe autoregressive video diffusion as chunkwise generation and review the three-stage few-step pipeline used as a reference. Teacher forcing conditions chunks on ground-truth prefixes, while self-rollout DMD trains on student-generated prefixes.

3 PRELIMINARIES

  • The reference few-step AR video diffusion pipeline comprises AR diffusion training, a separate few-step initialization stage, and self-rollout DMD.LongTake uses this pipeline as a reference for examining direct initialization for DMD.Source: §3 · p. 3, Figure 2 · p. 3
  • AR video diffusion generates a video as consecutive chunks, with each chunk predicted from preceding chunks.The paper uses chunk-level notation; frame-wise AR is the single-frame chunk case.Source: §3 · p. 3
  • In standard teacher forcing, each noisy target chunk is conditioned on keys and values computed from the preceding clean ground-truth video prefix.Short-horizon TF trains on clips of at most N frames, with losses for target chunks evaluated in parallel using causal masking.Source: §3 · p. 3
  • The conventional second stage uses ODE or consistency distillation with the AR teacher and student sharing a ground-truth prefix, producing a few-step initialization for later DMD.This stage precedes self-rollout training on student-generated videos.Source: §3 · p. 3
  • Self-rollout DMD trains the generator on student-generated videos, exposing it to KV-cache states built from its own chunks and reducing the training–inference gap.The frozen bidirectional teacher supervises the joint distribution of the student rollout.Source: §3 · p. 4

4 TRAINING FOR LONG-HORIZON VIDEO GENERATION

LongTake extends teacher-forcing supervision to long ground-truth video prefixes, then uses the resulting model to initialize self-rollout DMD and supervise later rollout frames with Hybrid DMD.

4.1 LONG-HORIZON SUPERVISION FOR AUTOREGRESSIVE GENERATION

  • Short-clip training directly supervises predictions only within N, so later positions toward M rely on generalization during autoregressive rollout.Long-Horizon TF targets this teacher gap by pairing later ground-truth targets with cache states from long video prefixes.Source: §4.1 · p. 4

4.2 LONG-HORIZON TEACHER FORCING

  • Long-Horizon TF trains on later targets paired with KV-caches from long ground-truth prefixes, extending direct supervision beyond the short training horizon.With sink retention, longer prefixes span larger temporal separations between retained early frames and recent frames.Source: §4.2 · pp. 4–5, Figure 4 · p. 5
  • At 5s self-rollout DMD and 30s evaluation, Long TF improves dynamics over Short TF at comparable visual quality, without a separate CD stage.The models use the same DMD setup and differ in student initialization.Source: §4.2 · p. 5, Figure 5 · p. 6

4.3 EXTENDING SUPERVISION WITH HYBRID DMD

  • Hybrid DMD retains joint supervision over the initial window and adds conditional supervision from the Long-Horizon AR teacher for later student-generated frames.The student-generated prefix conditions the teacher’s later-frame supervision.Source: §4.3 · p. 6

4.4 PROPOSED TRAINING PIPELINE

  • The standard two-stage LongTake pipeline trains an AR model with Long-Horizon TF, then directly initializes self-rollout DMD without a separate few-step stage.Hybrid DMD extends the second stage to longer student rollouts while both teachers remain frozen.Source: §4.4 · p. 7

5 EXPERIMENTS

LongTake is evaluated on 30s and 60s rollouts against released 1.3B baselines, with measures of quality, motion, and visual stability. LongTake lies on the motion–quality Pareto front, while Hybrid DMD leads in dynamic degree at both durations.

Baselines and evaluation

  • Evaluation uses 1.3B checkpoints on 30s and 60s generations, with 128 extended MovieGen prompts and 251 VBench-Long prompts, respectively.Long-Horizon TF uses 43K video–text pairs and 3,000 updates to supervise generation up to 30s; joint DMD uses 21-latent-frame rollouts, while Hybrid DMD extends to 42-latent-frame rollouts.Source: §5 · p. 7

Qualitative results

  • At 30s, LongTake keeps the requested walk progressing as the scene changes; at 60s, it preserves the dog’s appearance beyond its 30s supervision horizon.Compared baselines show limited progression or appearance degradation, including color artifacts, subject duplication, and drift into close-ups at 60s.Source: §5 · p. 7, Figure 7 · p. 8

Quantitative results

  • Hybrid DMD achieves the highest dynamic degree at both durations and highest Quality at 30s, while joint DMD retains higher Quality at 60s.LongTake with 5s joint DMD also exceeds every baseline in Quality at both durations.Source: §5 · p. 7, Table 1 · p. 9

Long-horizon visual stability

  • LongTake matches the highest VLM score at 30s and has ∆IQ comparable to Context Forcing at both durations while producing stronger dynamics.Both variants have lower ∆IQ than Rolling Forcing at both durations; Hybrid DMD slightly increases endpoint quality changes relative to joint DMD but exceeds Rolling Forcing in VLM while producing stronger dynamics.Source: §5 · p. 8

Quantitative results

  • LongTake lies on the dynamic-degree–aesthetic-quality Pareto front at both durations; baselines above 90 dynamic degree trail it by 3.9 aesthetic score.Baselines with higher aesthetic quality than LongTake reach at most 58 dynamic degree; among other methods on the front, LongTake gains 33–42 dynamic-degree scores for under two aesthetic scores.Source: §5 · p. 7, Figure 8 · p. 8

Ablation Study

  • Long-Horizon TF improves quality at each tested prefix length over standard TF, while boundary metrics suggest better appearance preservation at the immediate transition.With the same initialization and 1,200 distillation iterations, Hybrid DMD at λ = 0.2 achieves the highest dynamic degree and Quality among tested later-frame supervision settings, with modest reductions in subject consistency and aesthetic quality.Source: §5 · p. 9, Table 2 · p. 9

6 CONCLUSION

LongTake uses real long-video supervision to initialize autoregressive video generation directly for self-rollout DMD, and Hybrid DMD reuses that model to supervise later frames. The related work situates this approach among methods for long-horizon generation, supervision, and distillation.

6 CONCLUSION

  • LongTake’s Long-Horizon TF predicts later frames from long ground-truth video prefixes and directly initializes self-rollout DMD without a separate few-step stage.Hybrid DMD reuses the model as a teacher for later self-rollout frames; both variants lie on the dynamic-degree–aesthetic-quality Pareto front at 30s and 60s.Source: §6 · p. 9
  • The evaluation is limited to a 1.3B backbone and rollouts up to 60s, leaving larger backbones and longer rollouts untested.Hybrid DMD’s effectiveness also depends on the predictive quality of its Long-Horizon AR teacher.Source: §6 · p. 9

A EXTENDED RELATED WORK

  • LongTake studies whether real long-video supervision strengthens direct TF initialization before self-rollout DMD, alongside prior AR diffusion and long-horizon generation methods.Related work also covers long-video supervision and distillation, including methods that extend DMD to student rollouts.Source: A EXTENDED RELATED WORK · p. 15

A.1 COMPARISON WITH CLOSELY RELATED WORK.

LongTake uses Long-Horizon TF on curated real long videos to strengthen direct initialization for DMD, and differs from related methods in both training setup and later-frame supervision. Its Hybrid DMD retains bidirectional joint supervision over the initial window while adding AR-teacher supervision of later frames.

Long-Horizon TF initialization

  • Under the same subsequent 5s joint DMD configuration, Long-Horizon TF yields stronger dynamics than short-horizon TF at comparable visual quality and higher aesthetic quality than causal CD initialization.This supports the two-stage pipeline without a separate few-step initialization stage.Source: §A.1 · p. 15

Long-Horizon TF implementation

  • Figure 9’s cache operators construct conditioning KV-cache states in parallel across chunks within each layer, while transformer layers remain sequential.One operator selects the initial sink and recent chunks; the other also merges evicted entries into the sink through a weighted prefix scan.Source: Figure 9 · p. 16

Context Forcing

  • Unlike Context Forcing’s shared-prefix target window, LongTake’s conditional DMD matches each next chunk under the student self-rollout prefix updated before that chunk.Context Forcing also trains a long-context AR teacher on real long videos and uses contextual DMD over extended rollouts.Source: §A.1 · pp. 15–16

Context-Matched Distillation

  • Compared with Context-Matched Distillation’s multi-step causal teacher and prefix-conditioned scoring, LongTake strengthens direct initialization before changing bidirectional scoring or extending the DMD horizon.Hybrid DMD then adds AR supervision of later frames while retaining bidirectional joint supervision of the initial window.Source: §A.1 · p. 16

B IMPLEMENTATION DETAILS

The implementation parallelizes conditioning-cache construction for Long-Horizon TF, then uses replayed student self-rollouts to combine joint supervision on the initial window with conditional supervision on later frames.

B.1 PARALLEL CACHE COMPUTATION

  • An associative prefix scan or parallel indexed selection constructs per-query KV-cache states while preserving chunk causality.The Long-Horizon TF cache is built from known prefix chunks; FIFO retains the sink and recent entries, while the EMA extension merges evicted entries into the sink.Source: §B.1 · pp. 16–17
  • The saved prefix cache is detached before a second pass evaluates target-window losses, with gradients retained through clean-context computation inside that window.Clean and noisy streams share past-history selection, but noisy queries cannot access their own target chunk’s clean keys and values or any future chunk.Source: §B.1 · pp. 16–18

B.2 HYBRID DMD

  • Hybrid DMD’s update is estimated from detached rollout inputs through replay, combining score-difference contributions for the initial window and later chunks.The bidirectional and joint fake-score models evaluate the complete initial window; the conditional fake-score model evaluates later target chunks under the student’s updated rollout prefix.Source: §B.2 · p. 19
  • Hybrid DMD combines joint supervision of the initial window with conditional supervision of later rollout frames using the Long-Horizon AR teacher.The sampled self-rollout prefix is held fixed during the update; gradients pass through replayed prefix representations, not through generation of preceding chunks.Source: §B.2 · pp. 18–19

C.1 TRAINING CONFIGURATION

LongTake initializes distillation from a Long-Horizon TF model trained on curated long-video data, then trains with joint DMD and optionally extends supervision through Hybrid DMD.

C.1 TRAINING CONFIGURATION

  • Hybrid DMD branches after 800 joint-DMD iterations and adds 400 iterations on 42-latent-frame rollouts, while the comparison control keeps 1,200 joint-DMD iterations.The first 21 latent frames receive joint supervision from a frozen Wan2.1-T2V-14B teacher and the remaining 21 from the Long-Horizon AR teacher.Source: §C.1 · pp. 19, 21, Table 3 · p. 20
  • LongTake initializes its student and frozen Long-Horizon AR teacher from Long-Horizon TF trained for 3,000 additional updates from the Causal Forcing2 chunk-wise AR diffusion checkpoint.The student then receives 1,200 joint-DMD iterations over 21 latent frames using Self Gradient Forcing.Source: §C.1 · p. 19
  • The same final EMA student and KV-cache size are used for both 30-second and 60-second evaluation.EMA starts after iteration 200 and uses decay 0.99 after each subsequent student update.Source: §C.1 · p. 19
  • Long-Horizon TF samples prefixes up to 99 latent frames and predicts a fixed 21-frame target window, reaching supervision over 120 latent frames.The four-step student uses denoising timesteps 1, 0.9375, 0.8333, and 0.625; the student attention window contains 12 latent frames including the current chunk.Source: §C.1 · p. 20

C.2 TRAINING DATA

Long-Horizon TF uses a curated video corpus with long clips, while the baseline configurations specify each evaluated method’s released model and context setup.

C.2 TRAINING DATA

  • The training corpus contains 43,000 video–text pairs: 40,000 OpenVidHD clips and 3,000 short clips from released Causal Forcing clean-synthesized data.OpenVidHD clips span 21–120 latent frames, approximately 5–30 seconds at 16 FPS; training samples OpenVidHD with probability 0.9 and the short subset with probability 0.1.Source: §C.2 · p. 21
  • OpenVidHD curation retains 243,010 candidates, then selects 40,000 videos to match short-horizon TF reference motion statistics while filtering for visual quality and temporal consistency.The initial filters require at least 16 FPS and candidate latent lengths between 21 and 123.Source: §C.2 · p. 21

C.3 BASELINE CONFIGURATION

  • The evaluated baselines use distinct released checkpoints and context configurations, including 12-frame windows for LongLive and SGF, a 9-frame window for Reward Forcing, and a 24-frame rolling window for Rolling Forcing†.Context Forcing uses its released EMA checkpoint with native Slow-Fast Memory and a 21-frame cache.Source: §C.3 · p. 21

C.4 EVALUATION PROTOCOL

The evaluation uses matched prompts for long-horizon rollouts and compares teacher generation at different real-video prefix lengths. LongTake variants remain on the 60-second motion–quality Pareto front, while Long-Horizon TF improves teacher generation quality before distillation.

C.4 EVALUATION PROTOCOL

  • The 30s evaluation uses 128 MovieGen prompts and the 60s evaluation uses 251 VBench-Long prompts, with all methods sharing prompts within each horizon.The aggregate Quality score combines seven video-quality dimensions; ∆IQ and VLM Exposure are reported separately.Source: §C.4 · pp. 21, 23

D.1 TEACHER GENERATION BEFORE DISTILLATION

  • Long-Horizon TF improves evaluated generation quality at both 5s and 20s real-video prefixes, with boundary metrics suggesting better appearance preservation at the observed-to-generated transition.The boundary measures cover only the immediate transition and can favor static outputs.Source: §D.1 · p. 23
  • Near-saturated instruction-following scores provide limited evidence of differences in prompt alignment between the two teachers.Source: §D.1 · p. 24

D.2 MOTION-QUALITY TRADE-OFF

  • Both LongTake variants remain on the dynamic-degree/aesthetic-quality Pareto front at 60s, including when aesthetic quality is replaced by non-motion quality.The non-motion measure is the VBench Quality score recomputed without dynamic degree.Source: §D.2 · p. 24

D.3 ADDITIONAL QUALITATIVE RESULTS

The additional qualitative results compare long-horizon generations, Long-Horizon TF initialization, and conditional supervision. Hybrid DMD increases dynamics relative to joint DMD, with modest reductions in subject consistency and aesthetic quality.

D.3 ADDITIONAL QUALITATIVE RESULTS

  • In the illustrated 30s rollout, λ = 0.2 produces more scene progression than λ = 0 while retaining a coherent central subject.Source: §D.3 · p. 24
  • At both 30s and 60s, both LongTake variants lie on the motion–aesthetic-quality Pareto front; they are also the only methods on the non-motion-quality front with dynamic degree above 90.Non-motion quality is the VBench Quality score recomputed without dynamic degree.Source: Figure 13 · p. 25, Figure 14 · p. 25
  • Figures 15–17 add 30s and 60s baseline-generation comparisons using different subjects and scenes from the earlier examples.Source: §D.3 · p. 25, Figure 15 · p. 26, Figure 16 · p. 27
  • Figures 18–20 illustrate Long-Horizon TF initialization under the same 5s joint DMD procedure.Source: §D.3 · p. 25, Figure 18 · p. 28, Figure 19 · p. 29
  • Hybrid DMD (λ = 0.2) increases dynamics over joint DMD (λ = 0), with modest reductions in subject consistency and aesthetic quality.The comparison holds teacher training fixed and uses shared initialization and equal distillation iterations.Source: §D.3 · p. 25, Figure 21 · p. 29, Figure 22 · p. 29

Related papers

Loading 2609.38562v1…