In-Distribution Forcing for Long Video Generation at Test Time

1 INTRODUCTION

• Beyond the finite training horizon, extrapolation introduces conditioning configurations outside training, and errors accumulate over time.Simply extending the training horizon cannot resolve extrapolation beyond it in arbitrarily long video generation.Source: §1 · p. 2
• KV conditioning selects or modifies cached entries but assumes they are in-distribution, without ensuring that their construction context was seen during training.The paper calls the resulting issue the KV-provenance problem: beyond the training horizon, entries are cached under unseen provenance.Source: §1 · p. 2
• ID-Forcing introduces self-caching and aligns both KV operations with training: earlier entries attend only to themselves, while later entries are cached autoregressively on top of them.It also keeps the first chunk in the conditioning window so the rolling window remains in-distribution.Source: §1 · p. 2
• ID-Forcing demonstrates suppression of drift in minute-scale generation while preserving visual quality, temporal consistency, and active motion.The framework aligns KV caching and conditioning with configurations encountered during training.Source: §1 · p. 2
• Figure 2 holds the conditioning window fixed and shows that the caching provenance of the final chunk alone can determine whether the next chunk drifts.Self-caching avoids drift, while caching under an unseen window degrades the next chunk; retaining the first chunk does not help when the window remains unseen.Source: Figure 2 · p. 3

2 PRELIMINARY

• A latent video is represented as chunks generated sequentially under a text condition, with later chunks conditioned on KV entries rather than raw earlier latents.Each generated chunk is written into one KV entry for later chunks to attend to.Source: §2 · p. 3
• Training uses self-rollouts over a fixed horizon, with each chunk generated by attending to preceding KV entries and then cached using that same context.The conditioning and caching context grows from empty for the initial chunk to the preceding entries within the training horizon.Source: §2 · p. 3
• Beyond the training horizon, standard rolling-window methods evict the oldest entry and use the L most recent KV entries for generation and caching.This changes the context from the full training prefix and manifests as drifting: colors and textures shift, and motion dynamics decay.Source: §2 · pp. 3–4
• Figure 3 contrasts Self-Forcing's unseen beyond-horizon caching windows with ID-Forcing's windows, which remain seen during training.The figure depicts both the caching window for each KV entry and the conditioning window of size L used to generate a chunk.Source: Figure 3 · p. 4

3.1 LEVEL 1 – KV CACHING: ENFORCING IN-DISTRIBUTION PROVENANCE

• A KV entry’s distribution depends on the context it attended to when cached, so unseen provenance makes the entry out-of-distribution.The conditioning window is the only path through which drift can enter generation beyond the training horizon.Source: §3.1 · pp. 4–5
• Self-caching prevents out-of-distribution KV entries by restricting each entry’s provenance to contexts executed during training.Without this control, later generations inherit out-of-distribution entries and drift accumulates.Source: §3.1 · pp. 4–5

3.2 LEVEL 2 – KV CONDITIONING: ENABLING EXACT ROLLING WINDOW

• Conditioning keeps κ0 fixed and limits the window to L < N entries, evicting the oldest non-sink entry and re-rotating the rest.Self-caching lets eviction preserve in-distribution representations because those entries have no external provenance to lose.Source: §3.2 · pp. 5–6

4.1 EXPERIMENTAL SETUP

• Evaluations use 120- and 240-second videos from two base models, MovieGen prompts, VBench-Long and drift metrics, plus a 2AFC user study.The implementation uses Wan2.1-T2V-1.3B architecture and compares against Deep Forcing, ∞-RoPE, and MemRoPE.Source: §4.1 · p. 6, §4 · p. 6

4.2 MAIN RESULTS

• ID-Forcing improves both drift metrics and achieves the highest Dynamic Degree in every evaluated setting, while remaining competitive or superior on other metrics.The authors identify Motion Smoothness as an exception because it is biased toward lower Dynamic Degree.Source: §4.2 · pp. 6–7, Table 1 · p. 7
• On 2-minute videos, ID-Forcing preserves subject consistency and color fidelity on Self-Forcing, and sustains motion and accurate color on LongLive.The comparisons include Deep Forcing, ∞-RoPE, and MemRoPE.Source: §4.2 · p. 7

4.3 ABLATION STUDIES

• In the ablation, retaining the first chunk and re-rotation reduce drift, but self-caching is needed to bring KV caching in-distribution and greatly reduce remaining drift.Without the components, Self-Forcing collapses beyond its training horizon.Source: §4.3 · p. 8

5 RELATED WORKS

• AR generation produces chunks sequentially from preceding KV entries, enabling streaming and potentially arbitrary test-time length, but deviations from training can accumulate as drift.Bidirectional models attend over the sequence at once and have fixed output length at sampling time.Source: §5 · p. 9
• Training on longer contexts does not remove the finite-horizon boundary and becomes infeasible for extremely long videos, so test-time extrapolation remains necessary.The paper characterizes improved training and test-time extrapolation as orthogonal directions.Source: §5 · p. 9
• Existing test-time methods manage KV conditioning through sinks, temporal re-indexing, or memory tokens, while leaving KV-entry construction unchanged.ID-Forcing instead addresses both KV caching and KV conditioning.Source: §5 · p. 9

6 CONCLUSION

• ID-Forcing uses self-caching, a bounded conditioning window, and a permanently fixed first chunk to address drift while maintaining stable dynamics at minute scale.The conclusion reports competitive performance across diverse durations, backbones, and metrics.Source: §6 · p. 9

• Table 3 reports VBench-Long quantitative evaluation for 30- and 60-second videos.Source: Table 3 · p. 12

• At 30 and 60 seconds, ID-Forcing improves both Color Drift and Motion Drift and achieves the highest Dynamic Degree among compared methods.Source: A ADDITIONAL RESULTS · p. 12
• ID-Forcing maintains Dynamic Degree from 65.94 at 30 seconds to 64.76 at 240 seconds, while Self-Forcing falls from 44.58 to 29.61.The comparisons indicate less long-horizon performance decline for ID-Forcing.Source: A ADDITIONAL RESULTS · p. 12
• Additional qualitative comparisons are provided for 2-minute video generation in Fig. 6; separately, Table 4 reports ablation evaluations measured on Self-Forcing at 120 seconds, with components added cumulatively.Source: A ADDITIONAL RESULTS · p. 12
• Self-Forcing’s high Background Consistency and Motion Smoothness scores reflect near-identical consecutive frames, alongside Dynamic Degree 30.04 and Color Drift 25.54.Source: A ADDITIONAL RESULTS · p. 12
• Adding the sink and re-rotation improves Dynamic Degree and drift metrics, but Color Drift remains 60.66 before self-caching.After adding the sink, Color Drift is 48.65 and Motion Drift is 67.19; re-rotation improves the drift metrics further.Source: A ADDITIONAL RESULTS · p. 12
• With self-caching, ID-Forcing improves Color Drift from 60.66 to 72.37 and Aesthetic Quality from 54.14 to 58.23.It leads on every metric except Background Consistency and Motion Smoothness, which favor collapsed Self-Forcing output.Source: A ADDITIONAL RESULTS · p. 13, Table 4 · p. 13

• Color Drift measures the L1 distance between normalized 180-bin HSV Hue histograms from the first and last frames.The distance ranges from 0 to 2.Source: B METRIC DESCRIPTION · p. 13
• Motion Drift measures the difference between Dynamic Degree in the first and last five seconds, with larger differences indicating motion degradation.The difference ranges from 0 to 1.Source: B METRIC DESCRIPTION · p. 13
• Both drift measures are normalized by their maximum and converted to higher-is-better scores, averaged over 128 prompts.Higher scores indicate more stable color and motion dynamics.Source: B METRIC DESCRIPTION · p. 13

• For each of 20 prompts, participants compared anonymized ID-Forcing and baseline videos across six dimensions, including overall preference.The dimensions also covered color, background, and subject consistency, Dynamic Degree, and temporal flickering.Source: C USER STUDY DETAILS · p. 13
• A balanced Latin-square assignment showed each participant every baseline five times and matched every prompt against all four baselines across the study.Source: C USER STUDY DETAILS · p. 13
• The appendix includes a user-study interface figure and a criteria-explanation figure, plus additional qualitative results on two-minute videos.Source: Figure 6 · p. 14, Figure 7 · p. 15, Figure 8 · p. 15