Source-linked AI summary
PackForcing: Short Video Training Suffices for Long Video Sampling and Long Context Inference
Xiaofeng Mao, Shaohao Rui, Kaining Ying, Bo Zheng, Chuanhao Li, Mingmin Chi, Kaipeng Zhang
TL;DR
Long-video autoregressive diffusion is limited by compounding errors and unbounded KV-cache growth. PackForcing manages history with a three-partition cache, compressed mid tokens, dynamic selection, and temporal RoPE adjustment. It reports 24× temporal extrapolation from 5-second clips to coherent 2-minute videos with a bounded 4 GB cache, while leaving subject-consistency trade-offs and scaling challenges open.
Problem
Autoregressive video generation must retain extensive context to limit error accumulation, but its KV cache grows linearly and becomes impractical for minute-scale videos.
Method
PackForcing partitions history into sink, compressed mid, and recent tokens, combining dual-branch compression with dynamic top-k selection and incremental temporal RoPE adjustment.
Results
24× temporal extrapolation extends 5-second training clips to 120-second videos while bounding KV-cache memory to approximately 4 GB.
Takeaways & Limitations
The reported results support short-video supervision for high-quality long-video synthesis within a bounded-context generation framework.
Takeaways & Limitations
PackForcing trails LongLive in Subject Consistency, scoring 90.49 versus 92.00, while preserving higher Dynamic Degree of 56.25 versus 44.53.
Abstract
from arXiv · showhide
Autoregressive video diffusion models have demonstrated remarkable progress, yet they remain bottlenecked by intractable linear KV-cache growth, temporal repetition, and compounding errors during long-video generation. To address these challenges, we present PackForcing, a unified framework that efficiently manages the generation history through a novel three-partition KV-cache strategy. Specifically, we categorize the historical context into three distinct types: (1) Sink tokens, which preserve early anchor frames at full resolution to maintain global semantics; (2) Mid tokens, which achieve a massive spatiotemporal compression (32x token reduction) via a dual-branch network fusing progressive 3D convolutions with low-resolution VAE re-encoding; and (3) Recent tokens, kept at full resolution to ensure local temporal coherence. To strictly bound the memory footprint without sacrificing quality, we introduce a dynamic top-$k$ context selection mechanism for the mid tokens, coupled with a continuous Temporal RoPE Adjustment that seamlessly re-aligns position gaps caused by dropped tokens with negligible overhead. Empowered by this principled hierarchical context compression, PackForcing can generate coherent 2-minute, 832x480 videos at 16 FPS on a single H200 GPU. It achieves a bounded KV cache of just 4 GB and enables a remarkable 24x temporal extrapolation (5s to 120s), operating effectively either zero-shot or trained on merely 5-second clips. Extensive results on VBench demonstrate state-of-the-art temporal consistency (26.07) and dynamic degree (56.25), proving that short-video supervision is sufficient for high-quality, long-video synthesis. https://github.com/ShandaAI/PackForcing
1 Introduction
Short-video diffusion models struggle with streaming because bidirectional processing handles all frames jointly, while autoregressive methods face compounding errors and linearly growing KV caches. PackForcing addresses both constraints with a three-partition cache, compression, dynamic selection, and temporal position adjustment, enabling long-horizon generation from short supervision.
- Bidirectional video diffusion models process all frames jointly, hindering streaming or real-time generation for longer videos.
- Autoregressive generation accumulates errors over time, causing semantic drift and a CLIP-score decline from 33.89 to 27.12 within 60 s for Self-Forcing.
- Approximately 749K tokens and 138 GB of KV storage are required for a 2-minute 832×480 video at 16 FPS, creating a memory bottleneck.
- PackForcing partitions history into full-resolution sink tokens, compressed mid tokens, and full-resolution recent tokens to preserve global and local information.
- Dynamic top-k selection retrieves informative mid tokens, while incremental temporal RoPE adjustment corrects positional gaps without full cache recomputation.
- A dual-branch compression layer provides 128× spatiotemporal compression, approximately 32× token reduction, and over 27× greater effective memory capacity.
- 24× temporal extrapolation extends 5-second training clips to coherent 2-minute videos, with state-of-the-art VBench scores and stable CLIP trajectories.
2 Related Work
Related work spans video diffusion architectures, autoregressive KV-cached generation, KV-cache management, and long-video extensions. PackForcing is positioned within this landscape as a unified approach to historical-context management.
- Video Diffusion Models: Video diffusion models evolved from pseudo-3D U-Nets toward Diffusion Transformers that represent videos as spatiotemporal patches.
- Autoregressive Video Generation: Autoregressive video generation synthesizes frames block-by-block while caching historical key-value pairs to overcome fixed-length joint modeling.
- KV Cache Management: KV-cache research includes attention sinks, attention-score-based heavy-hitter selection, and RoPE interpolation, but primarily emphasizes selection or eviction rather than explicit compression.
- Long Video Generation: Traditional long-video methods modify noise schedules, use hierarchical planning, or build multi-stage extensions, whereas PackForcing uses a unified single-stage causal framework.
3 Method
PackForcing manages autoregressive video history with a three-partition KV cache that preserves anchors and recent frames while compressing and selectively routing intermediate history. The design bounds attention context independently of video length and addresses positional gaps created by history eviction.
- Three-Partition KV Cache: PackForcing partitions generation history into sink, compressed mid, and recent/current tokens, applying a tailored policy to each temporal role.Sink and recent/current tokens remain full resolution, while mid tokens are compressed and dynamically selected.
- Three-Partition KV Cache: Sink tokens preserve early frames at full resolution as semantic anchors that stabilize scene layout, subject identity, and global style.The method sets Nsink=8, or two blocks, consuming less than 2% of the total token budget for a 2-minute video.
- Three-Partition KV Cache: Dynamic context selection routes only the most informative compressed mid blocks into the active context, limiting the computational budget as the compressed buffer grows.The active set is formed by evaluating query-key affinities and retaining Nmid informative blocks.
- Three-Partition KV Cache: Recent frames remain uncompressed to preserve local temporal dynamics, while a concurrent low-resolution backup transitions aging recent tokens into long-term mid memory.This dual-resolution shifting pipeline hides compression overhead and supports seamless transitions.
- Bounded Attention and Positional Adjustment: The active attention context remains fixed-size and independent of total video length, while temporal-only RoPE adjustment restores continuity after compressed blocks are evicted.The adjustment is applied once per eviction event across all layers and costs less than 0.1% of total inference time.
4 Experiments
Experiments evaluate PackForcing on 60- and 120-second videos using VBench, CLIP trajectories, qualitative comparisons, ablations, and short-video generalization. Results show strong motion, temporal alignment, long-horizon stability, and benefits from the proposed cache components, with a remaining subject-consistency trade-off.
- Quantitative Comparison: PackForcing achieves the highest Dynamic Degree at both 60 s and 120 s, scoring 56.25 and 54.12 respectively.It outperforms CausVid by +7.82 and +4.12 at the two durations.
- Long-Range Consistency: PackForcing’s CLIP alignment declines by only 1.14 points over 60 seconds, from 34.04 to 32.90.Self-Forcing drops by 6.77 points, whereas CausVid declines by 1.86 points.
- Qualitative Comparison: PackForcing preserves subject identity, visual fidelity, and dynamic motion across 120 seconds, while baselines progressively lose details, motion, or subject consistency.Self-Forcing shows color shifts and collapse, CausVid loses background details, and LongLive restricts motion.
- Ablation Studies: Removing sink tokens causes severe semantic drift, while disabling RoPE adjustment or dynamic context selection introduces severe frame-reset artifacts.These ablations identify all three mechanisms as important to stable long-video generation.
- Ablation Studies: Nsink=8 balances motion and consistency, achieving Dynamic Degree 49.84, Image Quality 73.18, and Aesthetic Quality 66.46 with bounded memory.An excessively large sink reduces Dynamic Degree to 35.16 by over-conditioning on static early frames.
- Generalization from Short-Video Supervision: 24× temporal extrapolation from 5-second training clips to 120-second videos is attributed to bounded context size and representational compatibility.The context remains approximately 27,872 tokens, while joint dual-branch training aligns compressed and full-resolution representations.
5 Conclusion
PackForcing addresses error accumulation and unbounded memory growth in autoregressive video generation with a three-partition KV cache. It achieves bounded memory, 24× temporal extrapolation, and coherent 2-minute video generation.
- PackForcing partitions the KV cache into sink, compressed mid, and recent tokens to address error accumulation and unbounded memory growth.
- 4 GB KV cache enables 24× temporal extrapolation from 5 s to 120 s.
- PackForcing generates highly coherent 2-minute videos and establishes state-of-the-art VBench scores and text-video alignment among existing baselines.
A Extended Discussion on Limitations
The paper identifies fixed compression, incomplete visual-saliency modeling, and unresolved quality trade-offs at higher resolutions as open limitations.
- The fixed 128× volume compression and ∼32× token reduction could be made adaptive to scene complexity.
- Attention-based importance scoring may miss aspects of visual saliency, motivating learned importance predictors.
- Scaling to 1920×1080 requires investigating how spatial compression interacts with quality.
B Design Comparison of Causal Video Generation Methods
The design comparison highlights PackForcing’s integration of learned compression, bounded memory, and RoPE adjustment for persistent long-range memory.
- PackForcing uniquely integrates learned compression, bounded memory, and RoPE adjustment for extended causal video generation.
C Training Strategy
PackForcing follows a two-phase self-forcing training procedure with causal rollout and teacher-guided optimization, while jointly training its compression layer. Training uses only 20 latent frames, and streaming VAE decoding supports progressive output.
- Training Strategy: The causal student is initialized from a pretrained bidirectional prior through ODE trajectory alignment, then optimized against a frozen bidirectional teacher during block-wise rollout.
- Training Strategy: The HR compression layer is optimized end-to-end within the rollout graph to preserve semantic and structural cues for causal attention.
- Short-to-Long Generalization: 20 latent frames, approximately 5 s after 4× VAE temporal decompression, support generalization to 2-minute generation with 24× temporal extrapolation.
- Training Strategy: Streaming VAE decoding incrementally produces blocks while a temporal cache maintains seamless frame boundaries and enables progressive display.
E Additional Experimental Results
PackForcing combines block-wise causal generation with KV-cache updates, dynamic mid-context selection, and incremental RoPE adjustment. With sink sizes of at least 8, it maintains a stable CLIP trajectory through minute-scale generation.
- Additional Experimental Results: Nsink ≥8 maintains a stable CLIP score trajectory throughout 120 seconds, whereas Nsink=0 drops to 28.51.The breakdown evaluates scores at 20-second intervals for different sink sizes.
- Additional Experimental Results: PackForcing inference alternates block-wise multi-step denoising, cache updates, mid-range compression, RoPE adjustment, and streaming VAE decoding.The procedure outputs frames incrementally while managing the cache capacity.
- Additional Experimental Results: Dynamic context selection identifies informative compressed mid-blocks using query-key affinity and routes a top-K subset into the active attention context.Selected compact caches are concatenated with recent blocks for the flash-attention forward pass.
- Additional Experimental Results: Affinity scoring reduces overhead through query subsampling, half-head evaluation, and step-wise caching.These optimizations avoid evaluating the full query grid and all attention heads at every step.
H.1 Empirical Analysis of Temporal Attention Patterns
Temporal attention analysis finds that historical demand spans the full cache and that important mid-range blocks change unpredictably. These observations motivate retaining sink, mid, and recent regions while using adaptive selection and position adjustment.
- H.1 Empirical Analysis of Temporal Attention Patterns: Selected blocks remain distributed across the full temporal range, with near-uniform late-stage importance averaging 0.499.The result rules out eviction policies that retain only the earliest and latest context.
- H.1 Empirical Analysis of Temporal Attention Patterns: The Jaccard distance between consecutive selection sets averages 0.75, while position diversity stabilizes above 0.85.Important blocks are scattered and non-contiguous rather than forming stable temporal clusters.
- H.1 Empirical Analysis of Temporal Attention Patterns: Full-history demand motivates a sink-plus-mid-plus-recent cache, while sporadic mid-token use motivates representative compression.The strategy preserves coverage of the past without retaining every mid-range token at full resolution.
- H.1 Empirical Analysis of Temporal Attention Patterns: With RoPE Adjustment, the early-to-late CLIP gap falls from 2.53 to 0.95, a 62% reduction, at a cost below 0.1% of total FLOPs.The comparison attributes the unadjusted gap to semantic drift after FIFO eviction begins.
H.3 Computational Efficiency Analysis
PackForcing supports 120-second generation with qualitative long-range consistency while substantially reducing the KV-cache memory requirement through compression.
- H.3 Computational Efficiency Analysis: PackForcing bounds KV-cache memory to ∼4.2 GB regardless of video length, compared with ∼138 GB without compression.The comparison concerns 120-second generation at 832×480 and 16 FPS.
- H.3 Computational Efficiency Analysis: The framework provides qualitative evidence of robust long-range consistency and dynamic generation in extended 120-second comparisons.The comparisons are made against state-of-the-art baselines.
I More Qualitative Results
Qualitative results show PackForcing preserving subject identity, structural details, and continuous motion across diverse two-minute video scenarios.
- I More Qualitative Results: PackForcing preserves an otter’s identity, lifejacket, and tropical-water details across two minutes, while baselines show degradation or inconsistency.Self-Forcing collapses by 60 seconds; CausVid blurs backgrounds and Rolling Forcing loses subject consistency.
- I More Qualitative Results: PackForcing maintains a glass sphere’s transparency, a dwarf’s identity, and continuous raking motion throughout 120 seconds.LongLive maintains the subject but freezes the motion, while other baselines lose structure or collapse.
- I More Qualitative Results: PackForcing preserves sparkles, vibrant colors, and dynamic rhythmic motion in a prolonged kangaroo disco scene.Competing methods face a trade-off between collapse under complexity and near-static motion.