PaperlayerView exact revision on arXiv

Source-linked AI summary

Flow caching for autoregressive video generation

Yuexiao Ma, Xuzhe Zheng, Jing Xu, Xiwei Xu, Feng Ling, Xiawu Zheng, Huafeng Kuang, Huixia Li, Xing Wang, Xuefeng Xiao, Fei Chao, Rongrong Ji

arXiv:2602.10825v1cs.CVcs.AI

TL;DR

Autoregressive video generation needs efficient caching because long-video inference remains costly and chunks exhibit heterogeneous denoising behavior. FlowCache uses chunkwise adaptive caching together with importance-redundancy KV-cache compression, achieving substantial speedups with minimal VBench changes on MAGI-1 and SkyReels-V2.

  • Problem

    Autoregressive video generation remains computationally demanding, while uniform caching fails to account for heterogeneous denoising states across video chunks and importance-only KV selection can retain redundant video tokens.

  • Method

    FlowCache independently adapts caching policies for video chunks and compresses KV caches by balancing token importance with redundancy under a fixed budget.

  • Results

    FlowCache achieves 2.38× and 6.7× speedups on MAGI-1 and SkyReels-V2, respectively, with VBench changes of 0.87 ↑ and 0.79 ↓.

  • Takeaways & Limitations

    The results establish FlowCache as an efficient framework for autoregressive video generation that preserves generation quality while reducing inference latency.

  • Takeaways & Limitations

    The theoretical analysis assumes an optimal flow-matching velocity field, power-law scheduling, and heterogeneous chunk state norms at intermediate timesteps.

Abstract

from arXiv · show

Autoregressive models, often built on Transformer architectures, represent a powerful paradigm for generating ultra-long videos by synthesizing content in sequential chunks. However, this sequential generation process is notoriously slow. While caching strategies have proven effective for accelerating traditional video diffusion models, existing methods assume uniform denoising across all frames-an assumption that breaks down in autoregressive models where different video chunks exhibit varying similarity patterns at identical timesteps. In this paper, we present FlowCache, the first caching framework specifically designed for autoregressive video generation. Our key insight is that each video chunk should maintain independent caching policies, allowing fine-grained control over which chunks require recomputation at each timestep. We introduce a chunkwise caching strategy that dynamically adapts to the unique denoising characteristics of each chunk, complemented by a joint importance-redundancy optimized KV cache compression mechanism that maintains fixed memory bounds while preserving generation quality. Our method achieves remarkable speedups of 2.38 times on MAGI-1 and 6.7 times on SkyReels-V2, with negligible quality degradation (VBench: 0.87 increase and 0.79 decrease respectively). These results demonstrate that FlowCache successfully unlocks the potential of autoregressive models for real-time, ultra-long video generation-establishing a new benchmark for efficient video synthesis at scale. The code is available at https://github.com/mikeallen39/FlowCache.

1 INTRODUCTION

Autoregressive video generation reduces long-video inference complexity but remains computationally demanding, while heterogeneous chunk denoising undermines uniform caching. FlowCache addresses this with chunk-specific caching and KV-cache compression, achieving substantial acceleration with minimal quality impact.

  • Motivation: Autoregressive models partition videos into fixed-frame chunks and reduce long-video inference complexity from quadratic to linear scaling.They use causal autoregressive denoising with a fixed denoising window.
  • Motivation: Different video chunks exhibit distinct similarity patterns at identical denoising timesteps, making uniform caching inadequate.Similarity deteriorates toward clean video, varies across chunks at the same timestep, and remains high between model inputs and outputs.
  • Motivation: KV caches grow during autoregressive generation and support physical-dynamics modeling and temporal consistency across video sequences.The paper identifies KV-cache compression as an important gap because existing techniques primarily target language models.
  • Method: FlowCache treats each video chunk as an independent computational entity and dynamically adapts its caching policy to the chunk’s denoising state.This chunkwise strategy is designed to improve computational efficiency and generation flexibility.
  • Results: FlowCache achieves 2.38× and 6.7× inference speedups on MAGI-1 and SkyReels-V2, with VBench changes of 0.87 ↑ and 0.79 ↓, respectively.The reported results indicate higher efficiency with minimal quality impact compared with baseline methods.

2 RELATED WORK

Related work covers autoregressive video generation, training-free feature caching for diffusion models, and KV-cache compression methods developed largely for language-model inference.

  • Autoregressive Video Generation: Autoregressive video-generation methods model long videos as sequences of discrete chunks, adapting diffusion and language-modeling paradigms to address quadratic long-sequence complexity.Each video chunk functions as a fundamental generation unit.
  • Cache Strategy: Training-free diffusion-model caching methods exploit temporal redundancy across denoising timesteps through input estimation or token-wise importance assessment.Examples include timestep-modulated output estimation and attention-based token selection.
  • KV Cache Compression: KV-cache compression methods for language models selectively retain past key and value states using importance-based strategies such as heavy-hitter or observation-window selection.The cited approaches reduce memory and computation by retaining selected historical tokens.

3 METHODOLOGY

FlowCache addresses the mismatch between uniform caching and heterogeneous denoising in autoregressive video generation through chunkwise adaptive reuse and importance-redundancy-aware KV cache compression. The method preserves generation quality while substantially accelerating inference and reducing attention-memory overhead.

  • Autoregressive video models partition long videos into chunks and use causal denoising, reducing long-video inference complexity from quadratic to linear with a fixed denoising window.
  • Existing cache methods are incompatible with autoregressive inference because different chunks exhibit unequal relative L1 distances at the same timestep.
  • FlowCache: FlowCache independently evaluates each chunk’s similarity trajectory to decide whether to reuse cached activations or perform recomputation.
  • FlowCache: 2.38× acceleration on MAGI-1 accompanies a 0.87% VBench improvement, while 6.7× speedup on SkyReels-V2 accompanies a 0.79% VBench degradation.
  • KV cache compression: FlowCache compresses KV caches by selecting historical tokens that are both important to current denoising queries and non-redundant under a fixed per-head budget.
  • KV cache compression: Joint importance-redundancy selection preserves long-range temporal consistency while reducing DiT attention memory and computation.

4 EXPERIMENTS

Experiments evaluate FlowCache on two autoregressive video models using video quality and inference-efficiency metrics. Results examine overall performance, component ablations, and KV-cache compression trade-offs.

  • Settings: Experiments use MAGI-1-4.5B-distill and SkyReels-V2-1.3B-540P as representative autoregressive diffusion models.Their implementations differ in chunking and autoregressive structure.
  • Settings: Evaluation measures perceptual video quality with VBench, LPIPS, PSNR, and SSIM, and efficiency with FLOPs and inference latency.Video-quality assessments use the VBench-long benchmark.
  • Results: 2.38× speedup with VBench 77.93 is achieved by FlowCache-fast on MAGI-1, while FlowCache-fast reaches 6.7× speedup with VBench 83.05 on SkyReels-V2.FlowCache-fast maintains high visual quality across both evaluated models.
  • Ablation Study: Chunkwise reuse preserves generation quality better than TeaCache reuse, while KV-cache compression causes negligible quality loss on MAGI-1 and SkyReels-V2.The ablation compares reuse strategy and KV-cache compression under the fast variant.
  • Ablation Study: The compression study analyzes query and key selection granularity, importance–redundancy balancing through λ, and quality–GPU-memory trade-offs across cache budgets.These experiments target preservation of critical historical context during compression.

5 CONCLUSION

FlowCache is a framework for accelerating autoregressive video generation by adapting caching to heterogeneous chunk denoising states and compressing growing KV caches. Experiments on MAGI-1 and SkyReels-V2 demonstrate its effectiveness for efficient video synthesis.

  • Conclusion: FlowCache accelerates autoregressive video generation with chunkwise adaptive caching and dedicated KV-cache compression.The framework independently manages reuse policies for video chunks according to their denoising states.
  • Conclusion: The method addresses heterogeneous denoising trajectories and the memory overhead of KV caches used for long-range temporal consistency.These are the empirical and systems motivations identified for the framework.
  • Conclusion: Experiments on MAGI-1 and SkyReels-V2 establish FlowCache as an effective training-free approach for efficient autoregressive video generation.The conclusion characterizes the method as bridging autoregressive video models and practical real-time, ultra-long video synthesis.

B PROOF OF THEOREM 1

Under an optimal flow-matching velocity field and power-law schedule, the proof analyzes relative L1 distance as a product of a decreasing scheduler term and an increasing norm-ratio term. The scheduler’s hyperbolic decay is shown to dominate, making relative L1 distance decrease over time.

  • Assumptions: Theorem 1 assumes an optimal velocity field and power-law schedule σ(t) = (t/T)^p with p > 0.The proof considers 0 < t1 < t2 ≤ T for a data chunk Xi.
  • Analysis of A(t): A(t) = p/t is strictly monotonically decreasing for t ∈ (0, T].This term arises from the derivative-to-schedule ratio under the power-law schedule.
  • Analysis of B(t): B(t) increases because its numerator increases with t while its denominator decreases along the denoising trajectory.The numerator depends on the increasing schedule, whereas the state norm in the denominator decreases from noise toward data.
  • Combined Analysis: The decreasing factor A(t) dominates B(t)’s milder, saturating growth, so their product is monotonically decreasing.The argument relies on B(t) approaching a finite constant and its growth rate diminishing over time.
  • Conclusion: Therefore, relative L1 distance satisfies L1rel(X, t1, i) ≥ L1rel(X, t2, i) whenever 0 < t1 < t2 ≤ T.This completes the proof of the theorem’s temporal monotonicity claim.

C PROOF OF COROLLARY 1

The corollary establishes that heterogeneous video chunks have unequal relative L1 distances at the same timestep. The proof attributes this divergence to different chunk states along distinct denoising trajectories.

  • Assumptions: For heterogeneous chunks i ≠ j, the corollary assumes their state norms differ at intermediate timesteps.This content heterogeneity is the condition under which cross-chunk relative L1 distances are compared.
  • Numerators: At a fixed timestep, approximately constant update magnitude makes the ratio of the model-output numerators approximately one.The proof attributes this to prior-chunk conditioning steering update direction more than overall magnitude.
  • Denominators: Distinct target data distributions generate different denoising trajectories and unequal intermediate state norms.The flow-matching path from initial noise to final data is unique for each target data chunk.
  • Conclusion: Consequently, L1rel(X, t, i) ≠ L1rel(X, t, j) for distinct heterogeneous chunks at a fixed timestep.The unequal relative distances reflect differences in underlying chunk content and denoising states.

D.1 IMPLEMENTATION DETAILS OF KV CACHE COMPRESSION

FlowCache compresses KV caches only after reaching a fixed budget, while input video chunk activations are reused throughout generation. The implementation therefore separates cache filling from ongoing compression and reuse.

  • Cache Filling Phase: KV cache compression begins only after the cache reaches its specified budget.No compression occurs while the cache is still being filled.
  • Cache Filling Phase: During cache filling, new KV entries are added until the budget is reached.This phase preserves all newly added entries without compression.
  • Compression Phase: Once the cache is full, each arriving KV entry triggers compression of existing entries within the budget.The compression phase accommodates new entries while maintaining the specified cache bound.
  • Combined Acceleration: Input video chunk activation reuse applies to all chunks throughout generation, and end-to-end acceleration measures its combined effect with KV compression.The evaluation accounts for both caching mechanisms rather than compression alone.

D.2 EFFICIENCY ANALYSIS OF KV CACHE COMPRESSION AND FEATURE REUSE

FlowCache’s KV compression and input-cache retention add limited memory and computation overhead. The analysis reports small illustrative costs, infrequent selection operations, and negligible overall overhead.

  • KV Importance Computation: 0.30 GB and 0.077s are the optimized illustrative costs for online KV-cache importance computation, versus 17.79 GB and 10.26s for the naive approach.These costs are evaluated at Lk = 24,300 and d = 128.
  • Input Cache: 0.6 GB is the reported input-cache memory footprint when Lq = 108000 and hidden states = 3072.The paper describes this footprint as negligible within overall inference overhead.
  • Selection Overhead: 0.0013 PFLOPs and 0.05 PFLOPs are the token-level and frame-level costs of selecting and retaining KV-cache tokens.For a 10-second, 720×720 text-to-video sequence with a five-chunk budget, the operation runs only five times.
  • Overall Efficiency: FlowCache’s additional memory footprint and computational overhead are reported as negligible.The paper refers to experimental memory results in Table 6.

E.1 APPLICATION ON THE 16-STEP DISTILLED MODEL.

FlowCache complements distillation by improving inference efficiency while preserving generation quality on the distilled MAGI-1 model. Its plug-and-play design requires no large-scale autoregressive-model distillation training.

  • 2.38× vs. 3.56× speedup: the 16-step distilled baseline is faster, but its quality is lower than FlowCache-fast on the 64-step model.The reported quality scores are 71.69% for the distilled baseline and 70.26% for FlowCache-fast.
  • 1.92× vs. 1.17× speedup: FlowCache outperforms TeaCache on the distilled model while preserving generation quality.The corresponding quality scores are 70.72% and 70.60%.
  • FlowCache provides orthogonal gains to distillation as a plug-and-play online method without large-scale distillation training.

E.2.1 COMPRESSION GRANULARITY

The compression ablations show that coarse global reuse can severely damage performance, whereas chunkwise reuse and fine-grained cache representations better preserve historical context. The importance–redundancy balance is controlled through λ.

  • 43.10% vs. 13.69% accuracy: ChunkWise reuse substantially outperforms non-discriminative TeaCache reuse on Physics-IQ.The baseline without acceleration or compression achieves 47.60%.
  • 39.34% vs. 38.62% and 38.99% accuracy: token-level keys outperform frame- and chunk-level key aggregation when queries are token-level.The experiment fixes the query to the last 50 tokens of the last denoising chunk.
  • 39.53% vs. 39.34% accuracy: frame-level queries slightly outperform token-level queries with token-level keys, but require substantially more GPU memory.The paper therefore adopts token-level queries as sufficient for retrieving important KV entries.
  • Physics-IQ scores increase as λ decreases and stabilize beyond a threshold, motivating the uniform choice λ = 0.07.The trend indicates that KV-cache redundancy captures useful historical information beyond raw importance alone.

E.2.3 DIFFERENT KV CACHE BUDGET

FlowCache’s cache-budget experiments evaluate memory savings alongside physics reasoning, while qualitative comparisons examine efficiency, fidelity, and robustness across video-generation settings.

  • Table 3 compares inference efficiency and visual quality between 16-step and 64-step MAGI-1 configurations using weighted VBench* scores.VBench* is the weighted sum of eight selected representative metrics.
  • 33.6% peak-memory reduction with Physics-IQ stability within ±0.1 demonstrates substantial memory savings without compromising physics reasoning quality.At a 5-chunk budget, FlowCache slightly outperforms Vanilla.
  • FlowCache outperforms TeaCache in fine-grained details and overall image quality while avoiding TeaCache’s visible noise artifacts.The comparison covers qualitative results for MAGI-1 and SkyReels-V2.
  • FlowCache variants show greater robustness than TeaCache in complex scenes or videos with significant motion.
  • FlowCache-fast provides higher acceleration, whereas FlowCache-slow preserves higher fidelity and consistency with the reference quality.The supplied passage states this speed–fidelity distinction without reporting the specific ratios here.
Loading 2602.10825v1…