Source-linked AI summary
Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantization
Haocheng Xi, Shuo Yang, Yilong Zhao, Muyang Li, Han Cai, Xingyang Li, Yujun Lin, Zhuoyang Zhang, Jintao Zhang, Xiuyu Li, Zhiying Xu, Jun Wu, Chenfeng Xu, Ion Stoica, Song Han, Kurt Keutzer
TL;DR
Autoregressive video generation is constrained by KV-cache memory, which limits deployment and retained history for long-horizon consistency. QVG introduces training-free, video-specific residual quantization using Semantic-Aware Smoothing and Progressive Residual Quantization. Across multiple models and benchmarks, it reduces KV-cache memory by up to 7.04× with 4% latency overhead while preserving near-lossless visual quality.
Problem
KV-cache memory grows with generation history, constraining deployment and the retained context needed for long-horizon video consistency.
Method
QVG groups semantically similar tokens, subtracts group centroids to form quantization-friendly residuals, and progressively quantizes them in coarse-to-fine stages.
Results
QVG achieves up to 7.04× KV-cache compression with 4% latency overhead while preserving near-lossless visual quality across multiple video models and benchmarks.
Takeaways & Limitations
QVG enables practical, memory-efficient long-video and world generation within the evaluated autoregressive video systems.
Abstract
from arXiv · showhide
Despite rapid progress in autoregressive video diffusion, an emerging system algorithm bottleneck limits both deployability and generation capability: KV cache memory. In autoregressive video generation models, the KV cache grows with generation history and quickly dominates GPU memory, often exceeding 30 GB, preventing deployment on widely available hardware. More critically, constrained KV cache budgets restrict the effective working memory, directly degrading long horizon consistency in identity, layout, and motion. To address this challenge, we present Quant VideoGen (QVG), a training free KV cache quantization framework for autoregressive video diffusion models. QVG leverages video spatiotemporal redundancy through Semantic Aware Smoothing, producing low magnitude, quantization friendly residuals. It further introduces Progressive Residual Quantization, a coarse to fine multi stage scheme that reduces quantization error while enabling a smooth quality memory trade off. Across LongCat Video, HY WorldPlay, and Self Forcing benchmarks, QVG establishes a new Pareto frontier between quality and memory efficiency, reducing KV cache memory by up to 7.0 times with less than 4% end to end latency overhead while consistently outperforming existing baselines in generation quality. Code is available at: https://github.com/svg-project/Quant-VideoGen
1. Introduction
Autoregressive video generation supports incremental, long-horizon synthesis but makes KV-cache memory a central deployment and capability bottleneck. QVG addresses this with video-specific, training-free quantization that targets quality, memory, and latency together.
- Mainstream bidirectional video diffusion models largely remain limited to 5–10-second generations in practical settings, leaving longer continuity and interaction underserved.
- Autoregressive generation enables streaming and long-horizon applications by making output frames depend only on retained history.
- Approximately 38K tokens for a 5-second 480p LongCat-Video generation require roughly 34 GB of KV-cache, making memory the binding resource before raw compute saturates.
- Longer retained KV-cache history improves preservation of identity, scene layout, and motion semantics during extended generation.
- Naively porting LLM KV-cache quantization to video diffusion causes severe quality degradation because video activations are heterogeneous across token and channel dimensions.
- QVG groups similar latent-space tokens, subtracts group centroids, and progressively quantizes residuals to trade quality against compression rate.
- QVG reduces KV-cache memory by up to 7.0× with less than 4% end-to-end latency overhead while outperforming quantization baselines across evaluated models and benchmarks.
2. Related Work
Related work frames autoregressive video generation as both an algorithmic and systems memory problem. Existing KV-cache quantization methods mainly target LLM statistics, motivating video-specific compression methods.
- QVG differs by exploiting video-specific spatiotemporal redundancy, where adjacent tokens tend to be numerically similar in latent space.
- Recent video systems shift from bidirectional clip-level denoising toward chunk-level autoregressive generation for long-horizon synthesis and real-time interaction.
- Explicit memory mechanisms and chunked rollouts extend long-horizon generation, but limited device memory constrains how much KV-cache can be retained.
- LLM KV-cache quantization methods address heavy-tailed outliers, structured distribution smoothing, and token-heterogeneous precision allocation.
3. Motivation
Video KV-caches grow rapidly with resolution, duration, and model size, while short context windows limit long-horizon quality. Video-specific redundancy and progressive structure provide the basis for QVG’s residual quantization approach.
- KV-cache memory grows linearly with latent-token count and requires 2 · L · (HWT) · d · ByteBF16 storage for L layers and hidden dimension d.
- A 5-second 480p LongCat-Video context uses roughly 38K latent tokens and about 34 GB of KV-cache, exceeding its 27 GB model-parameter footprint.
- Semantic-Aware Smoothing groups similar tokens and subtracts centroids, producing lower-magnitude, more homogeneous residuals for quantization.
- Fixed context windows such as 21 latent frames in Wan distilled Self-Forcing and 20 frames in HY-WorldPlay are driven primarily by GPU-memory constraints.
- Video KV-cache activations have dynamic ranges and token-dependent channel outliers, with observed maxima near 1e2 for K and 1e3 for V.
- Adjacent temporal tokens and nearby spatial patches often have high similarity, revealing redundancy that makes KV-cache compression feasible.
- Progressive Residual Quantization repeatedly processes residuals from coarse semantic structure toward fine-grained variation, lowering quantization error.
4. Methodology
QVG combines Semantic-Aware Smoothing with Progressive Residual Quantization to make KV-cache values more quantization-friendly and progressively reduce residual error. The framework supports quality–compression trade-offs through multi-stage residual processing and achieves near-lossless quality at high compression ratios.
- Semantic-Aware Smoothing: QVG first applies Semantic-Aware Smoothing to group semantically similar tokens, subtract group centroids, and produce lower-magnitude residuals for low-bit quantization.K-means forms token groups from hidden representations; centroid subtraction makes within-group distributions more homogeneous.
- Semantic-Aware Smoothing: 6.9× and 2.6× reductions in quantization error are achieved for key and value caches, respectively, after centroid subtraction.The larger key-cache reduction reflects the greater irregularity of value-cache statistics.
- Quality–Compression Trade-off: QVG-Pro achieves 4.97×–5.20× compression, while QVG reaches 6.94×–7.05× compression and maintains near-lossless video quality.The reported trade-off favors QVG-Pro for fidelity and QVG for maximum compression.
- Progressive Residual Quantization: Progressive Residual Quantization repeatedly smooths and re-quantizes residuals from coarse to fine granularity to capture increasingly detailed information.Each stage operates on the remaining residual; intermediate residuals are discarded while quantized outputs, centroids, and assignments are stored.
5. Experiments
QVG is evaluated across autoregressive video models and benchmarks, where it preserves generation quality while substantially reducing KV-cache memory and adding modest latency overhead. Ablations examine long-horizon quality, memory composition, progressive stages, and quantization group size.
- Quality Evaluation: QVG-Pro achieves the best fidelity scores, while QVG provides the largest compression ratios with marginal quality degradation on LongCat-Video-13B and HY-WorldPlay-8B.Both methods remain near-lossless on VBench metrics, whereas baselines degrade substantially, especially under INT2 quantization.
- Quality Evaluation: 28.716 PSNR is achieved at 6.94× compression for LongCat-Video-13B, while 29.174 PSNR is achieved at 7.05× compression for HY-WorldPlay-8B.
- Quality Evaluation: QVG and QVG-Pro maintain near-lossless Image Quality through 700 frames, while other baselines degrade sharply after approximately 100 frames.The Self-Forcing evaluation measures Image Quality every 50 frames along long video sequences.
- Efficiency Evaluation: End-to-end generation time increases by 2.1% on LongCat, 1.5% on HY-World, and 4.3% on Self-Forcing.These measurements indicate modest latency overhead from quantization and dequantization.
- Sensitivity Test: The first progressive quantization stage yields a 5.83× MSE reduction compared with naive quantization, while later stages provide at least 1.10× additional reduction with diminishing contributions.
- Sensitivity Test: Larger quantization blocks increase compression but reduce quality; block size 64 gives the best trade-off, whereas block size 16 gives the best quality.
6. Conclusion
QVG is a training-free KV-cache quantization framework that uses video-specific spatiotemporal redundancy to improve memory efficiency in autoregressive video generation. Across models and benchmarks, it reaches high compression with near-lossless visual quality and modest latency overhead.
- QVG combines Semantic-Aware Smoothing with Progressive Residual Quantization to produce quantization-friendly residuals and reduce error in a coarse-to-fine manner.
- QVG achieves up to 7.04× KV-cache compression with 4% latency overhead while preserving near-lossless visual quality.
- The reported results support practical, memory-efficient long-video and world generation.
Impact Statement
The paper reports potential societal consequences but does not identify any that require specific highlighting.
- The authors state that the work has many potential societal consequences, none of which they feel must be specifically highlighted.
A.1. Long Video Generation
Additional evaluations test whether QVG preserves long-horizon quality at 1400 frames and scales from 480p to 720p. QVG tracks BF16 quality over longer videos and maintains its error bounds without increasing centroid count at higher resolution.
- Long Video Generation: At 1400 frames, QVG remains on-par with BF16 Image Quality, while KIVI falls from 67.57 at 350 frames to 35.85.
- Scalability to Higher Resolutions: At 720p, QVG outperforms baseline quantization methods and maintains the same error bounds as at 480p.
- Scalability to Higher Resolutions: Moving from 480p to 720p does not require more centroids because QVG quantizes fixed-size KV-cache chunks rather than all tokens in a frame.Higher spatial resolution reduces the number of frames per chunk while preserving the per-chunk clustering scale.
B. Additional Hyperparameter Ablations
The ablations examine how chunk size and codebook size affect QVG’s overhead, compression, and reconstruction quality. Larger chunks favor efficiency, while K=128 and K=256 offer good trade-offs, with K=256 selected for implementation reasons.
- Chunk Size: 3.3% is the peak relative overhead across chunk sizes on Self-Forcing with INT2 quantization.The overhead is measured for QVG’s k-means clustering and quantization pipeline against end-to-end runtime.
- Chunk Size: Larger chunk sizes reduce overhead and increase the compression ratio.Clustering and metadata costs are better amortized over larger chunks.
- Codebook Size: Increasing K improves clustering quality but increases index and codebook overhead, reducing the compression ratio.The sweep compares K ∈ {64, 128, 256, 512} under INT4 and INT2 with one Progressive Residual Quantization stage.
- Codebook Size: K=128 and K=256 provide good trade-offs between clustering quality and metadata overhead.The final design uses K=256 because each token’s cluster ID fits naturally in one uint8 byte.
C. Additional Hardware and System Overhead Analysis
The system analysis evaluates QVG overhead across GPU platforms and scaling conditions. Overhead remains small on both H100 and RTX 5090 hardware, while chunk size primarily determines per-chunk cost as workload dimensions grow.
- Hardware Comparison: Below 2% of end-to-end latency, QVG’s quantization overhead remains small on both H100 and RTX 5090 platforms.The overhead is slightly higher on the RTX 5090 than on the H100.
- Scaling Analysis: Chunk size, rather than resolution, sequence length, or batch size, mainly determines QVG’s relative overhead.QVG operates on fixed-size KV-cache chunks, so larger workloads mainly increase the number of chunks while per-chunk cost stays unchanged.
- Scaling Analysis: As batch size increases, model forward computation and QVG bookkeeping scale roughly linearly.The batch-size study reports latency and QVG overhead across different batch sizes.