PaperlayerView exact revision on arXiv

Source-linked AI summary

CORAL-KV: Compact Cross-Layer Prediction and Residual Coding for Autoregressive Video KV Caches

Albert Ge, Akash Haridas, Utkarsh Saxena, Ruijing Yang, Ian Colbert, Frederic Sala, Vikram Appia, Emad Barsoum

arXiv:2610.12514v1cs.CVcs.LG

TL;DR

Growing KV caches constrain the memory available for long autoregressive video generation, while clustering-based compression can accumulate reconstruction errors over time. Coral-KV predicts cache states across layers and quantizes the remaining residuals with compact predictors. Its reusable and video-adapted configurations improve fidelity over matched clustering baselines while using less stored cache memory.

  • Problem

    Growing KV caches can dominate memory in long video generation, and small compression errors may compound as later frames attend to compressed history.

    Source: §1 · p. 1, §3 · p. 3

  • Method

    Coral-KV predicts keys and values across cache layers with compact block-DPLR predictors, then quantizes the remaining residuals using reusable or video-adapted predictors.

    Source: §1 · p. 2

  • Results

    Reusable Coral-KV improves PSNR over QVG-4 by 0.32 dB on HY-WorldPlay and 4.95 dB on MAGI-1, using 48.4% and 37.3% less cache memory, respectively.

    Source: §6 · p. 9

  • Takeaways & Limitations

    On HY-WorldPlay and MAGI-1, both configurations improve fidelity over matched clustering baselines while requiring less stored cache state.

    Source: §7 · p. 10

    Evaluation covers two video models and one held-out prompt source; transfer to other models, prompt distributions, or longer horizons is not established.

    Source: §8 · p. 10

Abstract

from arXiv · show

Autoregressive video diffusion models maintain a growing attention key-value (KV) cache, making memory a key bottleneck for long video generation. Recently, clustering-based quantization has emerged as an effective method to compress video KV caches by grouping similar key and value vectors then quantizing residual errors. However, when later frames attend to the compressed history, small reconstruction errors can compound as the video grows. Beyond the within-layer similarity exploited by clustering, we find strong dependencies across cache layers in video models. This motivates Coral-KV - Calibrated Offline, Residual-coded, and Adapted Locally - a KV-cache compression framework that uses cache states in each layer to predict keys and values in subsequent layers, then quantizes the remaining prediction residuals. We construct compact block-diagonal-plus-low-rank (block-DPLR) predictors: block-diagonal components model channel relationships within each attention head, while a global low-rank term models dependencies between heads. Coral-KV exposes a flexible fidelity-memory trade-off: it can reuse its predictors across videos or refit lightweight block-diagonal predictors to the current video for higher fidelity. Across extensive evaluations on both HY-WorldPlay and MAGI-1, our reusable configuration improves PSNR over QVG-4 by 0.32 dB and 4.95 dB, respectively, while using 48% and 37% less cache memory. Against QVG-Pro, our video-adapted configuration improves MAGI-1 PSNR by 3.77 dB while using 33.7% less cache memory. Coral-KV also maintains higher reconstruction fidelity over extended generation and reduces temporal discontinuities while retaining comparable generation latency.

1 Introduction

Autoregressive video models need growing KV caches to preserve long histories, but cache memory can limit their use and retained context. CORAL-KV exploits cross-layer dependencies with compact predictors and residual quantization, improving the memory–quality trade-off over clustering-based compression.

1 Introduction

  • The KV cache grows with video duration, spatial resolution, and model depth, potentially dominating GPU memory and limiting retained history.Longer context supports identity, scene geometry, and motion over long rollouts.Source: §1 · p. 1
  • Clustering-based QVG compresses keys and values within layers, but small early quantization errors can propagate into reconstruction drift over long generation.The introduction illustrates the memory–quality frontier across 32 prompts per model and matched-frame comparisons of BF16, QVG-1, QVG-Pro, and reusable CORAL-KV.Source: §1 · p. 1, Figure 1 · p. 2
  • CORAL-KV predicts cache states across layers with compact block-DPLR predictors and quantizes the remaining residuals.The predictors can be calibrated offline for reuse or adapted for higher fidelity; the framework co-designs prediction and residual quantization.Source: §1 · p. 2
  • Block-DPLR predictors use 13.1× and 3.0× fewer matrix parameters than dense prediction on HY-WorldPlay and MAGI-1, respectively.Coral-KV improves visual fidelity and temporal stability over matched baselines, with comparable generation latency for the reusable configuration.Source: §1 · p. 2

2 Related work

Related work spans video-cache compression and attention reduction, LLM KV compression, and low-bit residual quantization. Coral-KV retains all historical token positions while reducing the memory needed to store their KV values.

2 Related work

  • Video-cache methods compress or reduce attention history through clustering, token pruning, sparse attention, low-rank representations, or online parameter updates.Coral-KV is compared directly with QVG, which fits per-head centroid codebooks online for each cache block [16].Source: §2 · p. 3
  • Unlike methods that change accessible tokens or attention use, Coral-KV retains every position while reducing KV storage memory.Source: §2 · p. 3
  • Prior LLM KV methods quantize or correct cached state; AQUA-KV uses dense cross-layer predictors with residual quantization.Coral-KV applies compact block-DPLR predictors, which reduce memory and inference cost relative to dense maps.Source: §2 · p. 3
  • Coral-KV rotates prediction residuals with fixed Hadamard transforms and encodes them using the Gaussian-tuned 2-bit EDEN-2 codebook [29].Transform width is selected using quantization and video-quality ablations.Source: §2 · p. 3

3 Problem setting and motivation

Cache compression can reduce storage, but reconstruction errors may compound during generation and create a visible quality drop at the first frame attending to compressed history. Cross-layer predictors offer an alternative to centroids, with lower reconstruction error in the reported comparisons.

3 Problem setting and motivation

  • A 285-frame, near-720p HY-WorldPlay video would require 46.71 GB to retain every completed BF16 cache block.For L attention layers, KV width C, and T retained patch tokens, storing keys and values requires 2LTC × 16 bits.Source: §3 · p. 3
  • Early quantization errors can magnify into oversmoothing, blended structures, and geometric drift; the first frame attending to a newly compressed block can show an abrupt quality drop.This transition is termed quantization-boundary discontinuity (QBD).Source: §3 · p. 3
  • A key-to-key map explains 94% of variance on HY-WorldPlay and 89% on MAGI-1; adding current-layer keys raises value-prediction variance explained to 97% and 82%, respectively.The corresponding value-prediction figures rise from 85% to 97% on HY-WorldPlay and from 65% to 82% on MAGI-1.Source: §3 · p. 3
  • Reconstructed keys and preceding-layer values predict current-layer values, while reconstructed preceding-layer states feed the layerwise predictors.Figure 2 also compares predictor and centroid reconstruction MSE before residual quantization and illustrates QVG’s first frame attending to compressed history.Source: §3 · p. 3, Figure 2 · p. 4
  • Predictors reduce relative MSE versus matched-storage centroids by 59% for keys and 41% for values on HY-WorldPlay, and 51% and 27% on MAGI-1.The comparisons use centroid codebooks with 705 centroids per key or value head on HY-WorldPlay and 513 on MAGI-1.Source: §3 · p. 4

4 CORAL-KV

CORAL-KV reconstructs compressed cache blocks layer by layer, using reconstructed states to predict subsequent keys and values and quantizing prediction residuals. Its block-DPLR predictors and tailored residual coding balance fidelity, memory, and fitting cost.

4.1 Reconstruction overview

  • At each layer above zero, reconstructed prior-layer keys predict current keys; reconstructed current keys and prior-layer values predict current values, with quantized residuals added to both predictions.Layer-zero keys and values are quantized directly; INT4 is reported as sufficient to preserve fidelity on both models.Source: §4.1 · pp. 4–5, Figure 3 · p. 5

4.2 Compact predictors and residual quantization

  • Block-DPLR combines head-aligned block-diagonal maps with a rank-r component to model within-head and cross-head relationships.Direct factorized application avoids forming dense predictor matrices; at rank 128, it uses 13.1× and 3.0× fewer matrix parameters than dense prediction on HY-WorldPlay and MAGI-1.Source: §4.2 · pp. 5–6
  • Offline predictors pooled from 32 MovieGen Extended videos are reused across videos; video-adapted predictors refit block-diagonal terms per filled cache block while keeping low-rank factors fixed.The adaptation improves fidelity at a memory cost, and fixed low-rank factors avoid the slower global online update.Source: §4.2 · p. 6
  • Block-DPLR outperforms block-diagonal and parameter-matched low-rank predictors on both models; the offline dense predictor gains only 0.10 and 0.79 dB over reusable rank-128 block-DPLR.The respective gains are on HY-WorldPlay and MAGI-1, with substantially more predictor memory for the dense control.Source: §4.2 · p. 6, Figure 4 · p. 6
  • H16 and H64 transforms perform most consistently across both models, while wider transforms have model-dependent effects and can give HY-WorldPlay heavier value-residual tails.Source: §4.2 · p. 7, Figure 5 · p. 7
  • At one scale per 64 values, H64 reduces key-residual quantization error by 7.49% on HY-WorldPlay and 2.96% on MAGI-1, improving mean video PSNR by 0.69 and 0.25 dB.Value-residual error changes by less than 0.03%; the transformed residual pairs use the fixed Gaussian-tuned EDEN-2 codebook [29].Source: §4.2 · p. 7

5 Experimental methodology

The evaluation tests Coral-KV on two autoregressive video generators using held-out prompts and matched BF16 generations. It measures reconstruction quality, boundary discontinuities, stored-cache memory, and generation time.

Models, data, and baselines

  • 355 held-out MovieGen Extended prompts evaluate HY-WorldPlay and MAGI-1, with fidelity scored on frames after compressed cache blocks begin affecting generation.HY-WorldPlay generates 285 frames at 704 × 1280 and retains the most recent 48 latent frames for fidelity evaluation; MAGI-1 generates 192 frames at 720 × 720.Source: §5 · p. 7
  • The baselines include BF16, three QVG configurations, and an AQUA-KV-based dense cross-layer predictor, with QVG-4 serving as the latency baseline.QVG-Pro is the strongest quality baseline; QVG-4 matches reusable Coral-KV’s residual-bit allocation.Source: §5 · p. 7

Quality evaluation

  • RGB PSNR, SSIM, and VGG-LPIPS measure reconstruction fidelity against matched BF16 generations, scored after compression begins affecting frames.Each filled cache block is compressed as a unit: 7,040 visual tokens on HY-WorldPlay and 12,150 on MAGI-1.Source: §5 · p. 8
  • QBD compares discontinuity near each quantized block’s first affected frame with typical within-video frame similarity, and higher QBD indicates a more pronounced change.Table 2 normalizes each method’s average per-video QBD by its paired BF16 generations at the same scheduled frame indices.Source: §5 · p. 8

Memory and latency evaluation

  • Memory totals include stored cache components and video-specific parameters, while generation time includes video-specific fitting time.HY-WorldPlay memory counts all completed blocks across the 285-frame generation; these totals describe stored cache representations, not peak GPU memory.Source: §5 · p. 8, Table 1 · p. 8

6 Main results

Across both models, reusable Coral-KV improves fidelity over QVG-4 while using less cache memory. Video-adapted Coral-KV improves fidelity over QVG-Pro and reduces boundary discontinuities, while shared predictors lower total memory across videos.

Reusable Coral-KV

  • 0.32 dB and 4.95 dB PSNR gains over QVG-4 accompany 48.4% and 37.3% less cache memory on HY-WorldPlay and MAGI-1, respectively.Reusable Coral-KV also lowers generation overhead on both models.Source: §6 · pp. 8–9

Video-adapted Coral-KV

  • 3.77 dB higher MAGI-1 PSNR with 33.7% less cache memory, and 32.7% less memory on HY-WorldPlay, characterize video-adapted Coral-KV versus QVG-Pro.Video-adapted Coral-KV improves all three fidelity metrics over QVG-Pro on both models, with higher generation overhead than reusable Coral-KV.Source: §6 · p. 9

Memory scaling

  • At 16 videos, shared predictors give reusable Coral-KV lower total memory than QVG-1 and QVG-4 on both models.HY-WorldPlay’s full-history total is 6.90 GB versus 13.36 GB for QVG-4; on wide HY-WorldPlay caches, video-adapted Coral-KV uses 23.1% less full-history memory than AQUA-KV.Source: §6 · p. 9

Boundary stability and VBench

  • Video-adapted Coral-KV nearly halves MAGI-1 QBD versus QVG-Pro, at 1.328 versus 2.527, while both configurations use 32.7–52.3% less single-video cache memory.Boundary-discontinuity reductions persist across all 12 tested settings, with broadly similar VBench scores.Source: §6 · p. 9, Table 2 · p. 9

Temporal variation

  • QVG shows 13% less frame-to-frame change than BF16 on HY-WorldPlay, whereas Coral-KV nearly matches BF16’s frame-to-frame change.The passage reports that unusually smooth frames can produce higher VBench motion-smoothness and flicker scores.Source: §6 · p. 9

7 Conclusion

Coral-KV predicts cache states across layers and quantizes prediction residuals. Its reusable and video-adapted configurations improve fidelity over matched clustering baselines while using less stored cache state.

Conclusion

  • Coral-KV predicts cache states across layers and quantizes the remaining residual to compress autoregressive video KV caches.Its predictors can be calibrated offline for reuse across videos or adapted per cache block for higher fidelity, at additional memory and fitting-time cost.Source: §7 · p. 10
  • Both Coral-KV configurations improve fidelity over matched clustering baselines on HY-WorldPlay and MAGI-1 while requiring less stored cache state.The results show that better reconstruction need not require a larger cache.Source: §7 · p. 10
  • Sharing predictor parameters amortizes their cost across concurrent video caches, so total cache memory grows more slowly with video count.The conclusion presents this as a direction toward high-fidelity video generation with smaller, more scalable KV caches.Source: §7 · p. 10

8 Limitations

The paper identifies limits in its evaluation scope and describes related work and the quantization choices used in the full reconstruction specification.

8 Limitations

  • Evaluation covers two autoregressive video models and one held-out prompt source; transfer to other models, prompt distributions, and longer horizons is not established.The authors also do not formally explain why linear cross-layer prediction works; memory totals describe stored cache state, not whole-process peak GPU memory.Source: §8 · p. 10

A Expanded related work

  • Related work spans video-cache compression, LLM cache quantization and cross-layer methods, residual coding, and evaluation protocols.The cited approaches include token selection and attention reduction alongside cache compression.Source: A Expanded related work · p. 17

B.1 Full quantization and reconstruction specification

  • The specification uses a fixed block Hadamard transform and selects its width empirically because it has no input-independent distributional guarantee.The key format uses (8, 64) scale bits and group size, and the value format uses (16, 128), each yielding 2.125 bits per value.Source: §B.1 · p. 18
  • Residual groups use four-bit EDEN-2 indices, with key and value formats each requiring 2.125 bits per value before alignment and metadata.The fixed codebook is shared; reported memory comes from allocated tensors rather than this bit-count calculation.Source: §B.1 · p. 19

B.2 Efficient inference implementation

The implementation uses factorized predictors and fused compressed-cache conversion to reduce memory and conversion overhead, while checking numerical fidelity before latency evaluation.

B.2 Efficient inference implementation

  • The block-DPLR implementation avoids materializing the combined predictor and uses less predictor-application time on HY-WorldPlay, but not MAGI-1.Dense-to-block-DPLR application ratios are 2.64× on HY-WorldPlay and 0.76× on MAGI-1, so the factorized form is selected for memory efficiency rather than assumed universal speedup.Source: §B.2 · p. 19
  • Fused conversion kernels combine supported encoding, decoding, residual addition, and layout operations without changing the stored representation or reconstruction rule.MAGI-1 uses fused codebook assignment but retains conservative transform and reconstruction dispatch where broader fusion fails numerical checks.Source: §B.2 · p. 19
  • Only implementations passing numerical checks enter the latency study, with attention implementations held fixed across methods within each model.HY-WorldPlay uses AITER and MAGI-1 uses native grouped-query attention.Source: §B.2 · p. 19, Table 3 · p. 20

C.1 Evaluation configurations

The evaluation tests cross-layer prediction, boundary stability, matched-memory fidelity, and memory accounting under specified protocols. Coral-KV reduces boundary discontinuities and outperforms token-selection controls on matched-memory fidelity in the tested settings.

C.1 Evaluation configurations

  • Cross-layer predictability is measured with held-out cache dumps; the matched-storage reconstruction comparison uses clean predecessor states and does not test recursive reconstruction through a generated video.Offline predictors are fitted only on the calibration split, and configuration choices are fixed before evaluation aggregates are examined.Source: §C.1 · p. 20, Table 5 · p. 21, Figure 6 · p. 21

D.1 QBD implementation and robustness

  • Both Coral-KV configurations have lower BF16-normalized QBD than QVG-Pro on both models across all 12 tested definitions.For the primary definition, prompt-paired 95% bootstrap intervals for the advantage exclude zero on both models.Source: §D.1 · p. 21

D.2 VBench and BF16-relative dynamics

  • The boundary-stability protocol reports QBD and six VBench dimensions separately, and checks temporal dynamics against paired BF16 because smoothness scores can reward suppressed motion.BF16-relative diagnostics include inter-frame difference, spatial total variation, and normalized frame-residual difference.Source: §D.2 · pp. 21–22

D.3 Per-boundary QBD

  • Across the 285-frame rollout, mean QBD is 1.035× BF16 for Coral-KV and 1.321× for QVG-4; at the final boundary, ratios are 1.166× and 1.747×.QVG’s fitted QBD slope is positive, while Coral-KV’s smaller slope interval includes zero.Source: §D.3 · p. 22, Figure 7 · p. 22

E Comparison with token selection

  • At matched memory, Coral-KV exceeds DensityKV by 3.27–5.11 dB and the recent-token control in every reported row.All four paired 95% prompt-bootstrap intervals for the DensityKV gains exclude zero; these tests measure fidelity, not perceptual quality on DensityKV’s original backbones.Source: E Comparison with token selection · pp. 22–23, Table 6 · p. 23

F Memory accounting and implementation controls

  • Reported compressed-history memory includes codes, scales, metadata, predictors, and online parameters, while excluding transient reconstructed tensors and workspaces.These analytical totals are not peak process-memory measurements.Source: F Memory accounting and implementation… · p. 23

F.1 Full-history cache scaling

  • The full-history HY-WorldPlay memory comparison retains every completed 7,040-token block across the nine-block, 285-frame generation.The totals include QVG codebooks or the reusable predictor parameters.Source: §F.1 · p. 23, Table 7 · p. 24

F.2 Memory sharing across concurrent videos

  • Concurrent Coral-KV memory is accounted as shared predictor memory plus per-video cache memory, B + NC; QVG memory scales as NQ.The 16-video total uses shared-parameter accounting rather than measured concurrent throughput.Source: §F.2 · pp. 23–24

F.3 Matched latency definition and results

In matched latency experiments, reusable Coral-KV has lower reported overhead than QVG-Pro on both models, while video adaptation increases overhead relative to reuse.

F.3 Matched latency definition and results

  • 10.7% versus 17.7% on HY-WorldPlay and 6.8% versus 9.6% on MAGI-1: reusable Coral-KV versus QVG-Pro latency overhead.Both execution orders favor Coral-KV, but two runs do not establish a statistically significant speedup; adaptation increases overhead on both models.Source: §F.3 · p. 24

G.1 Configuration selection

The configuration study selects predictor, residual-coding, and calibration choices, then compares reusable and video-adapted Coral-KV with QVG-Pro over extended rollouts. The reported gains persist through frame 191, while ablations characterize the quality, memory, and fitting trade-offs.

G.1 Configuration selection

  • The rank-128 offline predictor raises PSNR from 14.14 to 17.55 dB on HY-WorldPlay and from 25.19 to 31.99 dB on MAGI-1.At equal predictor-memory budgets, block-DPLR has positive paired PSNR intervals on both models; video-specific block-diagonal fitting reaches 17.89 and 33.85 dB at higher memory and fitting cost.Source: §G.1 · pp. 24–25, Table 8 · p. 25
  • BF16 layer-zero anchors add memory without a demonstrated quality advantage: PSNR changes by −0.31 dB on HY-WorldPlay and +0.09 dB on MAGI-1.BF16 raises cache memory from 4.85 to 5.18 bits per BF16 cache value on HY-WorldPlay and from 2.47 to 2.82 on MAGI-1; it worsens MAGI-1 LPIPS by 0.036.Source: §G.1 · p. 25, Figure 8 · p. 25

G.2 Residual transform and codebook

  • H64/EDEN-2 gives the highest PSNR on both models in the fixed-scale-group transform and assignment comparison; selected K FP8/64, V FP16/128 improves PSNR over reversed allocation by 0.21/0.67 dB.Sharing FP16 scales across 128 rather than 64 values saves only 2.1%/4.9% memory on HY-WorldPlay/MAGI-1 while lowering PSNR by 0.23/0.82 dB.Source: §G.2 · p. 25, Figure 9 · p. 26

G.3 Online centroids, offline codebooks, and reusable predictors

  • Against offline-trained codebooks, reusable Coral-KV raises PSNR by 0.86/1.67 dB on HY-WorldPlay and 0.36/3.73 dB on MAGI-1 across scalar INT2/H64-EDEN-2.It also reduces first-block base-reconstruction relative MSE by 36–53% versus online QVG-1 and 56–70% versus offline-trained codebooks; the MAGI-1 scalar-INT2 PSNR interval includes zero.Source: §G.3 · p. 26, Table 9 · p. 27

G.4 Calibration-set size

  • MAGI-1 improves across the calibration-size sweep, while HY-WorldPlay peaks at 16 prompts; the study uses 32 prompts for both models.The rank-128 predictors are evaluated on the same 32-prompt selection set in this separate calibration-size experiment.Source: §G.4 · p. 28, Table 10 · p. 28

H.1 Long-horizon robustness

  • Across MAGI-1 frames 25–191, video-adapted Coral-KV improves PSNR over QVG-Pro by 3.77 dB; its late-window gain is 3.36 dB.The reusable configuration’s corresponding gains are 2.39 dB over frames 25–191 and 1.32 dB in frames 144–191; both configurations retain advantages beyond the standard 96-frame horizon.Source: §H.1 · p. 28, Figure 10 · p. 28

H.2 Additional reconstruction examples

  • Long-range panels compare BF16, QVG-Pro, and video-adapted Coral-KV at late frames; first-boundary panels mark QVG’s largest local fidelity gaps at frames 29 and 24.The late-frame examples cover MAGI-1 and HY-WorldPlay; the first-boundary examples show frames 28–30 for HY-WorldPlay and 23–25 for MAGI-1.Source: Figure 11 · p. 29, §H.2 · p. 29, Figure 13 · p. 31
Loading 2610.12514v1…