PaperlayerView exact revision on arXiv

Source-linked AI summary

One Distribution, Two Channels: Coverage-Calibrated KV Retention and Compensation

Shuxin Liu, Qing Liu, Yi Du, Ou Wu

arXiv:2610.02235v2cs.ARcs.AI

TL;DR

Long-context decoding is constrained by KV-cache memory and bandwidth, and retention rankings do not specify discarded attention mass or the direction of output error. CORE distills query utility and log-determinant coverage into one allocation for retention and latent-memory writing. Across three backbones, CORE improves accuracy–KV-budget trade-offs, with a reported RULER gain of up to 3.78 points over the strongest baseline at 90% compression.

  • Problem

    Fixed-budget methods often separate retention from compensation, and a retention ranking alone specifies neither discarded attention mass nor the direction of induced output error.

    Source: §1 · pp. 1–2

  • Method

    CORE distills query utility and log-determinant coverage into one lightweight cache-aware allocation that guides Top-B retention and parameterizes latent-memory writes.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    At 90% compression, CORE improves over the strongest RULER baseline by up to 3.78 points across three backbones.

    Source: §5.2 · p. 7

  • Takeaways & Limitations

    Across long-context and repeated-eviction settings, CORE improves accuracy–KV-budget trade-offs.

    Source: §6 · pp. 9–10

    Because compression may discard information needed by later queries, deployment should evaluate budgets on representative workloads and monitor output quality, especially under repeated eviction and changing query demands.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Long-context decoding is increasingly constrained by key--value (KV) cache memory and bandwidth. Existing fixed-budget compression methods typically separate retention from compensation, while a retention ranking specifies neither discarded attention mass nor the direction of induced output error. We start from an exact factorization: eviction error equals evicted attention mass times the directional gap between the evicted centroid and retained output, highlighting the importance of set-level coverage in retention and mass-preserving memory writing. We introduce CORE COverage Calibration and Evicted-Mass REdistribution for KV Cache, which distills an offline allocation combining query utility and log-determinant coverage into a lightweight cache-aware indexer. At inference, one calibrated distribution drives both channels: its Top-$B$ ordering retains complementary KV states, while its excluded allocation mass and conditional weights parameterize latent-memory writes without a separate write-weight predictor or online log-determinant evaluation. Our analysis provides a four-term pre-compensation error certificate, characterizes non-additive coverage interactions, and establishes mass-independent write stability with hierarchical bounds through recurrent and query-adaptive normalization. Across three backbones, CORE exceeds the strongest RULER baseline by up to 3.78 points at 90% compression; LongBench and repeated-eviction evaluations further demonstrate strong effectiveness and decoding efficiency.

1 INTRODUCTION

Long-context decoding makes KV-cache memory and bandwidth a bottleneck, while fixed-budget eviction separates retention from compensation. CORE uses one coverage-calibrated allocation to guide retention and represent evicted information.

1 INTRODUCTION

  • Longer contexts make the KV cache a primary GPU-memory and bandwidth bottleneck, motivating fixed-budget eviction and latent-memory compensation.Existing approaches typically treat retention and compensation separately, leaving open which states to retain and how to represent evicted information.Source: §1 · p. 1
  • A ranking orders candidate states but does not quantify discarded attention mass or the direction of induced output deviation.High individual scores can still select redundant states, so fixed-budget retention is a set-level coverage problem.Source: §1 · pp. 1–2
  • Eviction error factors into evicted attention mass and the directional gap between the evicted centroid and retained output.This identity motivates jointly reducing both factors through retention and representing both through compensation.Source: §1 · p. 1
  • CORE distills query utility and log-determinant coverage into a lightweight cache-aware indexer whose allocation drives Top-B retention and latent-memory writes.The excluded allocation mass and conditional weights parameterize writes, without a separate compensation weighting rule or online matrix evaluation.Source: §1 · p. 2
  • CORE’s contributions include a four-term eviction certificate, non-additive coverage analysis, and mass-preserving writes with recurrent and query-adaptive stability bounds.The paper also reports gains of up to 3.78 points at 90% compression across long-context benchmarks and three backbones.Source: §1 · p. 2

2 RELATED WORK

Prior KV-management work uses diverse retention signals, coverage strategies, compensation mechanisms, and fixed-budget formulations. CORE focuses on distilling cache-conditioned coverage into an allocation that also determines evicted-state writing weights.

2 RELATED WORK

  • CORE distills cache-conditioned coverage into one allocation that determines retention and evicted-state writing weights.Related approaches include importance, recency, future-aware and context-conditioned signals, adaptive policies, retrieval, quantization, learned retainers, and memory-backed compensation.Source: §2 · pp. 2–3
  • Recent KV-retention work augments token utility with diversity and redundancy reduction to model interactions among retained states.CORE’s related-work framing places its cache-conditioned coverage allocation among these approaches.Source: §2 · p. 2
  • Log-determinant objectives preserve set-level coverage but require marginal-gain evaluation, while lightweight proxies trade fidelity for efficiency.Information-theoretic formulations preserve cache capacity under fixed budgets; evaluation audits emphasize matched query visibility and physically enforced memory budgets.Source: Figure 2 · p. 3
  • Memory-backed methods, residual caches, and merging evicted values address compensation, while MOMENTKV targets directional mismatch and IndexMem separately parameterizes retention and latent memory.CORE instead couples retention and compensation through one allocation distribution.Source: §2 · p. 3

3 METHOD

CORE builds one allocation that couples coverage-aware retention with evicted-mass compensation. It distills an offline coverage teacher into an indexer, then uses the allocation’s retained and excluded portions for cache selection and memory writing.

3 METHOD

  • CORE’s shared allocation makes retention set-dependent, uses excluded allocation for write mass and composition, and motivates gated correction from scalar–direction residuals.The method distills offline log-determinant coverage into a lightweight allocation whose Top-B and excluded restrictions drive retention and memory writing.Source: §3 · p. 3

3.1 COVERAGE-CALIBRATED SOFT ALLOCATION AND RETENTION DISTILLATION

  • CORE sorts the learned allocation to select the retained set, using query-dependent utility from the frozen backbone and a compact key-space representation for candidate interactions.Boundary supervision penalizes local retain–evict inversions around the Top-B cutoff, complementing KL matching of teacher and student allocations.Source: §3.1 · pp. 3–4
  • The coverage-calibrated teacher combines query utility with cache-centered representations to estimate query relevance and set-level directional coverage.A cache-conditioned coverage marginal discounts redundancy and favors under-represented directions; the lightweight indexer distills this structure using query-relevance, cache-geometry, and deployment-state features.Source: §3.1 · p. 4

3.2 EVICTED-MASS REDISTRIBUTION

  • The allocation restricted to evicted states supplies their total write mass and conditional composition, preserving excluded allocation in memory writes.A recurrent accumulator weights each evicted summary by excluded mass, discounts prior events, and retains normalized composition; a query-dependent readout and learned scalar gate reconstruct the residual.Source: §3.2 · p. 5

3.3 TRAINING, INFERENCE, AND COMPLEXITY

  • CORE trains first by distilling the allocation, then jointly trains retention and compensation to reconstruct attention residuals induced by the retention policy.Deployment uses only causally available queries; inference predicts the allocation, selects Top-B states, updates evicted-mass memory, and applies compensated attention.Source: §3.3 · p. 5
  • Before Top-B selection, scoring and memory updates are linear in the number of candidate states, while streaming query–key statistics scale with query count and candidate count.Teacher computation uses repeated matrix operations, with total cost O(R(Nd^2_z)) over R iterations.Source: §3.3 · p. 5

4 THEORETICAL ANALYSIS

The analysis characterizes eviction error through mass and direction, then bounds pre-compensation error and write sensitivity. Its certificates separate allocation, coverage, utility, and excluded-mass effects under stated assumptions.

4 THEORETICAL ANALYSIS

  • Every normalized aggregation readout admits an exact decomposition into evicted mass and the directional difference between evicted and retained readouts.Under condition (A3), the converse holds, characterizing this readout class; the result applies beyond softmax attention.Source: §4 · p. 6
  • Under the stated bounded-value and positive-temperature assumptions, the certificate is formulated for a specified query distribution and deployed eviction set.The setup defines allocation mismatch and excluded mass as part of the bound’s scope.Source: §4 · p. 6
  • The expected pre-compensation eviction certificate separates excluded mass, allocation mismatch, coverage-induced shift, and utility mismatch.Its directional term uses 2Bv; unknown mean attention prevents direct online evaluation, and directional benefits are assessed empirically.Source: §4 · p. 6
  • Mass-preserving writes bound write differences by Cη(p, q), avoiding the inverse-mass amplification of conditional-only writes.The write-stability proposition requires positive minimum evicted mass.Source: §4 · p. 6

5 EXPERIMENTS

CORE is evaluated across three backbones on long-context benchmarks, decode-time compression, and retrieval, followed by analyses of its retention, compensation, components, and efficiency. It leads RULER under aggressive compression and shows gains across additional evaluations.

5.1 EXPERIMENTAL SETUP

  • Evaluations use Mistral-7B, Llama-3.1-8B, and Qwen3-14B on RULER and LongBench, with comparisons against eviction baselines under unified settings.CORE is trained on LongAlpaca-12k with a frozen backbone in two stages.Source: §5.1 · pp. 6–7

5.2 MAIN RESULTS

  • At 90% compression, CORE improves over the strongest RULER baseline by 2.25 points on average, peaking at 3.78 points on Qwen3-14B at 16K.It leads all 12 settings at 75% and 90% compression across three backbones and both context lengths.Source: §5.2 · p. 7, Table 1 · p. 7
  • At 75% compression, CORE’s LongBench 16-task average is highest among compressed methods, 2.30 points above the strongest baseline.Figure 3 covers six representative tasks and four compression ratios; CORE maintains balanced performance with clearer advantages under aggressive compression.Source: §5.2 · pp. 7–8, Figure 3 · p. 8

5.3 COMPRESSION ROBUSTNESS

  • CORE maintains high needle-retrieval accuracy across most length–depth pairs and leads decode-time compression at every interval, reaching 53.33% and 52.60% accuracy at interval 512.The interval-512 values correspond respectively to AIME25 and Math500; the schedules vary eviction frequency and intermediate cache occupancy.Source: §5.3 · p. 8, Table 2 · p. 8

5.4 RETENTION AND COMPENSATION ANALYSIS

  • Coverage supervision reduces evicted mass and directional factors; distillation lowers allocation discrepancies, while mass-preserving write sensitivity stays near 0.4 across excluded-mass bins.At high compression, mass becomes similar to the no-coverage selector while CORE retains a smaller directional factor.Source: §5.4 · pp. 8–9

5.5 ABLATIONS, TRAINABLE BASELINES, AND EFFICIENCY

  • Removing coverage supervision, coverage cues, or boundary loss reduces performance, while memory removal causes the largest drops at 75% and 90% compression.The memory-removal drops are 2.19 and 4.16 points, respectively.Source: §5.5 · p. 9, Figure 6 · p. 9
  • At 16K and 90% compression, CORE exceeds IndexMem by 2.60 points on Mistral-7B and 10.53 on Llama-3.1-8B, outperforming trainable eviction baselines across tested settings.The comparison includes Locret, TRIM-KV, KVP, and IndexMem.Source: §5.5 · p. 9, Figure 6 · p. 9
  • At 75% compression on Qwen3-14B, CORE leads RULER-4K at approximately 34 ms/token; shared allocation also outperforms COREind by 0.62–1.96 points across tested contexts and compression ratios.Decode latency is token-averaged suite-wide.Source: §5.5 · p. 9, Figure 6 · p. 9

6 CONCLUSION

CORE couples KV retention and latent-memory compensation through one allocation grounded in an exact factorization of eviction error. The paper reports improved accuracy–KV-budget trade-offs across long-context and repeated-eviction settings.

6 CONCLUSION

  • CORE uses an exact eviction-error factorization to connect discarded attention mass and directional gap with both retention and compensation.A lightweight indexer distills offline log-determinant coverage; its allocation drives Top-B retention and latent-memory writes through excluded mass and conditional weights.Source: §6 · pp. 9–10
  • CORE’s theory provides a four-term pre-compensation certificate, mass-independent write stability, and hierarchical bounds through recurrent and query-adaptive normalization.Source: §6 · p. 10
  • Across long-context and repeated-eviction settings, CORE improves accuracy–KV-budget trade-offs.Source: §6 · p. 10

AI USE STATEMENT

The authors report using generative AI tools to support manuscript preparation and presentation. The stated uses included language polishing and limited assistance with technical wording and checking.

AI USE STATEMENT

  • Generative AI tools were used primarily to polish language and improve manuscript clarity, fluency, and readability.Source: AI USE STATEMENT · p. 10
  • The tools also provided minor assistance with translation, wording refinement, terminology consistency, and organization of technical descriptions.Source: AI USE STATEMENT · p. 10
  • The tools occasionally assisted with checking mathematical statements and proof arguments and clarifying experimental and technical presentation.The authors characterize these uses as limited to manuscript preparation and presentation.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The work evaluates inference efficiency using existing pretrained models and public datasets and benchmarks. The authors note that compression can discard information needed by later queries, motivating workload-specific evaluation and quality monitoring.

ETHICS STATEMENT

  • The study uses existing pretrained models and publicly available datasets and benchmarks to examine inference efficiency.Source: ETHICS STATEMENT · p. 10
  • KV-cache compression may discard information needed by later queries, potentially causing retrieval failures or incorrect outputs.Source: ETHICS STATEMENT · p. 10
  • The authors recommend evaluating compression budgets on representative workloads and monitoring output quality, especially under repeated eviction and changing query demands.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The appendices document CORE’s method, theory, implementation, and evaluation, with additional results and limitations. They detail how coverage-informed allocations are distilled into a cache-aware indexer and relate its ranking to fixed-budget selection.

REPRODUCIBILITY STATEMENT

  • The reproducibility materials document models, benchmarks, baselines, hardware, cache budgets, implementation, training, evaluation, and hyperparameter selection.They also specify random seeds, generation and scoring procedures, and per-task, sensitivity, latency, and memory results.Source: REPRODUCIBILITY STATEMENT · p. 10

Organization of the Appendix

  • The appendices cover related-method positioning, coverage-teacher and indexer details, theory, diagnostics, protocols, additional experiments, and limitations.Source: Organization of the Appendix · p. 16

A.1 POSITIONING RELATIVE TO CLOSELY RELATED KV-CACHE METHODS

  • CORE shares allocation weights across retention and writes: Top-B determines retained support, while excluded allocation weights determine evicted-state writes without a separate write-weight predictor.Source: §A.1 · p. 17

A.2 COVERAGE TEACHER AND FIXED-BUDGET APPROXIMATION

  • The coverage teacher combines query utility with cache-centered geometry; for equal vector norms, less-represented directions receive larger coverage marginal scores.Its objective has a unique interior maximizer under the stated conditions α > 0, β ≥ 0, and τent > 0.Source: §A.2.1 · p. 17, §A.2 · p. 17

A.2.1 COVERAGE MARGINALS AND TEACHER OPTIMALITY

  • Increasing the coverage weight makes the exact optimum’s log-determinant coverage nondecreasing, without implying a monotonic reduction in output-space directional factor DE.Source: §A.2.1 · p. 17

A.2.2 FINITE-STEP DISTILLATION TARGET

  • A finite-step distillation target has a certificate: a cutoff gap exceeding the L1 error bound prevents retained–excluded rank reversals.The finite-step numerical check reports small residuals and stable selections at R0 = 8, while overlap with the R = 16 reference alone does not certify exact Top-B recovery.Source: §A.2.2 · p. 18, Table 5 · p. 18

A.2.3 CONNECTION TO FIXED-BUDGET SELECTION

  • The finite teacher ranking uses shared cache-conditioned coverage marginals as a surrogate and need not maximize the non-additive fixed-budget set objective.Source: §A.2.3 · p. 18

A.3 CACHE-AWARE INDEXER SPECIFICATION

  • The indexer uses query–key, coverage, redundancy, and auxiliary cache-state features; its coverage cues are learned rather than an analytic reconstruction of the teacher marginal.Boundary supervision compares candidates around the teacher cutoff, but local ordering does not guarantee complete Top-B agreement; the appendix also proves the normalized aggregation identity’s factorization.Source: §A.3 · p. 18, §A.3.2 · p. 19, §A.4 · p. 20

B.1.2 NECESSITY: PROOF OF THEOREM 1

This section examines the conditions and stability limits underlying eviction-error bounds and memory writes, alongside empirical tests of retention quality, allocation fidelity, and the eviction certificate. Under shared selection constraints, CORE’s student selectors stay close to the greedy set-quality reference and have lower allocation errors than the compared students at every tested budget.

B.1.2 NECESSITY: PROOF OF THEOREM 1; B.1.3 BOUNDARY CASES

  • Under non-degeneracy, subset readouts have a positive-weight representation unique up to common scale; the converse yields canonical factors, while signed weights and non-subset compression change or escape the characterization.The converse identifies the normalized representation, not every auxiliary g; g(E) may remain undetermined when |I \ E| = 1. Unnormalized additive readouts require no subset renormalization.Source: §B.1.2 · pp. 20–21, §B.1.3 · p. 21

B.2.1 PER-QUERY IDENTITY AND BOUND; B.2.2 EXPECTED EVICTION CERTIFICATE

  • For softmax attention, eviction error equals evicted attention mass times a directional gap and is bounded by 2BvπE(q); the expected certificate uses excluded allocation mass, with query-aggregation and head mismatch potentially large.The measured direction factor is at most 2Bv; empirical averages over a finite query collection require no independence assumption but do not establish generalization to unseen queries.Source: §B.2.1 · p. 21, §B.2.2 · p. 22

B.2.3 DECODE-TIME, REPEATED-EVICTION, AND MULTI-HEAD EXTENSIONS

  • Current-query logits and values determine eviction mass and error exactly, while successive per-query eviction changes accumulate additively and head-specific bounds sum through output projection.Exact evaluation applies only to queries present at eviction and incurs computation; Xie & Alanwar (2026) preclude a consistent deterministic estimate of a scheme’s own unseen-query error from retained information alone.Source: §B.2.3 · p. 22

B.3.1 ALLOCATION DECOMPOSITION

  • Top-B is the exact B-support projection minimizing residual allocation ℓ1 mass, and excluded allocation mass with conditional weights supplies mass-preserving memory writes.The write bound is tight for bounded linear writes; with protected entries fixed, it applies to eligible candidates under the remaining budget, but does not guarantee realized post-compensation error.Source: §B.3.1 · p. 23

B.3.2 JOINT RETENTION AND EXCLUDED-MASS FIDELITY

  • Full-allocation KL controls retention regret and excluded-mass discrepancy; a sufficient cutoff-margin condition recovers the teacher Top-B set, while local boundary supervision alone does not guarantee full agreement.With protected sinks, conditional training KL bounds retention on non-sink candidates; vector-write changes also depend on the evicted-set difference and vanish when the excluded sets match.Source: §B.3.2 · pp. 23–24

B.4 COVERAGE INTERACTION AND SET AWARENESS

  • Log-determinant coverage has diminishing returns, with squared inverse-covariance overlap measuring local interactions; weak overlap yields near-additive gains, not an exact fixed-budget optimizer.Prototype distance, assignment margin, and occupancy provide compact cues for residual direction, assignment robustness, and local directional crowding.Source: §B.4 · p. 24

B.5.1 PROOF OF PROPOSITION 1; B.5.2 HIERARCHICAL STABILITY OF MASS-PRESERVING MEMORY

  • Allocation-preserving linear writes avoid inverse-mass amplification, whereas conditional normalization incurs a 1/µmin factor that is unavoidable in general; matching excluded sets removes the membership term.Through recurrent memory and query-adaptive readout, stability separates allocation perturbation, accumulated-mass normalization, and readout normalization; the scalar gate is non-expansive.Source: §B.5.1 · p. 25, §B.5.2 · pp. 25–26

C THEORY-ALIGNED EMPIRICAL DIAGNOSTICS FOR SEC. 5.4

  • The theory-aligned diagnostics use 104 RULER-16K sequences from Llama-3.1-8B, four layers, four compression rates, and up to 32 subsequent queries, with learned parameters frozen.Comparisons share reference QKV states, query visibility, protected positions, and tie-breaking; attention metrics are averaged across heads and layers, then sequences and tasks.Source: C THEORY-ALIGNED EMPIRICAL DIAGNOSTICS… · p. 26

C.1 RETENTION AND ALLOCATION FIDELITY

  • With memory correction disabled, CORE is compared with the β = 0 selector using empirical eviction-mass and directional-factor estimates on shared reference states.Fig. 5(a) reports relative reductions over β = 0 in the mean product of those factors; normalized errors are computed per state before averaging.Source: §C.1 · p. 26
  • The certificate audit’s mean distillation, coverage-shift, and utility-mismatch contributions are 0.014, 0.078, and 0.158, summing to 0.250.The gap between attention mass and RetErr reflects the conservative 2Bv directional envelope.Source: §C.1 · p. 27, Table 6 · p. 27
  • At 75%/90% compression, Student Top-B gaps to greedy are 0.96%/1.39%, Teacher Top-B gaps are 0.85%/1.02%, and utility-only gaps are 5.90%/7.02%.The comparison uses shared fixed-budget selection constraints and the utility–coverage objective.Source: §C.1 · p. 27, Table 7 · p. 27
  • CORE has lower RegretB and MassErrB than the no-coverage-cues and KL-only students at every tested budget.RegretB measures lost teacher-assigned retained mass; MassErrB measures excluded-mass discrepancy.Source: §C.1 · p. 27

C.2 MEMORY RECONSTRUCTION AND STABILITY

Mass-preserving writes improve compensation over repeated events, while their sensitivity to allocation perturbations remains approximately constant as excluded mass changes. The section evaluates these behaviors using matched replay variants and fixed-query perturbations.

C.2 MEMORY RECONSTRUCTION AND STABILITY

  • Mass-preserving sensitivity stays approximately constant across excluded-mass bins, whereas conditional-only sensitivity increases as excluded mass decreases.Records are grouped by excluded mass; the plot reports median and 90th-percentile sensitivity.Source: §C.2 · p. 27
  • The reconstruction comparison uses shared eviction events, memory capacity, decay, and stabilizers, with mass-preserving, equal-event, and uniform-token writing variants.Metrics are computed per sequence–layer group from concatenated residual and correction vectors over evaluated queries.Source: §C.2 · p. 27
  • 50.38% compensation gain after four events is 16.00 points above equal-event and 14.65 points above uniform-token writing.Mass-weighted writing also has higher later alignment than equal-event writing at similar correction magnitude; event mass controls relative weighting across subsequent writes.Source: §C.2 · p. 28, Figure 7 · p. 28
  • At δ = 0.05, fourth-event gains fall by 0.44–0.66 points, while mass-weighted writing remains highest after perturbation.The perturbation is applied at every write event with eviction sets fixed and noise draws shared across variants.Source: §C.2 · p. 28, Table 8 · p. 28

D.1 IMPLEMENTATION DETAILS

The implementation evaluates CORE with frozen backbones, trained indexer and compensation modules, and shared benchmark procedures. Training, evaluation, and hyperparameter selection use specified data, budgets, and protocols.

D.1 IMPLEMENTATION DETAILS

  • CORE is evaluated on Mistral-7B-v0.2, Llama-3.1-8B-Instruct, and Qwen3-14B, with only the indexer and compensation modules optimized.All three backbones remain frozen during training.Source: §D.1 · p. 28

D.2 TRAINING DETAILS

  • Training uses LongAlpaca-12k for 5,400 updates across indexer distillation and joint training, with maximum sequence length increasing from 2,048 to 4,096 tokens.Memory reconstruction uses up to 16 trailing response queries or a held-out prompt suffix, and gradients do not pass through hard selection or into indexer scores.Source: §D.2 · p. 29

D.3 EVALUATION PROTOCOL

  • CORE and baselines follow the same KVPress benchmark pipeline, while RULER and LongBench scores are unweighted means over 13 and 16 tasks, respectively.AIME25 and Math500 use all 30 and 500 examples with boxed-answer extraction and exact string matching.Source: §D.3 · p. 29

D.4 DEVELOPMENT SET AND HYPERPARAMETER SELECTION

  • Hyperparameters are selected on 320 held-out development examples using Llama-3.1-8B-Instruct on LongBench at 75% compression, then held fixed across evaluations.Each method tests at most ten candidate configurations; sensitivity analyses are conducted only after selection and do not guide revisions.Source: §D.4 · p. 29

E.1 PER-TASK RESULTS ON LONGBENCH AND RULER AT 75% COMPRESSION

CORE leads the compressed-method averages in the reported LongBench and RULER-16K comparisons, with further evaluations covering ultra-long contexts, sensitivity, allocation variants, training sources and seeds, efficiency, and limitations.

E.1 PER-TASK RESULTS ON LONGBENCH AND RULER AT 75% COMPRESSION

  • CORE exceeds ExpectedAttention by 2.30 points on LongBench’s 16-task average and narrowly leads KeyDiff on the RULER-16K average.The task-level results show gains spanning multiple QA and summarization tasks, alongside complementary strengths across retrieval configurations.Source: §E.1 · p. 30, Table 10 · p. 30, Table 11 · p. 30

E.2 ULTRA-LONG-CONTEXT EVALUATION

  • CORE exceeds IndexMem by 4.43, 4.93, and 4.43 percentage points across three LongBenchv2 budgets on selected 32,768–131,056-token prompts.The evaluation uses 203 examples on Llama-3.1-8B with chunked prefill and a 16-token generation limit.Source: §E.2 · p. 30

E.3 HYPERPARAMETER SENSITIVITY

  • Removing coverage or boundary supervision causes clear degradation; overly strong coverage weighting can also hurt, while tested entropy and allocation-temperature variations change performance less.The default configuration lies within the high-performing region of the evaluated settings.Source: §E.3 · pp. 30–31, Figure 8 · p. 31

E.4 SHARED AND ALTERNATIVE ALLOCATION STRATEGIES

  • CORE outperforms its independently learned write-allocation variant across evaluated settings and leads all four alternative-allocation settings.The alternative distributions derive from ExpectedAttention or KeyDiff scores; the gap over the ExpectedAttention replacement is particularly pronounced at 90% compression.Source: §E.4 · p. 31, Table 13 · p. 31

E.5 SENSITIVITY TO TRAINING DATA AND RANDOMNESS

  • Across four RULER settings and the 16-task LongBench average, training-source deviations are at most 0.94 points; LongAlpaca-12k leads three RULER settings and LongBench, while SWE-smith improves the code average to 50.12 versus 49.42.Across eight seed-sensitive settings, CORE leads the strongest baseline by 2.54–11.54 points, with standard deviations from 0.04 to 0.27 points.Source: §E.5 · pp. 31–32, Table 14 · p. 31, Table 15 · p. 32

E.6 FULL EFFICIENCY SCALING AND PARAMETER COST

  • At 64K, CORE latency is 41.41, 37.29, and 35.22 ms/token at 50%, 75%, and 90% compression, respectively, lower than CAPKV and IndexMem at each setting.At 64K and 75% compression, dynamic memory is 6.57 GiB for CORE versus 6.65 GiB for IndexMem; CORE's auxiliary state is 0.22 GiB and online overhead is 7.7%.Source: §E.6 · pp. 32–33, Table 16 · p. 32, Table 18 · p. 32
  • CORE's additional prefill overhead is concentrated in prompt processing and cache preparation; vocabulary projection and greedy token selection add 4–5 ms once the cache is ready.The measurements compare methods on Qwen3-14B with identical 8K–64K inputs and a shared attention backend.Source: §E.6 · p. 33, Table 19 · p. 33

F LIMITATIONS

  • Stability analysis is limited to a fixed query, the largest tested backbone has 14B parameters, and efficiency measurements cover specific contexts, hardware, and recompression schedules.The paper identifies full autoregressive stability analysis and evaluation of larger models under broader inference settings as future work.Source: F LIMITATIONS · p. 33
Loading 2610.02235v2…