PaperlayerView exact revision on arXiv

Source-linked AI summary

Monte Carlo Estimation for KV Cache Eviction

Ahsan Bilal, Muhammad Ahmed Mohsin, Muhammad Umer, Wajih Hassan Raza, Atta Ul Asad, Young D. Kwon, Michal Valko, Dean F. Hougen

arXiv:2610.07643v1cs.CLcs.AI

TL;DR

KV-cache eviction must select tokens before future decoding queries are known, and existing future-aware methods rely on pseudo-responses or synthetic estimates. LORE-KV samples multiple continuations from the frozen model and scores token utility across their query states. It wins 23 of 24 RULER settings, with its advantage largest under aggressive compression and longer contexts.

  • Problem

    Eviction precedes future decoding queries, leaving unclear how well prompt-side scores represent generation needs and what multiple plausible futures add.

    Source: §1 · p. 1

  • Method

    LORE-KV samples model-conditional autoregressive trajectories and estimates fixed-budget token utility from reliability-weighted projected deletion costs.

    Source: §1 · pp. 1–2

  • Results

    LORE-KV /E wins 23 of 24 RULER settings; on Mistral at B = 128, its gain over AnDPro grows to 5.85 points at 16K.

    Source: §4.3 · p. 8

  • Takeaways & Limitations

    Cache retention can be framed around tokens' likely usefulness during generation rather than only their importance during the prompt.

    Source: §5 · p. 9

    LORE-KV incurs one-time lookahead overhead, makes irreversible eviction decisions, and does not explicitly model cross-head or cross-layer interactions.

    Source: §5 · p. 9

Abstract

from arXiv · show

Most KV-cache eviction methods ask, in effect, which memory appeared important while reading the prompt? We instead ask, which memory will matter while answering? Since decoding queries are unavailable at eviction time, prior future-aware methods rely on pseudo-responses or synthetic future-query estimates. We cast fixed-budget future-aware eviction as distributional estimation over plausible model-conditional query trajectories and introduce LORE-KV (Lookahead Output-perturbation with Reliability-weighted Ensembles for Key-Value caches), a training-free method that samples short autoregressive continuations from the frozen target model and uses their response-side query states to estimate prompt-token utility. Tokens are scored by projected leave-one-out attention-output deletion cost and aggregated across sampled futures with optional trajectory weighting. The temporary continuations are discarded before final decoding, requiring no auxiliary model or training. Ablations isolate the mechanism: at B=128, a single response-side continuation recovers about 89% of the gain over the prompt-window control, while additional futures provide smaller improvements. At B=128, LORE-KV raises the LongBench average on Qwen2.5-14B from 45.49 to 48.24 (+2.75) and the 16K RULER average on Mistral-7B from 45.20 to 51.05 (+5.85). Gains diminish at larger cache budgets and coexist with task-level regressions. LORE-KV incurs 1.46-2.77x AnDPro's per-sample wall-clock time as a one-time compression overhead across six dense and hybrid-attention backbones.

1 INTRODUCTION

Prompt-side scores can miss tokens needed during generation because eviction precedes future queries, while existing future-aware approaches leave the value and cost of multiple futures unclear. LORE-KV estimates future token utility from sampled model-conditional trajectories and projected deletion costs.

1 INTRODUCTION

  • Future decoding queries are unavailable when eviction occurs, so prompt-based scores may retain tokens important during prefill but not generation.Earlier future-aware methods use pseudo or synthetic queries, while learned predictors require model-specific training.Source: §1 · p. 1
  • LORE-KV is training-free and estimates token importance using queries the model is likely to encounter during generation.It samples multiple autoregressive trajectories from the frozen target model and estimates fixed-budget utility through reliability-weighted projected deletion costs.Source: §1 · pp. 1–2
  • The paper characterizes prompt-to-future query mismatch, studies multiple short trajectories and reliability weighting, and evaluates across six backbones, budgets, contexts, and benchmarks.Its validation includes query controls, matched lookahead-token budgets, and future-query oracle analyses.Source: §1 · p. 2

2 RELATED WORK

Prior eviction work uses prompt-side information, while future-query estimation and output-aware scoring address distinct parts of the eviction problem; other cache-compression approaches operate along orthogonal dimensions.

2.1 PROMPT-SIDE KV EVICTION

  • Prompt-side eviction methods use recency, attention sinks, accumulated attention, prompt-end attention, or key-space geometry to estimate token importance.These approaches estimate importance from prefill information rather than plausible future decoding queries.Source: §2.1 · p. 2

2.2 FUTURE-QUERY ESTIMATION AND OUTPUT-AWARE SCORING

  • Future-query methods approximate decoding queries, while output-aware methods measure attention-output perturbation or incorporate projected values and inter-head interactions.LORE-KV combines multiple model-conditional autoregressive trajectories with reliability-weighted projected deletion cost under a fixed cache budget.Source: §2.2 · p. 2

2.3 ORTHOGONAL KV-CACHE COMPRESSION

  • Figure 1 compares lookahead-query coverage of future decoding queries and reports that coverage saturates near M = 4.LORE-KV's prompt-time token-selection criterion is distinct from methods that alter cache precision, representation, serving, or decode-time maintenance.Source: Figure 1 · p. 3

3 METHODOLOGY

LORE-KV estimates future query states from short sampled continuations, weights trajectories by reliability, and scores candidate tokens by projected deletion impact before selecting a fixed-budget cache.

3 METHODOLOGY

  • LORE-KV has three stages: Monte Carlo future-query estimation, reliability weighting, and projected counterfactual eviction scoring.The method is training-free and estimates token utility across sampled future query trajectories.Source: §3 · p. 3

3.1 PROBLEM SETUP

  • The setup forms cached keys and values during prefill, protects the first token and prompt-end window, and leaves other positions as eviction candidates.Structural tokens can optionally join the protected set.Source: §3.1 · p. 3

3.2 TEST-TIME LOOKAHEAD QUERY CONSTRUCTION

  • LORE-KV samples short autoregressive continuations from the frozen model to provide response-side query trajectories for estimating future token utility.The trajectories approximate future query states and improve coverage over prompt-only queries; additional lookahead compute eventually saturates in reported tables.Source: §3.2 · p. 3

3.3 PARTICLE RELIABILITY WEIGHTING: ENTROPY AND TRAJECTORY DISAGREEMENT

  • Trajectory reliability combines atypicality, uncertainty, and disagreement signals into soft weights, giving more reliable futures greater influence.LORE-KV /E uses entropy-based uncertainty, while LORE-KV /D additionally uses query-space disagreement.Source: §3.3 · p. 4

3.4 CLOSED-FORM COUNTERFACTUAL DELETION EFFECT

  • The projected deletion score measures residual-stream impact, capturing both a token's attention and how its value differs from the current attention output.This makes attention alone insufficient as an importance measure.Source: §3.4 · p. 4

3.5 CORE SCORE, SELECTION, AND COMPRESSED CACHE

  • Per-query perturbations are aggregated across trajectories into token scores; the core method uniformly allocates each layer's budget across heads and selects top-scoring tokens per head.At B = 128, about 20% of retained tokens differ from the prompt-only rule; lookahead continuations are discarded before final decoding.Source: §3.5 · pp. 4–5

4 EXPERIMENTS

LORE-KV is evaluated on LongBench, RULER, and Needle-in-a-Haystack across dense and hybrid-attention backbones under matched cache budgets. Its largest gains occur under aggressive compression, with smaller margins as cache budgets grow.

4.1 EVALUATION ON LONGBENCH

  • At B = 128 on Mistral, LORE-KV /E scores 40.28 versus AnDPro’s 38.96; Figure 2 reports 1.50–1.63× greater cache efficiency at equal LongBench accuracy.Across three seeds, the LongBench mean is 40.28 (95% CI [39.97, 40.59]).Source: Figure 2 · p. 5, §4.1 · p. 5, Table 1 · p. 6
  • At B = 128, a single response-side rollout raises the score from 35.87 to 39.81, and multiple trajectories add further gains; at M = 4, extending T from 2 to 16 raises it from 38.96 to 40.34.Deletion scoring and reliability weighting provide only minor refinements.Source: §4.1 · p. 6
  • At B = 128, gains over the strongest baseline are 2.75 points on Qwen2.5 and 0.56 on Ministral-3, while LORE-KV also leads across budgets on hybrid-attention backbones.Margins become very small by B = 1024; hybrid-backbone eviction is applied only to KV-growing layers.Source: §4.1 · pp. 6–7, Table 2 · p. 7

4.3 EVALUATION ON RULER: SWEEPING CONTEXT LENGTH ACROSS FIXED TASKS

  • On RULER, LORE-KV /E wins 23 of 24 settings; at B = 128 on Mistral, its margin over AnDPro grows from 3.55 points at 4K to 5.85 at 16K.The advantage becomes small at B = 1024; multiple futures outperform a single rollout, while /D closely tracks /E.Source: Figure 3 · p. 7, §4.3 · p. 8

4.2 EVALUATION ON NEEDLE-IN-A-HAYSTACK (NIAH)

  • At B = 1024 on Llama-3.1-8B-Instruct, LORE-KV /E and /D tie at 0.977 exact match, above CriticalKV (0.955) and AnDPro (0.932).The gains come from retrieving boundary-depth needles at 16K that baselines miss; LORE-KV also retains answer-bearing tokens more often.Source: §4.2 · p. 7, Figure 4 · p. 8

4.3 EVALUATION ON RULER: SWEEPING CONTEXT LENGTH ACROSS FIXED TASKS

  • In fixed 4K RULER comparisons, LORE-KV /E achieves higher quality at lower retention, outperforming OBCache at 12.5% versus 20% retention and Expected Attention at the same 1,024-token budget.The Expected Attention comparison is contextual because it is query-agnostic by design.Source: §4.3 · p. 8

4.4 MEMORY AND LATENCY

  • At fixed cache budget, all methods have the same steady-state decoding memory and per-token cost; LORE-KV adds one-time lookahead and scoring overhead.It remains above the strongest baseline at every matched lookahead-token budget.Source: §4.4 · p. 8

4.5 MECHANISM ANALYSIS: LOCATING THE GAIN

  • Output fidelity rises from 0.873 to 0.905 on LongBench and from 0.929 to 0.944 on RULER; gains positively correlate with prompt-to-future query mismatch.The reported Spearman correlations for oracle-retention recall are 0.28/0.57 and for output fidelity are 0.53/0.51 on LongBench/RULER.Source: Figure 5 · p. 9, §4.5 · p. 9, Figure 6 · p. 9

5 CONCLUSION

LORE-KV frames cache eviction as predicting which tokens will matter during generation, while acknowledging one-time lookahead cost, irreversible eviction, and unmodeled cross-head or cross-layer interactions. Its appendices connect this objective to prompt-side and future-aware methods, the cache-distortion objective, and implementation choices.

5 CONCLUSION

  • LORE-KV retains tokens by likely usefulness during generation, but incurs one-time lookahead overhead, makes irreversible decisions, and does not model cross-head or cross-layer interactions.The conclusion motivates adaptive lookahead, recoverable or dynamically updated caches, and globally coordinated retention as future directions.Source: §5 · p. 9

A.1 PROMPT-SIDE EVICTION AND BUDGET ALLOCATION

  • Prompt-side methods use signals available during prefill, including attention sinks, accumulated attention, prompt-end observations, and prompt-side budget allocation.The passage groups StreamingLLM, H2O, SnapKV, PyramidKV, Ada-KV, and DefensiveKV by these roles.Source: §A.1 · p. 18

A.2 FUTURE-AWARE AND OUTPUT-AWARE EVICTION

  • Future-aware and output-aware methods motivate two LORE-KV axes: which query states assess importance and how deletion distorts attention outputs.The passage says related methods generally address one axis independently or use learned, analytical, or query-agnostic proxies instead of explicit response-side sampling.Source: §A.2 · p. 18

A.3 QUANTIZATION AND REPRESENTATION COMPRESSION

  • Quantization and representation compression alter cache storage, while decode-time and serving methods manage later cache growth and can in principle combine with prompt-time LORE-KV eviction.These are presented as distinct memory-management approaches rather than competing prompt-time eviction criteria.Source: §A.4 · p. 18, §A.3 · p. 18

B BACKGROUND: THE KV CACHE AND THE EVICTION PROBLEM

  • KV-cache memory grows linearly with context length, and the ideal fixed-budget cache minimizes expected output distortion over future decoding queries.The passage gives an example of a 70B-class model requiring about 42.9 GB of KV-cache memory at 128K tokens.Source: B BACKGROUND: THE KV CACHE AND THE… · p. 18

C METHOD DETAILS

  • LORE-KV samples short lookahead continuations, reliability-weights their trajectories, scores candidate entries by projected deletion effect, and retains top-scoring entries under the cache budget.Lookahead states are used for scoring and discarded before decoding; final decoding uses the compressed cache.Source: C METHOD DETAILS · p. 19, §C.1 · p. 19

C.1 IMPLEMENTATION: PROTECTED SET, GROUPED-QUERY ATTENTION, FUSED KERNELS

  • Implementation optionally protects prompt markers, aggregates deletion effects across query heads sharing a KV head, and limits scoring to a small query set without changing fused kernels for prefill or decoding.The scoring-query count is w + MT, much less than n; protected markers include chat delimiters, tool-call markers, and document separators.Source: §C.1 · p. 19

C.2 RELATION TO PRIOR SURROGATE-QUERY AND OUTPUT-AWARE METHODS

  • At M=1, LORE-KV captures most of its gain; additional futures further improve robustness, while λL=0 is the prompt-only control.The passage attributes the main performance driver to correcting prompt-to-future query mismatch and describes broader future coverage as a robustness benefit.Source: §C.2 · p. 19

C.3 RELIABILITY WEIGHTING: DERIVATIONS, CALIBRATION, AND EDGE CASES

The reliability-weighting section describes trajectory-level signals and estimator behavior, then relates the additional lookahead computation to quality and latency. The overhead is a one-time compression cost tied to fixed lookahead settings.

C.3 RELIABILITY WEIGHTING: DERIVATIONS, CALIBRATION, AND EDGE CASES

  • For fixed-length trajectories, the variance calculation uses per-step varentropy; early termination replaces T with Tm in the corresponding per-trajectory averages.The optional normalization scales centered typicality by a trajectory-level standard error and is evaluated only as an ablation, not used in the default method.Source: §C.3 · p. 20
  • With uniform weights, the lookahead term is the Monte Carlo estimate under pθ(Y | X); finite-M weighting is not generally unbiased, and disagreement weighting makes it an interacting-particle estimator.For λD > 0, the disagreement score depends on the particle ensemble rather than a fixed single-trajectory target.Source: §C.3 · p. 21
  • Reliability weighting combines trajectory typicality, model uncertainty, and query-space disagreement to soften the influence of sampled futures that may be plausible but unrepresentative.The method presents these signals as a robustness safeguard, while λL limits the influence of imperfect lookahead.Source: §C.3 · pp. 19–21

C.4 COMPUTATIONAL OVERHEAD

  • 1.46×–2.77×: LORE-KV /E takes this multiple of AnDPro’s per-sample wall-clock time, as a one-time overhead that is independent of cache budget when lookahead settings are fixed.Figure 7 describes stage-wise latency and quality versus total latency across cache budgets; the passage reports higher quality at matched cache budgets.Source: §C.4 · p. 21, Figure 7 · p. 22, Table 3 · p. 22

D PLUG-IN COMPONENTS AND THE ABLATION PROTOCOL

The ablations isolate future-query selection as LORE-KV’s main source of improvement, while optional aggregation, allocation, and selection plug-ins modify separate pipeline stages. The core method disables those plug-ins so its gains can be attributed to the query reference and deletion score.

D PLUG-IN COMPONENTS AND THE ABLATION PROTOCOL

  • The plug-ins separately modify token-score aggregation, head-budget allocation, and token selection, leaving the other pipeline stages unchanged.The core configuration uses uniform head budgets and plain top-Bl,h selection.Source: D PLUG-IN COMPONENTS AND THE ABLATION… · pp. 22–23
  • Robust aggregation can emphasize high-importance query subsets, while adaptive budgets shift capacity toward heads with larger future-query perturbation mass.Setting α=0 recovers mean aggregation; τ=0 recovers uniform head allocation.Source: D PLUG-IN COMPONENTS AND THE ABLATION… · pp. 23–24
  • Diversity-aware selection is set-dependent and may suppress repeated but useful evidence; small penalties mainly break ties among similarly important tokens.Setting λk=λv=0 recovers plain top-Bl,h selection.Source: D PLUG-IN COMPONENTS AND THE ABLATION… · pp. 25–26
  • All three plug-ins are disabled for headline results, isolating gains attributable to the query reference and deletion score.Plug-in rows serve as robustness checks, with changes small relative to the core gain.Source: D PLUG-IN COMPONENTS AND THE ABLATION… · p. 26
  • Across three sampling seeds, LORE-KV /E averages 40.28 on LongBench, versus 38.96 for AnDPro and 35.87 without lookahead at B=128.Reliability-weighting variants reweight nested prefixes of one 8-particle draw, matching lookahead decoding compute; deletion-objective comparisons also use matched lookahead queries.Source: D PLUG-IN COMPONENTS AND THE ABLATION… · p. 26
  • At B=128, changing the deletion objective shifts LongBench average by at most 0.09 points; replacing prompt-side with response-side queries drives the dominant gain.The ablations locate the main improvement in correcting the query-reference mismatch rather than tuning auxiliary mechanisms.Source: D PLUG-IN COMPONENTS AND THE ABLATION… · p. 26

E.1 PROTOCOL, DATASETS AND BASELINES

The evaluation uses shared or explicitly qualified protocols across dense and hybrid-attention backbones. In hybrid models, eviction is restricted to full-attention layers, which still hold most cache memory at long context but can make lookahead cache cloning memory-intensive.

E.1 PROTOCOL, DATASETS AND BASELINES

  • Baselines are rerun in a common harness for Llama evaluations, while LAQ++ values are external and not budget-matched; no protocol-matched advantage over GVote is claimed.Controlled evidence instead uses rerun baselines and an AnDPro comparison with matched queries and lookahead-token budget.Source: §E.1 · pp. 26–27
  • The evaluation spans four dense-attention and two hybrid-attention backbones, with prompts following each model’s convention and budgets matched within each backbone.The six backbones include Qwen3.5-9B and Gemma-4-12B-it; the latter two use hybrid attention.Source: §E.1 · p. 27

E.2 HYBRID-ATTENTION BACKBONES: WHERE EVICTION APPLIES

  • Eviction applies only to full-attention layers in hybrid models, which hold 98.2% of Qwen3.5’s and 75.8% of Gemma-4-12B-it’s KV memory at 64K.Lookahead cache cloning can remain memory-intensive at very long context, particularly for Gemma-4-12B-it; this is a practical constraint, not a scoring-rule limitation.Source: §E.2 · pp. 27–28, Table 8 · p. 28, Table 9 · p. 28

E.3 LONGBENCH RESULTS ON ADDITIONAL BACKBONES

Across additional backbones, LORE-KV’s benefit is largest under aggressive compression and diminishes as cache budgets grow, while task-level gains vary. The trend also appears on hybrid-attention backbones, with different budget-dependent margins.

E.3 LONGBENCH RESULTS ON ADDITIONAL BACKBONES

  • On Llama at B=128, gains occur mainly on multi-hop and long-document QA, while code completion and some retrieval or few-shot tasks are less consistent.The broader trend is strongest under aggressive compression and narrows as cache budget increases.Source: §E.3 · p. 30
  • At B=128, LORE-KV improves over AnDPro by approximately 2.75 points on Qwen2.5-14B and 0.56 points on Ministral-3; the gap becomes small by B=1024.The two reliability variants remain close across the additional backbones.Source: §E.3 · p. 30, Table 11 · p. 30
  • Qwen3.5-9B and Gemma-4-12B-it show consistent hybrid-attention gains, strongest under aggressive compression and notable on answer-time evidence retrieval tasks.Qwen3.5 approaches full-cache quality as budget grows, whereas Gemma-4-12B-it maintains a more stable margin.Source: §E.3 · p. 30

E.4 RULER: PROTOCOL AND PER-TASK RESULTS

RULER results and ablations point to future-query estimation as the main source of improvement, particularly under tight cache budgets. The accompanying NIAH figure specifies its retrieval evaluation grid, while the measurement protocol describes controlled query comparisons and analysis-specific caveats.

E.4 RULER: PROTOCOL AND PER-TASK RESULTS

  • At 4K and B=128, increasing lookahead particles from M=1 to M=4 raises the RULER average from 56.90 to 58.40; gains are largest on aggregation-heavy tasks.Multi-query and multi-key retrieval remain more challenging, while reliability weighting and other plug-ins produce comparatively small changes.Source: §E.4 · pp. 30–31, Table 15 · p. 32

E.5 NIAH GRID

  • Figure 8 reports exact-match and word-recall scores for Llama-3.1-8B-Instruct at B=1024 across 4K–32K contexts and needle depths.Panel means are shown above each plot.Source: Figure 8 · p. 33

F.1 MEASUREMENT PROTOCOL

  • Controlled diagnostics compare query criteria using shared states and selection, separating scoring differences from downstream selection differences.Exact token-id matching for answer tokens may underestimate absolute evidence retention, though it affects all criteria equally; Equation 12 also changes Figure 12’s energy scale while preserving single-query rankings.Source: §F.1 · p. 33

F.2 FUTURE-QUERY COVERAGE

Sampled lookahead queries move toward the true future-query region and improve geometric coverage over prompt-window queries, with most coverage gains near M=4.

F.2 FUTURE-QUERY COVERAGE

  • Sampled trajectories move toward the region occupied by true future queries in the representative layer-and-head projection.The figure compares prompt-window, sampled lookahead, and retrospectively obtained future queries.Source: §F.2 · p. 34
  • Coverage rises from 0.785 for the prompt-window reference, with most gain obtained by approximately M=4.The sweep uses nested prefixes of the same eight sampled trajectories per prompt.Source: §F.2 · p. 34
  • Coverage measures geometric support rather than eviction quality; one rollout can provide a more relevant query reference, while more trajectories cover alternative plausible futures.The small M=0-to-M=1 coverage change (0.785→0.786) does not contradict the reported end-to-end gain from a single response-side rollout.Source: §F.2 · p. 34

F.3 RETENTION-ORDER SHIFT

Lookahead changes which tokens survive near the retention boundary and improves agreement with a future-query oracle, supporting query-reference choice as the main source of gain.

F.3 RETENTION-ORDER SHIFT

  • At B=128, attention and LORE-KV/E retain only 0.82 of the same LongBench tokens and 0.86 on RULER despite rank correlations of ρ=0.94 and ρ=0.87.Single-lookahead and LORE-KV/E have higher top-B overlap: 0.97 on LongBench and 0.92 on RULER.Source: §F.3 · p. 34, Figure 9 · p. 35
  • At B=128 on LongBench, LORE-KV/E raises oracle-retention agreement from 0.50 for the window-only control to 0.63 and reduces full-cache output error from 0.177 to 0.138.The advantage persists across budgets and on RULER.Source: §F.3 · p. 34, Figure 10 · p. 35
  • Controlled comparisons attribute the improved oracle agreement to shifting the query reference toward plausible future states, while prompt-side attention- and value-based criteria remain closer to each other.The result supports query-reference choice as more consequential than further refinement of prompt-side scoring.Source: §F.3 · pp. 34–35

F.4 RELIABILITY WEIGHTING: ROBUSTNESS BEYOND THE QUERY REFERENCE

Across the appendix analyses, multiple plausible futures provide the main robustness benefit, while reliability weighting is a secondary correction; lookahead also shifts retained tokens toward answer evidence and preserves behavioral fidelity across budgets.

F.4 RELIABILITY WEIGHTING: ROBUSTNESS BEYOND THE QUERY REFERENCE

  • At B=128 on LongBench, Recall @B against the future-query oracle increases from 0.50 for the λL=0 window-only control to 0.63 for LORE-KV/E, supporting its multiple-future robustness.Reliability signals are only weakly correlated with oracle retention utility, and hard-filtering low-reliability trajectories does not reduce residual query-coverage error; reliability weighting instead acts as a soft correction.Source: §F.4 · pp. 35–36

F.5 RETAINED-TOKEN COMPOSITION

  • LORE-KV/E raises answer/evidence enrichment to 2.51× on LongBench versus approximately 1× for prompt-side baselines, and to 20.3 on RULER versus 16.5–16.3.Protected sink and recent-token classes are identical across methods; other token classes change only modestly.Source: §F.5 · pp. 36–37, Figure 11 · p. 36

F.6 LAYER- AND HEAD-LEVEL STRUCTURE

  • Lookahead gains occur across most heads, with the largest concentrated in a subset of middle-depth heads; adaptive allocation adds only modest gains beyond the core score.Perturbation energy generally increases with depth, and the qualitative pattern appears on both LongBench and RULER.Source: Figure 12 · p. 36, §F.6 · p. 37

F.7 BUDGET EQUIVALENCE IN BEHAVIORAL FIDELITY

  • Across budgets, LORE-KV/E preserves full-cache behavior better, especially at tight budgets; matching its fidelity requires 33–43% more retained budget on LongBench and 27–34% on RULER.When a baseline does not reach the target fidelity within tested budgets, the required budget saving is reported only as a lower bound.Source: §F.7 · p. 37, Figure 13 · p. 37, Table 16 · p. 37
Loading 2610.07643v1…