Source-linked AI summary
ALLOT: Budgeted Hybrid-Memory Routing for Knowledge Updates in LLMs
Shanfeng Huang, Zhou Fang, Song Xiao, Hai Du
TL;DR
When retrieval already answers some facts, writing every fact to both external and parametric memory spends adaptation resources on facts that may not need a parametric write. ALLOT retains all facts in external memory and uses a reusable memory-aware ranking with a hard cutoff to select parametric writes under a budget. ALLOT recovers 78.4% of oracle gain at the same write budget, compared with 41.3% for uplift and 35.1% for GBT.
Problem
When retrieval already answers some facts, writing every fact to both external and parametric memory spends adaptation resources on facts that may not need a parametric write.
Source: §1 · p. 1Method
ALLOT retains all facts in external memory and uses a reusable memory-aware ranking with a hard cutoff to select parametric writes under a budget.
Source: §1 · p. 1, §2.3 · p. 2Results
ALLOT recovers 78.4% of oracle gain at the same write budget, compared with 41.3% for uplift and 35.1% for GBT.
Source: §4.1 · p. 3Takeaways & Limitations
The results support allocating adaptation capacity according to incremental value rather than treating every factual update as an equally valuable training target.
Source: §5 · p. 4The allocation objective assumes fixed per-item outcomes, so allocation quality is the only free variable.
Source: §2.1 · p. 2
Abstract
from arXiv · showhide
For large language models (LLMs), parametric adaptation is costly when retrieval already suffices. We introduce ALLOT, a hybrid-memory routing framework that separates learned write priority from a hard parametric budget. A memory-aware router combines frozen text representations, retrieval confidence, and relation metadata; a single ranking supports multiple write budgets while preserving all facts in external memory. On CounterFact with Qwen3-4B, ALLOT reaches 0.760 accuracy at a 20% parametric-write budget and recovers 78.4% of the budget-matched oracle gain, with 80% fewer parametric writes than dual-writing every fact. At this budget, jointly adding retrieval and relation features to text improves normalized oracle gain by 6.2 percentage points. Complementary Qwen3-0.6B shared-store results achieve dual-write-level accuracy with 6-14.5% parametric writes, and cross-benchmark transfer retains approximately 88% of in-domain gain. These results support allocating adaptation capacity according to its incremental value rather than treating every factual update as an equally valuable training target.
1. INTRODUCTION
ALLOT treats parametric writes as a budgeted choice about which facts merit a second memory, rather than writing every fact to both memories. Its memory-aware ranking separates learned priority from the hard write budget.
1. INTRODUCTION
- ALLOT asks which facts merit an additional parametric write when retrieval already answers some facts and adaptation resources are limited.External writes are inexpensive, while training and retaining a LoRA adapter adds compute and storage.Source: §1 · p. 1
- Unlike systems that decide how memories are formed, read, or combined, ALLOT selects which updates receive parametric writes under a global adaptation budget.The framing is related to learning to defer, with storage actions replacing predictor choices.Source: §1 · p. 1
- A useful router must rank the incremental value of a second memory: poor retrieval alone does not establish that adaptation helps.A fixed action classifier cannot enforce a workload-level write budget; ranking separates preference from available write capacity.Source: §1 · p. 1
- ALLOT writes every fact externally and gives adapters only to top-ranked facts, using text, retrieval statistics, and relation metadata in one ranking.Changing the budget changes the selection cutoff without retraining or selecting a new probability threshold.Source: §1 · p. 1, Figure 1 · p. 1
- The paper contributes exact write-count control with nested selections across budgets, memory-aware prioritization, and an empirical allocation frontier.The introduction reports 78.4% oracle-gain recovery at 20% budget on CounterFact/4B and highlights feature ablations, shared-store results, and transfer.Source: §1 · pp. 1–2
2. BUDGETED HYBRID-MEMORY ROUTING
ALLOT formulates selective parametric writing as a hard-budget allocation problem, trains a router from action outcomes and item features, then ranks items for exact-budget writes. Its analysis connects ranking errors to which items cross the budget cutoff.
2.1. Allocation objective
- For fixed outcomes, the capped oracle selects up to the budget's largest positive gains, while the exact-budget oracle selects the largest gains even when some are negative.The capped oracle is an upper reference for the exact-budget router.Source: §2.1 · p. 2
2.2. Supervision and feature representation
- The router is trained on four action labels from probe outcomes, using frozen text embeddings, top-5 retrieval-similarity statistics, and relation encodings as input features.The feature slots are zero-padded when inactive; the 4B CounterFact labels have a 53% external-only majority, which class weighting counters.Source: §2.2 · p. 2, §2.1 · p. 2
2.3. Budgeted routing algorithm
- The deployed router scores each candidate once, sorts by score, writes every item externally, and writes the top B items parametrically, with fixed tie-breaking for nested selections.Changing the budget changes only the cutoff for the fixed candidate pool and feature context.Source: §2.3 · p. 2
2.4. Why ranking quality matters
- Allocation error depends on which items cross the cutoff, not four-way classification accuracy alone; confusing para with both leaves the aggregated score unchanged.A low-gain promoted item can displace a higher-gain write, and additional writes can have positive or negative realized gains.Source: §2.4 · p. 2
3. EXPERIMENTAL SETUP
The experiments evaluate budget allocation on zsRE and CounterFact with Qwen3-0.6B and Qwen3-4B, using per-item and complementary shared-store, transfer, and sequential protocols.
3. EXPERIMENTAL SETUP
- The study uses 500 factual updates each from zsRE and CounterFact, with three paraphrase probes per item, on Qwen3-0.6B and Qwen3-4B.External memory uses top-5 cosine retrieval; rank-16 LoRA targets query/value projections, with different adapter packing and training setups for the two model sizes.Source: §3 · p. 3
- The main 4B study compares top-B allocation methods across three 80/20 partitions; complementary protocols test shared-store writes, transfer from zsRE to CounterFact, and sequential budget pacing.Transfer uses no target-benchmark labels, and the sequential protocol varies arrival order.Source: §3 · p. 3
4. RESULTS
ALLOT’s budgeted routing performs strongly on CounterFact/Qwen3-4B and complementary shared-store and transfer tests. The results also show that feature utility and the beneficial-write pool vary with budget and model scale.
4.2. Joint value of routing features
- At 20% budget, joint text, retrieval, and relation features reach 78.4% OG, adding 6.2 pp over text alone; feature utility varies with the budget cutoff.Text plus retrieval is best at 5%, while the full combination is strongest at 10–30%.Source: Table 2 · p. 3, §4.2 · p. 3
4.1. Accuracy under a hard write budget
- At 20% budget, ALLOT reaches 0.760 accuracy versus 0.732 for random, 0.747 for uplift, and 0.768 for the budget-matched oracle, recovering 78.4% of oracle gain.Its advantage over uplift has the same sign across all three partitions.Source: §4.1 · p. 3, Figure 2 · p. 4
4.3. Shared-store efficiency and transfer
- 0.646 and 0.585 accuracy were on par with dual-writing’s 0.644 and 0.585, using 6% and 14.5% parametric writes.Archived storage estimates were 553 MB for dual-writing and 37/63 MB for routing, estimated reductions of 14.9×/8.8×.Source: §4.3 · pp. 3–4, Table 3 · p. 4
- A router trained on zsRE achieved 0.577 CounterFact accuracy versus 0.516 for retrieval and 0.585 for in-domain routing, retaining approximately 88% of in-domain gain without CounterFact labels.The evaluation tests cross-benchmark transfer without target-benchmark labels.Source: §4.3 · p. 4
4.4. Scale and sequential allocation
- At 20% budget, adapter training’s dominant write-cost term is cut fivefold; the beneficial fraction of CounterFact updates rises from 19.6% at 0.6B to 47.2% at 4B.In the 500-item pool, a 20% budget covers at most 100/236, or about 42%, of beneficial updates.Source: §4.3 · p. 4, §4.4 · p. 4
- Across 30 sequential replays, adaptive pacing reached 0.75764 at 20% budget versus 0.75800 for full-pool selection, a 0.036 pp gap.Per-item priorities transferred across arrival orders.Source: §4.4 · p. 4
- On 0.6B, the beneficial fraction stayed within 17–21% across LoRA ranks and packing densities; adapter training degraded only 1% of 300 unrelated probes.With 100 labels (≈3.5 GPU-hours), one router recovered 55–61% of oracle gain and served every budget through the same ranking.Source: §4.4 · p. 4
5. CONCLUSION
ALLOT frames LLM knowledge updates as budgeted hybrid-memory allocation: facts remain in external memory while a reusable ranking selects parametric writes. The paper reports strong oracle-gain recovery at a 20% budget and results supporting selective adaptation.
5. CONCLUSION
- ALLOT retains facts externally and uses a reusable memory-aware ranking to budget parametric writes, recovering 78.4% of budget-matched oracle gain at 20% on CounterFact/Qwen3-4B with 80% fewer writes.One ranking supports changing budgets without retraining or threshold recalibration, and orders writes over time.Source: §5 · p. 4
6. COMPLIANCE WITH ETHICAL STANDARDS
The experiments used publicly available language models and factual-editing benchmarks released for research use, with no new data collected.
6. COMPLIANCE WITH ETHICAL STANDARDS
- No human participants or animals were involved, no ethical approval was required, and all experiments used publicly available models and research benchmarks without collecting new data.Source: §6 · p. 5