PaperlayerView exact revision on arXiv

Source-linked AI summary

CAMR: Compatibility-Aware Memory Replay for Continual Learning in Large Language Models

Jingpu Yang, Tianhao Li, Fengxian Ji, Yufeng Wang, Shuxin Zheng, Xiaoqing Zhang

arXiv:2610.36734v1cs.CL

TL;DR

Continual learning must acquire new knowledge without losing prior learning, yet replay leaves open which incoming examples need supervision and which history to revisit. CAMR combines compatibility checks and a learned memory gate to guide bounded-budget supervision and replay, leading evaluated methods on WikiBigEdit and selected TRACE tasks.

  • Problem

    Continual learning must add knowledge and capabilities while retaining prior learning, but replay selection alone leaves unresolved whether incoming examples need learning, joint training, or no direct supervision.

    Source: §1 · p. 1

  • Method

    CAMR combines compatibility checks, a learned memory gate, and an online policy to jointly allocate incoming-example supervision and historical replay within a bounded per-arrival replay budget.

    Source: §1 · pp. 1–2

  • Results

    CAMR consistently achieves the highest WikiBigEdit Original Match from 3K to 100K facts and the highest final scores on four TRACE tasks, reaching 93.66% after 3K facts.

    Source: §1 · p. 2

  • Takeaways & Limitations

    CAMR maintains or improves earlier-task performance relative to post-acquisition levels in the FinQA →GSM8K →APPS →SAMSum sequence.

    Source: §1 · p. 2

    Experiments focus on Qwen3-8B-Base, and a full-parameter comparison of all baselines is left for future work.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Continual learning requires large language models to acquire new knowledge and capabilities while retaining what they have learned. Fine-tuning and replay support acquisition and retention, but deciding whether incoming examples need supervision and which historical examples to revisit remains a challenge. We propose Compatibility-Aware Memory Replay (CAMR), an adaptive framework that jointly controls current-example supervision and historical replay. CAMR combines compatibility checks with a learned memory gate to guide an online policy over three actions: learning incoming examples with general replay, jointly reinforcing compatible related examples, and skipping direct supervision of verified mastered duplicates. The policy learns from acquisition, retention, and training-cost feedback under a bounded per-arrival replay budget. We evaluate CAMR with Qwen3-8B-Base on continual factual updating and cross-task and cross-domain learning. On WikiBigEdit, CAMR consistently leads evaluated methods in Original Match from 3K to 100K facts, reaching 93.66% after just 3K facts and surpassing Full-history Replay (LoRA) after 100K facts. It also achieves the highest final scores on all four selected TRACE tasks and maintains or improves earlier-task performance relative to post-acquisition levels in a sequence spanning financial question answering, mathematical reasoning, code generation, and summarization.

1 Introduction

Continual learning asks LLMs to acquire new knowledge while retaining prior learning, but successive updates can disrupt what they have learned. CAMR coordinates current-example supervision with historical replay and reports strong results across factual updating and continual task settings.

1 Introduction

  • Continual learning enables LLMs to acquire knowledge and capabilities while retaining prior knowledge, but successive updates can disrupt previously learned knowledge.This is the stability–plasticity problem (McCloskey and Cohen, 1989; De Lange et al., 2022).Source: §1 · p. 1
  • Adaptive historical-example selection still leaves open whether incoming examples need further learning, compatible joint training, or no direct supervision, while unrelated history needs replay for retention.CAMR addresses this coordination gap between current-example supervision and replay composition.Source: §1 · p. 1
  • CAMR combines compatibility checks and a learned memory gate to guide an online policy jointly controlling current-example supervision and historical replay under a bounded per-arrival replay budget.The policy uses model mastery and gated historical information to adapt supervision decisions through learning feedback.Source: §1 · pp. 1–2
  • The policy chooses ADD for incoming examples with general replay, MERGE to prioritize compatible related history, or IGNORE to skip direct supervision of verified mastered duplicates while allowing replay.It adapts online using acquisition, retention, and training-cost feedback.Source: §1 · p. 2
  • CAMR consistently achieved the highest Original Match from 3K to 100K facts on WikiBigEdit, reaching 93.66% after 3K facts and surpassing Full-history Replay (LoRA) after 100K facts (92.07%).It also improved final-stage generalization success over Random Replay (Full FT) by 1.25 percentage points on WikiFactDiff.Source: §1 · p. 2
  • On all four selected TRACE tasks, CAMR achieved the highest final scores and maintained or improved earlier-task performance relative to post-acquisition levels in the FinQA →GSM8K →APPS →SAMSum sequence.Replay-count scaling and ablations examine sensitivity to replay capacity and the contributions of key policy components.Source: §1 · p. 2

2 Related Work

Prior continual-learning approaches control parameter updates, replay selection, or training-example supervision. CAMR combines decisions about which history to revisit with decisions about how to supervise incoming examples.

2 Related Work

  • Parameter-space and knowledge-editing methods control how parameters change, whereas CAMR decides which examples supervise each update.The cited approaches include regularization (Kirkpatrick et al., 2017), gradient projection (Wang et al., 2021), orthogonal low-rank subspaces (Wang et al., 2023a), and factual editing (Meng et al., 2022, 2023; Fang et al., 2025).Source: §2 · p. 3
  • Adaptive replay selects historical examples by interference risk or transfer value, while CAMR also decides how the incoming example itself is supervised.Experience replay mixes stored examples with new data to support retention (Rolnick et al., 2019); adaptive variants include Aljundi et al. (2019) and Meng et al. (2026).Source: §2 · p. 3
  • CAMR’s writer uses an actor–critic policy rewarded by acquisition, retention, and training cost, with a gated history summary in its state.The passage situates this approach alongside learned example selection and bandit-based curricula (Fan et al., 2018; Graves et al., 2017), and adapts gated memory from Engram (Cheng et al., 2026).Source: §2 · p. 3

3 Problem Setting

The problem setting presents a stream of supervised examples and an online learner that must acquire new knowledge and capabilities while retaining prior learning that remains valid. The learner jointly controls current-example supervision and historical replay under a per-arrival cap.

3 Problem Setting

  • The learner receives a stream of supervised examples and can use current and previously observed examples, but has no access to future arrivals or held-out targets.Each example contains an input, target output, and optional identity, task, or version metadata.Source: §3 · p. 3
  • The objective is to acquire new knowledge and capabilities while retaining prior learning that remains valid.For factual updates, supervision is trusted: an authoritative replacement makes the new answer valid and retires the old one.Source: §3 · p. 3
  • The writer jointly controls current-example supervision and historical replay under a per-arrival replay cap K, which counts historical examples and excludes the current example.ADD uses general replay, MERGE prioritizes compatible related history within the same budget, and IGNORE skips direct supervision of a verified mastered duplicate while allowing history-only replay.Source: §3 · p. 3

4 Method

CAMR couples compatibility-filtered memory retrieval with a writer policy that selects current-example supervision and historical replay. The writer learns from acquisition, retention, and training-cost feedback.

4 Method

  • CAMR retrieves compatible history, combines it with model mastery and a gated history representation, and uses an online writer to choose supervision and replay.Figure 2 depicts retrieval and filtering before the writer selects an action and forms the training batch.Source: §4 · p. 3, §4.2 · p. 4, Figure 2 · p. 4

4.1 Memory Retrieval and Compatibility

  • Eligible history excludes retired, current-equivalent, and conflicting records; retrieval combines semantic search and exact lookups before filtering candidates by compatibility and similarity.A probe set is sampled from eligible history and excluded from the event’s training batch.Source: §4.1 · pp. 3–4

4.2 Writer Policy and Replay

  • The writer selects ADD, MERGE, or IGNORE; MERGE prioritizes compatible candidates, while ADD and IGNORE use general replay and IGNORE omits current-example supervision.MERGE allocates part of the replay budget to general history and the remainder to compatible candidates, filling unused candidate slots from the remaining pool.Source: §4.2 · pp. 4–5

4.3 Learning Objectives and Online Updates

  • CAMR trains the language model with an answer-prediction objective and the writer with actor–critic learning driven by acquisition, retention, and training-cost feedback.Writer parameters update at rollout boundaries; the reward weights retention and cost, and the critic fits returns while the actor adjusts action probabilities using advantages.Source: §4.3 · p. 5

5 Experiments and Discussion

CAMR is evaluated on factual updating and continual task streams, with results reported for scaling, cross-task performance, replay budgets, and writer ablations. Across these settings, it shows strong acquisition and retention, though the experiments use different tasks and baselines.

5.1 Experimental Setup

  • The experiments evaluate Qwen3-8B-Base on WikiBigEdit, WikiFactDiff, TRACE, and a four-task cross-domain sequence, alongside additional ablation tasks.The setup also includes a Qwen3-14B-Base unmodified WikiBigEdit reference and reports metrics appropriate to each task.Source: §5.1 · pp. 5–6

5.2 Single-Task Data Scaling

  • CAMR achieves the highest WikiBigEdit Original Match at 3K, 10K, 50K, and 100K facts, and at 150K surpasses Full-history Replay (LoRA) on all four Match metrics.The 150K comparison also reports declines in Rephrase, Persona, and Locality scores for Full-history Replay (LoRA).Source: §5.2 · p. 6

5.3 Cross-Task and Cross-Domain Performance

  • On TRACE, CAMR has the highest final scores on all four tasks; on FinQA →GSM8K →APPS introductory →SAMSum, earlier tasks retain or improve post-acquisition scores.TRACE final scores are 56.75%, 69.84%, 67.92%, and 95.40%, with earlier-task losses of 0.75, 0.55, and 0.42 points. Cross-domain final scores are 76.11%, 81.58%, 30.50%, and 45.10%; CAMR leads on FinQA and SAMSum, while O-LoRA and Adam-NSCL lead GSM8K and APPS, respectively.Source: §5.3 · pp. 6–7, Table 2 · p. 7, Table 3 · p. 8

5.4 Sensitivity to Replay Count

  • Increasing K from 4 to 16 raises CAMR’s Original/Rephrase Match from 93.98%/86.56% to 98.10%/90.42%; K = 8 gives its highest Persona/Locality scores, 72.62%/19.86%.CAMR leads Original and Rephrase Match at every tested replay budget; K = 4 is the default used to test a limited replay budget.Source: §5.4 · p. 7, Table 4 · p. 8

5.5 Writer Component Ablations

  • The complete writer performs best on all four ablation tasks, tying the version without online feedback on NumGLUE-cm; removing action selection or IGNORE degrades performance across tasks.NumGLUE-ds is particularly sensitive to removing memory gating or replay under IGNORE, while online feedback improves three tasks, with its largest gain on NumGLUE-ds.Source: §5.5 · p. 7

K Method Original Rephrase Persona Locality

Writer ablations associate adaptive supervision and replay decisions with stronger performance, while replay analysis shows that CAMR combines focused sampling with broad historical coverage.

K Method Original Rephrase Persona Locality

5.6 Replay Sample Selection Analysis

  • At 100K, CAMR replays the high-frequency core-3K set at 2.42× Random Replay’s share while covering 90.4% of arrived facts versus Semantic Replay’s 83.9%.Its hybrid sampling prioritizes compatible history in MERGE while reserving uniform slots; ADD samples uniformly. CAMR also outperforms Semantic Replay on Original and Rephrase Match at every tested K.Source: §5.6 · pp. 8–9

6 Conclusion

The conclusion presents CAMR as a joint policy for current-example supervision and bounded historical replay. Across the evaluated settings, the authors report strong acquisition and retention.

6 Conclusion

  • CAMR jointly controls current-example supervision and bounded historical replay through compatibility-aware memory and an online writer policy.The authors report strong acquisition and retention across factual updating, cross-task learning, cross-domain learning, and replay budgets.Source: §6 · p. 9

Limitations

The experiments are limited by computational constraints and focus on Qwen3-8B-Base.

Limitations

  • Computational constraints limit experiments to Qwen3-8B-Base; full-parameter comparisons for all baselines and broader model-family evaluation remain future work.Some baselines use LoRA rather than full-parameter fine-tuning to limit cost.Source: Limitations · p. 9

A Additional 150K WikiBigEdit Evaluation

At 150K WikiBigEdit updates, CAMR exceeds Full-history Replay (LoRA) across all four reported Match measures. The replay-volume account specifies how Full-history Replay accumulates historical presentations.

A Additional 150K WikiBigEdit Evaluation

B WikiBigEdit Replay Volume

  • Full-history Replay uses LoRA and trains new facts in blocks of up to 375, ending blocks at native-interval boundaries and evaluation checkpoints.It replays all previously observed facts at each block’s start; cumulative historical replay is the sum of previously observed facts across blocks.Source: B WikiBigEdit Replay Volume · p. 11
  • Table 7 counts each selected fact once, including both current facts and historical replay, in cumulative fact presentations.CAMR and Random Replay use full-parameter updates, unlike Full-history Replay’s LoRA updates.Source: B WikiBigEdit Replay Volume · p. 11

C Writer Action Utilization

The action audit reports when IGNORE was or was not selected, while the hyperparameter specification describes the reference CAMR configuration and its replay and learning controls.

C Writer Action Utilization

  • With the original IGNORE gate, no arrival in the 100K WikiBigEdit stream met the duplicate/equivalence prerequisite, so IGNORE was never selected.The gate also requires current NLL ≤ τm, no arrival-triggered retirement, and a fresh greedy answer matching the full target.Source: C Writer Action Utilization · p. 11
  • Figure 4 maps replay intensity relative to event-wise Random Replay expectation, with gray at 1×, red above, and blue below expectation on a shared log2 scale.Rows compare CAMR, count-matched Random Replay, count-matched cosine Top-8 Semantic Replay, and theoretical 375-record Full-history Replay across five checkpoints.Source: Figure 4 · p. 12
  • In a repeated-example APPS diagnostic, the writer selected ADD 2,947 times (58.94%), MERGE 1,096 times (21.92%), and IGNORE 957 times (19.14%).The stream appended 2,361 exact input–output duplicates to the original 2,639-example single-pass stream.Source: C Writer Action Utilization · p. 12

D Complete CAMR Hyperparameter Specification

  • The reference configuration sets P = 4 and K = 4, τs = 0.65, τm = 0.25, α = 0.5, and γ = 0.95, with reward and critic coefficients also specified.Cref is 128 when Lmax < 128 and 1024 otherwise; β = 0.01 and cv = 0.5.Source: D Complete CAMR Hyperparameter… · pp. 12–13

E Semantic Replay Trajectory Analysis

The replay trajectory analysis contrasts CAMR’s localized allocation with random, semantic, and full-history replay. As WikiBigEdit grows, CAMR also keeps a larger share of replay slots on the fixed core-3K set than Random Replay.

E Semantic Replay Trajectory Analysis

  • CAMR forms localized regions of elevated and reduced replay intensity, while Random Replay stays near neutral, Semantic Replay creates stronger hotspots and cold regions, and Full-history Replay reaches the upper range from 50K onward.The comparison corrects for cumulative advantage of earlier-arriving facts and event-wise variation in eligible history.Source: E Semantic Replay Trajectory Analysis · p. 13
  • At 100K, CAMR, Random Replay, and Semantic Replay had replayed 90.4%, 88.8%, and 83.9% of arrived facts, respectively; Full-history Replay reached 99.8% coverage.Under the matched budget, CAMR maintained broad support while concentrating additional presentations in selected neighborhoods.Source: E Semantic Replay Trajectory Analysis · p. 14

F Core-knowledge replay under stream growth.

  • CAMR’s share of replay slots for the core-3K set fell from 25.54% to 7.30%, versus Random Replay’s decline from 19.49% to 3.02%; at 100K, CAMR’s share was 2.42× higher.The comparison covers the fixed high-frequency core-3K set as WikiBigEdit history grows.Source: F Core-knowledge replay under stream… · p. 14, Table 10 · p. 14

Related papers

Loading 2610.36734v1…