Memory-Centric Continual Learning for AI Agents

1. The Landscape: Why Agents Need Continual Learning via Memory

• Existing memory systems often treat history as a retrieval cache, while benchmarks emphasize recall over improved actions under interference and drift.Growing unmanaged stores can add retrieval noise, pollution, and negative transfer as task distributions shift.Source: §1 · p. 3
• MCCL treats memory as an evolving learning infrastructure that can incorporate feedback and refine management policies without continuous updates to foundation-model weights.An inner loop updates memory state from deployment experience; an outer loop adapts management policies based on task performance.Source: §1 · p. 3
• The taxonomy separates memory substrates, assets, and policies: where experience persists, what reusable form it takes, and how operations are managed.The axes distinguish physical storage from cognitive representation and operational control.Source: §1 · p. 3
• The lifecycle formalizes selective formation, consolidation, reactivation, revision and forgetting, and feedback-driven policy evolution.It describes how raw interactions become generalizable knowledge while mitigating memory pollution and drift.Source: §1 · p. 3
• The evaluation framework distinguishes retrieval checks from learning outcomes using a four-gate retention cascade and matched stateful-versus-reset comparisons.The supplied passage introduces these measures but does not provide the full evaluation protocol.Source: §1 · p. 3

2. A Unified Framework: Architecting Memory for Continual Learning

• MCCL couples an inner loop that updates memory state with an outer loop that uses task feedback to adapt memory-management policies.Experience is organized as episodic, semantic, and procedural assets across contextual, activation, external, and parametric substrates.Source: Figure 2 · p. 4, §2 · p. 4

2 A Unified Framework: Architecting Memory for Continual Learning

• The three-axis taxonomy distinguishes physical substrates, reusable assets, and management policies, which can be combined rather than treated as isolated categories.An external substrate alone does not ensure transferable capability; assets and policies shape formation and use.Source: §2 · p. 4, §2.1 · p. 5

2.1 Memory Substrates: Physical Foundations for Stability and Plasticity

• Substrates trade off persistence and controllability: external stores allow durable, revisable records, while parameter updates can internalize stable skills but make writes harder to control.External stores still face retrieval noise and functional forgetting; parametric updates can risk forgetting and make deletion or provenance harder.Source: §2.1 · p. 6

2.2 Memory Assets: Representations for Cross-Task Transfer

• Assets are classified by how experience is used: episodes retain contextual evidence, semantic assets abstract reusable knowledge, and procedural assets preserve how to act.Abstraction can improve compression and reuse, but erroneous generalizations can affect a wider range of tasks; consolidated assets may lose links to supporting episodes.Source: §2.2 · pp. 7–8, §2.3 · p. 9

2.3 Memory Policies: Prescribed, Deliberative, and Learned Control

• Memory policies govern retention, consolidation, revision, and forgetting, spanning fixed prescribed rules, per-case deliberation, and outcome-driven learning.Prescribed approaches use predefined rules such as FIFO, LRU, or fixed Top-k retrieval; regimes can be composed.Source: §2.3 · p. 9

3. The Memory Lifecycle as the Engine of Continual Learning

• A fixed memory controller can be efficient and inspectable yet rigid because task outcomes do not subsequently revise its novelty, importance, or disposal criteria.Deliberative control adds flexible case-by-case judgments but remains inference-intensive and prompt-dependent; learned policies update control from outcomes.Source: §3 · p. 10

3 The Memory Lifecycle as the Engine of Continual Learning

• MCCL couples experience-driven changes to persistent memory with behavioral-feedback-driven changes to the policy governing future memory decisions.The state transition handles experience under the current policy; the policy can evolve based on downstream feedback.Source: §3 · p. 10, §3.1 · p. 11
• Memory changes count as continual learning only when they remain usable and improve or adapt later behavior without unnecessary damage to useful capabilities.Information may persist without being retrieved, or be retrieved but used inappropriately or harmfully.Source: §3.1 · p. 11
• The inner memory-state loop covers formation, organization, retrieval and utilization, and revision and forgetting; an outer loop governs these operations and may update their control rules from feedback.Persistent memory is the evolving state on which lifecycle operations act, not an additional operational stage.Source: §3.1 · p. 11

3.1 Formation and Selective Writing

• Selective writing sets a plasticity boundary: admitting unreliable experience can propagate errors, while rejecting useful experience can remove evidence needed for future learning.The goal is useful admission, not maximizing stored information.Source: §3.1 · p. 11
• Initial representation bounds what downstream memory operations can use, because consolidation cannot reliably recover evidence that was never recorded.Structure and granularity are separate choices: relational and temporal indexing can coexist with fine-grained detail.Source: §3.1 · p. 11

3.2 Organization, Compression, and Consolidation

• Experience units shape what evidence can later be compared or consolidated: fine-grained units preserve local detail but can split dependencies, while broader units may combine observations that should remain separate.Recent systems use dynamically detected events or temporally grounded episodes rather than only fixed turns or windows.Source: §3.2 · p. 13
• Consolidation timing balances premature generalization against fragmented experience: LycheeMemory V2 uses semantic segment boundaries, while RecMem uses recurrence across related observations.These triggers are prescribed rather than learned from downstream outcomes.Source: §3.2 · p. 13
• Consolidation can turn episodes into reusable abstractions, trading some contextual detail for broader applicability rather than simply reducing storage size.Episodes retain context and supporting evidence; abstractions can improve transfer but need not replace episodic memory.Source: §3.2 · p. 13
• Repeated LLM-based rewriting can initially improve and then reduce memory utility, while raw episodes can remain competitive; consolidation may also remove context needed to judge whether an abstraction applies.The paper argues for preserving links to supporting episodes so later retrieval, verification, or revision can recover from poor generalization.Source: §3.2 · p. 13

3.3 Retrieval and Utilization

• Retrieval should recover enough contextual evidence for the current decision, whose needs may differ across planning, tool execution, verification, and failure recovery.Retrieved evidence appearing in context does not establish that it shaped reasoning or improved behavior; preservation, access, utilization, and behavioral effect are distinct.Source: §3.3 · p. 14

3.4 Revision and Forgetting

• Revision changes what is recorded or treated as current, while forgetting regulates continued access and influence; conflicting evidence may reflect error, temporal change, or contextual difference.Those cases require different responses, such as correcting a false observation versus retaining historical evidence while marking it no longer current.Source: §3.4 · p. 14

3.5 Feedback-Driven Policy Evolution

• Maintenance must distinguish outdated claims from records that are merely costly or distracting: invalidation changes scope without erasing history, while forgetting changes access or influence.Forgetting may use deletion, expiration, or soft suppression; factual validity and retrieval availability are distinct.Source: §3.5 · p. 15
• When future usefulness is uncertain, reversible suppression or archival may be preferable to irreversible deletion, which may also leave influence in derived summaries or procedures.A DELETE operation alone does not establish an effective selective-forgetting policy.Source: §3.5 · p. 15
• Online policy evolution continually changes how future memory operations are selected or configured, unlike state-level feedback incorporation or offline policy training that leaves the controller frozen.Online policy evolution is an advanced adaptation layer, not a prerequisite for all memory-centric continual learning.Source: §3.5 · pp. 15–16
• Feedback can enrich memory without changing its management rules, or train a controller; closing the policy-evolution loop requires keeping that controller open to revision during deployment.Reflexion and Voyager illustrate state-level incorporation, while AgeMem and UMA optimize memory-action selection against downstream rewards.Source: §3.5 · p. 16
• Policy updates face delayed credit assignment: a write, consolidation, or deletion may show its value only in a later task, and final outcomes may not identify the responsible decision.Storage, latency, and computation costs also require weighing against future benefits.Source: §3.5 · p. 16
• The lifecycle comparison maps overlapping design choices, not solved learning problems: an operation or responsive mechanism does not establish that its decision rule is learned or appropriate.Coordination must preserve evidence for later use while reducing outdated influence without damaging valid knowledge.Source: §3.5 · p. 16

4. Evaluating Memory-Augmented Agents as Continual Learners

• Figure 5 maps representative systems by emphasized memory operations and decision-control basis; its positions describe design choices, not performance.Darker regions mark main design focus, lighter regions supporting operations, and colors distinguish design families.Source: Figure 5 · p. 17
• Evaluation must establish durable behavioral improvement, adaptation, and transfer while preserving useful capabilities, rather than infer learning from a changed memory store or single-task score.The protocol aims to separate these outcomes from expanded context, extra test-time computation, or transient success.Source: §4 · p. 17

• Continual learning requires experiential gains to persist under delay and interference, remain revisable when the world changes, and transfer beyond remembered episodes [71] [112].Content correctness and retrieval scores diagnose whether experience was written and remains findable; a single-task outcome shows only help on the current probe.Source: §4.1 · p. 17
• Component traces can localize failure, but they do not substitute for behavioral evidence of learning.Source: §4.1 · p. 17
• Access-level metrics answer a narrower question than learning by measuring content correctness or retrieval rather than durable behavioral effects.Source: §4.1 · p. 17

4.1 What to Measure: Retention, Adaptation, and Transfer

• Table 1 distinguishes internal storage-fidelity and retrieval-reachability diagnostics from behavioral retention, adaptive revision, and cross-task transfer needed to establish continual learning.Source: Table 1 · p. 18

4.1.1 Retention: Learning Must Survive Delay and Interference

• Behavioral retention means outperforming a matched agent deprived of the experience on the same probe after delay and subsequent experience.A stored entry or decodable latent state can persist even after its contribution to decisions vanishes; a useful procedure may persist without its originating episode.Source: §4.1.1 · p. 18
• A weakest-gate hypothesis proposes that behavioral retention is bottlenecked by the least reliable step, so greater capacity may improve preservation while worsening access through retrieval competition.Continual insertion and eviction can reduce earlier-task performance, and repeated extraction and injection can turn a single-round gain into progressive collapse [148] [15].Source: §4.1.1 · p. 18
• Retention protocols should measure immediate gain Δ0_i, then normalized retained gain Δ_i(d, m)/Δ0_i after specified delay and interference, with delay and intervening updates characterized.Substrate-specific traces can localize deletion, eviction, rank competition, overwriting, state drift, or readout mismatch.Source: §4.1.1 · p. 18

4.1.2 Adaptation: Learning Must Remain Revisable under Change

• Adaptation evaluation tests whether behavior follows a valid update after environmental change without indiscriminately damaging still-useful capabilities.The temporal chain is detection, revision, recovery, and preservation; proposed measures include detection delay, steps to recovery, stale-error duration, and collateral degradation on unchanged probes.Source: §4.1.2 · p. 18

4.2 How to Measure: Protocols for Experience, Controls, and Time

• Adaptation diagnostics should use matched pre-change and post-change probes and report recovery alongside selective preservation; exposed maintenance or update accuracy alone does not demonstrate behavioral recovery.Conversational QA, sequential fact updates, or verbal recognition of change may fail to show that an acting agent resumes correct tool use or propagates the change to prediction.Source: §4.2 · p. 19

4.1.3 Transfer: Learning Must Extend beyond Remembered Episodes

• Transfer protocols separate replay of the same or near-duplicate episode, unseen variants within a task family, and shared-structure reuse on a distinct target task.Source and target partitions should exclude duplicates and target answers, document shared structure, and match the base model, tools, context budget, and compute.Source: §4.1.3 · p. 19
• Abstraction supports reuse across related tasks and environments, while raw trajectory memory can yield negative forward transfer and dedicated memory methods are only selectively helpful [111] [121] [163] [47] [6] [40] [166].Transfer reporting should include zero-shot target gain or forward transfer, interactions or tokens to a target threshold, and negative-transfer rates.Source: §4.1.3 · p. 19

4.2.1 Benchmarks Must Contain Learnable Experience

• Pre-supplied dialogue logs, profiles, or knowledge bases test access to history, but usually cannot establish that evaluation formed a new capability.Such benchmarks diagnose long-horizon access rather than learning during evaluation.Source: §4.2.1 · p. 19
• Evidence grows stronger from fixed history, to continually written state, to feedback that improves later behavior under matched and sequential controls.Procedural, semantic or personalization, and dynamic-rule streams test different capabilities, so success on one does not validate the others.Source: §4.2.1 · p. 19

4.2.2 Matched Comparisons Reveal What Memory Contributes

• Matched stateful-versus-reset comparisons isolate accumulated experience, while controls address alternative explanations such as the backbone, longer context, static retrieval, or extra inference.The stateful condition retains experience; the reset condition removes experience-related state on a matched task.Source: §4.2.2 · p. 19

4.2 How to Measure: Protocols for Experience, Controls, and Time

• A positive ΔP_t indicates additional benefit associated with accumulated state under matched conditions, not a new composite metric.No-memory or reset, full-history, static RAG, and matched-compute baselines test alternative explanations.Source: §4.2 · p. 20
• For parametric memory, reset must distinguish online parameter updates, persistent-state preservation, and subsequent readout; a positive ΔP_t may bundle several mechanisms.A stronger design matches backbone, task input, context, retrieval, and inference budgets while varying experience-related state.Source: §4.2 · pp. 20–21

4.2.3 Sequential Evaluation Separates Durable Learning from Transient Success

• Endpoint scores after long histories show information can be used, but not when its benefit appeared or whether it disappeared and recovered.Probing before acquisition, immediately afterward, and after a controlled update offers stronger evidence than a single terminal score.Source: §4.2.3 · p. 21
• Interventional panels combine reprobes, an event ledger, and matched stateful–reset trajectories; probes can test retention, adaptation, and transfer.The ledger can record elapsed time, writes, overlap, conflicts, feedback, tool or rule changes, and retrieval or update cost.Source: §4.2.3 · p. 21
• The blueprint re-evaluates capability i after update t in R_t,i: diagonal entries measure acquisition, lower-triangular entries expose retention and backward influence, and held-out variants test transfer.It pairs pre- and post-acquisition anchors and delay–interference checkpoints with matched budgets and an event ledger.Source: §4.2.3 · p. 21

4.2.4 Fragmented Benchmarks Limit Cross-Study Conclusions

• Numerical comparisons across papers are unreliable unless protocols specify behavioral outcomes, experience regimes, and matched and sequential controls.Access histories, incremental ingestion, and sequential interactive tasks measure different constructs and often differ in context, retrieval, or compute budgets.Source: §4.2.4 · p. 21

4.3.1 Non-Parametric Memory Scaling Law: Experience, Capacity, and Compute

• The proposed frontier relates attainable performance to cumulative experience, persistent capacity, and write, maintenance, and reading resources; its exponents and applicable ranges are empirical quantities.A capacity-only sweep can plateau when reading, experience supply, or maintenance becomes limiting, and expanding the limiting resource can permit further gains.Source: §4.3.1 · p. 22
• For independent, equiprobable binary facts, keeping average reconstruction error below δ requires at least n[1 − h2(δ)] bits; shared rules and recurring task structures can reduce capacity demand.Excessive compression can harm performance, while exceptions and rare queries may need original details.Source: §4.3.1 · pp. 23, 25
• On LoCoMo, increasing Raw RAG reading from 512 to 2,048 tokens improves F1 by 0.89 points at one-eighth history retention and 3.82 points at full retention; their paired difference is 2.93 points.On selected LongMemEval samples, full-history Raw RAG accuracy rises from 35.00% to 44.00% over the same read-budget change.Source: §4.3.1 · p. 23, Figure 8 · p. 24
• At an 8,192-token capacity limit and R=1,024, Mem0’s stateful–reset gain falls from +7.10 points after 64 training dialogues to −13.93 at 256; consolidation lowers occupancy but final gains remain negative.The capacity limit does not bind because occupancy plateaus at 3,858 tokens.Source: Figure 8 · p. 24, §4.3.1 · p. 25
• With equal performance, lightweight writing and direct reading may cost less under limited reuse, while costly preparation pays off when cumulative query savings exceed its added cost.On the matched LoCoMo small-bank cohort, query-time summarization reduces serialized input but increases latency, and its paired F1 is 0.72 points below full-bank reading.Source: §4.3.1 · p. 25

4.3.2 Parametric Memory Scaling Law

• Parametric writes share parameters, so new writes can overwrite earlier content; the proposed interference term stays near zero below a substrate- and policy-dependent critical load ratio, then increases.The framework predicts that fixed-capacity loss rises after the critical ratio is crossed and that load curves should approximately collapse when plotted against the ratio.Source: §4.3.2 · pp. 26–27
• With one write step, GradMem exact match falls from 100.0% at 4 pairs to 32.6% at 96, while five write steps retain 88.4% at 96 pairs.The study uses a synthetic task; existing studies vary either capacity or load, leaving the predicted collapse of load curves across capacities untested.Source: §4.3.2 · p. 28, Figure 12 · p. 29
• At rank 128 on Qwen3-8B, memorization accuracy is 40.0% for SFT, 100.0% for MemFT-OT, and 94.4% for MemFT-SW; Memory+ capacity growth also yields task-dependent gains.For Memory+, NaturalQuestions accuracy rises from 13.68 to 20.78 and TriviaQA F1 from 42.89 to 62.14 as value count grows from 1M to 64M; the layers are trained jointly during pretraining.Source: §4.3.2 · p. 28, Figure 12 · p. 29

5. Memory-Centric Continual Learning Applications Across Digital and Physical Domains

• Parametric and non-parametric memory share resource-limited scaling, but parametric memory adds fixed read budgets and load interference, favoring external storage for rapidly changing experience.Stable, frequently reused experience is a better fit for parametric memory, whose average cost advantage grows with reuse.Source: §5 · p. 30

5.1 Lifelong Personalization and Companion Agents

• Personal agents update user models to guide dialogue, planning, and tool calls; assistants track changing preferences, while tutors adapt hints and lessons to student progress.In a ten-week study involving 147 users and 1,252 calls, CareCall revisited past health complaints and life events, encouraging more disclosure and trust.Source: §5.1 · p. 31

5.2 Evolving Problem-Solving in Long-Horizon Digital Environments

• 24.6% on Mind2Web and 51.1% on WebArena: AWM raised success rates across 200 domains by distilling reusable workflows from successful web trajectories.The approach also cut overall action steps, and learned routines transferred to websites with similar structures.Source: §5.2 · p. 32
• Desktop and software agents combine execution histories with procedural skills to resume interrupted work, navigate unfamiliar interfaces, and reuse debugging strategies across codebases.Examples include verified tool scripts, executable skill documents, and selective context compression during extended debugging.Source: §5.2 · p. 32

5.3 Continuous Exploration in Embodied and Open Worlds

• Embodied agents pair world-state memory, which tracks changing surroundings, with skill memory; structured spatial and temporal records support navigation and object interaction.RememBR indexes video clips with spatial and temporal tags, while EmbodiedVideoAgent updates object states from interactions.Source: §5.3 · p. 32, §5.4 · p. 33
• 3.3× more items and tech-tree milestones 15.3× faster than baselines: Voyager saves verified executable skills that also support unseen tasks in new worlds.In text worlds, CLIN abstracts action outcomes into causal rules for future planning.Source: §5.4 · p. 33
• Unseen-task success after 500 demonstrations rose from 36% with the initial RoboCat model to 74% with the final model; a 16-robot fleet reached 95% across eight manipulation tasks.RoboCat pools experience across simulated and real arms; the fleet pools autonomous and human-teleoperated trajectories into a shared policy.Source: §5.4 · p. 33

5.4 Collaborative Evolution via Shared Multi-Agent Memory

• Shared memory supports team workflows and role assignment: structured interaction histories improved embodied success and question-answering, while selective sharing reduced duplicated work without hurting accuracy.Scientific teams also use persistent shared workspaces to carry validation results and hypotheses through iterative research cycles.Source: §5.4 · p. 33

6.1 Predicting Experience Value and Long-Term Credit Assignment

• Delayed credit assignment makes memory-policy learning difficult: a write's value may emerge only after many later summarization, pruning, and task-shift steps.Feedback changes memory content while the management policy remains fixed, and the paper says there is no principled way to balance experience value against storage, search, and confusion costs.Source: §6.1 · p. 34

6.2 The Stability–Plasticity Dilemma in Dynamic Environments

• Memory policies must predict future utility from local clues, trace failures across admission, summarization, retrieval, and reasoning, and train despite path-dependent memory states.Proposed approaches include task-stream value models, source-linked summaries and ablations, and branching rollouts from identical memory checkpoints.Source: §6.2 · p. 35
• Loose writing increases plasticity but can clutter stores with duplicates and conflicts, crowding useful records out under fixed retrieval budgets and increasing retrieval noise as memory grows.The paper recommends evaluating task gains under fixed compute budgets, tracking retention, transfer, and interference against frozen-memory baselines.Source: §6.2 · p. 35

6.3 Synergistic Continual Learning through Experience Memory and Parametric Learning

• External records and model weights exchange knowledge mostly one way, while parametric knowledge is difficult to edit, roll back, or delete once written.The proposed bridge tests and summarizes experiences externally before consolidating verified patterns into weights, then reopens outdated knowledge for correction.Source: §6.3 · p. 36

6.4 Trustworthy Memory Governance in Lifelong Deployment

• Memory safety risks span formation, consolidation, retrieval, and forgetting: poisoned inputs can become trusted rules, corrupted records can hijack tools, and deletion may leave downstream traces.Governance also faces shifting attack surfaces, delayed impacts that hinder source tracing, and privacy checks or audit logs that slow search and raise token costs.Source: §6.4 · p. 36

7. Conclusion

• Governance must become a testable, built-in guarantee of agent systems rather than an afterthought.The paper identifies direct trade-offs: strict privacy checks and audit logs slow search and increase token costs.Source: §7 · p. 37
• Governance should cover the full lifecycle, with source and permission checks, preserved provenance during consolidation, sandboxed retrieval, and measurable deletion and recovery targets.The paper notes that task-success-only benchmarks provide little incentive to build safe memory systems.Source: §7 · p. 37
• The paper reframes memory as an evolving learning infrastructure and proposes a framework spanning substrates, reusable assets, lifecycle policies, behavior-centered evaluation, and memory scaling.Its central question shifts from whether an agent can remember to whether it can continually learn.Source: §7 · p. 37