PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents

Wei-Chieh Huang, Weizhi Zhang, Yueqing Liang, Yuanchen Bei, Yankai Chen, Tao Feng, Xinyu Pan, Zhen Tan, Yu Wang, Tianxin Wei, Shanglin Wu, Ruiyao Xu, Liangwei Yang, Rui Yang, Wooseong Yang, Chin-Yuan Yeh, Hanrong Zhang, Haozhen Zhang, Siqi Zhu, Henry Peng Zou, Wanjia Zhao, Song Wang, Wujiang Xu, Zixuan Ke, Zheng Hui, Dawei Li, Yaozu Wu, Langzhou He, Chen Wang, Xiongxiao Xu, Baixiang Huang, Juntao Tan, Shelby Heinecke, Huan Wang, Caiming Xiong, Ahmed A. Metwally, Jun Yan, Chen-Yu Lee, Hanqing Zeng, Yinglong Xia, Xiaokai Wei, Ali Payani, Yu Wang, Haitong Ma, Wenya Wang, Chenguang Wang, Yu Zhang, Xin Eric Wang, Yongfeng Zhang, Jiaxuan You, Hanghang Tong, Xiao Luo, Xue Liu, Yizhou Sun, Wei Wang, Julian McAuley, James Zou, Jiawei Han, Philip S. Yu, Kai Shu

arXiv:2602.06052v4cs.CLcs.AI

TL;DR

Long-horizon, interactive, and user-dependent tasks expose gaps between benchmark performance and real-world utility, while agents must manage information beyond fixed context windows. This survey unifies foundation-agent memory across substrate, cognitive mechanism, and subject, and synthesizes its evaluation, infrastructure, and self-evolving directions.

  • Problem

    Existing evaluations often use short, isolated tasks and simplified assumptions, while agents in real-world settings must manage accumulated information across long-horizon interactions.

  • Method

    The survey classifies agent memory by internal or external substrate, sensory-to-procedural cognitive mechanism, and user-centric or agent-centric subject, while reviewing system topologies, operations, benchmarks, and metrics.

  • Results

    The survey selects 218 key articles from 2023 Q1 through 2025 Q4 and identifies sharply increasing research activity throughout 2025, especially in Q4.

  • Takeaways & Limitations

    Memory is presented as a key component for agents operating in long-horizon, context-exploding, and user-dependent environments, with portable and shareable skills emerging as a central self-evolution challenge.

  • Takeaways & Limitations

    Persistent user memory remains constrained by privacy, security, and data-governance risks, including targeted extraction and exposure of stored private data.

Abstract

from arXiv · show

Research in artificial intelligence is shifting from model innovations and benchmark scores towards problem definition and rigorous real-world evaluation. As the field enters the "second half," the central challenge becomes real utility in long-horizon, dynamic, and user-dependent settings such as agentic coding, deep research, and computer use, where LLM-based agents face context explosion beyond fixed context windows and must continuously accumulate, manage, and selectively reuse information across extended interactions. Memory, with hundreds of papers released in 2025, therefore emerges as the critical solution to fill this utility gap. Beyond passive storage, memory is increasingly the substrate through which agents self-evolve: short-term memory gates which experiences are perceived and abstracted during execution, while long-term memory consolidates them into reusable knowledge and skills, forming the loop through which agents improve from their own experience. In this survey, we provide a unified view of foundation agent memory along three dimensions: memory substrate (internal parametric state and external retrieval-augmented stores), cognitive mechanism (sensory, working, episodic, semantic, and procedural), and memory subject (user-centric personalization and agent-centric experience). We then analyze how memory is operated under single- and multi-agent topologies and highlight learning policies over memory operations, showing how memory management itself is becoming a trainable capability spanning reinforcement-learned context curation, experience consolidation at decision time, and the emerging ecosystem of portable, shareable agent skills. Finally, we review evaluation benchmarks and metrics for memory utility, and outline open challenges and future directions.

1 Introduction

AI research is shifting toward real-world utility in long-horizon, dynamic, and user-dependent settings, where fixed-context agents need memory to accumulate and reuse experience. This survey organizes foundation agent memory by substrate, cognitive mechanism, and subject, then examines its operation, learning policies, scalability, evaluation, and open challenges.

  • 15?
  • Memory architectures are evolving from static prompt-based mechanisms toward self-adaptive units that store, load, summarize, and reuse expanding interaction histories.
  • Working-memory curation increasingly uses learned policies, while episodic experiences are consolidated into semantic facts and procedural skills, creating a self-evolution loop.
  • The survey classifies memory by substrate, cognitive mechanism, and subject, covering internal versus external memory, five functional types, and user-centric versus agent-centric perspectives.
  • The survey further analyzes memory operations across single- and multi-agent topologies, scalability across changing environments, evaluation metrics, and six open challenges.

2 Background

Foundation agents use foundation models as decision modules augmented with state estimation, action execution, and memory management. Memory supports long-horizon interaction by recording, consolidating, and reusing experience, while agents face challenges in reliability, evaluation, alignment, and safety.

  • Agents operate in a loop that interprets instructions, selects actions, observes outcomes, and updates internal state or memory.
  • Foundation agents are autonomous or semi-autonomous systems driven by foundation models and augmented with state estimation, action execution, and memory management.
  • Memory enables agents to retain short-term and long-term information, while summarization, reflection, and consolidation compress experience into reusable knowledge.
  • Foundation agents are explored in workflow automation, tutoring, web and GUI interaction, embodied control, and scientific assistance, but require stronger long-horizon reliability, dynamic evaluation, alignment, and safety.

3 Taxonomy of Memory in Foundation Agents

The survey organizes foundation-agent memory as a system of complementary substrates and storage structures, enabling agents to retain, update, and retrieve knowledge across long-horizon interactions.

  • Taxonomy: The survey categorizes memory along substrate, cognitive mechanism, and subject dimensions.Substrates are further divided into external and internal memory.
  • External Memory: External memory stores experience outside model parameters and supports scalable, editable, cross-session retention through retrieval and update operations.Its implementations include vector indexes, text records, structural stores, and hierarchical stores.
  • External Memory: External memory requires management because retrieval latency, unreliable ranking, and accumulated low-quality items can degrade decision-making.Recommended controls include summarization, selective retention, forgetting, deduplication, and pruning.
  • External Memory: Vector indexes retrieve top-k embedded items through approximate nearest-neighbor search and append them to the prompt as grounding context.This is the dominant external-memory implementation in current agent and LLM work.
  • External Memory: Text records, structural stores, and hierarchical stores trade transparency, modularity, and scalability against summarization, coordination, latency, and consistency costs.Hierarchical stores separate persistent, episodic, semantic, and procedural information into specialized modules.

Internal Memory

Internal memory retains information within model parameters or transient inference states, offering fast integration but imposing update, capacity, persistence, and context-budget constraints.

  • Internal Memory: Internal memory includes parametric knowledge stored in model weights and working states used during inference.The survey distinguishes weight, latent-state, and KV-cache forms.
  • Weight Memory: Weight memory embeds facts or experiences directly into parameters through pre-training, post-training, parameter editing, or distillation.Calibration slots can store corrected facts without altering original parameters.
  • Latent-State Memory: Latent-state memory carries intermediate hidden states across steps or segments so earlier activations influence later computation after raw tokens leave the window.It represents information through hidden-state tensors produced during forward passes.
  • KV Cache: KV caching reuses prior attention keys and values to accelerate autoregressive decoding, especially for long outputs and large models.The trade-off is higher memory usage and implementation complexity.
  • Trade-offs: Internal weights favor stable general-purpose knowledge, while latent states and KV caches suit fast within-session reasoning but cannot support cross-session retention.Latent substrates also consume context budget and scale in storage cost with model depth, batch size, and precision.
  • Hybrid Design: Because no substrate dominates across settings, effective systems increasingly combine parametric, latent, and external memory.The hybrid pattern assigns inherent knowledge, short-term reasoning, and scalable experience storage to different substrates.

3.2 Memory Cognitive Mechanisms

The survey uses five interacting cognitive memory systems to analyze how foundation agents perceive, maintain, and reuse information across short- and long-term processes.

  • Cognitive Taxonomy: The taxonomy defines sensory, working, episodic, semantic, and procedural memory as five atomic systems for foundation agents.Other constructs, such as autobiographical and prospective memory, are treated as compositions or abstractions over these systems.
  • Sensory Memory: Sensory memory briefly retains raw perceptual signals so attention and selection can determine what receives higher-level processing.It functions as a transient interface between perception and cognition.
  • Sensory Memory: Sensory memory is rarely modeled explicitly in current LLM agents because textual inputs already collapse perceptual processing into symbolic representations.Multimodal and embodied agents provide settings where analogous short-lived mechanisms become relevant.
  • Sensory Memory: Hierarchical and multi-stage systems such as HMT, LightMem, and M2PA use sensory memory as an initial buffer before selection, compression, or consolidation.These systems provide explicit implementations of a stage that is otherwise often implicit.
  • Sensory Memory: Embodied agents may need sensory memory to process continuous, high-bandwidth video, audio, and proprioceptive streams under real-time and memory constraints.This makes sensory memory increasingly relevant beyond text-based agents.

Working Memory

Working memory is the agent’s capacity-constrained online workspace: it maintains and manipulates task-relevant state across interaction steps. It is increasingly managed as a learned gateway that selects and transforms experiences for later consolidation.

  • Working memory temporarily stores and actively manipulates information needed for reasoning, comprehension, learning, and ongoing operations.
  • In LLM agents, working memory carries and updates task-relevant state because the models are inherently stateless.The goal is to support multi-step reasoning and action under strict online capacity constraints.
  • Working memory is commonly realized through active context containing prompts, reasoning traces, tool outputs, and runtime states such as key-value caches.
  • Existing approaches compress, restructure, or abstract state before context entry, or maintain, update, and evict accumulated state during execution.
  • Long-horizon progress depends on principled state selection and transformation rather than unbounded context expansion.
  • Learned working-memory policies select and transform short-term state before it is consolidated into episodic, semantic, or procedural memory.This makes working memory the gateway through which agents can learn from their own experience.

Episodic Memory

Episodic memory preserves concrete, context-rich interaction experiences across sessions and selectively retrieves them during decisions. These episodes can also be reflected on and consolidated into semantic facts and procedural skills, while long-term retention and influence remain open problems.

  • Episodic memory stores specific interactive events across sessions, including temporal and environmental context, trajectories, action sequences, and feedback.
  • Its primary role is preserving historical contexts and outcomes so agents can reconstruct backgrounds and maintain continuity across dynamic, extended interactions.
  • An explicit experience repository is the common implementation, with research focused on recording episodes and retrieving or reflecting on them.
  • Accumulated episodes provide the experiential substrate for self-evolution by being selectively accessed at decision time and distilled into semantic facts and procedural skills.
  • Open problems include defining episode boundaries, regulating episodic recall during reasoning, and managing retention as episodic memory scales.
  • Related long-term memory: Semantic memory complements episodic memory by consolidating fragmented experiences into stable, reusable, decontextualized facts and concepts.

Procedural Memory

Procedural memory captures reusable ways of performing tasks, from explicit templates and strategies to learned neural policies and portable skills. Its externalization as shareable artifacts broadens reuse while creating governance and self-maintenance challenges.

  • Procedural memory encodes operational skills, execution strategies, and routines by abstracting complex action sequences into reusable patterns.
  • In agent architectures, procedural knowledge shapes execution through action sequences, decision rules, and workflows distilled from prior executions or learned through optimization.
  • Procedural memory is evolving from explicit non-parametric templates toward implicit parametric neural policies through experience distillation and metacognitive control.
  • Explicit skill libraries package verified programs, instructions, code, and resources as composable artifacts that agents can load, inspect, share, and reuse.
  • Open challenge: Self-evolving procedural memory must autonomously induce, revise, merge, repair, retire, and govern skills rather than rely solely on human curation.
  • Procedural memory provides reusable action abstractions that support skill acquisition and efficient execution of complex decisions over long horizons.
  • Memory subjects: Figure 5 organizes papers by five cognitive mechanisms and two memory subjects, with cluster area indicating paper count and horizontal position indicating publication time.

Memory Management in Long-Horizon Dialogues.

Long-horizon dialogue requires persistent user memory because finite context windows cannot reliably carry multi-session histories without noise accumulation and topic drift. Such memory supports evolving personalization but introduces substantial privacy, security, and governance demands.

  • High-fidelity user simulators provide scalable, reproducible, and controlled environments for training and evaluating interactive systems while reducing privacy and live-study costs.
  • Long-term personalization optimizes one user’s experience across days, months, or years by maintaining a persistent profile or personal knowledge base.
  • Personalized agents must reconcile historical evidence with new interactions, distinguish persistent preferences from temporary states, and revise user knowledge over time.
  • Risks: Persistent user memory creates privacy, security, and data-governance demands because stored or retrieved sensitive attributes can be exposed through memorization and context leakage.
  • Agent-centric memory: Agent-centric memory stores distilled knowledge, skills, and task priors from an agent’s execution history or environment interaction to support long-running tasks.
  • Agent-centric memory: Agent-centric memory enables self-evolution by distilling successful and failed trajectories into knowledge, strategies, and executable skills transferable to unseen tasks.
  • Long-horizon demands: Coding, web navigation, multi-turn decisions, and sequential tool use can require hundreds or thousands of steps, overwhelming immediate working memory and motivating externalized storage and retrieval.

Long-Horizon Task Execution.

Agent-centric memory retains rare domain insights and distills interaction experience into reusable abstractions and procedural skills. These memories support continual improvement, cross-task generalization, and strategy evolution through repeated execution.

  • Agent-centric experience memory retains rare insights and domain-specific heuristics for reuse when similar cases arise.
  • Long-term memory supports continual improvement by accumulating durable knowledge across tasks and episodes without catastrophic forgetting.
  • Cross-task generalization: Cross-task memory distills trajectories into task-agnostic abstractions and reusable skill primitives that generalize across environments and objectives.
  • Continual strategy and skill evolution: Continual strategy and skill evolution creates, refines, composes, and maintains environment-grounded procedural memories through repeated execution.

4 Memory Operation Mechanism

Memory operation mechanisms actively construct, organize, retrieve, revise, compress, and remove information throughout long-horizon interaction. Single-agent systems use core lifecycle operations, while multi-agent systems add architecture, routing, permissions, and conflict controls for coordinated sharing.

  • Single-agent operations: Single-agent memory uses storage and indexing, loading and retrieval, updating and refreshing, compression and summarization, and forgetting and retention.
  • Storage and retrieval: Indexing associates entries with semantic embeddings and metadata, while loading filters candidates before similarity-based retrieval supplies task-relevant memories.
  • Updating and compression: Updates revise incomplete or outdated entries, whereas compression and summarization convert episodic records into compact, abstract representations that reduce redundancy.
  • Forgetting and retention: Forgetting reduces obsolete or low-utility information through heuristic or learned removal policies, while retention preserves memories prioritized for future reasoning.
  • Multi-agent operations: Multi-agent systems combine private memories or shared workspaces with architecture, routing, permissions, and conflict controls to coordinate access and reduce inconsistency.

Learning Policy

Learning policies make memory management a learned capability rather than a fixed heuristic. They govern what and when agents store, how memories are represented, and when or where they retrieve or discard them through prompting, fine-tuning, and reinforcement learning.

  • Learning policies determine what to store, when to store it, how to represent it, and when and where to retrieve or discard memories.
  • Prompting, fine-tuning, and reinforcement learning progressively improve memory decisions from imprecise management toward effective and accurate management.

5 Memory Evolution and Optimization Policies

The survey organizes memory evolution policies into prompting, fine-tuning, and reinforcement learning, which operate at inference time, in model parameters, or through long-term outcome optimization. Across these paradigms, the literature addresses adaptive control, policy stabilization, retrieval execution, and boundary management.

  • Prompting: Prompt-driven optimization manages memory at inference time through natural-language prompts, reflection, and summarization without modifying model parameters.
  • Static prompt-based control: Static prompt control offers interpretable and predictable fixed rules but cannot adapt to interaction feedback or distributional shifts.
  • Static prompt-based control: Static methods organize memory containers, constrain single-agent access through prompts, or assign and coordinate memory across agents using predefined roles and protocols.
  • Dynamic prompt-based control: Dynamic prompt control revises memory policies at test time through experience and feedback while leaving model parameters unchanged.
  • Dynamic prompt-based control: Dynamic approaches use reflection to correct memory policies and adapt compression, denoising, and structural reorganization under limited context budgets.
  • Fine-tuning: Supervised fine-tuning internalizes memory policies into model parameters, enabling stable and reusable memory behavior across tasks.
  • Policy stabilization and boundary control: Fine-tuning methods learn reflection, correction, suppression, and role or identity boundaries to regularize updates and prevent error accumulation, concept drift, and persona inconsistency.
  • Retrieval refinement: Parameterized retrieval policies can reduce inference-time overhead, but remain fixed after training and lack explicit credit assignment over extended decision horizons.

6 Evolving and Scaling: Memory, Contexts, and Environments

Real-world agents face context that expands across interaction horizons, environmental complexity, tools, and agents, exposing the limits of episodic, context-bound evaluation and memory. Scalable systems therefore need persistent, structured, and adaptable memory that can retrieve, update, reorganize, and share state across environments.

  • Scaling Context and Environments: Real-world context scales across interaction horizons, environmental complexity, and the number of environments, tools, and agents.Unlike static evaluations, deployment context also changes as users, environments, and tools evolve.
  • Context-Limited Evaluation: Existing benchmarks often use short-horizon, context-limited, episodic settings that under-specify persistent-memory demands.They may rely on frozen knowledge, fixed APIs, curated interfaces, or restricted interaction budgets.
  • Evolving User Memory: Long-horizon personalization must preserve enduring user characteristics while updating preferences and constraints when new evidence warrants revision.Fixed context windows can produce early-context forgetting and progressive context drift.
  • Accumulated Execution Memory: Multi-turn tool use accumulates reasoning traces, tool inputs, outputs, and intermediate states, creating persistent context burdens that finite windows cannot scale indefinitely.Agents should consolidate repeated tool-use experience into reusable procedures and skills rather than reproducing long traces.
  • Structured and Cross-Environment Memory: Structured artifacts and heterogeneous tool environments require memory interfaces that support schema-aware retrieval, versioning, targeted edits, access control, and durable cross-session state.Environmental change also requires revising representations and skills when schemas, interfaces, or action preconditions change.

7 Evaluation

Foundation-agent memory evaluation combines outcome, retrieval, similarity, and judge-based metrics because tasks differ in ground-truth availability and output openness. Benchmarks are organized around user-centric personalization and agent-centric abilities, with newer designs making memory operations more attributable.

  • Metrics: Memory evaluation uses accuracy-based, similarity-based, and LLM-as-a-judge metrics for tasks with different ground-truth and output conditions.Exact correctness suits clear-answer tasks, while open-ended dialogue and summarization often require alternative scoring approaches.
  • Metrics: Recall@K, MAP, and NDCG@K assess whether relevant memories are retrieved and ranked effectively.Separating retrieval or selection from generation makes performance more attributable to memory mechanisms.
  • Metrics: Similarity metrics can track surface overlap but may underestimate valid paraphrases, overestimate fluent ungrounded responses, and indirectly reflect memory performance.Likelihood-based generation metrics do not verify grounding in the correct historical evidence.
  • User-Centric Benchmarks: User-centric benchmarks assess conversational consistency, user facts and preferences, multi-session reasoning, updates, inference, and boundary handling over personalized histories.The survey defines ten user-centric memory abilities, including Fact Extraction, Multi-Session Reasoning, User Facts & Preferences, and Abstain & Boundary Handling.
  • User-Centric Benchmarks: MR and UP are the most consistently covered user-memory abilities, while newer benchmarks increasingly evaluate explicit memory records and operation-level searching.This shift improves attribution compared with similarity-based evaluations that can mask incorrect recall behind fluency or generic helpfulness.
  • Agent-Centric Benchmarks: Agent-centric benchmarks characterize temporal reasoning, state tracking, grounding, planning, tool use, multi-hop reasoning, dialogue, debugging, code generation, patching, and test-time learning.Their evaluations span answer-level Accuracy/F1, goal-based SR and GC, and execution-centric metrics such as Pass@1 and RR.

8 Applications

Agentic memory supports continuity, learning, and personalization across domains including education, research, gaming, robotics, healthcare, dialogue, workflow automation, software engineering, recommendation, and search. Across these applications, memory manages long-horizon context, accumulates experience, and supports domain-specific reasoning or adaptation.

  • Cross-Domain Role: Agentic memory is presented as a cognitive substrate for continuity, learning, and personalization rather than merely storage.The survey frames applications as connecting agents’ past experiences with future actions.
  • Education: Educational agents use memory to track learner progress, adapt instruction, and maintain pedagogical coherence across sustained personalized interactions.Without memory, interactions are treated as isolated events.
  • Scientific Research: Scientific-research agents use memory to synthesize literature, preserve provenance, reconstruct context, and maintain reasoning continuity across multi-stage investigations.Structured logs and page stores support reproducibility and transparent reasoning lineage.
  • Gaming and Simulation: Gaming and simulation systems use episodic and procedural memory for skill acquisition, exploration, social histories, reflection, and shared multi-agent state.Some systems store successful actions as executable skills or synthesize high-level insights into relationships and plans.
  • Robotics: Robotics requires compressed but sufficiently detailed memory to link visual inputs with semantic concepts and maintain spatial representations during long-horizon navigation.Periodic trajectory summarization is one approach to handling partial observability.
  • Other Applications: Healthcare, dialogue, workflow automation, software engineering, recommendation, and information search use memory for longitudinal tracking, personalization, procedural reuse, context management, and evolving knowledge synthesis.These domains include cross-session therapeutic updates, reusable workflow templates, project memory, dynamic multimodal memory, and folded web trajectories.
  • Legal and Consulting: Legal and consulting applications use memory to preserve case history, synthesize evolving statutes, profile project requirements, and support multi-step reasoning, while hallucinated memory creates high-stakes liability concerns.The survey identifies verifiable memory linked to source documents as a future direction for professional accountability.

9 Future Directions

The survey identifies six directions for making agent memory more adaptive, efficient, multimodal, personalized, collaborative, and realistic to evaluate. Across these directions, it emphasizes persistent experience, structured representations, trustworthy updates, and execution-grounded assessment.

  • Memory for Continual Learning and Self-Evolving Agents: Continual-learning agents must manage retention, compression, and forgetting across intra-task and cross-task timescales while linking semantic, episodic, and procedural memory.The survey calls for consolidation mechanisms that align memory with model parameters, mitigate forgetting, and preserve plasticity.
  • Multi-Human-Agent Memory Organization: Persistent collaboration requires social memory that records collaborator preferences, expertise, feedback patterns, and interaction outcomes.Such memory is intended to support adaptive signaling, trust calibration, and improved coordination across repeated interactions.
  • Memory Infrastructure and Efficiency: Efficient memory systems should progress from organized textual storage toward compressed latent and internalized parametric representations.The proposed trajectory prioritizes precision-oriented retrieval, compact persistent experience units, and eventually constant-sized internal memory.
  • Lifelong Personalization and Trustworthy Memory: Lifelong personalization needs dynamic memory that bridges episodic traces with preferences, habits, and long-term intents while controlling storage and retrieval costs.The survey also highlights benchmarks for preference change, long-term consistency, adaptability, and trustworthy handling of persistent user data.
  • Memory for Multimodal, Embodied, and World-Model Agents: Multimodal agents should represent sensory streams as coherent, predictive internal states and treat memory operations as controllable actions in a world model.This perspective supports proactive planning of storing, compressing, and forgetting, alongside structured spatial, object, and skill representations.
  • Real-World Benchmarking and Evaluations: Real-world evaluation should use closed-loop, longitudinal, execution-grounded settings that test persistent memory under preference drift, delayed feedback, evolving environments, and real consequences.The survey recommends measuring safe overwriting, selective forgetting, provenance, auditing, rollback, and sustained adaptation rather than static recall alone.

10 Conclusions

The survey positions memory as central infrastructure for foundation agents operating across long-horizon, context-heavy, and user-dependent settings. It organizes the field by substrate, cognitive mechanism, and subject, then connects memory operation to agent evolution, evaluation, and trustworthy deployment.

  • Foundation agent memory addresses long-horizon, context-exploding, and user-dependent environments including agentic coding, deep research, and computer use.
  • The survey unifies memory by substrate, cognitive mechanism, and subject, and analyzes operation across single- and multi-agent systems.It also reviews prompting-, fine-tuning-, and reinforcement-learning-based policies for memory evolution and optimization.
  • The paper presents memory as a substrate of agent self-evolution and closes with evaluation metrics, application domains, and six challenges for reliable, scalable, trustworthy infrastructure.
Loading 2602.06052v4…