PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on the Memory Mechanism of Large Language Model based Agents

Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyu Dai, Jieming Zhu, Zhenhua Dong, Ji-Rong Wen

arXiv:2404.13501v1cs.AI

TL;DR

LLM-based agents need memory to support long-term interaction and self-evolution, but existing memory mechanisms are scattered across studies without a holistic synthesis. This survey defines and motivates agent memory, reviews its design and evaluation, presents applications, and analyzes limitations and future directions. Its central outcome is a systematic review with taxonomies and insights spanning the memory mechanism of LLM-based agents.

  • Problem

    Existing LLM-agent memory mechanisms are scattered across papers, lacking a systematic holistic review of their definitions, purposes, designs, evaluations, and common patterns.

  • Method

    The survey synthesizes agent memory research through definitions, necessity analysis, design dimensions, evaluation approaches, applications, limitations, and future directions.

  • Results

    The survey provides systematic taxonomies and insights into memory sources, forms, operations, evaluations, and applications in LLM-based agents.

  • Takeaways & Limitations

    The synthesis offers references for newcomers and aims to inspire more advanced memory mechanisms for LLM-based agents.

  • Takeaways & Limitations

    Recency-based caching can miss crucial information from distant memories in long-term tasks.

Abstract

from arXiv · show

Large language model (LLM) based agents have recently attracted much attention from the research and industry communities. Compared with original LLMs, LLM-based agents are featured in their self-evolving capability, which is the basis for solving real-world problems that need long-term and complex agent-environment interactions. The key component to support agent-environment interactions is the memory of the agents. While previous studies have proposed many promising memory mechanisms, they are scattered in different papers, and there lacks a systematical review to summarize and compare these works from a holistic perspective, failing to abstract common and effective designing patterns for inspiring future studies. To bridge this gap, in this paper, we propose a comprehensive survey on the memory mechanism of LLM-based agents. In specific, we first discuss ''what is'' and ''why do we need'' the memory in LLM-based agents. Then, we systematically review previous studies on how to design and evaluate the memory module. In addition, we also present many agent applications, where the memory module plays an important role. At last, we analyze the limitations of existing work and show important future directions. To keep up with the latest advances in this field, we create a repository at \url{https://github.com/nuster1128/LLM_Agent_Memory_Survey}.

1 Introduction

LLM-based agents extend original LLMs through interaction with environments and self-evolution, with memory serving as a key component for accumulating and using experience. This survey addresses the lack of a holistic review by defining, motivating, designing, evaluating, and applying agent memory.

  • Motivation: LLM-based agents must interact with environments, observe responses, and adjust actions to pursue complex real-world goals and self-evolution.Examples include ticket booking and personalized assistance based on user feedback.
  • Role of Memory: Memory differentiates LLM-based agents from original LLMs by supporting knowledge accumulation, historical-experience processing, and informative retrieval for action.Prior work explores in-trial and cross-trial information, natural-language storage, and memory reading and writing operations.
  • Survey Gap: Existing memory studies are promising but scattered, motivating a systemic review from a holistic perspective.The survey organizes the field around what memory is, why it is needed, and how to implement and evaluate it.
  • Contributions: The survey contributes formal definitions, an analysis of memory’s necessity, taxonomies for design and evaluation, application examples, and limitations with future directions.It presents memory design through sources, forms, and operations, and discusses direct and indirect evaluation approaches.
  • Organization: The paper is organized around related surveys, memory concepts and necessity, design and evaluation, applications, limitations, and future directions.These topics are covered across Sections 2 through 9.

2 Related Surveys

The paper positions itself within surveys of LLMs and LLM-based agents while focusing specifically on the memory mechanism. It addresses a fundamental agent problem not covered holistically by prior surveys and aims to provide taxonomies and guidance for future work.

  • Surveys on Large Language Models: Prior LLM surveys cover broad foundations, evolution, architectures, training, evaluation, and specialized topics such as alignment, retrieval, tools, knowledge editing, and long context.These surveys use different taxonomies and perspectives on LLMs.
  • Surveys on Large Language Models: LLM surveys also address evaluation, applications, and trustworthiness challenges including hallucination, bias, unfairness, explainability, security, and privacy.Evaluation surveys cover tasks, methods, benchmarks, and capabilities beyond performance.
  • Survey Landscape: Figure 2 organizes related surveys on LLMs and LLM-based agents.The caption identifies the figure as an overview of the survey landscape.
  • Surveys on LLM-based Agents: Related surveys on LLM-based agents summarize agent construction, applications, and evaluation from broader perspectives.The cited agent surveys establish the surrounding research context for this work.
  • Position of This Work: This survey focuses on the agent’s memory mechanism as a fundamental problem and presents the first survey in this direction.It aims to inspire advanced memory architectures and provide newcomers with comprehensive starting materials.

3 What is the Memory of LLM-based Agent

The paper defines agent memory through the interaction process and distinguishes narrow, within-trial memory from broader memory incorporating cross-trial information and external knowledge. It organizes memory-assisted interaction into writing, management, and reading operations.

  • LLM-based agents perceive environmental information, store it in memory, process it, and use the processed information to choose subsequent actions.
  • A trial is a complete agent-environment interaction process, while each trial contains multiple steps represented by actions and observations.
  • In the narrow sense, memory is derived from historical information within the current trial before the next action.
  • In the broad sense, memory combines current-trial history, information from previous trials or tasks, and external knowledge.
  • Memory Writing: Memory writing projects raw action-observation pairs into concise stored contents, which may be expressed in natural language or parametric representations.
  • Memory Management and Reading: Memory management processes stored information into more effective forms, while memory reading retrieves memory in the context of the next action.

4 Why We Need the Memory in LLM-based Agent

Memory is presented as necessary for LLM-based agents because it supports human-like cognition, self-evolution in dynamic environments, and continuity across applications. These perspectives connect memory with learning from experience and maintaining context for action.

  • Cognitive Psychology: Memory provides a cognitive basis for learning, social behavior, perception, reasoning, and other human mental processes.
  • Cognitive Psychology: Because agents aim to behave like humans, human memory theories and architectures provide important guidance for designing agent memory modules.
  • Self-Evolution: In dynamic environments, memory supports self-evolution by accumulating past errors, inappropriate behaviors, and failed experiences for future tasks.
  • Agent Applications: Memory preserves conversation context for conversational agents and role profiles for simulation agents, enabling continuity across interactions.
  • Together, cognition, self-evolution, and applications motivate memory as a necessary component of LLM-based agents.

5 How to Implement the Memory of LLM-based Agent

The survey reviews memory implementation through three complementary perspectives: where memory contents originate, how they are represented, and how they are processed.

  • Memory sources describe where contents come from, memory forms describe how contents are represented, and memory operations describe how contents are processed.

5.1 Memory Sources

The survey classifies memory sources into inside-trial information, cross-trial information, and external knowledge. These sources support immediate task handling, experience accumulation, and access to information beyond the interaction environment.

  • Memory contents come from information inside a trial, information across different trials, or external knowledge outside the agent-environment loop.
  • Inside-trial Information: Historical steps within a trial are widely used because they are highly relevant to the agent’s future actions.
  • Inside-trial Information: Relying only on inside-trial information can limit accumulation of knowledge across tasks and learning of more generalizable information.
  • Cross-trial Information: Cross-trial memory commonly stores successful and failed actions together with insights such as failure reasons and recurring patterns for success.
  • Cross-trial Information: Cross-trial experiences function as longer-term memory that lets agents adjust actions using feedback from multiple trials.
  • External Knowledge: External knowledge can be drawn from private or public sources, often through APIs, and can provide information beyond what interaction alone can yield.
  • External Knowledge: Integrating external knowledge expands agents’ knowledge boundaries with information described as unlimited, up-to-date, and well-founded for decision-making.

5.2 Memory Forms

Memory contents are represented mainly in textual or parametric form, each with distinct trade-offs in effectiveness, efficiency, interpretability, and application suitability.

  • 5.2 Memory Forms: Memory contents take textual or parametric form: textual memory is explicitly retained in language, whereas parametric memory is encoded into model parameters.Textual form is the mainstream representation and includes both unstructured natural language and structured forms such as tuples and databases.
  • 5.2.1 Memory in Textual Form: Textual memory can store complete interactions, recent interactions, retrieved interactions, or external knowledge acquired through tools.These strategies differ in whether they preserve all history, prioritize recency, select relevant content, or incorporate information from sources such as APIs.
  • 5.2.1 Memory in Textual Form: Complete-interaction storage preserves comprehensive information but increases computation and latency, can require truncation, and may reduce inference robustness.Attention cost grows quadratically with sequence length, while truncation can cause information loss and positional effects can make memory use unstable.
  • 5.2.1 Memory in Textual Form: Recent-interaction caching improves efficiency and focus but can omit earlier information that remains crucial in long-term tasks.The method maintains a recent memory window, potentially excluding important distant memories.
  • 5.2.2 Memory in Parametric Form: Parametric fine-tuning adapts agents to specialized domains but risks overfitting, catastrophic forgetting, high cost, data requirements, and poor suitability for frequent online updates.Because backpropagation at every interaction step is unaffordable, most fine-tuning approaches target offline scenarios.
  • 5.2.2 Memory in Parametric Form: Knowledge editing updates targeted facts more efficiently while protecting non-targeted knowledge, but meta-training costs and preservation of unrelated memories remain challenges.The approach supports high-precision, real-time parameter updates, although unrelated-memory preservation is not guaranteed.
  • 5.2.3 Advantages and Disadvantages of Textual and Parametric Memory: Textual memory is more comprehensive and explainable and writes more efficiently, whereas parametric memory avoids prompt-length limits and reads more efficiently but may lose information during parameterization.Textual memory incurs context-integration costs during inference, while parametric memory requires more costly writing and complex training.
  • 5.2.3 Advantages and Disadvantages of Textual and Parametric Memory: Textual memory suits recent-interaction and context-specific tasks, while parametric memory can better support large-memory or well-established-knowledge tasks.The distinction follows the reported trade-offs between prompt capacity, information density, reading and writing efficiency, and interpretability.

5.3 Memory Operations

The survey organizes agent memory into writing, management, and reading operations that jointly process information for LLM inference, and it summarizes how these operations are designed and evaluated.

  • 5.3 Memory Operations: Memory writing, management, and reading are three collaborating operations that implement the agent’s memory function.Together they process perceived information and provide usable information for subsequent LLM inference.
  • 5.3.1 Memory Writing: Memory writing stores selected perceived information, using either raw observations or summaries of those observations.The central design issue is identifying which information is essential to retain.
  • 5.3.1 Memory Writing: Writing strategies include structured extraction, controller-based operation selection, and self-directed updates by the agent.TiM stores entity relations in grouped database entries, SCM uses a memory controller, and MemGPT lets agents autonomously update memory.
  • 5.3.1 Memory Writing: Effective writing requires extracting and representing useful information from lengthy, noisy observations and environment-specific feedback.The survey identifies information-extraction strategy as vital because environments provide different forms of feedback.
  • 5.3.2 Memory Management: Memory management abstracts and maintains stored information through reflection, redundancy merging, and forgetting unimportant early memories.Examples include daily-event summaries, personality insights, and memory refinement based on environmental feedback.
  • 5.3.3 Memory Reading: Memory reading retrieves information relevant to the current state for reasoning and decision-making, requiring task-oriented selection among many memory entities.Reported implementations include SQL-generated memory chains, memory-pool retrieval, and vector-store selection of successful trajectories.
  • 5.3.3 Memory Reading: For textual memory, reading commonly uses text similarity and auxiliary information, whereas parametric memory is read implicitly through updated model parameters.The reading method is therefore coupled to how information was written and represented.
  • 5.3.4 Memory Evaluation: Evaluation methods divide into direct evaluation of memory capability and indirect evaluation through end-to-end agent tasks.The survey presents this framework to organize diverse evaluation strategies across applications.

6 How to Evaluate the Memory in LLM-based Agent

The survey organizes memory evaluation into direct and indirect strategies. Direct evaluation measures memory independently through subjective or objective criteria, including effectiveness and efficiency.

  • Two broad strategies evaluate memory: direct evaluation measures the module independently, while indirect evaluation infers effectiveness from task completion.
  • Direct Evaluation: Direct evaluation divides into subjective human judgments and objective numerical metrics.Subjective evaluation is useful without objective ground truths, whereas objective evaluation supports numerical comparison.
  • Direct Evaluation: Subjective evaluation commonly assesses whether recalled memories are coherent with context and rational for the question.Examples include retrieving travel preferences for a travel plan and recalling that the Summer Palace is in Beijing.
  • Direct Evaluation: Subjective labeling requires suitable and diverse evaluators, with choices between absolute scoring and pairwise comparison and careful rating granularity.Absolute scores provide quantitative results, while pairwise comparisons can reduce independent-scoring noise.
  • Direct Evaluation: Objective evaluation measures effectiveness with result correctness and reference accuracy, and evaluates efficiency through operation time and hardware cost.Result correctness compares agent answers with ground truth; reference accuracy evaluates retrieved supporting memories, often with F1-score.
  • Objective Metrics: Result correctness compares each generated answer with ground truth using matching accuracy over N problems.The matching function returns 1 when the generated answer equals the ground truth and 0 otherwise.
  • Direct Evaluation: Objective evaluation provides numeric comparisons that help benchmark memory systems and promote future development.

6.2 Indirect Evaluation

Indirect evaluation tests memory through downstream tasks rather than isolating the memory module. The survey covers conversation, multi-source question-answering, long-context tasks, success rates, and ablation studies.

  • Indirect evaluation treats successful completion of memory-dependent tasks as evidence that the memory module is effective.
  • Conversation: Conversation tasks evaluate memory through consistency and engagement, while stored context supports personalized interactions.
  • Multi-source Question-answering: Multi-source question-answering tests integration of inside-trial, cross-trial, and external information.These tasks can expose contradictions between sources and problems caused by knowledge updates.
  • Long-context Applications: Long-context evaluation assesses memory functions in practical downstream scenarios through long-context understanding and retrieval benchmarks.
  • Other Metrics: Success rate measures the proportion of tasks agents solve successfully, including spatial tasks, Minecraft item production, and code-generation problems.
  • Other Metrics: Ablation studies compare agent performance with and without memory modules to assess memory significance in specific scenarios.

6.3 Discussions

Indirect evaluation is easier to conduct using public benchmarks, but task performance reflects multiple factors rather than memory alone. Direct evaluation isolates memory more reliably, although open-source evaluation resources remain limited.

  • Task performance can be biased because downstream outcomes depend on multiple factors, with memory being only one of them.
  • Direct evaluation improves reliability by assessing memory independently, but the survey notes a lack of open-sourced resources.

7 Memory-enhanced Agent Applications

Memory-enhanced agents span role-play, social simulation, personal assistance, games, code generation, recommendation, and domain-specific expert systems. Across applications, memory stores context, preferences, experiences, or specialized knowledge, while each setting introduces distinct design constraints.

  • Role-playing and Social Simulation: Role-play and social simulation require memories consistent with role characteristics and influential enough to maintain realistic, rational, and diverse behavior.Humanoid agents may also need forgetting and long- and short-term memory aligned with cognitive psychology.
  • Personal Assistant: Personal assistants commonly retrieve textual memories of factual information and user styles to produce relevant, consistent, personalized responses.
  • Open-world Game: Game and open-world agents store observations and successful experiences so later exploration can avoid repeated mistakes and use external knowledge.
  • Code Generation: Code-generation agents use external resources and past experiences to improve development knowledge, continuity, contextual understanding, and iterative optimization.
  • Recommendation: Recommendation agents retain personalized information and feedback, but aligning these data with LLMs remains a challenge linking conventional recommenders and LLMs.
  • Expert System in Specific Domains: Domain-specific expert agents must retain specialized knowledge, while accuracy, staleness, partial updating, and large memory volume complicate storage and retrieval.

8 Limitations & Future Directions

Future memory research spans more efficient and interpretable parametric memory, coordinated multi-agent systems, lifelong learning, and cognitively aligned humanoid agents.

  • 8.1 More Advances in Parametric Memory: Textual memory is interpretable and easy to edit, whereas parametric memory offers higher information density but lower interpretability.
  • 8.1 More Advances in Parametric Memory: Parametric memory remains constrained by the costly transformation of text into parameters and difficulties establishing credibility and interpretability.SFT requires extensive corpora and is unsuitable for situational knowledge; meta-learning is presented as one possible approach.
  • 8.2 Memory in LLM-based Multi-agent Applications: Multi-agent memory research emphasizes synchronization to establish unified knowledge and consistent decision-making, alongside improved communication and information management.
  • 8.3 Memory in Lifelong Learning: Agent memory supports lifelong learning by storing and applying past observations during continuous environmental interaction.
  • 8.4 Memory in Humanoid Agents: Humanoid-agent memory should align with human cognitive processes and psychological principles rather than simply maximizing task performance.

9 Conclusion

The survey systematically reviews LLM-agent memory by addressing what memory is, why it is needed, and how it can be designed and evaluated. It also presents applications to demonstrate memory’s role and aims to inform future mechanisms.

  • The survey organizes memory research around its definition, necessity, design and evaluation, applications, limitations, and future directions.
Loading 2404.13501v1…