PaperlayerView exact revision on arXiv

Source-linked AI summary

From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms

Jinghao Luo, Yuchen Tian, Chuxue Cao, Ziyang Luo, Hongzhan Lin, Kaixin Li, Chuyi Kong, Ruichao Yang, Jing Ma

arXiv:2605.06716v1cs.AIcs.CL

TL;DR

Research on LLM-agent memory lacks a clear roadmap for robust innovations. This survey organizes memory evolution into Storage, Reflection, and Experience, concluding that abstraction mechanisms support transferable behavioral experience.

  • Problem

    Research lacks a clear roadmap for robust LLM-agent memory innovations, while statelessness limits consistency across multi-step tasks and learning from prior interactions.

  • Method

    The survey formalizes memory evolution as Storage for recording trajectories, Reflection for refining records, and Experience for abstracting cross-trajectory behavioral patterns.

  • Results

    The analysis identifies active exploration and cross-trajectory abstraction as central Experience-stage mechanisms that support autonomous evolution and transferable behavioral experience.

  • Takeaways & Limitations

    Memory evolution involves increasing information density and cognitive abstraction, guiding the design of more advanced memory mechanisms for LLM agents.

  • Takeaways & Limitations

    The qualitative survey lacks comprehensive quantitative comparisons because no unified benchmark evaluates memory mechanisms across all three stages.

Abstract

from arXiv · show

Large Language Model (LLM)-based agents have fundamentally reshaped artificial intelligence by integrating external tools and planning capabilities. While memory mechanisms have emerged as the architectural cornerstone of these systems, current research remains fragmented, oscillating between operating system engineering and cognitive science. This theoretical divide prevents a unified view of technological synthesis and a coherent evolutionary perspective. To bridge this gap, this survey proposes a novel evolutionary framework for LLM agent memory mechanisms, formalizing the development process into three stages: Storage (trajectory preservation), Reflection (trajectory refinement), and Experience (trajectory abstraction). We first formally define these three stages before analyzing the three core drivers of this evolution: the necessity for long-range consistency, the challenges in dynamic environments, and the ultimate goal of continual learning. Furthermore, we specifically explore two transformative mechanisms in the frontier Experience stage: proactive exploration and cross-trajectory abstraction. By synthesizing these disparate views, this work offers robust design principles and a clear roadmap for the development of next-generation LLM agents.

1 Introduction

LLM agents extend LLMs with tools and modular components but remain limited by statelessness, which disrupts consistency and prevents learning from prior interactions. This survey addresses fragmented memory research with a Storage–Reflection–Experience evolutionary framework and identifies drivers, mechanisms, and future directions.

  • Motivation: LLM-based agents integrate external tools and modular components to enable planning, tool use, and environmental interaction.
  • Motivation: LLMs’ inherent statelessness hinders logical consistency across complex multi-step tasks, prevents learning from prior interactions, and can cause recurring reasoning errors.
  • Research gap: Existing surveys lack a unified evolutionary perspective, obscuring memory-development drivers and limiting guidance for next-generation agent architectures.Prior work is described as focusing on engineering-module classification rather than systematically explaining critical technological transformations.
  • Proposed framework: The survey formalizes memory evolution into Storage for recording interaction trajectories, Reflection for dynamically evaluating and refining records, and Experience for prospective guidance through abstraction.
  • Survey scope and outlook: Its research questions examine why memory evolves, how it progresses from Storage through Reflection to Experience, and what changes Experience introduces, alongside future directions for dynamic triggering, working memory, datasets, and distributed shared memory.

2 Background

The paper models an LLM agent as a policy-driven decision-making entity whose external memory connects frozen parametric knowledge with changing environments. It organizes memory mechanisms evolutionarily into Storage, Reflection, and Experience, distinguished by increasing abstraction and cognitive processing.

  • Agent Formalization: An LLM agent uses policy πθ to map context to actions while interacting with a dynamic environment E.The policy conditions action sampling on system instructions, observations, and retrieved memory.
  • Agent Formalization: External memory M is a global repository, while mt is the context-specific memory retrieved at time t.The repository bridges frozen parametric knowledge in θ and evolving environmental dynamics.
  • Storage: Storage preserves chronological observation-action trajectories with minimal transformation and one-to-one correspondence to execution traces.A trajectory is represented as τ = ⟨(o1, a1), . . . , (oT , aT )⟩, and raw storage accumulates historical trajectories.
  • Reflection: Reflection semantically transforms completed trajectories into evaluated or corrected reasoning paths, prioritizing quality density over raw fidelity.It acts as a semantic filter that injects processed insights into the global repository as refined memory entries.
  • Experience: Experience performs cross-trajectory abstraction by compressing redundant, topologically similar trajectories into generalized schemas under the Minimum Description Length principle.The resulting abstraction serves as a policy prior that enables decision-making beyond rule-consistent actions.
  • Evolutionary Framework: Storage, Reflection, and Experience form an evolutionary framework based on memory utilization rather than complete substitution among mechanisms.A memory mechanism may retain characteristics of multiple stages within this evolutionary perspective.

3 Evolutionary Drivers

Memory mechanisms evolve to support long-range consistency, adaptation to dynamic environments, and continual learning as LLM agents operate in the real world. These requirements drive a progression from preserving information toward scaling, refinement, and abstraction for actionable reuse.

  • Long-range consistency: Long-horizon consistency is a prerequisite for real-world deployment and addresses redundant exploration, error accumulation, and reasoning discontinuities.The analysis distinguishes consistency of state from consistency of goals.
  • Dynamic environments: Dynamic environments make fixed reasoning patterns and static storage fragile because knowledge has conditional temporal validity and strategies lose utility over time.Outdated knowledge may fail without overt indication while retaining semantic relevance.
  • Continual learning: Continual learning requires internalizing out-of-distribution patterns as actionable knowledge so agents can avoid repetitive trial-and-error cycles.Memory must support reproduction of historical trajectories while addressing scaling and abstraction in dense memory.
  • Continual learning: Memory storage grows linearly during extended real-world interaction, while unrestricted expansion can propagate errors and contaminate learning performance.Early mechanisms used vectorization to scale storage capacity, but recent research identifies uncontrolled growth as detrimental.

4 Evolutionary Path

LLM agent memory mechanisms evolve through three stages: Storage preserves interaction trajectories, Reflection corrects and denoises them, and Experience abstracts them into higher-order knowledge. Each stage addresses limitations of the preceding one, progressing from faithful recording to error correction and abstraction.

  • Evolutionary Path: The evolutionary path comprises three stages—Storage, Reflection, and Experience—corresponding to trajectory preservation, refinement, and abstraction.The survey presents these stages as the primary trajectory of memory-mechanism evolution.
  • Storage: Storage addresses limited context windows and expanding interaction histories by faithfully preserving trajectories τi to maintain action consistency.Linear storage records trajectories as time-ordered token streams, typically managed with FIFO.
  • Reflection: Reflection responds to storage’s contaminated raw trajectories by using feedback to correct and denoise memory, transforming it from passive recording into active criticism.Its methods include introspection, environmental signals, memory-lifecycle maintenance, and trajectory compression or distillation.
  • Experience: Experience addresses reflection’s fragmentation, contextual dependence, retrieval costs, and inference burden by converting memories into natural-language policies, executable entities, or evolvable skill libraries.Skill libraries couple procedural abstractions with induction, reuse, and refinement.
  • Experience: Experience can also internalize interaction histories as model parameters or hidden-layer latent variables, while hybrid mechanisms accumulate explicit memories before periodic compression into implicit weights.These forms support dynamic invocation during inference and experience transfer between explicit and implicit memory.

5 Transformative Experience

The Experience stage extends memory beyond history storage by combining active exploration with cross-trajectory abstraction. Together, these mechanisms turn episodic trajectories into reusable, abstract policy guidance that supports autonomous and continuous agent evolution.

  • Active Exploration: Experience transforms LLM agents from passive information recorders into goal-driven collectors of experience through active exploration.Prior experience guides exploration, and its outcomes are abstracted back into memory.
  • Cross-Trajectory Abstraction: Experience extracts a separate rule set K as a policy prior for unseen scenarios, shifting memory from trajectory-local refinement to cross-trajectory abstraction.Reflection injects refined units m′_i back into memory for similar future tasks, whereas Experience abstracts rules across trajectories.
  • Active Exploration: Active exploration is driven by intrinsic motivation and feedback rather than traditional random exploration.This memory-centric process uses prior experience to direct environmental exploration.
  • Cross-Trajectory Abstraction: Cross-trajectory abstraction compresses isolated trajectories into universal patterns and stable policy priors for higher-level decision making.Implementations include aggregating behavioral sequences, encapsulating recurring behaviors as reusable program functions, and internalizing trajectories through fine-tuning.
  • Experience Feedback Loop: The feedback loop between active exploration and cross-trajectory abstraction drives autonomous and continuous evolution during Experience.Highly abstract memories also help agents circumvent challenges associated with conventional memory mechanisms.

6 Future Directions

Future memory mechanisms for LLM agents should move toward active, autonomous memory management and multimodal representations. Key directions include selective retrieval and integration, synchronization between individual and collective knowledge, and unified multimodal memory units.

  • Active Memory Perception: Active memory perception should replace passive triggering that retrieves large portions of memory indiscriminately, reducing interference from irrelevant or obsolete memories.Persistent retrieval of irrelevant or obsolete memories can disrupt reasoning coherence.
  • Active Memory Perception: Autonomous retrieval controllers should evaluate whether a task needs additional memory and determine the specific memory type to integrate.This would make memory mechanisms resources invoked only when needed.
  • Active Memory Perception: Future systems should synchronize individual perspectives with collective knowledge to enable a more agile evolution of socialized experience.This direction is associated with emerging work on socialized experience evolution.
  • Multimodal Memory: Multimodal memory should integrate visual perception, linguistic reasoning, and other perceptual modalities into memory units with unified temporality and semantics.This is identified as a significant direction for future LLM-agent memory mechanisms.

7 Conclusion

The survey presents LLM-agent memory as an evolution through storage, reflection, and experience, extending beyond storage capacity toward denser information and greater cognitive abstraction. It highlights active exploration and cross-trajectory abstraction as mechanisms of this transformation.

  • 7 Conclusion: The proposed evolutionary framework organizes LLM-agent memory into three progressive stages: storage, reflection, and experience.These stages provide the survey’s systematic structure for reviewing memory mechanisms.
  • 7 Conclusion: Memory evolution involves more than expanding storage capacity; it increases information density and transforms cognitive abstraction.The conclusion explicitly characterizes this as a fundamental change in how memory mechanisms develop.
  • 7 Conclusion: Active exploration and cross-trajectory abstraction are identified as mechanisms advancing memory toward the experience stage.The passage introduces these mechanisms as part of the survey’s account of memory transformation.

Limitations

The survey is limited by the absence of unified quantitative evaluation across storage, reflection, and experience mechanisms, and by experience’s overlap with established learning paradigms.

  • Lack of Direct Quantitative Comparison: The qualitative framework lacks comprehensive performance comparisons because the three stages have different objectives and no unified benchmark evaluates them all.Differences in foundation models, environments, and prompts also make direct numerical comparisons potentially misleading.
  • Relation to Established Learning Paradigms: The experience stage, particularly implicit experience, technically intersects with fine-tuning, reinforcement learning, and meta-learning rather than representing an entirely novel learning paradigm.The taxonomy does not position experience as entirely novel.

A Overview · B Detail within the Evolutionary Path

The survey organizes LLM agent memory mechanisms into an evolutionary path from Storage to Reflection to Experience, based on increasing use of historical trajectories. It also analyzes the drivers, technological shifts, scope, and representative works shaping this path.

  • A Overview: The manuscript is organized around the three-stage framework to delineate development drivers, pathways, frontier characteristics, and representative works’ contributions, mechanisms, and evolutionary positions.The survey presents this structure as a foundation for future research and innovation in LLM agent memory mechanisms.
  • A Overview: The framework categorizes memory evolution by the depth of historical-trajectory utilization and identifies three primary dimensions driving transformations in memory mechanisms and agent capabilities.The three-stage framework is introduced as the organizing basis for the survey.
  • A Overview: Storage preserves interaction trajectories over long durations to address LLM agents’ memory-capacity constraints.It is the foundational evolutionary layer.
  • A Overview: Reflection adds dynamic evaluation loops, shifting memory from information recording toward evaluation while mitigating hallucinations and logical errors.This stage treats memory as an evaluator rather than only a recorder.
  • A Overview: Experience abstracts across multiple trajectories to extract higher-order behavioral patterns, compress redundant memory, and produce transferable, reusable heuristic strategies.The survey identifies this as the highest cognitive stage.
  • A Overview: Active exploration and cross-trajectory abstraction advance memory toward Experience by turning agents from passive information recipients into goal-driven experience collectors capable of proactive generalization on unknown tasks.These are presented as pivotal technological shifts for reaching the Experience stage.
  • A Overview: The survey offers comprehensive, forward-looking coverage, an evolutionary taxonomy, critical analysis of Experience-stage characteristics, and identification of underexplored or unresolved issues.Its coverage includes overlooked research, recent advances, and broader theoretical perspectives.

B.1 Storage · B.2 Reflection

Storage mechanisms preserve interaction trajectories to support long-term consistency, but raw memories remain vulnerable to stochasticity, hallucinations, and errors. Reflection addresses this limitation by evaluating, maintaining, compressing, and environmentally calibrating stored trajectories.

  • B.1 Storage: Storage preserves trajectories as accurately as possible, enabling agents to maintain perceptions of internal and external states for continuity and reasoning.However, stored memories remain susceptible to contamination from model stochasticity and hallucinations.
  • B.1 Storage: Storage architectures comprise linear, vector, and structured forms, respectively emphasizing direct records, retrieval from high-dimensional repositories, and relational integrity.Vector storage shifts attention toward efficient retrieval, while structured storage supports logical reasoning and multi-hop retrieval.
  • B.1 Storage: Vector methods improve access to relevant memories through semantic similarity or differentiated weighting based on signals such as relevance, recency, importance, and temporal decay.These approaches improve recall or prioritization but remain retrieval-centric rather than abstraction-driven.
  • B.2 Reflection: Because passive storage conflates successful trajectories with hallucinations, logical errors, and invalid attempts, storage alone does not fundamentally improve memory quality.This motivates the transition from preservation toward reflective evaluation and refinement.
  • B.2 Reflection: Introspective reflection rectifies errors, dynamically maintains memory content, and compresses trajectories into compact, reusable representations.Representative mechanisms include self-critique, validated-memory retention, schema updates, structured reflections, procedural memories, and multi-granularity abstraction.
  • B.2 Reflection: Environmental reflection calibrates internal memory against real-world outcomes by modeling environments, tools, and preferences and by optimizing decisions using rewards or execution feedback.It addresses the risk that introspective reflection becomes inconsistent with factual reality.
  • B.2 Reflection: Collaborative reflection distributes memory management and consensus across specialized agents, alleviating cognitive bottlenecks and hallucinations in complex trajectory processing.Multi-dimensional calibration coordinates heterogeneous memory modules and specialized perception, caching, and execution roles.

B.3 Experience · C Extended Discussion on Multimodal Memory Mechanisms · C.1 Current Approaches.

The Experience stage seeks to abstract trajectories into transferable strategies and executable or internalized capabilities, overcoming reflection’s trajectory-level limits and the burdens of growing memory banks. Multimodal memory research remains concentrated on Storage, with limited attention to Reflection and Experience, and is organized around representation and retrieval.

  • B.3 Experience: Reflection reduces hallucinations and noise through evaluation, but its corrective efficacy remains limited to trajectories rather than transferable strategic knowledge.This limitation motivates moving beyond reflection on the past toward experience abstraction.
  • B.3 Experience: Trajectory-focused reflection can linearly expand memory banks, increasing inference burden and potentially encouraging agents to follow prior trajectories.The passage identifies memory growth and trajectory imitation as consequences of trajectory-level reflection.
  • B.3 Experience: Explicit experience distills trajectories into heuristic guidelines, structured experience graphs, and modular compositional units for strategy representation and reuse.Heuristic guidelines crystallize implicit intuition into natural-language rules, while graphs capture logical dependencies and collaboration patterns.
  • B.3 Experience: Procedural primitives abstract complex reasoning chains into executable skills that agents can invoke as atomic actions, reducing planning overhead.The cited approach encapsulates high-frequency action sequences into functions and extends executable abstraction to hardware optimization.
  • B.3 Experience: Implicit experience converts interaction history into latent variables or neural-network parameters, with latent modulation transferring experience without parameter updates and parameter internalization embedding it into model weights.Latent modulation operates in continuous high-dimensional space, whereas parameter internalization uses gradient updates to instill adaptive priors.
  • B.3 Experience: Hybrid experience establishes an Accumulate-Internalize cycle to address storage explosion and retrieval latency while reducing the mismatch between parameter updates and environmental dynamics.Experience transfer progressively decouples agents from external retrieval by distilling trajectories into structured experience and transferring it into model parameters.
  • C.1 Current Approaches.: Multimodal memory expands text-centered memory to text, audio, images, and video, but current LLM-agent research remains predominantly at Storage with limited Reflection and Experience work.Existing methodologies are discussed through Multimodal Representation and Multimodal Retrieval, including agent-driven modality-specific querying across multiple turns.

C.2 Unique Challenges.

Multimodal memory faces distinctive challenges in aligning heterogeneous signals, maintaining temporal consistency, and consolidating information without losing perceptual detail. Existing research remains concentrated in Storage, while Reflection and Experience are comparatively scarce.

  • Multimodal Alignment: Multimodal memory must bind visual, auditory, and textual signals into a unified semantic unit, making multimodal alignment a central challenge.Misalignment can prevent complete memory recall and introduce irrelevant signals that impede reasoning.
  • Temporal Consistency: Because multimodal memory unfolds continuously over time, organizing memory-unit length and granularity is a significant temporal-consistency challenge.Different modalities also have divergent temporal validity, complicating decay-function adaptation when environments change.
  • Consolidation and Forgetting: Consolidation can discard perceptual details from nontextual signals, creating a distinctive multimodal forgetting challenge.The passage identifies perceptual-detail loss as a consequence of consolidation.
  • Research Stage: Existing multimodal-memory research is concentrated predominantly in Storage, whereas Reflection and Experience research remains exceedingly scarce.This distribution indicates that multimodal memory is at an early phase of development.

D Datasets and Benchmarks

The survey organizes LLM-agent memory benchmarks along the Storage, Reflection, and Experience evolutionary stages. Existing benchmarks emphasize static-data storage and retrieval, leaving dynamic interaction, memory transformation, and continual evolution less fully evaluated.

  • Storage Stage: Benchmarks are categorized by the proposed evolutionary path, but existing evaluations primarily test storage and retrieval of static data.This emphasis leaves critical memory capabilities in dynamic interaction scenarios insufficiently evaluated.
  • Storage Stage: Extreme-context benchmarks test memory limits by retrieving minute facts from massive distracting contexts and evaluating reasoning over million-word backgrounds.Some extend these challenges to long-context reliability and visual settings.
  • Storage Stage: Interactive-consistency benchmarks evaluate whether agents maintain coherent memory across sessions, including ten-million-word dialogue and continuous knowledge updating and rejection.These tasks directly probe consistency during extended interaction.
  • Storage Stage: Relational-fact benchmarks assess semantic association and multi-hop reasoning by integrating facts across documents and performing multi-step reasoning over personal trivia.Related fact-checking frameworks also evaluate knowledge recall and evidence integration.
  • Reflection Stage: Reflection-stage benchmarks evaluate how agents transform raw trajectories into high-quality memory through denoising, fidelity, personalization, and reasoning in complex environments.Representative categories include error correction, personalization, and dynamic reasoning.
  • Experience Stage: Experience-stage benchmarks remain relatively scarce and assess abstraction of general experience from fragmented trajectories in dynamic environments to support continual evolution through practical application.The cited work simulates environments intended to resemble authentic deployment.
Loading 2605.06716v1…