Source-linked AI summary
Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework
Yanchen Wu, Tenghui Lin, Yingli Zhou, Fangyuan Zhang, Qintian Guo, Xun Zhou, Sibo Wang, Xilin Liu, Yuchi Ma, Yixiang Fang
TL;DR
Existing agent memory methods lack systematic comparison under common settings and a unified account of their components. The paper introduces a four-module framework, evaluates representative methods across three benchmarks and multiple dimensions, and develops a combined method with strong accuracy and low overhead. The analysis also identifies scope and scalability challenges for future memory systems.
Problem
Agent memory methods have lacked a unified framework and comprehensive same-setting comparisons of their components, accuracy, and efficiency.
Method
The paper decomposes memory into information extraction, management, storage, and retrieval, then evaluates representative methods across three benchmarks and multiple robustness dimensions.
Results
The new memory variant achieves strong accuracy while maintaining low overhead, and the study reports substantial backbone-model gains on multi-session and temporal reasoning tasks.
Takeaways & Limitations
A unified modular analysis clarifies differences among memory methods and supports designing combined architectures with improved performance and cost efficiency.
Takeaways & Limitations
Existing methods often support only limited information modalities, while memory growth increases storage, management, and retrieval overhead.
Abstract
from arXiv · showhide
Memory emerges as the core module in the large language model (LLM)-based agents for long-horizon complex tasks (e.g., multi-turn dialogue, game playing, scientific discovery), where memory can enable knowledge accumulation, iterative reasoning and self-evolution. A number of memory methods have been proposed in the literature. However, these methods have not been systematically and comprehensively compared under the same experimental settings. In this paper, we first summarize a unified framework that covers existing representative agent memory methods from a high-level perspective. We then extensively compare representative agent memory methods on two long-term conversational benchmarks and an agentic memory benchmark, and examine the effectiveness of representative methods, providing a thorough analysis of those methods. As a byproduct of our experimental analysis, we also design a new memory method by exploiting modules in the existing methods, which outperforms the state-of-the-art methods. Finally, based on these findings, we offer promising future research opportunities. We believe that a deeper understanding of the behavior of existing methods can provide valuable new insights for future research.
1 INTRODUCTION
LLM agents need memory to retain and reuse information across interactions, enabling long-horizon planning, personalization, and adaptive decisions. This paper addresses missing unified analysis and systematic comparisons by proposing a modular framework and comprehensive evaluation.
- Memory lets LLM agents preserve relevant past information beyond naive long-context prompting, supporting contextual knowledge and more informed decisions.
- Existing memory methods aim to support long-term planning, personalization, and adaptive decision-making instead of stateless reasoning.
- Prior work lacks a unified framework for analyzing memory components, component-level studies, and comprehensive accuracy-efficiency comparisons.
- The paper decomposes memory into information extraction, memory management, memory storage, and information retrieval, then evaluates representative methods across conversational and agentic benchmarks.
- The study analyzes practical robustness dimensions and designs a new memory method with the best overall performance and cost efficiency.
3 A UNIFIED FRAMEWORK
The unified framework models agent memory as a workflow that transforms incoming messages into persistent, managed records and retrieves relevant context for the current query. Its four modules separate extraction, management, storage, and retrieval responsibilities.
- The framework comprises information extraction, memory management, memory storage, and information retrieval.
- At interaction step t, the system receives messages M_t, query q_t, logical records D_t−1, and materialized state H_t−1.
- Information extraction: Information extraction converts incoming messages and metadata into logical records E_t while filtering redundancy and producing downstream knowledge representations.
- Memory management: Memory management updates the existing state using E_t, producing a change set Δ_t that connects, integrates, migrates, updates, or removes records.
- Storage and retrieval: The workflow outputs an updated memory state H_t and retrieved context X_t through storage and retrieval operations.
3 H𝑡←Materialize𝜃𝑆(H𝑡−1, Δ𝑡) ; // persist records and maintain organization and indexes // ❹Information retrieval
The framework distinguishes extraction, management, storage, and retrieval design choices, including multiple extraction paradigms and storage organizations. Memory management maintains coherent, adaptive records through connection, integration, transformation, updating, and filtering.
- Information extraction: Information extraction methods include direct archiving, summarization-based extraction, and graph-based extraction.
- Information extraction: Summarization-based extraction prompts an LLM to produce concise structured information from one or more dialogue turns.
- Memory management: Memory management evolves records through connecting related experiences, integrating fragmented information, transforming short-term into long-term memory, updating content, and filtering obsolete knowledge.
- Memory storage: Memory storage varies by organization and representation, including flat or hierarchical structures and vector- or graph-based paradigms.
- Memory storage: Hierarchical storage can separate short-term conversations, mid-term topic summaries, and long-term user preferences to balance computational overhead with knowledge persistence.
7 INFORMATION RETRIEVAL
The paper examines information retrieval and related memory-design choices through systematic experiments, finding that retrieval quality depends on context scale, evidence position, backbone reasoning, and architectural organization. Its proposed framework achieves the best overall benchmark performance while keeping token overhead below 450 tokens per dialogue.
- Retrieval design: Retrieval ranks memory records relevant to query q_t from persistent state H_t, producing context X_t for downstream reasoning or response generation.The paper distinguishes lexical retrieval, which supports exact names and phrases, from vector retrieval, which addresses vocabulary mismatch through semantic similarity.
- Retrieval robustness: Nearly all memory architectures show steadily declining F1 as context expands from 50% to 200%, primarily because irrelevant information lowers retrieval signal-to-noise ratio.The result indicates that larger contexts introduce interference rather than uniformly improving access to historical information.
- Retrieval robustness: A clear recency bias appears when supporting evidence is placed earlier in LONGMEMEVAL contexts, as greater temporal distance increases interference from subsequent dialogue.The position-sensitivity experiment compares evidence in the first, middle, and last thirds of the context.
- Backbone dependence: Scaling Qwen3.5 from 9B to 27B yields substantial gains across memory methods, while most methods perform best with GPT-5.4-mini.These findings indicate that evaluated memory architectures remain dependent on the backbone model’s reasoning capability.
- Proposed method: The new framework achieves the best overall performance on both benchmarks with fewer than 450 tokens per dialogue, and removing Tree or SP lowers overall F1 by 4.39 and 2.25 points, respectively.The framework combines tree-based organization, hierarchical memory separation, and segment-level processing; all three contribute to performance, with temporal reasoning most affected by ablation.
- Memory organization: Replacing tree organization with a graph reduces overall F1 from 43.40 to 40.60, although graphs perform better on temporal reasoning questions.The comparison suggests a trade-off between stronger balanced overall performance from trees and better temporal-dependency modeling from graphs.
9 LESSONS AND OPPORTUNITIES
The paper distills lessons about memory architecture and identifies opportunities for multimodal integration and storage-efficient memory. It also presents a roadmap for selecting architectures across four core capabilities.
- Lessons: A roadmap guides memory-architecture selection across information persistence, memory association, temporal reasoning, and knowledge update.
- Lessons: Hierarchical organization captures structural relationships more effectively than flat memory through tree-based indices or multi-level storage.
- Lessons: Structured representations improve organization, but retaining raw dialogue context remains essential to prevent semantic loss during extraction or retrieval.
- Opportunities: Existing memory mechanisms often support only limited information forms, constraining utilization of textual, interaction-trace, and multimodal sources.
- Opportunities: Rapidly growing storage and management overhead makes compression without useful-information loss a major research challenge.
10 RELATED WORKS
Related work spans retrieval-augmented generation, LLM-based database systems, and agent-memory surveys and benchmarks. The paper positions itself as combining modular analysis with systematic empirical comparison.
- RAG frameworks: RAG has been applied to question answering, programming context, SQL rewriting, DBMS configuration debugging, and data cleaning.
- RAG frameworks: RAG retrieves query-relevant information from external knowledge bases to mitigate LLM hallucination and increasingly uses graphs to organize relationships.
- LLMs for database: LLM-based database research leverages developer knowledge and persistent vector databases to improve tuning, search, updates, and data organization.
- Agent Memory Surveys and Benchmarks: Recent surveys organize agent memory conceptually, while benchmarks establish evaluation protocols for conversational and agentic memory capabilities.
11 CONCLUSIONS
The paper unifies and evaluates existing agent-memory methods through a modular framework, then uses the analysis to develop a strong, low-overhead memory variant and identify future research opportunities.
- The framework decomposes memory mechanisms into information extraction, memory management, memory storage, and information retrieval.
- Representative methods are evaluated on three benchmarks with analyses of token cost, context scalability, evidence-position sensitivity, and LLM-backbone dependence.
- A new memory variant combines existing techniques to achieve strong accuracy while maintaining low overhead.
- The paper summarizes lessons and proposes practical research opportunities for future studies.
A PROMPT TEMPLATES
The appendix documents prompts, benchmark datasets, task categories, and evaluation metrics used in the study. It covers conversational and agentic settings, including long-context reasoning and tool-mediated tasks.
- A PROMPT TEMPLATES: The appendix includes prompts for graph-based information extraction, LLM-as-a-judge evaluation, and answer simplification.
- B.1 Dataset Details: LOCOMO contains ten long-term conversations averaging 198.6 questions, 27.2 sessions, and 588.2 dialogue turns per conversation.
- B.1 Dataset Details: LOCOMO evaluates single-hop, multi-hop, temporal-reasoning, and open-domain knowledge questions.
- B.1 Dataset Details: LONGMEMEVAL contains 500 questions grounded in conversations averaging 50.2 sessions and approximately 115,000 tokens.
- B.1 Dataset Details: LONGMEMEVAL assesses information extraction and multi-session reasoning across extended user–AI interaction histories.
- B.1 Dataset Details: MemoryArena evaluates multi-session agent–environment tasks averaging 6.9 interdependent subtasks and approximately 57 agent actions.
- B.1 Dataset Details: MemoryArena includes bundled web shopping, group travel planning, and progressive web search tasks requiring retention of earlier decisions and constraints.
- B.1 Dataset Details: MemoryArena reports Task Success Rate and Task Progress Score, with additional task-specific metrics including soft Progress Score, Accuracy, and #Search.
B.2 Dataset Variant Construction
The study constructs LONGMEMEVAL variants to test context scalability and evidence-position sensitivity, then analyzes how memory methods and task categories respond to these pressures.
- Dataset Design: LONGMEMEVAL supports controlled evaluation through configurable context length, topically similar session histories, and clearly defined ground-truth evidence.
- Context Scalability: 50%, 150%, and 200% context variants respectively prune history or append sessions while retaining the ground-truth session.
- Position Sensitivity: Evidence-position variants relocate the ground-truth session into Early, Middle, or Late thirds of the conversation.
- Context Scalability: LLM-as-OS methods such as MemOS and MemGPT face more tool-call failures and indexing conflicts as context reaches 200%, whereas MemoryOS remains stable through rule-based management.
- Task Effects: Knowledge Update degrades sharply under scaling because more obsolete records increase retrieval interference, while temporal tasks remain relatively stable.
- Position Sensitivity: Most methods show recency bias, with higher overall F1 when evidence appears late rather than early.
- Position Sensitivity: MemoryOS has a smaller Late–Early gap because stage-wise transfers preserve earlier evidence independently across memory levels.
- Task Effects: Position sensitivity is stronger for transient session-localized information than for persistent traits.
C DETAILS OF OUR NEWLY DESIGNED METHOD
The newly designed method combines tiered memory storage, semantic segmentation, hierarchical summaries, heat-based management, and multi-mode retrieval. Its broader design recommendations favor efficient granularity, preserved history, and adaptive retrieval.
- Memory Construction: New messages enter a FIFO short-term queue and move into semantically similar segments when capacity is reached.
- Memory Construction: The mid-term memory tree stores segment summaries at leaves and aggregated summaries at parent nodes, reducing token overhead compared with turn-level processing.
- Information Retrieval: Retrieval independently accesses three storage tiers, combining complete short-term retrieval with flat semantic search and beam traversal for mid-term memory.
- Implications: Well-designed memory frameworks can reduce smaller LLMs’ dependence on model scale for complex queries involving temporal dependencies.
- Future Directions: Non-destructive updates should preserve historical information while annotating validity instead of deleting older knowledge.
- Future Directions: Existing hierarchical systems mainly consolidate short-term memory into long-term storage, motivating bidirectional transformation across memory levels.
- Future Directions: Fixed retrieval strategies can be brittle for complex queries, motivating routing planners that adapt retrieval mechanisms to query contexts.
- Benchmark Limitations: Current benchmarks use static, text-centric histories and therefore motivate interaction-driven, multimodal evaluations of evolving memory.
E ANALYSIS OF EVALUATION METRICS
The evaluation analysis explains the lexical metrics used for generated answers and highlights their sensitivity to wording, verbosity, response prompts, and evaluation reliability.
- Evaluation Reliability: LLM-as-a-judge results can depend on the judge model and prompt, so the study treats them as complementary rather than primary metrics.
- F1: Token-level F1 measures overlap between generated and reference answers using precision, recall, and their harmonic mean.
- BLEU-1: BLEU-1 measures clipped unigram precision with a brevity penalty.
- Metric Details: Repeated overlapping tokens are counted according to their minimum occurrence in the generated and reference answers.
- Metric Limitations: F1 and BLEU-1 are sensitive to wording and verbosity because unmatched explanatory tokens can lower lexical scores even when the key answer is correct.
F DISCUSSION OF METHOD SELECTION
The method-selection roadmap maps memory architectures to persistence, association, temporal reasoning, and knowledge-update requirements, while documenting benchmark and implementation constraints.
- Selection Framework: The selected baselines span token-level memory organizations and diverse choices across the four-stage framework.
- Information Persistence: Information Persistence favors MemoryBank for user profiles, MemTree for episodic facts when overhead is acceptable, and MemoryOS for greater token efficiency.
- Memory Association: MemTree supports implicit cross-session associations, while explicit multi-hop associations require structured paths across multiple memory units.
- Temporal Reasoning: MemOS performs best for time retrieval, whereas time comparison requires comparing temporal relations across multiple pieces of information.
- Knowledge Update: MemTree offers stronger knowledge updates when update cost is acceptable, while LightMem suits tight token budgets.
- Efficiency: Zep did not complete LONGMEMEVAL memory construction within two days in the unified experimental environment.
- Efficiency: Zep’s measured average retrieval latency was 536.8 ms because the study used its open-source implementation rather than its production API.
- Efficiency: Zep’s construction overhead reflects entity extraction, resolution, deduplication, conflict handling, and an expanding graph candidate space.