Source-linked AI summary
How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior
Zidi Xiong, Yuping Lin, Wenya Xie, Pengfei He, Zirui Liu, Jiliang Tang, Himabindu Lakkaraju, Zhen Xiang
TL;DR
LLM-agent memory systems can improve future task execution, but the long-term effects of memory-management choices remain insufficiently understood. This paper empirically studies memory addition and deletion across controlled agent settings, finding experience-following and challenges from error propagation and misaligned replay. The experiments further show that evaluator signals support effective memory management under task shifts and capacity constraints.
Problem
Existing memory strategies provide limited general understanding of how dynamic, noisy memory banks and their addition and deletion operations affect long-term agent execution.
Method
The paper conducts controlled empirical experiments across four agents to analyze experience-following and evaluator-guided memory addition and deletion.
Results
Agents often follow retrieved experiences: similar inputs yield similar outputs, while noisy or misaligned records can propagate errors or produce poor executions.
Takeaways & Limitations
Evaluator signals and simple addition and deletion can support robust long-term performance under task distribution shifts and memory constraints.
Takeaways & Limitations
The study considers only memory addition and deletion, omitting structural transformation, merging, summarization, and reflection, and its findings lack formal theoretical guarantees.
Abstract
from arXiv · showhide
Memory is a critical component in large language model (LLM)-based agents, enabling them to store and retrieve past executions to improve task performance over time. In this paper, we conduct an empirical study on how memory management choices impact the LLM agents' behavior, especially their long-term performance. Specifically, we focus on two fundamental memory management operations that are widely used by many agent frameworks-memory addition and deletion-to systematically study their impact on the agent behavior. Through our quantitative analysis, we find that LLM agents display an experience-following property: high similarity between a task input and the input in a retrieved memory record often results in highly similar agent outputs. Our analysis further reveals two significant challenges associated with this property: error propagation, where inaccuracies in past experiences compound and degrade future performance, and misaligned experience replay, where some seemingly correct executions can provide limited or even misleading value as experiences. Through controlled experiments, we demonstrate the importance of regulating experience quality within the memory bank and show that future task evaluations can serve as free quality labels for stored memory. Our findings offer insights into the behavioral dynamics of LLM agent memory systems and provide practical guidance for designing memory components that support robust, long-term agent performance.
1 Introduction
This paper studies how dynamic memory addition and deletion shape long-term LLM-agent behavior. It identifies experience-following and associated risks, then evaluates evaluator-guided memory management under changing tasks and limited capacity.
- Prior memory strategies are often task-specific, leaving limited understanding of principles governing memory behavior across agentic systems.
- Dynamic agentic memory differs from static external knowledge bases because its retrieval pool evolves and contains noisy, often self-generated trajectories.
- High input similarity between a current query and a retrieved record often produces high output similarity, enabling experience reuse but creating error propagation and misaligned replay.
- Controlled experiments across four agents show that trajectory evaluators can improve addition and deletion, including adaptation under task shifts and memory constraints.
- The study systematically analyzes how memory addition and deletion affect the long-term performance of LLM agents.
2 Background and Related Works
LLM agents use episodic memory to retrieve past task executions as demonstrations, while memory management adds or deletes records. Existing strategies are diverse and often task-specific, motivating a more unified account of memory-bank dynamics.
- Episodic memory records task-specific experiences and is the memory type examined in this paper.
- Given a new query q and memory base D of N query-execution pairs, the agent retrieves relevant past pairs before execution.
- Memory reading selects K relevant query-execution pairs using input similarity, such as cosine similarity between text-encoder representations.
- Memory management decides whether to add new query-execution trajectories and deletes outdated or redundant pairs.
- Structural transformation, merging, summarization, and reflection strategies are often tailored to specific agent types and lack a unified general design.
- Existing optimization studies do not explain how basic addition and deletion affect long-term performance under noisy memory dynamics.
3 Addition of Memory
Memory addition strongly shapes long-term agent behavior: selective storage with high-quality evaluation supports improvement, while noisy additions can degrade performance. Across controlled agents, retrieved demonstrations also produce an experience-following pattern in which similar inputs yield similar outputs, enabling either self-improvement or error propagation.
- 3.1 Setup: Memory addition stores completed task executions selectively or indiscriminately, with fixed-memory, add-all, coarse-evaluation, and strict human-evaluation strategies compared across four agents.The study includes RegAgent, EHRAgent, AgentDriver, and CIC-IoT Agent, using a synthetic controllable task alongside representative real agents.
- 3.2 Execution quality and memory size jointly determine long-term agent performance.: Strict selective addition consistently yields superior long-term performance, while noisy or low-quality additions can make fixed memory outperform coarse evaluators.All strategies begin from the same initial memory, but their long-term trajectories diverge as memory evolves.
- 3.2 Execution quality and memory size jointly determine long-term agent performance.: Strict addition and the C3 coarse evaluator continue improving over time, whereas add-all and some coarse strategies remain flat or decline as flawed records accumulate.The improvement trend is also reported across different LLM backbones for strict addition.
- 3.3 Experience-Following Property: Higher input similarity between a query and retrieved memory correlates with higher output similarity for RegAgent and AgentDriver, defining the experience-following property.The pattern also appears for other agents and alternative backbones; under memory addition, output similarity rises as input similarity grows.
- 3.3 Experience-Following Property: Experience following can replicate correct executions for similar tasks, but indiscriminate addition can introduce incorrect executions that drive self-degradation.The reported relationship becomes near-perfect under some conditions, with Pearson r ≈1 between input and output similarity.
- 3.4 Error Propagation in Agent Memory: Erroneous retrieved demonstrations propagate into later memory records and widen performance gaps relative to error-free variants, especially under add-all and coarse selective addition.With strict selective addition for AgentDriver, performance eventually approaches and surpasses the ground-truth baseline after roughly 2000 executions.
4 Deletion of Memory
The paper compares periodic, history-based, and combined memory deletion strategies, showing that their effects depend strongly on evaluator reliability. History-based deletion can improve long-term performance when guided by reliable utility estimates, while combined deletion offers a stronger memory-size reduction trade-off.
- History-Based Deletion: History-based deletion removes records retrieved at least n times when their average historical utility falls below threshold β.The utility is computed across past retrievals and can use the evaluator employed for selective addition.
- Deletion strategies: Periodic deletion reduces memory substantially with generally small performance degradation, indicating that addition-only designs accumulate redundant entries.
- Evaluator reliability: History-based deletion produces variable outcomes because its effectiveness depends on the reliability of the evaluator used for addition and deletion.
- Evaluator reliability: With a strict evaluator, history-based deletion improves performance for the evaluated non-synthetic agent and causes only a small performance drop for RegAgent despite removing several hundred frequent executions.
- Experience quality: Selective retention based on downstream utility can improve long-term performance by removing misaligned or harmful demonstrations from memory.The paper links these gains to filtering experiences whose stored trajectories provide limited or misleading guidance for the current task distribution.
- Combined deletion: Fine-tuned evaluators yield more stable results, while combined deletion balances maintained performance with smaller memory sizes.The combined strategy applies periodic and history-based criteria together; periodic deletion also bounds memory size by M ≤ α(t − t′)K.
5 Memory Management under Challenging Scenarios
The study evaluates memory management under task distribution shifts and fixed capacity constraints. Effective policies generally preserve performance and use limited storage more efficiently, though outcomes vary by agent and configuration.
- Task Distribution Shift: Figure 7 compares fixed memory, strict addition, history-based deletion, and combined deletion under shifted task distributions.The figure also marks the distribution-shift point and shows the combined-deletion performance without shift as a reference.
- Task Distribution Shift: Performance gaps relative to the no-shift variant generally remain small across task distributions.AgentDriver’s strict-addition variant surpasses the no-shift variant, whereas EHRAgent’s history-based deletion underperforms combined deletion.
- Memory Resources Constraint: Under strict capacity constraints, memory policies achieve high performance compared with the fixed-memory variant.The constrained combined-deletion policy periodically deletes records and then removes only the least useful record when additions exceed capacity.
- Memory Resources Constraint: Selective retention of relevant, high-quality records enables efficient use of limited storage and improves long-term agent performance.On AgentDriver, performance gradually converges as constrained memory size increases, suggesting unbounded memory growth is unnecessary.
6 Conclusion
The paper studies how memory addition and deletion shape long-term LLM-agent behavior. It identifies experience-following, error propagation, and misaligned experience replay, highlighting evaluator reliability as central to effective memory management.
- Conclusion: The study analyzes memory addition and deletion to explain long-term agent memory dynamics.These operations are treated as fundamental components of memory management.
- Conclusion: The experiments identify experience-following and reveal error propagation and misaligned experience replay as key challenges.The findings connect these behavioral dynamics to the quality and usefulness of stored experiences.
- Conclusion: Evaluator reliability is critical, and evaluator signals are essential for effective memory management.The conclusion emphasizes evaluator information as a central design consideration for memory systems.
Limitations
The paper’s conclusions are limited by its focus on two memory operations and its reliance on empirical rather than formal theoretical analysis.
- Scope of Memory Management: The study examines memory addition and deletion while omitting structural transformation, merging, summarization, and reflection.Extending the findings to systems with advanced updating mechanisms or different agent architectures requires additional fine-grained analysis.
- Theoretical Guarantees: The findings provide no formal theoretical guarantees because the analysis is primarily empirical.The authors frame controlled experimentation as a foundation for future theoretical investigation.
A Detailed experimental setups.
The experiments cover synthetic prediction, code generation, autonomous driving, and IoT traffic detection agents, using task-specific retrieval, evaluation, and memory-management procedures. Similarity measures, evaluator thresholds, deletion rules, datasets, and prompts are specified for reproducible comparisons.
- RegAgent: RegAgent predicts y = w⊤x from six-dimensional inputs using six retrieved demonstrations generated from Gaussian-distributed vectors with bounded noise.A prediction succeeds when its absolute error is at most 1, with success rate as the primary metric.
- RegAgent: RegAgent measures retrieval similarity with cosine similarity and output similarity with output_similarity = exp(−γ|x1 −x2|2), using γ = 1.0.The task is designed to isolate retrieved-demonstration effects and permit clear error and memory-addition criteria.
- EHRAgent: EHRAgent is a code-generation system evaluated on 2,392 filtered MIMIC-III tasks, retrieving four experiences from an initial bank of 100 records.Retrieval uses maximum cosine similarity, while exact match evaluates answer correctness.
- AgentDriver: AgentDriver uses 2,000 nuScenes test cases and an initial memory of 180 training experiences, with top-1 vector-similarity retrieval.The implementation simplifies the original two-stage retrieval process to improve reproducibility.
- CIC-IoT Agent: CIC-IoT traffic detection retains eight distinguishable attack classes and evaluates accuracy over 1,000 sampled test cases.The agent retrieves three experiences and begins with 100 synthetic records generated from a disjoint training subset.
- CIC-IoT Agent: CIC-IoT input similarity uses average relative feature change for continuous features and a separate definition for discrete features.Output similarity is computed from embeddings, while deletion uses periodic, history-based, or combined rules with evaluator thresholds.
- Prompts and Agent Functionality: The experimental prompts specify task outputs and available EHR database operations, including filtering, value retrieval, SQL interpretation, and calendar calculations.The RegAgent prompt requires a boxed numerical guess, while the CIC-IoT prompt requires one traffic-type answer with reasoning.
B.1 Additional results on addition experiments
Additional experiments report accuracy trends for different addition strategies during long-term running on EhrAgent and CIC-IoT.
- Figure 8 compares the accuracy trends of different addition strategies during long-term running on EhrAgent and CIC-IoT.
B.2 Experiments on different LLM backbones
Experiments with GPT-4o and Deepseek-V3 show consistent trends across different memory-management strategies, while additional analyses compare input and output similarity.
- GPT-4o and Deepseek-V3 were evaluated with fixed-memory, strict addition, strict addition with history-based deletion, and strict addition with combined deletion.
- The two models exhibited consistent trends within the main experiments.
- Figure 10 presents GPT-4o accuracy trends during long-term running on AgentDriver.
B.3 Error-free variant of history-based deletion
The deletion experiments compare history-based deletion with error-free variants, evaluate different evaluators, and examine the quality of retained versus deleted records.
- Around task index 1000, combined deletion with strict addition surpasses its error-free variant on AgentDriver.The authors interpret this as evidence that history-based deletion can retain outputs suitable for later reuse.
- History-based deletion performance is reported on RegAgent and CIC-IoT Agent with different evaluators.
- Figure 13 compares history-based deletion with its error-free variants, with coarse denoting the C1 evaluator.
- Table 4 reports correctness rates for deleted and retained records on EhrAgent and CIC-IoT Agent under coarse evaluation.
- With GPT-4o-mini as evaluator, retained AgentDriver memory has lower average quality than deleted records.
B.6 Results under memory resource constraints.
Additional experiments examine agent performance under memory resource constraints, including unlimited versus limited memory and multiple limitation sizes.
- Figure 14 presents performance changes after history-based deletion on RegAgent and CIC-IoT Agent with different evaluators.
- Figure 16 compares deleted and retained AgentDriver records, where lower error indicates better execution quality.
- Figure 15 compares deleted and retained RegAgent records under history-based deletion using Coarse 1.2 and Coarse 1.4.Lower absolute error indicates better execution quality.
- Figure 17 compares unlimited and limited memory sizes for strict and coarse selective addition with combined deletion.Coarse denotes the C1 evaluator.
- Figure 18 compares different memory limitation sizes with AgentDriver performance.Horizontal dashed lines indicate the corresponding unlimited-variant performance, and coarse denotes the C1 evaluator.