Source-linked AI summary
MemEvolve: Meta-Evolution of Agent Memory Systems
Guibin Zhang, Haotian Ren, Chong Zhan, Zhenhong Zhou, Junhao Wang, He Zhu, Wangchunshu Zhou, Shuicheng Yan
TL;DR
Existing self-improving memories accumulate experience through fixed architectures, leaving open how memory itself can adapt across task contexts. MemEvolve jointly evolves agent experience and memory architecture, using EvolveLab’s modular codebase and standardized evaluation substrate. Experiments report substantial gains and transfer across tasks and backbone models, with transfer bounded when task families differ fundamentally.
Problem
Fixed memory architectures can support agent evolution but cannot meta-adapt their learning mechanisms to diverse task contexts.
Method
MemEvolve jointly evolves experiential knowledge and memory architecture through a bilevel process, grounded in EvolveLab’s unified four-component design space and codebase.
Results
MemEvolve improves frameworks such as SmolAgent and Flash-Searcher by up to 17.06% and generalizes across tasks and LLM backbones, including 2.0−9.09% gains on unseen settings.
Takeaways & Limitations
Architecture-level evolution discovers broadly applicable memory systems within shared task regimes while retaining capacity for task-specific adaptation.
Takeaways & Limitations
Memories evolved on TaskCraft are unlikely to transfer effectively to fundamentally different task families such as embodied action.
Abstract
from arXiv · showhide
Self-evolving memory systems are unprecedentedly reshaping the evolutionary paradigm of large language model (LLM)-based agents. Prior work has predominantly relied on manually engineered memory architectures to store trajectories, distill experience, and synthesize reusable tools, enabling agents to evolve on the fly within environment interactions. However, this paradigm is fundamentally constrained by the staticity of the memory system itself: while memory facilitates agent-level evolving, the underlying memory architecture cannot be meta-adapted to diverse task contexts. To address this gap, we propose MemEvolve, a meta-evolutionary framework that jointly evolves agents' experiential knowledge and their memory architecture, allowing agent systems not only to accumulate experience but also to progressively refine how they learn from it. To ground MemEvolve in prior research and foster openness in future self-evolving systems, we introduce EvolveLab, a unified self-evolving memory codebase that distills twelve representative memory systems into a modular design space (encode, store, retrieve, manage), providing both a standardized implementation substrate and a fair experimental arena. Extensive evaluations on four challenging agentic benchmarks demonstrate that MemEvolve achieves (I) substantial performance gains, improving frameworks such as SmolAgent and Flash-Searcher by up to $17.06\%$; and (II) strong cross-task and cross-LLM generalization, designing memory architectures that transfer effectively across diverse benchmarks and backbone models.
1 Introduction
Existing memory systems help agents accumulate experience but remain architecturally static across task contexts. MemEvolve addresses this gap by jointly evolving experiential knowledge and memory architecture, supported by EvolveLab and benchmark results.
- Motivation: Current memory systems use trajectories, textual artifacts, structured tools, and code repositories, but their manually fixed pipelines limit adaptation across domains.The paper contrasts memory paradigms that favor browsing, reasoning, coding, or tool-use tasks.
- Motivation: Adaptive memory should change learning strategies by subject, rather than merely recording experience or applying one fixed mechanism.The paper frames this as a transition from skillful to adaptive learning.
- MemEvolve: MemEvolve uses bilevel evolution: agents first accumulate experience under a memory system, while an outer loop meta-learns improved architectures for future learning.The framework jointly evolves agent experience and the mechanism used to learn from it.
- EvolveLab: EvolveLab provides a modular codebase covering encoding, storage, retrieval, and management for representative self-evolving memory systems.It re-implements twelve architectures in a unified design space and supports standardized benchmarking.
- Evaluation: 17.06%: MemEvolve improves frameworks such as SmolAgent and Flash-Searcher, while TaskCraft-evolved memories gain 2.0−9.09% on unseen benchmarks and backbone models.The evaluation spans four challenging agentic benchmarks and reports cross-domain, cross-framework, and cross-LLM generalization.
2 Related Work
Related work situates self-improving agent memory within broader progress in LLM-based agent systems. It distinguishes personalized memory from memory that distills knowledge and skills through continual environmental interaction.
- LLM Agent Systems: LLM-based agents have progressed from simple single-agent workflows to more complex systems with multiple agents, tools, and automated orchestration.This development accompanies advances across diverse agentic tasks.
- Agent Memory Architectures: Self-improving memory distills knowledge and skills from continual environment interactions, whereas personalized memory captures user-specific information and preferences.This work focuses on self-improving memory.
3 EvolveLab: A Unified Codebase for Self-Evolving Memory
EvolveLab formalizes agent memory as a modular architecture that processes experience, maintains persistent state, retrieves task-relevant context, and manages long-term memory. Its unified implementation and evaluation substrate supports systematic comparison and evolution of diverse memory systems.
- 3.1 Preliminary: An active agent observes state and query, retrieves context from evolving memory using interaction history, and produces an action through its policy.The formal system includes agents, shared state, joint action space, environment dynamics, and memory.
- 3.1 Preliminary: Executed trajectories receive terminal rewards, while the memory system assimilates experience units ranging from transitions to aggregated segments or complete trajectories.The memory abstraction covers how new experiences are integrated and organized.
- 3.2 Modular Design Space of Memory Systems: EvolveLab decomposes memory into encode, store, retrieve, and manage operations, representing each architecture as a programmatic combination of these components.The resulting combination forms a genotype for MemEvolve’s meta-evolutionary process.
- 3.2 Modular Design Space of Memory Systems: Encode transforms raw experiences into structured representations, and store integrates them into persistent memory through mechanisms such as vector databases or knowledge graphs.These modules define experience representation and retention.
- 3.2 Modular Design Space of Memory Systems: Retrieve supplies task-relevant tools, planning experience, or procedural knowledge, while manage performs consolidation, abstraction, or selective forgetting.Together these operations support context-aware recall and long-term memory quality.
- EvolveLab: EvolveLab re-implements diverse memory architectures under a common interface and provides standardized evaluation across challenging agentic benchmarks and online or offline settings.Its codebase is designed for systematic implementation, comparison, and future research.
4 MemEvolve: A Meta-Evolving Memory Framework
MemEvolve replaces fixed memory evolution with a dual process that co-evolves an agent’s experience base and memory architectures. It searches this architecture space through nested evaluation, selection, diagnosis, and constrained redesign.
- Dual-Evolution Process: Fixed memory interfaces accumulate experience but cannot adapt architecturally, limiting self-improvement across task contexts.Traditional systems update memory state through environment interactions while keeping the memory interface immutable.
- Dual-Evolution Process: MemEvolve maintains competing candidate memory systems and alternates experience evolution with architectural evolution across iterations.Each candidate is evaluated on independently generated trajectory batches, while later iterations evolve the architecture set.
- Dual-Evolution Process: The inner loop updates each candidate’s memory state from task trajectories and aggregates feedback across task success, token consumption, and latency.Candidate memory states begin empty at each iteration and are evaluated using trajectory-level feedback vectors.
- Architectural Evolution: The outer loop ranks candidates, retains the top-K parents, and generates new architectures by modifying or recombining their four components.Selection favors favorable effectiveness–efficiency trade-offs, with task performance prioritized among Pareto-equivalent candidates.
- Unified View: Repeated dual evolution co-evolves the memory base and governing architectures, yielding increasingly adaptive and resource-aware behavior.The process is designed to ground architectural updates empirically while constraining them structurally.
- Diagnose-and-Design Evolution: Diagnose-and-design evolution uses execution evidence to identify architectural bottlenecks, then produces compatible variants within the modular design space.Variants may alter encoding, storage, retrieval, or management policies while remaining executable through the unified interface.
5 Experiments
MemEvolve is evaluated across four agentic benchmarks, multiple LLM backbones, and both familiar and held-out agent frameworks. Results show substantial, generally robust gains and transferability, while existing memory systems and cross-domain transfer remain bounded.
- Experiment Setup: Experiments cover GAIA, WebWalkerQA, xBench-DS, and TaskCraft, using GPT-5-mini plus DeepSeek V3.2 and Kimi K2 backbones.
- Generalization: MemEvolve transfers memory systems evolved on TaskCraft to WebWalkerQA and xBench-DS without dataset-specific meta-evolution.The transferred systems are fixed before evaluation on the held-out benchmarks.
- Main Results: MemEvolve improves xBench performance from 69% to 74% for one GPT-5-mini framework and raises pass@1 by 6% with pass@3 reaching 68.0% for another.
- Memory Comparison: Existing memories are inconsistent: DILU degrades GAIA by 2.42%, Dynamic Cheatsheet performs poorly on GAIA and xBench, and ExpeL underperforms on all three benchmarks.
- Robustness and Cost: Across three evaluated benchmarks, MemEvolve achieves 3.54%–5.0% gains while maintaining API costs comparable to the No-Memory baseline.Reported examples are GAIA at $0.085 versus $0.086 and xBench at $0.136 versus $0.141.
- Scope Boundary: MemEvolve’s generalization is scoped to shared task regimes and is unlikely to transfer effectively to fundamentally different families such as embodied action.The paper retains task-specific adaptation for settings with substantially different environments, action spaces, and tool sets.
6 Conclusion
The paper presents EvolveLab as a unified design space and codebase for self-evolving agent memory, then builds MemEvolve to evolve memory architectures through interaction feedback. Experiments support effectiveness, robustness, and generalization, while analyses identify recurring design principles.
- EvolveLab unifies self-evolving agent memory implementations, and MemEvolve performs adaptive architecture-level evolution driven by empirical interaction feedback.
- Experiments across diverse agentic benchmarks and backbone models demonstrate the approach’s effectiveness, robustness, and generalization.
- Analysis of automatically evolved memories identifies increased agentic involvement, hierarchical organization, and multi-level abstraction as design principles.
A EvolveLab Implementation
EvolveLab provides a unified, modular interface for implementing and evaluating self-evolving memory systems. Its providers standardize memory ingestion, retrieval, initialization, configuration, and lifecycle management.
- EvolveLab abstracts diverse memory architectures behind a unified interface for standardized implementation, evaluation, and meta-evolution.
- Unified Interface: The BaseMemoryProvider abstract base class defines the fundamental protocol and exposes memory type, configuration, initialization, ingestion, and retrieval operations.
- Retrieve: Memory providers retrieve relevant memories from a MemoryRequest containing task query, execution context, and system status.
- Encode & Store: Providers ingest TrajectoryData by extracting structural insights or tools and persisting them into an underlying storage medium.
- Manage: Manage functionality supports offline consolidation or selective forgetting through auxiliary provider methods or lifecycle events.
A.2 Standardized Data Carriers
EvolveLab uses standardized data carriers to make heterogeneous memory systems interoperable. These carriers represent memory items, trajectory histories, and retrieval requests and responses.
- MemoryItem represents raw text, distilled insights, or executable code while retaining metadata such as timestamps, confidence scores, and source identifiers.
- TrajectoryData contains the initial query, complete interaction traces, and terminal rewards as the raw substrate for memory evolution.
- MemoryRequest and MemoryResponse standardize retrieval queries and results across agent systems and memory providers.
A.3 Implementation Examples: ExpeL and SkillWeaver
EvolveLab demonstrates its modular interface through twelve memory-system implementations, including insight-based ExpeL and tool-centric SkillWeaver. These systems differ in what they distill, store, and retrieve.
- EvolveLab implements twelve distinct memory systems within its unified interface.
- ExpeLProvider: ExpeL distills successful and failed trajectories into textual insights, stores them in a vector database, and retrieves them by semantic similarity.
- SkillWeaverProvider: SkillWeaver synthesizes reusable Python functions from successful trajectories and dynamically injects retrieved skills into the agent’s action space.
B.1 Dataset Details
The study evaluates MemEvolve across four agentic benchmarks with different task scopes and transfer settings. Representative evolved systems illustrate how meta-evolution transforms lightweight or knowledge-oriented initializations into more structured memory architectures.
- Dataset Details: The evaluation covers GAIA, WebWalkerQA, xBench-DeepSearch, and TaskCraft.
- GAIA: GAIA evaluation evolves memory systems using Level-1 tasks together with 67 TaskCraft queries across three rounds of meta-evolution.
- WebWalkerQA: WebWalkerQA evaluates complex multi-turn web interactions, while its evaluated memory systems are meta-evolved on TaskCraft.
- xBench-DeepSearch: xBench-DeepSearch assesses agentic planning, tool use, and reasoning, with evaluated memory systems entirely meta-evolved on TaskCraft.
- TaskCraft: TaskCraft contributes 300 collected queries, including 120 used for three rounds of meta-evolution with 40 queries per round.
- Evolved Architectures: MemEvolve transforms simple static mechanisms into more expressive and adaptive architectures by modifying memory encoding, retrieval, and management strategies.