PaperlayerView exact revision on arXiv

Source-linked AI summary

Mem$^2$Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation

Zihao Cheng, Zeming Liu, Yingyu Shan, Xinyi Wang, Xiangrong Zhu, Yunpu Ma, Hongru Wang, Yuhang Guo, Wei Lin, Yunhong Wang

arXiv:2604.10923v1cs.CLcs.AI

TL;DR

Existing self-evolving-agent frameworks generally either accumulate experience within fixed capabilities or create new assets without experiential guidance. Mem2Evolve combines Asset Memory and Experience Memory in a co-evolutionary loop, and experiments across 6 tasks and 8 benchmarks show consistent gains over experience-centric and capability-centric baselines. Its deployment scope is limited by reliance on a sandbox for executing autonomously generated code.

  • Problem

    Existing frameworks treat experience accumulation and dynamic asset creation in isolation, leaving the former bounded by static assets and the latter unguided by past experience.

  • Method

    Mem2Evolve coordinates Asset Memory and Experience Memory through forward inference and backward evolution to expand capabilities and distill strategic experience.

  • Results

    Experiments across 6 tasks and 8 benchmarks show Mem2Evolve consistently outperforms capability-centric and experience-centric baselines while supporting sustained single-task evolution and cross-task memory reuse.

  • Takeaways & Limitations

    Mem2Evolve provides a framework for coordinated capability expansion and experience distillation with stable, sustained improvement across single-task and cross-task settings.

  • Takeaways & Limitations

    Mem2Evolve relies on a sandbox to execute autonomously generated code, limiting deployment in open-world environments requiring local file-system interaction or unrestricted network access.

Abstract

from arXiv · show

While large language model--powered agents can self-evolve by accumulating experience or by dynamically creating new assets (i.e., tools or expert agents), existing frameworks typically treat these two evolutionary processes in isolation. This separation overlooks their intrinsic interdependence: the former is inherently bounded by a manually predefined static toolset, while the latter generates new assets from scratch without experiential guidance, leading to limited capability growth and unstable evolution. To address this limitation, we introduce a novel paradigm of co-evolutionary Capability Expansion and Experience Distillation. Guided by this paradigm, we propose the \textbf{Mem$^{\textbf{2}}$Evolve}, which integrates two core components: \textbf{Experience Memory} and \textbf{Asset Memory}. Specifically, Mem$^{2}$Evolve leverages accumulated experience to guide the dynamic creation of assets, thereby expanding the agent's capability space while simultaneously acquiring new experience to achieve co-evolution. Extensive experiments across 6 task categories and 8 benchmarks demonstrate that Mem$^{2}$Evolve achieves improvement of 18.53\% over standard LLMs, 11.80\% over agents evolving solely through experience, and 6.46\% over those evolving solely through asset creation, establishing it as a substantially more effective and stable self-evolving agent framework. Code is available at: https://buaa-irip-llm.github.io/Mem2Evolve.

1 Introduction

Existing self-evolving agents typically optimize experience or expand capabilities separately, leaving experience-centric systems bounded by fixed assets and capability-centric systems unguided by past experience. Mem2Evolve addresses this gap with coordinated memories and a forward-backward co-evolution loop, outperforming both baseline paradigms across broad evaluations.

  • Existing paradigms: Experience-centric evolution optimizes execution strategies, prompts, or repositories but remains bounded by a fixed tool and expert-agent set.
  • Existing paradigms: Capability-centric evolution dynamically creates tools or expert agents, but creating assets from scratch without experience prevents reuse of proven strategies and avoidance of known pitfalls.
  • Proposed paradigm: Mem2Evolve introduces co-evolutionary capability expansion and experience distillation, allowing expanded capabilities to generate experiences that guide later expansion.
  • Mem2Evolve: Mem2Evolve coordinates Asset Memory and Experience Memory through forward inference and backward evolution to reuse and expand capabilities while distilling strategic experience.
  • Evaluation: Experiments across 6 tasks and 8 benchmarks show superior overall performance, sustained single-task evolution, and effective cross-task memory reuse.
  • Contribution: The framework is presented as the first co-evolutionary agent paradigm coupling dynamic capability expansion with experience distillation.

2 Related Work

Prior self-evolving-agent research separates experience-centric optimization from capability-centric asset generation. Mem2Evolve instead couples experience distillation with capability expansion so past insights guide asset creation and new capabilities yield richer experiences.

  • Experience-Centric Evolving: Experience-centric frameworks optimize agents using accumulated experience, including team selection, modular composition, and other execution improvements.
  • Capability-Centric Evolving: Capability-centric frameworks expand agent boundaries by dynamically generating tools or agents, reducing dependence on manual design.
  • Co-Evolutionary Evolving: Mem2Evolve couples capability expansion with experience distillation so past insights guide asset creation and new capabilities yield richer experiences.

3 Mem2Evolve

Mem2Evolve coordinates capability expansion and experience distillation through Asset Memory and Experience Memory in a forward–backward task loop. It recruits or creates assets during execution, then validates assets and distills trajectory insights for future reuse.

  • Dual-Memory Mechanism: Mem2Evolve uses Asset Memory for reusable expert agents and tools, while Experience Memory stores lessons from successful and failed trajectories.The two memories support capability expansion and strategic guidance.
  • Forward Inference: The forward inference strategy decomposes tasks, recruits existing assets, and creates missing tools or agents when current capabilities are insufficient.Recruitment reuses matching assets; creation is guided by the sub-task, web results, and relevant experience.
  • Forward Inference: Execution assigns each sub-task to a recruited agent, which uses available tools and role-specific experiences within a ReAct process.The system aggregates the resulting sub-task outputs into a final answer.
  • Backward Evolution: Backward evolution evaluates trajectories, validates newly created assets through critique-driven tests, and preserves reliable assets for future reuse.Failed validation triggers a self-correction loop that revises the asset and regenerates tests until validation passes.
  • Backward Evolution: Experience Memory distills successful trajectories into strategic guidance and failed trajectories into antipatterns and failure–fix pairs.These memory items guide future task execution and asset creation.
  • Evaluation: The main evaluation reports Pass@1 across 6 tasks and 8 benchmarks.The results table compares frameworks across the benchmark suite.

4 Experiments

Mem2Evolve is evaluated across eight benchmarks against naive LLMs, experience-centric frameworks, and capability-centric frameworks. It achieves the strongest overall performance and improves capability expansion beyond static-agent boundaries.

  • The evaluation covers eight benchmarks spanning general assistance, embodied and web interaction, planning, multi-hop question answering, and mathematical reasoning.
  • Mem2Evolve attains an average Pass@1 of 70.24% across all benchmarks, outperforming Alita by 6.46%, Aflow by 11.80%, and naive LLMs by up to 18.53%.
  • Starting from the same minimal configuration, Mem2Evolve improves performance by 57.84% on GAIA and 10.03%/13.33% on AIME24/AIME25.
  • Experience-centric baselines with fixed toolsets improve GAIA Pass@1 by at most 1.28%, while AFLOW improves AIME25 by 3.33% and shows no improvement on AIME24.
  • Experience Memory further improves capability expansion, with Mem2Evolve outperforming Alita by 6.46% in average Pass@1 under matched conditions.

5 Analysis

The analysis tests module necessity, experience-guided asset creation, and memory reuse within and across tasks. Results show that both dual-memory components and prior experience improve reliability, performance, and evolutionary stability.

  • RQ1: Ablation Study: Removing Tool Creation causes the largest ablation drop, 10.28%, while removing Expert Agent Creation reduces performance by 1.72%.
  • RQ1: Ablation Study: Removing Agent Memory causes a 4.73% drop and removing Tool Memory causes a 3.13% drop, limiting reuse of validated successes and past failures.
  • RQ2: Experience-Guided Asset Creation: Experience guidance reduces AIME debugging iterations by nearly 68% and raises first-pass validity above 82%.
  • RQ2: Experience-Guided Asset Creation: On GAIA, experience guidance improves first-pass validity by 56.0% relative to the setting without experience guidance.
  • RQ3: Single Task Self-Evolving: Within-task initial memory consistently improves performance, with most gains appearing from relatively small memory amounts before diminishing incremental improvements.
  • RQ4: Cross Tasks Self-Evolving: Heterogeneous memory from GAIA improves performance across seven target benchmarks and reaches results comparable to 25% single-task initialization without negative transfer.

6 Conclusion

Mem2Evolve integrates Asset Memory and Experience Memory to coordinate capability expansion with experience distillation. The framework supports sustained improvement across single-task and cross-task settings, while relying on sandbox execution that limits deployment scope.

  • 6 Conclusion: Mem2Evolve coordinates Asset Memory and Experience Memory to expand capabilities while accumulating strategic experience.The framework is designed for self-evolution with reduced reliance on human intervention.
  • 6 Conclusion: The framework continuously expands its capability space and uses accumulated experience to support stable, sustained performance improvements.
  • 6 Conclusion: Mem2Evolve improves performance in both single-task and cross-task settings.
  • 6 Conclusion: Mem2Evolve relies on a sandbox environment to execute autonomously generated code, limiting deployment in open-world settings requiring local files or unrestricted network access.

A.2 Evaluation of Existing Self-Evolving Agent Frameworks

Existing self-evolving frameworks cover different combinations of optimization, agent creation, tool creation, and experience persistence. The surveyed methods therefore vary in whether they generate assets, retain experience, or guide creation with prior experience.

  • A.2 Evaluation of Existing Self-Evolving Agent Frameworks: DyLAN and DSPy optimize agent systems, but their approaches differ in collaboration topology optimization versus compiler-based instruction or weight refinement.
  • A.2 Evaluation of Existing Self-Evolving Agent Frameworks: DSPy relies on user-defined structures and does not autonomously synthesize new agent roles or tools or provide continuous experience persistence.
  • A.2 Evaluation of Existing Self-Evolving Agent Frameworks: ReasoningBank provides long-term memory by distilling reasoning patterns from successful and failed trajectories for cross-task transfer and error avoidance.
  • A.2 Evaluation of Existing Self-Evolving Agent Frameworks: AFlow and AgentSquare search over workflow or modular design spaces to optimize agent architectures and compositions.
  • A.2 Evaluation of Existing Self-Evolving Agent Frameworks: ANN updates agent prompts through textual backpropagation but operates on predefined agent structures rather than creating new agents or tools.
  • A.2 Evaluation of Existing Self-Evolving Agent Frameworks: Alita and ToolMaker generate executable tools, while ToolMaker verifies generated code with unit tests but does not synthesize new agent roles or preserve cross-task experience.
  • A.2 Evaluation of Existing Self-Evolving Agent Frameworks: AgentVerse, AutoAgents, and SwarmAgentic autonomously create specialized agents, with SwarmAgentic also synthesizing executable tools through swarm-based search.

A.3 Task Planning

Mem2Evolve plans complex tasks as dependent subtasks, recruits relevant stored assets, and creates new tools when existing capabilities are insufficient. Tool creation proceeds through specification, documentation and experience collection, then implementation.

  • A.3 Task Planning: Complex tasks are decomposed into subtasks with objectives, expected outputs, and dependencies on prerequisite results.
  • A.3 Task Planning: When a subtask exceeds current capabilities, Mem2Evolve initiates tool creation because descriptions alone produce unstable and insufficiently grounded tools.
  • A.3 Task Planning: Tool synthesis has three stages: specification generation, documentation and experience collection, and implementation.
  • A.3 Task Planning: Tool specifications formalize names, descriptions, inputs, outputs, and execution logic before implementation.
  • A.3 Task Planning: The framework grounds tool generation with external documentation and retrieved experience from Experience Memory.
  • A.3 Task Planning: Generated tools are encapsulated in an MCP-compliant format for model-agnostic integration and reuse.
  • A.3 Task Planning: Before execution, Assets Recruitment retrieves stored tools and expert agents at the subtask level using embedding-based relevance.
  • A.3 Task Planning: If no expert agent exceeds the similarity threshold, the system triggers new agent generation; retrieved agents then execute through a ReAct loop using available tools.

B Experimental Details

The evaluation compares direct, reasoning, acting, and self-evolving agent frameworks spanning optimization, asset creation, and experience use. Baselines differ in whether they rely on fixed structures, search, feedback, or autonomous synthesis.

  • B Experimental Details: The evaluation includes Direct, CoT, ReAct, OpenAI Deep Research, and multiple self-evolving frameworks.
  • B Experimental Details: Direct feeds the task directly to the LLM without intermediate reasoning or external tools, measuring zero-shot capability.
  • B Experimental Details: CoT prompts intermediate reasoning steps, while ReAct interleaves reasoning and actions to retrieve information and update context.
  • B Experimental Details: DyLAN optimizes collaboration topology by identifying high-contribution agents and pruning low-performing ones from a fixed pool.
  • B Experimental Details: EvoAgent evolves agent prompts through crossover and mutation to discover more effective personas and strategies.
  • B Experimental Details: AFlow uses Monte Carlo Tree Search to optimize code-represented workflow structures and prompts based on search history.
  • B Experimental Details: DSPy refines instructions or model weights with metric-driven feedback but relies on user-defined program structures.
  • B Experimental Details: Alita generates executable Python tools and uses prompts plus an Experience Pool for continuous adaptation.

B.2 Benchmarks

The experiments cover six task categories and eight benchmarks, using benchmark-specific test sizes and evaluation metrics to assess general-purpose agent capabilities.

  • Benchmark coverage: The benchmark suite spans general assistance, embodied interaction, multi-hop question answering, mathematical reasoning, travel planning, and e-commerce.It includes GAIA, ALFWorld, HotpotQA, AIME 24/25, TravelPlanner, and WebShop.
  • Benchmark details: GAIA evaluates real-world assistant tasks with 466 questions using Pass@1.Its questions cover daily tasks, scientific reasoning, web browsing, and tool usage.
  • Benchmark details: ALFWorld evaluates interactive reasoning and action across 134 tasks using Success Rate.Agents must translate high-level goals into low-level actions in embodied environments.
  • Benchmark details: HotpotQA uses 500 randomly sampled test instances and evaluates multi-hop document reasoning with Exact Match.The benchmark requires locating relevant facts across multiple documents.
  • Benchmark details: TravelPlanner uses 1,000 test instances and reports an average of five delivery and commonsense or hard-constraint metrics.The five metrics include Delivery Rate, Micro/Macro Commonsense Constraint Pass Rate, and Micro/Macro Hard Constraint Pass Rate.
  • Benchmark details: WebShop evaluates web navigation and product selection on 251 test instances using Success Rate, while AIME 24/25 measure mathematical accuracy with Pass@1.AIME 24/25 each contain 30 high-difficulty problems.

B.3 RQ1: Ablation Study

The ablation section reports Pass@1 drops relative to the full model and describes a staged tool-evolution workflow that prioritizes native reasoning, tool reuse, and minimal creation of missing capabilities.

  • Ablation design: The ablation study reports Pass@1 scores and shows each performance drop relative to the full Mem2Evolve model.The supplied table description does not include the individual ablation values.
  • Tool-selection policy: LLM-native reasoning has highest priority, covering planning, decisions, extraction, reformatting, filtering, sorting, summarization, and simple transformations.The framework avoids proposing tools for capabilities that the language model can perform from context.
  • Capability analysis: The workflow decomposes a plan into atomic required capabilities before evaluating how each capability can be covered.The process begins by analyzing the full plan and deconstructing each step.
  • Tool creation: New tools are limited to capabilities involving complex calculations, stateful simulation, complex data manipulation, external I/O, or deterministic logic.When all capabilities are covered, the workflow sets need_creation=false and returns no missing tools.
  • Tool-selection policy: Existing tools are reused before new creation, and a capability is considered a gap only when neither native reasoning nor available tools can cover it.This policy is intended to prevent duplicate or near-duplicate tools.
  • Tool specification: Tool specifications require typed inputs and outputs, explicit core logic, non-overlapping names, and one specific operation per tool.The specification and validation rules emphasize reusable interfaces and exact type contracts.

D.1 Tool Implementation for Simulate Piston Platform Game

The simulated piston-platform tool uses Monte Carlo trials to estimate each numbered ball’s probability of being ejected rather than released, then selects the highest-probability ball.

  • Tool purpose: The tool simulates a ping-pong mechanism with a ball queue, a three-position platform, and random piston actions.Its purpose is to estimate which ball is most likely to be ejected under the specified replacement rules.
  • Inputs and validation: Inputs are num_balls, the total number of numbered balls, and num_simulations, the number of Monte Carlo iterations.Both inputs are validated as positive integers.
  • Simulation procedure: Each simulation initializes the platform, repeatedly applies piston rules while balls remain, and tracks winning ejections by ball number.The platform is initially filled with up to three balls and is refilled from the queue after piston events.
  • Piston rules: Piston 1 ejects position 1, piston 2 ejects position 2 while releasing position 1, and piston 3 ejects position 3 while releasing position 1.The remaining platform positions shift and the queue supplies replacement balls according to each rule.
  • Outputs: The output contains win_probabilities for each ball and best_choice, the ball with the highest estimated probability.Probabilities are computed by dividing each win count by num_simulations.

D.2 Tool Implementation for Youtube Audio Transcriber

The YouTube Audio Transcriber downloads a video’s audio, processes it in chunks, recognizes speech, and returns transcription text with video and file metadata for questions requiring internal video content.

  • Motivation: The transcriber addresses questions requiring semantic analysis of YouTube audio, such as identifying a number mentioned after an event in a video.The motivating case asks about a March 2018 YouTube 360 VR video narrated by the voice actor of Gollum.
  • Inputs and setup: The tool accepts a YouTube URL, language code, and chunk duration, then validates the URL and extracts its video ID.It supports youtube.com and youtu.be URL forms and raises an error when no video ID can be extracted.
  • Audio acquisition: It downloads the best available audio, converts it to 192-kbps MP3, extracts metadata, and locates the downloaded file in a workspace.The workflow uses yt-dlp options for audio extraction and records fields such as title, uploader, and duration.
  • Transcription: The audio is split into chunks, each chunk is converted to WAV, and SpeechRecognition transcribes it with the selected language code.Unknown audio, API errors, and unexpected exceptions are handled during chunk processing.
  • Outputs and cleanup: The returned object includes transcription_text, video_metadata, local_path, and file_size_mb, while temporary files are cleaned up.Error handling also removes temporary directories and can return an empty transcription with available metadata.
  • Experience-guided extension: A related video-analysis tool samples frames at fixed intervals and applies GPT-4o with a task-specific prompt to analyze extracted screenshots.The case study connects this capability to experience-guided creation from an existing GPT-4o image-analysis memory item.

D.4 Comparison of Tool Generation With and Without Experience Guidance

Experience Memory guides Mem2Evolve to generate an Excel-parsing tool that preserves color attributes needed for color-coded tasks. Without this guidance, the generated basic reader omits style information and the task fails.

  • With Experience Guidance: Experience guidance enables Mem2Evolve to generate a tool that extracts color-coded cells together with their original color information in a standardized format.The case study describes this as an experience-guided tool-generation process for attribute-preserving Excel parsing.
  • Without Experience Guidance: Without experiential guidance, the pandas-based Excel reader returns content but fails to capture visual styles required for color-based riddles, leading to unsuccessful execution.The unguided implementation explicitly notes that visual styles, including colors and fonts, are not extracted.
  • With Experience Guidance: The guided implementation processes Excel workbooks with openpyxl and constructs styled HTML output from cell values and extracted styles.The implementation validates files, iterates through worksheets and cells, and embeds style information in the returned HTML.
  • With Experience Guidance: The tool extracts font and background colors, converts ARGB values to RGB, and preserves them as inline CSS styles.It also handles bold and italic formatting when constructing each cell's style string.
Loading 2604.10923v1…