PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoMem: A Text-Gradient Recursive Self-Improvement Framework for Automated Memory Architectures Search

Lin Du, Jie Zhou, Yuxuan Cai, Kai Chen, Qin Chen, Xin Li, Bo Zhang, Wei Li, Liang He

arXiv:2608.14621v1cs.CLcs.AI

TL;DR

LLM agent memory architectures are manually designed despite task- and backbone-dependent interactions among encoding, storage, retrieval, and management. AutoMem searches these modules through experience-guided proposals and failure-guided diagnosis, consistently outperforming manually designed baselines across benchmarks and backbones while improving efficiency in reported analyses.

  • Problem

    Manually fixed memory architectures neglect task-dependent interactions among encoding, storage, retrieval, and management modules, while exhaustive search is expensive.

  • Method

    AutoMem recursively searches factored memory architectures using experience-guided proposals and failure-guided diagnosis that converts rollout failures into module-level textual feedback.

  • Results

    AutoMem discovers task-adaptive architectures that outperform the strongest manually designed baselines across all three benchmarks and both backbones, improving accuracy by 2.8 points on average.

  • Takeaways & Limitations

    Task-adaptive memory architecture search can identify stronger memory designs across varied tasks and backbones under limited evaluation budgets.

  • Takeaways & Limitations

    AutoMem searches combinations of known memory mechanisms rather than generating entirely new modules, interfaces, update rules, or cross-module interaction patterns.

Abstract

from arXiv · show

Long-term memory is increasingly central to LLM agents, yet memory design remains a highly coupled architecture problem: what to encode, how to store it, how to retrieve it, and how to manage it can vary substantially across tasks and backbone models. We construct a discrete search space with 5 encoders, 5 stores, 6 retrievers, and 4 managers, and show that no single memory architecture consistently dominates: different tasks favor different module combinations, leading to substantial performance gaps. Motivated by this, we propose \textsc{AutoMem}, a text-gradient recursive self-improvement framework for task-adaptive memory architecture search. \textsc{AutoMem} optimizes over the factored space through two components: Experience-Guided Architecture Search, which proposes candidate architectures from historical search trajectories and accumulated reflections, and Failure-Guided Module Diagnosis, which localizes memory-related failures to specific modules and converts them into targeted textual feedback. Experiments on GAIA, WebWalkerQA, and xBench-DeepSearch across two LLM backbones show that \textsc{AutoMem} consistently discovers task-adaptive memory architectures that outperform the strongest human-designed memory baselines, improving accuracy by $2.8$ points on average across six benchmark-backbone settings. Further analysis shows that \textsc{AutoMem} achieves a favorable accuracy-efficiency trade-off, reducing token cost by $14.3\%$ over the strongest accuracy baselines under Qwen3.5-122B-A10B, while also finding stronger architectures than substantially larger random searches within only a few guided iterations.

1 Introduction

Long-term memory effectiveness depends on coupled choices across encoding, storage, retrieval, and management, yet most systems are manually designed and fixed. AutoMem addresses this problem with task-adaptive architecture search that uses experience-guided proposals and failure-guided module diagnosis.

  • Motivation: Memory systems can fail when information is poorly encoded, representations are difficult to retrieve, retrieval selects irrelevant memories, or memory pools contain stale, duplicate, or conflicting entries.These failure modes motivate optimizing the full memory architecture rather than isolated components.
  • Memory architecture search: The search space factorizes memory into Encode, Store, Retrieve, and Manage modules, with 5 encoders, 5 stores, 6 retrievers, and 4 managers.Retrieve selects relevant memories, while Manage handles consolidation, deduplication, eviction, and conflict resolution.
  • Pilot study: Pilot results show architecture choice causes substantial performance variation, with the best architecture differing across datasets and backbone models.Module effects are strongly coupled: a strong retriever can fail with low-quality encoded memories, and a useful store format can fail when retrieval is inadequate.
  • AutoMem: AutoMem proposes candidate architectures from historical search trajectories, evaluated architectures, performance records, and accumulated reflections.This experience-guided component is part of a text-gradient recursive self-improvement framework for task-adaptive memory architecture search.
  • AutoMem: Failure-Guided Module Diagnosis analyzes failed rollouts, localizes memory-related failures to Encode, Store, Retrieve, or Manage, and converts them into targeted feedback.The framework combines diagnosis with experience-guided architecture proposals to identify task-adaptive memory architectures under limited evaluation budgets.

2 Related Work

Prior work develops diverse memory mechanisms for LLM agents, adaptive memory systems, and textual-gradient optimization. AutoMem differs by searching decomposed memory modules rather than relying on monolithic implementations or optimizing a single text object or parameter.

  • Memory architectures for LLM agents: LLM-agent memory methods store episodic experience, distill reflections, induce procedural skills, or share structured experience across agents.These approaches summarize dialogues or trajectories, reuse verbal self-reflections and cross-task insights, create reusable workflows or cheatsheets, or share experience across agents.
  • Self-Evolving Memory: Self-evolving memory work meta-evolves monolithic implementations from execution logs, limiting module-level credit assignment and controlled edits.Zhang et al. synthesize candidate memory implementations and select them through tournament evaluation.
  • Memory architectures for LLM agents: Prior memory systems instantiate only a few components and leave remaining modules at generic defaults, whereas AutoMem searches all four modules.Table 1 decomposes systems into encoder, store, retriever, and manager modules, with lifecycle operations represented in the Manage column.
  • Text-gradient Optimization: Textual-gradient methods use natural-language feedback to optimize prompts, computation graphs, or pipeline parameters rather than decomposed memory architectures.PROTEGI critiques and edits prompts, TEXTGRAD propagates feedback through computation graphs, and OPRO and DSPY optimize prompts or pipeline parameters.

3 Preliminary Analysis

The analysis represents long-term memory as a feasible combination of Encode, Store, Retrieve, and Manage modules, then shows that strong architectures are task-specific, coupled, and costly to discover through blind search. These findings motivate task-adaptive architecture selection over the valid module-combination space.

  • Modular memory architecture: Memory architectures are tuples a = (e, s, r, m) spanning Encode, Store, Retrieve, and Manage modules.The modules respectively govern what is written, how it is represented, how it is selected, and how the memory pool evolves.
  • Feasible architecture space: The feasible search space filters the Cartesian product E × S × R × M by Valid(e, s, r, m) = 1.This excludes incompatible combinations, such as graph-based retrieval with a non-graph store.
  • Random-search findings: 69.7% on GAIA exceeds the strongest same-backbone fixed-memory baseline, 67.8% for MemoryBank.The random-search probe therefore finds architectures beyond the ceiling represented by fixed memory designs.
  • Task specificity: Top configurations differ across GAIA, WebWalkerQA, and xBench-DeepSearch in all four modules, and a strong system on one benchmark can underperform the no-memory baseline on another.The preferred combinations can vary from workflow encoding with hybrid storage and semantic retrieval to shortcut encoding with graph storage and contrastive retrieval.
  • Coupling and search limitations: Module effects are coupled: each module’s value depends on the information, representation, and operations supplied by the other modules.The study also finds that random search wastes evaluation budget because failed trials do not inform subsequent independently sampled candidates.

4 Method

AutoMem searches task-adaptive memory architectures through text-gradient recursive self-improvement over discrete encoder, store, retriever, and manager choices. It combines experience-guided candidate generation with failure-guided module diagnosis, evaluates candidates on accuracy, memory gain, and cost, and selects a validated architecture.

  • Overall framework: AutoMem evaluates feasible architectures a = (e, s, r, m) within a budget T, seeking maximum task accuracy with positive memory-induced gain over the no-memory agent.Search uses disjoint search, validation, and test batches.
  • Optimization loop: Each round evaluates K candidates on accuracy, memory-induced gain, cost, and rollout traces, updates the Pareto front, periodically validates its leader, and feeds failed traces into the next diagnosis.The final architecture is selected by validation performance and reported on the held-out test batch.
  • Experience-Guided Architecture Search: Experience-Guided Architecture Search proposes valid tuple-level edits using the Pareto front, experience ledger, observation graph, and latest textual feedback.A deterministic checker rejects invalid proposals before evaluation, while the ledger stores reusable evidence and the graph maintains task-pattern/module associations as soft priors.
  • Failure-Guided Module Diagnosis: Failure-Guided Module Diagnosis converts failed rollouts into targeted textual gradients by filtering non-memory failures, attributing in-scope failures to modules, and recommending valid edits.Attribution uses trace signals covering extraction, retrieval, consistency judgment, and staleness, with evidence, confidence, and cross-source agreement recorded in the synthesized diagnosis.

5 Experiments

Experiments evaluate AUTOMEM across three challenging search and reasoning benchmarks and two task-agent backbones. AUTOMEM achieves strong accuracy, efficiency, search efficiency, component effectiveness, and task-adaptive memory routes.

  • Datasets and Implementation: Experiments cover GAIA, WebWalkerQA, and xBench-DeepSearch using Qwen3.5-122B-A10B and gpt-5.1-mini backbones.GAIA supports difficulty-stratified analysis, WebWalkerQA tests deep web navigation, and xBench-DeepSearch provides an external robustness check.
  • Main Results: 71.5% on GAIA, 46.0% on xBench, and 72.5% on WebWalkerQA are AUTOMEM’s Qwen3.5-122B-A10B accuracies, besting manually designed baselines.The comparison is made within each backbone group to avoid conflating memory design with backbone capacity.
  • Search Efficiency: 71.5% after five evolution rounds surpasses 69.7% from ten random trials, while AUTOMEM improves monotonically from 55.8% to 71.5%.Random search is unstable, with most sampled architectures performing below the No-Memory baseline of 65.0%.
  • Search Efficiency: 0.55× and 0.47× are AUTOMEM’s search token budgets relative to random search on GAIA and xBench-DeepSearch, respectively.A full directed evolution run consumes 142.7M tokens on GAIA and 160.1M on xBench-DeepSearch, versus 259.0M and 339.0M for ten random architectures.
  • Ablation: 64.2% without FGMD and 57.9% without EGAS show that both module diagnosis and accumulated search experience are necessary.Removing FGMD nearly doubles token cost from 128.4k to 226.7k, while removing EGAS produces the highest token cost among variants.
  • Discovered Architectures: Different benchmarks favor different Encode/Store/Retrieve/Manage combinations, supporting task-adaptive rather than fixed memory architectures.GAIA benefits from relational graph organization, WebWalkerQA from structured factual storage and tool validation, and xBench-DS from case reuse and reranking.

6 Conclusion

The paper frames long-term memory design for LLM agents as a task-adaptive architecture search problem, motivated by the sensitivity of fixed human-designed architectures to tasks and backbone models. It factorizes memory into Encode, Store, Retrieve, and Manage modules and proposes AUTOMEM as a text-gradient recursive self-improvement framework.

  • Long-term memory design for LLM agents is formulated as a task-adaptive architecture search problem.
  • Agent memory is factorized into Encode, Store, Retrieve, and Manage modules.
  • Fixed human-designed memory architectures are highly sensitive to task distributions and backbone models, with no single design consistently dominating across benchmarks.
  • AUTOMEM is proposed as a text-gradient recursive self-improvement framework for this memory architecture search problem.

7 Limitations and Future Work · A Experience Extraction Prompts · B Worked Examples from the Search State

AutoMem’s current design searches one architecture per task and recombines known memory mechanisms, leaving finer-grained adaptation and open-ended module generation for future work. The appendices document the extraction prompt format and provide runtime search-state examples from three benchmarks.

  • 7 Limitations and Future Work: AutoMem currently searches memory architectures at the task or benchmark level, applying one Encode/Store/Retrieve/Manage path across samples.This is effective and efficient when tasks share similar memory requirements, but individual samples may differ in their need for memory granularity.
  • 7 Limitations and Future Work: Future work could support sample-level architecture selection to accommodate differing memory-granularity needs across individual samples.The current task-level design may not reflect variation among samples within the same task distribution.
  • 7 Limitations and Future Work: AutoMem’s search space uses components inspired by existing memory frameworks, improving interpretability, controllability, and comparability with human-designed systems.This construction also constrains discovery primarily to combinations of known memory mechanisms.
  • 7 Limitations and Future Work: Future work could explore open-ended memory architecture generation, including entirely new modules, interfaces, update rules, or retrieval mechanisms.This would move beyond recombining existing memory components.
  • A Experience Extraction Prompts: The experience-extraction prompt asks an agent to select tip, insight, trajectory, workflow, or shortcut memories and emit structured MemoryUnits from an observation graph and task.Example units produced by these prompts are shown in Appendix B.5.
  • B Worked Examples from the Search State: The worked examples present verbatim, lightly abridged AutoMem search-run artifacts from GAIA, WebWalkerQA, and xBench-DeepSearch.Ellipses mark omitted fields, floating-point digits are truncated, and non-ASCII symbols are rendered in ASCII; embedded accuracies are intermediate small-batch statistics, not reported results.

B.1 One Optimization Round, End to End

Across one end-to-end optimization round, AutoMem uses failure-guided diagnosis to identify retrieval as the bottleneck, proposes controlled retrieval-only edits, and promotes cbr_rerank after paired evaluation. The round also records a retrieval principle and a follow-up comparison against hybrid retrieval.

  • Round 2 diagnosis: At round 2, the incumbent extracts tip, trajectory, and insight units into all-JSON storage with hybrid retrieval and lightweight management, while budget_capped failures are excluded from edits.The scope gate tags budget_capped failures as out-of-scope.
  • Round 2 diagnosis: 9 retrieval_noise failures made Retrieve the primary bottleneck, while accuracy rose only from 0.40 to 0.433.The gradient recommends changing retrieval from hybrid to contrastive.
  • Round 3 proposals: Round 3 proposes single-coordinate retrieval edits—hybrid -> contrastive and hybrid -> cbr_rerank—while holding every other module fixed.The controlled comparison follows the diagnosed retrieval bottleneck and direction.
  • Round 3 evaluation: 0.500 accuracy for cbr-rerank surpassed the incumbent’s 0.433 and contrastive’s 0.467, making cbr-rerank the new front leader.Both round-3 edits improved on the incumbent; cbr_rerank became the front leader.
  • Ledger update: With extraction and storage held constant, cbr_rerank beat contrastive on hit_rate, accuracy, memory_lift, and fitness, prompting a paired test against hybrid retrieval.The follow-up compares identical setups on accuracy, hit_rate, retrieval_noise, and fitness.

B.2 A Synthesized Textual Gradient

The synthesized textual gradient encodes a dominant memory bottleneck, supporting evidence, confidence, and a recommended action, while filtering out-of-scope failures and recording diagnostic agreement. In a WebWalkerQA round, it identifies retrieval noise as the primary failure mode and recommends switching from hybrid to dense-only retrieval.

  • Gradient structure: The gradient packages the bottleneck, evidence task IDs, confidence, recommended action, scope flag, and cross-source agreement into a structured diagnostic.Its fields instantiate (q⋆_t, E_t, ρ_t, R_t), while scope filtering and agreement tracking support diagnosis.
  • Diagnostic finding: 10/15 failures identify retrieval noise as the dominant failure mode, with hybrid retrieval returning off-topic units that pass the judge and causing 24% task failure.Three domain_knowledge_gap tasks also have zero relevance, but retrieval quality is the immediate bottleneck.
  • Recommended intervention: The recommended action is switching from hybrid to dense-only retrieval to reduce noise from sparse matching when relevant memory units are absent.The synthesized gradient assigns high confidence to this recommendation and does not bypass memory or classify the issue as out of scope.
  • Evidence synthesis: Rule-based and LLM diagnostics agree that retrieval noise is primary, supported by a dominant count of 10, a hybrid-to-dense suggestion, and evidence of irrelevant units passing the judge.The evidence task IDs include EHA_2023_meeting, SIGCHI_EC_meeting, and SIGCHI_CARES_Mullers.

B.3 Experience-Ledger Entries · B.4 An Observation-Graph Excerpt · B.5 Example Memory Units

The appendices illustrate how AutoMem records validated and refuted architecture principles, represents task-conditioned module evidence, and extracts guarded memory units ranging from declarative heuristics to executable macros. Together, these artifacts show how search experience is structured for reuse and future proposal generation.

  • B.3 Experience-Ledger Entries: Each ledger entry stores a claim, supporting evidence, and lifecycle status, including a seven-round principle with 20 evidence entries and confidence 0.86.The entry encodes the insight that higher retrieval hit-rate does not necessarily imply higher accuracy.
  • B.3 Experience-Ledger Entries: A paired comparison supported the claim that higher hit_rate did not improve accuracy when failures were dominated by reasoning errors.The comparison recorded accuracy delta -0.04 despite a hit_rate advantage of 0.149 for the compared candidate.
  • B.3 Experience-Ledger Entries: A refuted principle records that a lightweight combo beat the hybrid + tool_manager combo on accuracy and fitness despite lower hit_rate.The evidence includes a round-6 accuracy delta of 0.08 and a round-7 accuracy delta of -0.10 across repeated paired comparisons.
  • B.3 Experience-Ledger Entries: A dead-end ledger entry excludes further retrieval edits when hit_rate=0.929 but 91% of failures were reasoning_error rather than retrieval failures.The recorded architecture achieved accuracy=0.36, memory_lift=-0.08, and fitness=0.286.
  • B.4 An Observation-Graph Excerpt: After six WebWalkerQA rounds, the observation graph contained two task-pattern nodes and 32 edges whose trial counts and running accuracies formed a soft prior for module choices.For Lcomplex, the best accuracy so far was 0.667 versus a baseline accuracy of 0.778.
  • B.5 Example Memory Units: Memory units span declarative tips and insights, procedural trajectories and workflows, and executable shortcuts, with use_when and avoid_when guards checked during injection.Insights can be extracted from failed tasks, while trajectories, workflows, and shortcuts increase abstraction from grounded traces to reusable macros.
  • B.5 Example Memory Units: The examples operationalize memory through root-cause correction, multi-step templates, and input-schema-driven tool macros for date-anchored factual lookup.The shortcut defines web_search followed by final_answer, while the insight recommends primary-repository evidence plus independent authority verification.

C Implementation Details · D Algorithm Details

The implementation defines a factored memory-architecture search space over encoders, stores, retrievers, and managers, with validity enforced by cross-module constraints. The algorithm details provide pseudocode for the optimization loop and diagnostic pipeline, alongside the observation-graph update rule and ledger schema.

  • C Implementation Details: The search space is factored over encoder, store, retriever, and manager modules.Architectures are represented as (E, S, R, M).
  • C Implementation Details: The implementation specifies memory schemas, storage backends, retrieval strategies, and management operations.These details define the concrete memory-system components used by the search space.
  • C Implementation Details: The implementation also defines an observation-graph update rule and a ledger schema.These structures support tracking and updating the architecture-search process.
  • C Implementation Details: Each module has a selectable component menu whose options draw on prior work.Table 5 documents the per-module menu and its lineage.
  • C Implementation Details: The proposer can select and route every component option within the factored search space.The component menu is designed for proposer-driven architecture construction.
  • C Implementation Details: Cross-module constraints keep every architecture path valid by construction.For example, graph retrieval requires a graph-family store.
  • D Algorithm Details: The algorithm details include pseudocode for the optimization loop.The pseudocode specifies the iterative optimization procedure.
  • D Algorithm Details: The diagnostic pipeline is likewise described through pseudocode.This provides an explicit procedural account of diagnosis within the framework.

E Supplementary Experiments and Reproducibility

The supplementary experiments establish transparent, comparable search budgets and reproducibility protocols for AutoMem, random search, and fixed baselines. Evaluation uses disjoint, stratified splits with a fixed seed, with dataset-specific memory seeding and unseen websites for WebWalkerQA.

  • Search budget and reproducibility: AutoMem, random search, and fixed baselines are compared under transparent, comparable search budgets.These details correspond to §5.3.
  • Search budget and reproducibility: All splits are disjoint and stratified, using fixed seed 42.The protocol is designed to support reproducible comparisons.
  • Dataset-specific protocols: On WebWalkerQA and xBench-DS, the memory pool is seeded by the first search round rather than a separate warm-up split.This dataset-specific procedure replaces a separate warm-up split.
  • Dataset-specific protocols: On WebWalkerQA, validation and held-out tasks come from websites unseen during search.The protocol separates evaluation websites from those encountered during search.
Loading 2608.14621v1…