Self-Evolving LLM Memory Extraction Across Heterogeneous Tasks

• Static extraction rules are tailored to narrow contexts: conversational agents prioritize personal facts and preferences, while agentic systems focus on reusable skills and strategies.Source: §1 · p. 1
• Extraction quality is measured by how well a generation model uses the extracted memory to answer associated target queries.Source: §1 · p. 1, Figure 1 · p. 2
• On BEHEMOTH, no single static extraction prompt dominates across task categories, and CluE consistently outperforms prior self-evolving frameworks on heterogeneous tasks.Source: §1 · p. 2
• BEHEMOTH covers 18 datasets across personalization, problem-solving, and agentic tasks for systematic evaluation of heterogeneous memory extraction.Source: §1 · p. 2
• CluE clusters training examples by summarized extraction scenarios, analyzes each cluster locally, then synthesizes cross-cluster insights to update prompts.Source: §1 · p. 2
• GEPA (Agrawal et al., 2025), ACE, and MemEvolve (Zhang et al., 2025c) assume homogeneous tasks; frequent updates risk overfitting, while infrequent ones slow adaptation.Infrequent updates also dilute feedback across dissimilar examples.Source: §1 · p. 2

2.1 LLM Memory

• Existing memory systems extract personalized facts or experiential strategies using fixed domain-specific rules, while benchmarks do not cover all interaction scenarios under a unified protocol.Examples include Mem0 (Chhikara et al., 2025) and A-Mem (Xu et al., 2025) for personalized or factual memory, and ReasoningBank (Ouyang et al., 2025) and Dynamic Cheatsheet (Suzgun et al., 2025) for experiential or strategic memory. The paper proposes a heterogeneous benchmark and treats extraction instructions as learnable.Source: §2.1 · p. 3

2.2 Self-Evolving Frameworks

• Prompt and memory self-evolution methods refine prompts, memory designs, content, or skills, but commonly assume homogeneous task distributions.General prompt optimizers include APE (Zhou et al., 2023), OPRO (Yang et al., 2024), GEPA (Agrawal et al., 2025), and ACE; memory-focused examples include MemEvolve (Zhang et al., 2025c), ALMA (Xiong et al., 2026), Evo-Memory (Wei et al., 2025), and MemSkill (Zhang et al., 2026).Source: §2.2 · p. 3

3.1 Single-Step Memory Extraction

• Memory extraction is defined as producing a memory string from one source conversation and evaluating it as context for answering a target query, isolating extraction from retrieval and management.The target response is scored with a dataset-specific reward between 0 and 1 rather than an LLM-based judge.Source: §3.1 · p. 3
• The study evaluates both a static extraction prompt and a self-evolving prompt iteratively optimized on training conversation-query-reward triplets.Source: §3.1 · p. 3

3.2 Creating BEHEMOTH

• The curation method addresses the lack of heterogeneous memory benchmarks by enabling systematic evaluation of extraction generalizability across diverse tasks under a unified protocol.Existing memory benchmarks cover only homogeneous task distributions.Source: §3.2 · p. 4
• BEHEMOTH repurposes 18 datasets into three categories: personalization (5), problem-solving (7), and agentic (6), using Qwen3-32B to complete source conversations.Each benchmark instance is a conversation, target query, and reward triplet; the curation pipeline can be extended by changing the datasets or generation model.Source: §3.2 · p. 4, Figure 2 · p. 4
• The benchmark holds out LongMemEval, GPQA-Diamond, and ToolBench as out-of-distribution tests, while 330 sampled examples from remaining datasets form training data and the rest serve as in-distribution tests.The training set uses 20 examples per dataset, except AIME, which contributes 50; examples are randomly shuffled.Source: §3.2 · p. 4
• Because memory needs vary within and across categories, and the extractor sees no category label, it must infer the appropriate extraction behavior from the conversation itself.For example, successful and failed agent trajectories call for reusable strategies and pitfalls, respectively.Source: §3.2 · p. 4

3.3 Evaluation Metrics

• Evaluation fixes the model configuration across experiments and treats average dataset reward as a function of the extraction prompt when measuring generalizability across datasets.The metrics apply to both static and evolving prompt setups.Source: §3.3 · p. 4
• Macro accuracy weights datasets equally to measure absolute cross-domain performance, while relative gain aggregates per-dataset improvement ratios geometrically over a baseline prompt.The geometric mean normalizes away absolute score differences and limits the influence of outlier ratios.Source: §3.3 · p. 4

4 Evaluating Static Memory Extraction

• The evaluation compares hand-crafted or literature-derived prompts, including category-specific and taxonomy-based designs, using Qwen3-32B for both extraction and generation.The prompt groups include Mem0 and ReasoningBank, plus OpenMemory and Survey.Source: §4 · p. 5
• Simple raises overall macro accuracy from 37.84% to 46.00%, with gains across all three categories over the no-memory baseline.The results confirm the value of memory extraction.Source: §4 · p. 5
• Category-specific prompts excel in their target domains but underperform elsewhere, while taxonomy-based prompts yield more balanced but moderate improvements.The evaluated category-specific prompts are Mem0 and ReasoningBank; taxonomy-based prompts are Survey and OpenMemory.Source: §4 · p. 5
• OpenMemory’s five-class taxonomy does not outperform Survey’s simpler two-class design, so greater taxonomy detail does not guarantee better extraction.Extraction quality also depends on how well the extraction model can interpret and follow the prompt.Source: §4 · p. 5

5.1 Baselines

• GEPA (Agrawal et al., 2025) refines prompts from small batches of logs; ACE curates tagged rules, while MemEvolve (Zhang et al., 2025c) analyzes retrievable logs before proposing refinements.GEPA’s updates are batch-limited because the prompt and training logs must fit in the LLM context.Source: §5.1 · pp. 5–6

5.2 CluE: Cluster-based Evolution

• Under heterogeneous tasks, updating too often destabilizes prompts through conflicting batches, while updating too rarely dilutes feedback by averaging away task-specific insights.The existing frameworks were designed for homogeneous distributions with similar examples.Source: §5.2 · p. 6
• CluE summarizes extraction scenarios, clusters examples across dataset boundaries with labels that can be created, merged, or split, and independently analyzes each cluster for success and failure patterns.Scenario summaries focus on extraction needs and challenges, such as procedural knowledge, implicit information, or long contexts; clustering follows scenario rather than dataset or category.Source: §5.2 · p. 6
• Each CluE round evaluates the current prompt on a training batch, then synthesizes cross-cluster principles and scoped scenario-specific guidance into targeted prompt modifications.The Proposer resolves conflicting recommendations by scoping them to relevant memory categories rather than rewriting the prompt wholesale.Source: §5.2 · pp. 6–7

5.3 Results

• Table 2 evaluates self-evolving frameworks on in-distribution test sets starting from the Simple prompt; its RG metric denotes relative gain, in percent, over the Simple prompt.Source: Table 2 · p. 7
• Existing methods gained overall relative to Simple but traded category performance: GEPA gained +14.08% on agentic tasks and regressed −2.16% on personalization, while MemEvolve gained +10.76% on personalization and fell −3.25% on agentic tasks.Their gains on some categories came at the cost of regressions on others.Source: Table 2 · p. 7, §5.3 · p. 7
• 63.07 vs. 56.82 on LongMemEval was CluE’s strongest out-of-distribution result, and it did not fall below Simple on GPQA-Diamond or ToolBench.GEPA degrades on ToolBench, while ACE degrades on GPQA-Diamond.Source: Table 3 · p. 7, §5.3 · p. 7

6.1 Evolution From a Stronger Seed

• CluE achieves +6.54% overall relative gain from the Survey seed, while ACE and MemEvolve fall −1.44% and −0.74%, respectively.GEPA returns the seed unchanged; CluE improves across all three categories, whereas ACE and MemEvolve have notable category regressions.Source: Table 4 · p. 8, §6.1 · p. 8

6.2 From Single-Step to Continual Memory Extraction

• In continual extraction, CluE scores 50.83 vs. 43.33 on Game of 24 and 67.25 vs. 62.57 on AlfWorld, maintaining its advantage over MemEvolve.Simple falls below the No Memory baseline on average for Game of 24 or within its variance on AlfWorld; low-quality memories can accumulate and compound over time.Source: Table 5 · p. 8, §6.2 · p. 9

6.3 Qualitative Analysis of Evolved Prompts

• CluE’s evolved prompt combines a memory taxonomy, category-specific guidelines, and domain-agnostic principles, unlike GEPA’s domain-specific rules or MemEvolve’s lack of taxonomy.ACE’s prompt reaches 1,403 tokens, compared with 1,243 for GEPA, 936 for CluE, and 539 for MemEvolve; its added verbosity may weaken instruction-following. GEPA’s fixed context budget favors recent examples, while MemEvolve’s category-ungrouped feedback can cancel across tasks.Source: §6.3 · p. 9, Figure 4 · p. 9

7 Discussion and Conclusions

• The work formalizes single-step memory extraction, introduces BEHEMOTH, and proposes CluE to enable stable learning under heterogeneous training distributions.Extensive experiments demonstrate CluE’s effectiveness.Source: §7 · pp. 9–10
• BEHEMOTH could evaluate routing-based, skill-based, and other memory extraction approaches, while CluE may help agents handle heterogeneous demands across users or projects.The proposed uses extend beyond the settings studied in this work.Source: §7 · p. 10
• Future work should build more realistic benchmarks for greater task diversity and longer histories, and extend generalizable performance across storage, retrieval, and management.The current benchmark covers 18 datasets, and this work focuses on memory extraction as the first stage of the memory lifecycle.Source: §7 · p. 10

A.1 BEHEMOTH Curation

• For personalization, BEHEMOTH uses supplied conversations and queries; for problem-solving and agentic tasks, LLMg-generated trajectories serve as conversations and similar queries are retrieved.Each source conversation originates from a distinct query, though multiple examples may share a target query.Source: §A.1 · p. 14
• BEHEMOTH samples 20 training examples per dataset (50 for AIME), caps test sets at 200, shuffles datasets during evolution, and repeats evaluation three times when test sets are smaller.The three-run results are averaged; each run extracts memory with LLMe and answers target queries with LLMg.Source: §A.1 · p. 14

A.2 Self-Evolving Frameworks

• GEPA optimizes on 230 examples after reserving 100 for validation, using five-log reflection minibatches and validating candidates that improve on the minibatch.Optimization LLMs are the same as LLMe.Source: §A.2 · p. 14
• ACE processes all 330 training examples in one online epoch without validation, allowing up to three reflection rounds per incorrect attempt and updating its playbook every 25 steps.Its Curator applies add, update, or delete operations based on helpful and harmful signals.Source: §A.2 · p. 15
• MemEvolve runs five tournament rounds, while CluE uses the same hyperparameters: each round analyzes 35 examples, proposes three candidates, then evaluates four prompts on 35 new examples plus 10 stability samples.Source: §A.2 · p. 15

A.3 Continual Memory Extraction

• Continual and single-step No Memory baselines differ because they evaluate different example pools; AlfWorld also uses 1-shot extraction in continual evaluation but 0-shot answering in single-step evaluation.In continual AlfWorld, every example undergoes extraction with the demonstration; in single-step evaluation, the demonstration is removed when answering the target query, yielding a substantially lower No Memory baseline.Source: §A.3 · p. 15
• Single-step evolution uses memory-free source conversations, whereas continual extraction receives conversations containing retrieved memories, a mismatch the current pipeline does not address.The authors use §6.2 to test whether a prompt evolved in the single-step setting retains its advantages in continual use, and leave direct evolution under continual and heterogeneous conditions to future work.Source: §A.3 · p. 15

B.1 Generalizing Across Extraction Backends

• The evaluation repeats the evolution pipeline with Gemini-3-Flash as both extraction and optimization backend, starting from the Simple prompt.The study treats the extraction model and optimization LLM as a single memory extraction system and uses the same backend for both.Source: §B.1 · p. 15
• With Gemini-3-Flash, CluE achieves the highest overall relative gain (+3.40%) and leads Personalization (+6.66%) and Problem-Solving (+9.59%), while baselines regress or barely improve.The Simple baseline is higher with Gemini-3-Flash than with Qwen3-32B (50.46 vs. 46.00 overall macro accuracy); ACE drops to −3.93%, MemEvolve to −0.29%, and GEPA gains +0.93%.Source: §B.1 · pp. 15–16, Table 7 · p. 16
• On Agentic tasks, all evolved prompts fall below the Simple baseline, but CluE still substantially outperforms the No Memory baseline.The authors attribute the regression to limited headroom from the stronger backend's high Simple accuracy and the risk that over-specification hurts performance.Source: §B.1 · p. 16

B.2 Efficiency Analysis

• Efficiency is compared by wall-clock time, optimization LLM calls used exclusively for prompt optimization, and evaluation calls for target queries answered by LLMg during evolution.Source: §B.2 · p. 16
• CluE uses 221 optimization calls versus MemEvolve’s 30 and ∼5.5h versus ∼5.0h, adding marginal wall time while delivering the strongest performance gains.The comparison builds on MemEvolve’s tournament-based evolution and adds per-example summarization and cluster-based error analysis.Source: Table 8 · p. 16, §B.2 · p. 16
• GEPA incurs the highest evaluation cost (1,645 calls), largely from validation-set scoring for candidate selection, while ACE has fewest evaluation calls (1,120).As noted by Agrawal et al. (2025), GEPA’s validation-set scoring produces no learning signals.Source: §B.2 · p. 16
• ACE is slowest overall (∼12.4h) because extraction, reflection, and optional retries run sequentially, limiting parallelism.ACE has the fewest evaluation calls because it forgoes maintaining a candidate pool.Source: §B.2 · p. 16

Cluster Evolution and Taxonomy Construction

• The cluster pool consolidates from seven fine-grained clusters to four, merging scenarios that share extraction patterns, including emotional context with user preferences and puzzle solving with technical problem-solving.The final clusters are User Preferences and Emotional Context (with Translation); Factual Data Disambiguation and Verification; Procedural Knowledge in Virtual Environments; and Technical and Scientific Problem-Solving.Source: §B.3 · p. 17
• In a separate run from the Survey prompt, CluE split Code-based Technical Workflows from Technical Problem-Solving at Round 2 because implementation-specific patterns differ from general reasoning strategies.Examples of code-specific patterns include dependency management and error handling.Source: §B.3 · p. 17
• The proposer turns four clusters into five taxonomy sections, splitting the user-preference and translation cluster into separate categories to produce clean, non-overlapping categories.The five sections cover factual and temporal disambiguation, user preferences and emotional context, procedural and technical knowledge, logical and combinatorial reasoning, and translation and stylistic requirements.Source: §B.3 · p. 17

B.4 Expanded Results

• Tables 9–11 report per-dataset in-distribution accuracy, while category-level and overall aggregates are presented as more reliable than fluctuating individual-dataset scores.The stated rationale is that aggregates smooth out dataset-specific variance; the supplied passages do not provide table cell values.Source: §B.4 · p. 17, Table 9 · p. 18, Table 10 · p. 18

C.1 Static Memory Extraction Prompts

• The five static prompts range from Simple’s empty-taxonomy baseline to category-specific Mem0 and ReasoningBank and broader-taxonomy OpenMemory and Survey.OpenMemory uses five memory classes, while Survey uses factual and experiential classes.Source: §C.1 · p. 17, Figure 5 · p. 19, Figure 6 · p. 21

C.2 Prompts for CluE

• CluE’s cluster manager groups examples by extraction-scenario patterns rather than dataset, surface task, or domain, and its proposer synthesizes cluster analyses into one improved prompt.The supplied role prompts illustrate the cluster manager and proposer stages.Source: §C.2 · pp. 23, 25, Figure 11 · p. 24, Figure 13 · p. 26
• An example CluE prompt organizes memories into factual data and temporal disambiguation, user preferences and emotional context, and procedural and technical knowledge.Its guidelines include temporal anchors for facts and preferences and verbatim command syntax for procedural memories.Source: §C.2 · pp. 26–27
• For action workflows, the prompt rejects nonexistent actions and records state transitions, error handling, preconditions, recovery steps, and only source-described technical details.These rules cover object manipulation and multi-step workflows.Source: §C.2 · p. 27
• For logical and combinatorial reasoning, it requires line-by-line source checks, explicit constraints, validation methods, and markers distinguishing valid paths from dead ends.Physics memories also check sign conventions and units; multi-step calculations require at least two validation methods.Source: §C.2 · p. 27
• For translation and stylistic requirements, it extracts only explicit style preferences, derives linguistic markers from examples, and documents explicit dynamic-style changes.The guidance names tone and formality as examples of preferences.Source: §C.2 · p. 27
• Figure 14 presents a prompt evolved by CluE from Simple using Qwen3-32B.Source: Figure 14 · p. 28