PaperlayerView exact revision on arXiv

Source-linked AI summary

GenMem: Generative Symbolic Memory for Self-Evolving Harness

Xinke Jiang, Tao Feng, Weixuan Xu, Zhixin Zhang, Zhibang Yang, Wentao Zhang, Runchuan Zhu, Xu Chu, Junfeng Zhao, Yasha Wang

arXiv:2609.34633v1cs.LG

TL;DR

Memory agents must select useful experience from a sparse, redundant bank despite delayed feedback and content changes that can destabilize retrieval. GENMEM uses stable, generated symbolic addresses to coordinate retrieval and updates. It improves downstream task performance over memory-free and memory-augmented baselines in the reported benchmarks, while its conclusion notes limitations in code utilization and cold-start quality.

  • Problem

    Experience memory is sparse, redundant, and task-dependent, while delayed outcomes make memory operations difficult to supervise and content revisions can destabilize content-coupled retrieval.

    Source: §1 · pp. 1–2

  • Method

    GENMEM uses hierarchical Symbolic Identifiers as stable generated addresses for retrieving and revising experience, coupling MemRetriever and MemEvolver in a trained multi-agent harness.

    Source: §1 · p. 2

  • Results

    GENMEM improves task performance on ALFWorld, WebShop, and search-augmented QA, with average gains of 17.0 percentage points over memory-free baselines before StackPlanner training.

    Source: §4.2 · p. 7

  • Takeaways & Limitations

    Stable hierarchical addresses keep addresses fixed while content is refined, and combining SIDs can address novel situations not covered by any individual memory entry.

    Source: §6 · p. 10

    Some leaf codes attract disproportionate traffic, and noise in early trajectories can propagate into the SID space.

    Source: §6 · p. 10

Abstract

from arXiv · show

Long-term memory supports the self-evolution of LLM agents by retaining experience and skills across tasks and enabling their retrieval, reuse, and revision in subsequent long-horizon decision-making. Yet existing memory management approaches remain limited to discriminative retrieval and to address the sparse, hierarchical, and highly redundant structure of reusable experience: only a small, task-dependent subset of trajectories and memories warrants retention, retrieval, or revision. Learning these operations is further complicated by sparse, delayed, and indirect task-level feedback, with weak supervision across the memory lifecycle. Moreover, continual memory evolution introduces an architectural tension as addressing invariance: stored experience is perpetually revised, yet the addressing interface consumed by learned retrieval policies must remain stable. To address, we present GenMem, which reformulates memory management as generative symbolic addressing. Its core mechanism is the Symbolic Identifier (SID), a multi-level discrete token tuple drawn from a Cartesian-product address space that factorizes a million-scale sparse memory space using fewer than one hundred discrete symbols. Instead of generating ever-changing raw content, the memory agent learns to generate SIDs, while memory evolution rewrites the payload at a fixed address without shifting the address itself. Architecturally, GenMem couples a MemRetriever and a MemEvolver within a multi-agent harness, trained via GRPO with dense process and outcome rewards with two-channels optimization. Under offline memory evolution, experiments spanning ALFWorld, WebShop, multi-hop QA, medical reasoning, and deep research evaluate GenMem against strong memory-augmented baselines...

1 INTRODUCTION

GENMEM addresses sparse, redundant experience and delayed feedback while keeping addresses stable as memory content evolves. It uses generated symbolic identifiers to unify selective memory access and updates.

1 INTRODUCTION

  • Reusable experience is redundant and clustered around shared sub-skills, while delayed task outcomes make supervision of individual memory operations difficult.Only a small fraction of experience is reusable, query-relevant, or repeatedly refined.Source: §1 · p. 2
  • Addressing invariance requires addresses to remain stable as experience changes, so learned policies can map problems to functional roles rather than outdated content versions.Content-derived retrieval representations can change which queries reach an entry after revision.Source: §1 · p. 2
  • GENMEM’s motivating architecture addresses both sparse, shared-subskill experience and retrieval shifts caused by evolving content.Figure 1 presents these as the two challenges in evolving experience memory.Source: Figure 1 · p. 2, §1 · p. 2
  • GENMEM unifies retrieval and evolution through generated hierarchical Symbolic Identifiers (SIDs) over a sparse experience space.SIDs are discrete token tuples in a Cartesian-product address space, and payloads can be revised in place while addresses remain fixed.Source: §1 · pp. 2–3
  • GENMEM couples MemRetriever, a task-execution harness, and MemEvolver in a closed loop, with staged training and GRPO using process and outcome rewards.Online self-evolution keeps model parameters and codebooks fixed while the agents refine memory through the stable SID interface.Source: §1 · p. 2
  • The paper evaluates GENMEM against memory-agent baselines across embodied planning, web navigation, search-augmented QA, medical reasoning, and deep research.The introduction also identifies SID compositionality as an emergent property for addressing situations no individual memory entry covers.Source: §1 · p. 3

2 GENMEM: SYMBOLIC MEMORY ADDRESSING SYSTEM

GENMEM represents each experience with a hierarchical discrete SID and separates that stable address from its editable payload. The shared address space supports selective retrieval and memory updates.

2 GENMEM: SYMBOLIC MEMORY ADDRESSING SYSTEM

  • Each memory entry pairs a textual experience payload with an SID, a tuple of discrete symbols drawn from a product address space.The same symbolic addressing policy can use a query, trajectory, or distilled experience as context for retrieval or update targeting.Source: §2 · p. 3
  • Online revision changes an experience payload while preserving its SID, so the address identifies a memory slot rather than a particular content version.The codebooks remain fixed after construction.Source: §2 · pp. 3–4
  • A gradient-free cold start distills reusable procedures and failure lessons from successful, failed, and unverified trajectories into seed experiences.The extractor preserves uncertainty when traces lack evidence.Source: §2 · p. 3
  • RQ-KMeans constructs hierarchical addresses by quantizing experience embeddings into successive residual codebooks, with shared prefixes grouping related experiences.GENMEM uses codebook sizes (48, 16, 8, 8), yielding 49,152 possible addresses from 80 tokens.Source: §2 · p. 4
  • MemRetriever generates a retrieval SID and rewrites its associated experience as task-adaptive support, while MemEvolver generates an update SID and inserts or revises its payload.Revision can retain, consolidate, or delete an entry through unchanged, merged, or empty payloads.Source: §2 · p. 4

3 MEMORY POLICY OPTIMIZATION AND SELF-EVOLUTION

GENMEM trains MemRetriever and MemEvolver through staged alignment and memory-operation training, then jointly optimizes them with process and outcome feedback. Online evolution applies the learned policies while keeping model parameters and SID codebooks fixed.

3 MEMORY POLICY OPTIMIZATION AND SELF-EVOLUTION

  • GENMEM develops its memory agents through SID alignment pretraining, memory-operation mid-training, joint policy optimization, and online self-evolution.The first three stages teach agents to generate, use, and update symbolic memory; the final stage evolves the bank.Source: §3 · p. 4
  • Alignment pretraining teaches forward mappings from queries, trajectories, and experience to SIDs, plus reverse mappings from SIDs to experience and semantic descriptions.This establishes bidirectional associations for subsequent MemRetriever and MemEvolver training.Source: §3 · pp. 4–5
  • Memory-operation mid-training uses ReAct-style trajectories and teacher-distilled targets to teach each agent to reason about and execute SID-based operations.MemRetriever rewrites retrieved experience into support, while MemEvolver identifies transferable lessons and revises or proposes entries.Source: §3 · p. 5
  • Joint post-training combines process feedback on SID accuracy and strategic experience quality with outcome feedback from task performance using memory support.GRPO optimizes both agents; outcome reward contrasts StackPlanner performance with and without memory support.Source: §3 · p. 5
  • During online self-evolution, MemRetriever fetches support, StackPlanner executes, and MemEvolver updates payloads while model parameters and SID codebooks stay fixed.Updates target generated SIDs directly, without reassignment of existing addresses or gradient updates.Source: §3 · pp. 5–6

4 EXPERIMENTS

GENMEM is evaluated on interactive tasks and search-augmented QA, with additional tests of transfer and online memory evolution. It improves retrieval and downstream performance, while its fixed SID addresses accompany rising retrieval hit rates and task accuracy as memory evolves.

4 EXPERIMENTS

  • The evaluation asks whether GENMEM improves downstream performance and whether online evolution improves performance while maintaining stable SID addressing.It covers ALFWorld and WebShop, search-augmented QA, and a separate GPQA experiment; the setup describes task metrics, comparison methods, and training.Source: §4 · p. 6, §4.1 · p. 6

4.2 MAIN RESULTS (RQ1)

  • GENMEM has the highest candidate and final Top-1/Top-5 retrieval hit rates; its final Top-5 reaches 13.15%, versus 10.28% for SkillRouter, 12.52% for TF-IDF, and 2.89% for Qwen3-Embedding-0.6B.The comparison uses the same candidate budget and BGE reranking protocol.Source: §4.2 · p. 7
  • GENMEM gains 17.0 and 15.3 percentage points over corresponding memory-free baselines before and after Harness-RL, and exceeds SDAR by 3.7 points on average token F1.The comparisons cover ALFWorld, WebShop, and search-augmented QA; Table 1 reports interactive-task metrics and Table 2 reports QA F1.Source: §4.2 · p. 7, Table 1 · p. 7, Table 2 · p. 8

4.3 ONLINE SELF-EVOLUTION (RQ2)

  • Candidate Hit rises from 20.25% at t=0 to 25.58% at t=9, while Task ACC rises from 30.56% to 36.20% as GENMEM evolves its memory.The held-out query stream is disjoint from the initial bank, and model parameters, the SID encoder, and codebooks remain frozen; Table 4 tracks retrieval and accuracy across evolution checkpoints.Source: §4.3 · pp. 7–8, Table 4 · p. 9
  • Figure 3 tracks SID lineage, update concentration, operation types, and payload edit distance; the reported dynamics include selective writing and substantial content churn alongside GENMEM's performance gains.The figure describes cross-case reuse of revised memories, concentration of updates, insertions versus revisions and retention, and average payload edit distance.Source: §4.3 · p. 8, Figure 3 · p. 9

5 SIDS AS SYMBOLIC LANGUAGE: COMPOSITION& EMERGENT REASONING

On GPQA, accuracy rises as more SID prefix levels are supplied without experience text, and complete addresses outperform the raw query. Rewritten and directly retrieved experience achieve similar ranges, with rewriting higher only at k = 10.

5 SIDS AS SYMBOLIC LANGUAGE: COMPOSITION& EMERGENT REASONING

  • 54.06–54.66% accuracy with complete SID addresses, up from 48.32% for the raw query, is a 5.74–6.34-point gain without experience text.Accuracy increases with SID depth at each beam setting; the experiment supplies progressively deeper prefixes and evaluates k = 1, 5, and 10.Source: §5 · p. 9, Figure 4 · p. 9
  • Rewritten experience reaches 55.65–56.91% accuracy and directly retrieved experience 56.09–56.63%; rewriting scores higher only at k = 10.Both experience conditions retrieve from the memory bank, comparing rewritten support with directly supplied experience.Source: §5 · p. 9

6 CONCLUSION, LIMITATIONS AND FUTURE WORK

GENMEM frames memory as generative symbolic addressing: SIDs provide hierarchical addresses that stay fixed while experience payloads evolve. The appendix grounds this design in sparse experience structure, distinguishes task utility from similarity, and details SID construction and alignment.

6 CONCLUSION, LIMITATIONS AND FUTURE WORK

  • GENMEM uses hierarchical SID addressing to keep addresses stable while content is refined, and reports compositional SIDs reaching novel situations no single entry covers.The conclusion evaluates the design across embodied planning, web navigation, and multi-hop QA.Source: §6 · pp. 9–10
  • Uneven traffic across leaf codes and noise in early trajectories can affect codebook utilization and propagate into the SID space.The authors propose adaptive codebook allocation and a background revisit policy as future directions.Source: §6 · p. 10

A RELATED WORK

  • Prior work organizes and manages reusable memories, but similarity-based access may not reflect strategic utility, and delayed outcomes provide limited credit assignment to memory operations.Generative retrieval and discrete identifiers offer precedents, but do not jointly address stable mappings and targeted updates in evolving banks.Source: A RELATED WORK · p. 15

B.1 THE SPARSITY STRUCTURE OF EXPERIENCE SPACE

  • Experience value is concentrated in a small fraction of trajectories, while useful memories, shared structure, and warranted updates are sparse in different ways.GENMEM motivates structured addressing and selective evolution rather than dense global retrieval.Source: §B.1 · p. 16

B.2 RETRIEVAL UTILITY AND ADDRESS STABILITY

  • Similarity ranks representational agreement, not necessarily execution utility; separating payload from SID lets revisions preserve the lookup address.With addressing policy and codebooks fixed, revising a payload does not itself change the SID generated for unchanged context, though usefulness may change.Source: §B.2 · p. 16

B.3 SID AS A GENERATIVE TOKEN SEQUENCE

  • An SID is a multi-level sequence of codebook tokens used as an address for textual experience, not as a lossless encoding of its text.MemRetriever generates retrieval addresses from query and reasoning context; MemEvolver uses trajectories to target updates.Source: §B.3 · pp. 16–17

B.4 EMBEDDING RECONSTRUCTION FROM SIDS

  • Residual codebooks approximate seed experience embeddings, but the fitting objective does not guarantee lower reconstruction error for every entry at every level.This reconstruction concerns offline seed quantization, not online evolution, which can revise payloads without re-encoding or changing the SID.Source: §B.4 · pp. 17–18

B.5 SID ALIGNMENT MAPPINGS

  • SID alignment pretraining links queries, trajectories, and experience texts to complete address sequences through three address-generation and two text-generation mappings.Reverse mappings generate experience text or descriptions from an SID; they do not perform memory lookup, and SID levels are not assigned fixed semantic roles.Source: §B.5 · p. 18

C ADDITIONAL METHOD DETAILS

These appendices detail how GenMem constructs symbolic addresses, updates memory, and trains its policies. They also specify reward components and the cold-start process that produces initial experience summaries.

C.1 RQ-KMEANS DETAILS

  • Frozen encoders turn prefixed rich summaries into embeddings, which sequential residual K-means codebooks quantize into level-specific SID symbols.Codebooks are fitted sequentially and held fixed during online operation.Source: §C.1 · pp. 18–19
  • UCR measures complete-address sharing, DUI measures marginal assignment balance, and the entropy-based independence ratio measures cross-level dependence.The independence ratio is computed from the empirical joint distribution and can be sensitive to sparse address-space sampling.Source: §C.1 · pp. 19–20
  • The stated quantization costs cover offline fitting and assignment, not end-to-end latency or learned online SID generation.Text encoding, experience extraction, and LLM decoding are excluded.Source: §C.1 · p. 20

C.2 ONLINE MEMORY SELF-EVOLUTION

  • MemEvolver generates insert or revise updates at a target SID, while revisions alter the payload without reassigning existing addresses.Address invariance does not guarantee that revised content remains useful for every query that previously retrieved it.Source: §C.2 · pp. 20–21

C.3 REWARD DESIGN DETAILS

  • Policy training combines structural validity and strategic-quality process rewards with outcome feedback comparing memory-supported and unsupported StackPlanner results.MemEvolver's strategic reward covers experience faithfulness, SID localization, and operation appropriateness; outcome reward measures comparative utility rather than absolute task accuracy.Source: §C.3 · p. 21, §C.3.2 · pp. 21–22, §C.3.3 · p. 22

C.4 DESIGN CHOICES AND RATIONALE

  • Mid-training uses teacher-provided traces, while post-training evaluates behavior with SID, strategic-quality, and outcome rewards; revision can also retain, merge, or delete content.The interface has insert and revise operations, but the operation vocabulary alone does not establish that the policy chooses appropriately.Source: §C.4 · p. 22

C.5 STABLE ADDRESSES AND EDITABLE MEMORY CONTENT

  • Each occupied SID holds one current experience text, and revisions may refine it without changing the SID or codebooks.SID levels arise from residual quantization; shared prefixes do not establish independent semantic axes.Source: §C.5 · p. 23, Figure 6 · p. 23

D.1 COLD-START EXPERIENCE GENERATION

  • Cold-start generation collects multiple attempts, assesses outcomes and evidence, then extracts reusable procedures while preserving uncertainty in unverified attempts.The extraction compares contrasting outcomes; it is not optimization of a contrastive loss.Source: §D.1 · pp. 23–24

D.2 MEMORY BANK DATA STATISTICS

The appendix quantifies the scale, composition, and compression of GenMem's experience bank. It reports how entries are consolidated into summaries and characterizes the remaining long-summary tail.

D.2 MEMORY BANK DATA STATISTICS

  • 138,243 pre-summary entries were compressed into 16,416 final summaries after skill crawling and cluster-level deduplication.Raw skill crawling produced 171,820 entries, reduced to 16,384 canonical skills.Source: §D.2 · p. 24
  • SOP and factual experience make up most pre-summary entries, while skill experience provides a smaller complementary source.The source distribution spans interactive, reasoning, and search-oriented datasets rather than a single source.Source: §D.2 · p. 24, Table 5 · p. 24, Table 6 · p. 25
  • Approximately 1.075 × 10^9 pre-summary characters became 1.18 × 10^8 post-summary characters, about 9.1× compression.Entry-length statistics also show reductions in typical and extreme lengths.Source: §D.2 · p. 25, Table 7 · p. 25
  • Each final summary merges an average of 4.52 raw entries, while most summaries remain below the larger character thresholds.A small number of summaries aggregate substantially larger groups.Source: §D.2 · p. 25, Table 8 · p. 25, Table 9 · p. 25

E.1 EVALUATION BENCHMARKS

The appendix maps benchmarks to evaluation metrics and specifies an online evaluation protocol. It distinguishes task performance from retrieval and update measures, and documents boundaries in the benchmark and reporting scope.

E.1 EVALUATION BENCHMARKS

  • Evaluation spans in-domain interactive and QA benchmarks, out-of-domain QA, and cross-domain search, research, SQL, and software tasks.GPQA is used separately for symbolic reasoning; exact benchmark versions, splits, sample counts, and seed-bank overlap checks remain to be documented.Source: §E.1 · pp. 25–26, Table 10 · p. 26

E.2 EVALUATION METRICS

  • Task success, QA F1, symbolic-reasoning accuracy, and cross-domain metrics are distinct measures, while retrieval hit rates test target presence rather than relevance precision.The appendix also distinguishes cumulative SID accuracy from independent level accuracy and notes that no statistical-significance claim is made without repeated-run results and a specified test.Source: §E.2 · pp. 26–27

E.3 ONLINE EVALUATION PROTOCOL

  • Online evaluation uses a fixed temporal query stream and matched initial bank, model, encoder, and codebooks for online and frozen-bank controls.Held-out answers are unavailable during updates; each query is scored before the evolution policy applies its emitted operation.Source: §E.3 · p. 27
  • Probe task performance is tracked against processed queries, with paired online-versus-frozen differences and bootstrap intervals over tasks and stream seeds.The protocol also records memory precision@k, SID prefix hit rates, and operation frequencies.Source: §E.3 · p. 27

F.1 COMPARISON METHODS AND SETTINGS

The appendix identifies the methods and settings used for the benchmark comparisons, including a matched retrieval protocol. It also reports the memory-policy backbone and SID configuration while noting reproduction details that remain undocumented.

F.1 COMPARISON METHODS AND SETTINGS

  • The comparisons include prompting, memory-based, RL, and memory-augmented RL baselines for interactive tasks, alongside prompting, retrieval, and RL methods for QA.GENMEM and GENMEM (RL) distinguish configurations before and after Harness-RL training of StackPlanner.Source: §F.1 · p. 28
  • Retrieval methods share a 50-candidate budget and BGE reranking to Top-5, enabling a matched-budget comparison of candidate and final hit rates.The comparison includes Qwen3-Embedding-0.6B, TF-IDF, SkillRouter-Embedding-0.6B, and GENMEM.Source: §F.1 · p. 28

F.2 TRAINING HYPERPARAMETERS

  • Both memory policies use Qwen2.5-7B-Instruct and GRPO with a fixed four-level 48×16×8×8 SID codebook.Per-row result provenance, checkpoint versions, inference budgets, decoding settings, and seeds remain to be recorded for controlled reproduction.Source: §F.2 · p. 28, §F.1 · p. 28, Table 11 · p. 28

G.1 RQ-KMEANS EVALUATION METRICS AND CONFIGURATION ANALYSIS

The analysis compares codebook configurations and distance settings, then characterizes trade-offs in the selected SID configuration. The prefix comparison uses a three-level codebook, not the final four-level system.

Prefix-prompt comparison

  • The prefix comparison holds the seed bank, encoder, and Weighted setting (w = 0.5) fixed, but uses a three-level 48 × 24 × 24 codebook rather than the final four-level configuration.The prefix reduces reported reconstruction loss while slightly reducing leaf occupancy; it is not a direct ablation of prefix guidance in the final system.Source: §G.1 · pp. 28–29, Table 12 · p. 29

RQ-KMeans codebook configuration analysis

  • The analysis compares 40 codebook configurations across two to four levels and capacities from 27,648 to 147,456, using three distance settings fitted on the same 138,243 embeddings.Metrics include reconstruction loss, leaf utilization, SID uniqueness, and DUI.Source: §G.1 · p. 29, Table 13 · p. 29

Configuration trade-offs

  • The selected four-level configuration uses 80 SID tokens for 49,152 possible addresses, but has higher reconstruction loss than every listed alternative.Its leaf utilization is 62.22%, so the selection favors a smaller vocabulary rather than minimum reconstruction error.Source: §G.1 · p. 30, Table 15 · p. 30
  • Across 40 architectures, the reported Weighted setting has higher mean leaf utilization than Euclidean (49.15% versus 14.92%), with similar reconstruction loss (3.32 versus 3.38, in units of 10−4).These aggregate comparisons do not isolate hierarchy-depth effects.Source: §G.1 · pp. 29–30, Table 14 · p. 29

G.2 PRETRAINING AND MID-TRAINING RETRIEVAL ANALYSIS

This section evaluates SID retrieval across pretraining and mid-training configurations, codebook depths, and decoding and reranking settings. It also reports utilization and training-loss behavior for the selected four-level design.

Full SID retrieval results

  • The retrieval evaluation compares pretraining and mid-training stages for MemRetriever and MemEvolver under beam search and reranking, using examples from six search benchmarks plus ALFWorld and WebShop.Reported measures include recall at multiple cutoffs and per-level hit rates; full SID retrieval results are tabulated across six configurations.Source: §G.2 · pp. 30–31, Table 18 · p. 31

Codebook configuration

  • The four-level factorization provides a larger theoretical address space with shorter per-level vocabularies, reducing per-token prediction entropy during autoregressive SID generation.Table 17 compares the three-level and four-level codebook configurations.Source: §G.2 · p. 31, Table 17 · p. 31

SID decoding configurations

  • Increasing deterministic beam width from 5 to 20 raises Recall@5 from 7.93% to 8.53%, while latency rises from 121.0 to 406.1 ms.No sampled configuration exceeds the highest deterministic Recall@5 in this comparison.Source: §G.2 · p. 31, Table 19 · p. 32

Beam-search and rerank hit rates

  • Tables 20 and 21 report independent and cumulative per-level hit rates, with reranking applied to Beam@50 candidates for Top-1 and Top-5.For three-level models, L4 is not applicable.Source: §G.2 · p. 31, Table 20 · p. 32, Table 21 · p. 33

MemE Update-Type Breakdown

  • MemE retrieval performance is broken down by update-operation type, distinguishing merge_existing from insert_new evaluation categories.merge_existing targets populated slots and corresponds to revise; insert_new targets unpopulated leaves and corresponds to insert.Source: §G.2 · p. 31, Table 22 · p. 33

SID Depth Selection: 3-Level vs. 4-Level

  • The four-level SID achieves lower alignment-pretraining loss throughout training and converges faster to a lower plateau than the three-level SID.The authors attribute this to shorter per-level prediction targets and sharper categorical distributions, motivating the four-level default.Source: §G.2 · p. 31

SID Codebook Utilization Analysis

  • All four codebook levels have normalized entropy ≥0.97, with effective code counts of 43.3/48, 15.6/16, 7.9/8, and 7.9/8.L1 is mildly imbalanced, while L2–L4 approach uniformity; the four-level design provides 49,152 addressable regions without balancing regularization.Source: §G.2 · pp. 32–33

G.3 TRAINING DYNAMIC

The training-dynamics materials describe optimization curves and their limits, while the ablation section defines component comparisons. The GPQA evaluation reports accuracy gains from complete SIDs, and the case-studies passage identifies the traces presented.

G.3 TRAINING DYNAMIC

  • Mid-training fine-tunes both memory agents on ReAct-style traces with embedded SID generation, and both modules converge smoothly.The reported StackPlanner curves are task-execution training diagnostics, not held-out performance of the complete system; their losses are not directly comparable across tasks and each task is a single run without across-seed uncertainty estimates.Source: §G.3 · pp. 33–34

G.4 ABLATION STUDY

  • The ablations separately compare iterative memory refinement, generative SID retrieval, external memory access, RL post-training, and beam search against alternatives.The protocol evaluates one component change at a time across 200 samples and three complete evolution rounds.Source: §G.4 · p. 37, Table 23 · p. 37

G.5 GPQA EVALUATION PROTOCOL

  • Complete SIDs raise GPQA accuracy from the raw-query baseline of 48.32% to 54.06–54.66%; rewritten and directly retrieved experience show no consistent advantage over each other.Direct retrieval performs better at k = 1 and k = 5, while rewriting performs better at k = 10.Source: §G.5 · pp. 37–38

H CASE STUDIES

  • The case studies present three representative execution traces involving MemRetriever, StackPlanner, and/or MemEvolver, following 40 initial MemRetriever training steps (23.5 hours).The supplied passage does not report the cases’ outcomes.Source: H CASE STUDIES · p. 38

H.1 SYMBOLIC REASONING AND ONLINE EVOLUTION TRACES

This section presents traces and appendix prompts showing how SIDs address reusable skills and experiences, how memory payloads are revised without changing retrieval addresses, and how agents generate, verify, summarize, generalize, and embed experience. It also covers outcome-reward tasks contrasting memory use with baselines and the runtime operation of MemRetriever and MemEvolver, with memories and retrieval focused on reusable, actionable experience.

H.1 SYMBOLIC REASONING AND ONLINE EVOLUTION TRACES

  • Fusion combines shared verification procedures from separate memories, while decomposition retrieves complementary subskills and makes a target task’s supporting steps explicit.The examples cover multi-source date verification and normalized occupation extraction.Source: §H.1 · pp. 38–39
  • Online evolution selectively corrects and enriches a literary-analysis payload while preserving its SID, so learned retrieval addresses remain usable.Updates add a source hierarchy and explicit checks of character names, roles, context, and timeline.Source: §H.1 · p. 39
  • Runtime traces show memory supporting geographic-name verification, fixed-member combination counting, and insertion of a new sequential-payment procedure at an unpopulated leaf.The tunnel query resolves to Switzerland; the basketball example yields 120 lineups, and the payment example yields $20 remaining.Source: §H.1 · pp. 40–41

I.1 EXPERIENCE GENERATION PROMPTS

  • Experience-generation prompts require evidence and verification, and specify benchmark-specific procedures for WebShop selection and ALFWorld interaction.WebShop instructions include search, inspection, selection, and reward checks; ALFWorld instructions require tool-driven steps and reporting environment completion signals.Source: §I.1 · pp. 42–43

I.2 TRAJECTORY SUMMARIZATION PROMPTS

  • Trajectory-summary prompts preserve observed execution and evidence, distinguish success from failure or uncertainty, and turn verified runs into grounded, reusable procedures.Interactive-task summaries must use actual tool observations and reward or completion signals rather than claiming unverified success.Source: §I.2 · pp. 43–44

I.3 SKILL-CLUSTER AND SID-LEAF SYNTHESIS PROMPTS

  • Skill-cluster synthesis prompts generalize shared procedures, decision rules, checks, and pitfalls while removing one-off case details.The resulting experience is intended to be broadly reusable rather than a vague label.Source: §I.3 · p. 45
  • The skill-cluster synthesis prompt merges summaries from the same semantic cluster into one concise, nonredundant experience preserving common strategies, useful techniques, decision rules, successes, and pitfalls.It frames the input experiences as sharing a SID and semantic cluster.Source: §I.3 · p. 45

I.4 EMBEDDING GENERATION PREFIX

  • The embedding prefix encodes experience as reusable memory for semantic retrieval and SID clustering, prioritizing problem-solving patterns and details such as preconditions, procedures, checks, outcomes, and recovery.It deprioritizes dataset names, run IDs, wording artifacts, and incidental entities unless essential to the skill.Source: §I.4 · pp. 45–46

I.5 STACKPLANNER OUTCOME-REWARD PROMPTS

  • Outcome-reward prompts compare tasks with retrieved experience against no-memory baselines, requiring WebShop product identification and tool-executed ALFWorld success, alongside a no-memory general-QA baseline.The ALFWorld prompt requires replanning from observations and stops after 30 step calls; only a tool response with won=true counts as success.Source: §I.5 · p. 46

I.6 MEMRETRIEVER AND MEMEVOLVER RUNTIME PROMPTS

  • MemRetriever selects exactly one four-layer SID, makes one memory lookup, and returns a concise actionable summary or a no-relevant-experience response.The lookup tool returns top-k relevant summaries for supplied SIDs; the prompt forbids multiple or partial SIDs and repeated calls.Source: §I.6 · p. 47
  • MemEvolver analyzes a trajectory, retrieves an existing experience under a chosen SID, compares old and new insights, and synthesizes a transferable, actionable update.When no existing experience is found, it derives the experience from the trajectory rather than copying the trajectory verbatim.Source: §I.6 · p. 48

Related papers

Loading 2609.34633v1…