PaperlayerView exact revision on arXiv

Source-linked AI summary

MemGen: Weaving Generative Latent Memory for Self-Evolving Agents

Guibin Zhang, Muxin Fu, Shuicheng Yan

arXiv:2509.24704v2cs.CL

TL;DR

Existing agent memory either modifies model parameters or retrieves structured experiences, leaving a gap in fluidly interleaving memory and reasoning. MemGen addresses this with a learned trigger and generative weaver that insert reconstructed latent memories during reasoning, achieving broad gains and distinct planning, procedural, and working memory faculties.

  • Problem

    Existing parametric and retrieval-based memories do not capture the fluid interweaving of reasoning and reconstructive memory needed for dynamic agent cognition.

  • Method

    MemGen uses a reinforcement-learned memory trigger to select invocation moments and a memory weaver to synthesize latent memories conditioning a frozen reasoner.

  • Results

    Across nine benchmarks, MemGen delivers substantial gains, cross-domain transfer, and stable retention, including up to 31.7% on ALFWorld and 27.1% on KodCode with Qwen3-8B.

  • Takeaways & Limitations

    MemGen’s latent memories develop distinct planning, procedural, and working memory specializations without external guidance, suggesting a path toward fluid reconstructive intelligence.

Abstract

from arXiv · show

Agent memory shapes how Large Language Model (LLM)-powered agents, akin to the human brain, progressively refine themselves through environment interactions. Existing paradigms remain constrained: parametric memory forcibly adjusts model parameters, and retrieval-based memory externalizes experience into structured databases, yet neither captures the fluid interweaving of reasoning and memory that underlies human cognition. To address this gap, we propose MemGen, a dynamic generative memory framework that equips agents with a human-esque cognitive faculty. It consists of a \textit{memory trigger}, which monitors the agent's reasoning state to decide explicit memory invocation, and a \textit{memory weaver}, which takes the agent's current state as stimulus to construct a latent token sequence as machine-native memory to enrich its reasoning. In this way, MemGen enables agents to recall and augment latent memory throughout reasoning, producing a tightly interwoven cycle of memory and cognition. Extensive experiments across eight benchmarks show that MemGen surpasses leading external memory systems such as ExpeL and AWM by up to $38.22\%$, exceeds GRPO by up to $13.44\%$, and exhibits strong cross-domain generalization ability. More importantly, we find that without explicit supervision, MemGen spontaneously evolves distinct human-like memory faculties, including planning memory, procedural memory, and working memory, suggesting an emergent trajectory toward more naturalistic forms of machine cognition.

1 Introduction

MemGen frames agent memory as a dynamic cognitive faculty that interleaves reasoning with generative reconstruction rather than relying solely on parameter updates or database retrieval. Its trigger-and-weaver design improves performance, generalization, continual learning, and yields distinct memory specializations.

  • Agent memory enables LLM agents to internalize environmental experience and progressively improve problem-solving, beyond conversational memory for dialogue coherence.
  • Parametric memory updates model parameters but risks catastrophic forgetting, whereas retrieval-based memory externalizes experiences into structured databases.
  • MemGen monitors the agent’s cognitive state and uses a reinforcement-learning-trained memory trigger plus a memory weaver to synthesize latent memory during reasoning.
  • 31.7% improvement on ALFWorld and 27.1% on KodCode with Qwen3-8B, while surpassing parametric memory by 5.8% and GRPO by 5.32%.
  • MemGen generalizes across domains and retains previously learned performance after fine-tuning on three additional domains.
  • Post-hoc interventions reveal unsupervised planning, procedural, and working memory specializations in MemGen’s latent representations.

2 Related Work

Related work spans parametric and retrieval-based agent memory, latent computation, decoding, and reinforcement learning. MemGen is positioned as addressing self-improving memory rather than only long-context handling or inference acceleration.

  • Prior agent memory methods include parameter-integrated or external parameter modules and retrieval systems that abstract experiences into transferable knowledge.
  • Latent computation either makes reasoning natively latent or uses latent states to steer language-model generation.
  • MemGen’s latent-token generation resembles speculative decoding operationally, but its stated purpose is carrying memory rather than accelerating inference.
  • MemAgent and MEM1 focus on long-context inputs rather than evolving self-improving memory mechanisms.

3 Preliminary

The preliminary formulation represents an agent as a policy interacting with an environment and distinguishes memory systems by when and how they generate conditioning representations. MemGen targets finer-grained, self-determined invocation timing.

  • An agent trajectory alternates environment states and high-level actions, with each action generated as an autoregressive token sequence.
  • After execution, actions transition the environment state, and trajectory success is evaluated by a reward function.
  • Given past task trajectories, the objective is to optimize the policy and memory system for expected reward on new tasks.
  • Memory conditions policy actions through a representation generated by fM, whose nature and timing vary across memory paradigms.
  • Task-level memory is generated once, step-level memory is updated every step, and MemGen instead designs finer-grained self-determined regeneration moments.

4 Methodology

MemGen interleaves autoregressive reasoning with on-demand latent-memory insertion. A learned trigger selects invocation points, while a weaver reconstructs and reintegrates fixed-length latent memories without modifying the frozen reasoner.

  • 4.1 MemGen: Interleaving Memory and Reasoning: Human reasoning and memory are presented as an interwoven continuum, motivating MemGen’s recursive integration of memory into reasoning.
  • 4.1 MemGen: Interleaving Memory and Reasoning: At each token step, the trigger monitors hidden states and decides whether to pause generation for memory invocation.
  • 4.1 MemGen: Interleaving Memory and Reasoning: The weaver uses the current cognitive state to synthesize a bespoke latent memory that may incorporate externally retrieved cues and is selectively reconstructed rather than copied verbatim.
  • 4.1 MemGen: Interleaving Memory and Reasoning: MemGen repeatedly generates, monitors, invokes, weaves, and reintegrates memory while keeping the core reasoner frozen.
  • 4.2 Memory Trigger: Learning to Invocate Memory: The trigger is a lightweight LoRA adapter that outputs invocation probabilities and acts only at delimiter tokens to preserve decoding efficiency.
  • 4.2 Memory Trigger: Learning to Invocate Memory: Reinforcement learning trains the trigger to balance performance gains against unnecessary activations through a reward-adaptive penalty.
  • 4.3 Memory Weaver: Synthesizing and Inserting Latent Memory: The weaver maps hidden-state hooks to a fixed-length latent memory matrix that is prepended to guide subsequent generation.
  • 4.3 Memory Weaver: Synthesizing and Inserting Latent Memory: The weaver alone internalizes experiential knowledge, leaving the reasoner unchanged and supporting SFT, GRPO, or DAPO optimization.

5 Experiments

The experiments evaluate MemGen across multiple domains, against prompt-based, parametric, and retrieval-based baselines, while testing generalization, continual learning, memory mechanisms, and efficiency. Results show broad performance gains, cross-domain transfer, improved retention, emergent memory specialization, and limited inference overhead.

  • Experimental Setup: The evaluation spans nine datasets across web search, embodied action, mathematics, scientific reasoning, and coding.Benchmarks include TriviaQA, PopQA, ALFWorld, AQuA, GSM8K, MATH, GPQA, KodCode, and BigCodeBench.
  • Experimental Setup: MemGen is compared with twelve baselines across prompt-based, parametric-memory, and retrieval-based categories.The implementation includes MemGen SFT and MemGen GRPO variants with multiple LLM backbones.
  • Main Results: MemGen improves performance across diverse domains, including 50.60% and 63.60% on ALFWorld with SmolLM3-3B, and +27.06% on KodCode and +28.17% on PopQA with Qwen3-8B.The reported gains contrast with retrieval methods’ weaker reasoning-task performance and parametric methods’ weaker knowledge-intensive reasoning.
  • Cross-Domain Generalization: MemGen transfers across domains: training on KodCode raises MATH performance from 36.6% to 54.2%, while baselines can drop sharply on FEVER by 16.2%.The study trains on one dataset and evaluates on TriviaQA, ALFWorld, ScienceWorld, and FEVER.
  • Continual Learning: MemGen retains knowledge more evenly across sequential tasks, preserving 40.34% on AQuA after KodCode training versus 27.14% for ExpeL and 28.61% for SFT.After GSM8K training, it reaches 38.43% on AQuA and 21.72% on GPQA.
  • Framework Analysis: Post-hoc interventions identify planning, procedural, and working-memory roles, while efficiency analysis reports up to 57.66% improvement with latency at 24%–94% of vanilla latency.Procedural clusters affect tool-use, parsing, and formatting; working-memory clusters affect task understanding and reasoning consistency.

6 Conclusion

MemGen concludes by presenting dynamic latent memory as a way to interleave reasoning with memory synthesis while preserving a frozen reasoner. Its training procedures update the memory weaver through supervised or reinforcement-learning objectives.

  • MemGen interleaves reasoning with memory synthesis through a reinforcement-learned trigger and generative memory weaver.
  • MemGen’s broader results include performance gains, cross-domain generalization, continual learning, and planning, procedural, and working memory.
  • SFT trains the weaver to generate latent memories that guide the frozen reasoner toward expert demonstration behavior.
  • GRPO trains the weaver from group-relative trajectory rewards while updating only its parameters and regularizing against a fixed reference policy.

C.1 Training Dataset Setup

MemGen trains its components in a staged setup using official dataset splits, with TriviaQA supporting PopQA evaluation because PopQA lacks a training set. The experiments vary insertion strategies before learning the trigger.

  • Official training splits supervise the memory weaver and trigger, except PopQA, which is evaluated using a model trained on conceptually related TriviaQA.
  • The weaver is first trained without a trigger using latent-memory insertion at every punctuation boundary or sampled boundary subsets.
  • Afterward, the fixed weaver supports trigger training, allowing the trigger to learn invocation decisions from the agent’s evolving cognitive state.

D.1 Continual Learning Result

Continual-learning experiments report stronger retention, forward transfer, and balanced cross-task generalization for MemGen than for SFT and ExpeL. Additional evaluations examine trigger frequency across datasets and backbones.

  • MemGen preserves earlier-task performance more effectively than SFT and ExpeL during sequential training across AQuA, GPQA, GSM8K, and KodCode.After KodCode training, MemGen retains 40.34% on AQuA versus 27.14% for ExpeL and 28.61% for SFT.
  • When trained on GPQA, MemGen reaches 47.96% on GSM8K and 28.80% on KodCode, exceeding SFT by +2.22% and +10.30%, respectively.
  • Figures 7 and 8 visualize invocation frequencies across GSM8K, KodCode, and GPQA for Qwen2.5-1.5B and SmolLM3-3B.

D.3.1 Ablation Study

The ablation compares random, delimiter-wide, and trained-trigger memory invocation. Results favor semantically aligned intervention and show the dedicated trigger achieves the strongest overall performance.

  • Activating the weaver at all delimiters yields 17.34%, 56.20%, and 64.15% on GPQA, Kodcode, and TriviaQA, respectively.
  • The all-delimiters strategy outperforms random invocation, for which p = 0.5 achieves 16.66%, 52.95%, and 57.28% on the same tasks.
  • MemGen’s dedicated trained trigger achieves the best overall performance among the tested invocation strategies.

D.3.2 Analysis of Memory Weaver

The memory weaver’s parameterization creates a trade-off between capacity and efficiency. Full-parameter SFT improves performance, while LoRA remains competitive with fewer trainable parameters.

  • Full-parameter SFT enhances the weaver’s memory and learning capacity, yielding higher task performance across benchmarks.The comparison is reported in Table 6 between LoRA and full-parameter SFT.
  • LoRA-based memory weaving achieves competitive results while retaining high parameter efficiency.This indicates that lightweight adaptation can remain effective despite using fewer trainable parameters.

D.3.3 Efficiency Analysis

MemGen improves reasoning efficiency by reducing inference time in some settings while adding little overhead in others. Its integration process retrieves textual snippets, merges them with the reasoner’s hidden state, and reconstructs them into latent memory.

  • On ALFWorld with Qwen3-8B, MemGen SFT adds only 1.6% inference time while increasing accuracy from 83.59% to 85.82%.Inference time rises from 19.76 s to 20.08 s compared with SFT.
  • MemGen’s retrieval integration is designed to combine external textual knowledge with the weaver’s internalized experiences.The framework supports a hybrid approach rather than relying exclusively on either source.
  • Retrieved snippets are encoded into embeddings and merged with the reasoner’s internal cognitive state before memory synthesis.The encoded snippets form E_t, which is combined with H_t,<j for the weaver.
  • The memory weaver reconstructs internal and retrieved information into a compact latent memory for the reasoner.The concatenated hidden-state sequences provide the combined input to W_weaver.

E.2 Experimental Results

MemGen strengthens retrieval-based memory by transforming retrieved text into task-relevant latent memory rather than merely appending it. Combining retrieval with the weaver’s internal parametric memory produces substantially higher reported performance.

  • On ALFWorld, disabling MemGen’s parametric memory still raises retrieval-baseline performance from 36.18% to 45.60%.Only retrieved textual snippets are fed into the memory weaver in this condition.
  • On PopQA, disabling MemGen’s parametric memory increases retrieval-baseline performance from 28.16% to 39.50%.This result supports the weaver’s role as a synthesizer of retrieved information.
  • With internal parametric memory enabled alongside external retrieval, performance reaches 75.90% on ALFWorld and 76.40% on TriviaQA.The results demonstrate the reported synergy between internalized experiences and external textual knowledge.
  • MemGen actively reconstructs retrieved information into more potent, task-relevant latent memory instead of simply appending textual snippets.The comparison is made against the retrieval baseline with MemGen’s own parametric memory disabled.

F Latent Memory Token Demonstration

The demonstrations show latent memory supporting factual retrieval, tool-mediated question answering, and code-generation behavior. MemGen also assigns synthesized memories a semantic affiliation by comparing their mean embeddings with vocabulary and cluster representations.

  • The memory system also demonstrates a search workflow for identifying Sunset Boulevard as the Lloyd Webber musical premiering in the US on 10 December 1993.The example includes a search query, retrieved information, and a final answer.
  • Latent memory supports a search-and-answer workflow for factual questions such as identifying Exile’s hit “Kiss You All Over.”The example shows reasoning, search invocation, observation processing, and the final answer “Exile.”
  • A second factual example uses search to identify Michael Jackson’s 1988 autobiography as “Moonwalk.”The answer follows reasoning over retrieved observational text.
  • Another demonstration shows search-based reasoning identifying Mount Kilimanjaro as Tanzania’s volcano that is Africa’s highest mountain.The retrieved observation describes Kilimanjaro as a dormant volcano in Tanzania.
  • Code-generation demonstrations cover filtering and sorting even integers, finding first and second maxima, and locating the longest constant-value subarray.The examples include explicit handling of empty inputs and arrays with fewer than two unique elements.
  • For semantic affiliation, MemGen averages the latent memory token embeddings and compares the resulting vector with vocabulary and memory-cluster representations using cosine similarity.The experiments select the top 10 nearest neighbors.
Loading 2509.24704v2…