PaperlayerView exact revision on arXiv

Source-linked AI summary

Remember Before You're Asked: MemDream for Self-Probing Memory Evolution

Mingfei Lu, Mengjia Wu, Runsong Jia, Zhe Luo, Yi Zhang

arXiv:2609.34545v1cs.AI

TL;DR

Long-horizon memory systems often discover weaknesses only when real queries expose them. MemDream runs offline cycles in which specialized agents probe, diagnose, and repair memory, improving results on LoCoMo and MemoryAgentBench over evaluated baselines.

  • Problem

    Existing memory systems repair weaknesses only after real queries expose them, so they lack a proactive test of whether stored information can be reliably retrieved.

    Source: §1 · p. 2

  • Method

    MemDream runs offline dream cycles in which a Dreamer probes memory, an Analyst diagnoses retrieval failures, and a Consolidator repairs the graph, with GRPO learning repair policies and soft decay enabling reversible forgetting.

    Source: §1 · p. 2

  • Results

    MemDream improves answer F1 by 3.98 points on LoCoMo and overall score by 9.1 points on MAB over FluxMem and O-Mem, respectively.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Across reader models on LoCoMo and MemoryAgentBench, MemDream improves over memory baselines, with the largest gains in temporal reasoning and cross-session integration.

    Source: §5 · p. 11

Abstract

from arXiv · show

Memory is essential for enabling LLM-based agents to maintain coherent, personalized behavior over long-horizon interactions. However, existing memory systems share a fundamental limitation: they never proactively test their own memory, repairing it only after real queries expose weaknesses. This reactive paradigm means every retrieval failure corresponds to a real interaction in which the cost has already been paid. We propose MemDream, a framework that enables self-probing memory evolution for LLM agents. Our framework periodically enters offline dream cycles where three specialized agents (Dreamer, Analyst, Consolidator) collaboratively probe, diagnose, and repair the memory graph before failures occur. A policy trained via Group Relative Policy Optimization learns which repair operations produce durable retrieval improvements, while a soft decay mechanism provides reversible forgetting driven by the same anticipatory signal. Experiments on LoCoMo and MemoryAgentBench demonstrate that MemDream improves answer F1 by 4.5 points on LoCoMo and achieves a 9.1-point higher overall score on MAB over the strongest reactive-evolution baselines.

1 Introduction

Long-horizon agents depend on memory to keep behavior coherent, but existing systems often repair memory only after a real query exposes a weakness. MemDream uses offline dream cycles to probe and repair memory before that failure reaches the user.

1 Introduction

  • Memory governs response coherence and behavioral consistency over long interactions, yet fragmented, disconnected, or stale facts can leave weaknesses undiscovered until a real query exposes them.Self-generated probes can test whether relevant evidence is retrievable before a user encounters the defect.Source: §1 · pp. 1–2
  • Prior work spans structured memory, multi-stage retrieval, and self-evolving memory, motivating the question of how to signal that stored information cannot be reliably retrieved.The cited approaches variously fix organization in advance, leave memory unchanged, or adapt it through reflection, self-questioning, retrieval tuning, or learned update policies.Source: §1 · p. 1
  • MemDream improves answer F1 by 3.98 points on LoCoMo and overall score by 9.1 points on MAB over FluxMem and O-Mem, respectively.The reported comparison is against the strongest evaluated baselines.Source: §1 · p. 2
  • MemDream’s offline dream cycle uses a Dreamer, Analyst, and Consolidator to probe the memory graph, diagnose retrieval breakdowns, and repair weak regions before a real query arrives.The Dreamer’s forward-looking queries turn unobserved future queries into a repair training signal.Source: §1 · p. 2
  • GRPO trains the repair policy to learn which operations yield durable retrieval gains, replacing hand-crafted heuristics and assigning credit before user queries arrive.The Consolidator is trained on changes in retrieval success for the Dreamer’s queries.Source: §1 · p. 2
  • Soft decay makes forgetting reversible: nodes are demoted rather than deleted and can be restored when later cycles reveal renewed relevance.The same anticipatory signal drives both decay and memory repair.Source: §1 · p. 2

2 Related Work

Prior work shapes memory organization and retrieval, learns memory updates, or studies structured evidence selection in other tasks. MemDream is positioned as complementary: it targets the repair signal for persistent conversational memory.

2 Related Work

  • Memory architectures range from flat retrieval to hierarchical and structured stores, but leave content maintenance to implicit or hand-crafted rules (Gao et al., 2023; Edge et al., 2024; Sarthi et al., 2024).MemDream operates on top of memory graphs rather than changing the storage layer.Source: §2 · p. 2
  • Adaptive memory systems learn update policies, reflect on stored experience, or consolidate between sessions, while MemDream uses self-generated queries to expose latent retrieval failures for targeted repair (Yan et al., 2025; Wang et al., 2025c; Yuan et al., 2025; Zhang et al., 2026; Park et al., 2023; Yang et al., 2026a; Ye et al., 2026; Yang et al., 2026b; Dai et al., 2026).The cited recent approaches include offline region rewriting, memory-transition verification, and recurrence-triggered consolidation.Source: §2 · pp. 2–3
  • Unlike forgetting methods treated as standalone operations, MemDream’s augment_decay uses the consolidation signal to demote nodes predicted to hurt future retrieval and restore them when later cycles find them relevant.Forgetting is reversible within the dream cycle rather than a separate permanent step.Source: §2 · p. 3
  • Work on structured evidence selection and verification spans legal reasoning, citation prediction, graph fraud detection, and LLM serving; MemDream applies related concerns to persistent conversational memory.Its prospective queries locate retrieval weaknesses and guide direct memory edits.Source: §2 · p. 3

3 Methodology

MEMDREAM maintains hierarchical memory through asynchronous dream cycles that probe likely retrieval failures and repair the memory graph before real queries expose them. Its three-agent workflow uses learned rewards to guide repairs and reversible forgetting.

3.1 Problem Formalization

  • MEMDREAM retrieves from a three-tier graph of recent interactions, session-level episodes, and compressed semantic knowledge, then updates that memory as dialogue continues.Nodes carry content embeddings, entities, access statistics, and retrieval weights; edges connect shared entities, sessions, or consolidation merges.Source: §3.1 · p. 4
  • The learned memory-evolution policy targets fragmentation, redundancy, and staleness to improve retrieval success on plausible future queries.Because the future-query distribution is unknown, the dream cycle samples from it to create a training signal.Source: §3.1 · p. 4

3.2 Multi-Agent Dream Cycle

  • Dreamer probes weak regions, Analyst classifies retrieval failures, and Consolidator applies targeted repairs; probes pair unlinked, frequently co-retrieved nodes and use answers copied from memory.The Analyst maps buried, scattered, redundant, and noisy failures to extract, synthesize, merge, and prune operations, respectively.Source: §3.2 · pp. 4–5

3.3 GRPO-Based Agent Training

  • The three agents are trained with separate GRPO rewards, with Dreamer and Consolidator objectives balancing retrieval impact, coverage, grounding, and other operation-specific criteria.Reward weights are fixed a priori rather than tuned on evaluation benchmarks; group-normalized advantages give positive gradients only to above-mean rollouts.Source: §3.3.1 · p. 6, §3.3 · p. 6

3.3.2 Training Procedure

  • Dreamer, Analyst, and Consolidator are pretrained with supervised data, independently optimized with GRPO, then Dreamer and Consolidator are jointly fine-tuned while Analyst is frozen.The joint reward measures QA-F1 change on fresh Dreamer-generated queries, and deployment updates external memory while model parameters remain fixed.Source: §3.3.2 · pp. 6–7
  • Figure 3 reports that dream cycles improve accuracy across tested decay factors, with γ=0.2 optimal; higher γ retains excess low-utility content, while lower γ decays nodes too aggressively.The figure also reports that increasing retrieval depth improves F1 with diminishing returns and linear latency growth, and that per-agent GRPO weights outperform equal weighting and single-term ablations.Source: Figure 3 · p. 7

4 Experiments

MemDream is evaluated on LoCoMo and MemoryAgentBench against memory-system baselines, with ablations and sensitivity tests examining its components and settings. It leads overall on both benchmarks, while results also identify trade-offs and the value of reversible decay.

4.2 Overall Performance

  • 3.98 average F1 over FluxMem and 8.57 temporal F1 over O-Mem are MemDream’s GPT-4.1 LoCoMo gains, though it trails FluxMem by 1.41 F1 on open-ended questions.With both GPT-4.1 and Llama readers, MemDream reaches the highest average F1, BLEU-1, and ROUGE-L.Source: Table 2 · p. 8, Table 3 · p. 8, §4.2 · p. 9
  • 54.2 with GPT-4.1 and 37.1 with Llama are MemDream’s highest overall MemoryAgentBench scores, and it leads on 7 of 9 tasks with GPT-4.1.With Llama, it leads FluxMem’s 25.6 by 11.5; MemoryOS leads Detective QA and O-Mem leads Summarization.Source: Table 4 · p. 9, Table 5 · p. 9, §4.2 · p. 9

4.3 Ablation Study

  • Removing soft decay drops average F1 from 55.14 to 41.04 (−14.10) and open-ended F1 by 18.54, the largest reported ablation effect.The authors attribute the loss to hard pruning of nodes needed by later sessions.Source: §4.3 · p. 10, Table 6 · p. 10
  • Removing the Dreamer, Analyst, or GRPO drops F1 by 3.05, 1.71, or 2.16, respectively; the full system leads on every category and metric.Without the Dreamer, the cycle lacks self-generated probes; without the Analyst, extraction is uniform; without GRPO, the Consolidator uses SFT-only weights.Source: §4.3 · p. 10

4.4 Sensitivity Analysis

  • The proposed reward weights give the best F1 for all three agents versus equal and single-term weighting variants.The weights are set a priori rather than tuned on the benchmarks.Source: §4.4 · p. 10
  • Increasing retrieval depth improves F1 with linear latency growth and diminishing gains per second; the study uses k=5 to balance accuracy and inference cost.Source: §4.4 · p. 10
  • Across tested decay factors, dream cycles outperform the no-dream baseline; γ=0.2 is optimal, while values near 0 or 1 decay too aggressively or too little.Near-zero decay approaches the w/o Decay ablation’s failure mode; near-one decay leaves superseded nodes with low-utility content.Source: §4.4 · pp. 10–11
  • Rule-based probes score higher than LLM probes on discriminative power (0.36 vs. 0.15) and alignment (0.37 vs. 0.35), with similar query-type coverage.The study retains the LLM Dreamer as a trainable generator and uses rule-based probes as a diagnostic reference.Source: §4.4 · pp. 10–11

5 Conclusion

MemDream uses prospective self-probing to identify and repair memory weaknesses before real interactions. Its three agents learn through GRPO, soft decay makes forgetting reversible, and evaluation across reader models supports proactive memory maintenance.

5 Conclusion

  • Across reader models on LoCoMo and MemoryAgentBench, MemDream improves over memory baselines, with its largest gains in temporal reasoning and cross-session integration.The conclusion presents these results as support for proactive maintenance of agent memory.Source: §5 · p. 11

A Dataset Details

  • LoCoMo evaluates four reasoning categories on 1,540 questions from 10 long-horizon conversations, while MemoryAgentBench covers four competencies across nine subtasks.The appendix lists MemoryAgentBench’s competencies as Accurate Retrieval, Test-Time Learning, Long-Range Understanding, and Selective Forgetting.Source: A Dataset Details · p. 17

B Baseline Descriptions

  • The ten-baseline comparison includes flat, graph, hierarchical, multi-granularity, and adaptive memory systems, using identical input pages, chunking, reader models, and temperature 0.Examples include LangMem, ZEP, MemoryOS, A-Mem, O-Mem, and FluxMem.Source: B Baseline Descriptions · pp. 17–18

C Dream Cycle Prompts

  • The Dreamer probes plausible future needs, the Analyst diagnoses retrieval failures, and the Consolidator applies extract, merge, synthesize, or soft-decay operations.The prompts target weakly connected, redundant, temporally stale, and cross-session memory regions; repairs preserve content and log changes for reversibility.Source: C Dream Cycle Prompts · pp. 18–19

D Training Details

The agents receive supervised pretraining followed by GRPO, with joint Dreamer–Consolidator fine-tuning based on changes in retrieval F1. A subset experiment finds gains through five dream rounds, followed by decline from over-consolidation.

D Training Details

  • Joint training alternates Dreamer and Consolidator updates using ΔF1 on dream queries, while the Analyst remains frozen.The Dreamer is rewarded for exposing fixable weaknesses and the Consolidator for durable retrieval improvements; total training takes about 21 hours on one A100.Source: D Training Details · pp. 19–20, Table 7 · p. 19

E.1 Dream Cycle Rounds Comparison

  • On three LoCoMo conversations, performance improves through five rounds (+5.17 F1), then declines at eight rounds as excessive merging reduces node diversity.The authors report convergence typically within 2–3 rounds on the full 10-conversation evaluation, as gains diminish after critical weaknesses are repaired.Source: §E.1 · p. 20, Table 9 · p. 20

E.2 Dream Quality Detailed Analysis

Rule-based probes outperform LLM-generated probes on discriminative power and slightly on downstream alignment, while dream-guided top-30% query selection provides the strongest non-oracle training signal.

E.2 Dream Quality Detailed Analysis

  • Rule-based probes achieve higher discriminative power (0.36 vs. 0.15) and downstream alignment (0.37 vs. 0.35), with similar query-type coverage (5.0 vs. 4.8).The paper retains the LLM Dreamer as the trainable generator and uses rule-based probes as a diagnostic reference.Source: §E.2 · p. 21, Table 10 · p. 21

E.3 Training Signal Comparison

  • Dream-guided top-30% query selection reaches a 0.241 held-out average reward, the highest non-oracle result and 0.043 above the no-dream baseline.The oracle signal from actual future queries provides an upper bound of 0.262; top-30% also exceeds top-50% at 0.232.Source: §E.3 · p. 21, Table 11 · p. 21

F Statistical Significance of Results

Paired tests show large, conversation-consistent gains over Mem0 and MemR3, while the O-Mem gain is concentrated in two conversations. Across repeated dream cycles, key metrics remain stable in the 2–3-cycle operating regime, with no catastrophic decline even at 20 cycles.

F Statistical Significance of Results

  • The reliability analysis uses 10,000-replicate conversation-level cluster bootstraps because questions within a conversation share memory state and are not independent.For the GPT-4.1 reader, the available outputs cover 9 of 10 conversations.Source: F Statistical Significance of Results · p. 21, Table 12 · p. 22
  • Against Mem0 and MemR3, improvements are significant at the conversation level (p < 0.001) and positive in every conversation; the O-Mem gain is not conversation-level significant.The O-Mem question-level micro-F1 gain is +5.18 (p < 0.001), driven by two conversations.Source: F Statistical Significance of Results · p. 21, Table 13 · p. 22

G Multi-Round Consolidation Stability

  • At K=3, rare-fact retention is 0.960 and QA F1 is 0.517; across 20 cycles, rare-fact retention and QA F1 decline roughly 12–13%, without a catastrophic drop.In the operating regime of 2–3 cycles, all metrics are stable.Source: G Multi-Round Consolidation Stability · p. 22
  • Even after 20 cycles (roughly 10× the 2–3 cycles used in practice), node count increases 7% (308→331), hallucination does not accumulate (0.266→0.247), and entity fidelity stays flat to slightly rising (0.781→0.824).Source: G Multi-Round Consolidation Stability · p. 22

H Dream Query Groundedness

The authors test whether dream probes are grounded in stored memory and whether they resemble evaluation questions. Expected answers are copied from source memory, and measured probe overlap with evaluation questions is low.

H Dream Query Groundedness

  • The evaluation addresses two concerns: probes might target hallucinated answers or paraphrase evaluation questions.The expected answer is copied verbatim from stored source memory rather than generated.Source: H Dream Query Groundedness · pp. 22–23
  • Every expected answer is an exact substring of source memory: 110/110 under the default setting and 4150/4150 across five seeds, with zero exceptions.The default setting uses 11 probes per conversation; the scaled run uses about 415 probes per conversation.Source: H Dream Query Groundedness · p. 22
  • There are zero exact probe–evaluation-question matches, and 0/50 probes exceed ROUGE-L 0.8 against any same-conversation evaluation question.The maximum ROUGE-L is 0.571 for the analyzed first five stored probes per conversation.Source: H Dream Query Groundedness · p. 23

I Case Study

The case study shows a dream cycle detecting that a stored baking preference is disconnected from a stored nut allergy, then merging them so retrieval can surface the safety constraint. A broader audit examines the quality and use of repairs across all 10 LoCoMo conversations.

I Case Study

  • The probe retrieved the baking-preference node at rank 1 but missed the allergy node at rank 11, below the top-5 cutoff.The allergy and baking preference were stored in separate, unlinked session nodes.Source: I Case Study · pp. 23–24
  • The Consolidator merged the allergy and baking-preference nodes into a single safety-aware memory, after which the same probe retrieved it at rank 1.Retrieval F1 for the probe rose from 0.28 to 0.94; a later snack-recipe request retrieved the merged node and avoided nut-containing recipes.Source: I Case Study · p. 24

J Repair Quality Audit

  • Across 837 repair actions, none of 59 merges introduced an unsupported number or date; 9 actions (∼1%) caused information loss, while the remainder preserved content.Five actions added correct information; 82.1% of synthesized nodes were used in at least one correct answer.Source: J Repair Quality Audit · p. 24

K Computational Cost Analysis

The computational-cost analysis estimates the offline dream-cycle runtime, while distinguishing background maintenance from online answering. The comparison with Claude Dreaming characterizes MemDream as anticipatory, learned, verified, and reversible rather than backward-looking and rule-based.

K Computational Cost Analysis

  • The full dream cycle is estimated at approximately 9 minutes for 2–3 rounds, including 53 seconds of startup.The listed processing stages total 8.2 minutes per round; Table 14 describes one round’s stage times and the full-cycle estimate.Source: K Computational Cost Analysis · p. 25, Table 14 · p. 25
  • Dream cycles run asynchronously, so query answering uses current context and existing memory without waiting for background consolidation.Deployment triggers are configurable: more than 20 pages added, retrieval confidence below a threshold, or an elapsed time interval.Source: K Computational Cost Analysis · p. 25

L Comparison with Claude Dreaming

  • Claude Dreaming reviews past interactions and applies rule-based deduplication, conflict resolution, and pattern extraction; MemDream generates hypothetical future queries and learns operation choices with GRPO.MemDream’s Consolidator selects among extract, merge, synthesize, and prune based on the Analyst’s diagnosis.Source: L Comparison with Claude Dreaming · p. 26
  • MemDream re-executes dream queries after repairs and continues until retrieval F1 converges, whereas Claude Dreaming does not test retrieval improvement.MemDream also demotes nodes rather than deleting them, allowing restoration when later cycles reveal renewed relevance.Source: L Comparison with Claude Dreaming · p. 26
  • The comparison describes Claude Dreaming as reactive in signal and mechanical in operation, and MemDream as anticipatory self-probing with learned operations, verification, and reversibility.The authors characterize the systems as spanning rule-based maintenance and learned, self-supervised memory evolution.Source: L Comparison with Claude Dreaming · p. 26

M Comparison with Recent Memory Consolidation Methods

The comparison distinguishes MemDream’s explicit retrieval-testing loop from consolidation driven by rewriting, transition verification, or recurrence. These mechanisms are compared across different evaluation settings, without ranking under matched conditions.

M Comparison with Recent Memory Consolidation Methods

  • Table 16 contrasts recent memory-consolidation methods by their signals and update mechanisms, including MemDream’s explicit retrieval-testing loop.The comparison concerns mechanisms, not a ranking under matched settings.Source: M Comparison with Recent Memory… · pp. 26–27, Table 16 · p. 27
  • Auto-Dreamer (Ye et al., 2026) rewrites selected memory regions using downstream task performance and counterfactual utility, while MemDream targets repairs through generated queries and failure diagnosis.Auto-Dreamer retains source trajectories as historical evidence.Source: M Comparison with Recent Memory… · p. 27
  • TrustMem (Yang et al., 2026b) learns from preferences over updates assessed for coverage, preservation, and faithfulness, whereas MemDream probes whether stored evidence is retrievable.The passage describes transition verification and retrieval probing as complementary; MemDream’s soft decay preserves demoted nodes, while synthesized content still requires factual verification.Source: M Comparison with Recent Memory… · p. 27
  • RecMem (Dai et al., 2026) triggers consolidation from sustained semantic recurrence, while MemDream directs repair using generated-probe failures, including queries requiring evidence across separate nodes.The methods use different signals to allocate consolidation or repair effort.Source: M Comparison with Recent Memory… · p. 27
  • MemDream evaluates retrieval-oriented long-term memory on LoCoMo and MemoryAgentBench; TrustMem overlaps on MemoryAgentBench and RecMem on LoCoMo, but settings are not matched for ranking.Reader models, memory budgets, and evaluation protocols are relevant to empirical comparisons.Source: M Comparison with Recent Memory… · p. 27

Related papers

Loading 2609.34545v1…