Source-linked AI summary
TRUSTMEM: Learning Trustworthy Memory Consolidation for LLM Agents with Long-Term Memory
Tianyu Yang, Sudipta Paul, Vijay Srinivasan, Vivek Kulkarni, Srinivas Chappidi
TL;DR
LLM agents use long-term memory to support extended interactions, but generated updates can introduce errors that persist and affect later reasoning. TRUSTMEM verifies and ranks memory transitions to train update behavior, achieving state-of-the-art results across three settings while reducing transition-level errors.
Problem
Autonomous memory updates can omit information, corrupt existing entries, or introduce unsupported content that persists and affects later reasoning.
Source: §1 · p. 1Method
TRUSTMEM evaluates memory transitions for coverage, preservation, and faithfulness, then uses Transition-Ranked GRPO to optimize updates by ranking candidates under the same memory state.
Source: §1 · p. 2Results
TRUSTMEM achieves state-of-the-art results across MemoryAgentBench, HaluMem, and Mem-α, including 12.14 F1 points over the strongest baseline in HaluMem memory extraction.
Source: §1 · p. 2Takeaways & Limitations
Transition-level verification helps reduce omission, corruption, and hallucination, improving operation-level reliability alongside memory utility.
Source: §5 · pp. 10–11Current experiments mainly focus on text-based long-term memory benchmarks, while real-world agent inputs and memory evidence can be multimodal.
Source: Limitations · p. 11
Abstract
from arXiv · showhide
Large language model (LLM) agents rely on long-term memory to support extended interactions and personalized assistance beyond finite context windows. Existing memory agents actively update external memory through generated write, revise, and delete operations, but these updates may omit important information, corrupt existing memory, or introduce unsupported hallucinated content. Once stored, such errors become persistent system-state failures that can affect future reasoning and generation. In this paper, we propose TrustMem, a framework designed to improve the trustworthiness of memory consolidation. TrustMem relies on a Memory Transition Verifier to evaluate the transition process of memory updates in terms of coverage, preservation, and faithfulness. It further constructs preference pairs among candidate updates under the same memory state, enabling preference-guided reinforcement learning to directly optimize memory updating behaviors. Extensive experiments demonstrate that TrustMem improves both memory utility and reliability: it achieves state-of-the-art results across MemoryAgentBench, HaluMem, and the Mem-alpha validation set, improves HaluMem memory extraction by 12.14 F1 points, and reduces transition-level omission, corruption, and hallucination by 40.1\%, 79.1\%, and 50.0\%, respectively, compared with the strongest baseline for each error type.
1 Introduction
Long-term memory lets LLM agents retain information beyond finite context windows, but autonomous updates can introduce persistent errors that affect later reasoning. TRUSTMEM evaluates and optimizes memory transitions, improving benchmark performance and reducing transition-level errors.
1 Introduction
- Memory consolidation can corrupt facts, omit key timelines, or add unsupported claims, and flawed entries may be repeatedly retrieved and amplified in later interactions.The examples come from Mem1 [46] and Mem-T [41] under the evaluation setting.Source: §1 · p. 1
- Terminal rewards cannot pinpoint which update in a long sequence was faulty, and correct final answers can mask unsafe intermediate edits that remain risks for future tasks.Errors may arise from omissions, destructive overwrites, or final-step reasoning flaws.Source: §1 · p. 2
- The Memory Transition Verifier scores each update for coverage, preservation, and faithfulness, while Transition-Ranked GRPO ranks candidate updates under the same memory state to train memory-editing behavior.This provides fine-grained supervision while maintaining the downstream utility of the final memory.Source: §1 · p. 2
- TRUSTMEM achieves state-of-the-art results across MemoryAgentBench, HaluMem, and Mem-α, including gains of 6.5 points and 12.14 F1 points over the strongest baseline on MemoryAgentBench and HaluMem memory extraction.It also reduces omission, corruption, and hallucination by 40.1%, 79.1%, and 50.0%, respectively, compared with the strongest baseline for each error type.Source: §1 · p. 2, Figure 2 · p. 3
2 Related Works
Prior work studies memory storage, retrieval, representation, and learnable memory management, while reliability research evaluates or defends memory systems. TRUSTMEM focuses on whether each generated update is trustworthy before storage.
2 Related Works
- Long-term memory systems study external storage and management, as well as parametric or latent representations; TRUSTMEM instead assesses whether each generated update is trustworthy before storage.The surveyed systems include Generative Agents [21], Reflexion [27], MemoryBank [45], and MemoChat [17], among others.Source: §2 · p. 3
- RL-based memory agents improve memory utility, but prior methods mainly optimize final task performance or trajectory-level rewards rather than the safety of each local update.TRUSTMEM introduces transition-level verification and preference ranking for that purpose.Source: §2 · p. 3
- Reliability work evaluates memory or addresses attacks, admission control, provenance, and governance, whereas TRUSTMEM optimizes updates to reduce omission, corruption, and hallucination before they persist.The related benchmarks include LoCoMo [19], LongMemEval [33], MemoryAgentBench [9], and HaluMem [3].Source: §2 · p. 3
3 Method
TRUSTMEM treats consolidation as a sequence of local memory transitions and trains agents to make trustworthy edits while retaining useful terminal memory. Its verifier and preference-guided optimization supervise memory updates at the transition level.
3.1 Overview of TRUSTMEM Framework
- At each chronological chunk, the agent applies WRITE, REVISE, or PRUNE actions to an evolving external memory, producing an auditable local transition.PRUNE removes entries that evidence shows are invalid, redundant, or no longer useful; no action is also allowed when an update is unnecessary.Source: §3.1 · p. 4
3.2 Memory Transition Verifier
- The Memory Transition Verifier scores each update for coverage, preservation, and faithfulness, providing local supervision intended to prevent persistent omission, corruption, and hallucination.It evaluates whether chunk information is retained, valid prior memory remains intact, and changed or added content is supported.Source: §3.2 · p. 5, §3.1 · p. 5
3.3 Multi-granularity Reward Design
- TRUSTMEM combines sample-level rewards for terminal-memory utility and compactness with transition-level rewards for executable, specific, trustworthy edits.The reward design also includes downstream task performance, action executability, content specificity, and verifier judgments.Source: §3.3 · pp. 5–6
3.4 Transition-Ranked GRPO
- GRPO optimizes the memory-construction trajectory using group-sampled rollouts and scalar rewards computed at each chunk step.The reward is group-normalized across rollouts for the same chunk step to compute the advantage.Source: §3.4 · p. 6
- Transition-Ranked GRPO uses verifier scores to rank candidate edits generated for the same chunk and memory state, forming preference pairs when the score gap exceeds a margin.The final objective combines scalar-reward optimization, KL regularization, and transition-level ranking.Source: §3.4 · p. 6
4 Experiment
Experiments assess memory utility and operation-level reliability across Mem-α, MemoryAgentBench, and HaluMem, then analyze reward components, transition errors, training dynamics, and representative cases.
4.1 Experiment Settings
- Training uses a balanced 562-instance Mem-α pool; evaluation covers held-out Mem-α, MemoryAgentBench retrieval, test-time learning and long-range understanding, and HaluMem stage-wise safety.The baselines span long-context and retrieval systems, RL-based or agentic memory methods, and system-level memory frameworks; task scores follow dataset-specific metrics.Source: §4.1 · pp. 7–8
4.2 Experiments on Memory Utility
- TrustMem scores 65.7 average on MemoryAgentBench, 6.5 points above Mem-α, and 66.3 average on Mem-α, 4.4 points above its strongest baseline.On MemoryAgentBench it also exceeds Long-Context by 19.6 points and RAG-Top2 by 15.5 points.Source: §4.2 · pp. 8–9
4.3 Experiments on HaluMem
- TrustMem achieves best HaluMem-Medium extraction F1 (69.45), improving the strongest prior F1 by 12.14 points, and has the lowest omission rates in updating and QA.The benchmark covers extraction, updating, and memory-based question answering.Source: Table 3 · p. 7, §4.3 · p. 9
4.4 Ablation Study
- Removing Ract, Rver, or Reff lowers the Mem-α average from 66.3 to 54.0, 58.7, or 61.8; Transition-Ranked GRPO scores 66.3 versus 60.3 for PRM-style GRPO.Removing Rver lowers TTL from 70.1 to 58.3; Transition-Ranked GRPO raises TTL from 57.6 to 70.1.Source: §4.4 · p. 9, Table 4 · p. 9
4.5 Reliability of Memory Consolidation
- TrustMem has the lowest transition error rates: 8.08% omission, 0.14% corruption, and 0.01% hallucination.Omission is the dominant error type because memory construction is compressive; corruption and hallucination involve writing incorrect or unsupported facts.Source: §4.5 · p. 10
4.6 Training Reward Dynamics
- TrustMem maintains higher rewards and reaches a stable plateau, while the variant without Transition-Ranked GRPO becomes unstable and sharply loses task and total reward.Both methods improve early in training.Source: §4.6 · p. 10, Figure 4 · p. 10
4.7 Case Study
- Case studies describe baseline memory errors—including reversed facts, omissions, and hallucinated events—as potentially affecting future retrieval and reasoning; TrustMem preserves evidence and avoids unsupported edits.The case-study table compares input chunks and memories stored by baseline agents and TrustMem, highlighting relevant evidence and corrupted or unsupported spans.Source: §4.7 · p. 10
5 Conclusion
The conclusion presents TrustMem as a transition-supervised approach to trustworthy memory consolidation and reports improved utility and operation-level reliability.
5 Conclusion
- TrustMem supervises intermediate memory transitions with a Memory Transition Verifier and optimizes memory-editing behavior with Transition-Ranked GRPO.Experiments on MemoryAgentBench, HaluMem, and Mem-α show improvements in memory utility and operation-level reliability.Source: §5 · p. 10
- Transition-level verification helps reduce omission, corruption, and hallucination in long-term memory systems.Representative case studies compare baseline and TrustMem memories, marking input evidence and corrupted or unsupported spans.Source: §5 · p. 11, Table 5 · p. 11
Limitations
The experiments mainly cover text-based long-term memory benchmarks, leaving multimodal memory agents and vision-language models as planned extensions.
Limitations
- Current experiments mainly focus on text-based long-term memory benchmarks; multimodal memory agents and vision-language models are planned extensions.The paper identifies multimodal user inputs and memory evidence, including images, videos, and documents, as a future direction.Source: Limitations · p. 11
A Experiment Settings
The evaluated baseline subset varies across benchmarks according to method applicability and comparable-results availability.
A Experiment Settings
- The baseline subset for each benchmark depends on method applicability and the availability of comparable results.The paper directs readers to the corresponding result tables for each benchmark’s reported baselines.Source: A Experiment Settings · p. 15
B Experiments on Memory Utility
TRUSTMEM outperforms the reported baselines on both the Mem-α validation set and out-of-distribution MemoryAgentBench evaluation. Its results include gains on test-time learning and long-range understanding tasks.
B Experiments on Memory Utility
- 66.3 average on the Mem-α validation set outperforms Mem-α by 4.4 points, with TRUSTMEM scoring 72.4 on TREC-C, 73.8 on NLU, and 64.0 on PubMed.BookSum improves from 18.7 to 22.6.Source: B Experiments on Memory Utility · p. 15
- 65.7 average on MemoryAgentBench out-of-distribution evaluation beats Mem-α by 6.5 points, Long-Context by 19.6, and RAG-Top2 by 15.5.TRUSTMEM scores 83.0 on Single-Doc, 85.0 on TREC-C, and 84.0 on Banking77.Source: B Experiments on Memory Utility · p. 15
C Experiments on HaluMem
On HaluMem-Medium, TRUSTMEM reports strong memory-extraction results and low omission rates in both updating and question answering. The benchmark covers extraction, updating, and memory-based question answering.
C Experiments on HaluMem
- HaluMem-Medium evaluates operation-level reliability across memory extraction, memory updating, and memory-based question answering.Source: C Experiments on HaluMem · p. 15
- 12.14 F1 points over Mem0 is TRUSTMEM’s memory-extraction gain, with best recall, weighted recall, and F1 scores of 56.73, 67.45, and 69.45.AtomMem has the highest extraction accuracy, but its recall and F1 are described as extremely low.Source: C Experiments on HaluMem · p. 15
- 48.64 is TRUSTMEM’s lowest memory-updating omission rate, alongside a second-best correctness score of 46.84.The verifier checks whether updates preserve valid prior memory, incorporate new evidence, and avoid unsupported changes.Source: C Experiments on HaluMem · p. 15
- 12.46 is TRUSTMEM’s lowest memory-based question-answering omission rate, while its hallucination rate of 6.07 is second-lowest.The paper reports competitive correctness and a stronger trade-off across correctness, hallucination, and omission than methods that hallucinate more or omit too much.Source: C Experiments on HaluMem · p. 16
D Memory Transition Error Judge Prompt
The prompt specifies a strict evaluator for chunk-level memory transitions, using prior memory, input, generated operations, and updated memory to identify substantive omission, corruption, or hallucination.
D Memory Transition Error Judge Prompt
- GPT-4o-mini at temperature 0 judges each transition and returns structured JSON labels for omission, corruption, and hallucination.The judge receives the prior memory, input chunk, memory operations, and updated memory.Source: D Memory Transition Error Judge Prompt · p. 17
- The reliability protocol records each chunk-level transition and calculates each error type independently, counting only major errors as failures.A transition can contain multiple error types.Source: D Memory Transition Error Judge Prompt · p. 17
- Omission means the updated memory misses key chunk information, judged by whether its absence could impair later answers or benchmark performance.Key information includes facts, constraints, preferences, entities, relations, dates, numbers, or outcomes needed for later tasks; concise summaries may omit nonessential details.Source: D Memory Transition Error Judge Prompt · pp. 17–18
- Corruption covers operations that change, contradict, misattribute, or mix facts supported by the input chunk or prior memory.Source: D Memory Transition Error Judge Prompt · p. 17
- Hallucination is a substantive new fact unsupported by the input chunk or prior memory; simply preserving a prior-supported fact is not hallucination.Source: D Memory Transition Error Judge Prompt · p. 17
- An error is major when it could plausibly harm later recall or QA; small issues are minor, and transitions without substantive errors are labeled none.When multiple error types are major, the primary error is marked mixed.Source: D Memory Transition Error Judge Prompt · p. 18