MemMachine: A Ground-Truth-Preserving Memory System for Personalized AI Agents

Shu Wang; Edwin Yu; Oscar Love; Tom Zhang; Tom Wong; Steve Scargall; Charles Fan

Source-linked AI summary

MemMachine: A Ground-Truth-Preserving Memory System for Personalized AI Agents

Shu Wang, Edwin Yu, Oscar Love, Tom Zhang, Tom Wong, Steve Scargall, Charles Fan

arXiv:2604.04853v1Apr 6, 2026cs.AI

TL;DR

LLM agents need persistent memory because fixed parameters and finite context windows limit learning from interactions and retaining long-range conversational evidence. MemMachine preserves raw episodes while layering profile memory and contextualized retrieval, achieving strong benchmark accuracy with lower token use and composable multi-hop retrieval. Its results are configuration- and workload-dependent, and procedural memory is not currently implemented.

Problem
Fixed LLM parameters and restricted context windows limit persistent personalization and retention of relevant interaction history.
Method
MemMachine combines short-term and long-term episodic memory with profile memory, preserves raw episodes, and expands retrieval matches with neighboring context.
Results
93.0% LongMemEvalS accuracy, 0.9169 LoCoMo performance with gpt-4.1-mini, and 93.2% HotpotQA-hard accuracy were reported alongside approximately 80% fewer tokens than Mem0.
Takeaways & Limitations
Ground-truth-preserving storage supports cost-efficient long-term memory while adaptive retrieval strategies can be layered on without changing the underlying storage model.
Takeaways & Limitations
Results depend on evaluation models, prompts, providers, comparison configurations, benchmark coverage, and workload-dependent token-efficiency conditions.

Abstract

from arXiv · show

Large Language Model (LLM) agents require persistent memory to maintain personalization, factual continuity, and long-horizon reasoning, yet standard context-window and retrieval-augmented generation (RAG) pipelines degrade over multi-session interactions. We present MemMachine, an open-source memory system that integrates short-term, long-term episodic, and profile memory within a ground-truth-preserving architecture that stores entire conversational episodes and reduces lossy LLM-based extraction. MemMachine uses contextualized retrieval that expands nucleus matches with surrounding context, improving recall when relevant evidence spans multiple dialogue turns. Across benchmarks, MemMachine achieves strong accuracy-efficiency tradeoffs: on LoCoMo it reaches 0.9169 using gpt4.1-mini; on LongMemEvalS (ICLR 2025), a six-dimension ablation yields 93.0 percent accuracy, with retrieval-stage optimizations -- retrieval depth tuning (+4.2 percent), context formatting (+2.0 percent), search prompt design (+1.8 percent), and query bias correction (+1.4 percent) -- outperforming ingestion-stage gains such as sentence chunking (+0.8 percent). GPT-5-mini exceeds GPT-5 by 2.6 percent when paired with optimized prompts, making it the most cost-efficient setup. Compared to Mem0, MemMachine uses roughly 80 percent fewer input tokens under matched conditions. A companion Retrieval Agent adaptively routes queries among direct retrieval, parallel decomposition, or iterative chain-of-query strategies, achieving 93.2 percent on HotpotQA-hard and 92.6 percent on WikiMultiHop under randomized-noise conditions. These results show that preserving episodic ground truth while layering adaptive retrieval yields robust, efficient long-term memory for personalized LLM agents.

1 Introduction

Persistent AI agents need memory because fixed LLM parameters and finite context windows limit learning from interactions and retaining relevant history. MemMachine addresses this gap by preserving conversational ground truth, supporting contextualized retrieval, and evaluating cost-efficient long-term memory.

Fixed LLM parameters prevent agents from acquiring interaction-derived knowledge without retraining or fine-tuning.
Finite context windows force applications to curate and compress inference data, risking loss of relevant historical context.
Conventional RAG targets static documents and does not support agents learning from evolving user contexts across sessions.
MemMachine preserves raw conversational episodes, uses contextualized retrieval, reduces routine LLM dependence, and maintains profile memory for personalization.
93.0% overall accuracy was achieved on LongMemEval across six optimization dimensions, while the Retrieval Agent reached 93.2% on HotpotQA hard.

2 Related Work

Prior agent-memory systems span virtual memory, extraction-based memory, temporal graphs, observational compression, and broader memory operating systems. The literature exposes a preservation-versus-compression trade-off and uses benchmarks targeting long-term, episodic, temporal, and multi-session abilities.

MemGPT manages context through an operating-system-inspired virtual memory hierarchy but relies on complex, potentially latency-inducing LLM-driven operations.
Mem0 extracts conversational facts with LLM calls, while Zep combines temporal knowledge graphs with vector search for evolving relationships.
Mastra compresses conversations into dated observations and achieves strong LongMemEval scores, but cannot search a broader external corpus.
MemOS unifies plaintext, activation, and parametric memory under MemCube, requiring model-internal access beyond application-layer systems.
Agent-memory research balances compression efficiency against raw-detail preservation, while benchmarks assess long-term, multi-session, temporal, episodic, and update capabilities.

3 Memory Types for AI Agents

MemMachine organizes agent memory around episodic experience, semantic profiles, and short-term context, with temporal awareness spanning these types. Episodic memory preserves factual records, whereas profile memory abstracts user attributes for personalization; procedural memory remains unimplemented.

Episodic Memory: Episodic memory stores conversational experiences with timestamps, participants, and session identifiers as records of what happened.
Episodic Memory: Episodic memory provides ground truth for factual recall, conversation reconstruction, evidence, and cross-session continuity.
Semantic/Profile Memory: Profile memory distills preferences, facts, and behavioral patterns into high-level user attributes for personalization.
Procedural Memory: Procedural memory would encode skills, workflows, and decision heuristics, but MemMachine does not currently implement it.
Temporal Awareness: Temporal awareness tags episodes and supports filtering for reasoning about ordering, recency, and duration without a dedicated temporal-memory module.
Retrieval Choice: Effective agents combine episodic retrieval for factual grounding with semantic/profile retrieval for personalization.

4 MemMachine Architecture

MemMachine uses a client-server architecture with episodic and profile memory backed by relational, vector, and graph storage. Its staged recall pipeline preserves provenance while contextualized retrieval expands semantically matched turns with neighboring conversational context.

System Architecture: Agents access MemMachine through REST, Python SDK, and MCP interfaces, while storage spans PostgreSQL, SQLite, and Neo4j.
Data Ingestion: Each message becomes an Episode carrying producer, timestamp, session, and custom metadata before dispatch to episodic and profile memory.
Short-Term Memory: Short-term memory keeps recent episodes and LLM-generated summaries, compressing older content before transfer to long-term memory.
Long-Term Memory: Long-term indexing extracts sentences, propagates metadata, preserves episode links, and generates configurable embeddings for searchable storage.
Memory Recall: Recall checks short-term context, searches long-term vectors, contextualizes and deduplicates candidates, reranks them, and restores chronological order.
Contextualized Retrieval: Contextualized retrieval adds one preceding and two following episodes around a nucleus match, then reranks clusters for inference.
Profile Memory: Profile memory extracts and updates user facts and preferences, including when new information contradicts existing profile data.

5 Retrieval Agent

The Retrieval Agent addresses multi-hop and fan-out queries by routing each query to a specialized strategy while preserving a shared declarative memory search. Its design combines adaptive decomposition, iterative evidence accumulation, multi-query reranking, and offline prompt optimization.

Motivation: Multi-hop dependency chains create a late binding problem because later retrieval queries depend on entities discovered in earlier steps.A single embedding cannot formulate later-hop queries when intermediate entities are unknown at query time.
Architecture: The Retrieval Agent augments baseline search with an LLM-orchestrated pipeline that routes queries to purpose-built strategies without changing disabled callers.Agent mode is opt-in and maintains bounded cost and latency.
Architecture: All strategy nodes delegate to the same declarative memory search, so index and reranker improvements propagate across the retrieval tree.The tree is built once and cached, while routing decisions occur per query.
Query Routing: A root ToolSelectAgent classifies queries as multi-hop, single-hop multi-entity, or single-hop direct, routing them to ChainOfQuery, SplitQuery, or baseline MemMachine search.The classifier uses calibration examples and prioritizes explicit dependency chains.
Prompt Optimization: Approximately 4% accuracy improvement came from tuning only the final answer prompt, while jointly tuning all agent prompts improved accuracy by approximately 6%.These gains were achieved offline and add no runtime token or latency overhead.
Strategy Details: ChainOfQuery iteratively retrieves, judges sufficiency, rewrites queries, and accumulates evidence, while SplitQuery decomposes fan-out questions into concurrent independent lookups.ChainOfQuery executes up to three iterations; SplitQuery creates two to six sub-queries and defaults conservatively to no-split when ambiguous.
Reranking: Multi-query reranking concatenates the original query with rewrites and sub-queries so intermediate facts relevant to any retrieval step can influence final ranking.This helps episodes associated with intermediate entities score well even when those entities are absent from the original query.

5.6 Benchmark Results

Across five benchmarks, Retrieval Agent orchestration improves performance most on multi-hop or noisy retrieval tasks, while gains are smaller or mixed on simpler tasks. These benefits require additional LLM calls and therefore higher token costs, so agent mode is intended for selective use.

HotpotQA: 93.2% accuracy and 92.31% gold-supporting-fact recall on HotpotQA-hard exceeded MemMachine by 2.0 and 1.3 percentage points.ChainOfQuery reached 95.31% recall on multi-hop bridge questions.
WikiMultiHop: 92.6% accuracy versus 87.4% for MemMachine on WikiMultiHop under fully randomized cross-question noise, a +5.2-point improvement.The result used gpt-5-mini with all question contexts pooled into one shared episodic store.
MRCR: 81.4% versus 79.6% on MRCR, with 99.4% recall, while the no-memory LLM baseline scored 32.3%.The comparison indicates strong dependence on memory retrieval for this co-reference task.
EpBench: 73.3% versus 71.4% with gpt-4o-mini, but 73.4% versus 71.8% with gpt-5-mini on EpBench, showing model-prompt sensitivity.Retrieval Agent performance is therefore mixed and benchmark-dependent.
Cost and deployment: 36% of queries routed directly to MemMachine incur approximately 1,244 tokens, whereas ChainOfQuery reaches approximately 5,732 tokens per question.The higher multi-hop cost is bounded by a three-iteration limit, and the paper notes that agent mode is not universally beneficial.

6 LLM Integration and Model Impact

MemMachine uses LLMs selectively for summarization, profile extraction, and agent-mode inference rather than for every memory operation. Model choice affects both benchmark performance and cost, while memory retrieval can reduce input-token consumption.

Selective LLM use: LLMs serve three functions: STM summarization, profile extraction, and agent-mode inference.They are used strategically rather than for every memory operation.
Model impact: A 3–4 percentage point improvement follows from switching from gpt-4o-mini to gpt-4.1-mini across both operating modes.The improvement occurs without changing the memory system.
Cost efficiency: Approximately 78% fewer input tokens than Mem0 reduces inference cost and time-to-first-token latency.Token usage is identified as a primary cost driver for LLM applications.
Context-window behavior: Memory-augmented systems outperform raw full-context baselines even when conversations fit within 16K–26K-token context windows.Selective retrieval can mitigate lost-in-the-middle effects by surfacing relevant episodes.

7 Experimental Setup

The evaluation measures long-term conversational memory across LoCoMo and LongMemEvalS using standardized LLM-judge and overlap-based metrics. The study also compares MemMachine with established memory systems and provides reproducibility materials.

Benchmarks: LoCoMo scores 1,540 questions across single-hop, multi-hop, temporal, and open-domain categories.Its evaluation code is based on Mem0’s published framework, with adversarial questions excluded from scoring.
Benchmarks: LongMemEvalS evaluates five long-term memory abilities on 500 questions embedded in chat histories of approximately 115k tokens.The study ingests histories session by session and tests 12 configurations across six optimization dimensions.
Metrics: LoCoMo uses LLM Judge Score as its primary metric, alongside BLEU and token-level F1.The judge score is a weighted mean across categories and measures semantic equivalence against reference answers.
Metrics: LongMemEvalS reports per-category and overall llm_score using the benchmark’s standard GPT-4o-mini judge.The benchmark’s evaluation procedure is retained for comparability.
Baselines: Comparisons include Mem0, Zep, Memobase, LangMem, and OpenAI’s native-memory baseline.Some comparison results are publicly reported, while Mem0 is also rerun with gpt-4.1-mini for fair comparison.
Reproducibility: Benchmark scripts, configurations, and run instructions are released for reproducible evaluation.The paper recommends recording repository versions, model settings, provider configurations, and raw per-question outputs.

8 Results and Analysis

MemMachine performs strongly on LoCoMo and LongMemEvalS while showing substantial efficiency gains. Retrieval-stage choices, prompt formatting, search prompts, and answer-model selection generally matter more than sentence-level ingestion changes.

LoCoMo results: MemMachine achieves the highest overall LoCoMo score, exceeding the next-best system by 9.7 points.The comparison is reported for MemMachine against competing systems in the stated evaluation setting.
LoCoMo results: Single-hop, multi-hop, temporal, and open-domain scores are 0.9465, 0.8759, 0.7352, and 0.7083, respectively.Temporal performance trails Memobase at 0.8505, while agent mode with gpt-4.1-mini reaches 0.9159 on temporal questions.
Efficiency: Approximately 80% fewer input tokens, 75% faster memory addition, and up to 75% faster search improve efficiency.These measurements are reported as efficiency advantages beyond accuracy.
Retrieval depth: +4.2 percentage points results from increasing retrieval depth from k=20 to k=30, while k=50 falls to 0.890.The pattern reflects a trade-off between recovering relevant episodes and introducing distracting context.
Answer-model selection: GPT-5-mini improves over GPT-5 by +2.6% under the evaluated prompt configuration.At k=30, GPT-5-mini scores 0.916 versus GPT-5’s 0.902; at k=50, the scores are 0.928 and 0.914.
Accuracy–cost trade-off: C12 reaches 0.922 with 2.58M input tokens, whereas C15 reaches 0.930 using 3.8× as many input tokens.The reported comparison identifies C12 as Pareto-optimal under the evaluated configurations.

9 Discussion

The results suggest that retrieval quality, model–prompt co-optimization, and preserving raw episodic evidence are central design considerations for personalized memory systems. These benefits are balanced by limitations involving evaluation scope, privacy, cacheability, infrastructure, and untested optimization interactions.

Design implications: Retrieval-stage improvements cumulatively exceed the gain from sentence chunking, emphasizing how memory is recalled over how it is stored.Reported retrieval-side gains include retrieval depth, formatting, search prompt design, chain-of-thought removal, and user-query bias correction.
Design implications: GPT-5-mini’s advantage over GPT-5 persists across retrieval depths, showing that model selection and prompt design must be co-optimized.The reported advantage is +1.4% at both k=30 and k=50.
Personalization: Episodic memory supplies factual grounding while profile memory captures distilled user identity and preferences.Together they support continuity, adaptation, trust through recall, and proactive suggestions.
Context design: MemMachine combines an STM summary with selectively retrieved raw episodes, balancing high-level context against uncompressed factual grounding.This trade-off is positioned for auditability, compliance, and multi-hop reasoning over exact conversational records.
Privacy and deployment: Privacy depends on whether local, hosted, or hybrid model providers handle conversational data.Hosted APIs send data through third-party infrastructure, while local providers can keep processing on premises.
Limitations: The reported results are configuration- and workload-dependent rather than universal across production settings.Limitations include eval-model and provider sensitivity, mixed comparison protocols, incomplete workload coverage, directional token comparisons, and unexplored interaction effects.
Architectural tensions: Retrieval provides scalable access to large memory stores but adds latency and can invalidate prompt caches.The STM component keeps recent context immediately available without retrieval.
Architectural tensions: Prompt cacheability remains an optimization opportunity because retrieved episodes vary by query, despite the semi-stable STM prefix.The paper suggests caching frequently retrieved episode clusters as a possible future direction.

10 Future Work

Future work extends MemMachine toward richer procedural and temporal reasoning, larger-scale evaluation, adaptive retrieval budgets, and executable code-based tool use.

10 Future Work: Future work includes storing and retrieving learned action patterns, tool-use strategies, and workflow recipes as procedural memory.
10 Future Work: The system could add temporal indexing and query-expansion techniques to improve performance on temporal benchmarks.
10 Future Work: Evaluation could extend to LongMemEvalM, which contains 500 sessions and approximately 1.5M tokens per question.
10 Future Work: Adaptive retrieval budgets could adjust per-sub-query limits using query complexity and accumulated evidence to reduce redundant retrieval.
10 Future Work: Function-calling code mode could let agents emit executable Python or TypeScript instead of invoking large predefined tool lists.

11 Conclusion

MemMachine combines ground-truth-preserving memory with efficient retrieval and profile personalization. Across benchmarks, it reports strong accuracy and token efficiency, while its retrieval strategies extend to noisy multi-hop queries.

11 Conclusion: MemMachine uses short-term and long-term episodic memory augmented by profile memory to store, recall, and reason over past experiences.
11 Conclusion: 0.9169 on LoCoMo with gpt-4.1-mini was achieved using approximately 80% fewer tokens than Mem0.
11 Conclusion: 93.0% overall accuracy on LongMemEvalS showed that retrieval-stage optimizations, especially retrieval depth tuning (+4.2%) and context formatting (+2.0%), dominated ingestion-stage changes.
11 Conclusion: 93.2% on HotpotQA hard and 92.6% on WikiMultiHop with randomized noise demonstrate composability of retrieval strategies over the underlying storage model.
11 Conclusion: MemMachine is presented as a foundation for production-oriented agents whose personalization, accuracy, and trustworthiness depend on memory quality.