Mem++: Non-Destructive Memory for Long-Term Organizational LLM Agents

1 INTRODUCTION

• Organizational decisions are recorded by many authors in documents, and revisions arrive as new documents, so earlier versions may still matter.A later statement cannot simply be assumed to update an earlier one as it might in a single-author conversation.Source: §1 · p. 1
• Write-time memory systems trade off retaining raw text against structuring it, often fixing what can be answered before a question arrives.Retrieval systems preserve text but may not distinguish superseded values; structured systems extract facts or relations and lose access to the source documents.Source: §1 · p. 2
• Mem++ stores each artifact whole, defers version selection until a question arrives, and can add links between versions without deleting them.The consolidation pass that marks older conflicting versions as superseded is optional and evaluated as an ablation.Source: §1 · p. 2
• The paper evaluates Mem++ on OrgMemBench and conversational benchmarks, compares retrieval and consolidation variants, and reports that it outperforms memory baselines on OrgMemBench.It also evaluates extracted-fact cards and an entity-graph variant.Source: §1 · p. 2

2.1 CONVERSATIONAL MEMORY ARCHITECTURES FOR LLM AGENTS

• Conversational memory architectures range from full-record retrieval to ingest-time compression, trading retained source context against compact structure.The paper distinguishes chunked retrieval, fact extraction, episode rewriting, and hierarchical or managed memory approaches.Source: §2.1 · p. 3

2.2 LONG-TERM TEMPORAL MEMORY SYSTEMS

• Temporal systems may discard or revise older values, or retain them with dates, but conversational approaches treat changes as one narrator updating their own facts.Organizational changes instead arrive as documents from different authors, and questions may concern past versions as well as the current state.Source: §2.2 · p. 3
• Mem++ keeps each version as a document, marks when a replaced version stopped holding, and bounds retrieval by the question’s date.This supports questions about what holds now, what held earlier, and what changed.Source: §2.2 · p. 3

3 METHOD

• Mem++ uses a write path, optional consolidation, and a read path; it preserves whole documents and leaves version choice to the answering model.The framework is designed to retrieve current values, past values, and information about what changed and who made the change.Source: §3 · p. 3, Figure 2 · p. 4

3.1 ARCHITECTURE

• Each document becomes a record containing its text, author, event date, embedding, and state; records are never deleted, and consolidation can mark older conflicting versions as superseded.Only the state may change after writing, and consolidation is disabled in headline runs.Source: §3.1 · p. 4

3.2 MEMORY BUILDING AND STORING

• The write path stores each document whole, creates an embedding, and indexes its text; it calls no generative model and adds new documents without reprocessing older rows.On OrgMemBench, the loader supplies no author field, while event dates are available; on conversational benchmarks, turns use session dates.Source: §3.2 · pp. 4–5

3.3 RETRIEVAL VIA MULTI-INDEX FUSION

• For dated questions, retrieval scopes all indexes to active rows dated by the as-of time, then fuses lexical, tag, and semantic rankings with weighted RRF.Lexical search recovers exact identifiers, semantic search recovers paraphrases, and three of the k slots are reserved for the latest-dated matches.Source: §3.3 · p. 5
• Memg++ appends up to five entity-relation triples, but those triples are not date-scoped and can come from artifacts later than the question’s as-of date.The graph adds context beyond the fused document rows.Source: §3.3 · pp. 5–6

3.4 EVIDENCE PRESENTATION

• The answering model receives retrieved records in rank order as original text preceded by event dates and, when available, author tags.On the reported OrgMemBench tier, artifact authors are unnamed, so each row carries the same placeholder author alongside its date.Source: §3.4 · p. 6

4.1 EXPERIMENTAL SETUP

• The evaluation uses OrgMemBench’s 443 dated artifacts and 73 rubric-graded questions, plus LoCoMo and LongMemEvalS; LLM-judge scores are primary across all three.Seven baselines span full context, retrieval, extracted memories, temporal graphs, evolving notes, and entity graphs.Source: §4.1 · pp. 6–7

4.2 MAIN RESULTS

• 57.6 overall with gpt-4.1-mini makes Mem++ the best OrgMemBench system, 2.6 points above RAG.With gpt-4o-mini, Memg++ and Mem++ score 44.4 and 44.2, ahead of RAG at 43.4.Source: Table 1 · p. 7, §4.2 · p. 7
• Mem++ leads OrgMemBench Supersession with both answerers and Audit Replay with gpt-4.1-mini, where its 83.0 is 6.2 points above Zep.The authors attribute the Supersession gap over Mem0 and Zep to retaining original documents with dates and authors; Mem0’s Supersession scores are 40.8 and 32.3.Source: §4.2 · pp. 7–8
• 81.5 with gpt-4.1-mini and 77.4 with gpt-4o-mini give Mem++ the best average LLM-judge score on LoCoMo with both answerers.It exceeds Nemori by 2.1 and 3.0 points, respectively.Source: Table 2 · p. 8, §4.2 · p. 8
• 72.2 with gpt-4o-mini and 74.7 with gpt-4.1-mini place Mem++ second on LongMemEvalS, behind Memg++ with both answerers.Both variants exceed Full Context by at least 9.1 points.Source: Table 3 · p. 8, §4.2 · p. 8
• RAG leads Mem++ on OrgMemBench Contradiction by 28.2 and 27.4 points and Bi-temporal by 16.7 with gpt-4.1-mini; all methods score below 23 on Justification Chain.On LoCoMo, Mem++ trails Nemori on Temporal F1 with gpt-4.1-mini (41.7 vs. 57.7), but the scores are level with gpt-4o-mini (56.9 vs. 56.7).Source: §4.2 · pp. 8–9

4.3 ABLATION STUDY

• Removing the vector leg lowers scores by 35.7 points on OrgMemBench and 76.0 on LongMemEvalS, while removing the lexical leg changes scores by at most 0.7.On LoCoMo, removing the vector leg lowers the score by 9.1 points; the lexical leg matches any query term there.Source: §4.3 · p. 9
• Mem++ performs as well without consolidation or a fact index, while whole-document indexing scores 2.6 points higher on LoCoMo than extracted-fact indexing.The entity graph’s gains are inconsistent across answerers; on OrgMemBench the answerer never sees the graph triples. The authors consider Mem++ the better accuracy-cost trade-off because the graph adds three LLM calls per document and question cycle.Source: §4.3 · p. 9, §4.2 · p. 9

5 CONCLUSION

• Experiments show that keeping every version and deferring the choice between them to read time outperforms write-time extraction on OrgMemBench.The authors conclude that this is a strong baseline for organizational memory.Source: §5 · p. 9
• Ablations identify semantic retrieval over original documents as the driver; consolidation, fact extraction, and entity graphs add cost without consistent gain.Source: §5 · p. 9
• The organizational evaluation covers 73 questions about one synthetic organization; the authors plan to collect more diverse organizational records.Source: §5 · p. 9
• The authors designed Mem++ and reviewed AI-assisted work; they used generative AI for avatar generation and agentic AI to help edit the paper.Source: AI USE STATEMENT · p. 10
• The authors state that the full system and experiments will be open-sourced upon publication.Source: REPRODUCIBILITY STATEMENT · p. 10

A METHOD DETAILS

• The write path skips empty or exact-repeat documents, stores each remaining document whole without a generative call, and indexes its text, author, and embedding.Source: A METHOD DETAILS · p. 12
• At read time, Mem++ filters active rows by an optional as-of date, ranks candidates with lexical, tag, and vector indexes, then combines their ranks into evidence scores.Source: A METHOD DETAILS · p. 12
• The evidence set takes the top k−3 scored rows, reserves three slots for latest-dated matched rows, and fills any remaining slots with recent unmatched rows.The answering model receives rows in rank order, each preceded by its date and author where recorded.Source: A METHOD DETAILS · p. 12
• The optional consolidation operator greedily groups current rows by embedding similarity and uses a language model to label groups with multiple rows.Source: A METHOD DETAILS · pp. 12–13
• For conflicts, the latest-event-date row stays current while older rows are marked superseded but retain their text and dates; restatements produce a merged row of at most 220 tokens.Source: A METHOD DETAILS · pp. 12–13
• A consolidation-generated merged row can carry later content past an as-of date because it takes the earliest group date; this caveat applies only to the disabled-by-default ablation.Source: A METHOD DETAILS · p. 13

B.1 BASELINES

• Full Context places the whole history in the answering model’s prompt and uses no external memory.Source: §B.1 · p. 13
• RAG splits history into fixed-size chunks and retrieves those whose embeddings are closest to the question; chunks can split a document.On LoCoMo, the suffix identifies the chunk size in tokens.Source: §B.1 · p. 13
• Mem0 extracts salient facts with an LLM and may add, update, or delete stored memories, allowing a newer fact to overwrite an older one.Source: §B.1 · p. 13
• Zep extracts entities and facts into a temporal knowledge graph, invalidates contradicted edges, and combines semantic, keyword, and graph search at query time.Source: §B.1 · p. 13
• A-Mem stores each interaction as an LLM-written note with keywords, tags, and context, links related notes, and can update older notes’ attributes.Source: §B.1 · p. 13
• Nemori segments conversations into episodes, rewrites them as narratives, and stores semantic knowledge missed when predicting a new episode from existing memory.Source: §B.1 · p. 13
• gbrain combines keyword, vector, and graph search over a store extracted from documents and is reported only on OrgMemBench.Source: §B.1 · p. 13

B.2 EVALUATION METRICS

• The primary LLM-Judge Score is the share of answers judged correct against gold answers on LoCoMo and LongMemEvalS.On OrgMemBench, gpt-4o-mini grades answers against rubric facets, allowing partial coverage.Source: §B.2 · p. 14
• OrgMemBench scores each answer against 247 weighted rubric facets across 73 questions, and the authors judge each run three times and report the mean.Source: §B.2 · p. 14
• F1 is the harmonic mean of token-level precision and recall after answer normalization; the paper reports it on LoCoMo, where exact-wording metrics can penalize correct paraphrases.Source: §B.2 · p. 14
• BLEU-1 measures unigram precision against the gold answer with a brevity penalty; it is reported on LoCoMo and can penalize correct paraphrases.Source: §B.2 · p. 14

• All runs use fusion constant γ = 60, k = 50 retrieved rows, and three recency slots.Source: §B.3 · p. 14
• A disjunctive lexical index returns rows matching any query term; a conjunctive index requires every term.Source: §B.3 · p. 14
• On OrgMemBench, the local encoder embeds at most 256 word pieces per document, while stored text and the lexical index retain the full document.Source: §B.3 · p. 14
• Only OrgMemBench’s 15 Audit Replay questions use an as-of date, and its 40,000-character context limit admits about 19 of 50 rows.Recency slots and Memg++ triples follow fused rows and therefore do not reach the answering model on this benchmark.Source: §B.3 · p. 14
• Memg++ calls gpt-4o-mini twice per document at write time and once per question at read time, but conversational-benchmark write calls run per ten-turn chunk.Source: §B.3 · p. 14

• For k ≥30, the answering model sees the same rows because fused results exceed the 40,000-character context budget; remaining changes are run-to-run variation, reaching 4.4 points with gpt-4o-mini.The budget holds about 19 rows and at most 25; the fused-row order does not depend on k in this range.Source: §B.4 · pp. 14–15
• 40.5 at k = 5 rises to 56.5 at k = 30 for gpt-4.1-mini, then stays between 56.5 and 57.0 through k = 100.Source: §B.4 · p. 14, Table 7 · p. 15, Figure 4 · p. 15
• 36.6 at k = 5 rises to 44.2–48.7 for gpt-4o-mini at every k ≥10, with no clear trend beyond k = 15.Source: §B.4 · p. 14, Table 7 · p. 15, Figure 4 · p. 15