Selective Forgetting: A Graph-Based Memory Framework for Long-Term LLM Agents

Theo Rusu; Sourena Khanzadeh; Manar Alalfi

Source-linked AI summary

Selective Forgetting: A Graph-Based Memory Framework for Long-Term LLM Agents

Theo Rusu, Sourena Khanzadeh, Manar Alalfi

arXiv:2608.28978v1Aug 29, 2026cs.AI

TL;DR

The paper asks whether knowledge-graph structure improves long-term conversational memory and whether memory can remain efficient over extended interactions. It builds a graph-based memory pipeline with selective forgetting and evaluates it on LongMemEval. Graph retrieval underperforms flat retrieval overall, while forgetting reduces persistent storage with little change in token F1 but some loss in judged correctness.

Problem
The paper investigates whether structuring long-term conversational memory as a knowledge graph improves retrieval and reasoning, while systems remain efficient over extended interactions.
Method
The framework extracts conversational turns into typed nodes and attributed edges, retrieves a two-hop subgraph from top-ranked nodes, and periodically prunes low-importance nodes using recency, frequency, centrality, and age.
Results
Graph RAG underperforms baseline RAG overall, with token F1 of 0.417 versus 0.468, while selective forgetting substantially reduces persistent graph storage without statistically significant loss across the reported metrics.
Takeaways & Limitations
Graph structure alone is insufficient for uniformly better conversational memory; effective systems should combine structured representations with stronger update policies and selective retention.
Takeaways & Limitations
The results characterize a single extraction-based pipeline evaluated with one small extractor model and one benchmark, rather than graph-structured memory in general.

Abstract

from arXiv · show

Knowledge graphs have been proposed as a structured alternative to flat retrieval-augmented generation for long-term agent memory, on the assumption that representing conversations as entities and relations improves recall. We evaluate that assumption directly. Our framework extracts each conversational turn into typed nodes and attributed edges, answers questions from a two-hop subgraph, and periodically prunes nodes that score low on a weighted combination of recency, access frequency, degree centrality, and age. On LongMemEval, the graph does not outperform a flat vector baseline at a matched candidate-generation budget of five retrieval roots: token F1 is $0.417$ against $0.468$, and a paired bootstrap over 500 questions gives $Δ= -0.050$ (95\% CI $[-0.085, -0.016]$). The gap is widest on questions that require recalling a specific prior assistant turn, where judged correctness falls from $0.911$ to $0.607$, suggesting that decomposing a turn into entities discards the surface form these questions depend on. The forgetting module is more successful. Applied once to a persistent 27{,}021-node graph, it removes 9.8\% of nodes and 9.5\% of stored bytes; token F1 is unchanged ($+0.001$, 95\% CI $[-0.015, +0.016]$) and judged correctness falls by $1.6$ points, with the 95\% interval bounding any loss at $3.8$ points ($[-0.038, +0.006]$). Because our extractor is a single small model evaluated on one benchmark, these results characterise this extraction-based pipeline rather than graph-structured memory in general. Code: https://github.com/skhanzad/Selective-Amnesia

1 Introduction

Long-term memory is critical for sustained agent interactions, but flat retrieval has known limitations and graph-based memory introduces an unresolved growth-management challenge. This work tests whether knowledge-graph structure improves retrieval while using selective forgetting to control memory size.

Long-term memory enables agents to incorporate information from past exchanges during prolonged interactions.
Flat retrieval-augmented generation stores memories as similarity-based vectors but can retrieve noisy or redundant context and support limited multi-hop reasoning.
Knowledge graphs organize memories as entities and relationships, offering structured semantic and relational representations.
Unbounded memory growth can degrade retrieval quality, increase computational cost, and retain low-utility information.
The study evaluates graph-based conversational memory and an explicit forgetting module that reduces memory size without materially affecting retrieval quality.

2 Related Work

Prior memory research established external read/write stores, retrieval-based agent memory, and structured graph representations. Forgetting is increasingly treated as necessary for efficient memory, and this work combines it with graph-structured storage evaluated on a long-term conversational benchmark.

External memory architectures established controller-based read/write abstractions later adapted to persistence across LLM-agent turns and sessions.
Retrieval-augmented generation grounds outputs by retrieving passages from non-parametric stores, while later systems add dense retrieval and retrieval control.
Knowledge-graph methods represent information as entities and relations, supporting retrieved triples, graph traversal, and community summaries.
Selective retention is motivated across cognition and neural learning because indefinite accumulation or naive sequential learning can degrade memory performance.
LongMemEval evaluates long-term interactive memory demands including multi-session reasoning and knowledge updates.

3 Methodology

The framework maintains a persistent evolving knowledge graph through retrieval, update, and retention stages. It extracts conversational entities and relationships, retrieves a two-hop context from top-ranked nodes, and periodically prunes low-importance nodes.

3 Methodology: The system models interactions as an evolving graph of entities and relationships, periodically pruning low-importance nodes to bound growth.
3 Methodology: Each conversational turn is extracted, embedded, de-duplicated, and written as nodes and edges; retrieval selects top-5 nodes and expands them through a 2-hop traversal.
3 Methodology: The three-stage pipeline retrieves and serializes relevant subgraphs, extracts and integrates new graph content, then removes nodes below a retention threshold.
3 Methodology: Nodes use typed labels, titles, content, attributes, and temporal, access, and retention fields, while edges carry typed predicates and relational attributes.
3 Methodology: A single GPT-4o-mini extraction call processes each turn, handling user and assistant information through distinct extraction modes.
3.6 Importance Scoring and Forgetting: Node importance combines recency, access frequency, degree centrality, and turn age, with weights 0.35, 0.25, 0.20, and 0.20 respectively.The recency and turn-age components use 90-day and 1,000-turn exponential half-lives.
3.6 Importance Scoring and Forgetting: Every 400 conversational turns, nodes with importance below 0.10 and their incident edges are pruned.

4 Experiments

The experiments compare graph retrieval with flat retrieval and test whether forgetting compresses a persistent graph without degrading retrieval. Both use LongMemEval, matched language models, and comparisons of answer quality and storage.

4 Experiments: The experiments vary only retrieval and memory mechanisms while holding the underlying language model constant.
4 Experiments: LongMemEval contains 500 questions paired with multi-session conversation histories spanning approximately 33 months.
4 Experiments: Its question types probe single-session user, assistant, and preference recall alongside knowledge updates and other long-term memory demands.
4.1 Experiment 1: Experiment 1 builds a fresh graph per question and compares top-5 graph retrieval with top-5 raw-text chunk retrieval from the same haystack.
4 Experiments: Performance is measured with token-level precision and F1-score plus binary LLM-judge correctness averaged across questions.
4.2 Experiment 2: Experiment 2 compares persistent graphs with and without one post-ingestion application of forgetting, evaluating retrieval quality and storage footprints.

5 Discussion and Limitations

The experiments show that graph structure does not uniformly improve long-term conversational retrieval, while selective forgetting can reduce persistent graph size with limited quality loss. Benefits depend on question type, and pruning introduces a memory-efficiency versus information-retention trade-off.

Graph RAG underperforms the flat baseline overall, with token F1 of 0.417 versus 0.468 and LLM-judge accuracy of 0.454 versus 0.536.
Graph RAG improves only on temporal-reasoning questions, where LLM-judge accuracy is 0.293 versus 0.278 for flat retrieval.Graph structure represents events, participants, ordering, and temporal relationships explicitly.
The largest deficit occurs on single-session assistant questions, with judged correctness of 0.607 versus 0.911, because graph extraction may discard details needed for precise recall.The flat baseline can retrieve the original assistant turn verbatim, whereas graph extraction decomposes it into entities and relationships.
Graph RAG also trails on knowledge-update questions, with F1 of 0.456 versus 0.511, partly because conflict resolution can retain outdated attributes.The passage identifies last-write-wins as a possible improvement for appropriate factual and numeric attributes.
Forgetting removes 2,653 nodes and reduces graph storage by 9.5%, while token-level F1 remains nearly unchanged at 0.293 versus 0.292.LLM-judge accuracy decreases from 0.300 to 0.284 in this persistent-graph comparison.
Experiment 2 compares persistent memory with and without forgetting rather than maximizing retrieval accuracy, because merging all 500 haystacks introduces cross-conversation interference.
Pruned nodes have low re-reference frequency, limited connectivity, and reduced recency, but some still contribute to correct answers.This creates a trade-off between memory efficiency and information retention.
Overall, graph structure is most useful when relational or temporal information matters, whereas flat text retrieval remains advantageous for precise or verbatim recall.The discussion recommends combining structured representations with stronger update policies, selective retention, and preservation of important verbatim context.

6 Conclusion

The conclusion finds that knowledge-graph memory offers both relational benefits and information-loss limitations. It therefore emphasizes extraction, updating, and retention mechanisms, including selective forgetting, rather than representation alone.

Knowledge-graph conversational memory supports reasoning over temporally and semantically connected information but can harm tasks requiring precise or verbatim recall.
Performance depends critically on how information is extracted, updated, and retained over time.
Pruning a 27,021-node store removes 9.8% of nodes and 9.5% of bytes, with no significant change detected across four metrics.The paired bootstrap covers 500 questions.
Effective long-term memory systems should combine structured representations with stronger update mechanisms and selective retention strategies.

A.1 Note to Reviewers on Experimental Scope and AI Use

The study reports scope limits from a fixed evaluation budget and distinguishes model use within the method from AI assistance in authoring.

Four complete runs at n = 500 were conducted instead of a broader sweep because each configuration required costly extraction, generation, and judging calls.The authors used temperature = 0 and paired bootstrap intervals over the runs.
The findings characterize this extraction-based graph memory pipeline at this model scale rather than graph-structured memory in general.The authors identify a matched-compression control, a second benchmark, and a stronger extraction model as priorities for additional budget.
GPT-4o-mini served as the extraction, answer-generation, and judging model, while other language models assisted with appendix prose, literature search, and implementation.The authors state that technical design, analysis, and interpretation remained their own and that AI-assisted output was reviewed.

A.2 Justification for parameter values chosen for the experiments and methodology.

The methodology fixes parameters using empirical probes, budget constraints, conventions, and structural reasoning, while acknowledging that retention settings were not fully swept.

The conservative de-duplication threshold prioritizes a low false-merge rate because false merges irreversibly collapse entities and corrupt incident edges.Missed merges leave redundant nodes that later de-duplication or retrieval can still surface.
A 0.75 retrieval cosine floor admits paraphrases and partial entity mentions while excluding merely topical neighbors before ranking and truncating candidates.Because candidates are truncated to the top 5, the floor matters mainly when fewer than five nodes clear it.
Five retrieval roots match the flat vector baseline's retrieved chunks, equalizing the candidate-generation budget between representations.This design attributes performance differences to representation rather than retrieval-hit count.
Two-hop expansion is chosen as the smallest depth supporting relational and multi-session cases without the super-linear expansion observed at greater depths.One hop usually adds only attributes of a single entity, whereas three or more hops often reach hub-mediated relations irrelevant to the query.
The forgetting module runs every 400 turns with a 0.10 threshold, targeting the low-score tail rather than a fixed compression ratio.Scoring costs O(N + M), and the interval allows recency and turn-decay terms to distinguish dormant nodes.
Retention parameters were not fully swept, so Experiment 2 represents a single point on the compression–quality trade-off.The individual scoring components were also not isolated against a simpler retention rule at matched compression.

A.3 Role-Aware Extraction Prompt

The role-aware extraction prompt explicitly labels each conversational turn by speaker and distinguishes user speech from assistant responses.

The role-aware variant distinguishes user and assistant turns and specifies which assistant-provided facts should be retained.
The extractor is instructed to return strict JSON without markdown or a preamble.
Each turn is prefixed with a speaker role identifying either the human user or the AI assistant.The prompt uses separate [Role: user] and [Role: assistant] labels.

A.4 Sample Knowledge-Graph Extraction Prompt

The appendix specifies the extraction prompt, graph-ingestion pipeline, computational costs, and bootstrap procedures used to evaluate the memory framework.

A.4 Sample Knowledge-Graph Extraction Prompt: The extraction prompt converts each conversational message into nodes, directed edges, and attributes, returning only supported facts in strict JSON.Nodes use predefined labels, edges use precise relationship labels, and attributes capture quantities, dates, durations, frequencies, or related qualifiers.
A.4 Sample Knowledge-Graph Extraction Prompt: The sample prompt represents nodes with labels, titles, content, and attributes, while edges specify source, target, relationship, and attributes.The prompt example exposes the JSON fields used for node and edge records.
A.5 Algorithms: The ingestion pipeline extracts and embeds entities, deduplicates them against existing memory, adds relations, retrieves relevant nodes, expands a subgraph, and optionally forgets nodes.Standard operation also converts the same message into retrievable context, while matching uses exact titles followed by semantic similarity.
A.5 Algorithms: The deduplication threshold is selected by testing whether varied references to the same entities are merged correctly without incorrectly merging distinct entities.The procedure searches for a trade-off between duplicate merging and avoiding false merges.
A.6 Computational Complexity: The complexity analysis parameterizes costs by graph size, embedding dimension, extracted entities, relations, query entities, and retrieved-subgraph size.It excludes internal LLM and embedding-call costs, which depend on token lengths rather than graph size.
A.6 Computational Complexity: Under exhaustive vector search, the worst-case per-turn cost is O(Nd + M), while space complexity is O(Nd + N + M).The traversal is capped at 15 nodes, and the N + M query-time term would disappear if adjacency were persisted.
A.6 Computational Complexity: The appendix identifies exhaustive linear scans as the only memory-size-growing components and notes approximate nearest-neighbor indexing could reduce the O(Nd) factor.The forgetting module instead bounds N by removing low-importance stored nodes.
A.7 Statistical Significance of Experimental Results: The statistical analysis uses temperature 0, resampling over questions, 95% bootstrap intervals, Wilson intervals for judge correctness, and paired bootstrap comparisons between matched conditions.A difference is significant at α = 0.05 when its 95% confidence interval excludes zero; the reported key evidence concerns no significant quality difference after forgetting.