PaperlayerView exact revision on arXiv

Source-linked AI summary

Event-Centric Memory with Query-Aware Graph Augmentation for Long-Term Conversational Agents

Yichen Liu, Chunfeng Yuan, Haowei Liu, Wenjuan Li, Zefeng Lin, Bing Li, Xu Chen, Weiming Hu

arXiv:2610.11920v1cs.CL

TL;DR

Flat memory is lightweight but leaves event relations and state changes implicit, while persistent graph memory adds construction cost and can introduce irrelevant relations. QGMEM stores event-indexed memories and evolving traces, activates query-relevant units for local graph reasoning, and improves retrieval, multi-hop reasoning, conflict resolution, and scalability across six benchmarks with compact contexts.

  • Problem

    Flat-structured memory leaves event relations and state changes implicit, while graph-based memory incurs construction cost and can introduce irrelevant relations over long histories.

    Source: §I · p. 1

  • Method

    QGMEM stores event-indexed atomic memories and dynamic traces, then uses query-aware activation to organize relevant units into a local graph for reasoning.

    Source: §I · pp. 1–2

  • Results

    Across six benchmarks and contexts up to 10M tokens, QGMEM improves retrieval, multi-hop reasoning, conflict resolution, and scalability; on LoCoMo it reaches 82.50% nDCG@10.

    Source: §I · p. 2, §V · p. 10

  • Takeaways & Limitations

    QGMEM combines compact generation contexts with query-relevant relational organization across the evaluated dialogue settings.

    Source: §I · p. 2, §V · p. 10

Abstract

from arXiv · show

For persistent and personalized conversational agents, memory systems can enable them to remember, update, and reason over long histories by storing past interactions and retrieving relevant information. Existing memory systems typically follow two paradigms: flat-structured memory and graph-based memory. The former is lightweight but leaves event relations and state updates implicit, while the latter explicitly models memory structure but incurs additional construction cost and introduces irrelevant relations over long histories. To address these limitations, we propose QGMem, a novel memory construction and activation framework motivated by human memory, in which experience is organized into events and query-relevant events are modeled by graph as working memory. QGMem converts long dialogue histories into event-indexed atomic memory units that preserve individual experiences and consolidates related units into dynamic memory traces that retain state trajectories and current states. When a query arrives, hybrid memory retrieval gathers complementary candidate memories, and query-aware reranking activates the most relevant units as a compact working memory. To expose relational dependencies in the working memory and support conflict-aware reasoning, QGMem organizes the working memory as a local graph, which is then encoded as a graph token and provided to the LLM together with the textual working memory to improve evidence utilization during answer generation. Experiments across six benchmarks validate the framework and show consistent gains in retrieval, multi-hop evidence composition, conflict resolution, and ultra-long dialogue reasoning with compact contexts and moderate inference cost.

I. INTRODUCTION

Long-term conversational memory must preserve and retrieve evolving experiences, but flat storage leaves event relations implicit while persistent graphs add construction cost and irrelevant relations. QGMEM stores event-indexed memories and evolving traces, then builds a local graph over query-relevant memories; on LoCoMo it combines retrieval and answer quality with a compact context.

I. INTRODUCTION

  • Flat-structured memory leaves event relations and state changes implicit, while graph-based memory adds construction cost and can introduce irrelevant relations over long histories.QGMEM instead retains source-grounded event units and state traces, then organizes query-relevant memories as a local graph.Source: §I · p. 1
  • On LoCoMo with Qwen3-8B, QGMEM reaches 82.50% nDCG@10 and an 82.77% LLM-judged score with 598.19 retrieved-context tokens.The comparison considers retrieval-augmented, flat-structured, and graph-based systems; higher axis values and smaller context bubbles indicate better overall performance.Source: Figure 2 · p. 2, §I · p. 2

A. Persistent Conversational Memory Representation

  • Persistent graphs materialize relations across an expanding history, whereas QGMEM instantiates a graph only over activated working memory.Its persistent substrate uses source-grounded atomic units and lightweight dynamic traces to retain evidence and longitudinal state evolution.Source: A. Persistent Conversational Memory… · p. 3

B. Query-Time Memory Access

  • QGMEM combines hybrid retrieval with query-aware reranking to form compact working memory before constructing a local graph for evidence composition.The retrieval process filters irrelevant and superseded candidates while avoiding persistent global-graph maintenance and query-irrelevant relations.Source: B. Query-Time Memory Access · p. 3

C. Memory-Augmented Language Generation

QGMEM combines event-indexed persistent memory and evolving state traces with query-local graph augmentation, rather than serializing or maintaining a graph over the full dialogue history. Its construction preserves source evidence and state history, while generation uses selected textual memories and their relational structure.

C. Memory-Augmented Language Generation

  • QGMEM addresses the loss of explicit topology in text-serialized memories and the cost and irrelevant relations of graphs built over long histories with query-local graph augmentation.It couples this with event-indexed atomic storage and dynamic memory traces for facts that may be revised, supplemented, or superseded over time.Source: C. Memory-Augmented Language Generation · pp. 3–4

1) Atomic Memory Unit Construction:

  • Event-indexed atomic units preserve a source dialogue span and structured fact for each independently retrievable event or state.Overlapping dialogue windows retain limited preceding context, and multiple independently retrievable relations become separate atomic units.Source: 1) Atomic Memory Unit Construction · p. 4
  • Atomic units retain explicit entity, location, and factual-time attributes, while separating factual time from utterance time for temporal grounding.The representation is designed to distinguish answer-bearing memories from topically related events and select valid memories under time-sensitive or conflicting records.Source: 1) Atomic Memory Unit Construction · p. 5

2) Dynamic Memory Trace Consolidation:

  • Dynamic memory traces connect related units into chronological state trajectories, treating later incompatible values as updates and compatible refinements as supplements.Mutually compatible values can coexist; each trace retains its latest supported state.Source: 2) Dynamic Memory Trace Consolidation · p. 5
  • Retrieved atomic units remain linked to their traces, letting the reranker and generator use state trajectories while the latest member represents the current state.This preserves source-level granularity for verification while linking distant updates for conflict resolution and multi-hop evidence composition.Source: 2) Dynamic Memory Trace Consolidation · p. 5

B. Query-Aware Working Memory Activation

QGMEM activates a compact working memory by fusing complementary retrieval channels and reranking candidates for relevance and consistency with the current state. It then organizes the activated facts into a local graph whose relational topology complements textual memory and retained temporal state information.

1) Hybrid Memory Retrieval:

2) Query-Aware Reranking:

  • A structured candidate document supplies fact summary, source dialogue, attributes, and available state trajectory to the reranker, which selects the highest-scoring units under budget k.The resulting ordering is used for both graph construction and answer generation.Source: 2) Query-Aware Reranking · pp. 5–6

1) Local Memory Graph Construction:

2) Graph Encoding:

QGMEM encodes the activated local memory graph with relation-aware message passing, then pools the node representations and projects the result into a graph token for the LLM.

2) Graph Encoding:

  • The graph encoder applies relation-aware TransformerConv layers with multiple attention heads to incorporate predicate information during message passing.The update uses learned transformations and attention over incoming edges.Source: 2) Graph Encoding · p. 6
  • Attention aggregation propagates both memory content and typed relations into updated node representations.The attention weights depend jointly on neighboring node states and predicate embeddings.Source: 2) Graph Encoding · p. 6
  • After L layers, global mean pooling summarizes node representations and a two-layer nonlinear projector maps the summary to graph token zq.The graph token is supplied to the LLM.Source: 2) Graph Encoding · p. 6

3) Graph-Text Fusion:

QGMEM supplies textual working memory alongside a graph token, training only its graph encoder and projector while keeping the sentence encoder and LLM fixed. The framework accommodates different backbone interfaces, and its offline construction and bounded query-time graph keep costs tied to dialogue size or working-memory budget rather than full-history graph construction.

3) Graph-Text Fusion:

  • Graph-text fusion concatenates graph token zq with the textual working-memory sequence and supplies both, with the query, to the frozen LLM.This retains readable, source-grounded memory content while adding graph-structural signals for relation-aware generation.Source: 3) Graph-Text Fusion · p. 6

4) Training Objective:

  • Training updates the relation-aware graph encoder and projector while keeping the sentence encoder and backbone LLM fixed.Answer supervision encourages the graph representation to retain relational information that contributes to generation without altering the pretrained LLM.Source: 4) Training Objective · pp. 6–7

1) Adaptability Across LLM Backbones:

  • For proprietary API backbones with inaccessible embedding spaces, QGMEM serializes the activated memory and local graph as text instead of supplying a graph token in the target embedding space.The memory construction, activation, and graph organization procedures remain in place across backbone interfaces.Source: 1) Adaptability Across LLM Backbones · p. 7

2) Efficiency Analysis:

  • Offline memory construction uses overlapping fixed-size windows and examines only 20 nearest fact-level neighbors per atomic unit during trace consolidation.At query time, the induced local graph has at most k directed edges and 2k nodes before deduplication, requiring O(k) materialization time and space.Source: 2) Efficiency Analysis · p. 7

IV. EXPERIMENTS

Experiments span six long-term-memory benchmarks covering retrieval, conflict resolution, and histories up to 10M tokens, comparing QGMEM with RAG and flat- and graph-based memory systems. QGMEM reports strong results across these evaluations and across three LLM backbones.

1) Benchmarks:

  • LoCoMo and LongMemEval-S assess retrieval and recall, FactConsolidation-SH/MH assess conflict resolution, and BEAM histories range from 100K to 10M tokens.The benchmarks cover multi-session evidence composition, persistent recall and updates, single- and multi-hop conflict selection, and ultra-long-history scalability.Source: 1) Benchmarks · p. 7

2) Baselines:

  • QGMEM is compared with RAG, flat-memory systems Mem0 and LightMem, and graph-memory systems EMem-G, LiCoMemory, MR-Agent, and AriadneMem.Available systems use the same questions and backbone model for each benchmark.Source: 2) Baselines · p. 7

3) Evaluation Metrics:

  • Answer quality is assessed with repeated LLM judgments and AlignScore-large semantic coverage, with uncertainty estimated by 10,000 bootstrap resamples.BEAM follows its official protocol, macro-averaging nine nugget-based abilities and Event Ordering scored with normalized Kendall’s τb.Source: 3) Evaluation Metrics · p. 7

1) Accurate Retrieval:

  • 82.77% on LoCoMo and 62.14% on LongMemEval-S LLM-judged scores exceed the strongest baselines by 8.30 and 5.72 points, respectively.Semantic coverage reaches 56.31% and 51.31%, with gains of 12.77 and 2.78 points.Source: 1) Accurate Retrieval · p. 7, Table I · p. 8

2) Conflict Resolution:

  • 78.10% on Fact-SH and 12.50% on Fact-MH LLM-judged scores surpass the strongest baselines by 2.18 and 6.00 points, respectively.Fact-MH semantic coverage improves by 5.69 points to 13.34%, while Fact-SH coverage reaches 80.58%.Source: 2) Conflict Resolution · p. 7, Table I · p. 8

4) Retrieval Quality Analysis:

  • 89.08% Recall@10 accompanies QGMEM’s best Precision@10 (20.92%), Sufficiency@10 (97.11%), and nDCG@10 (82.50%).Precision and nDCG improve by 3.00 and 14.34 points.Source: 4) Retrieval Quality Analysis · p. 8

3) Ultra-Long Dialogue History Scalability:

  • At 10M tokens, QGMEM scores 29.81%, exceeding LightMem and AriadneMem by 20.18 and 21.21 points, respectively.It remains above 35% through 1M tokens; histories and questions are shared within each scale, but evaluation sets differ across scales.Source: Figure 4 · p. 8, 3) Ultra-Long Dialogue History… · p. 8

5) Adaptability Across LLM Backbones:

  • 64.58%, 56.00%, and 70.11% average scores across Qwen3-8B, Llama3.1-8B-Instruct, and GPT-4o-mini exceed corresponding strongest baselines by 13.57, 5.31, and 13.80 points.GPT-4o-mini uses textual serialization because its embedding space is inaccessible; QGMEM leads on LoCoMo, LongMemEval-S, and BEAM-100K for all three backbones.Source: 5) Adaptability Across LLM Backbones · p. 8

6) Efficiency Analysis:

QGMEM reports stronger backbone adaptability, while efficiency results quantify its query-time costs and amortized construction. Ablations, a case study, and sensitivity analyses examine how its components and settings contribute to performance.

6) Efficiency Analysis:

  • Figure 5 presents adaptability across LLM backbones on four benchmarks, evaluating Qwen3-8B, Llama3.1-8B-Instruct, and GPT-4o-mini.Source: Figure 5 · p. 9
  • QGMEM uses 598.19 retrieved-context tokens and 1208.79 total online tokens per QA, takes 5.94 seconds end to end, and improves the LoCoMo score over EMem-G by 20.29 points.The 1.09-second offline construction latency is amortized per QA; fixed-budget working memory keeps the maximum graph supplied to the encoder or generator from growing with history length.Source: Table III · p. 9, 6) Efficiency Analysis · p. 9

1) Component ablation:

  • Removing atomic units, dynamic traces, reranking, and the graph token lowers Macro Avg. by 7.43, 2.08, 5.23, and 3.95 points, respectively.The trace ablation has larger effects on Fact-MH (2.89 points) and BEAM-500K (4.23 points); reranker removal lowers LoCoMo and LongMemEval-S by 8.95 and 9.58 points, while graph-token removal lowers Fact-SH by 7.81 points.Source: 1) Component ablation · p. 9
  • Removing hybrid retrieval lowers Macro Avg. by 1.44 points, with decreases of 2.78 on Fact-SH and 2.30 on BEAM-100K.The ablation analysis attributes candidate coverage to the three retrieval channels and query-specific prioritization to reranking.Source: 1) Component ablation · p. 9

2) Qualitative analysis:

  • In the LoCoMo case, QGMEM reranks the answer-bearing comparison to Rank 1 and connects Jon and Gina’s shared challenges to it in the local graph.Mem0, LightMem, and EMem-G respectively illustrate ranking, evidence-utilization, and relation-selection failures.Source: 2) Qualitative analysis · p. 9, Figure 6 · p. 10

3) Hyperparameter sensitivity:

  • Working-memory accuracy peaks at K=10, candidate-pool accuracy at n=100, construction-window accuracy at w=6, and relation-consolidation accuracy at τ=0.80.Larger working memories can add irrelevant or redundant evidence; larger candidate pools show no consistent improvement, while relation thresholds trade off coverage against relevance.Source: 3) Hyperparameter sensitivity · pp. 9–10, Figure 7 · p. 10

V. CONCLUSION:

  • Across six benchmarks and contexts up to 10M tokens, QGMEM improves retrieval, multi-hop reasoning, conflict resolution, and scalability across diverse backbones with compact contexts.Its atomic units and dynamic traces preserve grounded evidence and evolving states, while query-aware activation organizes relevant units for local graph reasoning.Source: §V · p. 10

APPENDIX

The appendix specifies QGMEM’s construction, retrieval, training, and evaluation procedures, then reports an ablation of retrieval and selection choices.

1) Memory construction:

  • Memory construction splits histories at session boundaries, then uses six-turn sliding windows with two-turn overlap and a four-turn step.Overlap provides context for resolving pronouns and local references but cannot alone support new memory units; different retrievable facts or state changes may yield multiple units per window.Source: 1) Memory construction · p. 13

2) Retrieval and generation:

  • Retrieval uses a 100-item RRF-fused pool from lexical, semantic, and event-attribute channels, then selects 10 units; relation clustering uses threshold 0.80 and minimum size 2.Entity and location matching uses a 0.60 dense-similarity threshold, and all LLM-driven steps use temperature 0.Source: 2) Retrieval and generation · p. 13

B. Training Settings:

  • Training draws 1,200 ConvoMem cases yielding 2,138 QA pairs, then retains 1,634 cases with correct final answers for graph-token supervision.Partitions are train/validation/test, checkpoint selection uses validation only, and no LoCoMo or LongMemEval-S examples are used in this training part.Source: B. Training Settings · p. 13

C. Evidence Retrieval Quality Evaluation Protocol:

  • Retrieval quality is evaluated against gold evidence rather than exact text overlap, using coverage, relevance, answer-evidence sufficiency, and nDCG measures.Sufficiency requires the retrieved set together to contain enough evidence for the answer; gold evidence and judge labels are unavailable to retrieval systems.Source: C. Evidence Retrieval Quality… · p. 13

1) Retrieval and Selection Ablation:

  • On LoCoMo with Qwen3-8B, removing reranking causes the largest retrieval-ablation drop; removing event-attribute retrieval also degrades performance, while hybrid retrieval has a smaller consistent effect.The analysis links event-attribute retrieval to matching entity, location, and time constraints, and hybrid retrieval to improving candidate recall before reranking.Source: Table VI · p. 14, 1) Retrieval and Selection Ablation · p. 14

1) Experimental Settings:

  • All systems use the same answer format, LLM-as-judge setting, and number of retrieved entries; semantic retrieval uses BGE-base-en-v1.5 and reranking uses Qwen3-Reranker-8B..Source: 1) Experimental Settings · p. 14

1) LoCoMo Judge Prompt:

The judge prompt accepts semantically equivalent answers and rejects predictions that contradict the reference, omit key information, are too vague, or add unsupported content.

1) LoCoMo Judge Prompt:

  • The judge accepts semantic paraphrases but rejects contradictions, missing key details, vague answers, and unsupported information.It evaluates a prediction against a question and reference answer, returning a correct/incorrect judgment with a brief reason.Source: 1) LoCoMo Judge Prompt · p. 14

2) LongMemEval-S Judge Prompt:

The judge prompts define how answers are marked correct across LongMemEval-S question types, including temporal answers, personalized responses, and unanswerable questions.

2) LongMemEval-S Judge Prompt:

  • Responses count as correct when they contain the answer, an equivalent answer, or all intermediate steps; a subset of required information is incorrect.Source: 2) LongMemEval-S Judge Prompt · p. 14
  • Off-by-one errors in predicted durations are accepted, and an updated answer can count as correct even when the response also includes previous information.Source: 2) LongMemEval-S Judge Prompt · p. 14

Single-session Preference Tasks

  • For preference tasks, a response is correct if it recalls and uses the user's personal information correctly, without needing to address every rubric point.Source: 2) LongMemEval-S Judge Prompt · p. 14

2) LongMemEval-S Judge Prompt:

1) Prompts for QGMem:

The prompts specify how QGMem constructs and indexes memory units, consolidates evolving information, parses query constraints, and answers from retrieved memory.

1) Prompts for QGMem:

  • Atomic memory construction produces one independently retrievable, source-grounded claim per unit, with turn bounds and a triple representing its core claim.A unit may span multiple turns only when needed to complete its single claim; multiple claims from one turn can become separate units.Source: 1) Prompts for QGMem · p. 15
  • Retrieval indexes capture discriminative entities, locations, and fact times, distinguishing event time from utterance time and leaving vague times unspecified.Source: 1) Prompts for QGMem · p. 15
  • Memory-trace consolidation keeps units describing the same evolving subject-relation, orders them by memory time, and records earlier states alongside the latest supported state.A later contradiction is treated as an update, and non-conflicting additions are retained.Source: 1) Prompts for QGMem · p. 16
  • Query parsing extracts concise, discriminative entity, location, and time constraints, distinguishing fact time from conversation time and resolving relative expressions against the question time.Source: 1) Prompts for QGMem · p. 16
  • Answer generation is grounded only in the provided memory context and uses the trajectory's latest state when it conflicts with older evidence.The output includes a brief reasoning path citing memory labels and a final answer.Source: 1) Prompts for QGMem · p. 16
Loading 2610.11920v1…