PaperlayerView exact revision on arXiv

Source-linked AI summary

Structured Episodic Event Memory

Zhengxuan Lu, Dongfang Li, Yukun Shi, Beilun Wang, Longyue Wang, Baotian Hu

arXiv:2601.06411v2cs.CL

TL;DR

Long-term memory systems for LLM agents often rely on flat retrieval that misses structural dependencies and dynamic event context. SEEM combines graph and episodic memory, provenance-linked event frames, associative fusion, and Reverse Provenance Expansion; it consistently outperforms competitive baselines on LoCoMo and LongMemEval. The framework improves the maintenance of coherent narratives and logical consistency, while incurring efficiency, error-propagation, and representation-scope limitations.

  • Problem

    LLM agents need memory that preserves factual consistency and contextual relevance across long-term interaction passages and queries.

  • Method

    SEEM builds graph memory for relational facts and episodic memory from provenance-linked event frames, then fuses them for context synthesis and reasoning.

  • Results

    SEEM consistently outperforms competitive memory-augmented and dense retrieval baselines on LoCoMo and LongMemEval, including a 4.4% absolute margin over HippoRAG 2 on LongMemEval.

  • Takeaways & Limitations

    SEEM provides a scalable approach for maintaining global context and enhancing long-term reasoning in LLM-based agents.

  • Takeaways & Limitations

    SEEM increases latency and token costs, risks error propagation in structured memory, and may omit abstract information that does not fit predefined semantic slots.

Abstract

from arXiv · show

Current approaches to memory in Large Language Models (LLMs) predominantly rely on static Retrieval-Augmented Generation (RAG), which often results in scattered retrieval and fails to capture the structural dependencies required for complex reasoning. For autonomous agents, these passive and flat architectures lack the cognitive organization necessary to model the dynamic and associative nature of long-term interaction. To address this, we propose Structured Episodic Event Memory (SEEM), a hierarchical framework that synergizes a graph memory layer for relational facts with a dynamic episodic memory layer for narrative progression. Grounded in cognitive frame theory, SEEM transforms interaction streams into structured Episodic Event Frames (EEFs) anchored by precise provenance pointers. Furthermore, we introduce an agentic associative fusion and Reverse Provenance Expansion (RPE) mechanism to reconstruct coherent narrative contexts from fragmented evidence. Experimental results on the LoCoMo and LongMemEval benchmarks demonstrate that SEEM significantly outperforms baselines, enabling agents to maintain superior narrative coherence and logical consistency.

1 Introduction

SEEM addresses long-term memory limitations in LLM agents by combining relational graph memory with structured episodic memory. Its provenance-linked event frames, associative fusion, and hybrid retrieval support coherent reasoning, outperforming competitive baselines on two benchmarks.

  • LLM agents struggle with long-term interaction because finite context windows and unstable memory hinder recall beyond immediate context.
  • Traditional RAG retrieves local passages through vector similarity but often misses structural dependencies needed for multi-hop reasoning.
  • 4.4% absolute margin over HippoRAG 2 on LongMemEval accompanies consistent outperformance of competitive memory-augmented and dense retrieval baselines.
  • SEEM combines an Episodic Memory Layer for narrative progression with a Graph Memory Layer for static relational facts.
  • Episodic Event Frames and Reverse Provenance Expansion link structured memory units to source passages and reconstruct coherent contexts for reasoning.
  • The authors report empirical validation that SEEM maintains logical consistency and narrative coherence across evaluated settings.

2 Related Work

Existing memory systems improve retrieval structure but remain limited by flat representations, entangled abstraction levels, and incomplete event-context modeling. These gaps motivate representations that preserve dependencies and situational dimensions together.

  • Vector-based RAG retrieves through flat vector spaces, often producing de-contextualized and scattered context for complex multi-hop reasoning.
  • Graph and summary-based methods link related text segments or support associative retrieval but can entangle thematic abstractions with fine-grained facts.
  • Interaction-history systems may fail to preserve coherent event contexts because selective summarization or rigid entity relations omit unified situational structure.

3 Methodology

SEEM builds a hierarchical memory representation that combines episodic narrative frames with graph-structured facts, preserving links to source passages. During inference, associative retrieval and provenance expansion synthesize coherent evidence for logically consistent generation.

  • Hierarchical memory architecture: SEEM maps interaction passages into a dual-layer memory: EML captures narrative progression, while GML organizes static factual relations.Both layers are grounded in original passages through provenance pointers.
  • Problem formulation: Memory-augmented generation constructs an intermediate representation M, retrieves a query-relevant subset, and generates a response from that memory.The formulation separates memory consolidation from conditioned generation.
  • Episodic Event Frame Extraction: EEF extraction converts each passage into a cognitive frame containing an event summary, semantic roles, and a provenance pointer to its source.The roles include participants, action, time, location, causality, and manner.
  • Associative Consolidation and Fusion: Associative fusion merges semantically related event frames and unions their provenance pointers so one consolidated frame can retrieve evidence scattered across turns.An LLM-based judge determines whether a new frame and a retrieved historical frame belong to the same event.
  • Graph Memory Layer: The GML extracts relational quadruples with temporal validity, links graph nodes to source passages, and merges similar nodes to bridge lexical variations.The quadruples represent entities, relations, and temporal validity.
  • Retrieval and Context Synthesis: Reverse Provenance Expansion adds all passages associated with retrieved event frames, after which expanded passages, event frames, and top relational facts are serialized into the reasoning context.This context supports cross-referencing high-level facts with episodic evidence during response generation.

4 Experimental Setup

SEEM is evaluated on LongMemEval and LoCoMo using lexical and semantic measures, against dense-retrieval and memory-based baselines under standardized backbone configurations.

  • Benchmarks: LongMemEval tests five memory competencies across 500 manually curated questions in evolving user-agent interactions.The competencies include information extraction, multi-session reasoning, temporal reasoning, and knowledge updates.
  • Benchmarks: LoCoMo evaluates comprehension of extremely long-term open-domain conversations spanning up to 32 sessions and averaging 16k tokens.
  • Evaluation Metrics: LoCoMo uses token-level F1, BLEU-1, and LLM-as-a-Judge to assess lexical similarity, semantic correctness, and factual accuracy.
  • Baselines: SEEM is compared with compact and generalist embedding models, HippoRAG 2, and other memory-augmented approaches.
  • Implementation: The experiments standardize backbone models and additionally test Pangu-Embedded-7B to assess model-agnostic robustness and efficiency.Qwen3-Next-80B-A3B-Instruct is the primary backbone, with Pangu-Embedded-7B serving as a smaller secondary backbone.

5 Results

Across LoCoMo and LongMemEval, SEEM achieves the strongest overall results, with gains in semantic reasoning, temporal understanding, smaller-backbone performance, and robustness to expanded retrieval.

  • Main Results: SEEM yields the highest scores across most evaluation metrics on LoCoMo and LongMemEval.Table 1 summarizes benchmark performance, while Table 2 breaks down LoCoMo performance by question category.
  • Dense Retrieval: SEEM exceeds NV-Embed-v2 by 3.2% in F1 and 3.3% in LLM-as-a-Judge score on LoCoMo.
  • Memory-based Frameworks: SEEM achieves 61.1 F1 and 78.0 J on LoCoMo, exceeding HippoRAG 2 by 2.8% and 1.5%, respectively.
  • Memory-based Frameworks: SEEM achieves 65.0% accuracy on LongMemEval, a 4.4% absolute improvement over HippoRAG 2.
  • Question Categories: SEEM outperforms in four of five LoCoMo question categories, with notable gains in single-hop, temporal, and adversarial reasoning.The reported temporal advantage is associated with event-centric indexing, while provenance grounding helps distinguish evidence from distractors.
  • Semantic vs. Lexical Performance: SEEM’s gains are especially evident in LLM-as-a-Judge and LongMemEval accuracy, which emphasize semantic alignment and factual correctness over word overlap.The results attribute narrative reconstruction to Reverse Provenance Expansion, which expands fragments into complete event contexts.
  • Parameter-Constrained Backbones: With Pangu-Embedded-7B, SEEM consistently outperforms baseline frameworks, indicating effective performance under parameter constraints.The LongMemEval results support maintaining long-term context without relying solely on massive parameter counts.
  • Retrieval Sensitivity: Increasing initial retrieval size from 3 to 10 produces a 5.9% F1 gain without the degradation typical of traditional RAG as context grows.

6 Conclusion

SEEM addresses scattered retrieval in long-term interactions by combining episodic event frames with associative fusion to synthesize coherent narratives from fragmented observations.

  • Conclusion: SEEM integrates episodic event frames and associative fusion to synthesize coherent narratives from fragmented observations.
  • Conclusion: The framework maintains global context and provides a scalable approach for enhancing long-term reasoning in complex environments.

Limitations

SEEM’s effectiveness is accompanied by computational, reliability, and representation-scope limitations.

  • Computational Efficiency: SEEM increases latency and token costs because frame extraction and associative fusion rely heavily on LLMs.These costs are higher than with standard vector retrieval.
  • Reliability: Errors in initial extraction or fusion can propagate and permanently corrupt the structured memory store.
  • Representation Scope: Predefined semantic slots may limit representation of abstract information that does not fit standard cognitive frame definitions.

Ethical Considerations

SEEM raises privacy, user-control, bias, and safety considerations because it persistently stores interaction-derived memory and relies on LLMs for extraction and generation. Cross-model validation also examines whether its gains depend on a particular backbone.

  • Ethical Considerations: Persistent episodic frames and relational quadruples require privacy safeguards when retaining user interaction histories over extended periods.The paper recommends anonymization and explicit user control over modifying or deleting stored memory frames.
  • Ethical Considerations: SEEM may inherit or amplify underlying-model social biases, potentially solidifying them in long-term memory and subsequent reasoning.The paper recommends content filtering and auditing during memory consolidation.
  • Ethical Considerations: The authors replace Qwen3-Next-80B-A3B-Instruct with GPT-OSS-120B to test whether improvements derive from the architecture rather than one model.This is framed as a controlled cross-model validation on LoCoMo.
  • Ethical Considerations: SEEM maintains performance leadership with GPT-OSS-120B, mirroring trends observed with the Qwen3-Next-80B-A3B-Instruct backbone.The authors describe the gains as reinforcing the model-agnostic character of the hierarchical episodic architecture.

A.2 Granular Category-wise Evaluation

Category-wise evaluations on LoCoMo and LongMemEval examine how SEEM performs across distinct reasoning challenges. The framework is especially strong in temporal, multi-hop, knowledge-update, and distractor-related settings.

  • LoCoMo Categories: SEEM achieves superior performance across four of five LoCoMo reasoning categories.The strongest advantages occur in Temporal and Multi-hop reasoning.
  • LoCoMo Categories: Structured EEFs capture chronological dependencies that dense retrieval and static graph-based approaches may overlook.The passage links this observation to SEEM’s advantages in Temporal and Multi-hop reasoning.
  • LoCoMo Categories: SEEM shows higher resilience to adversarial distractors, indicating lower vulnerability to hallucinations than traditional retrieval-based systems.The passage states this comparison directly without providing a numerical value.
  • LongMemEval Categories: LongMemEval evaluates Speaker-Specific, Multi-Session, Temporal, and Knowledge Update reasoning categories among its six task dimensions.The category analysis reports accuracy across these distinct reasoning settings.
  • LongMemEval Categories: SEEM achieves the highest average accuracy on LongMemEval, driven primarily by Knowledge Update and Temporal reasoning.The supplied passage does not include the corresponding numerical accuracy.

A.3 Evaluation of Incremental Memory Construction

SEEM is evaluated for sequential memory construction and for the structural organization supporting long-horizon reasoning. Its incremental mode preserves performance while its episodic and graph layers provide complementary forms of context.

  • Incremental Construction: SEEM processes four chronological passage segments sequentially in the incremental construction setting rather than building memory from one batch.This setup tests streaming interaction scenarios.
  • Incremental Construction: The marginal discrepancy between batch and incremental modes indicates stable performance across all evaluation metrics.The authors attribute this stability to associative fusion preserving narrative coherence and structural integrity across fragments.
  • Graph Memory Structure: Temporal anchors are prevalent in the GML, supporting chronological dependency resolution and relational propagation during hybrid retrieval.The passage presents the GML as a factual foundation for contextual reasoning.
  • Episodic Event Frames: EEF representations decompose interactions into roles such as Reason and Method, preserving intent-action distinctions for social and causal reasoning.The framework also fuses conversational inquiries with responses into cohesive episodic units.

B.3 Analysis of Associative Fusion

SEEM’s associative fusion consolidates fragmented interaction turns into structured episodic frames while retaining temporal, semantic, and provenance information. The resulting dual-layer memory supports coherent, less redundant retrieval and question answering.

  • Associative Fusion: Associative fusion reduces the number of memory units by synthesizing fragmented turns into unified episodic frames.This grouping mitigates semantic redundancy and improves retrieval density.
  • Associative Fusion: Multi-turn fusions bridge narrative sequences and help maintain a logically continuous memory state for long-horizon reasoning.The supplied passages connect structural continuity with context maintenance.
  • Dual-Layer Complementarity: The GML and EML have a mean cosine similarity of 0.46, indicating complementary semantic dimensions despite grounding in the same interaction context.The analysis uses 1,282 valid retrieval pairs from 1,986 LoCoMo queries.
Loading 2601.06411v2…