PaperlayerView exact revision on arXiv

Source-linked AI summary

Zep: A Temporal Knowledge Graph Architecture for Agent Memory

Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, Daniel Chalef

arXiv:2501.13956v1cs.CLcs.AIcs.IR

TL;DR

Agents require evolving conversational and business knowledge, but existing RAG approaches focus on largely static corpora. Zep[5] uses Graphiti[6], a temporally aware knowledge graph, to integrate these sources while preserving historical relationships, achieving state-of-the-art memory-benchmark performance with lower token costs and latency.

  • Problem

    RAG approaches focus on largely static corpora, while agents require continuously evolving interaction, business, and world data that cannot fit effectively within LLM context windows.

    Source: §1 · p. 1

  • Method

    Zep[5] uses Graphiti[6], a temporally aware knowledge graph, to synthesize conversational and structured business data while preserving historical facts, relationships, and validity periods.

    Source: §1 · p. 2

  • Results

    Zep achieves state-of-the-art memory-benchmark performance with reduced token costs and latency, including approximately 90% shorter LongMemEval response times.

    Source: §4.3.2 · p. 7, §5 · p. 8

  • Takeaways & Limitations

    Zep’s improvements extend across model scales and are most pronounced on complex, nuanced questions when paired with more capable models.

    Source: §4.3.2 · p. 7

    DMR’s single-turn fact-retrieval questions do not assess complex memory understanding and poorly represent real-world enterprise use cases.

    Source: §4.2 · p. 6

Abstract

from arXiv · show

We introduce Zep, a novel memory layer service for AI agents that outperforms the current state-of-the-art system, MemGPT, in the Deep Memory Retrieval (DMR) benchmark. Additionally, Zep excels in more comprehensive and challenging evaluations than DMR that better reflect real-world enterprise use cases. While existing retrieval-augmented generation (RAG) frameworks for large language model (LLM)-based agents are limited to static document retrieval, enterprise applications demand dynamic knowledge integration from diverse sources including ongoing conversations and business data. Zep addresses this fundamental limitation through its core component Graphiti -- a temporally-aware knowledge graph engine that dynamically synthesizes both unstructured conversational data and structured business data while maintaining historical relationships. In the DMR benchmark, which the MemGPT team established as their primary evaluation metric, Zep demonstrates superior performance (94.8% vs 93.4%). Beyond DMR, Zep's capabilities are further validated through the more challenging LongMemEval benchmark, which better reflects enterprise use cases through complex temporal reasoning tasks. In this evaluation, Zep achieves substantial results with accuracy improvements of up to 18.5% while simultaneously reducing response latency by 90% compared to baseline implementations. These results are particularly pronounced in enterprise-critical tasks such as cross-session information synthesis and long-term context maintenance, demonstrating Zep's effectiveness for deployment in real-world applications.

1 Introduction

Zep [5] addresses agents’ need for continuously evolving memory through Graphiti [6], a temporally aware knowledge graph engine. It synthesizes conversational and business data while preserving historical facts, relationships, and their validity periods.

1 Introduction

  • LLM-based agents face limits in context windows, effective context utilization, and pretrained knowledge, requiring additional context for out-of-domain knowledge and reduced hallucinations.Source: §1 · p. 1
  • Agents need continuously evolving interaction, business, and world data, whereas current RAG approaches primarily address broad knowledge in largely static corpora.RAG uses information retrieval techniques [2] to supply domain knowledge, but entire conversation histories and business datasets cannot fit effectively within LLM context windows.Source: §1 · p. 1
  • Zep [5] combines unstructured messages and structured business data through Graphiti [6], a dynamic, temporally aware knowledge graph engine.Graphiti updates the graph without losing information, preserving a timeline of facts and relationships, including their periods of validity.Source: §1 · p. 2
  • Zep evaluates memory retrieval using MemGPT’s Deep Memory Retrieval task [3] and LongMemEval [7], emphasizing accuracy, latency, and scalability as a production system.Source: §1 · p. 2

2 Knowledge Graph Construction

Zep constructs a temporally aware knowledge graph with episode, semantic entity, and community tiers. Bi-temporal timestamps and contradiction-based edge invalidation support evolving relationships while preserving their history.

2 Knowledge Graph Construction

  • Zep organizes memory into three hierarchical tiers: raw episodes, resolved semantic entities and relationships, and communities summarizing strongly connected entities.Episodes retain non-lossy source data, while edges connect episodes to entities, entities to one another, and communities to members. This organization draws on episodic/semantic memory models, AriGraph, GraphRAG, and hierarchical RAG strategies [11].Source: §2 · p. 2

2.1 Episodes

  • Episodes accept messages, text, or JSON; the experiments focus on conversational messages carrying an actor and reference timestamp.Separate event-time T and ingestion-time T′ timelines distinguish event chronology from ingestion order, with T′ supporting auditing and reference timestamps anchoring relative dates. Bidirectional episode–semantic-edge indices enable source tracing and retrieval of related entities and facts, although these connections were not evaluated experimentally.Source: §2.1 · pp. 2–3

2.2.1 Entities

  • 1024-dimensional entity-name embeddings support cosine-similarity candidate retrieval alongside full-text search, followed by LLM-based entity resolution using episode context.Extraction uses the current message plus n = 4 preceding messages, automatically includes the speaker, and applies reflection inspired by reflexion[12] to reduce hallucinations and improve coverage. Entity summaries support resolution and retrieval; duplicate matches receive updated names and summaries, while predefined Cypher queries maintain schema consistency instead of LLM-generated queries.Source: §2.2.1 · p. 3

2.2.2 Facts

  • Fact deduplication restricts hybrid edge searches to matching entity pairs, preventing cross-entity mismatches and reducing the search space.Facts receive embeddings before graph integration. Extracting the same fact between different entities enables a hyper-edge implementation for complex multi-entity facts.Source: §2.2.2 · p. 3

2.2.3 Temporal Extraction and Edge Invalidation

  • Graphiti combines temporal extraction and edge invalidation to update relationships as conversations evolve while retaining current states and historical changes.Using episode reference timestamps to resolve absolute and relative dates, edges store t′created/t′expired for system creation/invalidation and tvalid/tinvalid for fact validity. An LLM detects temporally overlapping contradictions, prioritizes newly ingested information, and sets an invalidated edge’s tinvalid to the incoming edge’s tvalid.Source: §2.2.3 · p. 3

2.3 Communities

  • Community detection uses label propagation [13] rather than Leiden [14], enabling incremental updates that delay full community refreshes.New entities join the plurality community of their neighbors and trigger summary updates, reducing latency and LLM inference costs but gradually diverging from a full run, so periodic refreshes remain necessary. Following GraphRAG, communities use iterative map-reduce-style member summaries; unlike its map-reduce retrieval, Zep embeds keyword-rich community names for cosine-similarity search.Source: §2.3 · p. 4

3 Memory Retrieval

Zep converts text queries into LLM context through a configurable search–rerank–construct pipeline, combining lexical, semantic, and graph-based search with rerankers that prioritize relevant results.

3 Memory Retrieval

  • Zep converts text queries into LLM-ready context through candidate search, result reranking, and context construction.Candidates comprise lists of semantic edges, entity nodes, and community nodes. Constructed context includes edge facts with tvalid and tinvalid fields, entity names and summaries, and community summaries.Source: §3 · p. 4

3.1 Search

  • Zep combines cosine semantic similarity, Okapi BM25 word matching, and breadth-first contextual search to broaden candidate coverage.Search targets edge facts, entity names, and community names containing relevant keywords and phrases. Cosine similarity and BM25 use Neo4j’s Lucene implementation [16].Source: §3.1 · pp. 4–5
  • Breadth-first search expands initial results within n-hops and can use recent episodes as seeds to retrieve recently mentioned entities and relationships.Accepting nodes as search parameters provides control over which graph neighborhoods contribute context.Source: §3.1 · p. 5

3.2 Reranker

  • Zep increases retrieval precision using Reciprocal Rank Fusion (RRF), Maximal Marginal Relevance (MMR), graph-based rerankers, and crossencoders.Graph-based rerankers prioritize frequent conversational mentions or proximity to a designated centroid node. Crossencoders score query relevance through cross-attention over nodes and edges, but incur the highest computational cost.Source: §3.2 · p. 5

4 Experiments

Zep improved LongMemEval accuracy and latency over baseline implementations, while its DMR gains were marginal. These evaluations covered only a subset of Graphiti’s search capabilities.

4 Experiments

  • The experiments retrieve relevant facts and entity summaries as context, testing only a subset of Graphiti’s search functionality.The general protocol retrieves the 20 most relevant edges and entity nodes and formats them into a context string. DMR specifically reports top-10 node-and-edge retrieval, with an LLM judge comparing responses against golden answers.Source: §4 · p. 5, §4.2 · p. 6

4.1 Choice of models

  • Zep ingests conversation histories into knowledge graphs using gpt-4o-mini-2024-07-18, with BGE-m3 handling embeddings and reranking.Response generation used gpt-4o-mini-2024-07-18 and gpt-4o-2024-11-20. DMR additionally used gpt-4-turbo-2024-04-09 for direct comparability with MemGPT’s results.Source: §4.1 · p. 5, §4 · p. 5

4.2 Deep Memory Retrieval (DMR)

  • 94.8% versus 94.4% accuracy with gpt-4-turbo and 98.2% versus 98.0% with gpt-4o-mini favored Zep over full-conversation baselines.MemGPT reported 93.4% accuracy with gpt-4-turbo; session-summary baselines achieved 78.6% with gpt-4-turbo and 88.0% with gpt-4o-mini. MemGPT’s gpt-4o-mini results could not be reproduced because methodological details were insufficient.Source: Table 1 · p. 6, §4.2 · p. 6
  • DMR’s short conversations, single-turn factual questions, and ambiguous wording limit its assessment of complex memory and enterprise applications.Each conversation contains only 60 messages, readily fitting current context windows. LongMemEval offers longer, more coherent conversations and more diverse questions, with reported LLM performance declining as conversation length increases.Source: §4.2 · p. 6

4.3 LongMemEval (LME)

  • Approximately 115,000 tokens per conversation characterize LongMemEvals, remaining within frontier-model context windows while supporting evaluation across six unevenly distributed question types.The types are single-session-user, single-session-assistant, single-session-preference, multi-session, knowledge-update, and temporal-reasoning. Answers were evaluated using GPT-4o and question-specific prompts from, reported to correlate highly with human evaluators.Source: §4.3 · pp. 6–7

4.3.1 LongMemEval and MemGPT

  • A direct LongMemEval comparison with MemGPT was unavailable because inserting existing conversations into archival history failed to produce successful question responses.The workaround was necessary because the framework did not support direct ingestion of existing message histories.Source: §4.3.1 · p. 7

4.3.2 LongMemEval results

  • 15.2% and 18.5% accuracy improvements over baseline were reported for Zep with gpt-4o-mini and gpt-4o, respectively, alongside approximately 90% lower response times.Reduced prompt sizes contributed to latency reductions. Zep also incurred additional network latency that was absent from baseline evaluations.Source: Table 2 · p. 7, §4.3.2 · p. 7, §4.3 · p. 7
  • 17.7% and 9.06% performance decreases on single-session-assistant questions occurred with gpt-4o and gpt-4o-mini, respectively, despite gains on other LongMemEval categories.Zep with gpt-4o-mini improved four of six categories, particularly single-session-preference, multi-session, and temporal-reasoning; gpt-4o additionally improved knowledge-update performance. Less capable models may require further development to interpret Zep’s temporal data.Source: Table 3 · p. 7, §4.3.2 · pp. 7–8

5 Conclusion

Zep reports state-of-the-art memory-benchmark performance with lower token costs and latency, while identifying opportunities for improved extraction, graph modeling, and evaluation.

5 Conclusion

  • Zep achieves state-of-the-art performance on existing memory benchmarks while reducing token costs and latency.Its graph-based LLM memory combines semantic and episodic memory with entity and community summaries.Source: §5 · p. 8
  • Fine-tuned models improve GraphRAG extraction accuracy while reducing costs and latency [25], motivating models tailored to Graphiti prompts.Graphiti-specific fine-tuning may improve knowledge extraction, particularly for complex conversations.Source: §5 · p. 8
  • Domain-specific ontologies warrant exploration in Graphiti, as LLM-generated knowledge-graph research has primarily operated without formal ontologies [4][17][19][26].Graphiti and Zep likely represent initial advances in graph-based memory; integrating other GraphRAG approaches and developing novel extensions remain additional research directions.Source: §5 · p. 8
  • Existing memory benchmarks often lack robustness and complexity, and none adequately assesses synthesis of conversation history with structured business data.Additional benchmarks should reflect business applications such as customer experience, beyond simple needle-in-a-haystack retrieval. Zep’s traditional RAG capabilities also require evaluation against established benchmarks,, and.Source: §5 · p. 8
  • Production scalability remains insufficiently addressed in LLM memory and RAG research, particularly regarding cost and latency.Zep includes retrieval-latency benchmarks to begin addressing this gap, following LightRAG’s emphasis on these metrics.Source: §5 · p. 8

6 Appendix

The appendix specifies prompts for extracting and resolving conversational entities and relationship facts, plus assigning timestamps that describe when relationships become valid or cease to hold.

6.1.1 Entity Extraction

  • Entity extraction identifies explicitly or implicitly mentioned entities in the current message, always placing the speaker first and excluding relationships, actions, and temporal information.Other significant entities, concepts, or actors are included, with node names made as explicit as possible using full names.Source: §6.1.1 · p. 8

6.1.2 Entity Resolution

  • Entity resolution uses node names and summaries to detect duplicates despite naming differences, returning the matched UUID and most complete full name.The response explicitly reports whether the new node duplicates an existing entity.Source: §6.1.2 · p. 9

6.1.3 Fact Extraction

  • Fact extraction captures relationships between two distinct provided entities from the current message, using concise all-caps relationship types and detailed factual descriptions.Descriptions retain all relevant information and consider temporal aspects when relevant.Source: §6.1.3 · p. 9

6.1.4 Fact Resolution

  • Fact resolution treats edges expressing the same information as duplicates even when wording differs, returning a duplicate flag and the matched existing edge’s UUID.Nonduplicate facts receive is_duplicate: false.Source: §6.1.4 · p. 10

6.1.5 Temporal Extraction

  • Temporal extraction dates relationship formation or alteration, resolving relative times against a reference timestamp without inferring dates from related events.valid_at records when a relationship began and invalid_at when it ended; non-spanning facts receive only valid_at, present-tense facts use the reference timestamp, and fields without qualifying temporal information remain null. Datetimes use ISO 8601 with a timezone, defaulting to UTC Z; date-only values use midnight, and year-only values use January 1 at midnight.Source: §6.1.5 · p. 10

Related papers

This paper cites

Loading 2501.13956v1…