Source-linked AI summary
Zep: A Temporal Knowledge Graph Architecture for Agent Memory
Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, Daniel Chalef
TL;DR
LLM agents need dynamic memory because static RAG and finite context windows limit access to evolving conversational and business knowledge. Zep addresses this gap with Graphiti, a temporally-aware knowledge graph that synthesizes diverse data while preserving temporal relationships; evaluations report higher accuracy and substantially lower latency, though the authors describe these results as initial advances in graph-based memory.
Problem
Static RAG and finite context windows limit agents’ access to continuously evolving conversational, business, and world data.
Method
Zep uses Graphiti, a dynamic temporally-aware knowledge graph that synthesizes unstructured messages and structured business data while tracking temporal fact validity.
Results
Zep achieves state-of-the-art memory-benchmark performance while reducing token costs and operating at significantly lower latencies.
Takeaways & Limitations
Zep’s evaluations support graph-based memory as a practical approach for agent retrieval across evolving information and long-term context.
Takeaways & Limitations
The authors characterize Graphiti and Zep’s results as initial advances, leaving multiple research avenues for future graph-based memory systems.
Abstract
from arXiv · showhide
We introduce Zep, a novel memory layer service for AI agents that outperforms the current state-of-the-art system, MemGPT, in the Deep Memory Retrieval (DMR) benchmark. Additionally, Zep excels in more comprehensive and challenging evaluations than DMR that better reflect real-world enterprise use cases. While existing retrieval-augmented generation (RAG) frameworks for large language model (LLM)-based agents are limited to static document retrieval, enterprise applications demand dynamic knowledge integration from diverse sources including ongoing conversations and business data. Zep addresses this fundamental limitation through its core component Graphiti -- a temporally-aware knowledge graph engine that dynamically synthesizes both unstructured conversational data and structured business data while maintaining historical relationships. In the DMR benchmark, which the MemGPT team established as their primary evaluation metric, Zep demonstrates superior performance (94.8% vs 93.4%). Beyond DMR, Zep's capabilities are further validated through the more challenging LongMemEval benchmark, which better reflects enterprise use cases through complex temporal reasoning tasks. In this evaluation, Zep achieves substantial results with accuracy improvements of up to 18.5% while simultaneously reducing response latency by 90% compared to baseline implementations. These results are particularly pronounced in enterprise-critical tasks such as cross-session information synthesis and long-term context maintenance, demonstrating Zep's effectiveness for deployment in real-world applications.
1 Introduction
LLM agents need additional context because context-window, utilization, and pre-training limits constrain their capabilities. Zep addresses the limitations of static RAG by providing dynamic memory from conversations and business data through a temporally-aware knowledge graph.
- LLM agents are limited by context windows, effective context utilization, and knowledge acquired during pre-training.
- Additional context supplies out-of-domain knowledge and helps reduce hallucinations.
- RAG supplies domain knowledge using information-retrieval techniques.
- Static RAG corpora cannot provide the continuously evolving conversational, business, and world data agents require.
- Zep uses Graphiti, a dynamic temporally-aware knowledge graph that synthesizes unstructured messages and structured business data while preserving historical relationships.
- Zep evaluates memory retrieval for accuracy, latency, and scalability using DMR and LongMemEval.
2 Knowledge Graph Construction
Zep constructs a hierarchical, temporally-aware knowledge graph from raw episodes into entities, facts, and communities. Graphiti preserves source data and historical relationship states while dynamically incorporating new information and resolving contradictions.
- Zep’s graph contains episode, semantic entity, and community subgraphs organized hierarchically.
- Episodes store raw messages, text, or JSON without loss, while semantic entities and relationships are extracted from them.
- Community nodes summarize clusters of strongly connected entities and provide a higher-level view of the semantic graph.
- Graphiti uses message timestamps and bi-temporal modeling to represent event chronology separately from data-ingestion order.
- Graphiti dynamically adds evolving conversational data while maintaining current relationship states and historical records.
- Temporal extraction records when facts are valid and when they are created or invalidated in the system.
- New temporally overlapping contradictions invalidate affected edges, with newer information prioritized by transactional order.
3 Memory Retrieval
Zep retrieves agent context through a three-stage search, reranking, and construction pipeline. Multiple complementary graph and text searches improve candidate coverage before results are formatted as context for the language model.
- Zep’s retrieval API maps a text query to formatted text context for an LLM agent.
- The retrieval pipeline composes search, reranking, and context construction.
- Search returns candidate semantic edges, entity nodes, and community nodes from the graph.
- The constructor formats facts with valid date ranges, entity names and summaries, and community summaries into agent context.
- Cosine similarity, BM25 full-text search, and breadth-first graph search target semantic, lexical, and contextual similarity.
- Breadth-first search expands candidates by finding additional nodes and edges within n-hops.
- Rerankers improve precision by reordering candidates, including through RRF, MMR, and episode-mention frequency.
4 Experiments
The experiments evaluate Zep’s retrieval mechanisms on DMR and LongMemEval, using conversation histories integrated into a Zep knowledge graph and reformatted as agent context. Zep improves accuracy and latency relative to baselines, especially on complex LongMemEval question types, while DMR’s short conversations limit its memory-system challenge.
- 4 Experiments: Zep integrates conversation histories into a knowledge graph, retrieves relevant edges and entity nodes, and reformats them into a context string for response generation.The evaluation retrieves the 20 most relevant facts and entity summaries through Zep’s APIs.
- 4.3 LongMemEval: LongMemEval uses conversations averaging approximately 115,000 tokens and six question types spanning single-session, multi-session, knowledge-update, and temporal-reasoning tasks.The benchmark is designed to represent real-world business applications and challenge existing LLMs and commercial memory solutions.
- 4.3.2 LongMemEval results: 18.5% accuracy improvement was achieved by Zep with gpt-4o over the baseline, while gpt-4o-mini achieved a 15.2% improvement.The reduced prompt size also produced significant latency cost reductions compared with baseline implementations.
- 4.3.2 LongMemEval results: Zep improved gpt-4o-mini performance in four of six question categories, with the largest gains in preference, multi-session, and temporal-reasoning questions.With gpt-4o, Zep also improved the knowledge-update category, while less capable models may need further development for temporal data.
- 4.3.2 LongMemEval results: Approximately 90% response-time reduction was achieved by Zep while maintaining higher accuracy than baseline implementations.The latency comparison is reported across the LongMemEval evaluation.
- 4.3.2 LongMemEval results: 17.7% lower performance for gpt-4o and 9.06% lower performance for gpt-4o-mini occurred on single-session-assistant questions.This was identified as an exception to Zep’s otherwise consistent improvements and as an area needing further research and engineering.
5 Conclusion
Zep combines graph-based memory with semantic and episodic representations, achieving strong benchmark performance while reducing token costs and latency. The paper also identifies limitations in current memory benchmarks and positions its results as early progress toward broader graph-based memory systems.
- Zep combines semantic and episodic memory with entity and community summaries in a graph-based approach to LLM memory.The conclusion describes this architecture as the basis of Zep’s memory system.
- The reported Graphiti and Zep results likely represent only initial advances in graph-based memory systems, leaving room for further extensions.Suggested directions include integrating other GraphRAG approaches into the Zep paradigm.
- Fine-tuned models for GraphRAG entity and edge extraction can improve accuracy while reducing costs and latency, suggesting a potential direction for Graphiti.The paper identifies fine-tuned extraction models as a possible way to enhance knowledge extraction, particularly for complex conversations.
- Existing memory benchmarks often lack robustness and complexity, motivating additional evaluations that reflect business applications such as customer experience tasks.The authors report that many benchmarks default to simple needle-in-a-haystack fact retrieval and do not adequately assess Zep’s processing capabilities.
- Production-oriented evaluations should address scalability in terms of cost and latency, an area the paper begins to examine through latency benchmarks.
6 Appendix
The appendix specifies prompt-based procedures for extracting, resolving, and relating entities and facts from conversations. It also defines temporal handling for facts using reference timestamps and ISO 8601 dates.
- 6.1.1 Entity Extraction: Entity extraction identifies the speaker first, then extracts significant entities and concepts from the current message while excluding relationships and temporal information.The instructions require explicit full names and exclude entities mentioned only outside the current message.
- 6.1.2 Entity Resolution: Entity resolution determines whether a new node duplicates an existing entity using both node names and summaries.Duplicate entities may have different names, and duplicate matches require returning the existing node identifier and most complete name.
- 6.1.3 Fact Extraction: Fact extraction produces concise relation types and detailed facts only between two distinct listed entities.The procedure also requires considering temporal aspects of relationships when relevant.
- 6.1.4 Edge Resolution: Edge deduplication checks whether a new edge expresses the same factual information as any existing edge, even when the wording is not identical.
- 6.1.5 Temporal Extraction: Temporal extraction sets relationship dates only when they concern the formation or alteration of the edge fact, using ISO 8601 timestamps and a reference timestamp.Present-tense facts use the reference timestamp for valid_at, while unrelated time mentions are ignored.