Source-linked AI summary
Memory OS of AI Agent
Jiazheng Kang, Mingming Ji, Zhe Zhao, Ting Bai
TL;DR
Fixed context windows limit long-term coherence and personalization in AI-agent dialogues. MemoryOS introduces a hierarchical memory-management system and achieves average improvements of 49.11% on F1 and 46.18% on BLEU-1 over baselines on LoCoMo with GPT-4o-mini.
Problem
Fixed-length context windows limit LLMs’ ability to maintain continuity, factual consistency, and personalization in long dialogues.
Method
MemoryOS combines storage, updating, retrieval, and generation through hierarchical short-, mid-, and long-term memory tiers.
Results
49.11% on F1 and 46.18% on BLEU-1 average improvement over baselines on LoCoMo with GPT-4o-mini.
Takeaways & Limitations
MemoryOS supports coherent, personalized responses and persistent user preferences in extended conversational interactions.
Abstract
from arXiv · showhide
Large Language Models (LLMs) face a crucial challenge from fixed context windows and inadequate memory management, leading to a severe shortage of long-term memory capabilities and limited personalization in the interactive experience with AI agents. To overcome this challenge, we innovatively propose a Memory Operating System, i.e., MemoryOS, to achieve comprehensive and efficient memory management for AI agents. Inspired by the memory management principles in operating systems, MemoryOS designs a hierarchical storage architecture and consists of four key modules: Memory Storage, Updating, Retrieval, and Generation. Specifically, the architecture comprises three levels of storage units: short-term memory, mid-term memory, and long-term personal memory. Key operations within MemoryOS include dynamic updates between storage units: short-term to mid-term updates follow a dialogue-chain-based FIFO principle, while mid-term to long-term updates use a segmented page organization strategy. Our pioneering MemoryOS enables hierarchical memory integration and dynamic updating. Extensive experiments on the LoCoMo benchmark show an average improvement of 49.11% on F1 and 46.18% on BLEU-1 over the baselines on GPT-4o-mini, showing contextual coherence and personalized memory retention in long conversations. The implementation code is open-sourced at https://github.com/BAI-LAB/MemoryOS.
1 Introduction
Fixed-length context windows make LLMs struggle with coherent, personalized long-term dialogue, especially across temporal gaps and extended interactions. MemoryOS addresses this limitation through an operating-system-inspired architecture combining hierarchical storage, dynamic updating, retrieval, and generation.
- Motivation: Fixed-length memory management causes disjointed dialogue memory, factual inconsistencies, and reduced personalization, undermining persistent user adaptation and multi-session knowledge retention.These limitations become particularly acute in extended interactions requiring stable persona representation.
- Related Work: Existing LLM memory mechanisms include knowledge-organization methods and retrieval mechanism-oriented approaches for structuring, managing, and querying information.Examples include interconnected semantic networks or notes and semantic retrieval systems.
- MemoryOS: MemoryOS introduces an operating-system-inspired memory framework with four modules—Storage, Updating, Retrieval, and Generation—and three tiers: short-term, mid-term, and long-term storage.Its architecture provides hierarchical storage, dynamic updating, adaptive retrieval, and contextual generation.
- Contributions: MemoryOS is designed to preserve long-term conversational coherence and user persona persistence by dynamically capturing and evolving user preferences across extended dialogues.The paper reports experiments validating response correctness, coherence, and handling of long conversational interactions across diverse benchmark datasets.
2 Related Work
Prior work addresses LLMs’ long-term coherence and personalization limits through knowledge-organization, retrieval-oriented, and architecture-driven memory frameworks. MemoryOS is inspired by operating-system segment-page management, organizing conversational memory into prioritized logical segments and pages for context management and personalization.
- Fixed-length LLM designs struggle with temporal gaps, causing fragmented memory, factual inconsistencies, and diminished personalization in long dialogues.
- LLM memory systems broadly fall into knowledge-organization, retrieval mechanism-oriented, and architecture-driven frameworks (Zhang et al., 2024; Wu et al., 2025; Du et al., 2025).
- Operating-system segment-page management balances logical structure with physical utilization, while segmentation supports protection and sharing and paging reduces internal fragmentation.
- MemoryOS’s overview comprises memory Store, Updating, Retrieval, and Response modules.
- MemoryOS applies this principle by dividing conversation-topic segments into pages, using heat-based prioritization to retain relevant content and discard or archive less-accessed information.
3 MemoryOS
MemoryOS is a hierarchical memory-management system for AI agents that dynamically updates and retrieves memory to support coherent, personalized long-conversation interactions. It organizes short-term, mid-term, and long-term personal memory through modular storage, updating, retrieval, and response generation.
- System Overview: The architecture comprises four modules—memory storage, updating, retrieval, and generation—that dynamically manage relevant context for coherent and personalized long-conversation interaction.MemoryOS is presented as a comprehensive system that updates memory dynamically and retrieves semantically relevant context.
- Memory Storage: MemoryOS uses three hierarchical stores: STM for timely conversations, MTM for recurring topic summaries, and LPM for persistent user or agent preferences.LPM contains separate user and agent personas, supporting persistent personal details, knowledge, traits, settings, and interaction-derived characteristics.
- Memory Updating: MTM groups topic-related dialogue pages into semantically coherent segments, evicts low-heat segments at capacity, and transfers segments with heat above τ=5 to LPM.Heat combines retrieval frequency, segment interaction length, and recency, preserving highly engaged topics while updating user and agent traits and knowledge.
- Memory Updating: STM retains recent dialogue pages and migrates the oldest page to MTM when its fixed-length queue reaches capacity, using FIFO updates.Each page records a user query, model response, and timestamp, while dialogue chains preserve contextual continuity across related exchanges.
- Memory Retrieval and Generation: MemoryOS retrieves all STM pages for recent context, selects MTM segments before relevant dialogue pages, and incorporates LPM knowledge for personalized responses.The retrieval process updates a segment’s visit counter and recency factor after access, while generation integrates STM, MTM, and LPM results into a coherent prompt.
4 Experiments
Experiments on GVD and LoCoMo evaluate MemoryOS for conversational memory, showing superior benchmark performance, efficiency, and long-dialogue personalization. Ablations and sensitivity analyses further identify the importance of hierarchical memory components and controlled retrieval.
- Benchmark Results: MemoryOS outperforms representative memory methods because hierarchical storage and coordinated management preserve cross-topic dependencies, unlike flat FIFO, single-stage retrieval, or costly graph linking.MemoryBank performs worst; TiM improves over it but cannot preserve cross-topic dependencies, while MemGPT and A-Mem lack systematic memory management mechanisms.
- Benchmark Results: 49.11% average F1 and 46.18% BLEU-1 improvements on LoCoMo with GPT-4o-mini, while GVD accuracy exceeds A-Mem by 3.2%.MemoryOS achieves superior performance across the benchmark datasets, with especially strong gains on the more challenging LoCoMo memory tasks.
- Ablation Study: Removing MemoryOS sharply reduces long-dialogue performance, with Mid-Term Memory contributing most, followed by the Long-Term Persona Module and Dialogue Page Chain.The ablation removes MTM, LPM, the dialogue page chain, or the entire memory system.
- Efficiency Analysis: 4.9 versus 13 average LLM calls and 3,874 versus 16,977 recalled tokens show lower efficiency costs than A-Mem* and MemGPT, respectively.Efficiency is measured by average LLM calls per response and tokens consumed in memory retrieval.
- Hyperparameter Analysis: On LoCoMo, increasing retrieved pages k from 5 to 40 improves performance until gains diminish, so MemoryOS sets k = 10 to balance useful context and noise.Excessive retrieved content can introduce noise and adversely affect performance.
- Case Study: Case studies show MemoryOS recalls weeks-old activities and preferences, supported by mid-term segment-page storage, dialogue-page chaining, and personalization.The system recalls wetland-park activities and the user’s goal of wanting to get fit.
5 Conclusion
MemoryOS is a novel OS-inspired memory management system for AI agents that uses hierarchical storage to address fixed context windows in long conversations. Its segment-paging storage, dynamic updating, semantic retrieval, and heat-driven eviction prioritize critical dialogue information.
- 5 Conclusion: MemoryOS uses a hierarchical memory storage architecture to address fixed context window limitations in long conversations.The system is inspired by memory management mechanisms in operating systems.
- 5 Conclusion: OS-style segment-paging storage for dialogue history supports efficient memory storage, updating, and semantic retrieval.
- 5 Conclusion: Heat-driven eviction dynamically prioritizes critical information across dialogue history.