PaperlayerView exact revision on arXiv

Source-linked AI summary

MemOS: A Memory OS for AI System

Zhiyu Li, Chenyang Xi, Chunyu Li, Ding Chen, Boyu Chen, Shichao Song, Simin Niu, Hanyu Wang, Jiawei Yang, Chen Tang, Qingchen Yu, Jihao Zhao, Yezhaohui Wang, Peng Liu, Zehao Lin, Pengyuan Wang, Jiahao Huo, Tianyi Chen, Kai Chen, Kehang Li, Zhen Tao, Huayi Lai, Hao Wu, Bo Tang, Zhengren Wang, Zhaoxin Fan, Ningyu Zhang, Linfeng Zhang, Junchi Yan, Mingchuan Yang, Tong Xu, Wei Xu, Huajun Chen, Haofen Wang, Hongkang Yang, Wentao Zhang, Zhi-Qin John Xu, Siheng Chen, Feiyu Xiong

arXiv:2507.03724v4cs.CL

TL;DR

LLMs lack persistent, controllable memory mechanisms for long-context reasoning, personalization, and evolving knowledge. MemOS addresses this gap with a memory operating system that unifies plaintext, activation, and parameter memories through scheduling and lifecycle management. The framework is intended to support controllable, plastic, and evolvable memory, while KV-based injection reduces time to first token without changing output sequences.

  • Problem

    LLMs rely on static parameters and short-lived context, while RAG remains stateless, limiting long-term personalization, knowledge evolution, and behavioral continuity.

  • Method

    MemOS unifies representation, scheduling, storage, provenance, lifecycle management, and transitions across plaintext, activation, and parameter memories.

  • Results

    KV-based memory injection substantially reduces TTFT while preserving identical output sequences; Qwen2.5-72B shows a 91.4% reduction under long-context, short-query conditions.

  • Takeaways & Limitations

    MemOS provides a memory-centric framework for controllability, plasticity, evolvability, and continual adaptation in LLM systems.

  • Takeaways & Limitations

    The paper identifies self-optimizing, self-reconstructing memory units and a scalable memory marketplace as future developments.

Abstract

from arXiv · show

Large Language Models (LLMs) have become an essential infrastructure for Artificial General Intelligence (AGI), yet their lack of well-defined memory management systems hinders the development of long-context reasoning, continual personalization, and knowledge consistency.Existing models mainly rely on static parameters and short-lived contextual states, limiting their ability to track user preferences or update knowledge over extended periods.While Retrieval-Augmented Generation (RAG) introduces external knowledge in plain text, it remains a stateless workaround without lifecycle control or integration with persistent representations.Recent work has modeled the training and inference cost of LLMs from a memory hierarchy perspective, showing that introducing an explicit memory layer between parameter memory and external retrieval can substantially reduce these costs by externalizing specific knowledge. Beyond computational efficiency, LLMs face broader challenges arising from how information is distributed over time and context, requiring systems capable of managing heterogeneous knowledge spanning different temporal scales and sources. To address this challenge, we propose MemOS, a memory operating system that treats memory as a manageable system resource. It unifies the representation, scheduling, and evolution of plaintext, activation-based, and parameter-level memories, enabling cost-efficient storage and retrieval. As the basic unit, a MemCube encapsulates both memory content and metadata such as provenance and versioning. MemCubes can be composed, migrated, and fused over time, enabling flexible transitions between memory types and bridging retrieval with parameter-based learning. MemOS establishes a memory-centric system framework that brings controllability, plasticity, and evolvability to LLMs, laying the foundation for continual learning and personalized modeling.

1 Introduction

LLMs lack system-level mechanisms for managing memory across long contexts, evolving knowledge, and users or tasks. MemOS addresses this gap by unifying memory representation, lifecycle control, scheduling, and adaptation across heterogeneous memory types.

  • Challenges: Static parameter memory is costly to update, difficult to interpret, and vulnerable to catastrophic forgetting, while RAG lacks unified lifecycle and temporal controls.RAG may retrieve outdated and new regulations simultaneously without reconciling, retiring, or prioritizing them.
  • Challenges: Long-range tasks expose limited context windows, quadratic attention costs, and fading adherence to user instructions or state.These constraints can cause models to forget user-defined code structures or writing styles during extended interactions.
  • MemOS: MemOS treats memory as an explicitly modeled and schedulable system resource rather than only implicit parameters or temporary retrievals.Its design targets information distributed across time and space and supports unified organization, internal-state maintenance, and long-term adaptation.
  • MemOS: MemOS provides an abstraction layer and Memory API for structured storage, provenance tagging, lifecycle tracking, permissions, and auditable access across users, tasks, and sessions.Its controllability mechanisms schedule memory creation, activation, fusion, and disposal, with scoped memories that can expire or archive.
  • MemOS: MemOS supports plastic restructuring and evolvable transitions among parameter, activation, and plaintext memories.Memories can be sliced, tagged, hierarchically mapped, context-bound, migrated, or transformed across tasks and roles.
  • MemOS: The framework aims to reconstruct memory representation, management, and scheduling for cross-task adaptation, cross-modal evolution, and cross-platform migration.Its broader goal is to support continual evolution by making memory a controllable infrastructure component.

2 Memory in Large Language Models

Research on LLM memory has progressed from defining memory and modeling human-like mechanisms to tool-based operations, but MemOS argues for systematic, full-lifecycle governance. It distinguishes implicit and explicit memory across temporal scales and unifies plaintext, activation, and parameter substrates.

  • Research trajectory: LLM memory research progresses through definition and exploration, human-like memory development, tool-based management, and systematic governance.Earlier tool-based systems largely provide basic insert, delete, and update operations, whereas MemOS targets structured evolution and secure control.
  • Memory taxonomy: LLM memory comprises implicit forms—including parameters, KV-caches, and hidden states—and explicit text- or context-based storage.The framework also organizes memory temporally as sensory, short-term, or long-term, with sensory memory grouped under short-term handling.
  • Implicit memory: Parameter memory provides generalized capabilities but has high update cost, limited interpretability, and reduced flexibility, with retraining risking catastrophic forgetting.These constraints motivate alternatives to relying solely on large-scale parameter updates for memory formation and maintenance.
  • Implicit memory: KV-caches and hidden states act as transient implicit short-term memory by influencing attention, output behavior, and contextual coherence during inference.Existing optimization work emphasizes compute and memory efficiency, while structured and controllable knowledge representation remains underexplored.
  • MemOS governance: MemOS unifies plaintext, activation, and parameter memory through hierarchical organization, integrated retrieval and scheduling, and the MemCube unit.MemCubes organize semantic fragments for query-based aggregation and multi-granularity activation, supporting explicit control and dynamic fusion.
  • MemOS governance: Tool-based memory management remains insufficient because CRUD interfaces do not provide systematic evolution, coordination, access control, or security governance.MemOS addresses this gap with scheduling, layering, API abstraction, permission control, exception handling, and cross-context integration.

3 MemOS Design Philosophy

MemOS frames memory as a first-class system resource and proposes OS-inspired infrastructure for scheduling, abstraction, governance, and evolution. Its Mem-training paradigm shifts continual adaptation toward runtime collection and propagation of explicit memory units across tasks, time horizons, and deployment environments.

  • Motivation: Current LLM architectures lack systematic support for persistent context, retained personalization, and knowledge updates across sessions.Knowledge is rigidly encoded in parameters while context and personalization are not preserved reliably.
  • Vision: Traditional scaling is shifting from pretraining toward post-training, but diminishing returns and growing system complexity motivate continuous memory modeling and dynamic scheduling.The proposed direction targets long-term knowledge accumulation, task adaptation, and behavioral evolution.
  • Mem-training Paradigm: Mem-training enables knowledge to be collected, restructured, and propagated at runtime instead of relying only on sporadic parameter updates.This supports self-adaptation across tasks, time horizons, and deployment environments.
  • Mem-training Paradigm: Continuous interaction extends training beyond static corpora toward dynamic knowledge accumulation, with MemOS supporting memory generation, scheduling, fusion, and updating.The paradigm emphasizes transforming experience into structured memory and repeatedly retrieving and reconstructing it.
  • Core pillars: MemOS treats memory as a first-class, schedulable resource spanning agents, users, applications, and sessions.This design aims to break down memory silos and improve the effectiveness and efficiency of memory access.
  • Core pillars: MemOS makes evolution a core capability through continuous learning, structural reorganization, task transfer, and co-evolution of models and memory.The stated goal is self-adaptation to changing tasks, environments, and feedback.
  • Governance: MemOS adds lifecycle-wide governance through access control, versioning, and provenance auditing to support controllability, traceability, explainability, and compliance.Its OS analogy includes abstraction, unified scheduling, and lifecycle governance for heterogeneous memory resources.

4 Memory Modeling in MemOS

MemOS models memory through three types—plaintext, activation, and parameter memory—and coordinates their evolution with the MemCube abstraction. MemCubes unify heterogeneous content, metadata, scheduling, and lifecycle governance.

  • Memory Types: MemOS defines Plaintext, Activation, and Parameter Memory as a semantic evolution trajectory from perception to consolidation.
  • Plaintext Memory: Plaintext memory is editable, traceable, independently storable knowledge retrieved through external interfaces for updates, customization, and personalization.
  • Plaintext Memory: MemOS integrates plaintext memory into inference and can transform frequently used plaintext into activation paths.
  • Activation Memory: Activation memory stores inference-generated intermediate states, especially KV-cache representations, to support long-range dependencies and reusable inference pathways.
  • Parameter Memory: Parameter memory stores knowledge in model weights, supports implicit activation, and can be modularly enhanced with LoRA or adapters.
  • MemCube: A MemCube combines semantic content with metadata for unified scheduling, lifecycle management, fusion, state transitions, and governance across memory types.Its metadata covers identity, control, behavioral metrics, and modification lineage.

5 Architecture of MemOS

MemOS organizes memory processing across interface, operation, and infrastructure layers. The architecture connects user interaction, memory scheduling, lifecycle control, retrieval, injection, and governance.

  • The MemOS framework spans interface, operation, and infrastructure layers across the memory-processing pipeline.
  • The pipeline moves from semantic parsing and API abstraction to memory scheduling and lifecycle control.
  • The infrastructure layer supports memory injection, retrieval, and governance during model execution.

5.1 Overview: Three-layer Architecture of MemOS

MemOS uses a modular three-layer architecture to invoke, schedule, store, migrate, and govern heterogeneous memory resources. Standard interfaces connect user-facing operations with control and infrastructure services.

  • MemOS comprises Interface, Operation, and Infrastructure layers with distinct responsibilities and collaborative interfaces.
  • Interface Layer: The Interface Layer exposes APIs for querying, writing, updating, transferring, and composing memory units.
  • Interface Layer: MemReader parses requests into structured memory calls while performing permission checks and coordinating compliance validation.
  • Operation Layer: The Operation Layer indexes heterogeneous memories, selects memory types, plans invocation strategies, and tracks lifecycle transitions.
  • Infrastructure Layer: The Infrastructure Layer manages storage, security, migration, access control, retention, auditing, and memory repositories.
  • Together, the layers form a complete operation loop and support multi-model, multi-task, and cross-platform memory sharing.

5.2 Execution Path and Interaction Flow of MemOS

MemOS executes a closed loop from prompt parsing through memory retrieval, scheduling, lifecycle transitions, archiving, and governance. Memory units are selected and managed according to task context, access patterns, and policy.

  • MemOS execution is triggered by user interaction or automated tasks and proceeds through parsing, scheduling, state management, and archiving.
  • Prompt Input and Memory API Packaging: MemReader identifies intent, time scope, entities, and contextual anchors, then packages memory access requirements into a structured MemoryCall.
  • Memory Retrieval and Organization: MemOperator semantically matches memory units and organizes them with task-specific indexes and memory graphs.
  • Memory Scheduling and Activation: MemScheduler uses similarity, access frequency, temporal decay, and priority tags to select memories and compute injection strategies.
  • Lifecycle Modeling and State Transitions: MemLifecycle moves memory through Generated, Activated, Merged, Archived, and Expired states based on access patterns, decay, and task labels.
  • Storage Archiving and Access Governance: Archiving organizes evolved memories by user, task, or context and applies access controls, retention policies, and compliance checks.

5.3 Interface Layer

The interface layer translates natural-language tasks into structured memory calls and exposes composable, governed APIs for memory operations and pipelines.

  • MemReader: MemReader parses prompts into memory features such as intent, temporal scope, entities, memory type, and contextual anchors.It also supports prompt rewriting, coreference resolution, and dialogue memory slot filling.
  • Memory API: The Memory API standardizes memory creation, updates, retrieval, and auditing while supporting provenance metadata and governance.
  • Memory Pipeline: Pipeline composition chains memory actions such as retrieve, augment, update, and archive through shared MemoryCube state and metadata.
  • Memory Pipeline: Compositional memory flows support higher-level cognition patterns, task-specific knowledge shaping, and auditable memory workflows.

5.4 Operation Layer

The operation layer structures, retrieves, transforms, schedules, and evolves memories across plaintext, activation, and parameter forms according to task and resource conditions.

  • MemOperator: MemOperator organizes memories through tags, knowledge-graph links, and hierarchical semantic abstraction, while supporting hybrid retrieval.
  • Hybrid Retrieval: Structured retrieval filters tags, time spans, Boolean conditions, and access policies, while semantic retrieval uses embedding similarity search.
  • MemScheduler: MemScheduler dynamically classifies, transforms, and dispatches Activation, Plaintext, and Parameter Memory based on task semantics and system constraints.
  • Memory Migration: Cross-type migration promotes frequently recalled plaintext to activation memory, distills stable templates into parameters, and archives underused KV entries.
  • Scheduling and Governance: Scheduling integrates task goals, model load, cache hits, access history, governance, and lifecycle policies to determine memory injection and movement.Decisions are logged to MemCube for traceability and interpretability.
  • MemLifecycle: MemLifecycle models memory as a finite-state process spanning Generated, Activated, Merged, and Archived states, with snapshots and rollback support.Lifecycle states also determine caching, archival, cleanup, compression, and migration behavior.

5.5 Infrastructure Layer

The infrastructure layer governs access, stores heterogeneous memories in namespaces, supports backend abstraction and migration, and enables controlled memory exchange.

  • MemGovernance: MemGovernance provides access control, compliance enforcement, and auditability throughout memory sharing, transfer, and inference.
  • Access Control: Permissions combine user identity, memory object, and calling context to enforce private, shared, and read-only access policies.
  • Compliance: Lifecycle enforcement, heat monitoring, garbage collection, archiving, redaction, and access logging protect memory and personal data.
  • MemVault: MemVault organizes memory into dedicated namespaces for private users, expert knowledge, shared industries, contextual pools, and pipeline caches.
  • Backend Integration: A unified MemoryAdapter connects vector stores, relational databases, and blob storage while preserving consistent querying, writing, and synchronization APIs.
  • Runtime Routing: MemVault coordinates with scheduling and lifecycle modules to load, migrate, and route memories according to relevance, access history, and state.
  • Memory Migration: MemLoader and MemDumper transfer structured memory units across platforms, supporting export, injection, synchronization, and knowledge continuity.Governance validates migration policies and isolates sensitive data.
  • MemStore: MemStore enables controlled publishing, subscription, and distribution of memory units across models, institutions, and industry networks.Push and pull exchange can use tags, semantic filters, licensing conditions, expiry policies, and audit traces.

6 Evaluation

Across long-context, personalization, API, and KV-injection evaluations, MemOS generally ranks at or near the top while maintaining context or latency control.

  • LoCoMo: MemOS achieves the best average performance across nearly all LoCoMo task categories and ranks first or second in every category.It also maintains strong F1 performance and reasonable context length control.
  • LongMemEval: MemOS achieves the best overall average performance on LongMemEval and ranks first or second except in knowledge updates.
  • Personalization: MemOS records the best Personalized Response performance in both PreFEval scenarios and the lowest preference-unaware error.It maintains acceptable context length control and low overall error rates.
  • Personalization: MemOS achieves the best precision on PersonaMem while maintaining acceptable context length control.
  • Ablation: Increasing memory capacity steadily improves LoCoMo performance, especially for multi-hop and temporal reasoning tasks.
  • Ablation: Hybrid semantic retrieval and memory-centric design support accurate, fluent, and contextually aligned responses under long-horizon constraints.
  • API Efficiency: MemOS maintains a 100% success rate and reasonable latency even under 100 QPS, with the lowest latency across nearly all reported metrics.
  • KV Acceleration: 91.4% reduction in TTFT is achieved by Qwen2.5-72B under long-context, short-query conditions with KV-based memory injection.Output sequences remain identical between KV-based and direct prompt injection.

7 MemOS for Architecture Innovation and Applications

MemOS treats memory as a first-class, composable system resource with standardized interfaces, governance, and lifecycle control. Its architecture supports memory reuse across tasks and faster KV-based injection without changing output semantics.

  • Architecture and Interfaces: MemOS treats memory as a first-class system resource, enabling lifecycle management and orchestration across multiple memory forms.This abstraction supports modular and reusable knowledge assets.
  • Architecture and Interfaces: Memory becomes a composable, user-controllable intelligence unit that can be uploaded, mounted, and invoked independently of training workflows.MemOS exposes memory through modularized and assetized interfaces.
  • Memory-as-a-Service: MemStore enables experts to publish structured memories that consumers can install through a standardized interface subject to permission controls.MemGovernance provides access conditions for published memory assets.
  • Efficiency: KV-based memory injection consistently achieves lower Time to First Token latency than direct prompt-based injection while preserving output semantics.The comparison spans models, context lengths, and query lengths.
  • Architecture and Interfaces: Standardized interfaces and persistence formats let runtime tasks read, write, mount, fuse, or replace memory blocks on demand.Task-level Memory API calls hide vector indexing, KV caching, and context orchestration from users and developers.
  • Task Adaptation: MemOS dynamically loads relevant memory sets across task phases and performs hot-swapping and cache eviction as contexts evolve.The legal-assistant scenario uses distinct memories for templates, case logs, and current regulations.

7.2 MemOS Application Scenarios

MemOS applies managed memory to dialogue continuity, evolving knowledge, personalization, role separation, and cross-platform reuse. It extracts structured memory, schedules relevant historical information, governs updates, and supports portable identity-aware memory.

  • Dialogue Continuity: MemOS extracts salient budgets, preferences, and constraints into structured conversation memory units linked to a task’s long-term memory path.MemLink connects these units to the ongoing task.
  • Dialogue Continuity: MemScheduler retrieves relevant historical fragments during inference and integrates them into the active reasoning path.The stated goal is continuity of semantic state despite context sliding.
  • Knowledge Evolution: MemOS models knowledge as lifecycle-governed memory with stages for generation, replacement, fusion, and deprecation.Updates are scheduled using usage frequency, contextual alignment, and semantic overlap.
  • Knowledge Evolution: For updated clinical guidelines, MemOS tags trusted sources, compares versions, suggests updates, prioritizes active versions, and archives obsolete entries.The described workflow avoids retraining while preserving prior knowledge structures.
  • Personalization and Roles: Each user identity receives dedicated memory spaces, allowing multiple roles to coexist under one account.The system dynamically loads the appropriate persona during inference.
  • Personalization and Roles: MemOS encodes interaction patterns into personal memory units capturing language tone, response preferences, and value leanings.These units are incorporated into inference for personalized behavior.
  • Portability: Standardized representations, encryption, and platform-agnostic mount protocols make memory blocks portable across mobile, cloud, and enterprise environments.This design addresses memory silos that fragment cross-device and cross-agent continuity.

8 Conclusion

MemOS is proposed as a memory operating system for heterogeneous LLM memory, centered on MemCube units and system modules for scheduling, lifecycle management, storage, and augmentation. The conclusion frames future work around interoperable, self-evolving, and decentralized memory infrastructure.

  • Core Contribution: MemOS introduces a memory operating system intended to provide foundational memory infrastructure for next-generation LLM applications.The proposal targets collaborative construction of this infrastructure.
  • Core Contribution: MemOS unifies parameter, activation, and explicit plaintext memories through MemCube and modules for scheduling, lifecycle management, structured storage, and transparent augmentation.The paper states that these components enhance reasoning coherence, adaptability, and scalability.
  • Future Directions: The paper envisions a decentralized memory marketplace built around modular memory resources for continual learning and long-term evolution.This is presented as a future intelligent ecosystem.
  • Future Directions: Planned cross-LLM memory sharing would support interoperability and module reuse across different foundation models.The proposed extension of MIP would define formats, compatibility rules, and trust mechanisms.
  • Future Directions: Planned self-evolving MemBlocks would optimize, reconstruct, and evolve from usage feedback, reducing manual maintenance and supervision.This is listed as a future direction rather than a demonstrated result.
  • Conclusion: The conclusion presents MemOS as a route from closed, static generation systems toward continuously evolving agents with long-term memory and behavioral plasticity.It also frames memory as a first-class computational resource for cross-task, cross-platform, and multi-agent intelligence.
Loading 2507.03724v4…