Source-linked AI summary
From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs
Yaxiong Wu, Sheng Liang, Chen Zhang, Yichao Wang, Yongyue Zhang, Huifeng Guo, Ruiming Tang, Yong Liu
TL;DR
Existing work lacks a systematic account of how LLM-driven AI memory relates to human memory and how human memory can inspire stronger AI systems. The paper surveys these relationships, organizes memory research through three dimensions and eight quadrants, and discusses open problems and future directions. It covers personal and system memory mechanisms, their applications, and constraints affecting long-term personal-memory scalability and privacy.
Problem
Existing reviews mainly analyze AI memory by time, leaving its relationships with human memory and its object and form dimensions insufficiently systematized.
Method
The paper analyzes human memory categories, maps them to LLM-driven AI memory, and organizes prior work using object, form, and time dimensions with eight quadrants.
Results
The survey systematically covers personal and system memory research, including personalization, reasoning, planning, memory management, and KV-cache management.
Takeaways & Limitations
Human memory provides a basis for designing more efficient and flexible AI memory architectures, while the taxonomy structures analysis of current systems and future development.
Takeaways & Limitations
Parametric long-term personal memory requires fine-tuning on individual user data and substantial computational resources, limiting scalability and practical deployment.
Abstract
from arXiv · showhide
Memory is the process of encoding, storing, and retrieving information, allowing humans to retain experiences, knowledge, skills, and facts over time, and serving as the foundation for growth and effective interaction with the world. It plays a crucial role in shaping our identity, making decisions, learning from past experiences, building relationships, and adapting to changes. In the era of large language models (LLMs), memory refers to the ability of an AI system to retain, recall, and use information from past interactions to improve future responses and interactions. Although previous research and reviews have provided detailed descriptions of memory mechanisms, there is still a lack of a systematic review that summarizes and analyzes the relationship between the memory of LLM-driven AI systems and human memory, as well as how we can be inspired by human memory to construct more powerful memory systems. To achieve this, in this paper, we propose a comprehensive survey on the memory of LLM-driven AI systems. In particular, we first conduct a detailed analysis of the categories of human memory and relate them to the memory of AI systems. Second, we systematically organize existing memory-related work and propose a categorization method based on three dimensions (object, form, and time) and eight quadrants. Finally, we illustrate some open problems regarding the memory of current AI systems and outline possible future directions for memory in the era of large language models.
1 Introduction
The paper frames memory as central to LLM-driven AI systems and identifies a gap in reviews that focus mainly on temporal categories. It surveys human–AI memory correspondences, introduces a three-dimensional eight-quadrant classification, and examines open issues and future directions.
- Human Memory: Human memory is broadly divided into short-term and long-term systems, encompassing sensory, working, explicit, and implicit memory.Short-term memory lasts seconds to minutes, whereas long-term memory can persist from minutes to years.
- Motivation: LLM-driven AI systems use memory to retain, retrieve, and apply information from past interactions, helping agents overcome context-window limits and make better decisions.Memory-enhanced agents can decompose complex tasks, remember interaction history, and invoke tools.
- Research Gap: Existing reviews largely classify AI memory by time, leaving object and form dimensions insufficiently addressed.The paper highlights the need to distinguish memories by what they concern and how they are represented or stored.
- Contributions: The survey relates human memory categories to personal and system memory, parametric and non-parametric memory, and short-term and long-term AI memory.It organizes prior work across these correspondences and examines both personalization and complex-task capabilities.
- Contributions: The paper analyzes open issues in current AI memory and outlines potential future directions for memory development in the LLM era.Its contributions include identifying challenges and directions alongside the proposed classification and literature synthesis.
2 Overview
The paper reviews human memory as interacting systems and processes that support temporary processing, durable storage, retrieval, updating, and reflection. It uses these human-memory parallels to motivate a structured AI-memory taxonomy based on three dimensions and eight quadrants.
- Short-Term and Long-Term Memory: Working memory temporarily holds and manipulates information for ongoing tasks, whereas episodic memory preserves personal experiences and events over long periods.Sensory information may be transferred into working memory and later into long-term memory.
- Short-Term and Long-Term Memory: Human memory includes short-term and long-term systems that operate together to retain, process, and use information for different purposes.Short-term memory includes sensory and working memory, while long-term memory includes explicit and implicit memory.
- Memory Mechanisms: Multiple memory systems interact across brain regions, including regions associated with working memory, explicit memory, semantic memory, and procedural memory.The prefrontal cortex, hippocampus, cerebral cortex, and cerebellum are linked to distinct memory functions.
- Memory Mechanisms: Human memory relies on encoding, storage, and retrieval, with consolidation, reconsolidation, reflection, and forgetting further shaping memory over time.These processes support stabilization, updating, evaluation, and filtering of stored information.
- AI Memory Taxonomy: The proposed 3D-8Q taxonomy classifies AI memory by object, form, and time, yielding eight quadrants organized by function, storage mechanism, and retention duration.The dimensions distinguish personal/system, non-parametric/parametric, and short-term/long-term memory.
3 Personal Memory
Personal memory stores and uses user interaction data to improve personalization through non-parametric retrieval and parametric memory mechanisms. Its lifecycle includes construction, management, retrieval, and application across personalized systems.
- Personal memory stores and utilizes human input and response data from interactions with LLM-driven AI systems.It is examined through non-parametric and parametric construction and implementation approaches.
- Non-Parametric Personal Memory: Non-parametric personal memory separates current-session dialogue from historical cross-session memories, which supplement context and recover missing information.Long-term memory can include behavioral history, preferences, and interaction records.
- Construction and Management: Personal-memory construction extracts and refines raw interaction data, while management performs deduplication, merging, and conflict resolution.These stages are compared with human memory consolidation, reconsolidation, and reflection.
- Retrieval: Retrieval methods depend on storage format, including SQL for key-value memory, graph-based recall, and dense vector retrieval indexed with FAISS.Different representations support different retrieval procedures for identifying memories relevant to a user’s request.
- Usage and Evaluation: Personal memory enhances individualized recommendations and responses and supports applications including software development, social-network simulation, and financial trading.Benchmarks cover long-term conversation, everyday life, proactive dialogue, multimodal dialogue, egocentric video, and long-context reasoning.
- Parametric Personal Memory: Parametric short-term personal memory caches intermediate attention states to accelerate inference, whereas parametric long-term memory encodes personal data into model parameters.Long-term parameterization requires fine-tuning on individual user data, creating substantial computational demands that hinder scalability and deployment.
4 System Memory
System memory comprises intermediate representations and results generated during task execution, supporting reasoning, planning, efficiency, and continual improvement. The survey distinguishes non-parametric and parametric system memory across short- and long-term forms.
- System memory consists of intermediate representations or results generated throughout task execution and supports reasoning, planning, self-evolution, and continual improvement.The section examines system memory from non-parametric and parametric perspectives.
- Non-Parametric System Memory: Non-parametric short-term system memory records reasoning and action results for the current task, while long-term memory consolidates successful experiences and self-reflection from historical interactions.Short-term memory supports current-task execution; long-term memory supports continual evolution and adaptive enhancement.
- Reasoning and Planning Enhancement: ReAct alternates intermediate reasoning steps with actions, while Reflexion uses dynamic memory and self-reflection to refine behavior based on prior errors or limitations.These mechanisms provide references for subsequent execution and support adaptive decision-making in complex problem solving.
- Experience Consolidation: Long-term non-parametric system memory reflects on and refines accumulated short-term traces, retaining effective strategies and extracting lessons from failures.Continuous updating and optimization are intended to reduce repeated errors.
- Parametric System Memory: Parametric short-term system memory stores inference-time knowledge such as KV Cache, whereas parametric long-term memory edits and stores knowledge in model parameters.These forms correspond respectively to human working memory and semantic memory.
- KV Management and Reuse: KV management and reuse address inference cost and latency through organization, compression, quantization, token-level KV Cache, and sentence-level Prompt Cache.PagedAttention enables flexible sharing and near-zero KV cache waste, while other methods reduce memory and computational costs.
- Personal–System Distinction: Parametric short-term personal memory processes individual input data, whereas parametric short-term system memory optimizes system-level context storage and reuse during task execution.The approaches overlap technically but differ in focus.
- Long-Term Parametric Memory: LLMs can function as long-term parametric memory by storing and integrating information over extended periods, while knowledge-editing systems support continual updates.WISE uses separate main and side memories with routing and knowledge sharding for edited information.
5 Open Problems and Future Directions
The paper identifies open directions for LLM memory across multimodal integration, temporal updating, comprehensive architectures, shared memory, collective privacy, and automated evolution. These directions aim to address current systems’ narrow scope and improve adaptability, collaboration, and real-world applicability.
- From Unimodal Memory to Multimodal Memory: Multimodal memory should integrate text, images, audio, video, and sensor data into a unified understanding.The paper connects this direction to stronger perception, complex real-world tasks, and more personalized interactions.
- From Static Memory to Stream Memory: Stream memory continuously updates information and responds to evolving contexts, complementing static memory’s stable long-term knowledge.The paper presents static and stream memory as complementary batch-processing and real-time paradigms.
- From Specific Memory to Comprehensive Memory: Future architectures should move beyond narrow task-specific components toward multilayered, multidimensional, and adaptive memory systems.The motivation draws on human memory’s interconnected subsystems, including sensory, working, explicit, and implicit memory.
- From Exclusive Memory to Shared Memory: Shared memory could connect domain-specific models, enabling cross-domain knowledge transfer and cooperative task solving.The paper describes shared memory as a way for multiple LLMs to access one another’s expertise across domains.
- From Individual Privacy to Collective Privacy: Interconnected AI memory creates a collective-privacy challenge because group-level data use can enable profiling, misuse, or excessive surveillance.The paper identifies balancing data utility and privacy preservation as a central requirement.
- From Rule-Based Evolution to Automated Evolution: Automated evolution is proposed as a direction beyond manually crafted rules and heuristic adjustments for self-reflection.The supplied passage frames rule-based evolution as limiting flexibility, scalability, and efficiency.
6 Conclusion
The conclusion presents memory as central to LLM-based AI advancement and surveys how human cognition can inform AI memory design. It introduces an eight-quadrant framework across object, form, and time, reviews current developments, and identifies future challenges.
- 6 Conclusion: The paper compares human memory categories with existing AI memory models and proposes an eight-quadrant framework based on object, form, and time.The framework is intended as a theoretical foundation for multi-level and comprehensive memory systems.
- 6 Conclusion: The survey reviews AI memory from personal-memory and system-memory perspectives while outlining open challenges and future research directions.The conclusion covers contemporary memory design in the LLM era.