Memory for Large Language Models

Sining Zhoubian; Dan Zhang; Evgeny Kharlamov; Jie Tang

Source-linked AI summary

Memory for Large Language Models

Sining Zhoubian, Dan Zhang, Evgeny Kharlamov, Jie Tang

arXiv:2607.25380v1Jul 28, 2026cs.CL

TL;DR

Memory mechanisms in LLMs remain fragmented across heterogeneous operational categories and lack a unified abstraction. This survey organizes them by representation, update dynamics, and persistence, finding that complementary implicit and explicit substrates motivate a shift toward hybrid architectures.

Problem
Memory mechanisms in LLMs are categorized operationally without a unifying abstraction that distinguishes their persistence, accessibility, update dynamics, and representation.
Method
The survey develops a mechanism-centric taxonomy of model-level memory along representation, update dynamics, and persistence, formalizing memory writing, routing, state transitions, and consolidation.
Results
The analysis finds that implicit memory is computation-coupled, explicit memory provides autonomous persistent storage, and hybrid architectures orchestrate these complementary substrates.
Takeaways & Limitations
LLM memory design is becoming an architectural axis in which hybrid systems combine complementary mechanisms while balancing recall fidelity, scalability, interference, persistence, and update complexity.
Takeaways & Limitations
Implicit memory remains limited in persistence across documents, tasks, or interactions without additional storage mechanisms.

Abstract

from arXiv · show

Memory has evolved into a foundational architectural dimension in large language models (LLMs), shifting from an implicit byproduct of computation to a spectrum of explicit, controllable mechanisms. While recent advances introduce diverse strategies---spanning transient attention, recurrent state dynamics, parameter-efficient adaptations, and scalable lookup storage---this rapid evolution has led to a highly fragmented research landscape. In this survey, we present a systematic, architecture-centric taxonomy of memory in LLMs. Our framework characterizes memory along three orthogonal axes: representation (implicit versus explicit), update dynamics (offline versus online), and persistence (short-term versus long-term). We further formalize the granular mechanisms dictating memory writing, routing, state transitions, and consolidation. This unified perspective elucidates the conceptual boundaries between computation-coupled and independently addressable memory, effectively bridging disparate architectural paradigms. Additionally, we critically analyze hybrid memory architectures, system-level efficiency trade-offs, and multi-dimensional evaluation methodologies. By consolidating these scattered advancements into a cohesive framework, this survey charts the trajectory of memory-centric LLM design and provides a principled foundation for future innovations in scalable and adaptive language modeling.

I. INTRODUCTION · II. A TAXONOMY OF MEMORY IN LARGE LANGUAGE MODELS · A. Representation: Implicit vs. Explicit Memory

The survey frames memory as a central architectural dimension in LLMs and organizes model-level mechanisms through a taxonomy of representation, persistence, and update dynamics. It specifically distinguishes implicit, computation-coupled memory from explicit memory with independent storage and access semantics.

I. INTRODUCTION: LLM memory spans transient attention, recurrent states, parameter updates, and persistent lookup storage, motivating memory as an explicit architectural dimension beyond scaling parameters, data, and compute.The survey presents this evolution as a shift from memory being an implicit computational byproduct toward diverse explicit mechanisms.
I. INTRODUCTION: Existing studies examine attention caches, recurrent states, test-time adaptation, retrieval, and conditional routing in isolation despite addressing shared questions about storage, updating, computation, and retention.The survey identifies the lack of a unified perspective as a central problem in the fragmented literature.
II. A TAXONOMY OF MEMORY IN LARGE LANGUAGE MODELS: The survey proposes an architecture-centric taxonomy with three orthogonal axes: representation, persistence, and update dynamics.Representation distinguishes implicit computational memory from explicit storage; persistence captures lifespan; update dynamics describes when and how representations evolve.
I. INTRODUCTION: The survey reviews computation-driven and parameterized systems, hybrid architectures, multi-timescale updates, conditional routing, system trade-offs, and multidimensional evaluation methods.It positions these analyses as contributions intended to clarify open challenges and future directions in memory-centric modeling.
II. A TAXONOMY OF MEMORY IN LARGE LANGUAGE MODELS: Its scope is model-level memory instantiated within model architecture or inference-time dynamics, excluding agent-level and prompt-based systems dependent on external orchestration.The scope focuses on mechanisms that evolve over time or actively store and retrieve contextual or experiential information beyond fixed base parameters.
A. Representation: Implicit vs. Explicit Memory: Implicit memory is stored as a byproduct of forward computation and accessed through the computation graph without an explicit read, write, or lookup interface.Examples include recurrent sequence states in RNNs, linear attention, and SSMs, alongside Transformer KV caches.
A. Representation: Implicit vs. Explicit Memory: Explicit memory is a distinct storage component with defined addressing, retrieval, and update operations that are not reducible to standard forward computation or offline backpropagation.It can comprise external tables, associative buffers, or dedicated memory modules integrated into the model.
A. Representation: Implicit vs. Explicit Memory: The representation axis places attention-based context accumulation, retrieval augmentation, and inference-time adaptation in one conceptual space without conflating architectural form with functional role.The broader taxonomy also distinguishes offline versus online updates and short-term versus long-term persistence.

B. Update Dynamics: Offline vs. Online Memory

Update dynamics distinguish offline memory, fixed after training, from online memory, which can incorporate information during inference. A finer-grained view further characterizes how memory is modified through diverse, potentially combined update rules.

Update Dynamics: The offline/online axis classifies memory by whether updates occur only during training or remain available during inference.This axis describes when memory can be modified, not the specific update mechanism.
Offline Memory: Offline memory is updated during training, typically through gradient-based optimization, and remains fixed during inference.Parameterized retrieval embeddings and end-to-end-trained memory-augmented layers are examples.
Online Memory: Online memory updates during inference at per-token, per-segment, or per-batch frequencies, using rules distinct from standard backpropagation.It supports on-the-fly incorporation of new information, adaptation to nonstationary inputs, and long-horizon dependencies.
Fine-Grained Update Rules: Fine-grained update rules distinguish mechanisms such as closed-form recurrent transitions, gradient-optimized fast weights, and threshold-gated KV updates.These rules clarify memory writing, retention, and forgetting, and may be combined within hybrid architectures.

C. Persistence: Short-Term vs. Long-Term Memory

Persistence measures the effective temporal horizon over which stored information influences computation, shaped by retention, storage granularity, and compression. Short-term memory supports precise local recall, whereas long-term memory extends influence through compression, selective retention, or persistence beyond local windows.

Persistence: Short-Term vs. Long-Term Memory: Persistence depends on effective temporal horizon, storage granularity, and compression rate—not physical retention alone.A large, high-fidelity store may remain short-horizon, while a compact recurrent or parameterized state may influence computation longer.
Short-Term Memory: Short-term memory stores fine-grained information within local contexts, segments, or inference episodes, with attention KV caches preserving token-level states at low compression.KV-cache storage grows with retained tokens, limiting indefinite scalability while supporting precise local recall.
Long-Term Memory: Long-term memory extends influence across long contexts or sessions through compressed recurrent states, explicit stores, memory slots, or test-time updated parameters.Its defining property is an extended effective horizon, often enabled by higher compression, selective retention, or persistent storage rather than storage size alone.
Interplay of Axes: Representation, update dynamics, and persistence are largely orthogonal, so explicit or online memories can exhibit either short-term or long-term persistence.Update and eviction policies help determine whether online memory remains short-term or becomes long-term.

D. Positioning Relative to Memory-Centered Surveys · III. IMPLICIT MEMORY VIA COMPUTATION DYNAMICS · A. Attention as Implicit Memory

The survey positions itself as a model-level architectural account that complements system- and cognition-centered memory surveys. It then treats implicit memory as computation-coupled, transient information processing, with attention providing content-addressable access but limited persistence and practical capacity.

D. Positioning Relative to Memory-Centered Surveys: Memory-centered surveys operate at different system levels, so their taxonomies are complementary rather than directly interchangeable.Table III compares representative surveys by primary scope and organizing lens rather than every topic covered.
D. Positioning Relative to Memory-Centered Surveys: This survey centers model-level architectural mechanisms and includes external or agent-system memory only when storage, access, or updates are tightly coupled to the model.Examples include conversational records, user profiles, reflection traces, tool outputs, and database-backed stores.
III. IMPLICIT MEMORY VIA COMPUTATION DYNAMICS: Implicit memory arises through transient activations, attention interactions, and evolving hidden states during forward execution rather than independently persistent parameters or external repositories.Such memory is tightly coupled to the computational graph and typically discarded when inference terminates.
III. IMPLICIT MEMORY VIA COMPUTATION DYNAMICS: The implicit-memory section also examines contextual retention and pruning under long-sequence or streaming conditions, recurrent sequence memory, and shared structural limitations.Recurrent sequence memory compresses history into structured dynamical states.
A. Attention as Implicit Memory: Self-attention is the canonical implicit-memory mechanism, retaining past-token representations as key–value pairs that incoming queries retrieve through content-based interaction.It realizes memory through forward computation without a separate storage module.
A. Attention as Implicit Memory: Attention memory capacity is tied to context length, but long-context models may underuse their nominal capacity because of optimization and inductive biases.Extending context increases available short-term memory, while approaches such as MLA reduce per-token storage through latent key–value compression.
A. Attention as Implicit Memory: StreamingLLM combines sliding-window attention with preserved attention-sink tokens to support stable generation over unbounded streams without retraining, implementing implicit eviction.This approach bounds memory while retaining selected information during streaming inference.
A. Attention as Implicit Memory: Attention-based implicit memory supports online updates and flexible content-based access, but remains short-term, architecturally bounded, and difficult to control or persist across extended horizons.Alternative formulations such as HyperMLP show that memory-access geometry and sequence mixing can be generalized within attention-style implicit memory.

B. Sparse, Selective, and Structured Memory · C. Recurrent Sequence Memory

The section frames sparse and recurrent mechanisms as forms of implicit memory that control access, retention, or state transitions within forward computation. These designs improve efficiency and support online processing, but compressed recurrent states limit flexible access and long-horizon fidelity.

B. Sparse, Selective, and Structured Memory: Sparse, selective, and structured attention controls which information is admitted, accessed, and retained within implicit memory.These controls remain embedded in attention patterns and internal dynamics rather than exposed as explicit memory operations.
B. Sparse, Selective, and Structured Memory: Sparse Transformers, BigBird, and Longformer restrict attention to local windows, strided patterns, or global tokens, reducing cost while preserving long-context performance.Their structured subsets make full attention over all past tokens unnecessary.
B. Sparse, Selective, and Structured Memory: MoBA and Native Sparse Attention replace fixed access patterns with adaptive routing over coarse memory blocks, summaries, retained fine-grained blocks, and local windows.Both remain attention-based implicit memory and primarily provide adaptive access control rather than persistent storage.
B. Sparse, Selective, and Structured Memory: Selective attention uses data-dependent gating or routing to decide which contextual information influences computation, while local–global designs balance recent detail with broader summaries.These mechanisms function as implicit admission and retention controls.
C. Recurrent Sequence Memory: Recurrent sequence memory compresses history into an evolving latent state, spanning RNNs, linear-attention recurrences, and structured state space models.RNNs learn nonlinear transitions, whereas SSMs derive structured recurrences that support efficient scans and controlled long-range dynamics.
C. Recurrent Sequence Memory: Hierarchical and hybrid variants allocate recurrent capacity across resolutions or combine recurrent compression with attention, chunkwise computation, or multiple routed states.Log-Linear Attention grows capacity logarithmically with sequence length, while RetNet, Jamba, and MoM balance state propagation with selective access or allocation.
C. Recurrent Sequence Memory: Recurrent memory enables online processing of unbounded streams with bounded storage, but computation-coupled states limit flexible read/write control and may lose long-horizon detail.Next-Latent Prediction shapes hidden representations through an offline objective rather than a new test-time write rule.

D. Limitations of Implicit Memory

Implicit memory is effective and efficient but remains constrained by its tight coupling to model computation. Its bounded capacity, inflexible access semantics, and limited cross-context persistence motivate explicit memory mechanisms.

D. Limitations of Implicit Memory: Implicit memory mechanisms are architecturally limited because they remain tightly coupled to model computation across attention-based and state-based designs.These limitations arise consistently from computation-coupled architectures.
D. Limitations of Implicit Memory: Implicit memory has bounded capacity, and compression trades memory span against representational fidelity, hindering selective preservation of fine-grained information over long horizons.Attention-based capacity depends on context-window and KV-cache resources constrained by complexity and hardware limits.
D. Limitations of Implicit Memory: Implicit memory lacks explicit, adaptable read/write operations because storage and retrieval are dictated by attention weights or state transitions.This computation-graph dependence can make implicit memory insufficiently adaptive to input.
D. Limitations of Implicit Memory: Implicit memory typically persists only across limited forward passes or inference sessions, while recurrent extensions mainly improve retention within longer sequences.Cached state snapshots and mixtures of recurrent states do not provide fully persistent memory across sessions.
D. Limitations of Implicit Memory: These limitations motivate memory systems that decouple storage from internal computation and expose broader explicit semantics for capacity control, persistence, and selective access.Such mechanisms form the basis of explicit memory systems examined in the next section.

IV. EXPLICIT MEMORY VIA ADDRESSABLE AND ADAPTIVE STORAGE · A. Parameterized External Memory Modules · D. Multi-Timescale and Nested Update

The paper defines explicit memory as model-integrated, independently stored structures that persist beyond transient computation and support controllable access or updates. It organizes these mechanisms around parameterized memory, addressable storage, conditional activation, and update dynamics spanning multiple timescales.

IV. EXPLICIT MEMORY VIA ADDRESSABLE AND ADAPTIVE STORAGE: Explicit memory decouples storage from transient forward states through parameterized modules, key–value repositories, or dedicated components with controllable access and persistence.It is distinct from prompt-level memory engineering, retrieval-augmented generation pipelines, and external agent frameworks.
IV. EXPLICIT MEMORY VIA ADDRESSABLE AND ADAPTIVE STORAGE: The taxonomy covers parameterized modules, lookup-based storage, conditional parameter memory such as MoE, and mechanisms organized by distinct temporal scales.These perspectives cover dedicated memory parameters, persistent addressable repositories, sparse expert activation, and related model-level update rules.
A. Parameterized External Memory Modules: Parameterized external memory uses dedicated memory parameters that persist across inference instances and can be selectively updated without modifying the pretrained backbone.This separation enables controlled online plasticity while preserving pretrained knowledge.
A. Parameterized External Memory Modules: Fast-weight systems place recent information in rapidly updated parameters while slower weights retain long-term knowledge, extending explicit memory through memory matrices and meta-learned mechanisms.MANNs and DNCs further develop this paradigm with explicit memory matrices.
D. Multi-Timescale and Nested Update: Test-time methods such as Titans, TTT, and TTT-E2E update auxiliary memory parameters online, while In-Place TTT reuses Transformer matrices as fast-weight memory.These approaches modify designated parameter subsets through optimization-based writing rather than standard pretraining.
D. Multi-Timescale and Nested Update: Gated Differentiable Working Memory selectively writes chunks according to contextual utility, addressing the problem of deciding which observations merit persistent or semi-persistent storage.LM2 instead provides dedicated memory slots accessed through cross-attention-like interactions and gating.
D. Multi-Timescale and Nested Update: Decoupling contextual memory parameters from general-knowledge parameters reduces catastrophic interference but introduces trade-offs involving memory capacity and learning dynamics.Across episodes, persistent learnable state provides a controllable balance between stability and plasticity, although parameterized memory can still participate in token-level computation.

B. Lookup-Based and Retrieval-Oriented Memory

Lookup-based memory provides persistent, independently addressable storage accessed through content-based retrieval, distinguishing it from computation-coupled attention and fixed hidden-state representations. The section covers datastore augmentation, model-integrated lookup, editable memory, and the architectural boundary between embedded memory and external RAG systems.

Core properties: Lookup-based memory stores independently addressable entries that persist across inference episodes and are retrieved by content rather than recomputed through attention.Many stores are constructed offline, while some architectures admit newly observed representations online.
Nearest-Neighbor and Datastore-Augmented Models: kNN-LM augments pretrained language models with a persistent datastore of context–target pairs, externalizing memory outside the standard forward computation graph.The datastore supports content-based retrieval and can improve perplexity and rare-token prediction, as described in the passage.
Boundary with Activation-Level Retrieval: Memorizing Transformers and LongMem retrieve stored key–value representations from nondifferentiable or cache-based memory banks, placing them near the boundary between model-level retrieval and activation-level retrieval.LongMem uses a SideNet to retrieve and fuse cached representations with the current context.
Conditional and Scalable Lookup Memory: Engram integrates conditional lookup into the model using sparsely activated, persistently stored slots retrieved through structured hashing and conditioned on the current hidden state.This approach selectively looks up memory instead of recomputing historical context through full attention.
Editable and Feed-Forward Lookup Memory: Editable and model-integrated lookup mechanisms include PlugLM’s updatable differentiable key–value memory and MemoryLLM’s precomputed token-level retrieval from decoupled feed-forward memories.PlugLM updates memory without full model retraining, while MemoryLLM reinterprets feed-forward networks as context-free token-wise retrieval modules.

C. Conditional Parameter Memory and Mixture-of-Experts · D. Multi-Timescale and Nested Update Mechanisms · E. Structural Implications and Risks

The paper frames conditional parameter memory, multi-timescale updates, and explicit persistent storage as architectural mechanisms that improve modular access and temporal adaptability. These benefits introduce trade-offs involving stability, scalability, optimization, lifecycle management, and interpretability.

C. Conditional Parameter Memory and Mixture-of-Experts: MoE models provide offline explicit memory through specialized parameterized experts selected by input-conditioned sparse routing.Each expert acts as a persistent memory block, while the router performs context-dependent addressing over modular knowledge storage.
C. Conditional Parameter Memory and Mixture-of-Experts: Sparse expert activation scales capacity while keeping per-token FLOPs nearly constant, with Mixtral and DeepSeek-MoE emphasizing specialization and efficiency.Mixtral uses top-2 routing over eight experts per layer; DeepSeek-MoE uses fine-grained partitioning and load balancing.
D. Multi-Timescale and Nested Update Mechanisms: Explicit memory parameters can evolve at multiple temporal frequencies because their update schedules are structurally distinct from backbone training and inference.Test-time mechanisms update memory during inference, while nested learning organizes parameter subsets into hierarchically differentiated schedules.
D. Multi-Timescale and Nested Update Mechanisms: Fast–slow decompositions use rapidly updated parameters for recent patterns and slowly updated weights for stable knowledge, limiting high-frequency updates to reduce catastrophic interference.The memory parameters are explicitly designated for temporal adaptation rather than arising incidentally from computation.
D. Multi-Timescale and Nested Update Mechanisms: Multi-timescale memory exposes a stability–plasticity trade-off: frequent updates improve adaptability but risk transient-signal overfitting, whereas slower updates preserve stability but reduce responsiveness.Explicit online memory must regulate update magnitude, frequency, and scope.
E. Structural Implications and Risks: Explicit persistent memory expands capacity beyond hidden-state dimensionality and context windows, but increasing slots, parameters, or datastore entries raises computational and storage costs.Lookup systems can grow as entries accumulate, while parameterized modules can expand for richer representations, potentially with diminishing returns.
E. Structural Implications and Risks: Online persistence creates risks of interference, drift, stale retrieval, biased updates, and difficult convergence across coupled learning processes operating at multiple temporal scales.Rapid test-time adaptation may destabilize representations, while additional objectives, update loops, and nested hierarchies complicate optimization analysis.
E. Structural Implications and Risks: Persistent storage also requires lifecycle and interpretability mechanisms, including reset policies, temporal-storage separation, component attribution, and analysis of update propagation.Compared with implicit memory, explicit memory removes computation-bound constraints but adds structural and training complexity.

V. MODEL-LEVEL MEMORY ARCHITECTURES: DESIGN, IMPLEMENTATION, AND EVALUATION · A. Hybrid Memory Architectures

Hybrid memory architectures combine complementary mechanisms within unified, end-to-end differentiable models to improve efficiency, scalability, recall, and long-term capability. They span interleaved attention–state layers, adaptive routing, explicit storage, multiple update timescales, and modular components while balancing recall, context, computational cost, and reuse.

V. MODEL-LEVEL MEMORY ARCHITECTURES: DESIGN, IMPLEMENTATION, AND EVALUATION: Standard Transformers rely on attention-based working memory, but integrating multiple memory mechanisms may improve practical systems’ efficiency, scalability, and long-term capability.This motivates diversifying beyond a comparatively simple attention-centered paradigm.
A. Hybrid Memory Architectures: Architectural hybridization combines distinct memory paradigms within one model to leverage complementary strengths without relying on external systems.This differs from hybridization among implicit memory mechanisms within a single representation.
A. Hybrid Memory Architectures: Interleaving selective state-space layers with sliding-window attention enables efficient ultra-long-context processing while preserving high-resolution recall for recent tokens.Samba exemplifies this attention–state pattern and lowers migration costs without requiring full pretraining from scratch.
A. Hybrid Memory Architectures: Hybrid models combine attention and recurrent or state-space memory across different depths or within the same layer, rather than using only a fixed layer ratio.Examples include OLMo Hybrid, Kimi Linear, Hymba, and Falcon-H1.
A. Hybrid Memory Architectures: Signal-gated architectures make memory access updates adaptive: AMOR gates attention refinement, whereas HAM gates token admission into the KV cache.Fixed serial and hybrid-head models instead use scheduled or structural allocation of high-resolution attention memory.
A. Hybrid Memory Architectures: Hybrid systems combine dense Transformer computation with explicit storage, enabling selective recall without uniformly increasing computational cost.Titans adds a test-time learnable memory that updates parameters during inference, while Engram-inspired systems route tokens through sparse key-based memory slots.
A. Hybrid Memory Architectures: Different update timescales create memory hierarchies: TTT-E2E performs batch-style inference updates, while Nested Learning uses nested optimization loops at distinct frequencies.These mechanisms allow activation states and other memory components to evolve at different granularities.
A. Hybrid Memory Architectures: Modular designs combine structured state-space backbones, sparse global attention, MoE routing, and separate workspace and factual memory mechanisms.Hybrid systems trade high-resolution local recall against compressed global context, while structured or streaming layers can provide linear rather than quadratic complexity and modular components enable selective specialization.

B. Memory Management and Efficiency

Efficient memory management is an architectural requirement for long-context LLM deployment, addressing attention-cache growth, bandwidth bottlenecks, fragmentation, and computational cost. The section organizes solutions around KV-cache compression, memory virtualization, working-memory consolidation, and structured sparsity.

KV Cache Compression and Quantization: KV-cache compression and quantization reduce storage and inter-device communication costs while aiming to preserve retrieval fidelity.The KV cache grows linearly with sequence length and often becomes the dominant memory bottleneck during long-context inference.
Memory Virtualization and Paged Attention: PagedAttention virtualizes KV-cache storage into pageable blocks, enabling allocation and reuse across variable-length sequences while reducing fragmentation.It decouples logical attention memory from contiguous physical allocation, supporting large-batch long-context inference.
Working-Memory Consolidation: Bottlenecked Transformers periodically consolidate or rewrite cached representations at structured generation boundaries through an auxiliary bottleneck module.This manipulation remains short-term implicit working memory rather than long-term explicit storage.
Sliding Window versus Sparse Global Attention: Sliding-window attention yields linear complexity by prioritizing local recency, whereas sparse global attention combines local windows with selected global tokens to retain theoretical expressivity guarantees.Longformer and BigBird exemplify sparse global attention mechanisms.
Architectural Tradeoffs: Compression, virtualization, and structured sparsity are architectural design axes that reshape memory representation, access, and scaling without proportionally increasing computational or storage cost.These techniques enable extended-context modeling while controlling operational resource usage.

C. Evaluation of Memory Systems … B. Continual and Lifelong Parametric Memory

Memory evaluation requires diagnostic tests spanning retrieval, reasoning, interference, persistence, and efficiency rather than a single scalar metric. Open challenges center on unifying memory theory and enabling stable online adaptation without forgetting or instability under distribution shift.

C. Evaluation of Memory Systems: Memory evaluation is difficult because it is not directly observable and instead appears through behaviors such as long-range resolution, recall fidelity, interference robustness, and contextual persistence.Targeted diagnostic tasks are therefore needed to isolate specific memory capabilities.
C. Evaluation of Memory Systems: Long-context benchmarks test retrieval and recall by measuring recovery of information amid distractors and utilization of extended contexts across question answering, summarization, and reasoning.Examples include Needle-in-a-Haystack, LongBench, RULER, and L-Eval.
C. Evaluation of Memory Systems: Structured dependency tests assess whether models integrate information across distant tokens while maintaining coherent intermediate representations over extended documents.SCROLLS and NarrativeQA-style tasks target reasoning beyond raw retrieval.
C. Evaluation of Memory Systems: Explicit memory evaluation must measure degradation speed, distraction sensitivity, temporal persistence, recall precision, key-value alignment, and continual-learning behavior.The relevant dimensions differ across retrieval-based and parameter-based memory updates.
C. Evaluation of Memory Systems: Memory benchmarks should report performance against context length and memory footprint because larger context windows can inflate results while remaining impractical under real-world constraints.Scaling curves are especially relevant for compressed or paged attention systems.
C. Evaluation of Memory Systems: Implicit and explicit memory require different protocols, including long-range accuracy and noise stability versus update consistency, retrieval precision and latency, and robustness to growth and interference.These distinctions motivate evaluation frameworks tailored to memory accessibility and update behavior.
C. Evaluation of Memory Systems: A unified memory metric remains unavailable, motivating orthogonal evaluation axes of capacity, fidelity, persistence, and efficiency to compare implicit, explicit, and hybrid architectures.Controlled evaluation is needed to separate architectural improvements from simply increasing context size or parameter count.
VI. OPEN CHALLENGES AND FUTURE DIRECTIONS: Memory research increasingly treats memory as a modular, controllable architectural axis, but lacks unified theory and must address stable lifelong parametric adaptation under forgetting and distribution shift.A state-transition framework could organize mechanisms by persistence, accessibility, update dynamics, representation, retention fidelity, interference robustness, and compression efficiency; online adaptation includes gradient-based test-time updates, nested parameter hierarchies, and modular memory blocks.

C. Robust and Interpretable Update Rules … VII. CONCLUSION

The paper identifies robust update rules, adaptive memory orchestration, hardware–algorithm co-design, and multidimensional evaluation as necessary for scalable memory systems. Its unified taxonomy frames memory through representation, update dynamics, persistence, and mechanisms governing information retention and forgetting.

C. Robust and Interpretable Update Rules: Optimization-based writing is adaptable but vulnerable to objective mismatch, accumulated bias, and catastrophic drift, while state-transition updates risk interference, saturation, and miscalibrated forgetting.These challenges arise from proxy test-time objectives and compressed state representations.
D. Adaptive Memory Allocation and Control: Hybrid memory architectures require coordination across attention, state-space, retrieval, and expert-routing pathways rather than fixed compositions.Static designs limit adaptation to task structure and contextual demands.
D. Adaptive Memory Allocation and Control: Learned controllers could allocate tokens, compress context representations, and modulate routing across memory subsystems according to task requirements.The proposed orchestration dynamically adjusts memory usage instead of fixing pathways at design time.
E. Hardware–Algorithm Co-Design for Scalable Memory: At contexts of hundreds of thousands or millions of tokens, memory bandwidth and storage hierarchies increasingly dominate system performance.The paper links scalable architectural innovation to GPU and accelerator memory constraints.
E. Hardware–Algorithm Co-Design for Scalable Memory: Hierarchical memory aligned with hardware tiers, multi-level caching, selective recomputation, in-memory attention, and hardware-aware sparsity are proposed co-design directions.These approaches seek to manage memory operations under expanding context windows.
F. Principled and Multi-Dimensional Memory Evaluation: Memory evaluation should separately measure storage capacity, retrieval fidelity, temporal persistence, interference robustness, and computational efficiency.Extended-context benchmarks can conflate memory capacity with reasoning ability and other scaling effects.
VII. CONCLUSION: The survey unifies memory architectures through representation, update dynamics, and persistence, while formalizing optimization writing, state transitions, signal-gated routing, and structural consolidation.The framework characterizes how information is retained, modified, and forgotten.
VII. CONCLUSION: Implicit memory is coupled to forward computation, whereas explicit memory provides autonomous persistent storage; no single paradigm satisfies every computational desideratum.Token-level attention favors high-fidelity recall but scalability costs, while recurrent sequence memory provides compressed long-horizon views.