PaperlayerView exact revision on arXiv

Source-linked AI summary

Sequence-to-Sequence Models with Attention Mechanistically Map to the Architecture of Human Memory Search

Nikolaus Salvatore, Qiong Zhang

arXiv:2506.17424v1q-bio.NCcs.LG

TL;DR

It remains unclear why human memory uses context-based architectures despite their ability to explain many memory phenomena. This paper maps RNN-based seq2seq models with attention to CMR and implements one as a model of memory search. The model captures human recall patterns as effectively as CMR and, when optimized, converges on optimal free-recall patterns.

  • Problem

    Although context-based models explain many memory behaviors, the functional role of context and why human memory uses this architecture rather than alternatives remain unclear.

    Source: Introduction · p. 2

  • Method

    The authors mathematically map RNN-based seq2seq attention mechanisms to CMR and implement the seq2seq model as a cognitive model of memory search.

    Source: Introduction · pp. 1–2

  • Results

    The seq2seq model captures and predicts human free-recall patterns as effectively as CMR, and its fully optimized version converges on optimal free-recall patterns.

    Source: Introduction · p. 2

  • Takeaways & Limitations

    The convergence suggests CMR’s context-based architectural assumptions may serve an adaptive purpose and offers a flexible, interpretable model of memory search.

    Source: Introduction · p. 2

    The model is evaluated on immediate free recall; delayed recall and recall with distractor tasks remain to be evaluated.

    Source: An alternative model of human memory… · pp. 16–17

Abstract

from arXiv · show

Past work has long recognized the important role of context in guiding how humans search their memory. While context-based memory models can explain many memory phenomena, it remains unclear why humans develop such architectures over possible alternatives in the first place. In this work, we demonstrate that foundational architectures in neural machine translation -- specifically, recurrent neural network (RNN)-based sequence-to-sequence models with attention -- exhibit mechanisms that directly correspond to those specified in the Context Maintenance and Retrieval (CMR) model of human memory. Since neural machine translation models have evolved to optimize task performance, their convergence with human memory models provides a deeper understanding of the functional role of context in human memory, as well as presenting new ways to model human memory. Leveraging this convergence, we implement a neural machine translation model as a cognitive model of human memory search that is both interpretable and capable of capturing complex dynamics of learning. We show that our model accounts for both averaged and optimal human behavioral patterns as effectively as context-based memory models. Further, we demonstrate additional strengths of the proposed model by evaluating how memory search performance emerges from the interaction of different model components.

Introduction

The paper maps RNN-based seq2seq models with attention onto CMR, linking translation architectures developed for task performance with mechanisms of human memory. It uses this connection to model memory search and examine context’s role and model flexibility.

Introduction

  • RNN-based seq2seq models with attention share architectural components with CMR, including a mapping between sequence encoding and recall and between attention and context reactivation.The models developed for translation performance correspond to mechanisms specified in a model developed to align with human behavior.Source: Introduction · pp. 1–2
  • The convergence suggests CMR’s context-based architecture may serve an adaptive purpose, because translation models developed through task-performance optimization converge on similar structures.The authors frame this as evidence that using context may be rational for organizing and retrieving information, not as proof that alternatives cannot work.Source: Introduction · p. 2
  • The seq2seq model offers a flexible, interpretable approach to memory search, addressing limits of parameter-fitting models and the interpretability of neural networks.The authors implement a neural translation model as a cognitive model of human memory search.Source: Introduction · p. 2
  • A basic seq2seq model with attention captures and predicts human free-recall patterns as effectively as CMR, while its fully optimized version converges on optimal free-recall patterns.Evaluations during training also show recency and temporal contiguity characteristics similar to those of human participants.Source: Introduction · p. 2
  • The paper examines how working-memory capacity and episodic-memory attention affect recall, including whether attention ablation captures patterns associated with hippocampal amnesia.Reduced hidden-state capacity is proposed to require greater reliance on context retrieved from episodic memory; attention ablation is examined as a model of amnesic recall characteristics.Source: Introduction · p. 2

Background: Parallels in Historical Developments

Historical developments in translation and human memory provide parallels between sequence processing and context-guided recall. Attention’s access to prior encoder states is compared with CMR’s reinstatement of encoding contexts.

The sequence-to-sequence model architecture and early formulations of human context models

  • Translation models progressed from rule-based and statistical approaches to neural methods, while seq2seq addressed RNN limits on variable-length inputs and outputs.An encoder maps the source sequence to a fixed-length context vector that a decoder uses to generate a variable-length target sequence.Source: The sequence-to-sequence model… · pp. 2–3
  • In free recall, studied items form the input sequence and recalled items the output; context drifts over time, cues recall, and overlaps more for items from the same list.The end-of-encoding context carries into recall when delay or distraction is minimal.Source: The sequence-to-sequence model… · p. 3

The attention mechanism and the context reinstatement mechanism

  • Attention gives the decoder access to prior encoder states, paralleling CMR’s reinstatement of an item’s encoding context to cue remaining recalls.CMR’s reinstatement mechanism is linked to the tendency for items studied close together to be recalled together.Source: The attention mechanism and the… · p. 3

Deriving a Detailed Mathematical Mapping

The section derives a component-level mapping between attention-based seq2seq models and CMR, aligning how both encode items into evolving states and use prior states to guide recall or output.

Seq2seq model with attention

  • At each encoding step, the seq2seq model combines the new item's embedding with its previous hidden state, stores all encoder states, and later attends over them during decoding.The decoder receives the previous recalled item's embedding and prior decoder state; dot-product scores and softmax weights produce a weighted sum of encoder states.Source: Seq2seq model with attention · p. 6

Context Maintenance and Retrieval (CMR) model

  • CMR gradually shifts context toward each studied item, forms item-context associations, and favors recalling items whose encoding contexts resemble the current recall context.A softmax converts activation strengths into recall probabilities; parameter k controls retrieval noise.Source: Context Maintenance and Retrieval… · pp. 7–8

Deriving a Detailed Mathematical Mapping

  • The mapping aligns encoding states that mix prior context and new-item input, then shows that both models' decoding states combine recurrent state, recalled-item input, and reactivated encoding information.CMR uses fixed mixing parameters for encoding, whereas seq2seq uses learned parameter matrices, a bias, and an activation function.Source: Putting everything together: Mapping… · pp. 8–9

Putting everything together: Mapping the seq2seq model with attention to CMR

  • Attention and context reinstatement are mathematically equivalent: each uses the current decoding state to retrieve a similarity-weighted average of encoding states that guides the next item.The decoding states also share the previous state, the most recently recalled item's embedding, and the reactivated encoding states.Source: Figure 2 · p. 5, Putting everything together: Mapping… · p. 9

Results

On the PEERS free-recall evaluation, the attention-based seq2seq model fit individual behavioral patterns better than CMR. Further analyses show how recall patterns vary with training, hidden-state capacity, and the attention mechanism.

Results

  • The analyses use PEERS task structures to assess individual prediction, learning-stage patterns, hidden-dimension effects, and attention ablation.Source: Results · p. 10

The seq2seq model can explain and predict recall behavior in memory search comparably to the CMR model.

The optimized seq2seq model demonstrates the same recall behavior as the optimal policy of the CMR model.

The intermediate training evaluations of the seq2seq model exhibit similar qualitative patterns as are typically observed in human participants.

The seq2seq model provides insight into how working memory capacity affects optimal recall strategies.

An ablation study of the attention mechanism provides insight into the performance difference between amnesia patients and healthy controls.

Discussion

RNN seq2seq models with attention map closely to CMR and can model human free recall, including optimal and sub-optimal behavior. The discussion examines how working-memory capacity and episodic context relate to recall, and whether task-optimized architectures offer a rational account of memory processes.

Discussion

  • Attention-enabled seq2seq models predict human recall characteristics more accurately than CMR while aligning with its optimal recall behavior and capturing sub-optimal behavior.The model’s attention mechanism can reactivate past encoding contexts, paralleling CMR; removing attention reduces backward contiguity and recall performance.Source: Discussion · p. 16

An alternative model of human memory search

  • The seq2seq model balances interpretability with flexibility, fits and predicts individual free-recall characteristics, and outperforms CMR in some predictive tasks.The authors also point to transformer-based language models as a possible future avenue for modeling memory.Source: An alternative model of human memory… · p. 16

Disentangling the contributions of working memory and episodic memory

  • Smaller hidden dimensions are associated with stronger attention-based reliance on episodic memory for primacy items, while increased working-memory capacity can compensate for attention ablation.The model combines evolving working-memory context with reactivated encoding context; unlike CMR’s fixed combination parameter, it uses a learned dense layer to combine them dynamically.Source: Disentangling the contributions of… · p. 17

A rational explanation of architectural assumptions in human memory search

  • Starting recall at the list’s beginning and proceeding forward produces optimal recall, matching the optimal behavior derived from CMR.The authors argue that convergence between task-optimized NMT architectures and behavior-aligned memory models supports an adaptive purpose for CMR’s architectural assumptions.Source: A rational explanation of… · pp. 17–18

Conclusions and Future Directions

The conclusion presents attention-based seq2seq models as an interpretable, flexible tool for modeling memory search. It identifies more naturalistic recall tasks and newer NMT architectures as directions for future work.

Conclusions and Future Directions

  • The seq2seq model with attention balances interpretability and flexibility, making it useful for simulating cognitive phenomena and exploring theoretical insights into memory.Compared with models built around lists of random words, it can handle more complex input and output structures such as sentences.Source: Conclusions and Future Directions · p. 18
  • Applying the model to recall narratives and stories is a proposed direction beyond recalling lists of words.Source: Conclusions and Future Directions · p. 18
  • Future work could examine transformer-based NMT architectures and further investigate links between neural architecture and human capacities such as working memory.Source: Conclusions and Future Directions · p. 18

Methods

The methods implement attention-based seq2seq models for individual recall prediction and optimal free recall, using PEERS data and a separate MTL-amnesia comparison. Training includes supervised pre-training followed by reinforcement learning for optimal behavior.

Additional Model Details

  • GRU seq2seq models use hidden dimensions of 32, 64, or 128; episodic memory stores encoding contexts and item embeddings, which attention weights to form retrievals for decoding.A two-layer MLP converts retrieved embeddings into discrete item predictions.Source: Additional Model Details · p. 18

Explain and Predict Individual Subject Behavior

  • Individual-subject models were trained on 90% of each subject’s presentation–recall pairs, with validation and test splits of 5% each, and compared with fitted CMR models.Seq2seq performance was reported on held-out test data; CMR parameters were optimized against serial-position, first-recall, and conditional-response patterns.Source: Explain and Predict Individual Subject… · pp. 18–19

Optimal Free Recall Behavior

  • For optimal free recall, the model learns to recall as many items as possible from a presented list using reinforcement learning after supervised pre-training.Source: Optimal Free Recall Behavior · p. 19

Supervised Pre-Training

  • Supervised pre-training uses random word lists from the PEERS vocabulary and a Sinkhorn-based set prediction loss to initialize models before reinforcement learning.The models were trained on 50,000 randomly generated 14-word sequences for 10 epochs.Source: Supervised Pre-Training · p. 19

Reinforcement Learning

  • PPO trains the recall policy with rewards of +1 for correct items, -1 for incorrect items, and -0.5 for repeating a correctly recalled item.Training used 50,000 iterations of four episodes each; evaluation used 10,000 randomly generated sequences.Source: Reinforcement Learning · p. 19

Penn Electrophysiology of Encoding and Retrieval Study

  • Behavior comparisons used immediate free-recall data from 171 young adults in Experiment 1 of the PEERS study, with 16-word lists and several encoding-task conditions.Participants had 75 seconds to recall after each list.Source: Penn Electrophysiology of Encoding and… · pp. 19–20

Medial Temporal Lobe Amnesia and Temporal Context Study

  • The attention-ablation comparison used a looped-list task with 12-word lists repeated four times to study temporal contiguity in MTL-amnesia patients and controls.The cited study included ten amnesic patients and 16 controls; patients showed disrupted temporal contiguity despite an intact recency effect.Source: Medial Temporal Lobe Amnesia and… · p. 20
Loading 2506.17424v1…