PaperlayerView exact revision on arXiv

Source-linked AI summary

Self-Keyed In-Parameter Memorization for External Memory-Free Dialogue Recall in Frozen Language Models

XinYu Piao, Myunghoon Ryu, Joongheon Kim, Jong-Kook Kim

arXiv:2610.36948v1cs.CLcs.AI

TL;DR

This paper asks whether a frozen language model can internalize dialogue and later answer questions without external memory or access to the future question and answer during storage. It proposes Skim, which writes self-generated questions as keys for dialogue spans in a closed-form parameter edit. Across five models, Skim recalls stored turns, and streaming storage reaches the same recall performance as a batch edit.

Abstract

from arXiv · show

A large language model maintains conversational content in an external storage that is outside the model and grows as the dialogue continues. This work assumes an environment where a language model has no external memory and no prior knowledge of the future question or its answer. The motivation is to research whether a dialogue can be stored in the model parameters and later answer a question about it. This work proposes self-keyed in-parameter memorization (SKIM), a closed-form method in which the model generates its own questions about each dialogue turn and writes those questions into its parameters as the keys that later retrieve the turn's content. Generating many varied questions for each turn allows a later user question to match one even when phrased differently, and its answer is recalled from the parameters alone. Evaluated on five language models, SKIM recalls most stored dialogue turns from later user questions, and it answers from the model parameters without any external storage.

Introduction

This section asks whether a frozen model can store dialogue in its parameters and answer later questions without external memory. It introduces Skim, which uses self-generated questions as keys, and reports recall across five language models.

Introduction

  • As dialogue lengthens, KV-cache memory and inference costs rise, while retrieval-augmented generation depends on finding the right passage and forming an effective query.The KV cache stores attention keys and values for past tokens; retrieval-augmented generation uses an external index.Source: Introduction · p. 1
  • The work asks whether a frozen model can internalize dialogue as it unfolds and answer later questions from its parameters alone, rather than improve external-memory systems.The stated aim is to test whether this internalization is feasible.Source: Introduction · p. 1
  • Dialogue weight editing lacks both a supplied answer label and a known fixed cue, while external caches and retrieval stores grow as turns accumulate.Prior editing methods require an object label and a subject–relation cue; external storage avoids these assumptions but retains dialogue outside the parameters.Source: Introduction · p. 1
  • Transformer feed-forward layers are treated as factual-association memories, motivating a closed-form edit to a model-specific mid-to-late layer’s down-projection.The cited prior work locates readily edited facts in a band of middle layers.Source: Introduction · p. 1
  • Skim extracts candidate spans, generates questions for each span, selects questions that distinguish same-turn rivals, and jointly installs question–span pairs through a closed-form edit.The generated questions replace the fixed subject–relation cue required by prior editing methods.Source: Introduction · p. 2
  • At a five-model mean of 63.1% and 75.0% on the strongest model, Skim recalls stored dialogue turns well above the no-cache floor that removes all memory.The evaluation covers five language models.Source: Introduction · p. 2

Problem Setting and Preliminaries

The setting requires a frozen model to store each arriving turn without access to its later question or answer, then answer from parameters alone. The preliminaries frame a feed-forward down-projection edit as a regularized least-squares update over keys and values.

Problem Setting and Preliminaries

  • The dialogue arrives as a stream of turns, each stating a fact that a later question will ask about, but the model receives neither that question nor its answer during storage.Unlike a cache or retrieval index, the model must decide from the turn itself what to store.Source: Problem Setting and Preliminaries · p. 2
  • Each turn must be stored without gradient fine-tuning at a cost independent of stored turns; recall uses one forward pass, no external memory, and preserves general behavior.The later question is the only recall input, and its phrasing may be unanticipated.Source: Problem Setting and Preliminaries · p. 2
  • A transformer feed-forward down-projection is modeled as key-value memory: hidden activations address content, and editing its weight matrix changes the retrieved values.For multiple associations, keys and values form the columns of matrices K and D.Source: Problem Setting and Preliminaries · p. 2
  • The edit minimizes a regularized least-squares objective, whose zero-gradient condition yields normal equations and a closed-form ridge solution.The stated second form inverts an n × n matrix, which is small when few turns are stored.Source: Problem Setting and Preliminaries · p. 2
  • A larger regularization strength favors small edits and preservation of base weights, while near-orthogonal keys bound how many associations a layer can hold before interference.The update stores values in superposition and reproduces each value at its key up to regularization.Source: Problem Setting and Preliminaries · p. 2

Proposed Method

Skim stores turns as question–span pairs through a closed-form edit to one feed-forward layer, then recalls spans from model parameters. It also re-solves accumulated writes so arrival order does not affect the final edit.

Self-Keyed Construction

  • Skim extracts short candidate spans as values, generates answer-free questions as keys, and keeps diverse questions that distinguish each span from its rivals.Top-k spans defer the choice of which content to retrieve until the later question arrives.Source: Self-Keyed Construction · p. 3, Figure 1 · p. 3

Joint Edit in One Feed-Forward Layer

  • A joint closed-form edit installs accumulated question–span pairs in one feed-forward projection, with a ridge term stabilizing matches between close keys.Unrelated inputs add little to the residual stream, while a matching key contributes the stored value.Source: Joint Edit in One Feed-Forward Layer · pp. 3–4

Recall and Self-Probed Selection

  • At recall, the user question is rendered in the key-building frame and the edited model decodes a short answer from its parameters.A self-probe selects storage-time settings using unused paraphrases of generated questions and clean span emission, without later questions or ground-truth answers.Source: Recall and Self-Probed Selection · p. 4

Mitigating Forgetting from Sequential Edits

  • Each new chunk replaces the previous edit with a re-solve over all accumulated pairs, keeping the stored edit and each recall pass fixed in size.The offline re-solve grows as turns accumulate, but the resulting matrix and one-pass recall do not.Source: Mitigating Forgetting from Sequential… · p. 4
  • Arrival order and grouping have no effect on the final edit, which equals the batch edit.The edit is re-solved from the original weights using all accumulated pairs.Source: Mitigating Forgetting from Sequential… · p. 4

Evaluation and Analysis

The evaluation tests SKIM on five models as dialogue turns accumulate, then traces where recall fails and whether streaming writes match a batch edit. Recall is strongest for proper-noun answers, while store-time extraction and selection account for most failures.

Experimental Setup

  • The evaluation uses five open-weight models and up to 32 answer-bearing turns from LongMemEval’s single-session-user subset, excluding numeric-answer questions.The subset contains 33 non-numeric facts; recall is measured from N=4 to N=32.Source: Experimental Setup · pp. 4–5, Table 1 · p. 5

Recall Attribution

  • At N=32, SKIM recalls about 46.9–75.0% across five models, versus about 90–100% for the KV cache and True-Q; its five-model average is 63.1%.The no-cache floor is 3.1–15.6%.Source: Figure 2 · p. 5, Recall Attribution · p. 5

Which Stage Fails

  • At N=32, 44 of 59 failures occur during storage—22 missing spans and 22 wrong selections—while routing failures are zero across the five models.The remaining 15 failures are decoding errors.Source: Figure 3 · p. 6, Which Stage Fails · p. 6

Failure by Position and Answer Type

Streaming Matches Batch

  • Streaming recall matches batch recall at every measured size from N=4 to N=32 because each chunk re-solves from the frozen base weights over accumulated pairs.This order-invariant solve avoids the forgetting that can occur when edits are stacked.Source: Streaming Matches Batch · pp. 6–7, Figure 5 · p. 7

Discussion

  • The main remaining limits are extracting descriptive multi-token spans beside same-turn rivals, forming keys for unforeseen questions, and storing or exactly reproducing numeric answers.The discussion notes that numeric answers are rarely selected and recovered numbers often miss the exact digits required by the containment metric.Source: Discussion · p. 7

Related Work

Prior weight-editing methods store predefined factual targets, while Skim constructs both the stored value and its key from the dialogue turn itself.

Related Work

  • ROME stores one subject-indexed factual association with a rank-one update, while MEMIT distributes updates across layers to edit thousands of associations simultaneously (Meng et al. 2022a,b).AlphaEdit uses null-space constraints to reduce interference during sequential edits (Fang et al. 2025).Source: Related Work · p. 7
  • These editing methods require predefined targets and desired outputs, whereas Skim derives both its stored value and key from the turn (Meng et al. 2022a,b; Fang et al. 2025).Skim recalls selected facts in response to later queries phrased differently.Source: Related Work · p. 7
  • Prior training-free methods modify model behavior or stored knowledge without gradient-based retraining (Meng et al. 2022a; Fang et al. 2025).Closed-form weight editing directly changes selected parameters to store factual associations (Meng et al. 2022a,b).Source: Related Work · p. 7

Conclusion

The study tests whether a frozen model can internalize dialogue turns through closed-form edits without external memory, future questions, or ground-truth answers. Skim recalls dialogue content across five models, and streaming storage matches storing all turns at once.

Conclusion

  • At 32 stored turns, Skim recalls a five-model mean of 63.1% and reaches 75.0% on the strongest model, above the no-cache floor of 3.1 to 15.6%.The study assumes no ground-truth answer, later question, or external memory as dialogue arrives one turn at a time.Source: Conclusion · p. 7
  • Streaming dialogue storage reaches the same recall performance as storing all turns at once, mitigating the forgetting caused by sequential storage.Skim extracts what to store from each turn and uses self-generated questions as keys.Source: Conclusion · p. 7
  • The experiments show Skim stores conversation content in fixed-size model parameters without external memory.The study evaluates whether real dialogue turns can be internalized by one closed-form edit.Source: Conclusion · p. 7

Related papers

Loading 2610.36948v1…