PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning a Fact Is Not Learning How to Retrieve It

Chaemin Jang, Jihee Kim, Dongman Lee

arXiv:2610.03251v2cs.AI

TL;DR

A fact learned in one wording may not be retrievable through another, and ordinary fine-tuning makes it difficult to distinguish weak fact learning from failure to learn a request form. The paper separates those stages and finds that context states link earlier form experience to later retrieval.

  • Problem

    Ordinary fine-tuning presents facts and request forms together, making it difficult to distinguish weak fact learning from failure to retrieve through a different form.

    Source: §1 · p. 1

  • Method

    The authors vary request-form training before identical statement-only training on new facts, then examine and intervene on the context state during retrieval and fact learning.

    Source: §1 · pp. 1–2, §5.2 · p. 9

  • Results

    The context state links earlier request-form experience to later retrieval, and changing it can affect retrieval across facts and factual relations.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Retrieval through other request forms can be learned separately from factual content and reused across facts in the experiments.

    Source: §6 · p. 9

    The experiments use controlled factual training, the context state does not explain every retrieval failure, and effects may vary with what pretrained models already know about request forms.

    Source: §6 · p. 9

Abstract

from arXiv · show

A model trained on "The capital of X is Y" may produce "Y" after "The capital of X is" but fail after "The capital of X:." We call these different ways of eliciting the same fact request forms. To separate learning a fact from retrieving it, we train two models in two stages. In the first stage (request-form training), one model sees each fact in five forms and the other sees the same facts only as statements. In the second stage (target-fact training), both receive identical training on new facts, all as statements. Both then retrieve the new facts almost equally well from statements, but differ sharply on other request forms. Thus, a model can learn how to retrieve through a request form before it learns the facts. To understand this difference, we examine the hidden state immediately before the answer, which we call the context state. When given two different request forms for the same fact, the model trained on five forms in stage one produces more similar context states than the model trained on statements alone in that stage. Changing this state at retrieval time can enable or prevent retrieval of an already learned fact, and the same effect transfers across facts and factual relations, such as capitals and currencies. To test its role during learning, we change the context state only during target-fact training. This intervention changes later retrieval without intervention at test time. Together, these results show that later retrieval depends on earlier request-form experience and the context state during fact learning.

1 INTRODUCTION

The paper separates learning a fact from learning how to retrieve it, addressing the ambiguity that ordinary fine-tuning teaches both together. It identifies the context state before the answer as a link between earlier request-form experience and later retrieval.

1 INTRODUCTION

  • Identical target-fact training yields sharply different retrieval across request forms depending on earlier request-form training.Both models retrieve facts from the declarative training form, but only the five-form model retrieves them reliably from the colon-ended form.Source: §1 · p. 1
  • Replacing the context state at retrieval can reverse success, and a state from one fact can help retrieve another fact across relations.The paper identifies the hidden state immediately before the answer as the context state.Source: §1 · pp. 1–2
  • Shifting the context state during target-fact training changes later retrieval through a request form, even when the shift is absent at test time.Declarative and list request forms produce more similar context states after five-form stage-one training than after statement-only training.Source: §1 · p. 2
  • The paper’s central distinction is between learning factual content and learning how to retrieve that content through a request form.The context state is presented as the link between earlier request-form learning and later retrieval of new facts.Source: §1 · p. 2

2 RELATED WORK

Related work examines how format experience, presentation, and internal representations affect knowledge use. This paper positions its contribution as identifying an answer-position context state that controls retrieval.

2 RELATED WORK

3 LEARNING A FACT AND LEARNING HOW TO RETRIEVE IT CAN BE SEPARATED

The two-stage experiment separates learning new facts from learning how to retrieve them: identical fact training yields similar performance in the training form but sharply different retrieval through other forms. Earlier request-form experience transfers to new facts, while retrieval is especially sensitive to the request’s final words.

3.1 TWO-STAGE TRAINING

  • The two-stage design gives one model five forms for earlier facts and the other statements alone, then trains both identically on new facts presented as statements.The target facts are disjoint from stage-one facts, and evaluation includes forms absent from both stages.Source: §3.1 · p. 4

3.2 LEARNING A FACT DOES NOT TEACH RETRIEVAL THROUGH NEW REQUEST FORMS

  • Earlier request-form training preserves retrieval of new facts across forms: list-form accuracy rises from 0.000 to 0.977 despite identical target-fact training.The declarative training form performs similarly across histories; the gap extends across relations, while possessive and short key–value forms improve little and a relative-clause paraphrase declines.Source: §3.2 · p. 4, Table 2 · p. 4, Figure 2 · p. 5
  • The cross-form retrieval gap is not accompanied by detectable differences in declarative fact learning and recurs across model families, relations, entity types, and training methods.The continuous learning measures’ intervals include zero.Source: §3.2 · p. 5

3.3 EARLIER REQUEST-FORM TRAINING TRANSFERS TO NEW FACTS

  • Earlier request-form training improves retrieval of new capitals even when earlier answers are incorrect or come from another relation; retrieval also rises with request-form exposure.At 5% exposure, request-form training yields 0.381 versus 0.003 for declarative paraphrases; Figure 3 tracks list-form exact-generation accuracy and marks Pythia-1.4B at 20% exposure.Source: §3.3 · pp. 5–6, Figure 3 · p. 6, Table 3 · p. 6

3.4 RETRIEVAL DEPENDS ON THE REQUEST NEAR THE ANSWER

  • Retrieval tracks the request’s final words: sharing more of the training line’s final words is associated with higher retrieval, while an unrelated 30-token preamble does not change which forms succeed.The hidden state immediately before the answer is identified as a natural place to investigate what controls retrieval.Source: §3.4 · p. 6

4 A REUSABLE CONTEXT STATE CONTROLS RETRIEVAL

Changing the context state can enable or prevent retrieval of an already learned fact. Its retrieval-supporting effect transfers across facts and relations and is largely captured by a fact-independent direction.

4.1 CHANGING THE CONTEXT STATE CHANGES RETRIEVAL

  • Replacing only the list request’s context state with the declarative-form state raises exact-generation accuracy from 0.000 to 0.963; the reverse replacement reduces it from 0.997 to 0.023.A norm-matched random state restores nothing, and replacing the state at the entity token has no effect.Source: §4.1 · p. 7

4.2 THE CONTEXT STATE TRANSFERS ACROSS FACTS AND RELATIONS

  • Context states from other facts—and even other relations—restore retrieval nearly as well as same-fact states, including across three models and two-token answers.For Pythia-410M, currency- and population-derived states reach 0.946 and 0.928, compared with 0.963 for the same-fact state.Source: §4.2 · p. 7
  • The retrieval-supporting state is not merely the request’s final token: at later layers, useful information persists when the state carries an entity.At layer 6, entity-free states recover 0.02, while entity-carrying states recover 0.70–0.78; the reverse Qwen intervention lowers declarative retrieval from 0.999 to 0.550.Source: §4.2 · p. 7

4.3 A FACT-INDEPENDENT DIRECTION SUPPORTS RETRIEVAL

  • Adding a single direction raises list-form accuracy from 0.000 to 0.921, and removing it from the restoring vector reduces accuracy to 0.010.The direction also supports colon-ended declarative retrieval and transfers between capital and birth-year domains; a learned linear map adds little beyond it.Source: §4.3 · pp. 7–8
  • Declarative and list context states begin with similar centered cosine similarity across layers, then separate in later layers; earlier five-form training keeps them more aligned.The comparison is shown for both Pythia models, with layer 24 measured after the final layer norm.Source: Figure 4 · p. 8

5 CONTEXT STATES LINK FACTS TO FUTURE RETRIEVAL DURING LEARNING

Earlier request-form training keeps context states for other forms closer to the declarative form used to teach new facts. Changing the context state during fact learning changes later retrieval, even when the model is tested without intervention.

5.1 REQUEST FORMS PRODUCE ALIGNED CONTEXT STATES

  • Without earlier request-form training, list and declarative context states grow more different in later layers; with it, they remain more similar in both Pythia models.The difference is small at the first layer, indicating that the forms separate as they pass through the model.Source: §5.1 · p. 8

5.2 CHANGING THE CONTEXT STATE DURING LEARNING CHANGES FUTURE RETRIEVAL

  • During fine-tuning, each fact remains a declarative sentence, but its context state is moved toward one produced by another request form; testing afterward leaves the state untouched.The shift is computed on facts other than those evaluated, so the target entity-answer pairs were not shown together in that request form.Source: §5.2 · pp. 8–9
  • Shifting context states during factual fine-tuning changes later colon-ended retrieval across models and domains, including from 0.024 to 0.943 for Pythia-410M.The shift’s direction was not used to construct the tested colon-ended form; Table 5 reports exact-generation accuracy there.Source: §5.2 · p. 9, Table 5 · p. 9
  • The particular shift matters: in Qwen2.5-1.5B, it improves accuracy by an additional 0.27 over a random change and reaches 0.94 versus 0.72 from showing 5% of target facts in the new forms.The shift improves retrieval without writing any target fact in a request form.Source: §5.2 · p. 9

6 IMPLICATIONS AND LIMITATIONS

The paper argues that factual learning and retrieval can be separated: identical training on new facts can yield different retrieval depending on earlier request-form experience. It also identifies limits to the findings and documents AI assistance and reproducibility materials.

6 IMPLICATIONS AND LIMITATIONS

  • Changing the context state during factual fine-tuning changes later retrieval, even after the intervention stops.The paper presents this state as linking earlier request-form training to later retrieval.Source: §6 · p. 9
  • The findings suggest evaluating factual fine-tuning by separating whether a fact was learned from whether it can be retrieved through a particular request form.A fact may be retrievable through one form while failing through another.Source: §6 · p. 9
  • The evidence comes from controlled factual training; natural-text learning may involve additional mechanisms, and the context state does not explain every retrieval failure.Effects may also depend on pretrained knowledge of a request form, so larger models, different pretraining, or instruction tuning may change their strength.Source: §6 · p. 9

AI USE STATEMENT

  • Generative AI assisted experiment planning, code review, analysis organization, and manuscript editing; the authors report that they ran and verified the experiments.The code, prompts, per-cell results, and analysis scripts are available, and reported statistics can be recomputed from the artifacts.Source: AI USE STATEMENT · p. 10, REPRODUCIBILITY STATEMENT · p. 10

A EXACT PROTOCOL

The protocol uses synthetic factual worlds and a two-stage training design, then evaluates retrieval under multiple forms. A preamble condition tests whether unrelated text placed before the request affects evaluation.

A EXACT PROTOCOL

  • The fact world contains 2,000 synthetic country–capital pairs, with 300 fixed evaluation facts drawn from one disjoint set.The candidate pool is fixed, and the main experiments use capitals with unique first tokens.Source: A EXACT PROTOCOL · pp. 13–14
  • Training uses Pythia-410M and specified optimization settings, with five-form or statement-only earlier training followed by identical target-fact training.The target-stage corpus is byte-identical across histories within a seed; each target fact is seen about eleven times.Source: A EXACT PROTOCOL · p. 14

B REQUESTS WITH A PREAMBLE

C MORE TARGET-FACT TRAINING

The supplementary experiments vary the amount and rate of target-fact training, test request-form exposure, and replicate retrieval gains in independently generated fact worlds.

C MORE TARGET-FACT TRAINING

  • In statement-only-history Pythia-410M experiments across eight seeds with exact generation, X4800 used 4,800 updates at the standard rate, while XLR used learning rate 10−5.XLR ran until declarative generation matched the standard run’s final value (−0.01), checked every 200 updates; it took 1,000–6,800 updates, with a maximum of 9,600.Source: C MORE TARGET-FACT TRAINING · p. 14

D REQUEST-FORM EXPOSURE

  • The request-form exposure experiment trains a fraction of target facts in five forms while evaluating target facts presented only as statements.At 100% exposure, 768 of 1,000 target facts are written in five forms, including all 300 evaluation facts; the reported evaluation forms remain unwritten.Source: D REQUEST-FORM EXPOSURE · p. 14

E REPLICATIONS

  • In two independent fact worlds, five-form earlier training exceeded statement-only training on held-out list retrieval by +0.974 and +0.887; an additional seed set gave +0.972.All reported comparisons were positive in 8/8 seeds in each setting.Source: E REPLICATIONS · p. 15

F GENERALITY

The generality experiments examine keying and restoration across relations, including a five-relation run. The results report separation across relations and an exception in statement-only birth-year retrieval.

F GENERALITY

  • The experiments compare held-out list retrieval after identical target-fact training, test request-final-word keying, and attempt restoration by rewriting or adding a restoring vector.The restoring-vector direction alone gives 0.921 in Pythia-410M.Source: F GENERALITY · p. 15
  • Every relation separated in 8/8 seeds in the Pythia-1.4B five-relation run, with matched declarative accuracy.The comparison is for list-form exact generation.Source: F GENERALITY · p. 15
  • The statement-only birth-year list value was 0.344, higher than for other relations, while the five-form value was unaffected.Source: F GENERALITY · p. 16

G SUFFIX BANK

The suffix bank evaluates exact generation across 16 request forms grouped by overlap with the training line. The statement-only model’s accuracy varies sharply across forms, even within the same overlap level.

G SUFFIX BANK

  • The evaluation fixes 16 request forms, with one to four forms per overlap level, and reports exact-generation means across eight seeds.Overlap counts how many final tokens of the training line recur immediately before the answer.Source: G SUFFIX BANK · p. 16
  • 0.995 for “Indeed, the capital of {c} is” and 0.000 for “Capital of {c}:” show sharply different generation across request forms.The former has overlap 4 and the latter overlap 0.Source: G SUFFIX BANK · p. 16
  • Generation is not uniform within overlap levels: overlap-2 forms range from 0.085 to 0.891, while overlap-3 forms range from 0.187 to 0.948.For example, “The capital in {c} is” scores 0.891, whereas “The chief city in {c} is” scores 0.085.Source: G SUFFIX BANK · p. 16

H PATCHING DETAILS

Patching the answer-position context state can restore or reduce request-form generation, with results varying by model and patch source. In Qwen2.5-1.5B, several “is”-final sources restore list-form generation, while a random state does not.

H PATCHING DETAILS

  • The patching analysis compares answer-position states from declarative prompts, random vectors, other facts, and prompts involving other relations or no entity.States are held at the final prompt position during decoding; ranking is read from the first forward pass.Source: H PATCHING DETAILS · p. 16
  • In Pythia-410M, unpatched list-form generation is 0.000 and rewrite generation is 0.948; patching at the country token leaves generation at 0.000 at every layer.The layer-0 cells and rewrite are scored on all 300 facts; other layer/source results use facts correct under the declarative prompt by pool top-1.Source: H PATCHING DETAILS · p. 16
  • In Qwen2.5-1.5B, same-fact, different-fact, and cross-relation “is”-final states restore list-form generation to 0.998–0.989, while a random state yields 0.000.Unpatched generation is 0.603 and rewrite generation is 0.998.Source: H PATCHING DETAILS · p. 17
  • In the shared-first-token world, unpatched generation is 0.001 and rewrite generation is 0.941, with both capital tokens scored by layer and source.The protocol uses all 300 facts and same-fact, different-fact, and random sources.Source: H PATCHING DETAILS · p. 17

I CONTEXT-STATE INTERVENTION DURING LEARNING

The paper intervenes on context states during target-fact training and examines how those states are constructed and how results are assessed. The statistical protocol treats training seeds as the inferential unit.

I CONTEXT-STATE INTERVENTION DURING LEARNING

  • At layer 4, the targeted request-form shift was added at each target line’s answer position; for Pythia-410M, held-out WikiText loss was 5.584 versus 5.595 under a norm-matched random shift.Exact-generation results are eight-seed means (four seeds for birth years); the main-text values use the colon-ended declarative form, whose direction was not used in the shift.Source: I CONTEXT-STATE INTERVENTION DURING… · p. 17

J CONTEXT-DIRECTION VECTOR DETAILS

  • An affine map fit on earlier-training facts gives 0.949 list and 0.995 declarative generation on evA; a rank-16 map gives values within 0.01.The direction-only condition uses β = 1 in every seed; matched-cosine removal leaves 0.388 for the mirror direction, 0.151 for random directions, and 0.010 for the target direction.Source: J CONTEXT-DIRECTION VECTOR DETAILS · p. 18

K STATISTICAL PROTOCOL

  • The training seed is the inferential unit, with core comparisons using eight paired seeds and exact paired sign-flip tests over all 28 assignments.Interventions changing declarative accuracy by more than 0.05 in any seed are treated as uninterpretable, not null.Source: K STATISTICAL PROTOCOL · p. 18
Loading 2610.03251v2…