PaperlayerView exact revision on arXiv

Source-linked AI summary

Writability and the Pathway: A Controlled Study of Knowledge Injection into Addressed Memory

Wei Gao, Cheng Tian, Duan Yang, Jianqing Yin

arXiv:2610.21553v1cs.CL

TL;DR

Addressed conditional memory offers a local surface for post-hoc writes, but a write may not transfer to new questions or preserve untargeted behavior. The paper compares update scopes and residual anchoring in controlled GPT-2 experiments and audits archived production-memory writes. Row-only writes reproduce trained completions, while other outcomes vary across question-form behavior, drift, and production cases, supporting bounded rather than universal claims.

  • Problem

    A local addressed-row write does not guarantee access under a new question or preserve behavior on untargeted inputs.

    Source: §1 · pp. 1–2

  • Method

    The study compares row-only and row-plus-projection updates with and without fixed module-residual anchoring, alongside archived production-memory writing experiments.

    Source: §3 · p. 3, §7 · p. 10

  • Results

    Row-only updates reproduce all trained canonical completions; projection updates change question-form diagnostics and increase measured drift, while anchoring lowers drift without changing canonical reproduction.

    Source: §1 · p. 2, §10 · p. 12

  • Takeaways & Limitations

    The results support bounded writing feasibility and treating trained expression, question-form behavior, and preservation as separate measured outcomes.

    Source: §8 · p. 11, §10 · p. 12

    Lexical target-string hits do not establish correct entity-specific retrieval, and the controlled experiment fixes one backbone, initialization, synthetic fact set, and penalty coefficient.

    Source: §8 · p. 11

Abstract

from arXiv · show

Addressed conditional memory provides a local surface for post-hoc knowledge injection, but a successful write need not transfer to a new question or preserve untargeted behavior. This study combines a controlled GPT-2 comparison with production-memory experiments on Qwen3.8-Flash-Next-FP8 and DeepSeek-V4.1-Flash. Newly initialized modules attached to frozen GPT-2 are tested in a 2 x 2 design: row-only versus row-plus-projection updates, each with or without fixed module-residual anchoring. All arms reproduce 50 trained completions in each of three sampling seeds. Natural-question target-string hits average 5.3% for both row-only arms, 28.7% for unanchored projection updates, and 22.0% with anchoring, while strict answer scores remain zero. Anchoring lowers measured drift; lexical hits do not establish entity-specific retrieval. In the archived Qwen experiments, gradient writes make the designated first answer token top-ranked for 132 independently reset targets, comprising 93 new promotions and 39 already top-ranked targets. A separate matched 24-target comparison gives 13 new promotions with Adam versus two with SPSA at equal iteration counts but unequal computational procedures. An 80-row case includes saved continuations and a replay record, while reformulations bound its scope. Six DeepSeek zero-order cases yield one new target-token promotion and one regression. These results support bounded writing feasibility and separate trained expression, question-form behavior, and preservation; first-token rankings do not establish complete-answer success or universal conditions for writable memory.

1 Introduction

The study asks whether local addressed-row writes suffice for trained completions and how writes behave under new questions and on untargeted inputs. A controlled comparison separates row updates from shared readout updates and residual anchoring, finding that canonical reproduction alone does not characterize writability.

1 Introduction

  • A locally addressable write does not guarantee local effects or access under a new question, because other inputs can reuse rows and shared projections and gates mediate retrieval.The study therefore measures trained completion, question-form behavior, and collateral changes separately.Source: §1 · pp. 1–2

2 Related work

  • Prior work studies local row edits, multiple fact forms, and retention-preserving conditional activation; this study examines shared readout updates and drift constraints from a common mounted starting state.The related-work comparison also distinguishes addressed n-gram memory from other editing targets and notes that cross-paper numerical rankings are precluded by differing setups.Source: §1 · pp. 1–2, §2 · pp. 2–3

1 Introduction

  • The controlled GPT-2 study crosses row-only versus row-plus-projection updates with the presence or absence of output-anchored writing, using shared starting conditions and 50 canonical training sentences.Three seeds vary optimization sampling on one initialization and fact set; anchor forward passes make compute unequal despite equal update counts.Source: §1 · p. 2
  • Row-only updates reproduce every trained canonical completion, while shared projection updates change question-form diagnostics and increase measured drift; anchoring lowers drift without changing canonical reproduction.These findings characterize tested interfaces rather than establishing universal prerequisites for writable memory.Source: §1 · p. 2
  • The production-memory studies provide complementary evidence from token-level outcomes, saved continuations, reformulations, and implementation diagnostics under their original protocols.They examine the boundaries of writing claims rather than defining a universal writability criterion.Source: §1 · p. 2

3 Background and problem formulation

Addressed memory rows are components in a readout pathway, not independently queriable facts. The controlled setting varies which parameters are updated and whether module residuals are anchored while separating canonical expression from question-form behavior and preservation.

3 Background and problem formulation

  • Addressed rows pass through shared projections and a hidden-state-dependent gate before contributing to the backbone, so a new formulation can alter addresses, gate state, or both.Row updates are local to selected parameters, but shared projection updates can change readout for rows outside the write set.Source: §3 · p. 3
  • The writing problem measures canonical completions, an untrained question form, unwritten-entity controls, and pre-write distribution changes as distinct aspects of writability.The preservation reference is the model after new modules are mounted; OAW constrains module residuals, while final-distribution drift and perplexity are measured separately.Source: §3 · p. 3

4 Writing protocol and controlled setting

  • Two memory modules are inserted after GPT-2 blocks 3 and 8, each with 24 heads, 400,000 rows per head, and 32 dimensions per row.Half the heads use causal 2-gram addressing and half use causal 3-gram addressing, and each layer has an independent table.Source: §4 · p. 3
  • The independent causal polynomial address uses token IDs and layer-specific initialization, with its ordered constants shared across layers and addressed rows deterministically enumerated from fixed training inputs.The implementation is not a bitwise reproduction of a production XOR scheme.Source: §4 · p. 4
  • Row-only updates train addressed embedding rows, whereas row-plus-projection updates additionally train shared key and value weights and biases, potentially affecting unaddressed rows.The eligible row sets contain 500,480 scalars; the additional shared projection surface contains 2,362,368 scalars.Source: §4 · p. 4
  • The objective combines causal cross-entropy over real next-token positions with an optional penalty matching module residuals to fixed pre-write references.The residual penalty uses two sampled anchor windows per step and λ=0 or 5 by arm; CE arms perform no anchor forward passes.Source: §4 · p. 4
  • All arms use shared write inputs and update counts, but OAW adds anchor-window computation and row-plus-projection trains additional shared parameters, so computational costs are not matched.Runs reconstruct the same pre-write model and share fact and anchor sampling streams across arms at each seed.Source: §4 · pp. 5–6, Figure 1 · p. 5, Table 1 · p. 6

5 Evaluation

Evaluation separates trained completions from untrained question forms, unwritten-entity controls, and preservation measurements. Scoring combines exact-match and lexical diagnostics with retained continuations, while audits clarify which addresses and fixed slices the measurements cover.

5 Evaluation

  • The fixed synthetic fact file writes items 0–49 and reserves items 50–99 as controls, with no subjects shared across splits but some target strings shared.The records are synthetic assertions rather than a benchmark of established world knowledge.Source: §5 · p. 6
  • Canonical evaluation uses a prefix of each training sentence, whereas the relation-template question form is not included in training.The expected continuation follows the canonical subject-and-relation prompt.Source: §5 · p. 6
  • Exact match scores the normalized full response, while prefix, substring, target-likelihood, and saved-continuation measures distinguish formatting failures from lexical matches.Substring matches alone do not establish a correct factual assertion.Source: §5 · p. 6
  • Four fifty-item sets evaluate canonical completions, untrained natural questions for written facts, unwritten-entity controls, and natural questions for those controls.The control targets are not world knowledge, and the natural-question set is not a set of new facts.Source: §5 · p. 6
  • Preservation uses fixed, disjoint anchor, development, and perplexity slices; only anchor inputs enter OAW optimization, while development and perplexity slices are reserved for measurement.The table caption specifies per-position development KL and perplexity from accumulated negative log-likelihood over effective targets.Source: §5 · p. 7, Table 2 · p. 7
  • At final prompt positions, all 24 head addresses belong to the written-row union for every canonical prompt but none of the natural-question prompts.Subject-disjoint controls are therefore not address-disjoint controls; unwritten-entity exact match is not a forgetting measure.Source: §5 · p. 7
  • Intrusion counts a continuation containing another written target string from the same relation, while own-target hits and intrusions can co-occur.These are non-complementary diagnostic events, not measures of semantic accuracy.Source: §5 · p. 7

6 Controlled GPT-2 results

Across the fixed starting model and training exposure, the controlled comparison separates trained fact expression from question-form behavior, drift, and unwritten-entity responses. Row-only writes reproduce trained completions, while question-form lexical hits and preservation measurements differ across update scopes and anchoring conditions.

6 Controlled GPT-2 results

  • 100% canonical exact match in every write seed for all four arms, compared with 0% before writing; row-only updates suffice at 2500 updates for these trained continuations.The result concerns 50 synthetic training targets and does not establish capacity or transfer to pretrained memory modules.Source: §6 · p. 7
  • 5.3 ± 1.2% boundary-aware target-string hits for both row-only arms, versus 28.7 ± 6.4% for RK_CE and 22.0 ± 2.0% for RK_OAW; natural-question exact match and prefix hit are zero in all twelve runs.The hits are content diagnostics, not semantic accuracy; some responses contain a target in a sentence while failing strict short-answer EM.Source: §6 · pp. 7–8, Figure 2 · p. 8, Table 3 · p. 8
  • 0.001663 to 0.000816 nats per position in row-only scope and 0.009504 to 0.003930 in row-plus-projection scope for mean development-slice KL with anchoring.Anchoring also lowers measured KL and perplexity in every matched seed, while natural-question target NLL increases; projection-scope target-string hits decrease.Source: §6 · pp. 8–9, Figure 2 · p. 8, Table 4 · p. 9
  • 29.3 ± 1.2% canonical exact match on unwritten synthetic controls in each row-only arm, versus 18.7 ± 2.3% for RK_CE and 18.7 ± 1.2% for RK_OAW; these are not forgetting scores.Canonical intrusion rates are 68.7%, 68.0%, 81.3%, and 81.3% across R_CE, R_OAW, RK_CE, and RK_OAW; natural-question intrusion rates are 4.0%, 4.0%, 31.3%, and 22.0%. These distinct lexical events cannot be subtracted into net accuracy or establish entity-specific retrieval.Source: Figure 2 · p. 8, §6 · p. 9, Table 5 · p. 9

7 Writing into production memory tables

The production-memory studies examine whether local writes change designated outputs, while separating token-level promotions from complete-answer success and transfer. Their archived records support bounded feasibility, with important limits on comparison, replay, and generalization.

7 Writing into production memory tables

  • These archived production records concern different protocols from the controlled GPT-2 comparison and do not test production-scale OAW.They analyze retained runs rather than fresh production-model replay.Source: §7 · pp. 9–10

7.1 Local writes and the measured target

  • The archived Qwen score ranks the supplied answer’s first tokenizer token, so a target-token top-1 outcome does not establish full-answer accuracy.The table’s New and Lost counts use initially non-top-1 and initially top-1 targets, respectively; trials are independent write–measure–restore runs.Source: §7.1 · p. 10, Table 6 · p. 10

7.2 Qwen: write feasibility and optimizer dependence

  • At 150 iterations on the same 24 prompts, Adam produced 13 additional target-token top-1 outcomes versus two with SPSA, but the comparison does not equalize computation.Different optimization-time representations also prevent attributing the difference solely to derivative access.Source: §7.2 · p. 10
  • 132 independently restored Qwen trials promoted all 93 initially non-top-1 target tokens and retained 39 already top-ranked tokens, demonstrating designated next-token changes rather than joint storage or transfer.The deepest promoted token began at rank 2,153; the trials do not measure joint storage capacity, sequential retention, or transfer to new questions.Source: §7.2 · p. 10

7.3 An 80-row write, reformulations, and controls

  • The saved 80-row gold-symbol write moved Au from rank 1 to rank 0, but the overlapping reformulation probes do not establish general question-answering success.The designed shared-row exclusion produced zero changes on four controls; the six probes do not test a zero-overlap reformulation, and another run changed three control distributions.Source: §7.3 · pp. 10–11

7.4 DeepSeek: mixed outcomes and implementation limits

  • Six DeepSeek SPSA trials yielded one new target-token promotion and one regression, supporting limited response perturbation rather than a cross-architecture success rate.The promoted Rio target was imposed, not evidence that the resulting answer was factually correct; gradient diagnostics require further implementation validation.Source: §7.4 · p. 11

8 Discussion and limitations

The discussion treats writability as several distinct outcomes: trained expression, expression under new questions, and behavior on untargeted inputs. It limits the conclusions to the tested settings and measurements.

8 Discussion and limitations

  • Local rows suffice for canonical reproduction, while shared projection training changes question-form diagnostics and increases measured output drift; OAW lowers drift without changing canonical exact match.OAW’s question-form effects depend on update scope and metric.Source: §8 · p. 11
  • Canonical success does not establish correct, entity-specific retrieval: lexical hits can misidentify the subject, and exact match is sensitive to response format.Semantic assessment would be needed to quantify correct, entity-specific retrieval.Source: §8 · p. 11
  • Production studies complement, but do not replicate, the controlled comparison: they use trained tables, different addressing and readouts, and mostly token-level objectives.Their matched Qwen sets do not establish compute-normalized superiority, and DeepSeek regressions and reformulation probes retain adverse outcomes.Source: §8 · p. 11
  • The controlled study fixes one backbone, initialization, synthetic fact set, and penalty coefficient; three seeds vary only optimization sampling, and preservation uses fixed slices of one WikiText stream.The RK intervention changes parameter count and cross-address sharing, OAW adds compute, and final-position analysis does not exclude earlier-position retrieval or hidden-state-dependent gating.Source: §8 · pp. 11–12
  • Independent initializations, additional preservation data, and matched replay controls could test reach; the studies do not establish universal writability conditions or scale independence.Production experiments do not test OAW or multi-hop composition, and pathway width, backbone scale, and prior training strength are not isolated.Source: §8 · p. 12

9 Reproducibility

The reproducibility section documents hashed experimental inputs and archived records while distinguishing artifact checks from recomputation. The conclusion frames the findings as bounded feasibility and recommends reporting writing and evaluation conditions together.

9 Reproducibility

  • The frozen protocol identifies the checkpoint, tokenizer, facts, slices, runner, and sampling streams by content hash and verifies the shared pre-write state and anchor reference before optimization.Per-seed results, prompt construction, scoring, and memory-disabled conversion checks are reported in appendices.Source: §9 · p. 12
  • A checksum-verified private bundle supports text-based rescoring and arithmetic checks, but recomputing likelihoods, output KL, or perplexity requires model states and forward passes.The stream was reconstructed from raw WikiText validation text, but historical processing, dependency versions, and synthetic-fact creation remain unresolved; hashes do not make private materials accessible.Source: §9 · p. 12
  • Production counts are recomputed from retained records rather than new inference, with saved replay outputs distinguished from fresh reproduction.The archived studies retain original result JSONs, code snapshots, continuations, and available write-vector metadata.Source: §9 · p. 12

10 Conclusion

  • In controlled GPT-2, addressed-row updates learn all trained canonical completions; projection updates change question-form diagnostics and increase drift, while residual anchoring lowers drift without changing canonical reproduction.The controlled findings concern newly initialized modules attached to a frozen backbone.Source: §10 · p. 12
  • Production writes changed selected Qwen and DeepSeek outputs, but the evidence does not establish complete-answer accuracy, simultaneous storage of all tested facts, or production-scale OAW superiority.The paper recommends reporting update surface, prediction target, transfer probes, and preservation measurements together; it makes no universal three-condition or scale-independence claim.Source: §10 · p. 12

Use of AI-assisted tools

The controlled runs reproduce trained canonical completions across all seeds, while question-form hits and preservation measurements vary by writing scope and anchoring. These diagnostics distinguish lexical matches, output drift, and behavior on unwritten entities.

Use of AI-assisted tools

  • OpenAI Codex and DeepSeek Harness were used during the research and manuscript preparation.The Harness used deepseek-v4.1-flash.Source: Use of AI-assisted tools · p. 12

A All runs and per-seed metrics

  • All runs reproduce every trained canonical completion, while target NLL still separates writing scopes after discrete scores saturate.The reported measurements are on fixed training targets across three sampling seeds.Source: A All runs and per-seed metrics · p. 13, Table 7 · p. 13
  • Projection updates raise natural-question target-string Hit over row-only updates; anchoring leaves row-only Hit unchanged and lowers projection-scope Hit.Short-answer and content diagnostics remain distinct, and Hit is not semantic accuracy.Source: A All runs and per-seed metrics · p. 13, Table 8 · p. 13
  • Unwritten canonical scores describe changes on untrained entities, not forgetting of previously mastered facts.The corresponding table reports boundary-aware target-string matches, not semantic accuracy.Source: A All runs and per-seed metrics · p. 14, Table 9 · p. 14
  • For unwritten natural questions, own-target Hit and intrusion match different answer sets and need not be mutually exclusive.Hit denotes a boundary-aware target-string match, not semantic accuracy.Source: A All runs and per-seed metrics · p. 14, Table 10 · p. 14
  • Anchoring lowers KL and perplexity within each writing scope in every matched seed, with conclusions limited to fixed recovered slices.KL and perplexity compare final output distributions with the common mounted pre-write state.Source: A All runs and per-seed metrics · p. 15, Table 11 · p. 15
  • Anchoring lowers both layers’ module-residual MSE, while wall-clock records show added work; equal update counts do not mean equal compute.The residual comparison uses a fixed pre-write reference.Source: A All runs and per-seed metrics · p. 15, Table 12 · p. 15
  • Control intrusion matches another written target of the same relation, while negative ΔNLL means a control’s assigned target became more likely.Neither measure establishes entity-specific knowledge, and intrusion rates cannot be subtracted into an accuracy.Source: A All runs and per-seed metrics · p. 16, Table 13 · p. 16

B Representative continuations

The representative continuations show that target-string presence and strict short-answer scoring can diverge. The accompanying diagnostics describe answer concentration and lexical hits without establishing entity-specific retrieval.

B Representative continuations

  • At index 2, row-only arms return a different place, while projection arms return the written target within a sentence; all four responses fail strict EM.The examples were selected by a fixed rule and illustrate format and content rather than overall accuracy.Source: Table 14 · p. 16, B Representative continuations · p. 16
  • At index 24, all four arms produce a continuation containing the correct target, fine porcelain, but fail short-answer EM.The scored continuation says Yewdale specializes in the production of fine porcelain.Source: B Representative continuations · p. 16

C Answer-distribution diagnostics

  • The relation-only frequency reference matches 19/50 written targets and 13/50 controls under exact match, prefix match, and boundary-aware Hit.It ignores the subject, and its fixed short-answer output is not fairly ranked against free-form generation or evidence of entity-specific retrieval.Source: C Answer-distribution diagnostics · pp. 16–17
  • Among 13 written facts with targets absent from the control pool, row-only runs have zero natural-question target-string hits and projection runs have 4/13.The three seeds query the same facts; target-string hits do not establish semantic accuracy or explain the writing pathway.Source: C Answer-distribution diagnostics · p. 17

D Address audit and constants

The address audit characterizes which rows are addressed at the final prompt position for this fixed prompt set, while the accompanying question templates and text rules define the evaluation forms. These audits are descriptive and do not establish a causal mechanism.

D Address audit and constants

  • At the final prompt position, all 24 head addresses in each layer belong to written training rows for canonical prompts, but none do for natural-question prompts.The audit covers the fixed set and does not rule out earlier-position access or hidden-state-dependent module access.Source: D Address audit and constants · p. 17
  • The audit enumerates rows using the training path’s tokenizer, addressing function, and buffer constants, without a model forward pass.It is a descriptive measurement, not evidence of a causal mechanism.Source: D Address audit and constants · p. 17
  • The 24 heads use fixed ordered constants: heads 0–11 have order 2, and heads 12–23 have order 3.The multipliers were drawn with a seeded CPU generator and held fixed; each layer uses its zero-based block index as the initial salt.Source: D Address audit and constants · p. 17, Table 15 · p. 17

E Question templates and normalization

  • Natural-question forms are fixed dataset templates applied to the same written and control entities, without added few-shot examples.Normalization and exact, prefix, and substring matches follow fixed text rules; substring matches alone do not establish a correct assertion.Source: E Question templates and normalization · p. 18

F Evaluation limits of short-answer scoring

The evaluation checks expose how short-answer scoring, numerical conversion checks, and archived production records delimit the reported evidence. Together they document tested inputs and saved cases without claiming universal identity or independently reproduced production runs.

F Evaluation limits of short-answer scoring

  • With the canonical fact supplied, the native-backbone control yields 2/50 exact matches, 5/50 prefix hits, and 49/50 boundary-aware target-string hits under the fixed question and stopping protocol.The example with target 1380 shows that added words can cause strict short-answer scoring to reject a continuation containing the target.Source: F Evaluation limits of short-answer… · p. 18

G Memory-disabled conversion checks

  • Memory-disabled conversion checks on fixed inputs meet the reported tolerances, but the initial maximum-logit-difference tolerance of 10^-4 was not met on checked float32 inputs.The tolerances are not a claim of universal numerical identity; zero float64 difference on a few segments does not establish universal identity.Source: G Memory-disabled conversion checks · pp. 18–19

H Recovered validation stream and artifact identity

  • A source audit exactly reconstructs the retained 247,290-token array by encoding consecutive one-million-character segments and reproduces every token ID.Whole-string encoding yields 247,289 tokens and does not match; the audit recovers content and a reconstruction procedure, not the original processing program.Source: H Recovered validation stream and… · p. 19
  • The twelve post-write state files remain on a private host, and file statistics establish their presence rather than verifying their contents or model outputs.Historical dependency versions and the synthetic-fact creation process remain unresolved, and no public repository is provided.Source: H Recovered validation stream and… · p. 19

I Production protocols and saved case records

  • In the matched 24-target Qwen comparison, gradient writing yields 24 post-write top-1 targets, while SPSA records 2 and 0 new promotions in its two sets of 12.Equal iteration counts do not remove procedural differences: Adam uses a floating-point additive delta, whereas SPSA evaluates two quantized perturbations per iteration; the comparison is not compute-normalized.Source: I Production protocols and saved case… · p. 19
  • The 80-row archived write has internally consistent saved metadata and controls, but its controls are design-informed locality checks rather than held-out controls.A separate specificity field is not a valid distribution-change metric, and the archived record does not prove an independently reproduced run.Source: I Production protocols and saved case… · pp. 19–20
  • The six saved reformulation/entity probes rank the fixed Au target; target rank, generated answer, and entity-specific correctness remain distinct.The four-token snippets do not provide full-answer or general-reasoning scores.Source: I Production protocols and saved case… · p. 20, Table 16 · p. 20
  • The six DeepSeek SPSA cases are intervention targets, not a correctness benchmark; the gradient file has different baselines and is not merged for optimizer ranking.The diagnostics do not establish a universal gate or kernel theorem.Source: I Production protocols and saved case… · p. 20, Table 17 · p. 20

J Prior training and exploratory controlled-bed results

The retained training-dose experiment and earlier GPT-2 records offer bounded completion diagnostics, not evidence of universal writing conditions or verified cross-bed causal effects. The section distinguishes writing, expression, and preservation while noting limits in the archived evaluations.

J Prior training and exploratory controlled-bed results

  • Prior-training, width, write-load, and preservation experiments on self-trained beds and earlier GPT-2 conversions remain distinct from the new-module experiment and do not establish scale-independent necessary conditions.Differences in initialization, token-key construction, exposure, or stacked objectives remain unresolved for cross-bed causal claims; replay/isolation runs that include OAW are not independent no-OAW baselines.Source: J Prior training and exploratory… · pp. 20–21
  • Fifteen saved results span five checkpoints on one 12-layer, width-768 pretraining trajectory, with three writing seeds per checkpoint.The bed was historically labelled 1B, while retained architecture code specifies approximately 779.4 million parameters, including 614.4 million memory-table parameters.Source: J Prior training and exploratory… · p. 21
  • Table 18 counts case-sensitive expected strings anywhere in the next 15 greedy tokens across three trained prompt forms for 50 facts, not held-out transfer or 150 independent facts.The three forms were explicitly included in the writing text, and seeds varied writing randomness rather than the pretraining trajectory.Source: J Prior training and exploratory… · p. 21, Table 18 · p. 21
  • Higher-dose endpoints showed stronger completion response in the retained trajectory, but its intermediate point was nonmonotone and does not establish a universal prior threshold.The observation motivates explicit testing of training state; it does not show that perplexity alone predicts writing or resolve initialization and implementation effects.Source: J Prior training and exploratory… · p. 21, Table 18 · p. 21
  • Earlier GPT-2 logs recorded 95.78 ± 0.77% substring-hit probes, while a separate replay batch recorded 94.67 ± 0.67%; neither verifies performance from the new experiment’s common start.The batches remain separate because a seed-42 file was overwritten, some added storage was insufficiently initialized, and defense comparisons used different mounted states.Source: J Prior training and exploratory… · p. 21
  • The original question-form substring count averaged 52%, and the cross-entity hit count averaged 46.44%, but saved aggregates cannot reconstruct full-answer EM.The 49-pair “multi-hop” set lacked a required entity chain and mismatched its two relations in 39 pairs, so its approximately 13% count is not evidence of multi-hop ability.Source: J Prior training and exploratory… · p. 21
Loading 2610.21553v1…