PaperlayerView exact revision on arXiv

Source-linked AI summary

Sequential knowledge editing breaks a model's ability to tell good evidence from bad, without costing it accuracy

Atul Anand

arXiv:2609.29587v1cs.AI

TL;DR

Standard editing evaluations test whether answers change, not whether models still know which retrieved evidence to trust on untouched facts. This paper measures paired arbitration and finds selective prediction degrades after sequential editing even when MMLU remains unchanged. The result identifies an editing-related evaluation blind spot rather than a loss of multiple-choice capability.

  • Problem

    Standard editing evaluations measure edited answers, paraphrases, and locality, but do not ask whether editing changes how a model weighs retrieved evidence on untouched facts.

    Source: §1 · p. 1

  • Method

    The paper defines a paired arbitration measurement for individual facts and contexts, using a forced two-candidate contrast to assess evidence choice.

    Source: §1 · p. 2, §3 · p. 2

  • Results

    At 1,000 edits, LoRA-gentle has MMLU 0.6275 and ∆AURC +0.1067, versus +0.0054 for the matched control.

    Source: §5 · p. 5

  • Takeaways & Limitations

    Standard editing evaluations can miss worsened evidence judgment because they ask what the model answers rather than how it chooses.

    Source: §10 · p. 8

    Per-fact prediction of retrieval failure works for LoRA but not MEMIT, and Mistral effect magnitude varies more than threefold across seeds.

    Source: §5.3 · p. 6, §9 · p. 8

Abstract

from arXiv · show

Knowledge editing is evaluated on whether the edited fact changed, whether paraphrases follow, and whether unrelated answers stayed put. A model can pass all three and still lose something none of them measures: the ability to decide, on facts that were never edited, which retrieved documents to believe. We score the log odds a model assigns to its remembered answer against the answer an injected passage asserts, before and after editing, holding the query, the passage and both candidate strings fixed. Our cleanest arm is a conservatively tuned LoRA: after 1,000 sequential edits on Qwen2.5-7B-Instruct it leaves MMLU unchanged to four decimal places, yet the spread of the arbitration quantity across untouched facts falls by 36%. Selective prediction degrades with it. Area under the risk-coverage curve rises by 0.107, against 0.005 for a norm-matched perturbation at the same MMLU, and error on the model's most confident quarter of arbitration decisions goes from 0.217 to 0.342. This is not capability loss. Sweeping random perturbation over five severities, damage bad enough to cut MMLU from 0.6275 to 0.3725 produces less harm (0.088) than MEMIT does at 0.6050 (0.102). The effect holds across three seeds, two model families, two datasets, two probe-disjointness criteria, three prompt templates and paraphrased queries. Layer ablation on saved weight deltas shows it is distributed: no single layer reproduces it, and removing any one recovers about half. Under retrieval with a frozen retriever, accuracy falls from 0.592 to 0.46. A secondary finding may matter more in practice. Three of five model and method pairings we ran collapse to chance MMLU at 1,000 sequential edits under published hyperparameters, while edit success stays at 1.00 and locality reads clean. Sequential-editing evaluations that never measure capability cannot see this.

1 Introduction

The paper asks whether sequential editing changes how a model weighs retrieved evidence against its own knowledge on facts that were never edited. It finds this arbitration can deteriorate even when multiple-choice accuracy is unchanged, exposing a gap in standard editing evaluations.

1 Introduction

  • Standard editing evaluations miss changes in evidence arbitration because they test answers, not how the model chooses which answer to trust.The target is arbitration on facts untouched by editing.Source: §1 · p. 1
  • After 1,000 sequential edits, the clearest arm preserved MMLU while selective-prediction error rose twentyfold relative to a matched random perturbation.The paper frames this as worse evidence judgment, not general capability loss.Source: §1 · pp. 1–2
  • The paper pairs measurements at the individual fact and context level, finding that the mean misses a collapse in spread rather than a shift in center.It also compares editing-specific damage with a damage-response curve, localizes effects using exact weight deltas, and connects the measurement to retrieval failure.Source: §1 · p. 2

2 Related work

This work distinguishes arbitration on untouched facts from prior studies of edited-fact robustness, calibration, and knowledge retention. It argues that these lines address different quantities and can yield compatible findings.

2 Related work

  • Context-Robust Editing asks whether an edited fact survives contradicting context [Context-Robust Editing, 2025], whereas this work studies context reliance on facts never edited.The paper characterizes its question as the reverse of that prior work.Source: §2 · p. 2
  • Edited-fact underconfidence [Underconfidence, 2025] and knowledge-focused noise or spectral-collapse findings [Noise Accumulation, 2025, Spectral Collapse, 2026] concern different quantities from external-evidence arbitration.The paper says the results are compatible and that its degradation controls show these effects can come apart.Source: §2 · p. 2
  • AlphaEdit’s null-space constraints intended to preserve unrelated knowledge do not imply stable arbitration; on Qwen2.5-7B, it reaches chance MMLU after 1,000 sequential edits [AlphaEdit, 2024].Source: §2 · p. 2

3 Measuring arbitration

The paper measures arbitration on untouched factual queries by comparing a remembered answer with an alternative under the same query and context. It separates evidence conditions and evaluates whether confidence rankings support selective prediction.

3 Measuring arbitration

  • The arbitration score contrasts the probabilities of an alternative and the model’s remembered answer under the same query and context; paired drift holds both candidate strings fixed before and after editing.The experimental design keeps edit subjects separate from probe subjects and replays the same retrieval at each checkpoint.Source: §3 · p. 2, Figure 1 · p. 3

3.1 Four evidence conditions, never collapsed

  • Four evidence conditions distinguish agreement with memory, a genuine update, a stale false claim, and an unsupported-authority claim; collapsing them would hide opposing effects.A model may become more receptive to true updates while degrading under adversarial text.Source: §3.1 · p. 3

3.2 Why the mean is the wrong statistic

  • At 1,000 LoRA edits, stale-false drift had mean −2.70, median −6.06, and A standard deviation fell from 12.08 to 5.56, so the mean understates the change.The heavy-tailed distribution lets facts moving in opposite directions cancel in the mean; the paper therefore reports dispersion ratio and AURC.Source: §3.2 · p. 3

3.3 Selective prediction

  • AURC tests whether |A| ranks decisions by correctness: the method sorts by |A|, sweeps coverage, and measures risk among answered queries.Unlike dispersion, AURC is unchanged by rescaling that preserves ranking; it changes when editing reorders confidence across facts.Source: §3.3 · pp. 3–4

4 Setup

The experiments test sequential editors on untouched probes across two model families and use controls to separate editing effects from generic damage. The control suite also exposes settings where editor-matched comparisons become uninformative.

4 Setup

  • The setup uses Qwen2.5-7B-Instruct and Mistral-7B-Instruct-v0.3, with CounterFact edits and ZsRE replication probes filtered for subject or relation disjointness.The primary Qwen filter retains 174 of 500 probes; the Mistral filter retains 167.Source: §4 · p. 4
  • Editors run in their natural regimes: LoRA edits one fact at a time, while MEMIT uses closed-form batches of 100.The second-moment sample count is reduced from 100,000 to 20,000, with both locate-then-edit methods using the same cached statistics.Source: §4 · p. 4
  • Seeds reorder the same 1,000 edits, which distinguishes order sensitivity from deterministic closed-form solves that otherwise repeat identical results.Source: §4 · p. 4

4.1 Controls

  • Controls include no edit, norm-matched noise, a five-severity damage sweep, shuffled targets, and confidence-matched temperature scaling.Shuffled targets reproduce the aggressive LoRA dispersion collapse at 100 edits (0.45 against 0.49), but become uninformative at 1,000 edits when the closed-form solve destroys the model.Source: §4.1 · p. 4
  • A norm-matched perturbation is informative only when its matched editing arm remains sane; the aggressive Mistral LoRA setting and its matched perturbation both fail.Source: §4.1 · p. 4
  • Figure 2 shows all three editors above the random-perturbation curve at comparable or better MMLU, while the norm-matched control stays near the floor.Even noise severe enough to reduce MMLU to 0.3725 causes less arbitration harm than MEMIT at 0.6050.Source: Figure 2 · p. 5

5 Results

The main result is that sequential editing can substantially impair arbitration even when benchmark capability is preserved. Damage-matched comparisons, a joint model, and seed replications support an editing-specific effect, while its magnitude varies on Mistral.

5.1 Dispersion separates from damage with intervals

  • MEMIT's dispersion-ratio difference versus damage-matched perturbation is −0.366 on stale-false evidence (95% cluster-bootstrap interval [−0.452, −0.276]); median drift is +1.69 versus +1.25.The dispersion differences are also −0.259 [−0.363, −0.151] on true updates and −0.247 [−0.328, −0.160] on adversarial text; on agreeing evidence the interval contains zero.Source: §5.1 · p. 5

5 Results

  • 0.6275 MMLU after 1,000 LoRA-gentle edits matches baseline, while ∆AURC is +0.1067 versus +0.0054 for the matched control.Table 1 reports the central comparison; AlphaEdit's MMLU of 0.2750 is chance for four-way multiple choice, so its ∆AURC is not an arbitration result.Source: §5 · p. 5, Table 1 · p. 6

5.2 A joint model over editors and conditions

  • In a fact-intercept mixed model, every editor separates from no-edit except the norm-matched control (+0.14, p = 0.60); seed is a fixed effect.The authors use a fixed rather than random seed effect because three levels cannot support a stable variance component.Source: §5.2 · p. 6

5.3 Generality

  • Across edit-order seeds, Qwen MEMIT dispersion is 0.622 and 0.628; Mistral LoRA-gentle ∆AURC ranges from +0.0467 to +0.1493, leaving its magnitude unpinned.The Mistral values differ by more than threefold, so the authors report existence and direction rather than a mean.Source: §5.3 · p. 6
  • AURC is more seed-stable than the median statistic, which swings from −6.06 to −12.09 for the same arm.Source: §5.3 · p. 6

6 The effect is distributed across layers

The dispersion collapse is distributed across MEMIT’s five updated matrices: no single layer reproduces it, while removing one layer recovers only about half. The saved-delta ablation shows the update must be read jointly.

6 The effect is distributed across layers

  • MEMIT’s saved deltas were reapplied to the base model in subsets, isolating weight-update effects without replaying edits.MEMIT writes to five mlp.down_proj matrices.Source: §6 · p. 6
  • 0.62 for the full update contrasts with 0.97–1.03 for any single layer alone; removing one layer leaves dispersion ratios at 0.79–0.85.The five individual contributions sum to roughly 0.03 of compression, versus 0.38 jointly.Source: §6 · p. 6
  • The joint effect is consistent with the norm-matched perturbation being inert; a layer-by-layer analysis would have missed it.Source: §6 · p. 6
  • Risk-coverage ordering also degrades after editing, with the gap widest at low coverage.The unedited model keeps error low when answering only its most confident queries.Source: Figure 3 · p. 7

7 End-to-end retrieval

Under frozen-retriever retrieval, accuracy falls after editing, and Stage-1 ∆AURC orders end-to-end accuracy across five checkpoints. Per-fact prediction is mixed: it works for LoRA-gentle but not MEMIT.

7 End-to-end retrieval

  • The frozen-retriever stress test retrieves all three competing documents for each query, using the same passage set for every checkpoint.The corpus includes a correct, stale, and adversarial document plus 600 distractors; retrieval quality is held fixed.Source: §7 · p. 7
  • Accuracy falls from 0.592 unedited to 0.454 for MEMIT and 0.471 for LoRA-gentle at 1,000 edits; AlphaEdit reaches 0.029.The model follows a misleading document on the majority of queries.Source: §7 · p. 7
  • Post-edit arbitration margin predicts which facts fail for LoRA-gentle (AUC 0.701 versus 0.523 for pre-edit confidence), but not for MEMIT (0.513 versus 0.603).The controlled measurement indicates which models fail end-to-end, not reliably which individual facts.Source: §7 · p. 7

8 Published configurations that destroy the model

At 1,000 sequential edits, three of five tested model-method pairings fall to chance MMLU despite edit success remaining 1.00 and locality appearing unremarkable. The dispersion response differs by evidence condition.

8 Published configurations that destroy the model

  • Three of five pairings fall to chance MMLU at 1,000 edits: AlphaEdit on Qwen (0.2750), MEMIT on Mistral (0.2425), and aggressive LoRA on Mistral (0.2125).MEMIT survives on Qwen at 0.6050; which pairing holds up appears idiosyncratic rather than principled.Source: §8 · p. 7
  • Both editors compress dispersion to roughly 0.6 in three conflicting evidence conditions, while matched noise stays at 1.0.For agreeing evidence, the editor-versus-noise interval contains zero.Source: Figure 4 · p. 8
  • Edit success remains 1.00 and neighborhood locality looks unremarkable across these runs.Evaluations that omit capability cannot distinguish a working editor from a destroyed model.Source: §8 · p. 8

9 Limitations

The limitations constrain per-fact prediction, seed interpretation, and the strength of replication evidence. Several design choices also narrow which editor comparisons the study can make.

9 Limitations

  • Per-fact retrieval-failure prediction works for LoRA but not MEMIT, so the deployment claim concerns models rather than individual facts.Source: §9 · p. 8
  • Mistral effect magnitude varies threefold across seeds, and Stage 4 is maximally conflicted by construction.Source: §9 · p. 8
  • The ZsRE panel has 66 probes after filtering, supporting agreement in direction rather than a second independent estimate.Source: §9 · p. 8
  • MEMIT’s second-moment sample count was reduced; AlphaEdit has no Mistral configuration, and the aggressive-LoRA artifact conclusion relies on a control that fails at higher edit load.Source: §9 · p. 8

10 Conclusion

After 1,000 edits, the conservatively tuned LoRA retains its multiple-choice performance but becomes substantially worse at judging retrieved evidence. The controlled measurement warns at the checkpoint level, not reliably for individual answers.

10 Conclusion

  • After 1,000 edits, the conservatively tuned LoRA answers multiple-choice questions as well as before but is substantially worse at judging which retrieved document to believe.The mixed-effects analysis models paired drift with a random intercept per fact.Source: §10 · p. 8, Table 2 · p. 9
  • The controlled measurement ranks checkpoints by retrieval degradation but does not reliably identify individual answers to distrust, and it does not beat pre-edit confidence for MEMIT.The mixed-effects fit uses 58,464 observations over 174 facts; seed is treated as a fixed effect because three levels are too few for a stable variance estimate.Source: §10 · pp. 8–9, Table 2 · p. 9

A Capability at every checkpoint

The appendix provides a complete capability table for all 57 Qwen checkpoints so readers can inspect whether capability remains constant.

A Capability at every checkpoint

Loading 2609.29587v1…