PaperlayerView exact revision on arXiv

Source-linked AI summary

Memory-Based Model Editing at Scale

Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, Chelsea Finn

arXiv:2206.06520v1cs.AIcs.CL

TL;DR

Model editors must make targeted post-deployment updates while preserving behavior on unrelated inputs, but existing methods struggle with edit scope and repeated edits. SERAC uses an explicit edit memory with learned scope and counterfactual components, and the paper evaluates it on challenging question-answering, fact-checking, and dialogue tasks. SERAC consistently outperforms prior approaches across these problems, while its memory growth and training-data requirements remain limitations.

  • Problem

    Existing model editors struggle to model an edit’s intended scope accurately and often fail after many edits, limiting reliable post-deployment updates.

  • Method

    SERAC stores edits in an external memory and uses a scope classifier and counterfactual model to modulate a black-box base model’s predictions.

  • Results

    SERAC consistently outperforms existing model-editing approaches by a substantial margin across challenging question-answering, fact-checking, and dialogue-generation problems.

  • Takeaways & Limitations

    SERAC supports effective multi-editing, complex edit scopes, and natural-language edit specifications without requiring base-model parameters or gradients during editing.

  • Takeaways & Limitations

    SERAC requires an edit dataset for training, may need resource-intensive auxiliary models, and can accumulate an unbounded edit memory during continual editing.

Abstract

from arXiv · show

Even the largest neural networks make errors, and once-correct predictions can become invalid as the world changes. Model editors make local updates to the behavior of base (pre-trained) models to inject updated knowledge or correct undesirable behaviors. Existing model editors have shown promise, but also suffer from insufficient expressiveness: they struggle to accurately model an edit's intended scope (examples affected by the edit), leading to inaccurate predictions for test inputs loosely related to the edit, and they often fail altogether after many edits. As a higher-capacity alternative, we propose Semi-Parametric Editing with a Retrieval-Augmented Counterfactual Model (SERAC), which stores edits in an explicit memory and learns to reason over them to modulate the base model's predictions as needed. To enable more rigorous evaluation of model editors, we introduce three challenging language model editing problems based on question answering, fact-checking, and dialogue generation. We find that only SERAC achieves high performance on all three problems, consistently outperforming existing approaches to model editing by a significant margin. Code, data, and additional project information will be made available at https://sites.google.com/view/serac-editing.

1. Introduction

Model editing aims to update a deployed model locally and efficiently, but existing parameter-based editors struggle to preserve edit scope and handle many edits. SERAC addresses these limitations with an explicit edit memory, scope classifier, and counterfactual model, and is evaluated on three challenging tasks.

  • Model editing seeks fast, data-efficient updates to a small region of a pretrained model’s behavior without damaging other inputs.
  • Existing learnable editors produce updates that fail to discriminate entailed from non-entailed facts and cannot handle large numbers of edits.
  • SERAC stores user-provided edit descriptors in an explicit cache instead of modifying the base model’s parameters.
  • A scope classifier determines whether a test input matches a cached edit, while a counterfactual model predicts the edited output when it does.
  • The evaluation introduces challenging editing problems based on question answering, fact-checking, and dialogue generation.
  • SERAC consistently outperforms prior approaches by a substantial margin on the three most difficult problems.

2. The Model Editing Problem

The model editing problem is to alter predictions within an edit’s intended scope while preserving behavior outside it. SERAC formalizes scope using in-scope and out-of-scope examples, including difficult entailment and semantic-neighborhood cases.

  • An edit descriptor specifies a desired change to a base model, producing an edited model whose behavior differs on selected inputs.
  • Edit scoping: An edit’s scope is the set of inputs whose true labels are affected, requiring altered in-scope predictions and unchanged out-of-scope predictions.
  • Edit scoping: Hard in-scope examples require non-trivial reasoning from the edit, while hard out-of-scope examples are semantically close but should remain unchanged.
  • The benchmark covers basic rephrases, logical entailments, difficult fact-based reasoning, and dialogue sentiment changes from natural-language behavior descriptions.
  • Edit scoping: For multiple edits, the intended scope is defined as the union of the individual edit scopes.

3. Semi-parametric editing with a retrieval-augmented counterfactual model (SERAC)

SERAC is a semi-parametric editor that preserves the base model while storing edits explicitly and using separate learned components to determine when and how predictions should change. It retrieves relevant edits and delegates scoped prediction changes to a counterfactual model, reducing interference between edits.

  • SERAC does not modify base-model parameters; it stores edit descriptors in an explicit cache and wraps the base model.
  • SERAC uses an explicit edit cache, a scope classifier, and a counterfactual model that overrides the base model when necessary.
  • The scope classifier estimates whether a new input falls within each cached edit’s scope, while the counterfactual model predicts the desired output under that edit.
  • SERAC limits retrieval to the most relevant edit example, using the base model when no edit applies, which reduces interference between edits.
  • SERAC’s scope classifier primarily uses fixed-length input and edit embeddings with negative squared Euclidean distance for computational efficiency.
  • The scope classifier and counterfactual model are trained separately with supervised objectives using in-scope and out-of-scope examples.

4. Datasets & Evaluation

The evaluation introduces challenging question-answering, fact-checking, and conversational-sentiment editing settings, measuring both desired in-scope changes and preservation of out-of-scope behavior. The benchmark applies ten simultaneous edits and reports SERAC as the only method producing meaningful edits across all problems.

  • The evaluation covers question answering, fact-checking, and conversational dialogue, including harder in-scope and out-of-scope examples.
  • QA & QA-hard: QA-hard tests logically entailed or true/false questions whose labels differ from the edit example, while hard out-of-scope inputs are semantically similar to the edited input.
  • FC: The FC setting updates an out-of-date fact-checking model using evidence-claim examples, with contradictory evidence producing semantically related hard out-of-scope claims.
  • ConvSent: ConvSent edits dialogue sentiment from arbitrary topic-and-sentiment descriptions rather than labeled input-output pairs, while preserving generations on other topics.
  • Metrics: Edit success measures agreement with desired in-scope behavior, whereas drawdown measures disagreement between pre-edit and post-edit behavior on out-of-scope inputs.
  • Results: Ten simultaneous edits are applied to every problem, and SERAC is the only method reported to produce meaningful edits across all evaluation settings.

5. Experiments

Experiments evaluate SERAC and competing editors across challenging editing tasks, multiple simultaneous edits, component behavior, architecture, reuse across base models, and computational costs. SERAC generally maintains stronger editing performance, especially as edit counts increase and on difficult fact-checking and dialogue settings, while exposing scope-classification and resource trade-offs.

  • Model Editing Benchmarking: SERAC consistently outperforms prior editors across QA, QA-hard, fact-checking, and ConvSent, with especially strong gains on the most difficult tasks.For QA-hard, SERAC improves over other approaches; for FC and ConvSent, competing methods are near random or random while SERAC performs substantially better.
  • Model Editing Benchmarking: SERAC shows almost no degradation when applying 75 edits, whereas drawdown exceeds edit success for ENN and MEND at that edit count.Training ENN and MEND with 10 rather than 1 edit does not reliably improve their performance at 75 test edits.
  • Further Empirical Analysis of SERAC: SERAC’s component analysis identifies scope estimation as the primary QA-hard challenge and hard out-of-scope classification as the FC bottleneck.Hard examples account for most classifier errors, while low out-of-scope counterfactual accuracy is usually avoided when inputs are correctly routed to the base model.
  • Further Empirical Analysis of SERAC: Cross-attention improves SERAC performance over dense embeddings, while increasing the classifier backbone size provides relatively little improvement.The experiments compare Embed and Cross classifiers with distilBERT and BERT-base backbones on QA-hard and FC using 10 edits.
  • Further Empirical Analysis of SERAC: SERAC editors transfer across T5 base-model sizes with near-identical edit success and drawdown, reflecting separation between the editor and base model.The reported comparison evaluates previously trained SERAC editors on multiple T5 base models.
  • Further Empirical Analysis of SERAC: SERAC’s edit memory uses about 3KB per edit, while most additional memory is a fixed cost from the classifier and counterfactual model.Memory overhead is relatively small for QA but larger for FC and ConvSent, where auxiliary models are similar in size to the base model.

6. Related Work

Prior model-editing work includes parameter fine-tuning, meta-learned adaptation, external gradient editors, neuron-level edits, and memory mechanisms. SERAC differs by explicitly storing user-provided edit descriptors and retrieving them semi-parametrically.

  • Constrained fine-tuning updates pretrained parameters using new examples.
  • Meta-learning methods use special pre-training objectives to enable rapid, targeted fine-tuning for edits.
  • External learned editors modify fine-tuning gradients without changing the base model that processes edits.
  • Some approaches attribute knowledge to particular neurons and manually edit their activations.
  • Unlike earlier memory-augmented neural systems, SERAC directly stores user-provided edit descriptors and retrieves them semi-parametrically.

7. Discussion

SERAC stores edits externally rather than in model parameters and is presented as a more practical editor across difficult editing conditions. Its limitations include training-data dependence, potentially costly architectures, unbounded continual-edit memory, and misuse risks.

  • SERAC stores model edits in external memory rather than directly modifying model parameters.
  • SERAC enables more effective edits with multiple edits, complex edit scopes, and natural-language edit specifications.
  • SERAC does not require base-model access during editor training or gradients during editing, and one trained editor can edit models with different architectures.
  • SERAC requires a dataset of edits for training its scope classifier and counterfactual model.
  • Continual editing can make the edit memory grow without bound, while some settings may require more resource-intensive architectures.
  • Model editors may be misused to craft agents that amplify particular viewpoints.

A. Additional Sentiment Editing Example and Broader Impacts

A vaccine-sentiment editing example shows SERAC changing BlenderBot-90M’s responses from nearly always negative to consistently positive and on-topic. The example also underscores model editing’s dual-use governance risks.

  • SERAC changes BlenderBot-90M responses about vaccines from nearly always negative sentiment to consistently positive, on-topic responses.
  • Editing dialogue models to reduce vaccine hesitancy may support public health, but general opinion modulation has broader governance implications.
  • Adversaries could reuse SERAC with essentially zero configuration to edit newly open-sourced dialogue models.
  • The authors characterize model editing as dual-use and call for monitoring how editors are distributed and deployed.

B. Baselines

The baselines include gradient-based parameter-editing methods and the gradient-free Cache+lookup method. Their implementations differ in which transformer layers are edited, how updates are learned, and how cached representations trigger edits.

  • Gradient-based methods adapt fully connected layers in the final transformer blocks, with the number of blocks depending on model architecture.
  • Fine-tuning minimizes negative log-likelihood on edit samples using Adam with model-specific learning rates.
  • Cache+lookup stores hidden representations of edit samples and their labels in an external memory.
  • Cache+lookup returns the nearest stored edit label when representation distance falls below a threshold; otherwise it uses the pretrained model’s prediction.
  • Cache+lookup uses δ = 2.75 for question answering and δ = 4 for fact checking.
  • Editable Neural Networks train a subset of parameters with a bi-level optimization objective, while MEND predicts a rank-1 decomposition of a fine-tuning gradient.

C. SERAC Implementation Details

SERAC experiments use publicly available Hugging Face implementations and checkpoints, with DistilBERT as the classifier and task-specific counterfactual models.

  • All experiments use publicly available Hugging Face implementations and checkpoints.
  • The SERAC scope classifier uses distilbert-base-cased across models and experimental settings.
  • Counterfactual models vary by task: T5-small for question answering, BERT-base-uncased for fact-checking, and BlenderBot small-90M for conversational sentiment modulation.
  • All scope-classifier and counterfactual models are trained with Adam at a learning rate of 1 × 10−5.

D.1. QA-hard

The QA-hard data-generation pipeline constructs entailed, true/false, and hard-negative questions using automated converters, nearest-neighbor sampling, and BlenderBot-based conversational prompts.

  • Question generation: Entailed questions are generated by passing each question as both the question and context to an entailment-question generator.The resulting questions are typically interpretable but not always grammatically correct.
  • Conversational prompts: Table 7 presents pre-edit and post-edit BlenderBot-90M samples about vaccines, a topic absent from editor training and validation sets.
  • Conversational prompts: Table 8 defines ConvSent prompt templates by combining SENTIMENT and POSITION values with entities sampled from zsRE.
  • Question generation: True/false questions are produced with a rule-based converter, using the edited question-answer pair for true statements and alternative answers for false statements.
  • Hard negatives: Hard negatives are sampled uniformly from the top 100 nearest test examples while excluding the top 50 neighbors.Nearest neighbors are computed with all-MiniLM-L6-v2 embeddings to avoid true positives and rephrases.
  • Conversational prompts: Conversational sentiment completions use a 3-billion-parameter BlenderBot with manually generated prompts filled from zsRE and GPT-3 entities.The 15,000 zsRE entities were randomly selected from entries beginning with alphabetic characters.
Loading 2206.06520v1…