PaperlayerView exact revision on arXiv

Source-linked AI summary

Transformer-Patcher: One Mistake worth One Neuron

Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, Zhang Xiong

arXiv:2301.09785v1cs.CLcs.AI

TL;DR

Existing model-editing methods largely target one mistake, while deployed PLMs face continuously emerging and recurring errors. The paper formalizes Sequential Model Editing and introduces Transformer-Patcher, which adds trainable neurons to the last FFN layer. Across classification and generation tasks, it corrects up to thousands of mistakes while retaining overall model performance and achieves state-of-the-art SME performance.

  • Problem

    Existing Model Editing methods mostly fix one mistake, but deployed PLMs encounter different and recurring errors that require prompt, repeated correction.

  • Method

    Transformer-Patcher freezes original parameters and adds a few trainable neurons to the last FFN layer, training patches to target specific mistakes.

  • Results

    Transformer-Patcher achieves strong performance across five metrics on classification and generation tasks, correcting up to thousands of mistakes while retaining overall model performance.

  • Takeaways & Limitations

    The paper provides a sequential editing task, evaluation pipeline, and practical editor for repeatedly revising transformer-based language models.

Abstract

from arXiv · show

Large Transformer-based Pretrained Language Models (PLMs) dominate almost all Natural Language Processing (NLP) tasks. Nevertheless, they still make mistakes from time to time. For a model deployed in an industrial environment, fixing these mistakes quickly and robustly is vital to improve user experiences. Previous works formalize such problems as Model Editing (ME) and mostly focus on fixing one mistake. However, the one-mistake-fixing scenario is not an accurate abstraction of the real-world challenge. In the deployment of AI services, there are ever-emerging mistakes, and the same mistake may recur if not corrected in time. Thus a preferable solution is to rectify the mistakes as soon as they appear nonstop. Therefore, we extend the existing ME into Sequential Model Editing (SME) to help develop more practical editing methods. Our study shows that most current ME methods could yield unsatisfying results in this scenario. We then introduce Transformer-Patcher, a novel model editor that can shift the behavior of transformer-based models by simply adding and training a few neurons in the last Feed-Forward Network layer. Experimental results on both classification and generation tasks show that Transformer-Patcher can successively correct up to thousands of errors (Reliability) and generalize to their equivalent inputs (Generality) while retaining the model's accuracy on irrelevant inputs (Locality). Our method outperforms previous fine-tuning and HyperNetwork-based methods and achieves state-of-the-art performance for Sequential Model Editing (SME). The code is available at https://github.com/ZeroYuHuang/Transformer-Patcher.

1 INTRODUCTION

Transformer-Patcher frames model editing as a sequential problem because deployed language models encounter recurring mistakes that can affect many users. It proposes adding trainable patches to the final FFN layer, aiming to correct errors while preserving broader model behavior.

  • Motivation: Deployed PLMs still produce undesirable outputs, and uncorrected online QA errors can mislead many users.Manual caches can overrule problematic predictions but lack robustness and generality.
  • Motivation: Existing model-editing methods mostly address one mistake, whereas practical deployments require continuous correction of newly emerging and recurring errors.This motivates extending Model Editing into Sequential Model Editing.
  • Sequential Model Editing: Sequential Model Editing evaluates Reliability, Generality, and Locality while requiring a series of mistakes to be fixed as soon as they appear.The paper also introduces a standard experiment pipeline and five evaluation metrics.
  • Transformer-Patcher: Transformer-Patcher freezes original parameters and adds a handful of trainable neurons to the last FFN layer to revise problematic behavior at low editing cost.Activation and memory losses make patches respond to specific inputs while limiting effects on irrelevant examples.
  • Contributions: The paper contributes the SME task, its experiment pipeline and metrics, and Transformer-Patcher as a state-of-the-art sequential editor.Its experiments cover classification and generation tasks.

2 RELATED WORKS

The related work situates Sequential Model Editing among FFN memory interpretations, existing fine-tuning and HyperNetwork editors, and continual learning. SME differs from continual learning because it edits one example at a time within a single task.

  • Feed-forward Network: FFNs can be interpreted as key-value neural memories, with hidden dimensions representing memories and activations weighting retrieved values.The input acts as a query, the first layer as keys, and the second as values.
  • Model editors: Fine-tuning-based editors modify model parameters with extra losses to reduce overfitting or constrain changes to the original output space.Examples use parameter-distance or KL-divergence constraints.
  • Model editors: HyperNetwork-based editors learn to produce model updates, using approaches such as meta-learning or constrained optimization.These methods require additional training phases.
  • Continual Learning: SME resembles continual learning but handles one example at a time, with all examples drawn from the same task rather than different datasets and tasks.The distinct setting introduces challenges that general continual-learning methods may not properly address.

3 SEQUENTIAL MODEL EDITING PROBLEM

Sequential Model Editing repeatedly updates a model when it produces undesirable outputs on a stream of edit examples. Each post-edit model must preserve the correction, generalize it appropriately, and remain accurate on irrelevant inputs.

  • Task formulation: At step t, the editor receives the previous model f_t−1 and mistake pair (x_t, y_x_t), then outputs the revised model f_t.The process operates over a data stream and an initial model f_0.
  • Task formulation: The sequential setting requires edits to occur successively when the model produces undesirable outputs.The editor is applied to a stream of mistakes rather than a single isolated example.
  • Evaluation properties: After every edit, Reliability requires the post-edit model to output the desired prediction.This property directly evaluates whether the current mistake has been corrected.
  • Evaluation properties: Generality requires the corrected behavior to extend from an edit example to equivalent inputs, including rephrased sentences.Equivalent inputs share the desired label with the edit example.
  • Evaluation properties: Locality requires the edit to remain precise so the post-edit model stays accurate on irrelevant examples.Past edits must also remain undisturbed in the SME setting.

4 TRANSFORMER-PATCHER

Transformer-Patcher edits transformer behavior by adding trainable neurons to the last FFN layer while preserving original parameters. Its training objectives make patches respond to mistakes, produce target outputs, and avoid irrelevant inputs.

  • Patch design: Transformer-Patcher adds one trainable patch neuron to the last FFN layer for each mistake while freezing all original parameters.Classification uses one patch, whereas auto-regressive generation uses one patch per wrongly generated token under teacher forcing.
  • Patch design: A patch is an extra key-value neuron whose activation depends on the input query matching its learned key and bias.The patch contributes a bias term ap · vp to the last-layer output when activated.
  • Patch training: The activation loss maximizes the patch response for the mistaken query, approximating the requirement that its key and bias activate that input.The approximation uses the condition qe · kp + bp > 0 for ReLU or GeLU activations.
  • Patch training: The edit loss trains the activated patch to produce the desired output using the task’s original loss function.Because the patch is added in the last layer, its output adjustment is not canceled by later model components.
  • Patch training: The memory loss suppresses patch activation on queries retained from previously seen examples, supporting locality on irrelevant inputs.Its second term separates the mistaken example’s activation from irrelevant examples by a margin γ, and the total patch loss combines edit, activation, and memory losses.

5 EXPERIMENTS

The experiments evaluate Sequential Model Editing on classification and generation tasks using authentic mistakes, multi-step edits, and metrics for reliability, generality, and locality. Transformer-Patcher performs strongly across tasks, scales to thousands of edits, and benefits from memory loss and last-layer patching.

  • Experimental Settings and Evaluation Metrics: The SME pipeline uses authentic model mistakes and sequentially edits an initial model while evaluating reliability, generality, and locality.Its metrics include SR, GR, ER, TrainR, and TestR.
  • Experimental Settings and Evaluation Metrics: The evaluation covers fact-checking on FEVER with BERT and closed-book question answering on zsRE with BART.Both classification and auto-regressive generation tasks are included.
  • Experimental Results: Transformer-Patcher achieves SR≈1, ER≈1, TrainR≈1, and TestR≈1 across the two tasks while requiring 7.1 seconds per FC edit and 18.9 seconds per QA edit.Reducing memory examples to 10,000 lowers the reported edit times to 4.7 and 12.4 seconds.
  • Experimental Results: Transformer-Patcher effectively corrects up to thousands of mistakes while retaining overall model performance, and increases model size by 1.4% for FC and 4.5% for QA.The authors suggest online patching followed by later fine-tuning to remove accumulated patches.
  • Analyses: MEND and KE work in the first few edits but soon fail because their HyperNetworks are coupled to the original model parameters.Retraining the HyperNetworks after each edit is reported to require unacceptable hours of training.
  • Analyses: Memory loss is critical: removing it severely harms overall performance, while KL divergence only partially alleviates the problem and remains unsatisfying on QA.The main contribution comes from lm1, with lm2 providing additional improvement.
  • Analyses: Patching the last decoder layer improves efficiency and can improve generality, with GR increasing from 0.74 at layer 2 to 0.81 at layer 5.Bottom-layer patches cannot make effective edits, while last-layer computation allows previous-layer results to be cached and reused.
  • Analyses: Reducing memory size from 40,000 to 5,000 examples causes only slight changes in SR, ER, TrainR, TestR, and GR.The method is therefore reported as not very sensitive to memory-set size.

6 CONCLUSION

The paper formalizes Sequential Model Editing and introduces Transformer-Patcher for continuously editing transformer-based language models. Experiments show editing up to a thousand times while addressing serious model mistakes.

  • Transformer-Patcher edits transformer-based language models sequentially and supports up to a thousand continuous edits.Experiments cover classification and autoregressive generation tasks.
  • Sequential Model Editing provides an experiment pipeline and evaluation metrics for assessing repeated model edits.
  • The method targets serious errors such as biased predictions and hate speech.

A MULTIPLE NEURON PATCHING

For examples containing multiple generation mistakes, Transformer-Patcher adds multiple patches to the last FFN layer and trains them to activate selectively for corresponding queries.

  • Autoregressive generation mistakes can be assigned to separate queries because the FFN is position-wise.An example with n mistakes is handled through n corresponding queries.
  • Transformer-Patcher adds n patches to the last FFN layer to handle n mistakes in an example.The patched FFN combines patch keys, values, biases, and activations with its original computation.
  • Each patch is constrained to activate on its corresponding query during activation-loss training.
  • The activation vector A selects diagonal patch activations, and activation loss penalizes insufficient activation using hyper-parameter ka.
  • For memory loss, each patch is restricted to have lower activation on memory queries than on its corresponding query.
  • Patch keys are initialized from normalized related queries, with initialization chosen so the initial activation value is 1.

B EXPERIMENTAL DETAILS

The experiments evaluate sequential editing on FEVER fact-checking and zsRE question answering using repeated edit folders, trained initial models, baselines, and specified memory procedures.

  • Data splits: FEVER is split into training, validation, and edit subsets, with 10,496 instances assigned to the edit set.
  • Data splits: zsRE is filtered and split into 5,317 edit examples, 15,982 validation examples, and 24,051 test examples.
  • Data splits: The edit set is divided into 20 folders, and memory-dependent methods use a 40,000-example memory pool updated during editing.
  • Initial models training: Initial models are BERT-base for fact-checking and BART-base for question answering, trained by maximizing model likelihood.
  • Transformer-Patcher training details: Transformer-Patcher adds one patch per fact-checking edit and up to five patches per question-answering edit under teacher forcing.
  • Baseline implementation details: Fine-tuning baselines use learning rate 1e-5 with Adam until the mistaken example is corrected, while KL constraints sample 512 memory examples.
  • Baseline implementation details: SERA is implemented as a SERAC variant that directly returns the cached edit label for both fact-checking and question answering.
  • Environment details: SME experiments run 20 times on separate edit folders using 8 NVIDIA Tesla V100 GPUs.

C EXTRA EXPERIMENT RESULTS

Additional experiments examine locality during editing, variability, patch activations, fixed-memory behavior, and scaling to thousands of edits. Transformer-Patcher generally preserves performance and remains robust under these settings.

  • Variation of locality with the number of edits: As editing continues, other baselines accumulate model damage, whereas Transformer-Patcher does not.This comparison concerns locality on the QA task.
  • Standard deviation of experiment results: Transformer-Patcher achieves the smallest standard deviation on ER, TrainR, and TestR.
  • Statistics of activation values of different patches: Patch activation statistics distinguish edit, past-edit, and randomly sampled mistakes, with GeLU allowing both positive and negative activations to activate patches.
  • Scale up to thousands of edits: Using all edit data as one stream, Transformer-Patcher effectively corrects up to thousands of mistakes while retaining overall model performance.
  • Statistics of activation values of different patches: KL Patch has the lowest activation value on edit queries for both tasks, corresponding to lower QA SR.
  • Editing results with fixed memory set: With a fixed memory set, Transformer-Patcher shows a slight ER decline and slight TrainR rise, supporting robustness.
  • Editing results with fixed memory set: Transformer-Patcher can apply hundreds of thousands of memory vectors in one batch with minimal GPU memory and computation resources.
  • Contradictory of lower E and lower TestR: Lower E and lower TestR may reflect edit-test distribution gaps or slight overfitting, so metrics should be considered comprehensively.
Loading 2301.09785v1…