Modifying Memories in Transformer Models

Chen Zhu; Ankit Singh Rawat; Manzil Zaheer; Srinadh Bhojanapalli; Daliang Li; Felix Yu; Sanjiv Kumar

Source-linked AI summary

Modifying Memories in Transformer Models

Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, Sanjiv Kumar

arXiv:2012.00363v1Dec 1, 2020cs.CLcs.LG

TL;DR

Transformer models memorize factual knowledge in distributed parameters, but selectively changing old facts without harming unmodified knowledge remains difficult. The paper introduces a benchmark and constrained fine-tuning approaches for this task, finding that layer-specific updates are effective and explicit memory does not necessarily make modification easier.

Problem
The paper addresses the limited ability to modify specific memorized facts in Transformer models while preserving performance on unmodified facts.
Method
The paper benchmarks knowledge-modification methods, formulates the task as constrained optimization, and studies constrained fine-tuning across Transformer architectures and layers.
Results
Constrained layer-wise fine-tuning is effective, while models with explicit memory modules are not necessarily easier to modify than implicitly memorizing Transformers.
Takeaways & Limitations
The study identifies Transformer components and training factors relevant to modifying selected factual knowledge while preserving unmodified knowledge.
Takeaways & Limitations
Constrained single-block modification degrades as the number of modified facts increases, causing larger accuracy losses on unmodified facts.

Abstract

from arXiv · show

Large Transformer models have achieved impressive performance in many natural language tasks. In particular, Transformer based language models have been shown to have great capabilities in encoding factual knowledge in their vast amount of parameters. While the tasks of improving the memorization and generalization of Transformers have been widely studied, it is not well known how to make transformers forget specific old facts and memorize new ones. In this paper, we propose a new task of \emph{explicitly modifying specific factual knowledge in Transformer models while ensuring the model performance does not degrade on the unmodified facts}. This task is useful in many scenarios, such as updating stale knowledge, protecting privacy, and eliminating unintended biases stored in the models. We benchmarked several approaches that provide natural baseline performances on this task. This leads to the discovery of key components of a Transformer model that are especially effective for knowledge modifications. The work also provides insights into the role that different training phases (such as pretraining and fine-tuning) play towards memorization and knowledge modification.

1 Introduction

Transformer models implicitly memorize factual knowledge but lack databases’ efficient editability. The paper introduces a benchmark and constrained methods for changing selected facts while preserving unmodified knowledge.

Motivation: Transformers implicitly store factual knowledge in parameters, making targeted updates less direct than in conventional databases.Conventional databases explicitly store knowledge, whereas Transformer knowledge is distributed across model parameters.
Task: The paper defines knowledge modification as changing selected facts while preventing performance degradation on unmodified facts.The task targets changing stale or user-specified knowledge without disturbing the remaining factual knowledge.
Contributions: The authors introduce a benchmark for modifying factual knowledge while preserving performance on unmodified facts.The benchmark evaluates whether candidate methods achieve both desired modification and preservation.
Contributions: Knowledge modification is formulated as constrained optimization with a constraint on loss for unmodified facts.The paper also explores baseline methods that approximately enforce this constraint.
Findings: Constrained layer-wise fine-tuning is reported as a simple, effective method for modifying memorized knowledge.The approach focuses updates on selected Transformer layers rather than necessarily updating the whole model.
Findings: Modification is not necessarily easier in models with explicit memory modules than in Transformers relying on implicit memorization.This finding challenges the assumption that explicit memory automatically simplifies factual editing.

2 Related Works

Prior work established that language models can serve as sources of factual knowledge and explored augmenting them with explicit memories. Knowledge modification differs from continual learning because it targets selected stored facts while preserving the rest.

Language models as factual knowledge sources: Probing studies found that pretrained Transformer models contain factual knowledge and can perform on open-domain factual knowledge tasks.LAMA converted facts and question-answer pairs into cloze sentences, while later work probed T5 models.
Language models as factual knowledge sources: Some approaches combine language models with structured knowledge bases to complement knowledge learned from unstructured text.Related work includes retrofitting representations and training models that leverage both curated knowledge and corpora.
Memory augmented models: Memory-augmented Transformers add explicit long-term or symbolic memory modules for storing entity representations or factual triples.FaE was motivated partly by the prospect of updating knowledge directly in explicit symbolic memory.
Memory augmented models: FaE can inject new facts but performs poorly when modifying facts already encountered during training because implicit and explicit knowledge can conflict.This motivates studying modification of knowledge retained in the underlying Transformer parameters.
Memory modification vs. continual learning: Continual learning preserves prior-task performance while learning new tasks, often without access to earlier data.Memory modification similarly seeks efficient prediction updates and preservation, but focuses on modifying selected knowledge.

3 Modifying implicit factual knowledge of Transformer models

The paper frames editing implicit factual knowledge as changing selected facts in distributed Transformer weights while preserving the rest. It compares natural baselines with constrained fine-tuning, including layer-specific updates.

Task formulation: Knowledge modification changes a selected subset of facts stored implicitly in model weights while preserving the remaining knowledge.Because facts are distributed across many weights and weights affect many facts, selective modification is challenging.
Task formulation: The formal objective updates facts S to new facts M, producing θ_new that retains performance on F\S.The paper illustrates this with updating the holder of a marathon world record while keeping other knowledge intact.
Baseline approaches: Retraining on revised pretraining and fine-tuning data is reliable in principle but impractical for small knowledge edits.Finding and updating facts in unstructured corpora is difficult, and retraining from scratch is expensive.
Baseline approaches: Fine-tuning only on modified-fact evidence can achieve high modified-fact accuracy but may severely degrade unmodified-fact knowledge.The reported causes are overfitting and catastrophic forgetting.
Constrained fine-tuning: The proposed constrained strategy fine-tunes on modified facts while constraining parameter changes to minimize interference with unmodified facts.Its complexity scales only with the number of modifications and it is reported to preserve unmodified knowledge effectively.
Constrained fine-tuning: The constraint approximates loss preservation around θ0, using a parameter-space norm and projected gradient descent.The experiments found ℓ∞ more stable than ℓ2, while a Fisher-information alternative is more costly.
Constrained fine-tuning: The constraint parameter δ governs the trade-off between changing modified facts and preserving unmodified-fact accuracy.Very small δ limits modification, whereas very large δ approaches unconstrained fine-tuning and harms unmodified accuracy.
Fine-tuning specific Transformer blocks: Fine-tuning initial and final Transformer blocks improves adaptation and preservation, outperforming whole-network updates in the reported experiments.The paper notes that initial layers sometimes perform better, despite common emphasis on top-layer fine-tuning.

4 Experiments

Experiments introduce modified-fact benchmarks from T-REx and zsRE, then evaluate constrained and unconstrained updates across Transformer models, training states, and parameter subsets. Results show constrained updates reduce forgetting, while the best layer and achievable trade-off depend on model state, modification size, and architecture.

Datasets and benchmarks: The benchmark modifies a small subset of T-REx or zsRE facts by changing their objects and consistently updating their supporting evidences.Each fact is represented by subject–relation–object triples with multiple masked or cloze-style evidences.
Datasets and benchmarks: Performance averages accuracy on modified facts, AM, and unmodified facts, AF\S, to measure their trade-off.The constraint strength δ is selected to maximize their average accuracy.
Results: Constrained finetuning on modified facts with an ℓ∞ weight constraint works better than natural strategies such as mixed modified and unmodified batches.The experiments compare BERT, ALBERT, and FaE across several modification settings.
Results: Unconstrained updates can raise accuracy on modified facts but cause catastrophic forgetting of unmodified facts, including after updating only one BERT-Base layer.Pretraining and finetuning improve original-fact accuracy before modification, but unconstrained FTM degrades it substantially.
Results: The best performances may come from modifying specific Transformer layers rather than the entire model, and the preferred layer changes with model state and modification count.In FT+FTM, the best block shifts from later to earlier blocks as the number of modified facts increases.
Results: With constrained single-block updates, performance degrades as more facts are modified, with larger losses on unmodified facts.The authors attribute this pattern to limited capacity when only one layer is modified; ALBERT cannot use this strategy because its layer weights are shared.
Results: For FaE, modifying symbolic links alone preserves unmodified-fact accuracy at 60.38% but reaches only 46.88% on modified facts, while finetuning additional weights improves the trade-off.Finetuning the layers directly mapping symbolic memory to predictions gives the best reported trade-off among FaE components.

5 Conclusion

The paper introduces factual-knowledge modification for Transformer models and benchmarks constrained updates that preserve unmodified knowledge. Evaluations cover implicit and symbolic memory, while results show modification remains necessary even with symbolic memory and becomes harder as more facts change.

Two benchmarks evaluate modifying factual knowledge while preserving performance on unmodified facts.
~50% accuracy is achieved on modified facts when minibatches are randomly sampled from DF′ after fine-tuning pretrained BERT-Base.
Modifying Transformer parameters remains necessary for networks with symbolic memory.
A comprehensive mechanism for consistently modifying both implicit and explicit knowledge remains future work.

A Dataset details

The benchmarks construct modification datasets from T-REx and zsRE, pairing original facts with altered objects and separate evidence for training and testing. T-REx uses masked Wikipedia evidence and relation templates, while zsRE partitions multiple questions per fact under defined token and correctness constraints.

The datasets contain facts F and modifications M for a subset S ⊂ F, using T-REx and zsRE as sources.
Two benchmarks use LAMA's T-REx and KILT's zsRE versions, replacing objects in cloze statements for selected facts.
T-REx contains 34,039 facts across 41 Wikipedia relations, with masked evidence and relation-specific cloze questions.
T-REx examples pair a fact with masked training and test evidence, such as Della Pia Glacier's continent.
For T-REx modification, Sydney is replaced by another birthplace object such as London, sampled according to training-set frequencies.
zsRE splits questions within each fact into training and test sets, limits inputs to 512 tokens and answers to 20, and requires exact token matches.

B Fine-tuning on a mixture of modified and unmodified facts

The constrained optimization updates the model using modified evidence while limiting parameter movement. In the 512-fact experiment, mixing unmodified samples into training supplies an additional loss signal from a subset of unmodified evidence.

The constrained objective minimizes L(x′; θ) subject to ∥θ − θ0∥ ≤ δ.
For |M| = 512, training runs for 10 epochs with minibatches of 128 and 112 iterations per epoch on modified evidence.
When unmodified samples are used, each iteration adds 128 samples from DF\S and computes loss gradients over 256 samples.
This mixture uses approximately 10% of the unmodified training evidence.

C The Small Modification Limit

The small-modification analysis approximates the unmodified-fact loss constraint when few facts change and parameter updates are small. Under this assumption, the first-order term vanishes, while Fisher-metric approximations remain costly and underperform the ℓ∞ constraint in experiments.

Evaluating the loss constraint over all DF′ is expensive, but it simplifies when few facts change and weight updates are small.
Assuming the model remains at the unmodified-fact loss minimum makes the linear term in ∆θ vanish, so the second-order term dominates.
For cross-entropy loss, the bracketed quantity is the Fisher metric.
A batch-size-128 Fisher-information approximation did not outperform the ℓ∞ norm with constraint (3), and detailed exploration is deferred.

D Solving constrained optimization with projected gradient descent

The paper uses projected gradient descent to fine-tune model parameters while constraining their distance from the initial parameters. The implementation combines norm-based projection with Adam gradient steps.

D Solving constrained optimization with projected gradient descent: Algorithm 1 applies Adam gradient updates while enforcing a norm constraint around the initial parameters.The algorithm initializes moment estimates, samples training batches, computes gradients, and updates parameters under the constraint.
D Solving constrained optimization with projected gradient descent: Projected gradient descent projects each iterate into the constraint set after every gradient step.The projection selects the nearest point within the permitted set to the current iterate.
D Solving constrained optimization with projected gradient descent: Under an ℓ2 constraint, parameters remain within distance δ of the initial parameter vector θ0.The constraint set is defined as {θ : ∥θ − θ0∥2 ≤ δ}.
D Solving constrained optimization with projected gradient descent: Under an ℓ∞ constraint, the projection is applied element-wise to keep each parameter change within δ of θ0.The paper states that the max and min operations in this projection are applied element-wise.

E Additional results for fine-tuning without constraints

The paper reports additional unconstrained fine-tuning results on T-REx, comparing randomly initialized, pretrained, and fine-tuned pretrained models across different numbers of modified facts.

E Additional results for fine-tuning without constraints: Figure 4 reports mean and standard deviation of test accuracies after unconstrained fine-tuning on different numbers of modified T-REx facts.The compared settings are RI+FTM, FTM, and FT+FTM.
E Additional results for fine-tuning without constraints: RI+FTM starts from a randomly initialized model before fine-tuning on modified facts.RI denotes random initialization, while FTM denotes fine-tuning on modified facts.
E Additional results for fine-tuning without constraints: FTM starts from an off-the-shelf pretrained model and fine-tunes it on modified facts.The figure distinguishes this setting from FT+FTM, which first fine-tunes the pretrained model on unmodified T-REx facts.
E Additional results for fine-tuning without constraints: FT+FTM starts from a pretrained model fine-tuned on unmodified T-REx facts, then fine-tunes it on modified facts.This setting evaluates the additional effect of prior fine-tuning on unmodified data.

F kNN-LM for modification?

The paper evaluates kNN-LM as a knowledge-modification approach for masked language models. Its nearest-neighbor mechanism struggles with modified facts and can alter unrelated facts sharing the same object, motivating fine-tuning.

F kNN-LM for modification?: kNN-LM augments language-model predictions with a key-value datastore of contextual embeddings and following-token values.In the masked-language-model setting, keys are contextual embeddings of [MASK] tokens and values are object-token labels.
F kNN-LM for modification?: For masked-language-model modification, the datastore can contain only training examples associated with the modified facts.The paper focuses on constructing the datastore for modified facts rather than the entire training set.
F kNN-LM for modification?: The model uses the nearest-neighbor prediction only when its contextual-embedding distance is below ϵ; otherwise it retains the original prediction.The thresholded rule is defined by Eq. 10.
F kNN-LM for modification?: Even with a very large ϵ, kNN-LM does not achieve reasonable accuracy on modified facts because nearest neighbors often represent the wrong fact.The paper attributes this to discrepancies between training and test questions for the same fact.
F kNN-LM for modification?: Changing datastore values can modify predictions for all facts sharing the same object, creating an unavoidable tradeoff between modified and unmodified accuracy.The paper gives changing Charles Darwin’s birthplace as an example that can also affect William Shakespeare’s birthplace prediction.
F kNN-LM for modification?: Including unmodified facts in the datastore requires identifying corresponding training samples, while changing only values still creates conflicts with facts sharing objects.The paper concludes that fine-tuning remains essential for knowledge modification in kNN-LM.