Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models

Motivation

• The approach is motivated by conflicts between existing and new knowledge, which can make inconsistent information difficult to learn.The paper illustrates this with previously learned but erroneous beliefs conflicting with correct updates.Source: §1 · pp. 1–2

F-Learning

• F-Learning first forgets old knowledge through parameter subtraction, then fine-tunes the resulting model on new knowledge.The method defines these as sequential stages: old knowledge forgetting and new knowledge learning.Source: §1 · p. 2

Experimental findings

• F-Learning improves knowledge-updating performance for full fine-tuning and LoRA and outperforms existing baselines in most cases.The reported experiments identify improvements regardless of whether full fine-tuning or LoRA is used.Source: §1 · p. 2
• Subtracting LoRA parameters can approximately match subtracting full-fine-tuning parameters when forgetting old knowledge.This finding is reported as an additional experimental result of the proposed paradigm.Source: §1 · p. 2

Parameter-preserving methods

• Retrieve augmentation preserves model parameters by connecting the model to external knowledge bases containing new or corrected information.Other parameter-preserving methods add trainable modules or use demonstrations and memory to revise outputs.Source: §2 · p. 2

Parameter-modifying methods

• Fine-tuning modifies model parameters to learn new knowledge, including parameter-efficient methods such as Prefix-Tuning and LoRA.The related work also covers regularized fine-tuning, meta-learning, and methods that locate and edit internal model components.Source: §2 · pp. 2–3

Parameter-preserving methods

• Existing knowledge-updating approaches commonly add knowledge bases, network modules, or parameters, increasing practical complexity and inference consumption.The paper contrasts these approaches with direct fine-tuning-based updating.Source: §2 · pp. 2–3

Paper focus

• The paper positions its contribution as improving fine-tuning methods rather than adding external knowledge bases, modules, or model parameters.This focus is presented as a response to the practical burden of many existing updating methods.Source: §2 · p. 3

Task formulation

• Knowledge updating transforms a model fθ trained on old input-output pairs into fθ∗ that produces specified new answers.The updated set contains the same inputs with replacement outputs, and i denotes the number of knowledge pairs updated.Source: §3 · p. 3

Evaluation indicators

• Reliability measures accuracy on the new knowledge, while generalization measures accuracy on equivalent rephrasings in the neighborhood N(x_i).Both indicators assess whether the intended knowledge change is applied to direct and equivalent queries.Source: §3 · p. 3
• Locality measures whether predictions for irrelevant examples remain unchanged from the pre-update model.The task therefore requires targeted updates without changing out-of-scope knowledge.Source: §3 · p. 3

4.1 Forgetting old knowledge

• The method derives old-knowledge parameters by fine-tuning the original model on Kold and subtracting its original parameters from the fine-tuned parameters.Kold is the dataset containing the old knowledge targeted for forgetting.Source: §4.1 · p. 4

4 Proposed method: F-Learning

• F-Learning first forgets old knowledge by subtracting an old-knowledge parameter increment from the original model, then learns new knowledge through supervised fine-tuning.The forgetting rate is controlled by λ, and the second stage fine-tunes the resulting model on Knew.Source: §4.1 · p. 4, §4.2 · p. 4

4.1 Forgetting old knowledge

• Forgetting is intended only when the model has already learned the old knowledge; otherwise, the operation may damage normal knowledge.The paper states that forgetting has no purpose without prior old-knowledge learning and may have a destructive effect.Source: §4.1 · p. 4

4.2 Learning new knowledge

• After forgetting, supervised fine-tuning on Knew produces a model whose parameters incorporate the new knowledge for updating.The learning stage operates on the model produced by the forgetting stage.Source: §4.2 · p. 4

5.1 Datasets

• The experiments use ZsRE and COUNTERFACT, split into old-knowledge and new-knowledge data for two-stage updating.ZsRE uses question rephrasings, whereas COUNTERFACT contains counterfactual data.Source: §5.1 · p. 4

5.2 Baselines

• Baselines include Full-FT, LoRA, FT-c, ROME, and MEMIT, covering fine-tuning and locate-based editing approaches.FT-c uses an L∞ constraint, while ROME and MEMIT update factual associations or large-scale memories.Source: §5.2 · p. 5

5.4 Experimental Results

• F-Learning improves knowledge updating for both full fine-tuning and LoRA, outperforming existing baselines in most cases.Table 1 reports results across three metrics on two datasets and two LLaMA models.Source: §5.4 · pp. 5–6, Table 1 · p. 5
• Full fine-tuning learns new knowledge more effectively than LoRA, while ROME largely preserves locality because it performs limited updating.The authors attribute LoRA’s lower learning capacity to its restricted parameter subset and ROME’s behavior to editing few parameters.Source: §5.4 · p. 6

5.5 Forgetting with LoRA and Learning with Full Fine-tuning

• Forgetting with LoRA followed by full fine-tuning surpasses direct full fine-tuning and nearly matches or sometimes exceeds full F-Learning, while generally preserving locality.The authors associate this trade-off with LoRA’s smaller parameter impact.Source: §5.5 · p. 6, Table 2 · p. 6
• LoRA subtraction approximates full-fine-tuning subtraction by altering attention-layer patterns and relationships, despite not removing old knowledge from MLP layers.The smaller computational cost of LoRA makes this finding practically valuable.Source: §5.5 · p. 6, §5.8 · p. 7

5.6 Adaptability Testing

• F-Learning remains effective on zsRE with BLOOM-7B, where locality improves substantially while Reliability and Generality remain roughly stable.This result supports adaptability beyond the primary LLaMA-based experiments.Source: §5.6 · p. 6

5.7 Time Testing

• F-Learning takes about twice as long as Full-FT but remains faster than locate-based methods, and its forgetting parameters can be reused for later optimization and inference.Fine-tuning methods are reported as significantly faster than locate-based approaches.Source: §5.7 · p. 7, Table 3 · p. 7

• F-Learning sometimes completed knowledge updating even when the forgetting operation failed to remove the old knowledge.This occurred in example 4.Source: §6 · p. 8
• The forgetting operation produced irrelevant outputs in some examples, indicating that the model gradually forgot old knowledge.The forgetting rate λ was set to 0.3 in this llama2-7B zsRE case study.Source: §6 · p. 8
• In example 3, the model never learned the new knowledge, illustrating a limitation of the method.The authors identify this as a bad case requiring improvement.Source: §6 · p. 8

• F-Learning surpasses other baselines in most cases on the zsRE and COUNTERFACT datasets.Source: §7 · p. 8
• Forgetting old knowledge by subtracting LoRA parameters can achieve a similar effect to subtracting full-fine-tuning parameters.The paper identifies this finding as motivating further investigation of knowledge updating.Source: §7 · p. 8

• F-Learning adds extra computation because knowledge updating includes an additional forgetting process.Source: §8 · p. 8

A.1 Datasets and Examples

• ZsRE uses back-translation question rephrasings as an equivalence neighborhood, while COUNTERFACT contains more challenging counterfactual data.The evaluation and edit sets contain 19,085 and 10,000 examples, respectively.Source: §A.1 · p. 11
• A ZsRE example changes the answer about Watts Humphrey’s university from Trinity College to University of Michigan.Source: §A.1 · p. 11
• Rephrased prompts measure Generalization, while localized prompts measure Locality.Source: §A.1 · p. 11
• The datasets contain paired old and new knowledge for the same questions, preserving the question format while changing the answer.Source: §A.1 · p. 11
• Both datasets are counterfactual: old knowledge is real-world correct, whereas target new knowledge is real-world wrong.Source: §A.1 · p. 11
• The old and new knowledge in the datasets are artificially given and are not intended to represent real-world time or correctness.Source: §A.1 · p. 11
• Artificially wrong target knowledge helps ensure that the original LLM has not already learned it, avoiding ambiguity about whether an output came from training data or prior model knowledge.Source: §A.1 · p. 11

A.2 Implementation Details of Experiments

• Experiments use LLAMA2-7B and LLAMA-7B as base models, with old-knowledge pretraining followed by three training epochs for the experimental setup.The study also reports BLOOM-7B settings for zsRE experiments.Source: §A.2 · p. 11, §A.2.1 · p. 11

A.2.1 F-learning

• F-Learning experiments vary the forgetting-rate hyperparameter λ across models and datasets, with learning rates, batch sizes, epochs, and gradient accumulation specified separately.For LLAMA2-7B, λ differs across zsRE and COUNTERFACT and across F-Learning variants.Source: §A.2.1 · pp. 11–12

A.2.2 Full-FT and LoRA

• Full-FT and LoRA update knowledge without forgetting old knowledge, whereas FT-c, ROME, and MEMIT use separate layer, optimization, or covariance-statistics settings.Full-FT and LoRA use instruction fine-tuning data, while the editing baselines specify their own optimization configurations.Source: §A.2.2 · p. 12, §A.2.3 · p. 12, §A.2.4 · p. 12

A.3 Impact to Other Capabilities within LLMs Testing

• F-Learning slightly improves mathematical ability after forgetting but reduces it after new-knowledge learning, while other capabilities change little and college chemistry and medicine improve significantly.The evaluation covers GSM8K, MATH, and MMLU using accuracy, with LLAMA2-7B as an example.Source: §A.3 · p. 12, Table 5 · p. 12

A.4 Interpretability of Parametric Arithmetic

• Parameter subtraction moves loss opposite to gradient descent, resembling gradient ascent or loss maximization while being described as more stable and controllable for limiting effects on unrelated knowledge.The paper relates this analysis to prior parametric arithmetic work and presents the relationship in Figure 3.Source: §A.4 · p. 13, Figure 3 · p. 13

A.5 Prospects and Application Scenarios

• The paper identifies harmful-knowledge forgetting, copyright-content elimination, and user-privacy protection as potential application scenarios for old-knowledge forgetting.Knowledge updating is distinguished from continual learning and machine unlearning by its focus on correcting old or wrong knowledge.Source: §A.5 · p. 13