PaperlayerView exact revision on arXiv

Source-linked AI summary

Can We Edit Factual Knowledge by In-Context Learning?

Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, Baobao Chang

arXiv:2305.12740v1cs.CL

TL;DR

Large language models can generate fake content, and this paper asks whether in-context learning can edit factual knowledge while generalizing edits and preserving unrelated facts. It designs demonstration strategies for IKE and reports competitive editing efficacy without parameter modifications, including on GPT-J (6B).

  • Problem

    Whether in-context learning can edit LLM knowledge while generalizing updated facts and preserving unrelated facts remains an open question.

    Source: §1 · p. 2

  • Method

    IKE uses demonstration formatting and organization strategies to guide LLMs toward knowledge edits that generalize across text surfaces while preserving irrelevant facts.

    Source: §1 · p. 2

  • Results

    On GPT-J (6B), IKE achieves comparable overall editing performance, outperforming MEND by an absolute 10% editing success rate and gaining 30 points in specificity over ROME.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results provide evidence that ICL has potential for knowledge editing on language models without parameter modifications.

    Source: §7 · p. 8

    Real-world applicability remains under-explored because IKE uses consistent fact and prompt domains and formats, whereas real-world inputs are diverse.

    Source: §6 · p. 8

Abstract

from arXiv · show

Previous studies have shown that large language models (LLMs) like GPTs store massive factual knowledge in their parameters. However, the stored knowledge could be false or out-dated. Traditional knowledge editing methods refine LLMs via fine-tuning on texts containing specific knowledge. However, with the increasing scales of LLMs, these gradient-based approaches bring large computation costs. The trend of model-as-a-service also makes it impossible to modify knowledge in black-box LMs. Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without parameter updating, we explore whether ICL can edit factual knowledge. To answer this question, we give a comprehensive empirical study of ICL strategies. Experiments show that in-context knowledge editing (IKE), without any gradient and parameter updating, achieves a competitive success rate compared to gradient-based methods on GPT-J (6B) but with much fewer side effects, including less over-editing on similar but unrelated facts and less knowledge forgetting on previously stored knowledge. We also apply the method to larger LMs with tens or hundreds of parameters like OPT-175B, which shows the scalability of our method. The code is available at https://github.com/Zce1112zslx/IKE.

1 Introduction

The introduction frames knowledge editing as balancing generalization across prompts with specificity to a target fact, and proposes in-context demonstrations as a parameter-free way to pursue both. It reports comparable editing performance on GPT-J and applicability to OPT-175B.

1 Introduction

  • Traditional gradient-based editing incurs heavy computation for identifying target neurons and can cause forgetting or over-editing, limiting scalability and generalizability.Parameter changes may also dramatically influence end users when a large model is deployed as a black-box service.Source: §1 · pp. 1–2
  • In-context learning avoids parameter modifications and their associated computation and side-effect risks, offers an interpretable way to calibrate behavior, and motivates testing its still-unclear applicability to knowledge editing.ICL represents task descriptions and demonstrations in natural language context for model predictions.Source: §1 · p. 2
  • IKE targets generalization across prompts and specificity to the edited fact using copy, update, and retain demonstrations, with relevant facts retrieved from the training corpus as inputs.Copy demonstrations inject facts, update demonstrations improve their generalization, and retain demonstrations guide preservation of unrelated knowledge.Source: §1 · pp. 1–2, Figure 1 · p. 1
  • IKE achieves overall comparable editing performance on GPT-J (6B), exceeding MEND by an absolute 10% in editing success rate and ROME by 30 specificity points.The introduction also reports less computation overhead and fewer side effects for IKE than parameter-updating approaches.Source: §1 · p. 2
  • IKE applies to OPT-175B, where nearly 50% of edited knowledge facts retain relatively high probability; selection and retain demonstrations aid specificity, while update demonstrations improve generalization.The introduction also identifies real-world feasibility and potential challenges as topics of discussion.Source: §1 · p. 2
  • The paper presents the first systematic exploration of ICL for knowledge editing, with comprehensive strategy studies and discussion of real-world feasibility and potential challenges.The introduction also frames editing as a response to language models generating fake, outdated, biased, or offensive content.Source: §1 · pp. 1–2

2 Related Work

Related work covers parameter-editing approaches, benchmarks for evaluating editing efficacy and specificity, and early research on in-context knowledge editing. The cited ICL study found that demonstrations improve editing success but left its potential and side effects underexplored.

Knowledge Editing Methods

  • Prior editors include hypernetwork methods (Cao et al., 2021b; Mitchell et al., 2022a), edit-memory retrieval (Mitchell et al., 2022b), and attribution-based MLP edits (Dai et al., 2022; Meng et al., 2022a).Hypernetworks predict parameter shifts or transform fine-tuning gradients; attribution-based methods identify relevant neurons or layers and update targeted MLP parameters.Source: §2 · p. 2

Knowledge Editing Benchmarks

  • BERT editing commonly uses FEVER and zsRE, while GPT editing uses Wikitext; this study uses COUNTERFACT, whose counterfactual targets and harder out-of-scope questions make desired answers more challenging to predict.COUNTERFACT targets are also described as facts that pretrained LMs are unlikely to have captured, reducing the influence of prior knowledge.Source: §2 · p. 3

In-context Learning

  • ICL learns from demonstrations in context without training, and Si et al. (2022) found demonstrations improve editing success but studied only GPT-3, leaving its potential and side effects underexplored.Given related examples and a query, the model predicts by analogy (Brown et al., 2020).Source: §2 · p. 3

3 Task Formulation

Knowledge editing injects a new fact by increasing its target probability, while requiring related prompts to adopt the edit and unrelated prompts to retain their original predictions.

3 Task Formulation

  • Editing injects a new fact (x∗, y∗) into a language model M by maximizing PM(y∗|x∗).Here, x∗ is a prompt probing factual knowledge and y∗ is the editing target.Source: §3 · p. 3
  • For prompts related to the new fact, predictions should generalize to the editing target y∗.For example, asking who the US president is should yield Joe Biden after editing.Source: §3 · p. 3
  • For prompts outside the edit scope, predictions should retain their original value yo.For example, editing the US president fact should leave the prediction for the president of Russia unchanged.Source: §3 · p. 3

4 Method: IKE

IKE edits factual knowledge by constructing natural-language demonstrations rather than updating model parameters. Its demonstrations teach the model to copy, update, and retain predictions, and are selected and ordered by similarity.

4.1 In-Context Learning

  • In-context learning predicts an output from demonstrations without parameter updates, using templates to express inputs and labels as natural-language text.The paper describes ICL as predicting y given x and k demonstrations.Source: §4.1 · p. 3

4.2 In-Context Knowledge Editing

  • IKE aims to raise the target prediction for prompts within the editing scope while preserving original predictions for prompts outside it.The method frames these goals as generalization within the scope and specificity outside it.Source: §4.2 · pp. 3–4

4.2.1 Demonstration Formatting

  • IKE demonstrations teach the model to copy the target prompt’s new prediction, update predictions for in-scope prompts, and retain original answers for out-of-scope prompts.Copy demonstrations use the target prompt; update demonstrations use prompts in the editing scope; retain demonstrations use prompts outside it.Source: §4.2.1 · p. 4

4.2.2 Demonstration Organization

  • IKE selects k demonstrations using an unsupervised nearest-neighbor retriever and orders them from lowest to highest cosine similarity to the new fact.A pretrained sentence encoder represents the target prompt, its original answer, and the targeted prediction; training-corpus records are encoded the same way.Source: §4.2.2 · p. 4

4.3 Discussion: Gradient-based methods and gradient-free methods

  • IKE requires no gradient estimation, leaves the original LM untouched, and represents edits in interpretable natural text, reducing computation and parameter-update side effects.The authors state this makes editing applicable to LMs with trillion-level parameters and provides a human-understandable interface for calibrating behavior.Source: §4.3 · p. 4, Table 1 · p. 4

5 Experiment

On COUNTERFACT, IKE achieves a GPT-J overall score comparable to ROME without parameter modifications and improves over PROMPT on OPT-175B. The analyses examine demonstration design, model scale, over-editing, and retention of prior knowledge.

5.1.1 Baselines

  • GPT-J (6B) is the main evaluation backbone, compared against FT, MEND, ROME, and PROMPT, which probes the model using the edit fact as context.FT fine-tunes on the edit fact; MEND transforms fine-tuning gradients, while ROME writes new key-value pairs into an MLP module.Source: §5.1.1 · p. 5

5.1.2 Evaluation Setup

  • IKE is evaluated on five GPT-like models from 1.5B to 175B parameters and COUNTERFACT, using efficacy, generalization, specificity, and their harmonic-mean score.COUNTERFACT has 21,919 records; the first 2,000 are the test set, and its prompts distinguish target, paraphrase, and neighborhood scopes.Source: §5.1.2 · pp. 5–6

5.2 Main Results

  • IKE scores 89.6 versus ROME’s 91.5 on GPT-J and improves over PROMPT by 36.0 points on OPT-175B without parameter modifications.ROME is the strongest baseline overall but has high computational overhead; IKE excels in specificity while performing well in efficacy and generalization.Source: §5.2 · p. 6, Table 2 · p. 6

5.3.1 Ablation on Demonstration

  • More demonstrations improve the generalization-specificity trade-off; removing selection lowers NS from 77.0 to 45.0, while removing retain demonstrations drops NM from 35.2 to -47.6.Without demonstrations, PROMPT over-generalizes (NS 37.9); 4 or 8 demonstrations initially lag PROMPT in efficacy and generalization. Random ordering has negligible impact, copy demonstrations have a slight effect, and removing update demonstrations hurts generalization.Source: Table 3 · p. 6, §5.3.1 · p. 7

5.3.2 IKE Benefits from Model Scaling

  • IKE’s generalization and specificity improve with model scale, with OPT-175B achieving the strongest results among the evaluated models.The scaling evaluation focuses on these metrics because they capture side effects that can affect end users.Source: §5.3.2 · p. 7, Table 4 · p. 7

5.3.3 Resilience to Over-Editing

  • Under a contrastive test of similar but unrelated relations, IKE has less over-editing, while ROME has the lowest average CKA score and highest false rate.The CKA assessment compares the target-relation probability with probabilities for other relations sharing the subject; a low score indicates over-generalization.Source: §5.3.3 · pp. 7–8, Table 5 · p. 8

5.3.4 Maintenance for Original Knowledge

  • All editing methods lower the probability of the original fact, and ROME forgets almost all such facts; IKE causes less forgetting on time-aware knowledge.The paper notes that historical facts may need to remain available when a prediction is updated over time, and constructs a TEMPLAMA-based benchmark for this case.Source: §5.3.4 · p. 8

6 Discussions

The discussion reports that IKE edits individual facts with improved generalization and specificity, fewer side effects, and no parameter changes. It identifies scaling and format diversity as open challenges and outlines retrieval and prompt variations as directions to address them.

Discussions

  • IKE’s individual-fact editing showed better generalization and specificity, fewer side effects, and no parameter modification.The prior experiments followed Meng et al. (2022a) for a fair comparison.Source: §6 · p. 8
  • IKE’s real-world applicability remains uncertain for numerous edits that exceed context limits and for facts or prompts with diverse formats and domains.The discussion raises these as under-explored questions rather than reporting solutions.Source: §6 · p. 8
  • IKE could use external memory to retrieve relevant edits for a prompt, avoiding the need to prepend every factual edit; Mitchell et al. (2022b) propose a retrieval-based method for multiple facts.The passage presents external memory as a possible extension of IKE.Source: §6 · p. 8
  • To test generalization across fact and prompt forms, the authors replaced facts with neutral Wikipedia data or used prompts asking the LM to generate text related to the new object.The detailed discussion is in §D.Source: §6 · p. 8

7 Conclusion

The paper examines in-context learning for knowledge editing in large language models through IKE, which uses demonstration strategies without modifying model parameters. The conclusion reports competitive editing efficacy, decent generalization and specificity, scalability, resilience to over-editing, and preservation of time-aware facts across multiple editing rounds.

7 Conclusion

  • IKE achieves competitive knowledge-editing efficacy without modifying model parameters while maintaining decent generalization and specificity.The method is evaluated on large-scale language models.Source: §7 · p. 8
  • IKE uses three types of demonstration formatting and retrieval-based organization to construct prompting demonstrations for language models.Source: §7 · p. 8
  • Further analysis finds IKE scalable to large language models, resilient to over-editing, and able to maintain time-aware facts through multiple editing rounds.Source: §7 · p. 8

A Implementation Details … A.3 Other Baselines

The appendices describe IKE’s implementation and demonstration design, including its formatting and selection choices, and explain how the other baselines were run.

A.1 IKE

  • IKE is implemented with PyTorch, Huggingface transformers (Wolf et al., 2019), and sentence transformers.PyTorch uses a modified BSD license; the other two libraries use Apache License 2.0.Source: §A.1 · pp. 10–11
  • IKE with 32 examples runs for about 3 GPU hours on a 40 GB NVIDIA A40 GPU.Source: §A.1 · p. 11

A.2 Demonstration Designing

  • Demonstrations are selected by k-NN from the training corpus and encoded with all-MiniLM-L6-v2; k is 32 for 2048-token context limits and 16 for 1024.The selection procedure follows Liu et al. (2022).Source: §A.2 · p. 11

A.2.1 Demonstration Formatting

  • IKE reformats COUNTERFACT examples as copy, update, and retain demonstrations, using a 1:3:4 ratio, respectively.The format is New Fact: f. Prompt: x y, where x is the probing prompt and y is the model prediction; Table 7 shows examples.Source: §A.2.1 · p. 11, Table 7 · p. 11
  • The order of demonstration types is predefined so each type’s position is distributed as uniformly as possible.The authors identify ordering as an under-explored factor affecting IKE.Source: §A.2.1 · p. 11

A.3 Other Baselines

  • For other baselines, the authors use code implemented by Meng et al. (2022a), add the prefix “Prompt:” to prompts, and report results they run.Source: §A.3 · p. 11

B Details of COUNTERFACT Dataset · C Time-aware Knowledge Editing · D Detailed Discussions

The appendices describe the COUNTERFACT evaluation split and prompt types, then test whether time-aware edits preserve earlier facts; IKE retains those facts after multiple edits, unlike ROME, which forgets them.

B Details of COUNTERFACT Dataset

C Time-aware Knowledge Editing

  • For time-aware editing, the study collects 2,067 facts across three TEMPLAMA relations: sports-team membership, position held, and employer.Source: C Time-aware Knowledge Editing · p. 12
  • The TEMPLAMA evaluation injects changing objects for the same subject-relation pair chronologically, then tests whether the LM preserves the editing history.Source: C Time-aware Knowledge Editing · p. 12
  • Old-fact memorization is measured by the probability of recalling the first injected object after all edits divided by its probability immediately after the first edit.The example sequentially injects Obama for 2010, Trump for 2017, and Biden for 2021, then probes the 2010 fact.Source: C Time-aware Knowledge Editing · p. 12
  • IKE retains previously injected time-aware facts after multiple editing rounds, whereas ROME forgets them at an extremely low memorization ratio.The authors suggest that parameter updates for time-aware facts may conflict within the same FFN module and cause forgetting.Source: C Time-aware Knowledge Editing · p. 12

D.1 Scale up to more factual edits · D.2 Generalization on facts and prompts

IKE can generalize to prompts and facts with different formats, and retrieval augmentation offers a way to scale it beyond the context-length limit on multiple edits. On longer, more complex neutral texts, IKE scores higher than PROMPT on the reported target and neighborhood metrics.

D.1 Scale up to more factual edits

  • IKE and gradient-based editing both face difficulties with many simultaneous factual edits; IKE cannot keep all corresponding demonstrations in context indefinitely because of input-length limits.Source: §D.1 · p. 12
  • IKE can pair with external memory to retrieve only facts relevant to a prompt and construct their demonstrations in context, using the original LM when no edit is relevant.Mitchell et al. (2022b) instead describe retrieving relevant edits with a scope classifier and applying updated parameters.Source: §D.1 · p. 12

D.2 Generalization on facts and prompts

  • Because facts and prompts can vary in domain and format, the section asks whether IKE generalizes when their formats are inconsistent.Source: §D.2 · p. 12
  • IKE generalizes across differing fact and prompt formats, producing text related to new objects rather than only the objects themselves.Figure 3 shows GPT-J generation examples of this behavior.Source: §D.2 · p. 13, Figure 3 · p. 13
  • 75 PS on target prompts and 73 NS on neighborhood prompts for IKE, versus PROMPT’s 65 and 64, on edits using longer, more complex neutral texts.The evaluation replaced facts with longer Wikipedia neutral data in 100 cases.Source: §D.2 · p. 13

Related papers

Loading 2305.12740v1…