PaperlayerView exact revision on arXiv

Source-linked AI summary

Can We Edit Factual Knowledge by In-Context Learning?

Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, Baobao Chang

arXiv:2305.12740v1cs.CL

TL;DR

Large language models may contain false or outdated factual knowledge, while gradient-based editing becomes computationally costly at scale. This paper studies in-context knowledge editing with structured demonstrations and finds competitive editing efficacy without parameter updates, fewer side effects, and scalability to larger models.

  • Problem

    The paper asks whether ICL can edit factual knowledge while addressing the computation and side-effect challenges of gradient-based editing.

  • Method

    IKE constructs natural-language demonstrations using copy, update, and retain templates plus retrieval-based organization to guide knowledge editing.

  • Results

    IKE achieves competitive knowledge-editing efficacy without parameter modifications, with reported gains over MEND and ROME on GPT-J (6B) and scalability to OPT-175B.

  • Takeaways & Limitations

    The results provide evidence that ICL has potential for knowledge editing while reducing computation and parameter-update side effects.

  • Takeaways & Limitations

    Real-world deployment remains under-explored for large numbers of editing facts and for diverse fact and prompt formats or domains.

Abstract

from arXiv · show

Previous studies have shown that large language models (LLMs) like GPTs store massive factual knowledge in their parameters. However, the stored knowledge could be false or out-dated. Traditional knowledge editing methods refine LLMs via fine-tuning on texts containing specific knowledge. However, with the increasing scales of LLMs, these gradient-based approaches bring large computation costs. The trend of model-as-a-service also makes it impossible to modify knowledge in black-box LMs. Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without parameter updating, we explore whether ICL can edit factual knowledge. To answer this question, we give a comprehensive empirical study of ICL strategies. Experiments show that in-context knowledge editing (IKE), without any gradient and parameter updating, achieves a competitive success rate compared to gradient-based methods on GPT-J (6B) but with much fewer side effects, including less over-editing on similar but unrelated facts and less knowledge forgetting on previously stored knowledge. We also apply the method to larger LMs with tens or hundreds of parameters like OPT-175B, which shows the scalability of our method. The code is available at https://github.com/Zce1112zslx/IKE.

1 Introduction

Knowledge editing seeks to update targeted LM facts while generalizing across prompts and avoiding interference with unrelated knowledge. This paper investigates ICL-based editing through systematic strategy studies and reports competitive efficacy with fewer side effects and scalability to larger LMs.

  • Knowledge editing must generalize across prompts describing the same fact while preserving unrelated facts.
  • Gradient-based editing can incur heavy computation and cause forgetting of existing facts or over-editing of unrelated facts.
  • ICL offers parameter-free, interpretable knowledge editing by conditioning predictions on natural-language demonstrations.
  • IKE uses copy, update, and retain demonstration templates plus retrieval-based organization to target editing, generalization, and specificity.
  • IKE achieves an absolute 10% higher editing success rate than MEND and a 30-point specificity gain over ROME on GPT-J (6B).
  • IKE scales to OPT-175B, where nearly 50% of knowledge facts retain relatively high probability after editing.

2 Related Work

Related work covers gradient-based knowledge editing, benchmark design, and early ICL-based editing studies. The paper distinguishes its broader investigation of ICL strategies and side effects from prior limited exploration.

  • Prior knowledge-editing methods include hyper-network-based approaches that predict or transform parameter updates for targeted edits.
  • Knowledge-editing benchmarks evaluate efficacy and specificity across datasets such as FEVER, zsRE, Wikitext editing, and COUNTERFACT.
  • ICL learns from demonstrations concatenated in the input context without parameter updates, but earlier editing work focused on GPT-3 without deeply examining side effects.

3 Task Formulation

Knowledge editing injects a target fact while requiring generalization across related prompts and specificity on unrelated prompts. The task evaluates whether edited predictions change appropriately within scope while preserving original knowledge outside it.

  • Task objective: Knowledge editing injects a new fact (x*, y*) by maximizing its probability for a target prompt.The target prompt probes factual knowledge, while the target answer specifies the edited prediction.
  • Task examples: Demonstrations illustrate copying the new fact and updating a related question to the desired answer.The examples use the US presidency and Einstein’s field of study to show these behaviors.
  • Generalization: Generalization requires related prompts to produce the edited answer, not only the exact target prompt.For example, prompts asking who the US president is should also yield Joe Biden after editing.
  • Specificity: Specificity requires unrelated prompts to retain their original predictions.The example preserves the original answer for a prompt about the president of Russia.

4 Method: IKE

IKE uses in-context demonstrations to edit knowledge without updating model parameters. Its demonstrations encode copying, updating, and retaining behaviors, while retrieval and ranking organize examples for generalization and specificity.

  • 4.2.1 Demonstration Formatting: IKE constructs natural-language demonstrations that teach models to copy, update, or retain predictions.Copy demonstrations target the new fact, update demonstrations cover related prompts, and retain demonstrations preserve out-of-scope answers.
  • 4.2.1 Demonstration Formatting: Update demonstrations support generalization by changing predictions for prompts within the editing scope.These prompts are related to the target fact rather than identical to its original probing prompt.
  • 4.2.1 Demonstration Formatting: Retain demonstrations support specificity by keeping original predictions for prompts outside the editing scope.Their expected answers remain the model’s original responses.
  • 4.2.2 Demonstration Organization: IKE selects nearest-neighbor demonstrations with an unsupervised sentence-encoder retriever and ranks them by cosine similarity.The prompt, original answer, and target prediction are encoded before retrieval and ordering.
  • 4.2 In-Context Knowledge Editing: Unlike parameter editing, IKE evaluates the unchanged model with demonstrations, avoiding gradient estimation and parameter-update side effects.The natural-language context also provides a more interpretable interface for calibrating behavior.

5 Experiment

The experiments evaluate IKE against parameter-updating baselines across editing performance, demonstration design, model scale, over-editing, and forgetting. IKE achieves competitive editing while improving specificity, scaling to larger models, and reducing some side effects.

  • Main Results: IKE achieves a comparable overall score with ROME on GPT-J, 89.6 versus 91.5, without modifying model parameters.IKE also improves over PROMPT by 36.0 points on OPT-175B.
  • Demonstration Design: Increasing demonstrations helps IKE balance generalization and specificity, while selected demonstrations improve specificity and update demonstrations improve generalization.Removing selection lowers NS from 77.0 to 45.0; removing retain demonstrations lowers NM from 35.2 to -47.6.
  • Scaling: IKE performance positively correlates with model scale, with OPT-175B achieving the strongest generalization and specificity results.The evaluated GPT-like models range from 1.5B to 175B parameters.
  • Side Effects: IKE has less influence on over-editing than the baselines under contrastive knowledge assessment, while ROME has the lowest average CKA score and highest false rate.The CKA evaluation tests whether edits affect similar but unrelated relations.
  • Side Effects: All editing methods reduce the probability of original facts, with ROME forgetting almost all original facts.The study notes that time-aware updates should preserve older factual knowledge rather than erase it.

6 Discussions

The discussion reports that IKE can improve generalization and specificity with fewer side effects and no parameter modification. It also identifies unresolved deployment questions involving many edits and heterogeneous fact or prompt formats.

  • Discussion: IKE can obtain better generalization and specificity with fewer side effects while requiring no parameter modifications.The experiments mainly evaluate editing individual facts for fair comparison with prior work.
  • Open Questions: A key open question is whether IKE can accommodate many editing facts despite language models' limited input length.An external memory may retrieve relevant factual edits instead of placing all edits in the context.
  • Open Questions: IKE's real-world scope remains uncertain because its facts and prompts currently use consistent domains and formats, unlike diverse real-world inputs.Additional experiments replace facts with neutral Wikipedia data and prompts with generation prompts.

7 Conclusion

The paper concludes that IKE demonstrates the potential of in-context learning for knowledge editing on large language models. Its results cover competitive efficacy without parameter updates, generalization and specificity, scalability, reduced over-editing, and preservation of time-aware knowledge across editing rounds.

  • Conclusion: IKE uses demonstration formatting and retrieval-based organization to guide knowledge editing without modifying model parameters.The demonstrations include three formatting types and retrieval-based organization.
  • Conclusion: IKE maintains decent generalization and specificity while showing scalability, resilience to over-editing, and preservation of time-aware knowledge through multiple editing rounds.These findings provide evidence that ICL has potential for knowledge editing on language models.

A.1 IKE

IKE constructs in-context demonstrations for knowledge editing without parameter updates. Its demonstrations use copy, update, and retain examples, with retrieval and ordering strategies to organize them.

  • Implementation: The implementation uses PyTorch, Hugging Face Transformers, and Sentence Transformers, and 32-example IKE runs take about 3 GPU hours on an NVIDIA A40.The reported hardware has 40 GB of GPU memory.
  • Demonstration Formatting: IKE uses three demonstration types—copy, update, and retain—to teach language models different editing behaviors.The demonstrations are reformatted from COUNTERFACT examples into these three types.
  • Demonstration Formatting: IKE sets the copy, update, and retain demonstration ratio to 1:3:4.The ratio is chosen based on the reported importance of each demonstration type.
  • Demonstration Ordering: The method distributes demonstration types as uniformly as possible through a predefined type order.Demonstration ordering is treated as an influencing factor of IKE.

B Details of COUNTERFACT Dataset

COUNTERFACT evaluates knowledge edits with paraphrase and neighborhood prompts, while TEMPLAMA tests time-aware factual changes and retention across sequential edits.

  • COUNTERFACT: COUNTERFACT entries change a factual statement and provide paraphrase and neighborhood prompts for evaluation.Paraphrase prompts are semantically equivalent to the original; neighborhood prompts share the relation and object but change the subject.
  • TEMPLAMA: TEMPLAMA contains time-aware relations whose objects can change across years, including sports team, position, and employer relations.The dataset includes 2067 facts represented as (t, s, r, o).
  • Sequential Editing: Sequential editing probes whether an earlier fact remains memorized after later edits to the same relation.The memorization ratio compares the probability of the oldest object after the first edit with its probability after all edits.
  • Sequential Editing: ROME exhibits extremely low memorization ratios for previously injected time-aware facts, whereas IKE retains those older facts after multiple editing rounds.The passage attributes the ROME behavior to parameter conflicts in the same feed-forward network module.

D.1 Scale up to more factual edits

IKE is examined for scaling to many factual edits, retrieval-augmented use, and generalization across fact and prompt formats. Results support stronger generalization and specificity than retrieval-only prompting on complex facts.

  • Scale-up Limitations: IKE faces input-length limits because it cannot prepend context demonstrations for an indefinitely growing number of factual edits.The passage notes that gradient-based methods also encounter difficulties when updating multiple facts simultaneously.
  • Retrieval Augmentation: Retrieval augmentation could store multiple factual edits externally and place only prompt-relevant facts and demonstrations into IKE’s context.If no relevant edit is retrieved, the original language model can generate the answer directly.
  • Generalization Limitations: IKE’s real-world generalization is bounded by experiments in which fact and prompt domains and formats are kept consistent.The paper identifies adaptation to diverse formats and domains as an open question.
  • Prompt Generalization: IKE generates text related to a new object for prompts with different forms, rather than merely outputting the new object itself.The examples use generation prompts from COUNTERFACT.
  • Complex Facts: On 100 longer, more complex neutral-text cases, IKE obtains 75 PS on target prompts and 73 NS on neighborhood prompts.PROMPT obtains 65 PS and 64 NS, respectively, under retrieval augmentation without examples.
Loading 2305.12740v1…