PaperlayerView exact revision on arXiv

Source-linked AI summary

Knowledge Editing for Large Language Models: A Survey

Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, Jundong Li

arXiv:2310.16218v4cs.CLcs.AI

TL;DR

Large language models are costly to update as knowledge changes, while naive fine-tuning can damage unrelated knowledge. This survey organizes knowledge-based model editing through a constrained formulation and taxonomy, then reviews methods, evaluation resources, applications, and open challenges.

  • Problem

    Updating pre-trained LLMs is computationally costly, and naive fine-tuning risks losing valuable knowledge unrelated to the update.

  • Method

    The survey formulates KME as constrained optimization, categorizes strategies into external memorization, global optimization, and local modification, and reviews their methods, metrics, datasets, and applications.

  • Results

    The survey provides a structured synthesis of KME techniques, including their strengths, weaknesses, evaluation resources, applications, and future research challenges.

  • Takeaways & Limitations

    The taxonomy and analysis support selecting KME methods for specific tasks while highlighting practical challenges that require further research.

  • Takeaways & Limitations

    KME still faces a potential trade-off between locality and generality, while global optimization methods incur high costs from fine-tuning all parameters.

Abstract

from arXiv · show

Large language models (LLMs) have recently transformed both the academic and industrial landscapes due to their remarkable capacity to understand, analyze, and generate texts based on their vast knowledge and reasoning ability. Nevertheless, one major drawback of LLMs is their substantial computational cost for pre-training due to their unprecedented amounts of parameters. The disadvantage is exacerbated when new knowledge frequently needs to be introduced into the pre-trained model. Therefore, it is imperative to develop effective and efficient techniques to update pre-trained LLMs. Traditional methods encode new knowledge in pre-trained LLMs through direct fine-tuning. However, naively re-training LLMs can be computationally intensive and risks degenerating valuable pre-trained knowledge irrelevant to the update in the model. Recently, Knowledge-based Model Editing (KME) has attracted increasing attention, which aims to precisely modify the LLMs to incorporate specific knowledge, without negatively influencing other irrelevant knowledge. In this survey, we aim to provide a comprehensive and in-depth overview of recent advances in the field of KME. We first introduce a general formulation of KME to encompass different KME strategies. Afterward, we provide an innovative taxonomy of KME techniques based on how the new knowledge is introduced into pre-trained LLMs, and investigate existing KME strategies while analyzing key insights, advantages, and limitations of methods from each category. Moreover, representative metrics, datasets, and applications of KME are introduced accordingly. Finally, we provide an in-depth analysis regarding the practicality and remaining challenges of KME and suggest promising research directions for further advancement in this field.

1 INTRODUCTION

Knowledge-based Model Editing (KME) targets efficient, precise updates to pre-trained LLMs while preserving unrelated knowledge. The survey formulates, categorizes, and analyzes KME methods, metrics, datasets, applications, practicality, and future challenges.

  • Motivation: LLMs require frequent knowledge updates, but their large parameter counts make retraining computationally expensive.Updates may rectify obsolete information or incorporate new knowledge to maintain relevance.
  • Motivation: Naive fine-tuning injects new knowledge but can require intensive resources and cause valuable existing knowledge to be lost through unconstrained updates.The survey identifies computational cost and overfitting as central disadvantages.
  • Knowledge-based Model Editing: KME precisely modifies model behavior for specific edits while limiting negative effects on irrelevant pre-trained knowledge.Edits may use auxiliary parameters or update partial model parameters, with some methods adding constraints for unmodified knowledge.
  • Knowledge-based Model Editing: KME emphasizes locality and generality: unrelated inputs should remain consistent, while related inputs should adapt to the edited knowledge.These objectives make KME distinct from ordinary fine-tuning and require specialized strategies.
  • Survey scope: The survey categorizes KME into external memorization, global optimization, and local modification, and analyzes their relationships, insights, advantages, and limitations.Its constrained-optimization formulation treats methods from each category as specialized cases.
  • Survey scope: The survey also examines KME metrics, datasets, applications, practicality, challenges, and promising future research directions.The paper positions these analyses as support for method selection and continued progress in KME.

2 BACKGROUND

The background introduces model editing, transformer-based LLMs, and related approaches. It frames editing as precise model modification that incorporates targeted knowledge while preserving behavior on unrelated inputs.

  • Editing of Machine Learning Models: Model editing modifies pre-trained models precisely to encode specific knowledge while maximally preserving existing knowledge and unrelated-input behavior.The background traces applications from computer vision editing to textual information in NLP.
  • Transformers: Transformers use stacked self-attention modules with residual connections to capture contextual information from textual sequences.Self-attention directs prediction toward different sequence parts and relationships among tokens.
  • Transformers: In transformers, feed-forward networks add information based on knowledge stored in fully connected-layer weights, while self-attention aggregates value vectors from other tokens.The final hidden representations support token-level and sequence-level NLP predictions.
  • Large Language Models: LLMs are transformer-based models with billions of parameters trained on large corpora that demonstrate factual and commonsense knowledge.They differ from traditional small-scale language models through their scale and emergent abilities.
  • Related approaches: Fine-tuning and parameter-efficient fine-tuning are related updating approaches, but remain computationally intensive and often impractical for black-box LLMs.These limitations motivate KME-specific techniques.

3 PROBLEM FORMULATION

The survey formulates KME as constrained optimization: edit targeted knowledge toward desired outputs while preserving outputs for unrelated inputs. It represents edits through knowledge triples, prompts, output spaces, and model modification.

  • Editing Target: An edit represents a knowledge triple t=(s,r,o) whose original object o is replaced by a target object o* while subject s and relation r remain fixed.The triple captures the knowledge targeted for injection into the model.
  • Input and Output Space: A subject-relation pair is converted into a natural-language prompt x, and the textual model output y can be converted back into the queried object.Thus, textual input-output pairs correspond to knowledge triples.
  • Input and Output Space: For each edit, KME seeks to move model behavior from the original output space Y_e to the target output space Y*_e for inputs in the edit scope.Multiple edits combine their in-scope inputs through a union over the edit set.
  • Formulation: KME transforms a pre-trained model f into an edited model f* according to an edit set while minimizing changes to input-output pairs irrelevant to the edits.The formulation uses a loss for discrepancy from desirable responses and a modification operator M(f; E).
  • KME Objectives: The objective requires generality on in-scope inputs and locality on unrelated inputs so edited knowledge applies broadly while valuable knowledge is preserved.These are identified as two crucial perspectives of KME.

4 EVALUATION METRICS

The survey evaluates knowledge-based model editing through metrics covering edit success, preservation of unrelated knowledge, generalization, logical transfer, sequential stability, and large-scale editing. It also highlights practical difficulties in defining evaluation sets and computing metrics over enormous input spaces.

  • 4.1 Accuracy: Accuracy measures the success rate of edits on predefined input-output pairs.It supports fair comparisons between KME methods but is difficult to apply consistently to long-form generation.
  • 4.2 Locality: Locality measures whether the edited model preserves pretrained knowledge unrelated to the edits.Because the full input space may be enormous or infinite, evaluations usually predefine out-of-scope inputs.
  • 4.3 Generality and 4.4 Portability: Generality measures whether updated knowledge transfers to semantically similar prompts, reducing overfitting to a particular input.Portability instead evaluates transfer to logically related edits, such as reversed or neighboring relations.
  • 4.5 Retainability: Retainability measures performance changes after consecutive edits, and existing approaches often fall short as updates accumulate.The metric quantifies how sequential editing affects overall editing performance.
  • 4.6 Scalability: Scalability measures an editing strategy’s ability to incorporate many edits simultaneously.SERAC supports a maximum of 75 edits, whereas MEMIT enables thousands without significant influence on editing accuracy.

5 METHODOLOGIES

The methodologies section organizes existing KME strategies by how and where new knowledge is injected into pretrained LLMs, while emphasizing their technical details, insights, shortcomings, and relationships.

  • 5 METHODOLOGIES: The survey introduces a taxonomy of KME strategies based on the location and mechanism of knowledge injection.It analyzes methods from each category together with their technical details, insights, shortcomings, and relationships.

5.1 Categorization of KME Methods

KME methods are categorized into external memorization, global optimization, and local modification according to where and how new knowledge enters the model. The categories differ in their trade-offs across evaluation metrics and deployment settings.

  • 5.1 Categorization of KME Methods: The survey categorizes KME methods into external memorization, global optimization, and local modification.The taxonomy is based on whether knowledge enters external parameters or internal weights, and whether it is optimized or directly incorporated.
  • 5.1 Categorization of KME Methods: External memorization stores new knowledge outside the pretrained weights, while global optimization and local modification alter internal model components through different mechanisms.The figure and taxonomy organize corresponding representative works under these categories.
  • 5.1 Categorization of KME Methods: External memorization is advantageous for massive editing under limited computational resources because its memory size is controllable.The categories exhibit different strengths and weaknesses across the survey’s evaluation metrics.

5.2 External Memorization

External memorization edits LLM behavior by storing new knowledge in external memory or supplementary parameters while preserving the original weights. Memory-based methods retrieve stored edits during inference, whereas extension-based methods incorporate added components into the model process.

  • 5.2 External Memorization: External memorization introduces additional trainable parameters while leaving the pretrained parameters unchanged.This minimizes influence on existing knowledge and allows the memory capacity to expand with additional edits.
  • 5.2.2 Memory-based Strategies: Memory-based methods store edited knowledge outside the intrinsic model architecture and let the LLM access it during inference.Edits are generally converted into text using predefined templates.
  • 5.2.2 Memory-based Strategies: SERAC caches edits, uses a scope classifier to detect relevant prompts, and applies a counterfactual model only when an edit is in scope.Otherwise, the original model produces the output; SERAC is gradient-free with respect to the target label and pretrained parameters.
  • 5.2.2 Memory-based Strategies: Prompt-based methods such as IKE store factual demonstrations as in-context prompts to balance generality and locality.They can update factual knowledge efficiently, but the survey notes limitations for questions involving multiple relations.
  • 5.2.3 Extension-based Strategies: Extension-based methods add supplementary parameters, including modified FFN outputs, trainable patches, or adapters, to represent new knowledge.Examples include CALINET, T-Patcher, and GRACE, with patches designed to target specific knowledge while preserving other parameters.
  • 5.2 External Memorization: External memorization minimally perturbs the original model and supports precise adjustments without fully overhauling its architecture.Its principal trade-off is the need to acknowledge limitations associated with external interventions and added components.

5.3 Global Optimization

Global optimization methods update all model parameters while adding constraints or intermediate mechanisms to preserve unedited knowledge. They offer generality for related knowledge but face locality, instability, computational-cost, and scalability challenges.

  • Overview: Global optimization strategies inject new knowledge by updating all LLM parameters while designing preservation mechanisms for irrelevant knowledge.
  • Constrained Fine-tuning: Constrained fine-tuning restricts loss changes or parameter updates on unmodified knowledge to reduce overfitting and forgetting.Methods constrain losses, parameter norms, or the fraction of parameters updated; RecAdam additionally uses annealing to avoid massive early updates.
  • Meta-learning: Meta-learning regulates edits across varied tasks so models can adapt quickly to new edits while reducing overfitting to individual tasks.Editable Training uses a model-agnostic training approach to rectify errors while preserving performance on irrelevant instances.
  • Intermediate Fine-tuning: Intermediate fine-tuning uses a smaller intermediate model to obtain updated parameters more efficiently than altering all parameters directly.Hyper-networks can calculate weight deviations, while MEND transforms standard fine-tuning gradients into low-rank weight edits using small auxiliary networks.
  • Summary: Global optimization provides generality for relevant knowledge but cannot easily preserve locality because fine-tuning affects all model parameters.These methods can also be unstable, computationally expensive, time-consuming, and difficult to scale to multiple edits.

5.4 Local Modification

Local modification methods locate weights or neurons associated with target knowledge and edit only those components. This reduces interference with unrelated knowledge while supporting groundtruth-based, prompt-based, and layer-level editing strategies.

  • Overview: Local modification decomposes KME into locating knowledge-related weights and editing them to produce the desired answer.Updating only a small fraction of weights is intended to avoid negatively influencing irrelevant information.
  • Knowledge Neurons: Knowledge-neuron approaches treat parts of FFN layers as storage locations for factual information, although identifying them is difficult in transformer architectures.The FFN’s second-layer weights are described as memory retrieved using activation patterns produced by earlier weights.
  • Groundtruth-based Strategies: Groundtruth-based strategies use the correct target answer to identify and edit relevant knowledge neurons in a top-down manner.They can modify individual FFN weights or other selected components according to supervision from the desired output.
  • Layer-level Editing: ROME edits FFN memory using key-value associations, whereas MEMIT distributes updates across multiple MLP layers for mass knowledge editing.MEMIT shares updates across layers and spreads residuals over critical FFN layers; PMET later improves residual attribution.
  • Prompt-based Strategies: Prompt-based strategies mask or corrupt key information and use activation or probability changes to locate components causally associated with the target knowledge.ROME identifies layers through mediated effects of hidden representations, while MEMIT extends related FFN editing to multiple facts.
  • Summary: Local modification enables efficient updates without adding weights or optimizing the whole network, but locating the correct knowledge-bearing components remains central.

6 DATASETS

KME evaluation datasets are divided into generation and classification groups and draw on knowledge graphs, Wikipedia, and crowd-sourced data. Generation benchmarks emphasize precise factual outputs, while classification benchmarks use discrete labels for exact evaluation.

  • Dataset Types: KME datasets comprise generation datasets with textual outputs and classification datasets with categorical outputs, adapted from several data sources.Sources include knowledge graphs, Wikipedia pages, and crowd-sourced responses.
  • Generation Datasets: Generation datasets primarily evaluate factual knowledge editing because relational records provide definitive answers and precise evaluation targets.They are generally curated with diverse relational contexts and prompt forms.
  • Generation Datasets: zsRE uses human-generated question paraphrases to test whether edited models remain effective on semantically equivalent inputs.Each entry links a question to a Wikipedia sentence that supplies the factual basis.
  • Generation Datasets: CounterFact tests whether edits produce generalized factual changes rather than superficial alterations in word choice.Its records contain knowledge triples and crafted prompt templates involving recognized Wikidata entities.
  • Multimodal Datasets: VLKEB targets vision-language model editing by combining multimodal knowledge-graph data with an extended Portability metric.It addresses modality differences, complex model components, and limited data for LVLM editing.
  • Classification Datasets: Classification datasets pair prompts with discrete labels, enabling precise evaluation without defining a specific textual output space.FEVER evaluates binary fact verification using sentence-level evidence.

7 APPLICATIONS

KME supports applications that require models to remain current as information changes. The survey highlights question answering, fact-checking, and natural language generation as key downstream settings.

  • Overview: KME can inject knowledge into downstream models precisely and efficiently across question answering, fact-checking, and natural language generation.
  • Question Answering: In question answering, KME updates specific answers while preserving responses to unrelated inputs, helping models remain current as new information appears.Relation-based QA is especially suitable for evaluation because it requires retrieving precise real-world facts.
  • Fact Checking: Fact-checking models require continual updates because evolving evidence and guidance can make previously learned facts outdated.The survey connects KME with maintaining currentness, trustworthiness, and accuracy in changing information environments.
  • Observed Outcomes: An evaluated KME application reports significant improvements over baselines and generalization to other in-scope inputs from the same Wikipedia page.
  • Natural Language Generation: Natural language generation can use KME to keep generated content relevant and accurate as the global information landscape evolves.The application also aims to avoid false statements that could mislead users.

8 DISCUSSION

The survey identifies unresolved challenges in KME, including balancing locality with generality, scaling edits, and developing more practical editing paradigms. It also highlights future directions involving optimization-free, automated, continual, robust, and fair editing.

  • Trade-off between Locality and Generality: KME methods must balance locality against generality because improving one property can compromise the other.MEMIT and ROME generally preserve locality, while increasing memory size can improve locality but reduce generality.
  • Editing at Scale: Editing at scale remains difficult because many edits can contradict pre-existing knowledge and damage model coherence.As edit counts increase, locality and generality performance can both decline.
  • Optimization-Free Editing: Optimization-free KME could avoid parameter updates, but demonstration-based approaches are difficult to scale and often retain knowledge unsatisfactorily.IKE uses demonstration contexts to guide copying, updating, and retaining predictions without parameter updating.
  • Auto-Discovery of Editing Targets: Automating editing-target discovery is needed because identifying and prioritizing new knowledge from rapidly expanding raw resources is labor-intensive and costly.Potential sources include websites and social media.
  • Continual Editing: Continual editing must correct newly encountered issues while preventing edited models from forgetting or contradicting previous edits.This requirement is especially relevant to online question-answering systems that receive reports of incorrect answers during use.
  • Robust Editing: Robust KME should preserve factual accuracy while resisting adversarial perturbations, misinformation, and maliciously injected harmful knowledge.The survey also identifies fairness as a future concern because pre-training can incorporate biases.

9 CONCLUSIONS

The survey reviews KME techniques for efficiently updating specific knowledge in pretrained LLMs. It unifies the field through a constrained formulation, structured taxonomy, evaluation resources, applications, and research directions.

  • 9 CONCLUSIONS: The survey provides a comprehensive review of KME techniques for precise and efficient knowledge updates in pretrained LLMs.It covers methods, challenges, opportunities, datasets, metrics, and applications.
  • 9 CONCLUSIONS: It formulates KME as a constrained optimization objective that jointly addresses editing accuracy and retention.The formulation is intended to generalize across different KME strategies.
  • 9 CONCLUSIONS: The survey organizes existing techniques into a structured taxonomy and identifies future challenges and directions for advancing practical KME.It also notes that some techniques require specific dataset structures for training or evaluation and highlights real-world applications.
Loading 2310.16218v4…