Source-linked AI summary
Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge
Yasumasa Onoe, Michael J. Q. Zhang, Shankar Padmanabhan, Greg Durrett, Eunsol Choi
TL;DR
Language models can become outdated, and prior knowledge-editing evaluations largely test whether edited facts are reproduced rather than whether models can reason from them. The paper evaluates propagation from new-entity definitions using ECBD and a controlled cloze benchmark. Parameter updates show limited propagation, while prepending definitions improves performance across settings.
Problem
Prior knowledge-editing evaluations test whether injected facts are reproduced and unrelated predictions remain unaffected, but provide limited evidence about whether models can reason from updated facts.
Source: §1 · p. 1Method
The paper injects definitions of new entities using knowledge-editing methods and evaluates cloze completions on ECBD and a manually designed controlled benchmark.
Source: §1 · p. 2Results
Takeaways & Limitations
The results show substantial challenges for current update strategies in propagating knowledge from entity descriptions.
Source: §8 · p. 9The experiments cover English models and emerging entities unseen during pretraining, excluding updates to existing entities.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
Pre-trained language models (LMs) are used for knowledge intensive tasks like question answering, but their knowledge gets continuously outdated as the world changes. Prior work has studied targeted updates to LMs, injecting individual facts and evaluating whether the model learns these facts while not changing predictions on other contexts. We take a step forward and study LMs' abilities to make inferences based on injected facts (or propagate those facts): for example, after learning that something is a TV show, does an LM predict that you can watch it? We study this with two cloze-style tasks: an existing dataset of real-world sentences about novel entities (ECBD) as well as a new controlled benchmark with manually designed templates requiring varying levels of inference about injected knowledge. Surprisingly, we find that existing methods for updating knowledge (gradient-based fine-tuning and modifications of this approach) show little propagation of injected knowledge. These methods improve performance on cloze instances only when there is lexical overlap between injected facts and target inferences. Yet, prepending entity definitions in an LM's context improves performance across all settings, suggesting that there is substantial headroom for parameter-updating approaches for knowledge injection.
1 Introduction
The paper asks whether language models can reason from injected knowledge about new entities, beyond reproducing edited facts. It evaluates this with two cloze benchmarks and finds that parameter updates struggle to propagate knowledge, while prepending definitions works well.
1 Introduction
- Prior editing evaluations test whether injected facts are reproduced and unrelated sentences remain unaffected, but not whether models can reason from the new facts.The paper frames this missing evaluation as entity knowledge propagation.Source: §1 · pp. 1–2
- The study evaluates knowledge propagation by injecting entity definitions with editing methods and testing cloze completions across sentences about the entity.The authors call this task entity knowledge propagation.Source: §1 · p. 2
- ECBD uses novel entities, definitions, and Wikipedia probe sentences, while ENTITY INFERENCES provides manually designed probes for a more controlled evaluation.The paper notes that much of Wikipedia's information may not be inferable from an entity definition.Source: §1 · p. 2
- Existing parameter-updating methods handle simpler inferences in ENTITY INFERENCES but fail to improve performance on ECBD.The paper reports that prepending entity definitions works well, while matching its performance through parameter updates is challenging.Source: §1 · p. 2
2 Entity Knowledge Propagation
Entity Knowledge Propagation (EKP) studies whether language models can use definitions of emerging entities to make later inferences about them. It formalizes the task around an entity, definition, probe, and gold completion, with update success and specificity as evaluation criteria.
2 Entity Knowledge Propagation
- EKP updates model parameters using an emerging entity’s definition, then tests whether the model can reason about that entity in later probes.The task targets entities unseen during pre-training.Source: §2 · p. 2
2.1 Task Definition
- Each example pairs an emerging entity and definition with a probe sentence and gold completion, and the update aims to raise the completion’s probability.The probe references the entity by name; the entity is assumed to come with a short definition.Source: §2.1 · p. 2
- Update success measures improvement on the target completion, while specificity measures how much the update changes perplexity for other entities.For specificity, the ideal perplexity difference is near zero; a positive difference means performance worsened on other entities.Source: §2.1 · pp. 2–3
- Unlike prior editing tasks that reveal the answer during the update, EKP tests inferences whose answers may have little overlap with the injected definition.Earlier work evaluates whether an injected answer is preserved for paraphrased queries.Source: §2.1 · p. 3
3 Constructing benchmarks for EKP
The benchmarks pair an entity definition with probe sentences and gold completions to test propagation of entity knowledge. They combine real-world ECBD examples with a controlled benchmark designed to make the target inference more explicit.
3.1 ECBD
- ECBD repurposes Wikipedia definitions and probe sentences for entities originating between January 2020 and September 2021, which were selected to be unseen by the studied models.Its probes come from the entity’s Wikipedia page and require predicting masked spans.Source: §3.1 · pp. 3–4
- ECBD-EASY retains cases where the masked target span appears verbatim in the entity definition, making propagation easier than in the full dataset.Boosting the probability of definition tokens can improve perplexity on the gold span.Source: §3.1 · p. 4
3 Constructing benchmarks for EKP
- The benchmarks measure propagation with per-token perplexity on ECBD masked spans or accuracy on ENTITY INFERENCES, and assess specificity on other entities’ probes.ECBD’s perplexity is not comparable across base models because architectures and tokenizers differ.Source: §3.1 · p. 4, §3.2 · p. 4
3.2 ENTITY INFERENCES
- Because ECBD probes can require rich knowledge and reasoning, the authors construct ENTITY INFERENCES for a more controlled study of knowledge propagation.Its handcrafted probes offer multiple-choice answer candidates rather than open-vocabulary span prediction.Source: §3.2 · p. 4
- ENTITY INFERENCES separates explicit probes, whose answers are stated in the definition, from implicit probes requiring commonsense-like information.An example of an implicit probe asks what people do with a TV show.Source: §3.2 · p. 4
4 Experimental Setup
The experiments use left-to-right and sequence-to-sequence language models, several parameter-update approaches, and definition prepending as an in-context comparison. Reported results vary by dataset, while the methods also differ in their computational costs.
4.1 Base Language Models
- The study uses GPT-Neo 1.3B and T5-large, and adds GPT2-XL to follow the ROME evaluation protocol.These cover left-to-right and sequence-to-sequence architectures.Source: §4.1 · p. 4
4.2 Parameter Updating Methods
- Fine-tuning minimizes loss on an entity-definition example, either across all parameters or only the last transformer layer; MEND transforms gradients into parameter updates, while ROME rewrites an MLP key-value pair.Each fine-tuning example starts from the original model checkpoint.Source: §4.2 · pp. 4–5
- On ENTITY INFERENCES, fine-tuning and ROME raise accuracy with specificity costs, while MEND is ineffective; on ECBD, no technique lowers perplexity, though ECBD-EASY shows some gains.The table caption reports the comparison across the evaluation datasets.Source: Table 3 · p. 5
4.3 Input Augmentation
- Input augmentation prepends an entity definition to the probe, with a randomly chosen other entity’s definition as a comparison baseline.The longer input increases inference-time computation and does not update model parameters.Source: §4.3 · pp. 5–6
4.4 Computational Cost
- Input augmentation increases inference-time cost and scales poorly for updating models with many new entities, whereas gradient updates incur upfront costs and MEND adds editor-training costs.After MEND’s meta-network is trained, updating individual facts becomes cheaper, but the meta-network must be retrained for each language model.Source: §4.4 · p. 6
5 Results
Results vary with inference difficulty: parameter updates help on ENTITY INFERENCES and ECBD-EASY but not ECBD, while prepending definitions brings robust gains across settings.
5 Results
- Prepending definitions yields robust, consistent performance gains across the tested settings and base models.Random definitions hurt GPT-Neo but do not affect T5, suggesting the injected definitions contain relevant information.Source: §5 · p. 6
5.1 ENTITY INFERENCES
- Full-model fine-tuning raises post-edit accuracy by more than 20 points for all three base models, while specificity drops by 15.9 points on GPT-Neo and 8.8 points.Source: §5.1 · p. 6
- MEND changes specificity by only +0.3, while on GPT2-XL ROME improves editing performance by +30 versus fine-tuning’s +23.5, with specificity changes of -8.8 versus -2.0.Source: §5.1 · p. 6
5.2 ECBD
- On ECBD, no editing method substantially decreases perplexity or increases specificity; MEND instead increases perplexity for GPT-Neo.ROME was not reported because its entity-relation-object format poorly fits general definitions; forcing that format produced perplexities over 100 on both ECBD sets.Source: §5.2 · p. 6
5.3 ECBD-EASY
- On ECBD-EASY, GPT-Neo fine-tuning and MEND decrease perplexity by 9.0 and 8.5, respectively, while T5-large’s post-edit perplexity is unchanged.Figure 2 reports that fine-tuning and MEND approach train-on-test on ECBD-EASY but fail on ECBD.Source: §5.3 · p. 6, Figure 2 · p. 7
- The easier-subset gains suggest propagation is achievable, but existing editing methods may only reproduce knowledge directly injected into the model.The authors use this finding to motivate further analysis of the task’s challenges.Source: §5.3 · p. 7
6 Analysis
The analysis compares editing with an upper-bound setting and probes how definition–probe overlap relates to performance. Parameter updates fare differently across datasets, and overlap is informative but not uniformly predictive.
6.1 Targeted Update / Specificity Tradeoff
- Train-on-Test estimates a fine-tuning upper bound by making each definition sentence identical to its probe sentence.The analysis varies the number of update epochs; its ECBD and ECBD-EASY comparisons track perplexity and specificity.Source: §6.1 · p. 7
- On ECBD, longer fine-tuning worsens both specificity and perplexity, while prepending definitions performs robustly.The authors suggest the problem may lie in how the data is used during learning rather than in the data itself.Source: §6.1 · p. 7
- On ECBD-EASY, MEND and fine-tuning approach the train-on-test upper bound with substantial perplexity improvement and only mild specificity degradation.The authors note that this setting resembles exact-fact editing, and caution that success there need not transfer to realistic propagation.Source: §6.1 · p. 7
6.2 Information Overlap
- Performance improves on average in both datasets when the gold span appears in the definition; ECBD’s overlap trend is stronger with input augmentation.Figure 3 groups GPT-Neo examples by whether the gold span is included; most ECBD probes are not included, and few of those show substantial perplexity gains.Source: Figure 3 · p. 8, §6.2 · p. 8
6.1 Targeted Update / Specificity Tradeoff
- On ENTITY INFERENCES, fine-tuning can match prepending definitions in accuracy, but a substantial gap from the gold setting remains.The gold setting saturates accuracy quickly, and some examples are not readily learnable in the current data setup.Source: §6.1 · pp. 7–8
6.2 Information Overlap
- ENTITY INFERENCES is slightly more optimistic than ECBD when the answer is absent from the definition.Source: §6.2 · p. 8
- The overlap analysis bins examples by lexical and semantic similarity, measuring gold-answer rank change on ENTITY INFERENCES and perplexity change on ECBD.Figure 4 visualizes performance gaps across Jaccard-similarity bins, where larger values mean more similar definition and probe sentences.Source: §6.2 · pp. 8–9
- For ENTITY INFERENCES, higher definition–probe Jaccard similarity tracks progressively more negative changes in gold-answer rank, but ECBD shows almost no such correlation with perplexity change.Figure 4 breaks performance down by lexical-similarity bins; the authors suggest ECBD’s inference patterns may be qualitatively more challenging.Source: Figure 4 · p. 8, §6.2 · p. 9
7 Related Work
Related work covers methods for keeping language models current and studies of content transfer from descriptions. This paper’s setting is close to the latter, but adds parameter updates.
7 Related Work
- Prior work evaluates changing facts about existing entities through open-retrieval or cloze-style question answering, and reports success from continued pretraining and domain adaptation.The cited work also explores updating localized parameter sets associated with memorized facts.Source: §7 · p. 9
- Choi et al. (2022) contrast prepending information with fine-tuning and find that fine-tuning generally works worse, framing their approach as distillation.Source: §7 · p. 9
- West et al. (2022) study whether language models can generate statements consistent with an entity description, but do not update model parameters from those descriptions.Source: §7 · p. 9
8 Conclusion
The paper studies whether descriptions of new entities can be injected into language models and then used for inference. Its conclusion is that current update strategies lag behind simply prepending the definition.
8 Conclusion
- The paper frames entity knowledge propagation as injecting descriptions of new entities into language models and testing inference based on them.Source: §8 · p. 9
- Fine-tuning and efficient update strategies enable reproduction of exact facts from descriptions, but inference from those facts is substantially harder.Source: §8 · p. 9
- Across the two datasets, update strategies lag behind simply prepending the definition in context, and the authors call for further work.Source: §8 · p. 9
Limitations
The study focuses on updating knowledge about emerging entities, not changes to existing entities. Its experiments also cover only English-language models; broader settings remain to be tested.
Limitations
- The study excludes updates to existing entities, focusing instead on emerging entities whose information is completely unseen during pretraining.The authors cite the complexity of existing-entity changes as the reason for this scope boundary.Source: Limitations · p. 9
- An organization’s name change could affect many related entities, illustrating why existing-entity updates may become intractable.Source: Limitations · p. 9
- Experiments use English-language models only; larger models and newer parameter-tuning approaches remain unbenchmarked.The authors say multilingual results may differ and call for broader benchmarking.Source: Limitations · p. 9
A.2 Harmful Data Instances
The appendix describes dataset screening and modeling settings, details the controlled inference benchmark, and compares dataset complexity and ROME’s behavior under different knowledge conditions.
A.2 Harmful Data Instances
- The authors screen manually created entity-inference examples to exclude offensive or harmful content; the data and experiments are in English.Other data are publicly available from Wikipedia.Source: §A.2 · p. 11
A.3 Modeling Details
- Training uses batch size 1, with 5 epochs on ECBD and 10 on ENTITY INFERENCES; each experiment reflects a single run.Learning rates are 3e-6 for ECBD and 5e-4 for ENTITY INFERENCES; experiments used less than 4 GPU hours.Source: §A.3 · p. 11
A.4 ENTITY INFERENCES
- ENTITY INFERENCES uses Wikipedia entities tagged with TV shows and natural disasters plus fabricated people, with explicit definition-based and implicit commonsense-like probes and 6–12 answer candidates per probe; examples average 10 candidates.Source: §A.4 · pp. 11–12
A.5 More Similarity Scores
- The appendix compares performance across lexical similarity levels using Jaccard, Rouge-L, and BERT Score.Figure 5 breaks down performance by similarity between probe and definition sentences.Source: §A.5 · p. 12, Figure 5 · p. 13
A.6.1 Comparison of datasets
- ECBD combines unknown entities with sometimes non-descriptive labels, while ENTITY INFERENCES usually has unknown entities but known labels.The authors suggest this greater complexity may explain ECBD’s drastic perplexity increases.Source: §A.6.1 · p. 12, Table 6 · p. 14
A.6.2 ROME Test Generation
- ROME generates reasonable text when either the subject or label is known, but fails to incorporate prompt information when both are unknown.The authors relate this pattern to ROME’s difficulty locating editable parameters when neither key nor value is well known.Source: §A.6.2 · p. 12, Table 8 · p. 15