Source-linked AI summary
Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases
Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingyong Yan, Meng Liao, Tong Xue, Jin Xu
TL;DR
The paper addresses the lack of rigorous understanding of why masked language models achieve factual knowledge extraction success and whether their predictions are reliable. It studies their mechanisms across three extraction paradigms and finds that prompt bias, type guidance, and answer leakage explain much of the observed performance, questioning their reliability as factual knowledge bases.
Problem
The lack of rigorous studies of MLM prediction mechanisms leaves the reliability and conditions of their factual knowledge predictions unclear.
Method
The paper thoroughly studies MLM mechanisms across three representative factual knowledge extraction paradigms, including prompt-based retrieval, illustrative cases, and external contexts.
Results
Previous performance mainly reflects prompt fit to dataset answer distributions, while illustrative cases provide type guidance and external contexts contain correct answers explicitly or implicitly.
Takeaways & Limitations
These findings strongly question whether current MLMs can serve as reliable factual knowledge bases.
Takeaways & Limitations
The study focuses on masked language models and leaves generative pretrained models such as GPT-2 and GPT-3 for future work.
Abstract
from arXiv · showhide
Previous literatures show that pre-trained masked language models (MLMs) such as BERT can achieve competitive factual knowledge extraction performance on some datasets, indicating that MLMs can potentially be a reliable knowledge source. In this paper, we conduct a rigorous study to explore the underlying predicting mechanisms of MLMs over different extraction paradigms. By investigating the behaviors of MLMs, we find that previous decent performance mainly owes to the biased prompts which overfit dataset artifacts. Furthermore, incorporating illustrative cases and external contexts improve knowledge prediction mainly due to entity type guidance and golden answer leakage. Our findings shed light on the underlying predicting mechanisms of MLMs, and strongly question the previous conclusion that current MLMs can potentially serve as reliable factual knowledge bases.
1 Introduction
The paper investigates whether masked language models can serve as reliable factual knowledge bases by examining three extraction paradigms and their prediction mechanisms. It finds that apparent success is largely explained by prompt bias, type guidance, and answer leakage rather than reliable factual extraction.
- Research scope: The study examines prompt-based retrieval, case-based analogy, and context-based inference as three factual knowledge extraction paradigms.These paradigms query a masked language model with a relation prompt alone, illustrative cases, or external relevant context, respectively.
- Prompt-based retrieval: Prompt-based retrieval produces predictions that are biased toward the applied prompt rather than the facts being extracted.The paper reports that previous decent performance mainly stems from prompts overfitting dataset answer distributions.
- Case-based analogy: Illustrative cases mainly improve object-type recognition, but do not reliably identify the exact target entity among entities sharing that type.The rank of the answer within its entity type changes randomly after cases are introduced.
- Context-based inference: External context improves extraction mainly when it explicitly or implicitly contains evidence for reconstructing the correct answer.Even masked answers can help when the model correctly reconstructs them from the remaining context.
- Overall implication: The findings indicate that current masked language models are not reliable factual knowledge sources under these extraction settings.The study presents its analysis as evidence against treating current MLMs as reliable factual knowledge bases.
2 Related Work
Related work frames factual knowledge extraction as part of broader efforts to probe pretrained language models and improve their prompting, demonstrations, and contextual inputs. It also documents growing concerns about reliability and spurious dependence on surface forms.
- Knowledge probing: Prior research probes pretrained language models for linguistic, semantic, and world knowledge.The cited work includes studies of factual and other forms of knowledge encoded in pretrained models.
- Reliability concerns: Recent studies question pretrained models' reliability because predictions can depend spuriously on surface forms and be sensitive to negation or mispriming.These findings motivate closer examination of whether apparent knowledge extraction performance reflects factual knowledge.
- Extraction paradigms: Knowledge extraction research commonly uses prompt-based retrieval, case-based analogy, and context-based inference.These paradigms differ in whether they add learned or mined prompts, illustrative cases, or external relevant contexts.
3 Prompt-based Retrieval
Prompt-based retrieval is strongly shaped by prompt-induced prediction distributions rather than dataset facts. Across comparisons, apparent performance gains reflect prompt–dataset fit, while subjects provide only limited additional guidance.
- 3 Prompt-based Retrieval: Prompt-based retrieval queries an MLM with a subject, relation prompt, and masked object, using manually created, mined, or automatically searched prompts.The paper illustrates retrieval with the query “Steve Jobs was born in [MASK].”
- 3.1 Overall Conclusion: Prompt-based retrieval is prompt-biased, so earlier performance measures how well prompts fit dataset answer distributions rather than factual extraction ability.The paper states this as its first overall conclusion.
- 3.2 Different Answers, Similar Predictions: MLMs produce highly similar prediction distributions for LAMA and WIKI-UNI despite the datasets having different answer distributions.Across prompt types, prediction-distribution correlations exceed 0.8 for more than half of relations.
- 3.2 Different Answers, Similar Predictions: In WIKI-UNI, top-5 answers cover 7.78% of instances, whereas top-5 predictions cover more than 52%, yielding mean P@1 values of 16.47 versus 30.36 on LAMA.The mismatch shows that prediction distributions track LAMA-like concentration rather than WIKI-UNI’s answer distribution.
- 3.3 Prompts Dominates Predictions: Prompt-only distributions correlate strongly with full predictions, with coefficients above 0.6 for more than half of relations across all three prompt types.The paper characterizes this behavior as guided guessing: prompt-biased distributions are moderated by subjects.
- 3.3 Prompts Dominates Predictions: For a minority of relations, lower prompt-only correlations arise mainly from subject entities’ type-selectional preferences.This type-guidance mechanism is developed further in the case-based analysis.
- 3.4 Better Prompts are Over-Fitting: Smaller KL divergence between a prompt-only distribution and LAMA’s answer distribution predicts better prompt performance.Thus, better-performing prompts can reflect compatibility with the test-set distribution rather than stronger retrieval ability.
- 3.4 Better Prompts are Over-Fitting: Prompt improvements on LAMA are attributed to over-fitting dataset answer distributions, not to prompts with stronger semantic associations to relations.The paper therefore characterizes prior prompt-search studies as optimizing prompt–dataset compatibility.
4 Case-based Analogy
Case-based analogy improves extraction mainly by helping MLMs recognize object types, not by selecting the correct entity within a type. The analysis uses induced relation types and in-type rank to separate these effects.
- Overall conclusion: Illustrative cases guide MLMs toward the correct object type rather than directly improving factual entity prediction.The paper induces relation object types from Wikidata taxonomy to analyze this mechanism.
- Entity-set type induction: The induced type is the finest-grained type covering more than 80% of a relation’s entities after constructing and sorting an entity type graph.The algorithm recursively adds ancestor types, then selects the first sufficiently covering type in fine-to-coarse order.
- Overall performance: Case-based analogy raises mean precision from 30.36% to 36.23%, with 11.81% of instances benefiting and 5.94% being undermined.These results are obtained after adding ten illustrative cases whose objects exclude the query’s golden answer.
- Entity prediction: Illustrative cases improve overall ranking mainly through type recognition, while providing limited help for selecting the answer among entities of the same type.The overall-rank improvement does not reflect better within-type entity prediction.
- Entity prediction: The change in in-type rank is nearly random, with Raised/Unchanged/Dropped proportions of 33.05%/35.47%/31.47%.In-type MRR changes only from 0.491 to 0.494, indicating little improvement in choosing among same-type entities.
5 Context-based Inference
External context improves MLM factual prediction mainly when it provides sufficient evidence for the answer, either explicitly or through reconstructable implicit leakage. Masking the answer preserves gains primarily when the remaining context can recover it.
- Overall conclusion: Additional context improves knowledge prediction mainly because it contains the correct answer explicitly or implicitly.The evaluation separates contexts according to whether the answer appears and whether masked answers can be reconstructed.
- Explicit answer leakage: Contexts containing the answer significantly improve extraction, whereas contexts without it can produce a slight performance drop.The comparison groups instances by whether the additional context contains the answer.
- Implicit answer leakage: Masking the answer still leaves a significant performance gain when the surrounding context provides enough information to reconstruct it.This tests whether context helps beyond simply displaying the answer.
- Implicit answer leakage: Most masked-context improvement comes from instances where MLMs correctly reconstruct the masked answer from the remaining context.Contexts whose masked answers cannot be reconstructed yield relatively minor improvements.
6 Conclusions and Discussions
The paper concludes that MLM factual extraction is driven by prompt bias, type guidance, and explicit or implicit answer leakage rather than reliable fact retrieval. It discusses implications for probing and external-context use, while limiting its scope to masked language models.
- Conclusions: Across three extraction paradigms, prompt retrieval is prompt-biased, illustrative cases mainly provide type guidance, and context helps mainly through answer leakage.These mechanisms collectively question whether current MLMs can serve as reliable factual knowledge bases.
- Implications: The findings provide explanations for knowledge-intensive techniques and suggest debiasing evaluation of probing datasets.The paper connects its mechanism analysis to external-context methods and PLM probing reliability.
- Future directions: The mechanisms identified may support type-centric prediction tasks and the evaluation, selection, and denoising of external contexts.The paper names NER, factoid QA, and CommonsenseQA as examples of affected directions.
- Limitations: The study focuses on masked language models and leaves generative pretrained models such as GPT-2 and GPT-3 for future work.The authors identify this as a scope boundary attributed to page limitations.
- Evaluation design: The WIKI-UNI dataset contains 70K facts with a uniform answer distribution, contrasting with the 34K facts in LAMA.It uses the same 41 relations and balances object frequencies to test whether predictions follow dataset artifacts.
B Results on RoBERTa-large
The authors report that the paper’s conclusions are similar for BERT-large and RoBERTa-large, while presenting BERT-large results in the main article.
- Results on RoBERTa-large: The conclusions are similar on BERT-large and RoBERTa-large, although the article reports BERT-large results in the main text.RoBERTa-large results are provided separately.
B.1 Promp-based Retrieval
For RoBERTa-large, prediction distributions on LAMA and WIKI-UNI are highly correlated across prompts, indicating that prompt-based retrieval can produce similar outputs across different datasets.
- Pearson correlations between LAMA and WIKI-UNI prediction distributions exceed 0.9 for most relations across all three prompt types.The comparison concerns RoBERTa-large.
- The analysis compares prediction distributions across datasets and prompt types rather than only individual factual answers.Figure 7 summarizes correlations for RoBERTa-large across LAMA and WIKI-UNI.
B.2 Case-based Analogy
Illustrative cases substantially improve RoBERTa-large’s extraction performance, while the analysis indicates that prediction changes are tied to entity-type and surface-form guidance.
- Illustrative cases significantly increase knowledge-extraction performance for RoBERTa-large.Table 9 reports performance for the case-based analogy paradigm.
- The supplementary analysis reports top-k object-entity coverage and changes in overall and in-type ranks for RoBERTa-large.These analyses characterize how illustrative cases alter candidate rankings and coverage.
- BERT-large achieves 68% type precision versus 51% for RoBERTa-large in the prompt-based paradigm.The type-precision difference partly explains RoBERTa-large’s significantly worse performance than BERT-large on LAMA.
- Illustrative cases provide type guidance and can also provide surface-form guidance in some relations.Surface-form guidance occurs when an object entity name is a substring of the subject entity name.
B.3 Context-based Inference
Context-based inference improves factual extraction mainly when external contexts contain the answer or allow it to be reconstructed; contexts without those signals can reduce performance.
- Context-based inference significantly improves factual extraction when the external context contains the answer.The comparison groups RoBERTa-large instances by whether the answer is present in context.
- Performance drops for instances whose context does not contain the answer.The result is reported in the grouped comparison for RoBERTa-large.
- Overall improvements from external contexts mainly come from cases where the masked answer can be reconstructed from context.The analysis compares masked contexts according to whether answer reconstruction is possible.
- The study reports overall performance after introducing different contexts for RoBERTa-large.Table 11 summarizes the aggregate effect across context types.
C Full Version of the Type Prediction Results
The full type-prediction analysis examines case-based analogy results for BERT-large and RoBERTa-large, including how illustrative cases alter predictions across relations.
- Illustrative cases can supply type guidance and surface-form guidance beyond the effects discussed in the article.The additional finding concerns relation-level behavior in the case-based analogy paradigm.