Source-linked AI summary
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, Hannaneh Hajishirzi
TL;DR
Language models’ parametric memories may be insufficient for less popular factual knowledge, raising questions about when their outputs should be trusted. The paper probes ten models and retrieval approaches on long-tail QA, finding that retrieval helps where memorization is weak but can mislead on popular entities, motivating Adaptive Retrieval. Adaptive Retrieval improves performance while reducing inference costs, though the findings’ applicability beyond the controlled entity-centric setting remains limited.
Problem
The paper investigates when language models’ parametric factual memories are reliable and how scaling or non-parametric memories address weak long-tail memorization.
Method
The study evaluates ten language models and retrieval augmentations on POPQA, a 14k-question Wikidata-derived dataset, and EntityQuestions.
Results
Retrieval helps substantially on long-tail questions, while retrieval can hurt large models on popular entities; Adaptive Retrieval improves POPQA performance by up to 10%.
Takeaways & Limitations
Combining parametric and non-parametric memories selectively can improve retrieval-augmented LM performance while reducing inference costs, including halving GPT-3 API costs.
Takeaways & Limitations
The controlled experiments use synthetic datasets, so applicability to naturally occurring factual knowledge has not been firmly established.
Abstract
from arXiv · showhide
Despite their impressive performance on diverse tasks, large language models (LMs) still struggle with tasks requiring rich world knowledge, implying the limitations of relying solely on their parameters to encode a wealth of world knowledge. This paper aims to understand LMs' strengths and limitations in memorizing factual knowledge, by conducting large-scale knowledge probing experiments of 10 models and 4 augmentation methods on PopQA, our new open-domain QA dataset with 14k questions. We find that LMs struggle with less popular factual knowledge, and that scaling fails to appreciably improve memorization of factual knowledge in the long tail. We then show that retrieval-augmented LMs largely outperform orders of magnitude larger LMs, while unassisted LMs remain competitive in questions about high-popularity entities. Based on those findings, we devise a simple, yet effective, method for powerful and efficient retrieval-augmented LMs, which retrieves non-parametric memories only when necessary. Experimental results show that this significantly improves models' performance while reducing the inference costs.
1 Introduction
The paper probes when language models’ parametric factual memories are reliable and whether retrieval can complement them. It finds weak long-tail memorization, limited gains from scaling, and benefits from retrieving only when necessary.
- Motivation: The work frames reliable deployment around identifying when LM outputs should not be trusted.The study asks how factual memorization varies and how scaling and retrieval affect reliance on parametric knowledge.
- Findings: LM memorization is concentrated on popular factual knowledge, while GPT-3 davinci-003 answers most long-tail questions incorrectly.For the 4,000 least popular POPQA questions, GPT-j 6B reaches 16% accuracy and GPT-3 davinci-003 reaches 19%.
- Findings: Scaling models does not significantly improve factual memorization on long-tail questions.The paper reports only a small difference between GPT-j 6B and GPT-3 davinci-003 on the least popular POPQA questions.
- Findings: Retrieval-augmented models substantially improve performance on long-tail questions and can outperform much larger unassisted models.A Contriever-augmented GPT-Neo 2.7B outperforms GPT-3 davinci-003 on the 4,000 least popular questions.
- Findings: Retrieved context can harm large models on popular-entity questions because it may be misleading.This motivates treating parametric and non-parametric memories as complementary rather than making retrieval universal.
- Adaptive Retrieval: Adaptive Retrieval combines parametric and non-parametric memories according to popularity and retrieves passages only when necessary.It improves POPQA performance by up to 10% while reducing inference costs, including halving GPT-3 API costs.
2 Related Work
Prior work established that pretrained and fine-tuned language models store substantial world knowledge and that corpus frequency correlates with memorization. This paper extends that line by predicting memorization from question-visible variables and studying query-level retrieval decisions.
- Parametric and non-parametric knowledge: Prior studies show that pretrained and closed-book language models can memorize substantial world knowledge in their parameters.This literature includes BERT probing and competitive closed-book QA with fine-tuned T5.
- Understanding memorization: String frequency and question-answer co-occurrence in pretraining corpora are positively associated with memorization and QA accuracy.The paper instead examines variables available directly in the input question.
- Benchmark context: POPQA is constructed from Wikidata triples to support analysis of factual knowledge across entity popularity.The figure caption describes sampling triples, converting them into natural-language questions, and calculating popularity.
- Retrieval-augmented LMs: This work introduces Adaptive Retrieval and studies whether to retrieve for each query rather than at the token level.It presents query-level retrieval decisions for retrieval-augmented LM prompting.
3 Evaluation Setup
The evaluation focuses on entity-centric factual knowledge in open-domain QA, using controlled benchmarks with long-tail entity popularity. POPQA is built from Wikidata triples, while the analysis uses popularity and relationship type to study memorization.
- 3.3 Benchmarks: POPQA and EntityQuestions provide long-tail open-domain QA distributions for testing factual memorization.POPQA contains entity-centric questions, while EntityQuestions supplies a second long-tail benchmark.
- 3.1 Focus and Task: The study defines factual knowledge as a subject–relationship–object triplet about an entity.This representation supports analysis of specific entity details and relationship types.
- 3.1 Focus and Task: Models answer open-domain questions without a provided ground-truth paragraph, using few-shot prompting without parameter updates.A prediction is correct when any substring exactly matches a gold answer.
- 3.2 Dimensions of Analysis: Entity popularity is used to analyze memorization, with Wikipedia page views serving as the popularity measure.The study emphasizes subject entity and relationship type because input questions expose those variables.
- 3.2 Dimensions of Analysis: The analysis acknowledges that GPT-3-style private pretraining corpora are unavailable, so corpus frequencies may not reflect the data used by the models.This limits direct interpretation of pretraining-frequency relationships.
- 3.3 Benchmarks: POPQA samples Wikidata triples across 16 relationship types and verbalizes them with manually written natural-language templates.Wikidata links enable reliable analysis of popularity and relationship types.
4 Memorization Depends on Popularity and Relationship Type
Factual memorization varies with subject popularity and relationship type: scaling improves overall performance mainly for popular knowledge, while long-tail accuracy remains limited. Popularity also predicts accuracy across relationship types, although some relationships permit surface-level guessing.
- Overall performance: GPT-3 achieves 35% accuracy and GPT-Neo 20B achieves 25% accuracy on POPQA without in-context examples.The evaluation covers ten models spanning OPT, GPT-Neo, and GPT-3 at varying scales.
- Popularity: Subject entity popularity positively correlates with accuracy across almost all relationship types, with correlation roughly 0.4 for GPT-3 003 versus approximately 0.1 for GPT-Neo-1.3B.The correlation is generally stronger for larger language models.
- Relationship type: Relationship type affects memorization, but some high-accuracy relationship types may be solved using surface-level clues or dominant answer patterns rather than memorized triples.Lower-correlation types such as country and sport often have higher accuracy, whereas occupation, author, and director show lower accuracy.
- Scaling: Scaling mostly improves memorization for questions about entities with log10(popularity) larger than 4, while lower-popularity performance remains relatively constant.This pattern appears on both POPQA and EntityQuestions.
- Scaling: 15%, 16%, and 19% accuracy are achieved by GPT-Neo 6B, GPT-Neo 20B, and GPT-3 davinci-003, respectively, on the 4,000 least popular questions.These results indicate that practical scaling does not move reliable memorization far into the long tail.
5 Non-parametric Memory Complements Parametric Memory
Retrieval-augmented LMs address parametric-memory gaps, especially for less popular entities, but retrieved evidence can hurt when models already know the answer.
- Retrieval augmentation significantly improves smaller models, with Contriever-augmented GPT-Neo 2.7B outperforming vanilla GPT-3.
- Contriever provides a 7% accuracy gain on GPT-3 davinci-003, while GenRead helps GPT-3 but offers little improvement for smaller models.GenRead can also impose substantial latency, reaching 70 seconds per GPT-NeoX 20B query.
- Retrieval-augmented models have the clearest advantage on less popular entities, although Contriever generally outperforms BM25 on POPQA.BM25 performs better on the least popular entities.
- For popular facts, retrieved context often fails to help or hurts because state-of-the-art LMs have already memorized the answers.
- Retrieval can cause errors when evidence quality is poor: 10% of questions become incorrect despite correct unassisted answers, with recall@1 of 0.14 versus 0.42 overall.
6 Adaptive Retrieval: Using Retrieval Only Where It Helps
Adaptive Retrieval combines parametric and non-parametric memory by retrieving only for questions below popularity thresholds. It improves accuracy while reducing inference costs, particularly for larger models.
- 6.1 Method: Adaptive Retrieval uses retrieval below a popularity threshold and parametric predictions for more popular entities.Thresholds are selected on development data to maximize adaptive accuracy independently for each relationship type.
- 6.2 Results: Adaptive Retrieval is especially effective for larger models, whereas smaller models retrieve almost all questions because their parametric memories are less reliable.
- 6.2 Results: GPT-3 davinci-003 with adaptive GenRead and Contriever achieves 46.5% POPQA accuracy, 5.3% above any non-adaptive method.
- 6.3 Efficiency: Adaptive Retrieval reduces inference time by up to 9% from standard retrieval and lowers API costs by 15% on EntityQuestions while maintaining equivalent performance to retrieval-only systems.Skipping retrieval shortens inputs and reduces latency in both retrieval and language-model components.
7 Discussion and Conclusions
The paper finds that factual memorization tracks entity popularity, scaling provides marginal long-tail gains, and retrieval helps long-tail questions but can mislead models on well-known entities. Adaptive Retrieval addresses this trade-off by retrieving only when necessary and is reported as both more powerful and more efficient.
- LM memorization correlates strongly with entity popularity, while scaling offers only marginal improvement on long-tail distributions.
- Non-parametric memories greatly aid long-tail questions but can mislead powerful LMs on well-known entities whose answers are already stored parametrically.
- Adaptive Retrieval uses entity popularity and relationship types to retrieve only when necessary, improving both effectiveness and efficiency.
Limitations
The study’s conclusions are scoped to entity-centric factual knowledge and synthetic datasets, and Adaptive Retrieval depends on a time-dependent popularity definition and QA-pipeline details.
- The results rely on two synthetic datasets, so their applicability to naturally occurring factual knowledge is not firmly established.
- Adaptive Retrieval’s effectiveness depends on details of the question-answering pipeline and on the time-dependent definition of popularity.
- Whether the findings apply to other world knowledge, such as commonsense, remains an open question.
Ethical Considerations
The evaluation focuses on general entity-centric knowledge, but the authors connect its findings to privacy, security, and potential reliability concerns affecting minority groups.
- Ethical Considerations: The evaluation focuses on memorization of general entity-centric knowledge, while noting that findings may apply to personal information memorized from the web.The paper cites security concerns associated with models memorizing personal information available online.
- Ethical Considerations: The findings suggest that language models may have less reliable knowledge of minority groups and disproportionately rely on stereotypes when uncertain.The authors identify retrieval augmentation as a possible direction for future bias investigation, not an established result.
B Experimental Details
The experiments combine cost- and resource-constrained evaluations with few-shot prompting, relation-specific adaptive thresholds, and broad accuracy analyses across datasets and models.
- Computational resources and API costs: GPT-3 API usage totaled $275 across 14,282 questions and four methods, with vanilla experiments costing $0.46 per 1000 questions.Contriever-, BM25-, and GenRead-augmented experiments cost $3.08, $2.80, and $3.25 per 1000 questions, respectively.
- Computational resources and API costs: Larger-than-2B-parameter models were run on a single 32GB V100 GPU using int8 quantization for OPT 13B and GPT-Neo 20B.Preliminary GPT-Neo 6B experiments found no notable performance drop from quantization.
- Experimental setup: Few-shot POPQA contexts use one sampled question-answer pair from each of 15 other relationship types, whereas EntityQuestions uses 15 random examples.The differing strategies reflect the datasets' relationship-type coverage.
- Adaptive Retrieval: Adaptive thresholds are selected on 75% of POPQA by brute-force maximization of accuracy, using retrieval below each relation-specific popularity threshold and parametric predictions above it.The learned thresholds are evaluated on the remaining 25% across 100 random splits.
- Results: The appendix reports full per-relationship accuracy and correlation results for POPQA and EntityQuestions, including a negative-correlation analysis for capital relations in EntityQuestions.The capital relationship shows a U-shaped popularity-performance pattern because some low-popularity answers occur in subject names.
- Results: Retrieval augmentation improves performance across models, with GPT-Neo 1.3B performing on par with GPT-3 on POPQA; smaller models rely especially heavily on retrieval for EntityQuestions.The appendix reports full relationship-type results and notes that EntityQuestions contains many low-popularity questions.
C.3 Qualitative Results
Qualitative analyses examine when retrieval augmentation helps or harms, attributing failures mainly to mistaken or irrelevant entity retrieval and successes to high-recall relevant evidence.
- Relationship-level analyses: Figures 16–19 report accuracy and correlations across all relationship types for POPQA and EntityQuestions.These results extend the relationship-level analyses beyond the primary figures.
- Retrieval successes: Retrieval helps when it supplies relevant information directly, with recall@1 of 0.88 for cases where Contriever augmentation enables a correct answer.One example is correct only by coincidence because “Pierre” refers to a South Dakota city rather than a basketball player.