Source-linked AI summary
Measuring and Narrowing the Compositionality Gap in Language Models
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, Mike Lewis
TL;DR
The paper asks whether language models’ question-answering success reflects memorization or compositional reasoning over separately learned facts. It measures this distinction with multi-hop datasets, finds a roughly constant compositionality gap as GPT-3 scales, and shows that elicitive prompting—especially self-ask with search—improves compositional answering.
Problem
It remains unclear how much language models’ strong question answering reflects memorization of huge corpora versus reasoning.
Method
The paper measures compositional reasoning with multi-hop datasets and introduces self-ask, which decomposes complex questions into answered follow-up questions before answering the original.
Results
The compositionality gap remains roughly 40% across GPT-3 model sizes and training techniques, while elicitive prompting and self-ask improve compositional question answering.
Takeaways & Limitations
Larger-scale pretraining improves factual knowledge without a corresponding improvement in composing that knowledge, whereas explicit elicitation narrows the gap.
Takeaways & Limitations
Experiments focus on English 2-hop question answering datasets, and models larger than 175 billion parameters were not evaluated.
Abstract
from arXiv · showhide
We investigate the ability of language models to perform compositional reasoning tasks where the overall solution depends on correctly composing the answers to sub-problems. We measure how often models can correctly answer all sub-problems but not generate the overall solution, a ratio we call the compositionality gap. We evaluate this ratio by asking multi-hop questions with answers that require composing multiple facts unlikely to have been observed together during pretraining. In the GPT-3 family of models, as model size increases we show that the single-hop question answering performance improves faster than the multi-hop performance does, therefore the compositionality gap does not decrease. This surprising result suggests that while more powerful models memorize and recall more factual knowledge, they show no corresponding improvement in their ability to perform this kind of compositional reasoning. We then demonstrate how elicitive prompting (such as chain of thought) narrows the compositionality gap by reasoning explicitly. We present a new method, self-ask, that further improves on chain of thought. In our method, the model explicitly asks itself (and answers) follow-up questions before answering the initial question. We finally show that self-ask's structured prompting lets us easily plug in a search engine to answer the follow-up questions, which additionally improves accuracy.
1 Introduction
The paper measures whether language models can compose separately known facts into answers to novel multi-hop questions. It finds a persistent compositionality gap, then shows that elicitive prompting, self-ask, and search-supported self-ask improve compositional question answering.
- Motivation: Compositional reasoning requires combining separately learned facts to deduce knowledge not directly observed during training.The paper contrasts this capability with rote memorization of directly observed facts.
- Measurement: The Compositional Celebrities dataset contains 8.6k automatically generated 2-hop questions combining frequently stated facts in improbable ways.Its questions are designed to separate memorization of individual facts from reasoning over their composition.
- Findings: The compositionality gap remains roughly 40% across GPT-3 model sizes and training techniques, despite improved single- and multi-hop question answering performance with scale.The gap is the fraction of compositional questions answered incorrectly despite correct answers to their sub-questions.
- Prompting: Elicitive prompts, including chain of thought, let models reason before producing an answer and markedly improve compositional question answering.The paper evaluates elicitive prompting beyond CC on 2WikiMultiHopQA, Musique, and the manually constructed Bamboogle dataset.
- Prompting: Self-ask improves further by decomposing complex questions into easier follow-up questions that the model answers before answering the main question.The method explicitly structures intermediate question generation and answering.
- Prompting: Self-ask can be combined with an internet search engine to answer follow-up questions and further improve results.This extends the structured prompting method with external retrieval for the sub-questions.
2 Systematically Measuring the Compositionality Gap
The paper measures compositionality by testing whether models can combine familiar facts into unlikely 2-hop questions. GPT-3 models improve on single- and multi-hop accuracy with scale, but their compositionality gap remains roughly constant.
- Dataset and measure: Compositional Celebrities contains grammatical 2-hop questions that combine familiar facts in unlikely ways to test compositional reasoning.The dataset includes 17 categories and is designed so individual facts are likely familiar while their combinations are unlikely to have appeared together.
- Results: GPT-3 answers 45.4% of CC 2-hop questions and 80% of their sub-questions, revealing a substantial failure to compose recalled facts.Accuracy varies widely by category, from 84.6% for Birthplace/Domain Name to 1.2% for Birth Year/Literature Nobel Prize Winner.
- Dataset and measure: The compositionality gap is the fraction of compositional questions answered incorrectly despite correct answers to their individual sub-questions.
- Results: The compositionality gap remains roughly 40% across GPT-3 model sizes and training techniques, despite monotonic improvements in single- and multi-hop accuracy.
- Results: Models compose facts more reliably when they recall the correct sub-question answers with lower perplexity.The paper suggests reporting perplexity on correct answers alongside accuracy for downstream question-answering evaluation.
3 Elicitive Prompts Narrow the Compositionality Gap and Improve Question Answering Performance
The paper evaluates elicitive prompting on four 2-hop question-answering datasets, letting models decompose questions and reason before producing answers. Chain of thought improves over direct prompting, self-ask improves further, and adding search to self-ask yields additional gains.
- 3.2 Bamboogle: The evaluation covers Compositional Celebrities, 2WikiMultiHopQA, Musique, and the manually constructed 125-question Bamboogle dataset.Bamboogle uses varied questions designed to challenge decomposition and to resist incorrect answers from a popular search engine.
- 3 Elicitive Prompts Narrow the Compositionality Gap and Improve Question Answering Performance: Direct-answer prompting requests an immediate answer, whereas elicitive prompts let the model reason before answering.Chain of thought and scratchpad are examples of elicitive prompts.
- 3 Elicitive Prompts Narrow the Compositionality Gap and Improve Question Answering Performance: Elicitive prompts allocate more computation to difficult questions and produce much higher compositional-question accuracy than direct prompting.On CC, elicitive prompts can sometimes answer more compositional questions correctly than direct prompts answer sub-questions correctly.
- 3.1 Self-ask: Self-ask explicitly formulates and answers follow-up questions before producing the final answer, using scaffolds that improve parseability.The method builds on chain of thought but separates question decomposition from answering the resulting sub-questions.
- 3.3 Improving Self-ask with a Search Engine: Self-ask’s explicit sub-question boundaries allow a search engine to answer follow-ups while the language model continues decomposition and final answering.Search results are inserted into the prompt without fine-tuning or architectural changes.
- 3.6 Results: Chain of thought improves over direct prompting, while search alone struggles with most compositional questions and LM postprocessing only matches direct prompting on 2WikiMultiHopQA.
- 3.6 Results: Self-ask improves over chain of thought by 11% absolute on Bamboogle, and integrating search improves performance on all datasets by up to 10% absolute.
- 3.6 Results: Self-ask achieves similar or better performance than least-to-most prompting while running more than 30% faster.Least-to-most requires multiple forward passes with different prompts, whereas self-ask uses one prompt in one forward pass.
4 Related Work
Related work covers methods that provide extra computation, decompose complex questions, and connect language models to search systems. The paper distinguishes self-ask through its causal-LM prompting, compositionality-gap analysis, datasets, and search integration.
- Additional computation: Earlier work showed that generating rationales or allocating additional computation can improve solutions to difficult language and vision problems.
- Question decomposition: Prior decomposition methods trained supervised models to split compositional questions into sub-questions rather than using a causal pretrained language model.
- Question decomposition: Self-ask is described as faster and more scalable because it automatically performs decomposition rather than relying on separate supervised decomposition systems.
- Question decomposition: Compositional Celebrities differs from earlier 2-hop datasets by combining facts likely seen separately but unlikely seen together during pretraining.
- Experimental caveat: The paper’s later Table 2 experiments used a different davinci-002 deployment, and the authors note that self-ask improved with that newer deployment.
- Search integration: Earlier systems integrated language models with web or information-retrieval systems through imitation learning, reinforcement learning, or special querying languages.
- Relation to concurrent work: Concurrent methods resembled self-ask but did not present the paper’s compositionality-gap findings, datasets, or web-search integration.
5 Conclusion
The conclusion highlights the paper’s compositionality gap and two ways to improve compositional question answering: self-ask and search-assisted follow-up questions.
- The paper presents the compositionality gap as a central finding about language models’ difficulty composing known facts.
- Self-ask improves over chain of thought by having the language model explicitly state and answer follow-up questions.
- A search engine can answer self-ask’s sub-questions, further improving the method.
6 Limitations
The limitations constrain the main compositionality-gap result to models up to 175 billion parameters and to primarily English 2-hop question-answering datasets.
- The reported roughly 40% compositionality gap covers models from 1 billion to 175 billion parameters, excluding larger models.The authors note that models larger than 175 billion parameters could behave differently.
- The experiments focus on 2-hop question-answering datasets in English.
- Limited manual experiments suggest self-ask also works on semantic parsing, arithmetic, and logical puzzles, but broader evaluation could yield different results.
A.1 GPT-3 Model Sizes
The GPT-3 model sizes used in the paper were taken from an external source.
- GPT-3 model sizes were taken from the EleutherAI blog.
- The model-size information is attributed to a linked external webpage.
- The cited source provides the model-size information used in the paper.
A.2 Constructing Compositional Celebrities
The Compositional Celebrities dataset was constructed by crawling celebrity and property lists, then checking a random sample of questions for correctness.
- The dataset construction crawled celebrity lists for birth-date-based compositional questions.
- Celebrity lists were also crawled for compositional questions based on countries of birth.
- The authors manually checked a random sample of questions to verify answer correctness.
- Table 3 provides an example question-and-answer pair for each of the dataset’s 17 categories.
A.3 CC prompts
The evaluation uses dedicated prompts for both 1-hop and 2-hop questions in Compositional Celebrities.
- Tables 4 and 5 provide the prompts used to evaluate 1-hop and 2-hop questions, respectively.
- The prompts are part of the evaluation setup for measuring language-model performance on Compositional Celebrities.
- The prompt examples distinguish single-hop questions from compositional 2-hop questions.
A.4 Evaluating LMs on CC
The appendix documents datasets, prompts, metrics, and diagnostic examples used to evaluate direct prompting, chain of thought, and self-ask on multi-hop question answering.
- Compositional Celebrities: Compositional Celebrities contains 17 categories, with Table 3 providing an example question-answer pair from each category.
- Bamboogle: Bamboogle consists of varied 2-hop questions, constructed from random vital Wikipedia articles and checked against incorrect search-engine featured snippets.The construction was intended to identify questions unlikely to appear on the web or in language-model training data.
- Prompting methods: Tables 8–13 list direct, chain-of-thought, and self-ask prompts for Musique and 2WikiMultiHopQA.
- Diagnostics: As confidence in correct sub-question answers increases, Davinci-002 answers the corresponding compositional questions correctly more often.
- Prompting results: Elicitive prompting narrows and sometimes closes the compositionality gap on Compositional Celebrities.The figure compares regular GPT models with the 001 and 002 instruct models; chain of thought performed within 1% of self-ask on this dataset.
- Evaluation results: Exact match, F1, and Cover-EM show the same pattern, while self-ask more often produces the requested short-form final answers than chain of thought on Bamboogle.Cover-EM counts an answer as correct when it contains the ground-truth answer.