Source-linked AI summary
Active Retrieval Augmented Generation
Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, Graham Neubig
TL;DR
Language models hallucinate, and single-time retrieval does not fit long-form generation whose information needs evolve during output. The paper proposes active retrieval, specifically FLARE, which uses predicted future sentences and low-confidence tokens to guide retrieval and regeneration. Across four long-form tasks, FLARE performs superiorly or competitively, although gains are limited on some datasets and naive interleaving increases cost.
Problem
Language models can hallucinate, while single-time retrieval is limited for long-form generation requiring information to be gathered throughout generation.
Method
FLARE iteratively predicts the upcoming sentence, retrieves relevant documents when it contains low-confidence tokens, and regenerates the sentence.
Results
Across 4 tasks/datasets, FLARE achieves superior or competitive performance compared to single-time and multi-time retrieval baselines.
Takeaways & Limitations
Active retrieval can support long-form generation by deciding when and what information to retrieve during generation.
Takeaways & Limitations
FLARE did not provide significant gains on Wizard of Wikipedia or ELI5, and naive interleaving of generation and retrieval increases overhead and generation cost.
Abstract
from arXiv · showhide
Despite the remarkable ability of large language models (LMs) to comprehend and generate language, they have a tendency to hallucinate and create factually inaccurate output. Augmenting LMs by retrieving information from external knowledge resources is one promising solution. Most existing retrieval augmented LMs employ a retrieve-and-generate setup that only retrieves information once based on the input. This is limiting, however, in more general scenarios involving generation of long texts, where continually gathering information throughout generation is essential. In this work, we provide a generalized view of active retrieval augmented generation, methods that actively decide when and what to retrieve across the course of the generation. We propose Forward-Looking Active REtrieval augmented generation (FLARE), a generic method which iteratively uses a prediction of the upcoming sentence to anticipate future content, which is then utilized as a query to retrieve relevant documents to regenerate the sentence if it contains low-confidence tokens. We test FLARE along with baselines comprehensively over 4 long-form knowledge-intensive generation tasks/datasets. FLARE achieves superior or competitive performance on all tasks, demonstrating the effectiveness of our method. Code and datasets are available at https://github.com/jzbjyb/FLARE.
1 Introduction
Language models can hallucinate, while single-time retrieval is poorly suited to long-form generation requiring information gathered throughout the process. The paper proposes active retrieval and evaluates FLARE across four long-output tasks.
- Language models may hallucinate despite memorizing some world knowledge, motivating retrieval from external knowledge resources.
- Single-time retrieval uses the input to retrieve documents once before generating a complete answer, mainly benefiting short-form knowledge-intensive tasks.
- Long-form generation has evolving information needs, requiring multiple knowledge pieces to be gathered during generation rather than inferred from the input alone.
- Active retrieval augmented generation decides when and what to retrieve throughout generation, retrieving when the model lacks required knowledge.
- FLARE anticipates future content with a temporary next sentence, retrieves documents when it contains low-probability tokens, and regenerates that sentence.
- Across 4 tasks/datasets, FLARE achieves superior or competitive performance against single-time and multi-time retrieval baselines.
2 Retrieval Augmented Generation
Retrieval-augmented language models combine a retriever with an LM that conditions generation on retrieved documents. The paper generalizes this setup by interleaving retrieval and generation throughout the answer.
- Retrieval-augmented LMs retrieve documents for a query and condition answer generation on the retrieved documents.
- The common single-time setup uses the user input as the retrieval query and generates the complete answer at once.
- Active retrieval augmented generation decides when and what to retrieve during generation, interleaving retrieval with generation.
- At each step, the query can use the user input and previously generated output, beginning with the user input when no previous generation exists.
- The model conditions the next generation on current-step retrieved documents, the user input, and previous output while discarding earlier retrieved documents to avoid input-length limits.
3 FLARE: Forward-Looking Active REtrieval Augmented Generation
FLARE actively retrieves information during generation by anticipating future content and retrieving only when the predicted sentence contains low-confidence tokens. It implements this idea through retrieval instructions and direct, confidence-based query formulation.
- 3 FLARE: Forward-Looking Active REtrieval Augmented Generation: FLARE actively decides when and what to retrieve throughout generation, targeting future content rather than relying only on the initial input.Its two methods are FLAREinstruct, which prompts search-query generation, and FLAREdirect, which uses the predicted next sentence as a query.
- 3.1 FLARE with Retrieval Instructions: FLAREinstruct combines retrieval instructions with downstream task instructions so the LM generates search queries while performing the task.When it emits a search query, generation pauses, retrieved documents are prepended to the user input, and generation resumes.
- 3.2 Direct FLARE: Black-box LM constraints make FLAREinstruct queries potentially unreliable, motivating FLAREdirect’s use of the predicted next sentence for retrieval.FLAREdirect avoids depending on retrieval instructions to elicit reliable queries.
- 3.2 Direct FLARE: FLAREdirect first generates a temporary next sentence, retrieves documents when it lacks confidence, and regenerates that sentence using the retrieved documents.If every token in the temporary sentence meets the confidence threshold, the sentence is accepted without retrieval.
- 3.2.2 Confidence-based Query Formulation: Using the next sentence as a query performs better than using previous context but can perpetuate errors contained in the predicted sentence.An incorrect sentence about Joe Biden’s university could retrieve misleading information.
- 3.2.2 Confidence-based Query Formulation: Implicit queries mask low-confidence tokens, while explicit queries generate questions targeting uncertain spans to improve retrieval accuracy.The masking threshold β controls aggressiveness; question generation can ask, for example, which university Joe Biden attended.
4 Multi-time Retrieval Baselines
The paper organizes multi-time retrieval baselines by when and what they retrieve. These baselines use previous windows, previous sentences, or task-specific question decomposition, each with distinct limitations.
- Previous-window approaches: Previous-window approaches retrieve every l tokens using the generated tokens from the previous window as the query.The implementation follows a window size of l = 16.
- Previous-sentence approaches: Previous-sentence approaches retrieve once per sentence and use the previous sentence as the query.IRCoT is identified as belonging to this category.
- Question decomposition approaches: Question decomposition approaches use manually annotated, task-specific exemplars to guide sub-question generation and trigger retrieval dynamically.Self-ask retrieves when the model generates a sub-question.
- Limitations of multi-time retrieval baselines: Existing baselines may misrepresent future intent, retrieve at inefficient fixed intervals, or require task-specific prompt engineering that limits generalizability.These drawbacks motivate active retrieval that adapts both retrieval timing and query content.
5 Experimental Setup
FLARE is evaluated with few-shot in-context learning across four diverse knowledge-intensive tasks, using task-specific datasets and metrics. The experiments include multihop QA, commonsense reasoning, long-form QA, and open-domain summarization, with overall comparisons reported across tasks and datasets.
- Evaluation design: Evaluation uses few-shot in-context learning across four diverse knowledge-intensive tasks.The study subsamples at most 500 examples from each dataset because of experiment costs.
- Task and dataset setup: 2WikiMultihopQA tests complex questions requiring composition, comparison, or inference, with chain-of-thought and final-answer generation evaluated using exact match and token-level metrics.The reported metrics include exact match, token-level F1, precision, and recall.
- Task and dataset setup: StrategyQA consists of crowdsourced yes/no questions requiring world and commonsense knowledge, with chain-of-thought and final answers evaluated by exact match.The final answer is extracted and matched against the gold answer.
- Task and dataset setup: ASQA evaluates comprehensive answers to ambiguous questions, including a hint-based setting, using EM, Disambig-F1, ROUGE, and an overall DR score.The hint setting guides language models toward the intended aspect of an ambiguous question.
- Task and dataset setup: WikiAsp evaluates aspect-based summaries about entities across 20 Wikipedia domains using ROUGE, named entity-based F1, and UniEval for factual consistency.The summaries are generated about specified aspects such as academics and history.
- Overall comparison: Figure 4 compares FLARE and baselines using EM for 2WikiMultihopQA, StrategyQA, and ASQA, and UniEval for WikiAsp.The overall results are reported across all four tasks and datasets.
6 Experimental Results
Across four knowledge-intensive generation tasks, FLARE consistently outperforms retrieval baselines. Ablations show that forward-looking queries and confidence-triggered retrieval are central, while excessive or poorly formed retrieval can hurt performance.
- Overall results: FLARE outperforms all baselines on every evaluated task and dataset.The experiments cover multihop QA, commonsense reasoning, long-form QA, and open-domain summarization.
- Comparison with baselines: FLARE outperforms all baselines by a large margin on 2WikiMultihopQA.Question decomposition is the strongest baseline, but FLARE surpasses it without manually annotated decomposed-question exemplars.
- Comparison with baselines: FLARE outperforms baselines across all reported metrics on StrategyQA, ASQA, ASQA-hint, and WikiAsp.Previous-window retrieval underperforms single-time retrieval on ASQA, consistent with its weak reflection of future intent.
- Importance of forward-looking retrieval: Using the next sentence for retrieval is clearly better than using the previous sentence on 2WikiMultihopQA and ASQA-hint.The comparison directly tests forward-looking versus past-context retrieval under matched sentence-level retrieval settings.
- Importance of active retrieval: Retrieval percentages above 60% plateau performance on 2WikiMultihopQA and above 50% reduce performance on StrategyQA.Retrieval triggered for 40%-80% of sentences usually provides good performance across tasks and datasets.
- Query formulation: Masking low-probability tokens improves retrieval over using the complete predicted sentence, while implicit and explicit query formulation perform similarly.The results support removing potentially erroneous low-confidence tokens before retrieval, without establishing a clear advantage between the two query-formulation methods.
7 Related Work
Related work spans iterative retrieval and adaptive retrieval, while FLARE focuses on long-form generation and can potentially combine text retrievers with browser interaction.
- Iterative and adaptive retrieval: Iterative retrieval and refinement have been studied for text and code generation, but FLARE differs in generation granularity and retrieval strategy.The cited methods include Peng et al. (2023), Zhang et al. (2023), Zemlyanskiy et al. (2022), and Yu et al. (2023).
- Iterative and adaptive retrieval: Adaptive retrieval in single-time scenarios uses question popularity or generation probabilities, whereas FLARE targets long-form generation.
- Browser-enhanced LMs: WebGPT and WebCPM train language models to use browsers for factuality enhancement, allowing multiple queries before generation.These systems use reinforcement learning or supervised training.
- Browser-enhanced LMs: FLARE uses text-based retrievers but can potentially be combined with a browser to improve retrieval quality.
8 Conclusion
The paper concludes that active retrieval augmentation should decide when and what to retrieve during generation. FLARE operationalizes this through forward-looking sentence retrieval and shows effectiveness across four tasks or datasets.
- FLARE iteratively uses the upcoming sentence to retrieve relevant information when it contains low-confidence tokens, then regenerates that sentence.
- Experimental results on 4 tasks/datasets demonstrate the effectiveness of the proposed methods.
- Future directions include better active-retrieval strategies and efficient architectures for integrating retrieved information.
9 Limitations
FLARE did not provide significant gains on Wizard of Wikipedia or ELI5, and naive interleaving of retrieval with generation increases computational overhead and cost.
- FLARE did not provide significant gains on Wizard of Wikipedia or ELI5.
- Wizard of Wikipedia outputs average approximately 20 tokens, so retrieving multiple disparate information pieces might not be necessary.
- On ELI5, both single-time retrieval and FLARE did not significantly outperform not using retrieval because grounding and evaluation are difficult.
- Naively interleaving retrieval and generation increases overhead and generation cost because the language model is activated repeatedly.Without caching, previous activations must also be recomputed after each retrieval.
A FLARE Implementation Details
FLARE implementation bootstraps retrieval from the user input, predicts temporary future sentences, and selectively retrieves during iterative generation. Retrieval is triggered for only a portion of sentences, though this still adds overhead versus single-time retrieval.
- FLARE starts by using the user input as the initial query to retrieve documents for generating the first sentence.
- Subsequent steps generate a temporary forward-looking sentence without retrieved documents before deciding on retrieval.
- Each step generates 64 tokens, extracts the first sentence with NLTK, and discards the remaining tokens.
- Efficiency: Retrieval is triggered for 30% ∼60% of sentences depending on the downstream task.
- Efficiency: Compared with single-time retrieval, interleaving retrieval and generation with a naive implementation increases overheads.
B Datasets and Settings
The experiments cover long-form QA, open-domain summarization, multihop QA, and commonsense reasoning, using task-specific exemplars, retrievers, corpora, and document counts. The settings also adapt WikiAsp to open-domain summarization by replacing references with information gathered from the open web.
- Multihop QA: Multihop QA uses eight exemplars, BM25 retrieval, and Wikipedia articles as the retrieval corpus.The task example requires connecting the founder of Versus with the circumstances of Gianni Versace’s death.
- Commonsense Reasoning: Commonsense reasoning uses six exemplars, BM25 over Wikipedia, and three retrieved documents.The example reasons from a pear’s density to whether it floats in water.
- Tasks: The evaluation includes long-form QA in original and hinted ASQA settings, open-domain summarization, multihop QA, and commonsense reasoning.The passages describe task-specific outputs and settings for each evaluation scenario.
- Long-form QA: ASQA experiments use manually annotated exemplars, BM25 retrieval over Wikipedia, and three retrieved documents.The same setup is used for both the original ASQA and ASQA-hint settings.
- Open-domain Summarization: WikiAsp is converted from multi-document summarization to an open-domain setting by removing references and gathering information from the open web.The setup uses manually annotated exemplars and Bing search while excluding specified Wikipedia-related domains to avoid leakage.
C Hyperparameters
The appendix lists FLARE’s hyperparameters separately for each dataset, but the supplied passage does not report their values or settings.
- Hyperparameters: FLARE hyperparameters for different datasets are listed in Table 9.The supplied passage identifies the table but does not provide the individual hyperparameter values.
D Prompts and Few-shot exemplars
The appendix organizes task prompts, retrieval instructions, document formatting, and few-shot exemplars for the evaluated datasets. Examples illustrate search calls, step-by-step reasoning, and combined reasoning-plus-search behavior.
- Retrieval instructions: The retrieval instructions teach models to use the Search API by writing [Search(term)] with a chosen search term.Examples include searching for nanomaterial production risks, Ghana’s flag meanings, and metformin’s first-line use.
- Step-by-step reasoning: The reasoning prompts instruct models to write reasoning steps before drawing a conclusion.The exemplars compare entities, dates, nationalities, and relationships before producing final answers.
- Combined reasoning and search: Combined prompts require step-by-step reasoning while using the Search API whenever possible.The appendix includes a combined reasoning-and-search example and separate self-ask instructions for multihop questions.