Source-linked AI summary
A Survey on Retrieval-Augmented Text Generation
Huayang Li, Yixuan Su, Deng Cai, Yan Wang, Lemao Liu
TL;DR
Retrieval-augmented text generation addresses the challenge of generating useful text by combining neural generation with externally retrieved information. This survey formalizes the paradigm, reviews methods across dialogue, translation, and other tasks, and identifies future directions including multimodal retrieval and improved retrieval control. It also highlights unresolved gaps in training-versus-inference optimization and dialogue retrieval scope.
Problem
Conventional generation does not explicitly access external knowledge or retrieved references, while dialogue generation additionally faces one-to-many responses and missing contextual knowledge.
Method
The paper surveys retrieval-augmented generation through its retrieval sources, retrieval metrics, integration paradigms, and applications across dialogue response generation, machine translation, and other tasks.
Results
The survey organizes representative retrieval-augmented approaches across dialogue, translation, and other generation tasks, including methods using retrieved documents and multimodal information.
Takeaways & Limitations
Future work should explore multimodal retrieval, diverse and controllable retrieval metrics, and broader retrieval pools for generation tasks.
Takeaways & Limitations
Jointly learning retrieval metrics and generation models faces a gap between locally back-propagated training retrieval and globally conducted inference retrieval.
Abstract
from arXiv · showhide
Recently, retrieval-augmented text generation attracted increasing attention of the computational linguistics community. Compared with conventional generation models, retrieval-augmented text generation has remarkable advantages and particularly has achieved state-of-the-art performance in many NLP tasks. This paper aims to conduct a survey about retrieval-augmented text generation. It firstly highlights the generic paradigm of retrieval-augmented generation, and then it reviews notable approaches according to different tasks including dialogue response generation, machine translation, and other generation tasks. Finally, it points out some important directions on top of recent methods to facilitate future research.
1 Introduction
Retrieval-augmented text generation combines deep learning with traditional retrieval, acquiring knowledge explicitly and using retrieved references to support generation. The survey presents a generic paradigm and reviews representative approaches across dialogue, translation, and other generation tasks.
- Retrieval-augmented generation explicitly acquires knowledge through a plug-and-play retrieval process rather than storing it only in model parameters.This supports scalability by making knowledge externally accessible.
- The survey reviews representative approaches for dialogue response generation, machine translation, and other generation tasks.
- It first presents the generic retrieval-augmented generation paradigm and its three components: retrieval sources, retrieval metrics, and generation models.
- For dialogue response generation, exemplar or template retrieval has been shown beneficial for producing informative responses, alongside growing work on knowledge-grounded generation.
2 Retrieval-Augmented Paradigm
The retrieval-augmented paradigm maps an input and retrieved relevant instances to an output, organizing system design around retrieval sources, metrics, and integration methods. The survey distinguishes supervised and unsupervised memories, lexical and dense retrieval, task-specific learning, and several integration strategies.
- Retrieval-augmented generation extends ordinary input-output mapping by conditioning output y on input x and retrieved instances z.The retrieved set can contain relevant input-output pairs from training or external datasets.
- Retrieval sources: Retrieval memories come from training corpora, external datasets, or large-scale unsupervised corpora.External data can support domain adaptation and knowledge updates, while unsupervised target-language data can serve as memory in machine translation.
- Retrieval metrics: Sparse-vector methods such as TF-IDF and BM25 efficiently match keywords, whereas dense-vector retrieval targets semantic relevance beyond surface overlap.Dense retrieval commonly encodes text into low-dimensional vectors and scores them with inner products.
- Retrieval metrics: Task-specific retrieval learns metrics with the downstream generator so that retrieved memories are selected for their contribution to generation quality.
- Integration methods: Retrieved memory can be integrated through data augmentation, attention mechanisms, or explicit skeleton extraction.These methods respectively concatenate retrieved spans, encode retrieved targets for attention, or extract useful partial content before generation.
3 Dialogue Response Generation
Chit-chat response generation must handle one-to-many responses and knowledge absent from dialogue history. Retrieval augmentation combines informative retrieved material with generative flexibility, but existing dialogue methods face limitations in retrieval diversity, metric customization, and retrieval-pool scope.
- Background: Chit-chat systems face a one-to-many response problem because dialogue history alone cannot determine a meaningful, specific response.External knowledge may also be needed to avoid safe but boring responses.
- Background: Retrieval-based models provide informative and grammatical responses but generalize poorly to histories unlike the retrieval pool, while generation-based models generalize better but tend toward dull responses.
- Integration: Retrieval-augmented dialogue methods combine retrieval and generation through reranking, extra encoders, concatenated inputs, or other deep integration designs.
- Integration: Skeleton-based methods first extract useful structure from retrieved responses and then generate responses from that skeleton, helping control the inflow of erroneous information.
- Limitations: Retrieved responses or skeletons can fit a query yet remain inconsistent with the current target response because dialogue is one-to-many.
- Limitations: Existing dialogue methods are limited by single-response retrieval, universal relevance scores, and retrieval pools restricted mainly to dialogue corpora or documents.The survey suggests combining multiple responses, customized metrics, and broader-domain or multimodal corpora.
4 Machine Translation
The survey traces retrieval-augmented translation from human translation memory through SMT and NMT, organizing methods by where retrieval enters the translation pipeline. It also identifies limitations involving retrieval quality, integration, efficiency, and decisions about when to use retrieved information.
- Retrieval-augmented translation originates in human translation, where translation memory retrieves similar bilingual sentences to improve translation quality and efficiency.
- Statistical Machine Translation: SMT methods integrate translation memory through constrained decoding, phrase-table aggregation, or parameter tuning.Constrained decoding reuses useful target segments, phrase-table aggregation extracts translation rules, and parameter tuning adapts model parameters for retrieved sentences.
- Statistical Machine Translation: Constrained decoding is plug-and-play but degrades substantially when the input is not highly similar to the retrieved source sentence.
- Neural Machine Translation: NMT approaches either train the generation model to cooperate with retrieval or expose retrieved data only during inference.
- Neural Machine Translation: Inference-time NMT methods reward target words using sentence-level, alignment-based, token-level, or learned retrieval signals, but dense retrieval can impose high latency.
- Neural Machine Translation: NMT limitations include relying mainly on input–retrieval similarity for rewards and uncertainty about when retrieved information should be used.Excessive integration, such as using translation memory at every time step, can reduce translation efficiency and dampen output fluency.
5 Other Tasks
Retrieval-augmented generation has been applied beyond dialogue and translation to language modelling, summarization, paraphrase generation, text style transfer, and data-to-text generation. The survey also highlights multimodal extensions involving image-text retrieval and speech-to-text transcription.
- Language Modelling: Language modelling can use indexed hidden states and k-NN retrieval, while RETRO achieves performance comparable to GPT-3 with 25× fewer parameters.
- Summarization: Summarization methods retrieve exemplar documents or templates and use adaptive generation or reranking to produce summaries.
- Paraphrase Generation: Paraphrase generation retrieves a similar sentence and uses a neural editor, with related work controlling the syntax of generated text.
- Text Style Transfer: Text style transfer retrieves lexically similar texts, removes source-irrelevant tokens, and derives output from the edited template.
- Data-to-Text Generation: Data-to-text generation retrieves candidate texts from an unlabelled corpus and applies a neural selector to measure source–candidate similarity.
- Multimodal Extensions: Retrieval-augmented generation is suggested for multimodal tasks because image-text retrieval can associate images with relevant text, including for image captioning and speech transcription.
6 Future Directions
The survey identifies future directions centered on retrieval sensitivity, efficiency, optimization gaps, multimodality, and more diverse and controllable retrieval. These directions respond to weaknesses in retrieval quality, computational overhead, training–inference mismatch, and universal retrieval metrics.
- Retrieval Sensitivity: Retrieval-augmented models are sensitive to retrieval quality: they perform well with highly similar examples but can underperform non-retrieval models with dissimilar examples.
- Retrieval Efficiency: Larger retrieval memories may improve similarity of retrieved examples but increase inference overhead, motivating trade-offs such as memory compression.
- Local vs. Global Optimization: Training optimizes retrieval metrics locally over a few examples, whereas inference evaluates them globally across the memory, creating an essential gap.
- Multi-Modalities: Future work could extend retrieval-based generation to multimodal tasks such as image captioning and speech-to-text transcription.
- Diverse & Controllable Retrieval: Customized retrieval metrics could support controlled generation by favoring emotional or stylistic dialogue examples and terminology-specific translation data.
- Diverse & Controllable Retrieval: Using multiple retrieval metrics may increase retrieval diversity and improve coverage of useful information for higher-quality generation.
7 Conclusion
The paper surveys recent retrieval-augmented text-generation approaches, covering their components, applications, and future research directions.
- The survey reviews retrieval metrics, retrieval sources, and integration paradigms as core components of retrieval-augmented text generation.
- It discusses retrieval-augmented generation across dialogue response generation, machine translation, and other generation tasks.
- The paper identifies future directions for retrieval-augmented text generation.