PaperlayerView exact revision on arXiv

Source-linked AI summary

RetGen: A Joint framework for Retrieval and Grounded Text Generation Modeling

Yizhe Zhang, Siqi Sun, Xiang Gao, Yuwei Fang, Chris Brockett, Michel Galley, Jianfeng Gao, Bill Dolan

arXiv:2105.06597v4cs.CLcs.AI

TL;DR

Large language models may hallucinate or lack useful external information, while grounded generation commonly depends on scarce context–document pairs. RetGen jointly trains a retriever and grounded generator from the language-model signal, and the authors report informative, relevant text with effectiveness demonstrated in automatic and human evaluations.

  • Problem

    Pretrained language models may generate unfaithful text or lack current and personalized information, while grounded generation typically relies on scarce context–oracle-document pairs.

    Source: §1 · p. 1

  • Method

    RetGen jointly trains a document retriever and multi-document grounded generator from the language-model signal, using document utility to guide retrieval and a Mixture-of-Experts decoder to combine documents.

    Source: §1 · p. 2

  • Results

    RetGen’s retriever and generator cooperate to produce more informative, relevant text, with effectiveness demonstrated through automatic and crowd-sourced human evaluation.

    Source: §1 · p. 2, §6 · p. 7

  • Takeaways & Limitations

    The framework can use massive non-parallel corpora rather than requiring oracle context–document pairs for grounded generation.

    Source: §1 · p. 2

    Retrieved passages may be irrelevant or incorrect, and generation may miss correct information or introduce similar but incorrect information; these issues did not dominate the experiments but warrant further investigation.

    Source: §5 · p. 6

Abstract

from arXiv · show

Recent advances in large-scale pre-training such as GPT-3 allow seemingly high quality text to be generated from a given prompt. However, such generation systems often suffer from problems of hallucinated facts, and are not inherently designed to incorporate useful external information. Grounded generation models appear to offer remedies, but their training typically relies on rarely-available parallel data where information-relevant documents are provided for context. We propose a framework that alleviates this data constraint by jointly training a grounded generator and document retriever on the language model signal. The model learns to reward retrieval of the documents with the highest utility in generation, and attentively combines them using a Mixture-of-Experts (MoE) ensemble to generate follow-on text. We demonstrate that both generator and retriever can take advantage of this joint training and work synergistically to produce more informative and relevant text in both prose and dialogue generation.

1 Introduction

Large pretrained language models may produce unfaithful text and lack access to current or personalized information, while grounded generation often depends on scarce context–document pairs. RetGen jointly trains retrieval and grounded generation using the language-model signal, without requiring those oracle pairs.

1 Introduction

  • Pretrained language models may be unfaithful and lack current, personalized, or comprehensive information needed for information-demanding generation.The paper also notes that large models cannot effectively represent information about every person or entity.Source: §1 · p. 1
  • Training grounded generators on unstructured documents typically requires context–oracle-document pairs, which are seldom available.Unstructured documents offer richer information and more flexible maintenance than structured grounding data.Source: §1 · p. 1
  • Information-aware Text Generation predicts follow-on text that may use retrieved documents as contextual influence rather than as direct answers.The paper distinguishes this task from open-domain question answering and names dialogue generation and text auto-completion as applications.Source: §1 · p. 1
  • RetGen jointly trains a retriever and multi-document grounded generator on the language-model signal, avoiding oracle parallel data and combining documents through document-specific attention.The retriever is optimized for document utility in generation, while the generator combines retrieved documents in a Mixture-of-Experts manner.Source: §1 · p. 2

2 Related Work

Prior work applies retrieval to generation through editing, fixed generators, or retrieval-augmented models, while grounded generation also draws on structured and unstructured knowledge. RetGen is distinguished by jointly optimizing retrieval and generation without annotated text-reference pairs.

2 Related Work

  • Retrieve-then-edit methods edit retrieved text or feed it to a fixed generator; REALM and RAG use retrieval machinery for knowledge-intensive tasks, especially open-domain question answering.The paper contrasts these approaches with its trainable document encoder, MMI and retriever correction, and focus on information-aware text generation.Source: §2 · p. 2
  • Dense retrieval maps queries and documents into an embedding space for semantic matching, and can be fine-tuned for customized needs.The cited work applies dense retrieval to tasks including open-domain question answering and relevance ranking.Source: §2 · p. 2
  • Grounded-generation work uses structured sources such as knowledge graphs as well as raw text, which RetGen uses without requiring preconstructed graphs.Prior examples include graph-based and hybrid graph-path methods.Source: §2 · p. 2
  • Earlier unstructured-grounding approaches often use ad-hoc annotations, whereas RetGen jointly trains retriever and generator without annotated text-reference parallel data.The authors state that this allows training on target datasets without additional annotation.Source: §2 · p. 2

3 Method

RetGen optimizes the probability of upcoming text given a prompt and a non-parallel reference collection by coupling dense retrieval with grounded generation. It trains the retriever from generation likelihood and combines document-conditioned predictions at decoding time.

Method Overview

  • RetGen models the probability of upcoming text from a prompt and reference set, retrieving a small top-K subset when the full collection is too large to enumerate.The retrieved documents and prompt condition a grounded generator, and the objective updates retriever and generator parameters.Source: Method Overview · pp. 2–3, Figure 1 · p. 3

Document Retriever

  • The dense retriever encodes queries and documents in a shared embedding space, scores them by inner product, and uses approximate MIPS with indexed document vectors for sub-linear search.ANCE initializes the retriever; LSH is used to index precomputed document embeddings.Source: Document Retriever · p. 3

Knowledge-Grounded Text Generator

  • The GPT-2-like grounded generator conditions on a concatenation of one document and the query, then predicts the target with a language-model objective.Distinct token-type embeddings identify document and context; separating their position ranges is intended to reduce cues that the document is preceding context.Source: Knowledge-Grounded Text Generator · pp. 3–4

Joint Training

  • The generator is optimized directly, while an Actor-Critic-like estimate updates the retriever using document-conditioned likelihood relative to a constant baseline.The retriever needs a good pretrained initialization, and retrieved documents are refreshed asynchronously every 200 steps to reduce the cost of re-encoding the collection.Source: Joint Training · p. 4

Multi-document Decoding

  • At inference, RetGen ensembles document-specific output distributions token by token, fusing information at the output level rather than concatenating documents.A generator trained with single-document inputs can be used without additional training or fine-tuning, allowing flexibility in the number of documents; MMI reranks hypotheses to encourage stronger ties to the input document and context.Source: Multi-document Decoding · p. 4

4 Experimental Setups

The experiments cover response generation on Reddit and prose generation on arXiv, using Wikipedia documents as reference material. They compare RetGen variants and baselines with text-quality, diversity, and external-information-use metrics.

  • Reddit response generation and arXiv prose generation provide two information-demanding evaluation scenarios.Source: §4 · p. 4
  • Reddit includes 2.56M/2K/2K training/validation/test instances, and its selected examples have oracle documents for grounded baselines and retrieval evaluation.The test set uses 2,000 examples from 2018–2019, each with five references; one of six human responses is reserved for human evaluation.Source: §4 · p. 4
  • The arXiv corpus yields 9.6M/57K/2K train/validation/test abstract pairs from 1.67M abstracts, with no parallel oracle documents.The corpus is based on articles collected from 1991 to 2019.Source: §4 · pp. 4–5
  • The reference collection contains about 5.7 million Wikipedia documents, represented by each entry’s first two paragraphs.Overlong sentences are truncated to 100 words, and entries containing only one sentence are removed.Source: §4 · p. 5
  • Evaluation uses BLEU, METEOR, NIST, Entropy, Dist-n, and KMR, which measures the share of reference-only keywords covered by the prediction.KMR assesses use of external information, not relevance; Reddit relevance metrics are max-pooled across five references.Source: §4 · p. 5
  • DialoGPT and GPT-2 are fine-tuned baselines, while oracle- and random-document gDPT variants provide upper and lower bounds for the grounded generator.Source: §4 · p. 5
  • RetGen comparisons include top-1 and top-4 retrieval, frozen-retriever Fixed Φ, and an MMI reranking variant.Grounded generators share pretrained 345M DialoGPT/GPT-2 initialization, retrievers start from ANCE, and training uses K=4 for RetGen.Source: §4 · p. 5

5 Results

RetGen’s evaluation reports gains from retriever fine-tuning and multiple retrieved documents, while MMI trades a slight BLEU drop for greater diversity and higher NIST and METEOR scores. Human judgments and document-utility measures provide additional evidence, with limitations in evaluation and occasional grounding errors.

Generation Evaluation

  • Freezing the pretrained retriever yields suboptimal metrics versus RetGen (K=4), while Reddit RetGen (K=4) performs comparably to the oracle-document variant.MMI produces more diverse text and higher NIST and METEOR scores, with a slight BLEU drop.Source: §5 · p. 5, Table 1 · p. 6

KMR Evaluation

  • RetGen with MMI generally achieves the highest KMR; across both datasets, more documents and a trainable retriever yield higher KMR.Except for MMI, higher KMR indicates more effective use of external documents for LM objectives, but need not track generation quality.Source: §5 · p. 5

Generated examples

  • Generated examples show relevant retrieved documents covering orthogonal topic aspects, but irrelevant retrieval or generation errors can still introduce hallucinated or incorrect details.These failure modes do not dominate the experiments, but the authors say resolving them warrants further investigation.Source: §5 · p. 6

Impact of number of documents K

  • Evaluating RetGen with K ranging from 1 to 4 demonstrates monotonic improvement as K increases, while no significant improvement is observed with K > 4.Source: §5 · p. 7

Retrieval Evaluation

  • During retriever-only training, Reddit recall generally improves with fluctuations; increasing batch size improves recall more than increasing K from 4 to 8, which brings only marginal gain.For arXiv, expected reward can improve during training, which the authors take as evidence that retrieval benefits from LM signals.Source: §5 · p. 7, Figure 2 · p. 7

Human Evaluation

  • Human judges moderately prefer RetGen with MMI over vanilla RetGen, but human evaluation remains challenging and some metrics above the human oracle do not establish human parity.The authors attribute such metric scores presumably to randomness in human responses.Source: §5 · pp. 5, 7, Table 3 · p. 7

6 Conclusion

The conclusion presents joint retriever-generator training as a way to use the language-model signal for retrieval and combine multiple documents during decoding. It also discusses retriever correction and broader-impact risks.

6 Conclusion

  • The framework jointly optimizes a dense retriever and grounded generator, using multiple retrieved documents to produce text by selectively combining their information.The authors report evaluating it with crowd-sourced human evaluation and automatic generation and retrieval metrics, and plan to use QA and cloze objectives for future factuality evaluation.Source: §6 · p. 7
  • Crowd-sourced human evaluation may be error-prone and less accurate than some automatic metrics, according to cited machine-translation findings.The conclusion reports both crowd-sourced human and automatic evaluation for RetGen.Source: §6 · p. 7

A Broader Impact

  • The work applies no additional aggressive filtering beyond source-dataset filtering, so its data may retain toxicity, fairness, or bias issues that could appear in generations.The authors also note that outputs may, though not likely, reflect gender and other historical biases or exhibit mild unethical, biased, or offensive attitudes.Source: A Broader Impact · p. 10

B Retriever Correction

  • During autoregressive generation, a correction factor updates document relevance beliefs using the text generated so far.Documents that improve the probability of the generated text receive higher probability; the correction factor is multiplied into the retriever update at each step.Source: B Retriever Correction · p. 10

C Dataset details

The evaluation uses Reddit for response generation and arXiv for prose generation, with dataset-specific pipelines for constructing training and test instances.

C Dataset details

  • Reddit and arXiv are used to evaluate the methods on response generation and prose generation, respectively.The paper describes Reddit as the response-generation dataset and arXiv as the prose-generation dataset.Source: C Dataset details · p. 10
  • Reddit provides 2.56M instances built from 2011–2017 discussion threads containing Wikipedia links, with the linked oracle passage selected by ROUGE-F1 relevance to context.The pipeline follows one similar to the DSTC-7 grounded response generation challenge.Source: C Dataset details · p. 10
  • For arXiv, preceding abstract sentences provide context for the next sentence, and test instances are filtered for named entities shared with the target sentence using Tagme at a 0.7 threshold.The final train/validation/test sets contain 9.6M/57K/2K instances from 1.67M unique abstracts; citations, URLs, and equations are replaced with special tokens.Source: C Dataset details · p. 10

D Additional details of experimental setup

The experimental setup fixes several training and decoding choices, while a separate evaluation varies the number of retrieved documents at decoding time.

D Additional details of experimental setup

  • RetGen uses K = 4 during training, and experiments test no larger K because of GPU memory constraints.Retriever checkpoints and document indexes are refreshed every 200 iterations; generations use greedy decoding except for MMI.Source: D Additional details of experimental… · p. 10

E Impact of number of document K

  • In general, incorporating more documents yields better automatic metrics on Reddit and arXiv as decoding-time K varies.Table 4 reports automatic evaluation results for both datasets at different numbers of retrieved documents.Source: E Impact of number of document K · p. 11, Table 4 · p. 11

F Standard deviation of automatic metrics

The reported standard deviations for automatic metrics are estimated by bootstrapping, and the stated pairwise comparisons are statistically significant.

F Standard deviation of automatic metrics

  • All pairwise comparisons are significant at p<0.0001 using an unpaired two-sided t-test with Bonferroni correction.Standard deviations are estimated by bootstrapping the test set with 80% resampling without replacement across 10 sets per method.Source: F Standard deviation of automatic… · p. 11

G Issues with generation

RetGen can produce factually incorrect text through retrieval or generation failures; the paper describes these outputs as uncommon but still an important challenge.

H Attention Visualization of multi-document MoE decoder

The attention visualization shows the MoE decoder shifting from relatively even attention across retrieved documents to a more focused distribution as generation becomes specific.

  • The example uses an Iron Man 3 context sentence, and Table 7 lists the retrieved top-four documents for that context.Source: H Attention Visualization of… · p. 12, Table 7 · p. 13
  • Initially, the MoE decoder attends to all retrieved documents with relatively even probability; as generation becomes more specific, it focuses on the first two, assigning negligible attention to documents #3 and #4.During generation, attention typically becomes more peaked as the decoder reinforces attention to one or two documents based on its existing generation. This typically reduces the likelihood that irrelevant document #4 in Table 7 will have a large impact.Source: H Attention Visualization of… · p. 12
  • The example generation says Shane Black made Iron Man 3, rather than creating Iron Man 2.Source: Figure 3 · p. 12

I Additional Details of Human Evaluation

The human evaluation used screened judges, provided instructions and a template, and included payment and participant-exposure details.

Loading 2105.06597v4…