Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering

Motivation

• Original RAG keeps passage encodings and its external knowledge base fixed during QA finetuning, and its Wikipedia-trained retriever leaves adaptation to specialized domains such as research papers and news uncertain.Re-encoding is computationally expensive, while the retriever's prior Wikipedia training supports performance on Wikipedia-like datasets.Source: §1 · p. 2

Proposed approach

• RAG-end2end updates the DPR retriever, BART generator, and external knowledge-base encodings during training, jointly training retrieval and generation for QA and domain adaptation.The knowledge encodings are updated asynchronously; the auxiliary signal generates a concise, factual statement about a document using self-retrieved domain passages.Source: §1 · p. 2

Evaluation

• The approach is evaluated on datasets from COVID-19 research, Conversations, and News domains.The cited datasets are Wang et al. (2020), Wu et al. (2021b), and Trischler et al. (2016), respectively.Source: §1 · p. 2

Results

• Finetuning the retriever within RAG-end2end gives better overall results than separate DPR finetuning; question-encoder-only updates can degrade performance, while the auxiliary signal improves retrieval and accuracy.The authors identify retriever adaptation as critical to domain adaptation in RAG-like architectures.Source: §1 · p. 2

Implementation

• The authors open-source the RAG-end2end implementation through the HuggingFace Transformers library for community use, testing, and development.The implementation is provided through the library cited as Wolf et al. (2019).Source: §1 · p. 2

2 Background and Related Work

• ODQA typically has passage retrieval followed by machine comprehension; methods shifted from sparse TF-IDF and BM25 toward dense DPR embeddings, whose higher precision improves end-to-end QA accuracy.DPR (Karpukhin et al., 2020) uses separate BERT models for questions and passages and scores them by embedding dot product; answer generation may use extractive BERT or generative BART/GPT-2 (Min et al., 2021; Lewis et al., 2021).Source: §2 · p. 2

2.1 Retrieval Augmented Architecture

• RAG (Lewis et al., 2020b) differentiably combines a BART generator with a FAISS-indexed Wikipedia store, jointly fine-tuning the generator and question encoder on retrieved passages.Prior work reported strong performance on Natural Questions and more factual or grounded outputs, including reduced hallucinations in dialogue and improved conversation recall and summarization (Shuster et al., 2021; Xu et al., 2021).Source: §2.1 · p. 3
• This paper investigates extending RAG to end-to-end training and domain adaptation, which prior retrieval-augmented ODQA work had not explored.The paper identifies its focus as adapting retrieval-augmented models beyond general question answering with Wikipedia-based knowledge bases.Source: §2.1 · p. 3, §2.2 · p. 3

2.2 REALM-like end2end Retrieval Augment Architectures

• REALM (Guu et al., 2020) trains an end-to-end retriever during masked-language pretraining, then freezes it for QA fine-tuning; RAG instead starts with a pretrained DPR retriever and partially trains with BART.The authors cite REALM’s computational complexity and unavailable end-to-end training code, while noting RAG’s pretrained components avoid a heavy pretraining stage and are open-source.Source: §2.2 · p. 3
• Sachan et al. (2021) and Singh et al. (2021) extended retrieval-augmented models end-to-end, but focused on retriever training or reading and generation rather than domain adaptation.Sachan et al. report improved QA performance on datasets such as Natural Questions; Singh et al. report improved QA accuracy.Source: §2.2 · p. 3

3 Model Architecture and Training Procedure

• The system jointly trains answer generation and statement reconstruction while asynchronous processes refresh passage embeddings and the knowledge-base index.The BART reader uses a control token to distinguish answer generation from statement reconstruction.Source: §3 · p. 3, Figure 1 · p. 4

3.1 RAG Retriever and Generator

• RAG uses DPR (Karpukhin et al., 2020) BERT question and passage encoders with dot-product similarity, and a BART (Lewis et al., 2019) generator that marginalizes token loss over retrieved documents.The retrieved passages are treated as a latent variable in the sequence-to-sequence loss.Source: §3.1 · p. 4

3.2 Indexing of the External Knowledge Base

• The external passages are encoded and matched to question embeddings by dot product, while FAISS indexing reduces repeated retrieval computation.The indexing step addresses the retrieval bottleneck that can arise when the knowledge base contains millions of passages.Source: §3.1 · p. 4, §3.2 · p. 4

3.3 End-to-End Retriever Training

• RAG-end2end fine-tunes both DPR encoders and updates the knowledge-base index, unlike RAG’s fixed passage encoder and Wikipedia-based setup.Re-encoding and re-indexing can each take several hours, so the authors run those processes asynchronously alongside the main training loop. For a 250,000-passage knowledge base using four dedicated GPUs, updates took an average of 750 training steps; asynchronous updates may produce stale gradients, which the authors report do not significantly degrade performance according to prior research (Guu et al., 2020).Source: §3.3 · pp. 4–5

3.4 Statement Reconstruction

• The auxiliary signal encodes a statement, retrieves supporting passages, and trains the generator to reconstruct it without placing the statement itself in the knowledge base.A special <p> control token distinguishes reconstruction input from QA input, which prepends the question to retrieved passages.Source: §3.4 · pp. 5–6

4.1 Domain Specific Dataset Setup

• The experiments use domain-specific QA data and knowledge bases for COVID-19, News, and Conversation, with abstract or summary statements as reconstruction signals.The COVID-19 knowledge base contains 250,000 passages from 5,000 articles, News uses 85,000 passages from 10,000 articles, and Conversation uses 110,000 passages from 10,000 conversations. Models are evaluated with EM, F1, and Top-K retrieval accuracy.Source: §4.1 · pp. 6–7, §4.2 · p. 7

4.4 Effect of Adding the Statement-Reconstruction Auxiliary Task

• 13-point EM score, 27-point top 5, and 16-point top 20 improvements for Conversation QA distinguish RAG-end2end-QA + R from RAG-Original-QA.The paper presents these as the overall improvements of its contributions.Source: Table 1 · p. 8, §4.4 · p. 8

4.3 Effect of End-to-End Retriever Training on Domain Adaptation

• RAG-end2end-QA significantly outperforms RAG-original-QA on EM, F1, Top-5, and Top-20 across all three domains.EM improvements range from 1.13 points in News to 12.16 points in Conversation; Top-5 and Top-20 retrieval gains are around 25 points in Conversation and 4.7 to 6.6 points in the other domains.Source: §4.3 · pp. 7–8, Table 1 · p. 8

4.4 Effect of Adding the Statement-Reconstruction Auxiliary Task

• RAG-end2end-QA + R outperforms RAG-end2end-QA in all three domains, with EM gains of 1.7 to 8.39 points and Top-20 retrieval gains of 3.2 to 8 points.The comparison isolates the effect of adding statement reconstruction to RAG-end2end trained with QA pairs.Source: Table 1 · p. 8, §4.4 · p. 8
• Adding reconstruction to RAG-original-QA also improves EM, by 0.84 points in COVID-19 and 3.12 points in Conversation.The paper describes these gains as moderate and observes them even without the end-to-end extension.Source: §4.4 · p. 8
• Generated reconstructions are close enough to their input statements, though examples are not always factually 100% correct.Table 2 shows the input statements, retrieved top-5 documents, and reconstructed statements.Source: §4.4 · p. 8, Table 2 · p. 9

Standalone DPR fine tuning with domain specific data

• Domain-adapted DPR improves retrieval over DPR-original for COVID-19 and Conversation, while News results are almost unchanged; RAG-end2end's loss performs better than DPR fine-tuning with domain data.The paper notes that long correct passages in News and Conversation may explain why supervised DPR fine-tuning did not initially improve results; it generated synthetic QA pairs for those domains. End-to-end retriever fine-tuning can be computationally costly because the external knowledge base must be re-encoded and re-indexed.Source: §4.5 · p. 8, Table 3 · p. 10, Standalone DPR fine tuning with domain… · p. 10
• RAG-original-QA + R initialized with a domain-adapted DPR outperforms its usual-checkpoint counterpart, but RAG-end2end-QA + R outperforms that adapted-DPR model.The comparisons support domain-adapting the retriever as useful while showing that the RAG-end2end mechanism does not require separately training DPR.Source: Standalone DPR fine tuning with domain… · pp. 10–11, Table 4 · p. 11

5.1 Role of retriever in domain adaptation

• RAG-end2end improved on original RAG fine-tuning across all datasets, with gains attributed to updating knowledge-base embeddings and indexing and outperforming standalone DPR fine-tuning.The authors suggest that neural retrievers such as DPR, trained on public datasets, may struggle on domain-specific datasets.Source: §5.1 · p. 11

5.2 Cost of end2end retriever adaptation

• End-to-end adaptation can be expensive for knowledge bases with millions of passages, requiring GPU re-encoding and CPU resources for FAISS re-indexing.With sufficient compute, it enables training the retriever and reader directly from knowledge-base passages and question-answer pairs, without generating synthetic passage-related QA pairs.Source: §5.2 · p. 11
• Although RETRO's authors claim frozen BERT embeddings suffice, the authors argue domain-specific models benefit from a domain-adapted retriever and call for evaluating RETRO in such settings.The passage attributes the frozen-embedding claim to Borgeaud et al. (2021).Source: §5.2 · p. 11

5.3 Comparing the RAG-original with RAG-end2end on an In-domain dataset

• RAG-end2end outperformed RAG-original on answer generation and correct-document retrieval, including on Wikipedia-based SQuAD adapted to ODQA.The SQuAD knowledge base was built from question-answer contexts, split into 100-word passages, and contained 30K passages.Source: Table 5 · p. 11, §5.3 · pp. 11–12

6 Conclusion and Future Work

• RAG-end2end significantly outperformed original RAG across COVID-19, News, and Conversations, with results described as stable and generalizable across domains.The experiments also highlight the importance of the retriever in domain-specific question answering.Source: §6 · p. 12
• Joint training improved DPR performance over independently fine-tuning DPR and enabled training with QA pairs without gold-standard question passages.Source: §6 · p. 12
• The reconstruction auxiliary signal further improved both retriever and final answer generation accuracy.Source: §6 · p. 12
• Future work could test RAG-end2end on fact checking (Lewis et al., 2020b), summarisation (Shuster et al., 2021), and conversational response generation (Xu et al., 2021).Source: §6 · p. 12
• The authors suggest exploring generative capabilities with qualitative metrics for factual consistency (Kry´sci´nski et al., 2019; Cao et al., 2022) and hallucinations (Cao et al., 2022; Shuster et al., 2021), and whether updating the retriever and document embeddings during training could improve factual consistency.Source: §6 · p. 12
• They also propose further improving the retriever and exploring other auxiliary signals beyond statement reconstruction.Source: §6 · p. 12

• Figure 2 shows predicted answers and retrieved passages for conversational-domain questions (Wu et al., 2021b).Source: Figure 2 · p. 15