PaperlayerView exact revision on arXiv

Source-linked AI summary

InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales

Zhepei Wei, Wei-Lin Chen, Yu Meng

arXiv:2406.13629v3cs.CLcs.LG

TL;DR

Noisy retrieval makes RAG’s implicit denoising difficult to verify, while explicit denoising supervision can be costly. InstructRAG synthesizes rationales for in-context learning or fine-tuning, and outperforms existing RAG baselines across five benchmarks.

  • Problem

    Retrieved documents may contain misleading or erroneous information, and standard RAG implicitly denoises through direct answer prediction, making the process difficult to interpret and verify.

    Source: §1 · pp. 1–2

  • Method

    InstructRAG has an instruction-tuned model synthesize denoising rationales from questions, answers, and retrieved documents, then uses them for in-context learning or supervised fine-tuning.

    Source: §1 · p. 2, §2 · p. 2, §2.3 · p. 4

  • Results

    8.3% relative improvement over the best baseline on average across five knowledge-intensive benchmarks, with consistent outperformance in training-free and trainable settings.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Self-synthesized rationales provide explicit denoising supervision without additional supervision and are reported to support out-of-domain generalization.

    Source: §1 · p. 2

    Rationale consistency is 89% across all training samples, versus 98% when at least one relevant document is retrieved; samples with no relevant document compromise rationale quality.

    Source: A LIMITATIONS AND FUTURE WORKS · p. 20

Abstract

from arXiv · show

Retrieval-augmented generation (RAG) has shown promising potential to enhance the accuracy and factuality of language models (LMs). However, imperfect retrievers or noisy corpora can introduce misleading or even erroneous information to the retrieved contents, posing a significant challenge to the generation quality. Existing RAG methods typically address this challenge by directly predicting final answers despite potentially noisy inputs, resulting in an implicit denoising process that is difficult to interpret and verify. On the other hand, the acquisition of explicit denoising supervision is often costly, involving significant human efforts. In this work, we propose InstructRAG, where LMs explicitly learn the denoising process through self-synthesized rationales -- First, we instruct the LM to explain how the ground-truth answer is derived from retrieved documents. Then, these rationales can be used either as demonstrations for in-context learning of explicit denoising or as supervised fine-tuning data to train the model. Compared to standard RAG approaches, InstructRAG requires no additional supervision, allows for easier verification of the predicted answers, and effectively improves generation accuracy. Experiments show InstructRAG consistently outperforms existing RAG methods in both training-free and trainable scenarios, achieving a relative improvement of 8.3% over the best baseline method on average across five knowledge-intensive benchmarks. Extensive analysis indicates that InstructRAG scales well with increased numbers of retrieved documents and consistently exhibits robust denoising ability even in out-of-domain datasets, demonstrating strong generalizability.

1 INTRODUCTION

RAG can improve language-model accuracy with external knowledge, but noisy retrieved documents make implicit denoising difficult to verify and explicit supervision costly. InstructRAG uses self-synthesized rationales to teach denoising without additional supervision.

1 INTRODUCTION

  • Language models can produce factually incorrect content, particularly for current or out-of-domain knowledge, limiting reliable deployment where factuality is crucial.RAG uses external sources to improve accuracy on knowledge-intensive tasks, but retrieved content may still be noisy.Source: §1 · p. 1
  • Retrieved documents can contain irrelevant or erroneous information, while vanilla RAG implicitly denoises by predicting answers directly, a process that is difficult to interpret and verify.Explicit denoising supervision often requires substantial human effort.Source: §1 · pp. 1–2
  • InstructRAG instructs a language model to synthesize rationales explaining how noisy retrieved documents support ground-truth answers, then uses them for in-context learning or supervised fine-tuning.The framework requires no additional supervision compared with standard RAG methods.Source: §1 · p. 2
  • InstructRAG improves over the best baseline by 8.3% on average across five knowledge-intensive benchmarks and supports training-free and trainable use.The paper also reports robust denoising as retrieved-document noise increases and transfer to out-of-domain datasets.Source: §1 · p. 2

2 OUR METHOD: INSTRUCTRAG

InstructRAG synthesizes rationales from question-answer pairs and potentially noisy retrieved documents, then uses those rationales to train explicit denoising through in-context learning or fine-tuning.

2 OUR METHOD: INSTRUCTRAG

  • The framework first synthesizes denoising rationales from training questions, answers, and retrieved documents, then uses them as in-context demonstrations or fine-tuning data; the same model is used by default.The generator and learner may also be instantiated with different models.Source: §2 · pp. 2–3, Figure 2 · p. 3

2.1 PROBLEM SETTING

  • The problem setting uses annotated downstream-task data and an off-the-shelf retriever over an external knowledge base, without extra GPT-3 or GPT-4 supervision or document filtering and reranking.The model predicts answers from the retrieved documents and its parametric knowledge.Source: §2.1 · p. 3

2.2 RATIONALE GENERATION VIA INSTRUCTION-FOLLOWING

  • An instruction-following language model generates rationales that distinguish useful from noisy documents and connect context to the ground-truth answer; consistency averages 98% when at least one relevant document is retrieved across five benchmarks.Consistency is checked by substring matching against the answer.Source: §2.2 · p. 4
  • Template rationales based on substring matching can select semantically inaccurate documents and significantly degrade performance, while the language-model generator works without the answer with only a minor performance drop.Source: §2.2 · p. 4

2.3 LEARNING DENOISING RATIONALES IN RAG

  • INSTRUCTRAG-ICL uses sampled question-rationale demonstrations, while INSTRUCTRAG-FT fine-tunes a rationale learner on retrieved documents and questions using a standard language-modeling objective.For ICL, only example questions and rationales are shown to save memory and improve inference efficiency.Source: §2.3 · pp. 4–5

3 EXPERIMENTS

Across five knowledge-intensive benchmarks, INSTRUCTRAG-ICL consistently beats training-free baselines, while INSTRUCTRAG-FT surpasses non-retrieval and training-free baselines and nearly all trainable baselines. The experiments also examine rationale-generation choices, inference demonstrations, noise robustness, generalization, and evaluation with an LLM judge.

3.1 EXPERIMENTAL SETTING

  • Experiments cover five knowledge-intensive benchmarks, Wikipedia retrieval with sparse and dense retrievers, and training-free and trainable RAG baselines.The baselines include retrieval-free prompting, retrieval-augmented prompting, supervised fine-tuning, RetRobust, and Self-RAG.Source: §3.1 · p. 5

3.2 MAIN RESULT

  • INSTRUCTRAG-ICL consistently outperforms training-free baselines across metrics, and its performance improves with larger backbone models.The comparison is reported across the benchmarks and metrics in Table 3.Source: §3.2 · pp. 6–7, Table 3 · p. 6
  • INSTRUCTRAG-FT surpasses non-retrieval and training-free baselines across five benchmarks and most trainable-baseline metrics, but slightly trails Self-RAG on ASQA citation precision and recall.The reported exception is for citation metrics; the paper focuses on generation correctness and reports competitive citation performance.Source: Table 3 · p. 6, §3.2 · p. 7

3.3 ABLATION STUDY

  • Ground-truth answers and retrieved documents improve rationale generation; larger LM generators outperform template-generated rationales and smaller generators.The template approach relies on lexical matching and can miss semantic meaning, introducing noise into rationales.Source: §3.3 · pp. 7–8
  • Inference demonstrations help INSTRUCTRAG-ICL but hurt INSTRUCTRAG-FT, whose training already teaches direct rationale generation from noisy inputs.The authors attribute the FT decline to redundant demonstrations and a training–inference discrepancy.Source: §3.3 · p. 8

3.4 ANALYSIS

  • More demonstrations steadily help INSTRUCTRAG-ICL, while plain-answer few-shot prompting peaks at one; adding documents lowers retrieval precision but benefits INSTRUCTRAG-ICL and FT.The two InstructRAG variants gain rather than degrade as the number of retrieved documents and associated noise increase.Source: §3.4 · p. 8, Figure 3 · p. 8
  • INSTRUCTRAG outperforms baselines across in-domain and out-of-domain task transfers, and QA-trained INSTRUCTRAG-FT also improves unseen code-generation performance.The code-generation comparison uses pass@k in both non-retrieval and RAG settings.Source: §3.4 · pp. 8–9, Table 5 · p. 9
  • GPT-4o judging generally yields higher evaluation results than pattern-matching metrics, and INSTRUCTRAG outperforms baselines under both evaluation approaches.The authors attribute the higher judge scores mostly to matching semantically equivalent phrasings.Source: §3.4 · p. 9

4 RELATED WORK

Prior RAG work often reduces noise exposure through retrieval improvements, while LM reasoning work has generally not addressed potentially misleading retrieved content. INSTRUCTRAG instead targets explicit denoising in RAG, an underexplored use of instruction-following models.

4.1 RETRIEVAL-AUGMENTED GENERATION

  • INSTRUCTRAG takes an orthogonal approach to retrieval-quality improvements by explicitly denoising retrieved content, aiming to improve noise robustness and generation accuracy even in highly noisy contexts.Existing work commonly improves retrieval quality or reduces noise exposure; examples include adaptive retrieval and query rewriting.Source: §4.1 · p. 10

4.2 ELICITING REASONING IN LARGE LANGUAGE MODELS

  • LM reasoning research has rarely addressed RAG, where noisy retrieved content can mislead reasoning, and using instruction-following LMs for explicit RAG denoising remains underexplored.CoT demonstrations often require costly manual crafting, while other work selects or self-curates demonstrations or elicits zero-shot rationales.Source: §4.2 · p. 10

5 CONCLUSION

INSTRUCTRAG uses synthetic rationales to teach explicit denoising in RAG. Across five knowledge-intensive benchmarks, it outperforms state-of-the-art RAG approaches in training-free and trainable settings, with an average improvement of 8.3% over the best baseline.

5 CONCLUSION

  • 8.3% average improvement over the best baseline across five knowledge-intensive benchmarks, with INSTRUCTRAG outperforming state-of-the-art RAG approaches in both training-free and trainable settings.The paper characterizes the gains as improving the noise robustness of retrieval-augmented generation.Source: §5 · p. 10
  • INSTRUCTRAG generates detailed rationales explaining how ground-truth answers can be derived from retrieved documents, enabling explicit denoising.The rationales serve as either in-context learning examples or supervised fine-tuning data.Source: §5 · p. 10
  • The method requires no additional supervision compared with standard RAG methods and applies to both in-context learning and supervised fine-tuning.These synthetic rationales provide the supervision for the model to learn explicit denoising.Source: §5 · p. 10

REPRODUCIBILITY STATEMENT

The reproducibility statement lists source code, implementation details, and experiment prompt templates as available materials.

REPRODUCIBILITY STATEMENT

A LIMITATIONS AND FUTURE WORKS

The experiments focus mainly on question-answering, leaving generalization to scenarios such as open-ended generation unclear. The authors also identify evaluation-metric and training-sample biases, and outline future work on rationale quality, filtering, long-context settings, and retrieval integration.

A LIMITATIONS AND FUTURE WORKS

  • Experiments mainly cover question-answering, so generalization to other scenarios such as open-ended generation remains unclear.Source: A LIMITATIONS AND FUTURE WORKS · p. 20
  • Accuracy and exact match can miss semantically equivalent phrasings and favor longer generations, limiting how well they reflect generation quality.The authors suggest exploring LLM-as-judge metrics.Source: A LIMITATIONS AND FUTURE WORKS · p. 20
  • Rationale quality is lower when retrieved documents contain no relevant evidence: consistency is 89% across training samples versus 98% for samples with at least one relevant document.The authors propose exploring a filtering mechanism and better rationale-generation techniques, including domain-specific knowledge or multi-task learning.Source: A LIMITATIONS AND FUTURE WORKS · p. 20
  • Future work includes evaluating long-context settings with dynamic or very large document sets and integrating advanced retrieval techniques such as active retrieval.Source: A LIMITATIONS AND FUTURE WORKS · p. 20

B IMPLEMENTATION DETAILS

The implementation uses benchmark-specific retrievers and document counts, with a common fine-tuning setup and two-shot greedy inference by default.

Retrieval setup

  • Wikipedia passages are retrieved with task-specific sparse or dense retrievers; the default is top 5 documents, except top 10 for 2WikiMultiHopQA.The retriever is Contriever-MS MARCO for PopQA and TriviaQA, DPR for Natural Questions, GTR for ASQA, and BM25 for 2WikiMultiHopQA.Source: B IMPLEMENTATION DETAILS · p. 20

Training details

  • Fine-tuning uses Adam for 2 epochs with batch size 128, learning rate 2.5e-5, and a cosine schedule with 3% warmup.Vanilla SFT uses learning rate 2e-5; Self-RAG and RetRobust are re-implemented and tuned, with RetRobust trained for 5 epochs on 2WikiMultiHopQA.Source: B IMPLEMENTATION DETAILS · p. 20

Inference details

  • INSTRUCTRAG-ICL and few-shot demonstrations use 2 examples by default, and generation uses greedy decoding.Models are loaded with vLLM for memory-efficient inference.Source: B IMPLEMENTATION DETAILS · p. 20

C CASE STUDY

The case studies compare INSTRUCTRAG with other RAG approaches through attention and generated responses, illustrating document denoising and answer generation from model knowledge when needed.

Generation comparison

  • Figure 6 marks irrelevant or inaccurate generations in red and question-relevant content in green.The case study reports that INSTRUCTRAG-FT can leverage its own knowledge when required.Source: Figure 6 · p. 21

Attention visualization

  • Figure 5's attention visualization identifies Doc 2, the only relevant document containing the answer, as the benign document in noisy input.The visualization uses answer-to-document attention from the last 10 layers and shows a stronger denoising signal than vanilla SFT.Source: C CASE STUDY · p. 21, Figure 5 · p. 21

Generation comparison

  • INSTRUCTRAG-FT alone answers the ASQA example correctly while giving comprehensive denoising details, unlike in-context RALM and vanilla SFT.It identifies potentially relevant documents, lays out candidate information, and can draw on parametric knowledge when no relevant document remains in context.Source: C CASE STUDY · p. 21, Figure 6 · p. 21

D PROMPT TEMPLATES

The prompts use a shared instruction-tuned model template, task-adapted rationale generation prompts, and matched inference instructions whose demonstrations distinguish INSTRUCTRAG from its baselines.

Prompt setup

  • The prompts instantiate INSTRUCTRAG with Llama-3-Instruct models and apply the official Meta-Llama-3-Instruct chat template.The models are the 8B and 70B instruction-tuned variants.Source: D PROMPT TEMPLATES · p. 22

Rationale generation

  • Rationale generation uses a shared prompt structure across five benchmarks, with task-specific instructions and a separate template for ablation.Tables 7 and 8 present the shared structure and task instructions, while Table 6 gives the ablation template.Source: D PROMPT TEMPLATES · p. 22, Table 7 · p. 22, Table 8 · p. 22

Inference prompts

  • Inference instructions are matched within each comparison; INSTRUCTRAG-ICL uses question-rationale demonstrations while few-shot instruction uses question-answer demonstrations.The prompt pairs are Table 9 for in-context RALM, Table 10 for INSTRUCTRAG-ICL and few-shot instruction, and Table 11 for INSTRUCTRAG-FT and vanilla SFT.Source: D PROMPT TEMPLATES · p. 23, Table 10 · p. 23, Table 11 · p. 23

E EXAMPLE OF LLM-AS-A-JUDGE

Rationale quality is evaluated with an LLM judge that checks consistency between the rationale and the final answer.

LLM-as-a-judge

Related papers

Loading 2406.13629v3…