PaperlayerView exact revision on arXiv

Source-linked AI summary

The Distracting Effect: Understanding Irrelevant Passages in RAG

Chen Amiraz, Florin Cuconasu, Simone Filice, Zohar Karnin

arXiv:2505.06914v1cs.CLcs.IR

TL;DR

Irrelevant retrieved passages can distract answer-generating LLMs, but the distracting effect of a passage has lacked a quantifiable measure and diverse collection methods. This paper defines a measure and develops methods to collect distractors; combined methods provide more distracting passages, and fine-tuning on them outperforms an analogous standard-retrieval training set.

  • Problem

    The paper asks how to evaluate a passage’s distracting effect on an LLM for a query, an issue arising when irrelevant but semantically related passages may mislead answer generation.

    Source: §1 · p. 1

  • Method

    The paper defines a quantifiable, passage-isolating measure and collects distractors using a skewed retriever and LLM-generated categories.

    Source: §1 · pp. 1–2

  • Results

    The combined methods produce more distracting passages than any single method, and fine-tuning on the resulting set achieves superior results to an analogous standard-retrieval training set.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The collected passages can form more challenging and diverse RAG datasets and support fine-tuning LLMs to be more robust to distracting passages.

    Source: §6 · p. 9

    Experiments used English-language benchmarks; generalization to other languages is suggested but remains formally unverified.

    Source: Limitations · p. 10

Abstract

from arXiv · show

A well-known issue with Retrieval Augmented Generation (RAG) is that retrieved passages that are irrelevant to the query sometimes distract the answer-generating LLM, causing it to provide an incorrect response. In this paper, we shed light on this core issue and formulate the distracting effect of a passage w.r.t. a query (and an LLM). We provide a quantifiable measure of the distracting effect of a passage and demonstrate its robustness across LLMs. Our research introduces novel methods for identifying and using hard distracting passages to improve RAG systems. By fine-tuning LLMs with these carefully selected distracting passages, we achieve up to a 7.5% increase in answering accuracy compared to counterparts fine-tuned on conventional RAG datasets. Our contribution is two-fold: first, we move beyond the simple binary classification of irrelevant passages as either completely unrelated vs. distracting, and second, we develop and analyze multiple methods for finding hard distracting passages. To our knowledge, no other research has provided such a comprehensive framework for identifying and utilizing hard distracting passages.

1 Introduction

RAG can be misled by irrelevant but semantically related passages, motivating a way to evaluate their distracting effect. The paper measures that effect, studies ways to collect distractors, and uses them for fine-tuning.

1 Introduction

  • The proposed measure isolates a passage’s distracting effect for a query and LLM; scores correlate across LLMs, and higher-scoring passages reduce accuracy alongside the gold passage.The effect varies by LLM, despite the reported cross-LLM score correlations.Source: §1 · p. 1
  • Irrelevant results from stronger retrievers are more distracting than those from weaker retrievers, and higher-ranking irrelevant results are more likely to distract.Source: §1 · pp. 1–2
  • The collection methods target queries where standard retrieval fails, using a skewed retriever or LLM-generated passages in several categories.Retrieval may fail to return distractors, or no such passages may exist in the corpus.Source: §1 · p. 2
  • Combining the collection methods yields more distracting passages for queries than any single method, particularly standard retrieval.Source: §1 · p. 2
  • Fine-tuning on the resulting training set achieves superior results to fine-tuning on an analogous set obtained via standard retrieval.Source: §1 · p. 2

2 Related work

Related work examines irrelevant passages, ways to obtain hard examples, and methods for robust answer generation. This paper treats distraction as a continuous property and develops additional ways to obtain such passages.

2 Related work

  • Cuconasu et al. (2024) distinguish random from distracting irrelevant passages; this paper extends that framing by quantifying distraction as a continuous property.Source: §2 · p. 2
  • Passages for answer generation have commonly come from retrieval; Abdumalikov et al. (2024) also synthesize passages containing the question but not the answer, or vice versa.Yoran et al.’s method samples uniformly from the top K results rather than taking the top-ranked passages [1].Source: §2 · p. 2
  • Hard negatives target a retrieval or reranking system’s judgment of relevance, whereas this paper focuses on passages that can distract an answer generator; known answers also help filter false negatives.The cited work also discusses threshold-based filtering (Moreira et al., 2024) and instruction-mismatched negatives (Weller et al., 2024).Source: §2 · pp. 2–3
  • Robust answer-generation approaches include chain-of-thought methods, faithfulness-based relevance decisions, and fine-tuning with relevant and irrelevant passages.Prior fine-tuning work uses standard-retrieval passages, while Yoran et al. (2024) also use intentionally irrelevant passages sampled from top results.Source: §2 · p. 3

3 Distracting Passages

Section 3 defines the distracting effect of an irrelevant passage relative to a query and an LLM, then describes ways to obtain such passages.

3.1 Measuring the Distracting Effect

  • The measure scores an irrelevant passage by the probability that the LLM does not abstain when answering the query from it, and is used to rank passages per query.The score is bounded between 0 and 1 and does not require extra passages or a reference model.Source: §3.1 · p. 3

3.2 Obtaining Distracting Passages

  • Candidate construction combines retrieval and LLM generation: retrieval resembles inference, while synthetic examples may add robustness for rare distractors and queries with no corpus examples.Passages that include the ground-truth answer or entail it are excluded as relevant.Source: §3.2 · pp. 3–4

3.2.1 Retrieving Distracting Passages

  • Retrieval methods use top-ranked irrelevant passages; answer-skewed dense retrieval alters the query embedding to find documents related to the query but unrelated to its answer.Its λ parameter determines how aggressively answer-related documents are excluded.Source: §3.2.1 · p. 4

3.2.2 Generating Distracting Passages

  • Few-shot prompting of a strong LLM generates distractor categories including related-topic, hypothetical, negation, and modal-statement passages.These categories respectively include related information without the answer, a hypothetical alternative answer, a negated wrong answer, and a qualified wrong answer.Source: §3.2.2 · p. 4

4 Analyzing the Distracting Effect

Across the tested datasets, higher-ranked retrieved passages tend to be more distracting, and reranking increases this effect. Combining methods yields highly distracting passages, while hard distractors lower answer accuracy even when a gold passage is present.

4.1 Experimental Setting

  • Experiments use four question-answering datasets and evaluate distracting effects with open-source LLMs ranging from 3B to 70B parameters.The datasets are NQ, PopQA, TriviaQA, and WebQA.Source: §4.1 · p. 5

4.2 Distracting Effect of Retrieved Passages

  • Distracting effect decreases with retrieval rank, but reranking consistently raises it among the top five passages.The authors argue that passages surviving reranking are more likely to mislead the LLM.Source: §4.2 · pp. 5–6, Figure 2 · p. 6

4.3 Comparing Distracting Effects of Different Methodologies

  • Rst+ and Gmodal are among the most distracting approaches, while Rsk and Grel are among the least; reranking increases distraction for both retrieval methods.These method strengths and weaknesses persist across datasets.Source: Figure 1 · p. 5, §4.3 · p. 6
  • Jointly selecting the most distracting passage across methods produces passages with higher distracting effects than any individual method, and retrieval-plus-generation combinations contribute to this set.Rst+ most often provides the top passage, but other methods do so for about 48% of queries.Source: §4.3 · pp. 6–7, Figure 3 · p. 6

4.4 Prompting with Relevant and Distracting Passages

  • Hard distracting passages reduce answer accuracy by 6 to 11 accuracy points versus gold-only prompting, depending on the LLM.Weak distractors also reduce accuracy, but less; the degradation persists for Llama-3.3-70B.Source: Table 1 · p. 7, §4.4 · p. 7

5 Application: RAG Fine-Tuning

The section uses constructed hard-distractor training data to fine-tune LLMs and reports accuracy gains, especially on ungrounded cases.

5 Application: RAG Fine-Tuning

  • For each query, the training prompt combines a relevant passage with highly distracting passages to train a robust RAG generation component.These query-and-passage prompts form the training set.Source: §5 · p. 8

5.1 Experimental Setting

  • Training uses 800 NQ queries; the remaining NQ queries form the in-distribution test set, while PopQA, TriviaQA, and WebQA supply out-of-distribution tests.The models fine-tuned are Llama-3.2-3B and Llama-3.1-8B, compared with their non-fine-tuned counterparts.Source: §5.1 · pp. 8–9
  • Retrieve uses the top five results without reranking, while Rerank enables the reranking module.The three strategies produce corresponding training sets.Source: §5.1 · p. 8
  • Hard uses the first relevant Rerank passage plus four highly distracting passages in half of cases, and five highly distracting passages in the rest.The five passages are shuffled to form the prompt passage list.Source: §5.1 · p. 8

5.2 Results

  • For Llama-3.2-3B, Hard lifts overall accuracy by 6.7 points on TriviaQA and 7.6 on WebQA; gains are stronger on ungrounded cases but slightly trade off grounded accuracy.For Llama-3.1-8B, Hard has better overall performance in three of four benchmarks.Source: §5.2 · p. 9
  • Hard-example fine-tuning lifts all baselines by 5.3–16.1 absolute accuracy points for Llama-3.2-3B and 3.6–11.0 points for Llama-3.1-8B across test sets.Table 2 reports overall accuracy averaged over four test sets and indicates statistically significant Hard advantages except for Llama-3.1-8B on PopQA, where the result is slightly below Rerank and not significant.Source: Table 2 · p. 9, §5.2 · p. 9

6 Conclusions

The paper measures passage-level distracting effects, develops ways to obtain hard distracting passages, and uses them to fine-tune LLMs for greater robustness.

6 Conclusions

  • The paper provides an algorithm to measure a passage’s distracting effect with respect to a query and LLM, and reports robustness across LLM types and alternative implementations.The conclusion describes this as a measure of the passage’s effect in context.Source: §6 · p. 9
  • Combining methods for obtaining distracting passages produces more distracting passages and supports more challenging, diverse RAG datasets.The paper explores approaches beyond standard retrieval.Source: §6 · p. 9
  • The authors demonstrate that the collected passages can be used to fine-tune LLMs to be more robust to distracting passages.They describe this application as one of potentially many uses of their findings.Source: §6 · p. 9

Limitations

The paper identifies limits in its distracting-passage taxonomy, task coverage, and experimental language coverage.

Limitations

  • The generation categories do not capture the full range of distracting passage types, leaving rhetorical strategies and domain-specific use cases for future taxonomy expansion.The paper frames this expansion as an open research question.Source: Limitations · pp. 9–10
  • The study primarily investigates question answering, so applying the analysis to additional RAG tasks remains future work.The paper says other RAG use cases exist and that additional tasks would provide a more complete picture.Source: Limitations · p. 10
  • Experiments use English-language benchmarks; generalization to other languages is suggested by the methodology but remains formally unverified.The paper leaves verification of this hypothesis open.Source: Limitations · p. 10

A Additional Details on Distracting Effect

The appendix describes a measure based on whether an LLM is likely to abstain when given a query and passage.

A Additional Details on Distracting Effect

  • The procedure constructs a prompt that instructs the LLM to answer “NO-RESPONSE” when the context contains no relevant answer.The prompt uses the target answer in the template shown in Figure 6.Source: A Additional Details on Distracting… · p. 11
  • The measure uses the LLM’s probability of generating the first token of “NO-RESPONSE” as a confidence signal.This probability is evaluated for the query and passage.Source: A Additional Details on Distracting… · p. 12
  • High “NO-RESPONSE” probability yields a distracting-effect score near 0, while low probability yields a score near 1.A low probability indicates greater inclination to answer based on the passage.Source: A Additional Details on Distracting… · p. 12

A.1 Distracting Effect on Other LLMs and Datasets

Across the tested LLMs, distracting-effect distributions on TriviaQA and PopQA follow patterns observed on NQ and WebQuestions. Model families differ in classification behavior, while smaller models are more susceptible to distraction and can still improve with fine-tuning.

A.1 Distracting Effect on Other LLMs and Datasets

  • Distraction distributions on TriviaQA and PopQA follow NQ and WebQuestions patterns across all seven tested LLMs.The appendix reports similar patterns across the tested datasets and models.Source: §A.1 · p. 12
  • Qwen models concentrate about 90% of probability mass at the weak- and hard-distraction extremes, while Falcon models vary more in confidence and behavior.Falcon-3-3B often fails to abstain and answers regardless of passage relevance, explaining its distributions skewed toward maximum distracting effects.Source: §A.1 · p. 12
  • The 3B versions across model families are more susceptible to distraction than larger counterparts, but fine-tuning improves Llama-3.2-3B’s handling of distracting passages.The paper suggests larger models generally develop more robust mechanisms for handling irrelevant information during training.Source: §A.1 · p. 12

A.2 Answer-Skewed Retriever Hyper-parameters

The answer-skewed retriever’s configuration balances obtaining query-related but answer-unrelated passages against producing passages distinct from standard retrieval. The authors select λ = 1 and the Esub formulation.

A.2 Answer-Skewed Retriever Hyper-parameters

  • λ = 1 provided the selected balance for retrieving answer-skewed distracting passages; λ > 1 pushed results too far from the query topic.At λ = 2.0, average distracting effect was 0.08 for Esub and 0.19 for Eproj.Source: §A.2 · p. 12
  • With λ < 1, the answer-skewed retriever behaved too much like the standard retriever and returned almost the same documents.The authors conclude this would provide no additional contribution.Source: §A.2 · p. 12
  • Esub and Eproj produced quite similar results, but Esub had a higher proportion of unique wins across the tested LLMs.Source: §A.2 · p. 12

B Additional Details on RAG Fine-Tuning

The appendix supplies the distracting-effect evaluation and fine-tuning prompts, implementation settings, and examples illustrating how hard and weak distractors affect model answers. The examples also show that a passage’s distracting effect can vary across LLMs.

B Additional Details on RAG Fine-Tuning

  • Fine-tuning used three epochs, NEFTune noise α = 5, and maximum gradient norm 0.3, with model-specific LoRA ranks and learning-rate schedules.Llama-3.2-3B used rank 64 and a constant 3e −5 learning rate; Llama-3.1-8B used rank 128 and a cosine schedule beginning at 5e −5.Source: B Additional Details on RAG Fine-Tuning · p. 12
  • The evaluation algorithm builds a prompt from a query and passage and locates the NO-RESPONSE target position before computing distracting effect.The provided algorithm excerpt specifies the query, passage, LLM, and tokenizer as inputs, and distracting effect as its output.Source: B Additional Details on RAG Fine-Tuning · p. 13
  • A hard distractor can produce an incorrect answer despite relevant information being present, while weak distractors in the examples allow correct answers.For Llama-3.2-3B, the hard-distractor example scores 1.0 across tested models, while its weak-distractor example has consistently low scores.Source: Figure 8 · p. 14, B Additional Details on RAG Fine-Tuning · p. 14
  • A passage classed as weakly distracting for some models can still mislead others, including Qwen-2.5-3B in the example.The same passage is weakly distracting for Qwen-2.5-7B, Llama-3.1-8B, and Llama-3.3-70B, but other models score it above 0.68.Source: Figure 9 · p. 15, B Additional Details on RAG Fine-Tuning · p. 15

B.1 Accuracy By Training Set

The appendix reports accuracy breakdowns by test-set slice and training strategy, and documents the prompts used to generate distracting-passage categories. It also includes figures comparing distracting-effect distributions and retrieval-rank positions.

B.1 Accuracy By Training Set

  • Hard training yields its most significant improvement on ungrounded samples across test sets, datasets, and models; accuracy is generally highest when training and test-set types match.For Llama-3.2-3B on TriviaQA and WebQA, Hard outperforms None, Retrieve, and Rerank across all three accuracy metrics.Source: §B.1 · p. 17, Table 3 · p. 27

C Generation Prompts

  • The appendix provides generation prompts for related-topic, hypothetical, negation, and modal-statement distracting passages.Source: C Generation Prompts · p. 17, Figure 22 · p. 28, Figure 23 · p. 29
  • Figures 11–16 show distributions of distracting effects for passages from different methods across all datasets, measured by six LLMs; methods are ordered by mean, with higher means indicating greater distraction.Source: Figure 11 · p. 18, Figure 12 · p. 19, Figure 13 · p. 20
  • Higher-ranked retrieved passages consistently have greater potential to mislead models than lower-ranked passages.Figure 20 reports average distracting effect by retrieval rank across models, averaged over datasets.Source: Figure 20 · p. 26

Related papers

Loading 2505.06914v1…