Learning to Filter Context for Retrieval-Augmented Generation

• FILCO targets retrieval-augmented generation systems whose irrelevant or distracting passages can encourage hallucination, spurious memorization, or incorrect extraction.Source: §1 · p. 1
• The method filters context sentence by sentence using STRINC, lexical overlap, and conditional cross-mutual information (CXMI), then trains filtering models for test-time use.Source: §1 · p. 2
• Across six datasets with FLAN-T5 and LLAMA2, FILCO outperforms full-context augmentation and passage-wise filtering.Source: §1 · p. 2
• FILCO reduces prompt length by 44 −64% across tasks while improving generation with both positive and negative retrieved passages.Source: §1 · p. 2
• STRINC works best for extractive QA, LEXICAL for dialog generation, and CXMI for more complex tasks; FILCO also retains its advantage in multi-passage settings.Source: §1 · p. 2

2.1 Problem Statement

• FILCO represents retrieved passages as candidate text spans and selects filtered context T before conditioning the generation model on the query and T.Source: §2.1 · p. 2

2.2 Obtaining Oracle Contexts

• Oracle filtering selects a single best span using a filtering function, because single-span filtering outperformed multi-span filtering in preliminary studies.Source: §2.2 · p. 2
• STRINC selects the first span containing the exact output, whereas LEXICAL selects a high-overlap sentence; STRINC may select spurious matches and fails when no exact answer span exists.Source: §2.2 · p. 3
• Lexical overlap can reinforce misinformation when queries are factually incorrect, because misleading spans may match erroneous claims.Source: §2.2 · p. 3
• CXMI compares output probabilities with and without a candidate span, selecting the highest-scoring span above a threshold; it avoids lexical barriers but requires more computation.Source: §2.2 · p. 3

2.3 Learning to Filter Contexts

• At training time, a context filtering model learns to generate silver filtered context from the query and retrieved passages, approximating filtering when gold answers are unavailable.Source: §2.3 · p. 3
• At test time, the filtering model predicts context that is passed with the query to the generation model.Source: §2.3 · p. 3, §2.4 · pp. 3–4

2.4 Generation With Filtered Contexts

• Generation is trained and evaluated with filtered context rather than all retrieved spans, reducing computational cost in proportion to the amount of text removed.Source: §2.4 · p. 4

3.1 Tasks and Datasets

• The benchmark covers extractive, multi-hop, and long-form question answering, fact verification, and knowledge-grounded dialog generation using six Wikipedia-based datasets.The datasets are NQ, TQA, HotpotQA, ELI5, FEVER, and WoW, with task-specific evaluation metrics.Source: §3.1 · p. 4

3.2 Wikipedia Passage Retrieval

• Wikipedia passages are retrieved with adversarial Dense Passage Retriever, selecting the top five passages for every example before evaluating retrieval quality.The retrieval analysis reports top-1 and top-5 recall and precision on positive passages.Source: §3.2 · pp. 4–5, Table 2 · p. 5

A Mixture of Positive and Negative Passages

• Top-1 retrieved passages support the canonical output in at most half of examples across all six datasets, while adding passages increases coverage but also computation.The evaluation measures answer-string coverage for extractive QA and provenance-article coverage for the other tasks.Source: A Mixture of Positive and Negative… · p. 5
• Retrieved positive passages contain limited precisely supporting content: unigram precision is below 20.0 for WoW and below 5.0 for NQ, TQA, HotpotQA, and FEVER.ELI5 has unusually high top-1 precision, but adding four passages reduces its precision by over 40 points.Source: A Mixture of Positive and Negative… · p. 5
• These retrieval deficiencies motivate filtering sufficient and precisely necessary context so generation can focus on relevant knowledge more efficiently.The paper connects low precision and redundant content with potential distraction and degraded generation.Source: A Mixture of Positive and Negative… · p. 5

4.3 Generation Performance

• FILCO improves results over FULL and PSG across all six datasets, while achieving performance comparable to silver-filtered contexts.The comparison uses four methods and two backbone models; finer-grained filtering outperforms passage-wise filtering.Source: §4.3 · p. 6
• +4.3 and +8.6 EM in NQ and +1.1 and +0.2 EM in TQA show FILCO’s gains on extractive QA with FLAN-T5 and LLaMA2, respectively.The filter removes distracting alternative answers and irrelevant passages, helping models identify the correct answer span.Source: §4.3 · p. 6
• +1.0 and +1.3 F1 in HotpotQA and +0.6 and +2.6 EM in ELI5 show gains on more complex QA tasks with FLAN-T5 and LLaMA2.The reported improvement is smaller than for extractive QA, presumably because these tasks are more difficult.Source: §4.3 · p. 6
• +6.2 and +4.3 accuracy in FEVER and +3.5 and +1.1 F1 in WoW show larger gains on abstractive generation tasks with FLAN-T5 and LLaMA2.Filtering irrelevant content helps models focus on the concerned knowledge.Source: §4.3 · p. 6

4.4 Generation With Filtered Positive and Negative Passages

• FILCO improves end-generation results for both positive and negative retrieved passages, particularly on FEVER and WoW.It removes distracting content from positive passages and removes negative passages.Source: §4.4 · pp. 6–7

4.5 Evaluating Filtered Contexts

• FILCO reduces input length by 44−64% and produces higher context precision than unfiltered contexts across tasks.Silver filtering raises precision by +14.5 on HotpotQA and +60.7 on WoW, while FILCO remains largely comparable and reaches +3.8 points over silver filtering in TQA; PSG can have lower precision than FULL.Source: §4.5 · p. 7

5.1 Results with Different Strategies

• Different tasks favor different filters: STRINC performs best on NQ and TQA, LEXICAL on WoW, and CXMI on FEVER, HotpotQA, and ELI5.FLAN-T5 and LLAMA2 agree on most task preferences, with slight divergence on ELI5.Source: §5.1 · p. 8, Table 4 · p. 8

5.2 In-Depth Analysis for Different Tasks

• STRINC suits extractive QA because sentences lexically entailing the answer usually provide the ground-truth support, apart from spurious contexts.The analysis identifies NQ and TQA as the extractive tasks benefiting most from STRINC.Source: §5.2 · p. 8
• STRINC falls short on abstractive FEVER and WoW because canonical outputs may not appear in supporting documents, producing empty context.LEXICAL F1 provides more flexible unigram matching and is most suitable for dialog generation.Source: §5.2 · p. 8
• CXMI works best for multi-hop QA, long-form QA, and fact verification, where lexical overlap can select misleading but irrelevant spans.In FEVER, CXMI can favor evidence supporting the factual judgment rather than an incidental matching span such as “2000 BC.”Source: §5.2 · p. 8
• Figure 7 contrasts STRINC’s empty context with the highlighted LEXICAL and CXMI-filtered contexts in a FEVER example.The figure visualizes how filtering outcomes differ for the same example.Source: Figure 7 · p. 8

6.1 Baseline and Settings

• The experiment uses K = 5 retrieved passages to limit truncation losses and support fairer comparisons.FILCO is compared with unfiltered FULL, passage-filtering PSGS, RAG, FiD, EVI., and the silver-context setting.Source: §6.1 · p. 8

6.2 Generation Performance

• +1.2–14.2 points: FILCO surpasses FULL and PSGS across all six tasks and also outperforms existing performant baselines.The comparison uses top-5 retrieved passages filtered at the sentence level, with Table 5 reporting the generation results.Source: §6.2 · p. 9, Table 5 · p. 9
• Aggregating multiple top-ranked passages improves extractive-task performance relative to using only top-1 passages.The reported comparison specifically concerns extractive tasks in the multiple-passage setting.Source: §6.2 · p. 9
• FEVER and WoW performance drops by −3.2 and −2.3 points, potentially because lower-ranked passages have poorer retrieval quality.The passage notes that top-1 retrieval recall is relatively high for these tasks.Source: §6.2 · p. 9

Augmented Generation

• Retrieval-augmented generation spans paragraph, phrase, and token granularities, exposing a trade-off between retrieval difficulty and generation accuracy.Longer sequences are easier to retrieve but harder for generators to use correctly.Source: §7 · p. 9
• FILCO permits arbitrary passage sizes at retrieval time and provides precisely useful content to reduce distraction within output-supporting documents.This fine-grained filtering complements prior answer-sentence selection while optimizing context for the task of interest.Source: §7 · p. 9

Optimizing Retrieval for Augmentation

• Prior approaches rerank or select retrieved passages, but coarse passage-level filtering still leaves in-passage distractions.Existing methods use query similarity, reader predictions, generation utility, evidentiality, or retrieval skipping.Source: §7 · p. 9

8 Conclusion and Future Work

• FILCO removes distracting content from partially supporting and irrelevant passages, improving generation with precisely supportive context.The method applies across question answering, fact verification, and knowledge-grounded dialog generation.Source: §8 · p. 9
• Across six datasets, FILCO improves performance by an average of 2.8 points with FLAN-T5 and 3.0 points with LLAMA2.The conclusion reports these averages across the three task types evaluated in the paper.Source: §8 · p. 9
• The effective recipe for filtering context varies across tasks rather than following one universally best measure.This conclusion follows the paper’s task-specific strategy analysis.Source: §8 · p. 9

Domain scope

• The experiments use datasets built from Wikipedia articles, so effectiveness on special-domain datasets should be verified before extrapolating the conclusions.The authors mention news, biomedical, and fictional-story domains as potential applications but do not directly evaluate them.Source: Limitations · pp. 9–10

Evaluation

• Evaluation relies on automatic metrics such as Exact Match and Unigram F1, whose accuracy may be limited for complex tasks and larger-capacity models.The authors leave neural- or human-based evaluation open as a complement to automatic measures.Source: Limitations · p. 10

Computational requirements

• FILCO requires computational resources to train context-filtering and generation models, although filtered-content generation uses at least 4.7 times less computation than full-passage augmentation.The reduction applies at both training and inference time.Source: Limitations · p. 10