PaperlayerView exact revision on arXiv

Source-linked AI summary

Retrieval-Augmented Generation with Conflicting Evidence

Han Wang, Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal

arXiv:2504.13079v2cs.CLcs.AI

TL;DR

RAG systems must handle ambiguity, misinformation, and noise jointly, although prior datasets and methods largely isolate these conflicts. The paper introduces RAMDocs and MADAM-RAG, combining a challenging benchmark with document-specific multi-agent debate and aggregation. MADAM-RAG improves over strong baselines on ambiguity and misinformation tasks, while RAMDocs remains difficult, especially under imbalanced evidence.

  • Problem

    Existing RAG evaluations and methods generally address ambiguity, misinformation, or noise separately, despite their joint occurrence in retrieved evidence.

  • Method

    The paper introduces RAMDocs and MADAM-RAG, which assigns documents to independent agents that debate across rounds before an aggregator synthesizes the response.

  • Results

    MADAM-RAG outperforms strong baselines across ambiguity and misinformation evaluations, with gains up to 11.40% on AmbigDocs and 15.80% on FaithEval.

  • Takeaways & Limitations

    RAMDocs provides a unified test of multiple conflicting-evidence behaviors, while MADAM-RAG begins to address these conflicts across diverse models and tasks.

Abstract

from arXiv · show

Large language model (LLM) agents are increasingly employing retrieval-augmented generation (RAG) to improve the factuality of their responses. However, in practice, these systems often need to handle ambiguous user queries and potentially conflicting information from multiple sources while also suppressing inaccurate information from noisy or irrelevant documents. Prior work has generally studied and addressed these challenges in isolation, considering only one aspect at a time, such as handling ambiguity or robustness to noise and misinformation. We instead consider multiple factors simultaneously, proposing (i) RAMDocs (Retrieval with Ambiguity and Misinformation in Documents), a new dataset that simulates complex and realistic scenarios for conflicting evidence for a user query, including ambiguity, misinformation, and noise; and (ii) MADAM-RAG, a multi-agent approach in which LLM agents debate over the merits of an answer over multiple rounds, allowing an aggregator to collate responses corresponding to disambiguated entities while discarding misinformation and noise, thereby handling diverse sources of conflict jointly. We demonstrate the effectiveness of MADAM-RAG using both closed and open-source models on AmbigDocs -- which requires presenting all valid answers for ambiguous queries -- improving over strong RAG baselines by up to 11.40% and on FaithEval -- which requires suppressing misinformation -- where we improve by up to 15.80% (absolute) with Llama3.3-70B-Instruct. Furthermore, we find that RAMDocs poses a challenge for existing RAG baselines (Llama3.3-70B-Instruct only obtains 32.60 exact match score). While MADAM-RAG begins to address these conflicting factors, our analysis indicates that a substantial gap remains especially when increasing the level of imbalance in supporting evidence and misinformation.

1 Introduction

RAG systems must jointly resolve ambiguity, misinformation, and noise because these conflicts require different response behaviors. The paper introduces RAMDocs and MADAM-RAG to evaluate and address these challenges together.

  • Ambiguous queries can retrieve multiple factually correct answers, whereas misinformation and noise should be filtered out.
  • MADAM-RAG assigns each retrieved document to an independent agent, whose response is debated across multiple rounds before aggregation.
  • RAMDocs combines ambiguity, misinformation, and noise in one dataset while varying the amount of supporting evidence for each valid answer.
  • 11.40% absolute accuracy improvement over Astute-RAG was achieved on AmbigDocs with Llama3.3-70B-Inst.
  • 13.10% improvement over Astute-RAG was achieved on FaithEval with Qwen2.5-72B-Inst.

2 Related Work

Prior RAG benchmarks study ambiguity, noise, misinformation, attribution, or relevance largely as separate concerns. RAMDocs unifies these conflict types while explicitly supporting multiple valid answers and uneven evidence.

  • AmbigNQ and AmbigDocs evaluate questions with multiple valid answers, but do not consider noise or misinformation.
  • RGB evaluates retrieval noise, while other benchmarks address attribution, broad question settings, or stance and relevance.
  • Existing datasets generally assume a single correct answer, leaving multiple conflicting yet valid answers insufficiently addressed.
  • RAMDocs combines ambiguity, uneven supporting-document counts, misinformation, and noisy evidence in a unified benchmark.

3 RAMDocs: Retrieval with Ambiguity & Misinformation in Documents

RAMDocs simulates inter-context conflict by combining ambiguity, misinformation, and noise while requiring different behaviors for valid and invalid evidence. Its uneven support distribution tests whether models avoid favoring the most frequent answer.

  • RAMDocs treats ambiguity as multiple valid answers, but expects only correct answers when conflict comes from misinformation or noise.
  • Each query has 1–3 disambiguated answers, with each answer supported by 1–3 documents and an average of 2.20 valid answers.
  • RAMDocs contains 500 queries averaging 5.53 retrieved documents, including 3.84 supporting documents and 1.70 misinformation or noise documents per query.
  • The dataset randomizes correct-answer counts, support counts, misinformation, and noise to reflect variable real-world retrieval conditions.

4 MADAM-RAG: Multi-agent Debate for Ambiguity and Misinformation in RAG

MADAM-RAG separates document processing across agents, uses iterative debate to revise and challenge responses, and applies an aggregator to reconcile evidence. This design aims to preserve distinct valid answers while filtering unsupported claims.

  • Role of a Single LLM Agent: Each retrieved document is assigned to an independent LLM agent that generates an intermediate response from the query and that document alone.
  • Role of a Single LLM Agent: Independent processing lets agents review a document thoroughly and reduces dependence on document frequency.
  • Multi-round Debate: In later rounds, agents receive an aggregator-generated summary and may defend, challenge, or revise their claims.
  • Aggregator Module: The aggregator compares agent responses, resolves inconsistencies, and synthesizes answers based on the available evidence.
  • Aggregator Module: For ambiguous Michael Jordan queries, the aggregator retains 1963 and 1956 for distinct entities while flagging 1998 as misinformation.
  • Multi-round Debate: Debate convergence uses an early-stopping criterion when agents retain their previous answers, after which the aggregator determines the final answer.

5 Experiments and Results

MADAM-RAG is evaluated on FaithEval, AmbigDocs, and RAMDocs against multiple baselines and language models. It consistently outperforms competing RAG approaches, while RAMDocs remains challenging because conflicting evidence must be jointly resolved.

  • Datasets: RAMDocs evaluates ambiguity, misinformation, and noise together, requiring systems to retain valid alternatives while rejecting incorrect evidence.The dataset is designed to simulate real-world retrieval conditions involving multiple conflicting sources.
  • Main Results: MADAM-RAG consistently outperforms concatenated-prompt and Astute RAG across all three datasets and model backbones.The evaluation covers FaithEval, AmbigDocs, and RAMDocs, using open-source and closed-source models.
  • Main Results: 11.40% absolute accuracy separates MADAM-RAG from Astute RAG on AmbigDocs with Llama3.3-70B-Inst.The comparison measures performance on ambiguous queries requiring multiple valid answers.
  • Main Results: 15.80% improvement over concatenated-prompt on FaithEval is achieved with Llama3.3-70B, where only one correct answer is required despite misinformation.The result indicates robustness to subtle misinformation in retrieved evidence.
  • Main Results: MADAM-RAG exceeds single-agent self-reflection by 17.40% on FaithEval, 3.70% on AmbigDocs, and 4.00% on RAMDocs with Llama3.3-70B-Inst.The comparison highlights gains over a single-agent baseline across misinformation, ambiguity, and combined-conflict settings.
  • Analysis: Single-agent systems remain vulnerable to context length limits, frequency bias, and order effects because they process concatenated input.Parametric knowledge alone avoids retrieved misinformation but tends to provide only one answer for ambiguous queries.

6 Ablations and Analysis

Ablations and controlled analyses examine the aggregator, debate rounds, evidence imbalance, and misinformation. Multi-round debate and aggregation improve performance, while increasing imbalance or misinformation degrades competing systems and remains challenging overall.

  • 6.1 Importance of Using the Aggregator and Multiple Rounds of Debate: Both enabling the aggregator and increasing debate rounds produce substantial gains in MADAM-RAG ablations.Without the aggregator, accuracy rises by 21.10 points from round 1 to round 3 on FaithEval, while F1 increases by 3.62 on RAMDocs.
  • 6.2 Impact of varying the number of Supporting Documents: Up to 8% performance loss affects baselines as supporting-document imbalance increases, especially for concatenated-prompt.The baselines increasingly favor the answer with more supporting documents and may suppress an underrepresented valid answer.
  • 6.2 Impact of varying the number of Supporting Documents: A variance of 3.33 across rounds indicates that a single agent can champion a valid answer with minimal interference from other agents and evidence imbalance.This observation concerns the multi-agent discussion under imbalanced document support.
  • 6.3 Impact of Increasing Misinformation: MADAM-RAG maintains the highest performance at each fixed misinformation level, although its performance also decreases as misinformation increases.The experiment introduces one to three documents supporting a factually incorrect alternative while measuring exact match.
  • 6.3 Impact of Increasing Misinformation: 46% decreases occur for both concatenated-prompt and Astute RAG as misinformation increases.The setting uses 200 queries with two correct answers and incrementally adds one to three misinformation documents.

7 Conclusion

The paper introduces RAMDocs and MADAM-RAG to address conflicting evidence from ambiguity, misinformation, noise, and imbalanced support. Its analyses find the aggregator important and MADAM-RAG strongest in high-conflict settings, though the conclusion passage does not quantify the remaining gap.

  • Conclusion: RAMDocs combines ambiguity, misinformation, noise, and varying support levels while assigning different expected behaviors to each conflict source.Ambiguous queries require multiple correct answers, whereas misinformation and noise require returning only correct answers.
  • Conclusion: MADAM-RAG assigns separate documents to agents, uses debate, and synthesizes their reasoning with an aggregator model.The approach improves performance across standard RAG datasets and RAMDocs.
  • Conclusion: The aggregator is important, and MADAM-RAG best handles scenarios with high conflict as evidence imbalance and misinformation vary.These findings come from the paper’s ablation and conflict-level analyses.

A RAMDocs: Dataset Statistics

RAMDocs summarizes retrieved evidence along document- and answer-level dimensions, capturing ambiguity, evidence imbalance, misinformation, and irrelevant noise. Its statistics quantify the mixture of supporting, incorrect, and noisy documents and answers.

  • RAMDocs reports document-level counts for retrieved, correct-supporting, incorrect-supporting, and noisy or irrelevant documents per example.The corresponding averages are 5.53, 3.84, 0.61, and 1.08.
  • RAMDocs reports answer-level counts for correct and wrong answers and the documents supporting each answer.The corresponding averages are 2.20 correct answers, 0.86 wrong answers, 1.77 documents per correct answer, and 0.73 documents per wrong answer.

B Speculative RAG

Speculative RAG clusters retrieved documents by perspective, samples documents into subsets, and uses specialist drafters to produce candidate answers and rationales for aggregation.

  • Speculative RAG clusters retrieved documents into k perspective-based groups using InBedder-RoBERTa-large.It samples one document from each cluster to construct distinct subsets δj.
  • Each document subset is passed to a specialist RAG drafter that generates a draft answer αj and rationale βj.The aggregator receives all generated answers and rationales because the setting may contain multiple correct answers.

C Astute RAG

Astute RAG addresses imperfect or incomplete retrieval by generating internal knowledge, consolidating it with retrieved passages, and selecting a final answer from candidate groups.

  • Astute RAG is evaluated as a method for conflicts between retrieved context and the LLM’s parametric knowledge.Its motivation is imperfect or incomplete retrieval.
  • Astute RAG first generates internal passages from the query without accessing external documents.The model determines how many passages to generate, particularly when retrieved documents are sparse, noisy, or misleading.
  • It consolidates internal and external passages using source tags to identify consistency, conflicts, and irrelevant or unreliable information.The LLM reorganizes the inputs into a smaller set of refined, source-attributed passages.
  • Astute RAG generates one answer per refined passage group and selects among candidates using source reliability, confirmation, frequency, thoroughness, and confidence.This is the method’s answer-finalization stage.

D Computational Efficiency of MADAM-RAG

MADAM-RAG reduces debate cost through early stopping while using more tokens than prompt-based RAG and a comparable token budget to single-agent self-reflection.

  • MADAM-RAG terminates debate early when agents retain their answers across consecutive rounds, with convergence observed within 1-2 rounds for most examples.The stopping rule is designed to improve efficiency.
  • 3186 input tokens and 1547 output tokens are required on average by MADAM-RAG, versus 822/126 for prompt-based RAG.MADAM-RAG is comparable to the Single Agent with Self-reflection baseline when input and output tokens are considered together.
  • MADAM-RAG introduces overhead from its multi-agent setup while remaining computationally tractable relative to strong single-agent iterative alternatives.The cited comparison also reports that it outperforms zero-shot and single-agent approaches.

No RAG

The no-RAG approaches prompt a model to answer from multiple documents, optionally with reasoning, review, refinement, or aggregation. Some variants consolidate consistent evidence, separate conflicting evidence, assign confidence, and select all well-supported answers.

  • The basic prompt asks an expert question-answering model to answer using multiple documents and list all possible correct answers.
  • Single Agent with Self-reflection: Self-reflection variants ask the model to review its answer, identify problems, and improve or refine it.
  • Aggregation: Multi-response aggregation instructs an aggregator to read several drafted responses and return all possible correct answers with explanations.
  • Knowledge consolidation: Knowledge-consolidation variants cluster consistent documents, separate conflicting documents, exclude irrelevant information, assign confidence to candidate answers, and select accurate supported answers.
  • Multi-agent prompting: Agent-based prompting has individual agents answer from separate documents, then an aggregator combines their responses while allowing multiple answers or unknown when appropriate.
Loading 2504.13079v2…