PaperlayerView exact revision on arXiv

Source-linked AI summary

ReliableRAG: Combating Misinformation in Retrieval-Augmented Generation via Reliability-Guided Reasoning Chains

Jinpu Jiang, Xuan Wu, Wenhao Song, Bo Yang, You Zhou, Hongwei Ge, Heow Pueh Lee, Yanchun Liang, Chunguo Wu

arXiv:2608.25487v1cs.CLcs.IR

TL;DR

RAG systems can be misled by deceptive misinformation during multi-hop reasoning, while existing approaches have limited fine-grained reliability assessment. ReliableRAG evaluates individual structured triples and constructs reliability-guided reasoning chains; across three adversarially augmented datasets, it outperforms existing methods and improves factual reliability and robustness.

  • Problem

    Existing RAG approaches have limited ability to assess fine-grained information reliability, leaving multi-hop reasoning vulnerable to deceptive misinformation.

  • Method

    ReliableRAG extracts structured triples, evaluates their credibility and query-specific relevance, and uses reliable triples to construct autoregressive reasoning chains.

  • Results

    ReliableRAG outperforms existing methods across three multi-hop QA datasets adversarially augmented with deceptive misinformation, improving factual reliability and robustness.

  • Takeaways & Limitations

    Anchoring multi-hop reasoning in reliable triples provides a robust approach for filtering deceptive misinformation before final answer generation.

  • Takeaways & Limitations

    The chain-construction exemplars are curated to maintain no overlap with the test datasets.

Abstract

from arXiv · show

Retrieval-Augmented Generation (RAG) has emerged as a powerful architecture for Question Answering (QA) by integrating external information into Large Language Models (LLMs). However, false, inaccurate, and misleading information in news and social media poses a serious challenge to real-world RAG systems, especially in multi-hop QA, where complex multi-step reasoning can be misled by even a single deceptive misinformation segment in the retrieved documents. Existing approaches mainly rely on implicit alignment or explicit regulation, but their limited ability to assess fine-grained information reliability makes them vulnerable to deceptive misinformation that is semantically relevant to the question yet factually incorrect, leading to erroneous answers. To address this limitation, we propose ReliableRAG, which, to the best of our knowledge, is the first reliability-driven framework that mitigates deceptive misinformation in multi-hop QA through fine-grained evaluation of individual triples. ReliableRAG first extracts information segments from source documents and represents them as structured triples. It then quantifies triple reliability by combining query-triple semantic relevance with triple credibility, retaining only the top-$K$ reliable and non-redundant triples. Based on these refined triples, ReliableRAG autoregressively constructs robust reasoning chains to consolidate trustworthy evidence and filter deceptive misinformation, producing accurate answers faithful to reliable information. Experiments on three multi-hop QA datasets show that ReliableRAG outperforms existing methods, substantially improving the factual reliability and robustness of RAG systems under deceptive misinformation injection.

1 Introduction

ReliableRAG addresses deceptive misinformation in multi-hop QA by evaluating fine-grained triples and constructing reasoning chains from reliable evidence. Experiments on three adversarially augmented datasets show improved factual reliability and robustness.

  • Motivation: RAG retrieves relevant documents and prepends them to questions so an LLM can aggregate complementary information for multi-hop QA.Multi-hop tasks require complex, multi-step reasoning across disparate sources.
  • Limitations of Existing Methods: Existing implicit alignment and explicit regulation methods can struggle with semantically similar, fine-grained deceptive misinformation.Implicit alignment also requires substantial computational resources and high-quality training data, while explicit regulation has limited fine-grained reliability perception.
  • ReliableRAG: ReliableRAG evaluates fine-grained individual triples to prevent deceptive misinformation from misleading multi-hop question answering.The framework is presented as a reliability-driven approach and introduces a Triple Evaluation module for dynamically quantifying triple-level reliability.
  • ReliableRAG: The framework extracts structured triples, pre-assesses their credibility, and dynamically evaluates chain-specific queries before reasoning.Its modules are Triple Extraction, Triple Evaluation, and Chain Construction, operating across offline and online phases.
  • Experiments: ReliableRAG outperforms existing methods across three multi-hop QA datasets augmented with deceptive misinformation.The experiments report improved factual reliability, robustness, and answer accuracy under misinformation interference.
  • ReliableRAG: ReliableRAG autoregressively constructs robust reasoning chains anchored in reliable triples before generating the final answer.This design provides a highly trustworthy search space and aims to prevent misinformation propagation during reasoning.

2 Related Work

Related work improves RAG robustness through implicit alignment or explicit regulation, but ReliableRAG instead quantifies reliability at the triple level without training.

  • Prior RAG Methods: RAG research improves multi-hop QA through iterative search, Reasoning with Attributions, and preference optimization.These techniques facilitate the reasoning process but do not by themselves resolve misinformation vulnerability.
  • Implicit Alignment: Implicit alignment trains models to internalize self-reflection, decision-policy, or credibility criteria for regulating reliance on retrieved information.Such methods aim to adapt generation based on retrieval necessity and output relevance.
  • ReliableRAG: ReliableRAG is a training-free alternative that transforms documents into structured triples and combines query-triple relevance with triple credibility.It uses these reliability signals to construct robust reasoning chains and filter deceptive misinformation.

3 ReliableRAG

ReliableRAG converts retrieved documents into fine-grained triples, evaluates their semantic relevance and credibility, and uses reliable non-redundant triples to construct multi-hop reasoning chains. This design addresses misinformation and lost-in-the-middle risks by concentrating inference on a trustworthy search space.

  • Motivation: RAG can be misled when semantically relevant documents contain factual misinformation, especially during multi-hop reasoning over scattered evidence.Direct reasoning over coarse-grained documents can also cause models to overlook critical information in long contexts.
  • Triple Extraction: Triple Extraction transforms each document into multiple structured subject-predicate-object triples to support fine-grained processing.The resulting triple set contains M ≥ K representations, enabling structured selection rather than document-level reasoning.
  • Triple Evaluation: Triple Evaluation ranks triples by combining query-triple semantic relevance with triple credibility, selecting the top-K reliable triples while masking previously used information to avoid redundancy.Semantic relevance is computed with cosine similarity, while credibility evaluates entities and their relational claim at the triple level.
  • Triple Evaluation: Credibility analysis separately examines whether entities are authentic and whether the predicate expresses a coherent relationship between them.The evaluator synthesizes these analyses into a credibility score from 0 to 10.
  • Chain Construction: Chain Construction uses a multiple-option selector to choose probable triples, appends them to existing chains, and retains only the top-F chains by confidence across hops.Construction continues until the maximum hop is reached or chains emit a termination signal, using reliable triples to reduce deceptive misinformation.

4 Experiments

Experiments evaluate ReliableRAG on three multi-hop QA datasets with injected deceptive misinformation, comparing its performance, robustness, component contributions, and sensitivity to key settings.

  • Experimental Setup: ReliableRAG is evaluated on HotPotQA, 2WikiMultiHopQA, and MuSiQue using EM and F1 under ideal and evaluator-generated settings.The study injects low-credibility documents and compares ReliableRAG with diverse fine-tuning and non-fine-tuning baselines.
  • Main Results: 55.10%, 49.50%, and 31.60% EM are achieved on HotPotQA, 2WikiMultiHopQA, and MuSiQue under the Ideal setting, respectively.These are the highest EM results across the three tested LLMs and all test sets.
  • Main Results: 40.30%, 33.80%, and 19.50% EM are achieved on HotPotQA, 2WikiMultiHopQA, and MuSiQue under the evaluator-generated setting, respectively.The results are reported across all tested LLMs and support robustness against deceptive misinformation.
  • Robustness to Misinformation: ReliableRAG outperforms six compared methods with minimal performance drops as the number of low-credibility documents increases.This pattern also appears under evaluator-generated settings; ReliableRAG-SBS on HotPotQA improves slightly as misinformation rises.
  • Ablation Studies: Removing Triple Extraction causes a 34.1% EM drop on HotPotQA, while excluding Triple Evaluation causes a 15.6% EM decrease.The complete ReliableRAG-TBS performs best across all test sets, and the ablation results emphasize the importance of both modules.
  • Sensitivity Analyses: EM begins declining when the number of triples exceeds typically 15 to 25, while reasoning depth beyond L > 4 yields stagnant or slightly lower EM.These findings indicate that additional triples or reasoning steps do not necessarily improve performance and may introduce misinformation.
  • Sensitivity Analyses: α*=0.4 provides the most balanced performance by trading off query-triple semantic relevance against triple credibility.The coefficient regulates the dual-factor perception mechanism in Triple Evaluation.

5 Conclusion

ReliableRAG combats deceptive misinformation in multi-hop QA by evaluating fine-grained triple reliability and constructing reliable autoregressive reasoning chains. Experiments report improved performance, factual reliability, and robustness against misinformation propagation.

  • Conclusion: ReliableRAG dynamically evaluates reliability at the triple level, filters deceptive misinformation, and constructs reliable autoregressive reasoning chains.The framework is presented as a solution for preventing misinformation during multi-hop reasoning.
  • Conclusion: ReliableRAG significantly outperforms existing methods and provides a robust approach for factual reliability under deceptive misinformation.The conclusion emphasizes preventing misinformation propagation throughout multi-hop reasoning processes.
  • Conclusion: The framework is compared with diverse methods, including Naive LLM, Vanilla RAG, Prompt-based, Exclusion, Self-RAG, CAG, Knowledgeable-R1, CrAM, and TruthfulRAG.These comparisons include approaches with and without fine-tuning.

A.3 Implementation Details

ReliableRAG converts source documents into structured triples, evaluates their reliability, constructs reasoning chains, and synthesizes evidence for generation. Its analyses also examine indirect effects and robustness under evaluator-generated misinformation.

  • Context synthesis: Triple-based Synthesis concatenates triple components into concise text, whereas Source-based Synthesis maps triples to source documents and ranks them by voting frequency.TBS emphasizes information density, while SBS restores richer source-document details.
  • Framework workflow: Triple Evaluation combines query-triple semantic relevance with credibility, retaining the top-K reliable and non-redundant triples for chain construction.This creates a more trustworthy search space for subsequent reasoning.
  • Framework workflow: Beam search autoregressively constructs reasoning chains from the filtered triple space, reducing exposure to deceptive misinformation during multi-hop reasoning.The case study shows the final chains are synthesized into context for answer generation.
  • Analytical procedures: The indirect-effect analysis uses a baseline probability P0 and an optimized probability under α* to quantify the influence of the balancing coefficient.A positive shift in the indirect-effect distribution indicates enhanced misinformation mitigation.
  • Robustness analysis: As misinformation proportions increase, CrAM and Exclusion degrade substantially, while ReliableRAG maintains robust performance with minimal degradation.ReliableRAG-TBS EM on MuSiQue remains stable as misinformation scales up.

E.2 Robustness of ReliableRAG

ReliableRAG remains robust as deceptive misinformation becomes more prevalent, while compared methods deteriorate. Increasing maximum chain length beyond four provides little additional benefit on MuSiQue.

  • Evaluator-generated setting: ReliableRAG maintains robust performance with minimal degradation as the proportion of low-credibility documents increases.ReliableRAG-TBS EM on MuSiQue remains remarkably stable under rising misinformation.
  • Chain-length sensitivity: When L > 4, average chain-length growth slows significantly while EM remains virtually stagnant or slightly declines.The same pattern is reported across MuSiQue, HotpotQA, and 2WikiMultiHopQA.
  • Bi-encoder sensitivity: E5-Mistral delivers strong performance across datasets and synthesis strategies, with notable advantages on complex multi-hop settings.The comparison includes DRAGON+, E5, and E5-Mistral.

F.3 Ablation study for ReliableRAG-SBS

Ablation results show that ReliableRAG-SBS depends substantially on Triple Extraction, Triple Evaluation, and Chain Construction for performance and misinformation resistance.

  • Module ablations: ReliableRAG-SBS outperforms all ablated variants across the three multi-hop QA test sets.The complete system is the strongest configuration in the reported ablation.
  • Module ablations: 35.8% EM is lost when Triple Extraction is removed on HotpotQA, the largest reported ablation drop.The result supports the role of triples in organizing information for multi-hop QA.
  • Module ablations: 31.0% EM is lost when Triple Evaluation is excluded.The penalty is more pronounced for SBS than TBS because SBS maps triples back to coarse-grained source documents.
  • Module ablations: Without Chain Construction, EM declines once the triple count exceeds a threshold typically between 15 and 20.Blindly increasing the number of triples does not yield further performance gains.

F.4 Analysis of Input Computational Overhead

ReliableRAG-TBS produces the most concise input context, whereas SBS incurs greater overhead to preserve richer source-document semantics.

  • Context length: ReliableRAG-TBS maintains the most concise average input context among the compared methods.It concatenates each triple’s subject, predicate, and object into a single textual string.
  • Context length: ReliableRAG-SBS has a larger average input context because it maps triples back to their original source documents.The additional context preserves richer semantic nuances and comprehensive textual background.

F.5 Extra Results

Supplementary experiments test ReliableRAG with alternative generators and a coarse-grained document variant. The framework remains competitive across models, while Qwen2.5-7B with SBS reaches 71.3% F1 on HotPotQA.

  • Different Generators: ReliableRAG achieves competitive performance across datasets with Llama-2-7B and Qwen2.5-7B generators.The evaluation covers both TBS and SBS variants under the ideal setting.
  • Different Generators: 71.3% F1 on HotPotQA is achieved by ReliableRAG with Qwen2.5-7B under the SBS strategy.
  • Different Generators: The supplementary generator results indicate that ReliableRAG maintains robust performance without relying on a specific generator.
  • Coarse-Grained Documents: The study also evaluates dual-factor perception on coarse-grained documents using offline document credibility scores and α*=0.4.These evaluations are reported under the optimal configuration in Table 9.
  • Functional Units: The extractor, evaluator, and selector are examined as separate functional units in additional comparative experiments.

G.1 Impact of Different Extractors

Additional analyses examine ReliableRAG’s sensitivity to extractors, selectors, evaluators, and credibility-score distributions. Results suggest limited dependence on extractor or evaluator choice, while selector performance varies across models.

  • Impact of Different Extractors: Overall performance is not sensitive to the specific choice of extractor.The extractor comparison uses Llama3-8B-Instruct, Qwen-7B-Instruct, and Qwen-14B-Instruct in an ablated configuration.
  • Impact of Different Selectors: 53.4% EM on HotPotQA is achieved by Llama3-8B in the TBS variant, exceeding Mistral-7B and Gemma-7B.
  • Impact of Different Selectors: Llama3-8B-Instruct achieves the highest scores across all reported metrics and datasets in the selector comparison.The comparison includes Llama3-8B-Instruct, Mistral-7B, and Gemma-7B.
  • Impact of Different Evaluators: Performance differences among GLM-4-Flash, Mistral-Small-24B, and Qwen3.5-35B-A3B evaluators are relatively small.Qwen3.5-35B-A3B achieves marginally higher metrics across datasets.
  • Credibility-Score Distributions: The evaluator’s discriminative capability is assessed for both documents and extracted triples across three multi-hop QA test sets.The analysis uses credibility-score distributions before quantitative evaluation.
  • Credibility-Score Distributions: Wikipedia-sourced samples receive predominantly high credibility scores, whereas low-credibility samples have more dispersed scores with some high-valued cases.The latter pattern suggests that some segments in low-credibility documents may still contain correct information.

H.2 Quantitative Evaluation via ROC Curves

The ROC analysis evaluates how well credibility scores distinguish low- and high-credibility documents and triples across three multi-hop QA test sets. The reported results show that ReliableRAG remains reliable and robust with GLM-4-Flash as evaluator.

  • ROC-Curve Evaluation: Credibility scores are converted into low- versus high-credibility labels across thresholds from 0 to 10.Samples below each threshold are categorized as low-credibility, and the rest as high-credibility.
  • ROC-Curve Evaluation: ROC curves evaluate the evaluator’s discriminative performance for documents and triples across three multi-hop QA datasets.True Positive Rate and False Positive Rate are calculated across the threshold range.
  • ROC-Curve Evaluation: The reported results demonstrate high reliability and robustness with GLM-4-Flash as the evaluator, mitigating deceptive misinformation’s impact.
  • Credibility Assessment: The assessment framework distinguishes authentic entities from inaccurate relational claims when assigning a final triple credibility score.The example concerns the triple claiming Albert Einstein was the first recipient of the 1921 Nobel Prize in Physics.
Loading 2608.25487v1…