PaperlayerView exact revision on arXiv

Source-linked AI summary

HANRAG: Heuristic Accurate Noise-resistant Retrieval-Augmented Generation for Multi-hop Question Answering

Duolin Sun, Dan Yang, Yue Shen, Yihan Jiao, Zhehao Tan, Jie Feng, Lianzhen Zhong, Jian Wang, Peng Wei, Jinjie Gu

arXiv:2509.09713v1cs.CLcs.AI

TL;DR

Multi-hop RAG faces inefficient retrieval for compound queries, difficulty targeting sub-queries, and noise in retrieved documents. HANRAG uses Revelator to route queries, process compound and complex questions differently, and filter retrieval noise. The paper reports strong results on both single-hop and multi-hop benchmarks.

  • Problem

    Multi-hop RAG methods can spend multiple retrieval rounds on compound queries, miss sub-query-relevant information when retrieving from the original query, and accumulate noise.

    Source: §1 · p. 2

  • Method

    HANRAG uses Revelator to route queries, parallelize retrieval for compound sub-queries, refine seed questions for complex queries, and filter irrelevant documents.

    Source: §1 · p. 2, §4.1 · p. 5, §7 · p. 8

  • Results

    HANRAG performs excellently on multi-hop benchmarks and achieves impressive performance on single-hop benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The paper presents HANRAG as an adaptive framework that routes diverse queries to corresponding reasoning chains for targeted resolution.

    Source: §7 · p. 8

    Constructing corresponding training data for each agent increases the practical application cost.

    Source: §8 · p. 9

Abstract

from arXiv · show

The Retrieval-Augmented Generation (RAG) approach enhances question-answering systems and dialogue generation tasks by integrating information retrieval (IR) technologies with large language models (LLMs). This strategy, which retrieves information from external knowledge bases to bolster the response capabilities of generative models, has achieved certain successes. However, current RAG methods still face numerous challenges when dealing with multi-hop queries. For instance, some approaches overly rely on iterative retrieval, wasting too many retrieval steps on compound queries. Additionally, using the original complex query for retrieval may fail to capture content relevant to specific sub-queries, resulting in noisy retrieved content. If the noise is not managed, it can lead to the problem of noise accumulation. To address these issues, we introduce HANRAG, a novel heuristic-based framework designed to efficiently tackle problems of varying complexity. Driven by a powerful revelator, HANRAG routes queries, decomposes them into sub-queries, and filters noise from retrieved documents. This enhances the system's adaptability and noise resistance, making it highly capable of handling diverse queries. We compare the proposed framework against other leading industry methods across various benchmarks. The results demonstrate that our framework obtains superior performance in both single-hop and multi-hop question-answering tasks.

1 Introduction

The introduction identifies inefficiencies and noise in multi-hop RAG, then presents HANRAG as a Revelator-driven framework for routing and resolving varied queries. It reports strong performance on both single-hop and multi-hop benchmarks.

1 Introduction

  • Prior multi-hop RAG systems often use iterative retrieval for compound queries, creating multiple alternating retrieval-and-generation rounds and inefficiency.Compound queries seek multiple facts about one subject, whereas complex queries involve closely connected sub-queries.Source: Figure 1 · p. 1, §1 · p. 2
  • Using the original query for repeated retrieval can miss information needed for sub-queries, particularly in queries with three or more hops.The introduction also notes that basing next-step decisions on the original query and refined chunk content can be inaccurate.Source: §1 · p. 2
  • Without postprocessing, retrieved documents can add irrelevant noise, while fine-grained character- or word-level filtering is described as inefficient.The introduction says accumulated noise can hinder subsequent LLM performance.Source: §1 · p. 2
  • HANRAG uses Revelator to route queries, decompose compound problems, refine complex questions, and filter irrelevant retrieved documents.For compound questions it uses parallel retrieval; for complex questions, seed-question refinement guides iterative retrieval and generation.Source: §1 · p. 2
  • The authors introduce a heuristic training method for Revelator and a benchmark of compound multi-hop queries.They describe the framework as noise-resistant and adaptive.Source: §1 · p. 2
  • HANRAG performs excellently on multi-hop benchmarks and achieves impressive performance on single-hop benchmarks.The introduction presents this as an experimental result across both task types.Source: §1 · p. 2

2 Related work

The related-work section outlines multi-hop QA and its retrieval-generation workflow, then reviews iterative and adaptive retrieval approaches alongside challenges in accuracy, reasoning, and noise.

2 Related work

  • Multi-hop QA retrieves distributed knowledge through a multi-step reasoning chain, requiring interleaved retrieval, generation, and decisions about whether to continue.It extends beyond single-document QA by drawing on multiple documents.Source: §2 · pp. 2–3
  • A typical multi-hop QA system retrieves relevant documents, generates an answer from them, and assesses whether further retrieval is needed.The process repeats retrieval when the original query remains unanswered.Source: §2 · p. 3
  • Multi-hop QA faces challenges in cross-domain retrieval accuracy, generation reasoning, and noise accumulation over retrieval iterations.These are contrasted with single-hop QA systems.Source: §2 · p. 3
  • Iterative RAG adjusts retrieval based on preliminary generations, for example by rephrasing the query or expanding retrieval, and is described as useful for complex multi-step queries.The passage characterizes repeated self-correction as improving answer coherence and factual accuracy.Source: §2 · p. 3
  • Adaptive retrieval selects a retrieval strategy based on query complexity, but the reviewed approaches include inefficiency on compound queries and difficulty with complex multi-hop tasks.The section also describes retrieval delays and a binary retrieval decision that struggles with more complex tasks.Source: §2 · p. 3

3 Preamble

The preamble distinguishes queries that need no retrieval, single-step queries, compound queries, and complex queries. It describes direct answering, separate retrieval for compound sub-queries, and iterative refinement for complex questions.

3 Preamble

  • Straightforward queries can be answered from the LLM’s internal knowledge without external retrieval, which the paper presents as the time-efficient choice.Examples include identity-related and general-knowledge questions.Source: §3 · p. 3
  • Single-step queries may require external retrieval because incomplete internal knowledge can lead LLMs to give incorrect answers.The passage cites a birthdate question as an example.Source: §3 · pp. 3–4
  • Compound queries ask about multiple attributes of one entity and are decomposed so each sub-query becomes an individual retrieval task.The corresponding answers are combined with the original query and sub-query predictions to generate the final response.Source: §3 · p. 4
  • Complex queries contain closely connected sub-queries requiring logical reasoning, so the system refines seed questions and alternates retrieval and generation until reaching an answer.The passage identifies the first seed question as a prerequisite for answering subsequent questions.Source: §3 · p. 4

4 Methdology

HANRAG uses the Revelator to route queries to different processing chains and uses ANRAG to filter retrieved documents before generation. It trains Revelator capabilities with task-specific examples and keeps training data separate from test data.

4.1 Framework

  • Revelator coordinates HANRAG, while ANRAG retrieves passages, filters them for query relevance, and passes relevant documents to the LLM for answering.Straightforward queries go directly to the LLM; single-step queries use ANRAG.Source: §4.1 · p. 4
  • Compound queries are decomposed into independent sub-questions whose retrieval chains run asynchronously before their results are combined; complex queries use iterative retrieval and stop when sufficient information is available.For complex queries, Revelator refines a prerequisite seed question and evaluates whether its answer resolves the original query.Source: §4.1 · pp. 4–5, Figure 2 · p. 5

4.2 Data construction for Revelator

  • The training data includes four query types and relevance-annotated query-document pairs; the authors state that training and test data do not overlap.Qwen2-72B-instruct is used for relevance annotation.Source: §4.2 · pp. 5–6
  • Revelator training examples supervise decomposition from compound queries, refinement from reasoning steps, relevance judgments, and end-or-continue decisions based on last- versus earlier-hop answers.The end decision treats the final hop as complete and earlier hops as requiring further retrieval.Source: §4.2 · p. 6

5 Experiments

The experiments evaluate HANRAG on single-hop and multi-hop benchmarks, including a newly introduced compound-query dataset. They compare it with no-retrieval, single-step, iterative, and adaptive retrieval methods using effectiveness and retrieval-step measures.

Datasets

  • Evaluation covers three single-hop datasets, three complex multi-hop datasets, and a newly introduced compound-query dataset.The named single-hop datasets are SQuAD v1.1, Natural Questions, and TriviaQA; the complex-query datasets are HotpotQA, 2WikiMultihopQA, and MuSiQue.Source: §5 · p. 6

Methods

  • Comparisons include direct LLM answering, BM25 single-step retrieval, IRCoT, Adaptive Retrieval, self-RAG, and Adaptive-RAG.The stated goal is to assess HANRAG's versatility and generalizability.Source: §5 · p. 6

Metrics

  • Effectiveness is measured with EM, F1, and accuracy, while efficiency is measured by the number of retrieval-generation cycles per query.Accuracy denotes whether the ground truth is included in the prediction.Source: §5 · pp. 6–7

6 Results

HANRAG outperforms Adaptive-RAG across reported single-hop, complex, and compound-query results, while reducing retrieval steps in the reported settings. Ablations also show a substantial accuracy drop when the Refiner is removed.

Effect and efficiency of single-hop queries

  • Across single-hop benchmarks, HANRAG exceeds Adaptive-RAG by 12.2%, 6.83%, and 20.13% on EM, F1, and Accuracy, respectively.The passage also reports an average reduction of approximately 0.13 retrieval steps across the three datasets.Source: §6 · p. 7, Figure 3 · p. 7, Table 1 · p. 8

Effect and efficiency of complex queries

  • Across complex-query datasets, HANRAG improves over Adaptive-RAG by 6.67%, 6.34%, and 16.17% on EM, F1, and Accuracy, respectively.It also reduces average retrieval steps by 0.52 compared with Adaptive-RAG.Source: §6 · p. 7, Figure 3 · p. 7, Table 2 · p. 8

Effect and efficiency of multi-hop compound queries

  • On the compound-query dataset, HANRAG exceeds Adaptive-RAG by 19.63% in accuracy and reduces retrieval steps by nearly 1.5.Accuracy is calculated by splitting each ground truth into entities and counting predicted entities present as positive samples.Source: §6 · p. 8, Table 2 · p. 8

Ablation Study

  • Removing any single Revelator module lowers complex-query performance; removing the Refiner causes a 10% drop in overall accuracy.The authors link the Refiner's role to extracting seed problems for complex questions.Source: §6 · p. 8

7 Conclusion

HANRAG uses the Revelator to route queries into different reasoning paths, including parallel retrieval for compound questions and iterative refinement for complex ones. It also filters retrieved documents for relevance and reports outperforming several state-of-the-art methods.

7 Conclusion

  • The Revelator routes queries, decomposes compound questions for parallel retrieval, and iteratively refines seed queries for complex questions.Answers to compound sub-queries are aggregated; complex-query processing alternates refinement, retrieval, generation, and an ending judgment.Source: §7 · p. 8
  • A trained relevance discriminator filters retrieved documents, while high-quality training data and multi-task learning integrate the modules into the Revelator.Source: §7 · p. 8
  • HANRAG outperforms several state-of-the-art methods in comparative evaluations, according to the authors.Source: §7 · p. 8

8 Limitations

The authors identify per-agent training-data construction as a practical cost of HANRAG. The accompanying appendix passages describe single-step retrieval, an ending discriminator, and a relevance discriminator.

8 Limitations

  • Constructing training data for each agent increases practical application costs; developing a lighter, high-performance RAG system is left to future work.Source: §8 · p. 9

A.1 Single-step Retrieval

  • Single-step retrieval selects a fixed number of documents from an external knowledge base using the user’s query, then supplies them to the LLM.The passage identifies the Retriever as a model often implemented by embedding models.Source: §A.1 · p. 11

A.2 Ending discriminator

  • The ending discriminator determines whether iterative retrieval-generation continues or the current reasoning step suffices to answer the original complex query.A value of 0 continues the process; a value of 1 ends it.Source: §A.2 · p. 11

A.3 Relevance discriminator

  • The relevance discriminator retains retrieved documents relevant to query q and discards irrelevant ones.Source: §A.3 · p. 11

B Details of data construction

The appendix describes construction of a compound-query benchmark and training data for the Router, Decomposer, and Refiner.

B.1 Compound Queries Benchmark

  • The compound benchmark combines questions about the same entity into 2–4-hop queries, with 50,000 training, 8,000 development, and 2,000 test samples.The process begins with 10,000 randomly selected Wikipedia entities and extracts 10 documents per entity.Source: §B.1 · p. 11

B.2 Router

  • Router training covers straightforward, single-step retrieval, complex multi-hop, and compound multi-hop queries, drawing on the listed datasets and the compound benchmark.The passages report 12,247 straightforward samples and 50,000 samples each for single-step retrieval and complex multi-hop queries.Source: §B.2 · p. 11

B.3 Decomposer

  • The Decomposer is trained on compound-benchmark samples, pairing compound queries with their constituent sub-queries retrieved from storage.The training format is <Q, q1, q2, ...>.Source: §B.3 · pp. 11–12

B.4 Refiner

  • The Refiner uses step-by-step reasoning from MuSiQue and 2Wiki as seed problems, and includes single-step problems as both input and output.Source: §B.4 · p. 12

C Additional Experimental Results

HANRAG-Oracle comparisons show modest gains over HANRAG on the compound benchmark and improvements across the three complex benchmarks. On single-step retrieval, Oracle gains are smaller, while the Revelator’s four-way routing test accuracy is 83.93%.

C Additional Experimental Results

  • The Revelator achieves 83.93% accuracy when routing the test set among four query types.The evaluation uses 1,500 queries for each type: straightforward, single-step, complex, and compound.Source: C Additional Experimental Results · p. 12
  • On the three single-step retrieval test sets, HANRAG-Oracle increases EM by 1.17%, F1 by 1.8%, and Accuracy by 1.8% compared with HANRAG.The passage attributes most errors in these tasks to retrieval inaccuracies rather than routing errors.Source: C Additional Experimental Results · p. 12
  • On the three complex benchmarks, HANRAG-Oracle increases EM by 2.6%, F1 by 2.47%, and Accuracy by 3.27% compared with HANRAG.The reported average-steps improvement is 0.1.Source: C Additional Experimental Results · p. 12
  • On the compound benchmark, HANRAG-Oracle improves Accuracy by 1.36% and reduces retrieval steps by 0.24 compared with HANRAG.The passage says each correctly classified compound query needed one retrieval step under Oracle.Source: C Additional Experimental Results · pp. 12–13, Table 5 · p. 13

D Implementation details

The implementation uses eight Nvidia A100 GPUs and fine-tunes with LoRA for one epoch. It specifies the training framework, learning rate, scheduler, and warm-up share.

D Implementation details

E Ablation study

The ablations examine the relevance discriminator, ending discriminator, and refiner; case studies contrast HANRAG with Adaptive-RAG on single-step and compound queries. The prompt list covers query classification, seed-question extraction, and concise answering from retrieved documents.

E Ablation study

  • Removing the relevance discriminator reduces performance, while removing the ending discriminator leaves overall performance virtually unchanged but raises retrieval steps to 4.5.The reported maximum retrieval-step limit is 4.5; the authors attribute continued retrieval after reaching the final answer to the missing ending discriminator.Source: E Ablation study · p. 13
  • Without the refiner, using the original query for each retrieval round leads to inaccurate retrievals that can mislead the LLM into incorrect content.Source: E Ablation study · p. 13

F Case study

  • In the single-step case, HANRAG filters noisy documents that Adaptive-RAG passes to the generator, enabling a correct answer where Adaptive-RAG produces an incorrect one.The comparison displays three retrieved documents.Source: Table 7 · p. 14
  • For compound queries, HANRAG retrieves information for both sub-queries simultaneously, whereas Adaptive-RAG handles them sequentially in two retrieval steps.The caption describes HANRAG's approach as a space-for-time trade-off that reduces overall time for LLM generation.Source: Table 8 · p. 15

G Prompt list

  • The prompt list defines question categories, asks the decomposer to extract seed questions for complex queries, and directs the generator to answer concisely from retrieved documents.Source: G Prompt list · pp. 18, 19, 21
Loading 2509.09713v1…