Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicing

• Multi-round RAG systems can stop with insufficient evidence or retrieve distracting excess, making information sufficiency and self-awareness central challenges.The paper characterizes these errors as Over-Confidence and Over-Retrieval.Source: §1 · pp. 1–2
• Human-labeled process supervision is costly, and LLM-independent information-seeking chains may not match a system’s behavior or knowledge.Source: §1 · p. 2
• SIM-RAG addresses the data shortage with Self-Practicing followed by Critic Training on synthetic process-supervision data.The generated inner monologues capture the system’s reasoning trajectory across components.Source: §1 · p. 2
• Self-Practicing generates system-capability-aligned data, while the Critic repeatedly checks knowledge sufficiency and guides behavior through in-context reinforcement learning.Source: §1 · p. 2
• SIM-RAG uses information sufficiency to guide dynamic, multi-round retrieval and reasoning for complex tasks.Source: §1 · p. 2
• Experiments on three standard benchmarks report SIM-RAG as a lightweight, effective solution across diverse question-answering scenarios.Source: §1 · p. 3

2.1 Retrieval Augmented Generation

• Single-round RAG serves straightforward tasks, but complex questions may require synthesizing documents, resolving ambiguity, or filling retrieval gaps through iteration.Source: §2.1 · p. 3

2.2 Multi-Round RAG

• Training-free multi-round methods rely on LLM self-critique or internal signals, which can be unreliable or require access to model weights.Latent representations may also lack interpretability, limiting trustworthiness in applications such as healthcare.Source: §2.2 · p. 3
• Learned approaches include query-difficulty classification, Self-RAG, and IM-RAG, but reported trade-offs include substantial data and computational costs or costly annotations and no principled stopping mechanism.Source: §2.2 · p. 3

2.3 LLM Self-Training for Complex Reasoning

• Self-training uses model-generated outputs and reward signals, but RAG adds changing knowledge boundaries, noisy context, and the need to decide both when and what to retrieve.Source: §2.3 · pp. 3–4
• Because LLM self-critique may lack sufficient self-awareness, SIM-RAG uses an unattached lightweight Critic trained specifically to check information sufficiency.Source: §2.3 · p. 4

3.1 SIM-RAG

• SIM-RAG cycles an LLM Reasoner, Retriever, and lightweight Critic: supported answers return, while unsupported ones trigger retrieval rounds until grounding or the iteration cap.The Reasoner proposes an answer and rationale using the question and accumulated context.Source: §3.1 · pp. 4–5

3.2 Self-Practicing

• Self-Practicing uses known question-answer pairs to generate model-specific, context-aware multi-round traces, labeling answers Accept or Reject and producing T examples per pair.The labels capture answer correctness and information sufficiency; varying available information can produce different behaviors and labels from the same pair.Source: §3.2 · pp. 5–6

3.3 Reasoning-Enhanced Critic Learning

• The Critic is trained as a binary classifier by fine-tuning a generative language model to predict Accept/Reject labels from instruction-augmented inputs.The training objective maximizes the conditional likelihood of the target labels.Source: §3.3 · p. 6

3.4 Reasoning-Enhanced Inference

• At inference, a separate Critic supplies textual Accept/Reject feedback that the Reasoner incorporates in-context, adapting decisions without parameter updates and avoiding self-critique bias.The feedback is appended to the Reasoner’s input as verbal reinforcement.Source: §3.4 · p. 6

4.1 Task and Datasets

• Evaluation covers TriviaQA, HotpotQA, and 2WikiMultiHopQA, reporting Exact Match and F1 with each dataset’s Wikimedia dump as the retrieval corpus.The tasks span single-hop factoid questions and multi-hop evidence synthesis or fine-grained entity discrimination.Source: §4.1 · p. 6

4.2 Implementation Details

• Experiments use Llama3-8B or GPT-4 as Reasoner, Flan-T5-2.85B or Flan-T5-783M as Critic, and BM25 with Elasticsearch for retrieval.The authors report that the pipeline can be replicated with two NVIDIA 3090 GPUs or equivalent hardware.Source: §4.2 · p. 6

4.3 Baselines

• The comparison includes basic generation and Standard RAG alongside advanced multi-round methods, divided between systems requiring model-weight access and those usable with API-based LLMs.Table 1 identifies the two groups and marks results reproduced with newer models using an asterisk.Source: §4.3 · pp. 6–7, Table 1 · p. 7

4.4 Results

• SIM-RAGfull with GPT-4 consistently performs best on all three RAG datasets, outperforming every baseline by a large margin, including Self-RAG and Auto-RAG, which require extensive full-model fine-tuning.On HotPotQA, SIM-RAGlite scores 27.8% versus Self-RAG’s 17.1% while using one-tenth the training parameters (783M versus 7B).Source: §4.4 · p. 7

4.5 Analysis on Critic Predictions

• The Critic’s accuracy in rejecting incorrect answers is 63.9% on HotpotQA and 65.0% on 2Wiki, while correct-answer acceptance is 60.3% on TriviaQA versus 13.6% on both multi-hop datasets.The paper attributes this difference in part to the synthetic training data containing a higher proportion of positive examples for TriviaQA.Source: §4.5 · pp. 7–8, Figure 3 · p. 8

4.6 Ablation Study and Analysis

• Flan-T5 as Critic outperforms GPT-4 self-critique on both EM and F1 for complex multi-hop tasks, though GPT-4 achieves strong TriviaQA results.Both configurations use GPT-4 as Reasoner; the authors suggest general-purpose LLMs may produce too many false positives as Critic on multi-round QA.Source: Figure 4 · p. 8, §4.6 · p. 8
• On HotpotQA, larger turn limits improve SIM-RAGlite, with a 10.1% final-round gain over Naive Generation and a 47.2% Oracle Critic score.The Oracle Critic uses ground-truth labels and represents the stated upper limit without modifying the Reasoner; experiments cap turns at six because of the Critic’s context length.Source: Figure 5 · p. 8, §4.6 · pp. 8–9
• Self-practicing traces include errors from missing evidence at zero or one retrieval round, a correct two-round prediction, and distraction from over-retrieval at three rounds.The authors use these traces to illustrate why more retrieved information is not always beneficial for LLM reasoning.Source: §4.6 · p. 9

5.1 Strengths and Weaknesses

• SIM-RAG’s flexibility enables consistent outperformance of traditional baselines across tasks ranging from single-round retrieval to multi-round reasoning.Source: §5.1 · p. 9
• SIM-RAG may struggle with domain-specific jargon, and its system-specific Critic may require retraining if the Reasoner or Retriever changes substantially.Source: §5.1 · p. 9
• Data generation requires T × N large-model inferences, while unfamiliar domains may require more inference turns and make domain adaptation important for reducing inference time.Source: §5.1 · p. 9
• False Critic rejections can trigger over-retrieval mistakes, while low accuracy with an Oracle Critic implicates dataset difficulty, Reasoner knowledge and query formulation, and Retriever quality.Source: §5.1 · p. 9

5.2 Critic Learning

• Flan-T5-783M scores 14.7% EM on HotPotQA versus 20.1% for zero-shot Llama3, yet can serve as a Critic by assessing output coherence and validity rather than generating correct answers.Source: §5.2 · p. 9

6 CONCLUSION AND FUTURE WORK

• SIM-RAG optimizes multi-round RAG by adding a lightweight Critic, without modifying LLM parameters, and supports both open-source and closed-source systems.Source: §6 · p. 9
• Its Self-Practicing Algorithm generates synthetic data to address shortages of labeled, system-specific multi-round reasoning and retrieval training data.Source: §6 · p. 9
• Experiments on three standard benchmarks validate SIM-RAG’s effectiveness; the authors suggest future work could use the Critic as a reward model to optimize other RAG components.Source: §6 · p. 9
• The authors expect Self-Practicing and Critic training techniques to extend to other AI problems, while emphasizing that recognizing system limitations matters for reducing hallucination and enhancing trustworthiness and reliability.Source: §6 · p. 9