PaperlayerView exact revision on arXiv

Source-linked AI summary

Retrieval as Generation: A Unified Framework with Self-Triggered Information Planning

Bo Li, Mingda Wang, Gexiang Fang, Shikun Zhang, Wei Ye

arXiv:2604.11407v2cs.CLcs.AI

TL;DR

Existing RAG systems often rely on one-shot retrieval or external control despite information needs emerging during reasoning. GRIP embeds retrieval planning into token-level generation, and across five QA benchmarks it consistently outperforms existing RAG methods while achieving performance comparable to GPT-4o. Its main scope boundary is sensitivity to fixed retrieval-interface choices such as budget and evidence packing.

  • Problem

    Existing RAG systems commonly retrieve once from the initial query and generate from fixed context, which can fail when information needs emerge gradually or queries contain ambiguity and hidden dependencies.

  • Method

    GRIP uses explicit control tokens and self-triggered information planning to decide when to retrieve, reformulate queries, and terminate within one autoregressive trajectory.

  • Results

    Across five QA benchmarks, GRIP consistently outperforms existing RAG methods and achieves performance comparable to GPT-4o.

  • Takeaways & Limitations

    GRIP provides a unified framework for controllable and efficient retrieval-augmented generation without relying on external retrieval-control modules.

  • Takeaways & Limitations

    GRIP remains sensitive to retrieval-interface choices, including the maximum retrieval budget and evidence-packing strategy, which are fixed in this work.

Abstract

from arXiv · show

We revisit retrieval-augmented generation (RAG) by embedding retrieval control directly into generation. Instead of treating retrieval as an external intervention, we express retrieval decisions within token-level decoding, enabling end-to-end coordination without additional controllers or classifiers. Under the paradigm of Retrieval as Generation, we propose \textbf{GRIP} (\textbf{G}eneration-guided \textbf{R}etrieval with \textbf{I}nformation \textbf{P}lanning), a unified framework in which the model regulates retrieval behavior through control-token emission. Central to GRIP is \textit{Self-Triggered Information Planning}, which allows the model to decide when to retrieve, how to reformulate queries, and when to terminate, all within a single autoregressive trajectory. This design tightly couples retrieval and reasoning and supports dynamic multi-step inference with on-the-fly evidence integration. To supervise these behaviors, we construct a structured training set covering answerable, partially answerable, and multi-hop queries, each aligned with specific token patterns. Experiments on five QA benchmarks show that GRIP surpasses strong RAG baselines and is competitive with GPT-4o while using substantially fewer parameters.

1 Introduction

GRIP addresses the limits of one-shot, externally controlled retrieval by embedding retrieval decisions into token-level generation. Its self-triggered planning coordinates retrieval timing, query reformulation, and termination, with experiments showing strong performance and adaptive behavior across five QA benchmarks.

  • One-shot RAG can fail when information needs emerge during reasoning or queries contain ambiguity and hidden dependencies.
  • GRIP embeds retrieval behavior in the generative policy instead of relying on external modules for discrete retrieval decisions.
  • Self-triggered information planning lets GRIP decide when to retrieve, formulate follow-up queries, and stop within one autoregressive trajectory.
  • Structured supervision covers four answer-ability types and teaches the model to judge information sufficiency, trigger retrieval, and terminate when the question is resolved.
  • GRIP is evaluated on five QA benchmarks, with consistent gains over strong baselines and learned task-aware retrieval depth and stopping.

2 Generation-guided Retrieval with Information Planning

GRIP turns retrieval control into a small set of autoregressive control tokens and trains structured behaviors for direct answering, uncertainty, multi-hop search, and evidence synthesis. Reinforcement fine-tuning further reduces unnecessary retrieval while preserving adaptive multi-step planning.

  • 2.1 Token-Level Control for Unified Retrieval and Generation: GRIP adds [RETRIEVE], [INTERMEDIARY], [ANSWER], and [SOLVED] to control retrieval, partial reasoning, answer generation, and termination.
  • 2.2 Self-Triggered Information Planning: The [INTERMEDIARY]→[RETRIEVE] and [ANSWER]→[SOLVED] patterns support recursive evidence acquisition and completion within autoregressive decoding.
  • 2.2 Self-Triggered Information Planning: GRIP initially assesses whether internal knowledge suffices, then either answers directly or emits an intermediate state and retrieval query.
  • 2.2 Self-Triggered Information Planning: Follow-up retrieval queries condition on the original question and intermediate response to target unresolved aspects of the task.
  • 2.3 Learning Token-Controlled Retrieval Behavior via Structured Supervision: Four structured supervision types teach direct answering, uncertainty-triggered retrieval, multi-hop planning, and selective synthesis of noisy evidence.
  • 2.4 Training Objective and Optimization: Reinforcement fine-tuning addresses over-triggered retrieval, while GRIP’s token-level planning avoids external controllers and long-horizon search-policy optimization.

3 Experimental Setup and Main Results

GRIP is evaluated on five open-domain and multi-hop QA benchmarks using standardized metrics and matched retrieval settings. It achieves the strongest overall performance among open-source systems, reaches an Avg.Score of 41.0 close to GPT-4o, and shows particularly strong gains on multi-hop tasks.

  • 3.1 Datasets and Metrics: The evaluation covers HotpotQA, PopQA, Natural Questions, WebQuestions, and TriviaQA across factual recall, contextual understanding, and multi-step reasoning.
  • 3.1 Datasets and Metrics: The study reports EM, ROUGE, F1, and Avg.Score, with Avg.Score defined as the unweighted mean across datasets and metrics.
  • 3.2 Experimental Details: GRIP is trained with 40,000 structured samples for supervised fine-tuning and 5,000 additional samples for reinforcement learning.
  • 3.4 Main Results: GRIP consistently achieves the best overall performance among open-source systems across all five benchmarks and surpasses R1-Searcher in Avg.Score.
  • 3.4 Main Results: 41.0 Avg.Score places GRIP close to GPT-4o despite using the smaller LLaMA3-8B backbone.
  • 3.4 Main Results: Gains are most pronounced on HotpotQA and TriviaQA, while higher ROUGE and F1 indicate better answer completeness and alignment.

4 Analysis

GRIP adapts retrieval depth and query formulation to the information available during reasoning, reducing redundant retrieval while extending search when evidence is insufficient. Its generated queries improve answer coverage, and its performance remains strong under controllable budgets, open-source teacher supervision, and non-RAG evaluations.

  • 4.1 Retrieval-Depth Adaptivity: GRIP answers some examples directly despite baselines retrieving, achieving higher accuracy with fewer retrieval calls and avoiding redundant retrieval.This subset covers cases where DRAGIN or GainRAG retrieve at least once while GRIP answers without retrieval.
  • 4.1 Retrieval-Depth Adaptivity: GRIP improves accuracy on harder examples by performing two retrievals when baselines retrieve at most once, indicating evidence-sensitive depth control.The results support learning context-sensitive retrieval depth from structured token supervision.
  • 4.2 Adaptive Retrieval Across Tasks: 1.44 and 1.58 mean retrieval calls occur on HotpotQA and PopQA, versus 0.76 on NQ, showing task-aware retrieval frequency.GRIP retrieves more on tasks requiring multi-hop reasoning or long-tail evidence and less where parametric knowledge often suffices.
  • 4.2 Adaptive Retrieval Across Tasks: The overall average retrieval count drops from 1.60 to 1.24, a ∼22.5% reduction, after rule-based RL while preserving task-aware retrieval patterns.The refinement consistently reduces redundant calls across datasets.
  • 4.3 Improving Retrieval Quality by Generating New Queries: GRIP-generated queries significantly improve top-1 and top-3 answer coverage on NQ and WebQ, with larger gains at top-1 indicating improved evidence ranking.The improvement is attributed to iterative reasoning that uses intermediate answers and retrieved content to target unresolved aspects of the question.
  • 4.4 Controllable Retrieval Budget and Depth Extrapolation: As B increases from 3 to 10, Avg.Score rises from 41.0 to 41.8 while Avg.Count grows from 1.24 to 1.62, remaining below the budget cap.GRIP benefits from larger test-time budgets beyond its maximum three-step training trajectories, with diminishing returns and a cost-accuracy trade-off.
  • 4.5 Replacing Proprietary Teachers with Open-Source Supervision: Qwen3-32B teacher replacement yields comparable final performance, 40.3 vs. 41.0, while LLaMA3-8B-Instruct causes a larger drop.The authors attribute the difference to trajectory-instantiation quality rather than the existence of the learned retrieval policy.
  • 4.6 General-Purpose Capability Preservation: GRIP drops only 0.83 points on MMLU and 0.8 points on MBPP relative to the base INSTRUCT model, while achieving a 51.0% win rate versus 11.5% loss rate on CNN/DailyMail.These non-RAG evaluations indicate minimal degradation in general-purpose capabilities relative to the base model.

5 Related Work

Related RAG systems commonly use static one-shot retrieval or heuristic inference-time triggers, while newer methods learn retrieval behavior through supervision and multi-step search routines. These approaches motivate GRIP’s unified token-level retrieval planning.

  • Static and Heuristic Retrieval: Standard RAG pipelines usually retrieve documents once from the initial query, while heuristic methods estimate retrieval necessity using uncertainty signals.Examples include entropy-based filtering and logit-margin thresholds.
  • Learned and Multi-Step Retrieval: Recent methods make retrieval learnable through reflection or contrastive supervision, while deep-search and planning systems support iterative multi-step retrieval for complex queries.These lines of work provide learned control and search-and-reason routines related to GRIP’s objective.

6 Conclusion

GRIP integrates retrieval behavior into generation through self-triggered information planning, enabling dynamic retrieval control without external modules. Across five QA benchmarks, it consistently outperforms existing RAG methods and performs comparably to GPT-4o.

  • Framework: GRIP is a unified retrieval-augmented generation framework that integrates retrieval behavior into generation through self-triggered information planning.The design enables dynamic control over retrieval without relying on external modules.
  • Results: Across five QA benchmarks, GRIP consistently outperforms existing RAG methods and achieves performance comparable to GPT-4o.The authors position the framework as a foundation for controllable and efficient generation.

Limitations

GRIP’s robustness is bounded by fixed retrieval-interface choices, while its evaluation spans diverse QA settings and metrics. The paper evaluates multi-hop, open-domain, reading-comprehension, and biomedical tasks using exact-match, token-overlap, and ROUGE-based measures.

  • Scope Boundary: GRIP remains sensitive to the maximum retrieval budget and evidence-packing choices such as chunking and top-k.The paper keeps these interface choices fixed for fair comparison and identifies adaptive budgeting and context structuring as future directions.
  • Evaluation Scope: The evaluation suite covers multi-hop, open-domain, reading-comprehension, and biomedical QA tasks with varying dataset sizes and reasoning demands.The main suite contains five QA benchmarks, with BioASQ additionally reported as a domain-specific benchmark.
  • Evaluation Metrics: QA performance is assessed using Exact Match, token-level F1, and ROUGE, with scores averaged over evaluation instances.Exact Match requires a normalized exact string match, F1 measures token overlap, and ROUGE averages ROUGE-1, ROUGE-2, and ROUGE-L.

B Performance On The Domain-specific Dataset

On BioASQ, a domain-specific biomedical QA benchmark, GRIP achieves the strongest reported overall performance. It records the highest ROUGE, F1, and Avg.Score in the table.

  • Results: 69.6 Avg.Score is GRIP’s best overall BioASQ performance, exceeding the reported ROUGE and F1 averages of the comparison systems.Avg.Score is the unweighted mean of ROUGE and F1 for this benchmark.
  • Results: 54.8 ROUGE is GRIP’s highest reported BioASQ ROUGE score.The result is reported alongside GRIP’s best overall BioASQ performance.
  • Results: 84.4 F1 is GRIP’s highest reported BioASQ F1 score.The score is reported for the domain-specific biomedical QA benchmark.

C Cover Match Analysis for Answer Stability

GRIP’s CoverEM analysis evaluates whether outputs contain the normalized reference answer despite surface-form variation. Across five QA datasets, GRIP significantly outperforms the baselines, including GPT-4o and GainRAG.

  • CoverEM Definition: CoverEM equals 1 when a normalized prediction contains a normalized reference answer as a contiguous substring, making it a coverage-based semantic-correctness measure.The final CoverEM score is averaged across instances.
  • Results: GRIP significantly outperforms all baselines, including GPT-4o and GainRAG, on CoverEM across five QA datasets.The table caption identifies GRIP as achieving the best average CoverEM.
  • Interpretation: Consistent CoverEM gains indicate that GRIP produces correct-content spans despite surface-form differences.The paper interprets the cross-dataset gains as evidence of robustness to surface-level variation.
  • Training Design: GRIP’s training data includes directly answerable, partially answerable, and multi-hop cases mapped to distinct control-token behaviors.The structured examples include [ANSWER], [SOLVED], [INTERMEDIARY], and [RETRIEVE] patterns, alongside experiments with alternative token implementations.

F Ablation on the Reward Metric

The reward-metric ablation finds that changing the answer-fidelity overlap metric has little effect on overall performance or retrieval frequency. BLEU is retained as the default because it provides a smoother graded signal and the best Avg.Score under essentially the same retrieval budget.

  • Ablation Results: Within 0.5 Avg.Score, BLEU, ROUGE, token-F1, and EM produce highly consistent RL performance.The RL pipeline remains unchanged while only the overlap metric in the answer-fidelity reward is replaced.
  • Retrieval Behavior: Average retrieval count is nearly unchanged across reward metrics, indicating that the control reward dominates the retrieval policy.The answer-fidelity metric primarily shapes answer quality under the reported ablation.
  • Default Metric: BLEU is used by default because it offers a smoother graded signal than strict EM while achieving the best Avg.Score under essentially the same retrieval budget.The comparison concerns answer-fidelity reward choices, not changes to the retrieval-control setup.

G Behavioral Shift after Rule-based RL

Rule-based reinforcement learning shifts GRIP toward earlier stopping and fewer unnecessary retrieval steps, while preserving zero-retrieval behavior on easier questions. Qualitative and latent-space analyses further show adaptive query planning, fallback reasoning, and separable retrieval-control states.

  • Behavioral shift after RL: 59.6% to 32.5%: WebQ Retrieve=2 cases fall after RL, while Retrieve=1 rises from 6.6% to 37.5%.The shift indicates earlier stopping once sufficient context is available.
  • Behavioral shift after RL: 60.9%: GRIP’s NQ Retrieve=0 rate after RL remains high, increasing from 57.8% and preserving efficient behavior on easier questions.RL mainly converts Retrieve=2 cases into Retrieve=1 rather than increasing Retrieve=0.
  • Fallback and control: GRIP emits [INTERMEDIARY] and partial internal-knowledge reasoning before [RETRIEVE] when initial evidence is unavailable, delaying final commitment.This fallback pattern supports cautious progress instead of confident unsupported answering.
  • Iterative retrieval planning: Iterative query reformulation lets GRIP recover from an uninformative first retrieval and reach [ANSWER] and [SOLVED] after a more specific third-round query.The case illustrates retrieval planning based on evolving context and accumulated evidence.
  • Failure analysis: GRIP sometimes retrieves relevant evidence but still produces incorrect answers when implicit or multifaceted information requires deeper inference.Other failures involve redundant retrievals that dilute answer quality when sufficient information is already available.
  • Internal decision dynamics: GRIP’s hidden states form distinct clusters before [ANSWER] and [RETRIEVE], indicating separable internal states for the two control decisions.The visualization supports token-level behavioral modulation rather than a single undifferentiated decoding state.

L Retriever Ablation

Retriever ablations and backbone checks indicate that GRIP’s gains are not tied to one retriever or backbone. Its token-level planning selectively controls retrieval depth, extrapolates beyond training trajectories, and maintains efficiency across settings.

  • Retriever ablation: GRIP remains substantially stronger than Single-RAG and RobustRAG across BM25, DPR, and hybrid retrieval choices.The comparison keeps the LLaMA3-8B backbone, Wikipedia corpus, and k=3 passages fixed.
  • Retriever ablation: Hybrid retrieval improves both Single-RAG and RobustRAG, while GRIP’s hybrid result matches its best single-retriever result.For GRIP, DPR alone is slightly worse than BM25.
  • Retrieval budget: Most examples use r≤2 retrieval calls even as the maximum budget B increases, showing selective triggering rather than a fixed retrieval schedule.The realized retrieval-count distribution remains concentrated at small r across datasets.
  • Retrieval budget: At B=10, retrieval beyond the three-step training depth occurs in 17.0% of WebQ, 11.7% of NQ, 7.8% of PopQA, 7.3% of HotpotQA, and 3.4% of TriviaQA examples.This demonstrates extrapolative generalization beyond the maximum depth used in training trajectories.
  • Retrieval budget: Task difficulty shapes extrapolation: evidence-uncertain datasets have heavier r>3 tails, whereas factoid benchmarks concentrate at r=1 or r=2.Budget control provides a practical trade-off between retrieval cost and accuracy while preserving adaptive behavior.
  • Backbone robustness: On Qwen2.5-7B, GRIP again outperforms training-free and training-based RAG baselines across datasets while maintaining retrieval efficiency.The stable ranking suggests gains from retrieval-as-generation and scenario-typed supervision rather than backbone-specific effects.
Loading 2604.11407v2…