PaperlayerView exact revision on arXiv

Source-linked AI summary

R$^{2}$Adapter: A Routing and Rewriting Adapter for Efficient Hybrid RAG

Yucan Guo, Miao Su, Saiping Guan, Long Bai, Zhongni Hou, Zixuan Li, Xiaolong Jin, Jiafeng Guo, Xueqi Cheng

arXiv:2609.02894v1cs.CLcs.IR

TL;DR

Vanilla RAG is efficient for simple queries, whereas graph-based RAG better supports multi-hop reasoning but adds latency and computational overhead; existing hybrid routing also relies on brittle heuristics or costly LLMs. R2ADAPTER uses a model-agnostic lightweight router and selective query rewriting to allocate queries between the two paradigms, reducing graph retrieval usage while maintaining comparable answer accuracy.

  • Problem

    Vanilla RAG struggles with relational and multi-hop queries, while graph-based RAG adds inference cost; existing hybrid methods use brittle heuristics or costly LLM-based routing.

  • Method

    R2ADAPTER is a plug-in adapter that routes queries with a lightweight classifier and selectively rewrites uncertain graph-routed queries into reasoning-oriented representations.

  • Results

    R2ADAPTER reduces graph-based RAG usage while maintaining comparable QA performance across multi-hop benchmarks and representative graph-based RAG systems.

  • Takeaways & Limitations

    R2ADAPTER provides an efficient, adaptive, and model-agnostic approach for integrating vanilla and graph-based RAG without modifying the underlying systems.

  • Takeaways & Limitations

    Performance depends on the plugged-in retrievers and LLMs, and replacing the dense retriever may require router retraining for optimal alignment.

Abstract

from arXiv · show

Retrieval-Augmented Generation (RAG) has become a prevailing paradigm for enhancing Large Language Models (LLMs) with non-parametric knowledge. Vanilla RAG efficiently handles simple queries but struggles with relational or multi-hop reasoning. Graph-based RAG alleviates this issue but incurs higher inference complexity and latency. In practice, user queries can differ significantly in their complexity, rendering a fixed RAG strategy suboptimal. However, existing hybrid text-graph RAG methods typically rely on heuristic and LLM-based routing, resulting in unnecessary overhead and strong dependence on the underlying LLM. To address these challenges, we propose R$^{2}$Adapter, a lightweight plug-in Routing and Rewriting Adapter designed to allocate queries between vanilla and graph-based RAG dynamically. By routing only the queries that genuinely benefit from graph-based reasoning, R$^{2}$Adapter reduces unnecessary graph retrieval overhead. Additionally, uncertain graph-routed queries are rewritten to better expose their multi-hop reasoning requirements, improving retrieval quality without additional supervision. Extensive experiments on three multi-hop QA benchmarks demonstrate that R$^{2}$Adapter reduces graph-based RAG usage by up to 59% while maintaining comparable answer accuracy. This adapter is model-agnostic and can be seamlessly integrated into diverse vanilla and graph-based RAG pipelines, providing an efficient and adaptive solution for hybrid RAG systems.

1 Introduction

RAG combines external knowledge with LLMs, but vanilla and graph-based approaches trade efficiency against multi-hop reasoning capability. R2ADAPTER dynamically routes queries and selectively rewrites uncertain graph-routed queries to reduce overhead while preserving answer accuracy.

  • Vanilla RAG efficiently handles single-hop or surface-level queries but struggles with multi-hop inference and compositional evidence aggregation.
  • Graph-based RAG supports complex relational reasoning by modeling entities and relations, but graph construction and retrieval increase latency and computational overhead.
  • Existing hybrid methods use brittle heuristics or costly LLM-based routing, limiting generalization, efficiency, or independence from the underlying LLM.
  • R2ADAPTER uses a lightweight router to select between vanilla and graph-based RAG without invoking an LLM, plus a rewriter for uncertain graph-routed queries.
  • R2ADAPTER is model-agnostic and requires no modification to the underlying RAG systems, supporting practical integration across hybrid deployments.
  • Extensive multi-hop QA experiments show reduced graph-based RAG usage while preserving comparable answer accuracy across representative graph-based RAG systems.

2 Related Work

Related work contrasts unstructured and graph-based retrieval, then motivates hybrid RAG because query complexity varies and no single retrieval strategy is uniformly optimal.

  • Graph-based RAG augments passage retrieval with structured graphs to support explicit reasoning over entities and relations.
  • Graph-based RAG methods incur higher computational cost and are unnecessary for many simple queries that vanilla RAG can resolve.
  • Hybrid RAG dynamically selects retrieval strategies because user queries vary in complexity and no single strategy is uniformly optimal.

3 R2ADAPTER

R2ADAPTER is a plug-in adapter that routes queries between vanilla and graph-based RAG, constructs router supervision automatically, and selectively rewrites uncertain graph-routed queries.

  • 3.1 Overall Framework: R2ADAPTER combines a lightweight router with a rewriter as a pre-retrieval adapter for hybrid RAG.The router selects the retrieval paradigm, while the rewriter refines uncertain queries sent to graph-based RAG.
  • 3.1 Overall Framework: Queries predicted suitable for vanilla RAG bypass graph traversal, while uncertain graph-routed queries are reformulated before graph retrieval.Rewriting exposes latent relational structure by converting the query into relational triplets with intermediate placeholders and a target marker.
  • 3.2.1 Training Corpus Construction: The router training corpus compares passage-based vanilla RAG with HippoRAG 2 using relevant-document average rank to assign suitability labels.The lower average rank determines suitability; tied ranks mark both paradigms suitable, and vanilla is preferred when both qualify because of lower cost and latency.
  • 3.2.2 Training and Routing Process: The router is a dual-output DeBERTa-v3-base classifier that independently predicts vanilla and graph-based RAG suitability probabilities.Independent focal BCE losses address class imbalance and hard examples while allowing overlapping applicability between the two paradigms.
  • 3.2.2 Training and Routing Process: Threshold selection balances routing F1 and graph-based recall because misrouting graph-required queries to vanilla RAG is especially harmful.The best threshold is selected on validation data rather than fixed in advance.
  • 3.3 The Rewriter: The LLM-based rewriter activates only when graph-based routing is uncertain, limiting rewriting to cases below a predefined confidence threshold.This conservative policy reduces unnecessary LLM calls for queries that can be handled without rewriting.

4 Experiments

Experiments evaluate R2ADAPTER on three multi-hop QA benchmarks against vanilla RAG, graph-based RAG, and alternative routers. Results show reduced graph usage with comparable or improved QA and retrieval performance, generalization across systems, and gains from query rewriting.

  • Experimental Setup: Experiments use HotpotQA, 2WikiMultihopQA, and MuSiQue, comparing vanilla retrievers, graph-based RAG systems, and hybrid routing baselines.The router uses DeBERTa-v3-base; graph-based systems use NV-Embed-v2 and Llama-3.3-70B-Instruct under a unified setup.
  • Main Results: R2ADAPTER reduces graph-based RAG usage while maintaining comparable QA performance across integrated systems.With HippoRAG 2, it achieves the best average QA performance; with GraphRAG, it improves over GraphRAG while routing approximately 45% of queries to vanilla RAG on average.
  • Main Results: R2ADAPTER generalizes across HippoRAG 2, GraphRAG, and unseen QA benchmarks despite router supervision from HotpotQA and HippoRAG 2.The two graph-based systems use substantially different graph construction and reasoning paradigms.
  • Main Results: R2ADAPTER with HippoRAG 2 slightly improves Recall@2 and yields comparable Recall@5 on average.The results indicate a balance between retrieval cost and retrieval effectiveness through selective graph retrieval.
  • Router Analysis: R2ADAPTER-Router provides better or comparable QA performance to NER-Router and LLM-Router, whose behavior is unstable across datasets.At similar average graph-routing ratios, R2ADAPTER-Router achieves markedly stronger answer accuracy and reduces graph usage by up to 59% on HotpotQA.
  • Rewriter Analysis: Query rewriting consistently improves answer accuracy on graph-routed queries, with increasing threshold values producing steady EM and F1 gains across datasets.Rewriting is applied only after routing and is intended to expose latent relational structure while avoiding unnecessary perturbations to well-formed queries.

5 Conclusions

R2ADAPTER is a lightweight, model-agnostic adapter that dynamically assigns queries to vanilla or graph-based RAG and selectively rewrites uncertain graph-routed queries. Experiments show reduced graph retrieval usage with comparable answer accuracy across graph-based RAG methods.

  • 5 Conclusions: R2ADAPTER dynamically allocates queries between vanilla and graph-based RAG according to their reasoning requirements.It routes only queries likely to benefit from graph-based reasoning and selectively rewrites uncertain graph-routed queries.
  • 5 Conclusions: R2ADAPTER reduces graph retrieval usage while maintaining comparable answer accuracy across mainstream graph-based RAG methods.The adapter avoids costly LLM-based query classification and is presented as a practical solution for scalable hybrid text-graph RAG.

Limitations

R2ADAPTER improves efficiency while remaining dependent on the component RAG systems and retriever used underneath. Its router may require retraining after retriever changes, and its static rewriting threshold may not adapt fully to query diversity.

  • Pipeline performance is bounded by the vanilla and graph-based RAG systems into which R2ADAPTER is integrated.Retriever and underlying LLM limitations or biases can propagate to retrieval coverage and answer accuracy.
  • Replacing the dense retriever may require router retraining to match the new retrieval distribution.The existing router can still provide reasonable routing after replacement, but optimal alignment may be lost.
  • The rewriter uses a static threshold rather than adaptive learning to select queries for rewriting.The authors identify more sophisticated learned rewriting strategies as future work.

A.1 Evaluation Datasets

The evaluation setup covers retrievers, vanilla and graph-based RAG systems, routing baselines, and router-corpus statistics. These components define comparisons across sparse and dense retrieval, graph variants, and heuristic or LLM-based routing.

  • Retrievers: The evaluated vanilla RAG systems use BM25, Contriever, and NV-Embed-v2 as retrievers.BM25 is sparse, Contriever is contrastively trained and dense, and NV-Embed-v2 is a strong dense retriever based on an LLM embedding model.
  • RAG systems: GraphRAG, LightRAG, RAPTOR, HippoRAG, and HippoRAG 2 represent the evaluated graph- or structure-enhanced RAG systems.Their structures include community graphs, dual-level graphs, recursive trees, associative-memory graphs, and passage-node graphs.
  • Routing baselines: NER-Router sends queries with fewer than two named entities to vanilla RAG and otherwise to graph-based RAG.This baseline implements a lightweight entity-count heuristic inspired by HybGRAG.
  • Routing baselines: LLM-Router classifies queries as fact-based or reasoning-based, routing them to vanilla or graph-based RAG respectively.The classifier uses a fixed prompt template and is more computationally expensive than the rule-based baseline.
  • Evaluation statistics: The evaluation materials include router-corpus statistics by question type, routing preference, and entity-type distribution.Table 8 provides corpus statistics, while Tables 9–11 organize question types, preferences, and named-entity patterns.

B.1 Router Training Corpus

The router corpus identifies when graph-based RAG is preferred by comparing retrieval outcomes across query types and balancing reasoning needs against efficiency. The analysis finds bridge queries favor graph traversal, while comparison queries and entity counts do not reliably determine routing.

  • Corpus construction: Router training uses automatically labeled data partitioned into training, validation, and testing sets in an 8:1:1 ratio.The corpus is analyzed to identify scenarios where graph-based RAG has a decisive advantage over vanilla RAG.
  • Routing preferences: Comparison-type queries are more efficiently handled by passage-level synthesis when both RAG paradigms achieve identical retrieval performance.The representative comparison example gives both systems an average rank of 1.5 and prefers passage-based vanilla RAG.
  • Routing preferences: Bridge-type queries favor graph-based RAG when explicit multi-hop path traversal is required.A representative bridge query has average ranks of 2.5 for vanilla RAG and 1.5 for graph-based RAG.
  • NER feature analysis: Entity count is not a reliable routing proxy: vanilla-preferred queries average 2.30 entities versus 1.97 for graph-preferred queries.The analysis concludes that reasoning-chain complexity matters more than surface-level entity density.
  • NER feature analysis: Graph-preferred queries contain more DATE entities, at 16.0% versus 11.2% for vanilla-preferred queries.This pattern is associated with temporal and chronologically constrained reasoning, for which graph-based RAG may have greater potential.

B.2 Router Training Details

The router is fine-tuned from DeBERTa-v3-base on an automatically constructed corpus, with focal-loss training and validation-based threshold optimization. The setup addresses imbalanced preferences and difficult boundary samples while using an LLM-based rewriter.

  • Router optimization: R2ADAPTER initializes the router with a pre-trained DeBERTa-v3-base backbone and fine-tunes it on the constructed training corpus.Algorithm 2 specifies the training and threshold-optimization procedure.
  • Training challenges: Vanilla preferences dominate the corpus, and boundary samples are difficult because acceptable answers can still favor vanilla RAG for efficiency.This creates an efficiency-driven preference-learning challenge for the router.
  • Router optimization: The router predicts suitability scores for vanilla and graph-based RAG using an independent focal loss.Focal loss addresses class imbalance and emphasizes hard examples by down-weighting easy negatives.
  • Threshold optimization: A validation-set threshold search is performed periodically to account for asymmetric misrouting risks.The threshold is optimized on validation predictions rather than fixed in advance.
  • Implementation: The reported implementation uses AdamW for one epoch with a linear scheduler and 10% warmup, while rewriting uses an LLM prompt template.Experiments use Llama-3.3-70B-Instruct for downstream reasoning and NV-Embed-v2 as the dense retriever.

C.1 Efficiency Analysis

R2ADAPTER improves hybrid RAG efficiency by routing queries between vanilla and graph-based retrieval while limiting the latency and overhead associated with graph reasoning. Its router achieves strong routing performance with lower overhead than an LLM-based router, and rewriting adds only moderate latency.

  • End-to-End Latency: Routing reduces latency relative to always applying graph-based RAG when integrated with GraphRAG and HippoRAG 2.The router consistently lowers end-to-end latency, while vanilla-routed queries retain substantially lower latency.
  • End-to-End Latency: Rewriting adds moderate latency because it uses an LLM rewriter, but vanilla-routed queries offset part of this cost.The full adapter therefore balances effectiveness and efficiency with only a modest latency increase.
  • End-to-End Latency: Figure 6 compares per-query end-to-end latency for vanilla-routed, router-only graph-routed, and adapter-based graph-routed queries.The comparison covers routing configurations across datasets.
  • Routing Overhead: Figure 7 compares routing performance with HippoRAG 2, using bubble size to represent routing time per query.The figure jointly presents routing quality and routing overhead.
  • Routing Overhead: R2ADAPTER-Router achieves the highest F1 scores across all three datasets and reaches the routing performance upper bound.It outperforms the LLM-Router in accuracy while being approximately 4.6× faster.
  • Routing Overhead: R2ADAPTER improves average F1 by 3.9% over the faster NER-Router while adding only a marginal routing-time increase.NER-Router takes 7ms, whereas R2ADAPTER takes 16ms and provides more stable identification of complex queries.

C.2 Impact of Query Rewriting on Retrieval

Query rewriting produces only marginal retrieval-recall improvements across thresholds, but its main benefit is exposing implicit multi-hop structure for graph-based reasoning. The rewriter decomposes queries into relational triplets without adding external facts, improving structural alignment with the knowledge graph.

  • Threshold Effects: Recall@2 and Recall@5 increase consistently across rewriting thresholds, but gains remain below 1.5% in most settings.The threshold analysis is conducted across datasets with HippoRAG 2.
  • Rewriting Mechanism: The rewriter decomposes queries into explicit relational triplets without introducing external facts.This transforms implicit dependencies into the reasoning-oriented representation qr.
  • Rewriting Mechanism: The primary rewriting benefit is structural alignment with the knowledge graph rather than substantially expanding the scope of retrieved entities.The rewritten query provides a clearer logical roadmap for graph-based reasoning.
Loading 2609.02894v1…