Source-linked AI summary
OmniRetrieval: Unified Retrieval across Heterogeneous Knowledge Sources
Jinheon Baek, Soyeong Jeong, Sangwoo Park, Woongyeong Yeo, Minki Kang, Patara Trirat, Heejun Lee, Sung Ju Hwang
TL;DR
Retrieval systems often cannot span heterogeneous sources without flattening the schemas, ontologies, and operators that make those sources expressive. OmniRetrieval coordinates source selection, native query generation and execution, and cross-source evidence selection; across 13 datasets and 309 knowledge bases, it consistently outperforms relevant baselines. The framework’s remaining boundaries include evidence-selection improvements, shared-LLM reliance, and the infeasibility of materializing unified representations at benchmark scale.
Problem
Existing retrieval systems are tied to individual source types and fixed query languages, while shared representations erase structural affordances needed for heterogeneous retrieval.
Method
OmniRetrieval selects relevant sources, generates executable native queries conditioned on their structural contexts, executes them, and consolidates the outputs.
Results
OmniRetrieval consistently outperforms relevant single-source baselines across 13 datasets and 309 knowledge bases spanning unstructured, relational, and graph sources.
Takeaways & Limitations
Native-interface coordination provides a general framework for unified retrieval while preserving source-specific structural operators and enabling registration-based source extension.
Takeaways & Limitations
The instantiation relies on a single shared LLM and leaves stronger cross-source evidence selection and operator-specific specialization for future work.
Abstract
from arXiv · showhide
Real-world information needs require access to structurally diverse knowledge sources, from unstructured text and relational tables to knowledge graphs and property graphs. Existing retrievers, however, operate over one source at a time under a fixed query language, leaving the broader landscape of available knowledge fragmented behind incompatible interfaces. A natural attempt at unification would collapse these sources into a shared space, but this erases the structural affordances (such as schemas, ontologies, compositional operators) that give each source its expressive power. Effective retrieval over diverse knowledge, therefore, requires not homogenization but an overarching layer that meets each source on its own terms. To achieve this, we present OmniRetrieval, a framework that takes any natural-language query, identifies appropriate knowledge sources, and dispatches source-native queries to their native execution engines. Across an extensive benchmark spanning 13 datasets and 309 distinct knowledge bases over text, relational, and graph-structured sources, OmniRetrieval exceeds single-source baselines, demonstrating that it can serve as a general-purpose interface to the heterogeneous sources while preserving the structural distinctions that make each source valuable.
1. Introduction
Real-world questions span heterogeneous knowledge sources, but existing retrievers usually access only one source through a fixed interface. OmniRetrieval instead coordinates source selection, native query formulation, and cross-source consolidation while preserving each source’s structural affordances.
- Motivation: Real-world questions may require passages, relational joins, knowledge-graph triples, or property-graph traversals across different sources.The relevant source, query language, and structural operation depend on the question.
- Motivation: Existing retrievers are typically restricted to one source type and its associated query language.Document retrievers rank passages, text-to-SQL systems target one database, and graph generators target one graph backend.
- Motivation: A shared representation can flatten source-specific structure, bias retrieval toward similar source forms, and lose native query operations.The paper contrasts this homogenization strategy with preserving each source’s schemas, ontologies, joins, traversals, and paths.
- OmniRetrieval: OmniRetrieval identifies relevant sources, formulates executable native-language queries using their structural context, executes them, and consolidates multi-source outputs.Its source-native design allows a new knowledge base to be added by registration without retraining a shared encoder or rebuilding an embedding space.
- Evaluation: 309 knowledge bases across 13 datasets are evaluated, with OmniRetrieval consistently exceeding single-source baselines while producing valid native queries.The benchmark covers unstructured corpora, relational databases, RDF knowledge graphs, and labeled property graphs.
2. Method
OmniRetrieval selects candidate sources, generates source-native queries conditioned on structural context, executes them, and filters the resulting heterogeneous outputs into relevant evidence. The method uses broad candidate exploration so final selection can rely on retrieved evidence rather than an irreversible early routing decision.
- Problem Formulation: Each registered source exposes a native query language, execution engine, and structural context such as a schema, ontology, or corpus descriptor.The retrieval task is defined over independently maintained heterogeneous sources.
- Source Selection: Source selection reads the query and heterogeneous source descriptors to return a ranked subset of candidate sources.Descriptors may include schemas, ontologies, and corpus summaries, and the catalog can grow by appending new descriptors.
- Source Selection: Returning multiple candidates supports queries requiring multiple sources or queries whose target source is ambiguous, deferring commitment to evidence selection.The short list is ordered by relevance and contains at most k sources.
- Query Formulation: For each selected source, the framework translates the question into a query valid in that source’s native language and grounded in its structural context.SQL, SPARQL, Cypher, and free-form text correspond to relational, RDF, property-graph, and unstructured sources respectively.
- Evidence Selection: After execution, cross-source selection filters heterogeneous outputs to retain the subset relevant to the question.The selector verbalizes rows, triples, paths, and passages while preserving the structural work already performed by native execution.
3. Experimental Setup
The evaluation spans heterogeneous retrieval settings, datasets, knowledge bases, metrics, and baselines while keeping backbone models and execution engines fixed. OmniRetrieval is compared with single-backend routing, oracle, and feasibility-constrained unified-representation methods.
- Datasets and Knowledge Bases: 13 datasets provide 309 knowledge bases spanning document search, relational databases, RDF knowledge graphs, and labeled property graphs.The benchmark includes seven BEIR datasets, Spider and BIRD, three RDF datasets targeting Wikidata, and 15 Neo4j graphs.
- Datasets and Knowledge Bases: Each evaluation question is paired with structural context, using topical descriptors for document collections and schemas for structured backends.The benchmark samples 300 questions per dataset and executes generated structured queries against the corresponding databases or graph endpoints.
- Methods and Baselines: The comparison holds backbone models and per-backend execution engines fixed so differences reflect how methods engage the source pool.Backbones include GPT-5.4, Gemini-3.1 Pro, Sonnet-4.6, Qwen-3.5, and Gemma-4; document retrieval uses all-MiniLM-L6-v2.
- Methods and Baselines: OmniRetrieval selects multiple candidate sources, formulates and executes native queries for each, and consolidates results through cross-source evidence selection.KB Routing selects one knowledge base per query, while Oracle uses the gold knowledge base as a non-comparable upper bound.
- Methods and Baselines: Unified-representation methods are evaluated only under feasibility constraints because materializing a shared representation is infeasible at the benchmark scale.The paper cites billions of triples, tens of billions of graph paths, and databases with over 70 million rows as scale barriers.
- Evaluation Metrics: Evaluation uses macro-averaged source selection accuracy, retrieval accuracy, and LLM-as-a-Judge scores across the four native retrieval paradigms.Retrieval accuracy uses NDCG@10 for document search and Execution Match for SQL, SPARQL, and Cypher; the judge provides a softer assessment of equivalence.
4. Experimental Results and Analyses
OmniRetrieval consistently outperforms single-source baselines across backbones by exploring multiple candidate sources and selecting evidence across them. Analyses show that candidate breadth, backbone scale, and reliable evidence selection jointly shape performance, while native retrieval retains an advantage over unified representations for structurally compositional queries.
- Main Results: OmniRetrieval consistently outperforms all baselines across the five backbones.Its multi-candidate strategy and cross-source evidence selection improve on single-backend systems and KB Routing.
- Source Candidate Size: Increasing the candidate list size k improves OmniRetrieval monotonically, but the evidence-selection oracle improves faster as k grows.The widening oracle gap indicates that selecting correctly among more candidates remains challenging.
- Backbone Scale: At larger backbone scales, Top 3 separates from Top 1 as candidate diversity increases, while source selection remains the largest gap to the gold-source ceiling.The two settings are essentially tied at 2B; beyond 4B, candidates span more paradigms and sources, and the gap is clear at 27B.
- Cross-Source Evidence Selection: The gold source is included at a high rate across backbones, and evidence selection chooses it at a high rate once included, above random selection.This supports broad upstream exploration followed by deferred final commitment.
- Cross-Paradigm Coverage: Document Search has the widest cross-paradigm coverage, with a 28.2% off-diagonal mean versus 15.2 to 22.1% for the three structured paradigms.Its advantage is largely driven by SPARQL questions, where Wikipedia-derived corpora overlap with Wikidata content.
- Native vs Unified Retrieval: Unified-representation retrieval remains far below KB Routing and OmniRetrieval because atomic-unit retrieval cannot express structural composition such as joins, traversals, and multi-hop chains.The constrained setup gives the unified method access to a reduced pool containing gold-touched structured items plus random distractors.
5. Related Work
Prior work largely retrieves from one backend at a time, while OmniRetrieval extends tool-use-style dispatch to heterogeneous sources through native query languages and cross-source evidence consolidation.
- Classical retrieval methods are organized around a single corpus and representation, including lexical, dense, and modality-specific retrievers.
- Text-to-SQL, text-to-SPARQL, and related systems translate natural-language queries into the native language of a single backend.
- OmniRetrieval differs from generic tool use by synthesizing schema-grounded programs for large backends and consolidating structurally heterogeneous outputs.
6. Conclusion
OmniRetrieval retrieves from heterogeneous knowledge sources through their native query languages and consolidates outputs without collapsing them into a shared representation. Across 13 datasets and 309 knowledge bases, it consistently outperforms relevant baselines while preserving source-specific structural affordances.
- OmniRetrieval engages relevant sources through native query languages and consolidates executor outputs through cross-source evidence selection.
- 13 datasets and 309 knowledge bases span unstructured corpora, relational databases, RDF graphs, and labeled property graphs.
- OmniRetrieval consistently outperforms relevant baselines across this heterogeneous benchmark.
- Broad source exploration followed by evidence-based final selection supports the framework’s reported scaling behavior.
- The framework exposes a single natural-language interface while preserving the structural affordances of each source.
Limitations
The current instantiation performs reliably in cross-source evidence selection, but this component could be strengthened in future work. Suggested directions include supervised fine-tuning and reinforcement learning based on downstream answer quality.
- Cross-source evidence selection performs reliably but remains an area for further strengthening.
- Supervised fine-tuning on labeled cross-source selections is proposed as one improvement direction.
- Reinforcement learning using downstream answer quality as the reward signal is another proposed improvement direction.
Ethical Considerations
The work relies on publicly available datasets and standard LLMs, so risks inherited from connected sources and model knowledge remain relevant to deployment.
- The authors identify no ethical concerns beyond those inherited from the publicly available datasets and standard LLMs used.
- Private, harmful, or biased content in connected knowledge bases or internalized model knowledge may carry over into outputs.
- The authors recommend safeguards and filtering for connected sources and generated queries during deployment.
A. Benchmark Details
The benchmark details document the examples, structural contexts, and document-search corpus used to evaluate heterogeneous retrieval. Document collections are represented with topical and stylistic descriptors, illustrated through NFCorpus.
- Benchmark Details: The benchmark section includes example questions and verbatim structural contexts for each backend.These materials are provided alongside a note on the use of existing artifacts.
- Document Search: Document corpora use a short topical descriptor describing the domain and typical document style.
- Document Search: NFCorpus represents medical and nutritional information retrieval with terse consumer-health queries or lay-audience headlines.
- Document Search: NFCorpus documents are PubMed-style biomedical research abstracts written in a technical register.
A.2. Relational Databases
Structured backends expose source-specific structural contexts rather than a single common representation. Relational databases provide schemas, while Wikidata and property graphs expose graph-specific entities, predicates, labels, properties, and relationships.
- Relational Databases: Relational database contexts expose CREATE TABLE declarations with column types, primary keys, and foreign keys.
- Relational Databases: The concert_singer example includes singer attributes such as ID, name, country, song, release year, age, and gender.
- Knowledge Graphs: Wikidata contexts combine SPARQL prefixes and templates with question-linked topic entities and candidate predicates.Candidate predicates are ranked by semantic similarity between predicate labels and the question.
- Knowledge Graphs: Wikidata query examples cover single-hop patterns, two-hop patterns, and ASK queries over entities and properties.
- Property Graphs: Text2Cypher contexts expose node labels, typed property keys, relationship types, and graph edges as triples.The movie-database example includes properties for movies and people and relationships such as ACTED_IN and DIRECTED.
- Benchmark Assumptions: The benchmark uses existing public resources under their respective licenses for research, without additional filtering for personally identifying or offensive content.
B. Implementation Details
Implementation uses deterministic LLM calls, native retrieval paradigms, source-specific prompts, and cross-source evidence selection. Evaluation decomposes performance by source-selection accuracy, retrieval accuracy, and answer-judgment accuracy.
- Implementation Settings: Temperature 0.0 makes source selection, query formulation, and evidence-selection predictions deterministic, with one run per configuration.Generation is capped at 1024 tokens, and open-source backbones run locally on one NVIDIA H200 GPU.
- Evaluation: Table 5 decomposes results across Search, SQL, SPARQL, and Cypher for five backbones, with macro averages recovering Table 1.
- Source Selection: Under balanced weighting, top-1 SQL predictions range from 30% to 37% and Search predictions from 27% to 37% across backbones.The balanced reference is 25% per paradigm, while relational sources comprise approximately 93% of the catalog.
- Source Selection: Source selection returns up to k ranked candidate knowledge bases from a catalog grouped by backend.
- Query Formulation: Query formulation uses native prompts for three structured query languages and a shared template grounded in backend context.Document retrieval additionally rewrites the question into a hypothetical passage.
- Evidence and Evaluation: Cross-source evidence selection chooses among source-specific retrieval results, while LLM-as-a-Judge assesses direct-match or faithful-realization answers.