MoR: Better Handling Diverse Queries with a Mixture of Sparse, Dense, and Human Retrievers

• MoR dynamically selects and combines retrievers per query using pre- and post-retrieval trustworthiness signals, then re-ranks aggregated results.The framework also uses multi-granularity retrieval to exploit complementary semantic units.Source: §1 · p. 1
• MoR combines BM25, DPR, and SimCSE with zero-shot trustworthiness weights across databases of varying granularity.Source: Figure 1 · p. 2
• MoR variants outperform individual retrievers, including GritLM, across four scientific-domain retrieval and RAG tasks.Source: §1 · pp. 1–2
• Human retrievers extend the mixture beyond token- and embedding-based systems, achieving 1.4× MoR’s performance and a +58.9% relative gain over simulated humans alone.Source: §1 · p. 2
• The paper’s contributions include query-level comparative analysis, zero-shot deep fusion with pre-/post-retrieval signals, and improved human-LLM collaboration.Source: §1 · p. 2

• MoR dynamically combines complementary retrievers instead of selecting fixed retrievers for predefined query types.Source: §2 · p. 2
• Adaptive RAG research addresses knowledge-source selection, query complexity, retrieval cost, multimodality, and learned retriever ranking.Source: §2 · p. 2
• MoR draws on distributed information retrieval, query performance prediction, aggregated search, and rank fusion to combine multiple sources.Source: §2 · p. 2

3.1 Preliminary

• The retriever task assigns each document a model-specific relevance score, producing a score vector used to rank the corpus.Source: §3.1 · p. 3
• The test bed covers NFCorpus, SciDocs, SciFact, and SciQ, using NDCG@K to evaluate retrieval over diverse scientific query-document relations.Source: §3.1 · p. 3
• Figure 2 measures pairwise wins by whether the gold entry appears in one system’s top 20 but not the other’s, micro-averaged across queries.Source: Figure 2 · p. 3
• Candidate retrievers span unsupervised Contriever; supervised DPR, ANCE, TAS-B, GTR, and MPNet; and 7B references RepLLaMA and GritLM.Source: §3.1 · p. 3

3.2 Validating the folk wisdom

• DPR wins on 11.7% of SciFact queries against TAS-B despite lower overall performance, while DPR and SimCSE show complementary advantages despite similar aggregate results.Source: §3.2 · p. 3, Figure 2 · p. 3
• Route Oracle achieves +13.5% overall performance over GritLM, indicating substantial potential from query-level retriever mixtures.Source: §3.2 · pp. 3–4, Table 2 · p. 4

4 Mixture of Retrievers.

• MoR combines multiple retrievers without groundtruths by assigning query-specific weights and reranking documents using their weighted relevance scores.The framework computes per-retriever weights, forms adjusted document scores, and reranks the corpus.Source: §4 · p. 4
• Zero-shot embedding-space scores allow MoR to integrate diverse retrievers, including human information sources that provide ranked documents.The framework supports plug-and-play integration when suitable sentence-embedding models are available.Source: §4 · p. 4

4.1 Comprehensive retriever coverage

• Deep fusion expands each retriever across four index granularities, including questions, documents, sub-questions, passages, and propositions, without retriever adaptation.This increases the retriever pool fourfold to 4N and targets complementary semantic units.Source: §4.1 · pp. 4–5

4.2 Weighing retrievers’ effectiveness

• MoR estimates retriever effectiveness with complementary pre-retrieval and post-retrieval signals based on query-corpus proximity and retrieved-document correlation.Pre-retrieval signals down-weight outlier or poorly categorized queries, while higher Moran coefficients indicate more likely relevant retrieved documents.Source: §4.2 · p. 5

4.3 Parametric Combination.

• MoR-post combines pre-retrieval, Moran, and post-retrieval signals with fixed coefficients of 0.1, 0.3, and 0.6, while MoR variants outperform component and non-component LLM retrievers overall.The reported Table 3 comparison uses NDCG@5 and NDCG@20 macro-averages across tasks.Source: §4.3 · p. 6, Table 3 · p. 6

5.1 Main results: MoR is Effective

• MoR-post improves NDCG@20 by 10.8% over the best unsupervised component and 12.2% over the best supervised component.Source: §5.1 · p. 6
• MoR-post achieves an overall 3.9% relative NDCG@20 improvement over GritLM and outperforms it on NFCorpus, SciFact, and SciQ with 5 or 20 chunks considered.Source: §5.1 · p. 6
• MoR shows consistent RAG performance improvements over baselines on SciFact and SciQ without using downstream generation signals to assign weights.The evaluation feeds retrieved chunks to Llama-3-8B-Instruct and measures Exact Match.Source: §5.1 · p. 7

5.2 Human as a Retriever

• MoR assigns higher Vpost weights to simulated human experts on their corresponding domains and lower weights on non-expert domains.The setup uses four domain-specific experts with oracle retrieval in-domain and random rankings elsewhere.Source: §5.2 · p. 7
• MoR+Humans achieves the best performance across all evaluated domains and improves 58.9% relatively over equally weighted simulated human experts.This result supports incorporating non-oracle human information sources into collaborative retrieval.Source: §5.2 · p. 7, Table 6 · p. 7

5.3 Efficiency

• Mixing two retrievers can match the performance of mixing all eight, and the best pair is chosen for complementarity rather than individual ranking.This indicates subset selection can improve efficiency without substantial performance loss.Source: §5.3 · p. 8, Table 7 · p. 8
• At a 95th-percentile MoR-pre rejection threshold, MoR maintains strong performance while using around 20% of retrievers per query.Source: §5.3 · p. 8, Figure 3 · p. 8

5.4 Ablation Study

• Ablations show that the proposed retrieval and granularity signals outperform ad hoc mean or maximum score aggregation, while deep fusion contributes to performance.The comparison reports average retrieval performance using NDCG@20 across the task suite.Source: §5.4 · p. 8, Table 8 · p. 8

• MoR combines multi-granular deep fusion with pre- and post-retrieval signals that weight retriever outputs for each query.Source: §6 · p. 8
• Experiments validate consistent and robust performance improvements over component retrievers and state-of-the-art LLM-based retrievers.Further analyses examine human information sources and improved efficiency.Source: §6 · p. 9

• End-to-end post-presentation signals, such as exact-match performance or user satisfaction after using retrieved documents, remain a future direction.The authors suggest these signals could extend MoR from retrieval toward RAG and agentic retrieval.Source: Limitations · p. 9
• The current MoR design uses unsupervised signals, while supervised methods could learn query- and retriever-specific weights and signal coefficients.The authors describe neural alternatives that could use queries, retrievers, documents, and precomputed optimal weights.Source: Limitations · p. 9

• Documents are split into chunks of at most 128 words to avoid dense-retriever context overflow and map retrieved chunks back to original documents.This preprocessing is motivated by maximum-length constraints for models such as DPR.Source: §A.1 · p. 14
• Experiments run on a Google Cloud Platform instance with 8 NVIDIA L4 GPUs, each providing 24 GB of memory.Source: §A.1 · p. 14
• RAG experiments use temperature 0.1 and top_p 0.7 across all tasks.Source: §A.1 · p. 14

A.2 Propositionizer details and decompositions

• Propositioner is an off-the-shelf model distilled from GPT-4 into Flan-T5-Large using Wikipedia decomposition data.Source: §A.2 · p. 14
• The decomposition represents queries and documents as subqueries and propositions that express distinct semantic units.Source: §A.2 · p. 14
• Each proposition or subquery must be atomic, indivisible, self-contained, and contextually complete, including resolved coreferences when needed.Source: §A.2 · p. 14
• A biomedical query is decomposed into three subqueries covering externalization in neutrophil extracellular traps, inflammatory-cycle perpetuation, and autoantibody induction.Source: §A.2 · p. 14, Figure 4 · p. 14
• The appendix provides decomposition and RAG-template materials for SciFact and SciQ, including open-ended conversion for SciQ and three-way claim classification for SciFact.Source: §A.2 · p. 15

A.3 Extended Retriever Descriptions

• The retriever pool spans sparse BM25 and diverse BERT-sized dense models, while 7B RepLLaMA and GritLM provide larger-model references.The models range from 66M to 7B parameters and differ in architectures, training signals, and parameter sizes.Source: §A.3 · pp. 15–16

A.4 Extended design choice ablation

• MoR-post improves by 3.6% relatively with deep fusion, while vectorized-distance signals are strong without parametric combination.The classic Moran Index contributes to the combination but is not individually effective in this context.Source: §A.4 · p. 16

A.5 Qualitative Analysis

• MoR can recover relevant passages when most retrievers fail, but it cannot succeed when all base retrievers miss the correct passage.Source: §A.5 · p. 17

A.6 Extended Best Retriever Suite Analysis

• The best retriever suite can differ from the individually strongest retrievers; on SciQ, GTR and Contriever form the best suite despite MPNet and BM25 ranking highest individually by NDCG@20.Source: §A.6 · p. 17
• A 0.06 NDCG@5 gap separates Best of 2 from Best of 5 on SciFact, indicating that smaller curated retriever lists can retain similar retrieval performance on some tasks.Source: §A.6 · p. 17

A.7 Ethical Statements

• The experiments mainly cover English, and the authors report no ethical concerns because the models and datasets are open-sourced and the outputs are unlikely to contain harmful information.Source: §A.7 · p. 17

A.8 Licences of Scientific Artifacts

• The paper states that its scientific artifacts are used under their original purposes, with curated datasets and code/software under the MIT License.Source: §A.8 · p. 17, Table 14 · p. 19

A.9 Notation

• The appendix includes notation references and tables describing checkpoints, qualitative retrieval results, best retriever suites, datasets, and notation.Source: §A.9 · p. 17, Table 12 · p. 18, Table 13 · p. 18