PaperlayerView exact revision on arXiv

Source-linked AI summary

ReasonIR: Training Retrievers for Reasoning Tasks

Rulin Shao, Rui Qiao, Varsha Kishore, Niklas Muennighoff, Xi Victoria Lin, Daniela Rus, Bryan Kian Hsiang Low, Sewon Min, Wen-tau Yih, Pang Wei Koh, Luke Zettlemoyer

arXiv:2504.20595v1cs.AIcs.CLcs.IRcs.LG

TL;DR

Reasoning-intensive retrieval remains difficult because existing retriever training emphasizes short factual queries with direct document matches. ReasonIR-8B addresses this with synthetic reasoning-oriented queries and hard negatives, achieving state-of-the-art BRIGHT results and stronger RAG performance while benefiting from longer rewritten queries.

  • Problem

    Existing retrievers struggle with reasoning-intensive retrieval because their training datasets focus on short factual queries tied to documents with straightforward matches.

  • Method

    REASONIR-SYNTHESIZER generates varied-length and reasoning-intensive queries with synthesized documents and plausibly relevant hard negatives for contrastive training.

  • Results

    ReasonIR-8B achieves state-of-the-art BRIGHT scores of 29.9 nDCG@10 with rewritten queries and 36.9 with an LLM reranker, while improving MMLU and GPQA RAG performance over closed-book baselines.

  • Takeaways & Limitations

    ReasonIR-8B consistently benefits from longer rewritten queries and achieves strong retrieval performance with substantially lower compute than LLM rerankers.

Abstract

from arXiv · show

We present ReasonIR-8B, the first retriever specifically trained for general reasoning tasks. Existing retrievers have shown limited gains on reasoning tasks, in part because existing training datasets focus on short factual queries tied to documents that straightforwardly answer them. We develop a synthetic data generation pipeline that, for each document, our pipeline creates a challenging and relevant query, along with a plausibly related but ultimately unhelpful hard negative. By training on a mixture of our synthetic data and existing public data, ReasonIR-8B achieves a new state-of-the-art of 29.9 nDCG@10 without reranker and 36.9 nDCG@10 with reranker on BRIGHT, a widely-used reasoning-intensive information retrieval (IR) benchmark. When applied to RAG tasks, ReasonIR-8B improves MMLU and GPQA performance by 6.4% and 22.6% respectively, relative to the closed-book baseline, outperforming other retrievers and search engines. In addition, ReasonIR-8B uses test-time compute more effectively: on BRIGHT, its performance consistently increases with longer and more information-rich rewritten queries; it continues to outperform other retrievers when combined with an LLM reranker. Our training recipe is general and can be easily extended to future LLMs; to this end, we open-source our code, data, and model.

1 Introduction

ReasonIR-8B targets reasoning-intensive retrieval, where existing retrievers struggle because training data emphasizes short factual queries with direct document matches. Its synthetic-data recipe and evaluation show strong BRIGHT and RAG performance, including gains from richer rewritten queries and reranking.

  • Existing retrievers struggle because their training datasets focus on short factual queries with straightforward matches, unlike reasoning-intensive retrieval.
  • ReasonIR-8B is a bi-encoder retriever developed specifically for reasoning-intensive retrieval.
  • REASONIR-SYNTHESIZER creates varied-length queries, synthesized documents, hard queries, and superficially relevant but unhelpful hard negatives for contrastive training.
  • 24.4 nDCG@10 on original BRIGHT queries, 29.9 with GPT4-rewritten queries, and 36.9 with an LLM reranker establish state-of-the-art results.
  • ReasonIR-8B improves RAG performance over closed-book baselines by 6.39% on MMLU and 22.58% on GPQA while outperforming other retriever and search-engine baselines.
  • The method can incorporate newer LLMs for synthetic-data generation or base-model training, and the authors release code, model, and data recipe.

2 Preliminaries

RAG retrieves documents before generating an answer, while reasoning-intensive retrieval requires relevant supporting material despite low query-document overlap. The paper frames query rewriting, reranking, and contrastive training as complementary retrieval mechanisms.

  • A bi-encoder independently embeds queries and documents, then scores them using cosine similarity.
  • RAG retrieves the top-k documents and concatenates them with the query before generating the response.
  • Reasoning-intensive retrieval is harder because queries have low lexical and semantic overlap with relevant documents.
  • Query rewriting produces longer, more informative queries, while LLM reranking rescoring top-K documents selects the final k documents.
  • Contrastive training pulls queries toward positive documents and away from negative documents, with hard negatives approximating the full negative set.

3 Pilot Study: Examining Retrieval Datasets and Test-Time Scaling

The pilot study finds that public retriever-training datasets are much shorter and simpler than reasoning benchmarks. On BRIGHT, information-rich query rewriting is more effective than query decomposition, motivating test-time scaling toward longer queries.

  • BRIGHT spans 12 subjects, and its averaged nDCG@10 measures ranking quality among the top-10 retrieved documents.
  • Average queries contain 20 tokens in Natural Questions and 21 in MS MARCO, versus 194 tokens in BRIGHT, whose retrieval requires reasoning.
  • The pilot studies whether adding more reasoning information through rewritten queries improves reasoning-intensive retrieval on BRIGHT.
  • Query decomposition reduced Nomic performance from 12.1 to 10.5 and GRIT-7B performance from 20.4 to 17.3 on BRIGHT.
  • The study therefore treats long, information-rich queries as preferable to several decomposed short queries on BRIGHT.

4 ReasonIR: Synthesizing Hard and Varied-length Retriever Training Data

REASONIR-SYNTHESIZER augments public retrieval data with varied-length and hard-query examples to address reasoning-intensive retrieval. Its pipeline selects informative seed documents, generates challenging self-contained queries, and constructs difficult negatives for contrastive training.

  • Training-data recipe: The recipe combines public data with varied-length (VL) data and hard-query (HQ) data to improve reasoning-intensive retrieval.VL targets longer effective query context, while HQ targets reasoning-intensive queries.
  • Varied-length data: VL data uses 300–2,000-word queries and synthesized positive documents to expose the retriever to a wider range of query lengths.The generated queries are designed to encourage use of rich information from lengthy and complex inputs.
  • Hard-query data: HQ generation starts from educational, reasoning-worthy documents filtered from scientific and technical sources using the FineWeb-Edu classifier.Documents scoring below 2 are removed because they usually contain gibberish or subjective content.
  • Hard-query data: The document-to-query procedure asks an LLM to reason about background knowledge, problem-solving patterns, and realistic scenarios before formulating challenging, self-contained, diverse questions.This human-like brainstorming guideline is intended to improve query diversity without human effort.
  • Hard negatives: Hard negatives are generated because BM25-style top-ranked irrelevant documents do not work reliably for reasoning-intensive queries, while sufficiently difficult negatives support contrastive training.The paper analyzes negative-selection difficulty using the frequency that BM25 or GRIT-7B ranks the hard negative above the positive.
  • Evaluation: REASONIR-8B is fine-tuned on public and synthetic data, achieving 24.4 nDCG@10 on original BRIGHT queries, 29.9 with GPT4-rewritten queries, and 36.9 with an LLM reranker.The proposed QWENRERANK reranker uses interpolation to break ties and is described as requiring no training or long reasoning outputs.

5 Experiments

Experiments show that ReasonIR-8B improves reasoning-intensive retrieval and RAG, benefits from query rewriting and score combination, and achieves strong performance with substantially lower test-time compute than LLM rerankers.

  • Reasoning-intensive IR: 24.4 and 29.9 nDCG@10 on BRIGHT show strong performance with original and REASON-QUERY queries, respectively.The model also reaches 28.0 nDCG@10 with the smaller Llama3.1-8B-Instruct query rewriter.
  • Test-time scaling: Longer rewritten queries continue improving ReasonIR-8B, whereas GRIT-7B and nomic-v1.5-text show diminishing or negative gains.The result indicates better use of information-rich rewritten queries than competing dense retrievers.
  • Reasoning-intensive IR: 32.0 nDCG@10 results from interpolating ReasonIR-8B and BM25 scores, while QwenRerank combination reaches 36.8.These combinations improve retrieval beyond the standalone retriever.
  • Reasoning-intensive RAG: 3.9 and 7.1 absolute points are added to closed-book MMLU and GPQA performance, respectively, using ReasonIR-8B for RAG.ReasonIR-8B outperforms the previous SOTA retriever on both benchmarks.
  • Query rewriting: 0.8 and 0.6 absolute MMLU improvements result from query rewriting for GRIT-7B and ReasonIR-8B, while rewriting reduces dense-retriever performance on GPQA.The search engine’s GPQA score instead rises from 33.8 to 36.4 after rewriting.
  • Computational analysis: 200× less test-time compute lets ReasonIR-8B outperform the Rank1-32B LLM reranker.QwenRerank uses less compute than Rank1-32B while performing better.

6 Related Work

Prior work covers retrieval for factual and reasoning settings, synthetic training-data generation, and test-time scaling, but this paper targets reasoning-intensive retriever training directly.

  • Retrieval and reasoning: Retrieval has traditionally been less effective for reasoning-intensive tasks, while newer reasoning systems integrate retrieval into their reasoning processes.These systems often rely on retrievers optimized for semantic retrieval rather than reasoning-intensive retrieval.
  • Test-time scaling for RAG: Test-time RAG scaling has explored larger and more diverse datastores and longer retrieved contexts.These directions distribute additional computation across datastore and context components.
  • Synthetic data generation: Synthetic IR data generation commonly uses LLMs to create simple task-specific queries from documents or documents from queries.ReasonIR instead studies synthetic data generation for reasoning-task retriever training.

7 Discussion

The discussion presents ReasonIR-8B as a synthetic-data approach that improves reasoning-intensive retrieval while remaining adaptable to newer models and extensions.

  • Contribution: ReasonIR-8B achieves significant improvements on both reasoning-intensive IR and RAG tasks through synthetic data generation for retriever training.The authors identify this as the central direction of the work.
  • Future work: Future work includes scaling synthetic data, improving reasoning-worthy seed selection, and extending the approach to multilingual, multimodal, and multi-turn settings.These are proposed directions rather than evaluated capabilities.
  • Synthetic data recipe: Varied-length data generation creates instructions, queries, positive documents, and hard negatives through successive prompting stages.The final query combines the task instruction with the generated query before hard-negative generation.
  • Query rewriting: Controlled-length REASON-QUERY rewriting appends a maximum-token constraint to the original rewriting instruction.GPT4O-MINI is used for the query-scaling study.

C Query Decomposition

The query-decomposition appendix documents related prompts and reports that decomposing reasoning queries into short subtasks underperforms information-rich long queries on BRIGHT.

  • Query decomposition: Query decomposition assumes a reasoning retrieval task can be split into several simple subtasks.The appendix evaluates this approach with LangChain.
  • HQ data generation: The HQ generation prompt requests standalone reasoning-intensive questions answerable from a document’s supporting evidence.It directs questions toward core concepts and principles in the document’s domain.
  • Prompted data construction: The documented generation prompts specify JSON fields for an input and its relevant positive document.The hard-negative prompt receives incomplete data containing these fields.
  • Prompt inventory: The appendix includes prompts for varied-length generation, easy-query generation, REASON-QUERY rewriting, and task-specific reranking.The documented figures cover these prompt variants and reranking tasks.
  • Query decomposition: 0.121 to 0.105 with Nomic and 0.204 to 0.173 with GRIT-7B are the reported performance drops after decomposition.Both methods use GPT4 for query rewriting and decomposition.
  • Query decomposition: Information-rich long queries outperform three decomposed short queries on BRIGHT.The comparison is presented as a contrast between long-query rewriting and decomposition.

E.1 Contrastive Training Setup

The setup combines bi-encoder retrieval, query reranking, and benchmark evaluation across reasoning-intensive retrieval and RAG settings. It also reports Recall@100 and oracle-reranker performance to assess candidate quality and reranking potential.

  • Training configuration: REASONIR-8B uses contrastive training with hard negatives and in-batch negatives.The model is adapted from Llama3.1-8B with a bi-directional attention mask.
  • Reranking: QwenRerank produces normalized relevance scores from 0 to 1 using zero-shot prompting.Two tie-breaking combinations are evaluated: QwenRerank with BM25 and with the base retriever.
  • Evaluation: The evaluation covers BRIGHT reasoning-intensive retrieval, MMLU, and GPQA-based retrieval-augmented generation.BRIGHT uses nDCG@10, while MMLU is averaged across 56 subjects.
  • Evaluation: Recall@100 and oracle nDCG@10 evaluate candidate-pool quality and the theoretical ceiling for subsequent reranking.The oracle results indicate substantial remaining room for reranker improvement.

G Additional Reranker Results

Additional experiments examine reranker cost, tie-breaking, candidate pools, and retriever–reranker combinations. They show that candidate quality and retriever complementarity affect downstream reranking outcomes.

  • Reranker comparisons: REASONIR-8B’s proposed LLM reranker is training-free, cheaper, and more performant than existing LLM reranker baselines.The section also evaluates REASONIR-8B as the underlying retriever for reranking.
  • Tie-breaking: Tie-breaking improves results compared with using reranker scores directly.The experiments compare two tie-breaking strategies against direct score use.
  • Candidate pools: REASONIR-8B retrieves a better candidate pool for the Qwen-32B reranker.Candidate sets from different retrieval systems are compared for downstream reranking.
  • Retriever–reranker combinations: Rank1-7B performs best with BM25 candidates despite REASONIR-8B’s stronger standalone retrieval results.This shows that reranker performance depends on the retriever supplying its candidates.
  • Retriever complementarity: BM25 and REASONIR-8B share 28.2% of their top-100 documents, while gold-document overlap reaches only 53.5%.Their dissimilar candidate sets can complement one another in hybrid retrieval.

H.1 Training with Reasoning Rewritten Queries (RQ)

This section studies whether reasoning-rewritten queries should be included during retriever training. The results distinguish the benefits of query-length coverage from those of explicit reasoning-query training.

  • Training setup: The RQ training setup reuses the original queries’ positives and negatives for reasoning-rewritten queries.This reuse is explicitly described as an assumption made under computational constraints.
  • Individual synthetic-data types: Training with RQ is most beneficial for REASON-QUERY evaluation, while VL is more helpful for original-query evaluation.These conclusions apply when each synthetic-data type is combined with the public training set.
  • Combined mixtures: The HQ-plus-VL mixture achieves nDCG@10 scores of 24.4 on original queries and 29.9 on reasoning queries.This mixture performs best across both evaluation settings among the combined synthetic-data configurations described.
  • Ablation: Table 12 reports BRIGHT nDCG@10 for models trained on different data sources, including public and synthetic data.Each model is trained for 1000 steps from the LLAMA3.1-8B initialization checkpoint.
  • Combined mixtures: HQ plus VL outperforms RQ plus VL on REASON-QUERY evaluation.The authors attribute this pattern to sufficient coverage of query length and difficulty without directly training on rewritten queries.

I Test-time Compute Comparison between Retriever and Reranker

The compute comparison contrasts bi-encoder retrieval with cross-encoder reranking and illustrates reasoning-intensive retrieval through query and document examples. Bi-encoders are presented as substantially cheaper at test time, especially when rerankers process many documents or generate reasoning tokens.

  • Bi-encoder retriever: Bi-encoder query encoding costs approximately 2NL_q FLOPS, after datastore pre-indexing.Its fixed-model time complexity is O(L_q^2).
  • Cross-encoder reranker: Cross-encoder reranking costs 2Nk(L_q + L_d + L_o) FLOPS across k query-document pairs.The cost includes query, document, and generated output tokens.
  • Compute comparison: Reranking is significantly more expensive when k is large or generated reasoning output is long.The passage gives k = 100 and roughly 300 generated tokens for Rank1-7B as examples.
  • FLOPS calculation: REASONIR-8B has about 7.0 billion non-embedding parameters, compared with 6.5 billion for Rank1-7B and 31 billion for Rank1-32B.The parameter counts are used in the FLOPS estimates.
  • Compute comparison: Rank1-7B can be a few hundred times slower than REASONIR-8B for normal reasoning-intensive queries.The comparison is based on the estimated FLOPS in the compute analysis.
  • Reasoning-intensive retrieval: Reasoning-intensive queries require retrieving documents for complex scenarios rather than directly matching simple factual questions.Examples span biology, coding, economics, and legal retrieval data.
Loading 2504.20595v1…