PaperlayerView exact revision on arXiv

Source-linked AI summary

RaDeR: Reasoning-aware Dense Retrieval Models

Debrup Das, Sam O' Nuallain, Razieh Rahimi

arXiv:2505.18405v2cs.CLcs.IR

TL;DR

Retrieval models for language-model augmentation often struggle when relevance requires reasoning. RaDeR trains dense retrievers on retrieval-augmented mathematical reasoning data with self-reflective relevance judgments. It generalizes across reasoning benchmarks and outperforms strong baselines, including BM25 on reasoning-step queries.

  • Problem

    Retrieval models handle lexical and semantic matching reasonably well but face challenges when relevance prediction requires reasoning.

    Source: §1 · p. 1

  • Method

    RaDeR trains dense retrievers on mathematical reasoning trajectories with retrieved theorems, using self-reflection to assess relevance and form positive and hard-negative samples.

    Source: §4.2 · pp. 4–5

  • Results

    RaDeR outperforms strong baselines across reasoning-intensive benchmarks and is the first dense retriever reported to outperform BM25 on reasoning-step queries.

    Source: §1 · p. 2, §7 · p. 8

  • Takeaways & Limitations

    RaDeR’s results support reasoning-based retrieval for augmenting reasoning language models, including when their reasoning steps serve as retrieval queries.

    Source: §1 · p. 2

    Training primarily considers a retriever reasoning over a single document in isolation, rather than reasoning jointly across multiple documents.

    Source: §8 · p. 9

Abstract

from arXiv · show

We propose RaDeR, a set of reasoning-based dense retrieval models trained with data derived from mathematical problem solving using large language models (LLMs). Our method leverages retrieval-augmented reasoning trajectories of an LLM and self-reflective relevance evaluation, enabling the creation of both diverse and hard-negative samples for reasoning-intensive relevance. RaDeR retrievers, trained for mathematical reasoning, effectively generalize to diverse reasoning tasks in the BRIGHT and RAR-b benchmarks, consistently outperforming strong baselines in overall performance. Notably, RaDeR achieves significantly higher performance than baselines on the Math and Coding splits. In addition, RaDeR presents the first dense retriever that outperforms BM25 when queries are Chain-of-Thought reasoning steps, underscoring the critical role of reasoning-based retrieval to augment reasoning language models. Furthermore, RaDeR achieves comparable or superior performance while using only 2.5% of the training data used by the concurrent work REASONIR, highlighting the quality of our synthesized training data.

1 Introduction

Existing retrieval models handle lexical and semantic matching but struggle to judge relevance when reasoning is required. RaDeR generates reasoning-based retrieval training data from mathematical problem solving and performs strongly across reasoning-intensive benchmarks.

1 Introduction

  • Lexical and semantic term matching can fail when retrieval relevance depends on reasoning, while rerankers are constrained by the first-stage candidate set.Existing approaches include interleaved reasoning and retrieval, which relies on LLM reasoning steps, and reasoning-based reranking.Source: §1 · p. 1
  • Training reasoning-based retrievers requires diverse queries of varying lengths and complexity, as well as hard-negative reasoning samples.RaDeR uses retrieval-augmented mathematical reasoning trajectories and self-reflective relevance evaluation to generate training data.Source: §1 · p. 2
  • RaDeR generalizes from mathematical reasoning to BRIGHT and RAR-b, outperforming strong baselines and beating BM25 for the first time among dense retrievers on reasoning-step queries.It shows particularly strong results on BRIGHT Math and Coding splits.Source: §1 · p. 2
  • RaDeR gains 1.1 nDCG@10 points, a 4.5% relative gain, over concurrent REASONIR using 43,120 samples, about 2.5% of REASONIR’s 1,729,368.The authors present this as evidence for the effectiveness of their synthesized data.Source: §1 · p. 2

2 Related Works

Prior work interleaves reasoning with retrieval, optimizes retrieval queries, or reasons during reranking, but each approach retains constraints that RaDeR targets with reasoning-tailored synthetic data.

2 Related Works

  • Interleaved reasoning and retrieval use prior chain-of-thought steps as queries but rely on off-the-shelf retrievers.Search-R1 and R1-Searcher optimize query generation while keeping the retrieval model fixed.Source: §2 · p. 2
  • Reasoning-based rerankers such as RANK-1 and InteRank remain limited by the candidate set returned by initial retrievers.The cited methods use knowledge distillation from reasoning LLMs or reinforcement learning, respectively.Source: §2 · p. 3
  • Prior information-retrieval data augmentation expands terms or generates queries, while recent LLM pipelines use LLM-as-judge for quality and ReasonIR generates queries without labeled scalar rewards.These methods are not described as generating datasets specifically tailored for reasoning-intensive information retrieval.Source: §2 · p. 3
  • RaDeR directly integrates reasoning into first-stage retrieval and uses MCTS to generate sample-efficient synthetic data tailored to reasoning-intensive information retrieval.Source: §2 · p. 3

3 RaDeR: Reasoning-aware Retrievers

RaDeR combines a dense first-stage retriever with a pointwise cross-attention reranker; both predict relevance, without requiring reasoning at inference time for reranking.

3 RaDeR: Reasoning-aware Retrievers

  • The first-stage retriever uses a uni-embedding bi-encoder architecture for dense retrieval.The architecture is attributed to Lei et al. (2023).Source: §3 · p. 3
  • The reranker is a pointwise cross-attention model that takes the concatenated query and document and directly predicts relevance scores.Source: §3 · p. 3
  • RaDeR’s reranking approach is more efficient at inference than reasoning-based rerankers that rely on test-time compute for reasoning.The cited approaches are Weller et al. (2025) and Samarinas and Zamani (2025).Source: §3 · p. 3

4 Generating Retrieval Training Data

RaDeR uses retrieval-augmented MCTS over mathematical problems to produce varied reasoning trajectories and training samples with relevant theorems and hard negatives.

4 Generating Retrieval Training Data

  • Retrieval-augmented MCTS addresses the need for diverse, verifiable reasoning queries by using mathematical subproblems where retrieved theorems can support the original question.Checking generated answers against gold answers provides a proxy for evaluating the utility of retrieved theorems.Source: §4 · p. 3

4.1 Retrieval-Augmented Search-based Reasoning

  • MCTS explores reasoning trajectories through actions and reward-guided rollouts, with retrieval actions adding relevant theorems after self-reflection filters retrieved candidates.OST proposes one reasoning step, CRS completes the solution, and query generation is followed by theorem retrieval.Source: §4.1 · pp. 3–4

4.2 Synthesizing Training Data

  • High-reward trajectories containing retrieval nodes yield samples pairing a query with its retrieved relevant theorem and non-relevant retrieved theorems as hard negatives.Queries include partial MCTS reasoning paths, LLM-generated subproblem queries, and the original math questions.Source: §4.2 · pp. 4–5
  • Lexical training queries are generated from the retrieved theorem and retained only when BM25 retrieves that theorem in its top 20; negatives come from BM25 and RepLLaMA results.The final retrieval training data combines reasoning-based and lexical query types.Source: §4.2 · p. 5

5 Experimental Settings

The experimental setup uses Qwen2.5-based encoders and mathematical problem-solving data to train retrieval and reranking models for reasoning-intensive relevance.

Base language models

  • Qwen2.5 instruction-tuned models in 3B, 7B, and 14B sizes are used, with gte-Qwen2 and Llama-3.1-Instruct also included in encoder ablations.The model-size ablation tests parameter sizes from 3B to 14B.Source: §5 · p. 5

Math datasets for data generation

  • MATH and a NuminaMath subset supply problems; each question receives 16 retrieval-augmented MCTS rollouts using RepLLaMA and a ProofWiki theorem corpus.The framework uses a fixed parameter set described in Appendix D.Source: §5 · p. 5

Training details

  • The retriever uses contrastive InfoNCE with 12 synthesized hard negatives per query plus in-batch negatives, while the reranker is trained with contrastive loss.Training details are provided in Appendices F and G.Source: §5 · p. 5

6 Experimental Results

Experiments assess RaDeR on reasoning-intensive retrieval and reranking, standard term-matching retrieval, and retrieval-augmented question answering.

6 Experimental Results

  • Evaluation spans reasoning-intensive BRIGHT and RAR-b, standard MS MARCO, and QA with retrieval-augmented LLMs, using nDCG@10 to compare retrieval models.Recall and precision are also reported in Appendix I.Source: §6 · p. 5

6.1 BRIGHT Retrieval Performance

  • RaDeR-gte-Qwen2-7B achieves BRIGHT's best average performance of 25.5, outperforming strong baselines by at least 2 points in both query settings.The settings use original questions and GPT-4-generated CoT reasoning steps as retrieval queries.Source: §6.1 · p. 6, Table 1 · p. 6, Table 2 · p. 6

6.2 Retrieval Performance on RAR-b

  • RaDeR is comparable to the strongest open-source baseline on RAR-b Math and Coding tasks, exceeds it on MATH, and matches or surpasses top closed-source models.These results support generalization across reasoning-based retrieval tasks.Source: Table 3 · p. 6, §6.2 · p. 7

6.3 Retrieval on Traditional IR Benchmarks

  • RaDeR-gte-Qwen2-7B remains competitive with strong baselines on MS MARCO and TREC-DL term-matching benchmarks.The results indicate that reasoning-intensive training does not compromise effectiveness on standard IR benchmarks where reasoning is unnecessary.Source: §6.3 · p. 7, Table 4 · p. 7

6.4 BRIGHT Reranking Performance

  • On BRIGHT top-100 reranking, RaDeR models beat GPT-4 by +2.5 and +2.1 nDCG points; LeetCode and TheoremQA gains are +20.3 and +20.4 points, respectively.For GPT-4o CoT-based reranking, RaDeR is competitive with baselines including RANK1 while generating only relevance scores; RANK1 uses 635K examples and test-time reasoning.Source: §6.4 · pp. 7–8, Table 5 · p. 8, Table 6 · p. 8

6.5 RAG Performance using RaDeR

  • On TheoremQA, augmenting Qwen-2.5-7B-Instruct with RaDeR outperforms augmentation with RepLLaMA in both in-context RAG and retrieval-augmented MCTS settings.The MCTS setting uses majority voting for answer selection.Source: §6.5 · p. 8, Table 7 · p. 8

7 Conclusion

RaDeR combines retrieval and reranking models for reasoning-intensive relevance with MCTS-generated training data. The paper reports gains on reasoning-intensive benchmarks, broad generalization, and data efficiency.

7 Conclusion

  • RaDeR is a suite of retrievers and rerankers designed for reasoning-intensive relevance ranking.Its training data is generated with a retrieval-augmented MCTS reasoning framework.Source: §7 · p. 8
  • The paper reports substantial retrieval and reranking improvements over state-of-the-art models on reasoning-intensive benchmarks, alongside strong generalizability and data efficiency.The conclusion attributes training-data generation to retrieval-augmented MCTS.Source: §7 · p. 8
  • The reported evidence supports using mathematical reasoning to construct training data for models targeting reasoning-intensive relevance ranking.The conclusion characterizes the generated data as high quality.Source: §7 · p. 8

8 Limitations

The passages describe limitations in RaDeR’s single-document focus, lack of retrieval explanations, and answer-based reward signal, alongside related retrieval approaches and the MCTS rollout procedure.

8 Limitations

  • RaDeR training primarily treats documents in isolation, leaving joint multi-document relevance—such as in multi-hop QA—as future work.The relevance of each document in that setting may depend on the content of other documents.Source: §8 · p. 9
  • RaDeR returns relevance scores without explicit retrieval explanations, leaving explanation generation and interpretability as future-work directions.The authors frame maintaining first-stage ranking efficiency as a condition for such explanations.Source: §8 · p. 9
  • MCTS rewards depend on the final math answer, so an incorrect reasoning path that reaches a correct answer can make training data noisy.Source: §8 · p. 9

A.1 Mathematical Information Retrieval

A.2 CoT Reasoning

A.3 Reasoning-Based Re-Ranking Models

B Sampling MCTS Rollout Solutions

  • MCTS selection traverses to a leaf using maximum UCT values; simulation randomly rolls out actions to a terminal node, and correctness against the gold answer determines rewards used to update Q values.Source: B Sampling MCTS Rollout Solutions · p. 12

C Math Datasets for Data Generation

The data-generation setup uses mathematical problem datasets with question-answer pairs, and reports the MCTS hyperparameters in a table.

C Math Datasets for Data Generation

  • The pipeline uses MATH problems across eight subject types and five difficulty levels, plus selected NuminaMath splits containing competition-level problems paired with solutions.Gold step-by-step solutions are not required for data generation.Source: C Math Datasets for Data Generation · p. 13

D MCTS Parameters

E Statistics of Synthesized Data

The section points to synthesized-sample counts by source dataset and describes how RaDeR embeds inputs, scores relevance, and trains its retriever.

E Statistics of Synthesized Data

F Retriever Training Details

  • RaDeR uses the last-layer representation of an appended EOS token as the query or document embedding, then scores relevance with cosine similarity.Source: F Retriever Training Details · p. 13
  • The retriever is optimized with InfoNCE using a human-annotated relevant document as the positive and non-relevant documents as negatives.Source: F Retriever Training Details · p. 13

G Reranker Training Details

The section specifies pointwise reranker scoring and training, identifies hyperparameter and BRIGHT precision/recall reporting, and reports ablation gains from query diversity and model scaling.

G Reranker Training Details

  • The pointwise reranker scores concatenated query-document pairs by projecting the decoder’s final end-of-sequence hidden state to a scalar relevance score.Its training uses the same loss as retriever training, without in-batch negatives.Source: G Reranker Training Details · pp. 13–14

H Retriever/Reranker Training Hyperparameters

I Retrieval Performance on BRIGHT

J Ablation Studies

  • On BRIGHT, adding query types consistently improves retrieval; the 14B Qwen2.5-instruct model outperforms the 7B variant by 2 points on average.The query-type ablation compares models trained on subsets containing different synthesized query types.Source: J Ablation Studies · p. 14

K Analysis

K analyzes synthesized query examples and qualitative retrieval cases; the following sections provide prompts for MCTS actions, reflection, summarization, and reasoning-query generation.

K Analysis

M Prompt for OST thought action

N Prompt for CRS action

O Prompt for MCTS Query Generation

P Prompt for Self Reflection MCTS

Q Prompt for LLM Reasoning Query generation

Related papers

Loading 2505.18405v2…