PaperlayerView exact revision on arXiv

Source-linked AI summary

ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability

Wenhan Liu, Xinyu Ma, Weiwei Sun, Yutao Zhu, Yuchen Li, Dawei Yin, Zhicheng Dou

arXiv:2508.07050v3cs.IRcs.AIcs.CLcs.LG

TL;DR

Reasoning-intensive queries expose a training-data gap for passage rerankers. ReasonRank synthesizes diverse training data and trains with SFT and RL; it reports state-of-the-art results on BRIGHT and R2MED and greater efficiency than Rank1.

  • Problem

    Reasoning-intensive rerankers lack suitable training data, while complex queries require passages providing key evidence or solutions through similar methodologies.

    Source: §1 · p. 1

  • Method

    ReasonRank synthesizes 13K diverse reasoning-intensive examples with DeepSeek-R1, then uses cold-start SFT and RL with a multi-view reward for sliding-window ranking.

    Source: §1 · p. 2

  • Results

    ReasonRank achieves state-of-the-art performance on BRIGHT and R2MED; ReasonRank (32B) beats Rank-K (32B) on BRIGHT by about 5 points and is more efficient than Rank1.

    Source: §1 · p. 2, §5.2 · p. 7

  • Takeaways & Limitations

    Experiments support the effectiveness of synthesized reasoning-intensive data and two-stage training for reasoning-intensive passage ranking.

    Source: §5.2 · p. 7

    ReasonRank lacks non-reasoning training data, limiting its ability to switch seamlessly between reasoning and non-reasoning modes across search scenarios of varying difficulty.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large Language Model (LLM) based listwise ranking has shown superior performance in many passage ranking tasks. With the development of Large Reasoning Models (LRMs), many studies have demonstrated that step-by-step reasoning during test-time helps improve listwise ranking performance. However, due to the scarcity of reasoning-intensive training data, existing rerankers perform poorly in many complex ranking scenarios, and the ranking ability of reasoning-intensive rerankers remains largely underdeveloped. In this paper, we first propose an automated reasoning-intensive training data synthesis framework, which sources training queries and passages from diverse domains and applies DeepSeek-R1 to generate high-quality training labels. To empower the listwise reranker with strong reasoning ability, we further propose a two-stage training approach, which includes a cold-start supervised fine-tuning (SFT) stage and a reinforcement learning (RL) stage. During the RL stage, we design a novel multi-view ranking reward tailored to the multi-turn nature of listwise ranking. Extensive experiments demonstrate that our trained reasoning-intensive reranker \textbf{ReasonRank} outperforms existing baselines significantly and also achieves much lower latency than the pointwise reranker. Our codes are available at https://github.com/8421BCD/ReasonRank.

1 Introduction

Listwise ranking compares passages jointly, and test-time reasoning has motivated reasoning-intensive rerankers. ReasonRank addresses their training-data and ranking-reward challenges with synthesized data and two-stage training, reporting strong benchmark performance and improved efficiency.

1 Introduction

  • Listwise ranking evaluates passages together and has performed effectively, while test-time step-by-step reasoning has improved LLM performance on challenging tasks and motivated reasoning-based reranking.The introduction frames reasoning over query intent and multiple passages as desirable for accurate reranking.Source: §1 · p. 1
  • Existing rerankers often use traditional web-search data, while complex queries require evidence or solutions using similar methodologies and human annotation is prohibitively expensive to scale.The paper identifies these challenges as bottlenecks for training rerankers for complex search scenarios.Source: §1 · pp. 1–2
  • The authors synthesize 13K diverse reasoning-intensive training examples without human involvement, using DeepSeek-R1 to select passages and generate reasoning chains and gold rankings.A self-consistency filter keeps rankings in which R1-selected positive passages consistently appear near the top.Source: §1 · p. 2
  • ReasonRank achieves state-of-the-art performance on BRIGHT and R2MED; its 7B and 32B models exceed the previous BRIGHT state of the art by 3 and 5 points, respectively, and it is more efficient than Rank1.The 7B and 32B comparisons are reported for BRIGHT; the efficiency comparison is against the pointwise Rank1 reranker.Source: §1 · p. 2
  • ReasonRank’s two-stage training combines cold-start SFT for reasoning-pattern learning with reinforcement learning using a multi-view ranking reward.The reward considers local ranking quality while promoting relevant-passage recall to improve the final ranking metric.Source: §1 · p. 2

2 Related Work

Related work describes pointwise, pairwise, and listwise LLM ranking, then situates ReasonRank among reasoning-based rerankers trained for complex retrieval.

2 Related Work

  • Pointwise LLM ranking evaluates each query-document pair independently and is computationally efficient, but lacks cross-document comparison; pairwise ranking compares document pairs to establish relative relevance.Source: §2 · p. 2
  • Listwise methods rerank passage lists through global comparison and have achieved state-of-the-art performance on many IR benchmarks.Source: §2 · p. 3
  • Reasoning-based rerankers distill DeepSeek-R1 reasoning or optimize setwise ranking with RL, but the cited approaches use traditional web-search training data and perform poorly on reasoning-intensive benchmarks.ReasonRank instead proposes diverse-domain reasoning-intensive training data and a two-stage training approach.Source: §2 · p. 3

3 Preliminaries

The preliminaries define listwise passage reranking and its reasoning-based output, then describe sliding windows for processing long passage lists.

3 Preliminaries

  • Listwise ranking takes a query and passage list as input and outputs passage IDs ordered by relevance; this paper uses N = 100, window size 20, and step size 10.The sliding-window process promotes relevant passages from the back toward the front across iterative subsets.Source: §3 · p. 3
  • A reasoning-based listwise reranker outputs both a reasoning trace enclosed in <think> tags and a reranked passage-ID sequence enclosed in <answer> tags.Source: §3 · p. 3

4 Methodology

ReasonRank’s methodology combines automated synthesis of reasoning-intensive ranking data across diverse query domains with a two-stage training framework. The framework first uses listwise labels for SFT, then applies multi-view ranking rewards in RL.

4.1 Reasoning-intensive Data Synthesis

  • The synthesis framework covers complex QA, coding, math, and web search, then filters low-quality training labels using self-consistency.For filtering, samples whose listwise-label NDCG@10 falls below a predefined threshold α are removed, with pointwise labels as ground truth.Source: §4.1 · p. 3, §4.1.5 · p. 5

4.1.1 Complex QA

  • For complex QA, DeepSeek-R1 selects positives from passages linked to accepted answers and hard negatives from top-10 Google Search documents, then ranks lists capped at 20 passages.The generated labels include pointwise binary labels and listwise reasoning chains with gold rankings; the gold answer is withheld when generating reasoning chains.Source: §4.1.1 · pp. 3–4

4.1.2 Coding

  • For coding queries, E5-Mistral-7B-Instruct retrieves 40 candidate passages before R1 selects positives, samples negatives, and generates listwise reasoning and rankings.The passage corpus comes from the LeetCode dataset in the BRIGHT benchmark.Source: §4.1.2 · p. 4

4.1.3 Math

  • For math, the data cover problem–solution similarity and theorem applicability, using MATH queries and task-specific passage corpora before R1 creates listwise labels.The Math-Problem corpus is from Su et al. (2025), while Math-Theorem passages are theorem statements from ProofWiki; candidate retrieval follows the coding procedure.Source: §4.1.3 · pp. 4–5

4.1.4 Web Search

  • For web search, the framework samples 4K MSMARCO training queries, retrieves the top 20 passages with BM25, and uses R1 to generate listwise labels.The sampled passages have pointwise relevance labels.Source: §4.1.4 · p. 5

4.1.5 Self-Consistency Data Filtering

  • Self-consistency filtering removes training samples whose listwise-label NDCG@10, evaluated against pointwise labels, falls below a predefined threshold α.The retained samples form the quality-filtered dataset used for subsequent model training.Source: §4.1.5 · p. 5

4.2 Two-stage Training Framework

  • SFT learns from listwise labels containing reasoning chains and reranked passage lists; RL then uses multi-view rewards to optimize sliding-window ranking.The reward combines NDCG@10, Recall@10, and RBO; format rewards check the presence of <think> and <answer> tags and ranking syntax.Source: §4.2 · pp. 5–6

5 Experiment

Experiments evaluate ReasonRank on reasoning-intensive benchmarks, ablations, and traditional IR, with latency compared against Rank1. ReasonRank outperforms baselines on BRIGHT and R2MED and is also faster than Rank1 in the reported efficiency test.

5.1 Experimental Setup

  • BRIGHT and R2MED use distinct retrievers, rerank the top 100 passages, and use NDCG@10; ReasonRank is trained with Qwen2.5-7B-Instruct and Qwen2.5-32B-Instruct.The experiments compare reasoning and non-reasoning rerankers.Source: §5.1 · p. 6

5.2 Main Results

  • ReasonRank (7B and 32B) beats all baselines on average across BRIGHT and R2MED; 32B leads Rank-K (32B) by about 5 points on BRIGHT and Rank1 (32B) by about 4 on R2MED.The results are reported as NDCG@10 on top-100 passages retrieved by ReasonIR for BRIGHT and E5-Mistral for R2MED.Source: §5.2 · pp. 6–7, Table 1 · p. 7, Table 2 · p. 7
  • On BRIGHT, baselines generally struggle to improve initial retrieval, while the two non-reasoning rerankers underperform E5-Mistral on R2MED Avg.The authors suggest traditional training data and methods struggle to produce effective reasoning-intensive rerankers.Source: §5.2 · p. 7
  • On BRIGHT Avg., Rank-R1 (32B) exceeds Rank-R1 (7B) by 9 points, while ReasonRank (32B) exceeds ReasonRank (7B) by about 2.3 points.The authors interpret the scaling pattern as indicating stronger reasoning and ranking capabilities in larger models.Source: §5.2 · p. 7

5.3 Ablation Study

  • On BRIGHT, MSMARCO-only training drops 5.66 points, removing cold-start SFT drops 7.05, removing RL drops 2.59, and NDCG@10-only reward further harms performance.The ablations use ReasonRank (7B).Source: §5.3 · pp. 7–8
  • Without explicit reasoning, SFT and SFT+RL rerankers score 32.96 and 33.60, respectively, below ReasonRank at 35.74.These BRIGHT results support the authors’ conclusion that explicit reasoning is essential for listwise ranking.Source: §5.3 · p. 8

5.4 Traditional IR Benchmark

  • On averaged NDCG@10 across seven BEIR datasets, ReasonRank (32B) outperforms all listed baselines, with a smaller gap than on BRIGHT and R2MED.The selected BEIR datasets have fewer queries; the authors suggest their queries may be simpler and require less complex reasoning.Source: Table 4 · p. 8, §5.4 · p. 8

5.5 Efficiency Analysis

  • Across eight BRIGHT datasets, ReasonRank (7B) is 2–2.7× faster than Rank1 (7B), using one reasoning chain for multiple passages rather than one per passage.The comparison reranks the top 100 ReasonIR-retrieved passages on four A800 80G GPUs.Source: Figure 5 · p. 8, §5.5 · p. 8

6 Conclusion

ReasonRank combines reasoning-intensive ranking data synthesis with cold-start SFT and multi-view ranking-based RL. The paper concludes that it achieves superior performance and efficiency in its experiments.

6 Conclusion

  • ReasonRank pairs reasoning-intensive ranking data synthesis with a two-stage approach: cold-start SFT followed by multi-view ranking-based RL.The conclusion describes the experiments as demonstrating superior performance and efficiency.Source: §6 · p. 9

Limitations

The authors identify two limitations: training excludes non-reasoning data, and reranking still relies on a sliding-window strategy.

Limitations

  • Because training excludes non-reasoning data, ReasonRank cannot seamlessly switch between reasoning and non-reasoning modes across search scenarios of varying difficulty.The authors plan to add non-reasoning data to improve flexibility.Source: Limitations · p. 9
  • ReasonRank still uses sliding windows; the authors propose exploring full-list ranking to improve scalability and performance.They cite prior studies showing that LLMs can rank 100+ passages in one forward pass with better efficiency and effectiveness than sliding windows.Source: Limitations · p. 9

A.1 Benchmarks

The evaluation covers complex-reasoning, medical, and cross-domain retrieval benchmarks, alongside diverse reranking baselines and a filtered, multi-domain training-data synthesis process.

A.1 Benchmarks

  • BRIGHT tests reasoning-intensive retrieval with 1,384 diverse-domain queries, R2MED covers 876 queries across three medical retrieval tasks, and BEIR spans 18 datasets.The described tasks include fact-checking and question answering in BEIR, and Q&A reference, clinical evidence, and clinical case retrieval in R2MED.Source: §A.1 · p. 13

A.2 Baselines

  • The comparison baselines include pointwise and listwise rerankers, reasoning-based methods, and retrievers selected for the BRIGHT and R2MED evaluations.Examples include RankT5, RankZephyr, Rank1, Rank-R1, Rearank, Rank-K, ReasonIR, and E5-Mistral-7B-Instruct.Source: §A.2 · pp. 13–14

B.1 Details about Reasoning-intensive Ranking Data Synthesis

  • The synthesized reasoning-intensive training data totals 13.5k examples across Complex QA, Coding, Math, and Web Search; for Web Search, 4k training queries are sampled from the MS MARCO training set.Web Search applies the same listwise ranking prompt as Complex QA.Source: Table 5 · p. 14, §B.1 · p. 14

B Implementation Details

  • Self-consistency filtering removes low-quality synthesized data, with α=0.4 selected by validation grid search over [0.1, 0.7].The grid-search step size is 0.1.Source: §B.1 · p. 14

B.1 Details about Reasoning-intensive Ranking Data Synthesis

  • For complex QA, long documents are split at simple separators, and R1 selects positive and hard-negative passages before ranking the candidates.The splitting approach avoids assumptions about file structure; the selection prompts use the query and gold answer.Source: §B.1 · p. 14

B.2 Details about Two-Stage Training Framework

ReasonRank is trained with cold-start supervised fine-tuning followed by GRPO reinforcement learning, using R1-generated ranking data and a multi-view reward.

Cold-Start SFT

  • Cold-start SFT trains Qwen2.5-7B-Instruct and Qwen2.5-32B-Instruct backbones before the reinforcement-learning stage.The 7B model is trained for five epochs and the 32B model for four epochs; both use LoRA rank and alpha 32 for 32B SFT.Source: §B.2 · pp. 14, 16

Two-Stage Training Framework

  • R1 prompts support positive and hard-negative passage selection and listwise ranking, with complex-query positives judged by their explanatory relevance.The training prompt also appears in the figures for coding, math, and passage ranking.Source: Figure 6 · p. 15, Figure 7 · p. 15, Figure 8 · p. 15

Multi-view Ranking based RL

  • After SFT, GRPO trains both ReasonRank backbones, with rollout sequences sampled per input and rewards normalized within each group.Training uses eight rollouts per sample; the reported learning rates are 1e-6 for ReasonRank (7B) and 1e-5 for ReasonRank (32B).Source: §B.2 · pp. 16–17
  • The ranking reward includes an RBO component that compares the rollout ranking with the gold ranking list, with p as a predefined parameter.The paper describes the reward as multi-view and reports ϕ=0.2 and γ=0.1 for that reward.Source: §B.2 · pp. 16–17

C.1 Detailed Results of Ablation Study

The ablations indicate that the proposed components matter across BRIGHT datasets, and the framework is also evaluated on BEIR and across different backbones.

C.1 Detailed Results of Ablation Study

  • Ablated ReasonRank (7B) models decline on almost all BRIGHT datasets, supporting the effectiveness of the corresponding components.The reported ablation results use NDCG@10.Source: §C.1 · p. 17

C.2 Detailed Results on BEIR Benchmark

  • The BEIR comparison evaluates seven datasets by reranking BM25’s top-100 retrieved passages.RankT5 (3B), Rank-R1 (7B), and Rank1 (7B) are included as additional baselines.Source: §C.2 · p. 17

C.3 Different Backbones

  • ReasonRank consistently and significantly outperforms baselines across the tested backbones, and Qwen3-8B ReasonRank reaches the highest reported average score, 36.14%.ReasonRank also improves over its SFT-only version on the Avg. metric.Source: §C.3 · p. 17

C.4 Analysis of Reasoning Length

The analysis compares ReasonRank’s reasoning-chain lengths across BRIGHT datasets and reports longer chains for coding and theorem tasks than for StackExchange tasks.

C.4 Analysis of Reasoning Length

  • ReasonRank (7B) reranks the top-20 passages retrieved by ReasonIR for the reasoning-length analysis.Source: §C.4 · p. 17
  • Reasoning chains tend to be longer for coding and theorem tasks than for StackExchange tasks.The cited examples include Leetcode and Pony for coding, AoPS and TheoremQA for theorem tasks, and Robotics and Stack Overflow for StackExchange tasks.Source: §C.4 · pp. 17–18
  • The authors suggest that coding and theorem queries may require longer reasoning because they are more complex and challenging than StackExchange queries.Source: §C.4 · p. 18

C.5 Comparison with Teacher DeepSeek-R1

ReasonRank (32B) performs comparably to DeepSeek-R1 overall on BRIGHT and exceeds it on seven datasets. Further ranking gains are reported with stronger retrieval and smaller sliding windows, while case studies illustrate varied reasoning patterns.

C.5 Comparison with Teacher DeepSeek-R1

  • ReasonRank’s training labels use DeepSeek-R1 to mine passages based on gold answers, unlike pure teacher distillation, which does not use gold answers.The comparison reranks ReasonIR’s top 20 passages, rather than the top 100, because of DeepSeek-R1’s time and API cost.Source: §C.5 · pp. 18–19
  • ReasonRank (32B) scores 34.18 versus DeepSeek-R1’s 34.13 and surpasses it on seven BRIGHT datasets.The reported datasets are Economics, Robotics, Biology, Sustainable Living, Pony, AoPS, and TheoremQA-Q.Source: Table 9 · p. 19, §C.5 · pp. 19–20

C.6 Further Ranking Enhancement

  • With RaDeR and BM25 hybrid retrieval, ReasonRank (32B) reaches 39.08, one point above results based on ReasonIR.The authors attribute the improvement to higher-quality retrieval results; Table 10 covers the further ranking enhancements.Source: §C.6 · p. 20, Table 10 · p. 20
  • A window size of 10 and stride of 5 yields a 1.7-point improvement; ReasonRank (32B) averages 40.6 and is reported as BRIGHT leaderboard SOTA on August 9, 2025.The original settings used a window size of 20 and stride of 10; the authors suggest smaller windows may reduce ranking-context difficulty.Source: §C.6 · p. 20, Table 10 · p. 20

D Case Studies of ReasonRank (7B)

E Use of AI Assistants

D Case Studies of ReasonRank (7B)

  • Other cases show short reasoning for Robotics, repeated checking of top-ranked passages for LeetCode, and an initial attempt to solve TheoremQA-Theorems problems.The TheoremQA-Theorems case then checks candidate passages.Source: Table 13 · p. 22, Table 14 · p. 23, Table 16 · p. 25
Loading 2508.07050v3…