QReason: Query-Focused Decoupled Chain-of-Thought for Efficient Passage Reranking

1 Introduction

• For the same BRIGHT query, ReasonRank’s reasoning traces across windows exceed 90% average semantic similarity, indicating that much of the repeated trace could be reused.The traces contain global query reasoning and local passage reasoning, but existing methods regenerate both for each window.Source: §1 · p. 2
• QReason generates a ranking-oriented, query-focused reasoning query once and reuses it across windows with a non-reasoning reranker.This decouples global query reasoning from window-specific passage assessment.Source: §1 · p. 2
• The rewriter is optimized with supervised fine-tuning on query and relevant-passage pairs, followed by reinforcement learning with listwise and pointwise rewards.The rewards align rewriting with reranking quality while grounding it in relevant-passage evidence.Source: §1 · p. 2
• QReason’s strongest Qwen3.5-35B-A3B instantiation achieves superior ranking accuracy and computational efficiency compared with state-of-the-art reasoning-based rerankers.Experiments also show that QReason consistently improves non-reasoning rerankers.Source: §1 · p. 2

2 Related Work

• Explicit reasoning methods improve effectiveness on reasoning-intensive IR tasks but typically generate reasoning traces during inference, adding computational cost and redundancy.The passage distinguishes trace distillation methods from methods that optimize rerankers with GRPO.Source: §2 · p. 2
• Reasoning-enhanced query rewriting has been used for retrieval through direct prompting of large LLMs or reinforcement-learning-trained smaller rewriters.Examples named include GPT-4, LLaMA3-70B-Instruct, DeepRetrieval, and TongSearch.Source: §2 · p. 3
• Retrieval-oriented reasoning rewrites have demonstrated retrieval effectiveness but are primarily designed for the retrieval stage and do not transfer well to reranking.Source: §2 · p. 3

3 Task Formulation

• Context-length constraints divide candidate passages into overlapping windows for listwise reranking.Source: §3 · p. 3
• Existing reasoning-based rerankers independently generate a new reasoning trace for each window before producing its passage ordering, creating substantial redundancy.The reranker’s permutation orders passages in that window by estimated relevance to the query.Source: §3 · p. 3
• QReason decouples reusable query reasoning from window-level ranking by generating one reasoning query and reusing it across windows.The reranker is non-reasoning and uses the rewritten query for each window.Source: §3 · p. 3

4 Method

• Retrieval-oriented rewrites can introduce content that creates false relevance signals, and GPT-4 and TongSearch rewrites have unstable effects when applied to reranking.Reranking is described as more sensitive to fine-grained query-document interactions under a precision-oriented objective.Source: §4 · p. 3

4.1 Relevance-Grounded Supervised Tuning

• Relevance-Grounded Supervised Tuning uses gold relevant passages to guide a teacher-generated reasoning query that distills the chain connecting the original query to ranking-relevant intent.The rewriter is fine-tuned on input-output pairs consisting of the query and relevant passages as input and the reasoning query as output.Source: §4.1 · p. 3

4.2 Rank-Aligned Dual-Reward Refinement

• Rank-Aligned Dual-Reward Refinement combines a listwise NDCG@10 reward from a fixed reranker with a pointwise reward that keeps rewrites supported by relevant-passage evidence.The pointwise reward is intended to guard against overfitting to a fixed reranker and generations weakly related to relevance evidence.Source: §4.2 · p. 4

4.3 Training and Inference

• During reinforcement-learning rollouts, relevant passages are hidden from the policy model and used only for reward computation; at inference, the rewriter generates q′ from the original query alone.The reranker receives the original query with q′ appended and performs non-reasoning ranking.Source: §4.3 · p. 5

5.1 Dataset

• QReason’s training data derives from reasonrank_data_13k; BRIGHT is evaluated with NDCG@10, with supplementary FreshStack and NeuCLIRBench results reported in Appendix D.3.ReasonIR (8B) retrieves the top 100 passages using GPT-4-rewritten BRIGHT queries.Source: §5.1 · p. 5, §5.2 · p. 6

5.2 Baselines

• The comparison covers task-specific and general LLM rerankers, including reasoning and non-reasoning models, plus GPT-4, DeepRetrieval, TongSearch, and QReason rewrites.Rewritten queries are not paired with task-specific rerankers trained for original-query inputs.Source: §5.2 · pp. 5–6

5.3 Main Results

• 36.93 average score: QReason with Qwen3.5-35B-A3B is best overall on BRIGHT, surpassing Rank1 (32B), Rank-K (32B), and ReasonRank (32B).The comparison uses results reproduced under the same experimental environment.Source: Table 1 · p. 6, §5.3 · p. 6
• QReason improves original-query performance by 2.43, 1.05, and 1.44 points with Qwen2.5-32B-Instruct, Qwen3-30B-A3B-Instruct-2507, and Qwen3.5-35B-A3B, respectively.The authors report more stable behavior than existing rewriting baselines; GPT-4 drops 2.61 points with Qwen3-30B-A3B-Instruct-2507.Source: §5.3 · p. 6

5.4 Ablation Study

• Removing RGST or RADR costs 1.68 or 1.27 Avg. NDCG@10 points; removing relevant passages costs 2.20 points; removing pointwise or listwise rewards costs 1.33 or 1.28 points.The authors describe the two training stages and reward terms as complementary; relevant-passage guidance grounds rewrites in evidence and prevents drift from ranking facets.Source: §5.4 · p. 7

5.6 Parameter Sensitivity

• RADR performance rises and then falls as λ increases, with the best result at λ = 0.6.The authors report that a moderate pointwise reward can regularize the rewrite’s semantic scope without weakening its reranking utility.Source: §5.6 · pp. 7–8

5.5 Efficiency Analysis

• 7.9× and 2.6× speedups: QReason with Qwen3.5-35B-A3B and Qwen2.5-32B-Instruct, respectively, outpaces ReasonRank (32B) end-to-end on six BRIGHT subsets.With R ≈ R′, the efficiency gain mainly comes from avoiding reasoning generation in the remaining K −1 windows; appending q′ adds some input-prefill cost.Source: §5.5 · p. 7, Table 3 · p. 8

5.7 Case Study

• In the case study, QReason includes key input-output and supply-use evidence, while its rewrite also adds broader content beyond the gold evidence.The comparison describes TongSearch as covering part of the same evidence space while adding more specific allocation details not explicitly supported by the gold evidence.Source: §5.7 · p. 8, Table 4 · p. 8

• QReason reuses a ranking-oriented reasoning query across sliding windows, and experiments show it consistently improves different LLM rerankers without redundant reasoning generation.Source: §6 · pp. 8–9

• Listwise and pointwise relevance rewards do not explicitly capture coverage, faithfulness, or redundancy.Source: Limitations · p. 9

Ethical Considerations

• The work uses public datasets and pretrained models, collects no new user or sensitive data, and is intended for research rather than high-stakes deployment.Source: Ethical Considerations · p. 9

A.1 Evaluation Benchmark

• BRIGHT contains 1,384 diverse real-world queries and tests retrieval beyond lexical or shallow semantic matching through implicit query-document connections.Source: §A.1 · p. 11

A.2 Baselines

• Baselines include bi-encoder ReasonIR-8B, pointwise RankT5 and Rank1, listwise RankZephyr and REARANK, and setwise Rank-R1.Source: §A.2 · pp. 11–12
• Rank-K and ReasonRank use sliding-window reasoning reranking, while DeepRetrieval generates queries using retrieval metrics as rewards.Source: §A.2 · p. 12

B.1 Training Data Construction

• The authors sample 6,000 of 13,549 reasonrank_data_13k instances for RGST and use the remaining 7,549 for RADR.Source: §B.1 · p. 12

B.2 Training Details

• RGST fine-tunes Qwen2.5-7B-Instruct on teacher-synthesized reasoning queries, using each original query and its gold relevant passages as input; Figure 3 covers synthesis and training prompts.Source: §B.2 · p. 12, Figure 3 · p. 13
• RADR trains the rewriter with GRPO from the RGST checkpoint, hiding gold passages from the policy and using them for reward computation; Figure 4 shows the rewriting prompt.Source: §B.2 · p. 13, Figure 4 · p. 13

B.3 Experimental Setup and Inference Details

• At inference, the rewritten query is concatenated after the original query, and the rewriter and reranker use the RADR rewriting and listwise-reward reranking prompts.Source: §B.3 · p. 13
• General LLM rerankers use a window size of 20 and step size of 10, while baseline rerankers follow reported configurations and are reproduced in a unified environment.Source: §B.3 · p. 13
• QReason's rewriter and reranker use greedy decoding at temperature 0, making results nearly identical across multiple runs.Source: §B.3 · p. 14

C.1 Parameter Sensitivity

• A small λ lets the listwise reward dominate and may cause semantic drift, while a large λ weakens listwise alignment with the reranking objective.Source: §C.1 · p. 14
• λ = 0.6 gives the best NDCG@10 gain of reranking with rewritten queries over original queries; performance first rises and then declines as λ increases.Source: Figure 7 · p. 14, §C.1 · p. 14

C.2 Empirical Efficiency Analysis

• About 1/65 of ReasonRank (32B) latency, the QReason rewriter accounts for approximately 3.9% of total QReason latency with Qwen2.5-32B-Instruct.The latency comparison includes one-time rewriting and downstream reranking costs.Source: §C.2 · p. 14

D More Experiments

• The additional experiments examine QReason's effectiveness and generalizability in broader settings, with all results reported using NDCG@10.Source: D More Experiments · p. 15

D.1 Extended Main Results on BRIGHT

• QReason-generated reasoning queries yield consistent performance gains over original queries across five additional general LLM rerankers on BRIGHT.Table 6 compares original-query and QReason-query reranking with Qwen2.5-72B-Instruct, Llama3.3-70B-Instruct, Qwen3-32B, Qwen3-30B-A3B, and Qwen3.6-35B-A3B.Source: §D.1 · p. 15

D.2 Performance of QReason with Smaller General LLM Rerankers

• Under the main setting, smaller general LLM rerankers show limited ranking effectiveness, possibly from lacking task-specific reranking training; the follow-up uses window size 10 and step size 5.The passage identifies Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Qwen3-8B, Qwen3.5-9B, Qwen2.5-14B-Instruct, and Qwen3-14B as evaluated rerankers.Source: §D.2 · p. 15

E Case Details

• The Section 5.7 case details include the original query, gold relevant passages, and rewrites from QReason and TongSearch (7B), shown in Figures 8–14.URLs in relevant passages are replaced with [URL].Source: E Case Details · p. 16

G Artifact Licenses and Intended Use

• The work uses datasets, benchmarks, models, and software under their providers' terms, and intends its QReason rewriter and generated queries for research.Existing artifacts are used for research purposes, including training, reranking evaluation, and reproducibility analysis.Source: G Artifact Licenses and Intended Use · p. 16

D.3 Results on Additional Benchmarks

• On FreshStack, QReason gains 2.23, 2.53, and 2.26 NDCG@10 points over original queries with Qwen2.5-32B-Instruct, Qwen3-30B-A3B-Instruct-2507, and Qwen3.5-35B-A3B.With Qwen3.5-35B-A3B, QReason reaches average NDCG@10 33.63, 0.05 points below ReasonRank (32B).Source: §D.3 · p. 15, Table 8 · p. 18
• On NeuCLIRBench, QReason with Qwen3.5-35B-A3B reaches average NDCG@10 31.61, 1.01 points above ReasonRank (32B).QReason improves averages for all three evaluated rerankers: 29.73 to 31.13 (+1.40) with Qwen2.5-32B-Instruct, 30.24 to 31.11 (+0.87) with Qwen3-30B-A3B-Instruct-2507, and 30.68 to 31.61 (+0.93) with Qwen3.5-35B-A3B.Source: §D.3 · p. 16, Table 9 · p. 18