Source-linked AI summary
Beyond Polarization: The Generative Constraint of Chain-of-Thought in Pointwise Reranking
Xiaoyang Chen, Jie Liu, Haijin Liang, Haibo Shi, Jin Ma, Ben He, Yingfei Sun, Dezhi Ye
TL;DR
Pointwise Chain-of-Thought models typically trail direct scoring in document reranking, but whether targeted training can close this gap is unclear. This study tests the gap across scales and stress-test interventions, finding that classification and absolute scores improve while the relative ranking deficit persists. The authors suggest that routing continuous relevance semantics through discrete text constrains ranking-signal resolution within the pointwise paradigm.
Problem
Pointwise Chain-of-Thought models typically underperform direct scoring, and it remains unclear whether targeted training can bridge this gap.
Method
The study evaluates Qwen models across scales and settings, then applies reinforcement learning, fine-grained supervision, and architectural decoupling to test proposed causes of the gap.
Results
The relative ranking gap remains stable across model and data scales and persists after interventions improve classification accuracy and absolute scores.
Takeaways & Limitations
Within pointwise scoring, discrete text generation may constrain the resolution of ranking signals, leaving a stable bottleneck under current standard methods.
Takeaways & Limitations
The study does not exhaustively rule out task, data-distribution, training-regime, and rationale-style confounders, and leaves several repair paths uncovered.
Abstract
from arXiv · showhide
In pointwise document reranking, Chain-of-Thought models typically underperform direct scoring models. While existing diagnostics attribute this to inferior classification, score polarization, or calibration breakdown, whether targeted training can bridge this gap remains unclear. Our empirical study first confirms that this gap is stable across scales up to 32B parameters, ruling out model and data capacity confounders. We then apply stress tests utilizing reinforcement learning, fine-grained supervision, and architectural decoupling to explicitly repair these deviations. Although these interventions improve classification accuracy and absolute scores, the relative ranking gap persists. These findings suggest that, within the pointwise scoring paradigm, routing continuous relevance semantics through discrete text constrains ranking signal resolution, revealing a bottleneck that is stable and difficult to overcome under current standard methods, rather than an easily resolvable training bias.
1 Introduction
The study asks whether targeted training can close the pointwise reranking gap between Chain-of-Thought and direct scoring. Across scales and interventions, the gap remains, suggesting a constraint from routing relevance through discrete text.
- Pointwise CoT typically underperforms direct scoring despite proposed explanations involving classification, score polarization, and calibration.
- The study evaluates Qwen models from 0.6B to 32B across benchmarks in a two-stage empirical framework.
- The relative CoT-versus-direct-scoring gap remains stable across model sizes, data volumes, supervision settings, and reasoning sources.
- Reinforcement learning, fine-grained supervision, and prompt decoupling improve classification accuracy and absolute scores but do not eliminate the ranking gap.
- The analysis suggests that discrete text generation constrains the resolution of continuous ranking signals within pointwise scoring.
2 Experimental Setup
The experiments compare direct scoring and pointwise CoT across model families, scales, benchmarks, and supervision variants. CoT training data use either DeepSeek-R1 or Gemini-3-Pro distilled rationales, with optional fine-grained relevance labels.
- Qwen2.5 and Qwen3 models spanning seven parameter scales form the main experimental backbone, with Llama-3.1-8B-Instruct as a cross-family control.
- The study compares Direct Scoring (noCoT) with Pointwise CoT for pointwise reranking.
- For both variants, the ranking score is derived from candidate-token logits at the decoding position immediately following Answer:.
- The DS dataset contains approximately 380K MS MARCO samples with DeepSeek-R1-distilled reasoning chains, while the Gem variant uses Gemini-3-Pro chains and binary plus 0–4 relevance outputs.
3 Stage I: Persistence of the Gap
Stage I tests whether the CoT disadvantage reflects model scale, data scale, rationale quality, or training dynamics. Absolute CoT performance improves, but direct scoring retains a relative advantage across the evaluated settings.
- Insensitivity to Model Scaling: Scaling Qwen3 from 0.6B to 32B raises SFT DS CoT’s BRIGHT score from 9.6 to 23.0, an increase of over 2.4×.
- Insensitivity to Model Scaling: NoCoT consistently leads across model sizes; on DL19, Qwen3-32B scores 73.7 with SFT noCoT versus 70.1 with SFT DS CoT.
- Insensitivity to Data Scaling: Across 10K, 100K, 200K, and full-instance training scales, the performance gap remains stable after noCoT overtakes CoT as data volume expands.
- Insensitivity to Rationale Quality: Gemini-supervised CoT does not fundamentally improve ranking: on DL19 with Qwen3-14B, it scores 65.7 versus 66.8 for the DeepSeek-R1 variant.
- Asymmetric Gains from SFT: SFT produces larger gains for noCoT and leaves a 1–4-point margin across model scales, despite comprehensively improving both paradigms.
4 Stage II: Stress Tests on the Gap
Three targeted stress tests address hypothesized causes of the pointwise CoT ranking gap: classification accuracy, score polarization, and calibration breakdown. Although the interventions improve some underlying signals, the relative NDCG@10 deficit remains.
- Reinforcement Learning Alignment: GRPO uses structural validity, answer format, and classification correctness in a weighted reward, with classification receiving γ =0.8.The reward applies α=0.1 to structural validity and β =0.1 to answer format.
- Reinforcement Learning Alignment: GRPO aligns CoT classification with direct scoring, but NDCG@10 still trails across all evaluation units.The deficit narrows from −3.1 to −0.3 for 14B on DL19, while noCoT retains a 0.3–3.7 lead overall.
- Architectural Decoupling: Architectural decoupling generates a rationale with Mg, then feeds it to a separately scored Ms that predicts only the score token.Ms is continually finetuned from Mg and receives the pre-generated rationale in addition to the original input.
5 Analysis
The analysis tests whether the CoT ranking gap is robust across random seeds and model families, then probes how generated rationales affect ranking signals. It interprets the results as evidence for a discrete-text bottleneck that limits recoverable ranking resolution.
- Robustness and Generalization: The intergroup NDCG@10 gap consistently exceeds intra-group variance across three random seeds and all benchmarks.The test uses Qwen3-14B under noCoT and +Decouple settings.
- Robustness and Generalization: noCoT consistently outperforms CoT on DL19/20 and BRIGHT with Llama-3.1-8B-Instruct, showing the gap generalizes across model families.The replication uses identical settings to the core comparison.
- Discrete-Text Bottleneck: In the Label-CoT-Label probe, ranking by the label after the rationale falls from 72.7 to 68.2 on DL19 and from 22.0 to 19.5 on BRIGHT.The labels and generation paths are identical; only the intervening rationale differs in position.
- Discrete-Text Bottleneck: The analysis frames CoT as routing a continuous relevance state through a discrete rationale, whose finite vocabulary limits the resolution recoverable for ranking.Classification needs only the threshold sign, whereas ranking depends on fine-grained residual relevance between close pairs.
- Discrete-Text Bottleneck: The three interventions target different deviations, but the rationale remains on the critical path and caps recoverable ranking signal.GRPO can match or exceed noCoT classification accuracy while NDCG still lags.
6 Conclusion
The study finds that pointwise CoT underperforms direct scoring across capacity, data scale, and reasoning quality. Stress tests alleviate specific deviations but do not eliminate the ranking deficit, suggesting a stable discrete-text bottleneck under current standard methods.
- Pointwise CoT underperformance relative to direct scoring remains stable across model capacity, data scale, and reasoning quality.
- Reinforcement learning alignment, fine-grained supervision, and architectural decoupling alleviate specific deviations but fail to eliminate the ranking deficit.
- Routing continuous semantics through discrete text constrains ranking-signal resolution within the pointwise scoring paradigm.
- The bottleneck is stable and difficult to overcome under existing standard methods, while novel architectures or training strategies remain open possibilities.
Limitations
The study’s limitations concern scope, incomplete isolation of training variables, and the restricted pointwise scoring setup. The authors also note that external application factors and alternative decoupling methods remain outside the study.
- Scope: The study cannot fully rule out confounding effects from task types, data distributions, rationale length, or rationale style.These factors are difficult to decouple within a single paper.
- Scope: The 200-sample human evaluation provides only limited guarantees about rationale quality.
- Pointwise Scope: The study fixes scoring at the token after Answer: and excludes variants allowing broader attention or aggregation over hidden states.
- Uncovered Variables: The stress tests address classification accuracy, score polarization, and calibration breakdown rather than exhaustively isolating every training variable.
- Uncovered Variables: NoCoT-to-CoT distillation, continuous regression heads, and ranking-oriented losses remain uncovered and may narrow the gap further.
- External Factors: The diagnostic perspective does not fully consider training-resource constraints or complexities of real-world applications.
B.1 Data Statistics and Characteristics
The distilled datasets differ in rationale style and length, while Gemini prompting adds fine-grained scores alongside binary labels. Human evaluation and a confusable-negative case study assess these differences.
- Dataset Statistics: Gemini rationales are 30% longer than DeepSeek-R1 rationales, averaging 1489 versus 1151 characters.
- Dataset Statistics: DeepSeek-R1 rationales use conversational stream-of-consciousness reasoning, whereas Gemini rationales enforce a strict structured reasoning paradigm.
- Distillation Prompt: Gemini-3-Pro is prompted to output both a fine-grained relevance score from 0–4 and a binary label for every passage.
- Distillation Prompt: The prompt evaluates helpfulness, directness, and alignment of entities, time, and scope before assigning each passage a score and binary label.
- Human Evaluation: Human annotators preferred Gemini-distilled rationales in 70% of 200 randomly sampled query-document pairs.
- Case Study: A case study contrasts a query about post-hearing disability-claim approval with a document about initial-application approval.
- Case Study: DeepSeek-R1 identifies the mismatch but meanders through external knowledge, while Gemini explicitly separates topic match from aspect mismatch.
C Implementation Details and Prompt Templates
The implementation details cover SFT and GRPO training, composite score computation for Mtag, and prompt templates enforcing consistent output formats.
- C.1 Training Details: Experiments run on a cluster of four NVIDIA H20 GPUs.
- C.1 Training Details: SFT uses full-parameter LLaMA-Factory fine-tuning with bfloat16 precision and explicit <think> and </think> tokenizer tokens for Qwen2.5.
- C.1 Training Details: GRPO selects training samples answered correctly between 2 and 6 times across eight inference passes.
- C.1 Training Details: GRPO initializes from the CoT-SFT checkpoint to preserve established reasoning formats and prior capabilities during exploration.
- C.2 Final Score Computation for Mtag: Mtag’s fine-grained score applies softmax to logits for relevance tokens 0–4, then computes and normalizes their expected value.
- C.2 Final Score Computation for Mtag: Mtag’s answer score normalizes the True probability against the True and False probabilities.
- C.2 Final Score Computation for Mtag: The fine-grained and answer scores are averaged with equal weight to form the final score.
- C.3 Prompt Templates: Strict prompt templates are used across three paradigms to maintain consistent output formats during data construction and inference evaluation.
D Data Scaling Results on Qwen2.5 Series
The Qwen2.5 data-scaling results follow the main-paper trends and further support the persistence of the CoT gap across model series and sizes.
- Data Scaling: Figure 3 reports data-scaling results for Qwen2.5 series models.
- Data Scaling: The Qwen2.5 trends are consistent with those observed in the main paper.
- Data Scaling: The results further confirm persistence of the CoT gap across different model series and sizes.
E Full Results of BRIGHT Benchmark
This section reports BRIGHT pointwise-reranking results across Qwen2.5 and Qwen3 models, comparing Direct Scoring with Pointwise CoT under Zero-Shot and SFT settings. Results are organized by evaluation unit and averaged across the benchmark.
- BRIGHT benchmark results: Table 8 reports nDCG@10 for each of the 12 BRIGHT evaluation units and their average across all units.The table supports detailed comparisons across evaluated models and methods.
- Model and scaling coverage: The comparison covers Qwen2.5 and Qwen3 series models of varying sizes.Figure 3 presents data scaling on the Qwen2.5 series.
- Methods and settings: Direct Scoring (noCoT) and Pointwise CoT are evaluated under both Zero-Shot and Supervised Fine-Tuning settings.Under SFT, rationales are distilled from DeepSeek-R1 and Gemini-3-Pro.