Source-linked AI summary
Why Do Reasoning Models Lose Coverage? The Role of Data and Forks in the Road
Ngoc-Hieu Nguyen, Parshin Shojaee, Phuc Minh Nguyen, Nan Zhang, Chandan K Reddy, Khoa D Doan, Rui Zhang
TL;DR
Reasoning-model post-training can improve pass@1 while shrinking pass@k coverage, and the causes of this tradeoff remain unclear. This paper tests a data-centric explanation using controlled decision-point case studies and finds that forks in the road correlate with shrinkage, which diversity-aware data and decoding can partially mitigate.
Problem
Coverage shrinkage—higher pass@1 alongside lower pass@k—remains poorly understood, particularly whether fine-tuning data contributes beyond post-training optimization dynamics.
Method
The paper uses controlled graph-navigation and mathematical-reasoning case studies to isolate how indecipherable decision points and alternative reasoning paths affect post-training.
Results
Coverage shrinkage strongly correlates with decision-point structure in training data, while per-problem diversity design and diversity-promoting decoding partially recover lost coverage.
Takeaways & Limitations
The findings identify forks in the road as a data-centric driver of coverage shrinkage and support diversity-aware data and decoding as practical control strategies.
Takeaways & Limitations
Effectively operating across both instruct and backtracking/thinking reasoning modes remains an open problem.
Abstract
from arXiv · showhide
Recent progress in large language models has led to the emergence of reasoning models, which have shown strong performance on complex tasks through specialized fine-tuning procedures. While these methods reliably improve pass@1 accuracy, prior works have observed that they show a coverage shrinkage behavior, where pass@k degrades relative to the base model. In this paper, we investigate the reasoning shrinkage arise under SFT-based post-training. We hypothesize that this behavior is driven by properties of the fine-tuning data, specifically related to decision points or "forks in the road" scenarios where model faces indecipherable patterns with multiple valid reasoning paths. To test this hypothesis, we design controlled case studies that simulate such decision-point settings, spanning indecipherable nodes in graph branching, and reasoning modes. By tracking post-training dynamics in these settings, we find that the shrinkage phenomenon is tightly correlated with the prevalence of decision-point scenarios in the training data. We also demonstrate that this shrinkage behavior can be partially mitigated through targeted data synthesis design of decision-points, and a more systematic diversity-encouraging decoding mechanism. Our findings identify data-centric factors as a key driver of shrinkage in reasoning models and highlight diversity-aware designs as an effective lever for controlling it.
1 Introduction
The paper argues that coverage shrinkage—higher pass@1 but lower pass@k—arises substantially from decision-point structures in fine-tuning data, where models commit to fewer valid reasoning paths. Controlled case studies and data-design strategies investigate and mitigate this data-driven collapse.
- Motivation: Coverage shrinkage describes improved pass@1 accompanied by degraded pass@k, reflecting reduced diversity and coverage in learned reasoning procedures.The paper frames explaining this counterintuitive behavior as an open research question.
- Data-centric hypothesis: The paper shifts attention from post-training algorithms to data structure, hypothesizing that forks in the road drive shrinkage.These decision points contain multiple valid, indistinguishable reasoning paths requiring the model to commit to one.
- Controlled case studies: Controlled case studies examine decision points in graph navigation with indecipherable branching nodes and mathematical reasoning with multiple valid solution modes.The graph task requires traversing a star graph from a start node to a target while encountering branching points that provide no information.
- Mitigation strategies: Per-problem coverage of alternative decisions can significantly control shrinkage compared with distributing diversity across problems.This finding motivates synthetic data design that explicitly accounts for decision-point structure and within-problem coverage.
- Main contributions: The study identifies forks-in-the-road patterns in fine-tuning data as a key driver of coverage shrinkage through targeted case studies and training-dynamics analysis.Its stated aim is to provide a systematic, data-centric understanding of coverage shrinkage in reasoning post-trained models.
2 Preliminaries
This section defines coverage through pass@k and outlines how repeated sampling explores reasoning chains at test time. It also introduces SFT-based post-training, where datasets typically provide only one or a few annotated reasoning chains per problem.
- Coverage Definition: Coverage is the highest performance achievable when solutions are sampled repeatedly, commonly measured by pass@k.Coverage is described as a primary metric for evaluating test-time scaling.
- Test-Time Reasoning: In chain-of-thought generation, sampled reasoning chains can differ in phrasing, format, or the reasoning skills they invoke.These differences enable exploration of reasoning at test time.
- Post-training and Coverage Shrinkage: SFT reasoning fine-tuning uses triples (x_i, τ_i, y_i), but most problems have only one or a small number of annotated reasoning chains.The dataset represents each problem instance, reasoning chain, and final answer as x_i, τ_i, and y_i, respectively.
3 A Data-Centric View: Forks in the Road
The section frames reasoning as exploratory selection among multiple candidate strategies, then argues that training data often hides alternative paths and selection rationales. This missing rationale at decision points may cause models to commit to narrow, spuriously chosen trajectories, reducing inference-time coverage.
- Motivation: Reasoning requires exploring multiple candidate strategies, evaluating them, and selecting a promising path rather than merely executing one direction.The section characterizes strong problem solving as diverse approach generation plus path evaluation and selection.
- Data problem: Current reasoning fine-tuning datasets exhibit survivorship bias by exposing models mainly to one successful reasoning path per problem.Alternative strategies and the rationale for selecting the observed path are often hidden, creating a “missing rationale” problem.
- Fork-in-the-road Hypothesis: When viable paths exist but training obscures selection rationale at decision points, models may fail to learn a generalizable mechanism for resolving ambiguity.The proposed hypothesis says models instead commit to subsets of available paths using spurious, non-causal cues.
- Fork-in-the-road Hypothesis: The resulting suppression of alternative trajectories is proposed to produce coverage collapse at inference time.The paper tests this hypothesis with controlled case studies isolating decision-point structures in SFT data.
4 Experimental Framework
The experiments isolate decision points in controlled reasoning settings and show that their presence and data-level structure drive coverage shrinkage. Across graph navigation and reasoning-mode selection, decision points produce overconfident, brittle path choices that reduce broad coverage.
- Controlled decision-point setting: Controlled Forward and Reverse variants isolate coverage shrinkage by differing only in whether post-training includes decision points.Forward requires choosing among multiple continuations when only one leads to a correct solution; Reverse removes these decision points.
- Controlled decision-point setting: When decision points are removed, both models reach near-perfect pass@k across all k and remain stable during extended training.This broad coverage contrasts with the divergence observed in the Forward setting.
- Decision-point behavior: Training with decision points sharply increases confidence at both correct and incorrect branches, producing uncalibrated decisions that collapse coverage.Confidence rises throughout SFT epochs but is not selective between correct and incorrect choices.
- Robustness across post-training methods: The same coverage shrinkage appears under RLVR and is more pronounced than under SFT, implicating data structure and decision points beyond the learning algorithm.The comparison is reported for forward versus reverse settings.
- Spurious cues for branch selection: Semantically equivalent prompt perturbations can significantly shift branch selection, indicating that models learn spurious surface cues for reasoning-path decisions.The perturbations shuffle variable dependencies while preserving the underlying logic.
- Reasoning-mode diversity: With identical reasoning-mode ratios, problem-level diversity keeps coverage stable, whereas data-level diversity improves pass@1 while degrading pass@k.The difference arises from how modes are distributed across problems rather than from the overall amount of diversity.
5 Discussion on Data-inspired Shrinkage Mitigation Strategies
The discussion attributes coverage shrinkage to data-driven locking of alternative reasoning paths behind spurious decision points and examines mitigation through decision-point-aware data design and inference-time prefix perturbation. Experiments indicate that how diversity is distributed across problems matters, while Top-8 prefix sampling can recover coverage at larger k.
- Data Diversity Design: Coverage shrinkage is a data-driven artifact in which models lock alternative reasoning paths behind spurious decision points.The proposed mitigation strategies directly target these decision points.
- Data Diversity Design: Per-problem coverage of alternative decisions controls shrinkage more effectively than distributing the same diversity across problems.Post-training data design should account explicitly for reasoning decision-point structure and its coverage among problems.
- First-token Manipulation: Inference-time prefix perturbation tests whether alternative reasoning strategies remain present but locked behind collapsed decision points.The intervention forces generation from different high-probability initial reasoning tokens instead of relying on default generation.
- First-token Manipulation: Top-8 prefix sampling mitigates coverage shrinkage and improves pass@k at larger k.This result is reported for prefix perturbation in Figure 12.
6 Conclusion
The paper argues that coverage shrinkage in post-training reasoning models is primarily data-driven rather than caused by optimization dynamics or post-training algorithms. It attributes shrinkage to “forks in the road” in fine-tuning data, where multiple valid reasoning paths exist but the rationale for choosing among them is obscured.
- 6 Conclusion: Coverage shrinkage is highly driven by decision points in fine-tuning data where multiple valid reasoning paths exist without clear selection rationale.This data-centric hypothesis contrasts with prevailing explanations that blame optimization dynamics and post-training algorithms.
A Details on Synthetic Experiment · A.1 Task design
The synthetic task tests variable manipulation and dependency-structure planning by requiring models to compute a target variable from functional dependencies. Its path-star topology creates a decision point for selecting the relevant dependency path, while solutions can proceed forward with or backward without that decision point.
- A.1 Task design: The task defines functional dependencies among variables and trains models with intermediate computation steps to determine a designated target value.It combines chains of variable manipulation with planning over a dependency structure.
- A.1 Task design: An example query defines chained variables from n = 10 and asks for the resulting value of x.The example includes multiple dependency branches before reaching the target.
- A.1 Task design: The dependency graph is a path-star with a central root, multiple outgoing paths, and a target at the end of one unique path.Solving requires identifying the relevant dependency path and computing its intermediate values correctly.
- A.1 Task design: Prompts use the Alpaca instruction format, asking the model to solve a math problem and place the final answer within boxed notation.The input supplies a system of variable definitions and a target query.
- A.1 Task design: The paper presents forward and reverse solution variations for the same target computation.The forward solution is explicitly labeled as containing a decision point, whereas the reverse solution is labeled as lacking one.
- A.1 Task design: The forward derivation computes variables step by step and concludes with s = 110.This example is framed as a solution with a decision point.
A.2 Experimental settings · A.3 Probing confidence at decision points · A.4 Reinforcement Learning with Verification Reward (RLVR)
The experiments use controlled star-graph equations to study decision points, probing which branch models select and how confidently. RLVR experiments further test whether coverage shrinkage persists after supervised fine-tuning on data with or without decision points.
- A.2 Experimental settings: The dataset contains 6,400 training samples and 1,000 test samples generated from two-branch star graphs with root-to-leaf paths of length 10.Each example uses generated variables, equations, text templates, questions, and ground-truth solutions.
- A.2 Experimental settings: Qwen-2.5-0.5B and EvoLM-1B are supervised-fine-tuned for 16 epochs with lr=2e-5.A solution is correct when its boxed output matches the target variable’s value.
- A.2 Experimental settings: Evaluation marks a solution correct when the boxed output matches the target variable’s value.This criterion is applied to both supervised-fine-tuned models.
- A.3 Probing confidence at decision points: The probing procedure measures the probability of the next token at decision points in the forward solution.The prompt asks the model to solve a math problem and place its final answer within \boxed{}.
- A.3 Probing confidence at decision points: Given a variable-equation system and a target variable, the model is prompted to compute variables step by step.The input provides the equations and a known value, while the response begins by describing the sequential computation.
- A.3 Probing confidence at decision points: Predicting the first-step variable name identifies the branch the model will subsequently take and its confidence in following that branch.This links the model’s next-token prediction to branch selection at indecipherable decision points.
- A.4 Reinforcement Learning with Verification Reward (RLVR): Applying GRPO after one SFT epoch shows that coverage shrinkage also arises under RLVR when models are trained on forward data with decision points or reverse data without them.The comparison is reported in Figure 13 for models pretrained on forward and reverse (-DP) solutions.
B Details on Experiment with Reasoning Modes · B.1 Natural Language vs. Code Reasoning
The experiment compares natural-language and code-based mathematical reasoning using filtered GSM8K samples from two OpenMathInstruct datasets. Sampling uses controlled vLLM settings across three backbone models, generating 64 solutions per test sample.
- B.1 Natural Language vs. Code Reasoning: The experiment collects supervised fine-tuning samples from OpenMathInstruct-1 and OpenMathInstruct-2.Both datasets provide mathematical reasoning solutions used to construct the experiment’s training samples.
- B.1 Natural Language vs. Code Reasoning: OpenMathInstruct-2 contains natural-language solutions, whereas OpenMathInstruct-1 contains Python code solutions.The code solutions were generated by Mixtral-8x7B.
- B.1 Natural Language vs. Code Reasoning: The experiment filters questions to GSM8K before combining examples from the two datasets.This filtering creates the evaluation domain for comparing the two reasoning styles.
- B.1 Natural Language vs. Code Reasoning: The sampling procedure generates solutions in both natural-language and code reasoning styles.The supplied passage states that solutions are sampled for each reasoning style, NL versus Code.
- B.1 Natural Language vs. Code Reasoning: Inference uses the default vLLM server with temperature = 1.0, top p = 0.95, and max new tokens = 1024.These settings define the sampling implementation and generation limits.
- B.1 Natural Language vs. Code Reasoning: 64 solutions are generated per test sample.This is the specified number of sampled solutions for each test example.
- B.1 Natural Language vs. Code Reasoning: The experiment uses Qwen-2.5-0.5b, EvoLM-1B-160BP-8M42M, and EvoLM-4B-160BP-8M42M as backbone models.The passage lists these three models as the backbones used in the experiment.
B.2 Distilled Reasoning Models
This section evaluates DeepSeek-R1-Distilled reasoning models under controlled vLLM decoding settings. Figure 14 compares responses generated from different thinking prefixes across four reasoning benchmarks.
- Decoding Configuration: The experiment uses vLLM for efficient inference with specified decoding parameters.The configuration includes temperature 0.6, top-p 0.95, top-k -1, and 32768-token limits.
- Decoding Configuration: Decoding uses temperature 0.6, top-p 0.95, top-k -1, and 32768-token maximum and model-length limits.Remote code is trusted during inference.
- Thinking Prefixes: Figure 14 compares DeepSeek-R1-Distilled model performance when generating responses from different thinking prefixes.The evaluation spans GSM8k, MATH500, AIME24, and AIME25.
- Thinking Prefixes: The benchmark suite comprises GSM8k, MATH500, AIME24, and AIME25 reasoning benchmarks.The cited benchmarks are associated with Cobbe et al. (2021), Lightman et al. (2023), Zhang & Math-AI (2024), and Zhang & Math-AI (2025), respectively.
B.3 Examples of models’ responses
The examples show that changing the thinking-prefix wording can substantially alter models’ accuracy and response length. The responses also illustrate how prefix-dependent reasoning may produce either concise correct answers or lengthy, uncertain deliberation and errors.
- Base-7 addition examples: With the default prompt, DS-LLaMA-8B solved the base-7 addition correctly as 116 in base 7.Its response directly added the digits and carried the quotient after computing 6 + 5 = 11.
- Prefix-dependent behavior: Prompt prefixes produced up to 20% performance variance and up to 5.8x response-length variance across mathematical benchmarks.The figure compares DS-Qwen-1.5B, DS-Qwen-7B, and DS-LLaMA-8B on GSM8K, MATH500, AIME24, and AIME25.
- Base-7 addition examples: With the “Okay” prefix, DS-LLaMA-8B instead concluded that 66 + 50 equals 146 in base 7, after exploring multiple solution methods.The model converted both numbers to base 10 before also checking direct base-7 addition, with both methods yielding 146.
- Capital-of-Greece examples: For the Greece question, DS-Qwen-1.5B answered “Athens” concisely with both the “The” and “Okay” prefixes’ final responses.The “Okay” response’s reasoning was substantially more uncertain, considering Thessaloniki and conflicting possibilities before reaching the answer.