First Try Matters: Revisiting the Role of Reflection in Reasoning Models

1 INTRODUCTION

• The study addresses limited quantitative evidence on whether reflections correct errors or merely confirm earlier conclusions, despite mixed prior findings about their effects.Longer responses are associated with higher accuracy, while post-answer reflections are intuitively assumed to be productive.Source: §1 · pp. 1–2
• The study analyzes reflection patterns in eight reasoning models across five mathematical benchmarks, separating steps before the first candidate from subsequent reflections.An LLM-based extractor locates candidate answers in rollouts to support this comparison.Source: §1 · p. 2
• Across models and datasets, reflections are mostly confirmatory and rarely overturn a proposed candidate answer.Subsequent reasoning mainly reiterates or justifies the initial answer, challenging the view that reflections are the primary mechanism of self-correction.Source: §1 · p. 2
• 24.5% fewer tokens with less than a 2.9% accuracy drop from early stopping reflections at inference time.The method also allows the token-use and performance balance to be controlled through the early-stopping criteria.Source: §1 · p. 2
• Reflection-rich SFT data improve performance primarily by increasing first-candidate accuracy, rather than by teaching models to correct initially wrong answers through reflection.The authors hypothesize that additional reflections expose diverse reasoning paths and enrich the training distribution.Source: §1 · p. 2

2.1 REFLECTION EXTRACTION

• An LLM-based extractor identifies candidate answers and their positions in long chains of thought, defining reflections as content between successive candidates.The extracted candidates are organized by their positions for subsequent analysis.Source: §2.1 · p. 3

2.2 REFLECTION ANALYSIS

• Over 90% of reflections are confirmatory across tested models and datasets, while corrective reflections that improve performance are mostly below 2%.Figure 3 summarizes reflection types across models; the detailed dataset breakdown is in Appendix D.Source: Figure 3 · p. 4, §2.2 · p. 5

2.1 REFLECTION EXTRACTION

• The study collects 3,427 rollouts for each of eight reasoning models, spanning 7B to 685B and including models trained with reinforcement learning and distillation.Math500 is considered easier, with state-of-the-art models above 95% accuracy; AIME2024 and AIME2025 are harder, with pass@1 performance ranging from 30% to 80%.Source: §2.1 · p. 4
• Human evaluators judged 94% of candidate extractions correct across 100 sampled rollouts containing 426 candidates.Across four extractor variants, the distribution of the first candidate’s relative position was consistent for each model.Source: §2.1 · p. 4

2.2 REFLECTION ANALYSIS

• 16.8%–47.8% of tokens follow the first candidate answer, while reflections add 1.4%–3.5% accuracy across the five datasets.The comparison uses average accuracy across AIME2024, AIME2025, AMC, Olympiad Bench, and Math500.Source: Figure 4 · p. 5, §2.2 · p. 5
• On harder AIME2024 and AIME2025 problems, the first candidate appears later; on easier Math500 problems, it appears earlier, but models perform more reflections.The authors characterize this reflection pattern as not well aligned with task difficulty.Source: §2.2 · p. 5, Figure 5 · p. 5

3.1 TRAINING WITH VARYING AMOUNT OF REFLECTIONS

• Six cut-at-i SFT datasets vary reflection counts while holding each dataset to 28 million training tokens; cut-at-1 has 6,754 questions and cut-at-6 has 3,405.Rollouts are continued from the selected candidate answer to preserve coherence, and fewer than 0.5% are removed for a mismatched final answer.Source: §3.1 · p. 6
• Reflection-rich SFT rollouts yield higher accuracy and longer generations; cut-at-6 beats cut-at-1 by 4.05%, driven mainly by first-answer gains (+3.75%) rather than reflections (+0.3%).The cut-at-6 comparison averages Llama3.1-8B-Instruct and Qwen2.5-7B-Instruct; the first-answer/reflection breakdown is reported for Figure 7.Source: §3.1 · pp. 6–7, Figure 6 · p. 6, Figure 7 · p. 7
• +4.6% for the 32B model and +7.7% for the 7B model of post-RL performance gains came from first-answer accuracy, versus reflection gains of +0.3% and +0.1%.The comparison uses MiroMind-M1-RL models and their respective initializations.Source: §3.1 · p. 7, Figure 8 · p. 7

3.2 TRAINING WITH CORRECTIVE REFLECTION PATTERNS

• The corrective-reflection SFT datasets vary the share of problems using corrective rollouts from 0% to 100%, with confirmatory rollouts used for the remainder.The datasets are constructed from 6K problems that each have both a corrective and a confirmatory rollout.Source: §3.2 · p. 8
• Across corrective-reflection proportions, SFT models had similar accuracy and response lengths, and their probability of correcting an incorrect candidate, p(F →T), showed no improvement.The best-to-worst performance difference was 1.7%, and the maximum response-length difference was about 1K tokens.Source: §3.2 · p. 8, Table 1 · p. 8

Candidate Answer Detector

• CAD is trained to classify whether a rollout sentence contains a candidate answer, using question-and-sentence inputs and annotated examples.The CAD training sentences are labeled by gpt-oss-120b.Source: §4 · p. 8

Question-aware Adaptive Early-Stopping

• QRC uses the problem statement to choose whether to stop at the first candidate or allow more reflection, while CAD detects candidate-answer sentences during generation.The reported setting stops at the first candidate for QRC label 0 and at the third candidate otherwise.Source: §4 · pp. 8–9
• 24.5% fewer tokens with QRC comes with a 2.9% performance drop across five mathematical datasets.CAD alone saves 29.9% of tokens with a 3.8% drop; threshold changes provide additional token–performance trade-offs.Source: §4 · p. 9, Table 2 · p. 9, Figure 9 · p. 9

Inference-time scaling

• Inference-time scaling, including RLVR, has been studied as a route to stronger LLM reasoning on mathematical and coding benchmarks (Muennighoff et al., 2025; DeepSeek-AI et al., 2025).The passage also cites benchmark reports by Liu et al. (2025b), Ke et al. (2025), Xie et al. (2025), and Jain et al. (2025).Source: §5 · p. 9

Long chain-of-thought reasoning

• Accounts of long-CoT effectiveness emphasize reflection, verification, and revision, while other work notes verbose traces can add redundancy or support diverse-solution exploration (Yang et al., 2025b; Wang et al., 2025; Chen et al., 2025b; Zeng et al., 2025; Liu et al., 2025b;a).Source: §5 · p. 9
• The paper focuses on reflections within long CoTs, which the cited literature passage describes as underexplored across training and testing stages.Source: §5 · p. 9

6 CONCLUSIONS

• The paper concludes that reflections are mostly confirmatory yet helpful in training, and proposes early stopping to reduce excessive inference-time reflections while maintaining performance.The conclusion also says confirmatory reflections consume tokens while introducing marginal improvements.Source: §6 · p. 10

SYSTEM PROMPT 1

• The first prompt extracts candidate answers line by line, applies only trivial target-specific conversions, and emits line-number/value tuples without judging correctness.It also instructs the extractor to lock the problem’s target and required output form before evaluating candidate lines.Source: SYSTEM PROMPT 1 · pp. 14–15

SYSTEM PROMPT 2

• The second prompt counts explicit answers or values obtainable through a trivial final step as candidates, then outputs them as ordered line-number/answer tuples.The output follows the order in which the solution lines appear.Source: SYSTEM PROMPT 2 · pp. 16–17

• Table 3 presents statistics for datasets and rollouts.Source: Table 3 · p. 17

• Four annotators labeled candidate-answer correctness, with each candidate shown alongside its sentence, problem statement, and ground-truth solution.Source: C HUMAN EVALUATION · p. 17
• Figure 10 shows the user interface used to evaluate extraction correctness.Source: C HUMAN EVALUATION · p. 17, Figure 10 · p. 18
• The evaluation separates identifying whether a sentence contains a candidate answer from checking whether an extracted candidate answer answers the question, regardless of correctness.For a quantity that “can be represented as m/n, what is m+n”, “5” is valid in form, while “m=2, n=3” is invalid.Source: C HUMAN EVALUATION · pp. 17–18
• 94.1% accuracy on candidate-position identification (Q1) and 94.0% accuracy on target-format adherence (Q2) were measured on 100 sampled rollouts containing 426 candidates.Source: C HUMAN EVALUATION · p. 18

D BREAKDOWN OF REFLECTIONS

• Figure 11 summarizes reflection statistics for long CoTs collected across five mathematical datasets, with rollout counts varying by dataset.The caption specifies 32 rollouts per question for AIME24 and AIME25, four for AMC, and one for Olympiad Bench and Math500.Source: Figure 11 · p. 19

E REFLECTION STATISTICS

• Table 4 reports candidate-answer counts in rollouts across reasoning models and datasets.Source: Table 4 · p. 19

• Figure 12 illustrates the SFT data curation process in Section 3.1.Source: Figure 12 · p. 20

• The rollout analysis in Table 5 covers 8 models on 5 datasets.Source: Table 5 · p. 20
• In Table 5, “First” and “Final” denote the accuracy of the first appearing candidate and the rollout’s final answer.Source: Table 5 · p. 20
• In Table 6, “FC” denotes tokens to the first candidate, while “Refl.” denotes reflection tokens after that candidate.Source: Table 6 · p. 20

• 3.8% average improvement for cut-at-6 over cut-at-1 across two models, with training data controlled to the same number of tokens.Source: Figure 13 · p. 21
• First-answer accuracy contributes 2.65% improvement, while reflection contributes 1.15%.Source: Figure 13 · p. 21
• Orange bars represent tokens before the first candidate answer and blue bars represent reflection tokens; performance is marked on each segment.Source: Figure 13 · p. 21
• The figure separates token use and accuracy into first-candidate-answer and post-candidate reflection parts, with rows for models trained on rollouts truncated at different positions.Source: Figure 13 · p. 21