Source-linked AI summary
FoE: Forest of Errors Makes the First Solution the Best in Large Reasoning Models
Kehan Jiang, Haonan Dong, Zhaolu Kang, Zhengzhou Zhu, Guojie Song
TL;DR
The paper challenges test-time scaling by showing that reasoning errors grow with additional solutions through a Forest of Errors. It proposes RED to refine the first solution and discard later ones, outperforming seven baselines while reducing token use.
Problem
The paper investigates why alternative solutions can worsen reasoning as test time increases, identifying concurrent error growth and limited self-reflection as central issues.
Method
RED refines the first solution at high-entropy positions and discards subsequent solutions using dual-consistency, guided by FoE analysis and theoretical modeling.
Results
RED outperforms seven competitive baselines with up to 19.0% accuracy gains and 37.7%–70.4% token reduction across four datasets and six backbone models.
Takeaways & Limitations
FoE analysis suggests that suppressing error growth in the first solution and pruning later solutions can improve reasoning efficiency and effectiveness.
Takeaways & Limitations
RED introduces additional decoding latency, although worst-case profiling measured an average overhead of 4.6%.
Abstract
from arXiv · showhide
Recent Large Reasoning Models (LRMs) like DeepSeek-R1 have demonstrated remarkable success in complex reasoning tasks, exhibiting human-like patterns in exploring multiple alternative solutions. Upon closer inspection, however, we uncover a surprising phenomenon: The First is The Best, where alternative solutions are not merely suboptimal but potentially detrimental. This observation challenges widely accepted test-time scaling laws, leading us to hypothesize that errors within the reasoning path scale concurrently with test time. Through comprehensive empirical analysis, we characterize errors as a forest-structured Forest of Errors (FoE) and conclude that FoE makes the First the Best, which is underpinned by rigorous theoretical analysis. Leveraging these insights, we propose RED, a self-guided efficient reasoning framework comprising two components: I) Refining First, which suppresses FoE growth in the first solution; and II) Discarding Subs, which prunes subsequent FoE via dual-consistency. Extensive experiments across five benchmarks and six backbone models demonstrate that RED outperforms eight competitive baselines, achieving performance gains of up to 19.0% while reducing token consumption by 37.7% ~ 70.4%. Moreover, comparative experiments on FoE metrics shed light on how RED achieves effectiveness.
1 Introduction
The paper identifies a counterintuitive pattern in large reasoning models: later solutions can amplify reasoning errors rather than improve the first solution. It models these errors as a Forest of Errors and proposes RED to refine the first solution while pruning subsequent ones.
- Phenomenon Discovery: First is optimal in up to 93.7% of cases, while subsequent solutions can fail to correct errors or mislead a correct first solution.Subs fail to rectify an erroneous First with 75.4%–82.8% probability and can mislead a potentially correct First with probability up to 21.2%.
- Phenomenon Discovery: FoE errors propagate from multiple root causes into a forest-like structure whose growth increases with reasoning length.Root error nodes play a pivotal role, and error generation correlates with entropy and entropy variance.
- Practical Method: RED refines the First using entropy-based intervention and discards Subs using dual-consistency-based early stopping.The method targets positions prone to root errors and prevents inferior subsequent solutions from affecting the final answer.
- Insightful Analysis: A FoE-based probabilistic framework theoretically validates the optimality of the First alongside qualitative and quantitative empirical analysis.The analysis derives five observations about reasoning errors and their accumulation.
- Practical Method: 19.0% accuracy gains and 37.7%–70.4% token reduction are reported for RED across four datasets and six backbone models.The experiments also report that FoE metrics confirm RED effectively eliminates FoE.
2 The First is The Best
Across datasets and models, the experiments show that the first generated solution usually has greater influence and reliability than subsequent solutions. Subsequent solutions rarely repair an incorrect First and can instead introduce errors or waste computation.
- Influence of Subs: 75.4%–82.8% of Subs remain incorrect when First is incorrect.This indicates that subsequent solutions usually fail to rectify an incorrect First.
- Influence of Subs: 93.7% of cases favor First, meaning it is uniquely correct or more robust than Subs.The table excludes cases where both First and Subs are correct.
- Influence of Subs: 2.0%–7.1% is the maximum success rate for Subs rectifying an incorrect First, with many dataset-model combinations showing no successful cases.The reported correction rate is substantially lower than the frequency with which Subs fail to correct First.
- Influence of Subs: 18.8% is the maximum probability that Subs mislead a correct First into an incorrect answer.The reported risk of misleading a correct First exceeds the reported rate of correcting an incorrect First.
- Influence of Subs: After interruption-point resampling, First yields errors with probability only 3.7% of Subs’ probability.This result indicates greater stability for First under the reported sampling procedure.
3 FoE
The paper models reasoning errors as a Forest of Errors (FoE) and evaluates how this structure grows across solutions. First has smaller, slower-growing error forests, while reflection in subsequent solutions provides limited correction.
- FoE modeling: FoE models reasoning errors as trees linked by parent-child dependencies, with analyses covering structure, metrics, entropy, and reflection.The framework examines how errors originate, propagate, and are corrected across reasoning solutions.
- Error propagation: Root errors are pivotal because correcting descendants alone fails to stop propagation, whereas correcting roots slows subsequent error generation.Manual correction experiments compare grandchild, child, and root nodes.
- FoE metrics: First has smaller forests than Subs, with forest size 6.9 versus 8.1 and a 33.3% lower average error-node reproduction rate.Subs also show approximately 16.3% greater forest depth and more average nodes per tree, 7.1 versus 8.4.
- Reflection: Subs exhibit weaker reflection than First, including a 62.5% reduction in reflection frequency and decreases of 68.2% in completeness and 82.1% in depth for Qwen-8B-thinking.The comparison spans multiple backbones on the BS-17k-subset.
- Reflection: In First, true correction accounts for 67.1%, but in Subs fake correction and refusal rise to 64.2% and 31.1%, respectively.The final solution is described as essentially fake correction.
4 Method
RED improves the first solution at high-entropy, high-variance points and discards subsequent solutions using dual-consistency. Its design combines entropy-guided intervention with an early-exit rule requiring agreement within and across probe prompts.
- Method overview: RED is a self-guided reasoning method motivated by FoE analysis and designed to refine First while pruning Subs.The method section presents its motivation and two components.
- Refining First: Refining First intervenes at segments with high entropy and variance because these conditions are associated with root-error generation.The intervention is intended to suppress growth of the entire FoE by addressing root errors.
- Refining First: The entropy-based intervention monitors entropy variance and average maximum Top-K entropy within a sliding window, triggering when the variance exceeds threshold T.The mechanism uses a window of length L and a threshold-based trigger.
- Discarding Subs: Discarding Subs uses periodic probes with multiple prompt templates and parallel samples, followed by early exit only when dual consistency is satisfied.The design combines internal consistency within each template with cross-prompt agreement.
- Discarding Subs: Early exit requires each prompt’s dominant answer to reach frequency ≥P% and all prompt templates to produce the same dominant answer.This condition is intended to reject prompt-sensitive interim answers.
5 Experiments
Across five benchmarks and six backbone models, RED improves reasoning performance while substantially reducing token use and FoE metrics. The method is evaluated against vanilla models, training-free methods, and RL-based strategies.
- Experimental Setup: The experiments use representative Qwen- and Llama-family LRMs spanning 7B to 70B parameters.The compared backbones include Qwen3-thinking, DeepSeek-R1-Distill-Qwen, and DeepSeek-R1-Distill-Llama models.
- Main Results: RED improves performance by 3.2% ∼19.0% over the Vanilla model while reducing token consumption by 37.7% ∼70.4%.These results compare RED with eight baselines across five benchmarks.
- Main Results: On DeepSeek-R1-Distill-Llama-8B+AIME25, RED gains 5.5 score points and reduces tokens by 56.4%.
- FoE Metrics: RED reduces all FoE metrics by 41.0% ∼68.0%, whereas S-GRPO reduces static D/T by 3.4% and dynamic Repro by 7.4% on DeepSeek-R1-Distill-Qwen-32B.RED’s corresponding reductions are 37.9% for D/T and 57.9% for Repro.
6 Conclusion
The paper identifies The First is The Best in LRMs and attributes it to FoE growth as test-time reasoning expands. It introduces RED to refine the first solution and discard later computations.
- The First is The Best challenges test-time scaling laws by showing that reasoning errors scale concurrently with test time.
- RED combines Refining First to inhibit error growth with Discarding Subs to eliminate redundant, error-prone computations.
Limitations
The paper identifies additional decoding latency as a limitation of RED but reports that its measured overhead is small relative to the speedup from early exit.
- RED introduces additional latency from extra decoding operations, with worst-case profiling reporting an average overhead of 4.6%.The stress test disables early exit while retaining probe prompts and entropy-based intervention.
- The reported early-exit savings outweigh this overhead, producing a net speedup over the baseline.
B.1 Additional Results of FoE-related Metrics
Additional experiments show that RED improves accuracy while suppressing both static FoE structure and dynamic error reproduction across mathematical and scientific tasks. Its consistency-based pruning also improves sample efficiency and depends on combining both RED components.
- Mathematical Task: On MATH500, RED increases Pass@1 by +1.1 ∼ +2.1 and reduces all FoE metrics by 37.1% ∼68.0% across three backbones.The reduced metrics are FS, N/T, D/T, and Repro.
- Mathematical Task: On DeepSeek-R1-Distill-Qwen-32B, RED reduces D/T by 37.1% and Repro by 57.9%, compared with S-GRPO reductions of 2.9% and 7.0%.
- Scientific Task: On GPQA-Diamond, RED improves Pass@1 by +1.0 ∼+1.7 and reduces all FoE metrics by 38.6% ∼68.1% relative to Vanilla.
- Consistency Scaling: On AIME25 with Llama-70B, RED at k = 8 reaches 56.7%, matching Vanilla at k = 64 and indicating an ∼8× reduction in computational cost.
- Ablation Study: Removing Discarding Subs lowers Pass@1 and increases generation length, while removing Refining First remains below RED in accuracy and token efficiency.The ablation supports combining both components to improve the first trajectory and prevent later solutions from perturbing it.
E Experimental Results of Rollback in Subs.
Rollback sampling tests whether initially correct solution trajectories remain stable after interruption and continuation resampling. First is markedly more robust than Subs, supporting the claim that later solutions carry greater latent error risk.
- Rollback-sampling results: First maintains error rates below 1% across rollback ratios, whereas Subs show substantially higher, largely K-invariant error rates.Rollback ratios span K ∈ {10, 20, . . . , 80}%, with the KV cache restored at each truncation point.
- Experimental design: N = 100 independent continuation samples are generated from each restored intermediate state to estimate sampling error rates.Instances are retained only when the evaluated solution was correct in its original run, isolating stability under correctness.
- Rollback-sampling results: First averages approximately 0.69% error, compared with 18.20% for aggregated Subs under interruption and resampling.Solution 2, Solution 3, and Solution 4 average about 14.85%, 21.76%, and 18.0%, respectively.
- Interpretation: The results support a latent-risk interpretation: apparently correct Subs have weaker intermediate-state commitment to the correct answer.Resampling from an interruption point can therefore cause Subs to drift to incorrect final answers.
F.3 Empirical Findings on Error Node Fixing
The error-node fixing experiments test whether correcting descendants or roots changes subsequent FoE growth. They find that uncorrected ancestors continue generating errors, whereas root correction substantially suppresses downstream growth.
- Interpretation: The experiments support root correction as the decisive operation for slowing FoE reproduction because it removes the erroneous causal source from the downstream decoding state.This conclusion contrasts ancestor intervention with surface-level descendant patching.
- Descendant correction: Correcting a non-root node can induce sibling regeneration because its nearest uncorrected ancestor remains active.The experiment fixes a leaf while keeping its child and root intact, then detects newly generated errors linked to the same parent.
- Empirical findings: SPAWN@15 = 0.842: correcting a descendant without addressing its root cause fails to halt the error cascade in over 84% of cases.The metric measures whether an uncleared ancestor generates at least one new erroneous child within the subsequent 15 decoding steps.
- Root correction: Root correction substantially decelerates subsequent error-node generation even after an error tree has formed.KV-cache branching compares a Fix-Root continuation with a No-Fix continuation from the same decoding state.
- FoE construction: The FoE is constructed by linking each error to its nearest inducing predecessor when its parent–child score exceeds threshold τ.The procedure produces a forest of trees and a parent map over the chronological error list.
H The Reason for Node Generation: Further Analysis
Further analysis links root-error generation to the joint behavior of token entropy and entropy variance. High entropy and high variance together identify the strongest root-trigger regime, with First remaining more robust than Subs.
- Analysis setup: The analysis partitions decoding steps into LL, HL, LH, and HH using percentile thresholds for entropy and entropy variance.The default window length is L = 15, with robustness evaluated over L ∈ [10, 20].
- Entropy-quadrant findings: HH yields the highest root-trigger rate in both settings: 0.187 for First versus 0.264 for Subs.The result remains stable across entropy-window lengths L ∈ [10, 20].
- Statistical analysis: OR > 12: HH increases root-trigger odds by over an order of magnitude relative to LL, while neither entropy nor variance alone is sufficient.Compared with HL or LH, HH retains OR ≈3.4–4.7, with p < 10^-6.
- First versus Subs: First has lower root-trigger likelihood than Subs under identical HH conditions, with cross-setting OR = 0.64.The reported comparison is statistically significant at p < 0.01.
I Discarding Subs: Details and Analysis
The Discarding Subs analysis develops early stopping through convergence, cross-prompt robustness, and per-prompt parallel sampling. Dual consistency reduces confident wrong exits, while stricter checks trade coverage for lower risk; RED improves this tradeoff by refining First.
- Convergence: Baseline A can trigger on a sharply concentrated but wrong answer, because a wrong modal ratio may still exceed P.Its aggregate behavior is 73.20% ESC, 0.88% WESR, and 1.20% conditional risk.
- Cross-prompt robustness: Baseline B’s single-draw agreement across prompts can be spuriously correct-looking when all prompts happen to produce the same wrong answer.Figures 12 and 13 contrast genuine cross-prompt agreement with high-variance single-draw failure.
- Parallelism and dual consistency: Per-prompt parallel sampling stabilizes prompt-wise modes, but agreement still requires an internal consistency threshold to avoid premature wrong triggers.Figures 14 and 15 show correct aligned modes versus early wrong alignment with low internal modal ratios.
- Risk reduction: The probability of a confident prompt-invariant wrong answer decays exponentially in both N and M once the internal threshold is enforced.This explains the lower WESR and Crisk@Agree reported for DC and RED.
- Coverage tradeoff: Increasing P, M, or N generally lowers risk but increases expected trigger time, reducing early-stop coverage.The analysis formalizes this as a hitting-time tradeoff between safety and coverage.
- RED: Refining First shifts prompt-induced probability mass toward the correct answer earlier, decreasing expected hitting time while preserving or improving wrong-trigger concentration bounds.This provides the stated mechanism for RED’s improved coverage and reduced risk relative to DC.
J Analysis of Latency Overhead
RED’s monitoring and probing add only modest intrinsic latency, while adaptive termination yields substantially lower end-to-end inference time. A stress test disabling early exit measures the raw overhead separately from the acceleration benefit.
- Evaluation setup: The latency evaluation uses A100-80G accelerators and covers benchmarks including GSM8K, MATH, and GPQA-Diamond.The non-stopping configuration forces full-trajectory generation to quantify worst-case operational overhead.
- Intrinsic overhead: 4.6% Average Relative Latency Overhead across evaluated models and benchmarks under the non-stopping stress test.The test keeps entropy monitoring, interventions, probing, and consistency checks active while disabling termination.
- End-to-end runtime: RED consistently achieves the lowest or near-lowest total wall-clock time when early exit is enabled.Table 23 compares real-world inference runtime across baselines while excluding AlphaOne and GRPO because of their prohibitive computational costs.
- End-to-end runtime: 1.8× speedup reduces DeepSeek-R1-Llama-70B AIME24 inference time from 06:41 to 03:39.The reduction in generated tokens offsets the per-step probing and intervention costs.
K.2.3 From FoE size to answer error: a general bound.
The paper bounds answer-error probability by the expected size of the Forest of Errors, then shows why later solutions can carry a looser risk bound than the first. Continued exploration can additionally override a correct first solution through artifact reuse.
- General answer-error bound: Pr(Ŝ ≠ Y) ≤ Φ(T, P_R, ρ) := κ · M(S), linking answer-error probability to expected FoE size.The bound follows from a per-node criticality assumption and a union bound over error nodes.
- General answer-error bound: Φ is monotone in horizon T, root-trigger profile P_R, and reproduction rate ρ, so fewer roots or slower reproduction suppresses downstream errors.The branching-process formulation isolates root triggering and descendant reproduction as the two mechanisms controlling forest growth.
- First versus Subs: Subs has a strictly looser FoE-based risk upper bound because it combines higher root-triggering stress, weaker pruning, and a longer accumulation horizon.The comparison is stated as an expected structural dominance relation between First and Subs.
- Subs biases First: Artifact reuse across the continuation can amplify earlier errors, increasing root-triggering propensity or reproduction under the first trace’s history.The continuation is conditioned on the full First context, allowing previously introduced artifacts to be revisited and amplified.
- Subs biases First: Continued exploration creates an additional failure channel in which Subs override a correct First solution through newly introduced root errors.The paper defines this misguidance event as a correct first solution followed by an incorrect final decision.
- Conclusion: The theoretical analysis concludes that extending reasoning beyond First is not guaranteed to improve reliability and can strictly worsen it.This conclusion is attributed to FoE growth and the resulting difference between First and Subs risk bounds.