PaperlayerView exact revision on arXiv

Source-linked AI summary

Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning

Zhenni Bi, Kai Han, Chuanjian Liu, Yehui Tang, Yunhe Wang

arXiv:2412.09078v5cs.CLcs.AI

TL;DR

Complex reasoning remains challenging, and methods such as CoT and ToT may complete a single pass without revisiting flawed paths. FoT combines multiple reasoning trees with sparse activation, dynamic self-correction, and consensus-guided decisions. The paper reports improved reasoning performance across its evaluations, including a reported GSM8K accuracy of 97.33% for FoT (n=4) with QwQ, compared with 95.20% for rStar-Math.

  • Problem

    CoT and ToT can make a single reasoning pass without revisiting flawed paths, while solving complex reasoning problems remains challenging.

    Source: §1 · p. 1

  • Method

    FoT integrates multiple reasoning trees with sparse activation, dynamic self-correction, and consensus-guided decision-making.

    Source: §1 · p. 2

  • Results

    FoT (n=4) with QwQ achieved 97.33% accuracy on GSM8K, compared with 95.20% for rStar-Math.

    Source: §4.6 · p. 8

  • Takeaways & Limitations

    The authors report that FoT improves LLM reasoning performance while using sparse activation to improve efficiency and accuracy.

    Source: §1 · p. 2

Abstract

from arXiv · show

Large Language Models (LLMs) have demonstrated remarkable abilities across various language tasks, but solving complex reasoning problems remains a significant challenge. While existing methods, such as Chain-of-Thought (CoT) and Tree-of-Thought (ToT), enhance reasoning by decomposing problems or structuring prompts, they typically perform a single pass of reasoning and may fail to revisit flawed paths, compromising accuracy. To address this limitation, we propose a novel reasoning framework called Forest-of-Thought (FoT), which integrates multiple reasoning trees to leverage collective decision-making for solving complex logical problems. FoT employs sparse activation strategies to select the most relevant reasoning paths, improving both efficiency and accuracy. Additionally, we introduce a dynamic self-correction strategy that enables real-time error correction, along with consensus-guided decision-making strategies to optimize both correctness and computational resources. Experimental results demonstrate that the FoT framework, combined with these strategies, significantly enhances the reasoning capabilities of LLMs, enabling them to solve complex tasks with greater precision and efficiency. Code will be available at https://github.com/iamhankai/Forest-of-Thought.

1. Introduction

Existing reasoning approaches enrich prompts or decompose problems, but often make only one complete pass and may not revisit flawed paths. Forest-of-Thought (FoT) combines multiple reasoning trees with sparse activation, dynamic self-correction, and consensus-guided decisions.

1. Introduction

  • CoT uses demonstrations and ToT explores multiple paths with self-evaluation, while GoT structures thoughts as an interconnected graph.These approaches enrich prompts or decompose complex problems into simpler subproblems, but typically do not revisit reasoning after one complete pass.Source: §1 · p. 1
  • A single complete reasoning pass may leave flawed paths uncorrected and alternative approaches unexplored, compromising accuracy on complex problems.The passage uses a multi-step mathematical word problem as an example of intermediate errors and overlooked details.Source: §1 · p. 1
  • FoT integrates multiple reasoning trees and sparsely selects relevant paths, alongside dynamic self-correction and consensus-guided decisions.The authors say the strategies aim to improve efficiency, accuracy, and computational-resource use.Source: §1 · p. 2

2. Related Works

The related work reviews sequential and structured approaches to LLM reasoning, problem decomposition, and search. It highlights both their reasoning strategies and, for tree search, the potential computational burden of deep exploration.

2.1. XoT reasoning

  • CoT decomposes problems into intermediate steps, but complex mathematical and logical problems may require nonlinear reasoning beyond a sequential chain.The passage lists several subsequent CoT variants while noting that further exploration is needed for highly complex tasks.Source: §2.1 · p. 2

Least-to-Most Prompting (LtM) (Zhou et al., 2023)

  • Problem-decomposition approaches range from stepwise prompting and programmatic steps to AoT, which puts decomposition, solution generation, feasibility assessment, and next-step selection in one prompt.The passage says AoT reduces token consumption and improves efficiency.Source: Least-to-Most Prompting (LtM) (Zhou et… · p. 2

2.2. Monte Carlo Tree Search

  • MCTS uses randomized rollouts to build a local game tree, while LLM integrations combine probabilistic search with linguistic reasoning for multi-step logic and diverse solution paths.ToT tree depth can grow substantially on complex problems, increasing the search space exponentially and raising computational burden.Source: §2.2 · pp. 2–3, Least-to-Most Prompting (LtM) (Zhou et… · p. 2

3. Method

FoT combines multiple reasoning trees with sparse activation and decision-making, and adds dynamic self-correction. Its method also describes input augmentation, early termination, and a consensus-guided expert strategy for selecting final answers.

3. Method

  • FoT combines independent reasoning trees and sparse activation to use collective intelligence and improve reasoning in larger models.The method describes filtering results from key trees to compensate for individual shortcomings.Source: §3 · p. 3

3.1. The FoT Framework

  • Sparse activation selects relevant trees and top-scoring nodes, while invalid outputs can stop a tree early and exclude it from final decision-making.The framework also describes retrieving relevant questions from a knowledge base and increasing question-answer diversity within each tree; searches can terminate once a target solution is found.Source: §3.1 · pp. 3–4

3.2. Dynamic Self-Correction Strategy

  • Dynamic self-correction monitors predicted logits and triggers error detection and correction below a predefined threshold, rather than using a fixed iteration count.Predefined mathematical rules can support immediate correction, such as checking whether Game of 24 outputs derive from the input numbers.Source: §3.2 · p. 4

3.3. Decision Making Strategy

  • CGED combines majority consensus with LLM expert evaluation when activated trees produce inconsistent results, guiding selection of a final answer.The expert compares the trees’ reasoning processes and outcomes.Source: §3.3 · p. 5

4. Experiments

The experiments evaluate FoT variants across Game of 24, GSM8K, and MATH, using ToT- and MCTSr-based forests and several LLMs. Results cover component ablations, subtree scaling, stopping strategies, and comparisons with other methods.

4.1. Experimental setups

  • Experiments used ToT-based FoT for Game of 24 and MCTSr-based FoT for GSM8K and MATH, with tests also spanning Llama3-8B-Instruct, Mistral-7B, and GLM-4-9B.The Game of 24 test set contained 95 problems after duplicate and unsolvable problems were removed.Source: §4.1 · p. 5, §4.2 · p. 5, §4 · p. 5

4.2. Game of 24

  • On Game of 24, adding self-correction raised accuracy from 10.58% to 60.24%, input enhancement raised it to 77.98%, and sparse activation reduced invocations from 32.32 to 26.99.FoT’s accuracy remained at its highest reported level after sparse activation was added.Source: §4.2 · pp. 5–6, Table 1 · p. 5

4.3. GSM8K Benchmark

  • On GSM8K, 4-rollout MCTSr with 2 trees achieved 3.2% higher accuracy than 8-rollout MCTSr, supporting diversity across trees over simply extending an individual tree.The reported forest comparisons included Zero-Shot-CoT and MCTSr forests with different rollout counts.Source: §4.3 · p. 6
  • Across Mistral-7B, Llama3-8B, and GLM-4-9B, accuracy rose with activated subtrees, with diminishing yet consistent gains; a self-correction threshold of 0.5 performed best.These scaling results were reported on GSM8K.Source: §4.3 · pp. 6–7, Figure 4 · p. 8

4.4. MATH Benchmark

  • On MATH, FoT consistently outperformed MCTSr from Level 1 through Level 4, with FoT (n=4) showing approximately 10% performance improvement across difficulty levels.Figure 5 presents the comparison across the four levels.Source: §4.4 · p. 7, Figure 5 · p. 8

4.5. Ablation Studies of Stopping Strategies

  • At two activated subtrees, CGED accuracy was similar to Majority Vote and Math Expert; at five, CGED exceeded both by 2%.The comparison evaluates three stopping strategies across different numbers of activated subtrees.Source: §4.5 · p. 7, Table 4 · p. 7

4.6. Main Results

  • FoT outperformed reported baselines across math benchmarks (GSM8K 97.33% vs rStar-Math 95.20%; AIME 2024 53.33% vs 6.66%) and reached 96.84% on Game of 24.The reported Game of 24 result used 8 activated subtrees; the cross-benchmark comparisons are summarized in Table 5.Source: §4.6 · pp. 7–8, Table 5 · p. 8

5. Conclusion

Forest-of-Thought combines multiple reasoning paths with dynamic activation and sparse computation. The paper also examines how reasoning time scales with accuracy in LLMs.

5. Conclusion

  • FoT integrates multiple reasoning trees and dynamic activation to support robust, efficient problem-solving without backpropagation or fine-tuning.It generates diverse reasoning outcomes across complex tasks.Source: §5 · p. 8
  • FoT incorporates Tree of Thought and Monte Carlo Tree Search with sparse activation to improve computational efficiency.The paper describes sparse activation as part of its approach to complex problems.Source: §5 · p. 8
  • The paper explores the relationship between reasoning time and accuracy in LLMs as a basis for optimizing reasoning performance.Source: §5 · p. 9

Impact Statement

FoT combines multiple reasoning trees, dynamic self-correction, and consensus-based decision-making for complex problem-solving. The paper presents this adaptive process as a route to more precise and efficient reasoning.

Impact Statement

  • FoT uses multiple reasoning trees, dynamic self-correction, and consensus-based decision-making to explore paths, correct errors, and narrow the search space.Source: Impact Statement · p. 9
  • The paper states that FoT can support complex problem-solving with higher precision and reduced computational cost.Source: Impact Statement · p. 9
  • The authors identify mathematics, logic, and AI-driven problem solving as fields with potential implications for more robust and reliable AI systems.Source: Impact Statement · p. 9

A. Example Prompts - Game of 24

The Game of 24 prompts guide a solution through successive number-selection and calculation steps, then score the steps for reasonableness and efficiency.

A. Example Prompts - Game of 24

B. Example Prompts - GSM8K and MATH

The GSM8K prompts also apply to MATH with minor answer-extraction variations. The described mechanisms score and revise answers, iteratively refine outputs, and use expert selection when multiple answers align with majority consistency.

B. Example Prompts - GSM8K and MATH

  • The GSM8K prompts are used for MATH as well, with only minor variations in answer extraction.Source: B. Example Prompts - GSM8K and MATH · p. 12
  • The self-correction prompt scores an answer using criteria such as logical consistency, accuracy, and relevance, then prompts revision when flaws or suboptimal reasoning are indicated.Source: B. Example Prompts - GSM8K and MATH · p. 12
  • Iterative refinement is described as helping the model identify errors and generate more accurate, coherent responses; the cited experiments report improved performance across tasks.Source: B. Example Prompts - GSM8K and MATH · pp. 12–13
  • When multiple answers align with majority consistency, a mathematics expert evaluates them and selects the most appropriate or optimal answer.The selection is intended to align the final answer with logical consistency and mathematical rigor.Source: B. Example Prompts - GSM8K and MATH · p. 13

C. Evaluating the Scaling of FoT Compute Across Different Baseline Methods.

On GSM8K, FoT’s error rate decreases as more subtrees are activated; the scaling comparison includes several baseline methods.

C. Evaluating the Scaling of FoT Compute Across Different Baseline Methods.

  • As activated subtrees increase, FoT’s GSM8K error rate decreases, while other methods show diminishing returns or slower error reduction.The comparison includes zero-shot-CoT, one-turn-self-refine MCTSr, and 4- and 8-rollout MCTSr.Source: C. Evaluating the Scaling of FoT… · p. 13, Figure 6 · p. 14
  • Figure 6 plots the number of activated FoT subtrees against GSM8K error rate across the compared methods.The x-axis is activated subtree count; the y-axis is error rate.Source: Figure 6 · p. 14
Loading 2412.09078v5…