Is Depth All You Need? An Exploration of Iterative Reasoning in LLMs

1 Introduction

• CoT guides LLMs to construct intermediate reasoning steps before producing answers, and deep iterative reasoning reuses those steps and predictions as inputs across rounds.The paper frames iterative reasoning as a way to progressively activate relevant pre-trained knowledge.Source: §1 · p. 1
• Deep iterative reasoning faces challenges in ensuring each new round improves on the last and in choosing when to stop.The paper asks whether relevant pre-trained knowledge can instead be activated from the initial reasoning path.Source: §1 · p. 1
• With self-consistency, performance on samples initially solved by deep iterative reasoning improved as the number of paths increased, then plateaued.A single path correctly reclassified approximately half of those samples; the findings suggest diverse initial paths can activate relevant knowledge.Source: §1 · p. 2
• Self-consistency generates limited path diversity and cannot by itself fully replace the advantages of deep iterative reasoning.The paper identifies rephrasing the question or prompt, perturbations during generation, and multiple sampling as potential sources of diversity.Source: §1 · p. 2
• The proposed breadth method expands reasoning diversity by rephrasing questions or prompts while combining contextual exploration with self-consistency, and significantly outperforms deep iterative reasoning.The authors report that question or prompt perturbations significantly influence path diversity, while direct perturbations to the LLM do not.Source: §1 · p. 2

2.1 From Standard CoT to Depth Reasoning

• Deep iterative reasoning feeds the current reasoning steps and prediction back into the model, generates new reasoning, and repeats until a stopping criterion or iteration limit.The new prediction is generated from the question, prompt, and updated reasoning, rather than the earlier reasoning and prediction.Source: §2.1 · p. 2

2.2 Deep Iterative Reasoning as a Gradual Activation of Prior Knowledge

• 69.9% to 80.8%: arithmetic-task performance improves as iterations increase, whereas average performance on StrategyQA and CommonsenseQA remains stable.Arithmetic reasoning progressively activates relevant logical knowledge; commonsense tasks depend more on information retrieval, and deeper reasoning alone is unlikely to resolve missing pre-training knowledge.Source: §2.2 · p. 3, Figure 3 · p. 3, Figure 2 · p. 3

2.3 Can Depth Reasoning be Replaced?

• AddSub accuracy peaks at 84.8% with five reasoning paths, while AQuA improves from 38.4% with one path to 61.6% at three, then plateaus.These results come from samples misclassified by standard CoT but correctly predicted by deep iterative reasoning; Figure 4 presents the path-count comparison.Source: §2.3 · p. 4, Figure 4 · p. 4
• Self-consistency’s multiple initial reasoning paths can correctly reclassify some samples, but their limited diversity prevents it from fully replacing deep iterative reasoning.The paper calls the alternative of exploring diverse initial paths and voting across predictions breadth reasoning.Source: §2.3 · p. 4

3.1 What Influences Reasoning Diversity?

• Reasoning-path diversity can arise from rewording the question, rewording the prompt, perturbing the model, or sampling multiple paths from it.The paper frames these as four points in the CoT process where variation may be introduced.Source: §3.1 · p. 4, Figure 5 · p. 5

3.2 Impact Analysis of Factors

• Question and prompt rewording significantly increases reasoning-path diversity, while direct model perturbations have little impact and self-consistency produces less diversity than the two context changes.The analysis measures prediction entropy on questions the models cannot solve; it attributes the difference to context changes prompting reassessment, unlike sampling within a fixed context.Source: §3.2 · pp. 4–5

3.3 Proposed Method

• Breadth reasoning rephrases the question or prompt without changing its meaning, then combines contextual exploration with multiple samples per reformulation and votes across predictions.For example, three reformulations sampled twice each yield six reasoning paths. Table 1 covers accuracy on ten datasets across three reasoning-task categories, with breadth methods and deep iterative methods set to three paths or iterations.Source: §3.3 · p. 5, Table 1 · p. 6

4.1 Experimental Settings

• The evaluation uses GPT-3.5-turbo-0125 on ten arithmetic, commonsense, and symbolic reasoning datasets, comparing standard CoT, deep iterative reasoning, and breadth methods.Source: §4.1 · p. 6

4.2 Evaluating Different Breadth Methods

• SC performs particularly well on symbolic tasks, while question and prompt reformulations improve breadth performance, notably on arithmetic tasks.QuestionC-SC and PromptC-SC outperform their individual counterparts across arithmetic and symbolic examples.Source: §4.2 · pp. 6–7
• PromptC-SC attains the highest average accuracy of 85.1% across all datasets, a 4.3% improvement over PromptC, supporting the benefit of combining self-consistency with contextual prompt-space exploration.On arithmetic and symbolic tasks, QuestionC-SC and PromptC-SC significantly outperform their individual counterparts.Source: §4.2 · pp. 6–7

4.3 Breadth v.s. Depth

• Deep-CoT performs particularly well on GSM8K and AQuA, while ARI achieves 87.1% on AddSub.Source: §4.3 · p. 7
• 85.1% average accuracy for PromptC-SC versus 83.3% for ARI, a 1.8% improvement; breadth methods generally outperform deep iterative methods on symbolic and commonsense tasks.PromptC-SC exceeds ARI by 3.6% on LastLetter and 1.4% on CoinFlip.Source: §4.3 · p. 7

4.4 Breadth Analysis

• Prompt reformulations improve AQuA performance until gains plateau at five, whereas question reformulations begin to reduce performance at four.The figure evaluates reformulations alongside different levels of self-consistency sampling.Source: §4.4 · p. 7, Figure 6 · p. 7
• 68.1% with six self-consistency paths and 67.1% with nine are both surpassed by the proposed method using four paths.The four paths comprise two reformulations combined with two samples.Source: §4.4 · p. 7, Figure 6 · p. 7

• With fixed context, self-consistency paths may remain on one reasoning plane; exploring context space covers more strategies, while self-consistency supports stability under the same context.The figure depicts breadth as generating multiple paths and selecting by majority vote.Source: §5 · p. 8, Figure 7 · p. 8
• Deep iterative reasoning can stall when a new path adds no meaningful update, or introduce errors by continuing after reaching the correct plane.The authors also identify the difficulty of ensuring improvement at each iteration and choosing a stopping criterion.Source: §5 · p. 8
• A single-round breadth strategy avoids the challenges of iterative refinement described for deep reasoning.Source: §5 · p. 8

• The proposed breadth method modifies input expressions and uses self-consistency to expand reasoning diversity without multiple iterations.Source: §6 · p. 8
• Prompt modifications remain within the same semantic space, and more effective semantic exploration could expand reasoning breadth.Source: §6 · p. 8
• Multiple generated paths incur computational cost, motivating future work on combining depth and breadth efficiently while minimizing overhead.Source: §6 · p. 8

• Multiple generated reasoning paths increase computational cost and may be impractical for real-time or resource-constrained applications.Source: Limitations · p. 9
• Input-expression modifications explore a fixed semantic space and may miss deeper semantic variations that could broaden reasoning.Source: Limitations · p. 9
• Breadth reasoning does not explicitly ensure consistency across diverse paths.Source: Limitations · p. 9

A.1.1 Chain-of-Thought

• Manual prompting methods such as PAL achieve high performance but are costly and difficult to generalize, whereas automatic prompting methods such as Auto-CoT are cheaper and transferable but more error-prone.Semi-automatic methods including AutoMate CoT and BoostedPrompt aim to balance performance and efficiency for real-world use.Source: §A.1.1 · p. 11
• Self-Refine and Reflexion use iterative verification and refinement, while L2M decomposes problems, KD-CoT adds external knowledge, and RankPrompt uses self-ensemble prediction ranking.These approaches address reliability, complex questions, knowledge-sensitive tasks, and prediction accuracy, respectively.Source: §A.1.1 · p. 11
• Tree-based methods support broader exploration and backtracking, while graph-based methods support sub-problem aggregation, self-verification, and more coherent reasoning.The cited tree approaches are Yao et al. (2024b) and Chen et al. (2024); graph approaches are Lei et al. (2023) and Besta et al. (2024).Source: §A.1.1 · p. 11

A.1.2 Long-Chain Thought Reasoning

• Long-chain systems such as DeepSeek-R1, QwQ, and Marco-o1 iteratively refine reasoning; related work also explores accuracy-driven rewards and multimodal applications.The section describes these systems as correcting errors and exploring alternatives, and notes Mulberry’s use of o1-like principles in multimodal contexts.Source: §A.1.2 · p. 11

A.2 Algorithmic Pseudo-Code

• The algorithm generates N meaning-preserving reformulated questions, runs CoT reasoning and self-consistency sampling for each, then returns the majority-vote answer.Its inputs include the question, LLM, number of reformulations, and self-consistency samples per reformulation.Source: §A.2 · p. 12

A.3.1 Experimental Setting

• Experiments use Llama-2-7b-chat and challenging subsets of 20 AddSub and 20 AQuA questions to compare question and prompt reformulations, model perturbations, and sampling.The setup also varies prompt wording and perturbs the model’s final linear layer.Source: §A.3.1 · p. 12

A.3.2 Main Results

• Prompt modifications most effectively enhance reasoning diversity, while LLM perturbations are least impactful; the trends are similar across AQuA and AddSub.Table 3 reports prediction entropy for factors affecting reasoning-path diversity; higher entropy indicates broader exploration.Source: §A.3.2 · p. 13, Table 3 · p. 13

A.4 Experimental Demos of Deep Iterative Reasoning and Breadth Reasoning

• The demo tables distinguish problems solvable by both depth and breadth reasoning (Tables 4–6) from problems solvable by breadth reasoning but not depth reasoning (Tables 7–9).Source: Table 4 · p. 14, Table 5 · p. 15, Table 6 · p. 16
• Tables 10–12 present examples of problems unsolvable by both depth and breadth reasoning.The captions identify depth reasoning in Table 10 and breadth reasoning in Tables 11 and 12.Source: Table 10 · p. 20, Table 11 · p. 21, Table 12 · p. 22