Source-linked AI summary
Towards Reasoning in Large Language Models: A Survey
Jie Huang, Kevin Chen-Chuan Chang
TL;DR
Although LLMs show strong performance and elicited reasoning behaviors, it remains unclear whether they truly reason or rely on memorized patterns and heuristics. This paper reviews techniques, evaluation methods, benchmarks, and prior findings, concluding that further research and better benchmarks are needed.
Problem
It remains unclear to what extent LLMs are capable of true reasoning rather than relying on memorized patterns and heuristics.
Method
The paper provides a comprehensive review of techniques for eliciting and improving reasoning, evaluation methods and benchmarks, and prior findings and implications.
Results
The reviewed evidence shows that chain-of-thought prompting can improve reasoning-task performance, but generated steps may be wrong or incomplete and do not establish true reasoning.
Takeaways & Limitations
Further analysis of training data, model architecture, optimization objectives, and better benchmarks is needed to understand and improve LLM reasoning.
Takeaways & Limitations
The survey focuses on deductive reasoning and may not cover inductive or abductive reasoning in depth.
Abstract
from arXiv · showhide
Reasoning is a fundamental aspect of human intelligence that plays a crucial role in activities such as problem solving, decision making, and critical thinking. In recent years, large language models (LLMs) have made significant progress in natural language processing, and there is observation that these models may exhibit reasoning abilities when they are sufficiently large. However, it is not yet clear to what extent LLMs are capable of reasoning. This paper provides a comprehensive overview of the current state of knowledge on reasoning in LLMs, including techniques for improving and eliciting reasoning in these models, methods and benchmarks for evaluating reasoning abilities, findings and implications of previous research in this field, and suggestions on future directions. Our aim is to provide a detailed and up-to-date review of this topic and stimulate meaningful discussion and future work.
1 Introduction
Reasoning supports problem solving, decision making, and critical thinking, while LLMs have shown possible reasoning behaviors at sufficient scale. However, the extent to which LLMs truly reason remains unclear, motivating this survey’s comprehensive review.
- Reasoning uses evidence, arguments, and logic to reach conclusions or make judgments, supporting problem solving, decision making, and critical thinking.
- LLMs have advanced natural language processing and may exhibit emergent reasoning behaviors when they are sufficiently large.
- Chain-of-thought exemplars and step-by-step prompts can elicit explicit reasoning steps from LLMs.
- Despite strong performance on some reasoning tasks, it remains unclear whether LLMs are actually reasoning and how capable they are.
- The paper surveys reasoning concepts, enhancement and elicitation techniques, evaluation methods and benchmarks, findings, implications, and future directions.
2 What is Reasoning?
The paper treats reasoning as inference and conclusion-drawing from information, while distinguishing formal and informal reasoning and covering deductive, inductive, and abductive forms. Its focus is informal deductive reasoning because deductive reasoning is most commonly studied in the literature.
- Reasoning involves making inferences, evaluating arguments, and drawing logical conclusions from available information.
- Deductive reasoning derives conclusions that must follow from true premises.
- Inductive reasoning draws conclusions from observations or evidence that are likely, but not necessarily certain, to be true.
- Abductive reasoning selects the most likely explanation for a set of observations, without certainty.
- Formal reasoning follows structured rules, whereas informal reasoning relies more on intuition, experience, and common sense.
- The paper encompasses several reasoning forms but focuses particularly on informal deductive reasoning.
3 Towards Reasoning in Large Language Models
The survey focuses on large language models because reasoning is considered rarely present in smaller models and may emerge at larger scales. It therefore follows prior work in emphasizing models such as GPT-3 and PaLM.
- Reasoning, especially multi-step reasoning, is often viewed as a weakness in language and NLP models.
- Prior research suggests reasoning ability may emerge at a scale exceeding 100 billion parameters.
- The paper treats reasoning as rarely present in small-scale models such as GPT-2 and BERT and focuses on larger models including GPT-3 and PaLM.
3.1 Fully Supervised Finetuning
Fully supervised finetuning trains language models with explicit reasoning examples, but its data requirements and dataset-specific training constrain broader application and may encourage reliance on artifacts.
- Fully supervised finetuning can train models to generate rationales, step-by-step solutions, or intermediate computations for reasoning tasks.
- Models trained with explanations can perform better on commonsense question answering tasks.
- Fully supervised finetuning requires datasets containing explicit reasoning, which can be difficult and time-consuming to create.
- Training on a specific dataset limits application to that domain and may lead models to rely on training-data artifacts rather than actual reasoning.
3.2 Prompting & In-Context Learning
Prompting and in-context methods elicit explicit reasoning from LLMs by providing intermediate steps, refining rationales, exploring alternatives, verifying solutions, or decomposing problems. These techniques improve reasoning-task performance across several settings, while their effectiveness depends on exemplar design and task complexity.
- 3.2.1 Chain of Thought and Its Variants: Chain-of-thought prompting replaces standard demonstrations with input–rationale–output examples, teaching models to generate intermediate reasoning before answers.The approach has improved few-shot performance on arithmetic, symbolic, and commonsense reasoning tasks.
- 3.2.1 Chain of Thought and Its Variants: Zero-shot-CoT elicits reasoning with “Let’s think step by step,” while code-based reasoning, scratchpads, multilingual prompts, and multimodal applications adapt the approach to different settings.These variants remove few-shot requirements or tailor intermediate reasoning to programs, languages, and modalities.
- 3.2.2 Rationale Engineering: Rationale engineering extends chain-of-thought prompting through rationale refinement, exploration, and verification rather than relying only on manually crafted examples.The original method uses manually crafted reasoning examples and greedy decoding.
- 3.2.2 Rationale Engineering: More complex and diverse rationale exemplars can improve LLM reasoning performance, with exemplar diversity identified as important for producing better rationales.Complexity-based prompting reports improved performance as rationale complexity increases.
- 3.2.2 Rationale Engineering: Rationale exploration uses self-consistency to sample diverse reasoning paths and select among them instead of relying on a single greedy rationale.The method is motivated by the possibility that complex problems admit multiple valid ways of reaching the same answer.
- 3.2.2 Rationale Engineering: Rationale verification scores generated rationales and solutions, enabling selection of outputs whose reasoning leads to correct final answers.Incorrect rationales can produce incorrect final predictions, motivating trained verifiers.
- 3.2.3 Problem Decomposition: Problem decomposition addresses chain-of-thought weaknesses on complex tasks by breaking problems into smaller subproblems and solving them in an ordered sequence.Least-to-most prompting facilitates later subproblems with answers obtained from earlier ones.
- 3.2.4 Others: Other prompting approaches use modular selection and inference, simplified numerical representations, reasoning distillation, or language-model cascades for specialized settings.These methods extend reasoning facilitation beyond standard chain-of-thought prompting.
3.3 Hybrid Method
The hybrid approach combines methods that improve LLM reasoning capabilities with prompting techniques that exploit those capabilities during problem solving. This contrasts with prompting alone, which leaves model parameters unchanged.
- Prompting can elicit or better utilize reasoning in LLMs, but it does not improve their underlying reasoning capabilities because model parameters remain unchanged.
- The hybrid approach simultaneously enhances LLM reasoning capabilities and uses prompting to solve complex problems more effectively.
- Pretraining or finetuning on scientific, mathematical, SQL, or code data can improve performance on reasoning tasks when combined with reasoning-oriented prompting.The cited studies report gains in quantitative reasoning, SQL-related generalization, and related settings.
- Finetuning with scratchpad prompting significantly improves generalization to longer problems, unlike standard fully supervised finetuning.
- Bootstrapping lets LLMs self-improve by iteratively generating rationales, finetuning on rationales leading to correct answers, and repeating the process.Each iteration can produce better training data for subsequent refinement.
4 Measuring Reasoning in Large Language Models
Reasoning evaluation commonly measures downstream task performance, but accuracy alone does not reveal whether models reason or use heuristics. The paper therefore surveys task benchmarks alongside more direct rationale-analysis methods.
- End-task performance, such as accuracy, is a common way to measure LLM reasoning abilities.
- Arithmetic Reasoning: Arithmetic reasoning benchmarks include GSM8K, Math, MathQA, SVAMP, AS-Div, AQuA, and MAWPS.
- Commonsense Reasoning: Commonsense reasoning benchmarks include CSQA, StrategyQA, and ARC.
- Symbolic Reasoning: Symbolic reasoning can be evaluated with Last Letter Concatenation and Coin Flip.
- Others: Other benchmarks cover broad reasoning skills, generalization, and table reasoning, including BIG-bench, SCAN, Anil et al.’s datasets, and WikiTableQA.BIG-bench includes over 200 tasks spanning skills such as date understanding, word sorting, and causal judgement.
- Most evaluations emphasize final-task accuracy rather than directly assessing reasoning steps, leaving uncertainty about whether predictions rely on reasoning or heuristics.
- ROSCOE and PrOntoQA support deeper evaluation through interpretable step-level metrics and synthetic ontologies, respectively.ROSCOE assesses semantic alignment, logical inference, semantic similarity, and language coherence.
- The survey concludes that detailed examination of rationale quality remains limited and calls for more formal analysis of LLM reasoning abilities.
5 Findings and Implications
The surveyed evidence portrays reasoning as an emergent, prompt-elicited capability with some human-like patterns, but also substantial weaknesses on complex reasoning tasks. These findings leave the nature and limits of LLM reasoning unresolved.
- Reasoning appears to emerge in sufficiently large LLMs, with major reasoning-task improvements reported around 100 billion parameters.The evidence includes models such as GPT-3 175B.
- The emergence of reasoning suggests using large models for general reasoning may be more effective than training small models for specific tasks, although its cause remains unclear.
- Chain-of-thought prompts improve reasoning-task performance, can produce valid proof steps, and often yield dramatic gains where standard prompting scales smoothly.CoT can also improve out-of-distribution robustness, but models may choose incorrect steps when multiple options exist.
- LLMs exhibit human-like content effects: prior knowledge, abstract reasoning, and conclusion believability influence their judgments of logical validity.
- LLMs remain unskilled at complex reasoning, including tasks involving implicature and commonsense planning that humans find relatively simple.
- Poor performance on relatively simple planning domains suggests existing benchmarks may be too simple to gauge true reasoning abilities accurately.
6 Reflection, Discussion, and Future Directions
Reasoning can improve LLM performance, robustness, and interpretability, but current evidence does not establish that models truly reason. The paper therefore calls for better training approaches, more meaningful evaluations, and deeper analysis of model capabilities.
- Why reasoning?: Reasoning supports complex tasks such as problem solving, decision making, and planning, while explicit rationales can improve model explainability and interpretability.
- Right task/application?: Reasoning benchmarks should use realistic applications such as decision making, legal reasoning, and scientific reasoning rather than relying mainly on artificial tasks.
- Are language models really able to reason?: Evidence for LLM reasoning includes strong task performance, chain-of-thought reasoning, and human-like content effects, but these findings remain inconclusive.
- Are language models really able to reason?: Heuristic solutions, incorrect or inconsistent rationales, and human-like patterns without human-like behavior complicate claims that LLMs genuinely reason.
- Are language models really able to reason?: LLMs struggle with complex reasoning and simple commonsense planning, while benchmark performance can be sensitive to training-data term frequencies.
- Are language models really able to reason?: The paper concludes that stronger analysis of training data, architecture, optimization objectives, and benchmarks is needed before determining LLMs’ reasoning abilities.
- Improving reasoning capabilities of LLMs.: Chain-of-thought prompting may elicit reasoning but cannot exceed current capabilities; training data, architecture, and objectives should encourage reasoning, including through bootstrapping.
7 Conclusion
The paper reviews reasoning in large language models, covering methods for eliciting and improving it, evaluation approaches, and prior findings. It concludes that the extent of true reasoning remains unclear and that further research is needed.
- The paper provides a detailed, up-to-date review of techniques, evaluation methods, benchmarks, findings, and implications concerning reasoning in LLMs.
- It remains unclear whether LLMs truly reason or instead solve problems using memorized patterns and heuristics.
- Further research is needed to understand and improve LLM reasoning capabilities and determine their potential across applications.
Limitations
The survey narrows its scope to deductive reasoning because reasoning encompasses many forms, and its coverage is bounded by the field’s rapid evolution and publication timing.
- Scope: The survey focuses on deductive reasoning, leaving inductive and abductive reasoning outside its primary coverage.The paper explains that reasoning is too broad to summarize comprehensively in one paper and identifies deductive reasoning as the most commonly studied form.
- Recency: New contributions may have appeared while the survey was being written, limiting its coverage of the rapidly evolving field.The authors specifically flag work released during or after 2023 as potentially outside the survey’s coverage.
- Related coverage: A parallel survey by Qiao et al. emphasizes reasoning through language-model prompting, offering an additional resource for related coverage.