PaperlayerView exact revision on arXiv

Source-linked AI summary

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou

arXiv:2201.11903v6cs.CLcs.AI

TL;DR

Challenging arithmetic, commonsense, and symbolic reasoning remains difficult for language models despite scaling. This paper introduces chain-of-thought prompting and finds that it improves reasoning performance, with PaLM 540B achieving new state-of-the-art results on several benchmarks.

  • Problem

    Scaling language models alone has not reliably produced strong performance on challenging arithmetic, commonsense, and symbolic reasoning tasks.

  • Method

    The paper elicits multi-step reasoning by providing few-shot exemplars containing inputs, chains of thought, and outputs.

  • Results

    Chain-of-thought prompting improves arithmetic, commonsense, and symbolic reasoning performance; PaLM 540B reaches new state of the art on GSM8K, SVAMP, and MAWPS.

  • Takeaways & Limitations

    Chain-of-thought prompting broadens the reasoning tasks that sufficiently large language models can perform successfully without finetuning.

  • Takeaways & Limitations

    The method provides no guarantee that generated reasoning paths are correct, which can produce both correct and incorrect answers.

Abstract

from arXiv · show

We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning. In particular, we show how such reasoning abilities emerge naturally in sufficiently large language models via a simple method called chain of thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. The empirical gains can be striking. For instance, prompting a 540B-parameter language model with just eight chain of thought exemplars achieves state of the art accuracy on the GSM8K benchmark of math word problems, surpassing even finetuned GPT-3 with a verifier.

1 Introduction

The paper investigates chain-of-thought prompting as a simple way to unlock reasoning abilities in large language models. Across arithmetic, commonsense, and symbolic reasoning benchmarks, it outperforms standard prompting and achieves new state-of-the-art performance on GSM8K with PaLM 540B.

  • Motivation: Scaling model size alone has not been sufficient for high performance on challenging arithmetic, commonsense, and symbolic reasoning tasks.
  • Method: Chain-of-thought prompting uses generated natural-language rationales and intermediate reasoning steps to lead to final answers.The approach is motivated by the potential benefits of natural-language rationales for arithmetic reasoning.
  • Limitations of existing methods: Rationale-based training and finetuning require costly, high-quality rationale datasets, while traditional few-shot prompting performs poorly on reasoning tasks.Traditional few-shot prompting often does not improve substantially with increasing language-model scale.
  • Results: Empirical evaluations cover arithmetic, commonsense, and symbolic reasoning benchmarks, with chain-of-thought prompting outperforming standard prompting, sometimes strikingly.The paper presents this comparison as evidence that reasoning ability can be elicited through prompting.
  • Results: PaLM 540B achieves new state-of-the-art performance on GSM8K using chain-of-thought prompting.GSM8K is a benchmark of math word problems, and the result outperforms standard prompting by a large margin.

2 Chain-of-Thought Prompting

Chain-of-thought prompting provides few-shot exemplars that enable sufficiently large language models to generate step-by-step reasoning chains. These chains can decompose multi-step problems into intermediate steps and offer an interpretable window into model behavior, with utility evaluated across arithmetic, commonsense, and symbolic reasoning.

  • Method: Chain-of-thought reasoning decomposes multi-step problems into intermediate steps before producing a final answer.The passage illustrates this process with a multi-step math word problem.
  • Method: Few-shot exemplars containing chain-of-thought reasoning enable sufficiently large language models to generate chains of thought.The method provides demonstrations of chain-of-thought reasoning within few-shot prompting exemplars.
  • Experiments: The paper evaluates chain-of-thought prompting for arithmetic, commonsense, and symbolic reasoning.These experiments are presented in Sections 3, 4, and 5, respectively.

3 Arithmetic Reasoning

The section evaluates chain-of-thought prompting on five arithmetic word-problem benchmarks against standard few-shot prompting. Gains emerge primarily at large model scale and on harder problems, with PaLM 540B achieving new state-of-the-art results on GSM8K, SVAMP, and MAWPS.

  • Benchmarks and setup: The evaluation covers five benchmarks: GSM8K, SVAMP, ASDiv, AQuA, and MAWPS.The study uses manually composed eight-exemplar chain-of-thought prompts, except AQuA, which uses four training-set exemplars because it is multiple choice.
  • Benchmarks and setup: Chain-of-thought prompting augments standard few-shot input–output exemplars with reasoning steps before the answer.Standard prompting asks the model to give the answer directly, whereas the proposed prompts include a chain of thought for each exemplar.
  • Main findings: Chain-of-thought prompting improves performance only at sufficiently large scales, with gains appearing for models of approximately 100B parameters or more.The passage characterizes this as an emergent ability of model scale and reports no positive impact for small models.
  • Main findings: For GSM8K, performance more than doubled for the largest GPT and PaLM models, while gains on the easiest single-step MAWPS subset were negative or very small.This supports larger benefits on more-complicated problems than on easy problems.
  • Main findings: PaLM 540B with chain-of-thought prompting achieved new state-of-the-art results on GSM8K, SVAMP, and MAWPS, and came within 2% of the prior best on AQuA and ASDiv.Standard prompting had already surpassed the prior best on SVAMP.
  • Ablations and robustness: Across ablations and prompt variations, performance was about the same as the baseline when computation was represented only by dots or when the chain of thought followed the answer.These results suggest that the produced reasoning sequence, rather than variable computation or post-answer access to knowledge alone, is important.
  • Ablations and robustness: Different annotators, independently written chains, exemplar orders, and exemplar counts all retained substantial advantages over standard prompting, despite annotation variance.The robustness findings include LaMDA 137B results on GSM8K and MAWPS and additional appendix analyses across datasets and models.

4 Commonsense Reasoning

Chain-of-thought prompting applies beyond mathematics to commonsense reasoning, a broad class of tasks involving physical and human interactions. Across diverse benchmarks, scaling model size improved standard prompting, while chain-of-thought prompting produced further gains, especially for PaLM 540B.

  • Commonsense reasoning tasks involve physical and human interactions under the presumption of general background knowledge, making them suitable for language-based chain-of-thought prompting.The paper notes that commonsense reasoning remains beyond current natural language understanding systems.
  • Benchmarks: The evaluation covers five datasets spanning diverse commonsense reasoning types, including CSQA, StrategyQA, and specialized BIG-bench tasks.CSQA often requires prior knowledge, while StrategyQA requires inferring a multi-hop strategy.
  • Results: For all tasks, larger models improved standard-prompting performance, and chain-of-thought prompting delivered further gains that appeared largest for PaLM 540B.The setup used manually composed few-shot chains for CSQA and StrategyQA, while BIG-bench tasks used the first ten evaluation examples as exemplars.
  • Results: 75.6% vs 69.4%: PaLM 540B outperformed the prior state of the art on StrategyQA with chain-of-thought prompting.

5 Symbolic Reasoning

Chain-of-thought prompting enables symbolic reasoning on two toy tasks that challenge standard prompting and supports length generalization to inputs longer than the few-shot exemplars. With PaLM 540B, it produces almost 100% solve rates in-domain, while OOD performance shows upward scaling curves despite lower accuracy.

  • Tasks: The evaluation uses last letter concatenation and coin flip, two symbolic reasoning tasks that are simple for humans but potentially challenging for language models.Last letter concatenation combines the final letters of words, while coin flip tracks whether a coin remains heads up after people flip or do not flip it.
  • Evaluation setup: Models are tested both in-domain and out-of-domain, with OOD examples containing more reasoning steps than the few-shot exemplars.For last letter concatenation, exemplars contain two-word names, while evaluation uses names with three and four words.
  • Results: With PaLM 540B, chain-of-thought prompting leads to almost 100% solve rates on the symbolic reasoning evaluations.Standard prompting already solves coin flip with PaLM 540, but not with LaMDA 137B.
  • Results: On OOD evaluations, standard prompting fails for both tasks, whereas chain-of-thought prompting produces upward scaling curves with lower performance than in-domain.These results indicate length generalization beyond the chains of thought seen in exemplars for sufficiently large language models.

6 Discussion

The discussion presents chain-of-thought prompting as a simple, broadly applicable way to elicit multi-step reasoning without finetuning. It highlights substantial gains across reasoning tasks while noting unresolved questions about reasoning and annotation costs.

  • Chain-of-thought prompting substantially improves arithmetic reasoning, with gains stronger than ablations and robust across annotators, exemplars, and language models.These findings are reported in Section 3.
  • Chain-of-thought reasoning generalizes across commonsense and symbolic tasks, including out-of-distribution generalization to longer sequence lengths.Its linguistic form supports broad applicability, and symbolic prompting facilitates OOD generalization.
  • All experiments elicited chain-of-thought reasoning by prompting off-the-shelf language models, without finetuning any models.This keeps the method simple while covering arithmetic, commonsense, and symbolic reasoning experiments.
  • Chain-of-thought prompting can produce dramatically increasing scaling curves on tasks where standard prompting has a flat scaling curve.The discussion frames this as expanding the set of tasks large language models can perform successfully.
  • Whether chain-of-thought prompting reflects actual neural-network reasoning remains open, and manually annotating chains could be prohibitive for finetuning.The discussion notes synthetic data generation or zero-shot generation as potential ways to address annotation costs.

7 Related Work

The paper situates its approach within two especially relevant directions: using intermediate steps for reasoning and improving prompting methods. It connects natural-language rationales for math problems with few-shot, learned-prompt, and instruction-based approaches.

  • Intermediate-step reasoning: Intermediate-step reasoning includes natural-language rationales that solve math word problems through a series of steps.Ling et al. (2017) pioneered this direction.
  • Intermediate-step reasoning: Natural-language rationales contrast with prior work using formal languages to reason.The cited formal-language literature includes Roy et al. (2015), Chiang and Chen (2019), Amini et al. (2019), and Chen et al. (2019).
  • Prompting: The work also relates to prompting research following the popularization of few-shot prompting.The passage attributes this popularization to Brown et al. (2020).
  • Prompting: Recent prompting approaches include automatically learning prompts and giving models task-describing instructions.The passage cites Lester et al. (2021), Wei et al. (2022a), Sanh et al. (2022), and Ouyang et al. (2022).

8 Conclusions

The paper concludes that chain-of-thought prompting is a simple, broadly applicable method for enhancing language-model reasoning, with chain-of-thought reasoning emerging at sufficient model scale.

  • 8 Conclusions: Chain-of-thought prompting enhances language-model reasoning across arithmetic, symbolic, and commonsense tasks.The authors present it as a simple and broadly applicable method.
  • 8 Conclusions: Chain-of-thought reasoning emerges with model scale, enabling sufficiently large language models to perform tasks with otherwise flat scaling curves.The conclusion identifies model scale as the condition under which this reasoning ability appears.

Checklist … A.2 What is the role of prompt engineering?

The paper reports that chain-of-thought prompting depends on model scale and prompt construction, while remaining broadly effective across annotators, exemplars, exemplar orders, and models. It also documents reproducibility practices and limitations, including factual unreliability and prompt-dependent performance variation.

  • Checklist: The authors report their main claims and limitations, include reproducibility materials for LaMDA and GPT-3, and provide multiple-seed standard deviations in Tables 6 and 7.They describe resources in Appendix E.2 but did not estimate total compute.
  • Checklist: Generated chains of thought are not always factual, and the authors do not recommend using them factually or in real-world scenarios.The paper reports no expected negative societal impacts directly resulting from its contributions.
  • A.1 Why does increasing model scale improve chain-of-thought prompting?: Chain-of-thought success emerges only at certain scales, hurting performance for most models smaller than 10B parameters.The authors characterize this behavior as emergent rather than predictable by extrapolating smaller-model performance.
  • A.1 Why does increasing model scale improve chain-of-thought prompting?: Scaling PaLM from 62B to 540B fixed a substantial portion of semantic-understanding, one-step-missing, and other errors.The analysis manually categorized 45 PaLM 62B errors as 20 semantic-understanding, 18 one-step-missing, and 7 other errors.
  • A.1 Why does increasing model scale improve chain-of-thought prompting?: The authors attribute scale-related success to multiple possible emergent abilities, including semantic understanding, symbol mapping, staying on topic, arithmetic ability, and faithfulness.They call for further study of how pretraining data, architecture, and optimization causally enable these capabilities.
  • A.2 What is the role of prompt engineering?: Chain-of-thought prompting outperformed baseline prompting for all three annotators across eight arithmetic, commonsense, and symbolic reasoning datasets.Annotations were written as step-by-step reasoning without specific stylistic instructions, allowing each annotator’s linguistic style.
  • A.2 What is the role of prompt engineering?: Prompt-order variation was relatively minimal in almost all cases, and gains generally persisted across varying numbers of few-shot exemplars.Different GSM8K exemplar sets also worked across all four arithmetic datasets, suggesting exemplars need not match the test distribution.
  • A.2 What is the role of prompt engineering?: With identical prompts, chain-of-thought improved all three models on every dataset except CSQA and StrategyQA for GPT-3, but prompt engineering still caused substantial variation.On coin flip, performance ranged from 99.6% for Annotator A to 71.4% for Annotator C, and the authors suggest automatically generating annotations as future work.

A.3 Will chain-of-thought prompting improve performance for my task of interest? · A.4 Why is prompting with the equation only not enough for some arithmetic reasoning datasets?

Chain-of-thought prompting helps most on challenging, multi-step tasks with large models and relatively flat scaling curves, while offering smaller benefits when these conditions are absent. Equation-only prompting helps many few-step arithmetic datasets but is insufficient for GSM8K’s semantic difficulty because natural-language intermediate steps support decomposition.

  • A.3 Will chain-of-thought prompting improve performance for my task of interest?: Chain-of-thought prompting helps most when tasks require challenging multi-step reasoning, use a large language model, and have a relatively flat scaling curve.Benefits are smaller when one or more of these conditions are not met.
  • A.3 Will chain-of-thought prompting improve performance for my task of interest?: The largest arithmetic gain occurs for PaLM 540B on GSM8K, which combines challenging multi-step problems with a flat scaling curve.This setting meets the conditions identified as most favorable for chain-of-thought prompting.
  • A.3 Will chain-of-thought prompting improve performance for my task of interest?: PaLM 540B already achieves 90% or higher on MAWPS subsets requiring only one or two steps, leaving less headroom for improvement.The cited subsets are SingleOP, SingleEq, and AddSub.
  • A.3 Will chain-of-thought prompting improve performance for my task of interest?: Although the paper focuses on arithmetic, commonsense, and symbolic multi-step reasoning, chain-of-thought prompting can potentially apply to any task humans solve using a chain of thought.This applicability is stated as a possibility in principle rather than an empirically established result for every task.
  • A.3 Will chain-of-thought prompting improve performance for my task of interest?: Empirical evaluation of chain-of-thought prompting on diverse tasks such as machine translation is left for future work.The paper does not report those evaluations in the cited discussion.
  • A.4 Why is prompting with the equation only not enough for some arithmetic reasoning datasets?: Equation-only prompting helps on many datasets, especially SVAMP, ASDiv, and MAWPS, when they require only a few reasoning steps.For GSM8K, equation-only prompting does not improve performance substantially.
  • A.4 Why is prompting with the equation only not enough for some arithmetic reasoning datasets?: GSM8K is too semantically challenging for the model to translate directly into a single equation, whereas chain of thought reasons through each part using natural-language intermediate steps.The example contrasts an equation-only wrong answer with a chain-of-thought correct solution.

B All Experimental Results … D.1 Correct Chain of Thought Analysis

The paper reports broad experimental results showing that chain-of-thought prompting improves reasoning across arithmetic, commonsense, and symbolic tasks, while analyzing when generated reasoning is correct. Related work situates the method among prompting, explanations, program execution, numeric reasoning, and intermediate-step approaches, and highlights its broad, no-finetuning setting.

  • B All Experimental Results: The experiments compare standard prompting with chain-of-thought prompting across varying model sizes and all evaluated benchmarks.The results include arithmetic, commonsense, and symbolic reasoning tasks.
  • B All Experimental Results: External calculator evaluation can correct arithmetic operations in otherwise correct generated chains of thought.The calculator is applied post hoc to equations produced in arithmetic chains, propagating results when multiple equations occur.
  • C Extended Related Work: Chain-of-thought prompting relates to general prompting and natural-language explanations by augmenting task outputs with intermediate reasoning rather than merely adding instructions or explanations around predictions.The broader approach is described as drawing on prompting and natural-language explanations.
  • C.3 Program synthesis and execution: The method builds on program synthesis and execution research that uses intermediate reasoning steps and large language models.Prior work includes architectural innovations and language-model approaches to program execution.
  • C.4 Numeric and logical reasoning: Unlike often task-specific finetuning approaches for numeric and logical reasoning, chain-of-thought prompting works across a broad range of tasks without finetuning.This contrast is explicitly drawn against work showing finetuning for longhand module operations.
  • C.5 Intermediate language steps: Prior intermediate-language-step methods have used training or finetuning to improve performance, robustness, training speed, or bias mitigation.The paper places chain-of-thought prompting within this broader history of benefits from producing intermediate steps.
  • D.1 Correct Chain of Thought Analysis: 49 of 50 analyzed correct GSM8K chains from LaMDA 137B had correct logic and math, while one reached the correct answer through incorrect reasoning.Five of the 49 correct chains had minor imperfections but retained coherent and understandable logic.
  • D.1 Correct Chain of Thought Analysis: Correct chains are mostly reliable for free-response math, but correct answers in multiple-choice or binary tasks can arise through incorrect reasoning, creating a factuality limitation.The analysis notes this issue for the evaluated commonsense reasoning datasets and calls for future factuality analysis.

D.2 Incorrect Chain of Thought Analysis … E.2 Computational Resources

The analysis shows that incorrect chains of thought arise from calculator, semantic, and coherence errors, while prompting improvements remain robust across exemplar counts. The appendices also document version changes and inference-only computational setups.

  • D.2 Incorrect Chain of Thought Analysis: 50 randomly sampled LaMDA 137B outputs incorrect on GSM8K were manually analyzed by categorizing the modifications needed to make each chain correct.The categorization was designed to clarify how the model could be improved.
  • D.2 Incorrect Chain of Thought Analysis: 8% of chains were correct except for calculator errors, and adding an external Python calculator raised LaMDA 137B GSM8K solve rate from 14.3% to 17.3%.These chains could be corrected by applying an external calculator to their equations.
  • D.2 Incorrect Chain of Thought Analysis: 27 of 50; 54% of remaining chains required substantial edits, usually for semantic understanding errors, while 8 of 27 also contained incoherent reasoning.Incoherence included statements that did not follow from earlier statements or violated basic world knowledge.
  • D.2 Incorrect Chain of Thought Analysis: Generated reasoning processes are not guaranteed to be coherent or factually correct and can produce either incorrect final answers or accidentally correct final answers.The paper connects this limitation to prior work evaluating the factuality of language-model generations and explanations.
  • D.3 Additional Robustness Analysis: Chain-of-thought prompting remained more effective than standard prompting across varying numbers of few-shot exemplars in LaMDA 137B robustness experiments.The main experiments used 8 exemplars because the input length was constrained to 1024 tokens; Figure 11 reports robustness on five datasets with mostly flat standard-prompting scaling curves.
  • E.2 Computational Resources: All three language models used prompting-based inference without finetuning; LaMDA 137B used TPU v3, PaLM 540B used TPU v4, and GPT-3 experiments used the public API.LaMDA used 64 chips / 128 cores, while PaLM used 192 chips / 384 cores.

E.3 Dataset Details and Licenses … H Appendix: Alternate Annotators for MWP

The appendices document dataset sources and licenses, provide examples and full chain-of-thought prompts across tasks, and compare math-word-problem exemplars written by alternate annotators.

  • E.3 Dataset Details and Licenses: The paper lists sources and licensing information for the arithmetic and commonsense datasets, while symbolic reasoning datasets were created synthetically.Arithmetic datasets include AddSub, MultiArith, ASDiv, AQuA, GSM8K, and SVAMP; commonsense datasets include CSQA, StrategyQA, Date Understanding, and Sports Understanding.
  • Arithmetic reasoning: The arithmetic datasets are identified with repository links and, where specified, licenses including CC BY 4.0, MIT, and AQuA’s linked license.The listed datasets are AddSub, MultiArith, ASDiv, AQuA, GSM8K, and SVAMP.
  • Commonsense reasoning: The commonsense materials include CSQA, open-domain StrategyQA, and Date Understanding and Sports Understanding from BIG-Bench.StrategyQA uses the question-only set, and the cited original StrategyQA dataset is MIT-licensed.
  • G Appendix: Full Prompts: The full-prompt appendix provides few-shot chain-of-thought exemplars for math word problems, AQuA, letter concatenation, coinflip, CSQA, StrategyQA, Date Understanding, Sports Understanding, and SayCan planning.The math-word-problem exemplars in Table 20 apply to all math-word-problem datasets except AQuA.
  • G Appendix: Full Prompts: Additional full-prompt tables cover Date Understanding, Sports Understanding, and SayCan robot planning tasks.These tables specify the task-specific exemplars used for prompting.
  • H Appendix: Alternate Annotators for MWP: Alternate-annotator math-word-problem prompts reuse the Table 20 exemplars but replace Annotator A’s chains with chains written by Annotator B or Annotator C.Annotators B and C are presented as different annotators; the description of Annotator B notes that annotators were co-authors familiar with the goal of chain-of-thought prompting.
Loading 2201.11903v6…