Source-linked AI summary
Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango
Aman Madaan, Amir Yazdanbakhsh
TL;DR
The paper asks why chain-of-thought prompting helps few-shot language-model reasoning and investigates the question by independently altering symbols, patterns, and text in controlled counterfactual prompts. Across the study, symbol identity is largely immaterial, while the interaction between text and patterns is central to CoT's effectiveness.
Problem
The reasons for chain-of-thought prompting's success in few-shot language-model reasoning remain unclear despite its broad use.
Method
The study defines symbols, patterns, and text, then conducts controlled counterfactual experiments that alter one prompt component at a time across tasks and models.
Results
The exact type of symbols is virtually immaterial to performance, whereas deleting symbols removes CoT gains and text-pattern interplay is central to its success.
Takeaways & Limitations
Intermediate steps may primarily guide models toward replicating task-relevant symbols, with text extracting commonsense and patterns reinforcing task understanding.
Takeaways & Limitations
Counterfactual examples can be misleading, so the study relies on interpretations showing consistent and systematic performance divergence.
Abstract
from arXiv · showhide
The past decade has witnessed dramatic gains in natural language processing and an unprecedented scaling of large language models. These developments have been accelerated by the advent of few-shot techniques such as chain of thought (CoT) prompting. Specifically, CoT pushes the performance of large language models in a few-shot setup by augmenting the prompts with intermediate steps. Despite impressive results across various tasks, the reasons behind their success have not been explored. This work uses counterfactual prompting to develop a deeper understanding of CoT-based few-shot prompting mechanisms in large language models. We first systematically identify and define the key components of a prompt: symbols, patterns, and text. Then, we devise and conduct an exhaustive set of experiments across four different tasks, by querying the model with counterfactual prompts where only one of these components is altered. Our experiments across three models (PaLM, GPT-3, and CODEX) reveal several surprising findings and brings into question the conventional wisdom around few-shot prompting. First, the presence of factual patterns in a prompt is practically immaterial to the success of CoT. Second, our results conclude that the primary role of intermediate steps may not be to facilitate learning how to solve a task. The intermediate steps are rather a beacon for the model to realize what symbols to replicate in the output to form a factual answer. Further, text imbues patterns with commonsense knowledge and meaning. Our empirical and qualitative analysis reveals that a symbiotic relationship between text and patterns explains the success of few-shot prompting: text helps extract commonsense from the question to help patterns, and patterns enforce task understanding and direct text generation.
1. INTRODUCTION
Few-shot prompting, especially chain-of-thought (CoT), improves large language model performance by adding intermediate natural-language rationales, but the mechanism behind this success remains unclear. The paper studies that mechanism by separating prompts into symbols, patterns, and text and testing them with controlled counterfactual prompts.
- CoT prompting improves few-shot performance by augmenting prompts with intermediate natural-language rationales.
- The success of CoT remains unclear despite its wide use across complex reasoning tasks.
- The study separates few-shot examples into symbols, patterns, and text for controlled analysis.
- Counterfactual prompts alter one component while keeping the others fixed, enabling systematic comparison across four reasoning tasks.
- The experiments analyze how performance diverges across counterfactual prompts to identify mechanisms underlying CoT.
2. COUNTERFACTUAL EXPLANATION FOR CHAIN OF THOUGHT
The paper uses counterfactual prompts to isolate how individual components of in-context examples affect CoT performance. Its analysis finds that actual numbers and credible thoughts are not essential, while removing patterns or text eliminates CoT's effectiveness.
- Counterfactual prompts change one aspect of in-context examples to measure its role in task performance.
- On GSM-8K, neither actual numbers nor credible thoughts are paramount to CoT success, while altered wording has a modest effect.
- Eradicating either patterns or text nullifies the efficacy of CoT.
- Comparable performance between a counterfactual prompt and the original is accompanied by comparable attention patterns.
- The study treats counterfactual explanation as informative but potentially misleading, retaining interpretations supported by consistent and systematic performance divergence.
3. EXPERIMENTAL SETUP
The experiments center on reasoning tasks where CoT substantially improves over direct prompting and examine three semantic prompt components: symbols, patterns, and text. The study primarily uses PaLM-62B and checks whether findings concur on GPT-3 and CODEX.
- PaLM-62B provides the main analysis, with GPT-3 and CODEX used for reproducibility and cross-model comparison.
- The study focuses on mathematical, commonsense, and symbolic reasoning tasks where CoT improves over direct prompting.
- 3.1. SEMANTIC COMPONENTS OF PROMPTS: Symbols are task-related token sequences, such as numerical quantities, players and activities, or date expressions.
- 3.1. SEMANTIC COMPONENTS OF PROMPTS: Patterns combine symbols and operators or structure prompts to reinforce task understanding, including equations and relational forms.
- 3.1. SEMANTIC COMPONENTS OF PROMPTS: Text comprises remaining tokens that outline the task, connect patterns to symbols, and contextualize symbols.
4. ROLE OF SYMBOLS
Across counterfactual symbol replacements, the type of symbol generally has little effect on CoT performance, although deleting symbols removes its gains. Attention analyses likewise show similar token focus across original and altered prompts.
- 60.6% → 80.0%: using larger integers considerably improves SORTING task solve rate over vanilla CoT.
- Replacing symbols with abstract placeholders or out-of-distribution values generally leaves task solve rates relatively unaffected.
- Deleting all symbols, including numbers and dates, nullifies the gains carried over by CoT.
- Counterfactual and original prompts produce moderate-to-substantial agreement in generated answers, with Cohen’s κ exceeding 0.4 to 0.61.
- Attention analyses find similar token-attention patterns across prompts despite symbol alterations, including when both prompts generate the correct answer.
5. ROLE OF PATTERNS
Patterns are necessary for effective CoT, but their factual accuracy is often unimportant; removing them can cause premature or incorrect conclusions, with task-dependent sensitivity to wrong patterns.
- 5. ROLE OF PATTERNS: Patterns are necessary but not sufficient for CoT success, and wrong patterns can have task-dependent consequences.The paper characterizes pattern accuracy as broadly unimportant while emphasizing that pattern absence can be catastrophic.
- 5. ROLE OF PATTERNS: Removing patterns causes models to reach premature conclusions or produce incorrect final calculations despite semantically correct steps.Examples include an incorrect final multiplication without explicit patterns and premature conclusions when patterns are absent.
- 5. ROLE OF PATTERNS: Pattern-only prompts are futile and annul the gains of CoT.
- 5. ROLE OF PATTERNS: Text modifications hamper performance, showing that patterns alone do not explain prompt effectiveness.
- 5. ROLE OF PATTERNS: Wrong patterns preserve solve rates in GSM-8K, DATE, and SORTING but reduce SPORTS accuracy from 93.67% to 46.02%.The reported robust task solve rates are 24.39% for GSM-8K, 44.84% for DATE, and 64.80% for SORTING.
6. ROLE OF TEXT
Text is necessary for CoT because it connects symbols and patterns to task-relevant entities and meaning, while alterations increasingly harm performance as textual variation grows.
- 6. ROLE OF TEXT: Text is necessary for correct conclusions, and concordant entities in questions and thoughts are crucial for CoT success.
- 6. ROLE OF TEXT: Changing thought entities causes SPORTS performance to resemble DIRECT prompting because the model disregards the altered examples.Replacing entities also demolishes factual correctness and can confuse the model.
- 6. ROLE OF TEXT: Altering grammatical style produces moderate to drastically negative solve-rate effects, depending on how text and patterns interact across tasks.Yodish alterations had moderate effects on GSM-8K and drastically negative repercussions on SPORTS and DATE.
- 6. ROLE OF TEXT: Across tasks, performance drops roughly in proportion to the extent of textual modification, while SORTING lacks text for this analysis.
7. SYMBIOSIS BETWEEN TEXT AND PATTERNS
Text and patterns operate symbiotically in CoT: patterns structure meaningful intermediate reasoning, while text supplies commonsense knowledge and contextual meaning.
- 7. SYMBIOSIS BETWEEN TEXT AND PATTERNS: Without text, patterns cannot recover model performance; without patterns, text is insufficient to instruct the correct answer.
- 7. SYMBIOSIS BETWEEN TEXT AND PATTERNS: COT is more effective on questions containing more patterns, which require more intermediate steps to reach correct answers.For GSM-8K, exclusively COT-solved questions averaged 3.98 entities versus 3.62 overall, with p = 0.04.
- 7. SYMBIOSIS BETWEEN TEXT AND PATTERNS: Text elicits commonsense knowledge, such as animal attributes and the date associated with Christmas Eve, that supports final answers.
- 7. SYMBIOSIS BETWEEN TEXT AND PATTERNS: COT handles activities with lower web popularity than pattern-only prompts, with exclusively COT-correct activities averaging 68k results versus 399k overall.The paper describes this as activities four times rarer on the web.
- 7. SYMBIOSIS BETWEEN TEXT AND PATTERNS: Patterns help generate meaningful text and reinforce connections among clauses, driving knowledge elicitation and correct conclusions.
8. CCOT: CONCISE CHAIN OF THOUGHT
The paper proposes concise chain of thought (CCOT) prompts that retain strong task-solving performance while using fewer input and output tokens than standard CoT.
- 8. CCOT: CONCISE CHAIN OF THOUGHT: CCOT outperforms COT while employing prompts with fewer tokens.The comparison covers PaLM-62B and PaLM-540B across the evaluated tasks.
- 8. CCOT: CONCISE CHAIN OF THOUGHT: CCOT reduces average input tokens by 1.39ˆ and output tokens by 1.58ˆ relative to COT.
- 8. CCOT: CONCISE CHAIN OF THOUGHT: CCOT’s task solve rate remains relatively high when scaling to PaLM-540B, highlighting its reported efficiency.
9. RELATED WORK
The paper situates its counterfactual analysis within research on prompting, in-context examples, rationale generation, prompt selection, and model explanations.
- Prior work shows that few-shot performance can remain strong despite incorrect labels, motivating closer study of what prompt components actually matter.
- Related prompting methods improve reasoning by generating sub-questions, rationales, multiple samples, or more carefully designed prompts.
- The paper’s findings that useful sub-steps are central to CoT resonate with least-to-most prompting’s decomposition strategy.
- The study extends work on rationale generation by examining intermediate reasoning in few-shot prompting rather than only fine-tuned models.
- Counterfactual prompts provide a tractable way to probe large language models, although they can produce misleading explanations because of artifacts.
10. CONCLUSIONS
The paper uses controlled counterfactual experiments to study how text, patterns, and symbols support chain-of-thought reasoning in large language models. Across PaLM, GPT-3, and CODEX, it finds that text-pattern symbiosis is especially important for producing correct answers.
- The study evaluates CoT reasoning in PaLM, GPT-3, and CODEX using systematically controlled counterfactual experiments.
- The results indicate that the symbiosis between text and patterns carries substantial weight in the chain-of-thought reasoning process.
- Text serves as a channel for extracting semantic patterns that help models form correct answers.
REPRODUCIBILITY STATEMENT
The paper documents its experimental setup, randomization controls, models, prompts, outputs, and reproduction resources. Code, prompts, generated outputs, and one-click reproduction scripts are provided where available.
- Multiple random seeds, averages, standard deviations, significance tests, and Cohen’s kappa measure variation and agreement across prompt conditions.
- The authors open-source code and provide one-click reproduction scripts for publicly available models.
- Experiments use PaLM, GPT-3, and CODEX, with PaLM results constrained by availability and some API-based models subject to usage limits.
- Prompts and model outputs are included in directories with standardized fields for questions, generated answers, and true answers.
- The included examples show question-answer prompts with intermediate rationales and final yes-or-no answers.
ETHICS STATEMENT
The paper frames its contribution as identifying behavioral patterns rather than directly enhancing language-model reasoning. It reports that text and patterns jointly support CoT, while acknowledging methodological, task, model, compute, and societal boundaries.
- The paper identifies systematic behavioral patterns in few-shot models but does not directly enhance their reasoning capabilities.
- The study incurs substantial compute costs, although few-shot prompting is presented as a way to avoid additional training and dataset collection.
- Counterfactual conclusions may be misleading because symbols, patterns, text, and outcomes can be spuriously correlated, and the evaluated tasks and datasets limit generalization.
- The study reports similar attention to text and patterns, consistent with their comparable contributions to CoT success.
- Across four models, the findings hold broadly: pattern correctness is immaterial, abstract symbols remain useful, text sensitivity tracks randomness, and CCOT matches or exceeds COT while being 20% shorter.
- Exact symbol types can be abstracted, but deleting symbols removes CoT gains in DATE and GSM-8K, with accuracy dropping to 29.94% and 11.98%, respectively.
F.4. ROLE OF DATASET FOR EFFECTIVE CHAIN OF THOUGHT
The effectiveness of chain-of-thought prompting depends strongly on dataset characteristics, especially the prevalence of small numbers and the information encoded in intermediate steps. Counterfactual experiments show that changing numerical values can severely reduce accuracy, while altering symbols or factual content may have little effect.
- Role of small numbers for maths: Small single-digit numbers account for 50% of the digits in GSM-8K, helping explain why language models perform better on its arithmetic questions.The paper characterizes model mathematics as largely limited to simple operations involving small numbers.
- Abstracting symbols: Changing the number distribution has no bearing on performance in the symbolic counterfactual experiments, indicating that actual prompt symbols are not essential to COT success.The study also reports negligible impact from abstracting PERSON or ACTIVITY, while abstracting an unstated SPORT causes a substantial performance drop.
- Abstracting symbols: In SPORTS, replacing factual content already present in the question preserves extraction patterns, whereas replacing information that must be inferred substantially harms performance.This contrast separates copying information from extracting commonsense relations.
- Text and patterns: Forcing additional text can cause errors when the model makes faulty connections, showing that intermediate content can be harmful rather than uniformly beneficial.The paper locates detailed examples of these failures in its qualitative analysis.
G. ADDITIONAL EXPERIMENTS ON ROLE OF PATTERNS IN COT
Additional experiments show that patterns are necessary but not sufficient for effective chain-of-thought prompting, while their factual accuracy matters unevenly across tasks. Text and diverse patterns jointly help the model understand the task, but incorrect patterns can mislead or sometimes improve performance.
- G. ADDITIONAL EXPERIMENTS ON ROLE OF PATTERNS IN COT: The authors conclude that patterns are necessary but not sufficient, and that wrong patterns have task-dependent consequences.This conclusion summarizes the additional experiments rather than asserting universal robustness.
- G.1. IDENTIFYING PATTERNS: Patterns comprise symbol compositions or recurring prompt structures, but their isolation is dataset-dependent and may be difficult for tasks such as DATE.In SPORTS, the pattern explicitly links a person and activity to two sports in a shared sentence structure.
- G.2. ROLE OF PATTERNS: Cpat_none consistently underperforms COT and approaches DIRECT, while still exceeding DIRECT across datasets, showing that patterns are crucial but text also contributes.The corresponding table summarizes pattern-removal results across the evaluated datasets.
- G.2. ROLE OF PATTERNS: Pattern-only prompts annul COT gains and achieve solve rates similar to DIRECT, demonstrating that patterns without accompanying text are insufficient.In SORTING, pattern-only prompts are equivalent to DIRECT because numbers and less-than signs are classified as patterns.
- G.3. CONFUSING MODEL WITH INCORRECT THOUGHTS: Wrong patterns are relatively tolerated overall, but their effects depend on the task: SORTING rises from 46.0% to 64.8%, whereas misleading SPORTS facts can produce deceptive outputs.The SPORTS outputs remain structurally valid even when their factual associations are wrong.
- G.4. STRENGTH IN DIFFERENCES: Restricting SORTING thoughts to sequences beginning with only “1” or only “2” yields solve rates below 40%, whereas mixing both patterns reaches 87.78%.The comparison uses a balanced test set with 50 questions beginning with each number.
H. ADDITIONAL EXPERIMENTS ON ROLE OF TEXT
Text and patterns jointly support COT: text supplies semantic meaning and commonsense, while patterns guide task-relevant generation and factual connections. Altering text affects performance according to task structure, with grammatical changes especially harmful when thoughts require complex relationships.
- Semantically similar entity substitutions outperform direct prompting but remain below standard COT, because meaningful text and abstraction preserve some task structure.SPORTS is harder because substitutions can destroy factual correctness and confuse the model.
- Changing only the questions has little effect, whereas changing thought text causes moderate to severe degradation across GSM-8K, SPORTS, and DATE.The contrast indicates that thought text is more consequential than question wording in these variants.
- Grammar changes are tolerated in GSM-8K but harm SPORTS and DATE when altered sentence structures disrupt factual or temporal relationships.GSM-8K has simpler sentence structure, while DATE requires explicitly expressing complex time-date relations.
- Text and patterns form a symbiotic relationship: text supplies meaning, while patterns guide intermediate generation and factual connections.Removing either component prevents COT from fully recovering model performance.
- Patterns help the model extract relevant information and connect clauses, preventing semantically correct intermediate steps from ending in incorrect conclusions.In DATE and related examples, explicit patterns support factual and mathematically correct outcomes.