Source-linked AI summary
Thinking Fast and Slow in Large Language Models
Thilo Hagendorff, Sarah Fabi, Michal Kosinski
TL;DR
The paper asks how LLMs reason and whether they exhibit human-like intuitive errors. Using CRTs and semantic illusions across models, plus prompting and training-example interventions, it finds that larger ChatGPT models avoid these errors more often, while the authors note uncertainty about what drove the shift.
Problem
As LLM applications expand and their emergent abilities become harder to anticipate, the study examines how they reason and make decisions.
Method
The authors administer CRTs and semantic-illusion tasks across GPT models and humans, then test chain-of-thought prompting and training examples.
Results
ChatGPT-3.5 and ChatGPT-4 answered CRT tasks correctly at 59% and 96%, respectively, versus 5% for GPT-3-davinci-003; training examples also increased GPT-3-davinci-003 accuracy across CRT types.
Takeaways & Limitations
Psychological tests such as CRTs and semantic illusions can reveal emergent, human-like patterns and cognitive-error avoidance in LLM behavior.
Takeaways & Limitations
The authors can only speculate about the causes of the sharp GPT-3-to-ChatGPT shift because OpenAI provides limited technical information.
Abstract
from arXiv · showhide
Large language models (LLMs) are currently at the forefront of intertwining AI systems with human communication and everyday life. Therefore, it is of great importance to evaluate their emerging abilities. In this study, we show that LLMs like GPT-3 exhibit behavior that strikingly resembles human-like intuition - and the cognitive errors that come with it. However, LLMs with higher cognitive capabilities, in particular ChatGPT and GPT-4, learned to avoid succumbing to these errors and perform in a hyperrational manner. For our experiments, we probe LLMs with the Cognitive Reflection Test (CRT) as well as semantic illusions that were originally designed to investigate intuitive decision-making in humans. Our study demonstrates that investigating LLMs with methods from psychology has the potential to reveal otherwise unknown emergent traits.
Methods
The study administered CRT tasks to LLMs and humans, classifying responses as correct, intuitive, or atypical. It compared model generations and tested whether chain-of-thought prompting altered CRT performance.
- Methods: The researchers administered CRT tasks to GPT models ranging from GPT-1 to ChatGPT-4 and to human participants, with responses manually scored.Model temperature was set to 0; human participants were recruited through Prolific.io.
- Methods: CRT responses were categorized as correct, intuitive but incorrect, or atypical, with written chain-of-thought reasoning separately recorded.The intuitive category captured the trap targeted by each task, while atypical responses included other incorrect or evasive answers.
- Study 1: Cognitive Reflection Test: Early models often produced atypical answers, whereas GPT-3-davinci models produced intuitive errors on 70%–90% of tasks, compared with 55% for humans.Atypical responses declined as models’ task comprehension increased, while intuitive responses became more common.
- Study 1: Cognitive Reflection Test: ChatGPT-3.5 answered 59% of CRT tasks correctly and ChatGPT-4 answered 96%, versus 5% for GPT-3-davinci-003 and 38% for humans.Correct performance was accompanied by intuitive responses of 15% for ChatGPT-3.5 and 0% for ChatGPT-4, compared with 80% for GPT-3-davinci-003 and 55% for humans.
- Study 2: Chain-of-Thought Reasoning in CRT Tasks: The authors interpret chain-of-thought reasoning as a System-2-like process assembled through successive next-word generation, while ChatGPT can also solve CRT tasks without it.They compare this process with using a notebook to break mathematical problems into subtasks.
Study 3: Improving LLMs’ Intuition in CRT Tasks
Study 3 tested whether GPT-3-davinci-003 could develop more accurate, System-1-like responses to CRT tasks through exposure to solved examples. Accuracy increased with additional examples, with the size and speed of improvement depending on CRT task type.
- Study design: GPT-3-davinci-003 was prefixed with 0 to 49 solved CRT tasks of the same type to develop task-specific intuitions.The tasks were semantically similar within each type, enabling example-based exposure to support the intended System-1-like response pattern.
- Results by task type: 90 percentage points: CRT Type 2 accuracy rose from 2% to 92% after two examples.These tasks could be solved by repeating the duration listed in the task.
- Results by task type: 80 percentage points: CRT Type 3 accuracy increased from 12% to 92% after seven training examples.Type 3 tasks required subtracting one unit from the total time.
- Results by task type: 78 percentage points: CRT Type 1 accuracy increased from 0% to about 78% after 30 examples.Type 1 tasks required the greatest number of examples for the model’s accuracy to improve substantially.
- Overall pattern: Across CRT task types, GPT-3-davinci-003’s ability to answer correctly rather than intuitively increased with each additional example.The improvement was strongest for the simpler Type 2 tasks and slower for Type 1 tasks.
Supplementary Materials
The supplementary materials provide the task sets, response-scoring framework, and knowledge checks used across the CRT and semantic-illusion studies. They include bespoke task examples, task lists, and prompting variants used to examine intuitive and deliberate reasoning.
- Task materials: Table S1 lists all CRT tasks and semantic illusions used in the study.The supplementary task materials include numerical CRT items and semantically diverse illusion questions.
- CRT task types: CRT examples cover arithmetic traps involving price differences, proportional reasoning, production rates, and exponential doubling.The supplementary tasks include paired-cost problems, rate problems, and doubling scenarios involving viruses, mold, forests, lakes, and other settings.
- Task materials: The semantic-illusion materials use disguised factual errors, including questions about historical, cultural, scientific, and geographic knowledge.Examples include questions about South Korea, World War II, mythology, literature, science, and the location of la Sagrada Familia.
- Scoring and comparison: Responses were categorized as correct, intuitive but incorrect, or atypical, with Table S2 reporting fractions for LLMs and humans.The materials also distinguish responses preceded by written chain-of-thought reasoning using black-dot markers.
- Knowledge checks: Table S3 contains questions testing whether models possess the knowledge needed to recognize semantic-illusion errors.These checks separate an intuitive answer caused by overlooking the error from one given without the relevant factual knowledge.