Source-linked AI summary
Evaluating Large Language Models in Theory of Mind Tasks
Michal Kosinski
TL;DR
The paper asks whether LLMs can perform human-standard false-belief tasks and whether ToM-like abilities might arise without explicit engineering. It evaluates eleven models using diverse false-belief scenarios and matched controls, finding sharply improved performance in newer models, especially ChatGPT-4. The authors discuss the implications while noting that task success does not by itself establish genuine belief tracking.
Problem
AI has historically trailed humans on ToM despite strong performance in other tasks, motivating tests of whether LLMs can track unobservable mental states.
Method
The study administers 40 false-belief tasks with matched true-belief controls and reversed scenarios to eleven LLMs, requiring all 16 prompts in a task to be correct.
Results
ChatGPT-4 solved 75% of tasks, while GPT-3-davinci-003 and ChatGPT-3.5-turbo solved 20% and older models solved none; performance improved gradually across models.
Takeaways & Limitations
The findings indicate increasingly ToM-like behavior in LLMs and have implications for human–AI interaction, including improved communication as well as risks of deceit and manipulation.
Takeaways & Limitations
Success on false-belief tasks does not automatically demonstrate genuine belief tracking because guessing, memorization, and superficial task cues can produce correct responses.
Abstract
from arXiv · showhide
Eleven Large Language Models (LLMs) were assessed using a custom-made battery of false-belief tasks, considered a gold standard in testing Theory of Mind (ToM) in humans. The battery included 640 prompts spread across 40 diverse tasks, each one including a false-belief scenario, three closely matched true-belief control scenarios, and the reversed versions of all four. To solve a single task, a model needed to correctly answer 16 prompts across all eight scenarios. Smaller and older models solved no tasks; GPT-3-davinci-003 (from November 2022) and ChatGPT-3.5-turbo (from March 2023) solved 20% of the tasks; ChatGPT-4 (from June 2023) solved 75% of the tasks, matching the performance of six-year-old children observed in past studies. We explore the potential interpretation of these findings, including the intriguing possibility that ToM, previously considered exclusive to humans, may have spontaneously emerged as a byproduct of LLMs' improving language skills.
Significance
Theory of mind (ToM) is central to human social cognition, yet AI has historically lagged in tracking unobservable mental states. This work tests whether recent LLMs can solve human-standard false-belief tasks and considers whether such abilities may emerge through language training.
- Theory of mind tracks others’ unobservable knowledge, intentions, beliefs, and desires, supporting social interaction, communication, empathy, and moral judgment.
- AI agents with ToM could better anticipate human intentions, improving applications such as self-driving cars and virtual assistants.
- The paper discusses whether ToM-like behavior emerged as LLMs were trained for other goals, while distinguishing behavioral performance from attribution of human cognitive capabilities.
- The study evaluates recent LLMs on false-belief tasks regarded as a gold standard for assessing ToM in humans.
- Eleven LLMs were tested, with performance gradually improving and ChatGPT-4 solving 75% of false-belief tasks.
Study 1.1: Unexpected Contents Task (aka Smarties Task)
Study 1 introduces the Unexpected Contents, or Smarties, task, which tests whether models distinguish a container’s actual contents from a protagonist’s false belief. ChatGPT-4’s responses correctly tracked both the contents and the protagonist’s belief in the example task.
- The Unexpected Contents Task presents an opaque, mislabeled container and requires recognizing that an uninformed protagonist falsely assumes the label matches the contents.
- The study uses prompts targeting the container’s actual contents and the protagonist’s belief, with answer-completion formats chosen for efficient evaluation.
- ChatGPT-4 answered “popcorn” with Ppopcorn = 100% when asked to complete the story after Sam inspected the bag.
- ChatGPT-4 answered “chocolate” with Pchocolate = 100% when asked what Sam reported finding, indicating prediction of her false belief.
- Reversing container contents and labels, and balancing the words’ frequencies, were used to reduce order and frequency-based response biases.
Study 1.2: Reversed Scenario.
The reversed scenario swaps the container’s contents and label to test whether responses depend on fixed word associations or story order. The study also adds true-belief controls because correct false-belief answers can arise without tracking mental states.
- The reversed task changes the bag’s contents to chocolate and its label to popcorn while preserving the protagonist’s lack of visual access.
- ChatGPT-4 answered “chocolate” with Pchocolate = 99.7% for the actual contents and “popcorn” with Ppopcorn = 100% for Sam’s reported belief.
- False-belief tasks may be solved by assuming the protagonist is wrong or by using superficial cues, without tracking the protagonist’s mental state.
- Three true-belief controls minimally modify each false-belief story to test guessing, comprehension, and reliance on nonmental shortcuts.
- In one control, allowing the protagonist to inspect the container before reading its label removes the condition necessary for false-belief formation.
Open container.
The open-container tasks tested whether ChatGPT-4 tracked both the bag’s actual contents and Sam’s belief as story information unfolded. Its responses distinguished the true-belief controls from the false-belief scenario and adjusted predictions when labels, contents, or information changed.
- False-belief formation: 88.5% probability of popcorn and 11.5% probability of chocolate appeared when Sam reported the contents despite the contradictory label, indicating residual uncertainty about her belief.The corresponding direct-content prompt received 100% popcorn.
- True-belief controls: 99.7% probability of popcorn was assigned when the label matched the contents, showing that the model anticipated Sam’s true belief in this control.The same probability was produced for both the direct-content and friend-report prompts.
- True-belief controls: 100% probability of popcorn for direct content and 99.6% for Sam’s report followed information from a cousin that corrected the misleading label.This informed-protagonist control prevented false-belief formation by explicitly telling Sam the actual contents.
- Open-container tracking: ChatGPT-4 assigned 100% probability to popcorn after reading that the bag contained popcorn, while its initial completions favored unspecified valuables when only the prefix was available.The model’s content prediction remained stable as the story continued, even after the misleading label appeared.
- False-belief formation: ChatGPT-4 predicted Sam’s belief as popcorn before the false label was revealed, then changed its prediction once the label supplied evidence for a false belief.The passage describing the right panel identifies this shift as the model’s prediction of Sam’s belief rather than the bag’s actual contents.
- Transfer comparison: In the unexpected-transfer version, ChatGPT-4 predicted that John would search the basket, find it empty, discover the cat in the box, and be surprised.The reversed scenario swapped the initial and final containers, while the sentence-by-sentence procedure tested how predictions evolved as information was revealed.
Study 2.2: Reversed Scenario.
The reversed unexpected-transfer scenario exchanged the box and basket positions while preserving the protagonist’s absence during the transfer. ChatGPT-4’s prompts reflected the reversed final location and the protagonist’s outdated belief.
- Study 2.2: Reversed Scenario: 99.9% probability of the cat being in the basket was assigned after the reversed transfer.John initially placed the cat in the box, and Mark later moved it into the basket.
- Study 2.2: Reversed Scenario: 100% probability of John looking for the cat in the box showed that ChatGPT-4 tracked his false belief in the reversed scenario.The model’s predicted search location matched John’s last observed location for the cat.
- Study 2.2: Reversed Scenario: The task’s true-belief controls removed the conditions required for forming a false belief.The controls included allowing the protagonist to observe the transfer.
Present protagonist.
The present-protagonist control placed John in the room when the cat moved from the basket to the box, then tested his expected search location. ChatGPT-4 answered both control prompts with complete confidence.
- Present protagonist: 100% probability was assigned to the cat jumping out of the box after Mark placed it there.The scenario had John and Mark leave only after the transfer was completed.
- Present protagonist: 100% probability was assigned to John looking for the cat in the box, matching his knowledge of the transfer’s outcome.This control tested a protagonist who was present before leaving the room.
- Present protagonist: ChatGPT-4 passed the present-protagonist control, supporting its adjustment to a protagonist who was informed by direct observation.The passage describes this as a control for false-belief reasoning.
Informed protagonist.
The informed-protagonist control told John that Mark had moved the cat to the box after John left. ChatGPT-4 correctly predicted the cat’s location and John’s informed belief.
- Informed protagonist: 100% probability was assigned to the cat jumping out of the box in the informed-protagonist control.John was explicitly told that the cat had been moved and was now in the box.
- Informed protagonist: 100% probability was assigned to John looking for the cat in the box after Mark informed him of the transfer.The explicit phone call removed the protagonist’s false belief.
- Informed protagonist: ChatGPT-4 passed this control by anticipating that John’s belief should not be false after receiving the information.The control directly removed the protagonist’s unawareness of the transfer.
No transfer.
In the no-transfer scenario, ChatGPT-4 tracked both the cat’s actual location and John’s outdated belief about where the cat remained.
- ChatGPT-4 assigned 100% probability to the cat jumping out of the basket after John placed it there.
- The model reduced the predicted basket completion to 0% after Mark moved the cat to the box.
- Before location information was provided, ChatGPT-4 predicted that John would search in the box 61% of the time and in the basket 31% of the time.
- After John placed the cat in the basket, the probability that John would look there rose to about 100% and remained unchanged after Mark moved the cat.
Study 3: The Emergence of the Ability to Solve ToM Tasks.
Study 3 tested eleven LLMs with a conservative all-prompts scoring rule and found substantial performance differences across model generations. ChatGPT-4 solved most tasks, while true-belief controls and task revisions lowered measured performance.
- 40 false-belief tasks were evaluated across eleven LLMs, with one point requiring 16 correct answers across eight scenarios.
- About 1% of irregular responses were judged correct during manual review, while remaining irregular responses were classified as incorrect.
- 75% of tasks were solved by ChatGPT-4 (CI95% = [66%, 84%]), compared with 20% by GPT-3-davinci-003 and ChatGPT-3.5-turbo.
- Older models failed false-belief scenarios or controls in all tasks, whereas GPT-3-davinci-002 solved 5% of tasks (CI95% = [0%, 10%]).
- ChatGPT-4 solved 90% of Unexpected Contents tasks and 60% of Unexpected Transfer tasks (Δ = 30%; χ2 = 8.07, P = 0.01).
- ChatGPT-4’s performance remained high after item updates and true-belief controls, declining from 95% before modifications to 75% afterward (Δ = 20%; χ2 = 11, P < 0.001).
Discussion
The discussion interprets improving LLM performance as potentially related to language proficiency while emphasizing that success and failure on false-belief tasks do not by themselves establish or rule out human-like ToM.
- Performance pattern: ChatGPT-4 solved 75% of tasks, substantially outperforming earlier models and matching six-year-old children’s performance.
- Performance pattern: Performance improvement correlated with language proficiency, and results across task types were strongly correlated (R = 0.98; CI95% = [0.92, 0.99]).
- Limitations: Failures can reflect confounding factors such as missing commonsense knowledge or limitations of test items, procedures, and scoring keys.
- Interpretation: Random responding should succeed on a complete task only once in 65,536 tasks on average, but memorization, guessing, and other non-ToM strategies remain possible explanations.
- Interpretation: The bespoke battery used 40 scenarios, matched true-belief controls, and reversed versions to reduce the risk that models merely recalled previously seen solutions.
Conclusion
The conclusion frames apparent ToM-like behavior in LLMs as practically consequential and argues that psychological science can help interpret increasingly complex AI systems. It also suggests that studying LLMs may expand psychological science because language generation engages human-like psychological processes.
- Conclusion: Distinguishing genuine ToM from ToM-like behavior is philosophically important, but Turing argued that the distinction has limited practical value in interaction.The conclusion adopts a behaviorally oriented perspective on machine thought and ToM.
- Conclusion: Machines that track mental states may improve interaction and communication while also enabling deceit, manipulation, and psychological abuse.Such systems may additionally appear more human-like, affecting human–AI interactions and AI’s societal and legal role.
- Conclusion: Psychological science may help keep pace with rapidly evolving AI and improve the safe and effective use of these technologies.The conclusion compares understanding complex AI systems with studying the human brain as a black box.
- Conclusion: LLMs predicting human-generated language may model psychological processes, including theory of mind, learning, self-awareness, reasoning, emotions, and empathy.This suggests that studying AI can also advance psychological science.
Data, Materials, and Software Availability.
The paper acknowledges assistance with study materials and coding and identifies an earlier version as an arXiv preprint.
- Data, Materials, and Software Availability.: The authors acknowledge assistance with study materials and code preparation.The acknowledgment names Isabelle Abraham and Floriane Leynaud.
- Data, Materials, and Software Availability.: An earlier manuscript version was published as an arXiv preprint.The cited preprint is available at arXiv:2302.02083.