Source-linked AI summary
Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks
Tomer Ullman
TL;DR
The paper asks whether apparent LLM success on Theory-of-Mind tasks reflects robust mental-state reasoning, an important question because ToM is a pillar of common-sense reasoning and supports interaction with people. It tests a recent success case with directed task perturbations and argues that small, ToM-preserving changes expose failures, motivating skepticism about current and future machine ToM evaluations.
Problem
Whether machines robustly reproduce Theory-of-Mind reasoning remains uncertain despite its importance as a pillar of common-sense reasoning and its usefulness for interaction with people.
Method
The paper evaluates GPT-3.5 with the prior vignette-and-completion-probability setup, applying directed perturbations that preserve Theory-of-Mind principles.
Results
Current LLMs such as GPT-3.5 produce reasonable responses to basic ToM vignettes but fail when simple ToM-preserving perturbations are introduced.
Takeaways & Limitations
Evaluation of intuitive psychology in LLMs should begin skeptically, treating outlying failures as instructive rather than averaging them away.
Takeaways & Limitations
Robustness findings about a particular LLM can be overtaken as newer systems appear, and systematic benchmarks may be memorized rather than reflect ToM reasoning.
Abstract
from arXiv · showhide
Intuitive psychology is a pillar of common-sense reasoning. The replication of this reasoning in machine intelligence is an important stepping-stone on the way to human-like artificial intelligence. Several recent tasks and benchmarks for examining this reasoning in Large-Large Models have focused in particular on belief attribution in Theory-of-Mind tasks. These tasks have shown both successes and failures. We consider in particular a recent purported success case, and show that small variations that maintain the principles of ToM turn the results on their head. We argue that in general, the zero-hypothesis for model evaluation in intuitive psychology should be skeptical, and that outlying failure cases should outweigh average success rates. We also consider what possible future successes on Theory-of-Mind tasks by more powerful LLMs would mean for ToM tasks with people.
1 Introduction
Theory-of-Mind is a core form of common-sense reasoning whose machine evaluation has produced both successes and failures. This paper tests a purported LLM success with principled perturbations and argues that isolated failures should outweigh aggregate success rates when assessing robust reasoning.
- Theory-of-Mind describes adult-level reasoning about other people's mental states, including beliefs, false beliefs, and perceptual access.
- Theory-of-Mind is considered a pillar of common-sense reasoning and could help machines reason about people even without human-like mental states.
- Tests of Theory-of-Mind are increasingly applied to machines through visual, question-answering, and text-based tasks.
- Recent LLM work reported performance on ToM tasks comparable to 9-year-old children, while other tests produced more pessimistic evaluations.
- The paper examines whether reported LLM successes remain robust under directed perturbations that preserve the principles of Theory-of-Mind.
- A single instructive failure can outweigh more than 99% aggregate success when simple input alterations reveal that a system has memorized patterns rather than learned a general algorithm.
2 Examining the robustness of current LLMs on ToM tasks
The paper examines GPT-3.5 using the same vignette-and-completion-probability setup as prior work, while acknowledging that robustness assessment is difficult as new systems continually appear. Its evaluation is framed as an ongoing scientific comparison rather than a negative judgment of the earlier study.
- The study focuses on GPT-3.5, the strongest model in the prior work, and treats it as a threshold for evaluating less powerful models.
- The evaluation reproduces the prior setup by presenting vignettes to an LLM and examining probabilities for alternative completions.
- The authors characterize their assessment of GPT-3.5 as not responding robustly to ToM tasks, while treating the prior work as part of ongoing scientific comparison.
2.1 Unexpected Contents
The unexpected-contents task tests whether a model attributes beliefs from another person's information rather than from the true contents. GPT-3.5's apparent success reverses under small Theory-of-Mind-preserving variations, including perceptual access, unreadable labels, testimony, and self-authored information.
- Task: The smarties task requires distinguishing the participant's knowledge of the true contents from another person's potentially false belief.In the standard setup, an opaque container contains an unexpected item, while the uninformed person is expected to believe the label.
- Original result: In the original unexpected-contents evaluation, GPT-3.5 assigned popcorn with 100% probability for the content prompt but chocolate with 99% probability for the belief prompt.For the second belief prompt, it assigned 82% to chocolate and 14% to popcorn.
- Variation 1A: Transparent Access: With a transparent bag, GPT-3.5 still assigned Sam belief in chocolate with 95% probability, despite her seeing the popcorn directly.The variation changes perceptual access while retaining the misleading label.
- Variation 1B: Uninformative Label: When Sam cannot read the label, GPT-3.5 assigned chocolate with 98% probability, treating information unavailable to her as belief-relevant.The authors state that the label is meaningless to Sam in this variation.
- Variation 1C: Trustworthy Testimony: When a trusted friend tells Sam the bag contains popcorn and to ignore the label, GPT-3.5 still assigned chocolate with 97% probability.The authors interpret this as sensitivity to small irrelevant perturbations rather than a need to posit that Sam forgot or changed her mind.
- Pattern across variations: Across tested variations, placing the label-reading event at the end of the story strongly affected the belief-prompt answer.In the self-authored-label case, the model still assigned chocolate with 87% probability.
2.2 Unexpected Transfer
The authors perturb an unexpected-transfer Theory-of-Mind task in simple ways that preserve its principles, but GPT-3.5 continues attributing outdated beliefs and actions.
- Base task: The unexpected-transfer task requires tracking both the changed world state and the protagonist’s outdated belief to predict where the protagonist will look.In the classic Sally-Anne structure, the protagonist is unaware that an object has moved.
- Baseline: GPT-3.5 originally assigned P(basket) = 98% to both John’s belief and his expected search location.This result was presented as evidence that the model correctly inferred John’s mental states.
- Variation 2A: With transparent containers, GPT-3.5 still predicted that John believed the cat remained in the initial chest and would search there.The model assigned Pchest = 94% for John’s belief and Pchest = 90% for his search, despite direct perceptual access.
- Variation 2B: Changing the relation from ‘in’ to ‘on’ left GPT-3.5 predicting that John believed the cat was on the basket and would search there.The corresponding probabilities were Pbasket = 97% for belief and Pbasket = 74% for search.
- Variation 2C: Truthful testimony about the cat’s relocation failed to update GPT-3.5’s predictions: it still assigned Pbasket = 97% to John’s belief and Pbasket = 94% to his search.Similar mistakes occurred when John asked Mark to move the cat and Mark agreed.
- Overall pattern: Across these perturbations, the model continued predicting searches in the initial location even when the protagonist should no longer search there.The authors note that some variations add information or complexity, including changed objects and additional interactions.
- Variation 2D: When asked about Mark, the person who moved the cat, GPT-3.5 still predicted that Mark believed the cat was in the basket and might search there.The model assigned Pbasket = 99% to Mark’s belief and split the search probabilities between box and basket.
3 Discussion
The authors conclude that GPT-3.5’s success on basic Theory-of-Mind vignettes does not establish genuine Theory-of-Mind, because simple principled perturbations reverse its performance. They recommend skeptical evaluation while allowing that future systems could implement ToM through more explicit computational models.
- Conclusion: GPT-3.5 produces reasonable answers to basic ToM vignettes, but simple perturbations preserving ToM principles reverse its answers.The authors therefore judge that current LLMs have not learned anything like Theory-of-Mind.
- Skeptical evaluation: The authors recommend strongly presuming that LLMs lack Theory-of-Mind when evaluating claims of spontaneous emergence.They argue that people are biased to anthropomorphize behavior by attributing animacy and mental states.
- Future direction: The paper holds that Theory-of-Mind could in principle be replicated in silicon and suggests integrating computational ToM models with language models.This approach is contrasted with expecting ToM to emerge spontaneously from additional linguistic data.
- Evaluation limitation: A systematic generator or exhaustive benchmark could let LLMs memorize failure modes without resolving whether they learned Theory-of-Mind reasoning.The authors frame this as part of the broader problem of evaluating model generalization.
- Human versus machine assessment: Even if future LLMs pass classic ToM tests and their variations, ToM tests could remain valid for people while machine possession of ToM remains uncertain.The authors distinguish behavioral performance from the algorithms and processes generating that behavior.
- Interpretation: The authors describe current LLMs as opaque and argue that their successful labels or outputs should not be taken as evidence that they contain Theory-of-Mind.This reinforces the paper’s skeptical stance toward interpreting performance alone.