Source-linked AI summary
Understanding Social Reasoning in Language Models with Language Models
Kanishk Gandhi, Jan-Philipp Fränken, Tobias Gerstenberg, Noah D. Goodman
TL;DR
Prior LLM Theory-of-Mind evaluations have yielded inconsistent and difficult-to-interpret evidence about human-like social reasoning. This paper generates BigToM by populating causal templates and uses it to compare several LLMs with humans. GPT-4 most closely matches human inference patterns, although it is less reliable, while other models struggle.
Problem
Existing LLM Theory-of-Mind evaluations provide inconsistent evidence and raise concerns about methodological validity when assessing alignment with human reasoning.
Method
The paper builds causal templates, has a language model populate their variables, and combines them into controlled Theory-of-Mind evaluations.
Results
GPT-4 shows a more human-like inference pattern than other evaluated models, although it does not reach human-level performance on backward belief inference, where humans achieve 82% true-belief and 72% false-belief accuracy.
Takeaways & Limitations
Causal-template evaluations support systematic comparisons of LLM social reasoning across controlled inference conditions and human performance.
Takeaways & Limitations
Model-generated evaluations can encode stereotypes, content biases, and imbalanced coverage of situations from biases in the underlying model and training data.
Abstract
from arXiv · showhide
As Large Language Models (LLMs) become increasingly integrated into our everyday lives, understanding their ability to comprehend human mental states becomes critical for ensuring effective interactions. However, despite the recent attempts to assess the Theory-of-Mind (ToM) reasoning capabilities of LLMs, the degree to which these models can align with human ToM remains a nuanced topic of exploration. This is primarily due to two distinct challenges: (1) the presence of inconsistent results from previous evaluations, and (2) concerns surrounding the validity of existing evaluation methodologies. To address these challenges, we present a novel framework for procedurally generating evaluations with LLMs by populating causal templates. Using our framework, we create a new social reasoning benchmark (BigToM) for LLMs which consists of 25 controls and 5,000 model-written evaluations. We find that human participants rate the quality of our benchmark higher than previous crowd-sourced evaluations and comparable to expert-written evaluations. Using BigToM, we evaluate the social reasoning capabilities of a variety of LLMs and compare model performances with human performance. Our results suggest that GPT4 has ToM capabilities that mirror human inference patterns, though less reliable, while other LLMs struggle.
1 Introduction
The paper motivates more reliable assessment of LLM Theory-of-Mind reasoning and introduces a causal-template framework for systematic, controlled evaluations.
- LLM Theory of Mind matters because modeling human mental states supports effective teaching, communication, collaboration, and interaction.
- Existing evaluations produce difficult-to-interpret results because crowd-sourced examples can be ambiguous and synthetic or modified psychological tests have methodological limitations.
- The framework represents ToM scenarios as causal graphs, enabling interventions on variables, control conditions, and probes of distinct reasoning capabilities.
- The evaluation pipeline builds an abstract causal template, uses a language model to populate its variables, and combines those variables into diverse test conditions.
- The paper reports that GPT-4 shows human-like ToM inference patterns but is less reliable, whereas other evaluated LLMs struggle.
2 Related Work
Prior work has tested machine Theory of Mind with cognitive-science tasks, crowd-sourced data, and synthetic evaluations, but findings remain mixed and brittle.
- Theory-of-Mind in Humans: Human ToM research uses controlled false-belief and related experiments to study how people infer agents’ desires and beliefs.
- Theory-of-Mind in Machines: Machine ToM studies initially used task-specific neural models and later applied small cognitive-science test sets to LLMs.
- Theory-of-Mind in Machines: Altered and diverse scenarios revealed that apparent LLM ToM reasoning can be brittle, while crowd-sourced and synthetic evaluations produced mixed results.
- Model-Written Evaluations: Model-generated evaluations offer scalability, cost-effectiveness, and easy replication, but unstructured test generation makes their results challenging to interpret.
- Model-Written Evaluations: The paper combines structured causal test generation with the scalability and cost advantages of model-written evaluations.
3 Model-Written Evaluations with Causal Templates
This section develops a causal-template framework for generating controlled, scalable Theory-of-Mind evaluations with language models. It constructs BigToM items by populating template variables, composing condition-specific stories, and assessing their quality against human-written benchmarks.
- 3 Model-Written Evaluations with Causal Templates: Theory of Mind attributes mental states such as beliefs, intents, desires, emotions, and knowledge, with this work testing forward and backward inference over percepts, beliefs, desires, and actions.
- 3 Model-Written Evaluations with Causal Templates: The evaluations are designed to include control conditions, avoid directly using human-designed test items, and remain diverse and scalable while targeting ToM inference rather than unrelated common-sense errors.
- 3.1 Stage 1: Building a Causal Template: The causal template specifies context, agent, desire, percept, belief, causal event, and subsequent action, enabling systematic manipulation of what the agent perceives.
- 3.1 Stage 1: Building a Causal Template: The benchmark tests initial percept-to-belief inference, explicit versus implicit initial beliefs, forward belief, forward action, backward belief, and a random-event control.
- 3.2 Stage 2: Populating Causal Templates With Language Models: A language model populates prompt-template variables with generated sentences, after which selected variable sentences are combined into stories tailored to each inference condition.
- 3.4 Quality of Generated Data: Experts evaluated 2,500 model-written test items, finding 93.94% agreement on structural compliance and mean desired-behavior ratings of 4.33 and 4.35.
- 3.4 Quality of Generated Data: Participant evaluations compare 200 BigToM items with 50-item samples from SocialIQA and expert-written ToM scenarios, while Table 1 presents examples across datasets.
4 Experiments
The experiments evaluate five language models across multiple prompting conditions and compare them with human performance. Models generally struggle with belief reasoning, while GPT-4 more closely matches human patterns, and demonstrations improve performance.
- 4 Experiments: Five language models are evaluated with 0-shot, 0-shot-chain-of-thought, 1-shot, and 1-shot-chain-of-thought prompts using accuracy on comprehension questions.
- 4 Experiments: Figure 3 organizes 0-shot performance into forward belief, forward action, and backward belief conditions, with human error bars showing 95% bootstrapped confidence intervals.
- 4.1 Results and Discussion: All models perform proficiently on initial percept-to-initial-belief inference, indicating that they understand how percepts lead to beliefs.
- 4.1 Results and Discussion: Many models struggle with forward belief inference, particularly when an initial belief is explicitly stated; GPT-4 and, to a lesser extent, Claude approach human performance.
- 4.1 Results and Discussion: GPT-4 achieves human-level or slightly better performance on forward action inference in the critical false-belief condition, unlike most other models.
- 4.1 Results and Discussion: Humans achieve 82% true-belief and 72% false-belief accuracy in backward belief inference, whereas models are generally far below chance; GPT-4 shows a more human-like but subhuman 0-shot pattern.
- 4.1 Results and Discussion: One-shot prompting consistently improves performance across models and conditions, whereas zero-shot chain-of-thought does not consistently help.
- 4.1 Results and Discussion: One-shot chain-of-thought consistently improves performance, but the improvement may reflect mimicking the reasoning template rather than stronger Theory-of-Mind reasoning.
5 Discussion
The discussion presents BigToM as a structured benchmark for mapping LLM Theory-of-Mind reasoning while identifying important methodological and synthetic-data limitations. It argues that systematic benchmarking and more realistic interactive environments are needed for future research.
- 5 Discussion: BigToM uses abstract causal templates to control provided information and isolate what models must infer.The framework is intended to map ToM reasoning abilities while accounting for content effects and low-level confounds.
- 5 Discussion: Many tested models struggle to infer beliefs from percepts and events, especially when previously stated beliefs affect performance.GPT-4 aligns most closely with human inference patterns but remains below human accuracy when inferring beliefs from actions.
- 5 Discussion: The authors report that GPT-4 performs comparably on evaluations generated by Claude-2 and outperforms the model that generated that dataset.This result supports their view that using a model to generate evaluation items need not confound inferential testing.
- 5 Discussion: Synthetic evaluations can reflect biases in the generating model and training data, including stereotyped or imbalanced social situations.Detailed steering may help mitigate some biases, but diversity and balance require careful attention during generation.
- 5 Discussion: Future work should extend causal-template evaluations to domains such as morality, physical reasoning, medical diagnosis, and mathematics.The method is framed as applicable where hidden causes or underlying causes of effects must be inferred.
- 5 Discussion: The paper calls for realistic, dynamic social-reasoning environments that measure and potentially improve models in deployment-relevant interactions.The conclusion characterizes systematic and broad benchmarking of mental-state reasoning as a pressing concern.
B.1 Experiment 1: Human Quality Ratings
Experiment 1 evaluated the quality of model-written BigToM items against expert and crowd-sourced datasets through human ratings. BigToM received the strongest ratings, exceeding crowd-sourced items and matching or surpassing expert-written items.
- B.1 Experiment 1: Human Quality Ratings: 200 Prolific participants each rated 30 items, producing 20 independent ratings per item.The ratings covered Forward Belief True Belief, Forward Belief False Belief, Expert, and SocialIQA items.
- B.1 Experiment 1: Human Quality Ratings: BigToM items received the highest average ratings for every question across the evaluated datasets.Aggregate ratings averaged the three Likert measures for each item and participant.
- B.1 Experiment 1: Human Quality Ratings: 0.263 was the estimate for BigToM-Expert, while socialIQa-Expert was −0.889.The reported 95% credible intervals were 0.178–0.347 and −1.000–−0.782, respectively.
- B.1 Experiment 1: Human Quality Ratings: Model-written items were judged better than crowd-sourced items and comparable or better than expert-written items.The expert dataset included items drawn from established false-belief and related Theory-of-Mind task sources.
- B.1 Experiment 1: Human Quality Ratings: The benchmark used 25 conditions per generated template, with the omitted 25th condition testing belief inference from an initial percept.The displayed figure contained 24 conditions; the additional condition was described separately.
B.2 Experiment 2: Human Performance
Experiment 2 established a human baseline for the three main conditions using participants recruited through Prolific.
- B.2 Experiment 2: Human Performance: 20 Prolific participants provided the human baseline for the three main conditions in the “with initial belief” version.Participants were paid $12.05 per hour.
C Failure cases
The failure-case discussion describes generation weaknesses in smaller language models and the quality improvements obtained through verification, revision, and GPT-4 generation. It also presents a semi-autonomous interface for harder domains, while noting that BigToM itself was generated fully autonomously.
- C Failure cases: Text-davinci-003 and GPT-3.5-turbo frequently failed to follow instructions and made common-sense errors during template generation.Splitting generation into multiple stages improved results but did not achieve the expected reliability.
- C Failure cases: A semi-autonomous interface allows users to specify parameters and edit populated templates when a model cannot coherently complete them.The paper states that BigToM itself was generated fully autonomously; the interface is presented for more difficult domains.
- C Failure cases: Verifiers and revisers improved generated content quality, but model mistakes remained high.The pipeline used one model to judge and provide feedback, followed by revision based on that feedback.
- C Failure cases: Switching to GPT-4 with a single generation stage produced high-quality generations.This contrasts with the multi-stage mitigation attempts applied to weaker generators.
D Previous Benchmarks for ToM Reasoning in LLMs
Prior ToM benchmarks for LLMs suffer from narrow task designs, ambiguity, possible dataset leakage, and limited systematic coverage, complicating interpretation of model performance.
- ToMi: ToMi uses templates for Sally-Anne-like queries but narrowly varies object locations and leaves perceptual access unclear.Some stories are also ambiguous, with factors that may interfere with accurate ToM assessment.
- SocialIQA: SocialIQA is crowd-sourced, noisy, and structurally unorganized, making its ambiguous answers and behavioral coverage difficult to interpret.Several questions do not test the intended behaviors.
- Other false-belief benchmarks: False-belief benchmarks based on familiar human tasks may reward dataset leakage, fail under trivial alterations, and contain too few examples for systematic testing.These concerns are reported for tests discussed by Bubeck et al. and Kosinski.
- Chain-of-thought evaluations: One-shot chain-of-thought can produce perfect scores when all questions share a reasoning structure, making ToM interpretation difficult.The shared structure may reduce the generalization gap without demonstrating broader ToM capabilities.
- Adv-CSFB: Adv-CSFB increases the number of test examples, but the authors attribute model failures to underspecified perceptual access rather than ToM inference itself.They argue that explicitly stating what an agent can see can change performance.
E Evaluating models
The evaluation uses multiple prompting conditions and reports model performance across ToM methods and controls, with prompts and implementation resources documented separately.
- Resources: Models were evaluated using OpenAI and Anthropic APIs, while the Llama model ran on an internal cluster with one NVIDIA A40 GPU.These resources supported the model experiments.
- Prompts: The study compares 0-shot, 0-shot chain-of-thought, 1-shot, and 1-shot chain-of-thought prompting conditions.The corresponding prompts are referenced in Figures 9–12.
- Results: Model performance is reported across methods and evaluation conditions, with separate results for control conditions.Tables 7 and 8 provide the referenced model and control results.
F Addressing Concerns of Circularity
The paper addresses circularity by validating model-written evaluations across generators and examining the diversity and broader scope of the resulting benchmark.
- Cross-generator validation: GPT-4 performance on BigToM-Claude is similar to its performance on BigToM-gpt4, supporting consistency across template-populating models.Claude-2 performs worse than GPT-4 on the Claude-generated evaluation set.
- Dataset diversity: The dataset’s diversity is assessed across changeable states, mechanisms of change, and perception media.GPT-4 annotates 200 templates along these three dimensions.
- Dataset diversity: Examples span mechanisms such as rainfall and malfunctions, perception media such as tasting and hearing, and states such as temperature and cleanliness.These categories illustrate variation beyond object-location changes.
- Dataset diversity: Generated items are substantially diverse, whereas expert-written datasets are described as smaller and often concentrated on location changes perceived through visual observation.The comparison links diversity to the time and effort required for expert generation.
- Second-order beliefs: The template-based method is extended to testing second-order belief inference.The extension is illustrated in Figure 15.