ToMBench: Benchmarking Theory of Mind in Large Language Models

Motivation

• Existing ToM evaluations have narrow scope, subjective manual scoring, and contamination risks that can inflate apparent LLM performance.Prior studies often target isolated dimensions, open-ended answers require costly and potentially inconsistent judgments, and released inventories may have entered model training data.Source: §1 · pp. 1–2

Benchmark design

• ToMBench organizes evaluation around 8 tasks and 31 theory-of-mind abilities grounded in the ATOMS psychological framework.It provides both task-oriented and ability-oriented assessments.Source: §1 · p. 2, Figure 2 · p. 3
• Its multiple-choice format enables automated, efficient, and more consistent scoring than open-ended evaluation.Each sample contains a story, a question, and plausible options with one correct answer and misleading distractors.Source: §1 · p. 2

Data construction

• The inventory contains 2,860 original Chinese-English test samples created from scratch by trained workers and validated through strict annotation procedures.The samples cover diverse real-world social scenarios and are intended to reduce data leakage from existing inventories.Source: §1 · p. 2

Results

• GPT-4 leads the evaluated LLMs, yet its average ToM performance remains over 10 percentage points below the human baseline.The study evaluates 10 popular LLMs with vanilla and chain-of-thought prompting, and reports a widening gap under more rigorous social-scenario understanding.Source: §1 · p. 2

Implications

• ToMBench is presented as a systematic automated benchmark intended to support development of LLM social intelligence beyond surface-level language processing.The authors also report that LLM decision-making attention patterns differ notably from humans.Source: §1 · p. 2

2.1 8 Theory-of-Mind Tasks

• ToMBench adapts eight established psychology tasks to evaluate social understanding in stories and scenarios.The tasks are Unexpected Outcome, Scalar Implicature, Persuasion Story, False Belief, Ambiguous Story, Hinting, Strange Story, and Faux-pas Recognition.Source: §2.1 · p. 3

2.2 31 Theory-of-Mind Abilities

• The benchmark uses ATOMS to organize theory of mind into six retained dimensions and 31 specific abilities.The retained dimensions are Emotion, Desire, Intention, Knowledge, Belief, and Non-literal Communication; the visual summarizes their distribution across scenarios.Source: §2.2 · p. 3, Figure 3 · p. 4
• The eight tasks encompass 19/31 ATOMS abilities; extra ability-specific test samples address the remaining 12 and extend the evaluation scope to all 31 abilities.Source: §2.2 · p. 3
• The six ability dimensions capture emotional states, desires, goal-directed intentions, differentiated knowledge, non-reality-based beliefs, and meanings beyond literal words.Together they represent distinct ways people interpret mental states and social communication.Source: §2.2 · p. 3

3.1 Overview

• The inventory contains 2,860 original samples, ensuring at least 100 samples per task and at least 20 per ability.Source: §3.1 · p. 4

3.2 Data Collection

• Stories draw on everyday social scenarios across school, workplace, family, and neighborhood contexts, with questions targeting distinct aspects of each scenario.Source: §3.1 · p. 4, §3.2 · p. 4
• Each sample combines a story, question, and plausible answer options, using multiple-choice format to support consistent evaluation without manual scoring.Source: §3.2 · p. 4

3.3 Data Validation & Translation

• Two validation rounds produced 99.4% average agreement, followed by GPT-4-0613 translation into English and manual checking for bilingual evaluation.Source: §3.3 · p. 5

3.1 Overview

• ToMBench organizes evaluation into 8 theory-of-mind tasks and 31 abilities, with task-level and ability-level performance reported separately.Source: §3.1 · p. 4, §3.4 · p. 5

3.4 Evaluation Method

• Evaluation presents each story, question, and options to LLMs and averages accuracy over samples associated with each task or ability.Source: §3.4 · p. 5

4.2 Main Results

• The smallest human–LLM gap is 10.1% in task view and 10.8% in ability view, despite GPT-4-series models leading the LLMs.Source: §4.2 · p. 5, Table 2 · p. 6, Table 3 · p. 7
• GPT-4 exceeds GPT-3.5-Turbo by over 10% points, while Qwen-14B-Chat approaches GPT-3.5-Turbo and Mixtral-8x7B does not surpass Mistral-7B.Source: §4.2 · p. 6
• CoT prompting almost never improves ToM performance and can reduce it across both task-oriented and ability-oriented evaluations.Source: §4.2 · p. 6
• Faux-pas recognition and unexpected-outcome tasks perform relatively well, whereas scalar implicature and persuasion-story tasks are the most difficult.Source: §4.2 · p. 6
• Non-literal Communication and Emotion are the strongest ability categories, while Knowledge is weakest because it tests information propagation between characters.Source: §4.2 · p. 6

A Harder Coherent Test for LLMs

• Under the coherent test, all LLMs drop by at least 20% points versus a 13.6-point human decrease, widening the top-model gap from 10.1% to 16.2%.Source: A Harder Coherent Test for LLMs · p. 7, Figure 4 · p. 7
• On Knowledge/Pretend Links, humans reach 93.3% accuracy, while GPT-4-1106 and GPT-4-0613 score 26.7% and 3.3% on English samples.Source: A Harder Coherent Test for LLMs · p. 8
• Attention analysis suggests ChatGLM3-6B relies on semantic associations, whereas a student could answer correctly after reading only half the story.Source: A Harder Coherent Test for LLMs · p. 8, Figure 5 · p. 8

• Earlier studies alternately found LLM performance below humans or comparable to children on small false-belief and early-inventory tests.Source: §5 · p. 8
• Small test sets and templated false-belief scenarios produced strong results that did not establish broad theory-of-mind competence.Source: §5 · p. 8
• Subtle sample changes and higher-order or conversational tasks often sharply reduce performance, highlighting reliance on spurious correlations.Source: §5 · p. 8
• ToMBench addresses these gaps with 8 tasks, 31 abilities, and 2,860 from-scratch multiple-choice questions covering real-world social scenarios.Source: §5 · p. 8

• ToMBench is a systematic, automated, original benchmark designed to evaluate theory of mind efficiently across LLMs.Source: §6 · p. 8
• Even the most advanced LLMs significantly lag behind human ToM performance and struggle to comprehend social scenarios comprehensively.Source: §6 · p. 8
• LLMs tend to rely on semantic associations rather than human-like cognitive processes when answering ToM questions.Source: §6 · p. 8
• The benchmark is intended to support research on LLMs with innate ToM and more human-friendly artificial intelligence.Source: §6 · p. 8

Scope

• ToMBench covers broad textual ToM tasks but omits some psychological tasks and visual assessments, limiting coverage of multimodal and additional abilities.The authors suggest expanding tasks, abilities, and image- or video-based evaluation.Source: Limitations · p. 9

Inventory Size

• Some abilities have only 20 samples, which may limit reliable ability-specific analysis despite the inventory’s 2,860 total samples.The authors propose expanding sample sizes for more detailed analysis.Source: Limitations · p. 9

Language and Culture

• The English data is translated from Chinese, so residual cultural differences may remain despite prompt design and manual review.Future work could localize content and expand language coverage.Source: Limitations · p. 9

LLMs and Prompting Methods

• The experiments evaluate 10 LLMs with vanilla and chain-of-thought prompting, leaving additional models and prompting methods for future analysis.The authors specifically mention perspective-taking and foresee-and-reflect prompting.Source: Limitations · p. 9

ToM Understanding vs. Application

• Question-answering performance in ToMBench does not necessarily establish intrinsic ToM awareness or effective ToM application in human-AI interactions.The authors frame understanding and application as progressive but distinct aspects.Source: Limitations · p. 9

ToM and Anthropomorphism

• Evaluating LLM ToM may encourage anthropomorphic interpretations, although the stated aim is improved human-machine interaction rather than humanizing LLMs.The paper emphasizes preserving differences between humans and machines.Source: Ethical Considerations · p. 9

Content Safety

• Test stories are designed to avoid unsafe, toxic, biased, and offensive content, and use fictitious scenarios without identifiable information.The authors report professional training and psychological-literature review as part of content preparation.Source: Ethical Considerations · p. 9

Human Baseline

• The human baseline uses paid native Chinese-speaking graduate students recruited by registration order in an online document without sensitive-data collection.The paper states that this choice reflects available conditions rather than intended bias.Source: Ethical Considerations · p. 9

Contamination Example

• GPT-3.5-Turbo and GPT-4 generated samples closely matching original Sally-Anne examples, highlighting possible benchmark contamination.The Sally-Anne Test is presented as a commonly applied false-belief task.Source: A Inventory Contamination in LLMs · p. 12

B Details of Theory-of-Mind Tasks

• The benchmark adopts the ATOMS definition of ToM and identifies eight social-cognitive tasks covering mental-state representation and distinguishing self from others’ states.The task set includes unexpected outcomes, scalar implicature, persuasion, false belief, ambiguity, hints, strange stories, and faux pas.Source: B Details of Theory-of-Mind Tasks · pp. 12–13
• The eight tasks assess emotional discrepancies, nonliteral meanings, persuasion, false beliefs, ambiguous situations, indirect hints, complex social communication, and social faux pas.Examples include irony, misunderstandings, second-order beliefs, and recognition of social norms.Source: B Details of Theory-of-Mind Tasks · p. 13

C Details of Theory-of-Mind Abilities

• The ability framework contains 31 abilities in six dimensions, with 19 covered by existing tasks and 12 supported by additionally constructed test data.The cited passages identify emotion, desire, intention, knowledge, and belief among the dimensions.Source: C Details of Theory-of-Mind Abilities · pp. 13–14
• The abilities cover inferences about emotional reactions and regulation, desires and actions, intentions and goals, knowledge access, beliefs, and meanings beyond literal language.Examples include discrepant desires, failed-action completion, percept-knowledge links, false beliefs, irony, white lies, humor, and faux pas.Source: C Details of Theory-of-Mind Abilities · p. 14
• The belief dimension includes content, location, identity, second-order, action- or emotion-based, and sequence false beliefs.These abilities distinguish beliefs from reality, one’s own beliefs, or another person’s information state.Source: C Details of Theory-of-Mind Abilities · pp. 14–15

D Detailed Data Statistics

• For data statistics, each task has at least 100 samples across 19 abilities, while the remaining 12 abilities receive at least 20 additional samples each.The paper reports detailed statistics in Table 18.Source: D Detailed Data Statistics · p. 15

E Prompts for Data Translation

• The appendix documents prompts for translating Chinese data into English and for evaluating ToMBench responses.These materials are presented in Tables 19 and 20.Source: E Prompts for Data Translation · p. 15, F Prompts for ToM Evaluation · p. 15

• LLMs have not yet achieved general and robust human-level ToM performance across the 31 evaluated abilities.Source: G Detailed Results of 31 Abilities · p. 15
• The GPT-4 series performs best on most specific abilities and surpasses human performance in 9 of 31 abilities.Source: G Detailed Results of 31 Abilities · p. 15
• High performance may reflect direct, simple samples resembling training data, while small ability-specific sample sizes may understate human advantages.Source: G Detailed Results of 31 Abilities · p. 15

I Case Study for CoT Prompting

• GPT-4-0613 can decompose questions into detailed reasoning steps yet still provide the incorrect answer in a CoT case study.Source: I Case Study for CoT Prompting · p. 15, Figure 6 · p. 15

J Use of AI Assistants

• The appendix includes example data, extra ability samples, data statistics, translation and evaluation prompts, and detailed performance tables for the benchmark.Source: Table 14 · p. 20, Table 15 · p. 20, Table 18 · p. 22

H Detailed Results of Coherent Tests

• Coherent-test performance for all LLMs is reported under vanilla and CoT prompting, with no performance drop for the single-question persuasion story task.Source: H Detailed Results of Coherent Tests · p. 15, Table 22 · p. 25