PaperlayerView exact revision on arXiv

Source-linked AI summary

Theory of Mind in Large Language Models: Assessment and Enhancement

Ruirui Chen, Weifeng Jiang, Chengwei Qin, Cheston Tan

arXiv:2505.00026v2cs.CLcs.AI

TL;DR

LLM ToM research lacks consistent evidence about robust mental-state reasoning and needs broader evaluation and enhancement analysis. This survey reviews story-based benchmarks and improvement strategies, then identifies future research directions. It concludes that comprehensive benchmarks and reasoning-focused evaluation are needed to better assess and advance LLM ToM capabilities.

  • Problem

    LLMs’ ToM capabilities remain debated and insufficiently robust, while existing reviews do not comprehensively cover recent benchmarks and enhancement strategies.

    Source: §1 · p. 1

  • Method

    The survey analyzes recent story-based ToM benchmarks, enhancement methods, their evolution, and future directions.

    Source: §1 · pp. 1–2

  • Results

    The review finds that consistent assessment remains challenging because ToM complexity cannot be fully captured through a limited set of questions.

    Source: §6 · p. 9

  • Takeaways & Limitations

    More comprehensive benchmarks and reasoning-process evaluation are needed to support more accurate assessment and future advancement of LLM ToM.

    Source: §5 · p. 9, §6 · p. 9

    The survey primarily analyzes story-based benchmarks and does not delve into purely spatial scenarios.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Theory of Mind (ToM)-the ability to reason about the mental states of oneself and others-is a cornerstone of human social intelligence. As Large Language Models (LLMs) become increasingly integrated into daily life, understanding their ability to interpret and respond to human mental states is crucial for enabling effective interactions. In this paper, we review LLMs' ToM capabilities by analyzing both evaluation benchmarks and enhancement strategies. For evaluation, we focus on recently proposed and widely used story-based benchmarks. For enhancement, we provide an in-depth analysis of recent methods aimed at improving LLMs' ToM abilities. Furthermore, we outline promising directions for future research to further advance these capabilities and better adapt LLMs to more realistic and diverse scenarios. Our survey serves as a valuable resource for researchers interested in evaluating and advancing LLMs' ToM capabilities.

1 Introduction

ToM research in LLMs faces debate over whether apparent competence is robust, while existing benchmark and enhancement reviews remain incomplete. This survey addresses both evaluation and improvement, emphasizing recent story-based benchmarks, enhancement strategies, and future directions.

  • LLMs’ ToM abilities remain disputed, with studies reporting promising competence while others find superficial and unstable performance.Source: §1 · p. 1
  • Recent evaluations indicate that LLMs still lack robust ToM capabilities, while detailed reviews of new benchmarks and enhancement strategies remain limited.Source: §1 · p. 1
  • The survey reviews passive story-based and adaptable interactive benchmarks alongside prompt-only and technique-augmented enhancement methods.Source: §1 · p. 1, Figure 1 · p. 2
  • It highlights recently proposed and widely used story-based benchmarks from 2023–2024 and analyzes their evolution over time.Source: §1 · p. 2
  • The paper provides the first broad survey combining evaluation and enhancement of LLMs’ ToM capabilities.Source: §1 · p. 2
  • Its analysis organizes field progress and identifies future directions for benchmark development and strategy refinement.Source: §1 · p. 2

2 Theory of Mind

ToM concerns mental-state attribution, while its evaluation in LLMs commonly uses psychological-test-based benchmarks and focuses heavily on beliefs. Enhancement methods are classified by whether they use prompt engineering alone or additional techniques.

  • ToM attributes mental states such as beliefs, intentions, desires, emotions, knowledge, percepts, and non-literal communications.Source: §2 · p. 2
  • Orders measure the number of mental-state attributions required, with first-order and second-order questions requiring one and two nested attributions.Source: §2 · p. 2
  • More than ten recent benchmarks evaluate LLM ToM capabilities, mostly adapting established psychological tests such as Sally-Anne and Smarties.Source: §2 · p. 3
  • Recent enhancement approaches are classified as prompt-only methods or methods incorporating additional techniques, with most targeting belief-related questions.Source: §2 · p. 3

3 Benchmarks for Evaluating ToM Capabilities in LLMs

This section surveys story-based ToM benchmarks, tracing their expansion toward more complex reasoning, diverse mental states, realistic contexts, and multimodal inputs while identifying remaining scope limitations.

3.1.1 Overview of Text-Only Benchmarks

  • Text-only benchmarks span synthetic narratives, generated conversations, negotiation dialogues, bilingual datasets, and tests covering beliefs, emotions, perceptions, desires, intentions, and social commonsense.Examples include ToMi, FANTOM, NegotiationToM, TOMBENCH, and OpenToM, each emphasizing different reasoning settings or mental states.Source: §3.1.1 · pp. 3–4

3.1.2 Trends in Text-Only Benchmarks

  • Benchmarks have become increasingly complex, expanding from first-order to second- and fourth-order belief reasoning, broader mental states, richer questions, and more realistic contexts.This evolution includes multimodal inputs and movement beyond traditional psychological-test narratives.Source: §3.1.2 · p. 5, §3.2 · p. 5
  • Benchmark construction has shifted from template-only generation toward LLM-generated and human-built datasets, with manual construction addressing concerns about data contamination.TOMBENCH was built from scratch, whereas BigToM and OpenToM use LLM-generated content through template filling.Source: §3.1.1 · p. 4, §3.1.2 · p. 5

3.2 Multimodal Benchmarks

  • Multimodal benchmarks combine video and text to test belief and goal inference, with MuMA-ToM additionally separating modality-specific information and evaluating belief-of-goal reasoning.MMToM-QA focuses on one character finding household objects, while MuMA-ToM evaluates embodied multi-agent interactions.Source: §3.2 · p. 5
  • Current multimodal evaluations remain constrained by household settings, synthetic videos, limited agents or goals, and uncertain transfer to real-world applications.The survey calls for sim-to-real strategies or benchmarks using real-world videos and more diverse scenarios.Source: §3.2 · pp. 5–6
  • Most existing datasets are passive benchmarks in which LLMs observe scenarios rather than act, motivating interactive evaluations of dynamic ToM capabilities.This limitation applies across the surveyed benchmark landscape and is identified as a future research direction.Source: §3 · p. 3, §3.2 · p. 6

4 Strategies for Enhancing the ToM capabilities of LLMs

The paper groups ToM enhancement methods into prompt-only pipelines and approaches adding techniques such as fine-tuning, comparing their mechanisms, strengths, and limitations.

4.1 Methods Relying Exclusively on Prompt Strategies

  • These prompt pipelines can support higher-order reasoning, but errors in perspective, belief-graph, or temporal-state construction may propagate and limit performance.SYMBOLICTOM demonstrated effectiveness through third-order belief questions, while TIMETOM transforms higher-order reasoning into first-order reasoning through temporal belief intersections.Source: §4.1 · pp. 6–8
  • Prompt-only methods identify a target character’s perspective or beliefs before prompting an LLM, avoiding costly training and showing robustness to out-of-distribution samples.SYMBOLICTOM builds belief graphs, SIMTOM filters character-aware events, PercepToM selects perceived information, and TIMETOM uses temporal belief-state chains.Source: §4.1 · pp. 6–7

4.2 Methods Incorporating Additional Techniques

  • Additional-technique methods extend ToM reasoning with symbolic parsing, model checking, inverse planning, and fine-tuning beyond prompting alone.ToM-LM converts natural-language problems into symbolic formulations checked by SM-CDEL; BIP-ALM and LIMP estimate actions or utterance policies from mental-state hypotheses.Source: §4.2 · p. 8
  • LIMP outperforms BIP-ALM on MuMA-ToM and represents information in natural language, but hallucinations in VLM action recognition remain a significant error source.BIP-ALM achieved superior performance on MMTOM-QA, whose scenarios are limited to household object-searching and whose current setting does not support open questions.Source: §4.2 · p. 8
  • ToM-LM, BIP-ALM, and LIMP have been evaluated only in multiple-choice settings, leaving open-ended question answering as an unresolved challenge.Symbolic representations typically require fine-tuning for better performance, while ToM-LM additionally requires substantial logic expertise to prepare training pairs and may not generalize to open-ended questions.Source: §4.2 · p. 8

5 Future Directions

The paper calls for broader and more realistic ToM benchmarks, better evaluation of reasoning processes, and enhancement strategies that progressively and jointly address task complexity.

Benchmarks

  • Future benchmarks should cover a broader range of mental states beyond the belief-focused reasoning that dominates current benchmarks and strategies.The paper suggests adapting psychological tests spanning children’s development for LLM evaluation.Source: §5 · pp. 8–9
  • Multimodal benchmarks should use visual, auditory, and contextual cues while preserving information that can be lost when videos are converted to text.Suggested settings include short films, cartoons, real-world videos, and more diverse scenarios than current household multiple-choice tasks.Source: §5 · p. 9
  • Active benchmarks should evaluate LLMs as agents making decisions in complex environments rather than relying only on passive assessments.The paper presents agentic decision-making as necessary for deeper investigation of ToM capabilities.Source: §5 · p. 9

Enhancement Strategies

  • Curriculum or continual learning could address ToM complexity progressively, while joint or iterative pipelines could use feedback to reduce error propagation between stages.The paper illustrates this with feedback from inference to belief-graph construction when conflicts are detected.Source: §5 · p. 9

Evaluation

  • Answer correctness alone is insufficient, so datasets should support automated reasoning evaluation and probing of models’ internal mental-state representations.The paper identifies reasoning-process assessment and internal representation probing as directions for further exploration.Source: §5 · p. 9

6 Conclusion

The survey analyzes LLM ToM benchmarks and enhancement strategies, but consistent assessment remains difficult because ToM complexity cannot be captured by a limited set of questions.

6 Conclusion

  • Consistent assessment of LLM ToM capabilities remains challenging despite numerous benchmarks, because ToM complexity cannot be fully captured through a limited set of questions.The paper concludes that more comprehensive benchmarks may support more accurate evaluation and further ToM improvement.Source: §6 · p. 9

Limitations

The paper limits its main discussion to story-based ToM benchmarks and current LLM-tested strategies, while using the Sally-Anne test as a foundational example of belief evaluation.

Limitations

  • The survey primarily analyzes recently proposed or widely adopted story-based benchmarks and effective strategies tested with LLMs, excluding purely spatial scenarios from its main discussion.Interactive benchmarks and pre-LLM strategies with adaptation potential are discussed in appendices.Source: Limitations · p. 10

A.1 Sally-Anne Test

  • The Sally-Anne test presents Sally leaving while Anne hides a marble, then asks belief, reality, and memory questions to assess ToM.The belief question concerns where Sally will look, the reality question asks where the marble is, and the memory question asks its initial location.Source: §A.1 · p. 16

A.2 Ice Cream Van Experiment

The Ice Cream Van Experiment tests first- and second-order belief reasoning through a changing-location story, while the ATOMS framework organizes ToM into seven mental-state categories.

A.2 Ice Cream Van Experiment

  • The experiment’s scenario involves Mary, John, and an ice cream seller whose stated location changes after Mary leaves, creating the belief discrepancy.Source: §A.2 · p. 16
  • First-order questions ask whether Mary knows the van is at the church, whereas second-order questions ask where John thinks Mary will buy ice cream.Source: §A.2 · p. 16

A.3 Smarties Task

  • The Smarties task tests whether children distinguish an object’s apparent contents from its actual contents and predict another person’s mistaken belief.Source: §A.3 · p. 16
  • Children generally pass the Smarties task by age four or five, and task variations form the basis for multiple ToM benchmarks.Source: §A.3 · p. 16

B Abilities in Theory of Mind Space (ATOMS)

C Overview of Additional Text-Only Benchmarks

Additional text-only benchmarks extend ToM evaluation from first- and second-order belief tasks to higher-order reasoning, dialogue-based common ground, and applied mental-state prediction.

D Comparative Information on Benchmarks

The paper provides a detailed benchmark comparison covering foundations, story and question-answer formats, reasoning order, language coverage, and publication details.

E Interactive Benchmarks

Interactive ToM benchmarks situate reasoning in collaborative, visual, multimodal, and game-like environments, while highlighting gaps in agents, modalities, and belief-order coverage.

  • Interactive benchmarks draw on Minecraft, Dungeons and Dragons, and grid-world scenarios to evaluate or improve situated ToM beyond story-based tasks.Source: E Interactive Benchmarks · pp. 17, 19
  • MINDCRAFT uses asymmetric knowledge and skills in a collaborative Minecraft environment, emphasizing perception and knowledge through task and player-state questions.Source: E Interactive Benchmarks · p. 19
  • SymmToM models agents that communicate, perceive, navigate, and gather information in a grid world with limited hearing but perfect visual access.Source: E Interactive Benchmarks · p. 19
  • G4C incorporates a Dungeon Master’s intent into natural-language generation to examine whether intent and ToM improve computational communication.Source: E Interactive Benchmarks · p. 19
  • Situated ToM in Grid World evaluates all ATOMS dimensions and a reality check in MiniGrid, while addressing contamination and shortcut concerns associated with text-only datasets.Source: E Interactive Benchmarks · p. 19
  • Future interactive benchmarks should add more agents, multimodal information, higher-order beliefs, negotiation games, and human-AI collaboration tasks.Source: E Interactive Benchmarks · p. 19

F Methods Proposed in the Era of Small Language Models

The paper reviews earlier approaches for enhancing ToM in small language models and identifies future priorities spanning multilingual, higher-order, and smaller-model settings.

F Methods Proposed in the Era of Small Language Models

  • Textual Time Travel uses memory snapshots and retrieval of a relevant time step to generate answers, under local perception, recency, and reciprocity assumptions.The framework includes both heuristic rule-based and learning-based mechanisms.Source: F Methods Proposed in the Era of Small… · p. 19
  • Earlier ToM enhancement research also used sequence-to-sequence models and reinforcement learning, including Explicitly Modeling Symmetric Theory of Mind and ToM-Inspired RL.These methods may inform future improvements in LLMs’ ToM capabilities.Source: F Methods Proposed in the Era of Small… · pp. 19–20

G Additional Future Directions

  • Most ToM benchmarks remain English, motivating multilingual benchmarks to evaluate and advance comprehensive ToM capabilities across different languages.Chen et al. (2024) is identified as an example of multilingual benchmark work.Source: G Additional Future Directions · p. 20
  • Because existing strategies and benchmarks mainly target second-order beliefs, future work should advance higher-order ToM reasoning for settings such as multi-party negotiations.The paper connects this direction to developing “advanced ToM.”Source: G Additional Future Directions · p. 20
  • Smaller language models warrant investigation because current strategies mostly target models above 7 billion parameters and may not transfer to models with fewer parameters.Resource demands and smaller models’ reduced instruction-understanding capacity further motivate this direction.Source: G Additional Future Directions · p. 20
Loading 2505.00026v2…