PaperlayerView exact revision on arXiv

Source-linked AI summary

Diagnosing the Explicit-to-Applied Theory of Mind Gap in LLMs

Prajvi Saxena, Sabine Janzen, Wolfgang Maass

arXiv:2610.31060v1cs.AI

TL;DR

Theory of Mind benchmarks can obscure the distinction between recognizing another person’s mental state and using it to predict behavior, a distinction relevant to action-sensitive judgments. Across four benchmarks, the paper compares prompting, in-context learning, and BSTTT; recovery is substantial but varies across models, tasks, and interventions.

  • Problem

    Aggregate ToM accuracy obscures the distinction between recognizing a mental state and applying it to predict behavior, the action-relevant step in decisions about people.

    Source: §1 · p. 1, §A.3 · p. 11

  • Method

    Across four ToM benchmarks, the study compares structured prompting, matched in-context learning, and BSTTT to investigate recovery of applied reasoning.

    Source: §1 · p. 2, §8 · p. 8

  • Results

    Recovery is substantial but selective and intervention-dependent; on SimpleToM, Llama’s behavior prediction rises from 28.9% to 69.7% with Reminder+BSTTT.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Separate reporting of recognition and applied subtypes, alongside intervention comparisons, can show which performance gaps are recoverable and under what conditions.

    Source: §8 · p. 8

    The evaluation uses existing benchmarks, so performance on these tasks does not establish human-like social understanding or deployment readiness.

    Source: Ethical Considerations · p. 9

Abstract

from arXiv · show

As large language models are deployed in high-stakes roles such as therapy, education, and personal assistance, their social cognition must be assessed reliably. Current Theory of Mind (ToM) benchmarks report aggregate scores that conflate two distinct competencies: explicit ToM (recognizing what someone believes) and applied ToM (using that belief to predict behavior). Building on this distinction, we investigate the gap and its recoverability using four structurally diverse benchmarks. Aggregate reporting obscures uneven competence: on SimpleToM, Llama-3.1-8B scores 51.5 percent overall while recognizing mental states at 99.4 percent and predicting behavior at only 28.9 percent. To assess recovery, we compare structured prompting, matched in-context learning, and our proposed Belief-State Test-Time Training (BSTTT), which adapts lightweight parameters on answered support questions through action reconstruction. Across Qwen2.5-7B, Llama-3.1-8B, and GPT-oss-20B, recovery depends on the model, task, and intervention. On SimpleToM, reminders combined with BSTTT improve Llama's behavior prediction by 40.8 percentage points; reminders account for most of the overall improvement, with BSTTT adding 1.3 to 5.5 points beyond reminders across models. Contextual examples often outperform weight updates, while BSTTT remains stronger in specific model-benchmark settings. Temporal belief tracking remains difficult despite partial recovery. These findings show that aggregate ToM scores can overstate applied competence while hiding performance recoverable through targeted assistance. We recommend decomposed reporting of explicit and applied accuracy alongside intervention comparisons that establish which failures are recoverable and under what conditions.

1 Introduction

Aggregate ToM scores can conceal a gap between recognizing mental states and applying them to predict behavior. The paper compares prompting, matched in-context learning, and BSTTT across benchmarks to examine recovery and its limits.

1 Introduction

  • On SimpleToM, Llama’s behavior prediction rises from 28.9% to 63.2% with a mental-state reminder and 69.7% with Reminder+BSTTT.Across three model families and four benchmarks, substantial recovery is possible, though it varies by model, task, and intervention.Source: §1 · p. 2
  • On OpenToM, Llama reaches 56.4% with ICL versus 43.1% with BSTTT, while GPT-oss reaches 36.0% with BSTTT versus 28.9% with ICL.Weight updates help in some settings, but parameter adaptation is not a universal prerequisite for improved applied reasoning.Source: §1 · p. 2
  • On SimpleToM, Llama’s ICL behavior accuracy rises from 28.9% to 66.0%, while judgment accuracy falls from 26.2% to 14.0%.On DynToM, Llama improves from 19.7% to 32.6% with ICL, leaving substantial error and capacity limitations unresolved.Source: §1 · p. 2
  • The diagnostic protocol compares BSTTT, matched ICL, and structured prompting, including ToM-aligned and generic adaptation objectives, to investigate capacity-limitation and latent-competence hypotheses.The proposed evaluation compares interventions to characterize recovery across competencies and settings.Source: §1 · p. 2
  • The paper recommends separately reporting explicit and applied accuracy, treating temporal reasoning as a distinct axis, and comparing prompting, matched ICL, and weight adaptation.These comparisons are intended to characterize recoverable competence.Source: §1 · p. 2

2 The Explicit-to-Applied Theory of Mind Gap

The section formalizes the distinction between recognizing another agent’s mental state and applying it to reason about behavior. It operationalizes that distinction across four benchmarks and frames intervention-based recovery as a test of competing hypotheses.

2 The Explicit-to-Applied Theory of Mind Gap

  • The explicit-to-applied gap is explicit-question accuracy minus applied-question accuracy; a positive value means explicit accuracy is higher, while a near-zero gap can reflect either strong or weak performance.The authors therefore report both accuracies alongside the gap.Source: §2 · p. 2

2.1 Operationalization Across Benchmarks

  • SimpleToM pairs mental-state, behavior, and judgment questions; DynToM tests state relations and cross-scenario transformations, Hi-ToM higher-order beliefs, and OpenToM multihop belief reasoning against an explicit proxy.Hi-ToM lacks a paired explicit subset, while the other benchmarks provide explicit/applied decompositions or proxies.Source: §2.1 · p. 3, Table 1 · p. 3

2.2 Diagnostic Question

  • A positive gap motivates H1, that models lack some reasoning needed for reliable application, and H2, that they possess relevant knowledge but fail to apply it consistently under standard inference.Prompting, matched ICL, and BSTTT compare recovery from explicit belief cues, answered supports, and parameter updates; recovery does not establish that all needed knowledge was already encoded.Source: §2.2 · p. 3

3 Belief-State Test-Time Training

BSTTT adapts lightweight LoRA parameters on answered support questions to score a held-out query. It operates episodically and compares answer-supervised action reconstruction with next-token modeling of the same support narratives.

3 Belief-State Test-Time Training

  • BSTTT updates LoRA parameters on an episode’s answered support set, then uses the adapted model to score candidate answers for a held-out query.The backbone remains fixed, and LoRA parameters are reset before each episode.Source: §3 · p. 3

3.1 Episode Construction

  • Each episode pairs answered supports with a held-out query from the same benchmark-specific scenario family, narrative group, or trial.The query is excluded from supports, its label is used only for evaluation, and no external data is used.Source: §3.1 · pp. 3–4

3.2 Adaptation Objectives

  • Action Reconstruction minimizes cross-entropy over candidate-answer scores, using the correct answer among each support item’s candidates as supervision.The comparison holds the model, supports, LoRA configuration, optimizer, and gradient-step count constant.Source: §3.2 · p. 4
  • Next-Token Loss applies causal language modeling to the same support narratives, providing a contrast to answer-supervised adaptation.Adapters are attached to the top 25% of transformer layers; the backbone remains fixed and adapters are reset before each episode.Source: §3.2 · p. 4, §3.3 · p. 4

3.3 Implementation

  • The adapted model scores candidate answers conditioned on the selected prompt transformation, including MS-Reminder, SimToM, SymbolicToM, or Scratchpad.The default setup uses rank-8 adapters, three AdamW gradient steps, and a learning rate of 10^-4.Source: §3.3 · p. 4

4 Experimental Setup

The experiments evaluate three instruction-tuned open-weight models on four ToM benchmarks spanning varied reasoning settings. They compare prompting, matched in-context learning, and BSTTT configurations across three seeds.

4 Experimental Setup

  • The study evaluates Qwen-2.5-7B-Instruct, Llama-3.1-8B-Instruct, and GPT-oss-20B across four benchmarks covering static and temporal scenarios, first- and higher-order belief, and recognition and application.SimpleToM separates recognition, behavior prediction, and judgment; OpenToM and DynToM examine multihop inference and temporal belief tracking, while Hi-ToM tests higher-order reasoning.Source: §4 · p. 4
  • Twelve configurations compare prompting, matched ICL using the same answered supports as BSTTT, and BSTTT; experiments use three seeds and report mean accuracy.Cross-scenario ICL uses supports from other scenario families or trials; 95% bootstrap confidence intervals use 2,000 resamples.Source: §4 · p. 4

5 Results

Across benchmarks, explicit-to-applied gaps and recovery vary by model, task, and intervention. Results show partial recovery through prompting and adaptation, but persistent errors and uneven intervention benefits.

5.1 Finding 1: The Gap Is Substantial and Partially Recoverable

  • On SimpleToM, Llama’s applied accuracy rises from 27.5% to 51.7% with MS-Reminder+BSTTT and to 40.0% with matched ICL, while benchmark gaps vary substantially.Frozen gaps range from 21.0 to 71.9 pp on SimpleToM, whereas OpenToM gaps range from −0.8 to 8.5 pp; Figure 2 summarizes applied accuracy and intervention gains.Source: §5.1 · pp. 4–5, Figure 2 · p. 5
  • Answer-supervised adaptation exceeds narrative modeling by 13.0 pp for Llama and 15.7 pp for GPT-oss on Hi-ToM, but by only 1.8–2.5 pp overall on SimpleToM.The comparison does not establish that the required knowledge was already encoded in the original parameters; DynToM AR–NTL differences are −0.3 to +0.8 pp.Source: §5.1 · p. 5, Table 4 · p. 7
  • 40.8 percentage points: Llama’s SimpleToM behavior accuracy increases from 28.9% to 69.7% with MS-Reminder+BSTTT, while recognition remains near ceiling.MS-Reminder alone reaches 63.2%, whereas BSTTT-AR alone reaches 37.1%; GPT-oss judgment accuracy reaches 18.7% with the combined intervention and 26.8% with matched ICL.Source: §5.1 · p. 6

5.2 Finding 2: Recovery Depends on the Intervention

  • On OpenToM, matched ICL exceeds BSTTT-AR by 13.3 pp for Llama and 19.9 pp for Qwen, while GPT-oss performs 7.1 pp better with BSTTT-AR.Hi-ToM shows the same model-dependent ordering, with ICL stronger for Llama and Qwen but BSTTT-AR stronger for GPT-oss.Source: §5.2 · p. 6
  • Composing MS-Reminder with BSTTT adds 2.2 pp for Llama, 5.5 pp for Qwen, and 1.3 pp for GPT-oss over the reminder alone on SimpleToM.Composition can also underperform a component: GPT-oss Hi-ToM accuracy falls from 47.3% with BSTTT-AR to 46.5% when Scratchpad is added.Source: Table 5 · p. 7, §5.2 · p. 7

5.3 Finding 3: Recovery Varies Across Reasoning Tasks

  • Matched ICL raises Llama’s DynToM pooled applied accuracy from 15.9% to 26.8%, but substantial errors remain.With annotated mental-state tables, overall accuracy reaches only 42.2% for Qwen, 22.0% for Llama, and 14.5% for GPT-oss; available state information is insufficient for reliable performance in this prompting configuration.Source: §5.3 · pp. 7–8

5.4 Error Analysis

  • Remaining errors include limited transfer from support questions, multistep inference mistakes, and predictions based on world knowledge rather than the agent’s information.These recurring patterns characterize failures but do not establish an intrinsic capacity limit.Source: §5.4 · p. 8

6 Recommendations for Theory of Mind Evaluation

The paper recommends decomposing ToM evaluation and comparing recovery interventions under matched support. It also calls for separate temporal evaluation and component-level checks of intervention compositions.

6 Recommendations for Theory of Mind Evaluation

  • Report explicit and applied accuracy separately, distinguish applied subtypes, and evaluate temporal reasoning as a distinct axis.Behavior-prediction gains can coexist with judgment regressions; temporal belief tracking should be distinguished from static inference.Source: §6 · p. 8
  • Compare ICL and weight adaptation using the same answered supports, then test prompting-plus-adaptation against each component alone.These comparisons characterize recovery under tested interventions and account for benefits that vary by model and task.Source: §6 · p. 8

7 Related Work

Prior ToM evaluations report both strong false-belief performance and failures under varied probes. The paper situates its comparison of prompting, matched-support ICL, and BSTTT within work on social reasoning and test-time adaptation.

ToM evaluation in LLMs

Prompting and in-context learning for social reasoning

  • Prior approaches include reasoning instructions and examples, perspective-taking decomposition, symbolic belief tracking, and mental-state reminders, which this paper compares with matched-support ICL and BSTTT.The passage associates these approaches with Moghaddam and Honey (2023), Wilf et al. (2024), Sclar et al. (2023), and Gu et al. (2026), respectively.Source: §7 · p. 8

Test-time training and episodic adaptation

  • Test-time training adapts parameters during evaluation, and prior language-model variants use language-modeling-aligned updates or test-time reinforcement learning; BSTTT is compared through its adaptation objective and use of support information.The cited examples are In-Place TTT (Feng et al., 2026) and TTT-Discover (Yuksekgonul et al., 2026).Source: §7 · p. 8

8 Conclusion

The conclusion argues that aggregate ToM scores can mask differences between recognizing mental states and applying them. Recovery is possible but selective, and the authors recommend decomposed evaluation alongside matched-support and component comparisons.

8 Conclusion

  • Aggregate ToM scores obscure differences between recognizing mental states and applying them, motivating separate reporting of recognition and applied subtypes.Source: §8 · p. 8
  • Answered supports enable substantial recovery without weight updates, while BSTTT adds benefits in specific model-task settings.Source: §8 · p. 8
  • Cross-scenario ICL supports applied performance, while recovery remains selective: behavior gains can accompany judgment regressions.Source: §8 · p. 8
  • DynToM shows partial recovery and persistent errors under annotated-state prompting, leaving capacity limitations unresolved.Source: §8 · p. 8
  • The authors recommend decomposed reporting and matched-support and component comparisons to establish which failures are recoverable under what conditions.Source: §8 · p. 8

Limitations

The authors bound their conclusions by model scale, benchmark response format, and BSTTT's episodic reset. They identify open questions for larger models, open-ended tasks, and temporally extended adaptation.

Limitations

  • Experiments cover instruction-tuned models at 7–20B scale, leaving the gap's behavior in 70B+ or frontier models open.Source: Limitations · p. 9
  • All four benchmarks use multiple-choice formats, so open-ended generation may reveal a different gap, especially when applied reasoning requires constructing rather than selecting predictions.Source: Limitations · p. 9
  • BSTTT's episodic reset isolates per-scenario adaptation but is structurally mismatched to DynToM's temporally evolving beliefs; continual adaptation without reset is proposed for this regime.Source: Limitations · p. 9

Ethical Considerations

The authors caution that results on existing ToM benchmarks do not establish human-like social understanding or deployment readiness. Intervention gains depend on the model, task, and available support information.

Ethical Considerations

  • Performance on existing Theory of Mind benchmarks is not evidence of human-like social understanding or readiness for therapy, education, or personal-assistance deployment.Source: Ethical Considerations · p. 9
  • Improvements under test-time interventions depend on the model, task, and available support information.Source: Ethical Considerations · p. 9
  • Applications involving judgments about people's mental states require evaluation beyond these benchmarks.Source: Ethical Considerations · p. 9

AI Assistants Usage

This section limits AI-assistant use to language editing and specifies the paper’s prompting, adaptation, and evaluation procedures. It also reports uncertainty and cross-scenario results and illustrates how benchmarks distinguish identifying mental states from applying them to behavior and belief changes.

AI Assistants Usage

  • AI writing assistants were used only for grammar checking, language polishing, and clarity; they did not contribute to scientific content, experimental design, analysis, or conclusions.Source: AI Assistants Usage · p. 9

A.2 Explicit vs. Applied ToM: Grounding in Psychology

  • Explicit ToM identifies another agent’s mental state, whereas applied ToM uses that state to predict behavior, anticipate consequences, or evaluate actions.The paper operationalizes these as BE mental-state identification questions and BA questions about behavior prediction or action evaluation.Source: §A.2 · p. 11

A.3 Why the Gap Matters Beyond Benchmarks

  • Applied ToM matters in assistive, evaluative, and conversational AI because recognizing a user’s belief is not enough to predict actions, judge them fairly, or personalize communication.Source: §A.3 · p. 11

A.4 Connection to Classical ToM Tests

  • The four benchmarks span information-asymmetry inference in SimpleToM, recursive beliefs in Hi-ToM, temporal belief changes in DynToM, and multi-agent information access in OpenToM.SimpleToM removes explicit mentalizing cues; OpenToM uses attitude questions as explicit and multihop questions as applied, while DynToM separates local state tracking from state transformations.Source: §A.4 · p. 11, §B.1 · p. 12, §B.2 · p. 12

B.1 SimpleToM

  • >95% on mental-state questions, below 50% on behavior prediction, and below 25% on judgment are the frontier-model results reported for SimpleToM by Gu et al. (2026).The evaluated dataset contains 1,147 human-validated stories and 3,441 binary-choice questions; the paper maps mental-state questions to BE and behavior and judgment questions to BA.Source: §B.1 · p. 12

B.2 Hi-ToM; B.4 DynToM

  • Hi-ToM performance degrades with recursive order, while state-of-the-art LLMs underperform humans by 44.7% on average on DynToM (Xiao et al., 2025).Hi-ToM lacks a paired recognition–application decomposition; DynToM uses understanding questions as BE and transformation questions as BA.Source: §B.2 · p. 12, §B.4 · p. 13

C Baseline Methods and Prompt Templates

  • The methods include frozen option scoring, chain-of-thought prompting, mental-state reminders, SimToM perspective filtering, and SymbolicToM belief graphs.MS-Reminder prepends the model’s predicted mental-state answer to behavior or judgment prompts; SimToM filters events to a character’s perspective, whereas SymbolicToM tracks beliefs in a graph.Source: §C.1 · p. 13, §C.3 · p. 14, §C.4 · p. 14

B Benchmark Descriptions; C.2 Matched In-Context Learning

  • Matched ICL places answered support examples in the query prompt without changing parameters, while BSTTT adapts on answered supports; both are compared using the same supports.Held-out queries exclude support examples, and query reference labels are not supplied during prompting or adaptation.Source: B Benchmark Descriptions · p. 12, §C.2 · p. 13

C.5 SymbolicToM: Symbolic Belief-State Tracking

  • SymbolicToM operationalizes belief tracking as two-stage prompting: construct an explicit symbolic tracker, then answer conditioned on its state.It represents state updates and can include nested beliefs; unlike SimToM’s perspective filtering, it externalizes explicit state representations, and the methods’ relative effectiveness varies across models.Source: §C.5 · p. 15

C.6 Scratchpad: Structured Belief-State Scaffold

  • The Scratchpad baseline adds an internal structured mental-state scaffold to the story prompt without a separate generation pass.It directs the model to resolve conflicts between world state and agent belief, with question-specific focus on mental states, nested beliefs, or location transitions.Source: §C.6 · pp. 15–16

C.7 Composed Methods: Prompt+BSTTT Hybrids

  • Prompt+BSTTT hybrids apply the prompt scaffold at query time after AR adaptation on in-scenario support questions.For two-stage methods, frozen-model perspective notes or trackers are generated before adaptation; weight updates and input scaffolds are described as operating on orthogonal dimensions.Source: §C.7 · p. 16

C.8 DynToM-Specific Methods

  • DynToM methods use five-scenario trial context and type_a state-tracking questions to support transformation questions; Scratchpad-Oracle supplies gold states as a diagnostic upper bound.Scratchpad-Frozen fills a 5 × 4 belief, emotion, intention, and action table with frozen-model predictions; Scratchpad-TTT adapts on type_a questions and fills that table, while Hierarchical-TTT adapts but omits it. The trial-level adaptation is reset between trials.Source: §C.8 · p. 16

D.3 Action Reconstruction (AR) vs. Next-Token Loss (NTL)

  • AR adapts on answer-option likelihoods, whereas NTL adapts on narrative text without answer supervision; both use the same LoRA configuration, learning rate, and gradient-step count.Query-level BSTTT restores LoRA parameters and clears optimizer state before each episode; answers are scored by summed option-token log-likelihood, and a 7B/8B episode takes about 0.4 seconds versus 0.9 seconds for 20B.Source: §D.3 · p. 17, §D.2 · p. 17, §D.4 · p. 17

E.1 Bootstrap Confidence Intervals

  • GPT-oss-20B judgment accuracy rose from 3.1% to 18.7% with MS-Reminder+BSTTT and 26.8% with matched ICL, but remained below chance.The respective 95% bootstrap intervals were [2.2, 4.2], [16.5, 21.0], and [24.3, 29.4].Source: §E.1 · p. 18
  • 40.8 percentage points: Llama-8B behavior accuracy increased from 28.9% under frozen inference to 69.7% with MS-Reminder+BSTTT.The reported 95% bootstrap intervals are [26.3, 31.6] and [67.3, 72.3], respectively.Source: §E.1 · p. 18, Table 9 · p. 19

G.1 Cross-Scenario In-Context Learning

  • GPT-OSS cross-scenario ICL averaged 71.87% on SimpleToM, with 100.0% mental-state, 73.4% behavior, and 42.2% judgment accuracy.Compared with matched ICL, the reported difference was concentrated in judgment (26.8% versus 42.2%), while recognition and behavior remained essentially unchanged; the per-model tables report individual method configurations.Source: §G.1 · p. 20

G.2 Cross-Model Patterns in Full Results

Recovery varies by model, benchmark, and intervention: ICL is strongest for some model-task combinations, while BSTTT leads in others. Subtype results also show that gains in one ToM competency need not transfer to another.

G.2 Cross-Model Patterns in Full Results

  • Prompt rankings vary by model: SymbolicToM is strongest for Llama on Hi-ToM at 47.5%, while Scratchpad performs best for Qwen and GPT-oss.Composition also varies: Qwen benefits substantially from Scratchpad+BSTTT, while GPT-oss performs better with BSTTT-AR alone than with tested prompting compositions.Source: §G.2 · pp. 20–21
  • On SimpleToM, MS-Reminder+BSTTT has the highest overall accuracy for Qwen and Llama, while matched ICL is strongest for GPT-oss.The composed intervention improves over the reminder alone for all three models.Source: Table 12 · p. 21, §G.2 · p. 21
  • Hi-ToM accuracy improves 13.0–14.7 pp over frozen inference, with ICL strongest for Qwen and Llama and BSTTT-AR strongest for GPT-oss.Recursive belief reasoning is recoverable, but weight updates are not uniformly the most effective intervention.Source: §G.2 · p. 20, Table 13 · p. 22
  • On OpenToM, matched ICL reaches 60.4% for Qwen and 56.4% for Llama, while GPT-oss gains little from ICL and small gains from BSTTT.ICL outperforms every evaluated BSTTT configuration for Qwen and Llama; GPT-oss shows the opposite pattern relative to frozen inference.Source: §G.2 · p. 21, Table 14 · p. 22
  • On DynToM, matched ICL raises Llama accuracy from 19.7% to 32.6% and Qwen accuracy from 35.7% to 39.9%, while GPT-oss benefits little.Errors persist even with Scratchpad-Oracle, so local state information is insufficient for reliable performance; these results do not establish an intrinsic temporal-reasoning limit.Source: §G.2 · p. 21, Table 15 · p. 23

G.3 Per-Subtype and Gap Recovery Breakdown

  • On SimpleToM, Llama’s behavior accuracy rises from 28.9% to 69.7% with MS-Reminder+BSTTT-AR, but ICL lowers judgment accuracy from 26.2% to 14.0%.GPT-oss judgment rises from 3.1% to 18.7% with the composed intervention and to 26.8% with ICL; Qwen’s composed intervention improves all three question types, while ICL improves behavior but reduces recognition and judgment.Source: §G.3 · pp. 21–22, Table 16 · p. 23
Loading 2610.31060v1…