Source-linked AI summary
ToMStories: Interpreting Theory-of-Mind in Synthetic Narratives
Melanie Subbiah, Chia-En Jacklyn Tsai, Kathleen McKeown
TL;DR
Language models must infer characters’ feelings and motives from behavior and narrative subtext, areas that existing Theory-of-Mind benchmarks have relatively underrepresented. ToMStories evaluates these inferences using controlled synthetic narratives and multiple question formats. The largest models match humans on multiple-choice questions but not open-ended questions, while the authors caution that synthetic-data performance may overstate interpretation of human narratives.
Problem
Existing Theory-of-Mind benchmarks have given relatively little attention to feelings, desires, and intentions, which people may need to infer from behavior and narrative subtext.
Source: §1 · p. 1Method
ToMStories pairs synthetic narrative vignettes about psychological states with questions, using implicit, explicit, and conflicted variants and multiple evaluation formats.
Source: §1 · pp. 1–2Results
The largest models match humans on multiple-choice questions but fall short on open-ended questions.
Source: §1 · p. 2Takeaways & Limitations
Controlled synthetic data can help formulate and test hypotheses about model interpretation, but performance may not represent interpretation of real human psychology.
Source: §6 · p. 8All vignettes come from one model, so benchmark performance may partly reflect alignment with that generator’s conventions rather than general interpretive ability.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
Understanding people often requires inferring what they feel or want from how they behave rather than from what they state outright. We ask whether language models can make these inferences, and how they do so. We introduce Theory-of-Mind Stories (ToMStories), an evaluation dataset of ~6,000 short synthetically generated narrative vignettes, each paired with a multiple-choice or open-ended question about a character's emotion, fear, motivation, or emotional wound. These psychological states are conveyed implicitly (showing), explicitly (telling), or with conflict between dialogue and description. Evaluating thirteen open-weight models (1B-72B parameters), we find that the largest models match humans on multiple choice but not open-ended questions. Our analyses point to simplifications models make in both interpretation and generation of implicit psychological states, indicating that synthetic data can help us understand how models perform interpretation, but likely overestimates how well models interpret humans.
1 Introduction
The paper studies whether language models can infer emotions and motives from narrative behavior, where psychological meaning is often implicit. ToMStories provides controlled synthetic vignettes for evaluating and analyzing these inferences.
1 Introduction
- Feelings, desires, and intentions have received relatively little attention in existing Theory-of-Mind benchmarks, despite their relevance to interpreting people’s psychology.Narratives offer a context where psychological subtext can be expressed naturally and realistically.Source: §1 · p. 1
- ToMStories pairs synthetic vignettes with known psychological states, using implicit, explicit, and conflicted variants to study interpretation and generation of subtext.In conflicted variants, dialogue and description suggest different states.Source: §1 · p. 1, Figure 1 · p. 1
- The benchmark covers emotions, fears, motivations, and emotional wounds, and evaluates thirteen open-weight models from 1B–72B parameters.Its questions use multiple-choice and open-ended formats.Source: §1 · p. 2
- The largest models match humans on multiple-choice questions but fall short on open-ended questions.Source: §1 · p. 2
- Models share indicator phrases associated with psychological states, and overlap with the generator’s phrases is the strongest predictor of performance on implicit ToMStories.Source: §1 · p. 2
- Models treat implicitness partly as muted emotion and tend to rely more on dialogue than description when interpreting character states.Models that balance dialogue and description more evenly are more accurate on implicit ToMStories.Source: §1 · p. 2
2 Related Work
Prior LLM Theory-of-Mind work spans belief reasoning and broader mental-state benchmarks, while feelings, desires, and intentions remain comparatively underrepresented. Related research also studies narrative understanding and model representations.
2 Related Work
- LLM Theory-of-Mind benchmarks cover false-belief reasoning, higher-order belief tracking, causal templates, narrative ToM, and broader suites, but feelings, desires, and intentions remain underrepresented.Source: §2 · p. 2
- Narrative research reports difficulty inferring character psychology from implicit language, while commonsense resources generally use short event descriptions rather than narratives.Source: §2 · p. 2
- Mechanistic interpretability uses probing and steering to study model representations, including prior work decoding characters’ beliefs and emotions.Source: §2 · p. 2
3 ToMStories Dataset
ToMStories uses controlled synthetic narratives to test inference about psychological states across implicit, explicit, and conflicted presentations. Human annotations provide evidence that the implicit and conflicted vignettes are interpretable.
3 ToMStories Dataset
- The dataset draws ground-truth emotions, fears, motivations, and emotional wounds, plus settings and relationships, from a writers’ thesaurus.Its emotion list includes 135 states, broader than common emotion taxonomies; the thesaurus is available only through its free tier and should not be scraped or used commercially.Source: §3 · pp. 2–3
- Each vignette starts from a sampled setting, relationship, and psychological state, then receives implicit, explicit, and conflicted variants.Implicit variants convey states through subtext; explicit variants state them directly, while conflicted variants align dialogue and described behavior with different states.Source: §3 · p. 3
- The dataset contains 2,000 vignettes per variant, with implicit and explicit versions paired on setting, relationship, and state.Training and validation splits balance the four question types; Table 2 reports vignette counts by split and question type, plus average word count.Source: §3 · p. 3, Table 2 · p. 3
3.1 Human baselines
- For explicit variants, all tested models of 8B parameters or larger achieve at least 93% multiple-choice accuracy, supporting the claim that these questions are more straightforward.Source: §3 · p. 3
- Human annotators avoid distractors on more than 80% of implicit and conflicted multiple-choice questions, with substantial interannotator agreement (Cohen’s κ = 0.73).Perfect accuracy is not expected because synthetic vignettes can have distractors close to the correct answer.Source: §3.1 · p. 3
4 Methods
The methods benchmark models on implicit ToMStories questions and then investigate how models represent and communicate hidden psychological states. The analyses combine generated cue phrases with layer-wise representations and steering.
4 Methods
- The study first benchmarks implicit-vignette answers, then probes how models interpret hidden psychological states.The methods also investigate indicator phrases and internal representations.Source: §4 · p. 3
4.1 Benchmarking
- Models answer implicit questions with four-option choices or free-form responses, which are scored using accuracy, embedding similarity, and an LLM judge.The evaluated open-weight models span 1B–72B parameters across five model families.Source: §4.1 · pp. 3–4
4.2 Generating indicator phrases
- Models generate behavioral or physical indicator phrases for 100 psychological states, and the 13,798 resulting phrases are clustered by meaning.Average-linkage clustering uses a 0.5 cosine-distance threshold; the authors report that primary conclusions are robust to other thresholds.Source: §4.2 · pp. 4–5
4.3 Embedding-based interpretability
- For interpretability, the authors extract per-layer hidden states, train answer-index probes, and steer generation along the difference between implicit and explicit representations.They select a layer where probe accuracy starts to plateau; the steering coefficient controls movement toward more implicit or explicit expression.Source: §4.3 · p. 5
5 Results
The largest open models approach human performance on multiple-choice implicit vignettes but not open-ended questions. Further analyses link performance to shared indicator phrases and identify simplifications in how models interpret and generate implicit psychological states.
5.1 Benchmarking open models
- Multiple-choice: Llama-3.1-70B matches human-level performance; open-ended accuracy remains lower (64% humans vs. 59% Qwen-2.5-72B), with larger human advantages on wound and fear.Humans and models are similar on open-ended emotion and motivation vignettes; humans perform roughly 10% better on wound and fear questions.Source: §5.1 · pp. 5–6, Table 4 · p. 6
- On conflicted vignettes, models avoid distractors but favor dialogue-aligned answers; the two Llama 70B models most closely match humans, with a slight dialogue bias.The task contrasts answers implied by dialogue and description, alongside two distractors.Source: Figure 2 · p. 6, §5.1 · p. 6
5.2 Evidence of shared indicator phrases
- Only 7% of states have no open-model overlap with Claude’s indicator phrases; the strongest models overlap on over 40% of states.Claude’s top phrase for the motivation state “being accepted by others” overlaps with 9 of 13 open models.Source: §5.2 · p. 6, Figure 3 · p. 7
5.3 Sharing indicator phrases with the data generator predicts model performance
- Indicator overlap with Claude is the only significant joint predictor (β = 0.60 multiple choice; β = 1.03 open-ended) and explains the largest variance shares (45% and 66%).These are correlations and regression results across models, not evidence that phrase overlap causes stronger performance.Source: §5.3 · p. 7, Table 5 · p. 7
5.4 Implicit means less emotional to models
- Implicit steering reduces emotion words for seven of eight emotions (all but sadness) and lowers valence and arousal, suggesting models simplify implicit expression as less emotional.The authors note that implicit psychological expression can still be quite emotional.Source: Figure 4 · p. 8, §5.4 · p. 8
6 Conclusion
ToMStories provides a controlled synthetic setting for studying how models interpret and process character psychology in narratives. The analyses find shared psychological indicator phrases and simplifications in implicit expression, while cautioning that synthetic-data performance may not represent interpretation of real human narratives.
6 Conclusion
- Models learn shared indicator phrases associated with psychological states, and overlap with the generator’s phrases is the strongest predictor of implicit ToMStories performance.The conclusion also notes that suppressing emotional language is part of how models communicate psychology implicitly.Source: §6 · p. 8
- Models rely more on dialogue than description to infer character states, while synthetic data offers a controlled setting for formulating and testing interpretation hypotheses.The conclusion cautions against treating performance on synthetic human psychology as representative of performance on real human data.Source: §6 · p. 8
Limitations
The benchmark uses vignettes from one model and a Western, English-language thesaurus, limiting the conventions and cultural framing represented. The paper also limits probing and steering analyses to open-weight models and notes sensitivity to layer selection.
Limitations
- All vignettes come from Claude Sonnet 4.6, so accuracy may partly reflect its conventions and overestimate performance on more varied human-written narratives.Vignettes from other generators or written by humans would be needed to separate generator-convention alignment from interpretive ability.Source: Limitations · p. 9
- The tested psychological states and signaling behaviors reflect the Western, English-language framing of the thesaurus.Human baselines from a small, culturally and linguistically varied annotator group estimate human performance only roughly; differences in conventions may contribute.Source: Limitations · p. 9
- Probing and steering analyses cover only open-weight models; fixed smoothing and threshold choices shape layer selection, and steering quality is sensitive to that selection.Source: Limitations · p. 9
A Layer-wise Analysis
Layer-wise probes show distinct accuracy trajectories across model families and similar improvement patterns for implicit and explicit vignettes, with lower accuracy on implicit cases. The analyses select a layer at the onset of a probe-accuracy plateau and report hardware settings for model runs.
A Layer-wise Analysis
- Smaller models show an accuracy jump across about five layers before the midpoint; Qwen improves later, while DeepSeek’s embedding accuracy trails its ultimate accuracy.The paper suggests Qwen’s earlier layers may do more preprocessing and attributes DeepSeek’s later gains to inference-time reasoning and representation updates.Source: A Layer-wise Analysis · p. 12
- Llama-70B probe accuracy improves across more layers, which the authors describe as reflecting more complex processing for higher ultimate accuracy.Source: A Layer-wise Analysis · p. 12
- Implicit-vignette accuracy tracks explicit-vignette improvements across layers, but remains lower, suggesting similar processing mechanisms with a stronger answer signal in explicit cases.Source: A Layer-wise Analysis · p. 12
A.1 Layer selection
- The selected analysis layer is the earliest layer at or before peak accuracy whose smoothed probe accuracy reaches 90% of the rise and stays there for two layers.If no layer meets that condition, the peak layer is used; the two-layer requirement prevents a single-layer spike from moving selection shallower.Source: §A.1 · p. 12
B Hardware Details
- Most models run on one 40GB A100 GPU; models above 25B parameters use two such GPUs and 4-bit quantization, while generation limits differ for reasoning-native DeepSeek models.Most models can generate up to 400 tokens; DeepSeek models can generate up to 1500.Source: B Hardware Details · p. 12
C Embedding Analysis
The embedding analysis examines variant and question-type representations, including probe accuracy across layers and statistics of implicit-versus-explicit activation shifts. The steering procedure scales directions by each model’s residual-stream norm and tunes coefficients against generation coherence.
C Embedding Analysis
- The analysis plots vignette embeddings across variants and types, and defines each type’s variant-shift vector as the difference between implicit and explicit activation centroids.It also computes the mean shift and uses shift-magnitude variation and pairwise cosine similarity to assess consistency across question types.Source: C Embedding Analysis · p. 12
- Linear probes predict correct answers from each layer’s outputs, reporting overall accuracy and separate implicit and explicit accuracy curves.Source: Figure 5 · p. 13
D Steering Coefficient Tuning
- For each steering direction, the procedure searches for the largest coefficient that keeps coherence within one point of baseline on 12 generic writing prompts, then uses the smaller direction-specific magnitude as shared strength.The shared coefficient is checked in both directions.Source: C Embedding Analysis · p. 13, D Steering Coefficient Tuning · p. 13
- Steering strength is normalized by the median residual-stream norm at the steering layer, measured over probe-prompt tokens except each sequence’s first token.Source: D Steering Coefficient Tuning · p. 13
E Benchmark Scoring
The paper defines multiple scoring procedures for model answers, describes human-baseline screening, and details how generated indicator phrases are parsed and cleaned.
E Benchmark Scoring
- Multiple-choice answers count as correct when the correct option number appears in the instructed tags or is the last number in the response’s final 30 characters.Open-ended responses that violate the specified format are marked unparseable and excluded from accuracy calculations.Source: E Benchmark Scoring · p. 13
- Embedding-based scoring compares a free-form response with all four answer options using cosine similarity, then selects the most similar option for accuracy.The fixed encoder is all-MiniLM-L6-v2.Source: E Benchmark Scoring · pp. 13–14
- LLM-based scoring asks a judge to rate the similarity of a model response to the correct answer on a 0–4 Likert scale.The judge is gpt-4o-mini.Source: E Benchmark Scoring · p. 14
F Human Baseline Interface
- Human annotators reviewed 80 implicit and 80 conflicted vignettes; two implicit items were flagged as incoherent, while none were flagged as inappropriate or refusals.No conflicted items were flagged as incoherent.Source: F Human Baseline Interface · p. 14
G.1 Parsing and cleaning generated phrases
- Generated phrases are parsed from JSON or list formats, cleaned of malformed or redundant entries, and reduced to core behaviors averaging 7.3 words.Cleaning removes overly short or long phrases, punctuation-only entries, conversational preambles, and duplicates.Source: §G.1 · p. 14
G.2 Clustering threshold robustness
Indicator-phrase overlap findings were tested across clustering thresholds, with the substantive associations remaining largely consistent. The paper also reports how AI tools were used in the work.
G.2 Clustering threshold robustness
- Higher clustering thresholds merge more phrases and increase overlap values, while the substantive findings hold across thresholds.The main text uses the 0.50 threshold.Source: §G.2 · pp. 14–15, Table 8 · p. 15
- Same-state overlap correlated with implicit accuracy across thresholds (r = .74–.89 for multiple choice and .86–.92 for open-ended; all p < .01).Controlling for model size and description preference, overlap remained a significant predictor at every threshold for open-ended accuracy and most thresholds for multiple choice.Source: Table 8 · p. 15, §G.2 · p. 15
H AI Use
- Claude Code supported experiment implementation and visualization under author instructions; Claude and ChatGPT assisted with literature review, claim summaries, and sentence rewriting.The authors report running the experiments themselves and verifying the code.Source: H AI Use · p. 15