Source-linked AI summary
Evaluating Theory of Mind in Question Answering
Aida Nematzadeh, Kaylee Burns, Erin Grant, Alison Gopnik, Thomas L. Griffiths
TL;DR
Question-answering benchmarks had limited coverage of higher-order beliefs and did not ensure that a correct answer reflected understanding of the world state. This paper builds a theory-of-mind-inspired dataset and evaluates recent neural models on its tasks. None of the models succeed fully on tasks requiring them to track inconsistent beliefs or world states.
Problem
Prior benchmarks covered only first-order beliefs, and answering one question correctly did not guarantee that a model understood the world state.
Source: §1 · p. 2Method
The paper constructs a theory-of-mind-inspired dataset and evaluates recent neural question-answering models on tasks involving inconsistent beliefs and world states.
Source: §7 · p. 9Results
None of the evaluated models succeed fully on the task suite requiring them to track inconsistent beliefs or world states.
Source: §7 · p. 9Takeaways & Limitations
The authors characterize the dataset as diagnostic and conclude that good performance on similar toy tasks is not sufficient to establish reasoning capability.
Source: §7 · p. 9
Abstract
from arXiv · showhide
We propose a new dataset for evaluating question answering models with respect to their capacity to reason about beliefs. Our tasks are inspired by theory-of-mind experiments that examine whether children are able to reason about the beliefs of others, in particular when those beliefs differ from reality. We evaluate a number of recent neural models with memory augmentation. We find that all fail on our tasks, which require keeping track of inconsistent states of the world; moreover, the models' accuracy decreases notably when random sentences are introduced to the tasks at test.
1 Reasoning About Beliefs
Question-answering models need to reason about world states, including beliefs that differ from reality. Existing benchmarks leave gaps in testing higher-order beliefs and whether correct answers reflect understanding of the state of the world; the proposed dataset targets these gaps, and evaluated models do not succeed on its tasks.
1 Reasoning About Beliefs
- Answering a simple story question requires tracking the current state, rather than retrieving a nearby word associated with the question.In the example, Mary is at the office rather than the bathroom.Source: §1 · p. 1
- Most bAbi reasoning tasks required only transitive inference, and existing models failed on only one or two of its 20 tasks.The paper describes bAbi as already too simple for current models.Source: §1 · p. 1
- Prior belief-reasoning benchmarks tested only first-order beliefs, and success on a single question did not ensure understanding of the world state.The paper notes that developmental theory-of-mind experiments use additional questions, such as asking where an object really is, to check whether a correct belief answer was due to chance.Source: §1 · p. 2
- None of the evaluated models succeed at the tasks, which require tracking beliefs or world states that can be inconsistent.The tasks include differences between past and present states and agents’ false beliefs about the world or other agents’ mental states.Source: §1 · p. 2
2 Theory of Mind Experiments
The paper draws on developmental experiments that test whether people can reason about others’ mental states. The examples progress from false first-order beliefs to higher-order beliefs, using control questions to check understanding of the world as well as the belief in question.
2 Theory of Mind Experiments
- The paper uses developmental experiments as guidelines for designing tasks that evaluate question-answering models’ reasoning about others’ mental states.The background describes children’s understanding of desires, perceptions, and beliefs as developing over time.Source: §2 · p. 2
2.1 The Sally-Anne Experiment
- In the Sally-Anne experiment, Sally leaves after placing a marble in her basket, Anne moves it to a box, and participants answer belief, reality, and memory questions.Most children under three are described as answering the belief question with the marble’s actual location; the reality and memory questions help check that a correct belief answer reflects understanding rather than chance.Source: §2.1 · p. 2
2.2 The Icecream Van Experiment
- In the ice-cream-van experiment, John does not know Mary learned the van’s new location, creating a second-order false belief about her belief.Control questions check whether participants answer the second-order question by chance; the cited study found that six- and seven-year-olds could answer it.Source: §2.2 · p. 3
3 The Theory of Mind Task Dataset
The dataset adapts theory-of-mind experiments into three tasks and four question types to test belief reasoning alongside knowledge of an object’s initial and current locations. Its stories are generated from grammars and organized into variants with different task groupings.
3 The Theory of Mind Task Dataset
- The dataset covers true-belief, false-belief, and second-order false-belief tasks, distinguishing whether Sally’s belief matches reality and whether Anne’s belief about Sally’s belief is false.In the second-order task, Sally observes the change, but Anne does not know that she observed it.Source: §3 · p. 3
- The dataset assumes containers are transparent, so an agent becomes aware of an object’s true location on entering a location.The authors say this keeps tasks structurally similar and prevents a model from learning a task-specific answer from a sentence about looking inside a container.Source: §3 · p. 3
- Four question types test first- and second-order beliefs alongside the object’s initial and current locations, helping distinguish world-state understanding from a lucky answer.For true- and false-belief tasks, first- and second-order answers are the same; they differ in the second-order false-belief task because Anne has a false belief about Sally’s belief.Source: §3 · pp. 3–4
- ToM stories group tasks together, whereas ToM-easy stories contain one task; each dataset has 10,000 training examples for each of the 12 task-and-question combinations.ToM test stories contain four tasks and one question at the end; the testing procedure also separates question-position difficulty from the inherent difficulty of the question.Source: §3 · pp. 3–4
- Task grammars specify story entities, their properties, and predicates for actions and beliefs, using a restricted set of action and belief predicates.Stories are generated as grammar derivations.Source: §3 · p. 4
4 The Models
The paper evaluates memory-augmented and relational neural models selected for architectural novelty or strong bAbi results. The models differ in how they represent story information and relate it to questions.
4 The Models
- MemN2N embeds story sentences in memory and uses question similarity to weight them, stacking memory hops across layers.At each layer, the next question embedding combines the previous layer’s output and question embedding.Source: §4 · p. 4
- The Multiple Observer model adds a separate memory module for each agent, containing sentences the agent observed, and attends over those modules according to question relevance.This design addresses the need to represent agents’ differing experiences and knowledge.Source: §4 · p. 5
- EntNet uses gated recurrent units to learn entities and their properties, while the Relation Network considers pairs of sentences together with the question.The paper describes EntNet as capturing story-event sequence and the Relation Network as modeling relations among object pairs.Source: §4 · p. 5
5 Experimental Results
Across the evaluated models, success on ToM-easy did not extend to the more complex ToM tasks, which require tracking inconsistent states of the world. Adding random sentences at test time further reduced accuracy, while shrinking memory could artificially improve performance by limiting the story retained.
5 Experimental Results
- Models were trained jointly without noise and evaluated separately on task-question pairs; success required accuracy above 95% across the task suite.MemN2N results used the best of 10 runs, while EntNet and RelNet showed little initialization sensitivity and used averaged or single-seed results, respectively.Source: §5 · p. 5
5.1 Overall Performance on ToM-easy
- On ToM-easy, EntNet and RelNet achieved mean error of 0, while MemN2N often struggled with first-order false-belief questions.Multiple Observer was similar to MemN2N on most question-task pairs, with less run-to-run variation, but still performed poorly on the first-order question in the false-belief task.Source: §5.1 · pp. 5, 7
5.2 Overall Performance on ToM
- On ToM, Multiple Observer improved most questions, especially second-order questions, but did worse on memory; MemN2N worsened on all but reality questions.All models performed poorly on some first- or second-order questions, while all failed memory questions and succeeded on reality questions among the reported question patterns.Source: §5.2 · p. 7
- None of the evaluated models succeeded on ToM; EntNet and RelNet succeeded on ToM-easy but failed when tasks required handling inconsistent states.Among MemN2N and Multiple Observer, Multiple Observer performed better overall; RelNet performed best on ToM, according to Table 4.Source: §5.2 · p. 7, Table 4 · p. 8
- Memory-question errors were higher than reality-question errors for MemN2N, despite each requiring a single fact in the true-belief task.The authors interpret this as difficulty representing both an object's initial and current locations, as well as agents' mental states.Source: §5.2 · p. 7
5.3 Experimenting with Noise
- At p=.1 noise, every best model lost accuracy; RelNet was most sensitive, while Multiple Observer had the smallest accuracy decrease.The Table 4 comparison identifies Multiple Observer as best on noisy ToM-easy and EntNet as best on noisy ToM.Source: §5.3 · pp. 7–8, Table 4 · p. 8
5.4 Experimenting with Memory
- With memory size 10, MemN2N's hardest-task accuracy rose from 5.1% to 97.5%, but the authors note this retained at most two full tasks.They describe the apparent gain as artificial because ToM and ToM-easy become very similar with the smaller memory.Source: §5.4 · p. 8
6 Related Work
Prior work has approached mental-state reasoning through visual mistaken-belief identification, false-belief question answering, behavior prediction, and studies of people’s judgments about visual-question-answering systems.
6 Related Work
- Eysenbach et al. (2016) model mistaken-belief identification in visual scenes, whereas this work asks whether models can reason about true and false beliefs to answer questions.Their regression model identifies a mistaken belief and the person holding it in a scene.Source: §6 · p. 8
- Grant et al. (2017) find that MemN2N fails a false-belief test, while a Multiple Observer extension with separate agent memories and an observer outperforms it.Their benchmark examines interactions between beliefs and actions.Source: §6 · p. 8
- Rabinowitz et al. (2018) introduce ToMnet, a meta-learning model that predicts agent behavior from trajectories and succeeds when behavior reflects false belief rather than reality.Their model includes an observer with access to agents’ states and actions, which may be noisy or partial.Source: §6 · p. 8
- Chandrasekaran et al. (2017) study whether people can predict a visual-question-answering system’s failures and answers, finding that a few examples improve their responses.Source: §6 · p. 9
7 Discussion
The paper introduces a dataset for testing whether question-answering models reason about beliefs and inconsistent world states. The evaluated models do not fully succeed, and the dataset is intended as a diagnostic rather than a test of language fluency.
7 Discussion
- None of the evaluated models fully succeeds on tasks requiring tracking of inconsistent beliefs or world states.The evaluation includes three recent neural QA models and an extension of one.Source: §7 · p. 9
- The dataset tests whether question-answering models can reason about different types of beliefs while maintaining a correct understanding of the world.Source: §7 · p. 9
- Consistent linguistic structure is intended to isolate reasoning performance, and the dataset is a diagnostic: success on similar toy tasks is not sufficient evidence of reasoning capability.Source: §7 · p. 9