Source-linked AI summary
Do Large Language Models know what humans know?
Sean Trott, Cameron Jones, Tyler Chang, James Michaelov, Benjamin Bergen
TL;DR
The paper asks whether human ability to reason about others’ beliefs develops partly through experience such as language input. It tests GPT-3 on a text-based False Belief Task and finds lower accuracy than humans, with model behavior not fully explaining human responses.
Problem
The paper asks where the human ability to reason about others’ beliefs comes from, including whether it depends on experience such as language input.
Source: Conclusion · p. 23Method
The study tests whether GPT-3’s predictions on a text-based False Belief Task reflect characters’ implied knowledge states.
Source: Analysis of Large Language Model… · p. 10, Discussion · p. 13Results
82.7% human accuracy exceeded the largest LLM’s 74.5%, and LLM behavior did not fully explain human behavior.
Source: Conclusion · p. 23Takeaways & Limitations
The results help quantify language input’s contribution, while the LLM’s behavior does not fully explain human responses.
Source: Conclusion · p. 23The paper cautions that LLMs may not provide a psychologically plausible mechanism for human comprehension.
Source: Using LLMs to study human comprehenders · p. 21
Abstract
from arXiv · showhide
Humans can attribute beliefs to others. However, it is unknown to what extent this ability results from an innate biological endowment or from experience accrued through child development, particularly exposure to language describing others' mental states. We test the viability of the language exposure hypothesis by assessing whether models exposed to large quantities of human language display sensitivity to the implied knowledge states of characters in written passages. In pre-registered analyses, we present a linguistic version of the False Belief Task to both human participants and a Large Language Model, GPT-3. Both are sensitive to others' beliefs, but while the language model significantly exceeds chance behavior, it does not perform as well as the humans, nor does it explain the full extent of their behavior -- despite being exposed to more language than a human would in a lifetime. This suggests that while statistical learning from language exposure may in part explain how humans develop the ability to reason about the mental states of others, other mechanisms are also responsible.
Method
The study tested whether GPT-3 and humans use characters’ implied knowledge states to predict an object’s location in False Belief Task passages. It varied the cues and wording around belief and location, then compared model and human responses.
Method
- The passages followed False Belief Task structure: a character placed an object in a Start location, which another character either moved while the first was away or while present.The final sentence stated or implied where the main character believed the object to be, with Start as the belief-consistent response for false belief and End for true belief.Source: False Belief Passages · pp. 7–8
Analysis of Large Language Model Behavior
- Adding Knowledge State improved GPT-3 model fit (χ2(1) = 18.6, p < .001); in the True Belief condition, GPT-3 correctly predicted characters’ greater likelihood of looking in the End Location after the object was moved.Source: Analysis of Large Language Model… · p. 10
- The Knowledge State effect was stronger with implicit than explicit cues (β = −2.57, SE = 0.548), but was significant in both conditions.The Explicit condition had χ2(1) = 13.3, p < .001; the Implicit condition had χ2(1) = 18.8, p < .001.Source: Analysis of Large Language Model… · p. 10
Analysis of Human Responses
Human responses were influenced by characters’ Knowledge State, and the retained participants showed a stronger effect and higher accuracy than GPT-3. An exploratory analysis also examined accuracy before and after exclusions.
Analysis of Human Responses
- The human-response analysis tested whether Knowledge State explained responses beyond GPT-3 Log-Odds and other predictors.The base model included GPT-3 Log-Odds, Knowledge Cue, First and Recent Mention, and by-item effects.Source: Analysis of Human Responses · p. 11
- Among retained human participants, accuracy was 82.7%, higher than GPT-3’s 74.5%; although both were sensitive to Knowledge State, humans showed a much stronger effect across conditions.Adding Knowledge State significantly improved the human-response model (χ2(1) = 30.4, p < 0.001).Source: Analysis of Human Responses · p. 11
- Before exclusions, human accuracy was 55.8%; after excluding responses outside the Start and End locations, it was 73.1%.This exploratory estimate was not preregistered, and the authors noted concerns about Mechanical Turk data quality when interpreting the retained sample.Source: Analysis of Human Responses · p. 12
Analysis of GPT-3 Token Predictions
GPT-3’s predictions reflected characters’ implied knowledge states, but remained less accurate and less explanatory of human behavior. Performance was best among the largest models, and token-generation scoring produced a similar result.
Analysis of GPT-3 Token Predictions
- For token generation, GPT-3 completed the critical sentence with its highest-probability next word, which was then checked against the correct location.The analysis tested whether linguistic input could account for human sensitivity to knowledge states.Source: Analysis of GPT-3 Token Predictions · p. 12, Discussion · p. 13
- 74.5% accuracy using relative probabilities for Start versus End locations was close to the best model’s 73.4% token-generation accuracy.The token-generation procedure allowed GPT-3 to predict a word rather than restricting its prediction to the two locations; this did not qualitatively change the results.Source: Analysis of GPT-3 Token Predictions · pp. 12–13
- The largest models performed best: davinci answered correctly on 60.4% of items and text-davinci-002 on 73.4%.The smallest, lowest-performing models did not exceed chance performance.Source: Analysis of GPT-3 Token Predictions · p. 13
- 73.4% accuracy under token generation showed GPT-3’s predictions were sensitive to implied knowledge states, while humans reached 82.7%.The authors report that GPT-3 behavior did not fully explain human behavior, so language statistics alone did not account for the full human pattern.Source: Discussion · pp. 13–14
Do LLMs attribute beliefs?
The findings provide mixed evidence about LLM sensitivity to others’ beliefs: the strongest models showed sensitivity, but did not reach human-level performance. The paper’s conclusion is bounded to belief reasoning in the False Belief Task.
Do LLMs attribute beliefs?
- The False Belief Task measures reasoning about others’ beliefs to predict behavior, not every proposed aspect of Theory of Mind.The study does not address inferring implicit emotional states or reasoning about intended interpretations of utterances.Source: Do LLMs attribute beliefs? · p. 14
- State-of-the-art LLMs showed sensitivity to others’ beliefs in the False Belief Task, but did not achieve human-level performance.The paper says this behavior appeared only in the largest, highest-performing LLMs.Source: Do LLMs attribute beliefs? · p. 14
Competing Interpretations.
The discussion contrasts attributing belief reasoning from observable behavior with rejecting that possibility for language models in advance. It also considers whether the False Belief Task is valid for models, and leaves these interpretations unresolved.
Competing Interpretations.
- The duck-test view attributes cognitive properties from observable behavior, whereas axiomatic rejection denies LLMs can reason about others’ mental states because of their constitution.The cited passage names lack of embodiment, grounding, agency, and an interactive social environment as examples.Source: Competing Interpretations · pp. 14–15
- On the duck-test view, GPT-3’s better-than-chance False Belief Task performance supports ascribing belief reasoning to LLMs, though to a lesser extent than humans.This interpretation also suggests that language experience alone could undermine claims that innate endowment or social experience is necessary.Source: Competing Interpretations · p. 15
- On axiomatic rejection, GPT-3’s success could expose the False Belief Task as a flawed measure, because confounds might produce passing behavior without the capacity it targets.The authors connect this interpretation to existing critiques of the task.Source: Competing Interpretations · pp. 15–16
- A differential construct-validity position allows that LLMs may lack belief attribution while the task remains valid for humans; the authors endorse none of these interpretations.They say the evidence cannot adjudicate the debate and call for further empirical probing of whether humans and LLMs use similar processes.Source: Competing Interpretations · pp. 16–17
Insights from Comparative Cognition.
Comparative cognition faces a similar difficulty: behavior consistent with belief attribution may also reflect simpler responses to observable regularities. The discussion presents diverse experiments as a way to test whether varied cues are treated as equivalent situations of not seeing.
Insights from Comparative Cognition.
- Behavior consistent with belief attribution in nonhuman animals may instead reflect responses to observable regularities, a challenge known as the logical problem.The simpler alternative is described as complementary behavior reading rather than mindreading.Source: Insights from Comparative Cognition · p. 18
- Distinguishing belief attribution from behavioral regularities is difficult because belief attribution also relies on observable behavior.One proposed response is to use diverse experiments and techniques, testing whether varied cues belong to the same class of situations that lead to not seeing.Source: Insights from Comparative Cognition · p. 18
- The debate over belief attribution in comparative cognition suggests that resolving the analogous question for LLMs will likely be a serious challenge.The discussion adds that relevant philosophical theories may prove impossible to adjudicate between.Source: Insights from Comparative Cognition · p. 18
- A proposed approach is to test LLMs across diverse stimuli and experiments, then ascribe belief attribution if their behavior is consistently compatible with it, as for humans.This proposal assumes the same evidence would warrant that attribution for human participants.Source: Insights from Comparative Cognition · pp. 18–19
What can belief sensitivity tell us about Theory of Mind?
The discussion questions whether Theory of Mind is a coherent construct, given the broad range of abilities and limited convergence among measures. It proposes comparing LLMs and humans across a battery of tasks to examine validity and the role of distributional statistics.
What can belief sensitivity tell us about Theory of Mind?
- The authors restrict their claims to sensitivity to true and false beliefs, not Theory of Mind evidence more broadly.They note that belief representation is sometimes considered part of a wider set of abilities called mindreading or mentalizing.Source: What can belief sensitivity tell us… · p. 19
- Theory of Mind spans distinct abilities, but measures often show poor convergent validity and may not reliably predict social behavior.The discussion notes that low convergence raises questions about whether the tasks measure the same thing and whether the abilities belong under one umbrella term.Source: What can belief sensitivity tell us… · pp. 19–20
- Theory of Mind instruments assess different components, including false-belief reasoning, story interpretation, emotional-state inference, and mental-state attribution to animated shapes.Performance on the Short Story Task has been reported to correlate with performance on Reading the Mind in the Eyes task.Source: What can belief sensitivity tell us… · p. 19
- Future work could assess LLMs across a Theory of Mind task battery using a human benchmark for each task.This would allow examination of distributional-statistics explanations and whether measures converge in humans and LLMs.Source: What can belief sensitivity tell us… · p. 20
- Comparing task convergence across humans and LLMs could provide another test of Theory of Mind’s coherence as a construct.The results could inform debates about pluralist or eliminativist views of Theory of Mind.Source: What can belief sensitivity tell us… · p. 20
Using LLMs to study human comprehenders
The authors use LLMs to test whether distributional language information can support behavior consistent with belief attribution, while noting important limits on what model performance establishes about human cognition. They argue that evidence for such sufficiency is a critical step toward assessing experience-based theories of belief attribution.
Using LLMs to study human comprehenders
- The GPT-3 comparison tests whether distributional information alone can support behavior consistent with belief attribution, without humans’ rich social and multimodal scaffolding.The approach measures sufficiency in the absence of that scaffolding, not whether LLMs provide a psychologically plausible mechanism for humans.Source: Using LLMs to study human comprehenders · pp. 20–21
- GPT-3 was exposed to more than 200 billion words, approximately 2,000 times the estimated exposure of a 10-year-old, so its performance may overstate what humans learn from language alone.The passage contrasts this with estimates of 30–110 million words by age 10.Source: Using LLMs to study human comprehenders · p. 21
- The largest GPT-3 model tested, text-davinci-002, performed much better on the False Belief Task than smaller models.The differential effects of model parameter count and training-dataset size remain unclear.Source: Using LLMs to study human comprehenders · p. 21
- Even developmentally plausible model behavior would not show that humans use the same statistical mechanism, since distinct routes can produce the same behavior.Humans could use innate or domain-specific capacities to produce behavior models learn to imitate from language statistics.Source: Using LLMs to study human comprehenders · p. 22
- LLMs offer opportunities to test specific mechanisms, including which language inputs matter and whether training on dialogue or essays changes belief-reasoning behavior.The passage identifies mental-state verbs, interactive conversation, and syntactic constructions as possible sources of relevant linguistic experience.Source: Using LLMs to study human comprehenders · p. 23
- Evidence that distributional information suffices for competent False Belief Task performance is a critical step toward assessing experience-based theories of belief attribution in humans.The authors also emphasize that LLM comparisons focus on Theory of Mind specifically.Source: Using LLMs to study human comprehenders · pp. 20, 23
Conclusion
The text-based False Belief Task helps quantify language input’s contribution to belief reasoning: humans answered correctly 82.7% of the time, versus 74.5% for the largest LLM. LLM behavior did not fully explain human behavior, suggesting language statistics alone are insufficient to account for human sensitivity to false belief.
Conclusion
- Humans answered correctly 82.7% of the time, while the largest LLM responded correctly 74.5% of the time on the text-based False Belief Task.Correct responses were congruent with a character’s belief state.Source: Conclusion · p. 23
- LLM behavior did not fully explain human behavior, so language statistics alone were not sufficient to account for human sensitivity to false belief.The results suggest language statistics can generate some sensitivity to false belief, but not fully account for human sensitivity.Source: Conclusion · pp. 23–24
- Human ability to attribute mental states may involve linking linguistic input with innate capacities or embodied and social experiences.The results quantify language input’s contribution, while leaving these other possible contributors in view.Source: Conclusion · pp. 23–24
Knowledge State False Belief True Belief
Humans and GPT-3 showed sensitivity to characters’ belief states, but the effect was stronger in humans and GPT-3 predictions did not account for the human Knowledge State effect. Across models, text-davinci-002 came closest to human accuracy, while text-001-ada performed worst.
Knowledge State False Belief True Belief
- Humans showed a stronger tendency than GPT-3 to select the Start location when a character had not seen the object move, and a human Knowledge State effect was not accounted for by GPT-3 predictions.Figure 2 contrasts responses in False Belief and True Belief conditions.Source: Knowledge State False Belief True Belief · p. 34, Figure 2 · p. 34
- A response was knowledge-congruent when it matched the character’s likely location belief: End in True Belief and Start in False Belief.The character was present in the True Belief condition and absent in the False Belief condition.Source: Knowledge State False Belief True Belief · p. 35
- 73.4% accuracy made text-davinci-002 the model closest to human accuracy of 82.7%.The passage identifies text-davinci-002 as the largest fine-tuned model; text-001-ada, the smallest, performed worst.Source: Knowledge State False Belief True Belief · p. 35, Figure 3 · p. 35