Does ChatGPT have Theory of Mind?

• The study compares ChatGPT-3 and ChatGPT-4 on problems involving human biases in reasoning and decision-making.Source: §1 · p. 1
• The study tests responses to prompts with different levels of detail because ChatGPT is sensitive to subtle prompt differences.Source: §1 · p. 1
• The problems draw on representative examples from the main chapters of Kahneman’s overview of reasoning and decision-making biases.Source: §1 · p. 1
• An answer counts as correct when it predicts the answer people are most likely to give, even if that answer is wrong.For questions about typical human choices, the scoring target is the common response rather than the rational or factually correct response.Source: §1 · p. 1

• Theory of Mind involves reasoning about beliefs, desires, and intentions and how mental states inform explanations and predictions of behavior.Source: §2 · p. 1
• Prior studies of large language models’ Theory of Mind tend to use false-belief tasks such as the Sally-Anne Test.Source: §2 · p. 1
• The paper treats human fallacies as relevant to Theory of Mind because they show behavior that departs from what purely rational grounds would predict.Source: §2 · p. 1
• Unlike Binz and Schulz (2023), who asked what ChatGPT itself would choose, this study asks what a third person would be likely to choose.The authors frame this as a test of ChatGPT’s understanding of human tendencies.Source: §2 · p. 1

3 Experiments

• Each prompt was presented nine times to account for ChatGPT’s stochastic responses, and the prompt set varied the amount and choice of detail.The approach to prompt engineering is summarized in Table 1.Source: §3 · p. 2, Table 1 · p. 2

3.1 Experiments

• The experiments select six problem areas from Kahneman (2011), with their concrete prompt versions provided in Appendix A.Source: §3.1 · p. 2

3.1.1 The Mental Shotgun

• The Mental Shotgun prompts ask which word pair a person would recognize as rhyming faster, using familiar pairs in p1–p4 and different pairs in p5.The task is based on the proposal that concurrently computing other tasks can slow completion of the intended one.Source: §3.1.1 · p. 2

3.1.2 The Anchoring Effect

• The Anchoring Effect experiment adapts the original study, changing names and ages in p1–p4 and using an entirely different instance in p5.The task concerns estimates remaining close to a value considered beforehand.Source: §3.1.2 · p. 2

3.1.3 The Linda Problem

• The Linda Problem tests whether belief-consistent information leads people to favor a conjunction over a single trait, using a modified scenario without Linda’s name or the original occupations.Source: §3.1.3 · p. 2

3.1.4 The Planning Fallacy

• The Planning Fallacy prompts use a kitchen-remodeling project in p1–p4 and a new Amsterdam subway-line project in p5.Source: §3.1.4 · p. 2

3.1.5 Relative Wealth

• The Relative Wealth task asks whether a third person would judge two people with different wealth levels as differently happy; p5 replaces financial wealth with physical fitness.Versions p1–p4 use the same experiment described by Kahneman, while p5 uses a different instance.Source: §3.1.5 · pp. 2–3

3.1.6 Loss aversion

• The Loss Aversion scenario presents a person with a bet that has positive expected value, which Kahneman’s theory predicts an individual would decline.The experiment uses the same scenario described by Kahneman but changes the amount of money.Source: §3.1.6 · p. 3

• 224/270 ChatGPT-4 answers were correct (p < 0.0001), versus 147/270 for ChatGPT-3 (p > 0.05); ChatGPT-4 answered more correctly (two-proportion Z-Test, p < 0.0001).Source: §4 · p. 3
• Figure 1 shows similar average correct-answer counts across prompts for ChatGPT-3, while ChatGPT-4’s averages differ by prompt.Source: Figure 1 · p. 3

5 Conclusion

• ChatGPT-4 answered around 80% of questions correctly, whereas ChatGPT-3 did not answer significantly more than half correctly (p > 0.05).The authors describe ChatGPT-4 as showing a certain amount of comprehension of human biases, a key facet of ToM.Source: §5 · p. 3
• Correct answers sometimes rested on flawed reasoning, while varied prompts mitigated possible training-data parroting without resolving how novel a fair test must be.The authors cite flawed argumentation, concern about reproducing training material, and an open question about problem novelty.Source: §5 · p. 4

A.1 Prompts for the Mental Shotgun problem

• The Mental Shotgun prompts compare how quickly people recognize rhyming word pairs with different degrees of sound-pattern similarity.Versions include pairs such as “note” and “goat” versus “note” and “vote.”Source: §A.1 · p. 6

A.2 Prompts for the Anchoring effect problem

• The Anchoring prompts ask whether people exposed to different reference ages will give different estimates, with another version comparing sentences for identical crimes.The van Gogh versions contrast reference ages 105 and 55; another prompt contrasts prosecutors’ demands of 7 and 12 years.Source: §A.2 · p. 6

A.3 Prompts for the Linda Problem

• The Linda prompts contrast a conjunction such as “teacher and woman” with the simpler “teacher” option in scenarios describing a person.Versions vary the person and the amount of background detail.Source: §A.3 · p. 7

A.4 Prompts for the Planning Fallacy

• The Planning Fallacy prompts ask whether actual project costs are likely to match or exceed estimates, using kitchen renovations and an Amsterdam metro project.The kitchen versions include added details about planned renovations; another version concerns a new metro line.Source: §A.4 · p. 7

A.5 Prompts for the Relative Wealth problem

• The Relative Wealth prompts compare judgments about people with equal current amounts but different gains, losses, or earlier wealth.A variant also compares two people’s progress toward the same one-rep-max result.Source: §A.5 · p. 8

A.6 Prompts for the Loss Aversion problem

• The prompts ask whether a person would take a bet offering a 50% chance to lose $100 and a 50% chance to win $125, or decline it.Versions present the bet to an average American and to Julia; another presents Tom with a 50% chance to lose $200 or win $250.Source: §A.6 · p. 8

B.1 Results for the Mental Shotgun problem

• Table 2 gives the percentage of correct answers for each model and prompt.Source: Table 2 · p. 9
• The table reports a binomial-test p-value for each model and prompt.Source: Table 2 · p. 9
• For each p-value, the table gives the result that can be concluded.Source: Table 2 · p. 9

B.2 Results for the Anchoring effect problem

• Table 3 gives the percentage of correct answers for each model and prompt.Source: Table 3 · p. 9
• The table reports a binomial-test p-value for each model and prompt.Source: Table 3 · p. 9
• For each p-value, the table gives the result that can be concluded.Source: Table 3 · p. 9

B.3 Results for the Linda Problem

• Table 4 gives the percentage of correct answers for each model and prompt.Source: Table 4 · p. 9
• The table reports a binomial-test p-value for each model and prompt.Source: Table 4 · p. 9
• For each p-value, the table gives the result that can be concluded.Source: Table 4 · p. 9

B.4 Results for the Planning Fallacy

• Table 5 gives the percentage of correct answers for each model and prompt.Source: Table 5 · p. 9
• The table reports a binomial-test p-value for each model and prompt.Source: Table 5 · p. 9
• For each p-value, the table gives the result that can be concluded.Source: Table 5 · p. 9

B.5 Results for the Relative Wealth problem

• Table 6 reports the percentage of correct answers for each model and prompt.Source: Table 6 · p. 10
• The table gives a binomial-test p-value for each model and prompt.Source: Table 6 · p. 10
• The table states the conclusion drawn from each p-value.Source: Table 6 · p. 10

B.6 Results for the Loss Aversion problem

• Table 7 reports the percentage of correct answers for each model and prompt.Source: Table 7 · p. 10
• The table gives a binomial-test p-value for each model and prompt.Source: Table 7 · p. 10
• The table states the conclusion drawn from each p-value.Source: Table 7 · p. 10