PaperlayerView exact revision on arXiv

Source-linked AI summary

PHAnToM: Persona-based Prompting Has An Effect on Theory-of-Mind Reasoning in Large Language Models

Fiona Anting Tan, Gerard Christopher Yeo, Kokil Jaidka, Fanyou Wu, Weijie Xu, Vinija Jain, Aman Chadha, Yang Liu, See-Kiong Ng

arXiv:2403.02246v3cs.CL

TL;DR

Prior work reports that LLMs lag humans in ToM and has largely studied ToM assessment and role-oriented prompting separately, with limited explanation of prompting effects. PHAnToM evaluates persona-based prompts across ToM tasks and reports that Dark Triad traits have a larger impact than Big Five traits across models and tasks.

  • Problem

    LLMs remain inferior to humans in ToM, while prior studies have largely treated ToM assessment and role-oriented prompting separately and have offered few explanations for varying prompt effects.

    Source: Introduction · p. 1

  • Method

    The study evaluates personality-trait personas on ToM tasks across LLMs and proposes a mechanism for why persona prompting styles may yield different performance.

    Source: Introduction · p. 2, Theory-of-Mind Reasoning Task · p. 4

  • Results

    Dark Triad traits have a larger effect than Big Five OCEAN traits on ToM performance across models and tasks.

    Source: Conclusion · p. 8

  • Takeaways & Limitations

    Persona assignments can have positive or negative effects on LLM social-cognitive reasoning, warranting attention to which traits are induced.

    Source: Introduction · p. 2, Conclusion · p. 8

Abstract

from arXiv · show

The use of LLMs in natural language reasoning has shown mixed results, sometimes rivaling or even surpassing human performance in simpler classification tasks while struggling with social-cognitive reasoning, a domain where humans naturally excel. These differences have been attributed to many factors, such as variations in prompting and the specific LLMs used. However, no reasons appear conclusive, and no clear mechanisms have been established in prior work. In this study, we empirically evaluate how role-playing prompting influences Theory-of-Mind (ToM) reasoning capabilities. Grounding our rsearch in psychological theory, we propose the mechanism that, beyond the inherent variance in the complexity of reasoning tasks, performance differences arise because of socially-motivated prompting differences. In an era where prompt engineering with role-play is a typical approach to adapt LLMs to new contexts, our research advocates caution as models that adopt specific personas might potentially result in errors in social-cognitive reasoning.

Introduction

LLMs have shown inadequate social-cognitive reasoning despite their capabilities on other NLP tasks, and prior work leaves gaps in connecting role-oriented prompting to ToM performance and explaining its effects. PHAnToM audits persona-based prompting across ToM tasks and reports that Dark Triad traits have a larger impact than Big Five traits.

Introduction

  • LLMs show generally inadequate social-cognitive reasoning, including ToM, despite demonstrated capabilities across NLP tasks.ToM involves attributing mental states such as beliefs, intentions, thoughts, and emotions to oneself and others.Source: Introduction · p. 1
  • Prior research has largely studied ToM assessment and role-oriented prompt engineering separately, while offering few mechanisms for why prompting styles yield different ToM performance.This gap is salient as LLMs are increasingly used in settings requiring sophisticated human interaction.Source: Introduction · pp. 1–2
  • PHAnToM examines how persona-based prompts affect three ToM tasks and asks how effects vary across models, tasks, and cumulative prompting.The study also proposes a mechanism for why different persona styles elicit varying ToM performance.Source: Figure 1 · p. 1, Introduction · p. 2
  • Dark Triad traits have a larger impact than Big Five traits on ToM performance across models and tasks.The study also reports that LLMs with higher ToM performance variance across persona prompts tend to be more controllable in personality tests.Source: Introduction · p. 2

Related Work

Prior work documents LLM sensitivity to prompts and explores persona induction and ToM separately, leaving task-performance variability under induced personas and cross-task effects underexplored.

Sensitivity of LLMs to Prompts

  • Prompt variations, role-play, and other prompt strategies have been associated with changes in LLM performance, motivating study of personality role-play in social-cognitive reasoning.Prior work also reports that persona prompts can affect reasoning across socio-demographic groups.Source: Sensitivity of LLMs to Prompts · p. 2

Inducing Personas in LLMs with Prompts

  • Personality traits are used to shape LLM personas, and prior studies report reliable personality measurements under prompts but do not account for possible task-performance variability.The paper distinguishes a persona—a constructed identity or role shaped by traits—from personality, described as relatively enduring patterns of cognition, feeling, and behavior.Source: Inducing Personas in LLMs with Prompts · pp. 2–3

Theory-of-Mind Reasoning

  • ToM is commonly assessed through false-belief tasks that test whether people recognize that agents can hold beliefs inconsistent with reality.The Sally and Ann task is a prototypical example, and ToM supports social communication and relationships.Source: Theory-of-Mind Reasoning · p. 3
  • Prior LLM ToM studies report performance below humans and describe models adopting an omniscient view, while often focusing on one reasoning task.The paper identifies induced-persona effects across ToM tasks of varying complexity and cognitive demand as underexplored.Source: Theory-of-Mind Reasoning · p. 3

Methodology

The methodology tests persona-based prompting across eight personas and three ToM tasks, using trait descriptions and task prompts alongside model-specific evaluations.

Methodology

  • The study examines eight personas and three ToM tasks to assess persona-based prompting effects on LLM reasoning.The overview names Information Access (IA), Answerability (AA), and Belief Understanding (BU).Source: Methodology · p. 3

Prompting strategies

  • Eight trait-based persona descriptions draw on personality theory and validated psychometric measures, with wording reviewed by an author trained in personality psychology.The eight traits comprise the Big Five OCEAN and Dark Triad.Source: Prompting strategies · p. 3

Theory-of-Mind Reasoning Task

  • IA tests whether a character has information access, AA adds whether they can answer correctly, and BU requires inferring characters’ beliefs.IA and AA each contain 3,571 examples, and BU contains 993; BU is described as the most challenging task.Source: Theory-of-Mind Reasoning Task · p. 4

Triggering Personalities in LLMs for ToM

  • Personas were ordered from least to most social, from Psychopathy through Machiavellianism and Narcissism to the Big Five traits, ending with Agreeableness.The ordering reflects their described relationships to social behavior and interpersonal interaction.Source: Triggering Personalities in LLMs for ToM · p. 4

Experimental setup

  • The experimental setup used five instructed LLMs and weighted F1 for IA and AA, accuracy for BU, and greedy decoding for Huggingface models.The models were Mistral 7B, Llama 2, Falcon 7B, Zephyr 7B Beta, and GPT-3.5; experiments used random seed 99.Source: Experimental setup · p. 4

Results

Persona-based prompts shift ToM performance, with effects varying by model, trait, and task; Answerability is especially sensitive. The reported patterns include adverse shifts for Dark Triad traits and improvements associated with more socially positive traits, alongside mixed results for conventional task-specific role-play.

Manipulation Checks of Persona-based Prompts

  • GPT-3.5 showed the strongest correspondence between prompted OCEAN traits and MPI scores, while Falcon’s scores remained consistent across persona prompts.The manipulation check administered the 120-question Machine Personality Inventory to measure the Big Five traits.Source: Manipulation Checks of Persona-based… · pp. 4–5

Main Findings

  • In Answerability, Machiavellianism and Psychopathy reduced Llama 2’s weighted F1 by 33.0 and 27.8 points, while Agreeableness raised GPT-3.5’s by 4.3 points.Table 1 reports weighted F1 for Information Access and Answerability and accuracy for Belief Understanding, with changes measured against no-prompt baselines.Source: Main Findings · p. 5, Table 1 · p. 6
  • Llama 2 was most sensitive to persona prompts, with a 33% F1 decrease; Falcon was least sensitive, shifting by at most 1.8%.Reported sensitivities were 9.1% for Mistral, 6.7% for GPT-3.5, and 4.5% for Zephyr; the authors suggest training differences may contribute.Source: Main Findings · pp. 6–7
  • For Mistral 7B, Llama 2, and GPT-3.5, cumulative Answerability changes began negative under Dark Triad traits and Neuroticism, then rose with Conscientiousness and Agreeableness.Similar patterns for Information Access and Belief Understanding are reported in the Appendix.Source: Main Findings · p. 7, Figure 5 · p. 7

Sensitivity to Personality Description

  • Mistral 7B’s ToM performance showed no major changes between the authors’ personality descriptions and Jiang et al.’s alternative descriptions.The comparison covered Information Access, Answerability, and Belief Understanding scores.Source: Sensitivity to Personality Description · p. 7, Table 2 · p. 8

Comparison with Traditional Role-play

  • Traditional task-specific role-play increased Answerability performance for all models except GPT-3.5, while its effects on the other tasks were mixed.The authors conclude that such prompts are not always effective for ToM tasks.Source: Comparison with Traditional Role-play · p. 7

Discussion

  • The discussion reports Agreeableness-associated gains in Answerability and lower ToM performance with Psychopathy and Machiavellianism prompts across tasks, but Narcissism results diverged from prior psychological findings.The authors relate the Agreeableness pattern to psychological work linking the trait with ToM, and describe the Narcissism result as inconsistent with previous findings.Source: Discussion · pp. 7–8

Conclusion

PHAnToM finds that personality-targeted prompts affect LLM Theory-of-Mind reasoning, with effects varying by trait and model. The authors advise caution when using persona induction and call for research on beneficial and detrimental traits.

Conclusion

  • Dark Triad traits have larger effects than Big Five OCEAN traits on ToM performance across models and tasks.The authors especially identify GPT-3.5, Llama 2, and Mistral as affected.Source: Conclusion · p. 8
  • Personality induction has both positive and negative effects on LLM social-cognitive reasoning.Effects differ across models; Falcon is described as less likely to be simulated and affected by these prompts.Source: Conclusion · p. 8
  • Users should exercise caution because personality and persona induction affect social-cognitive reasoning differently across LLMs.Source: Conclusion · p. 8
  • The authors call for future research to identify traits that benefit LLMs and mitigate detrimental ones.Source: Conclusion · p. 8

Paper Checklist

The checklist points to supplementary materials and an online repository for reproducibility details. It also directs readers to the paper’s Methods and Results, Discussion, Limitations, and Ethics sections for evaluation, limitations, and impact-related information.

Paper Checklist

  • The checklist reports that limitations and mitigation steps are discussed in Limitations, while potential negative societal impacts and misuse are addressed in the Ethics statement.Source: Paper Checklist · p. 10
  • Code, data, instructions, training details, and compute information are reported in supplementary materials and an online repository.Source: Paper Checklist · p. 11
  • The checklist locates evaluation justification in Methods and Results and Discussion, and the cost of misclassification and faults in Limitations.Source: Paper Checklist · p. 11

Supplementary Materials

The supplementary materials provide persona descriptions, prompt templates, and model-generation settings. The templates show how a persona description and task input are combined, while the model details specify a shared seed and decoding choices.

A. Persona Description

B. Prompts

  • The prompt templates combine conversation context and a task question, with persona prompting adding a personality description before those inputs.The materials also include a Narcissism persona example for an Answerability task.Source: B. Prompts · p. 12

C. Model Details

  • Experiments used random seed 99; Huggingface models except GPT-3.5 used greedy decoding.The listed models include Mistral, Llama 2, Falcon, and Zephyr; settings include temperature 0 and max new tokens 256.Source: C. Model Details · p. 13

D. Additional Results

The additional-results section presents scores for three Theory-of-Mind tasks across the experiments and compares them with scores reported in the original paper.

D. Additional Results

  • Table 3 presents scores for all experiments across three ToM tasks and compares them with the original paper’s reported scores.Source: D. Additional Results · p. 13

E. Comparison with Traditional Role-play

The study compares a task-specific role-play prompt with personality-based prompting and finds that its effects vary by model and ToM task. Traditional task-specific role-play is not consistently effective.

E. Comparison with Traditional Role-play

F. MPI Findings

Responses to the MPI statement “You trust others.” vary across models, including whether models refuse, explain, or embody a prompted personality in their wording.

F. MPI Findings

  • Zephyr without a personality prompt refuses to answer the MPI statement, saying it lacks personal beliefs or experiences.Source: F. MPI Findings · p. 13
  • Mistral, Llama 2, and Zephyr tend to explain their choices, while Falcon and GPT-3.5 tend to state only their choice.Source: F. MPI Findings · p. 13
  • Some responses use first-person or second-person pronouns, while others describe a person in the third person.Llama 2 with an Agreeableness prompt, for example, responds, “I believe that trust is a fundamental aspect of my personality.”Source: F. MPI Findings · p. 13

G. Sensitivity to persona-based prompting across tasks

Models differ in how strongly persona prompts affect their performance, and sensitivity varies across ToM tasks. GPT-3.5 and Llama show the greatest sensitivity overall, while Falcon shows the least.

G. Sensitivity to persona-based prompting across tasks

H. Cumulative effects of persona-based prompting on ToM tasks

Cumulative effects of personality traits differ between Information Access and Belief Understanding. For Information Access, most models show the opposite cumulative pattern to the one described for other ToM tasks.

H. Cumulative effects of persona-based prompting on ToM tasks

Loading 2403.02246v3…