PaperlayerView exact revision on arXiv

Source-linked AI summary

Can Large Language Model Agents Simulate Human Trust Behavior?

Chengxing Xie, Canyu Chen, Feiran Jia, Ziyu Ye, Shiyang Lai, Kai Shu, Jindong Gu, Adel Bibi, Ziniu Hu, David Jurgens, James Evans, Philip Torr, Bernard Ghanem, Guohao Li

arXiv:2402.04559v4cs.AIcs.CLcs.HC

TL;DR

LLM agents are increasingly used to simulate people, but whether they can simulate human behavior remains an open question. This paper studies trust using Trust Games and BDI modeling, finding that agents generally exhibit trust behavior and GPT-4 agents show high alignment with humans. It also examines trust biases and conditions that may influence agent trust.

  • Problem

    LLM agents are increasingly used to model humans, but the assumption that they behave like humans in simulation remains insufficiently validated.

    Source: §1 · p. 1

  • Method

    The paper uses Trust Games and BDI modeling to study agent trust, compare it with human trust, and probe trust across demographics, counterpart types, instructions, and reasoning strategies.

    Source: §1 · p. 2, §2.2 · p. 4

  • Results

    LLM agents generally exhibit trust behavior in Trust Games, GPT-4 agents show high behavioral alignment with humans, and smaller models show relatively lower alignment.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings suggest the feasibility of simulating human trust behavior with LLM agents and may inform study of human interactions and social institutions where trust is critical.

    Source: §7 · p. 10

    Trust Games simplify real-world scenarios, and the authors call for further study in complex, dynamic environments.

    Source: C Limitations and Future Works · p. 20

Abstract

from arXiv · show

Large Language Model (LLM) agents have been increasingly adopted as simulation tools to model humans in social science and role-playing applications. However, one fundamental question remains: can LLM agents really simulate human behavior? In this paper, we focus on one critical and elemental behavior in human interactions, trust, and investigate whether LLM agents can simulate human trust behavior. We first find that LLM agents generally exhibit trust behavior, referred to as agent trust, under the framework of Trust Games, which are widely recognized in behavioral economics. Then, we discover that GPT-4 agents manifest high behavioral alignment with humans in terms of trust behavior, indicating the feasibility of simulating human trust behavior with LLM agents. In addition, we probe the biases of agent trust and differences in agent trust towards other LLM agents and humans. We also explore the intrinsic properties of agent trust under conditions including external manipulations and advanced reasoning strategies. Our study provides new insights into the behaviors of LLM agents and the fundamental analogy between LLMs and humans beyond value alignment. We further illustrate broader implications of our discoveries for applications where trust is paramount.

1 Introduction

Although LLM agents are increasingly used to simulate people, the assumption that they behave like humans remains insufficiently validated. This paper studies trust as a fundamental human behavior, finding that agents generally exhibit trust in Trust Games and that GPT-4 agents align highly with human trust behavior.

1 Introduction

  • Trust is a critical human behavior, yet whether LLM agents can genuinely simulate human behavior remains an open question.The paper focuses on trust because it matters across social settings from daily communication to economic and political institutions.Source: §1 · p. 1
  • Using Trust Games and BDI modeling, the study examines agent trust and its behavioral alignment with human trust.Figure 1 presents the major components of this framework.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • LLM agents generally exhibit trust behavior in Trust Games, while GPT-4 agents show high behavioral alignment with humans.The paper defines behavioral alignment in terms of behavioral factors and dynamics; smaller models show relatively lower alignment.Source: §1 · p. 2
  • The study probes demographic bias, differences toward human versus LLM counterparts, explicit trust instructions, and reasoning strategies.These investigations report demographic bias, a relative preference for humans over agents, greater ease of undermining than enhancing trust, and possible influence from advanced reasoning.Source: §1 · p. 2
  • The authors point to implications for human simulation, role-playing, LLM cooperation, human-agent collaboration, and LLM safety.They also frame the findings as relevant to understanding analogies between LLMs and humans beyond value alignment.Source: §1 · p. 3

2 LLM Agents in Trust Games

The study uses Trust Games and their variations to examine trust behavior, then configures LLM agents with diverse models and personas. It uses BDI outputs to make agents’ decision reasoning explicit.

2.1 Trust Games

  • Trust Games include a trustor sending money to a trustee, with six game types varying reciprocity, risk, probability, or repeated interaction.The set comprises the Trust Game, Dictator Game, MAP Trust Game, Risky Dictator Game, Lottery Game, and Repeated Trust Game.Source: §2.1 · p. 3

2.2 LLM Agent Setting

  • Experiments use CAMEL with closed- and open-source LLMs at temperature 1, while GPT-4-generated personas provide diverse profiles.Personas include names, ages, genders, addresses, jobs, and backgrounds.Source: §2.2 · pp. 3–4
  • Agents explicitly output Beliefs, Desires, and Intentions as their decision-making reasoning process in Trust Games.The paper adopts BDI modeling to gain deeper insight into LLM-agent behavior.Source: §2.2 · p. 4

3 Do LLM Agents Manifest Trust Behavior?

The study assesses trust behavior through amounts sent in a Trust Game and whether those decisions can be interpreted through agents’ BDI reasoning. It finds that agents generally exhibit trust behavior, with substantial variation across models and personas.

3.1 Amount Sent

  • Most models have high Valid Response Rate, except Llama-7b, and amounts sent are predominantly positive.Valid Response Rate measures the share of personas whose transfer falls within the trustor’s initial $10.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 Belief-Desire-Intention (BDI)

  • BDI examples show how trust-related reasoning can justify different transfers, including high giving and a modest amount that balances risk and reciprocity.The authors conclude that agents’ amounts sent can be interpreted from their articulated BDI reasoning.Source: §3.2 · p. 5

3 Do LLM Agents Manifest Trust Behavior?

  • LLM agents generally exhibit trust behavior in Trust Games, based on positive transfers, valid responses, and decisions interpretable through BDI reasoning.The authors use both amount sent and articulated reasoning to assess trust, addressing the possibility that positive transfers could be random.Source: §3.1 · p. 4, §3 · p. 4, §3.2 · p. 5

3.3 Basic Analysis of Agent Trust

  • Vicuna-7b shows the highest trust level and GPT-3.5-turbo-0613 the lowest; most GPT-4 and Vicuna-7b personas send above the human average of $5.97.Most GPT-3.5-turbo-0613 personas send less than the human average; Llama2-70b and Llama2-13b distributions converge, while human and Vicuna-7b distributions diverge.Source: Figure 2 · p. 4, §3.3 · p. 5

4 Does Agent Trust Align with Human Trust?

The section assesses whether agent trust aligns with human trust across behavioral factors and repeated-game dynamics. GPT-4 shows high behavioral alignment with humans, while other LLM agents show relatively lower alignment.

4.1 Behavioral Alignment

  • Behavioral alignment compares agents and humans on factors affecting behavior and action dynamics, extending beyond value-focused alignment.The study examines reciprocity anticipation, risk perception, prosocial preference, and patterns across repeated turns.Source: §4.1 · pp. 5–6

4.2 Behavioral Factor 1: Reciprocity Anticipation

  • GPT-4 exhibits human-like reciprocity anticipation, while LLMs with fewer parameters such as Llama2-13b do not show awareness of reciprocity.The analysis compares average amounts sent in the Trust and Dictator Games and finds distinct GPT-4 BDI outputs across them.Source: §4.2 · p. 6, Figure 3 · p. 6

4.3 Behavioral Factor 2: Risk Perception

  • As trust becomes less risky, trust rates generally rise for GPT-4, GPT-3.5-turbo-0613, and text-davinci-003, with GPT-4 more aligned to humans.The curves plot the portion of trustors choosing trust against probability p; Vicuna-13b does not show the same trend.Source: §4.3 · pp. 6–7, Figure 4 · p. 6

4.4 Behavioral Factor 3: Prosocial Preference

  • LLM agents, especially GPT-4, show human-like prosocial preference in trust behavior, except Vicuna-13b.GPT-4 reasoning favors trust in social interactions and calculated risk in gambling.Source: §4.4 · p. 7, Figure 5 · p. 7

4.5 Behavioral Dynamics

  • Human repeated-game behavior typically shows returned amounts exceeding sent amounts, a stable return-to-sent ratio, and few fluctuations across turns.The authors use a fluctuation within 10% between successive turns as the criterion for a stable ratio.Source: §4.5 · pp. 7–8, Figure 6 · p. 8
  • GPT-4 agents exhibit high behavioral alignment with humans in Trust Games, whereas other LLM agents show relatively lower alignment.The authors describe the alignment as encompassing both actions and underlying reasoning processes.Source: §4.5 · p. 8

5 Probing Intrinsic Properties of Agent Trust

The section probes agent trust by comparing amounts sent across trustee demographics and identities, trust instructions, and reasoning strategies. It reports demographic bias, greater trust toward humans than agents, greater ease of undermining than enhancing trust, and possible effects of reasoning strategy.

5.1 Is Agent Trust Biased?

  • For most LLM agents, amounts sent to female trustees exceed those sent to male trustees; GPT-4’s difference is $0.55 vs. $ −0.21.The authors describe trustee gender as having a moderate impact and note that race-related bias results appear in Appendix F.Source: §5.1 · pp. 8–9

5.2 Agent Trust Towards Agents vs. Humans

  • Most LLM agents send more to human trustees than to agent trustees; Vicuna-33b sends $0.40 vs. $ −0.84, respectively.The authors characterize this as a relative preference for trusting humans over other agents.Source: §5.2 · p. 9

5.3 Can Agent Trust Be Manipulated?

  • Most LLM agents can follow instructions to decrease trust, but only a few follow both increase- and decrease-trust instructions.For text-davinci-003, the amount sent falls by $1.26 under the instruction not to trust the other player.Source: §5.3 · p. 9

5.4 Do Reasoning Strategies Impact Agent Trust?

  • Most LLM agents change the amount sent under zero-shot Chain-of-Thought reasoning, while GPT-4’s amount increases by only $0.02.The authors note that more research is needed to understand the relationship between reasoning strategies and agent behavior.Source: §5.4 · p. 9
  • Agent trust shows demographic biases, a relative preference for humans over other LLM agents, greater ease of undermining than enhancing trust, and possible influence from reasoning strategies.The authors summarize these as the section’s four properties of agent trust.Source: §5.4 · p. 10

6 Implications

The discussion connects trust behavior to human simulation, LLM-agent cooperation, human-agent collaboration, and agent safety. It highlights behavioral alignment and trust properties while noting implications and potential risks.

Implications for Human Simulation

  • GPT-4’s high behavioral alignment with human trust provides empirical evidence that LLM agents can simulate trust, a behavior central to human interactions and institutions.The authors describe this as a foundation for human simulations ranging from individual interactions to social networks and institutions.Source: §6 · p. 10

Implications for Agent Cooperation

  • Agent trust’s intrinsic properties may inform trust-dependent cooperation mechanisms and collective decision-making among LLM agents.The discussion identifies trust as a vital component of cooperation in multi-agent systems and human society.Source: §6 · p. 10

Implications for Human-Agent Collaboration

  • Agents’ relative preference for trusting humans over agents may inform human-agent collaboration, while demographic biases indicate potential collaboration risks.The discussion also notes that agent trust toward humans is under-explored and could recursively affect human trust.Source: §6 · p. 10

Implications for the Safety of LLM Agents

  • Understanding LLM-agent trust behavior may help maximize agents’ benefits and minimize potential risks to human society.The discussion frames this as relevant to ensuring agents cooperate, assist, and benefit rather than deceive, manipulate, or harm humans.Source: §6 · p. 10

7 Conclusion

The paper concludes that LLM agents exhibit trust behavior and that GPT-4’s trust is particularly aligned with human trust, supporting the feasibility of simulating human trust. It positions this contribution against prior work on LLM-based human simulation and game-theoretic behavior.

7 Conclusion

  • Trust Games reveal agent trust and behavioral alignment with human trust, particularly for GPT-4, supporting the feasibility of simulating human trust behavior with LLM agents.The authors say this opens a path to simulating interactions and institutions where trust is critical.Source: §7 · p. 10

A Related Work

  • Prior work has used LLM agents for individual- and society-level human simulation, but often without sufficient validation that they behave like humans.The paper introduces behavioral alignment and reports that LLMs, particularly GPT-4, can largely simulate human trust; it also distinguishes its focus on trust from prior game-theory studies.Source: A Related Work · p. 19

B Impact Statement

The impact statement presents LLM agents as promising human-simulation tools while pointing to trust-simulation evidence, role-playing applications, and limitations from model accuracy and demographic bias.

B Impact Statement

  • The paper’s trust findings provide empirical support for simulating more complex interactions and institutions where trust is essential.The impact statement says these findings pave the way for such simulations.Source: B Impact Statement · p. 19
  • LLM agents may help address conventional simulation’s utility-function limits and support hypotheses about social phenomena and policy effects.The paper frames these as potential applications of human simulation, not demonstrated outcomes.Source: B Impact Statement · p. 19
  • Role-playing simulations include interlocutors for conflict-resolution practice and virtual students for collaborative mathematical modeling practice.The paper gives these as examples of human-facing role-playing applications.Source: B Impact Statement · pp. 19–20
  • Some smaller-scale LLMs remain deficient at accurately simulating human trust, and gender and race biases pose potential risks in human behavior simulation.The authors identify improving behavioral alignment and mitigating these biases as areas for further research.Source: B Impact Statement · p. 20

C Limitations and Future Works

The study uses Trust Games, which simplify real-world scenarios, and calls for further study in more complex and dynamic environments. It also argues that understanding trust behavior requires attention to both actions and underlying reasoning processes.

C Limitations and Future Works

  • Trust Games simplify real-world scenarios, so the study calls for further research on agent trust in complex and dynamic environments.The authors also identify understanding both trust actions and their underlying reasoning as an interdisciplinary research need.Source: C Limitations and Future Works · p. 20
  • A fuller account of agent trust requires studying both behavior and the reasoning processes underlying it.The authors call for collective work across behavioral science, cognitive science, psychology, and sociology.Source: C Limitations and Future Works · p. 20
  • The paper identifies interdisciplinary collaboration as a need for understanding agent trust and its relationship to human trust.The named fields include behavioral science, cognitive science, psychology, and sociology.Source: C Limitations and Future Works · p. 20

D Additional Illustration for Experiments on Risk Perception

The appendix describes an experiment for estimating trust rates across probabilities and identifies a statistical test comparing amounts sent in Trust and Dictator Games.

D Additional Illustration for Experiments on Risk Perception

  • Trust rates are estimated across 10 probability conditions from 0.1 to 1.0 because LLM agents have difficulty directly stating their minimum acceptable probability of trust.At each probability, the study measures the share of trustor agents choosing to trust.Source: D Additional Illustration for… · p. 20

E Statistical Testing

  • Table 1 reports one-tailed independent-samples t-tests of changes in LLM agents’ amounts sent between the Trust Game and Dictator Game.The supplied caption identifies the comparison and test, but gives no test outcomes.Source: Table 1 · p. 21

F More Experiments on Probing Intrinsic Properties of Agent Trust

Figure 8 concerns changes in LLM agents’ average amounts sent when trustors are informed of trustees’ race attributes.

F More Experiments on Probing Intrinsic Properties

  • Figure 8 presents changes in average amounts sent when trustors learn trustees’ race attributes, reflecting demographic biases in LLM agents’ trust behavior.The caption does not specify the direction or size of the changes.Source: Figure 8 · p. 22

G The Complete Results for the Repeated Trust Game

The appendix provides repeated Trust Game results for humans and two LLM groups, alongside persona examples and prompts for game settings and trust-probing conditions.

G.1 Human

  • Figure 9 presents the human participants’ results in the Repeated Trust Game.The data were collected from Cochard et al. (2004) and redrawn by the authors.Source: Figure 9 · p. 23, §G.1 · p. 23

G.2 GPT-4

  • Figure 10 presents all GPT-4 agents’ results in the Repeated Trust Game.The caption does not report the results’ values or pattern.Source: Figure 10 · p. 24

G.3 GPT-3.5

  • Figure 11 presents all GPT-3.5 agents’ results in the Repeated Trust Game.The caption does not report the results’ values or pattern.Source: Figure 11 · p. 25

H.1 Persona Prompt

  • The persona-prompt examples describe varied fictional people, including profiles with different occupations, ages, genders, and backgrounds.Examples include a software engineer, chef, pediatrician, police officer, and fashion designer.Source: §H.1 · p. 26

H.2 Game Setting Prompt

  • The game-setting prompts cover Trust Game choices and repeated rounds, including prompts that give players preceding-round outcomes.The appendix includes prompts for trustors and trustees, plus lottery-style trust and gamble choices.Source: §H.2 · pp. 27–28

H.3 Prompts for Probing Intrinsic Properties

  • The probing prompts vary whether the counterpart is human or an LLM, identify race or gender, add step-by-step reasoning, or explicitly direct trust.The LLM-player prompt explicitly names the counterpart as an LLM.Source: §H.3 · pp. 29–30

I Belief-Desire-Intention (BDI) Analysis

This section covers BDI analysis of trust-game reasoning across LLMs and game conditions, alongside the paper’s note that it conducted no crowdsourcing or human-subject experiments and therefore marks participant-instructions and IRB checklist items not applicable.

I.1–I.4 Trust Game (Low Amount Sent vs. High Amount Sent)

I.5–I.9 Trust Game (Low Amount Sent vs. High Amount Sent)

I.10–I.12 Game Variants

I.13 the Repeated Trust Game

  • In a typical GPT-4 persona pair over seven rounds, both trustor amounts sent and trustee amounts returned increased while maintaining a consistent ratio.The analyzed pair was Group 10, selected from 16 randomly sampled persona pairs.Source: §I.13 · p. 43

I.14–I.17 Trust Game Conditions

NeurIPS Paper Checklist

  • The paper says it did not conduct crowdsourcing or research with human subjects, marking the participant-instructions and compensation item not applicable.The checklist asks for participant instructions, screenshots where applicable, and compensation details.Source: NeurIPS Paper Checklist · p. 56
  • The paper also marks the human-subject risks and IRB-approval checklist item not applicable because it did not conduct this kind of experiment.The checklist item concerns participant risks, disclosure of those risks, and IRB or equivalent review.Source: NeurIPS Paper Checklist · p. 56
Loading 2402.04559v4…