Source-linked AI summary
Playing repeated games with Large Language Models
Elif Akata, Lion Schulz, Julian Coda-Forno, Seong Joon Oh, Matthias Bethge, Eric Schulz
TL;DR
LLMs increasingly interact repeatedly with humans and other agents, motivating principled study of their social behavior. This paper uses repeated 2×2 games to examine cooperation and coordination, finding strong performance in self-interested games but suboptimal behavior in coordination games, with behavior modulated by additional opponent information and social chain-of-thought.
Problem
The paper asks how LLMs behave in repeated social interactions with humans and other agents, an urgent question given their rapid adoption and widespread interactions.
Method
The authors use behavioral game theory and have different LLMs play finitely repeated 2×2 games with one another, human-like strategies, and human players.
Results
LLMs perform well in self-interested games, especially the Prisoner’s Dilemma family, but behave suboptimally in coordination games such as the Battle of the Sexes; these signatures remain stable across robustness checks.
Takeaways & Limitations
GPT-4’s social behavior can be modulated by information about its opponent and social chain-of-thought, improving cooperation, scores, and coordination with human players.
Takeaways & Limitations
The investigation is constrained to simple 2×2 games and mainly finite games whose duration is known to the agents, while human experiments cover only the Battle of the Sexes and Prisoner’s Dilemma.
Abstract
from arXiv · showhide
LLMs are increasingly used in applications where they interact with humans and other agents. We propose to use behavioural game theory to study LLM's cooperation and coordination behaviour. We let different LLMs play finitely repeated $2\times2$ games with each other, with human-like strategies, and actual human players. Our results show that LLMs perform particularly well at self-interested games like the iterated Prisoner's Dilemma family. However, they behave sub-optimally in games that require coordination, like the Battle of the Sexes. We verify that these behavioural signatures are stable across robustness checks. We additionally show how GPT-4's behaviour can be modulated by providing additional information about its opponent and by using a "social chain-of-thought" (SCoT) strategy. This also leads to better scores and more successful coordination when interacting with human players. These results enrich our understanding of LLM's social behaviour and pave the way for a behavioural game theory for machines.
Introduction
The paper studies how LLMs behave in repeated social interactions using behavioural game theory and finitely repeated 2×2-games. Across game families, LLMs perform well in self-interested games but underperform when coordination is required.
- Motivation: Behavioural game theory provides a framework for studying cooperation and coordination in repeated interactions among diverse agents.It examines deviations from purely rational, selfish utility maximization through social preferences and psychological factors.
- Approach: The study lets LLMs play finitely repeated, fully informed 2×2-games against other LLMs, human-like strategies, and human players.The experiments focus on two players, each choosing between two discrete actions.
- Evaluation: The experiments evaluate LLM behaviour across multiple game families, including the Prisoner’s Dilemma family and coordination games such as Battle of the Sexes.The study analyzes cooperation, defection, and coordination across repeated interactions.
- Findings: LLMs perform well in games that reward pure self-interest, especially games from the Prisoner’s Dilemma family.This pattern is reported across the paper’s analysis of game families.
- Findings: LLMs underperform in games requiring coordination, including Battle of the Sexes.The paper identifies coordination as a distinct weakness relative to self-interested games.
Results
Across repeated 2×2 games, LLMs performed well in self-interested settings, especially Prisoner’s Dilemma games, but underperformed when success required coordination. GPT-4’s behavior was unforgiving in the Prisoner’s Dilemma, while social chain-of-thought improved coordination and human-player outcomes in the Battle of the Sexes.
- Analysing behaviour across families of games: Across 1224 repeated games, the study evaluated five LLMs across win-win, Prisoner’s Dilemma, unfair, cyclic, biased, and second-best game families.Each game was played for 10 rounds, with models facing every other model, including themselves, as either player.
- Analysing behaviour across families of games: GPT-4 performed best overall, outperforming Claude 2, davinci-003, davinci-002, and Llama 2.The reported comparisons were significant against all four models, with effect sizes ranging from d = 0.20 to d = 0.70.
- Analysing behaviour across families of games: LLMs performed well in win-win and Prisoner’s Dilemma games but worse when the best choice was not aligned with their own preferences.The authors focus on the Battle of the Sexes as a canonical convention-formation and coordination problem.
- Prisoner’s Dilemma: GPT-4 never resumed cooperation after an agent defected once and then cooperated thereafter, indicating repeated retaliation in the canonical Prisoner’s Dilemma.Its performance in these games generally stemmed from defecting, especially after observing one defection.
- Prisoner’s Dilemma: Informing GPT-4 that other players may make mistakes led it to cooperate again after defection, unlike its baseline unforgiving behavior.This manipulation was inspired by findings on forgiveness in human Prisoner’s Dilemma play.
- Battle of the Sexes: GPT-4 failed to coordinate with an alternating Battle-of-the-Sexes agent because it repeatedly selected its preferred option rather than adapting.This inability to alternate persisted across prompt, labeling, utility, and cover-story robustness checks.
- Battle of the Sexes: Social chain-of-thought prompting improved GPT-4’s behavior by having it predict the opponent’s move before choosing, leading it to alternate from round 5 onward.The approach was designed to reason about possible actions and outcomes before making a decision.
- Human experiments: Among human participants, SCoT increased Battle-of-the-Sexes scores and successful coordination, while it did not improve Prisoner’s-Dilemma scores.The Battle-of-the-Sexes score difference was β = 0.74, whereas the Prisoner’s-Dilemma difference was β = 0.10, p = 0.64.
Discussion
The study shows that LLMs can perform well in self-interested repeated games but behave suboptimally in coordination settings. GPT-4’s behaviour can be improved through prediction-based prompting, while the analysis remains bounded by simple finite 2×2 games and limited human experiments.
- Main findings: LLMs perform well on many game-theoretic tasks measured by individual reward, particularly when explicit coordination is unnecessary, but behave suboptimally in coordination games.These behavioural signatures remained stable across extensive robustness checks.
- Prisoner’s Dilemma: GPT-4 permanently defects after one negative interaction in the Prisoner’s Dilemma, even when the other agent later cooperates.The paper notes that continual defection is the equilibrium policy in the finitely repeated game, although it reduces joint payoff.
- Human experiments: In human experiments, SCoT prompting improved Battle of the Sexes scores and successful coordination, while Prisoner’s Dilemma score differences were small and non-significant.Battle of the Sexes average scores differed by β = 0.74, t(193) = 3.49, p < .001; Prisoner’s Dilemma scores differed by β = 0.10, t(193) = 0.47, p = 0.64.
- Intervention: Prediction-based prompting can alleviate behavioural flaws and oversight of simple strategies by explicitly encouraging social cognition or theory-of-mind reasoning.The paper suggests that social chain-of-thought prompting could improve human–LLM interaction similarly to chain-of-thought methods for non-social reasoning.
- Limitations: The study is limited to simple 2×2 games, mainly finite games with known durations, and human experiments covering only the Battle of the Sexes and Prisoner’s Dilemma.The authors identify indefinite games and additional human studies as important directions for further investigation.
Methods
The study evaluates LLM behavior in finitely repeated 2×2 games through prompt-based interactions, human-like strategies, and human participants. It uses controlled game descriptions, repeated histories, and statistical analyses to examine cooperation and coordination.
- Human study: The human-participant study recruited 195 Prolific participants and used comprehension checks, counterbalanced game order, and audience-adapted instructions.Participants played both games under content-wise consistent cover stories, while the presentation was adapted for human participants.
- Experimental design: Each game is presented as a prompt, and the interaction history is concatenated into subsequent prompts for in-context learning.Two LLMs make choices through prompt-chaining, with learning about prior rounds occurring within context.
- Repeated-game procedure: Interactions continue for 10 rounds, with payoffs represented as sums of stage-game rewards across repeated plays.The experiments set the discount factor δ = 1, so each stage payoff contributes without discounting.
- Game set: The study examines 144 2×2-games grouped into six families: Win-Win, Prisoner’s Dilemma Family, Unfair, Cyclic, Biased, and Second-Best.The families represent distinct payoff structures and strategic outcomes, including cooperation, unequal outcomes, cycling, and coordination.
- Focused games: Additional analyses focus on the Prisoner’s Dilemma and Battle of the Sexes, including matches between LLMs and simple hand-coded strategies.GPT-4 receives particular attention in these cooperation and coordination analyses.
- Experimental design: The experiments compare five LLMs across repeated 2×2 games, including interactions with other LLMs, human-like strategies, and human participants.The models include GPT-4, text-davinci-003, text-davinci-002, Claude 2, and Llama 2 70B.