Are ChatGPT and GPT-4 Good Poker Players? -- A Pre-Flop Analysis

1 Introduction

• Evaluating LLMs in games remains understudied, and poker tests mathematical analysis, reasoning, strategy, and understanding of human behavior.The paper presents itself as the first evaluation of LLMs playing incomplete-information games.Source: §1 · pp. 1–2
• Poker decisions use private cards, possible future outcomes, and clues about opponents’ hands, making poker an incomplete-information game.The paper notes that profitable strategies are evaluated in expectation over large samples, not from isolated decisions.Source: §1 · pp. 1–2
• The study evaluates ChatGPT and GPT-4 making pre-flop decisions in a 9-player Texas No-limit Hold’em game when first to act.The pre-flop decision depends on private cards, position, and any previous bets; Figure 1 names the table positions and action order.Source: Figure 1 · p. 1, §1 · p. 2
• Both models understand poker concepts, but ChatGPT tends to fold most hands and play premium hands while GPT-4 raises more hands than optimal.The authors characterize ChatGPT as conservative and GPT-4 as overaggressive; neither is game theory optimal.Source: §1 · p. 2

2 A Crash Course in No-Limit Hold’em Poker

• The pre-flop round follows the private-card deal, and voluntary action begins UTG before proceeding through the named table positions.In the 9-player game, the positions include UTG+1, UTG+2, Lojack, Hijack, Cutoff, and Button.Source: §2 · p. 3
• Raise-first-in (RFI) is the first pre-flop decision point when a player acts first or everyone before them has folded.The available actions are calling or limping, raising, and folding.Source: §2 · p. 3

2.1 Poker Charts

• Poker charts encode starting-hand choices by position; their 13-by-13 layout compresses 1,326 possible hands using suited, offsuit, and pair cells.The charts guide pre-flop choices such as folding, limping, or raising.Source: §2.1 · p. 3

2.2 GTO Pre-flop Strategy

• GTO RFI play folds approximately 90% of hands UTG and 50% on the Button, with no limping recommended except from the small blind.The resulting general pattern is to play a narrower range early and a wider range later, usually by raising or folding.Source: §2.2 · p. 3
• In standard cash games, the big blind also names the minimum bet, and the usual GTO raise size is 3 BB.The same term therefore refers either to a table position or, depending on context, a bet amount.Source: §2.2 · p. 3

3 ChatGPT Playing Poker

• The section examines ChatGPT’s RFI decisions by testing prompt variations and analyzing the resulting pre-flop charts.Source: §3 · p. 4

3.1 Basic System Prompt

• The selected system prompt describes a nine-player Texas no-limit RFI scenario, specifies fold, raise, or limp, and requires a fixed decision-and-bet-size format.Source: §3.1 · pp. 4–5

3.2 Analysing ChatGPT’s Decision Matrix

• Across 1,352 hand-position queries, six temperature/top-p settings and ten repetitions per hand, temperature 0.2 and top-p 0.95 produced the most robust results closest to GTO.Source: §3.2 · pp. 5–6
• ChatGPT’s baseline charts raise pocket Aces, fold 27 offsuit, and show fewer folds from later positions, patterns the authors read as poker understanding.Source: §3.2 · p. 6

3.1.1 User Prompts

• Short, poker-standard hand notation and ranking the higher card first produced more GTO-like decisions; unranked input caused erratic play, including folding premium hands such as AQs.Source: §3.1.1 · p. 5, §3.2 · p. 6

3.3 GTO System Prompt

• The GTO system prompt adds an explicit request for game-theory-optimal decisions while retaining the chosen short, ranked hand format and sampling settings.Source: §3.3 · pp. 6–7

3.3.1 ChatGPT GTO Decision Matrix Analysis

• When prompted for GTO play, ChatGPT nearly halves its limping range and raises more, yet still raises unusual hands and plays too few hands, especially from the button.The authors characterize its tight, conservative style as a Nit.Source: §3.3.1 · p. 7

4 GPT-4 Playing Poker

• The charts use the majority decision from five GPT-4 queries per hand, with temperature 0.2 and top-p 0.95.The experiment uses basic and GTO system prompts and provides hands in ranked order through the short user prompt.Source: §4 · p. 7
• GPT-4 never limps, and its basic- and GTO-prompt decision charts are similar except in later positions.Even with the basic prompt, GPT-4 raises more hands and plays more hands than GTO-prompted ChatGPT; the GTO prompt makes GPT-4 more aggressive still.Source: §4 · pp. 7–8
• From the Lojack onward, GPT-4 raises more hands than GTO calls for, reaching 90% of hands from the Button under the GTO prompt.The paper describes this as maniac-like aggression and concludes that GPT-4 is not GTO.Source: §4 · p. 8

5 Conclusion

• Both models understand starting-hand strength and position, but neither plays game theory optimal poker.They play differently according to position, yet their poker knowledge does not make their decisions GTO.Source: §5 · p. 8
• A GTO prompt makes ChatGPT tighter and more aggressive, while GPT-4 raises even more hands, mainly from later positions.The paper suggests the models may be unaware of their own playing tendencies.Source: §5 · p. 8
• ChatGPT’s tight, conservative play and GPT-4’s excessive raising are opposite departures from GTO.ChatGPT also limps weaker hands, while GPT-4 does not limp and raises the hands it plays.Source: §5 · p. 8

• The study evaluates only raise-first-in decisions at poker’s first decision point, which the authors describe as easiest to assess computationally and conceptually.Source: §6 · p. 9
• Because pre-flop is the simplest stage for GTO play, the authors expect deviations to worsen at later decision points as errors propagate.Source: §6 · p. 9

Pre-Flop

• Pre-flop betting follows the deal of each player’s two private cards.Source: §A.1 · p. 10

Post-Flop

• After pre-flop betting ends, three community cards are dealt face-up, beginning the post-flop betting round.Source: §A.1 · p. 10

Post-Turn

• The post-turn round follows a fourth face-up community card, called the turn card.Source: §A.1 · p. 10

Post-River

• The river is the fifth and final face-up community card, followed by the final betting round.Source: §A.1 · p. 10

• In a nine-player game, the small and big blinds post chips before seeing their cards, and UTG is first to act voluntarily.Source: §A.1.1 · p. 10
• Raise-first-in occurs when a player is first to put chips in the pot, either from UTG or after earlier players fold.Source: §A.1.1 · p. 10
• A bet is the first voluntary wager in a round with no prior wager, as when UTG acts pre-flop.Source: §A.1.1 · p. 10
• Starting-hand charts distinguish suited cards from unsuited cards, with pocket pairs on the diagonal.Source: §A.1.1 · p. 11
• RFI choices are limp, raise, or fold; a limp matches the big blind, while a raise increases the wager and a fold exits the hand.Source: §A.1.1 · pp. 10–11

• Starting-hand charts suggest fold, limp, or raise by hand and position, and earlier positions generally play fewer hands.Source: §A.1.2 · p. 11
• The charts compress 1,326 possible two-card starting hands into a 13-by-13 square, distinguishing suited and unsuited combinations.Source: §A.1.2 · p. 11
• Figure 9 maps ChatGPT’s raised, limped, and folded hands under a verbose prompt; Figure 10 shows the same actions for a short prompt with unranked card order.Source: Figure 9 · p. 12, Figure 10 · p. 12