Evaluating Language Models' Evaluations of Games

1 INTRODUCTION

• The paper frames reasoning as judging whether problems are worth pursuing, extending game-based AI assessment beyond models’ ability to play.Prior assessments have repeatedly used games, from chess and Go to newer games, to test reasoning flexibility.Source: §1 · p. 1
• Game evaluations pose distinct challenges: expected payoff is difficult to compute, while funness is difficult both to compute and to quantify.These queries allow evaluation of models on judgments made before anyone plays the game.Source: §1 · pp. 1–2
• The study compares model judgments on 121 novel games with people’s evaluations and explicit gameplay baselines, including game-theoretic optimal payoff where computable.It examines expected payoff and perceived funness as two evaluation queries.Source: §1 · p. 2
• Reasoning models are generally more aligned with human judgments of expected payoff and funness than non-reasoning language models, but closer alignment with game-theoretic optimality can coincide with weaker human fit.Funness evaluations also vary inconsistently across models, and reasoning models use highly variable amounts of reasoning tokens across queries.Source: §1 · pp. 2–3

2 FROM EVALUATING SOLUTIONS TO EVALUATING EVALUATION

• Game-solving is commonly assessed through an agent’s policy for selecting actions from states to optimize reward, while evaluation concerns properties of the game as a whole.A game is represented through feasible states, actions, transition rules, and goal functions mapping states to rewards.Source: §2 · p. 3
• A game-evaluation query may be decomposed into factors such as fairness, strategic demand, or expected duration, with computation varying across factors.The answers to subqueries must then be aggregated to address the original query.Source: §2 · p. 3
• Game evaluation uses less precise criteria than player evaluation and may not permit objective judgments, so evaluators can be compared with people or rational benchmarks and assessed by resource cost.Resource cost may be measured in wall-clock time, simulations, or reasoning tokens.Source: §2 · p. 3

3.1 EVALUATIONS OVER NOVEL GAMES

• The study uses 121 two-player competitive grid strategy games, mostly novel variants, and gathers novice human judgments on expected value and funness from over 450 participants.Approximately 20 people evaluated each game per query before playing.Source: §3.1 · pp. 3–4

3.2 ELICITING MODEL GAME EVALUATIONS

• Language and reasoning models evaluate expected payoff and funness with 20 rollouts, alongside gameplay agents ranging from random and heuristic play to tree-search and MCTS methods.Main-text reasoning-model results use medium reasoning effort.Source: §3.2 · p. 4

3.3 EVALUATION MEASURES

• The primary measure is R2 between average model and human judgments across all 121 games, supplemented by payoff comparisons with estimated game-theoretic optima on 78 games.The study also measures accuracy, distance from optimal payoff, and similarity among models and agent classes.Source: §3.3 · p. 4

4.1 EVALUATING EXPECTED PAYOFF (FAIRNESS) OF GAMES

• Reasoning models generally aligned better with human payoff judgments, but in the OpenAI family, fit to humans declined from o3 to GPT-5 as fit to game-theoretic judgments improved.Non-reasoning models were more similar to one another than to people or tree-search models.Source: §4.1 · p. 4, Figure 2 · p. 5
• More advanced reasoning models resembled search-based game reasoners, but explicit simulation appeared in only some DeepSeek R1 games, and per-game judgments still differed from people’s.The authors suggest some models may simulate gameplay; the inspected traces showed simulation less often than analogy-based or mathematical reasoning.Source: §4.1 · p. 5

4.2 EVALUATING GAME FUNNESS

• Without intermediate reasoning, models poorly matched human funness judgments; reasoning captured more human variance, but fits across model sophistication were jagged.Participants and models were free to define funness, reflecting the query’s difficulty to quantify and compute.Source: §4.2 · pp. 6–7, Figure 3 · p. 7
• Models differed in how often they considered balance and game length when judging funness, alongside commonly considered challengingness and strategic richness.The authors suggest differences in factor computation or aggregation may contribute to disparities in scalar funness judgments.Source: Table 2 · p. 6, §4.2 · p. 7

4.3 RESOURCE USAGE WHEN EVALUATING GAMES

• Models generally used fewer reasoning tokens for funness than payoff despite funness’s ambiguity; token use showed no measurable relation to game novelty or prediction distance.Novelty was measured by the number of game features differing from Tic-Tac-Toe.Source: §4.3 · p. 8, Figure 4 · p. 8

• Cognitive-science work studies how people select problems and strategies under limited cognitive resources, including deciding whether to engage with a task or quit.Resource-rational analysis has served as a framework for computational models of problem selection.Source: §5 · pp. 8–9
• Language-model reasoning research spans many domains but has predominantly evaluated problem solving rather than problem evaluation.Common comparisons assess reasoning models or prompted chain-of-thought against non-reasoning baselines.Source: §5 · p. 9
• The paper follows psychological studies that replace participants with language models and compare their behavior with people and rational cognitive models.This research also informs the design and improvement of LLM-based agents.Source: §5 · p. 9

• Language can bring associative knowledge to evaluating new systems, but intuition without deliberative, iterative thinking has limits; simulation or explicit reasoning seems essential for aligning with human judgments and computing optimal game-theoretic values.Source: §6 · p. 9
• Open questions include whose judgments to target, how to budget evaluation compute, what fairness biases models learn, whether they simulate internally, and which other evaluation queries to study.Source: §6 · pp. 9–10
• The findings’ generalization to other games, settings, and queries remains open, and assessments are sensitive to prompts and hyperparameters; the study focuses on novice game reasoners.How well models relate to people with different skills, cultures, and experience also remains an open question.Source: §6 · p. 10
• Evaluating AI evaluations may inform human-beneficial thought partners for deciding what problems to solve and assessing fairness in systems people use or follow.The discussion gives automated mechanism design as an example of where systems should be evaluated for fairness and engagement.Source: §6 · p. 10

• The paper studies two-player competitive strategy board games, assessing language and reasoning models on payoff and funness queries spanning difficulty to compute and difficulty to quantify.Source: §7 · p. 10
• Its perspective treats evaluating whether problems are worth solving as part of reasoning, alongside solving them.Source: §7 · p. 10
• The work raises questions about designing human-beneficial, resource-efficient machine reasoners and evaluating their judgments about new problems.Source: §7 · p. 10

• The paper frames alignment here as whether AI matches human judgments of game fairness and funness.Source: ETHICS STATEMENT · p. 10
• Models that better anticipate what people find engaging could be used to optimize highly engaging games, which could cross into harmful addictiveness.Source: ETHICS STATEMENT · p. 10
• The statement calls for careful consideration of how human-aligned engagement models are used in practice.Source: ETHICS STATEMENT · p. 10

• The listed appendix sections cover additional game-theoretic payoff estimates and comparisons with human payoff evaluations.Source: APPENDIX · p. 17

A1 ADDITIONAL RELATED WORK

• Games combine precise, encodable rules and rewards with demands such as long-range planning, semantic understanding, and social inference.These properties have made games useful environments for evaluating AI systems and studying language-model decision making.Source: A1 ADDITIONAL RELATED WORK · p. 17
• The paper studies how language models evaluate games before play and compares those evaluations with human judgments, using variants unlikely to appear in existing training corpora.This focus differs from research on language-model play in games and aligns with work on novel or generated games.Source: A1 ADDITIONAL RELATED WORK · pp. 17–18
• Unlike LLM-as-a-judge work aimed at scalable assessments, this study uses game judgments to examine models’ reasoning about queries that are difficult to compute or quantify.The related work describes judge systems producing scores, yes/no answers, and pairwise comparisons used to improve models, data, agents, and reasoning.Source: A1 ADDITIONAL RELATED WORK · p. 18

A2 EXAMPLE GAMES

• Table 3 groups 121 games by board shape and rules, with example games for each category.Source: Table 3 · p. 18

A3.1 PROMPTS AND ADDITIONAL LANGUAGE MODEL GENERATION DETAILS

• For payoff, models separately estimated P(P1 wins|not draw) and P(ends in a draw), which were combined as P(P1 wins) = P(P1 wins|not draw)×(1−P(ends in a draw)).Models were asked to provide the estimates simultaneously; future work could elicit payoff directly in one query.Source: A3.1 PROMPTS AND ADDITIONAL LANGUAGE… · p. 18
• Models were also asked to estimate game funness relative to the broader class of games.Source: A3.1 PROMPTS AND ADDITIONAL LANGUAGE… · p. 18
• Non-thinking models were prompted either for a direct numeric answer or to use chain-of-thought, and direct-answer runs producing a natural-language rationale first were filtered out.Source: A3.1 PROMPTS AND ADDITIONAL LANGUAGE… · p. 19
• Thinking models used chain-of-thought prompts, while DeepSeek-R1 received prompt and token-limit adjustments, and runs exceeding the limit were filtered out.The R1 maximum was set to 32,000 tokens after longer-than-default responses were observed.Source: A3.1 PROMPTS AND ADDITIONAL LANGUAGE… · p. 19

A3.2 SYSTEM PROMPT

• Payoff prompts ask, assuming both players play reasonably, how likely the first player wins if the game does not end in a draw and how likely a draw is; funness prompts rate grid-based games from 0 to 100, with fun defined freely.Source: A3.2 SYSTEM PROMPT · p. 19

A3.3 TASK PROMPT

• The task prompts use scratchpad language for the CoT variant but remove it for Direct, and filter Direct runs that include chain-of-thought before the answer for three models.Payoff questions use separate 0–100 scales for first-player win likelihood and draw likelihood; funness is rated from least to most fun in the class of grid-based games.Source: A3.3 TASK PROMPT · p. 20

A3.4 ALTERNATE MODELS

• Payoff comparisons include the Intuitive Gamer and Expert Gamer cognitive models, Monte Carlo Tree Search, and random agents; funness comparisons exclude these alternatives because the models are regressions fit to some human data.The Intuitive Gamer models fast, depth-limited probabilistic reasoning, while the Expert Gamer uses deeper tree search.Source: A3.4 ALTERNATE MODELS · pp. 20–21

A4 ADDITIONAL ANALYSIS DETAILS

• Supplementary rank correlations are less discriminative across models, though broad model-family trends persist; category-level payoff analyses also find per-game variability despite o3’s stronger aggregate alignment.The reported variation includes games with infinite or rectangular boards; supplementary funness comparisons examine absolute deviation by game category.Source: A4 ADDITIONAL ANALYSIS DETAILS · p. 21, Figure 5 · p. 21, A4.2 ADDITIONAL COMPARISONS TO HUMAN… · p. 22

A4.1 GAME-THEORETIC PAYOFF ESTIMATES AND ADDITIONAL ANALYSES

• Game-theoretic payoff estimates cover 78 of 121 games, using mathematical optima where possible and MCTS outcomes otherwise when 50 trials converge to the same Player 1 outcome.MCTS used 1,000 iterations per agent; each 50-match trial averaged approximately 4.2k seconds and over 7 million nodes.Source: A4.1 GAME-THEORETIC PAYOFF ESTIMATES… · p. 21

A5 ANALYZING REASONING TOKEN USAGE

• Reasoning-token usage was not well correlated with human judgments, model judgments, or deviation from game-theoretic optimality; traces sometimes showed plausible strategies but also implausible claims or conclusions.Examples of strategies included analogy to familiar games and consideration of first-mover advantage; errors included misestimating Player 1 win rate or underestimating funness.Source: A5 ANALYZING REASONING TOKEN USAGE · p. 23

A5.1 VARYING REASONING AMOUNT

• Changing o3 and GPT-5 reasoning amounts minimally affected aggregate fit to human data but changed proximity to game-theoretic optimal predictions; models could agree on simple games yet still miss human judgments.Models were generally consistent across reasoning amounts on some simpler games, while games with the largest differences generally changed monotonically with reasoning amount.Source: A5.1 VARYING REASONING AMOUNT · pp. 23, 27, Figure 17 · p. 27

A6 ADDITIONAL DETAILS ON REASONING TRACE CODING

• Reasoning-trace coding used o3 to classify explicit simulation, analogy, and mathematical computation, but the authors caution that the codes may miss unexpressed methods and that the analysis is approximate.The coding queried each rollout and game independently; two authors manually inspected some traces, and verification was still being expanded.Source: A6 ADDITIONAL DETAILS ON REASONING… · p. 28, A6.1 CODING METHODS EXPLICITLY ENGAGED… · p. 28

A6.1 CODING METHODS EXPLICITLY ENGAGED IN REASONING MODELS’ TRACES

• Trace-coding prompts distinguish explicit mathematical calculations from explicit game simulation, with calculations requiring precise operations and simulations requiring turn-taking and move selection.The prompts accept incorrect calculations or simulations, but exclude mere play simulation from the mathematics category.Source: A6.1 CODING METHODS EXPLICITLY ENGAGED… · p. 29

A6.1.1 EXAMPLE REASONING TRACES ACROSS EVALUATIVE METHODS

• In DeepSeek-R1’s analogical-reasoning trace, it estimates the first player’s win chance at 10% and draw probability at 90%, reasoning that decisive wins are unlikely and draws dominate.Source: A6.1.1 EXAMPLE REASONING TRACES ACROSS… · p. 29
• In a simulated-play example, DeepSeek-R1 reasons through a 2x10, three-in-a-row game and ultimately gives a 100 draw estimate and a conditional 50 estimate for the first player.The trace sketches alternating moves and a filled-board draw, while also noting that the conditional first-player estimate is moot if perfect play always draws.Source: A6.1.1 EXAMPLE REASONING TRACES ACROSS… · pp. 29–30
• A mathematical-reasoning example analyzes a 5x10, six-in-a-row game, argues only horizontal wins are possible, and estimates a 70–80% draw probability.The trace later settles on an 80% draw estimate and a 60% first-player win chance conditional on a non-draw, yielding a stated 12% overall first-player win rate.Source: A6.1.1 EXAMPLE REASONING TRACES ACROSS… · pp. 31–32

A6.2 CODING FUNNESS MEASURES

• Funness traces were coded for mentions of balance, strategic richness, challenge, game length, and novelty; coding was applied per rollout, with o3 also used on selected non-reasoning-model rationales.The passage says each funness query was asked independently for each rollout for each game.Source: A6.2 CODING FUNNESS MEASURES · p. 32

A6.3 ADDITIONAL EXAMPLE REASONING TRACES

• An additional DeepSeek-R1 example labels its fairness judgment unreasonable and estimates 5 for the first-player win chance and 95 for the draw chance in a 10x10, seven-in-a-row game.Its rationale invokes defensive blocking and draw dominance, including claims about comparable games; these are presented as the trace’s reasoning, not as verified results.Source: A6.3 ADDITIONAL EXAMPLE REASONING TRACES · p. 33
• The same example’s rationale claims the second player can equalize quickly, reducing the first player’s theoretical edge to near zero.The provided text also labels a second example as “Underestimate fun,” but supplies only its heading here.Source: A6.3 ADDITIONAL EXAMPLE REASONING TRACES · pp. 33–34
• The traces characterize the 4x9, four-in-a-row game as offering horizontal strategy but limited vertical depth, with quick vertical wins potentially favoring the first player.The recap also asserts a roughly 75% first-player win rate and average game length of 8–12 moves.Source: A6.3 ADDITIONAL EXAMPLE REASONING TRACES · pp. 34–35
• The 4x9 trace gives a fun rating of 45, then its recap gives 38/100, citing low height and concerns about first-player advantage and replayability.It describes the wide board as offering some strategic depth but judges that this does not offset the constraints on vertical play.Source: A6.3 ADDITIONAL EXAMPLE REASONING TRACES · p. 35
• In the 10x10 trace, the first player can win only horizontally or vertically, while the second player can also win diagonally; the recap rates the game 30/100.The trace characterizes the asymmetric rules as imbalanced and likely to reduce fairness and enjoyment.Source: A6.3 ADDITIONAL EXAMPLE REASONING TRACES · pp. 35–37

A7.1 ASSUMPTION OF “REASONABLENESS”

• When prompted to assume optimal play, all tested models fit human payoff predictions less well; o3’s estimates approach game-theoretic optimal values comparably to GPT-5’s.Neither o1 nor GPT-5 substantially improves its closeness to optimal values, while o1 struggles to estimate them and GPT-5 struggles to produce human-like, suboptimal responses.Source: A7.1 ASSUMPTION OF “REASONABLENESS” · p. 37, Figure 18 · p. 38

A7.2 QUESTION ORDER

• For o3, reversing the order of draw-probability and first-player-win questions produces little difference in payoff predictions.The analysis cautions that this does not establish order-robustness for people or other language models.Source: A7.2 QUESTION ORDER · pp. 38–39, Figure 19 · p. 38

A8 PRELIMINARY EXPLORATIONS INTO AN EXPANDED GAME SET

• The preliminary set added 16 variants each for Reversi, Hex, Yavalath, and cooperative tic-tac-toe to explore four game categories.The cooperative game had both players win if they made their respective patterns, or both lose otherwise.Source: A8 PRELIMINARY EXPLORATIONS INTO AN… · p. 39
• Competitive variants changed board size and turn dynamics, while cooperative variants changed board size and winning patterns.One competitive example let Player 2 play twice on the first turn; one cooperative example used a 4 × 5 board with different four-in-a-row goals.Source: A8 PRELIMINARY EXPLORATIONS INTO AN… · p. 39
• The study tested LLaMA 3.1 70B, DeepSeek v3, GPT-4, o1, o3, and GPT-5 on expected-payoff predictions under assumed reasonable play.The non-reasoning models used chain-of-thought, and prediction differences were measured by mean absolute deviation.Source: A8 PRELIMINARY EXPLORATIONS INTO AN… · p. 39
• Cooperative variants showed stark differences in predicted payoffs between reasoning and non-reasoning models, while competitive variants showed more variable and less stark differences.Reasoning models generally made similar predictions to one another on competitive variants.Source: A8 PRELIMINARY EXPLORATIONS INTO AN… · p. 39
• The authors caution that the observed differences across expanded variants are preliminary.They are exploring non-language-model reasoners such as MCTS and suggest gathering human judgments on novel variants, potentially prioritizing games with the biggest model differences.Source: A8 PRELIMINARY EXPLORATIONS INTO AN… · p. 39