Source-linked AI summary
Correlated Quantal Response Equilibrium: A New Equilibrium Notion for Human-AI Strategic Interactions
Zehao Zhao, Rahul Jain, Vijay Subramanian
TL;DR
Classical equilibrium notions have struggled to predict human strategic play, while AI decision-makers broaden the question to mixed human–AI interaction. CQRE relaxes correlated-equilibrium obedience and selects a maximum-entropy joint-play prediction; empirically, it fits better than Logit-QRE across the reported settings.
Problem
The paper asks how to model and predict outcomes in strategic interactions involving human and AI agents, against the background that classical equilibrium concepts fall short of accurately predicting human play.
Source: §1 · p. 1Method
Results
CQRE fits better than Logit-QRE on twelve human-play games, has better aggregate fit on AI–AI targets, wins six of seven larger-action games, and is favored in all 56 three-player comparisons.
Source: §1 · p. 2Takeaways & Limitations
Unlike product-form Logit-QRE, CQRE can represent cross-player dependence in joint action distributions.
Source: §1 · p. 2, §2 · p. 3The finite exponential representation assumes full support; when the prediction lies on the simplex boundary, the representation applies on its support.
Source: §2 · p. 5
Abstract
from arXiv · showhide
The modeling of the behavior of agents in strategic environments and the prediction of the resulting outcomes has long been a challenging problem; this has been the focus of game theory. The introduction of AI agents as human-aids or as sole decision-makers expands the underlying question: Can we model behavior and predict outcomes of strategic games that involve a mix of AI agents and human players? As an initial answer, we propose Correlated Quantal Response Equilibrium (CQRE) that combines and generalizes classical solution concepts such as Quantal Response Equilibrium and Correlated Equilibrium. We answer questions of existence and propose a method for computing the CQRE that is a best fit to a given joint action distribution. We demonstrate the empirical effectiveness of CQRE predictions using a diverse collection of complex general-sum games, including high-rank and large-action two-player games, challenging multiplayer games, and a large-scale extension of the Selten--Chmura setting to AI--AI and human--AI play.
1 INTRODUCTION
The paper asks how to predict strategic outcomes across human and AI play, where existing equilibrium concepts do not jointly capture dependence between players and boundedly rational choices. It introduces CQRE and evaluates its fit to observed play across several game settings.
1 INTRODUCTION
- The work frames CQRE as a shared equilibrium framework for strategic interactions involving human players and AI agents.Prior work on bounded rationality developed QRE and variants as empirically better predictors of human behavior than classical concepts.Source: §1 · p. 1
- Nash equilibrium imposes exact best responses and independent randomization, QRE softens responses but retains independence, and correlated equilibrium permits dependence but requires exact obedience.These contrasts motivate a point prediction that represents dependence while allowing heterogeneous deviations from exact obedience.Source: §1 · p. 1
- CQRE relaxes correlated-equilibrium obedience and selects the maximum-entropy joint distribution for each fixed slack vector, yielding a unique prediction and establishing existence.The slack can vary by player and action, by player, or be common across players and actions.Source: §1 · p. 2
- CQRE achieves better empirical fit than Logit-QRE on twelve human-play games, AI–AI targets in aggregate, six of seven larger-action games, and all 56 three-player comparisons, but not constructed product-form human–AI games.The evaluation also contributes 201,600 accepted one-shot decisions from six GPT-5.x deployments.Source: §1 · p. 2
- The paper proposes a computational method for fitting CQRE to observed play and reports that it provides better fit than extant notions across a diverse collection of games.Its contributions also include a stable, efficient implementation of the best-fit optimization problem.Source: §1 · pp. 1–2
2 CORRELATED QUANTAL RESPONSE EQUILIBRIUM
CQRE predicts joint play by maximizing entropy subject to obedience constraints relaxed by player-and-action slacks. The section establishes existence and uniqueness for fixed slacks, relates CQRE to CE and Logit-QRE, and describes slack parameterizations and limiting cases.
Figure 1
- Figure 1 reports mean ∆ values of −.0906 for β ∈[0, 20] and −.0821 for β ∈[0, ∞] across 49 ordered SC7 targets.The paired 95% intervals are [−.0949, −.0848] and [−.0863, −.0764], respectively.Source: Figure 1 · p. 3
Slack parameterizations
- Scalar, per-player, and per-player-and-action slack parameterizations are nested, with each broader specification able to reproduce the narrower ones.CQRE-pa is the most flexible specification; CQRE-c and CQRE-pp serve as restricted benchmarks.Source: §2 · pp. 3–4
Correlated QRE
- CQRE relaxes CE's exact obedience while retaining joint distributions, whereas Logit-QRE uses independent, product-form randomization.At zero slack, CQRE's feasible set is the CE set and its prediction is that set's unique maximum-entropy element.Source: §2 · pp. 3–5
- For finite N-player games, CQRE selects the unique maximum-entropy joint distribution satisfying obedience constraints relaxed by nonnegative slacks.Each slack bounds the recommendation-weighted deviation gain; the maximum-entropy criterion selects the least concentrated feasible distribution.Source: §2 · p. 4
Limiting cases
- When every slack reaches its maximum possible deviation gain, all obedience constraints become redundant and CQRE is uniform over joint profiles.As slacks increase componentwise, the feasible set expands and maximum attainable entropy is non-decreasing.Source: §2 · p. 4
Comparison to Logit-QRE beyond the analogy
- CQRE and Logit-QRE both have exponential forms, but CQRE's fitted slacks and constraint multipliers do not directly correspond to Logit-QRE's payoff-sensitivity parameter.CQRE multipliers indicate which relaxed obedience constraints influence the maximum-entropy solution.Source: §2 · pp. 4–5
3 DETERMINING BEST-FIT CQRE
Best-fit CQRE-pa estimates slacks from an observed joint-action distribution through a bi-level optimization. For each candidate slack vector, an inner maximum-entropy problem produces a unique prediction, and an outer problem selects the fit.
Outer optimization
- For each candidate slack vector, the inner problem computes a unique maximum-entropy prediction and the outer problem selects the slack minimizing fit loss to the target distribution.The fitted prediction is the inner solution at the selected slack vector, not the result of jointly optimizing the distribution and slacks.Source: §3 · p. 5
Payoff rescaling
- Payoffs are rescaled so unilateral payoff differences have magnitude at most one, matching the utility units in which obedience slacks are measured.When the game's payoff scale is zero, no rescaling is needed.Source: §3 · p. 5
Slack domain
- Per-action slacks are bounded by the largest possible deviation gains, since larger values are redundant and can be clipped without changing predictions.This restriction gives the optimization a compact slack domain.Source: §3 · p. 5
Inner maximum-entropy problem
- The inner problem uses probability variables over joint profiles and linear obedience inequalities, with Theorem 1 guaranteeing a unique optimizer.The stated dimensions are |A| probability variables and dD = Σ_i |Ai|(|Ai| −1) obedience inequalities.Source: §3 · p. 6
Outer optimization
- Because changing slacks can change which constraints bind, the outer loss is not guaranteed convex, so the method combines a grid search with multistart box-constrained L-BFGS-B.The best grid solution is retained as an incumbent, preventing continuous optimization from returning a worse fit up to numerical tolerance.Source: §3 · p. 6
Other equilibrium benchmarks
- The benchmark set includes restricted CQRE variants, maximum-entropy CE, mixed Nash, and Logit-QRE, with mixed Nash selecting the lowest-KL equilibrium when multiple equilibria exist.Common and per-player CQRE use the same continuous bi-level procedure and are ablations of per-player-and-action slack.Source: §3 · p. 6
4 EMPIRICAL RESULTS
The evaluation fits CQRE-pa and equilibrium benchmarks to four panels spanning human and LLM behavior, two- and three-player games, and varied action spaces and payoff structures. Across these settings, CQRE-pa’s relative fit varies by population and game, with strong results on established human play, larger-action games, and three-player games.
4.1 Established Human Behavior
- CQRE-pa’s mean KL is 0.0040 versus 0.0093 for Logit-QRE on the twelve established human games, a 56.9% reduction.It has the lowest point KL in nine games, and the aggregate paired interval for the difference is strictly negative.Source: §4.1 · p. 7
4.2 AI vs AI and Human vs AI extension
- Across 588 Selten–Chmura targets, CQRE-pa’s target-weighted mean KL is 1.36% lower overall, but Logit-QRE’s mean KL is 11.1% lower on constructed HAI targets.CQRE-pa reduces mean KL by 56.9% on HH and 3.0% on AAI targets; all four population-and-orientation aggregates have negative mean differences.Source: §4.2 · p. 7
4.3 Larger-Action Games
- CQRE-pa is favored in 111 of 147 larger-action comparisons (75.5%) and has lower mean KL in six of seven games.It sweeps all 21 pairings in sm n5, sm n7, and rank1 n5; selected Figure 2 comparisons also show lower aggregate KL on rank4 n5.Source: §4.3 · p. 8, Figure 2 · p. 8
- CQRE-pa wins all 12 GPT-5-family comparisons across sm n5, sm n7, rank1 n5, and rank4 n5.The broader larger-action results are not uniform across targets: 24 comparisons favor Logit-QRE and 12 are unresolved.Source: §4.3 · p. 8
- Across eight strongly-monotone games with n = 3 to 10, the fitted slack vectors’ r90 ranges from 2 to 7, or 22–50% of the nominal 2n-dimensional slack space.r90 is the smallest number of singular directions explaining at least 90% of centered variation.Source: §4.3 · p. 8
4.4 Three-Player Games
- On the three-player game, CQRE-pa reduces mean KL by 21.6% relative to Logit-QRE and wins all 56 model-triplet comparisons.Every target-specific paired interval lies below zero.Source: §4.4 · p. 8
5 CONCLUSIONS
The paper introduces CQRE as a maximum-entropy model of joint play with per-player and per-action relaxation of correlated-equilibrium obedience, proves a unique prediction for fixed slacks, and estimates slacks from observed play. It reports lower mean KL than Logit-QRE across several evaluated settings and contributes a reproducible dataset of 201,600 one-shot AI decisions.
5 CONCLUSIONS
- CQRE yields a unique prediction for every fixed slack vector, and a continuous bilevel procedure estimates the slacks from observed play.Empirically, CQRE has lower mean KL than Logit-QRE on established human and AI–AI play, in six of seven larger-action games, and in all three-player comparisons.Source: §5 · p. 8
AI use statement
- The authors used generative AI for language editing, formatting, figures, and generating one-shot decisions for the AI datasets, while reviewing text and cross-checking data and plots.They state that AI was not used for final decisions about theoretical claims, empirical conclusions, or interpretation.Source: AI use statement · p. 9
CHECKLIST
The checklist reports documentation for theoretical claims, algorithms, empirical results, and assets, alongside applicable human-subject and infrastructure disclosures.
CHECKLIST
- The checklist marks assumptions, proofs, and explanations for theoretical results as included.It points to Sections 2 and 3 for the CQRE definition and estimation procedure, and Appendix B for solver details and limitations of the nonconvex outer optimization.Source: CHECKLIST · p. 10
- The checklist marks code, data, reproduction instructions, training details, and empirical measures as documented; computing infrastructure is not applicable.It also marks source-code dependencies and reproduction instructions as included.Source: CHECKLIST · p. 10
- The checklist marks citations, applicable asset licenses, and descriptions of new assets as included; consent, sensitive-content discussion, and human-subject materials are not applicable.Source: CHECKLIST · p. 10
A ADDITIONAL THEORETICAL RESULTS
The appendices establish fixed-slack CQRE properties and regularity of the fitted prediction, then detail numerical procedures for computing and estimating CQRE-pa. They also state conditions and limitations for the local derivative used in continuous optimization.
A.1 Fixed-Slack Existence, Uniqueness, and Nesting
- For every finite normal-form game and nonnegative slack vector, the fixed-slack CQRE problem has a unique solution.The proof uses a nonempty compact convex feasible set and strict concavity of entropy.Source: §A.1 · p. 11
- Scalar, per-player, and per-action slack embeddings preserve the relaxed-obedience feasible set and its unique maximum-entropy element.The argument establishes nesting across these slack specifications.Source: §A.1 · p. 11
A.2 Exponential-Family Characterization
- The entropy-maximizing distribution is unique on its attainable support, although its exponential-family multipliers need not be unique.When the attainable support is the full action-profile set, the characterization reduces to Proposition 2.Source: §A.2 · pp. 11–12
A.3 Compact Slack Domain and Canonical Slacks
- Clipping each per-action slack at its maximum unilateral gain preserves the feasible set and removes no attainable CQRE-pa prediction.A fitted distribution also has a componentwise-smallest feasible canonical slack representative.Source: §A.3 · p. 12
A.4 Continuity and Existence of an Outer Minimizer
- On the compact per-action slack domain, the inner prediction is continuous and the outer KL loss attains a finite minimum, though minimizers need not be unique.The set of outer minimizers is nonempty and compact.Source: §A.4 · p. 13
A.5 Implicit Differentiation of the Inner KKT System
- The inner KKT system yields a local CQRE-pa hypergradient when the prediction has full support, strict complementarity holds, active rows have full rank, and the active set is locally unchanged.The formulas do not imply a closed form or a unique outer-optimization solution.Source: §A.5 · pp. 13–14
B.1 Inner maximum-entropy solve
- The inner numerical solver minimizes negative Shannon entropy with SLSQP and checks returned solutions for feasibility before accepting them.It uses a numerical probability floor, warm starts during continuous optimization, and recomputes final KL from the selected distribution.Source: §B.1 · pp. 14–15
B.2 Outer domains and discrete incumbent; B.3 Continuous outer optimization
- The outer estimator uses a discrete grid incumbent followed by multistart L-BFGS-B, with safeguarded finite differences when analytic-gradient checks fail.The final selector retains the grid fit unless a continuous candidate has no larger recomputed KL within numerical tolerance; these are incumbent, not global-optimality, guarantees.Source: §B.2 · p. 15, §B.3 · pp. 15–16
B.4 Equilibrium benchmarks
The benchmark appendix specifies how equilibrium predictions are selected and fitted. It follows a principal Logit-QRE branch from uniform play and includes a limiting mixed-Nash candidate only when convergence is validated.
B.4 Equilibrium benchmarks
- Exact CQRE fixes slack at zero, while scalar and per-player CQRE use the same inner solver and continuous outer procedure as CQRE-pa.All benchmark comparisons use KL( p-hat || prediction).Source: §B.4 · p. 16
- Mixed Nash benchmarks use validated equilibria and, when several are found, report the product distribution with the lowest in-sample KL as an oracle selection.This selection is not a unique rule implied by Nash equilibrium.Source: §B.4 · p. 16
B.5 Logit-QRE computation
- Logit-QRE computation traces the principal branch from its unique uniform solution at β = 0 and selects the branch point with minimum KL over the specified domain.For the branch-audited two-player analysis, β = ∞ is included only if the limiting mixed-Nash prediction passes distance and convergence checks; the best candidate from [0, 20] is retained.Source: §B.5 · pp. 16–17
C DATA AND EXPERIMENTAL PROTOCOL
The protocol combines published human-game data with one-shot LLM decisions across two- and three-player panels. Targets are constructed from panel-specific observations or marginals, and comparisons use paired bootstrap procedures to assess descriptive fit.
C.1 Collection interface and common LLM protocol
- Six OpenAI deployments made stateless one-shot decisions from payoff prompts, without interaction history, opponent identity, or a public signal.The two-player prompts showed the full payoff matrix; three-player prompts showed only the responding player's payoffs.Source: §C.1 · p. 17
C.2 Target definitions and counts
- The human benchmark uses twelve Selten–Chmura games, with 108 subject groups contributing 800 realized action pairs per group across 200 periods.The main analysis uses periods 1–200 and weights the twelve games equally.Source: §C.2 · pp. 17–18
- The all-12 extension defines 588 ordered-agent targets, including 432 AI–AI, 144 constructed human–AI, and 12 human–human targets.The dataset contains 201,600 accepted decisions from six deployments; constructed human–AI targets are outer products, not observed interactions.Source: §C.2 · p. 18
C.2 Target definitions and counts; C.3 Game construction
- The larger-action panel comprises seven two-player games and 147 targets, while the three-player panel covers 56 deployment multisets in one fixed three-player, three-action game.The larger-action games include strongly-monotone and rank-r constructions; the three-player targets retain a canonical player assignment.Source: §C.2 · p. 18, §C.3 · p. 18
C.4 Bootstrap and inference
- Bootstrap comparisons use paired targets, but the reported comparisons measure descriptive fit rather than out-of-sample predictive performance.The human benchmark resamples subject groups and within-group pairs; the three-player analysis resamples action triples and refits both models.Source: §C.4 · pp. 18–19
C.5 Data, Code, and Licenses
This appendix section records licensing for the replication data and software and introduces additional empirical results. The reported aggregates use percentile bootstrap intervals and equal target weighting unless stated otherwise.
C.5 Data, Code, and Licenses
- The Selten–Chmura replication data are licensed CC BY 4.0, and the accompanying software uses the BSD 3-Clause license.The data are available from openICPSR, Project 113254.Source: §C.5 · p. 19
D ADDITIONAL EMPIRICAL RESULTS
- Appendix D reports additional results supporting the main empirical section.Source: D ADDITIONAL EMPIRICAL RESULTS · p. 19
- Unless stated otherwise, reported intervals are percentile intervals from synchronized bootstrap replicates, and aggregate quantities weight targets equally.Source: D ADDITIONAL EMPIRICAL RESULTS · p. 19
D.1 Established Human Benchmark
On the twelve Selten–Chmura human games, CQRE-pa lowers mean KL relative to Logit-QRE. The extension distinguishes constructed AI–human targets from observed human–human interaction.
D.1 Established Human Benchmark
- CQRE-pa has the lowest point KL in nine of twelve games and reduces equal-game mean KL from .0093 for Logit-QRE to .0040, a 56.9% reduction.Fit also improves from .0072 for common-slack CQRE to .0057 for per-player CQRE and .0040 for CQRE-pa; the aggregate paired interval for ∆ is strictly below zero, though game-level results are heterogeneous.Source: §D.1 · p. 19
D.2 All-12 Selten–Chmura Extension
- Figure 3 plots full-sample KL on logarithmic axes; points below the diagonal favor CQRE-pa, and SC10’s CQRE-pa estimate is below 10^-4.Intervals extending below 10^-4 are clipped at the plot boundary.Source: §D.1 · p. 19, Figure 3 · p. 20
- Human–AI targets are products of corresponding human and AI marginals, so they do not represent observed human–AI interaction.The targets are constructed rather than observed interaction data.Source: §D.2 · p. 20
- The all-12 extension separates AI–AI, constructed human–AI, and human–human targets; each AI–AI row averages 36 ordered targets, each human–AI row 12 directional targets, and each human–human row one target.The two human–AI directions distinguish which player occupies the row role.Source: §D.2 · pp. 19–20
D.3 Larger-Action Games
Larger-action comparisons show different relative fits across games and populations, with GPT-5-family results differing from the remaining targets. Extending Logit-QRE’s β domain leaves target-level verdicts unchanged in the matched implementation.
D.3 Larger-Action Games
- The seven-game comparison uses continuous CQRE-pa point fits and stored Logit-QRE fits over β ∈ [0, 20]; paired intervals classify wins, losses, or unresolved comparisons.Intervals are fixed-prediction replays: observations are resampled and KL is reevaluated against stored predictions without refitting either model.Source: §D.3 · p. 20
- CQRE-pa wins all 12 GPT-5-family comparisons across sm n5, sm n7, rank1 n5, and rank4 n5, while Logit-QRE wins all such comparisons in sm n3 and rank2 n5.The 21-target GPT-5-family subgroup has mean ∆= +0.25 versus −2.78 for the remaining 126 targets; its mean CQRE-pa KL is 1.7–3.3 times the within-game mean for other targets.Source: §D.3 · p. 21
- GPT-5-family play is more concentrated in every game; in sm n3, those targets put 98–99% of their mass on the unique pure Nash profile (A, A).The concentration pattern is a useful diagnostic, but game structure also materially affects relative fit.Source: §D.3 · p. 21
D.3.1 Extended Logit-QRE Domain
- Extending Logit-QRE to β ∈ [0, ∞], with β = ∞ as the limiting mixed-Nash prediction, leaves all target-level verdicts unchanged; only rank3 n5 changes at displayed game-level precision.CQRE-pa remains favored on sm n5 (mean KL .8648 versus .9031; ∆= −.0383, 95% interval [−.0486, −.0351]) and rank4 n5 (1.0759 versus 1.1351; ∆= −.0592, 95% interval [−.0717, −.0520]).Source: §D.3.1 · p. 21
D.4 Three-Player Paired-Refit Results
In the three-player analysis, every target-specific paired interval favors CQRE-pa. Across strongly-monotone games, most centered fitted-slack variation lies in a small number of singular directions.
D.4 Three-Player Paired-Refit Results
- All 56 target-specific paired intervals lie strictly below zero in the three-player results.The analysis uses 56 model triplets with replacement and refits both CQRE-pa and Logit-QRE for each of 1,000 synchronized bootstrap replicates.Source: §D.4 · p. 22, Table 7 · p. 22
D.5 Effective Slack Dimension
- Across eight games, 90% of centered fitted-slack variation is captured by 2–7 singular directions, or 22–50% of the nominal dimension.The spectra are computed from centered matrices of componentwise-minimal fitted CQRE-pa slack vectors, one per model pairing.Source: §D.5 · p. 22, Figure 5 · p. 24
- CQRE-pa has lower mean KL in six of seven larger-action games and sweeps sm n5, sm n7, and rank1 n5; rank2 n5 is the principal inversion.The comparison uses continuous CQRE-pa and stored Logit-QRE fits over β ∈ [0, 20].Source: Figure 4 · p. 23