Source-linked AI summary
Choosing LLMs for Strategic Agent Interactions
Arastun Mammadli, Herbert Woisetschläger, Jawad Fayaz, Shiqiang Wang
TL;DR
Strategic benchmarks fix the model being evaluated, while conventional routers select models without accounting for strategic structure or an unknown, adaptive opponent. This paper formalizes selection as an opponent-conditioned meta game and evaluates selector methods across games and models. Strategic competence varies by game and opponent; simple bandit selection is more reliable than several heavier approaches, but does not solve selection against adaptive opponents.
Problem
Strategic benchmarks evaluate fixed models and conventional routers omit strategic structure and opponents, leaving model selection against unknown, adaptive opponents unaddressed.
Source: §1 · p. 2Method
The paper formalizes model choice as a repeated opponent-conditioned meta game layered over the base game and evaluates selectors across games and model portfolios.
Source: §1 · p. 2, §3.2 · p. 3Results
Across games and models, strategic competence varies substantially; plain bandits can outperform embedding and LLM-reflection methods, while simple adaptive bandit variants do not outperform stationary Thompson sampling.
Source: §4.2 · pp. 5–6, §6 · p. 8Takeaways & Limitations
Game-specific signal and interaction noise make careful replication important when evaluating strategic selection.
Source: §3.3 · p. 4The evaluation covers five games and uses parsed game actions, rather than natural-language agent interactions.
Source: §6 · p. 8
Abstract
from arXiv · showhide
Game-theoretic benchmarks claim that large language model (LLM) strategic decision-making varies across game types, interaction structures, and capability dimensions. However, they only evaluate strategic performance across individual fixed models. In parallel, contemporary LLM routers pick models under quality-cost trade-offs, but with no notion of a strategic structure. They do not account for an opponent that might be unknown, and that can adapt over time and shift the feedback signal. To address this, we extend the game-theoretic evaluations and define strategic selection as the meta game layered on top of the base game. In each round, a strategic selector fields an LLM from a portfolio of models to navigate a game payoff matrix that is latent and learned online. We find that the evaluation signal is game-specific, and that extracting a reliable signal requires a much higher number of seed runs $N$ compared to traditional benchmarking setups. Evaluating across 5 games and 10 LLMs, we show a model's strategic competence varies significantly depending on the game setup and the opponent faced. Building on this, we implement a suite of selector methods and find that approaches that rely on query embeddings or LLM self-reflection do not consistently capture the underlying strategic signal. A lightweight multi-armed bandit algorithm performs more reliably across games, though it still falls short of the best fixed model. Finally, we show that simple non-stationary and adversarial bandit variants fail to outperform a stationary bandit baseline, pointing to a gap in how current methods handle a shifting opponent in strategic settings.
1 Introduction
Existing strategic benchmarks fix the model, while LLM routers select under static task distributions without modeling opponents or strategic structure. This work treats model selection as an opponent-conditioned meta game and investigates how selectors learn against unknown, potentially adaptive opponents.
1 Introduction
- Strategic benchmarks study fixed-model play, whereas routers optimize performance-cost trade-offs over static queries without modeling strategic structure or opponents.LLM portfolios also vary in the type and quality of strategic behavior they exhibit.Source: §1 · p. 2
- Strategic selection layers a repeated meta game between selectors choosing fielded LLMs on top of the base game they play.Against adaptive opponents, selection must account for changing interactions, partial observability, and stochastic feedback.Source: §1 · p. 2
- The central question is whether a selector can learn which model to field against an unknown, potentially adaptive opponent.The opportunity arises because model capabilities vary across games.Source: §1 · p. 2
- The paper formalizes strategic selection, characterizes game-specific signal and noise to guide seed counts, and evaluates adapted routing methods across 10 models.Its empirical investigation tests scenario-dependent strategic capabilities and selector performance against fixed models and selector opponents.Source: §1 · p. 2
2 Related Work
Prior work studies strategic play with a fixed model per player, while routing work predicts a model for a query, generally without explicitly representing evolving strategic interaction. The paper considers selector methods that differ in what information and learning mechanisms they use.
2.1 Strategic LLM Decision-Making
- Game-theoretic benchmarks and frameworks find uneven, structure-dependent LLM performance, but existing strategic evaluations fix each player’s model [28] [36].Other work studies higher-level capabilities [3] [24] [32] [35] and strategic dispositions.Source: §2.1 · p. 2
- Table 1 classifies selector methods by cost awareness, opponent identity, online learning, embeddings, and non-stationary design.These dimensions describe which trade-offs, information, and learning mechanisms each method uses.Source: Table 1 · p. 3
2.2 LLM Routing
- Routing methods predict a best-fit model for a query, often using embeddings and performance-cost trade-offs [1] [6]; strategic selection instead depends on evolving game history.Near-identical query embeddings can call for opposite model choices, motivating per-round bandit feedback.Source: §2.2 · p. 3
3 Strategic LLM Selection
Strategic selection treats model choice as a repeated game between deployers, layered over the base game played by their selected LLMs. The formulation incorporates online updates, noisy interaction, and a net-gain measure balancing game utility and inference cost.
3.1 What is Strategic Selection?
- The meta game makes interaction itself a design consideration: strategic selection accounts for agent coordination, negotiation, and competitive outcomes, beyond system performance or cost alone.In competitive settings, it can give deployers more control over interaction outcomes and alignment with their preferences.Source: §3.1 · p. 3
3.2 Problem Formulation
- Each deployer selects a portfolio model as its meta-game action; the fielded models then play the base game and yield utility, inference cost, and net gain.Selectors update parameters from observed context and game outcomes, while the opponent’s selector, model, and action remain unobserved.Source: §3.2 · pp. 3–4
3.3 Noise in Strategic Play
- Strategic LLM evaluations require more replications than traditional benchmarks because opponent-conditioned interaction adds noise to stochastic, prompt-sensitive model play.The paper measures signal and noise across games and develops a way to choose seed counts for each game-portfolio pair.Source: §3.3 · p. 4
3.4 Evaluation Metrics
- Net gain combines min-max-normalized game utility and inference cost, with the cost trade-off weight fixed at λ = 1 in the reported runs.Across N seed runs, results are summarized using the mean, standard error, and 95% confidence interval.Source: §3.4 · p. 4
4 Experiments
The experiments test whether model selection can exploit game-specific strategic differences, first against fixed unknown opponents and then adaptive ones. Bandit selection performs best among selectors, but simple adaptations do not overcome the challenges of strategic interaction.
4 Experiments
- Experiments compare 10 open-weight models across five games and test bandits, LLM routers, self-reflection, and selection against fixed and adaptive opponents.Performance across Tables 2–4 should not be compared directly because the opponent pools differ across settings.Source: §4 · pp. 4–5, §4.1 · p. 5
4.1 Design
- Detectable pairwise comparisons stabilize near N ≈150 seeds; experiments use N=300 for fixed matchups and T=300 rounds with N=1 for selectors.Selectors’ context is limited to the current round, making each round an independent draw of opponent play.Source: §4.1 · p. 5
4.2 Results
- Strategic competence and disposition vary by game and model: scale is unreliable, and even models within a family can behave sharply differently.For example, Granite4.1 8B dares in 96% of Chicken Game rounds, while its 3B and 30B versions never dare.Source: §4.2 · p. 5, Figure 2 · p. 6
- Thompson sampling achieves the highest net gain among selectors in every game, yet fails to beat the best fixed model even in Cournot Duopoly.The comparison is against Llama3.1 8B in Cournot Duopoly.Source: §4.2 · p. 6
- Against adaptive selector opponents, plain multi-armed bandits remain dominant; Sliding-Window TS, Discounted TS, and EXP3 do not outperform the stationary baseline.The authors conclude that these simple adaptations do not capture implicit learning signals shaped by repeated strategic interaction.Source: §4.2 · p. 6
5 Lessons Learned
Strategic selectors need to account for game instructions, portfolio-model dispositions, and opponent actions, but current methods do not use this context consistently. Model performance varies across games, while games can lose their ability to distinguish models near equilibrium.
5.1 How to Learn from Strategic Context?
- Strategic context includes game instructions, portfolio-model dispositions, and opponent actions, but embedding the full context or relying only on outcomes misses parts of the scenario.A better selector should infer game-specific strategic cues and relate selection choices to future game states.Source: §5.1 · pp. 6–7
5.2 Games as Strategic Testbeds
- Best-performing models vary across five games; scale-performance correlation is strongest in Cournot Duopoly (r = 0.66), versus Stag Hunt (r = 0.12) and Colonel Blotto (r = 0.24).The discussion calls for methods to classify game difficulty and strategic value to guide deployment and selector design.Source: §5.2 · pp. 7–8
5.3 Games as Dynamic Testbeds
- Games provide dynamic interactive evaluations and are described as more resistant to training-data contamination than static benchmarks.Near-equilibrium play can erase discrimination: in zero-sum Colonel Blotto, equilibrium strategies yield indistinguishable expected payoffs, while general-sum games typically have CCE outcomes with varied payoffs; the authors suggest favoring general-sum games for discriminative inventories.Source: §5.3 · p. 8
6 Conclusion and Future Work
Across 5 games and 10 models, strategic capability varies across games and models, and selector methods differ in how well they exploit it. The section also describes limits on the study’s scope and methods for assessing signal and seed requirements.
6 Conclusion and Future Work
- Across 5 games and 10 models, strategic capability varies substantially; a plain bandit can surpass heavier encoders and LLM reflection, while discounted and adversarial variants fail to outclass the plain sampler.The authors motivate selectors that learn from model capability, usage cost, and strategic disposition in noisy interactions.Source: §6 · p. 8
- The study covers 5 games and uses parsed game actions, which can underestimate strategic capabilities such as persuasion and associated deployment risks.Expected gain also omits high-stakes risks such as catastrophic losses, regulatory penalties, and reputational damage.Source: §6 · p. 8
B.1 Signal and Noise Ratio
- Signal measures the largest gap between model means relative to the game-wide mean, while noise captures seed-to-seed score spread relative to a model’s mean.SNR is high when model separation exceeds the amount individual model scores jitter across seeds.Source: §B.1 · p. 13
B.2 Number of Seeds
- Choosing the seed count trades inference cost against confidence, and decision accuracy estimates whether small-batch pairwise rankings agree with a large-reference batch.The reference batch is self-generated in game-theoretic evaluations rather than drawn from an external test suite.Source: §B.2 · pp. 13–14
- The minimum detectable effect estimates whether a given seed count and opponent-pool size can resolve pairwise model gaps, with detectable fractions swept across seed counts.Its estimates use variation in model gaps across shared opponents and each model’s within-opponent seed variation.Source: §B.2 · p. 14
B.3 Analysis
The analysis finds that strategic-evaluation signal strength differs by game and requires substantial replication to estimate reliably. Its signal and seed-stability analyses use fixed-model, single-round matchups, so they do not directly characterize evolving selector runs.
B.3 Analysis
- These signal, noise, and seed-stabilization estimates use fixed-model, single-round matchups and do not directly transfer to selector runs whose choices evolve over rounds.Each seed is an independent draw of the base game in this analysis.Source: §B.3 · p. 14
- Across five games, SNR and decision accuracy correlate at r ≈0.81–0.85 for n ≥75; Cournot Duopoly has the highest SNR and Colonel Blotto the lowest.The SNR ranking is independently confirmed by decision accuracy.Source: §B.3 · p. 14, Figure B.1 · p. 14
- 95% decision accuracy requires n = 20 seeds in Cournot Duopoly but n = 200 in Colonel Blotto; no game-wide seed count below 200 clears 95% across all games.The decision-accuracy figure presents results across candidate seed counts.Source: §B.3 · pp. 14–15, Figure B.2 · p. 15
- Power analysis finds some games never clear the significance bar at the full seed budget; detection stabilizes around ≥150 seeds in games that do, and the main investigation uses N = 300.Cournot Duopoly has the highest ratio of detectable pairwise comparisons.Source: Figure B.2 · p. 15, §B.3 · p. 15
C.1 Models
The experiments use a portfolio of open-weight models across five strategic games and compare fixed, informed, reflective, embedding-based, and bandit selector approaches.
C.1 Models
- The portfolio contains 10 open-weight models from three size tiers; every model is served locally through Ollama.Calls use temperature 1.0 and top-p 0.9.Source: §C.1 · p. 15, Table C.1 · p. 15
C.2 Games
- The five games span zero-sum allocation, quantity-setting, coordination, second-price bidding, and anti-coordination through brinkmanship.The games are Colonel Blotto, Cournot Duopoly, Stag Hunt, Vickrey Auction, and Chicken Game.Source: §C.2 · pp. 15–16
C.3 Selectors
- Fixed Model fields one model throughout; Per-Opponent Full-Information uses a round-robin prior to choose a model against a known fixed opponent and then commits to it.Fixed Model retains context and past outcomes across rounds unless otherwise stated.Source: §C.3 · p. 16
- The LLM Selector uses Reflexion with a rolling history, while RouterDC selects by cosine similarity between the current history embedding and offline-trained candidate embeddings.The LLM Selector is queried each round; RouterDC's checkpoint is fit once per game.Source: §C.3 · p. 16
- Causal LLM Router samples from per-candidate, per-opponent beliefs and applies a doubly robust correction using its own pick probability as propensity.A propensity floor keeps every candidate's pick probability nonzero.Source: §C.3 · p. 16
- Thompson Sampler chooses the model with the highest sample from per-model Gaussian beliefs; its discounted and sliding-window variants decay or discard older observations.The discounted variant uses 0.99; the sliding window retains the 50 most recent observations.Source: §C.3 · p. 16
- EXP3 updates candidate weights multiplicatively from importance-weighted observed signals and adds a uniform exploration floor.The exploration rate is 0.1.Source: §C.3 · p. 17
D More Results
Appendix D reports complete disposition analyses and fixed-model and selector matchup results.
D More Results
- Appendix D.1 gives in-game disposition analyses, D.2 reports fixed-model pairwise matchups across five games, and D.3 reports selector-versus-fixed-model matchups.Source: D More Results · p. 17
D.1 Fixed Model Disposition Scores
The appendix summarizes fixed-model dispositions with principal components, then provides complete fixed-model and selector matchup tables.
D.1 Fixed Model Disposition Scores
- The first two principal components capture 69% of model disposition differences, interpreted as competitiveness and trust in the opponent.Disposition scores are standardized across models before projection.Source: §D.1 · p. 17, Table D.1 · p. 17
D.2 Fixed Model vs. Fixed Model
- Table D.2 reports each model's net gain, normalized utility, and normalized cost across five games, aggregated over pairwise matchups.The table reports mean ± 95% CI, ×100, for T = 1 and N = 300 seeds per run.Source: Table D.2 · p. 18
D.3 Selector vs. Fixed Model
- Table D.3 reports normalized net gain and in-game utility for selectors and fixed models across full pairwise matchups and five games.It reports mean ± 95% CI, ×100, for T = 300 rounds and N = 1 seed per run, highlighting the best score for each game.Source: Table D.3 · p. 19