Source-linked AI summary
Social Gym and SPaRTan: Benchmarking and Improving LLM Social Reasoning via Multi-Agent Game Tournaments
Keyu He, Xuhui Zhou, Maarten Sap
TL;DR
Social-reasoning evaluation lacks a method that is simultaneously multi-turn, broad, and rule-verifiable. The paper introduces Social Gym and SPARTAN, and reports that playbooks lift weaker roles for GPT-5-mini while gains largely vanish for Qwen3-32B.
Problem
Existing social-reasoning evaluations are either static, depend on subjective LLM judging, or cover a single game or domain, leaving no approach that is simultaneously multi-turn, broad, and rule-verifiable.
Source: §1 · p. 1Method
Social Gym evaluates agents across 21 rule-decided social games, while SPARTAN turns self-play trajectories and outcomes into a transferable playbook used in subsequent prompts.
Source: §1 · pp. 1–2, §4 · p. 5Results
Takeaways & Limitations
The findings indicate that model rankings, role advantages, and reflection gains depend on interaction structure rather than a single scalar social-ability measure.
Source: §6 · p. 8Because Social Gym uses fixed rules, role structures, and rule-decided outcomes, whether its measured capabilities and playbooks transfer to real interactions without clear win/loss criteria remains open.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
LLM agents are increasingly deployed in multi-agent social settings where they must cooperate, negotiate, and adapt to other agents. Measuring and improving these social skills is hard because, unlike math or logic, social interaction offers no objective ground truth: evaluations fall back on LLM judges, which are costly, subjective, and noisy, and models get no reliable signal to learn from. To address both, we first introduce Social Gym, an environment of 21 multi-agent social games (e.g., Werewolves, Resistance, Spyfall) whose rule-decided outcomes make agent performance verifiable and objective, with an Elo tournament that produces a cross-game leaderboard. Benchmarking experiments show that while GPT-5-mini tops the leaderboard, no model excels at all games uniformly or in all game roles, pointing to limitations of social reasoning. Motivated by this, we additionally propose SPaRTan (Self-Play and Reflect-Transfer), a training-free self-improvement loop: a model plays a game, reflects on its trajectories and their outcomes to produce a transferable playbook, and applies that playbook in subsequent games. Our results show that SPaRTan playbooks help GPT-5-mini agents level their performance on weaker roles, but largely do not improve Qwen3-32B's performance. Together, Social Gym and SPaRTan offer a reproducible, verifiable foundation for measuring and improving LLM social reasoning without weight updates.
1 Introduction
Existing social-reasoning evaluations lack a single approach that is simultaneously multi-turn, broad in domain coverage, and rule-verifiable. The paper introduces Social Gym and SPARTAN to measure social-game performance and test training-free improvement.
1 Introduction
- Existing evaluations trade off sustained interaction, objective scoring, and breadth: static probes are single-turn, interactive evaluations rely on subjective LLM judges, and verifiable studies target isolated games or domains.The authors identify the remaining gap as evaluation that is multi-turn, broad, and determined by interaction rules.Source: §1 · p. 1
- Social Gym spans 21 multi-agent games across competitive, cooperative, and mixed-motive structures, with sustained play requiring role-playing, coalition management, deception, and strategic information control.Its unified Elo tournament provides per-game win-rate tables and a cross-game leaderboard.Source: §1 · pp. 1–2
- SPARTAN is a training-free loop in which a model plays self-play games, reflects on trajectories to write a transferable playbook, and uses it in subsequent games.The playbook is injected into the model’s system prompt.Source: §1 · p. 2
- Across four evaluation setups, GPT-5-mini-generated playbooks lift the structurally weaker side of asymmetric games and transfer across games and into weaker LLMs, while gains largely vanish for Qwen3-32B.The setups include within-game iteration, cross-game transfer, multigame transfer, and distillation to weaker models.Source: §1 · p. 2
2 Related Work
Prior work evaluates social reasoning through static probes, interactive judging, or isolated games, while self-improvement methods range from prompt-only reflection to weight updates. The paper positions Social Gym and SPARTAN as broad evaluation and cross-game playbook-transfer contributions.
LLM-based evaluations of social intelligence
- Static social-reasoning probes provide reproducible scores but cannot exercise sustained behavior; interactive frameworks preserve multi-turn dynamics but use LLM judges or human raters, while single-game studies verify outcomes but isolate one social dynamic.Broader text-game environments emphasize competitive strategy rather than the breadth of social-cognitive demands.Source: §2 · pp. 2–3
- Social Gym fills the evaluation gap with a multi-turn, rule-decided suite of 21 games organized along social-cognitive axes.The suite combines broad coverage with verifiable outcomes.Source: §2 · p. 3
Self-improvement methods for LLMs
- Prompt-only self-improvement includes reflection and transferable natural-language skill libraries, whereas weight-update approaches such as SPIRAL use multi-agent self-play with reinforcement learning.SPARTAN belongs to the prompt-only family and is closest to skill-library methods.Source: §2 · p. 3
- Earlier skill-library work accumulated strategies within a single domain, leaving cross-game playbook transfer unstudied; SPARTAN investigates it through iterated-round, multi-source, and strong-to-weak experiments.The paper also tests one-source-to-many-target transfer, as described in its related-work passage.Source: §2 · p. 3
LLM agents in multi-player games
- Multi-player LLM studies have commonly covered one game, equilibrium class, or skill, while Social Gym aggregates 21 games and SPARTAN tests cross-game transfer.Prior work includes game-specific strategic scaffolding and isolated behavioral probes.Source: §2 · p. 3
3 Social Gym Benchmark
Social Gym combines a shared multi-agent game engine, a suite spanning distinct social-cognitive demands, and an Elo tournament with per-game and overall results. Its leaderboard shows broad model-ranking variation across games and roles.
3 Social Gym Benchmark
- Social Gym requires algorithmically verifiable outcomes and games spanning social intelligence, from strategic primitives to long-horizon group deception.These principles support unambiguous rewards and expose distinct failure modes.Source: §3 · p. 3
3.1 System Architecture
- The system architecture uses a finite-state-machine engine, visibility scopes for public, team-private, and private information, and a shared game interface that supports adding games through game-specific state transitions and rewards.The visibility layer tests inference about hidden states from permitted observations.Source: §3.1 · p. 3
3.2 Game Suite
- The 21 games span five categories across information structures and communication modes, covering strategic primitives, collective action, bluffing, hidden-role deduction, and alliance or reputation dynamics.Categories include Normal-Form, Economic, Bluffing, Hidden-Role Deduction, and Social Strategy.Source: §3.2 · pp. 3–4
3.3 Measuring success via Elo Tournament
- The Elo tournament fits ratings from pairwise outcomes, balancing roles and seats; cooperative games without a well-defined ranking are scored by win rate instead.The tournament fits Elo on 17 competitive games and separately reports ratings for majority/cooperative and minority/deceptive roles in hidden-role games.Source: §3.3 · p. 4
3.4 Leaderboard Results
- Per-game ranking inversions include Qwen3-32B at 1328 on Chicken but 817 on Werewolves; no model leads every category, and hidden-role deduction most sharply distinguishes model capabilities.Overall Elo and win rate track general capability, while game-specific scores reveal variation masked by a single aggregate.Source: §3.4 · pp. 4–5
- GPT-5-mini leads the overall leaderboard at 1110 Elo, while Qwen2.5-3B places last at 926; per-game rankings nevertheless invert sharply, with no model uniformly strong.Figure 2 reports results across 17 competitive games and provides per-game Elo alongside overall Elo and win rate.Source: §3.4 · p. 4, Figure 2 · p. 5
- Across six hidden-role games, no model plays both sides at the same level; the gap usually favors the minority/deceptive role but reverses for the strongest models in matched-capability self-play.Trajectory inspection also finds frequent parroting by Qwen2.5-3B, which the authors say likely contributes to its last-place overall Elo.Source: §3.4 · p. 5
4 SPARTAN: Self-Play and Reflect-Transfer
SPARTAN uses self-play trajectories and outcomes to create a transferable strategic playbook, then adds it to prompts for later games. Its evaluation includes within-game iteration, transfer, and distillation setups.
4 SPARTAN: Self-Play and Reflect-Transfer
- SPARTAN addresses per-game performance gaps with a training-free play, reflect, and transfer loop that extracts reusable strategies from a model’s own gameplay.The motivation is that top-ranked models still underperform peers on particular games.Source: §4 · p. 5
- During reflection, the model reviews trajectories and role-specific outcomes to produce a game-agnostic playbook covering deception, detection, persuasion, information management, coalitions, and timing.The playbook is prepended to the system prompt for subsequent games.Source: §4 · p. 5
- The evaluation tests within-game iteration, one-source-to-many-target transfer, many-source-to-one-target transfer, and strong-to-weak distillation using GPT-5-mini as the source model.The distillation setup uses six student models on Resistance.Source: §4 · p. 5
- In within-game iteration, the reflected side’s win rate rises while the opposing side’s falls across rounds, with the reflected side peaking at different rounds across Werewolves, Spyfall, and Resistance.The reported peaks are R3 for Werewolves, R2 for Spyfall, and R3 for Resistance.Source: Table 1 · p. 6
5 SPARTAN Experiments and Results
SPARTAN tests whether self-play reflection and transferable playbooks improve win rates without parameter updates, especially for structurally weaker roles. GPT-5-mini playbooks often lift disadvantaged sides, while Qwen3-32B shows mostly null results outside simpler action-channel games.
5 SPARTAN Experiments and Results
- In vanilla GPT-5-mini self-play, alt-role win rates are consistently lower than main-role rates on Werewolves, Spyfall, Undercover, and Resistance; each condition uses n=30 games, with binomial 95% CIs ≈±18 pp.Source: §5 · p. 6
5.1 Same-model reflection
- Across GPT-5-mini’s iterated reflections, the alt-side win rate averages 24% at baseline and 36%, 36%, 43%, and 35% under R1–R4, while the main side falls from 76% to 66%, 59%, 67%, and 63%.The alt-side peak is R3; per-game peaks vary, and most gains arrive at R1 before later rounds redistribute performance.Source: §5.1 · p. 6
- Cross-game R1 transfer skews positive for alt-side play and negative for main-side play, matching the within-game pattern of helping the disadvantaged side while leaving the advantaged side unchanged or hurting it.The Chameleon target is saturated at 100% on the alt side, so its target column carries no alt-direction signal.Source: §5.1 · pp. 6–7, Figure 3 · p. 7
- Broader multigame playbooks usually do not beat the target’s own Single-R1, but held-out Resistance improves from 23% with Single-R1 to 47% with Rwcsu.Chameleon remains at its 100% alt-side ceiling; Resistance is the exception and has the most baseline headroom.Source: §5.1 · p. 7, Figure 4 · p. 7
5.2 Cross-model distillation
- On Resistance, the held-out GPT-5-mini Rwcsu playbook raises three students’ weaker Spies-side win rates by 13 to 24 percentage points, while favored-side changes stay within ±5 points for every student.Qwen2.5-3B and Gemma3-27B do not gain on either side.Source: Figure 5 · p. 7, §5.2 · p. 8
5.3.1 Self-reflection (Open-weights model)
- For Qwen3-32B, Prisoner’s Dilemma is the only clean self-reflection lift, rising from 13% to 58% at R1; Resistance shows a smaller positive effect, while four other games are flat.The authors attribute the mostly null result to difficulty learning from long, complex social-deduction trajectories, except when actions collapse to a discrete token.Source: §5.3.1 · p. 8
- Qwen3-32B discussion outputs often parrot or paraphrase the prior speaker, and reflection codifies this behavior without eliminating it in subsequent play.This is reported as one concrete symptom associated with the model’s limited improvement on complex social-deduction games.Source: §5.3.1 · p. 8
5.3.2 Distillation (Open-weights model)
- Qwen3-32B teacher playbooks distill positively to all three students in Prisoner’s Dilemma, but Resistance results are mixed, ranging from +10 points to −20 points on the Resistance side.The authors report that the action-channel game transfers cleanly while the discussion-heavy game does not.Source: §5.3.2 · p. 8
6 Conclusion and Discussion
Social Gym’s 21 games and unified Elo expose game- and role-specific variation beneath overall rankings, while SPARTAN tests training-free playbook improvement. Across its evaluated settings, playbooks lift the structurally weaker side of asymmetric games; the authors identify future tests in realistic domains and with verifiable rewards.
6 Conclusion and Discussion
- SPARTAN playbooks lift the structurally weaker side of asymmetric games across within-model iteration, cross-game transfer, and cross-model distillation.The authors also report that Social Gym’s 21-game leaderboard reveals per-game ranking inversions and role-conditioned imbalances beneath overall rankings.Source: §6 · p. 8
- The results suggest that social ability is not a single scalar: model rankings, role advantages, and reflection gains depend on interaction structure.The authors say combining a broad game suite with targeted playbook interventions can separate structural properties of settings from model-specific failures.Source: §6 · p. 8
- Social Gym is proposed as a testbed for future reinforcement learning with verifiable rewards, using rule-computed episode outcomes without LLM judges.The authors identify testing transfer across cooperation, negotiation, bluffing, and hidden-role deduction as a future direction.Source: §6 · p. 9
- Whether playbooks learned in rule-based games carry over to realistic deployment settings remains a question for future work.Suggested domains include professional negotiation, customer-service de-escalation, and collaborative multi-agent work.Source: §6 · p. 9
Limitations
The evaluation is bounded by fixed-rule games with rule-decided outcomes, modest per-condition samples, and a missing placebo-playbook control. SPARTAN also confines reflection to natural-language system-prompt text.
Limitations
- Social Gym’s fixed rules, role structures, and rule-decided outcomes do not represent interactions such as resolving disagreements or building long-term trust, which lack clear win/loss criteria.Whether the measured capabilities and playbooks carry over to such settings remains open.Source: Limitations · p. 9
- With 30 evaluation games per condition, binomial 95% confidence intervals are roughly ±18 percentage points, and several reported effects fall within that range.The authors recommend replication with larger samples before strong conclusions are drawn.Source: Limitations · p. 9
- Without a content-matched placebo-playbook control, the study cannot fully separate playbook-content effects from generic prompt-perturbation effects.The authors point to structural patterns and Undercover’s monotonic regression toward 0% as evidence against a pure prompt-perturbation reading.Source: Limitations · p. 9
- SPARTAN reflection is limited to natural-language prose in the system prompt, without tool updates, external retrieval, or structured reasoning beyond that text.Methods that add retrieval or scratchpads may show qualitatively different transfer behavior.Source: Limitations · p. 9
Ethics / Broader Impacts
The paper flags dual-use risks from measuring and improving socially manipulative capabilities, while describing synthetic-game safeguards and releasing its code. Its Elo aggregation uses Bradley–Terry fitting and bootstrap confidence intervals, with cooperative games reported by normalized win rate.
Ethics / Broader Impacts
- The measured capabilities include deception, persuasion, and coalition manipulation, creating possible dual-use risk if transferred to human interactions.Experiments use synthetic multi-agent games; the authors characterize improvements as modest and largely null for open-weights models, and discourage adversarial human-facing use of playbook distillation.Source: Ethics / Broader Impacts · p. 9
A Code and Data Release
- The released repository includes the game engine and 21 games, Elo tournament infrastructure, and SPARTAN pipeline scripts; the project’s code and game implementations are MIT-licensed.The bundled SOTOPIA snapshot is also MIT-licensed, while evaluated proprietary models are accessed under provider terms and open-weights models under their published licenses.Source: A Code and Data Release · p. 12
B Elo Aggregation Details
- The Elo fit uses L2-regularized logistic regression with C=0.1, scale 400, and mean anchored at 1000; confidence intervals come from 1,000 multinomial bootstraps.The authors prefer Bradley–Terry fitting because it is path-independent and online updates would not converge with roughly 30 episodes per pair; cooperative games use normalized win rates instead of Elo.Source: B Elo Aggregation Details · p. 12
- Pairs assigned the same outcome bucket by design are excluded from the Bradley–Terry fit rather than treated as draws.This avoids capping ratings for strong models in games where co-winners or co-losers share a score without competing in a tie.Source: B Elo Aggregation Details · p. 12
C Capability-Profile Elo
Capability-profile Elo refits expose how model rankings vary across game categories, skill tags, and structural axes. The appendices also describe the shared game engine, game prompts, and reflection-playbook generation.
C Capability-Profile Elo
- Capability-profile ratings refit Bradley–Terry models on pooled outcomes within each game group, using the same hyperparameters and bootstrap; only 17 competitive games enter, and skill tags overlap.Frontier models gain Elo as information becomes more hidden while Qwen2.5-3B falls 72 Elo from complete information to hidden roles; open-ended games separate 19 of 21 model pairs at 95% confidence.Source: C Capability-Profile Elo · pp. 12–13
- Six hidden-role games separate the same 19 of 21 model pairs as the full 17-game tournament, despite using 43% of its episodes; economic and normal-form games reorder or compress rankings.Bluffing preserves the top two ranks but leaves the other five models within 32 Elo; coordination-tagged games barely separate the field. Figure 6 presents independent category and skill-tag refits, and the table captions identify the corresponding Elo profiles.Source: C Capability-Profile Elo · p. 13, Figure 6 · p. 14, Table 3 · p. 14
D Game Engine Details
- The shared engine runs games from declarative configurations and thin Python components, so adding a game requires no engine changes or game-specific engine code.Configurations specify the state machine, acting roles, legal actions, scheduler, message visibility, goals, and secrets; the scheduler advances states and the evaluator checks rule-based end conditions after each turn.Source: D Game Engine Details · pp. 13–14
E Game-Play Prompt
- A shared prompt template supplies game description, role goal and secrets, filtered history, available actions, output instructions, and—when enabled—the playbook.Game-specific variation is carried by the description, action vocabulary, and format instructions.Source: E Game-Play Prompt · p. 15
F Reflection Prompt
- SPARTAN’s reflection model reads full self-play trajectories and outcomes to write a first-person, transferable strategic playbook; later rounds revise the prior playbook using new games.The prompt asks for actionable, broadly applicable lessons without game-number references, then prepends the playbook to the system prompt for future games.Source: F Reflection Prompt · pp. 15–16
G Reflection Examples
- The appendix illustrates how reflection content changes across iterations using excerpts from GPT-5-mini and GPT-5 Werewolves playbooks.Source: G Reflection Examples · p. 17
G.1 GPT-5-mini Werewolves Playbooks
Cross-play role gaps are large and usually favor minority/deceptive roles, but their signs can mix structural role advantages with differences in model capability. The Werewolves playbooks preserve an initial strategy while adding tactical refinements across reflection rounds.
G.1 GPT-5-mini Werewolves Playbooks
- GPT-5-mini’s Werewolves playbook keeps its structure across rounds, refining a consistent persona, truthful anchors, calibrated deception, and later tactical safeguards.R2 adds an exit plan if exposed, R3 pairs claims with contextual detail, and R4 proposes a verifiable trade when bluffing a role.Source: §G.1 · p. 17
G.2 GPT-5 Werewolves Playbooks
- GPT-5’s playbook shifts from persona consistency toward detecting agenda and scoring claims against timing, specificity, rationale, and public events.The later guidance contrasts talk with actions such as vote timing and recommends checking whether claims fit game mechanics.Source: §G.2 · pp. 17–18
I.1 Cross-play role gaps
- |∆| exceeds 100 Elo in 29 of 42 model-game cells, with a median gap of 194; the mean ∆ is +73 Elo, favoring the minority/deceptive role overall.Five of six games share a direction across models with non-negligible gaps; Resistance is the exception.Source: §I.1 · p. 19
- Cross-play role gaps combine intrinsic game asymmetry with opponent capability differences, so their signs alone do not identify a game’s intrinsic role balance.The appendix uses same-model self-play to separate the capability-gap confound.Source: §I.1 · p. 20
I.2 Separating role balance from capability
Same-model self-play holds capability constant, revealing how cross-model role gaps can be inflated by opponent strength. The appendix also reports parroting and mixed SPARTAN transfer outcomes across games and models.
I.2 Separating role balance from capability
- +66pp: GPT-5-mini’s Werewolves alt-side win rate is 89% in tournament play versus 23% in self-play; weaker models show negative ∆self on average.Same-model self-play holds capability constant, making its alt-side rates the cleaner role-balance estimates; Chameleon is near a 100% ceiling and excluded from later alt-direction analyses.Source: §I.2 · pp. 20–21, Figure 9 · p. 21
J.1 Parroting Effect (Qwen2.5-3B)
- Qwen2.5-3B often paraphrases the preceding speaker without adding evidence, a pattern the authors say likely contributes to its last-place Overall Elo.The appendix notes that selected episodes are representative and that counterexamples exist.Source: §J.1 · pp. 21–22, J Qualitative Trajectory Examples · p. 21
K.1 Iterated Reflection on Werewolves
- R1 raises the Werewolf win rate from 37% to 70% (+33pp), while R2 raises the Villager rate from 63% to 87% (+24pp) and returns wolves to baseline.The playbook was evaluated against vanilla GPT-5, with 30 games per condition.Source: §K.1 · p. 22, Table 8 · p. 22
K.2 Cross-Game Transfer to a Held-Out Game
- On held-out Resistance, GPT-5’s reflection raises the team win rate from 37% to 47% (+10pp), while the Spies side is unchanged.Transfer to other models varies: Gemini 3.1 Pro transfers cleanly, whereas GPT-5-mini’s effects include sign flips and regressions against GPT-4o.Source: §K.2 · pp. 22–23, Table 9 · p. 23
K.3 Distillation Across Model Strengths
- Qwen3-32B’s Resistance Spies gain +13pp against GPT-5 but lose −13pp against GPT-4o, showing opposite transfer effects by opponent strength.The authors offer overfitting to GPT-5’s playstyle and misfiring against weaker opponents as hypotheses; Gemini transfers without sign-flips.Source: §K.3 · p. 23
K.4 Qualitative Observation: Multi-Level ToM in GPT-5 Self-Play
GPT-5 self-play trajectories include reasoning about others’ private information, counterfactual incentives, and audience-facing deception. In the Qwen3-32B results, reflection effects vary by game and transfer direction, with consistent Resistance-side patterns across multigame sources.
K.4 Qualitative Observation: Multi-Level ToM in GPT-5 Self-Play
- In GPT-5 self-play, agents coordinate by intersecting private information, infer incentives through counterfactual behavior, and model audiences when accusing or defending.The examples span Resistance endgames and Werewolves, including a wolf’s tailored denial and a villager’s independent diagnosis.Source: §K.4 · p. 23
L.1 Common-Suspect Coordination from Intersected Private Information (Resistance)
- Two Resistance agents independently derive Jacob and Melissa as a mutually trustworthy pair by intersecting their private suspect sets.The paper relates this move to common-knowledge-of-rationality reasoning; the game rules do not prescribe it.Source: §L.1 · p. 24
L.2 Counterfactual-Revealed-Preference Accusation (Resistance)
- A Resistance accusation uses Gary’s proposal pattern under a counterfactual spy partnership to judge which partnership is more likely.The inference contrasts observed proposals with those expected under Gary’s alternative-faction incentives.Source: §L.2 · p. 24
L.3 Constructed Deniability vs. External Diagnosis (Werewolves)
- In one Werewolves R4 game, the villager diagnoses partner shielding before the wolf offers a tailored benign explanation for the same evidence.Both responses model how the audience may interpret the behavior.Source: §L.3 · p. 24
M.1 Within-game iterated reflection (R1–R4)
- On Qwen3-32B’s Prisoner’s Dilemma, R1 reaches 58% win rate by prescribing late defection after cooperative rounds, while R4 regresses to 30%.The authors characterize iterated reflection in this setup as non-monotonic.Source: §M.1 · p. 25
M.2 Cross-game transfer (1 →n)
- Every single-game source playbook helps the cooperative Resistance team and hurts the deceptive Spies team symmetrically.The authors attribute this pattern to Resistance’s vote-mechanic insulation.Source: §M.2 · p. 25
M.3 Multigame source →held-out / in-distribution (n →1)
- Across multigame sources, Resistance shows Spies↓/Resistance↑ in all three source conditions; wcsu reaches +27 on the Resistance main side.That main-side result exceeds wcs (+20) and wcsur (+10).Source: Table 12 · p. 25
M.4 Verbatim parroting examples
Resistance gains despite repeated dialogue echo because its mission-vote actions remain independent, while null effects in several free-discussion games coincide with parroting pathologies. Student distillation is mixed across games, and the cost appendix provides approximate API expenditure estimates.
M.4 Verbatim parroting examples
- Mean 5-gram Jaccard overlap is 0.27–0.49 on Undercover, 0.05–0.09 on Chameleon, and 0.06–0.13 on Resistance; PD has no chat phase.Higher overlap indicates more paraphrasing of the prior speaker rather than independent content.Source: §M.4 · p. 25
- Null effects in Werewolves, Chameleon, Spyfall, and Undercover coincide with identity-copying, leaked-word accusation parroting, and repeated generic templates.Examples include a Citizen copying a Werewolf’s self-incriminating line, four Chameleon players repeating an accusation, and a generic Undercover template appearing 19 times in 42 turns.Source: §M.4 · pp. 25–26
- Resistance shows the largest multigame transfer effect: Spies fall from 87% baseline to 57–70%, Resistance rises 13%→23–47%, and free-discussion targets stay near baseline.Resistance can still gain despite echo because mission-vote cards are private actions rather than free dialogue.Source: Figure 11 · p. 26, §M.4 · p. 26
M.5 Distillation to student models
- In PD, Qwen2.5-3B’s R-armed Player_A side rose from 7% to 77% and Gemma3-27B’s from 0% to 77%; Resistance distillation was mixed.Table 13 summarizes PD gains of +17 to +77pp across all three students, while Resistance has one slight positive and two negative outcomes.Source: §M.5 · p. 26, Table 13 · p. 27
- The playbook’s load-bearing content must map to a decision channel the student can execute.The passage identifies this as the pattern behind differing student outcomes.Source: §M.5 · p. 27
N Compute and API Cost
- Table 14 provides approximate API expenditures by experiment phase, based on per-token prices at run time; local-GPU open-weights costs are excluded.The estimates are intended to help future work budget similar experiments.Source: N Compute and API Cost · p. 27, Table 14 · p. 28