PaperlayerView exact revision on arXiv

Source-linked AI summary

Is Anyone Enforcing the Cartel? What Reinforcement Learning Teaches LLM Agents in Markets

Hieu Tran, Hong Yu

arXiv:2610.28787v1cs.CLcs.GTcs.LG

TL;DR

Outcome indices in LLM markets cannot reveal whether cartel-like behavior is enforced by punishing deviations. The paper audits enforcement in counterfactual market copies and studies RL-trained and prompted agents. Small RL-trained LLMs learn what rivals reward but only what they readily try; prompted LLMs’ cartel-like outcomes mostly arise from rules that ignore the market.

  • Problem

    Outcome indices place behavior between competitive and collusive benchmarks but cannot show whether rivals would punish a deviation, leaving enforcement uncertain.

    Source: §1 · pp. 1–2

  • Method

    The audit forces a one-period deviation in a cloned market and compares the rival’s response with the undeviated copy, using shared sampling seeds.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    In runs with small open models, RL-trained LLMs learn what training rivals reward but only what they readily try, while prompted LLMs rarely punish and mostly reach cartel-like outcomes through rules that ignore the market.

    Source: §1 · p. 2, §7 · p. 8

  • Takeaways & Limitations

    Deterred agents and agents that match cuts can earn equal cartel profits against their copies, but only matching agents enforce them, thinly and as a by-product.

    Source: §1 · p. 2, §7 · p. 8

    The study covers open-weight models up to 120B parameters and four OpenAI API models; it does not test other providers’ frontier models or agentic tool-using scaffolds.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Evaluations of large language model (LLM) agents report that pricing agents collude at others' expense. Yet collusion is defined by enforcement: a cartel, in which competitors share monopoly profits, holds only if a member who cheats is punished. We introduce an audit that tests enforcement by forcing a member to cheat in counterfactual copies of a market, and use it to study what reinforcement learning (RL) teaches agents in pricing and trading. Small LLMs trained by RL learn what their training rivals reward, but only what they readily try. In self-play, they learn fixed rules ignoring their rivals; against a punishing rival, to be deterred; against a cheating rival, to match its cuts, never the costly price war that would deter it, which they rarely try. A war forced into training is learned, but mostly only where it was forced. Against their own copies, deterred and matching agents can earn the same cartel profits, but only matching ones can enforce them, and then thinly: as a by-product and only against large cuts. Q-learners enforce collusion only in pricing. Prompted LLMs reach cartel-like outcomes mostly through rules that ignore the market. Outcome measures cannot tell these cases apart; the audit can.

1 Introduction

Outcome indices can place market behavior between competition and cartel benchmarks, but cannot establish whether a cartel is enforced. The paper introduces a counterfactual audit and uses it to study what reinforcement learning teaches market agents.

1 Introduction

  • A cartel requires punishment that makes cheating unprofitable, so prices or profits alone cannot establish enforcement.Observed price responses and written reasoning can also mislead: best-responding firms may move together, and stated reasoning need not drive actions.Source: §1 · pp. 1–2
  • The audit forces a one-period deviation in a cloned market and compares the rival’s response with the undeviated copy using shared sampling seeds.Placebos, scale sweeps, and positive controls test whether apparent collusion reflects strategy and whether known punishers are detected.Source: §1 · p. 2
  • The study audits tabular Q-learners, RL-trained LLMs, and prompted LLMs in a pricing game and the Kyle trading market.The testbed uses exact benchmarks and places LLMs in the Kyle market, alongside the pricing game of prior work.Source: §1 · pp. 2–3
  • In runs with small open models, RL teaches agents what training rivals reward, but only what agents readily try.LLMs learn fixed rules in self-play, become deterred by a punishing rival, or match a cheating rival’s cuts; they do not learn a deterring price war unless it is forced into training.Source: §1 · p. 2

2 Markets and Collusion

The section distinguishes collusive outcomes from the strategies that sustain them and introduces benchmarked trading and pricing settings in which the distinction can be tested.

2 Markets and Collusion

  • In the Kyle trading market, a trader’s intensity trades off higher earnings against price impact, with cartel intensity below the competitive level.At market depth s = 1, each duopoly trader’s intensity is 0.71 at Nash and 0.53 in the cartel; cartel profit is 9/8 of Nash profit.Source: §2 · p. 3
  • An outcome between Nash and cartel benchmarks is collusive only if a deviation is met with a response that makes it unprofitable.An outcome index locates play between benchmarks but does not reveal whether rivals enforce it.Source: §2 · p. 3
  • Trading-market scale invariance keeps equilibrium indices fixed as noise volume changes, whereas depth-insensitive fixed-order rules can appear to span competition and collusion.The latter rule’s index rises from −∞ to 2I/(I −1) as the market deepens.Source: §2 · p. 3
  • In the pricing game, currency rescaling leaves equilibrium indices unchanged, while a best-responding firm follows a rival’s price cut by 0.29–0.44 per unit.This baseline matters because a rival can respond to a cut without enforcing collusion.Source: §2 · p. 3

3 The Audit

The audit tests enforcement by comparing play after a forced deviation with an undeviated counterfactual, then checks whether apparent strategic behavior survives controls and scale changes.

3 The Audit

  • The paired deviation test clones a saved market, forces a one-period deviation, and compares the rival’s response and the deviator’s payoff with the undeviated copy.Copies share histories and future random draws; the deviation is a best response or a visible aggressive shift, such as a 10% price cut.Source: §3 · p. 4
  • Placebos show outcomes when agents cannot collude, including lone traders, myopic objectives, and prompts with or without a real rival..Source: §3 · p. 4
  • Scale sweeps vary market depth, currency units, and trader counts to expose rules that do not preserve equilibrium indices under rescaling.A sweep can expose a rule, but cannot by itself certify a strategy.Source: §3 · p. 4
  • Positive controls test whether the audit detects punishment by instructed LLMs and pricing-game Q-learners.The pricing-game Q-learners are included because punishment is documented in prior work.Source: §3 · p. 4

4 Experimental Setup

The experiments compare tabular Q-learners, RL-trained LLMs, and prompted LLMs in trading and pricing settings, using repeated sessions and deviation tests. Outcomes and responses are reported with uncertainty intervals and agent-level audit verdicts.

4 Experimental Setup

  • The setup includes tabular Q-learners, prompted trading and pricing LLMs, and Qwen3 and Llama models trained by RL in self-play or against scripted rivals.RL credits actions with discounted return-to-go; with γ > 0, that credit includes later rival responses, which learning to punish requires.Source: §4 · p. 4
  • Each LLM condition runs 6–40 sessions of 60–200 periods, scores the second half, and includes one to three deviation events per session.Gains compare the deviator’s discounted profit with its copy’s over the deviation period and subsequent periods.Source: §4 · p. 4
  • The audit table reports market or price indices, profit indices, rival responses, deviation payoffs, and verdicts, with 95% confidence intervals.Verdicts distinguish punishing agents, deterred agents, and anchoring behavior.Source: Table 1 · p. 5
  • For trained policies, the seed is the replication unit because the policies are nearly deterministic.Outcome intervals use Student-t across sessions, while deviation-test intervals are normal across events.Source: §4 · pp. 4–5

5 Results

The results show that RL-trained agents can reach cartel-like outcomes without learning to deter cheating, while enforcement appears only in some matching agents and is thin. Q-learners punish in pricing but not trading, and prompted LLMs mostly reach cartel-like outcomes through market-ignoring rules.

5.1 RQ1: Does the Audit Catch Punishment?

  • Q-learners punish deviations in pricing, but trading Q-learners never elicit a rival response and deviations remain profitable.In trading, forward-looking Q-learners close 0.86 of the Nash-to-cartel gap, yet deviation pays 23% of a period’s Nash profit.Source: §5.1 · p. 5, §5.2 · p. 5

5.2 RQ2: What Does Reinforcement Learning Teach?

  • In self-play, GRPO learns fixed pricing markups and value-based trading rules that ignore rivals, even when trading profits reach cartel levels.Pricing agents’ copies do not cheat systematically, so self-play does not tie their prices to the rival’s behavior.Source: §5.2 · pp. 5–6
  • Against tit-for-tat, forward-looking agents keep prices near monopoly levels but do not punish cuts when paired with themselves, so cheating pays.The copies earn 1.50 times Nash profit, yet a 10% cut pays +0.39±0.00.Source: §5.2 · p. 6
  • Cheater-trained GRPO agents match rivals’ cuts rather than learn deterrent wars; some self-paired copies enforce high-price outcomes, but only thinly and mainly against large cuts.Cuts up to 5% can still pay, while 20% cuts lose; 10% cuts are near the enforcement threshold.Source: §5.2 · p. 6
  • With a war price shown in the prompt, most policies copy it, often posting it unprovoked rather than using it to answer a cheat.Training with the shown price can instead lead the policy to undercut from the start and settle at Nash.Source: Figure 4 · p. 7, §5.2 · p. 7
  • No LLM run learns a price war unaided; forced war responses teach it, mostly in states like those where they were forced.Wars left out of the policy update, or forced when no cheat occurred, do not teach the war; three forced wars in 100 also sufficed in some runs.Source: Figure 3 · p. 6, §5.2 · p. 7

5.3 RQ3: What Do Untrained LLMs Do in Context?

  • Untrained LLM traders do not punish; their cartel-like indices arise from rules that ignore the market, including weak scaling of orders with market depth.The reported elasticity is 0.03–0.34, against one for the optimum.Source: §5.3 · pp. 7–8
  • Among prompted pricing agents, only GPT-5.2 punishes, and it reaches cartel-like prices in just 2 of 8 sessions; agents’ written threats also often fail to match behavior.GPT-5.2’s mean profit was not significantly above Nash; traders mostly proposed acting together but did not threaten or answer deviations.Source: §5.3 · p. 8

6 Related Work

Related work studies LLM agents in games and markets, including reported collusion, and examines collusion through outcomes, regret, or reasoning. This paper focuses instead on testing enforcement, while situating its contribution alongside work on algorithmic collusion and reinforcement learning.

6 Related Work

  • Prior work evaluates LLM agents across games, negotiation, coordination, and markets, reporting collusion in pricing, auctions, market division, and communication.Studies also examine regret, monitoring of chain-of-thought, and how market outcomes change when agents differ.Source: §6 · p. 8
  • Rather than screen outcomes or read regret or reasoning, the paper tests the enforcement that defines collusion, using closed-form benchmarks and several prompts (Sclar et al., 2024; Mizrahi et al., 2024).Source: §6 · p. 8
  • Research on learning algorithms links Q-learning to pricing reward–punishment schemes and trading trigger strategies, while debating how learning and deviation responses shape collusion.Other work studies imperfect monitoring, over-pruning in trading, and opponent-aware reinforcement learning.Source: §6 · p. 8

7 Conclusion

The conclusion argues that outcomes alone cannot establish cartel enforcement and that counterfactual deviations provide a more direct test. Across the studied agents, high-profit outcomes can coexist with weak or absent punishment.

7 Conclusion

  • Outcome measures cannot show whether an agent enforces a cartel; the audit tests enforcement through counterfactual copies.Source: §7 · p. 8
  • RL-trained agents learn fixed rules, deterrence, or cut-matching depending on training, but the deterring price war is rarely tried unless forced.Forced wars are learned mostly in the states where they were introduced.Source: §7 · p. 8
  • Deterred and matching agents can earn equal cartel profits, but only matching agents enforce them, and then thinly as a by-product.Prompted LLMs mostly reach cartel-like outcomes through rules that ignore the market.Source: §7 · p. 8

Limitations

The study’s conclusions are bounded by its stylised markets, model and prompt coverage, and a deviation test that samples only particular departures from play. Training runs are also short and small, while strategy findings can vary across seeds and prompts; the analysis of model-written notes is secondary.

Market scope and prompts

  • The Kyle market omits order books, inventories, multiple assets, and heterogeneous horizons, and the fixed market-maker rule makes any trading-cartel gain come at the market maker’s expense.Punishment findings are specific to the two markets and prompts; negotiation, longer memories, or explicit coordination devices may enable collusion not observed here.Source: Limitations · p. 9

Deviation-test scope

  • The audit tests one-period deviations; punishment triggered only by persistent or larger deviations may escape detection, although larger shifts and perfect-monitoring runs mitigate this concern.Shared-seed copies are identical before deviation only for backends that honor seeds; OpenAI API seeds are best-effort.Source: Limitations · p. 9
  • The test’s intervals use multiple events per session, which are not independent, and cloning is required; matched-market or sandbox audits would be needed when deployed agents cannot be copied.Treating each session as one observation leaves positive-control results significant except two best-response tests, and makes no null response significant.Source: Limitations · p. 9

Training scope

  • RL experiments train 0.6B–4B models with LoRA for at most a few hundred iterations, versus millions of periods for Q-learners, so larger models or longer and different training may reach unobserved strategies.No LLM run learned unaided the price war; tabular Q-learners found it in 49 of 50 runs with thirty times the experience.Source: Limitations · p. 9
  • Enforcement findings are seed- and prompt-sensitive: forced-war generalization varies by seed, few-seed counts are coarse, and adding a price to the prompt can replace the learned policy.The study’s note-and-trace coding is secondary; the judge’s labels for inference about the rival and scaling down are weak, while punishment conclusions rest on deviation tests.Source: Limitations · p. 10

Ethical Considerations

The paper flags dual-use risk because an audit that detects collusion can also tell designers what auditors may look for. It argues the studied setting is limited and its findings are not evidence that deployed trading systems are safe.

Dual use

  • The audit may reveal what auditors look for, while some simulated agents learn to answer price cuts in ways that can sustain collusion between their copies.The trained agents are small models facing scripted rivals in simulated markets, and are studied to understand training rather than for deployment.Source: Ethical Considerations · p. 10

Interpretation

  • The findings neither show that LLM agents cannot collude nor establish that deployed trading systems are safe; outcome-based evidence is insufficient in either direction..Source: Ethical Considerations · p. 10

Data and research practice

  • The study uses no human participants or personal data, and releases code and data under the MIT licence for research on evaluating multi-agent systems.Experiments ran on shared GPUs, and OpenAI API usage cost about $245.Source: Ethical Considerations · p. 10

A Theory

The theory shows how changing noise volume rescales trading without changing prices or collusion indices, and derives when punishment can sustain a cartel. It also examines how market depth and prompt wording relate to traders’ behavior.

A Theory

  • Scaling noise volume by k scales orders and flow by k and price impact by 1/k, leaving prices unchanged and preserving equilibrium correspondence and collusion indices.This scale invariance applies to static Nash and history-dependent punishment equilibria.Source: A Theory · p. 14
  • At s=0.5, 1, and 2, Qwen3-8B traders’ intensities are 0.80, 0.91, and 0.93; their collusion indices are −5.10, −1.16, and +1.37.A constant b=0.88 instead gives indices −6.0, −1.0, and +1.5 under the frozen-impact formula.Source: A Theory · p. 15, Figure A.1 · p. 15
  • A level-1 trader facing one level-0 rival orders the cartel quantity, but facing at least two such rivals orders nothing.A level-0 trader orders as if alone; a level-1 trader best-responds to rivals expected to be level-0.Source: A Theory · p. 15
  • A grim-trigger strategy sustains the cartel when the discount factor exceeds 0.529 for two traders or 0.571 for three, at every depth.The learners’ γ=0.9–0.95 exceeds both thresholds when deviations are observed.Source: A Theory · p. 15

B Prompts

  • Across four prompt wordings, lone-trader intensity levels move, while depth elasticity remains far from one.The test uses greedy decoding and states the market maker’s price rule in the prompt.Source: Table B.1 · p. 16
  • Δ=0.79 and profit index 0.48 ± 0.75 for Qwen3 cartel-level control traders indicate partial compliance; Qwen3-32B keeps cartel orders (Δ=1.03; profit index 0.97 ± 0.66).The instructed cartel-level control’s weak response to a best-response deviation still leaves that deviation profitable; the Qwen3-32B response makes it unprofitable.Source: B Prompts · pp. 16–17

C Models, Runs and Compute

This section examines enforcement in counterfactual cloned markets and how training conditions shape whether agents learn costly price wars. It covers one-period deviations, forced-war training, learning across seeds, and rival-cheating designs that change the gains from war.

C Models, Runs and Compute

  • The audit clones the market after five ordinary periods, forces one trader to deviate for one period, and measures discounted profit gains and rivals’ response intensity; GRPO’s gradient vanishes when group samples agree.The deviation gain is calculated over the horizon with γ = 0.95; identical group returns can leave learned habits absorbing even when rival behavior changes.Source: C Models, Runs and Compute · p. 17, D Reinforcement-Learning Details · p. 18

D.1 Self-Play in the Pricing Game

  • In pricing self-play, trained policies settle on fixed markups and barely follow a 10% cut, unlike a best-responding rival.For the Qwen3-0.6B forward-looking policy, cut response is 0.00 ± 0.00 per unit, versus 0.41 for a rival that only best-responds at the same prices.Source: §D.1 · p. 19

D.2 Against Tit-for-Tat

  • Against tit-for-tat, the forward-looking Qwen3-0.6B policy posts twice its cost, but its copies do not respond to cuts and both tested deviations pay.The best-response deviation gains +0.47 ± 0.00 of a period’s Nash profit, and a 10% cut gains +0.39 ± 0.00.Source: §D.2 · p. 20

D.3 Against a Rival That Cheats

  • Against the cheating rival, GRPO agents mostly match cuts rather than wage the costly war, although sampled wars would earn positive advantages early in episodes.The war costs 27% of a period’s Nash profit; the policy samples it in at most 1% of answers, while a sampled war’s return advantage is +1.0 Nash profits in periods 3–6.Source: §D.3 · p. 20, §D.4 · pp. 20–21

D.5 A Price Shown in the Prompt

  • Adding “Reference price: 1.34.” anchors trained and untrained policies to that price, so showing the war price makes policies post it unprovoked rather than answer a cheat.The untrained Qwen3-0.6B opens at exactly 1.34 in 51% of first periods; most tested trained Qwen3-0.6B policies open at it in at least 97%.Source: §D.5 · p. 21

D.6 A Learner That Tries Every Price

  • With γ = 0.9, Q-learners wage wars in a median 28% of cheat responses and earn 1.39 times Nash; paired copies enforce a premium in 10 of 50 pairs.At the shorter 100,000- or 300,000-period budgets, only one learner in 50 wages the war, and most pairs settle near Nash.Source: §D.6 · p. 22

D.7 PPO

  • PPO without forced wars, like GRPO, matches the cheat and samples few wars; PPO with forced wars learns to wage them, with 58% and 56% of answers in two seeds.In the first unforced PPO seed, copies follow a 10% cut in full and the cut loses −0.83 ± 0.12; forced-war PPO copies show seed-dependent enforcement.Source: §D.7 · pp. 22–23

D.8 Forcing the War

  • Forced-war training teaches Qwen3-0.6B, Qwen3-4B, and Llama-3.2-1B to answer cheats with wars, but copied policies often fail to preserve cartel prices or deter small cuts.Across seeds, wars can become rules tied to a particular price or period, or trigger price declines between copies rather than sustain high prices.Source: §D.8 · p. 24
  • Randomizing forced-war prices taught a war in two of three seeds, but learning was delayed or absent despite frequent war draws.One seed began waging wars around iteration 160 after about 1,100 draws, another never learned within 200 iterations despite about 1,400 draws, and the third learned late. (D.8 Forcing the War)Source: §D.8 · p. 25
  • At twice-cost prices, agent copies followed 10% cuts in full, while a late 20% cut at that price was matched rather than punished.Across the three seeds, following the 10% cut paid slightly, neither paid nor lost, or paid, depending on the seed. (D.8 Forcing the War)Source: §D.8 · p. 25

D.9 Other Designs of the Cheating Rival

  • Cheating-rival variants changed war incentives: at cheat probability 0.3, war earned only 2% more than matching; specified 0.8 variants made war earn 28% or 19% more than Nash.In the 28% case, matching earned 16% more than Nash; with 20–40% cuts and 14 periods of deterrence, every answer short of war earned less than Nash. (D.9 Other Designs of the Cheating Rival)Source: §D.9 · p. 25
  • Despite larger war premiums, the policy learned wars no more readily: it matched the first variant’s cheats but priced at Nash against the other two, giving up cartel gains.After 200 iterations against the first variant, it waged wars in 0% of answers; paired with itself, copies followed 10% cuts that lost −1.47 ± 0.07 and drifted down from twice cost. Against the other variants, it priced at Nash from about iteration 20, and against the second it also forwent the 16% matching premium. (D.9 Other Designs of the Cheating Rival)Source: §D.9 · p. 25

D.10 The Trained Policies Paired with Themselves

Across trained LLM pricing agents, matching a rival’s cuts can sustain high prices but often deters only large deviations; training against a punisher teaches deterrence, not enforcement. Results vary across seeds and models, while cross-play attributes enforcement to the cheater-trained policy and Q-learners in trading fail to punish deviations.

D.10 The Trained Policies Paired with Themselves

  • Forcing a price war into Qwen3-0.6B training tests whether it learns the response beyond the circumstances where the war was forced.The figure compares credited and uncredited forced wars, forcing a war when no cheat occurs, random-price controls, and training without forced wars.Source: Figure D.3 · p. 26
  • In four of nine Qwen3-0.6B seeds, matching a 10% cut makes it unprofitable, but smaller cuts still pay, so enforcement is thin.The cut’s gain is +0.395 of a period’s Nash profit and matching costs 0.434 the next period; the pair’s price is above monopoly, so a 2% cut pays.Source: §D.10 · pp. 25–26
  • With every sample in every group agreeing, the first seed’s policy gets no reward signal beyond entropy bonus; its copies post twice their cost and copy a cut at once, but the echo fades: a 10% cut gains +0.08 ± 0.02 over nine periods, while a best-response deviation loses −0.33 ± 0.02.Self-play does not reinforce the enforcement it was given.Source: §D.10 · p. 27

D.11 Other Model Sizes and a Second Family

  • Across Qwen3-4B and Qwen3-1.7B seeds trained against a cheater, agents match cuts, but paired outcomes range from near Nash to premium prices, with mixed deterrence.For Qwen3-4B, one seed’s cut pays (+0.17 ± 0.02) while another’s loses (−0.15 ± 0.12); Qwen3-1.7B pairs drift near Nash and do not hold a cartel premium.Source: §D.11 · p. 27
  • Llama-3.2-1B seeds trained with a format penalty match cheats, but outcomes differ: one pair’s cuts trigger lasting price responses, while another barely responds.The second and third seeds hold premiums and enforce; the fourth settles near Nash and has no premium to enforce.Source: §D.11 · p. 28

D.12 Cross-Play

  • In cross-play, the cheater-trained firm matches a 10% cut and makes it lose, while the tit-for-tat-trained firm ignores the cut and it pays.Both firms post twice their cost; the audit attributes enforcement to the cheater-trained firm.Source: §D.12 · p. 28

D.13 A Punishing Rival in Trading

  • In trading, forward-looking Q-learners reach near-cartel intensity against the scripted punisher but do not respond to profitable deviations; myopic learners also fail to punish.The forward-looking policy reaches Δ = 0.95, yet a best-response deviation gains +0.06 ± 0.05 Nash profits.Source: §D.13 · p. 29

E Tabular Q-Learning Traders

Tabular Q-learners in trading can reach collusive outcomes without enforcing them. In the tested low-noise market, forward-looking learners do respond to deviations, but those deviations remain profitable.

Method

  • The study uses tabular Q-learning over trader memory, private value, and a 31-point order grid, then evaluates greedy policies after training.The market maker continues adapting during 20,000 evaluation periods.Source: E Tabular Q-Learning Traders · p. 31

Results

  • In the standard-noise market, forward-looking learners reach Δ = 0.86 ± 0.01, but a best-response deviation pays and draws no rival response.The response is −0.000 ± 0.003 and the deviation gains 0.053 ± 0.003; myopic learners reach Δ = 1.24 with no response, while counterfactual updates yield Δ = −0.10 ± 0.01.Source: E Tabular Q-Learning Traders · p. 31
  • In the low-noise market, forward-looking learners respond by 0.41 ± 0.15 in intensity, but the deviation gains 0.57 ± 0.05.The response is positive in all five seeds and significant in three; myopic learners respond by −0.02 ± 0.09.Source: E Tabular Q-Learning Traders · p. 31
  • With no rival and γ = 0, standard Q-learning under-trades by about 0.91√α, while counterfactual updates learn the optimal order at every step size.The shortfall shrinks as the discount factor rises.Source: Figure E.1 · p. 32

F Full Results

The full results show that prompted LLM trading often follows fixed rules rather than market conditions, while uninstructed agents show little evidence of punishment. In pricing, deviations and symmetric tests distinguish matching or price-following from responses that actually deter cuts.

F Full Results

  • Most tested reasoning and non-reasoning LLM traders do not adapt trading intensity to market depth, and the resulting duopoly index shifts from far below Nash to beyond the cartel.Reasoning models scale intensity with depth; GPT-5.5 alone reaches Nash in the tested markets.Source: F Full Results · p. 32, Figure F.2 · p. 33
  • Telling traders the market maker’s price rule barely improves their response to market depth.Elasticity changes from 0.09 to 0.13 for Mistral-7B and from 0.15 to 0.21 for Qwen3-8B.Source: F Full Results · p. 32
  • Across 140 uninstructed-trader deviation tests, responses are generally weak; instructed traders respond more strongly when they see rivals’ orders, but much less when they see only order flow.Under the worst-case dependence adjustment, no uninstructed response remains significant; 5 of 7 significant instructed responses remain significant.Source: F Full Results · pp. 32–33, Figure F.1 · p. 32, Table F.1 · p. 34
  • Qwen3-8B’s trading aggression follows prompt wording more than the presence of a rival.Being told others may be informed raises intensity by 0.30 ± 0.16, whereas an actual rival changes it by −0.07 ± 0.15.Source: F Full Results · p. 33, Table F.2 · p. 34

G The Pricing Game

  • The pricing game evaluates deviations by comparing rival price changes with the static best-response benchmark, which follows a cut by 0.29–0.43 per unit.The test also measures the deviator’s discounted profit gain; punishment requires a response beyond best response that makes deviation unprofitable.Source: G The Pricing Game · p. 37
  • Forward-looking Q-learners in pricing respond to both price cuts and rises, indicating a response to departures from the learned path rather than undercutting alone.Myopic Q-learners respond to neither; Mistral-7B’s responses in both directions remain within the best-response benchmark.Source: G The Pricing Game · p. 37, Table G.2 · p. 38

H The Protocol of Fish et al.

The GPT-5.2 protocol tests whether observed pricing patterns reflect enforcement by forcing cuts in cloned markets. GPT-5.2’s rivals often matched cuts, but on-path outcomes and matching alone did not establish punishment.

H The Protocol of Fish et al.

  • The audit reproduces Fish et al. (2026)’s pricing agents using their Appendix G prompt template and screens gpt-oss-20b, which achieves a 0.00 mean gap to the 1.66 best response and 100% of its profit.It audits gpt-oss-20b in 20 sessions of 150 periods, scored on the second half, using Appendix G deviation tests. The two sessions were chosen after the fact for their prices, and events within a session are not independent.Source: H The Protocol of Fish et al · pp. 38, 40
  • At high reasoning effort, gpt-oss-20b priced near Nash (index 0.11), yet its rival followed a 10% cut by only 0.05 ± 0.18 per unit, below the 0.42 best-response benchmark.Its on-path coefficient was 0.11, the same as GPT-4’s, illustrating that on-path coefficients need not reveal off-path responses.Source: H The Protocol of Fish et al · p. 39
  • A 10% cut drew a 1.01 ± 0.11 per-unit rival price fall; the deviator’s gain was −0.49 ± 0.62 Nash profits, not significantly different from zero.Afterward, the rival’s price fell by 0.49–0.83 per unit; the paired-copy responses before deviation were close to zero, though seeds were only approximately honored.Source: H The Protocol of Fish et al · p. 39
  • Six of eight GPT-5.2 sessions earned more than Nash, but mean profit was 1.14 times Nash and not significantly above it.Sessions varied: two settled at cartel-like prices, one above monopoly, two low, one asymmetric, and two were still falling at period 100.Source: H The Protocol of Fish et al · p. 39, Figure H.1 · p. 40, Table H.2 · p. 41
  • In GPT-5.2’s cartel-like sessions, rivals matched every tested cut, but session 4’s myopic placebo also followed cuts near the best-response amount and returned quickly.The forward-looking model in the same state matched each cut in full; matching therefore did not alone show punishment.Source: H The Protocol of Fish et al · pp. 39–41
  • On-path cut-versus-rise response differences were not significant with eight sessions, and the myopic placebo leaned in the same direction.The off-path tests distinguish behavior that on-path asymmetry could not settle.Source: H The Protocol of Fish et al · p. 41

I Coding Notes and Reasoning Traces

The section checks what agents’ notes and reasoning traces say against coder validation and behavioral evidence. The coding supports the absence of cooperation and punishment in uninstructed traders, while some labels are less reliable than others.

I Coding Notes and Reasoning Traces

  • Notes and traces indicate what agents attend to, not what causes their actions, so they serve as checks on the behavioral audit rather than substitutes for it.In 382,700 uninstructed-trader notes and traces, all 48 uses of “punish” referred to the market’s response.Source: I Coding Notes and Reasoning Traces · p. 41
  • GPT-5.4-mini labeled 150 sampled responses per run with six yes/no questions about price impact, rival reasoning, cooperation, punishment, and scaling down orders.Traces longer than 12,000 characters were shortened; GPT-5.4 reasoning was hidden, so only its notes were labeled.Source: I Coding Notes and Reasoning Traces · p. 41
  • The keyword cross-check and judge both found cooperation and punishment absent outside the instructed control, and the authors inspected every keyword match.They agreed on price impact in 78% of cases and on rival reasoning in 90%, with Cohen’s κ of 0.52 and 0.51, respectively.Source: I Coding Notes and Reasoning Traces · p. 41
  • On the 200-item validation sample, the judge had perfect recall against the two stronger-model consensus for cooperation and punishment, but precision was 0.86 and 0.78.The judge tended to flag more cases than the stronger models; its price-impact and rival-reasoning labels had other reliability limits.Source: I Coding Notes and Reasoning Traces · p. 42, Table I.2 · p. 42
  • Human annotators found no cooperation in 75 uninstructed responses, and their labels support the conclusion that uninstructed traders did not cooperate or punish.The judge flagged three uninstructed responses for each label, while the humans differed on only one possible punishment case.Source: I Coding Notes and Reasoning Traces · pp. 41–42
  • Human validation likewise supported the absence of cooperation and punishment, but judge labels for rival inference were lower bounds and its scaling-down label was unreliable.For rival inference the judge’s recall was 0.37; for scaling down, agreement was κ = 0.13 in the human comparison.Source: I Coding Notes and Reasoning Traces · p. 42, Table I.3 · p. 43

J Cheap Talk

The cheap-talk condition adds messages to trading agents’ interaction, then examines message content alongside deviation tests. Messages often proposed coordination, but the texts did not threaten punishment for trading beyond an agreement.

J Cheap Talk

  • The talk condition gave Qwen3-8B and Mistral-7B a message channel alongside orders, with the rival’s previous message visible and messages unable to affect prices or profits.Qwen3-8B was run with rival orders shown and with order flow only; Mistral-7B was run with rival orders shown, each for 40 sessions of 200 periods.Source: J Cheap Talk · p. 43
  • Among judged messages not stating V was zero, Qwen3-8B’s messages often proposed coordination; Mistral-7B rarely did, and its conditional messages were mostly contrarian or taunting.Across four runs, no message used punishment-related terms or threatened a rival for trading more than agreed.Source: J Cheap Talk · p. 43
  • The table reports collusion index, rival response, deviation gain, and message-content shares; a second Mistral-7B seed run had different market results and its messages were not judged.The second run’s reported collusion index was −2.89 ± 1.21, rival response −0.053 ± 0.706, and gain 29.10 ± 16.61.Source: Table J.1 · p. 44

K Implications for Evaluation

The evaluation implications emphasize off-path behavior, appropriate controls, and testing beyond training states. The glossary distinguishes outcome measures from enforcement and defines the audit’s key terms.

K Implications for Evaluation

  • The paper recommends paired deviation tests, benchmark-moving parameter sweeps, non-colluding placebos, checking written plans against behavior, and testing trained agents in unseen states.An agent’s behavior in states its training rivals never produced is not fixed by its training outcome.Source: K Implications for Evaluation · p. 44
  • The audit requires a clonable environment and a benchmark rival that does not punish; without honored seeds it remains usable but yields wider intervals.The authors note that price-reactive rules can raise prices without collusion and call for tests of more frontier models, longer horizons, and agentic settings.Source: K Implications for Evaluation · p. 44

L Glossary

  • The glossary separates outcome indices from enforcement: an index places outcomes between Nash and monopoly, while enforcement additionally requires punishment that makes a deviation unprofitable.It also defines paired deviation tests as comparing cloned markets after forcing a one-period deviation, and defines a price war by its response and deterrent effect.Source: L Glossary · pp. 44–45
Loading 2610.28787v1…