Source-linked AI summary
Competitive Market Behavior of LLMs
Pawel Struski, Jakub Swistak, Inez Okulska, Przemyslaw Biecek
TL;DR
The paper examines whether LLM agents can participate in human-designed market mechanisms while delivering efficient outcomes. It replicates Smith-style double-auction experiments with LLM traders and finds incomplete, uneven convergence, alongside heterogeneous trading behavior and urgency-related reasoning shifts.
Problem
Evidence is limited on whether LLM agents are suited to human-designed market mechanisms and whether those mechanisms deliver their intended outcomes with LLM participants.
Method
The authors replace human subjects with LLM agents in Smith-style double-auction experiments and analyze market dynamics, individual trading decisions, and reasoning traces.
Results
No tested condition fully converges to competitive equilibrium; convergence is partial and uneven, with GPT Small reproducing the human pattern of steadily declining price dispersion.
Takeaways & Limitations
LLM agents are at best partially aligned with the double auction, so LLMs used as human stand-ins should be validated against human and theoretical benchmarks.
Takeaways & Limitations
Whether incomplete convergence reflects a fundamental limitation or merely a slower path to equilibrium remains open, with longer horizons and richer feedback potentially changing outcomes.
Abstract
from arXiv · showhide
Large language models (LLMs) are increasingly deployed as economic agents, yet there is little evidence whether LLM agents are suited for participating in market mechanisms designed for humans, and whether these mechanisms deliver desired outcomes when faced with LLM agents. We address this question by replicating seminal economic experiments, replacing human subjects with LLM agents. We place agents in a double auction environment, which is a widely-used market mechanism. We check whether such a market is able to deliver an efficient allocation of resources, thereby testing a novel dimension of alignment of LLM agents -- their compatibility with a fundamental market mechanism. We find that markets populated by LLM agents exhibit slower or no convergence towards market equilibrium, thus providing less efficient allocations than markets populated by humans. We then analyze agents' individual trading decisions and find substantial heterogeneity both across model families and market roles. We also run a lexical analysis of Chain-of-Thought (CoT) traces generated by the agents. We find that the decision to execute a trade rather than continue incrementally adjusting prices is associated with a shift from strategic considerations toward urgency. We publicly release our testing framework, which can be used for future evaluations.
1 Introduction
The paper asks whether LLM agents can reproduce the convergence and efficiency that humans achieve in double auctions. It frames this as compatibility between LLM behavior and a foundational market institution, and introduces a reusable experimental framework to test it.
- The study asks whether autonomous LLM traders converge to the competitive equilibrium in double auctions as human traders do.
- Double auctions matter because their equilibrium allocates goods efficiently, while failure to converge leaves mutually beneficial trades unrealized.
- The paper defines market-institution alignment as achieving the outcomes that a market institution delivers for humans, and tests this alignment for double auctions.
- The authors replicate Smith-style experiments with LLM agents across model families and capability tiers, analyzing both market dynamics and individual decisions.
- The reusable framework provides measurable human and theoretical benchmarks for studying convergence, behavioral heterogeneity, and reasoning traces in LLM traders.
2 Related Work
Related work studies LLM behavior in economic tasks, market interactions, convergence, collusion, and agent-based modeling. This paper extends that literature by examining general market-institution alignment against human and theoretical benchmarks.
- Research on LLM agents in markets spans individual decisions, emergent multi-agent phenomena, laboratory replications, narrow markets, and bidding strategies.
- Prior work on Smith-style convergence reports that LLM agents do not converge fully, while this study reports partial convergence in some experiments.
- Collusion studies often interpret supracompetitive prices and incremental undercutting as evidence of collusion, but this paper questions that inference when convergence is not guaranteed.
- The paper broadens market alignment beyond collusion to ask whether LLM-populated markets attain the intended allocative efficiency of the institution.
- Unlike traditional ABMs, which treat agents as instruments for studying economies, this study treats LLM agents’ behavior and alignment with double auctions as the object of study.
3 Method
The authors simulate continuous double auctions with privately informed LLM buyers and sellers, using symmetric reservation prices and a bounded sequential trading mechanism. Agents observe local and market histories, seek profit, and trade through bids or asks.
- The framework simulates continuous double-auction markets populated by LLM agents trading a single good.
- Market Environment: Eleven buyers and eleven sellers have reservation prices from $0.75 to $3.25, yielding competitive equilibrium p⋆ = $2.00 and q⋆ = 6 trades per round.
- Market Environment: The equilibrium maximizes total surplus, although agents observe only their own reservation values rather than the full supply and demand schedules.
- Market Environment: The symmetric design makes observed buyer–seller differences attributable to model behavior rather than asymmetry in market schedules.
- Trading Mechanism: Each round randomly selects agents for up to T = 300 iterations, with trades executing when an offer crosses the standing spread and quotes otherwise improving or being rejected.
- Information and Objective: Agents condition decisions on their reservation value, standing bid and ask, market history, and private action history, while maximizing realized trading profit.
- Experimental Setup: The experiments evaluate GPT Large, GPT Small, and Gemini Large across 10 independent runs with different random seeds, and release replication code and data.
4 Results
LLM markets do not fully converge to the competitive equilibrium, although GPT Small shows the clearest partial convergence. Individual trading behavior varies substantially across models and market roles, with incremental rent-seeking and spread-crossing choices shaping market outcomes.
- 4.1 Market-Wide Outcomes: No condition reaches full convergence: allocative efficiency peaks at 0.91, while trade volume falls short of the equilibrium quantity of 6 in every round but one.Price dispersion remains above the human benchmark across all model populations, and only GPT Small reproduces a declining dispersion pattern.
- 4.1 Market-Wide Outcomes: GPT Small shows the clearest partial convergence, with mean prices declining from $2.33 to $2.13, α from 27.6 to 11.7, and efficiency rising from 0.79 to 0.91.In round 5, dispersion is 11.7 versus 3.5 for humans, and mean price is $0.13 above equilibrium versus $0.03 for humans.
- 4.1 Market-Wide Outcomes: GPT Large averages only 2.2–3.8 trades per round and efficiency of 0.36–0.67, as agents repeatedly make $0.01 improvements without closing the spread.The iteration cap may affect transaction counts, but convergence speed remains a substantive outcome in this experiment.
- 4.2 Individual Behavior: Agents overwhelmingly use small $0.01 order improvements, prioritizing profit preservation over transaction completion despite the risk that rounds end before they trade.Small improvements preserve potential rent but are more likely to be undercut.
- 4.2 Individual Behavior: Spread tolerance differs by model: Gemini Large crosses spreads near $0.50, GPT Large waits near zero, and GPT Small buyers cross around $0.25 while sellers mostly wait.These choices reflect different trade-offs between per-trade profit and completing transactions.
- 4.2 Individual Behavior: GPT Small exhibits strong role asymmetry: over 90% of ask improvements are $0.01, while nearly 40% of bid improvements exceed $0.25.Sellers submit roughly three times as many orders as buyers, whereas buyers’ larger improvements are more likely to transact.
5 Analysis of Reasoning of LLM Traders
The analysis compares incremental quote improvements with spread-crossing trades in Gemini Large’s reasoning traces. Crossing is associated with more urgency and execution language, but the lexical evidence is suggestive rather than causal or generalizable across models.
- Reasoning phases: Gemini Large traces distinguish incremental quote improvements from spread-crossing trades, which secure a transaction.The analysis pools traces from both phases and compares terms characteristic of crossing versus incrementing.
- Scope and interpretation: Because only Gemini Large exposes reasoning traces, the observed pattern is treated as a window into one mechanism rather than a general claim about LLM traders.
- Lexical shift: As agents move toward trading, language about margin protection recedes while urgency and execution language rises.The authors interpret this shift as profit weighting early and completion weighting later.
- Scope and interpretation: The lexical analysis identifies terms separating phases but does not establish what causes agents to switch, and traces may not faithfully reflect the driving computation.
- Lexical shift: Figure 4 organizes lexical differences by urgency, execution, optimization, and strategy across move types.
6 Conclusions and Future Work
Across the tested LLM populations, double auctions do not fully reproduce the human convergence pattern or reliably deliver the intended efficient allocation. The authors therefore recommend validation against human and theoretical benchmarks while treating the findings as an early step open to broader testing.
- Main conclusions: No tested condition fully converges to competitive equilibrium, and price dispersion remains above the human benchmark in every trading round.
- Main conclusions: GPT Small is the smallest and least capable model yet produces the most human-like and efficient markets among the tested models.The authors caution that the design was not built to isolate model scale, so this result requires a separate study.
- Implications: LLM substitution for human agents in agent-based models cannot be assumed innocuous, even where human behavior is well documented and robust.
- Implications: Simulations using LLM agents as human stand-ins should be validated against human and theoretical benchmarks when those benchmarks exist.
- Implications: Supracompetitive prices and incremental undercutting cannot be cleanly attributed to collusion until convergence without collusion is established for LLM traders.
- Future work: The study remains an early step, with future work targeting longer horizons, richer feedback, interventions, other institutions, and broader model coverage.
A Role Asymmetries in the Fill Rate
GPT Small exhibits strong buyer–seller asymmetries in order improvements and fill rates. Sellers submit many more small improvements, while buyers post less often but more frequently clear the market.
- Fill rates: Buyer and seller trade initiations are broadly equal in number, but buyers have a markedly higher fill rate.The fill rate is the fraction of an agent’s orders that result in a transaction.
- Mechanism: Sellers submit roughly three times as many orders as buyers, while buyers’ larger improvements are more likely to clear.
B Prompt Templates
The prompt templates define role-specific objectives, private reservation-price constraints, market state and history inputs, and rules for improving orders and executing trades. They operationalize a Smith-style double auction through reusable system, user, announcement, outcome, and history templates.
- B.1 System Prompt: System prompts assign each agent a market role and instruct it to buy or sell one unit at the best possible price without violating its reservation price.Role and verb placeholders are filled at runtime, while buyer- and seller-specific wording comes from keyword substitutions.
- B.1 System Prompt: Profit is defined from reservation and transaction prices, with the order reversed for sellers so the signed difference is correct in each role.Each agent is told that its reservation price is private and that its goal is to maximize profit.
- B.1 System Prompt: The market has configurable numbers of buyers and sellers, runs for multiple rounds, and permits at most one transaction per agent per round.Agents seek an additional unit in each round and may transact regardless of what happened in the previous round.
- B.1 System Prompt: Each round maintains a standing best bid and ask; it ends when no improving order is possible or an order cap is reached.Either standing price may be absent before an order is posted on that side.
- B.2 User Prompt: User prompts provide market history, the agent’s own action history, the current round, standing bid, standing ask, and reservation price.The reservation-price instruction reiterates that the agent cannot trade beyond its constraint.
- B.3 Announcement Prompts: Buyer announcements must strictly improve the standing bid and remain at or below reservation value, whereas seller announcements must strictly improve the standing ask and remain at or above reservation value.A first announcement is allowed when no standing order exists on that side.
- B.4 Order Outcomes Block: A buyer bid crossing the standing ask or a seller ask crossing the standing bid executes immediately and removes both parties from the round.Noncrossing bids or asks become new standing orders only when they improve the relevant side; otherwise they have no effect.
- B.5 History Line Templates: History templates record accepted announcements, posted noncrossing best orders, and each agent’s general or non-improving action outcome.These templates expose market and personal order outcomes in structured textual form.
C Experimental Framework
The experiments use a Python multi-agent framework with LangGraph orchestration and require access to Gemini and OpenAI model-provider APIs. Simulations ran on a laptop with a CPU and 32GB of RAM.
- C Experimental Framework: The simulations were implemented in Python, orchestrated with LangGraph, and run using Gemini and OpenAI provider APIs.The reported hardware environment was a laptop with a CPU and 32GB of RAM.
D Typical Order Flow
Typical order flow differs substantially across models in when agents cross the bid–ask spread. Gemini Large crosses when profitable opportunities arise, GPT Large waits until the spread reaches zero, and GPT Small mixes both patterns.
- D Typical Order Flow: Gemini Large crosses the spread once profitable trading opportunities arise, whereas GPT Large waits until the spread goes to zero.The distinction is between spread-crossing moves and incremental order improvements.
- D Typical Order Flow: Figure 6 plots individual order prices across five rounds, with a red dashed line marking the $2.00 competitive equilibrium price.Vertical dashed lines separate the experiment’s five rounds.
- D Typical Order Flow: GPT Small exhibits a mixture of Gemini Large’s early spread crossing and GPT Large’s waiting pattern.The caption and accompanying discussion characterize this as model-level heterogeneity in willingness to cross the spread.