Why Better Models Can Create Riskier Systems: Evidence from LLM Agents in Financial Markets

1 Introduction

• The central question is whether improving individual model capability can degrade system-level outcomes when models are deployed at scale alongside similar models.The paper argues that collective behavior may decouple from individual model quality in ways individual evaluations cannot detect.Source: §1 · p. 2
• In financial markets, correlated non-corrective actions can produce persistent mispricing, excess volatility, or shock amplification; systemic risk refers to widespread instability impairing normal functioning.Source: §1 · p. 2
• Agent-based market simulations use LLM traders of varying capability to measure market-level properties in a setting with fundamental value and measurable price efficiency and volatility.The study asks whether frontier LLMs exhibit correlated behavior, whether greater participation produces systemic risk, and whether shared information amplifies it.Source: §1 · p. 2
• Frontier LLMs show significantly correlated non-corrective behavior that increases with capability; participation improves convergence, but shared misaligned information can make markets less stable than with noise traders alone.The stabilizing participation result holds across tested model families; under shared misinformation, coordinated reliance on misaligned information reverses that benefit.Source: §1 · p. 2

2 Systemic Risk in LLM Agents

• A system state evolves through population actions and is evaluated by how closely it tracks ground truth; in financial markets, these correspond to market price and fundamental asset value.Source: §2 · p. 2

2.1 Decomposing agent actions

• Each agent’s scalar action is split into corrective pressure tied to the contemporaneous state-ground-truth gap and a non-corrective residual, a statistical description of behavior.The residual is not necessarily irrational; it can reflect framing, heuristics, constraints, or priors that are informative on average but misaligned in a given context.Source: §2.1 · p. 3

2.2 Correlated action creates a non-diversifiable floor

• Positively correlated non-corrective actions cannot be diversified away as the population grows, and can destabilize the system by preventing return to ground truth or amplifying post-shock deviations.The variance decomposition’s correlation-driven floor is invariant to population size.Source: §2.2 · p. 3

2.3 Estimating correlated action

• The estimation procedure fits a separate OLS corrective component for each agent, computes residual actions on a disjoint test set, and measures pairwise residual correlation across scenarios.The study examines how this correlation varies with model capability and whether same-provider pairs are more correlated than cross-provider pairs.Source: §2.3 · p. 4

3 The Capability Paradox

• Capability means general-purpose reasoning and knowledge performance measured independently of financial-task performance.The paper uses MMLU-Pro and LMSYS Chatbot Arena ELO scores; neither evaluates financial knowledge or trading performance.Source: §3 · p. 4
• More capable models may increase non-corrective correlation while reducing residual variance through improved reasoning.The risk floor rises when increased correlation dominates the reduction in residual variance.Source: §3 · p. 4
• The authors show convergence empirically but leave its mechanism for future work.Overlapping training data and objectives, or alignment tuning, are proposed explanations rather than established mechanisms.Source: §3 · p. 4

4.1 RQ1: Do frontier LLMs exhibit correlated behavior?

• The RQ1 experiment measures seven models across 64 scenarios, averages five responses per scenario, and correlates their OLS residual behaviors across model pairs.Pairwise regressions relate correlation to average capability while controlling for shared provider family.Source: §4.1 · pp. 4–5
• Average pairwise non-corrective correlation is significantly predicted by pair capability under ELO (t = 3.94, p = 0.001; R2 = 0.475) and MMLU-Pro (t = 4.88, p < 0.001; R2 = 0.580).Shared provider was not significant in either specification (p = 0.39 for ELO; p = 0.28 for MMLU).Source: §4.1 · p. 5, Figure 2 · p. 5
• Even a large ensemble retains a residual-variance floor: ρ = 0.010, roughly half the single-model variance of 0.021.The empirical bootstrap tracks the theoretical curve, while an independence benchmark predicts variance declining as 1/N without a floor.Source: §4.1 · p. 5

4 Studying Systemic Risk

• The market simulations use an exogenous fundamental with two shocks, LLM traders, noise traders, and a market maker; RQ3 varies shared commentary conditions.RQ3 compares neutral generic commentary with adversarial commentary that claims fair value overstates intrinsic value by 25–30%.Source: §4.2 · pp. 5–6, §4.3 · p. 7

4.2 RQ2: Does increasing LLM market participation produce systemic risk?

• LLM participation monotonically lowers mean tracking error relative to noise traders, with more aggressive gap-responsive trading associated with lower residual error.At full participation, Gemini 2.0 Flash Lite has λ ≈1.51 and error ≈1.33; Sonnet 4.6 has λ ≈1.15 and error ≈2.10; Haiku 4.5 falls between them.Source: Figure 3 · p. 6, §4.2 · p. 6
• With accurate shared information, LLM agreement accelerates price discovery; the shared misinformation condition instead makes the market worse than uninformed noise traders alone.The section contrasts RQ2’s mixed outcome with RQ3’s elimination of the stabilizing benefit.Source: §4.2 · p. 6, §4.3 · p. 8

4.3 RQ3: Does a shared information environment amplify systemic risk?

• At full LLM penetration, adversarial distractors raise mean tracking error to five to seven times baseline, while volatility also increases with participation.Under neutral sentiment, all three models track fundamentals better than the noise-only baseline; adversarial commentary directly impugns the fundamental signal.Source: Figure 4 · p. 7, §4.3 · p. 7
• Because all agents draw from the same rotating distractor pool each round, commentary-induced reasoning errors are fully correlated, so adding LLM agents amplifies distortion rather than diversifying misinformation.Source: §4.3 · pp. 7–8

5 Related Work

• The contribution is a framework connecting individual capability, correlated errors, and dynamic system stability, with correlated errors non-diversifiable as populations grow.The paper tests the framework’s predictions in financial markets.Source: §5 · pp. 8–9
• Prior capability-scaling work measures individual performance, while this paper extends the question to collective outcomes in dynamic environments.The cited literature includes compute-scaling laws, non-monotonic scaling findings, and observational scaling of agentic behaviors.Source: §5 · p. 8
• Algorithmic monoculture research documents correlated errors in static decisions, whereas this paper studies how dynamic feedback lets correlated agents update a shared state.The paper frames this as a way for errors to compound over time.Source: §5 · p. 8
• Financial systemic-risk research studies contagion, common exposures, and endogenous imbalances, but the paper focuses on behavioral aggregation through shared models.Similar model-driven behavior may become more homogeneous and reliant on unreliable outputs.Source: §5 · p. 8

• Individual evaluations can miss aggregation floors, while provider diversity alone does not address the dominant driver of non-corrective correlation.High-capability models from different companies are more correlated than low-capability models from the same company.Source: §6 · p. 9
• Under adversarial information, LLM participation produces tracking error five to seven times the noise-only baseline, and population-size scaling remains an empirical test.The framework predicts correlated residual risk persists as the population grows.Source: §6 · p. 9
• AI governance should add population-level correlation audits alongside individual evaluations, while behavioral decorrelation and deployer coordination remain open challenges.The discussion identifies training for behavioral diversity, input perturbation, and provenance disclosure as possible approaches.Source: §6 · p. 9
• The empirical evidence is limited to simulated, simplified single-asset markets; whether these dynamics arise in other domains remains open.Real markets add correlated assets, endogenous liquidity, strategic adaptation, and institutional constraints that could amplify or dampen the effects.Source: §6 · p. 9

• More capable models show more correlated non-corrective behavior; accurate shared reasoning improves price discovery, but shared misinformation makes coordination a liability.These results underpin the paper’s capability paradox: individual performance improvements do not necessarily translate into better market-level outcomes.Source: §7 · p. 9
• The results motivate population-level evaluation alongside individual model evaluation, while whether the same dynamics arise in other domains remains open.Source: §7 · p. 9

• Provider and payer agents move aggregate billing intensity in opposite directions relative to the appropriate billing level.The system state is prevailing claims relative to care delivered; ground truth is the appropriate billing level.Source: A Application to Insurance Billing in… · p. 12
• Each agent’s action separates into a corrective response to a genuine billing gap and a non-corrective residual that includes responses to exploitable ambiguities or irrelevant documentation patterns.For providers, correction addresses undercoding; for payers, it addresses overbilling.Source: A Application to Insurance Billing in… · p. 12
• Similar training corpora and architectural biases can correlate non-corrective residuals within each side, so shared heuristics compound rather than cancel across agents.The resulting aggregation floor cannot be eliminated by deploying more agents.Source: A Application to Insurance Billing in… · p. 12
• Adversarial feedback can make more capable provider agents more effective at exploiting ambiguities, raising non-corrective correlation on both sides despite better individual benchmark accuracy.Provider behavior shapes the claims distribution that payer agents learn to respond to.Source: A Application to Insurance Billing in… · p. 12
• Billing instability arises in the dangerous regime of large non-corrective residuals, high within-side correlation, and adversarial coupling across provider and payer agents.The described outcomes include billing inflation, denial inflation, and increased administrative overhead, risks not detected by evaluating either side individually.Source: A Application to Insurance Billing in… · p. 12

• The residual decomposition distinguishes a family-shared component from a population-shared component that affects agents regardless of family.The components are described as mutually uncorrelated, with family-shared components independent across families.Source: B Residual Decomposition and Sources… · p. 13
• Cross-family residual correlation equals the share of total residual variance attributable to the population-shared component.Family mixing eliminates the family-shared variance component but leaves the population-shared source.Source: B Residual Decomposition and Sources… · p. 13
• Symmetric portfolio initialization can leave a non-diversifiable floor even in a fully provider-diversified deployment.Source: B Residual Decomposition and Sources… · p. 13
• A shared information environment exposes every agent to the same distractor pool each round, so induced reasoning errors are fully correlated and tracking error scales with LLM share.Source: B Residual Decomposition and Sources… · p. 13

C Simulation Details and Scope

• Reference prices update from normalized net order imbalance, while the fixed bid/ask spread separates execution pricing from reference-price evolution.The fundamental value is exogenous and piecewise constant; agents observe prices, their portfolio state, and a noisy fair-value signal before submitting buy, sell, or hold decisions and quantities.Source: C Simulation Details and Scope · p. 13
• Low δ represents deeper markets where prices respond less to order flow; high δ represents thinner markets where the same flow moves prices more.Source: C Simulation Details and Scope · p. 13
• Regime shifts add tracking transients to the stationary residual-variance component, so the predicted floor characterizes the noise component, not those transients.Positive residual serial correlation makes the floor a lower bound relative to the i.i.d. benchmark.Source: C Simulation Details and Scope · pp. 13–14
• When cash or inventory constraints bind frequently, linear state-variance predictions are approximate because the AR(1) derivation assumes unconstrained agents.Source: C Simulation Details and Scope · p. 14

D Capability Benchmarks

• Table 1 reports capability benchmarks for the eight models used in the panel experiments.MMLU scores for models in RQ1–3 are also reported in Figure 1.Source: Table 1 · p. 14, D Capability Benchmarks · p. 14

E LLM Usage

• Claude Code assisted implementation of the market simulation and analysis, and Claude helped polish the paper’s writing.Source: E LLM Usage · p. 14

F External Assets and Licenses

• The experiments use authorized commercial and open model APIs, public benchmark scores, and author-written code; released artifacts will retain third-party software attribution under a repository license.Model weights and proprietary provider assets are not redistributed.Source: F External Assets and Licenses · p. 14