Source-linked AI summary
WolfSociety: Understanding Collective Risk from Harmful-Agent Scaling in Financial Agent Societies
Lejun Zhang, Sarah Lu-Liang, Xin Jiang, Muning Wen, Weinan Zhang, Shangding Gu
TL;DR
Individual-agent safety evaluations may miss risks that emerge as agents interact and alter shared conditions. We study collective failure in a controlled financial society, finding a size-dependent collapse transition and weaker fixed-count disruption in larger societies.
Problem
Prior work does not establish how the harmful population required for collective failure changes as the surrounding society grows, a gap relevant because interacting agents can influence one another and shared environments.
Source: §1 · p. 3Method
We vary harmful-agent fraction and society size in a controlled financial agent society and use Agent Society Dynamics to relate collapse boundaries, fixed-count response, and interaction effects.
Source: §1 · p. 3Results
From N=100 to N=2000, the 50% collapse boundary falls from 4.7% to 2.2%, while the corresponding harmful count rises from about 5 to 44.
Source: §1 · p. 3Takeaways & Limitations
In this setting, collective failure depends on society size and interaction structure as well as harmful-agent prevalence, supporting evaluation across population sizes and harmful compositions.
Source: §6 · p. 14The study uses thirty-day episodes, static within-episode graphs, archetypal roles, and an S1 primary scaling analysis; future work can test fully LLM-controlled populations, adaptive networks, longer horizons, and other domains.
Source: Scope, Limitations, and Ethics · p. 14
Abstract
from arXiv · showhide
Safety evaluations typically focus on individual agents, but interacting agents can spread harmful information and influence the environment in which later decisions are made. We study how collective failure changes with harmful-agent fraction and society size in a controlled financial agent society, where agents communicate over a social network and trade in a shared market. In the primary financial scenario, collective failure requires broad harmful diffusion together with severe price dislocation or liquidity stress. Across all tested society sizes, failure remains rare at low harmful fractions but rises sharply over a narrow range. As society size grows from N=100 to N=2000, the harmful fraction associated with a 50% failure probability decreases from 4.7% to 2.2%, while the corresponding number of harmful agents increases from approximately 5 to 44. In contrast, when the number of harmful agents is held fixed, their impact becomes weaker as the society grows. Controlled interventions further show that broader network reach shifts the collapse boundary toward lower harmful fractions, whereas stronger conformity alone has little effect. To characterize these effects, we introduce Agent Society Dynamics, a finite-size framework for relating harmful-agent fraction, society size, and interaction structure to collective failure. Overall, our results reveal a nonlinear, size-dependent collapse transition in financial agent societies, showing that collective failure depends not only on the prevalence of harmful agents but also on the size and interaction structure of the surrounding society. Code is available at https://github.com/SAIL-Research-Lab/WolfSociety.
1 Introduction
The introduction asks how the harmful population required for collective failure changes as a society grows, then studies this question in a controlled financial agent society. It reports a nonlinear size-dependent collapse pattern and introduces Agent Society Dynamics to organize the scaling and interaction effects.
1 Introduction
- Prior work documents interaction failures and collective transitions, but does not establish how the harmful population required for failure changes as the surrounding society grows.The motivation is that harmful influence can spread through interactions and alter shared conditions that shape later decisions.Source: §1 · p. 3
- The study varies society size and harmful-agent fraction in a financial society where communication and trades affect conditions observed by later agents.WolfBench packages the controlled environment, four manipulation scenarios, and evaluation interfaces.Source: §1 · pp. 3–4
- From N=100 to N=2000, the 50% collapse boundary falls from 4.7% to 2.2%, while the corresponding harmful count rises from about 5 to 44.Collapse is rare at low harmful fractions at every tested size, then rises sharply over a narrow range.Source: §1 · p. 3, Figure 1 · p. 4
- Neither a constant harmful count nor a constant harmful fraction explains the transition; the critical harmful population grows more slowly than society size.The introduction estimates bν = 0.222 across six primary society sizes.Source: §1 · p. 3
- Agent Society Dynamics links the collapse boundary, fixed-count response, and interaction-driven shifts in a finite-size framework.Its simple description relates αc(N) ∝ N^-ν to Kc(N) ∝ N^(1−ν), with 0 < ν < 1 matching the observed pattern.Source: §1 · p. 3
- At a fixed harmful-agent count, diffusion and market disruption weaken as society grows, and this decline persists under fixed, square-root, and per-capita liquidity rules.The liquidity experiments use all three directly implemented scaling rules.Source: §1 · p. 4
- Broader network reach shifts collapse toward lower harmful fractions, whereas stronger conformity alone changes the collapse boundary little.The primary size-dependent trend also holds across rule-based and hybrid controllers and five alternative LLM backbones.Source: §1 · p. 4
2 Related Work
Related work connects multi-agent interaction, social influence, financial interdependence, and finite-size effects to collective risk. The paper positions its contribution as jointly measuring harmful-fraction boundaries and harmful counts as society size changes, then testing that pattern through fixed-count analyses and interventions.
Multi-agent safety and population-scale simulation
- Prior multi-agent studies report interaction failures and collective risks, while population-scale simulations offer tractable settings for studying emergent behavior (Hammond et al., 2025; Cemri et al., 2025; Erisken et al., 2025; Motwani et al., 2024; Nakamura et al., 2026; Huang et al., 2026; Park et al., 2023; Piao et al., 2025; Akkil et al., 2026; Feng et al., 2026).Adversarial minorities have been reported to push communicating agents toward persistent misalignment (De Marzo et al., 2026), and changing misaligned-value prevalence can alter collective regimes (Zhang et al., 2026).Source: §2 · p. 5
Collective transitions and social influence
- Threshold, cascade, and complex-contagion models describe how local changes and repeated exposure can produce discontinuous collective outcomes.Related decision models and recent agent studies distinguish individual social response from the breadth of information propagation (Bellina et al., 2026; Liu et al., 2026).Source: §2 · p. 5
Systemic risk and financial agent environments
- Financial-network models link structure and interdependence to systemic fragility, while agent-based and LLM-agent environments study market dynamics and coordinated behavior (Haldane and May, 2011; Elliott et al., 2014; Acemoglu et al., 2015; Byrd et al., 2019; Ren et al., 2025; Bracale Syrnikov et al., 2026).The paper uses a closed loop in which communication affects trading, market conditions respond, and later agents observe those changes.Source: §2 · p. 5
Finite-size scaling of collective failure
- Finite-size scaling distinguishes collective transitions from finite-population effects, but prior LLM-agent work does not directly characterize how the harmful population required for failure changes with society size (Flint et al., 2025; Willis et al., 2026).The paper jointly varies harmful fraction and population size, estimates the boundary and corresponding count, and tests the pattern without assuming a known universality class.Source: §2 · pp. 5–6
3 Agent Society Dynamics: Collapse and Finite-Size Scaling
The section defines collective collapse through harmful diffusion and market stress, then relates society size, social response, and harmful-agent composition to the collapse boundary. It also separates social responsiveness from information reach and introduces a fixed-count analysis of severity.
3.1 Society-Level Collapse
- Collapse requires broad harmful diffusion together with severe price dislocation or liquidity stress, measured using prespecified operational cutoffs.The episode severity is the maximum daily severity over the thirty-day episode; the cutoffs are simulator definitions, not real-world regulatory thresholds.Source: §3.1 · p. 6
3.2 Social Response and Repeated Interaction
- Social messages and private evidence jointly inform agent responses, while trust, attention, and network structure shape which social evidence reaches agents.D_N measures how strongly received social information is reflected in actions, not how widely that information spreads; χ summarizes accumulated response over a finite horizon.Source: §3.2 · p. 7
3.3 Finite-Size Scaling
- The finite-size relation αc(N) ∝ N^-ν and Kc(N) ∝ N^(1-ν) describes a regime where the collapse fraction falls while the corresponding harmful count rises.The experiments estimate the overall exponent ν, not δ and ζ as separate causal quantities.Source: §3.3 · p. 7
- Holding harmful-agent count K fixed, the severity surface relates episode severity to society size and harmful count, with bN < 0 indicating weaker disruption in larger societies.The analysis directly compares fixed, square-root, and per-capita liquidity scaling implemented in the market maker.Source: §3.3 · p. 8
4 Experimental Setup
The experiments use a controlled financial society where communication and trading share a market, and package the simulator and evaluation interfaces as WolfBench. They vary harmful-agent composition, society size, and interaction properties, then estimate collapse boundaries and fixed-count severity.
4.1 Society and Scenarios
- WolfBench packages the simulator, manipulation scenarios, and evaluation interfaces for studying collective failure across harmful-agent composition and society size.Agents communicate over a social graph and trade in a shared market, with market conditions observed in later rounds.Source: §4.1 · pp. 8–9
4.2 Scaling and Interventions
- The scaling study holds the scenario and key society parameters fixed across sizes, while interventions vary network reach, attention, conformity, or response precision individually and jointly.The increased-reach condition doubles mean graph degree; matched intervention conditions reuse random seeds.Source: §4.2 · p. 9
4.3 Fixed-Count Analysis and Estimation
- Fixed-count severity is fitted at the episode level using log RS1 = a + bN log N + bK log K + ε, with subcritical cells and paired bootstrap resampling.Liquidity scaling is implemented in the market maker before complete trajectories are simulated; collapse boundaries are interpolated between adjacent harmful fractions bracketing p = 1/2.Source: §4.3 · pp. 9–10
5 Results
The results show a nonlinear, finite-size collapse transition: larger societies reach the 50% boundary at smaller harmful fractions but larger harmful counts. Fixed-count disruption weakens with size, and network reach is more closely associated with boundary shifts than conformity alone.
5.1 Nonlinear Collapse and Finite-Size Scaling
- From N = 100 to N = 2000, αc falls from 0.047 to 0.022 while Kc rises from 4.7 to 44.0; collapse rises sharply over a narrow fraction range.The fitted boundary exponent is 0.222, and the 10%–90% transition width ranges from 0.013 to 0.028 for four sizes.Source: §5.1 · pp. 10–11, Figure 3 · p. 11
5.2 Fixed-Count Severity Scaling
- At fixed harmful count, the subcritical size coefficient is bN = −0.894, and severity declines with society size under all three liquidity rules.Each rule's 95% confidence interval excludes zero, while the intervals overlap substantially.Source: §5.2 · p. 11, Table 2 · p. 11
5.3 Interaction Conditions and Network Reach
- Doubling mean graph degree shifts the boundary by −0.017, while high conformity raises D_N by 0.183 but barely changes the boundary.Changing attention, conformity, or response precision individually produces smaller boundary shifts whose confidence intervals include zero.Source: Figure 4 · p. 12, §5.3 · p. 12
5.4 Controller Realization Audits
- Across tested controller allocations and five society sizes, the endpoint pattern persists; five alternative LLM backbones also show a decreasing boundary from N = 100 to N = 1000.The backbone two-point exponents range from 0.273 to 0.325; controller-allocation intervals remain wide.Source: Table 3 · p. 12, §5.4 · p. 13, Table 4 · p. 13
5.5 Transfer Across Manipulation Mechanisms
- S2 and S3 also show nonlinear transitions, whereas S4 does not reach the collapse criterion for α ≤ 0.12 within the thirty-day horizon.Because manipulation channels and severity definitions differ, these experiments compare transition presence rather than absolute boundary locations.Source: §5.5 · p. 13
6 Discussion
The discussion frames collective fragility as dependent on both society size and interaction structure. Fixed-count disruption weakens in larger societies, while broader network reach shifts the collapse boundary more clearly than conformity does.
Why the fraction falls while the count grows
- At a fixed harmful count, larger societies show less severe harmful diffusion and market disruption, including across three liquidity rules.The decline persists under fixed, square-root, and per-capita liquidity scaling.Source: §6 · p. 13
Why network reach differs from conformity
- Broader network reach moves the collapse boundary despite little change in DN, while conformity barely changes the boundary.Conformity makes individual actions more dependent on social evidence, distinguishing response strength from information spread.Source: §6 · p. 13
- Among individually varied components, network reach has the clearest relationship with where collective failure occurs.Source: §6 · p. 14
Implications for the safety of agent systems
- Isolated-agent evaluations can miss failures emerging through interaction at scale, motivating tests that vary population size and harmful composition.Source: §6 · p. 14
7 Conclusion
The conclusion presents collective safety as a society-level problem in a controlled financial agent society. It links the size-dependent collapse pattern to fixed-count disruption and interaction structure.
- Larger societies collapse at smaller harmful fractions even though the corresponding number of harmful agents increases.Source: §7 · p. 14
- At the same harmful count, disruption weakens in larger societies under fixed, square-root, and per-capita market-depth scaling.Source: §7 · p. 14
- The conclusion identifies harmful composition, population size, market design, and interaction structure as joint dimensions of multi-agent safety.Agent Society Dynamics connects harmful influence, society size, and interactions to collective failure.Source: §7 · p. 14
Scope, Limitations, and Ethics
The findings concern a controlled financial society with thirty-day episodes, static within-episode graphs, and archetypal roles. The fixed-count and liquidity analyses also have defined experimental scope.
- The primary scaling analysis uses S1; S2–S4 examine whether similar transitions occur under other manipulation mechanisms.Source: Scope, Limitations, and Ethics · p. 14
- The simulated societies use thirty-day episodes, static within-episode graphs, and archetypal roles.Source: Scope, Limitations, and Ethics · p. 14
- Liquidity experiments use a common subset of society sizes and harmful counts with the all-rule controller configuration.Source: Scope, Limitations, and Ethics · p. 14
- Manipulation scenarios abstract public cases at the mechanism level without naming securities or reproducing operational procedures.Source: Scope, Limitations, and Ethics · p. 14
Supplementary Material
The supplementary material develops the local-response and finite-size arguments, then details the scenario mechanisms and their abstraction from public cases.
A Theory Details and Boundary Conditions
- The supplement uses a reduction to expose how direct harmful input, social response, and shared-environment changes combine over a finite horizon.Source: A Theory Details and Boundary Conditions · p. 18
A.1 Quantal-response reduction and Jacobian
- The local action map is an analytical reduction, not the literal policy executed by every simulator role.The simulator also includes hold, silence, threshold, satisficing, imitation, and impulse-prone behaviors.Source: §A.1 · p. 18
- The condition ∥BΓWN∥∞ < 1 is sufficient for contraction of the fixed-state action block.The closed-loop linearization separates direct social response, environmental feedback to action, and the update of shared state.Source: §A.1 · p. 18
A.2 Information bounds
- The information bounds motivate, but do not replace, the empirical comparison; DN is defined from corrected conditional mutual-information estimates.Source: §A.2 · p. 19
A.3 Finite-size midpoint bridge
- The finite-size midpoint bridge connects persistent-input gain to the collapse midpoint, while identifying only the combined size dependence δ + ζ.A separate interpretation of the components requires fixed definitions of direct attack input and unit-direction amplification response.Source: §A.3 · p. 20
B Complete Scenario Descriptions
- The four scenarios abstract public manipulation patterns by preserving communication, trading, timing, and incentive structure while removing names, amounts, and operational detail.Source: B Complete Scenario Descriptions · p. 20
B.1 Public-case provenance and abstraction boundary
- Scenario collapse occurs when its corresponding score reaches one on an eligible day; S1–S2 are evaluated throughout, S3 from day 2, and S4 from day 6.The scenario scores normalize observed components by prespecified thresholds; the source cases motivate mechanism classes, while numerical settings are simulation choices.Source: §B.1 · pp. 20–21
C.1 Information estimator and finite-sample correction
Information is measured at agent–asset decision events, and finite-sample bias is corrected using conditional permutations. The scaling analysis estimates collapse midpoints on frozen grids and quantifies uncertainty by resampling paired experiment seeds.
C.1 Information estimator and finite-sample correction
- Each information-estimator row represents one agent–asset decision and records its action, signals, market/history state, role, and social-proof state.The raw conditional mutual-information estimator uses a plug-in entropy identity.Source: §C.1 · p. 21
- Sixteen conditional permutations estimate finite-sample bias by shuffling Y within each observed Z stratum while holding Z and marginal sample counts fixed.The permutation generator uses a fixed metric seed for reproducibility.Source: §C.1 · p. 21
C.2 Frozen grids, midpoint estimation, and bootstrap
- The harmful-fraction grids were fixed before final collapse-boundary computation, with dedicated pilot grids retained for N = 100 and N = 300.The remaining grids were recorded as part of the frozen main sweep.Source: §C.2 · p. 22
- Twelve common-random-number seeds estimate each size’s response curve; the first adjacent grid points bracketing 50% collapse are linearly interpolated to obtain αc(N).The primary exponent is the negative ordinary-least-squares slope of log αc(N) on log N across six resolved sizes.Source: §C.2 · p. 22
- Uncertainty is estimated by paired seed bootstrap, reconstructing response curves and refitting the exponent; non-bracketing midpoints are censored rather than extrapolated.Bootstrap resampling uses the same seed identities across sizes and harmful-fraction cells.Source: §C.2 · p. 22
D Additional Numerical Results
Additional tests distinguish propagation and interaction effects from robustness checks on model backbone, controller realization, and harmful-fraction measurement. Multi-hop restriction suppresses the transition, while the tested alternative backbones and controller conditions preserve the endpoint pattern.
D Additional Numerical Results
- High conformity produces the largest increase in social-information dominance but only a small collapse-boundary shift; increased network reach moves the boundary toward lower harmful fractions.Joint conditions alter several interaction parameters, so their shifts describe combined settings rather than a single causal pathway.Source: D Additional Numerical Results · p. 22
- Restricting harmful messages to their first hop prevents collapse probability from reaching 50% on the tested grid, whereas removing market-state feedback produces no clear boundary shift.The boundary changes from 0.0429 to 0.0429 at N = 300 and from 0.0275 to 0.0267 at N = 1000 under the paired clean-market trajectory.Source: D Additional Numerical Results · p. 22
- Across five alternative LLM backbones, the collapse boundary decreases from N = 100 to N = 1000, with two-point exponents between 0.273 and 0.325.The primary finite-size estimate still comes from the six-size sweep.Source: D Additional Numerical Results · p. 22
- Behavioral-only, standard hybrid-quota, and increased-quota controllers all preserve a decreasing boundary and increasing corresponding harmful count from N = 100 to N = 1000.The N = 2000 grid is right-censored and excluded from the common-support exponent comparison; point estimates are similar with wide six-seed intervals.Source: D Additional Numerical Results · p. 22
- Using all 10,000 bootstrap draws gives mean bν = 0.223 with a 95% interval of [0.152, 0.283]; the all-six-size rule retains 3,726 draws.The all-six-size rule gives the narrower conditional interval reported in the main paper.Source: S4 wash-trading system prompt · pp. 23–24
- Replacing target harmful fractions with realized fractions changes the fitted exponent from 0.222 to 0.224 without changing the directions of the boundary and harmful-count trends.The realized fraction is K/N.Source: S4 wash-trading system prompt · p. 24
E Response-Surface and Sensitivity Analyses
Response-surface and sensitivity analyses examine fixed-count effects, threshold choices, boundary prediction, normalization, and role robustness. The fixed-count response captures the direction of the size effect, but its no-midpoint contour changes more steeply than the observed boundary.
E Response-Surface and Sensitivity Analyses
- Changing normalization parameter ℓ shifts bζfg from −0.784 to 0.216 while δ + bζfg remains 0.216, with simulator responses held fixed.The empirical normalized-response exponent is distinct from theoretical amplification exponent ζ.Source: E Response-Surface and Sensitivity… · p. 24
- The fixed-count response-contour exponent is 0.469 [0.435, 0.501], and direct size slopes remain negative, ranging from −1.007 to −0.573.The contour estimate remains similar when restricted to zero-collapse cells: 0.477 [0.437, 0.516].Source: E Response-Surface and Sensitivity… · p. 24
- Across resolved threshold-sensitivity fits, 0 < bν < 1; stricter cutoffs push the largest sizes beyond the tested grid.Under the S1 conjunction, the social component alone reaches the crossing, while price or liquidity alone does not reach 50% on the tested grid.Source: E Response-Surface and Sensitivity… · p. 25
- No-midpoint predictions preserve the observed boundary’s direction and ordering (Spearman correlation 0.943) but yield exponent 0.463 versus the observed 0.222.Mean absolute log error is 0.390; the constant-boundary and direct boundary-power-law baselines have errors 0.212 and 0.092.Source: E Response-Surface and Sensitivity… · p. 25
- Boundary-supported calibration reduces leave-one-size-out error from 0.351 to 0.089, with predicted exponent changing from 0.139 to 0.152.Because calibration uses training-size collapse midpoints, this comparison measures boundary interpolation rather than no-midpoint prediction.Source: E Response-Surface and Sensitivity… · p. 25
- The transition remains present across role-homogeneous ablations, although mixed-role agents behave differently from legacy-score agents.The experiments test whether the transition depends on a homogeneous population of legacy risk-score traders.Source: E Response-Surface and Sensitivity… · p. 25
F Supplementary Tables
- The fitted null exponent is bν = 0.692 with the linear midpoint and 0.694 with the logistic midpoint.The cited table caption identifies the comparison as a Watts-style threshold null on matched society sizes and S1 grids.Source: F Supplementary Tables · p. 29, Table 17 · p. 29