PaperlayerView exact revision on arXiv

Source-linked AI summary

LLM Multi-Agent Systems Are Vulnerable to Herding and Runaway Cascades

Farnoosh Hashemi, Shiyu Ji, Joice Chen, Michael W. Macy

arXiv:2610.30769v1cs.AIcs.MA

TL;DR

LLM agents increasingly make decisions that become inputs for other agents, but the resulting endogenous feedback and cascade risks are not well understood. The paper adapts the Music Lab design to large LLM populations and finds generally greater LLM susceptibility to herding and less consistent self-correction than in a human benchmark. These comparisons are qualitative because the human and LLM studies differed in stimuli, response tasks, and presentation.

  • Problem

    Evidence about LLM susceptibility to cascades with endogenous feedback is limited, although individual-model evaluations do not show how collective behavior shapes later choices.

    Source: §1 · pp. 1–2

  • Method

    The study used sequential LLM recommendation populations with popularity counts visible in influence conditions and compared them with independent baselines and a Music Lab human benchmark.

    Source: §1 · p. 2, §2.1 · p. 3, §2.2 · p. 3

  • Results

    LLM populations were generally more susceptible to herding and cascades and less consistently self-correcting than the human benchmark.

    Source: §4 · p. 10

  • Takeaways & Limitations

    Evaluating collective AI behavior requires testing how model outputs shape subsequent inputs, not only how individual models respond to fixed signals.

    Source: §4 · p. 10

    The human–LLM comparison is qualitative because stimuli, response tasks, and presentation differed, and the study used homogeneous single-model populations without explicit collaboration or conformity incentives.

    Source: §2.5 · p. 5, §4 · p. 10

Abstract

from arXiv · show

Large language models (LLMs) increasingly operate in multi-agent systems where they collaborate by responding to others' choices, creating endogenous feedback loops with vulnerability to social influence, herding behavior and runaway cascades. Following the design of the classic Music Lab experiment, we tested this vulnerability in large populations of LLM agents that recommended unfamiliar songs, with or without exposure to cumulative recommendation counts. Exposure led to runaway cascades that generated extreme and unpredictable collective recommendations increasingly divergent from a song's intrinsic appeal. Compared to humans, LLM populations were generally far more susceptible to herding behavior, and unlike humans, LLM cascades were less able to self-correct. This vulnerability has implications for multi-agent collaboration in recommender systems, automated evaluation, complex problem-solving, and unauthorized conspiracies.

1 Introduction

The paper asks whether endogenous social feedback makes LLM populations vulnerable to runaway cascades that fail to self-correct, a dynamic that isolated-model evaluations cannot reveal. It tests this with an independent baseline and replicate influence worlds, comparing influence strength and collective behavior with a human benchmark.

1 Introduction

  • LLM multi-agent systems create endogenous feedback when one model’s decisions become inputs for later models, but their vulnerability to runaway, non-self-correcting cascades remains little known.Evaluating isolated models against fixed signals does not reveal how collective behavior shapes subsequent individual choices.Source: §1 · pp. 1–2
  • Prior work shows that social influence can make early popularity self-reinforcing, producing unequal, unpredictable outcomes that may nevertheless self-correct.In the Music Lab, popularity ordering strengthened these effects; later analyses found that popularity could realign with intrinsic appeal as choices accumulated.Source: §1 · p. 1
  • Feedback loops also arise in recommender systems, where past behavior shapes visibility and subsequent behavior, with reported effects on diversity, choice homogeneity, utility, and popularity bias.The paper situates this concern alongside work on LLM conformity and human–AI feedback amplification.Source: §1 · p. 2
  • Prior LLM social-influence studies use externally supplied majority answers, peer judgments, or user opinions, rather than an endogenous feedback loop.A cascade may fail to self-correct depending on the strength of social influence relative to independent judgment.Source: §1 · p. 2
  • The study tests whether population-generated popularity counts produce herding, errant and unpredictable cascades, stronger failures under stronger influence, and eventual self-correction.It also asks whether LLMs differ from humans and whether reasoning or higher-intelligence models are less susceptible.Source: §1 · p. 2
  • The study uses replicate influence worlds and an independent baseline to test LLM cascades, vary social-influence strength, and compare collective behavior with a human benchmark.Earlier agents’ recommendations generate the popularity counts later agents observe.Source: §1 · p. 2

2 Methods

The study adapts the Music Lab’s parallel-world design to test how endogenous recommendation feedback changes LLM agents’ choices and collective outcomes. Its methods compare independent and influence conditions across models, measure divergence, unpredictability, and self-correction, and use Music Lab data as a qualitative human benchmark.

2 Methods

  • Sequential recommendations updated counts visible to later agents in influence worlds, while independent decisions supplied a model-specific baseline and replicate worlds measured variation in collective outcomes.Worlds shared stimuli and procedures but accumulated separate recommendation histories.Source: §2 · p. 2

2.1 Task and stimuli

  • Agents assessed 48 newly generated lyric-based songs without audio; each selected four titles, then made yes/no recommendations after reading lyrics.The fixed selection budget standardized evaluation opportunities and modeled the constraint that downstream evaluation acts only on selected candidates.Source: §2.1 · p. 3

2.2 Parallel worlds and feedback conditions

  • Each model had one independent world and eight replicate worlds per influence condition; counts were hidden independently, shown with randomized ordering under weak influence, and shown with popularity ordering under strong influence.All worlds began with zero counts, evolved separately, and agents did not message one another; Figure 1 depicts the conditions and agent sequence.Source: §2.2 · p. 3, Figure 1 · p. 4

2.3 Model populations

  • The study evaluated five reasoning and eight non-reasoning models, classifying reasoning status by developer designation and assessing capability with the Artificial Analysis Intelligence Index v4.1.1.Each population used a single model, and personas combined demographic attributes with preferred music genres.Source: §2.3 · p. 4

2.4 Outcome measures

  • KL divergence measured how influence-world recommendation distributions departed from the independent distribution, while unpredictability measured differences among replicate worlds.The independent-world market shares served as a population-specific measure of each song’s relative intrinsic appeal; KL divergence used Jeffreys smoothing and was averaged across eight worlds.Source: §2.4 · pp. 4–5
  • Self-correction was assessed by comparing how closely recommendation counts in each half of an influence world’s sequence aligned with independent-world counts.A positive change in Spearman correlation indicated that later recommendations aligned more closely with intrinsic appeal; the study also reports Gini inequality, correlation, and persistence measures in supplementary materials.Source: §2.4 · p. 5

2.5 Human benchmark

  • The human benchmark reapplied the outcome definitions to Music Lab downloads, but differences in stimuli, response tasks, and presentation make it qualitative rather than a controlled human–LLM comparison.Weak and strong influence corresponded to the Music Lab experiments with randomized-grid and popularity-ordered presentations, respectively.Source: §2.5 · p. 5

3 Results

In the Music Lab-style recommendation task, social influence pushed LLM collective outcomes away from independent judgments and made them more variable. These effects generally exceeded the human benchmark, while LLM cascades were less consistently self-correcting.

3.1 LLM susceptibility to herding

  • Mean KL divergence reached about nine times the human benchmark under weak influence and four times under strong influence; strong influence raised mean divergence by 65% in 12 of 13 models.GPT-5 approached human divergence, while Gemma 3 4B diverged more than eight times as far under strong influence.Source: §3.1 · p. 5, Figure 2 · p. 6

3.2 Social feedback can turn herding behavior into a stampede

  • Non-reasoning models were more than twice as unpredictable as humans under both influence levels, and early movers predicted later popularity better than intrinsic appeal in 12 of 13 weak- and all 13 strong-influence models.Unpredictability was near zero under independent evaluation; social influence increased it in every model, with the stated exceptions for humans and DeepSeek-R1 under weak influence.Source: §3.2 · p. 6, Figure 3 · p. 7

3.3 Susceptibility to herding and cascades decreased with model intelligence scores.

  • Across 13 models, divergence and relative unpredictability declined as intelligence scores increased in both influence conditions, though the divergence association was stronger under strong influence.Spearman correlations for divergence were −0.851 under strong and −0.551 under weak influence; for relative unpredictability they were −0.733 and −0.719, respectively.Source: §3.3 · p. 7, Figure 4 · p. 8

3.4 Human cascades self-corrected; LLM cascades generally did not

  • Human cascades self-corrected in 15 of 16 influence worlds, while LLM changes varied across models and worlds and averaged near zero under strong influence.Human popularity-appeal correlations increased from 0.633 to 0.773 under weak influence and from 0.575 to 0.660 under strong influence; the study does not establish whether correction occurs over longer sequences.Source: §3.4 · p. 8, Figure 5 · p. 9

3.5 Supplementary results

  • Social influence raised Gini inequality in all 13 models under strong influence and 12 under weak influence; mean increases were larger than humans' for non-reasoning models and smaller for reasoning models.Source: §3.5 · p. 9
  • Selections were roughly uniform across positions for most LLMs, but human position bias was much stronger, potentially understating the LLM-human susceptibility gap.Source: §3.5 · p. 9
  • DeepSeek-R1 chain-of-thought traces mentioned song popularity in every log and both influence conditions, consistent with responsiveness to social influence.Source: §3.5 · p. 9
  • At strong influence, lowering temperature from 1.0 to 0.7 did not attenuate non-reasoning-model divergence or unpredictability; assessing 12 rather than four titles reduced both while preserving model patterns.Source: §3.5 · p. 9

4 Discussion

The discussion finds that LLM populations generally show greater herding and cascades, with less consistent self-correction, than the human benchmark. It frames these findings as a reason to study endogenous population feedback and notes that the experiment’s design limits attribution of human–LLM differences.

4 Discussion

  • Collective AI evaluation must go beyond individual responses to fixed signals and test how model outputs shape subsequent inputs.The discussion situates this need in multi-agent systems where model-generated judgments become inputs for later agents.Source: §4 · p. 10
  • Across 13 experiments, LLM populations showed generally greater susceptibility to herding and cascades and less consistent self-correction than the human benchmark.Popularity made songs with an early advantage more likely to be selected for assessment and then assessed more favorably.Source: §4 · p. 10
  • Reasoning and higher-intelligence models were less susceptible to herding, but that lower susceptibility did not ensure self-correction.The experiment omitted stronger influence mechanisms such as communication, task incentives, and social sanctioning; susceptibility may increase if greater task complexity combines with social pressure and incentives.Source: §4 · p. 10
  • The human–LLM contrast is limited because the study adapted Music Lab with different stimuli, response tasks, and presentation, so a matched human replication is needed.The study also used homogeneous single-model populations, without explicit collaboration or incentives for conformity or ground-truth accuracy.Source: §4 · p. 10

A.1 Conformity and sycophancy in individual LLMs

  • Prior conformity and sycophancy studies tested responses to supplied social information, whereas this study examines how successive recommendations create a cumulative signal and population-level cascades.Its outcomes include divergence from independent judgment, unpredictability across replicate worlds, and self-correction.Source: §A.1 · p. 16

A.2 Collective dynamics in interacting LLM populations

Prior work finds conformity, consensus, and cascades in interacting LLM populations, including in settings with sequential decisions and accumulating social signals. This study connects individual susceptibility to collective outcomes, while identifying hybrid human–AI populations as an open extension.

A.2 Collective dynamics in interacting LLM populations

  • Prior interacting-agent studies report conformity, consensus, fragmentation, shared conventions, collective biases, and populations trapped against intrinsic preferences.Some report greater conformity in smaller models and greater resistance in reasoning-optimized models.Source: §A.2 · p. 16
  • Sequential studies have found anchoring, majority dominance, and shifts toward socially favored alternatives when agents observe earlier ratings or popularity signals.These studies include sequential film ratings, endogenous microblog popularity counts, and choices shaped by engagement counts or seeded popularity.Source: §A.2 · p. 16
  • The study connects individual susceptibility to the distribution, unpredictability, and self-correction of collective recommendations in one controlled design.Agents act sequentially among fixed alternatives with independently measured intrinsic appeal; replicated worlds and temporal comparisons assess whether early choices redirect outcomes and whether recommendations regain alignment with independent appeal.Source: §A.2 · pp. 16–17

A.3 Hybrid human–AI societies and feedback loops

  • The experiment isolates a cumulative-recommendation feedback loop in homogeneous LLM populations without changing model weights or the available alternatives.Extending the design to mixed human–AI populations would test whether population composition changes cascade strength and correction.Source: §A.3 · p. 17

B Materials and Methods

The methods supplement details song stimuli, persona conditioning, prompt templates, and model inference settings. The design uses structured preference and demographic variation, with prompt and output formats held consistent across conditions apart from the influence manipulations.

B Materials and Methods

  • The supplement provides additional design details, implementation procedures, and statistical methods beyond the main-text account of worlds, population sizes, and influence conditions.The main text describes those core design elements.Source: B Materials and Methods · p. 17

B.1 Song stimuli

  • The study used 48 song stimuli, with lyric lengths ranging from 131 to 172 words and a mean of 153.8 words.Table S1 lists song titles and lyric lengths; word counts use whitespace-separated tokens, including section markers.Source: §B.1 · p. 17, Table S1 · p. 17

B.2 Persona construction and agent heterogeneity

  • Persona conditioning varied agents’ preference inputs to create a population with heterogeneous prompts, beyond variation from stochastic decoding alone.Personas included preferred genres and demographic characteristics sampled to approximate the original Music Lab participants; the authors do not claim agents reproduced individual human listeners.Source: §B.2 · p. 18

B.3 Prompts

  • Prompts used separate selection and recommendation stages, and the influence-condition templates displayed cumulative counts at both stages.In the weak-influence condition, the popularity-ordering clause was omitted and title order randomized; the independent condition omitted counts and earlier-choice references and also randomized title order.Source: Figure S1 · p. 18, §B.3 · p. 18, Figure S2 · p. 19

B.4 Models

  • Non-reasoning models used temperature 1.0, while reasoning models used medium effort where supported; settings were identical across agents within each model and condition.Other generation parameters remained at provider defaults unless otherwise specified.Source: §B.4 · p. 19

B.5 Human benchmark and design differences

The design adapted Music Lab’s parallel-world feedback structure for LLM agents and treated human results as a qualitative benchmark. It also defined measures for unpredictability, alignment with intrinsic appeal, self-correction, inequality, and displacement from independent recommendations.

B.5 Human benchmark and design differences

  • Newly generated songs reduced the risk of training exposure importing popularity signals, while lyrics enabled common text inputs; multimodal cascade dynamics remain future work.Audio evaluation was constrained by model availability and the cost of evaluating over 160,000 agents.Source: §B.5 · p. 19
  • Recommendations replaced downloads while preserving feedback: each affirmative choice increased a visible count for later agents.Source: §B.5 · p. 20
  • A fixed four-song selection budget standardized evaluation opportunities and approximated Music Lab sampling; raising it to 12 reduced divergence and unpredictability but preserved the main patterns.Music Lab participants sampled 3.8 and 3.6 songs on average in experiments 1 and 2.Source: §B.5 · p. 20
  • The list interface randomized titles under independent and weak influence but popularity-ordered them under strong influence, so weak influence preserved counts without matching the human interface.Human position bias under weak influence was not directly comparable and was not reported.Source: §B.5 · p. 20
  • Human–LLM contrasts are qualitative because stimuli, response tasks, and presentation differed; both LLM influence conditions shared one model-specific independent baseline.The human benchmark mapped weak and strong influence to Music Lab experiments 1 and 2, respectively.Source: §B.5 · pp. 19–20

B.6 Outcome measures

  • Unpredictability was the average song-level difference in market share across pairs of influence worlds; its independent baseline came from 1,000 splits into 720-agent samples.Relative unpredictability divided influence-condition unpredictability by the same model’s independent estimate; values above one indicate proportional amplification.Source: §B.6 · p. 21
  • Intrinsic-appeal alignment correlated song shares in independent and influence worlds, while early-mover prediction correlated the first 30 agents’ shares with those of later agents.The pooled correlations were descriptive; dependence among observations made conventional Pearson significance tests misleading.Source: §B.6 · pp. 21–22
  • Self-correction was measured as the change in correlation with independent-world counts between sequence halves, with second-half counts excluding first-half recommendations.Positive changes indicate closer alignment with intrinsic appeal; negative changes indicate movement away from the independent ranking.Source: §B.6 · p. 22

B.7 Robustness checks

Robustness checks varied temperature and selection budget with matched independent baselines. The intelligence analysis used a continuous benchmark score, while noting that its associations were descriptive and confounded.

B.7 Robustness checks

  • Robustness checks repeated strong influence at temperature 0.7 for eight non-reasoning models and increased the selection budget from four to 12 for all 13 models, each with a matched baseline.Other procedures were unchanged; results were reported in figures S7–S9.Source: §B.7 · p. 23

C.1 Model intelligence and susceptibility to social influence

  • Reasoning models averaged 32.0 on the Artificial Analysis Intelligence Index versus 11.4 for non-reasoning models, but the classes overlapped and associations were descriptive, not causal.The analysis did not control for factors including reasoning class, scale, architecture, training, and post-training.Source: §C.1 · p. 23
  • Table S3 reports Artificial Analysis Intelligence Index scores by developer-designated reasoning class; higher scores indicate greater benchmarked intelligence.Source: Table S3 · p. 23

C.2 Social influence increased inequality of market shares

Social influence increased market-share inequality relative to independent evaluation, though the magnitude varied across reasoning classes. Inequality did not significantly rise as influence strengthened.

C.2 Social influence increased inequality of market shares

  • Non-reasoning models’ mean Gini rose from 0.395 independently to 0.737 under weak and 0.838 under strong influence, while reasoning models rose from 0.528 to 0.657 and 0.722.The strong-influence increase was larger for non-reasoning models than reasoning models (p < 0.05); non-reasoning increases exceeded the human benchmark, while reasoning increases were smaller.Source: §C.2 · p. 24
  • Gini was higher under strong than weak influence in 9 of 13 models, but the difference was not significant (p > 0.05); similar inequality can concentrate recommendations on different songs.Source: §C.2 · p. 24
  • Gini averaged 0.706 under weak influence and 0.794 under strong influence, versus 0.446 independently; increases were 0.260 and 0.348 (p < 0.001 for both).Inequality exceeded baseline in all 13 models under strong influence and 12 of 13 under weak influence; GPT-5 nano was the exception.Source: Figure S3 · p. 24, §C.2 · p. 24

C.3 Social influence attenuated but did not eliminate the correspondence between intrinsic appeal and popularity

Cumulative recommendation counts weakened the alignment between popularity and intrinsic appeal, while early agents’ choices often predicted later popularity better than intrinsic appeal did. This pattern was stronger among non-reasoning models, whereas humans showed it only under strong influence.

C.3 Social influence attenuated but did not eliminate the correspondence between intrinsic appeal and popularity

  • Mean correlation between popularity and independent evaluation fell from r = 0.584 under weak influence to r = 0.472 under strong influence.Correlations were lower under strong influence in 11 of 13 models; the difference was statistically significant across models (p < 0.01).Source: §C.3 · p. 25
  • Reasoning models retained stronger alignment with independent distributions than non-reasoning models under both weak and strong influence.Mean correlations were 0.738 and 0.689 for reasoning models versus 0.487 and 0.336 for non-reasoning models; humans were close to reasoning models under weak influence and between the two classes under strong influence.Source: §C.3 · p. 25
  • Popularity generally remained positively associated with intrinsic appeal, despite the weakening under social influence.Songs favored under independent evaluation generally remained more popular under social influence.Source: §C.3 · p. 25

C.4 Early movers predicted later popularity better than intrinsic appeal

  • Early movers’ choices predicted later popularity better than intrinsic appeal in all 13 models under strong influence and 12 of 13 under weak influence.The exception under weak influence was GPT-5 (0.823 vs. 0.885); the comparisons were significant across models under both influence conditions.Source: §C.4 · p. 25
  • Humans showed early movers’ choices outweighing intrinsic appeal only under strong influence, unlike LLMs, where early movers outweighed it under both influence levels.For humans, the early-mover correlation was 0.316 versus 0.740 for intrinsic appeal under weak influence, and 0.723 versus 0.557 under strong influence.Source: §C.4 · p. 25

C.5 Position bias could provide an alternative explanation for the effects of strong influence

The position-bias analysis suggests that strong-influence outcomes may partly reflect agents’ preferences for early list positions, while most LLMs showed little such bias in randomized conditions. Across trajectories, divergence persisted, whereas unpredictability generally stabilized or declined over time.

C.5 Position bias could provide an alternative explanation for the effects of strong influence

  • Most LLMs’ selections were approximately uniform across positions under randomized independent and weak-influence conditions, while humans showed stronger position bias than most agents.Gemma 3 4B and GPT-4o mini showed modest early-position preferences, still closer to uniformity than the strong-influence distributions.Source: §C.5 · p. 26, Figure S4 · p. 27
  • If position bias generalizes from the independent condition, it could explain part of strong-influence concentration, especially among humans, and make measured human–LLM differences appear smaller than they are.The text notes that this inference awaits an experiment controlling for position bias.Source: §C.5 · p. 26
  • Even if position bias contributes to strong-influence effects, it cannot explain the weak-influence divergence, where list order was randomized and every model diverged from its independent baseline.The passage also notes that path-dependent sensitivity could let even a small position bias tip a system toward whatever appeared at the top early on.Source: §C.5 · p. 26

C.6 Trajectories of cumulative advantage

  • KL divergence from independent distributions persisted over the observed trajectories, though its development varied across models and worlds.Trajectories plot accumulated recommendations for LLMs and downloads for humans; final values match the averages reported in the main results.Source: §C.6 · p. 27, Figure S5 · p. 28
  • Unpredictability generally stabilized or declined over time, and strong influence produced greater final unpredictability than weak influence in most models.Unpredictability compares song market shares across pairs of worlds as recommendations accumulated.Source: Figure S6 · p. 29

C.7 Robustness analyses

Robustness checks varied sampling temperature and the number of songs agents could assess, comparing independent and strong-influence conditions. Lower temperature did not attenuate social influence, and expanding the consideration set reduced but did not eliminate divergence and unpredictability.

Robustness analyses

  • The temperature check varied the non-reasoning models’ sampling temperature, while the consideration-set check changed selections from four to 12 songs across all 13 models.Both checks used independent and strong-influence conditions.Source: §C.7 · pp. 29–30

Robustness to sampling temperature

  • Mean KL divergence was 1.62 at temperature 1.0 and 1.75 at temperature 0.7, with greater divergence at the lower temperature in seven of eight models.Model ordering was identical across temperatures (Spearman ρ = 1.00); influence-world unpredictability was 0.0292 and 0.0306, respectively.Source: §C.7 · p. 29, Figure S7 · p. 30

Robustness to the size of the consideration set

  • Mean KL divergence fell from 1.19 with four selections to 0.75 with 12, but remained nonzero in every model and preserved model ordering (Spearman ρ = 0.95).With 12 selections, mean KL divergence was 0.31 among reasoning models and 1.03 among non-reasoning models.Source: §C.7 · p. 30, Figure S8 · p. 31
  • Influence-world unpredictability fell from U = 0.0239 to U = 0.0201 with 12 selections, yet remained above independent-condition unpredictability at both selection sizes.With 12 selections, the lowest influence value (U = 0.0048) was approximately 2.6 times the highest independent value (U = 0.0019); mean Uinf/Uind was 8.6 with four selections and 13.0 with 12.Source: §C.7 · p. 30, Figure S9 · p. 31

C.8 Limitations and directions for future work

The study identifies limits in its interaction setup and population composition, and points to testing recommendations in a consequential domain beyond songs.

C.8 Limitations and directions for future work

  • Agents observed accumulated counts without discussing choices or retaining shared conversational history, leaving those interaction features untested.Future work could examine whether argument exchange, memory, reconsideration, or conformity incentives amplify or attenuate cumulative-advantage dynamics.Source: §C.8 · p. 32
  • Each population used a single model, with heterogeneity introduced through personas rather than mixtures of models or humans and AI agents.The authors identify mixed-capability and mixed human–AI populations as directions for future study.Source: §C.8 · p. 32
  • Whether exposure to earlier recommendations improves evaluation quality or drives acceptance-and-rejection cascades remains to be tested in AI-assisted peer review.The proposed application substitutes academic-paper recommendations for song recommendations.Source: §C.8 · p. 32
Loading 2610.30769v1…