Peer Influence across Heterogeneous AI Models

1 Introduction

• Prior work on LLM opinion simulations often tracks general stances rather than joint task performance, and often uses agents with the same underlying model.Research on heterogeneous systems has largely emphasized negotiation or overall system performance rather than microscopic influence dynamics.Source: §1 · p. 2
• The study introduces a framework for measuring peer influence in heterogeneous agents performing joint decision-making tasks.A judge and peer first perform a task independently; the judge then repeats it while considering both outputs, and influence is estimated from its output across rounds.Source: §1 · p. 2
• The framework tests seven models across three binary text-classification tasks to assess how influence relates to model pairing, size, and baseline confidence.The models span different architectural families and parameter sizes; the evaluation covers 238,682 text instances.Source: §1 · p. 2
• Peer influence is strong across tasks, and the shift depends more on the judge’s susceptibility than on the peer’s influence power.Some models remain susceptible despite high confidence in their initial judgments.Source: §1 · p. 2
• Influence is asymmetric, with smaller models often matching larger models’ persuasive power.The magnitude of shifts varies across model families.Source: §1 · p. 2
• Model heterogeneity has non-uniform effects: some pairings produce larger shifts than homogeneous systems, while others produce smaller ones.The findings also include cases where a dissenting peer prompts a model to reinforce its prior judgment.Source: §1 · pp. 2–3
• The authors provide a methodology to test reciprocal influence of AI agents in heterogeneous decision-making systems and highlight model heterogeneity’s effect on output stability, enriching the toolkit for assessing and enhancing multi-agent security [26] [46].Source: §1 · p. 3

2.1 Decision-making framework

• The judge reconsiders its first-round decision after receiving a disagreeing peer’s label and explanation; repeated trials estimate the probabilistic influence of that peer.The framework supports heterogeneous or homogeneous model pairs and can swap judge and peer roles.Source: §2.1 · p. 3

2.2 Datasets

• Three binary classification datasets span sentiment analysis, CommonsenseQA 2.0, and sarcasm detection, with reasonably balanced labels across tasks.Sentiment is described as relatively easy, while sarcasm detection is the most challenging task.Source: §2.2 · pp. 3–4

2.3 Models and parametrization

• The evaluation uses seven open-weight models from four providers, including smaller and larger models within the Gemma, Llama, and Qwen families.All models use 4-bit bitsandbytes quantization; the study compares largest-family models pairwise and also pairs sizes within families.Source: §2.3 · p. 4

3 Results

• Classification is generally consistent across repetitions, but gpt-20b labels 16% to 58% of samples inconsistently by task and llama-8b does so for 43% of Commonsense samples.Among samples with fully certain labels, class balance varies by task, skewing positive in Sarcasm despite balanced ground truth.Source: §3 · p. 4

3.1 Peer Influence

• Influence scores exceed 0.5 in most model-pair cases and often peak above 0.75, with judges’ probability mass concentrating on the peer’s suggested label.The reported margin of error is at most 0.01 for all judge, peer, and dataset combinations.Source: §3.1 · p. 5, Figure 1 · p. 5
• Influence tends to be directional within fixed model pairs: an influential peer is likely less susceptible as judge, yet aggregate high persuasiveness need not mean low susceptibility; Gemma-27b is the clearest example.Source: §3.1 · pp. 5–6
• The most susceptible judges can also be highly certain in isolation: gemma-27b labels 99.95% of Sarcasm samples consistently yet has susceptibility 0.89.Across pooled model pairs and datasets, certainty and susceptibility have a positive Spearman correlation (ρ = 0.61, p = 0.036).Source: §3.1 · p. 6
• Peer-judge interactions backfire at about 15% on average in Sarcasm, 12% in Commonsense, and 7% in Sentiment, with peaks exceeding 40% for some model combinations.Source: §3.1 · p. 6

3.2 Homogeneous vs. heterogeneous interactions

• Heterogeneity does not uniformly raise or lower susceptibility: gpt-20b becomes 17% to 79% less susceptible, while llama-70b becomes 13% to 27% more susceptible.The direction and extent of the change depend on model and dataset; llama-70b also becomes less likely to backfire in heterogeneous systems.Source: §3.2 · pp. 6–7

3.3 Impact of model size

• Among heterogeneous same-family interactions, the systematic size advantage in persuasiveness disappears; smaller Gemma and Llama models are comparable or more influential, unlike smaller Qwen models.Smaller peers trigger stronger backfiring overall, clearest in Gemma and Qwen systems and weaker in Llama systems.Source: §3.3 · p. 7

3.4 Sensitivity analyses

• Peer influence scores showed little to no difference across temperatures 0.2, model default, and 1.2; in comparison with the main experiment, Qwen-72b showed a substantial increase in positive influence.Source: §3.4 · pp. 8–9
• Without explanations, Llama-70b remains stable, while Qwen-72b shows higher positive influence and reduced backfiring.The Qwen-72b pattern mirrors the memoryless condition; the results suggest reasoning contributes to stability and removing it can leave a model more vulnerable.Source: §3.4 · p. 9
• In the explanation swap, both models favored GPT-20b’s label-explanation pairs regardless of presenter, suggesting its reasoning was more persuasive.The experiment swapped label-explanation pairs between peer and judge to test whether explanation content mattered more than agent identity.Source: §3.4 · p. 9
• Llama-70b became less susceptible to its own reasoning, while GPT-20b became more susceptible and less prone to backfire when shown its own reasoning.This suggests Llama-70b’s explanations are less compelling as external input than GPT-20b’s.Source: §3.4 · p. 9

• Influence patterns depend on the model pairing: shift magnitude reflects receiver susceptibility more than speaker persuasiveness, grows on harder tasks, and can sometimes reinforce prior opinions.Source: §4 · p. 9
• Neither model scale nor isolated certainty reliably predicts resistance: small models can persuade and resist as effectively as larger models, while certain agents may remain susceptible.Source: §4 · p. 9
• Compared with homogeneous counterparts, heterogeneous systems can either reduce or increase final-outcome variability, depending on model choice.Source: §4 · p. 9
• Round 2 interaction mostly improved F1 on harder tasks, especially Sarcasm, while Sentiment generally declined; heterogeneous pairs performed better, and smaller judges benefited more from larger peers.Source: §4 · p. 9
• The framework is limited to single-turn interactions on binary classification tasks, leaving broader, more complex systems outside its scope.Source: §4 · p. 10
• Evidence comes from a limited pool of open-source quantized models and sampling settings, while equal configuration weighting differs from the concentrated empirical distribution.Source: §4 · p. 10
• Model mix is a first-order design choice: it shapes system stability, which increases with intrinsic task uncertainty, so easy-task stability does not assure hard-task stability.Source: §4 · p. 10
• In shared workflows, a contrarian agent powered by a relatively small model can effectively reverse outcomes; adversarial capability is not gated by scale.Source: §4 · p. 10

5.1 Influence estimation

• Repeated M-trial rounds compare the judge’s initial label distribution with its distribution after reconsidering its own output alongside the peer’s label and explanation.The judge and peer first answer independently; the judge then produces a final label without an explanation.Source: §5.1 · p. 10
• Positive shifts toward either label contribute to total influence, while shifts reinforcing the judge’s initial preference are reported separately as backfiring scores.The authors also report the probability of backfiring.Source: §5.1 · p. 11

5.2 Match Cases

• Seven match cases represent the possible round-1 label patterns that permit disagreement, including cases where one or both agents produce both labels.Each instance falls into exactly one case; homogeneous pairs can disagree only in the B:B case.Source: §5.2 · p. 12
• Upsampling gives each available influence direction M repetitions, keeping interaction counts consistent despite skewed label distributions.Upsampling is used because label distributions can be skewed for some model-sample pairs.Source: §5.2 · p. 12
• Influence is computed separately for each match case and macro-averaged across cases; each case is capped at 7,000 samples.With M=10, this amounts to 70,000 generations per model, match case, and dataset.Source: §5.2 · p. 12
• Agreeing interactions serve as a robustness check, with almost no opinion shifts observed.These interactions begin with both judge and peer producing the same label.Source: §5.2 · p. 12

• Commonsense was already well curated and required no additional preprocessing.Source: S1 Data preprocessing · p. 16
• Sarcasm claims shorter than six words and claims touching on specified sensitive topics were filtered out to provide semantic context and avoid guardrail effects.Source: S1 Data preprocessing · p. 16
• Sentiment preprocessing removes semi-duplicates with the same label but retains examples where added text changes the sentiment.For instance, an appended phrase that flips a positive example to negative is retained.Source: S1 Data preprocessing · p. 16

• Further model details are available on the models’ original Hugging Face repository pages.Source: S2 Model Specifications · p. 16
• Table S1 specifies the models and defines “Sampling params” as whether a repository provides a generation configuration file with default sampling parameters.Source: Table S1 · p. 16, S2 Model Specifications · p. 16
• The Gemma models and GPT-20b lack sampling parameters specified in their repositories, so the authors used recommendations from online sources and the official GPT-OSS repository, respectively.Source: S2 Model Specifications · pp. 16–17

• Figure S1 presents the system prompt used for all model calls.Source: Figure S1 · p. 17, S3 Prompts · p. 17
• Figure S2 presents round-1 and round-2 prompts, with slightly different prompts for Sentiment, Commonsense, and Sarcasm.Source: Figure S2 · p. 17, S3 Prompts · p. 17
• In Figure S2’s round-2 prompt illustration, blue and red distinguish judge and peer label-explanation pairs from round 1.Source: Figure S2 · p. 17

S4 Discarded claims from invalid outputs

• Only a very small number of round-1 outputs with invalid formatting were discarded, and those instances were excluded from later analyses.Table S2 reports discarded-instance counts by model and dataset.Source: S4 Discarded claims from invalid outputs · p. 17, Table S2 · p. 18

S5 Influence heatmaps

• Low diagonal mass across model combinations indicates instability, and only GPT-20B shows appreciable correlation between baseline and post-interaction certainty.For other models, post-interaction certainty mass concentrates at 0 or 1.Source: S5 Influence heatmaps · p. 18, Figure S3 · p. 19, Figure S4 · p. 20

S6 Influence scores on agreeing interactions

• When peers propose labels matching judges’ opinions, models remain stable and interactions yield near-zero backfiring.The agreeing-interaction check uses Sarcasm instances where the judge already assigns all probability mass to the peer’s label.Source: S6 Influence scores on agreeing… · p. 22, Figure S6 · p. 22

S7 Backfiring probability

• The supplement reports the probability that peers induce backfiring across model pairs and datasets, including pairs from the same family with varying sizes.Figures S7 and S8 present these probabilities.Source: S7 Backfiring probability · p. 23, Figure S7 · p. 23, Figure S8 · p. 23

S8 Influence Comparison between Homogeneous and Heterogeneous Systems

• The homogeneous-versus-heterogeneous comparison reports receiver susceptibility differences, while Figure S9 plots positive influence against backfiring by peer agent.Stars and circles distinguish homogeneous and heterogeneous systems; positions near the origin indicate non-influential peers.Source: Table S3 · p. 24, S8 Influence Comparison between… · p. 24, Figure S9 · p. 24

S9 Prompt Variations for Sensitivity analysis

• The sensitivity analysis varies prompts to use a memoryless judge, omit explanations, or swap the judge’s and peer’s label-explanation pairs.These prompt variants are shown in Figure S10.Source: Figure S10 · p. 25, S9 Prompt Variations for Sensitivity… · p. 25

S10 Comparison of Model Explanations

• GPT-20B’s explanations are judged more compelling than Llama-70B’s, and both models systematically prefer GPT-20B’s reasoning regardless of whether it is framed as a peer opinion or a model’s own prior.GPT-20B averages 23 words per explanation versus 7 for Llama-70B; its longer explanations support more elaborate arguments, while Llama-70B tends to assert labels briefly.Source: S10 Comparison of Model Explanations · p. 25

S11 Classification Performance

• The classification-performance figures compare round-2 F1 after peer interaction with the judge’s round-1 F1 in isolation; positive differences indicate improvement.One figure covers model pairs generally and another pairs from the same family with different sizes.Source: Figure S11 · p. 26, Figure S12 · p. 26, S11 Classification Performance · p. 26

S12 Influence Scores on Match Type-Level

• The match-type analysis reports influence and backfiring scores with instance counts, distinguishing agreeing 0:0 and 1:1 cases from disagreeing configurations.A dash marks cases where judge certainty makes positive and negative shifts impossible to compute.Source: S12 Influence Scores on Match Type-Level · pp. 27, 30, Table S4 · p. 30