PaperlayerView exact revision on arXiv

Source-linked AI summary

Gossip Agents: Reputation through Opinion Dynamics over Evolving Networks

Yutong Wu, Arlei Silva

arXiv:2610.22674v1cs.MA

TL;DR

Existing reputation approaches often make reputations public or keep them private to direct observers, leaving how opinions spread through changing networks comparatively unexplored. This paper models private-opinion exchange over evolving networks and tests its effects in two social dilemmas. Reputation promotes cooperation in both settings, while in Harvest broadcast reach shapes utility, sustainability, and other collective outcomes.

  • Problem

    Gossip-based reputation must address differing private opinions and changing communication networks, while prior gossip approaches typically assume a fixed graph.

    Source: §1 · p. 1

  • Method

    Agents form private assessments, aggregate reputation signals into rewards, and exchange opinions through DeGroot averaging, with broadcast radius setting information reach.

    Source: §1 · p. 2, §4.1 · p. 4

  • Results

    Reputation promotes cooperation across broadcasting settings in the Prisoner’s Dilemma; in Harvest, utility and sustainability increase with broadcast radius, alongside trade-offs across metrics.

    Source: §5.4.1 · p. 7, §5.4.2 · p. 8

  • Takeaways & Limitations

    In Harvest, Local Broadcast largely avoids the extremes’ trade-offs, retaining high equality and peacefulness while recovering most sustainability benefits and consistency associated with Broadcast All.

    Source: §5.4.2 · p. 8

    Assessments and opinion aggregation are fixed rather than learned, leaving open whether learned mechanisms remain effective as communication networks evolve.

    Source: §6 · p. 8

Abstract

from arXiv · show

Sustained cooperation among self-interested learning agents is essential in many shared environments. Reputation systems are among the best-studied mechanisms for promoting such cooperation. Existing multi-agent reinforcement learning approaches, however, typically assume that reputation is either globally available to all agents or derived solely from an agent's own observations, leaving largely unexplored how reputational information propagates between agents. We study gossip, the exchange of information about third parties, as a mechanism for reputation diffusion over dynamic communication networks. Agents form private opinions from limited observations and exchange them with their neighbors following DeGroot opinion dynamics, while the communication network changes as agents interact and move. A tunable broadcast radius controls the reach of information, spanning private reputation, local gossip, and globally shared reputation. We study this framework in two \(n\)-player social dilemmas with distinct network dynamics: a Prisoner's Dilemma with randomly resampled interaction groups and Harvest, where communication neighborhoods emerge from agents' movements. Our results show that the reach of reputational information shapes cooperation in both environments. Moreover, in Harvest, reputation diffusion increases environmental rewards and leads to more peaceful and sustainable behavior.

1 Introduction

Reputation can promote cooperation among self-interested agents, but existing approaches often make reputations either universally available or private to the observer. This work studies how gossip carries private opinions through changing communication networks.

1 Introduction

  • Reputation systems promote cooperation by making past conduct affect how others interact with an agent.Under indirect reciprocity, defection can carry costs through others’ later withdrawal of cooperation rather than direct retaliation.Source: §1 · p. 1
  • Existing approaches typically make reputation globally available or limit it to the agent who observed the behavior.Gossip offers an intermediate route by sharing third-party assessments with agents who did not witness the underlying behavior.Source: §1 · p. 1
  • Different private observations can produce conflicting opinions, and gossip may spread disagreement or consensus around an inaccurate assessment.Communication networks also change as agents interact, whereas existing gossip approaches typically assume a fixed graph.Source: §1 · p. 1
  • The framework separates private assessment, reputation aggregation, and DeGroot-based opinion exchange, with broadcast radius controlling information reach.The radius spans private, local, and global reputation; communication networks are redrawn through random regrouping or agent movement.Source: §1 · pp. 1–2
  • The study examines reputation diffusion in an n-player Prisoner’s Dilemma and Harvest, and reports that it encourages cooperation in both games.In Harvest, it additionally encourages more peaceful and sustainable behavior.Source: §1 · p. 2

2 Related Work

Prior cooperation research uses mechanisms that alter learning, incentives, preferences, or partner choice. Reputation research has examined both public scores and private assessments, while this work focuses on opinion diffusion over evolving communication networks.

Cooperation in Mixed-Motive Games

  • Mixed-motive game approaches include opponent-shaping, reciprocity-based policies, and partner selection as ways to promote cooperation.Partner selection allows agents to choose whom to interact with.Source: §2 · p. 2
  • Harvest research has used social preferences and incentive mechanisms, whereas this work studies cooperation emerging through reputation and gossip.Altruistic agents can increase collective return at the cost of equality.Source: §2 · p. 2

Reputation in Multi-Agent Reinforcement Learning

  • Public-reputation methods share a common score, while this work models private opinions spreading through neighboring agents before aggregation.Prior public approaches do not make reputation access depend on the communication network [26].Source: §2 · p. 2
  • LR2 confines assessments to directly encountered agents, while COOPER uses a static graph and two-player games; this work studies n-player dilemmas with changing networks.The changes may be exogenous through random regrouping or endogenous through learned movement [48].Source: §2 · p. 2

Evolutionary Biology

  • Evolutionary-biology work studies gossip and cooperation across populations, while this paper examines private-assessment diffusion among reinforcement-learning agents.Prior work includes analysis of the gossip needed for sufficient consensus to stabilize cooperation.Source: §2 · p. 2

Opinion Dynamics and Evolving Networks

  • The paper brings opinion dynamics and evolving-network research together to study how reputation opinions shape cooperation among learning agents.It uses DeGroot averaging, a model in which agents repeatedly average neighbors’ opinions.Source: §2 · pp. 2–3

3 Preliminaries

The preliminaries define the two social-dilemma settings used in the study. The Prisoner’s Dilemma is extended to randomly formed n-agent groups, while Harvest models resource depletion and regrowth in a partially observable environment.

3.1 Social Dilemmas

  • A Prisoner’s Dilemma has temptation, reward, punishment, and sucker payoffs ordered T > R > P > S, making defection dominant despite mutual cooperation being preferred to mutual defection.In the n-agent extension, agents are randomly partitioned into neighborhoods and receive mean pairwise payoffs; defection remains dominant at neighborhood level.Source: §3.1 · p. 3

3.2 Sequential Social Dilemmas

  • Harvest is a partially observable gridworld where agents collect apples that regrow according to local density but not in depleted regions or on occupied tiles.Aggressive harvesting can deplete the shared resource, while sustainable harvesting preserves it and yields higher long-term returns; agents can also use time-out beams to remove others temporarily.Source: §3.2 · p. 3
  • Harvest’s prior three-stage dynamic moves from exploration to resource-depleting overharvesting and then strategic time-outs that let remaining agents harvest regrowth.Sustainability emerges through exclusion rather than voluntary restraint, motivating reputation as an alternative mechanism.Source: §3.2 · p. 3

4 Methods

The framework lets learning agents form private opinions from local observations, share them through parameterized gossip, and aggregate reputations that shape rewards and decisions.

4 Methods

  • Agents form private opinions, exchange them with neighbors, and aggregate them into reputation signals that shape rewards; opinions of others can also inform decisions.Agents are not given hand-engineered prosocial strategies; they must discover them in response to reputation signals.Source: §4 · p. 3, §4.1 · p. 3

4.1 Reputation Framework

  • Agents update opinions from directly observed prosocial or antisocial behavior; absent new evidence, the prior opinion carries forward unchanged.An exponential moving average weights the previous opinion against a new nonzero observation, and events outside an agent’s view do not affect its signal.Source: §4.1 · pp. 3–4
  • The architecture proceeds through local opinion formation, temporal aggregation into current-step reputation, and broadcast of reputation information.Source: Figure 1 · p. 4
  • DeGroot-style averaging combines neighbors’ raw opinions, while broadcast radius controls how far reputational information propagates.Each agent’s communication neighborhood includes itself and agents within distance ρ.Source: §4.1 · p. 4
  • The radius spans private information with no broadcast, local word-of-mouth, and population-wide sharing, while opinion formation and aggregation remain fixed.These protocols correspond to ρ = 0, 0 < ρ < ∞, and ρ = ∞, respectively.Source: §4.1 · p. 4
  • A new reputation entry is added when an opinion about an agent is updated, and its reputation averages the K most recent entries.If the agent is not observed at a step, no entry is added and its previous reputation carries forward.Source: §4.1 · p. 4

5 Experiments

The experiments compare reputation diffusion at private, local, and population-wide broadcast radii against a no-reputation baseline in a randomly regrouped Prisoner’s Dilemma and Harvest. Reputation methods outperform the baseline, while Harvest results reveal trade-offs across utility, sustainability, peacefulness, and equality.

5.1 Environment

  • The experiments use 15 agents in randomly redrawn five-agent Prisoner’s Dilemma groups and 10 agents in Harvest, where opinions enter observations as perceptual features.The No Reputation baseline uses only extrinsic reward; the three reputation methods differ by broadcast radius.Source: §5 · p. 5

5.2 Opinion Formation

  • Private assessments are aggregated into reputation signals and exchanged using DeGroot opinion dynamics; in the Prisoner’s Dilemma, assessments condemn exploitation rather than defection itself.In Harvest, rules treat resource exploitation, failure to conserve when no apples are visible, and aggression toward agents in good standing as antisocial; sanctioning low-reputation agents carries no reputational reward or penalty.Source: §5.2 · p. 5, §5.2.2 · p. 6

5.4 Results & Discussion

  • Reputation promotes Prisoner’s Dilemma cooperation across broadcast settings, while the broadcast radius has little effect on the final cooperation level.The settings converge similarly; Broadcast All converges more slowly and varies most, while Local Broadcast at ρ=10 converges slightly faster than at ρ=5.Source: Figure 3 · p. 6, §5.4.1 · p. 7
  • In Harvest, all reputation methods converge above 110 utility, versus roughly 50 for No Reputation; Broadcast All eventually surpasses the other methods after about 5000 episodes.Broadcast All improves more slowly early in training.Source: Figure 4 · p. 6, §5.4.2 · p. 8
  • Harvest sustainability increases with broadcast radius, and both broadcasting methods show less run-to-run variation than No Broadcast.The No Reputation baseline declines from approximately 480 to 260 on sustainability.Source: Figure 7 · p. 7, §5.4.2 · p. 8
  • Local Broadcast provides the most balanced Harvest performance, retaining high equality and peacefulness while recovering most sustainability benefits and consistency associated with Broadcast All.Broadcast All has the highest utility, sustainability, and punishment accuracy, but lower equality and slightly lower peacefulness; No Broadcast has substantial run-to-run variation.Source: §5.4.2 · p. 8

6 Conclusion & Future Work

The framework combines private opinions, DeGroot exchange, and reputation-shaped rewards over evolving networks; agents learned cooperative strategies in both studied environments without cooperation being explicitly prescribed.

6 Conclusion & Future Work

  • The framework has agents form local opinions, exchange them through DeGroot averaging, and receive reputation-shaped rewards as communication networks evolve.It was studied in a randomly regrouped n-player Prisoner’s Dilemma and Harvest, where networks change with agent movement.Source: §6 · p. 8
  • Reputation-enabled Prisoner’s Dilemma agents cooperated in approximately 95% of rounds across broadcast radii, while agents without reputation converged to defection.The broadcast radius had little effect on average cooperation, although population-wide broadcasting produced greater variation across runs.Source: §6 · p. 8
  • In Harvest, reputation more than doubled utility relative to baseline and promoted more peaceful and sustainable behavior.Larger broadcast radii further improved sustainability and reduced variance, while population-wide broadcasting had the highest utility and sustainability but the lowest equality; local broadcasting retained most benefits while preserving equality and peacefulness.Source: §6 · p. 8
  • Assessment rules and opinion aggregation are fixed, leaving open whether learned assessment and aggregation remain effective as communication networks evolve.Agents are also assumed to communicate opinions truthfully; strategic or noisy gossip remains a future direction.Source: §6 · p. 8

B.1 Prisoner’s Dilemma

The Prisoner’s Dilemma agents have two available actions and use a compact two-hidden-layer network architecture.

B.1 Prisoner’s Dilemma

  • Each agent chooses between two actions: cooperate or defect.The default network has two hidden layers of 64 units, rectified linear activations, and a two-unit output layer.Source: §B.1 · p. 10
  • The default neural network uses two hidden layers of 64 units with rectified linear layers and a two-unit output layer.The output units correspond to the two available actions.Source: §B.1 · p. 10

B.2 Harvest

Harvest agents use an eight-action space and a convolutional network followed by two hidden layers.

B.2 Harvest

  • Each Harvest agent has eight actions, including movement, turning, staying, and firing a beam.The listed actions are move left, right, up, or down; stay; turn clockwise or counter-clockwise; and fire beam.Source: §B.2 · p. 10
  • The default network uses a 32-filter convolution layer with a 3 × 3 kernel, then hidden layers of 32 and 16 units and an eight-unit output layer.The layers use rectified linear activations; output units correspond to actions.Source: §B.2 · p. 10

C.1 Ablation Study on Harvest Reputation Rules

The Harvest reputation-rule ablation fixes the broadcast radius at infinity and reports results across five runs.

C.1 Ablation Study on Harvest Reputation Rules

  • The Harvest ablation on rule 2 fixes ρ at infinity and averages five independent runs.Shaded regions show variance across runs; the supplied passage does not state the ablation outcome.Source: §C.1 · p. 10
Loading 2610.22674v1…