PaperlayerView exact revision on arXiv

Source-linked AI summary

Evolutionary dynamics with game transitions

Qi Su, Alex McAvoy, Long Wang, Martin A. Nowak

arXiv:1905.10269v2q-bio.PE

TL;DR

The paper asks how cooperation can evolve when environments change rather than remain fixed. It models behavior-dependent game transitions on structured populations and finds that weak selection favors cooperation when b/c > k − k′, with transitions potentially lowering the threshold even when individual games disfavor cooperation.

  • Problem

    Fixed-game assumptions may oversimplify environments that change over time, while cooperation is difficult to establish in highly connected populations.

  • Method

    The paper models cooperators and defectors on graphs whose interactions use games that update according to individuals’ behaviors and previously played games.

  • Results

    Weak selection favors cooperation when b/c > k − k′, where k′ captures game-transition effects, and slight game differences can markedly reduce the cooperation threshold.

  • Takeaways & Limitations

    Game transitions can support cooperation in highly connected populations, including cases where cooperation is disfavored in each individual game.

  • Takeaways & Limitations

    The analysis assumes donation games in its multi-state treatment and focuses particularly on weak selection.

Abstract

from arXiv · show

The environment has a strong influence on a population's evolutionary dynamics. Driven by both intrinsic and external factors, the environment is subject to continual change in nature. To capture an ever-changing environment, we consider a model of evolutionary dynamics with game transitions, where individuals' behaviors together with the games they play in one time step influence the games to be played next time step. Within this model, we study the evolution of cooperation in structured populations and find a simple rule: weak selection favors cooperation over defection if the ratio of the benefit provided by an altruistic behavior, $b$, to the corresponding cost, $c$, exceeds $k-k'$, where $k$ is the average number of neighbors of an individual and $k'$ captures the effects of the game transitions. Even if cooperation cannot be favored in each individual game, allowing for a transition to a relatively valuable game after mutual cooperation and to a less valuable game after defection can result in a favorable outcome for cooperation. In particular, small variations in different games being played can promote cooperation markedly. Our results suggest that simple game transitions can serve as a mechanism for supporting prosocial behaviors in highly-connected populations.

1. Introduction

The paper addresses how game transitions may relax the high cooperation threshold in highly connected populations. It proposes coupling changing environments to individuals’ behaviors and finds that even slight differences between games can strongly favor cooperation.

  • Motivation: Highly connected populations can require benefits at least 36 times greater than costs under the rule b/c > k.A French high-school contact network has k = 36, while collegiate Facebook networks can have still larger mean degrees.
  • Contribution: Game transitions are proposed as a way to relax the cooperation threshold below the mean number of neighbors.The model links the games played next to individuals’ behaviors and the games played in the current time step.
  • Motivation: Many evolutionary models assume a fixed game, although experimental evidence shows that environments often change over time.The paper illustrates this with pasture degradation caused by overgrazing and subsequent resource constraints.
  • Contribution: Under game transitions, cooperation evolves when b/c > k − k′, where k′ captures transition effects.The paper reports that cooperation can be favored even when it is disfavored in every individual game.
  • Contribution: Slight differences between games can dramatically lower the barrier to cooperation and support prosocial behavior.The proposed mechanism is intended to explain cooperation in structured populations, including highly connected networks.

2. Model

The model places cooperators and defectors on a graph, lets them play potentially different games with each neighbor, and updates reproduction and games over time. Behaviors and prior games jointly determine subsequent games, coupling evolutionary change to environmental change.

  • Population and payoffs: Individuals occupy graph nodes, interact with every neighbor, and can play distinct games across different interactions.Mutual cooperation, mutual defection, and unilateral cooperation receive game-specific rewards, punishments, sucker’s payoffs, and temptations.
  • Population and payoffs: The model assumes each game is a prisoner’s dilemma with payoff ranking Ti > Ri > Pi > Si.Each player sums payoffs across interactions before those payoffs are translated into reproductive fitness.
  • Selection: Under weak selection, fitness is f = 1 − δ + δπ with 0 < δ ≪ 1.The parameter δ represents selection intensity.
  • Updating: A death-birth update randomly removes one player, after which neighboring offspring compete for the vacant site in proportion to fitness.Games then update based on the prior games and actions; the replacement player inherits games determined by the prior occupant’s interactions.
  • Game transitions: Game transitions may be deterministic or stochastic and may depend on prior games, behaviors, both, or neither.The constant single-game model is recovered as a special case.

3. Results

Game transitions can lower the threshold for cooperation in structured populations, including highly connected graphs where static games make cooperation difficult. The outcome depends on how actions change future games, especially when mutual cooperation leads to a more valuable game and defection to a less valuable one.

  • Under death-birth updating, cooperation can be favored even when both games individually oppose it, provided the benefit difference between games is sufficiently large.For graphs of any degree, b1 − b2 > 2c warrants cooperation over defection.
  • For k = 100 and c = 1, ∆b = 1.0 lowers the critical ratio (b1/c)∗ from 100 to 50.5.The reduction occurs despite only a slight difference between the two games.
  • When mutual cooperation leads to game 1 and other action profiles to game 2, cooperation can evolve under birth-death and pairwise-comparison updating.Without transitions, cooperation is never favored under these updating rules; with transitions, b1 − b2 > c/ξ supports cooperation, with ξ = 1/2.
  • The model applies to regular, random, and scale-free population structures and extends to global and local game transitions with modified transition effects.For randomly selected updates, the original threshold equations still predict the evolutionary outcome; local transitions preserve the rules when ξi is modified.
  • Across deterministic transition patterns, cooperation is promoted when mutual cooperation leads to the more valuable game and unilateral defection leads to the less valuable game.Transitions after mutual defection have negligible effects, whereas transitions after mutual cooperation or unilateral defection can substantially change the critical ratio.
  • Weak selection favors cooperation when b/c exceeds the effective neighbor threshold k − k′.Here, k′ summarizes the effects of game-transition patterns and differences in game benefits.

4. Pure versus stochastic strategies

The model also examines stochastic strategies, in which players cooperate with specified probabilities rather than choosing cooperation or defection unconditionally. Under the studied transition patterns, the cooperation-favoring conditions retain the same basic forms as for pure strategies.

  • Stochastic strategy s_p cooperates with probability p and defects otherwise, with s_1 and s_0 representing pure cooperation and defection.
  • The condition favoring s_p over s_q retains the format of the pure-strategy rules, with ξ_i modified for the updating rule.
  • When mutual cooperation leads to a more valuable game and other action profiles lead to a less valuable game, transitions lower the threshold for sufficiently cooperative stochastic strategies under death-birth updating.

5. Discussion

The discussion presents game transitions as a feedback mechanism coupling behavior and environmental states, yielding cooperation-favoring conditions even when individual games do not favor cooperation. It also connects these effects to partner-fidelity feedback and identifies dependence on initial game distributions for some transition patterns.

  • The paper’s central rule is b/c > k − k′, where k′ captures how game transitions affect cooperation across environmental states.
  • Mutual cooperation can generate a synergistic benefit analogous to partner-fidelity feedback, which promotes cooperation without requiring a partner’s conditional response.
  • Small differences between games can considerably reduce the cooperation threshold when mutual cooperation leads to a more profitable game and other actions lead to a less profitable one.
  • Game transitions can stabilize cooperation under mutation or random strategy exploration by causing environments exploited by defectors to deteriorate rapidly.
  • Game transitions can create environmental reciprocity, allowing cooperation to be favored even when cooperators are disfavored in each individual environment.
  • For some transition patterns, evolutionary dynamics and ξ_i may depend on the initial fractions of the various games.

Appendix B. Global versus local game transitions

The appendix distinguishes global transitions, which can affect games throughout the population, from local transitions restricted to games associated with nearby updating events. It shows that the cooperation rules persist with modified ξ values, while global transitions are generally more effective under some update rules.

  • Global transitions update games broadly, whereas local transitions impose updates only on games induced by the nearest neighbors of a deceased player.
  • Under birth-death updating, global and local transitions are identical because reproduction competition occurs at the population level.
  • Under death-birth updating, cooperation is favored when b_1/c > k − ξΔb/c, with ξ modified for local transitions.
  • Under pairwise-comparison updating, cooperation is favored when ξΔb/c > 1, again using a transition-dependent ξ.
  • Global transitions are more effective cooperation promoters than local transitions under the stated death-birth and pairwise-comparison thresholds.
  • Both transition types can promote cooperation, amplify beneficial game variations, and respond strongly to mutual or unilateral cooperation and defection.

SI.1.2.3. Change in pA.

This appendix derives the change in strategy and edge frequencies for local game-transition dynamics by tracking replacement events and game-induced edge switching. Under weak selection, the system can reduce to slower dynamics in p_A after faster conditional edge variables equilibrate.

  • The time derivative of p_A is obtained by combining strategy-replacement events after either an A-player or B-player is selected to die.
  • Changes in edge frequencies include switching on edges linking the focal player to neighbors and on edges linking neighbors to next-nearest neighbors.
  • When a neighboring A-player replaces a focal B-player, the associated edge can change through both strategy replacement and a transition from game j to game i.
  • Under weak selection, q_A|A reaches equilibrium much faster than p_A, producing a slow manifold with q̇_A|A = 0.
  • After reduction, the edge frequencies p_XY and conditional probabilities q_X|Y become functions of p_A.
  • If the reduced system is asymptotically stable, it has a single equilibrium and the initial fractions of games do not affect the evolutionary outcome; zero eigenvalues can instead permit initial-condition dependence.

SI.1.2.9. Fixation probability.

This section derives fixation-probability conditions for comparing strategies under several evolutionary updating rules. For donation games, the analysis yields conditions for one strategy to be favored over another.

  • The fixation probability of a fraction x of B-players is introduced as the starting quantity for the analysis.
  • The ratio of fixation probabilities is used to compare the evolutionary success of the two strategies.
  • For sufficiently small x, the fixation-probability ratio admits a simplifying approximation.
  • For sufficiently large populations with x = 1/N, A-players are favored over B-players when the derived condition holds.
  • The same analytical condition applies under death-birth, imitation, and pairwise-comparison updating, although the coefficients differ across rules.
  • For donation games, the framework specializes to explicit conditions for A-players to be favored over B-players under each updating rule.

SI.2. Approach to evaluate the sensitivity of evolutionary dynamics to the initial condition

This section represents edge states as a Markov chain and uses its communicating-class structure to determine whether evolutionary outcomes depend on initial game assignments. Multiple closed classes imply sensitivity, whereas one closed class implies independence.

  • The approach defines a Markov chain with state space E = {1, 2, . . . , 3n} from the game-transition matrices.
  • The entry in row i and column j of M is the transition probability from state i to state j.
  • One closed communicating class makes the evolutionary outcome independent of the initial condition, whereas multiple classes make it sensitive to the initial condition.
  • The sign of each entry in the accumulated matrix indicates whether a state can reach another within at most 3n transitions.
  • Zero entries in every column indicate more than one closed communicating class and hence dependence on initial game fractions.
  • When all entries except those in one column are positive, there is one closed communicating class and the outcome is insensitive to initial conditions.
  • Initial game assignments can constrain the long-term games played by an edge through the closed communicating class it enters.
  • The matrix-based approach predicts sensitivity by checking column positivity or the presence of zero eigenvalues.

SI.3.1. Global game transitions.

This section analyzes global game transitions, where every interaction can update its game each time step. The dynamics can be reduced using stationary game distributions and an effective game representation.

  • Global game transitions allow games in all interactions to update at every time step.
  • For sufficiently large populations, game frequencies reach stationary distributions faster than strategy frequencies.
  • Game transitions create an effective game whose payoff structure is determined by the stationary distributions of games associated with interaction types.
  • The resulting evolutionary conditions apply under death-birth, imitation, pairwise-comparison, and birth-death updating.
  • For donation games, substituting the donation-game payoffs reduces the general conditions to cooperation-selection rules involving the game-transition effects.
  • Stationary distributions u(s) are obtained by solving u(s) = u(s)P(s) for each action profile.
  • With local transitions, transitioning a fraction p of games yields the same results as global transitions because the stationary-distribution equation is unchanged.

SI.3.4. Stochastic strategies.

This section extends the analysis from pure strategies to stochastic strategies that cooperate with different probabilities. It derives selection conditions by combining expected payoffs with stationary distributions of interaction scenarios.

  • A stochastic strategy sp cooperates with probability p and defects with probability 1 − p, while sq is defined analogously.
  • The model recovers pure cooperators and defectors by setting p = 1 and q = 0.
  • Stationary distributions track the fractions of interactions combining game i with the two players’ realized actions.
  • The expected payoff in an sp–sq interaction averages the four action profiles across the games played.
  • Under death-birth updating, sp is favored over sq when (k + 1) fpp + (k −1) fpq > (k −1) fqp + (k + 1) fqq.
  • A stochastic strategy is more cooperative than another when it assigns a larger probability to cooperation.
  • The selection conditions for stochastic strategies under imitation, birth-death, and pairwise-comparison updating follow the corresponding pure-strategy donation-game conditions.

SI.3.5. Intuition based on “sigma rule”.

Game transitions can be interpreted through an effective payoff game: transitions and payoff differences add a cooperation-specific payoff, altering the usual sigma-rule condition. This framework also identifies when stochastic or strategy-independent transitions cannot promote cooperation.

  • Effective game: Game transitions make evolution proceed as if interactions used an effective game with an altered mutual-cooperation payoff.The added payoff depends on transition patterns and differences among games.
  • Effective game: The extra mutual-cooperation payoff is determined jointly by transition-pattern coefficients ξ_i and game variations Δb_1i.
  • General conditions: For general transition models, deriving which strategy is favored requires solving a set of equations, with explicit expressions provided for representative interaction scenarios.
  • Stochastic transitions: A fully stochastic transition makes any game equally likely in the next time step, while diverse stochastic games can be approximated by a static unified game.
  • Strategy-independent transitions: If transitions are independent of strategic actions, game transitions cannot promote cooperation under pairwise-comparison or birth-death updating.This corresponds to ξ_i = 0, regardless of the donation-game benefit.

SI.4.3. Evolutionary dynamics with game transitions between two states (n = 2).

The two-state analysis derives how transition patterns affect cooperation when mutual cooperation leads to one game and other action profiles lead to another. The outcome can depend on transition structure, initial game frequencies, and the updating rule.

  • Two-state transition dynamics: The analysis derives general cooperation conditions for two-state game transitions and examines patterns whose outcomes may or may not depend on initial conditions.
  • Local transitions: Under selected local transition patterns, the evolutionary outcome is characterized through transition matrices and reduced subsystems.
  • Transition patterns: For the specified two-state transition pattern, local and global transitions yield distinct ξ values, with corresponding conditions under death-birth and pairwise-comparison updating.
  • Stochastic strategies: Game transitions can favor cooperative stochastic strategies over less cooperative stochastic strategies under birth-death or pairwise-comparison updating, unlike fixed-game settings.
  • Initial conditions: The evolutionary outcome can depend on the initial fractions of different games when ξ is determined by those initial frequencies.

SI.4.4. Evolutionary dynamics with probabilistic game transitions among three states (n = 3).

The three-state analysis examines probabilistic transitions among games with different values and studies how transition probability changes cooperation thresholds. The effect of probabilistic transitions depends on game variations and can be non-monotonic.

  • Three-state transitions: The three-state model orders games by value as b1 > b2 > b3 and allows probabilistic transitions among them.Mutual cooperation tends toward the most valuable game, mutual defection toward the least valuable, and unilateral actions toward an intermediate game.
  • Transition probability: The critical benefit-to-cost ratio can decrease monotonically or vary non-monotonically with transition probability, depending on the ratio of game-benefit differences.
  • Transition probability: As transition probability increases from slightly above zero, the cooperation threshold first decreases and then increases in the non-monotonic case.
  • Implications: Probabilistic transitions can strengthen or weaken their cooperation-promoting effects depending on variations among games, with the conclusion extending to imitation and pairwise-comparison updating.
  • Global transitions: For global transitions, ξ2 = (k −1)/2 and ξ3 = 0, so probabilistic transitions do not alter the effects of game transitions on cooperation.
Loading 1905.10269v2…