Source-linked AI summary
Human-AI Coevolution
Dino Pedreschi, Luca Pappalardo, Emanuele Ferragina, Ricardo Baeza-Yates, Albert-Laszlo Barabasi, Frank Dignum, Virginia Dignum, Tina Eliassi-Rad, Fosca Giannotti, Janos Kertesz, Alistair Knott, Yannis Ioannidis, Paul Lukowicz, Andrea Passarella, Alex Sandy Pentland, John Shawe-Taylor, Alessandro Vespignani
TL;DR
Recommenders shape everyday choices through a feedback loop in which users’ behavior also supplies data for later training, yet the loop and its social effects remain understudied. The paper introduces Coevolution AI to examine these dynamics, concluding that existing evidence lacks iteration-by-iteration data on recommendations, user reactions, and retraining.
Problem
The feedback loop between users and recommenders is understudied, and existing studies often lack data capturing the full interaction over time.
Source: §1 · p. 2, Modelling the feedback loop · p. 6Method
The paper proposes Coevolution AI as a field for theoretical, empirical, and mathematical study of human-AI feedback loops.
Source: §1 · p. 3Results
The paper identifies a need for longitudinal data that capture recommendations, users’ reactions, and retraining at each feedback-loop iteration.
Source: Modelling the feedback loop · p. 6Takeaways & Limitations
Studying coevolution requires new methods and a bidirectional account of how humans and recommenders influence each other.
Source: §1 · p. 4, OPEN CHALLENGES OF COEVOLUTION AI · p. 10Existing studies typically describe users’ choices without prior recommendations or platform retraining, leaving the feedback loop incompletely observed.
Source: Modelling the feedback loop · p. 6
Abstract
from arXiv · showhide
Human-AI coevolution, defined as a process in which humans and AI algorithms continuously influence each other, increasingly characterises our society, but is understudied in artificial intelligence and complexity science literature. Recommender systems and assistants play a prominent role in human-AI coevolution, as they permeate many facets of daily life and influence human choices on online platforms. The interaction between users and AI results in a potentially endless feedback loop, wherein users' choices generate data to train AI models, which, in turn, shape subsequent user preferences. This human-AI feedback loop has peculiar characteristics compared to traditional human-machine interaction and gives rise to complex and often ``unintended'' social outcomes. This paper introduces Coevolution AI as the cornerstone for a new field of study at the intersection between AI and complexity science focused on the theoretical, empirical, and mathematical investigation of the human-AI feedback loop. In doing so, we: (i) outline the pros and cons of existing methodologies and highlight shortcomings and potential ways for capturing feedback loop mechanisms; (ii) propose a reflection at the intersection between complexity science, AI and society; (iii) provide real-world examples for different human-AI ecosystems; and (iv) illustrate challenges to the creation of such a field of study, conceptualising them at increasing levels of abstraction, i.e., technical, epistemological, legal and socio-political.
1 Introduction
Recommenders create a continuing feedback loop between users’ choices and AI training, with effects that can extend beyond individuals to society. The paper introduces Coevolution AI to study this loop across AI, complexity science, and society.
1 Introduction
- Recommenders are a useful focus because they are ubiquitous in online platforms and can shape users’ decisions and interactions instantly and profoundly.The paper identifies pervasiveness, persuasiveness, traceability, speed, and complexity as aspects magnified by AI-based recommenders.Source: §1 · p. 2
- Users’ choices train recommenders, whose suggestions influence subsequent choices and generate data for retraining, creating a potentially endless feedback loop.The loop may have unintended effects, including echo chambers, inequality, and congestion.Source: §1 · p. 2, Figure 1 · p. 3
- AI and complexity science, working in isolation, cannot address the challenges of understanding human-AI coevolution.AI research often treats machines as solitary agents, while little is known about whether feedback loops amplify or mitigate undesired collective outcomes or which parameters govern their network dynamics.Source: §1 · p. 3
- Coevolution AI proposes a field at the intersection of AI and complexity science for theoretical, empirical, and mathematical investigation of the human-AI feedback loop.Its society-centred perspective adds societal well-being, new methodological and epistemological approaches, and legal and political interventions to existing AI debates.Source: §1 · pp. 3–4
- Society-centred AI considers feedback-loop effects on well-being at both individual and societal levels and calls for new methods and legal and political interventions.The paper situates this concern in a political economy that privileges individual utility over collective goods and concentrates recommendation power.Source: §1 · p. 4
- The paper previews reviews of research methods, examples from four ecosystems, discussion of social impacts, and open challenges for Coevolution AI.The ecosystems named are social media, online retail, urban mapping, and generative AI.Source: §1 · p. 4
2 Methods for human-AI coevolution
The section reviews empirical and simulation methods, controlled and observational studies, and the small body of research that explicitly models feedback loops. It identifies trade-offs in existing approaches and argues that longitudinal data capturing recommendations, user reactions, and retraining are needed to study bidirectional causation.
2 Methods for human-AI coevolution
- Only a few studies move beyond static recommender effects to investigate human-AI feedback-loop mechanisms theoretically or empirically.The section reviews methodological strengths and shortcomings and highlights ways to capture those mechanisms.Source: §2 · p. 4, Modelling the feedback loop · p. 6
Empirical vs Simulation studies
- Empirical studies use real-platform behavior data, while simulations generate data from mechanistic, AI, or digital-twin models.Empirical work can support generalisation but is time- and condition-specific and often hard to reproduce; simulations are reproducible and allow parameter manipulation but rely on assumptions that may miss real-world or unintended dynamics.Source: Empirical vs Simulation studies · p. 5
Controlled vs observational studies
- Controlled studies compare groups exposed to different recommendations, helping isolate effects and establish causal relationships, but can limit generalisability and flexibility.They are also difficult to design because they require direct access to platform users and recommenders.Source: Controlled vs observational studies · p. 5
- Controlled examples report that Twitter personalization amplified political messages, with mainstream right-wing parties benefiting more, and navigation services may increase travel time and CO2 emissions above an adoption threshold.These examples illustrate findings from controlled studies, not general outcomes for all platforms or settings.Source: Controlled vs observational studies · p. 5
- Observational studies can support broader generalisations when data are sufficiently large and representative, but causal attribution is challenging and findings may be affected by bias, measurement error, or confounding.Examples include YouTube polarization, linked mainly to users’ preferences, and music-streaming recommendations associated with more purchases and greater similarity in purchase variety.Source: Controlled vs observational studies · pp. 5–6
Modelling the feedback loop
- Existing feedback-loop studies provide insights, but their data typically omit prior recommendations and platform retraining, limiting the picture of the interaction.Longitudinal data at each iteration should capture recommendations, user reactions, and retraining to study causation bidirectionally; existing data have mainly been used to validate theoretical approaches.Source: Modelling the feedback loop · p. 6
3 Outcomes of human-AI coevolution
Human-AI coevolution produces outcomes at individual, item, model, and systemic levels, often unintentionally. Across four ecosystems, feedback loops can create trade-offs between individual benefits and collective effects.
3 Outcomes of human-AI coevolution
- Outcomes span individual, item, model, and systemic levels, with systemic examples including polarization, echo chambers, inequality, concentration, and segregation.Other outcomes include individual-level filter bubbles, model-level collapse, and changes in volume and diversity across levels.Source: §3 · pp. 6–7
Social media
- Social-media feedback loops can connect user interactions with posts and followers to recommendations, with outcomes including filter bubbles, homophily, polarization, fragmentation, and echo chambers.Recommendations can help users find content and like-minded people while potentially confining them to a filter bubble; personalized Twitter recommendations overexposed users to some political content.Source: §3 · p. 7
Online retail
- Collaborative filtering may increase sales volume and individual consumption variety while decreasing aggregate consumption diversity and amplifying popular products.Recommenders can reduce choice effort and overload, but may also increase concentration and reduce product variety and competition.Source: §3 · p. 7
Urban mapping
- Shared route recommendations may contribute to congestion, longer travel times, and higher CO2 emissions, while drivers’ route choices feed back into later recommendations.If too many drivers choose the same “eco-friendly” route, it will cease to be eco-friendly.Source: §3 · p. 8
4 Social impact of human-AI coevolution
Coevolution AI connects AI and complexity science with society, where recommender feedback may interact with existing social trends. The discussion highlights trade-offs around collective utility, recommender ownership, and inequality across ecosystems.
4 Social impact of human-AI coevolution
- Coevolution AI treats recommender effects on social phenomena as central because they may amplify trends already underway, motivating input from social sciences.The discussion focuses on individual utility versus common goods, ownership of “the means of recommendation,” and inequality and concentration.Source: §4 · p. 8
Individual utility vs common goods
- Recommenders are mostly designed to maximize individual utility and company profits, leaving common goods underconsidered.The paper links this approach to methodological individualism, which may overlook collective utility.Source: §4 · p. 8
Ownership of the means of recommendation
- The paper asks who owns the “means of recommendations” and how that ownership shapes choices and collective outcomes, while noting political economy is outside this article’s scope.The authors argue that a different socioeconomic context geared toward shared collective objectives can foster positive societal outcomes through coevolution.Source: §4 · p. 9
Inequality and concentration
- The article notes that AI’s uneven impacts are stronger in low- and middle-income countries and hypothesizes that coevolution effects may be underestimated without feedback-loop research [172].It cautions against techno-solutionism and calls for more studies to steer positive outcomes.Source: §4 · p. 9
- Coevolution may foster inequality and concentration across ecosystems, including exposure inequalities, popular-product concentration, and diversity loss in generated content [126].Ride-hailing recommendations may also reinforce existing racial and socioeconomic inequalities in poor and Black neighbourhoods.Source: §4 · p. 9
5 Open Challenges of Coevolution AI
Coevolution AI faces challenges spanning technical, epistemological, legal, and socio-political levels. Addressing them requires understanding feedback-loop dynamics and their implications for research and policy.
OPEN CHALLENGES OF COEVOLUTION AI
- Step-wise measurement should track how feedback-loop iterations and recommender retraining change human behavior and recommender outcomes, alongside mathematical modeling of those mechanisms.Examples include tracking changes in product variety across successive times and how long generative AI takes to lose linguistic diversity.Source: OPEN CHALLENGES OF COEVOLUTION AI · p. 10
- Controlled studies and bidirectional analysis are needed to examine how humans and recommenders continuously influence each other.Existing literature often overlooks coevolution, and a unidirectional view of causality misses this reciprocal influence.Source: OPEN CHALLENGES OF COEVOLUTION AI · p. 10
- Limited external access to platform data and opaque recommender design impair reproducibility and replicability, while specialized APIs could support controlled experiments.The paper also notes uncertainty about how vetted researchers will access privately owned platforms and calls for impact evaluation of feedback-loop mechanisms.Source: OPEN CHALLENGES OF COEVOLUTION AI · pp. 10–11
- Oligopolistic control of the means of recommendation may hinder socially desirable development, while small changes in recommenders or human behavior may substantially affect social outcomes.Redistribution across smaller market players and society could support more transparent data-access rules and a fairer distribution of recommendation capacity.Source: OPEN CHALLENGES OF COEVOLUTION AI · p. 11
- Challenges persist even under potential platform transformations, and accurately understanding recommender influence is presented as necessary to inform policy and address potential negative externalities.The paper’s stated aim is society-centric AI that contributes to addressing societal problems.Source: OPEN CHALLENGES OF COEVOLUTION AI · p. 11