PaperlayerView exact revision on arXiv

Source-linked AI summary

Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms

Petter Törnberg, Diliara Valeeva, Justus Uitermark, Christopher Bail

arXiv:2310.05984v1cs.SIcs.AIcs.MA

TL;DR

The paper asks whether news-feed algorithms can increase interaction across political divides without increasing toxic or uncivil behavior. It simulates social-media users with ANES-based personas across different feed designs, then finds the bridging design combines cross-party interaction with the lowest toxicity. The authors call for further validation before treating these findings as established.

  • Problem

    The paper asks whether platforms can increase interaction between people with opposing views without increasing toxic or uncivil behavior.

    Source: §1 · p. 3

  • Method

    It simulates ANES-persona agents across feed designs, including a bridging algorithm that promotes posts liked by people with different views.

    Source: §1 · p. 3, §2 · p. 4, §3 · p. 5

  • Results

    The bridging platform had more interpartisan than intrapartisan comments and the lowest toxicity; the all-user high-engagement timeline showed higher interaction and toxicity.

    Source: §4 · p. 6

  • Takeaways & Limitations

    The model produced plausible group behavior and replicated emergent patterns observed in prior empirical research, supporting further simulation research.

    Source: §5 · p. 8

    Substantial additional validation and analysis are needed to confirm the results and identify the model’s capacity limits.

    Source: §5 · p. 7

Abstract

from arXiv · show

Social media is often criticized for amplifying toxic discourse and discouraging constructive conversations. But designing social media platforms to promote better conversations is inherently challenging. This paper asks whether simulating social media through a combination of Large Language Models (LLM) and Agent-Based Modeling can help researchers study how different news feed algorithms shape the quality of online conversations. We create realistic personas using data from the American National Election Study to populate simulated social media platforms. Next, we prompt the agents to read and share news articles - and like or comment upon each other's messages - within three platforms that use different news feed algorithms. In the first platform, users see the most liked and commented posts from users whom they follow. In the second, they see posts from all users - even those outside their own network. The third platform employs a novel "bridging" algorithm that highlights posts that are liked by people with opposing political views. We find this bridging algorithm promotes more constructive, non-toxic, conversation across political divides than the other two models. Though further research is needed to evaluate these findings, we argue that LLMs hold considerable potential to improve simulation research on social media and many other complex social settings.

1. Can Bridging Algorithms Improve Conversations on Social Media?

The section frames cross-partisan interaction as democratically valuable but potentially toxic, motivating a feed design intended to bridge viewpoints without worsening toxicity.

  • Interaction with people who hold different views is presented as important to constructive public conversation and democratic theory.The section notes that evidence about whether following controls limit exposure to alternative views is mixed.Source: §1 · p. 3
  • Exposure to opposing views may intensify attacks, especially when feeds reward engagement even when it is conflictual.The section defines toxic discourse as rude, disrespectful, or unreasonable conversation.Source: §1 · p. 3
  • Bridging algorithms are proposed to amplify posts receiving positive reactions across political views, potentially limiting extremist hijacking and encouraging overshadowed moderates to speak.The theory is that cross-spectrum approval can signal that moderate users’ messages are valued.Source: §1 · p. 3
  • The section asks whether a bridging algorithm can increase interaction across political divides without increasing toxicity, addressing a perceived trade-off between viewpoint range and civil discourse.Source: §1 · p. 3

2. Simulating Social Media Platforms Using Large Language Models

The paper combines agent-based modeling with LLMs to simulate social media discourse, using survey-based personas to represent users and their interests.

  • ABMs model how individual behaviors and interactions can generate group-level patterns, but their simplified agents have lacked realistic reasoning and social interaction.The paper presents them as tools for identifying mechanisms, not predicting precise outcomes or fully capturing social reality.Source: §2 · p. 3
  • LLMs’ capacity for realistic conversation offers a complement to ABMs for simulating public-discourse dynamics.Source: §2 · p. 3
  • Figure 1 depicts the combined LLM–agent-based model and ANES-based personas used to simulate the impact of bridging algorithms on discourse.Source: Figure 1 · p. 4
  • The simulations use U.S. Twitter users on a single day in July 2020, with personas based on 2020 ANES respondents and their social-media use, demographics, beliefs, interests, and news sources.Personas dynamically include relevant attributes, and LLM prompting adds non-political interests and personality traits.Source: §2 · p. 4
  • The 500 agents share the same personas, and the sample skews younger and more Democratic than Republican; most agents lack party affiliation and do not discuss politics on social media.Source: §2 · pp. 4–5

3. Simulating Three Social Media Platforms

The study compares three synthetic platforms that differ in which posts their news feeds prioritize, including a bridging design based on likes from opposing partisans.

  • The first platform shows high-engagement posts from followed users, with 500 users each following 30 people and greater likelihood of following politically similar users.It is designed to represent echo-chamber dynamics and is expected to limit both cross-party interaction and toxicity.Source: §3 · p. 5
  • The second platform exposes users to high-engagement posts from anyone, including people they do not follow, modeling a break from an echo chamber.The authors predict greater interaction but heightened toxicity under this design.Source: §3 · p. 5
  • The bridging feed shows posts from all agents and prioritizes those liked by users from the opposing political party.For example, a Republican author’s post gains visibility through Democratic likes.Source: §3 · p. 5
  • The authors predict that bridging will increase cross-party engagement without increasing toxicity relative to the other two platforms.Source: §3 · p. 5

A. The Simulation Process.

The simulation has agents share news and then respond to posts surfaced by their platform’s timeline algorithm, with interactions feeding back into later visibility.

  • Agents are selected to post according to ANES-based social-media use, then choose among 15 July 1, 2020 news stories matched to their persona’s sources and interests.The date followed GPT-3.5’s training cutoff to provide context while avoiding knowledge of future events.Source: A. The Simulation Process · p. 5
  • Agents view other users’ news posts through their assigned timeline algorithm, then like or comment based on persona interests and preferences.These actions feed back into the algorithm and shape what later agents see.Source: A. The Simulation Process · pp. 5–6
  • The run approximates one day of discourse, during which 30% of agents like or comment on posts.Source: A. The Simulation Process · p. 6
  • API rate limits and computing resources forced a brief simulation period and small sample, leaving larger and longer runs for future study.Source: A. The Simulation Process · p. 6

B. Measuring the Outcomes.

The study estimates discourse toxicity with Perspective API and measures cross-party interaction from agents’ observed likes and comments using an E-I Index.

B. Measuring the Outcomes.

  • Perspective API estimates the toxicity of discourse in each simulated platform.The authors plan to validate simulation realism and text quality with human respondents in a future iteration.Source: B. Measuring the Outcomes · p. 6
  • The E-I Index measures interpartisan interaction by comparing agents’ interactions with people who do and do not share their political views.It is calculated as inter-partisan interactions minus intra-partisan interactions, divided by their sum.Source: B. Measuring the Outcomes · p. 6
  • The E-I Index ranges from -1 for exclusively intra-partisan interactions to 1 for exclusively inter-partisan interactions.Source: B. Measuring the Outcomes · p. 6

4. Results

The bridging algorithm produced substantial cross-party interaction and the lowest toxicity across the three platforms. By contrast, broad high-engagement exposure increased both interaction and toxicity, while the follow-based feed showed little opposing-party interaction.

4. Results

  • Platform 1 showed low toxicity and nearly no interaction with messages from the opposing party.Source: §4 · p. 6, Table 2 · p. 6, Figure 2 · p. 7
  • Platform 2’s timeline was nearly 50% more toxic and reached about the average toxicity of U.S. Twitter messages, while comments and likes remained own-party biased.Its E-I indices were -0.70 for comments and -0.78 for likes; the authors suggest responses to upsetting opposing-side posts boosted their visibility, reinforcing conflict and toxicity.Source: §4 · p. 6, Figure 3 · p. 7
  • The bridging platform had more interpartisan than intrapartisan comments, nearly balanced likes, and the lowest toxicity of all three platforms.Its positive E-I index indicates more cross-party than within-party comments; the timeline also emphasized issues crossing political divides.Source: §4 · p. 6, Table 2 · p. 6, Figure 4 · p. 7

5. Discussion and Conclusion

The discussion presents cross-divide bridging as a promising but preliminary design direction, while emphasizing validation, model limitations, and the trade-off between richer simulations and interpretability.

5. Discussion and Conclusion

  • Survey respondents expressed a preference for content that is productive, useful, and positive.The authors cite this preference as context for considering alternatives to existing platform designs.Source: §5 · p. 7
  • Amplifying posts with positive reactions across political divides may help platforms optimize for consensus and make forums less attractive to trolls.The authors present bridging as one of many possible tools for improving social media.Source: §5 · p. 7
  • Results were validated only with automated content analyses, and technical constraints limited agent numbers and simulation duration.Human evaluation may better capture social-media interaction and LLM impersonation; longer simulations could reveal whether bridging effects strengthen or deteriorate.Source: §5 · p. 7
  • Calibrating agent behavior to real populations remains difficult because LLMs may underrepresent groups or reflect developers’ norms and politeness biases.The authors report that toxicity in this study corresponded closely with Twitter levels for the same period, while noting that rigorous calibration remains an open question.Source: §5 · pp. 7–8
  • Unknown proprietary training and fine-tuning processes limit transparency, and probabilistic LLM behavior may yield substantially different replications even with the same prompts.The authors identify repeated runs as a partial mitigation and an open-source research model as a more fundamental solution.Source: §5 · p. 8
  • The one-day design omits changing perceptions and relationships, while retrospective LLM training limits realistic reactions to current events and user adaptation to algorithms is unmodeled.The authors suggest longer simulations and prompt-based memory as possible avenues, though some behaviors may exceed current LLM capacities.Source: §5 · p. 8
  • The relatively simple model produced plausible group behavior and replicated empirical emergent patterns, but added parameters can make results harder to interpret.Potential realism improvements include more survey data, larger models, richer networks, and more complex feeds; the authors say these should be weighed against parsimony.Source: §5 · p. 8
  • The authors see LLM-powered agent models as a middle ground between traditional ABMs and human-subject experiments, with potential uses in studying discourse and collective behavior.They also propose combining these models with surveys or field experiments and exploring topics such as misinformation, extremism, and non-U.S. contexts.Source: §5 · pp. 8–9

A. Generating personas.

Personas combine recoded American National Election Study responses with LLM-generated traits, then use survey responses to shape agents’ political identities and platform behavior.

A. Generating personas.

  • ANES responses supply persona attributes including demographics, voting history, party affiliation, and views on political violence, recoded into descriptive text.Questions without a respondent answer are omitted from the profile.Source: A. Generating personas · p. 11
  • Temperature responses add descriptions of strong positive or negative feelings toward political figures or groups, while political-discussion frequency adds arguing or nonpolitical-interest traits.Figures rated neither warmly nor coldly are excluded as not salient.Source: A. Generating personas · p. 11
  • Survey-reported news sources determine which stories agents can see and share, while social-media-use frequency affects how often they post or like messages..Source: A. Generating personas · p. 11
  • A Democratic–Republican feeling-thermometer difference yields a partisan score from -1 to 1, distinguishing aligned independents from people who prefer neither party.The score is also translated into a short partisan-leaning statement in the persona.Source: A. Generating personas · p. 11
  • An LLM adds plausible personality traits and interests to the survey-based profile, producing a multisentence persona spanning political, personality, and lifestyle attributes.The authors use these attributes as factors shaping online behavior and discussion.Source: A. Generating personas · p. 11
  • Agents’ network is a homophilous random graph in which connection probability depends on political position, with H denoting the homophily parameter.The paper describes this probability as proportional to the distance-adjusted relationship between agents’ political positions.Source: A. Generating personas · p. 11
  • Comment lengths are sampled from a log-normal fit to 300,000 U.S. English-language Twitter messages posted from 2017 to 2021.The fitted distribution has shape 0.8066, location 0.2178, and scale 10.69.Source: A. Generating personas · p. 11
  • Prompts ask agents to choose a news headline and comment, respond emotionally to a message thread, and like only messages they endorse and feel positive about.The reaction prompt permits disagreement and emotionally charged responses, including sarcasm, swearing, and personal attacks.Source: A. Generating personas · p. 11
Loading 2310.05984v1…