Source-linked AI summary
The Fake News Spreading Plague: Was it Preventable?
Eni Mustafaraj, Panagiotis Takis Metaxas
TL;DR
Researchers and platforms need to understand how socio-technical systems enable misinformation and how humans and algorithms should divide responsibility for addressing it. The paper reconstructs the 2010 Twitter-bomb, summarizes its tactics in an easily memorable spreading recipe, and documents how repeated messages manipulated real-time search results. The 2010 Twitter-bomb reached an estimated 61,732 Twitter users through recipient retweets.
Problem
Researchers and platforms need to understand how socio-technical systems enable misinformation and how humans and algorithms should divide responsibility for addressing it.
Method
The paper reconstructs the 2010 Twitter-bomb, summarizes its tactics in an easily memorable spreading recipe, and documents how repeated messages manipulated real-time search results.
Results
The 2010 Twitter-bomb reached an estimated 61,732 Twitter users through recipient retweets.
Takeaways & Limitations
Misinformation research can inform platform design, but meaningful evaluation of interventions remains limited when independent researchers cannot access platform data.
Takeaways & Limitations
The effectiveness of Facebook’s newer disputed-content labeling remained unknown because independent researchers lacked access to the necessary Facebook data.
Abstract
from arXiv · showhide
In 2010, a paper entitled "From Obscurity to Prominence in Minutes: Political Speech and Real-time search" won the Best Paper Prize of the Web Science 2010 Conference. Among its findings were the discovery and documentation of what was termed a "Twitter-bomb", an organized effort to spread misinformation about the democratic candidate Martha Coakley through anonymous Twitter accounts. In this paper, after summarizing the details of that event, we outline the recipe of how social networks are used to spread misinformation. One of the most important steps in such a recipe is the "infiltration" of a community of users who are already engaged in conversations about a topic, to use them as organic spreaders of misinformation in their extended subnetworks. Then, we take this misinformation spreading recipe and indicate how it was successfully used to spread fake news during the 2016 U.S. Presidential Election. The main differences between the scenarios are the use of Facebook instead of Twitter, and the respective motivations (in 2010: political influence; in 2016: financial benefit through online advertising). After situating these events in the broader context of exploiting the Web, we seize this opportunity to address limitations of the reach of research findings and to start a conversation about how communities of researchers can increase their impact on real-world societal issues.
1. INTRODUCTION
The paper traces how organized actors exploit social platforms to spread misinformation, from a 2010 Twitter-bomb targeting Martha Coakley to fake news on Facebook during the 2016 election. It distills a reusable spreading recipe and highlights the difficulty of evaluating platform interventions when researchers lack access to data.
- 1.1 The Anatomy of a political Twitter-Bomb: 929 tweets from nine newly created accounts targeted 573 users in 138 minutes with a false claim about Martha Coakley.The accounts repeatedly linked to a website presenting her speech out of context and claiming she opposed employing Catholics in emergency rooms.
- 1.2 A recipe for spreading misinformation on Twitter: The paper turns the documented Twitter-bomb into a memorable recipe for spreading misinformation, newly summarized here beyond the original event report.The recipe includes infiltrating communities already engaged with a topic and using them as organic spreaders through their extended subnetworks.
- 1.3 Spreading Fake News on Facebook: Fake news presents falsehoods in a news-like format, using provocative emotional content and click bait to attract social-media users and generate advertising revenue.The paper notes that information technology enables such material to be produced and consumed at massive scale.
2. FROM PROPAGANDA TO FAKE NEWS
Online propaganda predates social media, but search engines and social platforms have enabled increasingly sophisticated ways to make biased content find users. The paper calls for researchers to document these techniques, inform platforms, and help journalists and the public evaluate them.
- Propaganda is older than social media, but search engines made it easier for propagandists to reach audiences through Web Spam.Advertisers, political activists, and religious zealots modified Web structure to promote biased results over organic ones.
- Spammers adapted to search-engine defenses by creating networks of mutually linking sites to increase their PageRank.These tactics included Link Farms and Mutual Admiration Societies, even as search engines expanded quality signals.
- Recent propaganda methods include fake news, featured-snippet manipulation, and autocomplete revelations across search and social platforms.These methods represent newer ways of ensuring propagandistic content reaches users.
- Researchers should document and understand these phenomena while making their findings known to platform providers, journalists, and non-specialists.The paper notes that credence given to conspiracy theories can contribute to public confusion.
3. RESEARCH THAT INFORMS DESIGN
Research findings influenced platform designs addressing manipulation in real-time search, retweeting, and Facebook news distribution, while limited Facebook data constrained independent evaluation.
- 3.1 The Evolution of Real-Time Search Results: Google eventually changed real-time search to show tweets from a person’s timeline rather than tweets about them, preventing adversarial messages from gaining unearned prominence.Earlier experiments showed that non-suspected users’ retweets could preserve spam in search results.
- 3.1 The Evolution of Real-Time Search Results: Experiments found that suspending spam accounts was insufficient because their retweets could remain visible through non-suspected users.The authors proposed retroactively deleting spam retweets and labeling users who enabled spam.
- 3.2 The Evolution of Retweeting: Twitter’s 2010 retweet design allowed users to edit original text, enabling spam to persist in retweets and allowing users to change the meaning of messages.Later designs delete retweets when originals are deleted, while quoted retweets display that the original tweet is unavailable.
- 3.3 The Evolution of Fake News on Facebook: Facebook replaced human Trending News editors with machine-learning algorithms in August 2016, after which fake news began reaching Trending News status.The change followed complaints about alleged anti-conservative bias and concerns about suppressing speech.
- 3.3 The Evolution of Fake News on Facebook: Facebook later introduced disputed-story labels and sharing warnings, but independent researchers still lacked access to data needed to evaluate their effect on fake-news spreading.Users remain allowed to share stories after receiving the warning.
4. DISCUSSION
The authors argue that platform interventions and private groups can prevent researchers from observing misinformation campaigns at their origins. They call on information-retrieval research communities to clarify which tasks humans and algorithms should perform in web-based socio-technical systems.
- 4. DISCUSSION: Deleting misinformation from suspended accounts prevents researchers and fact-checkers from reconstructing campaign origins and spreading mechanisms.Private Facebook groups create a further obstacle because outside researchers may miss fake-news cascades during their early stages.
- 4. DISCUSSION: The authors argue that researchers lacked a leading role in discovering the current fake-news plague, while journalists raised concerns that were not heard.They characterize this situation as worrisome and connect it to Facebook’s algorithmic replacement of humans and Google’s advertising incentives.
- 4. DISCUSSION: The paper poses an open question about which tasks humans and algorithms should perform in systems such as Facebook, Google, and Twitter.It urges research communities to lead efforts to answer this question.