Source-linked AI summary
Anatomy of an online misinformation network
Chengcheng Shao, Pik-Mai Hui, Lei Wang, Xinwen Jiang, Alessandro Flammini, Filippo Menczer, Giovanni Luca Ciampaglia
TL;DR
The paper examines how misinformation and fact-checking spread and compete on Twitter, and investigates the structure and purveyors of the misinformation diffusion network. Using Hoaxy and k-core decomposition of retweet data, it finds that fact-checking nearly disappears toward the core while social bots proliferate, with a dense, stable core emerging after the election.
Problem
The paper asks how misinformation and fact-checking compete online, what characterizes the misinformation network’s core and purveyors, and how misinformation might be reduced.
Method
The authors build Hoaxy to study Twitter diffusion, analyze retweet networks linking claims and fact-checking articles, and apply k-core decomposition to identify the network core.
Results
Fact-checking nearly disappears toward the network core while social bots become more prevalent; the main core stabilizes in size and membership while its density increases.
Takeaways & Limitations
The findings provide a first look at the structure and dynamics of a massive online misinformation diffusion network and identify central accounts relevant to network disruption.
Takeaways & Limitations
Hoaxy assesses claims using source lists compiled by trusted third-party organizations rather than manually evaluating every claim.
Abstract
from arXiv · showhide
Massive amounts of fake news and conspiratorial content have spread over social media before and after the 2016 US Presidential Elections despite intense fact-checking efforts. How do the spread of misinformation and fact-checking compete? What are the structural and dynamic characteristics of the core of the misinformation diffusion network, and who are its main purveyors? How to reduce the overall amount of misinformation? To explore these questions we built Hoaxy, an open platform that enables large-scale, systematic studies of how misinformation and fact-checking spread and compete on Twitter. Hoaxy filters public tweets that include links to unverified claims or fact-checking articles. We perform k-core decomposition on a diffusion network obtained from two million retweets produced by several hundred thousand accounts over the six months before the election. As we move from the periphery to the core of the network, fact-checking nearly disappears, while social bots proliferate. The number of users in the main core reaches equilibrium around the time of the election, with limited churn and increasingly dense connections. We conclude by quantifying how effectively the network can be disrupted by penalizing the most central nodes. These findings provide a first look at the anatomy of a massive online misinformation diffusion network.
1 Introduction
The paper introduces Hoaxy to study how misinformation and fact-checking spread and compete on Twitter, then uses it to characterize the 2016 election-period diffusion network and possible countermeasures.
- Contributions: Hoaxy is an open platform for large-scale, systematic studies of misinformation and fact-checking diffusion on Twitter.It collects tweets sharing claims and fact-checking articles and makes data available through an open API.
- Research questions: The study addresses competition between misinformation and fact-checking, the structure and purveyors of the network core, and ways to reduce misinformation.These are stated as research questions RQ1–RQ3.
- RQ1: Claims vs. fact-checking: Only 5.8% of dataset tweets shared fact-checking links, a 1:17 ratio with misinformation tweets.Fact-checking nearly disappears toward the inner core, and some core links were shared mockingly or misleadingly rather than for accurate information.
- RQ2: Network core: The main core initially grows, then stabilizes in size and membership while becoming denser, with more social bots present than elsewhere.Centrality measures identify different subsets, but the most central nodes show similar partisan slant across measures.
- RQ3: Countermeasures: A simple greedy strategy that penalizes the main purveyors of misinformation would significantly reduce the overall amount of misinformation.The paper evaluates this as a possible platform countermeasure.
2.1 Network core analysis
The paper uses k-core decomposition to identify progressively more densely embedded portions of the diffusion network and its main core.
- Definitions: A k-core is the maximal subgraph whose nodes all have degree at least k.The main core is the non-empty graph with the maximum k value.
- Procedure: k-core decomposition recursively removes degree-k nodes to extract the (k + 1)-core, with removed nodes forming the k-shell.The sequence of increasing k-cores reveals network structure and can identify influential users.
2.2 Bot detection
The study uses Botometer, a supervised classifier, to evaluate whether Twitter accounts exhibit characteristics of social bots.
- Botometer: Botometer classifies accounts using temporal, network, language, and sentiment features.It is trained on labeled examples discovered with honeypots and human raters.
- Classifier variants: Botometer offers standard and universal classifiers, with the latter omitting language features for use beyond English-speaking contexts.The classifiers are supervised and rely on labeled examples.
2.3 Claim verification
The paper identifies misinformation sources through third-party lists and validates the source-based assumption with manual review of a sample of articles.
- Source selection: The study merges independent lists of websites flagged as misinformation sources by journalistic and fact-checking organizations.The lists have uneven coverage but some overlap.
- Verification procedure: 50 sampled articles were independently evaluated by two reviewers, with disagreements resolved by a third reviewer.The sample included articles from sources whose content was tweeted at least once during the study period.
- Verification results: 27% of sampled claims could be verified, while unverifiable claims were grouped with misinformation.The verification rubric covered seven misinformation types plus inconclusive claims.
- Sampling caveat: The verification sample is biased toward prolific sources, and sampling by tweets instead favors popular sources.The fraction of verified claims is cut in half when claims are sampled by tweets.
- Fact-checking sources: The tracked fact-checking websites included PolitiFact, Snopes, FactCheck.org, and other independent organizations.Climate Feedback was added in April 2017 and does not affect the present analysis.
2.4 Hoaxy system architecture
Hoaxy combines Twitter and source-site collection with processing, storage, indexing, search, and visualizations for studying misinformation and fact-checking diffusion.
- Back-end: Hoaxy’s back-end collects tweets and source documents, stores them, indexes documents, and exposes data through an API.Twitter filtering and separate website crawling supply the collection pipeline; Lucene supports full-text search.
- Back-end: URL heuristics canonicalize about 30% of URLs and resolve shortened URLs for about 45% of URLs extracted from tweets.The pipeline focuses on common UTM parameters and follows HTTP redirects; snapshotting and domain aliases are ignored.
- Deployment: Hoaxy provides search, timeline, and interactive diffusion-network visualizations for claims and subsequent fact-checking.Users query indexed claims and fact-checking articles, select results, and visualize tweet growth or network information flow.
- Deployment: By October 2017, Hoaxy had collected 29,351,187 tweets and 653,911 documents, mostly from claim sources.Claim sources contributed 27,648,423 tweets and 628,350 documents, compared with 1,705,576 tweets and 25,561 documents from fact-checking sources.
- Deployment: The public interface launched on December 20, 2016, and user interests shifted among topics including vaccines, pizzagate, voter fraud, and Trump Russia.Daily query activity followed popular pieces of controversial information during the first six months of operation.
2.5 Datasets
The dataset analysis constructs weighted, directed retweet networks and separates complementary datasets to study diffusion, core evolution, central users, and robustness.
- Network construction: 66.9% of collected tweets were retweets or quoted retweets, with quoted retweets comprising approximately 1 in 10 retweets.The analysis therefore focuses on retweet and quoted-retweet relations to identify primary and secondary spreaders.
- Network construction: The network contains Twitter-account nodes and weighted directed edges whose weights count retweets from one account to another.Edges are labeled by retweeted content type, separating claim and fact-check counts as w(e) = wc(e) + wf(e).
- Datasets: The full six-month network combines claims and fact-checking and is decomposed into k-core shells to examine its densest subgraph.The analysis is split into pre- and post-Election Day periods.
- Datasets: A second dataset excludes fact-check edges and extends observation through October 9, 2017 to study long-term evolution of the misinformation core.Additional claim sources added in December 2016 are excluded for consistency across periods.
- Datasets: A third claims-only dataset covers the pre-election period to characterize central core users and assess network robustness.Node disconnection evaluates reductions in total claim-retweet volume and unique claim links after removing nodes.
3 Results
The pre-election misinformation network has a dense, increasingly stable core dominated by misinformation spreaders, with fact-checking rapidly diminishing toward its center and social bots more prevalent there.
- 3.1 Claims vs. fact-checking: k-core decomposition separates the network into fact-checker and misinformation-spreader communities, with fact-checking disappearing toward the densest main core.The main core is obtained at max{k} = 50; cross-community edges are mostly fact-checks.
- 3.1 Claims vs. fact-checking: For k > 20, the spread of fact-checking drops precipitously to a small, stable residual activity.Some fact-checking content is still retweeted inside the main core, often by accounts that mainly spread misinformation.
- 3.1 Claims vs. fact-checking: Fact-checking retweets made by main-core spreaders commonly use misleading wording, attack fact-checking sites, or contradict the article’s stance.These characteristics were identified through manual inspection of fact-checking retweets by core spreaders.
- 3.2.1 Core dynamics: The cumulative misinformation core grows larger and denser over time, reaching an equilibrium size of approximately 800 accounts around Election Day.The main core’s size stabilizes while its density continues to increase.
- 3.2.1 Core dynamics: After a peak in August, the main core has low churn, and 321 users remain members throughout the observation period.The low churn implies a stable set of users consistently driving network activities.
- 3.2.2 Core membership: Bot detection and centrality analysis identify social bots as more prevalent in the main core, while in-strength best captures its predominantly secondary-spreading accounts.Centrality measures produce different rankings, and in-strength indicates prolific retweeting accounts.
4 Discussion
The paper presents Hoaxy as an open platform for studying misinformation and fact-checking on Twitter, alongside an analysis of misinformation diffusion around the 2016 election. It reports a segregated network with a dense, stable post-election core, while acknowledging limits in source, platform, geographic, linguistic, and claim-verification coverage.
- Hoaxy enables large-scale studies of how misinformation and fact-checking spread and compete on Twitter, with data available through an open API.
- The analyzed network is strongly segregated by information type, and a dense, stable core emerged after the election.
- Hoaxy captures every collected tweet carrying each tracked misinformation claim, but its corpus covers only a fixed set of sources and Twitter activity.Facebook and other social media platforms are not included, so coverage of the broader misinformation ecosystem is incomplete.
- The analysis excludes unverified claims reported in the professional mainstream press, leaving broader media dynamics outside the study's scope.
- The study considers only US-based sources publishing English content, limiting direct coverage of misinformation in other national and linguistic contexts.The authors released Hoaxy's code to support study of non-US contexts.
- Hoaxy's collected claims are generally not verified, although its search interface shows relevant fact-checking articles to facilitate claim verification.