Source-linked AI summary
The Web of False Information: Rumors, Fake News, Hoaxes, Clickbait, and Various Other Shenanigans
Savvas Zannettou, Michael Sirivianos, Jeremy Blackburn, Nicolas Kourtellis
TL;DR
False information on OSNs can manipulate public opinion and produce societal harms, including effects on elections and crisis-related panic. This paper develops a typology of the ecosystem and reviews research on perception, propagation, detection and containment, and political false information. It concludes that automated solutions without human input cannot effectively mitigate the problem and calls for awareness and holistic research approaches.
Problem
False information campaigns on OSNs can manipulate individuals and cause serious societal consequences, including altered election outcomes, panic, and chaos.
Method
The paper constructs a typology of false-information types, actors, and motives and synthesizes research across four ecosystem research lines.
Results
The review finds that false-information research spans user perception, propagation, detection and containment, and political dynamics, with political false information propagating faster and farther than other types.
Takeaways & Limitations
The paper argues that effective mitigation requires human input, public awareness, and a holistic approach to studying and addressing the ecosystem.
Takeaways & Limitations
The reviewed literature lacks rigorous temporal analysis of user perception and does not consider interactions across multiple OSNs.
Abstract
from arXiv · showhide
A new era of Information Warfare has arrived. Various actors, including state-sponsored ones, are weaponizing information on Online Social Networks to run false information campaigns with targeted manipulation of public opinion on specific topics. These false information campaigns can have dire consequences to the public: mutating their opinions and actions, especially with respect to critical world events like major elections. Evidently, the problem of false information on the Web is a crucial one, and needs increased public awareness, as well as immediate attention from law enforcement agencies, public institutions, and in particular, the research community. In this paper, we make a step in this direction by providing a typology of the Web's false information ecosystem, comprising various types of false information, actors, and their motives. We report a comprehensive overview of existing research on the false information ecosystem by identifying several lines of work: 1) how the public perceives false information; 2) understanding the propagation of false information; 3) detecting and containing false information on the Web; and 4) false information on the political stage. In this work, we pay particular attention to political false information as: 1) it can have dire consequences to the community (e.g., when election results are mutated) and 2) previous work show that this type of false information propagates faster and further when compared to other types of false information. Finally, for each of these lines of work, we report several future research directions that can help us better understand and mitigate the emerging problem of false information dissemination on the Web.
1 Introduction
False information on OSNs enables targeted manipulation and can produce serious societal harms, including election effects, panic, and chaos. The paper organizes research around perception, propagation, detection and containment, and politics.
- False information campaigns on OSNs can manipulate individuals and affect elections, while crisis-related false information can cause panic and chaos.
- The paper proposes a typology addressing false information’s types, disseminating actors, and motives.
- The review covers user perception, propagation dynamics, detection and containment, and politics-related false information.
- Political false information receives separate attention because it can affect election outcomes, involves state-sponsored or coordinated actors, and has intensive effects on social media.
- The survey provides an overview of relevant research and future directions intended to address gaps and help alleviate false information dissemination.
2 False Information Ecosystem Typology
The paper presents a fine-grained typology of false information, its disseminating actors, and their motives. The categories can overlap, underscoring the ecosystem’s complexity.
- The typology distinguishes eight forms of false information and is designed as a roadmap based on an extensive literature review.
- Fabricated stories are fictional and disconnected from real facts, while propaganda is fabricated content intended to harm a party’s interests, usually politically.
- Conspiracy theories invoke unproven conspiracies and may present unsourced information as fact rather than using an evidence-based approach.
- Hoaxes present false or inaccurate facts as legitimate, whereas rumors remain ambiguous or unconfirmed.
- Clickbait uses misleading headlines or thumbnails to increase traffic, while biased stories are extremely one-sided toward a person, party, situation, or event.
- False-information categories can overlap, such as rumors using clickbait or propaganda combining fabrication with partisan bias.
- Actors include bots, criminal or terrorist organizations, activist or political organizations, governments, hidden paid posters, and state-sponsored trolls.
3 User Perception of False Information
Research on user perception examines how people encounter and assess false information through OSN data, questionnaires, interviews, and crowdsourcing. Findings span rumors, conspiracy theories, biased news, satire, and credibility assessment, while important temporal and cross-platform gaps remain.
- Studies use large-scale OSN datasets and direct user input from questionnaires, interviews, or crowdsourcing to examine perception and interaction with false information.
- OSN data-analysis research mainly examines how users perceive and interact with rumors and conspiracy theories.
- Rumors: Analysis of 1.7B tweets found rumor spreaders and non-rumor spreaders had similar registration age and follower counts, while rumors differed in writing style and social-relationship vocabulary.
- Rumors: Rumor studies report that true rumors resolved faster than false rumors, while users generally supported unverified rumors less often when information came from reputable accounts with evidence.
- Conspiracy Theories: Conspiracy-theory Facebook pages showed more negative sentiment than science pages, with sentiment becoming more negative as conversations grew larger.
- Future Directions: Reviewed work identifies gaps in rigorous temporal analysis and in understanding how users across multiple OSNs perceive the same false information.
4 Propagation of False Information
Research on false-information propagation spans data analysis, mathematical and statistical modeling, source detection, and visualization across online social networks. Findings describe rapid, bursty, persistent, and cross-community diffusion, while identifying credible accounts and source detection as potential containment aids.
- OSN Data Analysis: Propagation studies use data analysis, mathematical or statistical approaches, and visualization systems to examine false-information dynamics across OSNs.Table 2 organizes studies by methodology, platform, and false-information type.
- OSN Data Analysis: False rumors can propagate differently from confirmed news, and aggregate tweet analysis can distinguish the two.Studies examined earthquake and crisis-related rumors on Twitter.
- OSN Data Analysis: Credible or official accounts can help control rumor spread, although reputable accounts may also share false information.Active engagement and credibility are associated with rumor control, but credibility does not guarantee accurate sharing.
- Propagation Findings: False-information popularity is bursty and persistent, can form polarized communities and variants, and may spread more widely across interconnected online and offline layers.The review also identifies dedicated disseminator accounts and describes source-detection approaches as a first step toward stopping spread.
- OSN Data Analysis: False stories propagate faster, farther, and more broadly than true stories, with stronger effects for political false stories.This comparison comes from an 11-year Twitter diffusion study.
- Future Directions: Future work should study propagation across platforms and formats, develop visualization tools, and rigorously examine orchestrated campaigns.The review identifies these areas as gaps for understanding propagation and finding information sources.
5 Detection and Containment of False Information
Research on detecting and containing false information spans machine learning, credibility assessment, rumor systems, and containment strategies, but robust, timely mitigation remains unavailable. The review highlights generalization concerns, the need for human–machine collaboration, and future systems that handle multiple formats, user profiles, and platforms.
- Detection: Detection studies use handcrafted features, conventional machine learning, neural networks, and credibility models across multiple false-information categories.Approaches target rumors, hoaxes, satire, fabricated claims, clickbait, propaganda, and credibility assessment.
- Detection: 97% accuracy was achieved by a Tweet Latent Vector SVM Tree Kernel model on two Twitter datasets.The model represents tweet semantics with a proposed 100-dimensional vector.
- Detection: 99% accuracy was reported for identifying Facebook hoaxes from features based on users’ interactions.The study used Logistic Regression and user-like information.
- Systems: Rumor systems combine retrieval, classification, clustering, visualization, and human annotation to identify rumors and assess their validity.One collaborative system discarded 50% of 20M tweets for a Boston Marathon Bombings incident, while other systems used approximately 10k annotated tweets or semantic and syntactic features.
- Containment: Containment approaches add nodes that disseminate corrective information or block selected nodes while considering network structure and user experience.A sampling-based approach reported a 10x speed-up without compromising performance against state-of-the-art approaches.
- Findings and future directions: The review finds that feature-heavy machine learning may not generalize, effective mitigation requires human–machine collaboration, and future systems should integrate formats, user profiles, and OSNs.The paper notes that no robust platform currently mitigates false information effectively and efficiently in a timely manner.
6 False Information in the political stage
The political false-information literature examines propaganda, bots, political leaning, rumors, and coordinated campaigns across major online communities. The review emphasizes temporal coordination analysis, extensive bot use, machine-learning assistance with generalization concerns, and the need to study cross-platform political dissemination.
- Scope of political-stage research: Political-stage studies cover propaganda, biased information, rumors, bots, political leaning, and coordinated campaigns across Twitter and other online communities.Table 4 organizes the reviewed studies by methodology and considered online social network.
- Main insights: Temporal analysis can assess coordination among bots, state-sponsored actors, and orchestrated political false-information efforts.The review identifies bursty activity and coordinated behavior as relevant signals in political dissemination.
- Main insights: Bots are extensively used to disseminate political false information, including through coordinated posting, hashtags, retweets, and external URLs.Studies examined botnets associated with political agendas, conflicts, referenda, and terrorist propaganda.
- Main insights: Machine learning can assist in detecting political false information and users’ political leaning, but its generalization to other datasets and domains remains uncertain.The review reports machine-learning frameworks using topological, content-based, crowdsourced, and behavioral features.
- Future directions: Political campaigns substantially disseminate false information in mainstream Web communities, motivating cross-Web studies of its sources and containment.The paper proposes studying state-sponsored troll factories through user analytics and societal-impact perspectives.
7 Other related work
Related work extends beyond core detection and containment studies to credibility assessment, conspiracy theories, fabricated news, propaganda, bias, rumors, and manipulated or misleading images. These studies combine human judgments, interface systems, linguistic and structural analysis, and multimedia features.
- Scope: The review groups related studies into general studies, systems, and the use of images in the false-information ecosystem.These categories cover work that does not fit the paper’s main research lines.
- Actors and narratives: Qualitative and graph-based studies examine how websites, paid posters, and online communities promote conspiracy theories, propaganda, and political agendas.The reviewed work includes hidden paid posters assigned missions with deadlines and domain-link analysis of conspiracy theories.
- Fabricated information: Fake news differs structurally from real news through smaller content, simpler words, and longer clickbait headlines.The comparison was reported across three news-article datasets.
- Rumors: Linguistic patterns in 15M tweets from two crisis incidents can support timely rumor detection.The studies examined expressed uncertainty during the Boston Bombings and Sydney Siege.
- Credibility assessment: Credibility systems combine databases, disputed-claim warnings, user annotations, and machine processing to support real-time assessment of information.Examples include FactWatcher, Dispute Finder, and CREDBANK.
- Images: Images can increase perceived credibility while also being manipulated, motivating datasets and visual features for identifying false multimedia.Reviewed datasets include 12K labeled tweets and 50K Sina Weibo posts with 26K images.
8 Discussion & Conclusions
The paper concludes that false information requires a holistic research perspective spanning multiple Web communities and emphasizes real-time, cross-community monitoring. It also identifies gaps and future directions for improving understanding and mitigation.
- The overview identifies user perception, propagation dynamics, detection and containment, and political false information as major research areas.
- Current automated solutions without human input are unable to effectively mitigate false information on the Web.
- Researchers should develop models and methods that generalize across multiple Web communities and datasets.
- Real-time platforms should reveal how false information propagates across multiple Web communities.One example is alerting Twitter users when 4chan users promote a questionable-credibility hashtag.