Source-linked AI summary
FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media
Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, Huan Liu
TL;DR
Fake-news detection requires more than article text because misleading stories exploit social-media contexts, while existing datasets lack comprehensive multidimensional information. FakeNewsNet addresses this gap by integrating fact-checked news content with social context and spatiotemporal data, then exploring the repository through sentiment, engagement, propagation, and detection analyses. The repository demonstrates utility for fake-news research, including detection, propagation, mitigation, and related studies.
Problem
Existing fake-news datasets generally lack comprehensive news content, social context, and spatiotemporal information needed to study detection and propagation.
Method
The paper constructs FakeNewsNet by integrating fact-checked news content with social-context and spatiotemporal information, then performs exploratory and detection analyses.
Results
FakeNewsNet demonstrates utility through exploratory analyses of user responses, propagation patterns, and fake-news detection tasks using state-of-the-art baselines.
Takeaways & Limitations
The repository supports research on fake-news detection, propagation, mitigation, evolution, and related social-media studies.
Abstract
from arXiv · showhide
Social media has become a popular means for people to consume news. Meanwhile, it also enables the wide dissemination of fake news, i.e., news with intentionally false information, which brings significant negative effects to the society. Thus, fake news detection is attracting increasing attention. However, fake news detection is a non-trivial task, which requires multi-source information such as news content, social context, and dynamic information. First, fake news is written to fool people, which makes it difficult to detect fake news simply based on news contents. In addition to news contents, we need to explore social contexts such as user engagements and social behaviors. For example, a credible user's comment that "this is a fake news" is a strong signal for detecting fake news. Second, dynamic information such as how fake news and true news propagate and how users' opinions toward news pieces are very important for extracting useful patterns for (early) fake news detection and intervention. Thus, comprehensive datasets which contain news content, social context, and dynamic information could facilitate fake news propagation, detection, and mitigation; while to the best of our knowledge, existing datasets only contains one or two aspects. Therefore, in this paper, to facilitate fake news related researches, we provide a fake news data repository FakeNewsNet, which contains two comprehensive datasets that includes news content, social context, and dynamic information. We present a comprehensive description of datasets collection, demonstrate an exploratory analysis of this data repository from different perspectives, and discuss the benefits of FakeNewsNet for potential applications on fake news study on social media.
1 Introduction
FakeNewsNet addresses the difficulty of detecting intentionally misleading social-media news from content alone by integrating news content, social context, and spatiotemporal information. The paper presents the repository, explores its datasets, and discusses applications to detection, evolution, and mitigation.
- Motivation: Social media facilitates rapid, widespread information sharing but also enables fake news that can affect individuals, society, and real-world events.The paper identifies financial and political incentives for spreading false or biased stories and cites Pizzagate as an example of real-world impact.
- Research gap: Content-only detection is insufficient because fake news is intentionally written to mislead consumers.The paper therefore highlights user engagements, social behaviors, and spatiotemporal propagation patterns as additional signals.
- Research gap: Existing datasets generally cover only one or two aspects, leaving a need for comprehensive news-content, social-context, and spatiotemporal data.The missing temporal information limits study of propagation, user reactions, and early detection or intervention patterns.
- Contribution: FakeNewsNet provides two datasets combining news content, social context, and spatiotemporal information through an end-to-end data-collection system.The repository is constructed with FakeNewsTracker and is intended to support multiple fake-news research problems.
- Contribution: The paper explores dataset characteristics from multiple perspectives and provides baselines for future fake-news detection research.It also discusses potential uses for studying fake-news evolution, propagation, and mitigation.
2 Background and Related Work
Prior fake-news datasets often provide limited information, labels, or domain coverage. FakeNewsNet addresses these gaps by collecting multidimensional data across news content, social context, and spatiotemporal information from political and entertainment domains.
- Dataset limitations: Several benchmark datasets rely on restricted sources or labels, including BS Detector outputs rather than human annotations.Other datasets use limited samples, short claims, or specialized collections such as conspiracy and scientific-news pages.
- Existing datasets: CREDBANK provides around 60 million tweets covering 96 days and more than 1,000 news events, with credibility assessed by 30 annotators.Its scale and crowd-sourced credibility assessments illustrate a different coverage strategy from article-centered repositories.
- Dataset limitations: Existing fake-news datasets commonly omit parts of the news-content, social-context, and spatiotemporal information space.Examples include datasets focused on headlines and text, short statements, or narrow domains.
- FakeNewsNet: FakeNewsNet collects multidimensional information from political and entertainment news domains to address the disadvantages of existing datasets.The repository is designed to support fake-news detection, propagation, and intervention research.
3 Dataset Integration
FakeNewsNet integrates fact-checked news content with social-context and spatiotemporal information. Its collection process uses fact-checking sources, social-media search and engagement data, user metadata, locations, and timestamps.
- Integration overview: The integration process combines news content, social context, and spatiotemporal information into the FakeNewsNet repository.Figure 1 presents this collection and integration flow.
- News content: Fact-checking websites provide ground-truth labels and source news content for fake and real stories.PolitiFact uses journalist and domain-expert evaluations; removed pages are recovered through archives or automated web search.
- News content: GossipCop stories rated below 5 are collected as fake entertainment news, while E! Online articles are treated as real entertainment news.Because GossipCop may not provide source URLs, headlines are searched through Google or the Wayback Machine.
- Social context: Social context is collected through platform search APIs and expanded with replies, likes, reposts, and metadata for participating users.Twitter search queries are formed from news headlines and used to retrieve posts that directly spread the stories.
- Spatiotemporal information: Spatiotemporal information records user-profile locations and engagement timestamps, enabling analysis of propagation and topic changes over time.The repository is also dynamically updated with newly added fact-checked news articles.
4 Data Analysis
FakeNewsNet’s exploratory analysis examines news content, user profiles, sentiments, interactions, networks, temporal diffusion, and detection performance across its datasets. These analyses reveal distinguishing patterns between fake and real news and show the value of combining social context with article content.
- News Content: Fake and real articles cover similar broad topics, but their specific topics differ, making content alone insufficient for detection.PolitiFact articles mainly concern political campaigns, while GossipCop articles concern celebrity relationships.
- News Content: 301 PolitiFact publishers produced 432 fake news pieces, while 209 GossipCop publishers produced 6,048 fake news pieces.In PolitiFact, 191 publishers produced one fake piece; in GossipCop, 114 publishers did so.
- User Profiles: 22% of users involved in fake news were bots, compared with around 9% of users involved in real news.Botometer assigns scores in [0, 1], and the analysis uses 0.5 as the bot-account threshold.
- Post and Response: Fake-news replies show a larger negative-sentiment ratio, whereas real-news replies contain more neutral sentiment; GossipCop shows no significant sentiment difference.Sentiment ratios are computed for positive, negative, and neutral replies using VADER.
- Post and Response: Fake news tends to receive fewer replies and more retweets, while real news receives a higher ratio of likes.The authors interpret these differing engagement distributions as useful for studying user beliefs and diffusion patterns.
- Spatiotemporal Information: Fake news shows a sudden retweet increase that soon stabilizes, whereas real news shows steadily increasing retweets.Replies account for 5.76% of fake-news tweets and 7.93% of real-news tweets.
- Fake News Detection Performance: SAF improves accuracy over SAF/S and SAF/A by around 5.65% and 3.68%, respectively, on PolitiFact.The results indicate that user engagements add detection information beyond news articles on PolitiFact.
5 Data Structure
FakeNewsNet organizes each news item with news content, tweets, retweets, user profiles, and user timelines, while an API supports access to subsets of the large dataset.
- Data organization: Each news directory is named with an autogenerated news ID and contains news content, tweets, retweets, user profiles, and user timeline data.The news content file stores metadata collected from source URLs, while tweets and retweets are collected through the Twitter API.
- Data organization: The tweets folder stores metadata for tweets associated with each news article as separate Twitter API objects.
- Data organization: The retweets folder stores files named by tweet ID, each containing retweet objects associated with a particular tweet.
- Data access: The full dataset cannot be directly distributed because of Twitter policy, so the authors provide documented Python scripts and URL-to-tweet-ID files for selective downloads.Users initiate the API by running main.py with the required configuration.
6 Potential Applications
FakeNewsNet is positioned as a multi-dimensional resource for detection, propagation, mitigation, and malicious-account research across political and entertainment news.
- Potential applications: FakeNewsNet supports research on early fake news detection, fake news evolution, mitigation, and malicious account detection.
- Fake news detection: Its reliable journalist- and expert-annotated labels and multi-dimensional information address the need for benchmark data with comprehensive evidence.The repository combines news content, social context, and spatiotemporal information.
- Fake news detection: News attributes such as publishers, headlines, body texts, and videos support extraction of linguistic features for distinguishing fake news.
- Fake news detection: User profiles, posts, reposts, comments, and likes provide social-context signals about users, opinions, and news proliferation.
- Cross-domain analysis: The two datasets cover political and entertainment domains, enabling comparison of common and topic-specific fake-news patterns.
- Fake news evolution: Temporal user engagements enable analysis of diffusion trajectories and comparisons of changing patterns between fake and real news.
- Fake news mitigation: The repository supports mitigation research by identifying provenances, persuaders, and clarifiers and by building spatiotemporal user-diffusion networks.These data can support influence minimization and mitigation campaigns.
- Malicious account detection: User profile metadata, historical tweets, and social-network information can be used to analyze and cluster malicious accounts.A preliminary study found that bot users are more likely to appear in fake-news spreading processes.
7 Conclusion and Future Work
The paper presents FakeNewsNet as a comprehensive repository combining news, social, and spatiotemporal information, and outlines extensions to broaden sources and improve collection quality.
- Conclusion: FakeNewsNet combines news content, social context, and spatiotemporal information through a principled multi-source collection strategy.
- Conclusion: The authors explore repository features and demonstrate its utility through fake news detection tasks using several state-of-the-art baselines.
- Future work: Future work will extend the repository to additional reliable sources, reduce web-search noise, and integrate it with front-end software for an end-to-end system.