Source-linked AI summary
Ginger Cannot Cure Cancer: Battling Fake Health News with a Comprehensive Data Repository
Enyan Dai, Yiwei Sun, Suhang Wang
TL;DR
Fake health-news detection faces specialized challenges, including opaque methods and limited comprehensive health-focused datasets. The paper constructs FakeHealth, a repository combining rich news content, explanatory reviews, social engagements, and user networks; its analyses and baseline evaluations demonstrate dataset quality and identify useful patterns and challenges.
Problem
Fake health-news detection faces specialized challenges, while few studies address the area and comprehensive datasets are lacking.
Source: Introduction · p. 1Method
The authors construct FakeHealth, combining health-news content, reviews with explanations, social engagements, and user-user networks.
Source: Introduction · pp. 1–2Results
FakeHealth’s exploratory analyses and evaluated approaches demonstrate dataset quality and identify useful patterns and challenges for detection research.
Source: Conclusion · p. 9Takeaways & Limitations
The repository’s varied features support multiple detection approaches and potential research on explainable health fake-news detection.
Source: Introduction · p. 2
Abstract
from arXiv · showhide
Nowadays, Internet is a primary source of attaining health information. Massive fake health news which is spreading over the Internet, has become a severe threat to public health. Numerous studies and research works have been done in fake news detection domain, however, few of them are designed to cope with the challenges in health news. For instance, the development of explainable is required for fake health news detection. To mitigate these problems, we construct a comprehensive repository, FakeHealth, which includes news contents with rich features, news reviews with detailed explanations, social engagements and a user-user social network. Moreover, exploratory analyses are conducted to understand the characteristics of the datasets, analyze useful patterns and validate the quality of the datasets for health fake news detection. We also discuss the novel and potential future research directions for the health fake news detection.
Introduction
False health claims spread widely and can threaten public health, while health-news detection faces specialized challenges and a shortage of suitable datasets. FakeHealth addresses this gap with two feature-rich datasets, exploratory analyses, and directions for future work.
Introduction
- False health news can spread broadly and threaten public health, while subtle claims and readers’ limited ability to assess medical credibility complicate detection.Explanations are also important because existing detection methods, especially deep-learning models, are opaque.Source: Introduction · p. 1
- The field has few health-focused detection studies, in part because comprehensive fake-health-news datasets are lacking.The authors introduce FakeHealth as a repository containing HealthStory and HealthRelease.Source: Introduction · p. 1
- FakeHealth combines news content, reviews with explanations, social engagements, and user information to support detection, interpretability, dataset-quality analyses, and future research directions.The proposed directions include explainable and knowledge-based detection.Source: Introduction · p. 2
Related Work
Prior fake-news datasets cover political, general, war-related, and science topics, with varying combinations of article content and social context. The authors identify gaps in health coverage, user networks, and explanatory knowledge, motivating FakeHealth’s health-news datasets.
Related Work
- Existing datasets include political articles and statements, general news, and Syrian-war articles, but differ in scope and labeling approaches.BuzzFeedNews, LIAR, NELA-GT-2018, and FA-KES illustrate these varied dataset designs.Source: Related Work · p. 2
- Some datasets add engagement data, but coverage ranges from event posts without original articles to repositories with user networks and spatiotemporal information.FakeNewsNet covers political news; several engagement datasets lack user profiles.Source: Related Work · p. 2
- The authors identify insufficient health-news content, limited user-network information, and absent related knowledge and explanations as gaps addressed by FakeHealth.Their datasets combine health-news content, social engagements and networks, and reviews answering standard health-news criteria.Source: Related Work · p. 2
Dataset Collection
FakeHealth is built from HealthNewsReview material and associated news content, Twitter activity, and information about engaged users. Its collection process combines review and article acquisition with social-engagement searches and user-network collection.
HealthNewsReview
- HealthNewsReview.org reviews health-care claims using standard criteria and experienced reviewers from several relevant fields.The project operated from 2005 to 2018 and its reviewers signed industry-independent disclosure agreements.Source: HealthNewsReview · p. 2
Data Collection Pipeline
- The collection begins by crawling HealthNewsReview stories and releases, then scraping the corresponding source-news content.News reviews published before December 2009 were filtered out, and Wayback Machine was used when source URLs were missing or out of date.Source: Data Collection Pipeline · pp. 2–3
- Twitter engagements were collected using title, extracted-keyword, and long-URL searches, followed by scraping replies and retweets.The keyword search retained at least five keywords; retweets were capped at 200 per tweet by the Twitter API.Source: Data Collection Pipeline · p. 3
- The pipeline supplements engagement data with profiles, recent timelines, and follower/following information for users who interacted with or were mentioned in the news-related posts.Source: Data Collection Pipeline · p. 3
Dataset Description
FakeHealth separates health news stories from health news releases because they come from different sources, while organizing both datasets around content, reviews, social engagement, and user networks. Expert ratings use shared and dataset-specific criteria to assign overall scores and classify news pieces as fake or real.
Dataset Description
- HealthStory covers news stories from media outlets such as Reuters Health, while HealthRelease covers releases from institutes, universities, research centers, and companies.The datasets are kept separate because their sources differ.Source: Dataset Description · p. 3
- Both datasets organize multiple features into news contents, news reviews, social engagements, and user networks.Source: Dataset Description · p. 3
- Experts assess news on ten criteria: eight are shared, while HealthStory and HealthRelease each have two dataset-specific criteria.The overall rating ranges from 0 to 5 and is proportional to the number of criteria satisfied.Source: Dataset Description · p. 3
- News pieces scoring below 3 are treated as fake news, following Shu et al. (2018).Source: Dataset Description · p. 4
Exploratory Analysis
Exploratory analyses compare real and fake health news across review criteria, topics, sources, and social context. The findings identify candidate features and challenges for detection, while baseline results support the datasets’ use for future evaluations.
Exploratory Analysis
- The exploratory analysis compares real and fake health news across multiple dimensions to identify potential detection features and challenges, and provides baselines for future research.Source: Exploratory Analysis · p. 4
News Reviews
- Expert-assessed review criteria show minimal or no correlation and contribute equally to real/fake labels; unigram and POS features predict most criteria above chance.The authors say criterion prediction could support explainable detection and multi-aspect news ratings.Source: News Reviews · p. 4, Table 3 · p. 5
News Topics
- Real and fake news have broadly similar topic distributions in both datasets, although some topics, including stem-cell therapies, are more associated with fake news.The authors suggest tags and keywords may provide useful detection features.Source: News Topics · pp. 4–5, Figure 4 · p. 5, Figure 3 · p. 5
News Sources
- In HealthStory, publisher ratings differ substantially: Associated Press averages 3.87 versus WebMD’s 2.82 (p < 0.001), indicating source may be a useful feature.The comparison is limited to HealthStory because HealthRelease publishers are too diverse for meaningful comparison.Source: News Sources · p. 5, Figure 5 · p. 5
Social Context
- In HealthStory, real stories receive more tweets and retweets and more positive replies than fake stories; these differences are not significant in HealthRelease.The HealthRelease reply-score pattern is similar but not statistically significant; the authors note its relatively limited number of replies.Source: Social Context · pp. 5–6, Figure 6 · p. 6, Table 4 · p. 6
- Users propagating fake health news are slightly more likely to be bots in both datasets, and users involved in health news vary in bot-likelihood.Bot-likelihood scores were compared for randomly sampled groups of 2,000 users associated only with real news and 2,000 associated with fake news in each dataset.Source: Social Context · p. 6, Figure 7 · p. 6
- Fake-news engagement can spike while real-news engagement rises steadily; observed bot patterns include synchronized posting, short bursts, and repeated posts over a month.The authors note that health-news dissemination may last several months, compared with four days for general news.Source: Figure 8 · p. 6, Social Context · p. 7, Figure 9 · p. 7
Performance of Fake Health News Detection
- Simple baselines perform relatively well against random guessing, and added news-source, tag, and social-context features help, while limited performance leaves room for improvement.The authors say the results validate the labels and collected features and provide reference results for future evaluations.Source: Performance of Fake Health News… · p. 8, Table 5 · p. 8
Potential Research Directions
FakeHealth’s extensive news-side information supports a range of potential applications in health-news research. The proposed directions include explainable, knowledge-based, credibility-based, multimodal, early-detection, and propagation analyses.
Explainable Fake Health News Detection
- Ten binary health-criteria labels and review explanations could support identifying poorly illustrated aspects of news and help users and developers understand classification decisions.The authors suggest explanations could help users think critically and developers detect issues such as biased training data or adversarial samples.Source: Explainable Fake Health News Detection · p. 8
Knowledge-based Fake Health News Detection
- News contents and reviews provide background knowledge and corrections from which subject-predicate-object triples could form graphs for checking claims against retrieved values.The authors propose using retrieved knowledge to explain verification or falsification of health-news claims.Source: Knowledge-based Fake Health News… · p. 8
Credibility-based Fake Health News Detection
- Topic, source, and user-side information—including tags, keywords, and user profiles, timelines, and friends—could inform credibility-based detection.The paper notes that its analysis links news quality to sources and reports more fake news about stem cells in its analysis.Source: Credibility-based Fake Health News… · p. 8
Multi-Modal Fake News Detection
- Article, tweet, and user-profile image links could support multimodal models that examine visuals for clues or user credibility.The paper also notes that user profile images may reveal the credibility of users’ posts.Source: Multi-Modal Fake News Detection · p. 8
Fake Health News Early Detection
- Tweets, retweets, replies, user profiles, recent timelines, and friend profiles could support early detection when social engagements are limited.The authors describe early detection as identifying fake news before it has been widely spread.Source: Fake Health News Early Detection · p. 9
Fake Health News Propagation
- The datasets’ engagements could support studying how real and fake health news spread across topics and identifying high-impact fake news.The paper reports that both normal users and bots contribute to dissemination of real and fake news, while the impact of health news varies.Source: Fake Health News Propagation · p. 9
Conclusion
The paper releases FakeHealth, a repository of two feature-rich health-news datasets, and reports exploratory analyses, evaluations, and potential research directions.
Conclusion
- FakeHealth contains two health-news datasets with news contents, social engagements, user-user networks, and explanations supporting the ground truth.The repository is intended to facilitate research on fake health news.Source: Conclusion · p. 9
- Exploratory analyses describe dataset characteristics and identify patterns and challenges relevant to fake health news detection.Source: Conclusion · p. 9
- Evaluations of different detection approaches provide baselines for future studies and are reported as demonstrating dataset quality.The paper also presents potential future research directions enabled by the repository’s information.Source: Conclusion · p. 9
Data Format
FakeHealth organizes news contents, reviews, social engagements, and user-network information; the repository also describes review criteria and access to engagement data.
Data Format
- The repository separates contents, reviews, engagements, and user-network data into four categories, with HealthStory and HealthRelease content and engagement folders.Reviews are stored in dataset-specific JSON files; engagements include tweets, retweets, and replies, while user-network folders hold profiles, timelines, followers, and followings.Source: Data Format · p. 9
- Review criteria comprise eight common criteria, plus separate criteria for news stories and news releases.Table 6 labels the common criteria C1–C8, and the special criteria S9–S10 for stories and R9–R10 for releases.Source: Table 6 · p. 9
- Because Twitter policy prevents direct publication of full engagement and network contents, the repository stores IDs and provides an API to retrieve them from Twitter.The authors state they will maintain and update the repository to ensure usability.Source: Data Format · p. 9