Source-linked AI summary
Graph Neural Networks with Continual Learning for Fake News Detection from Social Media
Yi Han, Shanika Karunasekera, Christopher Leckie
TL;DR
Fake news spreads rapidly online, creating a need for accurate detection that is less dependent on manipulable text and can adapt to new datasets. The paper uses GNNs to classify propagation patterns with limited non-textual features and applies continual-learning techniques for incremental adaptation. GNNs match or outperform state-of-the-art methods without text, while GEM and EWC provide balanced performance across existing and new datasets.
Problem
The paper addresses accurate propagation-based fake-news detection without text and adaptation when a trained GNN encounters new, potentially different data.
Method
The paper uses GNNs to distinguish fake and real news through propagation patterns and applies GEM and EWC during incremental training.
Results
GNNs achieve comparable or superior performance without text, while continual-learning techniques achieve balanced performance on existing and new datasets.
Takeaways & Limitations
Propagation patterns and limited social-context features can support fake-news detection, while continual learning avoids retraining on the entire dataset as data grows.
Takeaways & Limitations
The approach may require more feature engineering to mitigate catastrophic forgetting across different graph structures.
Abstract
from arXiv · showhide
Although significant effort has been applied to fact-checking, the prevalence of fake news over social media, which has profound impact on justice, public trust and our society, remains a serious problem. In this work, we focus on propagation-based fake news detection, as recent studies have demonstrated that fake news and real news spread differently online. Specifically, considering the capability of graph neural networks (GNNs) in dealing with non-Euclidean data, we use GNNs to differentiate between the propagation patterns of fake and real news on social media. In particular, we concentrate on two questions: (1) Without relying on any text information, e.g., tweet content, replies and user descriptions, how accurately can GNNs identify fake news? Machine learning models are known to be vulnerable to adversarial attacks, and avoiding the dependence on text-based features can make the model less susceptible to the manipulation of advanced fake news fabricators. (2) How to deal with new, unseen data? In other words, how does a GNN trained on a given dataset perform on a new and potentially vastly different dataset? If it achieves unsatisfactory performance, how do we solve the problem without re-training the model on the entire data from scratch? We study the above questions on two datasets with thousands of labelled news items, and our results show that: (1) GNNs can achieve comparable or superior performance without any text information to state-of-the-art methods. (2) GNNs trained on a given dataset may perform poorly on new, unseen data, and direct incremental training cannot solve the problem---this issue has not been addressed in the previous work that applies GNNs for fake news detection. In order to solve the problem, we propose a method that achieves balanced performance on both existing and new datasets, by using techniques from continual learning to train GNNs incrementally.
1 INTRODUCTION
The paper investigates propagation-based fake-news detection with GNNs, asking whether non-textual signals suffice and how models can adapt to unseen datasets without full retraining.
- Approach: Propagation-based detection uses how news spreads on social media, motivated by evidence that fake and real news spread differently online.The approach is language-agnostic and can be less vulnerable to adversarial attacks than content-based methods.
- Related Work: The study builds on prior propagation-based and GNN-based fake-news detection work while selecting a graph-classification algorithm for its investigation.
- Question 1: The paper examines whether GNNs can identify fake news without tweet content, replies, or user descriptions.
- Question 1: GNNs can achieve comparable or superior performance to state-of-the-art methods using propagation patterns and restricted non-textual features rather than sophisticated text analyses.The features come from user profiles and timeline tweets, while the approach avoids tweet content and user replies.
- Question 2: GNNs trained on one dataset may perform poorly on new data, and direct incremental training does not solve this problem.The issue arises when the new data may differ substantially from the training data.
2 BACKGROUND ON GRAPH NEURAL NETWORKS
GNNs process non-Euclidean graph data by propagating node features through graph structure, and this paper uses DiffPool for graph-level propagation-pattern classification.
- GNNs: GNNs are designed for non-Euclidean data, unlike deep-learning methods that mainly process Euclidean data.
- Graph Representation: A graph is represented by adjacency matrix A and feature matrix F, with each node having d features; the GNN initializes H0 as F.
- Propagation Function: A GNN propagates graph information through parameterized layers, applying an activation function and learned weight matrix at each layer.
- Propagation Function: A popular propagation implementation adds self-loops and normalizes adjacency using the degree matrix before multiplying node representations by layer weights.
- Graph Classification: For graph classification, the paper selects DiffPool, which learns a coarsened graph while incorporating structural information.DiffPool extends existing GNN models by using the original node representations and adjacency structure.
3 PROPAGATION-BASED FAKE NEWS DETECTION
The paper represents each news item as a propagation graph and applies GNNs using non-textual features to detect fake news. Experiments show comparable or superior detection performance, including with limited early-propagation data, while additional timeline and follower relations provide little improvement.
- 3.1 Data Generation: Each news item becomes a graph whose nodes are root tweets or retweets, with edges representing inferred information flow within cascades.When Twitter does not expose immediate retweet sources, the method orders tweets by time and searches earlier tweets for a source.
- 3.1 Data Generation: The model excludes tweet content, replies, and user descriptions, using propagation structure plus profile features such as verification, follower counts, and temporal information.The selected features are directly available from tweet objects, supporting online detection.
- 3.2 Experimental Verification: Training uses random 75%/25% train-test splits, averages results over five repeats, and evaluates accuracy, precision, recall, and F1 score.Multiple random splits are used because performance can vary substantially across data partitions.
- 3.2 Experimental Verification: Non-textual GNN features achieve comparable performance on PolitiFact and the best result on GossipCop against methods requiring textual analysis.The comparisons use eight previously reported fake-news detection algorithms and the proposed propagation-based method.
- 3.2.2 Training on the Partial Dataset for Early Detection: With limited propagation histories, the model still achieves decent performance, especially on GossipCop, including when only the first 100 tweets are used.The authors associate the stronger partial-data performance with GossipCop’s larger dataset size.
- 3.2.3 Additional Non-textual Features from User Timeline Tweets: Timeline-tweet features and follower/following relations do not produce obvious or significant performance improvements, so follower/following relations are omitted.On GossipCop, timeline-tweet features alone perform equally well to the earlier models.
4 DEALING WITH NEW DATA
Models trained on one news dataset generalize poorly to another, and ordinary incremental training preserves only the newest dataset's performance. Continual-learning methods, especially GEM, provide more balanced performance across datasets.
- 4 DEALING WITH NEW DATA: Models trained on PolitiFact perform poorly on GossipCop because the datasets' propagation graphs differ substantially in node and edge counts.
- 4 DEALING WITH NEW DATA: Retraining from scratch on all accumulated data is undesirable because continually arriving datasets would make repeated full retraining increasingly expensive.
- 4.1 Incremental Training: Direct incremental training performs well on the newly trained dataset but degrades performance on the previously learned dataset, exhibiting catastrophic forgetting.
- 4.2 Continual Learning: The proposed continual-learning setup treats each dataset as a separate task and applies GEM and EWC while learning the second dataset.
- 4.2 Continual Learning: GEM and EWC achieve relatively balanced performance across both datasets, with GEM generally outperforming EWC; incremental GEM training on the whole dataset improves performance further.The comparison is reported in Figs. 6, 7, and Table 2.
- 4.2 Continual Learning: EWC requires additional fine-tuning, including early stopping to obtain balanced results, while GEM and EWC add only slightly more training time than normal training.
5 RELATED WORK
Prior fake-news detection methods use content, social context, or both, with approaches ranging from knowledge graphs and linguistic style analysis to interaction-based graph models. Recent work also considers explainability.
- 5 RELATED WORK: Existing methods are organized into content-based, context-based, and mixed approaches according to whether they use news content, social interactions, or both.
- 5.1 Content-based Approaches: Knowledge-based content methods compare extracted subject-predicate-object facts against a knowledge graph, using link prediction when a fact is missing.
- 5.1 Content-based Approaches: Style-based methods detect fake news through writing characteristics that distinguish fabricated statements from real-life accounts.
- 5.2 Context-based Approaches: Context-based methods exploit interactions such as tweets, retweets, replies, mentions, follows, or likes to infer credibility or detect fake news.
- 5.3 Mixed Approaches: Mixed approaches combine article text, user responses, profiles, publishers, subjects, and user relationships through recurrent, diffusive, or matrix-factorization models.
- 5 RELATED WORK: A smaller body of recent work investigates why models assign fake-news labels, focusing on explainability.
6 CONCLUSIONS AND FUTURE WORK
The paper uses GNNs to detect fake news from propagation patterns without text, achieving comparable or superior performance to text-intensive methods. It also applies continual-learning techniques to balance performance on existing and new datasets, while identifying feature choices as future work.
- GNNs using limited social-context features and no text achieve comparable or superior performance to state-of-the-art fake-news detectors requiring syntactic and semantic analyses.
- GEM and EWC support incremental GNN training that balances performance on existing and new data without retraining on the entire dataset.
- Future work will examine whether adding features or finding universal features can mitigate catastrophic forgetting across different graph structures.