Source-linked AI summary
CSI: A Hybrid Deep Model for Fake News Detection
Natali Ruchansky, Sungyong Seo, Yan Liu
TL;DR
Fake news detection is important but challenging because existing approaches often focus on only text, response, or source. CSI combines these characteristics through Capture, Score, and Integrate, and experiments report higher accuracy than existing models along with meaningful user and article representations.
Problem
Fake news detection is important and difficult because existing work has largely specialized in one of three characteristics: article text, user response, or source behavior.
Method
CSI uses Capture for temporal response and text, Score for user source behavior, and Integrate to classify articles using the combined information.
Results
Experiments on two real-world datasets demonstrate accurate fake-news classification and meaningful latent representations of both users and articles.
Takeaways & Limitations
CSI provides article classifications, user suspiciousness scores, and representations that support separate analysis of users and articles.
Takeaways & Limitations
Fake news detection remains challenging, and the authors identify incorporating reinforcement learning and crowdsourcing as an open direction for more accurate and timely predictions.
Abstract
from arXiv · showhide
The topic of fake news has drawn attention both from the public and the academic communities. Such misinformation has the potential of affecting public opinion, providing an opportunity for malicious parties to manipulate the outcomes of public events such as elections. Because such high stakes are at play, automatically detecting fake news is an important, yet challenging problem that is not yet well understood. Nevertheless, there are three generally agreed upon characteristics of fake news: the text of an article, the user response it receives, and the source users promoting it. Existing work has largely focused on tailoring solutions to one particular characteristic which has limited their success and generality. In this work, we propose a model that combines all three characteristics for a more accurate and automated prediction. Specifically, we incorporate the behavior of both parties, users and articles, and the group behavior of users who propagate fake news. Motivated by the three characteristics, we propose a model called CSI which is composed of three modules: Capture, Score, and Integrate. The first module is based on the response and text; it uses a Recurrent Neural Network to capture the temporal pattern of user activity on a given article. The second module learns the source characteristic based on the behavior of users, and the two are integrated with the third module to classify an article as fake or not. Experimental analysis on real-world data demonstrates that CSI achieves higher accuracy than existing models, and extracts meaningful latent representations of both users and articles.
1 INTRODUCTION
Fake news detection is important yet difficult because text, user response, and source each provide incomplete signals. CSI combines all three through Capture, Score, and Integrate to classify articles and characterize users.
- Fake news can manipulate public perception, while humans and fact-checking systems face difficulty or delays in distinguishing false stories.One study found respondents judged fake articles somewhat or very accurate 75% of the time.
- Text, response, and source are generally recognized as three characteristics relevant to fake news detection.Prior approaches often focus on only one characteristic, whose ambiguities limit automated detection.
- CSI uses Capture to model temporal user activity and text, Score to represent users and their group-promotion propensity, and Integrate to classify articles.Capture is implemented with an LSTM, while Score uses a neural network and implicit user graph.
- CSI explicitly identifies misinformation on both article and user sides while avoiding manual feature selection, domain assumptions, and distributional assumptions.Its features capture temporal behavior and textual content in a context-general way.
- Experiments on two real-world datasets show significantly higher classification accuracy than existing models and meaningful latent representations of users and articles.The model also rates users by group participation and requires fewer parameters and training than previous work.
2 RELATED WORK
Related work studies fake news through linguistic text cues, response propagation, and source or group behavior. CSI differs by modeling all three without hand-crafted features, domain knowledge, or distributional assumptions.
- Text-based methods use linguistic cues and traditional classifiers, but indicators vary across topics and platforms and hand-crafted features are laborious and dataset-dependent.Examples include pronouns, conjunctions, emotional words, swear words, and personal pronouns.
- Response-based approaches analyze temporal user reactions or propagation on social graphs, but graph access can be infeasible and many methods use hand-crafted social-network features.Temporal response patterns can help characterize article content.
- Source-oriented work identifies epidemic sources, isolates bots, or detects anomalous user groups using graph, behavioral, mixture-model, kernel, or Bayesian methods.Some approaches assume groups are known in advance, whereas later methods infer anomalous groups.
- CSI combines text, response, and source characteristics while producing article classifications, user scores, and representations for separate analysis.The model is presented as more general than approaches relying on hand-crafted features, domain knowledge, or distributional assumptions.
3 PROBLEM
The problem is defined over temporal user–article engagements that include relayed text. CSI aims to classify each article while assigning suspiciousness scores to users without assuming a behavior distribution or engagement context.
- Each engagement links a user, an article, and a time, and contains textual information relayed by the user.Examples include tweets or Facebook activity.
- The task jointly produces an article label and a suspiciousness score for every user.The article label indicates whether the article is fake, while the user score identifies suspicious users or groups.
- The model captures text, response, and source characteristics together while modeling temporal behavior of both users and articles.This formulation treats the activity as a temporal sequence of engagements.
- CSI makes no assumptions about the distribution of user behavior or the context of engagement activity.
4 MODEL
CSI combines temporal article-response patterns, textual information, and user-source behavior through Capture, Score, and Integrate modules. The model jointly represents articles and users, then combines these representations for article classification and broader analysis.
- Model overview: CSI has three parts: Capture extracts temporal article representations, Score represents and scores users, and Integrate combines their outputs for classification.Capture addresses response and text, while Score addresses source behavior.
- Capture: Capture uses an RNN, specifically an LSTM, to encode engagement timing, frequency, distribution, and associated text into article representations.Temporal engagements are vectorized, embedded, and processed sequentially; partitioning can aggregate large sequences before cell input.
- Capture: Capture’s article vectors summarize response and textual characteristics and can support article classification or stand-alone article analysis.The vectors are passed to Integrate for classification and can also be analyzed independently.
- Score: Score applies a fully connected transformation to user features, learns user representations, and produces scalar scores that compactly capture source-related behavior.User features can be derived from a weighted co-engagement graph, with SVD used to obtain lower-dimensional features and group behavior represented through dense graph blocks.
- Integrate: Integrate averages user scores for an article, concatenates the result with its Capture representation, and predicts the article label through a final fully connected layer.The Capture, Score, and Integrate parameters are jointly trained by back-propagation.
- Generality: CSI is presented as generalizable beyond fake news: Capture models temporal target engagements, Score identifies actor group participation, and Integrate combines them for target prediction.The framework can also incorporate additional domain knowledge.
5 EXPERIMENTS
Experiments on two real-world datasets evaluate CSI’s classification accuracy, model efficiency, and learned user and article representations. The results show strong classification performance, useful latent representations, robustness with limited labels, and interpretable suspicious-user behavior.
- Classification accuracy: CSI outperforms other models in both accuracy and F-score on the evaluated datasets.The experiments use 80% training data, 5% parameter tuning, 15% testing, and 5-fold cross-validation.
- Classification accuracy: More than 1% improvement in both accuracy and F-score over CI shows that engagement frequency and distribution provide useful signals beyond linguistic features.The comparison attributes the improvement to temporal information in user engagements.
- Classification accuracy: Up to 4.3% higher overall performance than GRU-2 makes CSI the best-performing comparison model and version.The results associate this improvement with integrating user features and the text, response, and source characteristics.
- Model complexity: CSI uses approximately 52K parameters, compared with 621K for GRU-2, because Capture uses compact features and Score provides condensed user representations.This gives CSI a substantially smaller parameter count than the compared stacked recurrent model.
- Model complexity: With only 10% of training samples, CSI achieves performance comparable to GRU-2, indicating usability with fewer labeled examples.The paper presents this result as evidence that CSI is lighter and easier to train when labeled data are limited.
- Interpreting user representations: Score outputs user scores and representations that correlate positively with fake-news engagement and capture suspicious group behavior.The analysis uses the fraction of fake-news engagements as a proxy because the datasets lack ground-truth user labels.
- Interpreting user representations: User representations correlate with engagement differences at 0.631 on Twitter and 0.867 on Weibo, with both correlations statistically significant at the 1% level.Projection results also show separation between users with extreme fake-news engagement fractions.
- Temporal user behavior: The most suspicious users are among the first to promote fake content, while suspicious users often show bursts of rapid engagements with an article.These temporal patterns differ more clearly from less suspicious users for fake news than for true news.
6 CONCLUSION
CSI combines text, user response, and source behavior through Capture, Score, and Integrate to predict fake news while representing users and articles. The authors report improved classification and identify human-in-the-loop learning as a direction for more accurate and timely predictions.
- CSI combines text, response, and source characteristics through Capture, Score, and Integrate to produce article predictions and user suspiciousness scores.Capture models temporal user–article activity and text, Score estimates source suspiciousness, and Integrate combines both modules.
- The modular design produces separate predictions for users and articles while combining their information for classification.The model also generates latent representations of both users and articles for separate analysis.
- CSI does not assume a particular distribution of user behavior, textual context, or underlying data structure.The model can incorporate richer information, such as user profile data, and capture temporal engagement evolution from users and articles.
- The work demonstrates the value of modeling the three intuitive characteristics of fake news together.
- Human participation through reinforcement learning and crowdsourcing is proposed as a direction for more accurate and timely predictions.The authors note that fake news detection remains challenging and has open questions despite encouraging results.