Source-linked AI summary
Detecting Fake News with Capsule Neural Networks
Mohammad Hadi Goldani, Saeedeh Momtazi, Reza Safabakhsh
TL;DR
Fake news detection is challenging because misinformation can cause societal problems. This paper applies capsule neural networks with length-specific embeddings and architectures, reporting higher accuracy than state-of-the-art methods on ISOT and LIAR datasets.
Problem
Fake news detection is challenging because misinformation can cause many problems in society.
Method
The paper proposes capsule neural-network architectures using different embedding models and strategies for news statements of different lengths.
Results
The proposed architectures outperform state-of-the-art methods by 7.8% on ISOT, 3.1% on LIAR validation, and 1% on LIAR test accuracy.
Takeaways & Limitations
Capsule neural networks show encouraging performance for fake news detection across the evaluated ISOT and LIAR datasets.
Takeaways & Limitations
Classification is weakest for the pants-fire label and for distinguishing true from mostly-true and barely-true from false.
Abstract
from arXiv · showhide
Fake news is dramatically increased in social media in recent years. This has prompted the need for effective fake news detection algorithms. Capsule neural networks have been successful in computer vision and are receiving attention for use in Natural Language Processing (NLP). This paper aims to use capsule neural networks in the fake news detection task. We use different embedding models for news items of different lengths. Static word embedding is used for short news items, whereas non-static word embeddings that allow incremental up-training and updating in the training phase are used for medium length or large news statements. Moreover, we apply different levels of n-grams for feature extraction. Our proposed architectures are evaluated on two recent well-known datasets in the field, namely ISOT and LIAR. The results show encouraging performance, outperforming the state-of-the-art methods by 7.8% on ISOT and 3.1% on the validation set, and 1% on the test set of the LIAR dataset.
1. Introduction
Fake news spreads through social media in varied forms and can cause societal problems, making automatic detection challenging. The paper proposes capsule-neural-network architectures tailored to news length and reports better results than state-of-the-art methods.
- Social media enables manipulated information motivated by political, economic, or social interests, producing fake advertisements, false statements, satires, and rumors.
- Fake news can cause many problems in society and is widespread enough to motivate effective automatic detection frameworks.
- Detecting fake news is challenging because models must predict intentionally misleading content from previously reviewed real and fake news.
- High-quality, large-size training data is important, while detection may involve binary or fine-grained classification across datasets such as ISOT and LIAR.
- The paper proposes capsule-neural-network models using different word embeddings and n-gram feature-extraction levels for news statements of different lengths.
- The proposed models achieve better results than state-of-the-art methods.
2. Related work
Prior fake-news research uses linguistic, network, metadata, user-interaction, and multimodal signals. Studies report benefits from combining text with social context, user information, speaker profiles, or event-invariant features.
- CSI combines text, response, and source features through Capture, Score, and Integrate modules for fake-news classification.
- User interactions and network information are used as features in logistic-regression, crowdsourcing, and other fake-news classification approaches.
- Linguistic features at lexical, syntactic, and semantic levels can distinguish fake from genuine news, with reported performance comparable to humans.
- LIAR contains 12,836 labeled short statements from contexts including Facebook posts, tweets, and political debates.
- Models improve classification by combining text with metadata, including user behavior or speaker profiles such as party affiliation and location.
- Other approaches include ISOT classification with TF-IDF and linear SVM, event-adversarial multimodal detection, and Bayesian selection to reduce exposure to fake news.
3. Capsule networks for fake news detection
The paper applies capsule neural networks to fake-news detection, using word-embedding variants and architectures adapted to statement length. The design extracts features at multiple n-gram levels and processes them through parallel capsule networks.
- The paper proposes two capsule-neural-network models for fake-news detection according to news-statement length.
- Word embedding models: Static word2vec keeps pretrained vectors fixed, whereas non-static word2vec fine-tunes them during training for the target task.
- The proposed approach targets improved fake-news identification by applying capsule networks to text classification.
- Capsule networks: Capsule networks preserve detailed entity information through vector-valued capsules and routing between capsule layers.
- Proposed architectures: For short statements, the static model uses two parallel networks with filter sizes 3 and 5 because word sequences are limited.
4. Evaluation
The evaluation uses the ISOT and LIAR datasets, which provide different news lengths and metadata profiles for fake-news classification. Experiments use classification accuracy as the evaluation metric and are implemented with Keras.
- Datasets: The study evaluates fake-news detection on the ISOT and LIAR datasets.ISOT contains medium or long news statements, whereas LIAR contains short statements and metadata about speakers and claims.
- ISOT dataset: ISOT contains articles collected from Reuters.com and Kaggle.com, with each article longer than 200 characters and labeled fake or real.Available metadata include article type, text, title, date, and label.
- LIAR dataset: LIAR contains 12.8K human-labeled short statements evaluated by PolitiFact.com editors using six truthfulness labels.The labels range from pants-fire and false to mostly-true and true.
- Evaluation metric: Classification accuracy is the evaluation metric, defined as the ratio of correct predictions to the total number of samples.The reported formula uses true positives, true negatives, false positives, and false negatives.
5. Results
The proposed capsule models were evaluated on ISOT and LIAR against baseline methods, with non-static embeddings strongest for ISOT and speaker-history metadata strongest for LIAR. The results also analyze wrong predictions and show that word-frequency patterns affect label prediction, while several LIAR labels remain difficult to distinguish.
- 5. Results: The experiments evaluated the proposed models on two datasets and compared their results with baseline methods.The datasets were analyzed separately, including ISOT and LIAR.
- 5.1. Classification for ISOT dataset: Non-static word embeddings achieved the best result for the medium- and long-length news statements in ISOT.The embedding comparison was performed on ISOT, which contains medium- and long-length news statements.
- 5.1. Classification for ISOT dataset: 7.8% higher accuracy was achieved by the non-static capsule network than the best LSVM result on ISOT.The comparison used the non-static capsule network against methods including SVM, LSVM, KNN, DT, SGD, and logistic regression.
- 5.2. Discussion: Word-frequency patterns strongly affected predictions in the analyzed wrong samples.The analysis compared normalized word frequencies in training data with real and fake labels and examined two misclassified samples.
- 5.3.1. Discussion: The LIAR model classified false, half-true, and mostly-true news more accurately, but struggled with true versus mostly-true, barely-true versus false, and pants-fire labels.Many pants-fire texts were predicted as false.
6. Conclusion
The paper applies capsule networks to fake news detection using architectures tailored to news length, embedding type, and n-gram features. Experiments on ISOT and LIAR report improved accuracy over prior methods.
- The proposed system applies capsule networks to fake news detection with two architectures for different news-statement lengths.
- For medium or long texts, four parallel capsule networks extract 2-, 3-, 4-, and 5-gram features.
- Medium- and long-text processing uses non-static embeddings that are incrementally up-trained and updated during training.
- Short-text detection uses two parallel networks with filter sizes 3 and 5 and a static embedding model.
- Accuracy improved by 7.8% on ISOT, 3.1% on the LIAR validation set, and 1% on the LIAR test set.