Source-linked AI summary
3HAN: A Deep Neural Network for Fake News Detection
Sneha Singhania, Nigel Fernandez, Shrisha Rao
TL;DR
Fake news can manipulate public opinion, motivating automated detection that accounts for article structure and informative content. The paper introduces 3HAN, a three-level hierarchical attention network that forms a news vector from words, sentences, and headline-body interactions. On a large real-world dataset, pre-trained 3HAN achieves 96.77% accuracy and exposes attention weights for human fact-checking.
Problem
Fake news may manipulate public opinion, while concatenating headlines with article bodies fails to exploit article structure.
Method
3HAN hierarchically encodes words, sentences, and headline-body interactions with attention, forming a news vector for classification and using headline-only supervised pre-training.
Results
96.77% accuracy is achieved by the pre-trained 3HAN model on a large real-world news dataset.
Takeaways & Limitations
Attention weights can be visualized to identify important words and sentences for further human fact-checking.
Takeaways & Limitations
Because manually fact-checked articles were too few for deep neural training, the dataset assumes every article from a website shares that website’s fake-or-genuine label.
Abstract
from arXiv · showhide
The rapid spread of fake news is a serious problem calling for AI solutions. We employ a deep learning based automated detector through a three level hierarchical attention network (3HAN) for fast, accurate detection of fake news. 3HAN has three levels, one each for words, sentences, and the headline, and constructs a news vector: an effective representation of an input news article, by processing an article in an hierarchical bottom-up manner. The headline is known to be a distinguishing feature of fake news, and furthermore, relatively few words and sentences in an article are more important than the rest. 3HAN gives a differential importance to parts of an article, on account of its three layers of attention. By experiments on a large real-world data set, we observe the effectiveness of 3HAN with an accuracy of 96.77%. Unlike some other deep learning models, 3HAN provides an understandable output through the attention weights given to different parts of an article, which can be visualized through a heatmap to enable further manual fact checking.
1 Introduction
3HAN addresses fake-news detection with a three-level hierarchical attention network that represents article structure and highlights informative content for classification and fact-checking. On a large real-world dataset, the pre-trained model reaches 96.77% accuracy.
- Approach: 3HAN constructs a news vector by hierarchically modeling words, sentences, the body, and the headline.Its attention mechanism assigns importance scores to words and sentences based on their relevance to the output probability.
- Motivation: Existing general-purpose classifiers can concatenate headlines with article bodies but do not exploit the article’s hierarchical structure.The paper positions 3HAN as a neural approach that models interactions among words, sentences, and the headline.
- Approach: The model adds a headline-specific level because headlines are distinctive, summarize the body, and convey stance toward it.This design operationalizes the paper’s headline premise within the hierarchy.
- Interpretability: Headline-only supervised pre-training initializes the first layers of 3HAN, while attention visualizations identify article parts for human investigation.The resulting heatmaps support targeted manual fact-checking of important words and sentences.
- Results: 96.77% accuracy is achieved by the pre-trained 3HAN model on a large real-world news dataset.The unpretrained 3HAN model achieves 96.24% accuracy and both are compared with traditional and neural baselines.
2 Model Design
3HAN encodes an article bottom-up through word, sentence, and headline-body levels, using bidirectional GRUs and attention to form a news vector for binary classification. Headline-only supervised pre-training provides a better initialization for the model.
- Architecture: 3HAN represents an article with three hierarchical components: word encoding and attention, sentence encoding and attention, and headline-body encoding and attention.The first two levels model the body hierarchy; the third incorporates the headline premise.
- Word Encoding: Bidirectional GRUs encode contextual information by processing word sequences in forward and backward directions.Word embeddings are used as inputs, and the resulting annotations summarize each word’s surrounding context.
- Word Attention: Word attention weights relevant word annotations to form sentence representations, using a learned relevance vector and softmax normalization.The sentence encoding is an attentive weighted sum of word annotations.
- Sentence Encoding and Attention: Sentence encoding applies a bidirectional GRU to sentence representations so annotations capture coherence with neighboring sentences.Sentence attention then identifies relevant sentences when forming the body vector.
- Headline Encoding: The headline encoder processes headline words together with the body vector to model headline stance relative to the body.A third bidirectional GRU and headline-body attention produce the headline-level annotations.
- Classification: The resulting news vector is passed to a sigmoid classifier trained with binary cross-entropy using fake or genuine article labels.The classifier outputs the predicted probability for the article class.
- Pre-training: Headline-only supervised pre-training initializes the word encoder and attention layer using each headline’s corresponding article label.The stated purpose is better initialization of the full model.
3 Experiments
The experiments evaluate 3HAN against traditional and neural baselines using website-level labels, standardized preprocessing, and separate data splits. Hierarchical attention and headline pre-training produce the strongest reported performance.
- Dataset: Website-level labels address the scarcity of manually fact-checked articles by assigning each article its containing website’s fake or genuine label.The dataset uses fake sites identified by PolitiFact and genuine sites compiled by Forbes.
- Baselines: The evaluation compares 3HAN with frequency-based classifiers, SVM variants, recurrent models, and hierarchical neural alternatives.The compared models include bag-of-words, bag-of-ngrams, GRU-based representations, HAN variants, and 3HAN variants.
- Experimental setup: The preprocessing pipeline tokenizes and cleans text, retains vocabulary words appearing more than five times, and initializes fine-tuned embeddings with 100-dimensional GloVe vectors.Sentences and headlines are padded or truncated to average lengths, and articles to an average sentence count.
- Experimental setup: Neural models use a 20% train, 10% validation, and 70% test split, while word-count models use 30% training and 70% test data.The split differs by model family.
- Results and analysis: Hierarchical models outperform other baselines, attention outperforms average and max pooling, and 3HAN+PT outperforms 3HAN.The results also support a separate headline level because 3HAN performs better than HAN, while pre-training improves initialization.
4 Discussion and Insights
3HAN combines hierarchical attention over words, sentences, and headlines with visual explanations of which article parts influence fake-news classification. Experiments indicate that attention, headline-specific modeling, vocabulary patterns, and article structure all contribute useful signals.
- Attention and interpretability: 3HAN assigns attention weights to words, sentences, and headlines, enabling heatmap-based inspection of influential article content.Human fact-checkers can prioritize sentences and words with high attention weights for verification.
- Attention and interpretability: Sentence 5 received the highest attention weight, 0.287, and was identified as factually incorrect.The example sentence concerned a purported 12 percent penalty on immigrant money in Canada.
- Observed signals: Word-count models perform well, indicating that vocabulary and usage patterns distinguish fake news from true news.These models do not account for word ordering or semantics.
- Observed signals: Experiments infer inverted-pyramid writing from a small accuracy improvement when padding sentence counts increase.The paper also notes that fake-news articles tend to repeat information.
5 Conclusion and Future Work
The paper presents 3HAN as a news-vector model for fake-news detection and reports superior accuracy over other state-of-the-art models. It also highlights attention visualization and proposes a future web service that learns from newly fact-checked articles.
- Conclusion: 3HAN creates a news vector that represents an article for fake-news detection.The conclusion describes the news vector as an effective article representation.
- Conclusion: 3HAN achieves superior accuracy to other state-of-the-art models.
- Conclusion: Attention-layer visualization is highlighted as part of the model's contribution.
- Future work: The authors plan a web application that offers fake-news detection as a service and learns online from newly fact-checked articles.