Source-linked AI summary
Interpretable Rumor Detection in Microblogs by Attending to User Interactions
Ling Min Serena Khoo, Hai Leong Chieu, Zhong Qian, Jing Jiang
TL;DR
The paper addresses rumor detection when replies may concern an entire thread rather than a specific parent-child link. It introduces attention-based models for broader tweet interactions and reports stronger performance than prior state-of-the-art models, while also exposing token- and post-level explanations.
Problem
Tree-based rumor detectors may miss interactions across conversation branches because social-media replies can address the broader thread rather than only a specific parent.
Method
The paper flattens conversation trees chronologically and applies transformer self-attention, including structure-aware and hierarchical token/post-level variants.
Results
The proposed models outperform current state-of-the-art models across PHEME, Twitter15, and Twitter16, with StA-PLAN best on Twitter15 and StA-HiTPLAN best on PHEME.
Takeaways & Limitations
Attention weights support possible explanations of rumor predictions at both token and post levels.
Takeaways & Limitations
The usefulness of structural information remains unclear across the evaluated data sets, and the authors leave further investigation to future work.
Abstract
from arXiv · showhide
We address rumor detection by learning to differentiate between the community's response to real and fake claims in microblogs. Existing state-of-the-art models are based on tree models that model conversational trees. However, in social media, a user posting a reply might be replying to the entire thread rather than to a specific user. We propose a post-level attention model (PLAN) to model long distance interactions between tweets with the multi-head attention mechanism in a transformer network. We investigated variants of this model: (1) a structure aware self-attention model (StA-PLAN) that incorporates tree structure information in the transformer network, and (2) a hierarchical token and post-level attention model (StA-HiTPLAN) that learns a sentence representation with token-level self-attention. To the best of our knowledge, we are the first to evaluate our models on two rumor detection data sets: the PHEME data set as well as the Twitter15 and Twitter16 data sets. We show that our best models outperform current state-of-the-art models for both data sets. Moreover, the attention mechanism allows us to explain rumor detection predictions at both token-level and post-level.
1 Introduction
The paper motivates rumor detection from community responses and identifies a limitation of tree models: replies may contain information relevant across conversation branches. It therefore proposes attention-based models that capture broader tweet interactions and provide explanations.
- Motivation: $130 billion in stock value was wiped out after a false tweet claimed Barack Obama was injured, illustrating the potential impact of fake news.The paper cites this event as motivation for effective automated detection.
- Motivation: Community-response approaches detect fake claims by applying natural language processing to comments directed toward the claim.These methods exploit opinions, conjectures, and evidence shared by social-media users.
- Existing approaches: Tree models represent source and responding tweets as nodes and recursively propagate information between parents and children.Their intended structure is a propagation tree, modeled with recursive neural networks.
- Research gap: Tree models do not explicitly model interactions between tweets on different branches, although replies may address the broader thread and provide evidence relevant to other nodes.The paper illustrates this with replies that debunk a source claim while appearing under separate branches.
- Proposed models: PLAN flattens the tree into chronological order and uses post-level self-attention for pairwise tweet interactions; StA-PLAN adds structure awareness, while StA-HiTPLAN adds token-level attention.The models aim to combine broader interaction modeling with structural information and richer tweet representations.
- Contributions: The paper reports token-level and post-level explanations, evaluations on PHEME, Twitter15, and Twitter16, and performance exceeding current state-of-the-art models across both data sets.These are presented as the paper’s main contributions.
2 Related Work
Related work approaches fake-news detection through claim content, source and social-network information, fact checking, or community responses. This paper belongs to the community-response line but replaces recursive tree models with a transformer network.
- Research directions: Prior fake-news detection uses claim content, source and social-network characteristics, fact checking, and community response.The paper organizes existing approaches into these four feature or evidence classes.
- Content information: Linguistic deception detection studies features such as pronoun percentages, word length, verb quantity, and word classes, but these cues may be domain- or topic-specific.The cited work covers fake reviews, witness accounts, satire, and fake news.
- Source and social network: Source-based work reports improved fake-news classification accuracy when source information is added to content, while fake-news accounts can exhibit different social-network characteristics.These findings motivate using source and network information as detection features.
- Fact checking: Fact checking verifies claims against trusted sources, but automated methods require an established corpus of facts and may not work for new claims with little evidence.The paper contrasts this principled approach with the rate and coverage challenges of manual verification.
- Community response: Community-response methods classify claim veracity from comments, replies, propagation patterns, and stance-aware features.Prior work includes classifiers and multi-task learning approaches using social-media responses.
- Closest related work: Earlier rumor-detection systems modeled posts sequentially or with recurrent, propagation-tree, recursive-tree, and LSTM architectures.The paper positions its transformer approach as an alternative to these recursive tree models.
3 Rumor Detection
The paper formulates rumor detection as predicting a claim’s rumor class from a thread’s source tweet, responding tweets, and their interactions. Its models allow broader post relationships than parent-child tree connections alone.
- Problem formulation: Rumor detection predicts a claim’s veracity from all responding tweets and the interactions between those tweets.The task is defined over a thread representation and its relation structure.
- Thread representation: A thread is ordered chronologically as X = {x1, x2, x3, ..., xn}, with x1 as the source tweet and xi as the ith tweet.Here, n denotes the number of tweets in the thread.
- Interaction structure: The proposed models let any post attend to any other post in the same thread, unlike tree models that relate posts only through replying parent-child pairs.This design targets interactions that cross conversation branches.
- Interaction structure: Structure-aware models label tweet-pair relations as parent, child, before, after, or self.The relation label R(i, j) is assigned to each pair of tweets xi and xj.
- Prediction target: Twitter15 and Twitter16 use four classes—non-rumor, false-rumor, true-rumor, and unverified—whereas PHEME uses three classes without non-rumor.The experiments cover these two Twitter data sets and the PHEME 5 data set.
4 Approaches
The paper replaces recursive tree-based rumor models with attention architectures that capture interactions among tweets, while optionally preserving conversation structure and learning token-level representations. It also investigates time-delay embeddings to account for how tweet timing affects interpretation.
- Motivation: Tree models may be suboptimal because shallow social-media threads do not explicitly model interactions between tweets in different branches.Users can observe the full thread, so a reply to one tweet may continue a broader conversation.
- Transformer Networks: Transformer attention models long-range dependencies by computing attention-weighted representations across positions.The output at each position is a weighted combination of information from other positions, with higher attention weights indicating greater compatibility.
- Post-Level Attention Network (PLAN): PLAN flattens the conversation tree chronologically, applies max pooling to each tweet, and uses post-level multi-head attention before prediction.The resulting tweet representations are transformed through multiple attention layers and then interpolated for a fully connected classifier.
- Structure Aware Post-Level Attention Network (StA-PLAN): StA-PLAN incorporates five structural relations—parent, child, before, after, and self—into attention compatibility and output propagation.Learned structural vectors supplement textual content when determining compatibility and propagating information.
- Structure Aware Hierarchical Token and Post-Level Attention Network (StA-HiTPLAN): StA-HiTPLAN replaces max pooling with token-level self-attention followed by post-level attention to learn more complex tweet representations.The model first enables interactions among tokens within each tweet, then uses attention to form the sentence representation.
- Time Delay Embedding: The models also encode tweet latency using 100 time bins of 10 minutes, with tweets beyond 1,000 minutes assigned to the final bin.The resulting time-delay embedding is added to each tweet’s sentence embedding.
5 Experiments and Results
The experiments evaluate PLAN variants on Twitter15, Twitter16, and PHEME, finding strong overall performance gains but dataset-dependent effects from structural and token-level attention. The analysis also examines preprocessing, dataset differences, and attention-based explanations.
- Experimental setup: The models were evaluated on Twitter15, Twitter16, and the PHEME 5-events dataset, using event-wise cross-validation for PHEME.Twitter15 and Twitter16 were preprocessed by removing retweets; claims without replies were excluded from testing.
- Overall results: 14.2% and 6.8% were the best-model improvements over prior work on Twitter15 and Twitter16, respectively, while PHEME improved by 1.6% F1-score.All proposed models outperformed tree recursive neural networks on Twitter15 and Twitter16.
- Dataset disparity: PHEME’s event-level split creates an out-of-domain classification setting because training and test events do not overlap.The authors identify event-specific features as a possible explanation for poorer test performance and plan to explore event-agnostic methods.
- Structural and temporal information: Structural attention was strongest on Twitter15, whereas it did not outperform PLAN on Twitter16 and its usefulness across datasets remained unclear.The authors suggest the larger Twitter15 dataset may be necessary to exploit complicated structural information.
- Token-level attention: StA-HiTPLAN performed best on PHEME but did not outperform the baseline on Twitter15 or Twitter16.The authors associate this pattern with weaker or more implicit rumor signals in PHEME and the prevalence of neutral replies.
- Interpretability: Attention weights provide token-level and tweet-level explanations, highlighting influential tweets and phrases such as “facts to write stories it appears” for a PHEME fake-claim prediction.The model uses post-level attention to weight tweet relatedness and token-level attention to identify important words within a tweet.
6 Conclusion
The paper proposes three attention-based models that outperform state-of-the-art models across three datasets while using attention to identify posts contributing to predictions. It also examines structural and temporal information, and limits its scope to data containing community responses.
- The three proposed models outperformed state-of-the-art models on three datasets.
- Self-attention models pairwise interactions between posts, while additional mechanisms capture structural and time information.
- Attention weights provide possible explanations by identifying important posts behind predictions.
- The study focuses only on data with community response.