Source-linked AI summary
MDFEND: Multi-domain Fake News Detection
Qiong Nan, Juan Cao, Yongchun Zhu, Yanyan Wang, Jintao Li
TL;DR
Multi-domain fake-news detection is difficult because existing methods often rely on single-domain data despite domain-specific distributions and limited labels. The paper introduces the nine-domain Weibo21 benchmark and MDFEND, which uses a domain-gated mixture of experts; experiments report significantly improved effectiveness over baselines.
Problem
Existing fake-news detection approaches mainly target single domains, while MFND faces domain shifts in word usage and propagation patterns plus limited labeled data.
Method
The paper constructs the nine-domain Weibo21 benchmark and proposes MDFEND, which uses a domain gate to aggregate representations from a mixture of experts.
Results
MDFEND significantly improves effectiveness compared with the evaluated baselines in multi-domain fake-news detection.
Takeaways & Limitations
Weibo21 enables systematic evaluation of MFND methods, while MDFEND provides an effective specialized model for this setting.
Abstract
from arXiv · showhide
Fake news spread widely on social media in various domains, which lead to real-world threats in many aspects like politics, disasters, and finance. Most existing approaches focus on single-domain fake news detection (SFND), which leads to unsatisfying performance when these methods are applied to multi-domain fake news detection. As an emerging field, multi-domain fake news detection (MFND) is increasingly attracting attention. However, data distributions, such as word frequency and propagation patterns, vary from domain to domain, namely domain shift. Facing the challenge of serious domain shift, existing fake news detection techniques perform poorly for multi-domain scenarios. Therefore, it is demanding to design a specialized model for MFND. In this paper, we first design a benchmark of fake news dataset for MFND with domain label annotated, namely Weibo21, which consists of 4,488 fake news and 4,640 real news from 9 different domains. We further propose an effective Multi-domain Fake News Detection Model (MDFEND) by utilizing a domain gate to aggregate multiple representations extracted by a mixture of experts. The experiments show that MDFEND can significantly improve the performance of multi-domain fake news detection. Our dataset and code are available at https://github.com/kennqiang/MDFEND-Weibo21.
1 INTRODUCTION
The paper frames multi-domain fake news detection as necessary because existing single-domain approaches struggle with sparse data and severe domain shift. It introduces the Weibo21 benchmark and MDFEND, a mixture-of-experts model using a domain gate, with experiments showing improved effectiveness.
- Social media enables widespread fake-news dissemination, making fake-news detection a critical problem because it can harm individuals and society.
- Single-domain fake-news detectors perform unsatisfactorily when applied across domains because individual domains may contain extremely limited fake-news data.
- Domain shift makes MFND challenging because domains differ in word usage, propagation patterns, and the amount of labeled data available.
- Weibo21 contains 4,488 fake and 4,640 real news items from nine domains, with content, timestamps, pictures, and comments.
- MDFEND uses a domain gate to aggregate multiple representations extracted by a mixture of experts for multi-domain fake-news detection.
- The study constructs Weibo21, proposes MDFEND, and systematically evaluates multi-domain fake-news detection methods on the benchmark.
2 RELATED WORK
Prior fake-news datasets generally lack sufficient multi-domain information, while existing detection research uses diverse textual, structural, multimodal, and auxiliary-task approaches. The paper positions Weibo21 as a response to this dataset gap.
- Earlier fake-news detection studies used handcrafted features, propagation patterns, multimodal features, or related tasks as signals.
- Existing datasets such as LIAR, CoAID, FakeHealth, Twitter16, and Weibo lack multi-domain information, while FakeNewsNet covers only two domains.
- Table 1 presents the data statistics of the proposed Weibo21 dataset.
3 WEIBO21: A NEW DATASET FOR MFND
Weibo21 is a Chinese multi-domain fake-news dataset collected from Sina Weibo, combining content, images, timestamps, comments, and domain labels. Its construction includes source verification, deduplication, expert annotation, and domain-difference analysis.
- Data Collection: The dataset collects Chinese fake and real news from Sina Weibo posted between December 2014 and March 2021.
- Data Collection: Each news item includes content, pictures, timestamps, and comments, providing textual, visual, sequential, and social-context information.
- Data Collection: Official Weibo judgments identify fake news, while NewsVerify-verified items provide real news collected from the same period.
- Data Collection: Deduplication yields 4,488 fake-news items and 4,640 real-news items.
- Domain Annotation: Ten experts annotate news into nine domains, accepting labels when more than eight experts agree and discussing unresolved cases.
- Preliminary Data Analysis: Analysis of Weibo21 finds significant differences in frequently used words across domains, demonstrating domain variation in news content.
4 MDFEND: MULTI-DOMAIN FAKE NEWS DETECTION MODEL
MDFEND addresses multi-domain fake news detection with representations tailored to domain differences. It combines multiple expert representations through a domain gate before classification.
- MDFEND treats multi-domain fake news detection as a binary classification problem.
- Representation Extraction: BERT and Mask-Attention produce sentence embeddings, while a learnable domain embedding individualizes representation extraction for each domain.
- Representation Extraction: Multiple expert networks extract complementary news representations because a single expert may capture only partial content characteristics.
- Domain Gate: A domain gate uses domain and sentence embeddings to assign expert weight ratios, enabling adaptive selection rather than simple averaging.
- Prediction: The weighted final feature vector is fed to an MLP with a softmax output layer for fake news detection.
5 EXPERIMENT
The experiments compare single-domain, mixed-domain, and multi-domain baselines on Weibo21, finding that additional data and domain-aware learning generally improve MFND performance. MDFEND performs better than other multi-domain models by modeling domain relationships through gated, soft combination of representations.
- 5.1 Baseline Methods: The experiments compare single-domain, mixed-domain, and multi-domain baselines on the Weibo21 dataset.Single-domain models train separately per domain, mixed-domain models train across all domains, and multi-domain models combine domains according to their structures.
- 5.2 Experimental Settings: Experiments average f1-score across 10 runs, with shared settings used to compare methods.The reported evaluation uses the same parameters for all methods and averages results over repeated runs.
- 5.3 Results: Additional data improves performance overall, as mixed-domain and multi-domain models generally outperform single-domain models.The result supports the importance of using data across domains for MFND.
- 5.3 Results: Multi-domain models generally outperform mixed-domain models, indicating that multi-domain learning is useful and necessary for MFND.The comparison distinguishes domain-aware multi-domain methods from simply pooling all domains into one model.
- 5.3 Results: Single-domain models can outperform corresponding mixed-domain models on specific domains, such as Health for TextCNN.Simply combining data from different domains can produce negative effects from additional data.
- 5.3 Results: MDFEND performs better than other multi-domain models by feeding domain embedding and content to its gate and softly combining multiple domains.The authors contrast this design with roughly decoupling domain-shared and domain-specific features in EDDFN.
6 CONCLUSION
The paper studies multi-domain fake news detection by introducing Weibo21 and MDFEND, then systematically evaluating methods on the dataset. Experiments show the effectiveness of MDFEND for MFND.
- 6 CONCLUSION: Weibo21 is introduced as an MFND dataset collected from one platform with the richest domains, to the authors’ knowledge.The paper evaluates different methods on this dataset.
- 6 CONCLUSION: MDFEND uses a domain gate to aggregate multiple representations extracted by a mixture of experts.The method is proposed specifically for multi-domain fake news detection.
- 6 CONCLUSION: Experiments systematically evaluate MFND methods on Weibo21 and show MDFEND’s effectiveness.The conclusion summarizes the dataset, method, and evaluation as the paper’s main contributions.