Source-linked AI summary
CED: Credible Early Detection of Social Media Rumors
Changhe Song, Cunchao Tu, Cheng Yang, Zhiyuan Liu, Maosong Sun
TL;DR
Rapid social-media rumor spread makes early, reliable detection important, while existing methods use all or fixed proportions of repost information. CED treats reposts as a sequence and identifies a microblog-specific Credible Detection Point for prediction. On three real-world datasets, it reduced prediction time by more than 85% while achieving better accuracy than state-of-the-art baselines.
Problem
Existing rumor-detection methods consider entire or fixed proportions of repost information, limiting early detection before rumors spread widely.
Method
CED treats reposts as a sequence and learns a microblog-specific Credible Detection Point for making a credible prediction.
Results
More than 85% reduction in prediction time was achieved on three real-world datasets, with better accuracy than all state-of-the-art baselines.
Takeaways & Limitations
CED supports reliable rumor detection using only repost information available up to an early, dynamically selected detection point.
Takeaways & Limitations
CED is not good at addressing conflicting repost sequences, which the authors identify as future work.
Abstract
from arXiv · showhide
Rumors spread dramatically fast through online social media services, and people are exploring methods to detect rumors automatically. Existing methods typically learn semantic representations of all reposts to a rumor candidate for prediction. However, it is crucial to efficiently detect rumors as early as possible before they cause severe social disruption, which has not been well addressed by previous works. In this paper, we present a novel early rumor detection model, Credible Early Detection (CED). By regarding all reposts to a rumor candidate as a sequence, the proposed model will seek an early point-in-time for making a credible prediction. We conduct experiments on three real-world datasets, and the results demonstrate that our proposed model can remarkably reduce the time span for prediction by more than 85%, with better accuracy performance than all state-of-the-art baselines.
1 INTRODUCTION
Rumors can spread rapidly on social media, creating risks before people or organizations can verify them. The introduction identifies the limits of existing detection methods and presents CED, which seeks a credible, microblog-specific early prediction point.
- Rumors can spread explosively before contradiction or detection, potentially causing economic loss or public panic during emergencies.
- Manual rumor reporting requires substantial effort and faces coverage and time-delay problems.
- Existing feature-engineering methods are biased, time-consuming, and difficult to generalize across rumor-detection scenarios.
- Existing neural models use entire or fixed proportions of repost information, limiting their ability to detect rumors as early as possible.
- CED learns a microblog-specific Credible Detection Point from repost sequences for reliable prediction using only information available up to that point.
- More than 85% reduction in detection time was achieved across experiments on three real-world datasets, with better accuracy than state-of-the-art baselines.
- The work also expands the Weibo rumors dataset to about twice its original size for further rumor-detection research.
2 RELATED WORK
Prior rumor-detection work includes manually engineered features, deep neural networks, and propagation-based methods, but early detection remains insufficiently addressed. CED combines neural feature learning with propagation-sequence modeling to target earlier rumor detection.
- Traditional methods manually design textual, user, temporal, structural, or linguistic features for rumor classification.
- Feature-based methods are biased, time-consuming, scenario-specific, and difficult to generalize across applications.
- Deep neural methods learn representations from repost sequences using recurrent or convolutional architectures.
- Neural models that combine text, user, statistical, or external information still typically use entire or fixed proportions of repost information, limiting early detection.
- Propagation-based studies identify differences between false and real information diffusion, but early propagation signals have not been established as sufficient for reliable rumor detection.
- CED applies deep neural networks to feature learning and is evaluated on real-world datasets for early rumor detection.
3 METHODOLOGY
CED represents reposts as a chronological sequence, extracts interval features, and models their evolution to identify when a credible prediction can be made. Its objective combines prediction, post-detection stability, and early-detection timing, with extensions for original-message and CNN features.
- 3.1 Problem Formalization: The task predicts whether each microblog is a rumor from its associated repost sequence, with rumors labeled 1 and non-rumors labeled 0.
- 3.2 Repost Sequence Conversion: Reposts are batched into consecutive units, using N = 10 to reduce sequence length while retaining fine time granularity.
- 3.3 Feature Representation: CED converts each repost interval into feature vectors and processes the resulting variable-length sequence with recurrent neural networks.
- 3.3.2 Convolutional Neural Network: CNN feature extraction applies convolution and ReLU to word-embedding matrices, then max-pooling selects the strongest feature for each filter.
- 3.5 Credible Early Detection: The Credible Detection Point marks the earliest time at which prediction reaches a threshold and later predictions are intended to remain stable.
- 3.5 Credible Early Detection: CED combines prediction likelihood, post-point prediction differences, and detection time through OCED = Opred + λ0 · Odiff + λ1 · Otime.
- 3.5.2 CED with Original Microblog: CED-OM incorporates original microblog information, while CED-CNN replaces TF-IDF with CNN-derived features for the original message and repost sequence.
4 EXPERIMENTS
The experiments evaluate CED on three real-world datasets, including Weibo and Twitter data, against representative text, CNN, GRU, and paragraph-vector baselines. The evaluation examines accuracy, early detection, and parameter sensitivity.
- CED is evaluated on three representative real-world datasets, with analyses of detection accuracy, early detection, and parameter sensitivity.
- The datasets include Weibo-stan and Twitter collections assembled from repost information, with some unavailable microblogs removed because posts or accounts were deleted.
- Weibo-all expands the Weibo data using known rumors and a similar number of non-rumor events collected from Sina sources.
- The baselines comprise CNN-OM, TF-IDF with SVM, a two-layer GRU, and CAMI.
4.3 Evaluation Metrics and Parameter Settings
The evaluation uses accuracy and Early Rate to measure detection correctness and how much repost information is needed, with specified data splits and model settings.
- Accuracy measures correctly predicted rumors and non-rumors, while macro precision, recall, and F-measure provide additional evaluation metrics.
- Early Rate measures the utilization ratio of repost information, with lower values indicating earlier detection using less information.The metric uses the first threshold-crossing prediction time relative to the total repost-sequence length.
- The datasets use a 10% validation split and divide the remainder into training and testing at a 3:1 ratio.
- CED evaluates prediction thresholds of 0.875 and 0.975, with λ0=0.01, λ1=0.2, and a 2-layer GRU sequence encoder.
- For fair comparison, hidden sizes are 200 for CED, GRU-2, and CAMI, and 100 for CED-OM and CED-CNN.The CNN variants use filter widths and 50 filters per size, producing a final hidden size of 200.
4.4 Results and Analysis
Across Weibo and Twitter experiments, the proposed CED variants improve accuracy while requiring substantially less repost information than baseline methods. The results also identify threshold and input representation choices that affect the early-detection trade-off.
- CED, CED-OM, and CED-CNN consistently outperform all baselines with higher accuracy and less repost-sequence information.
- Detection time decreases by around 86% in Weibo and 77% in Twitter while accuracy also improves over the baselines.
- CED-CNN achieves very high detection accuracy using only 13.2% of repost-sequence information.
- CED-OM improves Early Rate under both thresholds because original microblogs provide additional information when few reposts are available.
- CED-CNN achieves the best Early Rate across all three datasets, while CNN-based repost processing outperforms TF-IDF-based processing.
- A higher prediction threshold increases confidence but delays detection and produces a higher Early Rate.
- The experiments report that the model remains robust and flexible across datasets and parameter settings.
4.5 Early Detection.
The early-detection analysis examines when models reach a credible prediction and how performance changes with repost information. CED-CNN generally reaches credible detection using less repost information, while fixed-percentage evaluation is less suited to adaptive detection points.
- The Credible Detection Point is learned and inferred for each microblog using a threshold-based detection strategy.
- About 30% of microblogs are detected with less than 10% repost information, while CED-OM exceeds 40% and 60% below 10% and 20%, respectively.
- CED-CNN detects more than 60% of Weibo-all samples using 10% repost information, outperforming CED and CED-OM at that level.
- Less than 10% of CED-CNN cases require the whole repost sequence, indicating lower repost-information requirements for credible detection.
- CED-OM and CED-CNN perform better than the compared baselines when the fixed repost-information percentage is below 55%.
- Fixed-percentage evaluation is less suitable for CED because the model determines how much repost information each microblog needs.
4.6 Loss Comparison
The loss comparison isolates how Otime and Odiff contribute to CED-CNN's early, credible predictions. Otime advances detection without sacrificing accuracy, while Odiff supports both early detection and accuracy.
- Otime improves Early Rate without reducing Accuracy relative to the full CED-CNN objective.Removing Otime decreases Early Rate by 3% while producing almost the same Accuracy.
- Odiff is important for both Early Rate and Accuracy, as removing it significantly reduces both measures.The comparison uses O2 against CED-CNN with the complete objective.
- Otime is designed to advance the Credible Detection Point, while Odiff stabilizes post-point prediction probabilities beyond the relevant thresholds.These objective components jointly reflect the intended design of the full objective.
- The objective-function experiments validate introducing the Credible Detection Point into model training.The conclusion follows from comparisons of O1, O2, and the complete objective OCED.
4.7 Training process
During training, Accuracy increases toward stability while Early Rate decreases. CED stabilizes earlier, whereas CED-CNN can achieve higher accuracy and stronger early detection after sufficient training, especially on larger datasets.
- Accuracy gradually increases toward stability as training steps rise, while Early Rate decreases.This pattern is observed across the training-process analysis.
- CED reaches steady training-set accuracy at about 400 steps, while CED-CNN fluctuates before exceeding CED at about 700 steps.CED-CNN therefore requires more training steps but may achieve higher training-set accuracy.
- CED-CNN has better Early Rate than CED throughout Weibo-stan training, but initially performs worse on Weibo-all before about 300–400 steps.Its stronger early-detection performance on the larger dataset appears only after sufficient training.
- Validation accuracy is more stable for CED-CNN on Weibo-all, indicating that dataset size affects stable selection of the optimal model.The analysis compares validation performance for CED and CED-CNN across the two Weibo datasets.
4.8 Parameter Sensitivity.
Parameter sensitivity experiments examine α, λ0, and λ1 for CED and CED-CNN. Increasing α trades earlier detection for accuracy, while CED-CNN remains stable across α settings and both models remain stable across broad λ ranges.
- Increasing α improves prediction accuracy but lengthens detection time, requiring a scenario-dependent tradeoff between accuracy and Early Rate.At α = 0.975, the model performs better than baselines and reduces detection time by about 87%.
- CED-CNN maintains good results and stability across different α values, supporting reliable balance between Early Rate and Accuracy.The threshold directly determines the judgment of the early detection point.
- CED and CED-CNN remain stable when λ0 and λ1 range from 10^-5 to 10^-1.λ0 weights the difference loss, while λ1 weights prediction time in the objective.
- CED-CNN is reported as more stable than CED and flexible across parameter settings, supporting practical training.The overall sensitivity observations describe both models as robust, with especially strong stability for CED-CNN.
4.9 Case Study and Error Analysis
Case studies show that CED can predict correctly once repost signals become consistent, while conflicting, sparse, or irrelevant reposts make a credible detection point difficult to identify. Handling conflicting repost sequences remains a stated limitation.
- Case Study: CED can make a correct early prediction when repost information after the Credible Detection Point is consistent.The prediction probabilities become stable after the point, matching the model's design assumption.
- Error Analysis: Incorrect cases are associated with too few reposts, conflicting reposts, or reposts unrelated to the original microblog.The first and third conditions may be alleviated as relevant comments accumulate over time.
- Error Analysis: Conflicting repost sequences can prevent CED from finding a credible prediction point and making a credible prediction.The conflict causes early interval probabilities to vary substantially.
- Error Analysis: CED is not good at handling conflicting repost sequences, which the paper identifies as future work.This limitation follows from the difficulty of locating a credible prediction point in conflicting cases.
5 CONCLUSION AND FUTURE WORK
CED targets early social-media rumor detection by identifying a credible detection point and reducing interference from repost information. Future work proposes adding publisher profiles, propagation structure, and attention-based selection of critical reposts.
- CED makes credible predictions at a specific detection point while reducing interference from repost information.
- More than 85% reduction in prediction time was achieved with better accuracy on real-world datasets.
- Future Work: Future work will incorporate publisher profiles and propagation structure alongside repost information and original microblogs.This additional information is expected to address conflicts in repost sequences.
- Future Work: Future work will use attention mechanisms to distinguish critical signals or reposts and improve early rumor detection.