Source-linked AI summary
Predicting, Evaluating, and Explaining Top Misinformation Spreaders via Archetypal User Behavior
Enrico Verdolotti, Luca Luceri, Silvia Giordano
TL;DR
Misinformation moderation is often reactive, creating a need to identify risky accounts before false content spreads widely. This paper formalizes three overlapping behavioral archetypes, develops ranking and machine-learning methods with temporal and explainable features, and finds that super-spreaders dominate the highest ranks while other archetypes become more prominent lower down.
Problem
Reactive moderation can occur after misinformation has circulated widely, motivating research on whether behavioral traits reveal an account’s dissemination risk.
Method
The paper formalizes amplifiers, super-spreaders, and coordinated accounts and develops archetype-specific rankings plus an integrated machine-learning framework.
Results
Super-spreader traits dominate the highest-ranked positions, while multiple archetypal traits increasingly co-occur at middle and lower ranks.
Takeaways & Limitations
The approach provides tools for earlier monitoring and prioritization of potentially harmful accounts, with explainable AI improving interpretability.
Takeaways & Limitations
The study’s network omits indirect exposure pathways, uses domain-level credibility labels, and applies ranking models only to users observed during training.
Abstract
from arXiv · showhide
The spread of misinformation on social networks poses a significant challenge to online communities and society at large. Not all users contribute equally to this phenomenon: a small number of highly effective individuals can exert outsized influence, amplifying false narratives and contributing to significant societal harm. This paper seeks to mitigate the spread of misinformation by enabling proactive interventions, identifying and ranking users according to key behavioral indicators associated with harmful content dissemination. We examine three user archetypes -- amplifiers, super-spreaders, and coordinated accounts -- each characterized by distinct behavioral patterns in the dissemination of misinformation. These are not mutually exclusive, and individual users may exhibit characteristics of multiple archetypes. We develop and evaluate several user ranking models, each aligned with a specific archetype, and find that super-spreader traits consistently dominate the top ranks among the most influential misinformation spreaders. As we move down the ranking, however, the interplay of multiple archetypes becomes more prominent. Additionally, we demonstrate the critical role of temporal dynamics in predictive performance, and introduce methods that reduce data requirements by minimizing the observation window needed for accurate forecasting. Finally, we demonstrate the utility and benefits of explainable AI (XAI) techniques, integrating multiple archetypal traits into a unified model to enhance interpretability and offer deeper insight into the key factors driving misinformation propagation. Our findings provide actionable tools for identifying potentially harmful users and guiding content moderation strategies, enabling platforms to monitor accounts of concern more effectively.
Predicting, Evaluating, and Explaining Top Misinformation Spreaders via Archetypal User Behavior
The paper formalizes behavioral archetypes and develops ranking and explainability methods for identifying misinformation spreaders.
- The study formalizes three user behavioral archetypes: amplifiers, super-spreaders, and coordinated accounts.
- It develops and compares archetype-specific and combined methods for predicting top misinformation spreaders.
- An explainable AI methodology reveals diverse behavioral traits associated with actors responsible for misinformation propagation.
1. Introduction
The introduction frames misinformation moderation as too reactive and motivates behavioral prediction for earlier intervention. The paper combines archetype-based ranking with machine learning and finds that super-spreaders dominate the highest ranks, while other traits become more prominent lower down.
- Reactive moderation often acts after misinformation has already circulated widely, motivating proactive identification of potentially harmful accounts.
- The paper develops archetype-specific ranking systems and a machine-learning framework integrating traits from all three archetypes.
- Super-spreader traits dominate the highest-ranked positions among users most responsible for misinformation dissemination.
- At middle and lower ranks, multiple archetypal traits frequently co-occur, with resharing behavior becoming increasingly prevalent.
- The contributions include formalized archetypes, the time-aware TASH-Index, and explainable machine-learning methods for interpreting misinformation propagation.
2. Related Work
Related work spans content-centered and user-centered misinformation research, including graph and temporal models. The paper addresses underexplored archetype roles, dynamic behavior, and limited explainability through an interpretable, multi-archetype framework.
- Content-centered approaches analyze linguistic, semantic, visual, or virality features but can overlook user behavior and network dynamics.
- User-centered research studies behavioral, temporal, network, exposure, ideological, linguistic, and psychological dimensions of misinformation.
- This work models diffusion through behavioral archetypes and combines time-aware influence metrics with interpretable machine learning and SHAP explanations.
- Existing methods include GNN-based detection, attention and explanation modules, temporal forecasting, architecture benchmarks, and coordination modeling.
- The field distinguishes identifying active spreaders, predicting future spreaders, and characterizing user roles and strategies.
- Remaining challenges include underexplored amplifiers, dynamic user roles, and limited XAI adoption in misinformation research.
3. Behavioral Archetypes: Definition and Characterization
The paper defines amplifiers, super-spreaders, and coordinated accounts as distinct but interrelated diffusion behaviors. These archetypes underpin ranking models that score users’ potential to propagate low-credibility content.
- The three archetypes are not mutually exclusive and can describe both human-operated and automated accounts.
- Amplifiers primarily spread information through extensive resharing rather than creating original content.
- Coordinated accounts collaborate to amplify shared messages and create an illusion of public consensus.
- Super-spreaders consistently create original content that achieves viral reach through extensive resharing by others.
- The archetypes support ranking models that assign accounts scores reflecting their potential to propagate low-credibility content.
4. Methodology
The methodology assigns archetype-specific scores to users based on misinformation activity, resharing behavior, influence, and coordination. It also incorporates content credibility and temporal dynamics to produce rankings that balance recent and historical behavior.
- Credibility assessment: The framework labels content credibility using NewsGuard domain scores and focuses on original posts and reshares, excluding self-reshares, replies, and quotes.The ranking models operate on assigned credibility regardless of content modality.
- Amplifiers: Amplifier rankings use total misinformation reshares and an Early Reposter Index that rewards users who reshare earlier in diffusion cascades.The Early Reposter Index gives greater prominence to users appearing early, especially on posts receiving many reshares.
- Coordinated accounts: Coordination rankings measure users’ positions in a co-reshare similarity network through eigenvector centrality or the maximum incident edge weight.These scores capture embeddedness among similar users or strong coordination with at least one other account.
- Super-spreaders: Super-spreader rankings use received reshares as an influence proxy through Influence Score and H-Index methods.The Influence Score sums reshares received by a user’s low-credibility posts, while the H-Index requires multiple posts to reach corresponding reshare counts.
- Temporal modeling: Fixed observation windows can make rankings depend on window length and posting frequency, whereas time-aware aggregation is designed to reduce short-term fluctuations.The H-Index is presented as a more stable alternative because increasing it requires consistent engagement over time.
- Temporal modeling: Time-aware methods partition observations into contiguous intervals and apply an exponential moving average, weighting recent behavior while retaining historical influence.The framework defines time-aware variants of existing methods, including a Time-Aware Influence Score using a parameter α to control long-term versus short-term influence.
5. Evaluation
The evaluation uses two COVID-19 Twitter datasets, a temporal train–test split, and low-credibility reshare networks to compare user-ranking strategies. Performance is measured by how effectively rankings identify users whose removal reduces misinformation diffusion, while accounting for important scope and estimation constraints.
- Credibility analysis: The credibility distributions compare NewsGuard-based scores for labeled retweets in VaccinItaly and COVID-19 Multilanguage.VaccinItaly shows peaks at very low and very high credibility, with the low-credibility concentration spread across scores 0–40.
- Data sources: The study evaluates rankings on VaccinItaly and COVID-19 Multilanguage, which differ in geographic focus, linguistic diversity, scale, and observation period.VaccinItaly contains 371,586 retweets from 51,962 users over 306 days; the Multilanguage dataset contains 1,118,697 reshares from 129,507 users over 372 days.
- Evaluation design: The temporal split assigns the first 80% of chronologically ordered resharing events to observation and the remaining 20% to evaluation.This prevents future information from entering model training.
- Network construction: Low-credibility reshare networks retain content below the NewsGuard-based credibility threshold of 39 and represent users as nodes connected by weighted reshare edges.Edge weights count distinct pieces of content reshared between users, rather than repeated shares of the same content.
- Network dismantling: Network dismantling iteratively removes users according to a ranking and measures each removal’s incident-edge weight as a fraction of total network weight.The procedure estimates the share of misinformation no longer circulating when selected users are removed.
- Evaluation metrics: Quality@k estimates misinformation removed by moderating the top-k users, while nDCG@k evaluates ranked-user relevance against unseen test-set dissemination.The optimal dismantling curve ranks users by evaluation-set node strength, combining incoming and outgoing reshare connections.
6. Results
The results show that machine learning and time-aware super-spreader ranking perform strongest for top-ranked users, while combining archetypal signals improves interpretability and supports differentiated moderation. Performance remains comparatively robust across datasets and data-availability conditions, although retrained machine learning degrades with limited data.
- Ranking Performance: ML and TASH achieve performance comparable to optimal ranking for top-ranked users and consistently outperform alternative strategies.Their rankings align with high node strength, which combines incoming and outgoing reshare activity.
- Ranking Performance: The ML model achieves the highest nDCG score, 0.904, while TASH-Index closely follows, especially at small and moderate intervention levels.ML and TASH-Index perform particularly well when only a few top-ranked users are removed.
- Ranking Performance: TAI-Score attains the highest Quality score, 83.9%, at k = 1000, making it particularly effective for broad-scale intervention.The results indicate different methods may be preferable for concentrated versus wide-scale moderation.
- Cross-Dataset Robustness: The COVID-19 Multilanguage results closely mirror VaccinItaly findings, reinforcing robustness across linguistic and geographic contexts.ML and TASH-Index again provide the strongest top-ranking signals.
- Archetypal Feature Contributions: Repost Count is the dominant SHAP predictor, with higher values associated with higher predicted rankings, followed by TAI-Score and TASH-Index.The model combines amplifier and super-spreader signals rather than relying on a single archetype.
- Archetypal Feature Contributions: The model relies mainly on TASH-Index and TAI-Score at ranks 1–10, while Repost Count and Coordination-Centrality become more influential at lower ranks.Lower-ranked users increasingly reflect amplifier and coordinated-account behaviors rather than only super-spreader traits.
- Moderation Implications: The findings support targeted interventions for super-spreader-like top-ranked users and broader systemic responses to coordinated or repetitive resharing.The proposed moderation strategies distinguish behavioral patterns across ranking levels.
- Data Scarcity Robustness: The pre-trained ML model remains stable across most data-reduction levels, whereas the retrained model fluctuates and degrades as less data becomes available.The pre-trained model begins degrading at the smallest 30-day window, likely because archetypal feature vectors become too sparse.
7. Conclusion
The study formalizes three behavioral archetypes and develops interpretable ranking methods for identifying misinformation spreaders. It argues that behavior-based rankings can support human-supervised moderation while noting important data, scope, and evaluation limitations.
- The framework formalizes super-spreaders, amplifiers, and coordinated accounts and ranks users by observable behavioral patterns rather than identity or intent.
- Combining archetypal features in an interpretable model can provide competitive predictive performance and actionable insight into misinformation dynamics.
- The TASH-Index captures temporal and structural diffusion patterns efficiently, performs competitively in low-data settings, and can improve machine-learning models as a feature.
- Model-generated rankings are best deployed as decision-support tools in human-in-the-loop moderation, supporting prioritization while preserving oversight and accountability.
- The study is limited by incomplete reshare cascades, domain-level credibility labels, training-user coverage, and static intervention metrics.
Appendix A.1. Time-Aware Methods Optimization
The time-aware methods require tuning the smoothing factor α and time-slot size δ. Grid search selects α = 0.5 and δ = 14 days for TASH-Index, versus α = 0.6 and δ = 18 days for TAI-index.
- α = 0.5 and δ = 14 days are selected for TASH-Index, while α = 0.6 and δ = 18 days are selected for TAI-index.The values are obtained through a grid search over predefined parameter ranges.
Appendix A.2. Ranking Amplifiers
The appendix compares ranking strategies for amplifiers, coordinated users, and super-spreaders across datasets and evaluation settings. Reshare frequency outperforms early reposting for amplifiers, centrality better captures coordination structure, and temporal methods improve super-spreader identification.
- Ranking Amplifiers: Reshare frequency outperforms early reposting for identifying amplifiers.The Early Reposter method performs worse than the simpler Repost Count method, challenging early resharing as a strong influence proxy.
- Coordinated Users: Coordination Centrality outperforms Coordination Max Weight, indicating that dense cluster embedding is more informative than a single strong similarity link.The methods use cosine similarity between users’ reshare patterns as edge weights.
- Super-spreaders: Including temporal dynamics improves both the TASH-Index and TAI-Score in super-spreader dismantling trajectories.
- Cross-dataset comparison: The ML model achieves the highest nDCG scores on the COVID-19 Multilanguage dataset, while ML and TASH-index outperform the other methods overall.The appendix reports that these findings closely mirror the VaccinItaly results.
Appendix B.1. Sensitivity to Credibility Threshold
Credibility-threshold sensitivity analyses vary which reshares enter training while holding the test reference fixed. Performance is generally stable, except under extreme filtering and, for ML, when all available training data are included.
- Evaluation design: The fixed test set uses credibility threshold 39.0, isolating the effect of changing the training threshold.Threshold 39.0 labels content considered unreliable because it severely violates basic journalistic standards.
- Training-threshold sensitivity: Varying the credibility threshold has limited effects on model performance, with only small fluctuations across thresholds.The comparison evaluates H-index, TASH-index, and machine-learning models.
- Training-threshold sensitivity: A threshold of 0.0 significantly worsens performance because training retains only reshares of content with credibility score exactly 0.The passage attributes the decline likely to the sharp reduction in training-data size.
- Evaluation design: Table A.2 reports nDCG and Quality scores at different @k levels, with 4.8% of misinformation remaining unmitigated because new users emerge during testing.The maximum Quality score is identical across methods because test-phase new users remain after training-observed nodes are removed.
- Training-threshold sensitivity: At the maximum threshold, the ML model slightly drops toward H-index performance, whereas TASH-index maintains a favorable performance level.The maximum threshold applies no filtering and uses all available training data.
Appendix C.2. Model Selection
Because user behavior overlaps across archetypes, the paper evaluates multiple machine-learning models and adopts Random Forest as the baseline for assessing TASH-Index alongside other user features.
- Overlapping user behavior motivates machine-learning models that can help disentangle archetypal complexity.
- Hyperparameter grid search is used to compare multiple models on test-set performance.The comparison is reported in Table C.3.
- The selected Random Forest model serves as the machine-learning baseline for evaluating TASH-Index alongside other user features.
- Random Forest emerges as the most stable and reliable model overall, particularly under high target skewness and inter-feature correlation.