Source-linked AI summary
Predicting online extremism, content adopters, and interaction reciprocity
Emilio Ferrara, Wen-Qiang Wang, Onur Varol, Alessandro Flammini, Aram Galstyan
TL;DR
Online extremism raises the need to detect extremist users and forecast how regular users adopt extremist content or reciprocate extremist contacts. The paper applies metadata, network, and temporal features with existing learning models to curated Twitter data in static and simulated real-time settings. Across six task-modality combinations, the best AUC ranges from 72% to 93%.
Problem
The paper addresses how to detect extremist users and predict extremist-content adoption and interaction reciprocity in social media.
Method
The framework uses Logistic Regression and Random Forests with metadata, network, and temporal features on curated Twitter datasets, evaluated statically and in simulated real time.
Results
The best performance across the six task-modality combinations ranges between 72% and 93% AUC, depending on task complexity and training-data availability.
Takeaways & Limitations
Feature analysis highlights retweet-to-tweet ratio, average hashtags, tweet count, and average retweets as consistently strong predictive signals.
Takeaways & Limitations
The authors state that future work must address more realistic and complex prediction tasks and deploy a real-time abuse-detection prototype.
Abstract
from arXiv · showhide
We present a machine learning framework that leverages a mixture of metadata, network, and temporal features to detect extremist users, and predict content adopters and interaction reciprocity in social media. We exploit a unique dataset containing millions of tweets generated by more than 25 thousand users who have been manually identified, reported, and suspended by Twitter due to their involvement with extremist campaigns. We also leverage millions of tweets generated by a random sample of 25 thousand regular users who were exposed to, or consumed, extremist content. We carry out three forecasting tasks, (i) to detect extremist users, (ii) to estimate whether regular users will adopt extremist content, and finally (iii) to predict whether users will reciprocate contacts initiated by extremists. All forecasting tasks are set up in two scenarios: a post hoc (time independent) prediction task on aggregated data, and a simulated real-time prediction task. The performance of our framework is extremely promising, yielding in the different forecasting scenarios up to 93% AUC for extremist user detection, up to 80% AUC for content adoption prediction, and finally up to 72% AUC for interaction reciprocity forecasting. We conclude by providing a thorough feature analysis that helps determine which are the emerging signals that provide predictive power in different scenarios.
1. Introduction
The paper develops a computational framework for detecting extremist users and forecasting extremist-content adoption and interaction reciprocity. It combines multidimensional social-media features, a large curated dataset, and both post-hoc and simulated real-time prediction settings.
- The study formalizes three forecasting tasks: detecting extremist users, predicting extremist-content adoption, and forecasting interaction reciprocity between regular users and extremists.
- The framework extracts user metadata, network statistics, and temporal activity patterns to support all three prediction tasks.
- The dataset combines over 3 million tweets from more than 25 thousand extremist accounts with around 30 million tweets from 25 thousand exposed regular users.
- Each forecasting task is evaluated both on aggregated data and in simulated real time, with models trained on past data to predict future activity.
- The analysis identifies which employed features provide predictive power across the three forecasts.
2. Data and Preliminary Analysis
The study constructs labeled Twitter datasets through manual curation, expert verification, and gardenhose collection. It pairs identified ISIS-supporter accounts with a similarly sized sample of users exposed to ISIS content.
- Manual curation and expert verification produced a list of over 25 thousand Twitter accounts labeled as supporting the Islamic State.
- The ISIS-account dataset contains 3,395,901 tweets from 25,538 identified supporters collected between January and June 2015.
- The exposed-user dataset contains 29,193,267 tweets from 25 thousand users randomly sampled among followers of ISIS accounts.
- The exposed-user sample was chosen to match the ISIS-account group in size and avoid class-imbalance issues.
- ISIS accounts serve as positive labels, while users exposed to ISIS serve as negative labels for prediction.
3. Methodology
The methodology uses Logistic Regression and Random Forests with 52 handcrafted metadata, timing, and network features. Feature selection and cross-validation support scalable evaluation across the prediction tasks.
- Learning models: The system uses LASSO-regularized Logistic Regression as a scalable baseline and Random Forests with 100 estimators and Gini split quality.
- Evaluation: Performance is averaged over 5-fold cross-validation, with 80% of data used for training and 20% for testing, while hyperparameters are grid-searched.
- Evaluation: The benchmark reports Precision, Recall, F1, and AUC, while the study explicitly aims to demonstrate promising results from existing methods rather than introduce new algorithms.
- Feature engineering: The framework generates 52 features spanning user metadata and activity, timing patterns, and network statistics.
- Feature engineering: User features include tweet, follower, friend, hashtag, mention, URL, and profile descriptors, while timing features summarize event intervals and posting rates.
- Feature selection: Greedy forward selection adds the feature producing the best average classifier performance until further significant improvement ceases.
4. Experiments
Experiments evaluate extremist-user detection, extremist-content adoption, and interactions with extremists in static and simulated real-time settings. The supplied passages define the tasks and describe the feature-ranking analysis used in evaluation.
- The experiments cover three tasks: extremist-supporter detection, extremist-content adoption, and interaction prediction with extremists.
- Task I (T1): Extremist-supporter detection tests whether designed activity features distinguish ISIS accounts from regular users and provides a yardstick for later tasks.
- Task II (T2): Content adoption is labeled positive when a regular user retweets extremist content to which the user was exposed.
- Task III (T3): Interaction reciprocity is labeled positive when a regular user replies to contact initiated by an ISIS account.
- Prediction modalities: Each task has static and simulated real-time modalities, with static evaluation aggregating January–June 2015 data and using random 5-fold splits.
- Feature analysis: Feature-selection analysis examines the top 11 features and their rank distributions.
4.1. T1: Detection of extremist supporters
The framework detects extremist Twitter accounts using handcrafted metadata, network, and temporal features in static and simulated real-time settings. Random Forest performance improves substantially when temporal dependencies are incorporated.
- 4.1.1. T1A: Time-independent detection.: Above 87% AUC is achieved by Random Forests in time-independent extremist-account detection.The static task aggregates six months of data and evaluates Logistic Regression and Random Forest models with five-fold cross-validation.
- 4.1.2. T1B: Simulated real-time detection.: Real-time evaluation trains on earlier temporal slices and tests predictions on subsequent months.Training and prediction periods are separated chronologically, with scenarios using different amounts of historical data.
- 4.1.2. T1B: Simulated real-time detection.: 93.2% AUC is achieved by Random Forests with five months of training data in simulated real-time detection.Performance ranges from 83.8% AUC with one month of training data to 93.2% with five months.
4.2. T2: Predicting extremist content adoption
This section evaluates prediction of whether regular users adopt extremist content in aggregated and simulated real-time settings. Random Forests perform best, with real-time performance reaching 80.2% AUC at peak.
- T2A and T2B: The section covers both static and real-time scenarios for predicting extremist-content adoption by regular users.The real-time scenario preserves temporal dependencies rather than aggregating all observations.
- T2B: Feature selection analysis examines the distribution and ranking of the top 11 features in the real-time adoption task.Figure 2 presents the T2B feature-selection analysis and feature-rank distribution.
- T2A: Random Forests are the best performer for aggregated content-adoption prediction, although Logistic Regression is narrowly behind.Positive cases are users who retweeted at least one ISIS tweet, while negative cases were exposed users who retweeted none.
- T2A: The aggregated task predicts whether regular users retweet content produced by an ISIS account during January–June 2015.The task uses time-aggregated datasets and distinguishes adopters from exposed non-adopters.
- T2B: 80.2% AUC is achieved at peak in real-time content-adoption prediction, where Random Forests benefit from temporal correlations.The real-time setup preserves temporal ordering and uses month-long training and testing slices.
4.3. T3: Predicting interactions with extremists
The third task predicts whether regular users reciprocate interactions initiated by extremists, comparing time-independent and simulated real-time settings. Real-time modeling reaches 72.6% AUC with Random Forests and 68.3% with Logistic Regression, while retweeting and hashtag features rank highly.
- T3A: Time-independent prediction.: Interaction reciprocity asks whether a regular user mentioned by an extremist account will reply.Positive instances are users who reply to at least one extremist-initiated contact.
- T3A: Time-independent prediction.: The interaction-prediction task is presented as the most challenging of the three forecasting tasks.The paper motivates this difficulty by citing prior work on predicting reciprocity and interaction intensity.
- T3B: Simulated real-time prediction.: 72.6% AUC is achieved by Random Forests for real-time interaction-reciprocity prediction, while Logistic Regression reaches 68.3%.The real-time model trains on the first five months and evaluates on the last month.
- T3B: Simulated real-time prediction.: The top three interaction-reciprocity features are retweets-to-tweets ratio, average hashtags, and average retweets.All three also appeared among the top features in the two previous tasks.
4.4. Feature predictive-power analysis
Feature selection on real-time tasks identifies a stable set of highly predictive Twitter metadata features across all three forecasting tasks. These features are computationally inexpensive and therefore suitable for efficient real-time systems.
- Feature ranking across tasks: Retweets-to-tweets ratio, average hashtags, tweet count, and average retweets consistently rank among the top features across all three tasks.The top-11 ranking includes two tied positions, motivating the use of 11 rather than 10 features.
- Feature selection: Feature selection is performed on real-time rather than time-aggregated prediction tasks.The analysis is intended to capture predictive value in real-time forecasting.
- Practical implications: The repeatedly high-ranking features can be computed directly from Twitter API metadata for efficient stream-based prediction.The paper connects this computational simplicity to potential real-time detection and prediction systems.
5. Related Literature
Related work spans computational social-science studies of extremist propaganda and machine-learning approaches to detecting radicalization, recruitment, hate promotion, and extremist support. Prior studies use social, textual, sentiment, metadata, and network-based evidence across several platforms and datasets.
- Computational social sciences research: Computational social-science research uses online data to study extremist propaganda, supporter populations, narrative framing, and recruitment mechanisms.Examples examine ISIS supporters on Twitter and Arabic-speaking discussions about the Arab Spring.
- Computational social sciences research: Sentiment analysis of Dabiq examined how ISIS language evolved, including increasing attention to females and internet jargon.The study interpreted these patterns in relation to radicalization motives, emotions, concerns, and online youth identities.
- Computer science research: Computer-science research develops models to detect extremist content, cyber-recruitment, radicalization efforts, hate promotion, and extremist support.Approaches include network analysis, sentiment detection, Bayesian criteria, textual features, metadata, crawling, and unsupervised clustering.
- Computer science research: Recent studies also examine behavioral change and the survival dynamics of ISIS-related online groups.One reports about 700 potentially radicalizing users among 154,000, while another studies 196 VKontakte aggregates involving about 100,000 users.
6. Conclusions
The paper builds three extremism-forecasting tasks from large Twitter datasets and evaluates them in static and simulated real-time scenarios. Across six task-modality combinations, the framework achieves 72%–93% AUC, while recurring metadata features provide interpretable predictive signals.
- Conclusions: The study uses over 25,000 suspended extremist accounts, more than three million extremist tweets, and almost 30 million tweets from an equal-sized exposed regular-user sample.The extremist accounts were manually identified, reported, and suspended, providing the ground truth described by the paper.
- Conclusions: The three tasks detect extremist users, predict extremist-content adoption, and forecast interactions between extremist and regular users.They are evaluated under both time-independent and timeline-respecting prediction settings.
- Conclusions: 72%–93% AUC is obtained across six forecasts combining three tasks with static and simulated real-time prediction scenarios.Performance varies with task complexity and the amount of training data available.
- Conclusions: Retweet ratio, average hashtags, tweet count, and average retweets repeatedly rank highly, highlighting content-production, network, and timing patterns.The authors present these recurring features as informative signals in extremist online conversations.
- Future work: Future work targets more realistic prediction tasks, deeper network and temporal analysis, and a prototype for real-time abuse-signature detection.These are stated as planned extensions rather than completed capabilities.