RTbust: Exploiting Temporal Patterns for Botnet Detection on Twitter

Michele Mazza; Stefano Cresci; Marco Avvenuti; Walter Quattrociocchi; Maurizio Tesconi

Source-linked AI summary

RTbust: Exploiting Temporal Patterns for Botnet Detection on Twitter

Michele Mazza, Stefano Cresci, Marco Avvenuti, Walter Quattrociocchi, Maurizio Tesconi

arXiv:1902.04506v1Feb 12, 2019cs.SIcs.AIcs.CRcs.CY

TL;DR

Social bots can manipulate retweets for economic and sociopolitical purposes, while existing detectors face evasion, labeling, and scalability challenges. The paper introduces RTbust, which extracts temporal features from retweet activity and clusters accounts, achieving F1 = 0.87 versus F1 ≤0.76 for competitors and uncovering 2 previously unknown active botnets.

Problem
Retweeting bots can manipulate popularity and influence, while existing detectors face bot evasion, unreliable labels, and costly timeline or graph analysis.
Method
RTbust uses an LSTM variational autoencoder to extract latent features from account retweet time series, then applies hierarchical density-based clustering for group-level labeling.
Results
F1 = 0.87 for RTbust versus F1 ≤0.76 for competitors, and application to a large retweet dataset uncovered 2 previously unknown active botnets.
Takeaways & Limitations
RTbust detects retweeting social bots using timestamps and group temporal patterns, avoiding the need for full user timelines or social graphs.
Takeaways & Limitations
RTbust’s latent features are difficult to interpret because they come from a black-box variational autoencoder.

Abstract

from arXiv · show

Within OSNs, many of our supposedly online friends may instead be fake accounts called social bots, part of large groups that purposely re-share targeted content. Here, we study retweeting behaviors on Twitter, with the ultimate goal of detecting retweeting social bots. We collect a dataset of 10M retweets. We design a novel visualization that we leverage to highlight benign and malicious patterns of retweeting activity. In this way, we uncover a 'normal' retweeting pattern that is peculiar of human-operated accounts, and 3 suspicious patterns related to bot activities. Then, we propose a bot detection technique that stems from the previous exploration of retweeting behaviors. Our technique, called Retweet-Buster (RTbust), leverages unsupervised feature extraction and clustering. An LSTM autoencoder converts the retweet time series into compact and informative latent feature vectors, which are then clustered with a hierarchical density-based algorithm. Accounts belonging to large clusters characterized by malicious retweeting patterns are labeled as bots. RTbust obtains excellent detection results, with F1 = 0.87, whereas competitors achieve F1 < 0.76. Finally, we apply RTbust to a large dataset of retweets, uncovering 2 previously unknown active botnets with hundreds of accounts.

1 INTRODUCTION

Social bots can manipulate retweet activity for sociopolitical and economic gains, while increasingly sophisticated accounts evade individual detection. RTbust addresses this challenge by analyzing coordinated temporal patterns efficiently and unsupervised.

Artificially boosted retweets can reshape political debates, affect popularity or influence, defraud businesses, and motivate malicious bot activity.
Bot developers continually modify accounts to evade deployed detectors, making sophisticated bots nearly indistinguishable from legitimate accounts.
Large bot groups still leave automation traces because they share common goals, motivating unsupervised group-level detection.
Existing behavior- and graph-based methods require substantial data collection and computation, limiting large-scale applicability.
F1 = 0.87 for RTbust, compared with F1 ≤0.76 for competitors, using retweet timestamps rather than full timelines or social graphs.
The study introduces retweet visualization and group analysis, then uncovers 2 previously unknown active botnets.

2 RELATED WORK IN BOT DETECTION

Prior bot detectors commonly use supervised learning and account-level features, but labeled ground truth is problematic and sophisticated bots are difficult to identify. Group-level and adversarial approaches address coordination and adaptation in complementary ways.

Supervised bot detection often lacks reliable ground truth because labels are manually assigned and no standard definition of a social bot exists.
Bot detectors primarily exploit profile features, post features, or social and interaction graph features, with feature choice affecting performance and efficiency.
Group-level detection analyzes accounts collectively and is effective for coordinated, synchronized bots.
Adversarial machine learning shifts bot detection from reactive countermeasures toward proactive analysis of evolving evasion strategies.

3 DATA COLLECTION AND ANNOTATION

The study builds a large, complete Italian-retweet dataset spanning two weeks and supplements unsupervised detection with manual annotation for evaluation. The data show both typical and extreme user activity.

9,989,819 retweets from 1,446,250 users were collected between 17 and 30 June 2018, averaging 7 retweets per user per day.
The retweet-per-user distribution has a heavy-tailed shape, indicating many extreme users alongside typical activity.
The dataset contains 1,691,865 distinct original tweets and records hourly retweet volume across the two study weeks.
Twitter Premium Search API enabled a complete retweet dataset using the queries lang:IT and is:retweet.
Manual inspection identified highly automated but non-malicious accounts that neither disguised themselves as humans nor acted coordinately to inflate popularity.
Approximately 1,000 accounts were annotated, yielding an almost balanced set of 51% bots and 49% legitimate accounts.

4 PATTERNS OF RETWEETING ACTIVITY

RTT visualizes each retweet against its original tweet to reveal temporal behavior. Legitimate users show a near-diagonal droplet pattern, whereas bots exhibit straight-line, triangular, or waterfall regularities.

RTT plots retweet timestamps on the x axis against corresponding original-tweet timestamps on the y axis, with the diagonal encoding causality.
Normal behaviors – droplet pattern: Legitimate users typically produce a near-diagonal droplet pattern, reflecting short retweet delays and occasional vertically stacked sequences.
Normal behaviors – droplet pattern: Striking regularities in RTT plots are associated with automated actions and therefore indicate bot behavior.
Suspicious behaviors – straight line pattern: The straight-line pattern shows retweets occurring within seconds and activity divided into sessions separated by inactivity gaps.
Suspicious behaviors – triangular pattern: The triangular pattern consists of highly regular activity sessions with similar durations and inactivity intervals.
Suspicious behaviors – waterfall pattern: The waterfall pattern contains long vertical lines caused by systematic reverse-chronological retweeting of a feed or timeline.
RTT supports both human analysis and the subsequent automatic detection of malicious retweeting bots.

5 INTRODUCING RETWEET-BUSTER

RTbust detects retweeting bots by converting users’ temporal retweet activity into compact unsupervised features and clustering users with similar behaviors. It uses timestamp-based time series, lossless compression, an LSTM autoencoder, and density-based clustering to identify coordinated groups.

Data preparation and compression: RTbust organizes each user’s tweet and retweet timestamps into a retweet time series for bot detection.The series uses a reference timestamp and records retweet activity at each time point.
Data preparation and compression: One observation per second preserves fine temporal detail, although the resulting user time series are highly sparse.Users retweet, on average, only once in a few minutes.
Data preparation and compression: RTbust applies modified run-length encoding to compress consecutive zeros representing periods without retweeting activity.Only zero runs are compressed, using negative run lengths to distinguish them from positive retweet observations.
Unsupervised feature extraction: An LSTM autoencoder transforms variable-length retweet time series into fixed-length latent feature vectors through unsupervised feature extraction.The decoder supports training, while the trained encoder serves as the feature extractor; lower-dimensional vectors provide more compression but can lose information.
User clustering: HDBSCAN clusters users’ latent vectors to identify large groups with common retweeting behaviors that may constitute coordinated botnets.The approach uses density- and hierarchical-based clustering after LSTM encoding.

6 EXPERIMENTS AND RESULTS

Experiments evaluate RTbust configurations, feature extractors, baselines, and state-of-the-art detectors on annotated accounts, then inspect its discovered clusters. The VAE-based configuration performs best and uncovers two previously unknown active retweeting botnets.

Parameters configuration: 8 VAE features stabilize clustering and achieve good bot detection results, while increasing the latent vector beyond 10 features stops reducing the number of clusters.PCA and TICA features provide significantly less information for bot detection than VAE features.
Parameters configuration: Clusters larger than 10 accounts produce the best results, providing an operational threshold for excluding minuscule botnets.The threshold is described as both intuitive and operationally effective.
Quantitative evaluation of bot detection: F1 = 0.87 is achieved by VAE-based RTbust, outperforming Social fingerprinting at F1 = 0.76, Botometer at F1 = 0.43, and Holo-Scope at F1 = 0.01.RTbust beats all competitors on every evaluation metric except recall; PCA- and TICA-based RTbust variants obtain F1 ≤0.67.
Qualitative evaluation of bot detection: RTbust and t-SNE organize bots into different clusters, yet both preserve an overall distinction between bots and legitimate accounts.Most legitimate accounts are unclustered and central, whereas most bots are colored and positioned near other bots.
Qualitative evaluation of bot detection: RTbust discovered two previously unknown active retweeting botnets, including one with 44 accounts and another with almost 300 accounts.The smaller botnet focuses on three car-related accounts, while the larger one promotes tweets about Italian pop singer Valerio Scanu.

7 DISCUSSION

RTT plots provide a lightweight way to inspect suspicious retweeting behavior and support annotation, account verification, and explanation of bot-detector decisions. RTbust extends this analysis through unsupervised pattern discovery, while its latent-feature design leaves explanations difficult without visualization.

Visualizing suspicious behaviors: RTT plots reveal retweeting behavior for individual accounts and groups while requiring less data than graph- or cascade-based visualizations.Earlier visualizations may require full social graphs or retweet threads and cascades.
Visualizing suspicious behaviors: RTT plots can help human annotators label social bots and administrators verify automation before banning accounts.They may speed manual assessment and reduce human mistakes.
Generalizability and robustness: RTbust can potentially detect previously unseen retweeting-bot behaviors because it automatically identifies meaningful patterns without supervised labels.This matters because no standard definition of social bots or their expected behaviors is well agreed upon.
Generalizability and robustness: RTbust’s generalizability and robustness are desirable because social bots evolve to evade detection techniques.The method is intended to better withstand future changes in bot behavior.
Explainability: RTbust’s latent features are difficult to interpret, creating an explainability drawback that RTT plots can help mitigate after labeling.The difficulty stems from interpreting features produced by the black-box variational autoencoder.

8 CONCLUSIONS

The paper introduces RTT visualization and RTbust for analyzing retweeting activity and detecting malicious retweeting bots. RTbust outperformed competing techniques and revealed two previously unknown active botnets, while its current account-clustering decision is identified for improvement.

RTT visualization: RTT plots provide an effective and efficient visualization for examining Twitter retweeting behavior.The authors use them to analyze normal legitimate-user behavior and identify suspicious automated-retweeting patterns.
RTT visualization: Three suspicious retweeting behaviors are attributed to automated activity and treated as representative of bot activities.The paper contrasts these with the normal retweeting behavior of legitimate users.
RTT visualization: RTT plots can support dataset annotation, administrator verification, and explanation of black-box bot-detector decisions.They are presented as a tool for interpretable and explainable AI.
Retweet-Buster: RTbust uses an LSTM variational autoencoder and hierarchical density-based clustering to detect retweeting social bots through group analysis.Accounts in large clusters are labeled bots, while unclustered accounts are labeled legitimate.
Results: F1 = 0.87 for RTbust, compared with F1 ≤0.76 for other techniques.The comparison covers baselines and state-of-the-art social bot detection techniques.
Results: 2 previously unknown active botnets comprising hundreds of accounts were discovered by applying RTbust to a large retweet dataset.The discovery resulted from applying the technique beyond its evaluation comparison.
Limitations and future work: RTbust’s current account-clustering decision is described as naive, with recall identified as its bottleneck.Future refinements may prune clusters or accounts to improve precision or expand clusters to improve recall.