Source-linked AI summary
A Decade of Social Bot Detection
Stefano Cresci
TL;DR
Social bots are increasingly sophisticated and difficult to distinguish from legitimate accounts, challenging detection efforts. This paper surveys a decade of research and finds that bot evolution undermines individual-account detectors, motivating coordinated group-based approaches and future innovations.
Problem
Sophisticated, evolving bots increasingly resemble legitimate or hybrid accounts, limiting detection approaches that treat bots as clearly separable and individually classifiable.
Method
The paper conducts a longitudinal survey of the first decade of social bot detection research, analyzing trends, results, challenges, and proposed innovations.
Results
Group-based detectors became more prevalent than individual-account detectors in 2018, while earlier classifiers failed to detect later waves of evolving bots.
Takeaways & Limitations
Effective bot detection should account for coordinated group behavior and the evolving, hybrid nature of automated and human-driven accounts.
Takeaways & Limitations
Bots’ hybrid automated and human-driven behaviors weaken binary labels and make sophisticated accounts difficult to distinguish from legitimate accounts individually.
Abstract
from arXiv · showhide
On the morning of November 9th 2016, the world woke up to the shocking outcome of the US Presidential elections: Donald Trump was the 45th President of the United States of America. An unexpected event that still has tremendous consequences all over the world. Today, we know that a minority of social bots, automated social media accounts mimicking humans, played a central role in spreading divisive messages and disinformation, possibly contributing to Trump's victory. In the aftermath of the 2016 US elections, the world started to realize the gravity of widespread deception in social media. Following Trump's exploit, we witnessed to the emergence of a strident dissonance between the multitude of efforts for detecting and removing bots, and the increasing effects that these malicious actors seem to have on our societies. This paradox opens a burning question: What strategies should we enforce in order to stop this social bot pandemic? In these times, during the run-up to the 2020 US elections, the question appears as more crucial than ever. What stroke social, political and economic analysts after 2016, deception and automation, has been however a matter of study for computer scientists since at least 2010. In this work, we briefly survey the first decade of research in social bot detection. Via a longitudinal analysis, we discuss the main trends of research in the fight against bots, the major results that were achieved, and the factors that make this never-ending battle so challenging. Capitalizing on lessons learned from our extensive analysis, we suggest possible innovations that could give us the upper hand against deception and manipulation. Studying a decade of endeavours at social bot detection can also inform strategies for detecting and mitigating the effects of other, more recent, forms of online deception, such as strategic information operations and political trolls.
THE SOCIAL BOT PANDEMIC
Social bots remain difficult to define precisely, while studies report substantial and sometimes sharply elevated bot presence in major online platforms and politically or economically salient discussions. These concerns have driven rapidly increasing research on bot characterization, detection, and impact estimation.
- Definitions: Social bots lack a precise, widely agreed definition because different research communities emphasize their multifaceted and dynamic behavior.Definitions vary between technical perspectives and other community-specific approaches.
- Scale: 15% of active Twitter accounts in 2017 and 11% of Facebook accounts in 2019 were estimated to be bots.These estimates indicate a considerable share of accounts across both platforms.
- Scale: 71% of Twitter users mentioning trending US stocks in 2019 were likely to be bots, showing that bot presence increases dramatically around strong political or economic interests.Similar findings were reported for online cryptocurrency discussions.
- Research response: Research on bot characterization, detection, and impact estimation has followed a steeply rising publication trend.The paper attributes this growth partly to concerns about bots’ reported role in harms affecting online ecosystems.
THE DAWN OF SOCIAL BOT DETECTION
Social bot detection research began in January 2010, initially relying mainly on supervised machine learning applied separately to individual accounts. Although these approaches showed promise, unreliable ground truth, binary labels, and evolving bot behavior created persistent challenges.
- Early approaches: The first dedicated work on detecting automated social-network accounts dates to January 2010 and predominantly used supervised machine learning on individual accounts.Detectors were separately applied to each account and assigned a binary label.
- Early approaches: Early detectors could identify simplistic fake followers using only 19 inexpensive data and computation features.Fake followers did not need to produce content or engage in conversations, making detection relatively easy.
- Limitations: Supervised detection was hindered by the lack of real ground-truth datasets and inconsistent human labeling caused by diverse definitions of social bots.Labels were often assigned by human operators who manually analyzed the data.
- Limitations: Binary classifications failed to represent malicious accounts combining automated and human-driven behaviors.Such mixed behavior cannot be adequately captured by simple binary labels.
- Limitations: The evolutionary nature of social bots constituted another major drawback of individual detectors.As bots became more sophisticated, distinguishing them from legitimate human-operated accounts became more difficult.
THE ISSUE OF BOT EVOLUTION
Bot evolution has produced increasingly sophisticated and human-like accounts that undermine detection methods designed for earlier bot generations. As a result, bots are no longer reliably separable from legitimate accounts when analyzed individually.
- THE ISSUE OF BOT EVOLUTION: Initial detection successes prompted bot developers to adopt countermeasures, creating a cycle of increasingly sophisticated social bots known as bot evolution.Newer bots became harder to detect than older ones.
- THE ISSUE OF BOT EVOLUTION: Research from 2011–2013 provided the first evidence and theoretical foundations for studying social bot evolution.Early bots were relatively simplistic, with low reputation, few connections, few posts, and clear automation signals.
- THE ISSUE OF BOT EVOLUTION: A supervised classifier initially detected the second wave of evolving bots accurately, but failed as a third wave emerged across social networks from 2016 onward.The third wave is identified in the supplied passage as shown in panel C of Figure 4.
- THE ISSUE OF BOT EVOLUTION: Sophisticated bots are difficult to distinguish from legitimate accounts individually because newer bots increasingly resemble human-operated accounts and combine automated with human-driven behavior.This hybridization produces cyborg accounts.
- THE ISSUE OF BOT EVOLUTION: The evolution of online automation invalidates the early assumption that bots are clearly separable from legitimate accounts.The limitation applies especially to naïve, early supervised bot-detection approaches.
THE RISE OF GROUP APPROACHES
Since 2012–13, bot-detection research increasingly shifted from analyzing individual accounts to targeting groups, exploiting traces of coordination and larger data volumes. Group detectors also introduced task-specific algorithms, while the field’s approach shaped detection performance and reflected broader institutional attention to coordinated inauthentic behavior.
- From individual to group detection: Since 2012–13, multiple teams independently proposed detectors that targeted groups of accounts rather than individual accounts.The systems used different techniques and implementations but shared this group-oriented philosophy.
- From individual to group detection: Coordinated botnets leave more automation traces than sophisticated single bots, making synchronized behavior and larger account groups useful detection targets.Group analysis also supplies more data for computationally demanding AI algorithms.
- Algorithmic specialization: Group detectors commonly replaced general-purpose classifiers with ad-hoc algorithms designed specifically for bot detection to improve performance.Facebook and Twitter’s recognition of “coordinated inauthentic behaviors” underscored the importance of this focus.
- Longitudinal evidence: A longitudinal classification of more than 230 papers documented the rise of group approaches alongside steadily increasing annual bot-detector publication since 2015.The survey classified detectors by whether they targeted individual accounts or groups and by their high-level approach.
- Methods for group analysis: Researchers detected coordination through graph structures, dense adjacency-matrix blocks, spectral patterns, and anomalous temporal tweeting or retweeting behavior.These methods addressed interactions among users and users’ interactions with content.
- Consequences for detection: The shift from individual to group detectors can substantially affect detection performance, while disciplinary preferences can favor account-level textual classifiers.Natural language processing research often emphasizes content and supervised binary classifiers analyzing accounts individually.
A GLIMPSE INTO THE FUTURE OF DECEPTION DETECTION
Future deception detection must move beyond reactive systems and account for the nonstationary, adversarial nature of social bot detection. Adversarial machine learning, including GAN-based generation of challenging examples, could enable more proactive and robust detectors, but the research direction remains nascent and faces major open challenges.
- Reactive detection: Existing individual and group-based bot detectors follow a reactive schema, adapting only after new misbehaving accounts evade current techniques.Bot mischiefs have consistently driven the development of new detectors.
- Adversarial environment: Social bot detection violates machine learning assumptions of stationarity and neutrality because bot developers actively try to fool detectors.This violation makes previously strong algorithms less reliable for detecting bots in-the-wild.
- Adversarial detection: Adversarial bot detection uses meaningful adversarial examples to expose weaknesses and proactively test detectors instead of waiting for new bot mischiefs.Designing for adversaries can provide higher guarantees for deception detection.
- GAN-based methods: GANs could generate many plausible adversarial social-bot examples, overcoming limited applicability and example scarcity while improving discriminator training.A GAN jointly trains competing deep-learning networks, with the generator producing increasingly challenging malicious examples.
- Open challenges: Adversarial detection remains in its infancy and requires techniques for generating diverse adversarial examples and evaluating their effectiveness.Early efforts were researcher-driven and covered few cases with limited applicability.
OPEN CHALLENGES AND THE WAY AHEAD
The authors identify organization, standardized evaluation, generalizability, and reference datasets as priorities for advancing social bot detection. They recommend coordination-focused, nonbinary, less supervision-dependent, and adversary-aware approaches to produce more actionable results.
- The exponentially growing research literature needs better organization to improve knowledge exploitation and enable researchers to develop new solutions more efficiently.
- Benchmarks, frameworks, and reference datasets are increasingly necessary to evaluate and compare the growing range of disparate bot-detection techniques.
- Detectors should be evaluated across many bot types and collection times because favorable single-type, single-time evaluations may overestimate detection performance.The proposed bi-dimensional generalizability space covers bot-type and temporal dimensions.
- Reference datasets should include social bots, cyborgs, and political trolls, but limited data, ambiguous ground truth, and dataset obsolescence remain obstacles.
- Future techniques should identify suspicious coordination, replace binary labels with multifaceted indicators, favor unsupervised or semi-supervised methods, and account for adversaries by design.