Source-linked AI summary
The Rise of Social Bots
Emilio Ferrara, Onur Varol, Clayton Davis, Filippo Menczer, Alessandro Flammini
TL;DR
Sophisticated social bots increasingly mimic human behavior across content, networks, timing, and sentiment, while their prevalence and detection remain uncertain. This paper reviews bot characteristics and Twitter detection approaches, finding that behavioral features can distinguish synthetic from human activity and reveal engineered social tampering.
Problem
Social bots increasingly emulate human behavior across content, networks, timing, and sentiment, making their prevalence and detection uncertain despite incentives for their development.
Method
The paper reviews sophisticated social bots and proposes a taxonomy spanning network-based, crowdsourcing, machine-learning, and combined detection approaches.
Results
Behavioral features capturing content, network, sentiment, and temporal patterns can discriminate bots from humans; Bot or Not? exceeds 95% AUROC on its dataset.
Takeaways & Limitations
Content, network, sentiment, and temporal signatures provide practical signals for detecting engineered social tampering on Twitter.
Abstract
from arXiv · showhide
The Turing test aimed to recognize the behavior of a human from that of a computer algorithm. Such challenge is more relevant than ever in today's social media context, where limited attention and technology constrain the expressive power of humans, while incentives abound to develop software agents mimicking humans. These social bots interact, often unnoticed, with real people in social media ecosystems, but their abundance is uncertain. While many bots are benign, one can design harmful bots with the goals of persuading, smearing, or deceiving. Here we discuss the characteristics of modern, sophisticated social bots, and how their presence can endanger online ecosystems and our society. We then review current efforts to detect social bots on Twitter. Features related to content, network, sentiment, and temporal patterns of activity are imitated by bots but at the same time can help discriminate synthetic behaviors from human ones, yielding signatures of engineered social tampering.
The rise of the machines
Social media ecosystems create economic and political incentives to design algorithms that exhibit human-like behavior. Social bots automatically produce content and interact with humans while emulating and possibly altering their behavior across multiple dimensions.
- The rise of the machines: Social media creates economic and political incentives to design algorithms that exhibit human-like behavior.These ecosystems include hundreds of millions of individuals.
- The rise of the machines: Social bots must emulate content, social networks, temporal activity, diffusion patterns, and sentiment expression.Social media introduces these dimensions beyond content alone, raising the challenge of human-like behavior.
- The rise of the machines: Social bots automatically produce content and interact with humans on social media while trying to emulate and possibly alter their behavior.Such bots have inhabited social media platforms for a few years.
Engineered social tampering
Social bots can provide useful services, yet they may also spread unverified information and create broader risks. Their combination with social-media-informed automated trading systems is particularly concerning because bots can amplify misleading information while trading systems lack fact-checking capabilities.
- Engineered social tampering: Some social bots are benign or helpful, aggregating content and automatically responding to customer inquiries.These services are increasingly adopted by brands and companies for customer care.
- Engineered social tampering: Even service-oriented bots can contribute to the spread of unverified information.The passage notes that ostensibly useful bots can sometimes become harmful.
- Engineered social tampering: Widespread bot diffusion may have unwarranted consequences for market stability as operators increasingly react to social-media signals.The passage cites claims that Twitter signals can help predict stock markets and reports growing evidence that market operators pay attention and react.
- Engineered social tampering: Combining social bots with automated trading systems creates risks because bots amplify misleading information while trading systems lack fact-checking capabilities.The passage describes these systems as exploiting social-media information at least partially and characterizes their combination as ripe with risks.
The bot effect
Social bots can tamper with the social Web in ways that endanger society, from disrupting public systems and markets to exposing users’ private information.
- The bot effect: Social bots may endanger democracy, cause panic during emergencies, and affect the stock market.These are described as consequences of tampering with the social Web.
- The bot effect: A social botnet study demonstrated that social media users were vulnerable to having private information exposed.The exposed information included phone numbers and addresses.
Act like a human, think like a bot
Modern social bots have evolved from easily detectable automated posters into sophisticated agents that imitate human profiles, content choices, and activity rhythms. This increasing human-likeness makes bot detection more difficult and blurs the boundary between human and bot behavior.
- Act like a human, think like a bot: Early social bots mainly posted content automatically and were easy to detect using simple signals such as unusually high content volume.A 2011 honeypot trap detected thousands of social bots.
- Act like a human, think like a bot: Modern Twitter bots can search the Web for profile material, collect media, and schedule posts to imitate human content production and consumption.Their activity can reproduce circadian patterns and temporal spikes in information generation.
- Act like a human, think like a bot: As Twitter bots become more sophisticated, distinguishing human-like from bot-like behavior becomes increasingly difficult.The boundary between the two behavioral categories is increasingly fuzzy.
A taxonomy of social bot detection systems
The section presents a taxonomy of social bot detection approaches comprising social-network information, crowdsourcing, and machine-learning methods based on discriminative features. It also notes that existing social-media-service strategies appear inadequate and that academic efforts are still at an early stage.
- Motivation: Social-media services’ current detection strategies appear inadequate for addressing social bots.The passage frames this inadequacy as a reason for developing advanced automatic detection methods.
- Motivation: Academic efforts to automatically detect or distinguish social bots from humans have only recently begun.The passage characterizes the academic response as being at an early stage.
- Taxonomy of approaches: Detection approaches are divided into three classes: social-network information, crowdsourcing and human intelligence, and machine-learning methods using highly revealing discriminative features.The taxonomy covers approaches proposed in the literature for distinguishing bots from humans.
Graph-based social bot detection
Graph-based social bot detection uses community structure to identify tightly knit groups of controlled accounts, but its effectiveness depends on the detection algorithm and can be weakened by attackers mimicking legitimate connectivity. Systems such as SybilRank address this limitation by incorporating innocent-by-association reasoning.
- Graph-based social bot detection: Community detection methods can reveal tightly knit local communities of controlled social bots, but the choice of algorithm crucially affects detection performance.This approach is used within adversarial social bot detection frameworks.
- Graph-based social bot detection: Attackers can counterfeit sybil-account connectivity to mimic legitimate community structure, defeating methods that rely solely on community detection.Detection systems such as SybilRank address this shortcoming by also employing innocent-by-association reasoning.
Crowd-sourcing social bot detection
Crowd-sourcing social bot detection uses human workers to evaluate conversational nuances and emerging anomalies through an Online Social Turing Test. However, privacy concerns and limited profile information can constrain human judgment, while some advanced bots may no longer seek to mimic human behavior.
- Crowd-sourcing social bot detection: Wang et al. proposed crowd-sourcing social bot detection to large numbers of workers and created an Online Social Turing Test as a proof of concept.The approach treats human detection as a scalable evaluation task.
- Crowd-sourcing social bot detection: The proposal assumes humans outperform machines at judging sarcasm, persuasive language, emerging patterns, and anomalies in online interactions.These conversational and behavioral nuances were presented as especially difficult for machines to evaluate.
- Crowd-sourcing social bot detection: Using workers for validation raises privacy concerns, and Twitter’s less informative profiles give annotators less basis for judgments than Facebook or Renren profiles.Twitter profiles are more public than Facebook profiles but contain less information than Facebook or Renren profiles.
- Crowd-sourcing social bot detection: Manual analysis of a Syrian Twitter botnet active for 35 weeks suggested that some advanced social bots may no longer aim to mimic human behavior.The observation came from annotator analysis of the botnet’s interactions and content.
Feature-based social bot detection
Feature-based social bot detection encodes behavioral patterns as machine-learning features to distinguish human-like from bot-like accounts. Systems such as Bot or Not? use highly predictive behavioral features and visualizations, while user metadata is among the most predictive and interpretable feature types.
- Feature-based detection: Behavioral patterns can be encoded as features that machine-learning methods use to learn human-like and bot-like signatures and classify accounts.Feature classes capture orthogonal dimensions of user behavior.
- Bot or Not?: Bot or Not?, released in 2014, was Twitter’s first publicly available social bot detection interface and used highly predictive features to separate bots from humans.The system was designed to raise awareness of social bots.
- Feature types: Common detection features include hashtag co-occurrence networks, sentiment signals, and the volume of content produced and consumed over time.Examples include emoticon, happiness, and arousal-dominance-valence signals, plus tweeting and retweeting activity.
- Evolving bot behavior: Because bots continuously evolve, analyzing highly predictive behaviors can reveal patterns for discriminating bots from humans, with user metadata among the most predictive and interpretable features.The passage presents metadata-based rules of thumb as a way to infer whether an account is likely a bot.
Combining multiple approaches
Effective social-bot and Sybil detection benefits from combining complementary behavioral and network-based signals. The Renren detector uses activity, timing, and predictive interaction features to classify accounts as bot-like or human-like.
- Combining multiple approaches: Renren examines multiple behavioral dimensions, including users’ activity and timing information, to detect Sybil accounts.The approach builds on the need for complementary detection techniques against Sybil attacks in social networks.
- Combining multiple approaches: Real users spend more time messaging and viewing other users’ content, whereas Sybil accounts harvest profiles and befriend other accounts.The comparison is based on ground-truth clickstream data.
- Combining multiple approaches: Renren uses invitation frequency, accepted outgoing requests, and network clustering coefficient to classify accounts into bot-like and human-like profiles.The detector identifies these features as highly predictive of account type.
Master of puppets
The section argues that research must identify who controls social bots and reverse-engineer their targeting, content, timing, and topics, while major uncertainties about bot prevalence remain. It warns that increasingly bot-populated ecosystems require humans and bots to recognize one another to avoid dangerous false assumptions.
- Master of puppets: Researchers must identify social bots’ “masters” and reverse-engineer whom bots target, how they generate content, when they act, and which topics they discuss.Governments and other well-resourced entities have been alleged to use social bots to their advantage.
- Master of puppets: Simple automated mechanisms that generate content and boost followers can successfully infiltrate platforms and increase bots’ social influence.These findings come from efforts to reverse-engineer platform vulnerability and infiltration strategies.
- Master of puppets: Researchers still do not know how many social bots exist or what share of social-media content they produce, because current estimates vary widely.The paper suggests that observed bots may represent only the tip of the iceberg, while initiatives such as DARPA’s SMISC challenge can catalyze research.
- Master of puppets: Future social-media ecosystems may normalize machine-machine interaction, making mutual recognition between bots and humans necessary to prevent bizarre or dangerous situations.Such situations can arise when people make false assumptions about the humanity of their interlocutors.