Fame for sale: efficient detection of fake Twitter followers

Stefano Cresci; Roberto Di Pietro; Marinella Petrocchi; Angelo Spognardi; Maurizio Tesconi

Source-linked AI summary

Fame for sale: efficient detection of fake Twitter followers

Stefano Cresci, Roberto Di Pietro, Marinella Petrocchi, Angelo Spognardi, Maurizio Tesconi

arXiv:1509.04098v2Sep 14, 2015cs.SIcs.CRcs.LG

TL;DR

Fake followers can distort the apparent popularity and influence of Twitter accounts, yet the phenomenon has received limited dedicated investigation. The paper constructs a verified baseline, evaluates existing media rules and academic features, and revises classifiers for lower cost and overfitting. Its Class A approach retains comparable detection rates while using less resource-demanding features.

Problem
Fake followers can distort perceived Twitter popularity and influence, while the phenomenon has not been deeply investigated in the literature.
Method
The paper builds a baseline of human and fake-follower accounts, evaluates existing classifiers, and revises them to reduce overfitting and data-gathering cost.
Results
Class A achieves detection rates comparable with the best classifiers while avoiding their overhead-demanding features.
Takeaways & Limitations
Fake Twitter followers can be detected without leveraging resource-demanding features.
Takeaways & Limitations
Countermeasures could be taken to evade some Class A features, while relationship-based Class C features require more data.

Abstract

from arXiv · show

$\textit{Fake followers}$ are those Twitter accounts specifically created to inflate the number of followers of a target account. Fake followers are dangerous for the social platform and beyond, since they may alter concepts like popularity and influence in the Twittersphere - hence impacting on economy, politics, and society. In this paper, we contribute along different dimensions. First, we review some of the most relevant existing features and rules (proposed by Academia and Media) for anomalous Twitter accounts detection. Second, we create a baseline dataset of verified human and fake follower accounts. Such baseline dataset is publicly available to the scientific community. Then, we exploit the baseline dataset to train a set of machine-learning classifiers built over the reviewed rules and features. Our results show that most of the rules proposed by Media provide unsatisfactory performance in revealing fake followers, while features proposed in the past by Academia for spam detection provide good results. Building on the most promising features, we revise the classifiers both in terms of reduction of overfitting and cost for gathering the data needed to compute the features. The final result is a novel $\textit{Class A}$ classifier, general enough to thwart overfitting, lightweight thanks to the usage of the less costly features, and still able to correctly classify more than 95% of the accounts of the original training set. We ultimately perform an information fusion-based sensitivity analysis, to assess the global sensitivity of each of the features employed by the classifier. The findings reported in this paper, other than being supported by a thorough experimental methodology and interesting on their own, also pave the way for further investigation on the novel issue of fake Twitter followers.

1. Introduction

Twitter’s broad adoption has enabled anomalous accounts and made fake followers a socially and economically relevant problem. Existing media detection rules are often presented without analytical validation, while academic spam and bot research offers machine-learning features that motivate systematic evaluation.

Twitter’s widespread use has made it an arena for anomalous accounts behaving in unconventional ways.
Fake followers are accounts exploited to increase a target account’s follower count, potentially making it appear more trustworthy and influential.
Artificially inflated popularity can affect perceived creditworthiness and help malicious accounts publish authoritative messages or run more effective advertising campaigns.
Media and blogger rules propose intuitive red flags for fake followers, but they are commonly presented without analytic algorithms or validation mechanisms.
Academic research has mainly addressed spam and bot detection using machine-learning classifiers trained on manually annotated accounts and non-human features.
Despite its economic and social impacts, fake-follower detection has not been deeply investigated in the literature.

Contributions

The paper builds a verified baseline and evaluates media rules alongside academic spam-detection features for fake-follower classification. It then develops lighter classifiers that reduce data-gathering cost while retaining high training-set accuracy and tests them on separate account sets.

The study builds a baseline dataset in which human and fake-follower accounts are known a priori.
It tests blogger classification rules and feature sets from spam-detection research on the baseline dataset.
Media-style classification rules do not successfully detect fake followers in the baseline dataset, whereas spam-detection feature classifiers work well.
The analysis finds that the best-performing features are also the most costly to gather, limiting their practical applicability.
The lightweight classifiers use less costly features while correctly classifying more than 95% of accounts in the training dataset.
Detection performance is additionally validated on two account sets disjoint from the original training dataset.

Road map

The paper progresses from related work and baseline-data construction to empirical evaluation of media criteria and academic spam-detection features. It then analyzes feature-extraction cost, presents a lightweight classifier, and concludes with the study’s findings.

Section 2 reviews and compares related work on Twitter spam and bot detection.
Section 3 describes the baseline dataset, while Section 4 evaluates social-media criteria for fake-follower detection using that dataset.
Section 5 examines features previously used to detect spam on Twitter accounts.
Section 6 computes feature-extraction costs and presents a lightweight classifier balancing detection capability and crawling cost.
Section 7 concludes the paper.

2. Related Work

Related work distinguishes fake followers from other anomalous Twitter accounts and surveys media rules, academic spam and bot detectors, and account-market research. The paper uses this body of criteria to build and evaluate a specialized fake-follower approach.

The paper distinguishes fake followers from spammers, bots, and cyborgs, which respectively advertise harmful content, automate accounts, or combine manual and automated behavior.
Fake followers are accounts created and sold to follow target accounts, magnifying customers’ apparent influence and engagement through inflated follower counts.
The paper assesses whether intuitive media criteria can scientifically support fake-follower detection, rather than merely applying undisclosed commercial analyses.
Media reports score followers against common-sense human-behavior criteria and classify them as human, bot, or neutral, but the reported results lack validation.
Academic work detects spam using profile, behavioral, network, and content features, often combined in machine-learning or hybrid models.
Prior studies report higher detection rates from new spam criteria and large-scale classifier-based removal of spam profiles.
Other research clusters newly created malicious accounts, detects account markets and customers, and studies market operations and fraud.
The cited academic selection is not exhaustive, but it supplies a large collection of criteria leveraged for the paper’s fake-follower detection approach.

3. Baseline datasets

The paper constructs verified human and fake-follower datasets, combines them into a balanced 3,900-account baseline, and makes the baseline publicly available. Collection involved large-scale crawling and manual or CAPTCHA-based verification, while the fake-follower set is explicitly illustrative rather than exhaustive.

The study crawled 9 million Twitter accounts and about 3 million tweets to build its empirical datasets.
TFP dataset: The TFP dataset identified 469 certified humans among 574 followers after CAPTCHA verification.
Human accounts: The human reference set HUM contains 1,950 verified human accounts assembled from the TFP and E13 subsets.
Fake followers: The fake-account data came from three online markets, with 155 of 1,000 TwitterTechnology accounts suspended before crawling.
Scope: The authors state that the fake-follower dataset is illustrative rather than exhaustive and reflects accounts easily found on the Web at search time.
Baseline dataset: The balanced BAS dataset contains 1,950 fake followers and 1,950 humans, totaling 3,900 accounts, and is used for the experiments.

4. Fake detection with algorithms based on classification rules

The paper evaluates rule-based procedures proposed by media and social-media analysts for identifying suspicious Twitter accounts. It finds that many such rules perform poorly individually or in combination, although a small number of rules achieve stronger discrimination.

Rule-based procedures: The reviewed media proposals combine multiple account checks into rule-based algorithms, but often omit how their criteria should be aggregated.
Evaluation strategy: The authors evaluate each rule separately before combining rules and features, because individual criteria may gain classification power in combination.
Camisani-Calzolari algorithm: The Camisani-Calzolari algorithm assigns human and bot scores from public account features and classifies accounts using the score difference.
Operational constraints: Rules requiring continuous monitoring of follow or unfollow behavior were not applied because Twitter API limits make such monitoring practically infeasible.
Algorithm performance: The Camisani-Calzolari algorithm achieved accuracy close to 0.5, high precision, very low recall, F-Measure close to 0.1, and low MCC on the evaluated dataset.
Rule weaknesses: Many rules use relevant attributes but define them too simply, while some rules can be misleading because fake followers almost never tweet through APIs.

5. Fake detection with algorithms based on feature sets

The paper repurposes academic spam-detection features for fake-follower detection and evaluates them with multiple machine-learning classifiers. These feature-based classifiers perform strongly on the BAS dataset, while feature choice affects accuracy, interpretability, and robustness.

Spam-detection features are used for the first time to identify fake followers in Twitter accounts.
The study quantifies account properties as features and trains both interpretable glass-box and non-interpretable black-box classifiers.Decision trees and regression models expose their internal structure, whereas support vector machines do not provide direct human-explicable correspondence.
The most promising features include number of tweets, friends/(followers^2) ratio, bidirectional links ratio, and API ratio.Removing the bidirectional link ratio reduces effectiveness and increases decision-tree complexity from 33 to 101 nodes in the reported comparison.
Classifiers using Yang et al.’s features achieve MCC greater than 0.98 for RF, J48, and D, with precision and recall around 0.99.They also achieve higher AUC than classifiers using Stringhini et al.’s features; the latter still obtain accuracy, precision, and recall around 0.98 for several classifiers.
All classifiers almost correctly distinguish human and fake-follower accounts on the BAS dataset, and feature-based classifiers outperform the CC algorithm.The Yang et al. feature set performs slightly better overall than the Stringhini et al. set.
Fake followers appear more passive than spammers and less likely to use automated mechanisms for posting tweets.In the training dataset, 72% of fake followers have URL ratio below 0.05, while 78% have API ratio below 0.0001.

6. An efficient and lightweight classifier

The paper develops a lightweight Class A classifier by selecting features according to detection performance and crawling cost. Class A remains effective on test sets, although performance varies across classifiers and validation populations.

The lightweight-classifier analysis selects features and rules by balancing fake-follower detection capability against crawling cost.Crawling cost dominates computational cost because some features require downloading profiles, timelines, or relationship data.
Class A features require follower profiles, Class B features require followers’ tweets, and Class C features require their friends and followers.The categories are evaluated by estimating the number of Twitter API calls needed to obtain the required data.
Class C classifiers perform best overall, with RF, D, and AB reaching AUC 0.999, but Class C only slightly outperforms Class A.For RF, D, and J48, the MCC difference is around 0.02 and the AUC difference is 0.004; Class A with BN outperforms its Class C counterpart on most metrics.
Class A classifiers provide good performance while exploiting cost-efficient features, making them feasible for large-scale fake-follower investigation.
The classifier’s feature set may require continual updating because fake-follower generators can evade some of its simplest features.
Global sensitivity analysis finds that all 19 Class A features contribute significantly, with social-tie features playing a central role.The friends/(followers^2) ratio is the most important feature in the global analysis, while bidirectional link ratio is strongest among the compared Class C features.

7. Conclusions

The paper develops efficient techniques for detecting fake Twitter followers by combining effective features with attention to crawling cost. Its Class A classifiers retain detection performance while avoiding resource-demanding features, and sensitivity analysis identifies social-relation features as dominant.

Conclusions: The study constructs a baseline dataset of human and fake follower accounts and uses it to evaluate detection features and classifiers.The dataset was created specifically to support analysis of the novel fake-follower detection problem.
Conclusions: SVMs achieve results comparable to the best-performing classifiers, including Random Forest and Decorate.The paper identifies SVMs as a promising addition to the classifiers used in prior fake- and spam-detection studies.
Conclusions: Features based on friends and followers, including the bidirectional link ratio and friends/(followers^2) ratio, yield the best results.These relationship-oriented features evaluate social ties between an account and its neighbors.
Conclusions: Relationship-based Class C features are more effective than simpler profile features but require substantial data downloads and analysis time.Their cost makes them unsuitable for analyses involving massive numbers of followers.
Conclusions: Timeline-based Class B features are less time-demanding while remaining effective, offering a trade-off between efficient and accurate detection.The proposed Class A classifiers further show that currently available fake followers can be detected without resource-demanding features.
Conclusions: Information fusion-based sensitivity analysis shows that features based on social relations play a dominant role among the Class A features.The analysis assesses the global importance of features used across the classifiers.