PaperlayerView exact revision on arXiv

Source-linked AI summary

Twitter Bot Detection Using Bidirectional Long Short-term Memory Neural Networks and Word Embeddings

Feng Wei, Uyen Trang Nguyen

arXiv:2002.01336v1cs.CRcs.CLcs.LG

TL;DR

Twitter bots can provide benign updates or spread spam, and the paper addresses distinguishing bot from human accounts using tweet content alone. It applies a BiLSTM with word embeddings; on cresci-2017, it reports improvements over prior systems on selected test-set metrics and comparable performance overall.

Abstract

from arXiv · show

Twitter is a web application playing dual roles of online social networking and micro-blogging. The popularity and open structure of Twitter have attracted a large number of automated programs, known as bots. Legitimate bots generate a large amount of benign contextual content, i.e., tweets delivering news and updating feeds, while malicious bots spread spam or malicious contents. To assist human users in identifying who they are interacting with, this paper focuses on the classification of human and spambot accounts on Twitter, by employing recurrent neural networks, specifically bidirectional Long Short-term Memory (BiLSTM), to efficiently capture features across tweets. To the best of our knowledge, our work is the first that develops a recurrent neural model with word embeddings to distinguish Twitter bots from human accounts, that requires no prior knowledge or assumption about users' profiles, friendship networks, or historical behavior on the target account. Moreover, our model does not require any handcrafted features. The preliminary simulation results are very encouraging. Experiments on the cresci-2017 dataset show that our approach can achieve competitive performance compared with existing state-of-the-art bot detection systems.

I. INTRODUCTION

Twitter bots range from benign news and feed accounts to malicious accounts spreading spam, creating a need to distinguish bots from people. The paper proposes a tweet-content-only BiLSTM approach with word embeddings and reports competitive results without profile, network, or behavioral inputs.

  • Twitter’s open structure has attracted bots, which may provide benign updates or spread spam and other malicious content.Source: §I · p. 1
  • Prior bot-detection work includes tweet-text and URL analysis, while many account detectors combine profile, behavior, friendship-network, and timeline features.Source: §I · p. 1
  • The proposed BiLSTM with word embeddings distinguishes bots from human accounts without assumptions about profiles, friendship networks, or target-account history.Source: §I · p. 1
  • BiLSTM’s opposite-direction hidden layers provide information from past and future states, supporting contextual feature capture across tweets.Source: §I · pp. 1–2
  • Word embeddings replace traditional feature engineering and complex NLP tools, which the authors say makes implementation and deployment faster and easier.Source: §I · p. 2
  • Experiments on cresci-2017 show competitive performance against existing work while using only tweet contextual content as input.Source: §I · p. 2

II. RELATED WORK

The related work spans tweet-level spam detection, supervised and unsupervised account detection, and methods that study coordinated behavior or adversarial evasion. Many account-detection methods rely on handcrafted features from accounts and tweets.

II. RELATED WORK

  • Earlier work includes detecting spam tweets or their URLs, while coordinated-behavior studies analyze groups of accounts rather than individual spam accounts.Source: A. Twitter Bot Detection · p. 2

A. Twitter Bot Detection

  • Supervised bot detectors use handcrafted account, network, timing, automation, and content features, with approaches ranging from classifiers to neural networks.Source: A. Twitter Bot Detection · p. 2
  • Unsupervised methods cluster account-and-tweet features or compare behavior, including DeBot’s cross-user activity correlations and Cresci et al.’s digital DNA sequences.Source: A. Twitter Bot Detection · p. 2
  • Adversarial research tests how bots can evade detection, but the authors note it might also inspire bot creators and give them an advantage.Source: A. Twitter Bot Detection · p. 2

B. Recurrent Neural Networks

The paper frames bot detection as text classification and applies recurrent networks to tweet context. Its approach combines bidirectional sequence modeling with word embeddings, including fixed pretrained GloVe vectors and a shared trainable vector for out-of-vocabulary words.

B. Recurrent Neural Networks

  • LSTMs are designed to address long-term dependencies, and BiLSTMs combine forward and backward layers to use preceding and following tokens.Source: B. Recurrent Neural Networks · p. 3

III. OUR PROPOSED APPROACH

A. Word Embeddings

  • Word embeddings represent words as dense numeric vectors, with similar words near one another and vector arithmetic reflecting semantic or syntactic relationships.Source: A. Word Embeddings · p. 3
  • The model uses pretrained 200-dimensional Twitter GloVe vectors as fixed embeddings and a shared trainable vector for out-of-vocabulary words; tweets are tokenized and Twitter markers are mapped to special tokens.Source: A. Word Embeddings · p. 3

B. BiLSTM Neural Networks

The section introduces recurrent sequence models, explains LSTM and bidirectional context, then describes the paper’s BiLSTM classifier and training objective.

B. BiLSTM Neural Networks

  • RNNs process arbitrary embedding sequences with a hidden state updated from the previous state and current input.The hidden state provides internal memory for dynamic temporal behavior.Source: B. BiLSTM Neural Networks · p. 3
  • LSTMs use purpose-built memory cells to address vanishing gradients and exploit long-range context.The cell uses input, forget, and output gates alongside a cell activation vector.Source: B. BiLSTM Neural Networks · p. 4, Figure 2 · p. 4
  • BiLSTM processes tokens in both directions, concatenating forward and backward context representations.One LSTM runs left to right and the other right to left.Source: B. BiLSTM Neural Networks · p. 4, Figure 3 · p. 4
  • The proposed classifier uses three-layer BiLSTMs and a fully connected softmax layer to predict bot or human probabilities.Its input is a token sequence, and the model minimizes negative log likelihood in both directions.Source: B. BiLSTM Neural Networks · p. 4

IV. EXPERIMENTAL RESULTS

The section presents experimental setup and results comparing the neural model with existing bot-detection work.

IV. EXPERIMENTAL RESULTS

  • The section reports experimental setup and results for the proposed neural model.The supplied passage does not state the results themselves.Source: §IV · p. 4
  • The experiments compare the proposed model’s performance with existing work.Source: §IV · p. 4
  • The comparison includes systems associated with,,,, and.Source: §IV · p. 4

A. Existing Systems for Comparison

The comparison emphasizes the feature-engineering demands of existing systems and contrasts them with the proposed content-only RNN.

A. Existing Systems for Comparison

B. Dataset

The evaluation uses the public annotated cresci-2017 dataset and two test sets mixing human accounts with different bot groups.

B. Dataset

  • The cresci-2017 dataset contains 3,474 human accounts with 8.4 million tweets and 1,455 bots with 3 million tweets.The dataset is public and annotated.Source: B. Dataset · p. 5
  • Test set #1 mixes humans with social-bot-1 retweeters of an Italian political candidate; test set #2 mixes humans with social-bot-3 Amazon product spammers.The two test sets were prepared following.Source: B. Dataset · p. 5
  • Test set #1 contains 1,982 accounts and 4,061,598 tweets; test set #2 contains 928 accounts and 2,628,181 tweets.Source: B. Dataset · p. 5

C. Neural Network Model Setup

The model’s selected setup uses stochastic gradient descent with a 0.01 learning rate, a three-layer BiLSTM, and dropout that decreases during training.

D. Evaluation Metrics

The evaluation reports four standard metrics that capture different aspects of bot and human classification, including overall performance and the balance of errors.

  • Precision measures how many accounts predicted as bots are bots, while Recall measures how many actual bots are detected.Source: D. Evaluation Metrics · p. 5
  • Specificity measures correct identification of human accounts, while Accuracy measures the fraction of all users correctly classified.Source: D. Evaluation Metrics · p. 5
  • F-measure combines Precision and Recall, while MCC combines all four confusion-matrix elements and is described as an unbiased version of F-measure.MCC near 1 indicates accurate prediction, near 0 indicates no better than random guessing, and near -1 indicates strong disagreement with the real class.Source: D. Evaluation Metrics · pp. 5–6

E. Results and Discussion

On cresci-2017, the BiLSTM performs competitively with existing systems, while using tweet text without handcrafted features or prior account knowledge. The discussion also contrasts frequent words in human and social-bot-3 tweets.

E. Results and Discussion

  • The model avoids handcrafted features and prior profile, friendship-network, or historical-behavior knowledge by relying on tweet text.Cresci et al. use six user-behavior features and digital DNA sequences; the authors argue that avoiding feature engineering reduces data collection and preprocessing costs.Source: E. Results and Discussion · p. 6
  • The word-cloud discussion reports that social-bot-3 tweets often use exaggerated promotional language, while sampled human accounts commonly describe their owners and seldom link externally.Manual review of 100 randomly selected social-bot-3 tweets found that a majority contained links to external pages.Source: E. Results and Discussion · p. 6, Figure 6 · p. 7
  • The authors attribute the promising preliminary results to bidirectional recurrent modeling with word embeddings and the availability of a large public annotated training set [55].They state that avoiding feature engineering and prior knowledge can save time and money and enable faster, easier implementation and deployment.Source: E. Results and Discussion · pp. 6–7
  • Recall exceeds Cresci et al. by 0.4% absolute on test set #1, and F-Measure exceeds Cresci et al. and Ahmed et al. by 0.3% absolute on test set #2, where Accuracy ties the best.The model reports MCC = 0.920 for test set #1 and MCC = 0.857 for test set #2; most other scores are comparable to prior work.Source: E. Results and Discussion · p. 6, Table II · p. 7

V. CONCLUSION

The paper presents a BiLSTM with word embeddings for distinguishing Twitter bots from humans using tweets alone, without profile, friendship-network, historical-behavior, or handcrafted features. On cresci-2017, it reports performance similar to existing work and says the approach is easier to implement and deploy.

V. CONCLUSION

  • On cresci-2017, the model achieves performance similar to existing work without handcrafted feature engineering.The authors describe handcrafted feature engineering as labor intensive and time consuming.Source: §V · p. 7
  • The BiLSTM with word embeddings distinguishes Twitter bots from human accounts using tweets without assumptions about profiles, friendship networks, or historical behavior.Source: §V · p. 7
  • Avoiding handcrafted features allows faster and easier implementation and deployment of the bot-detection scheme.Source: §V · p. 7
  • The bidirectional recurrent architecture can be adapted to other detection problems, including phishing email, webpages, or SMS.Source: §V · p. 7
Loading 2002.01336v1…