PaperlayerView exact revision on arXiv

Source-linked AI summary

Detecting Cyberbullying and Cyberaggression in Social Media

Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Emiliano De Cristofaro, Gianluca Stringhini, Athena Vakali, Nicolas Kourtellis

arXiv:1907.08873v1cs.SIcs.CYcs.IR

TL;DR

Online abuse takes varied forms that complicate detection, and the paper examines behavioral differences among Twitter users while developing methods to identify abusive accounts. Its analysis finds network features especially useful for classifying aggression and reports changing suspension rates across account snapshots.

  • Problem

    Online abuse varies across users and can be transient, anonymous, or expressed through sarcasm and trolling, complicating effective detection.

    Source: §1.1 · p. 1

  • Method

    Crowdworkers labeled 1.5k users from roughly 10k tweets, and the study analyzed 38 user-, text-, and network-based features to characterize abusive behavior.

    Source: §9 · p. 29

  • Results

    Half of the top-12 features for aggression classification were network-based, while tweet characteristics helped distinguish aggressive, bully, and normal users when spam was removed.

    Source: §9 · p. 30

  • Takeaways & Limitations

    Suspension rates increased between the November 2016, December 2017, and September 2018 snapshots for aggressive and bully users, especially bullies.

    Source: §9 · p. 30

    More extensive study is needed to draw concrete conclusions about BBCpay and compare it with Gamergate, since BBCpay was newer and less organized.

    Source: §2.3.2 · p. 6, §3.2 · p. 10

Abstract

from arXiv · show

Cyberbullying and cyberaggression are increasingly worrisome phenomena affecting people across all demographics. More than half of young social media users worldwide have been exposed to such prolonged and/or coordinated digital harassment. Victims can experience a wide range of emotions, with negative consequences such as embarrassment, depression, isolation from other community members, which embed the risk to lead to even more critical consequences, such as suicide attempts. In this work, we take the first concrete steps to understand the characteristics of abusive behavior in Twitter, one of today's largest social media platforms. We analyze 1.2 million users and 2.1 million tweets, comparing users participating in discussions around seemingly normal topics like the NBA, to those more likely to be hate-related, such as the Gamergate controversy, or the gender pay inequality at the BBC station. We also explore specific manifestations of abusive behavior, i.e., cyberbullying and cyberaggression, in one of the hate-related communities (Gamergate). We present a robust methodology to distinguish bullies and aggressors from normal Twitter users by considering text, user, and network-based attributes. Using various state-of-the-art machine learning algorithms, we classify these accounts with over 90% accuracy and AUC. Finally, we discuss the current status of Twitter user accounts marked as abusive by our methodology, and study the performance of potential mechanisms that can be used by Twitter to suspend users in the future.

1 Introduction

The introduction frames cyberbullying and cyberaggression as widespread online harms that are difficult to define and detect. The paper studies user behavior across Twitter communities, develops methods to identify abusive users, and examines Twitter account suspensions.

1 Introduction

  • Cyberbullying is characterized by repeated hostile behavior, while cyberaggression is intentional harm delivered electronically; repetition and power imbalance indicate cyberbullying.The passage notes that cyberaggression is often considered a form of cyberbullying, though no generally accepted definition exists.Source: §1 · p. 1
  • 41% of Americans surveyed in 2017 reported personal online harassment, while 66% had witnessed it directed at others.The introduction also reports that about 50% of young social media users said they had been bullied online.Source: §1 · p. 1

1.1 Challenges

  • Detection is challenging because abuse varies across users, can emerge unpredictably, may exploit anonymity, and includes sarcasm or trolling beyond obvious abusive language.Twitter adds short, informal text, limited tweet-level context, and spam behavior that can resemble aggression or bullying.Source: §1.1 · pp. 1–2

1.2 Contributions

  • The study compares user activity and emotional attributes across established, hate-related, and newer topic communities to characterize patterns associated with abuse.The communities include NBA, Gamergate, and gender-pay-inequality discussions.Source: §1.2 · p. 2
  • The authors build a labeled dataset and train traditional classifiers and deep neural networks on behavioral attributes to distinguish bullies and aggressors from normal users.The classification contribution is framed as addressing the second research question on automatic detection.Source: §1.2 · p. 2
  • The paper also compares abusive labels with Twitter account statuses and tests whether machine-learning methods can approximate Twitter’s suspension mechanism.Source: §1.2 · p. 2

1.3 Roadmap

  • The roadmap covers data collection and community analysis, then session-based crowdsourced labeling, feature analysis, classification, and account-status modeling.It closes with related work and discussion of the study’s findings.Source: §1.3 · pp. 2–3

2 Datasets

The section describes how the study assembled and cleaned Twitter datasets spanning topic-focused communities and a random baseline, then compares their activity patterns and discussed topics.

2 Datasets

  • Twitter Streaming API data yielded two abuse-prone datasets with 700k tweets total and two expected-normal datasets with 1.4M tweets total.Gamergate and BBCpay comprise the first group; NBA and the random baseline comprise the second.Source: §2 · p. 3

2.1 Data Collection

  • Data collection began with manually selected seed keywords and expanded the filtering list over time using frequently occurring keywords from collected posts.The study used #GamerGate, #BBCpay, and #NBA as dataset-specific seeds, then maintained dynamic keyword lists.Source: §2.1 · pp. 3–4

2.2 Data Preprocessing

  • Preprocessing removed noisy text and spam, excluding users averaging more than 5 hashtags or an intra-tweet similarity above 0.8.The similarity cutoff excluded about 5% of users; the authors report the spam-removal approach was suitable for removing at least highly spam content.Source: §2.2 · pp. 4–5

2.3 Datasets Overview

  • Across the Gamergate, BBCpay, and NBA communities, posting activity followed a power-law pattern, while CCDF plots used equal-sized random samples of 64k users.For topic extraction, the authors tested 2–15 LDA topics and selected at most five as yielding distinct topics.Source: §2.3 · p. 5, Figure 2 · p. 5

2.3.1 Gamergate Controversy

  • Gamergate topics included Black Lives Matter, political issues, and hostility toward Hillary Clinton; top contributors also discussed Black Lives Matter and Clinton.The paper describes Gamergate as a mature community associated with organized harassment and uses it for later cyberbullying analysis.Source: §2.3.1 · p. 6, Table 3 · p. 6

2.3.2 BBC gender pay controversy (BBCpay)

  • BBCpay discussions featured gender salary inequality and Gary Lineker’s pay alongside Brexit, while the authors speculated that its newer, less organized community would have lower aggressive intensity than Gamergate.Source: §2.3.2 · p. 6, Table 5 · p. 6

2.3.3 NBA & Baseline

  • The baseline covered varied topics, while NBA discussions included sports, brands, and events; both were considered less hate-related than Gamergate and BBCpay.NBA also provided a topic-focused community for comparison with the random baseline and with strong but less hate-related groups.Source: §2.3.3 · pp. 6–7, Table 6 · p. 7, Table 7 · p. 7

3 A systematic analysis of users behavior traits

This section compares activity and emotional traits across Gamergate, BBCpay, NBA, and baseline Twitter users, using two-sample Kolmogorov–Smirnov tests and reporting statistically significant differences. The findings distinguish community activity from emotional expression: Gamergate users are highly engaged but do not uniformly display anger.

3.1 Activity Characteristics

  • Gamergate users had older accounts than baseline users (D = 0.20), posted more tweets (D = 0.35), and used more hashtags (D = 0.26).The Gamergate and BBCpay account-age distributions were also similar (D = 0.18).Source: §3.1 · p. 7, Figure 3 · p. 8, Table 8 · p. 8
  • Gamergate users resembled baseline users at the median for favorites but had almost twice their mean; they posted 1–2 more URLs at the median (D = 0.27).NBA and BBCpay users’ favoriting behavior was more similar to baseline users.Source: §3.1 · pp. 7–8, Figure 3 · p. 8, Table 8 · p. 8
  • Gamergate and baseline users’ mention activity was similar (mean difference 0.014), while BBCpay and NBA users used fewer mentions (D = 0.032 and D = 0.083, respectively).The differences from Gamergate were statistically significant for both BBCpay and NBA users.Source: Figure 3 · p. 8, §3.1 · p. 9
  • Gamergate users had almost 70% more followers and 23% more friends than baseline users; NBA users had the most followers, while Gamergate users had the most friends.BBCpay users were more similar to NBA users on both measures.Source: Figure 4 · p. 8, Table 8 · p. 8, §3.1 · p. 9

3.2 Emotional Characteristics

  • About 25% of Gamergate tweets were negative versus 15% of baseline tweets; Gamergate users were less joyful (D = 0.04) and more offensive than baseline and NBA users, but not significantly more than BBCpay users.The joy distributions did not differ significantly for the other five primary emotions; BBCpay users were less joyful than the other groups.Source: §3.2 · pp. 9–10, Figure 5 · p. 10
  • Gamergate and baseline users used emoticons similarly (D = 0.028, p = 0.96), while Gamergate users used uppercase less than baseline (D = 0.212); BBCpay resembled Gamergate and NBA resembled baseline (D = 0.06 and D = 0.04).Baseline users used emoticons more than BBCpay and NBA users, with subtle but statistically significant differences (D = 0.006 and D = 0.01, respectively).Source: §3.2 · p. 10
  • Overall, Gamergate users showed familiarity with Twitter as a broadcasting mechanism, sharing more favorites, URLs, and hashtags and having more followers and friends than baseline users.Their tweets were less joyful, despite the aggressive and hateful subject matter described in the analysis.Source: §3.2 · p. 10
  • The BBCpay comparison remains preliminary: the authors call for a dedicated, more extensive study to examine the topic’s dynamics and how its community matured and organized.The paper notes that BBCpay sometimes resembles Gamergate but differs on other measures.Source: §3.2 · pp. 10–11

4 Ground Truth Building for Cyberbullying Detection

The section builds a crowdsourced ground truth for classifying Twitter users as bullies, aggressors, spammers, or normal. It focuses on Gamergate and baseline users, grouping tweets into sessions and batches for human review.

4 Ground Truth Building for Cyberbullying Detection

  • Gamergate and baseline users were selected because Gamergate users appeared more aggressive and active, although the authors caution that not all Gamergate activity was abusive.The stated aim was to study user behavior classified as aggressive or normal and support further manual investigation.Source: §4 · p. 11

4.1 Preparing Data for Crowdsourcing

  • An 8-hour interarrival threshold defines sessions; 5–10-tweet batches achieved the best labeling agreement, yielding 1,500 batches after shorter sessions were excluded.Users with fewer than five tweets in the three-month period were removed before sessionization.Source: §4.1 · pp. 11–12

4.2 Crowdsourced Labeling

  • Workers labeled users, not individual tweets; an aggressor needed at least one harmful negative post, while a bully required repeated harmful posts on the same topic.Five workers labeled each batch, with majority vote determining its final label.Source: §4.2 · p. 12
  • Workers received batches, user profile descriptions when available, and behavior definitions; each worker labeled ten batches, including a control case.The task was delivered through an online survey tool developed by the authors.Source: §4.2 · pp. 12–13
  • 1,303 of 1,500 batches reached majority consensus across 9,484 tweets; labels were 4.5% bullies, 3.4% aggressors, 31.8% spammers, and 60.3% normal.Abusive users—bullies and aggressors together—made up about 8% of the dataset.Source: §4.2 · p. 12
  • 0.54 Fleiss’ κ indicated moderate inter-rater agreement; control-case accuracy was 67% overall (54% bully, 61% aggressive, and 84% spam).The authors note that moderate agreement reflects the difficulty of identifying bullying and aggression even by manual inspection.Source: §4.2 · p. 13

5 Feature Extraction and Selection

The section examines user-, text-, and network-based attributes to identify differences among bullies, aggressors, spammers, and normal users. The analysis is intended to inform feature selection for later classification.

5 Feature Extraction and Selection

  • The analysis considers user, text, and network attributes as candidate classification features, examining how those features vary across the four user categories.The authors use the feature analysis to inform later machine-learning modeling of user behavior.Source: §5 · p. 13

5.1 What do Bullies, Aggressors, Spammers, and Normal Users Post About?

  • Bullies and aggressors discussed different topics from normal users; bully topics included sensitive issues, while aggressive users expressed negativity on popular topics.Spammers’ topics often involved inappropriate content or attempts to attract followers; both abusive groups tended to use many hashtags to reach others or build communities.Source: §5.1 · pp. 13–14, Table 11 · p. 14, Table 12 · p. 14

5.2 User-based Features

  • Bullies and aggressors had less waiting time between posts than spammers and normal users, while their session-size distributions did not differ significantly from normal users.Normal users subscribed to more lists than other user categories.Source: §5.2 · p. 14

5.3 Text-based Features

  • Word2Vec distributions did not significantly separate abusive users from normal users; sentiment distinguished aggressive users but not bullies, and hate- and curse-word lists did not significantly distinguish abusive users.The authors caution that short tweets, modified words, URLs, and emoticons limit the suitability of such word lists for classification.Source: §5.3 · p. 15
  • Bullies used fewer adjectives and adverbs but longer words; aggressive users wrote more words, while bullies and aggressors used more hashtags and normal users fewer URLs.The authors suggest that bullies may favor direct attacks without additional descriptive language.Source: §5.3 · pp. 14–15

5.4 Network-based Features

  • Bullies had fewer followers than other classes and lower hub scores; bullies and aggressors had higher reciprocity than normal users and spammers.Aggressors’ network influence resembled that of normal users, while bullies’ resembled spammers.Source: §5.4 · pp. 16–17, Figure 9 · p. 17
  • Bully-versus-normal power difference was statistically significant, unlike the aggressive-versus-normal comparison; bullies and spammers were less prone to form clusters than aggressive and normal users.Louvain-based community features distinguished bullies from normal users, but not aggressors from normal users.Source: §5.4 · pp. 17–18

6 Machine Learning for Bullying & Aggression Detection

This section evaluates machine-learning approaches for distinguishing bullying, aggression, spam, and normal users across four classification setups, examining how results vary with target and spam handling. It also considers model choice for automated or human-reviewed detection and the contribution of user/network activity and linguistic-style features.

6.1 Experimental Setup

  • The neural classifier combines raw text with user-, text-, and network-based metadata, using a word-level RNN with GRUs and attention and alternating updates to the text and metadata paths.The two paths are concatenated for classification; interleaved training updates only one path per mini-batch to avoid imbalanced convergence and unwanted interactions.Source: §6.1 · pp. 19–20

6.2 Experimental Methodology

  • The experiments test four-class, three-class after spam removal, three-class offensive, and two-class offensive classification, using repeated 10-fold cross-validation and precision, recall, F1, AUC, and accuracy.The three-class offensive setup merges bully and aggressive labels; the two-class offensive setup also removes spam.Source: §6.2 · p. 21

6.3 Classification Results

  • In four-class classification, Naive Bayes detected 90.2% of bullies but only 19.8% of aggressive users; Random Forest achieved 72.9% precision, 74.6% recall, and 74.1% accuracy.The ensemble had the highest precision for aggressive users, an 18% increase over the result in [14]; Random Forest balanced precision and recall across classes and ran faster than the ensemble.Source: §6.3 · p. 21, Table 14 · p. 21
  • For four-class classification, the neural network detected 64% of bullies and 30% of aggressive users, with 70% precision, 63% recall, and 63.1% accuracy.Its aggressive-user detection rate exceeded Naive Bayes by 10.2%.Source: Table 14 · p. 21, §6.3 · p. 22
  • After spam removal, the ensemble achieved 90.2% precision, 97.1% recall, and 90.2% accuracy across bully, aggressive, and normal classes.Random Forest improved bully precision by 13.5% and recall by 12.2%; the confusion matrix showed 60% of aggressive cases labeled as bullies, while 30.89% of bullies were labeled aggressive and 7.68% normal.Source: §6.3 · p. 22, Table 16 · p. 23
  • When bullying and aggression were merged as offensive, Random Forest achieved the highest overall ROC area at 82.5%, while Naive Bayes had the highest offensive-class recall at 91.4%.Random Forest had the highest offensive-class precision (58.8%); the neural network detected 69% of offensive users.Source: §6.3 · p. 22
  • After spam removal, Random Forest achieved 93.4% precision, 93.7% recall, and 91.1% AUC for offensive-versus-normal classification.Naive Bayes detected the most offensive users (93.7%), while Random Forest detected 97.2% of normal users.Source: §6.3 · p. 22

6.4 Machine Learning Classification Takeaways

  • The authors characterize results across the four setups as reasonable based on their overall ROC AUC performance.They describe AUC as the probability that a classifier ranks a random positive case above a random negative case.Source: §6.4 · pp. 22–23
  • For aggressor detection after spam removal, Random Forest false-positive rates were 0.003 in the 4-class setup and 0.009 in the 3-class setup; model choice should reflect whether human review can reduce false positives.The authors suggest higher-precision models for fully automated systems and higher-recall models when humans review flagged users; they cite Random Forest for bully detection without human intervention and neural networks for aggressor detection with human monitoring in the 4-class setup.Source: §6.4 · p. 23
  • User- and network-based activity and connectivity features contributed most overall; verbs ranked highly in 4- and 3-class offensive experiments, while word-length and other style features mattered in the 3-class spam-removed and 2-class setups.In the 2-class experiment, average words per sentence and adjective count were also important; the authors report that linguistic style can help distinguish user behaviors.Source: §6.4 · p. 23
  • Random Forest was used for the reported confusion matrices and feature rankings because it balanced precision and recall across classes and had the highest overall AUC in three of four experimental phases.Source: §6.4 · p. 23

7 Abusive Behavior on Twitter: User vs. Platform Reaction

The section compares Twitter account statuses across communities, examines how active, suspended, and deleted users differ, and tests whether user features can predict status. It also tracks how the status of annotated offensive users changed across three snapshots.

7.1 Users Communities vs. Twitter Statuses

  • Gamergate users were suspended more often than baseline users (11.29% vs. 8.43%), while baseline users were deleted more often (25.86% vs. 16.22%).NBA and BBCpay users had higher active-account percentages and lower deletion rates than Gamergate and baseline users; their suspension rates were above baseline and closer to Gamergate’s.Source: §7.1 · p. 24, Table 21 · p. 25

7.2 Active, Suspended, and Deleted User Accounts

  • Suspended users had been on Twitter for less time than deleted users but had more followers and friends, with significant distribution differences.The reported Kolmogorov-Smirnov differences were D = 0.21404 for account age, D = 0.26208 for followers, and D = 0.22731 for friends.Source: §7.2 · p. 24, Figure 11 · p. 25
  • Suspended users posted more tweets and joined more lists than deleted users, while deleted users favorited more tweets than suspended and active users.Deleted and suspended users had similar negative sentiment; active users were most positive, while suspended users used more adjectives than active and deleted users.Source: §7.2 · pp. 24–25, Figure 11 · p. 25
  • Suspended users used more hashtags and URLs than deleted users, while active users used the fewest of both; deleted users’ hashtags often concerned social issues.Suspended users’ popular hashtags included Brexit and spam-related material, while active users discussed varied topics and sometimes posted aggressive or inappropriate content.Source: Figure 11 · p. 25, §7.2 · p. 26

7.3 Emulating the Twitter Suspension Engine

  • The status classifier achieved 97.6% accuracy and 99.5% AUC, detecting 87.3% of deleted accounts and 82.1% of suspended accounts.Misclassified deleted and suspended cases mostly fell into the active class, while the deleted and suspended classes were almost clearly separable.Source: §7.3 · pp. 26–27, Table 23 · p. 27, Table 24 · p. 27

7.4 Suppression of Offensive Behavior by Twitter

  • In September 2018, Twitter had suspended 55.17% of labeled bullies and 51.16% of aggressors; only 5.17% and 25.58%, respectively, remained active.In December 2017, 55.17% of labeled bullies and 53.49% of aggressors were still active; deleted-account rates changed little across snapshots.Source: §7.4 · p. 27, Table 26 · p. 27
  • From November 2016 to September 2018, suspension rates rose by 37.21% for aggressive users and 55.17% for bullies; spam suspension remained 39% in 2018.The authors interpret the high offensive-user suspension rate as evidence that Twitter increased efforts to address abusive behavior; the 2018 spam rate was below those for bullies and aggressors.Source: §7.4 · pp. 27–28, Table 26 · p. 27

8 Related Work

Prior work has detected abusive behavior across social media, gaming, and other online platforms using text, user, and network features. This paper extends that work with a Twitter-focused analysis and detection approach.

Abusive Behavior in Social Media

  • Researchers used word embeddings for hate-related Yahoo Finance comments, varied features for offensive YouTube comments, and machine learning for Yahoo Finance and News hate speech.Work on Yahoo Answers examined reported rule violations and prediction of user suspension [51].Source: §8 · p. 28

Cyberbullying in Social Media

  • Cyberbullying studies addressed sensitive topics in YouTube comments, bully-user scoring on YouTube, fine-grained bullying on Ask.fm, and gender bullying on Twitter.Other work combined Instagram comments and images to distinguish bullying from aggression, or detected bullying words and rumors on Twitter.Source: §8 · p. 28

Abusive Incidents in Game Communities / Abusive Detection Methods

  • Prior work also studied toxic behavior and sexism in online games [57], while abusive-detection methods used diverse supervised and graph-based approaches and text, user, and network features [47].This article extends its earlier Twitter work [15] [13] [12] by extracting user, text, and network features, analyzing tweets individually and in groups, comparing machine-learning methods, and examining platform statuses over time.Source: §8 · p. 28

9 Discussion & Conclusion

The study compares user behavior across Twitter communities and examines detection of cyberbullying and cyberaggression, then relates abusive-user profiles to account status over time. Its findings highlight network features for identifying aggression and later suspensions of many accounts labeled as abusive.

RQ1

  • Topic-community users had longer-running accounts than random users; Gamergate was more active and aggressive than BBCpay and NBA, while BBCpay posts also had aggressive connotations.The communities discussed varied topics, with users especially focusing on their community’s named issue.Source: §9 · p. 29
  • Bullies targeted sensitive social issues and were less popular and central; aggressors showed high reciprocity and centrality closer to normal users.Aggressors also displayed more negative sentiment than bullies or normal users, while bullies used aggressive and sometimes insulting language.Source: §9 · p. 29

RQ2

  • Crowdworkers labeled 1.5k users across four categories, and the analysis examined 38 user-, text-, and network-based features, including expanded network measures.The network measures included reciprocity, centrality, and community-related features; new text features included part-of-speech and stylistic attributes.Source: §9 · pp. 29–30
  • Traditional methods usually outperformed neural networks in accuracy and training time on limited annotated data; network features were most effective for aggression, comprising half the top 12 features.The top-12 count refers to features ranked by discriminatory power.Source: §9 · p. 30

RQ3

  • Gamergate and baseline users more often deleted accounts by choice than NBA or BBCpay users, while the study also tested whether features could identify likely suspensions or voluntary deletions.Gamergate users were suspended more often than baseline users.Source: §9 · p. 30
  • Suspended Gamergate users expressed more aggression but were more popular and active; deleted users had fewer friends and followers, and popularity could have delayed suspension.Source: §9 · p. 30
  • ∼5% of bully users remained active about two years later, as suspension rates rose from November 2016 through September 2018, especially for bullies.Earlier snapshots showed bullies more often deleting accounts while aggressors were suspended more often than bullies or normal users.Source: §9 · p. 30

Future Directions

  • Future work proposes testing the approach on other social platforms, adding linguistic features such as sarcasm, and developing real-time detection.The proposed real-time direction includes distributed stream and parallel-processing engines.Source: §9 · pp. 30–31
Loading 1907.08873v1…