Source-linked AI summary
Characterizing and Detecting Hateful Users on Twitter
Manoel Horta Ribeiro, Pedro H. Calais, Yuri A. Santos, Virgílio A. F. Almeida, Wagner Meira
TL;DR
Content-focused hate-speech detection is limited by noisy, incomplete text and subjective annotation. This paper characterizes users using profile, activity, and network information, then applies graph-based semi-supervised detection, which outperforms traditional approaches.
Problem
Content-focused hate-speech research faces incomplete, noisy text and subjective judgments, while isolated-post analysis can miss profile context.
Method
The paper samples Twitter’s retweet graph, crowdsources user-level annotations, characterizes activity and connections, and applies inductive graph embeddings for semi-supervised detection.
Results
Graph embedding outperforms widely used algorithms for detecting hateful users, while hateful users show distinctive activity, language, centrality, and dense retweet connectivity.
Takeaways & Limitations
User-level analysis provides contextual and network signals for characterizing and detecting hate speech in Twitter moderation workflows.
Takeaways & Limitations
The approach is evaluated only on Twitter, and user-level classification remains subjective and may blur the distinction between individuals and their speech.
Abstract
from arXiv · showhide
Most current approaches to characterize and detect hate speech focus on \textit{content} posted in Online Social Networks. They face shortcomings to collect and annotate hateful speech due to the incompleteness and noisiness of OSN text and the subjectivity of hate speech. These limitations are often aided with constraints that oversimplify the problem, such as considering only tweets containing hate-related words. In this work we partially address these issues by shifting the focus towards \textit{users}. We develop and employ a robust methodology to collect and annotate hateful users which does not depend directly on lexicon and where the users are annotated given their entire profile. This results in a sample of Twitter's retweet graph containing $100,386$ users, out of which $4,972$ were annotated. We also collect the users who were banned in the three months that followed the data collection. We show that hateful users differ from normal ones in terms of their activity patterns, word usage and as well as network structure. We obtain similar results comparing the neighbors of hateful vs. neighbors of normal users and also suspended users vs. active users, increasing the robustness of our analysis. We observe that hateful users are densely connected, and thus formulate the hate speech detection problem as a task of semi-supervised learning over a graph, exploiting the network of connections on Twitter. We find that a node embedding algorithm, which exploits the graph structure, outperforms content-based approaches for the detection of both hateful ($95\%$ AUC vs $88\%$ AUC) and suspended users ($93\%$ AUC vs $88\%$ AUC). Altogether, we present a user-centric view of hate speech, paving the way for better detection and understanding of this relevant and challenging issue.
Introduction
The paper shifts hate-speech analysis from isolated content to users, enabling analysis of activity, connections, and broader profile context. It finds distinctive behavioral, lexical, and network patterns among hateful users and uses dense connectivity to motivate graph-based detection.
- Introduction: User-level analysis incorporates content, activity, connections, and profile context that can clarify hate speech missed by isolated textual features.A profile may reveal abuse through contextual signals such as neo-Nazi imagery, even when a tweet lacks hate-related words.
- Introduction: Hateful users tweet more frequently, follow more people daily, and have more recent, shorter-lived accounts.The activity differences are also reported for related neighborhood and suspension comparisons.
- Introduction: Hateful users are more central in the sampled retweet network despite having fewer followers, contradicting the “lone wolves” stereotype.Similar centrality patterns appear among hateful users’ neighbors and suspended users.
- Introduction: Their vocabulary differs from normal users: hate-, anger-, and politics-related words occur less often, while masculinity, love, and curses occur more often.This challenges collection strategies that rely directly on hate-related words.
Background
The paper defines hateful users using Twitter’s hateful-conduct guidelines and models influence through a directed retweet graph. It also uses explicit abusive language and suspended accounts as related but distinct signals.
- Background: A hateful user is one whose annotators judge that they endorse content promoting violence against or attacking protected groups under Twitter’s guidelines.The guideline categories include race, ethnicity, nationality, sexual orientation, gender, religion, age, disability, and disease.
- Background: The retweet graph represents users as nodes and retweets as directed edges, with edge directions inverted because influence flows opposite to retweeting.The graph is used to model influence from retweeted users toward users who retweet them.
- Background: Explicit abusive language is unambiguous in its potential abusiveness, such as racial or homophobic slurs, but does not itself imply hate speech.The paper notes a correlation between explicit abusive language and hate speech without treating them as equivalent.
- Background: Suspended accounts provide an additional proxy for potentially hateful behavior, although most Twitter suspensions are attributed to spam.The study compares suspensions before and after a guideline change, including possible off-platform behavior under the newer guidelines.
- Background: The diffusion process starts from the sampled retweet graph, reverses influence edges, adds self-loops, marks lexicon users, and iteratively updates other users’ beliefs.This procedure supplies proximity values used for sampling users in the broader collection methodology.
Data Collection
The data-collection pipeline samples Twitter’s retweet network, uses a targeted lexicon with graph diffusion to stratify users, and manually annotates their full profiles. This design reaches users near lexicon users even when they did not use the lexicon themselves.
- Data Collection: The crawler samples Twitter’s retweet network despite API limitations that expose outgoing edges and prevent unbiased in-degree estimation without broad graph sampling.The study therefore uses a Direct Unbiased Random Walk algorithm to estimate out-degree distributions efficiently.
- Data Collection: The pipeline combines a hate-related lexicon, DeGroot-based diffusion, and four belief strata instead of sampling only tweets with obvious hate features.This balances the risk of too little hate speech under uniform sampling against lexical bias from narrowly targeted sampling.
- Data Collection: The lexicon contains 23 terms mostly used in hate-speech contexts, including “holohoax,” “racial treason,” and “white genocide.”The selection draws on Hatebase.org and the ADL hate symbol database.
- Data Collection: Diffusion constructs a row-normalized transition matrix after reversing retweet edges and adding self-loops, then updates beliefs through repeated matrix multiplication.Users receive belief values p_i in [0, 1], and sampling selects up to 1500 users from each of four intervals.
- Data Collection: Crowdsourcing annotates users as hateful or normal from their entire profiles rather than isolated tweets or words.Each profile receives three independent annotations, with up to five when annotators disagree; 544 hateful and 4,427 normal users are identified.
Characterizing Hateful Users
Hateful users differ from normal users in activity, vocabulary, sentiment, and network structure, while comparable patterns among neighbors and suspended users reinforce the findings. They are densely connected in the retweet graph, supporting graph-based detection.
- Study design: The study compares annotated hateful users with normal users, their respective neighbors, and suspended with active accounts to reinforce its characterization.The authors treat these paired populations as proxies for hateful speech and use the comparisons to increase robustness.
- Activity patterns: Hateful users tweet more frequently, follow more users daily, and have newer, shorter-lived accounts than normal users.These activity and account-age differences also appear in several comparisons involving their neighborhoods and suspended accounts.
- Spammer-related behavior: Hateful users use fewer URLs and hashtags per tweet, suggesting they do not behave like spammers.The lower URL and hashtag rates are statistically significant, and the broader analysis reports similar patterns for neighboring and suspended users.
- Network structure: The median hateful user is more central across network measures, despite having fewer followers, and hateful users have higher average out-degree.Influential normal users distort some average centrality comparisons, while similar neighbor and suspended-user patterns support the median finding.
- Vocabulary and sentiment: Hateful users use less hate-, anger-, shame-, terrorism-, violence-, and sadness-related vocabulary, but more emotional and swearing-related words.The analyzed Empath categories were selected beforehand, and error bars represent 95% confidence intervals.
- Vocabulary and sentiment: Hateful and suspended users are more negative, while hateful users are less subjective and use more profanity than counterparts.Neighbors of hateful users are also more negative, although they are not less subjective.
- Suspension patterns: After Twitter’s guideline change, banning increased from approximately 6.45 to 9.05 users per day, while banned users showed little difference in vocabulary, activity, or network structure.The analysis reports no statistically significant differences except for government-related words.
- Network structure: 41% of hateful-user retweets target other hateful users, making such retweets 71 times more likely than expected from their graph prevalence.Suspended users show a corresponding pattern: 7% of their retweets target suspended users, approximately 11 times their sampled prevalence.
Detecting Hateful Users
The paper detects hateful and suspended users using user activity, network, and text features, comparing traditional boosting models with semi-supervised GraphSage. The node-embedding approach performs best across assessed metrics, and exploiting the retweet network improves detection in both tasks.
- Features: User features include statuses, followers, followees, favorites, and network-centrality measures such as betweenness, eigenvector centrality, and node degree.These activity and network attributes are used as features for user-level classification.
- Features: GloVe features are formed by averaging 300-dimensional word representations across each user’s tweets.The representations come from spaCy’s off-the-shelf GloVe vectors.
- Models: The experiments compare Gradient Boosted Trees and AdaBoost with semi-supervised GraphSage, which aggregates features from sampled local neighborhoods.GraphSage learns node embeddings while using neighborhood information even when neighboring users are unlabeled.
- Results: GraphSage using both user and GloVe features achieves the best results across all assessed metrics for detecting hateful and suspended users.The evaluation uses five-fold cross-validation and reports F1-score, accuracy, and AUC.
- Results: Retweet-network information improves detection of both hateful and suspended users, indicating the gain is independent of the annotation process.User features alone make little difference in many settings, including identical AUC in some Gradient Boosting comparisons.
Related Work
Prior hate-speech research primarily samples and represents content, while this paper reviews approaches to data collection, annotation, feature extraction, and classification.
- Prior Studies: The reviewed literature includes studies characterizing hateful websites, hate-group organization, and hate targets across multiple online social networks.
- Data Collection: Previous studies commonly collect hate-speech data using lexicons, expanded co-occurrence terms, regular expressions, or tweets from known hate-speech users.
- Annotation: Human annotators are widely used, but hate speech is difficult to label because interpretation depends on context and subjective judgments.This paper gives annotators entire user profiles rather than individual tweets to provide more context.
- Features: Existing work represents online content with n-grams, bag-of-words, and word embeddings, alongside linguistic features such as part-of-speech and sentiment information.
- Classification: Researchers have applied supervised classifiers including Naive Bayes, logistic regression, support vector machines, rule-based classifiers, random forests, and gradient-boosted decision trees.
Conclusion and Discussion
The paper presents user-level hate-speech characterization and graph-based detection, finding differences in activity, language, and network structure. It also emphasizes unresolved limits involving platform scope, subjectivity, individual-versus-speech distinctions, and automated moderation bias.
- Method and Contribution: The proposed methodology samples Twitter’s retweet graph, identifies lexicon-associated users, applies diffusion to find neighbors, and crowdsources profile-level annotations.It annotated 4,988 users, including 544 considered hateful.
- Findings: Hateful users differ from normal users in activity patterns, account age, language, centrality, and retweet-network connectivity.Reported differences include more recent accounts, more negative sentences, and greater centrality among hateful users.
- Detection: Dense connections among hateful users motivate inductive graph embeddings, which outperform widely used algorithms such as Gradient Boosted Trees for detection.
- Limitations: The study is limited to Twitter, and hateful-user classification remains difficult because hate speech is subjective and annotators may disagree.
- Limitations: User-level models can blur distinctions between individuals and their speech, especially when suspension decisions incorporate contextual factors beyond specific platform content.The paper proposes using such models as a first step before human or more specific assessment.
- Limitations: Automated moderation raises concerns about accidental or intentional bias, fairness, accountability, transparency, and potential effects on free speech.