Source-linked AI summary
A Web of Hate: Tackling Hateful Speech in Online Social Spaces
Haji Mohammad Saleem, Kelly P Dillon, Susan Benesch, Derek Ruths
TL;DR
Online platforms lack reliable hateful-speech detection, while keyword methods miss indirect hatred and confuse hateful with supportive language. The paper trains classifiers from self-identifying hateful communities, achieving strong performance within and across platforms. The authors note that off-topic content in hateful communities introduces training noise.
Problem
Reliable hateful-speech detection is lacking, and keyword methods can miss indirect expressions while confusing hateful and support-community language.
Method
The paper uses language produced by self-identifying hateful communities as training data for hateful-speech classifiers.
Results
Community-based classifiers show higher precision than keyword-based methods and perform similarly when tested across platforms and on Reddit.
Takeaways & Limitations
Community-based training offers a way to detect hateful speech without relying solely on manual annotation or keyword searches.
Takeaways & Limitations
Content in hateful communities is not always hateful, introducing training noise and classification errors.
Abstract
from arXiv · showhide
Online social platforms are beset with hateful speech - content that expresses hatred for a person or group of people. Such content can frighten, intimidate, or silence platform users, and some of it can inspire other users to commit violence. Despite widespread recognition of the problems posed by such content, reliable solutions even for detecting hateful speech are lacking. In the present work, we establish why keyword-based methods are insufficient for detection. We then propose an approach to detecting hateful speech that uses content produced by self-identifying hateful communities as training data. Our approach bypasses the expensive annotation process often required to train keyword systems and performs well across several established platforms, making substantial improvements over current state-of-the-art approaches.
1. Introduction
Hateful speech is a widespread problem across online platforms, yet reliable detection remains lacking. The paper examines keyword-based limitations and proposes community-derived language models that work within and across platforms.
- Hateful speech affects diverse online platforms and can alienate users, support radicalization, and incite violence.
- Reliable detection is lacking, while user reporting requires labor-intensive review and can miss unreported harmful content.
- The paper identifies factors behind poor keyword-based performance and proposes language models derived from online communities.
- The proposed models perform well within platforms and across platforms, which the authors describe as a first achievement.
2. Background
The paper distinguishes hateful speech from broader or ambiguous forms of harmful expression by focusing on expressed group-based hatred. It grounds detection in the linguistic practices of self-identifying hateful communities.
- Hate speech lacks a consensus definition because social norms, context, and individual or collective interpretation affect its meaning.
- The term “hateful speech” focuses on expressed hatred against people based on group identity rather than the speaker’s intent.
- Hateful speech differs from cyber-bullying because it denigrates people based on group membership, whereas cyber-bullying centers on repetitive aggression and power imbalance.
- Hateful and support communities can share topics while using characteristic language conventions with different intentions.
- The study treats linguistic attributes of communities committed to degrading a target group as a principled definition of that hateful speech.
- Existing automated-detection research was limited, and one racism study found annotators agreed on purported hateful texts only 33% of the time.
3. The limits of keyword-based approaches
Keyword methods confuse hateful and supportive discussions because they share vocabulary and miss indirect expressions of hatred. The paper uses Reddit community data and topic modeling to expose this limitation.
- Hateful and support communities discuss similar topics, creating shared vocabulary that can confuse classifiers.
- Keywords miss hateful expressions that avoid standard slurs, such as calling migrants “parasites” or African-Americans “animals” in context.
- The study selects active hate and support subreddits for African-American, plus-sized, and women target groups from Reddit data.
- Labeled Latent Dirichlet Allocation learns subreddit-characterizing topics against a 460,000-comment random Reddit baseline.
- Topic terms align with target and support communities, while hate and support subreddits show substantial vocabulary overlap.
- Vocabulary overlap measured by Jaccard Index was 0.28 for black targets, 0.50 for female targets, and 0.76 for plus-size targets.
4. A community-driven model of hateful speech
The paper defines hateful speech through language produced by self-identifying hateful communities, avoiding manual annotation and keyword-based training data. Community-trained classifiers distinguish hateful content from random and support-community content, outperform keyword baselines, transfer across similar platforms, but remain target-specific and include community noise.
- Community-driven definition: Community language provides training data intended to avoid manual annotation and keyword searches while empirically defining hateful speech.The approach treats linguistic practices of communities committed to degrading a target group as a principled basis for classification.
- Baseline comparison: Community-based classifiers outperform keyword-based classifiers in all considered comparisons, with precision gains of 10% to 20%.The improvement reflects fewer incorrect hateful-speech detections, while community-based models also identify hateful posts that omit specialized slurs.
- Community discrimination: Community-based models distinguish hateful-community posts from both random Reddit comments and support-community comments despite shared vocabulary.The study explicitly evaluates the distinction between hate and support communities because topical overlap can make keywords misleading.
- Cross-platform transfer: The approach performs similarly across Reddit and other forum-based platforms with unrestricted posts, discussion threads, and clear textual context.The authors caution that substantially different platforms, especially short-text systems, may not generalize similarly.
- Target specificity: Hateful-speech classifiers perform poorly across different targets, indicating that training must remain relevant to the target community.The paper links this limitation to target-related themes in hateful conversations.
- Error analysis: Community-based training introduces noise because not every post in a hateful community is hateful, increasing some Type I errors.The authors describe this as an acceptable trade-off at this stage and identify narrower hateful-content selection as future work.
- Error analysis: Most Type II errors were non-racist posts correctly labeled non-hateful, suggesting reported performance may underestimate the method’s actual performance.Some missed hateful posts targeted groups other than the classifier’s specified target.
- Imbalanced data: Preliminary tests on hateful-to-non-hateful ratios of 1:10, 1:100, and 1:1000 produced results similar to balanced-test performance but require further analysis.These tests examine whether the method remains effective when hateful content is much less prevalent.
5. Conclusion
The paper identifies why keyword-based hateful-speech detection performs poorly and proposes self-identified hateful communities as training data, producing less labor-intensive and superior classifiers.
- Shared vocabulary between hateful and support communities causes keyword-based training positives to include non-hateful content.
- Keyword lists miss hateful speech expressed through less common or more nuanced constructions because they focus on widely known slurs.
- Self-identified hateful communities provide training data that requires far less collection effort and produces superior classifiers.
- Future work should extend community-based training beyond forum platforms to heavily used services such as Twitter and Facebook.