Source-linked AI summary
Automated Hate Speech Detection and the Problem of Offensive Language
Thomas Davidson, Dana Warmsley, Michael Macy, Ingmar Weber
TL;DR
Separating hate speech from other offensive language remains difficult because prior detection methods often conflate the two categories. This paper uses three-way tweet labeling and classification, finding that some racist and homophobic content is easier to identify as hate speech while sexist language is more often classified as offensive.
Problem
Prior hate-speech detection studies often conflate hate speech with offensive language, despite the legal and moral importance of distinguishing them.
Method
The paper crowdsources three-way tweet labels and trains a multi-class classifier to distinguish hate speech, offensive language, and neither.
Results
Fine-grained labels improve hate-speech detection, while racist and homophobic terms are more associated with hate speech than sexist language.
Takeaways & Limitations
Accurate classification requires accounting for context and the heterogeneous ways hate speech is expressed.
Takeaways & Limitations
The classifications reflect subjective social biases, with racist and homophobic slurs more readily identified as hateful than sexist language.
Abstract
from arXiv · showhide
A key challenge for automatic hate-speech detection on social media is the separation of hate speech from other instances of offensive language. Lexical detection methods tend to have low precision because they classify all messages containing particular terms as hate speech and previous work using supervised learning has failed to distinguish between the two categories. We used a crowd-sourced hate speech lexicon to collect tweets containing hate speech keywords. We use crowd-sourcing to label a sample of these tweets into three categories: those containing hate speech, only offensive language, and those with neither. We train a multi-class classifier to distinguish between these different categories. Close analysis of the predictions and the errors shows when we can reliably separate hate speech from other offensive language and when this differentiation is more difficult. We find that racist and homophobic tweets are more likely to be classified as hate speech but that sexist tweets are generally classified as offensive. Tweets without explicit hate keywords are also more difficult to classify.
Introduction
The introduction defines hate speech as harmful or derogatory language targeting disadvantaged groups, while distinguishing it from offensive language. The paper addresses this distinction by labeling tweets into three categories and training a model to differentiate them.
- Hate speech targets disadvantaged social groups in a potentially harmful manner, including characteristics such as race, ethnicity, gender, and sexual orientation.
- The paper defines hate speech as language expressing hatred toward, or intended to derogate, humiliate, or insult, a targeted group.In extreme cases, the definition includes language that threatens or incites violence, but is not limited to such cases.
- The study labels tweets as hate speech, offensive language, or neither, then trains a model to differentiate among these categories.The approach is designed to address prior work that often conflated hate speech with offensive language.
- The introduction reports that fine-grained labels can help improve hate-speech detection and support analysis of how hate speech differs from offensive language.
Related Work
Prior work shows that lexical and supervised approaches often conflate hate speech with offensive language, while syntactic features offer ways to identify targets and intensity. This distinction is challenging because offensive words frequently drive racist classifications and hate speech can depend on subtle linguistic differences.
- Lexical and supervised approaches: Bag-of-words methods achieve high recall but produce many false positives by misclassifying tweets containing offensive words as hate speech.This problem is attributed to the presence of offensive words in tweets that are not necessarily hate speech.
- Lexical and supervised approaches: 86% of anti-black-racist tweet classifications were attributed to the presence of offensive words.Kwok and Wang’s finding illustrates why the prevalence of offensive language and curse words makes hate-speech detection difficult.
- Lexical and supervised approaches: The distinction between hate speech and offensive language often depends on subtle linguistic differences.The passage gives a tweet containing a racist slur as an example of this challenge, though the quoted example is truncated.
- Feature-based approaches: Syntactic features have been used to identify hate-speech targets and intensity through relevant noun–verb combinations, POS trigrams, and explicit structures for intent and targets.Examples include “kill” with “Jews,” the POS trigram “DT jewish NN,” and a structure encoding intensity, user intent, and hate target.
- Lexical and supervised approaches: Supervised approaches have often conflated hate speech with offensive language, while neural language models have used similarly broad hate-speech definitions.These limitations make it difficult to determine whether such systems are actually identifying hate speech.
Data
The study used Hatebase terms to sample and crowd-code tweets, then trained a classifier on the resulting labels. Only 5% were majority-coded as hate speech, while most were offensive language, exposing the lexicon’s imprecision.
- Data collection: A Hatebase.org lexicon yielded tweets from 33,458 users, whose timelines produced an 85.4-million-tweet corpus and a random 25,000-tweet coded sample.Tweets containing lexicon terms were collected through the Twitter API and manually coded by CrowdFlower.
- Annotation results: 5% of tweets were coded as hate speech by the majority of coders, while only 1.3% were coded unanimously, demonstrating the lexicon’s imprecision.The hate-speech rate was lower than the 11.6% reported in a comparable Twitter study, likely because this study applied stricter criteria.
- Annotation results: Most tweets were labeled offensive language, at 76% under the 2/3 criterion and 53% under the 3/3 criterion.The remainder were labeled non-offensive at 16.6% under 2/3 and 11.8% under 3/3; these labels supported classifier training.
Features
The feature set combines normalized lexical n-grams, syntactic POS-tag patterns, and readability-based measures to represent tweet content, structure, and quality.
- Preprocessing: Tweets are lowercased and Porter-stemmed before feature extraction.
- Lexical features: Lexical features comprise TF-IDF-weighted unigrams, bigrams, and trigrams.
- Syntactic features: Syntactic structure is represented with Penn POS-tag unigrams, bigrams, and trigrams constructed using NLTK.
- Readability features: Tweet quality is captured using modified Flesch-Kincaid Grade Level and Flesch Reading Ease scores with the sentence count fixed at one.
Model
The study reduces feature dimensionality with L1-regularized logistic regression, then compares several prior models using cross-validation and grid search. Evaluation holds out 10% of the sample to help prevent over-fitting.
- The modeling pipeline first applies L1-regularized logistic regression to reduce data dimensionality.
- The study compares logistic regression, naïve Bayes, decision trees, random forests, and linear SVMs.
- Each model is tested with 5-fold cross-validation while holding out 10% of the sample for evaluation to help prevent over-fitting.
- A grid search iterates over the models and parameters.
Results
The best model performs strongly overall but struggles to identify hate speech, especially when hateful tweets lack explicit keywords. Errors reveal a bias toward less-hateful predictions, while multi-class classification reduces the tendency to label offensive language as hate speech.
- Overall performance: The best model achieves overall precision 0.91, recall 0.90, and F1 score 0.90, but hate-class precision and recall fall to 0.44 and 0.61.Almost 40% of hate speech is misclassified.
- Error patterns: The model is biased toward classifying tweets as less hateful or offensive than human coders, while approximately 5% of offensive and 2% of innocuous tweets are labeled hate speech.Most errors occur in the opposite direction, with relatively few tweets classified as more offensive or hateful than their true category.
- Hate-speech errors: Tweets most confidently identified as hate speech often contain multiple racial or homophobic slurs, whereas borderline hateful tweets are harder to classify without explicit hate terms.Some tweets containing slurs are actually anti-racist or less hateful, while many borderline cases still meet the hate-speech definition.
- Offensive-language errors: The multi-class framework limits offensive-language false positives: only 5% of true offensive language is labeled hate, and correctly labeled offensive tweets often contain curse words or sexist language.Misclassified offensive tweets frequently contain multiple slurs used in everyday communication, including racist language such as “n*gga” rather than “n*gger.”
- Neither-class errors: Tweets confidently assigned to the neither class are generally innocuous, positive, and readable, while errors often arise from references to race or sexuality and potentially offensive wording.Some neither-class tweets entered the sample because lexicon terms such as “charlie” and “bird” are not generally hateful.
Conclusions
The conclusions stress that hate speech must be distinguished from offensive language because conflating them produces serious classification errors. They also show that this distinction reflects social biases and depends on lexical cues and social context.
- Classification challenge: Conflating hate speech with offensive language misclassifies people as hate speakers and fails to distinguish commonplace offensiveness from serious hate speech.The paper emphasizes the legal and moral importance of making this distinction accurately.
- Lexical cues: Terms such as f*ggot and n*gger are generally associated with hate speech, whereas f*g, b*tch, and n*gga occur in both categories.Multiple racial and homophobic slurs make some egregious tweets easier to identify.
- Implementation: A smaller, higher-precision lexicon is preferable to a larger, higher-recall lexicon when a lexicon must be used.The authors provide a restricted version of the Hatebase lexicon.
- Future work: Future work should distinguish whether hate speech targets people directly, is addressed to nobody in particular, or appears in conversation.It should also examine the social contexts, conversations, individual characteristics, motivations, and social structures surrounding hate speech use.
- Social bias: People tend to identify racist and homophobic slurs as hateful but classify sexist language as merely offensive, reflecting subjective social biases.The results show better identification of particularly egregious anti-black racist and homophobic cases.