Source-linked AI summary
Racial Bias in Hate Speech and Abusive Language Detection Datasets
Thomas Davidson, Debasmita Bhattacharya, Ingmar Weber
TL;DR
The paper addresses limited consideration of bias in abusive-language detection and examines racial bias across five Twitter datasets. It trains classifiers and compares their predictions for African-American English and Standard American English tweets, finding systematic disparities that persist under keyword conditioning and may disproportionately penalize African-Americans.
Problem
Abusive-language detection systems may be biased, yet their potential racial bias has received limited attention despite possible harm to protected groups.
Method
The study trains classifiers on five Twitter hate-speech and abusive-language datasets and compares predictions across black-aligned and white-aligned tweet corpora.
Results
Classifiers trained on all five datasets systematically assign negative labels to black-aligned tweets more frequently, with disparities generally persisting after keyword conditioning.
Takeaways & Limitations
The authors caution against field deployment and enforcement use because these datasets may systematically penalize African-Americans and produce racial discrimination.
Takeaways & Limitations
The study lacks ground-truth racial identities, may miss codeswitching users, and cannot rule out differences in negative-class distributions as contributors to the disparities.
Abstract
from arXiv · showhide
Technologies for abusive language detection are being developed and applied with little consideration of their potential biases. We examine racial bias in five different sets of Twitter data annotated for hate speech and abusive language. We train classifiers on these datasets and compare the predictions of these classifiers on tweets written in African-American English with those written in Standard American English. The results show evidence of systematic racial bias in all datasets, as classifiers trained on them tend to predict that tweets written in African-American English are abusive at substantially higher rates. If these abusive language detection systems are used in the field they will therefore have a disproportionate negative impact on African-American social media users. Consequently, these systems may discriminate against the groups who are often the targets of the abuse we are trying to detect.
1 Introduction
The study examines racial bias in Twitter-based hate-speech and abusive-language datasets, motivated by the risk that biased systems could discriminate against the groups they are intended to protect.
- Bias in machine-learning systems often originates in training data and can affect both supervised and unsupervised language technologies.
- Abusive-language models aim to identify speech targeting protected groups, but bias may reduce accuracy or cause discrimination against those groups.
- The study trains classifiers on five Twitter datasets and compares predictions for African-American English and Standard American English tweets.
- All classifiers show systematic racial bias, predicting negative classes for African-American English tweets more frequently than for Standard American English tweets.
- Bias generally decreases after conditioning on relevant keywords but persists, and the authors expect deployment could produce racial discrimination.
2 Related works
Prior research documents bias in language technologies, but comparatively little work examines bias in hate-speech and abusive-language detection, where errors can harm targeted minorities.
- Bias in supervised and unsupervised language systems has been linked to training data, including disproportionate performance failures for darker-skinned women in facial recognition.
- Hate-speech detection research has paid limited attention to bias despite the risk of penalizing victims or failing to identify abuse against them.
- False positives have been associated with terms related to race, gender, and sexuality, while annotation schemes and annotator identities have been explored as possible sources of bias.
- The Perspective API was found to assign high toxicity scores to innocuous statements containing identity terms, reflecting overgeneralization from toxic training examples.
3 Research design
The research design combines classifiers trained on five Twitter abuse datasets with language-based racial-alignment groups and bootstrap comparisons of predicted negative-class rates.
- Datasets: The study uses all available English Twitter datasets labeled for various types of abuse and describes them in chronological order.
- Datasets: The five datasets differ in collection and annotation, including author, activist, crowdworker, and mixed sampling approaches.Waseem and Hovy collected hateful-term tweets and labeled racism, sexism, or neither; Waseem used feminist and anti-racism activists; Davidson used U.S. crowdworkers; Golbeck used author labels; Founta combined random and lexicon-based sampling.
- Classifier training: Classifiers use regularized logistic regression with bag-of-words features to reduce the risk of importing bias through pretrained embeddings.
- Classifier training: Tweets are converted into TF-IDF matrices with up to 10,000 features, while 80% of each dataset is used for training and the remainder for validation.
- Race dataset: The race dataset identifies likely non-Hispanic Black and white users using posterior language-model proportions of at least 0.80, yielding 1.1 million black-aligned and 14.5 million white-aligned tweets.
- Experiments: The primary test asks whether predicted negative-class probabilities are independent of racial alignment, comparing P(c_i = 1|black) with P(c_i = 1|white).
- Experiments: Bootstrap sampling estimates class-assignment proportions from 1,000 resampled tweets per racial corpus, followed by t-tests and black-to-white rate ratios.
- Experiments: A second experiment conditions comparisons on negative-associated keywords, while the planned “-er” analysis was excluded because its samples were too small.
4 Results
Across the five classifiers, racial disparities generally persisted: black-aligned tweets were more often assigned negative labels than white-aligned tweets, though the magnitude and direction varied by dataset, label, and experiment.
- Most classifiers assigned negative labels more frequently to black-aligned tweets, with statistically significant disparities in nearly all comparisons.The sole reversed comparison involved the racism class in the Waseem and Hovy (2016) classifier, where predicted rates were very low for both groups.
- 1.7 times as often, the Waseem and Hovy (2016) classifier labeled black-aligned tweets as sexist compared with white-aligned tweets.Its racism prediction rate was lower for black-aligned tweets, but the rate was extremely low for both groups.
- 17% versus 6.5% of tweets were predicted to contain offensive language by the Davidson et al. (2017) classifier for black-aligned and white-aligned corpora, respectively.The same classifier labeled around 5% of black-aligned tweets as hate speech, compared with 2% of white-aligned tweets.
- 1.4 times as frequently, the Golbeck et al. (2017) classifier predicted black-aligned tweets as harassment compared with white-aligned tweets.The Founta et al. (2018) classifier also predicted hate speech, abuse, and spam more frequently for black-aligned tweets.
5 Discussion
Classifiers trained on all five datasets show consistent, systematic, and substantial racial disparities, with black-aligned tweets more often assigned negative labels. These disparities persist after conditioning on selected keywords, although their magnitude and direction vary across datasets.
- All five datasets produce classifiers that more often classify black-aligned tweets as sexism, hate speech, harassment, or abuse than white-aligned tweets.
- Keyword-conditioned comparisons show that racial disparities generally persist, though they are usually smaller and occasionally reverse direction.
- Waseem and Hovy’s and Waseem’s classifiers consistently label black-aligned tweets as sexism more often, including approximately 1.5 times as often for tweets containing “n*gga”.
- The Davidson classifier shows the largest Experiment 1 disparities, likely reflecting abundant AAE tweets and frequent use of terms such as “n*gga” and “b*tch”.
- The Golbeck classifier significantly more often labels black-aligned tweets as harassment across experiments, while conditioning on keywords substantially reduces the disparity.
- The Founta classifier shows substantial disparities, including a 2.7-times higher hate-speech rate for black-aligned “b*tch” tweets.
6 Conclusion
The study finds substantial racial bias across all tested hate-speech and abusive-language datasets, with implications for data collection, annotation, class imbalance, and context-sensitive detection. The authors caution that field deployment, especially enforcement, could systematically penalize African-Americans.
- 6 Conclusion: All tested datasets showed substantial racial bias, and the authors caution that field use could systematically penalize African-Americans.They recommend focusing bias measurement and mitigation on how datasets are collected and labeled.
- 6 Conclusion: Keyword sampling can over-represent African-American English and produce datasets with biased population coverage.The authors link this risk to uneven keyword frequencies and oversampling.
- 6 Conclusion: Annotator identities and sociocultural contexts may introduce systematic bias, including in expert-labeled data.The discussion identifies authors, crowdworkers, activists, and geographically diverse annotators as relevant factors.
- 6 Conclusion: 1% to 18% of tweets were assigned to negative classes across classifiers in Experiment 1, indicating substantial variation across datasets and models.The authors associate low rates with possible false negatives and high rates with possible false positives.
- 6 Conclusion: Context and linguistic subtleties matter because the “n-word” can be racist or quotidian depending on speaker, context, and spelling.The authors note that the “-a” variant is typically innocuous among African-Americans but has sometimes been labeled hateful.
- 6 Conclusion: A context-independent, one-size-fits-all detector is inappropriate because different communities have different speech norms.The authors emphasize that a model suitable for one community may discriminate against another and that detection systems can have uneven effects.
7 Limitations
The study’s comparisons have several scope and interpretation limits, including uncertain race labels, possible corpus-distribution effects, model dependence, and unexplained predictions.
- 7 Limitations: The AAE corpus lacks ground-truth racial identities and may omit users who frequently codeswitch between AAE and SAE.These constraints affect how language-based demographic groupings should be interpreted.
- 7 Limitations: The observed disparities may partly reflect different negative-class distributions rather than dataset bias alone.The authors state that ground-truth labels for both abuse and race are needed for a fuller investigation.
- 7 Limitations: Results may vary with classifier or feature-set choices, and the study did not analyze why classifiers made particular predictions.More sophisticated models might alleviate or exacerbate bias.