Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT model

Marzieh Mozafari; Reza Farahbakhsh; Noel Crespi

Source-linked AI summary

Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT model

Marzieh Mozafari, Reza Farahbakhsh, Noel Crespi

arXiv:2008.06460v2Aug 14, 2020cs.SIcs.CLcs.IRcs.LG

TL;DR

The paper addresses biases in hate-speech datasets and trained classifiers, especially limited attention to classifier-level bias. It combines BERT transfer learning with sample reweighting and cross-domain evaluation, finding systematic racial bias against AAE-aligned tweets and significant reduction after mitigation.

Problem
Biases in hate-speech datasets and trained classifiers raise concerns, with classifier-level bias receiving less attention than biased datasets.
Method
The study fine-tunes a pre-trained BERT classifier using reweighted samples that reduce the influence of highly class-correlated training n-grams, then evaluates it cross-domain.
Results
The classifiers systematically assign AAE-aligned tweets to negative classes more often than SAE-aligned tweets, while reweighting significantly reduces racial bias.
Takeaways & Limitations
Cross-domain evaluation reveals racial bias in trained hate-speech classifiers, and the proposed mitigation decreases racial bias in the model.
Takeaways & Limitations
Future work should account jointly for sexual and gender identities, dialect, social identity, and highly correlated negative-class n-grams; BERT’s external knowledge may itself be a bias source.

Abstract

from arXiv · show

Disparate biases associated with datasets and trained classifiers in hateful and abusive content identification tasks have raised many concerns recently. Although the problem of biased datasets on abusive language detection has been addressed more frequently, biases arising from trained classifiers have not yet been a matter of concern. Here, we first introduce a transfer learning approach for hate speech detection based on an existing pre-trained language model called BERT and evaluate the proposed model on two publicly available datasets annotated for racism, sexism, hate or offensive content on Twitter. Next, we introduce a bias alleviation mechanism in hate speech detection task to mitigate the effect of bias in training set during the fine-tuning of our pre-trained BERT-based model. Toward that end, we use an existing regularization method to reweight input samples, thereby decreasing the effects of high correlated training set' s n-grams with class labels, and then fine-tune our pre-trained BERT-based model with the new re-weighted samples. To evaluate our bias alleviation mechanism, we employ a cross-domain approach in which we use the trained classifiers on the aforementioned datasets to predict the labels of two new datasets from Twitter, AAE-aligned and White-aligned groups, which indicate tweets written in African-American English (AAE) and Standard American English (SAE) respectively. The results show the existence of systematic racial bias in trained classifiers as they tend to assign tweets written in AAE from AAE-aligned group to negative classes such as racism, sexism, hate, and offensive more often than tweets written in SAE from White-aligned. However, the racial bias in our classifiers reduces significantly after our bias alleviation mechanism is incorporated. This work could institute the first step towards debiasing hate speech and abusive language detection systems.

Introduction

The study addresses hate-speech detection and the data-driven and algorithm-driven biases that can make automated classifications unfair. It proposes BERT-based transfer learning, reweighted training, and cross-domain validation to examine and mitigate these biases.

Hate speech and offensive language on social media are linked to trauma, hate crime, cyber-bullying, and discrimination, motivating automated detection efforts.
Hate-speech detection involves challenges in defining hate speech, building automatic identification tools, and addressing unintended data-driven and algorithm-driven biases.
The study extends a BERT-based transfer-learning approach for identifying hate speech in online social media.
A regularization mechanism reweights training data to mitigate data-driven and algorithm-driven bias, while a new loss-function strategy fine-tunes the pre-trained BERT model.
Cross-domain validation is used to evaluate the efficiency of the proposed bias-mitigation mechanism on two publicly available hate-speech and offensive-language datasets.

Previous works

Prior work developed keyword-, embedding-, neural-, transfer-learning-, and bias-analysis approaches for detecting hateful or abusive social-media content. The paper builds on evidence that both datasets and trained classifiers can encode unintended identity and racial biases.

Keyword-based hate-speech detection can perform well with resources such as HateBase, but maintaining and upgrading external lexicons is challenging.
Neural approaches used paragraph embeddings, word and character representations, CNNs, recurrent networks, and related classifiers to identify hateful content.
Multi-task transfer learning addressed differing annotation schemes, labels, and geographic or cultural sampling influences by jointly solving related hate-speech tasks.
Pre-trained embeddings improve NLP tasks with limited training data, while contextual models such as BERT address shortcomings in disambiguating words across contexts.
This paper proposes a pre-trained BERT-based model that targets hate-speech performance and unintended data or model bias through different fine-tuning strategies and bias mitigation.
Prior studies found unintended dataset bias from identity terms and sampling procedures, including classifiers that predict AAE texts as abusive more often than SAE texts.

Materials and methods

The framework contains shared BERTBASE-based hate-speech detection and bias-mitigation modules. Its detection component preprocesses tweets, fine-tunes BERT with alternative classifier architectures, and predicts test labels.

The framework has separate Hate Speech Detection and Bias Mitigation modules that share a pre-trained BERTBASE component but fine-tune it differently.
The detection pipeline extracts tweet text using preprocessing rules, feeds processed tweets into BERT, fine-tunes task-specific classifiers, and evaluates predictions on test data.
BERT is a multi-layer bidirectional transformer encoder trained on English Wikipedia and the Book Corpus, with BERTBASE and BERTLARGE variants.
BERTBASE contains 12 transformer layers, 12 self-attention heads, and 110 million parameters, whereas BERTLARGE contains 24 layers, 16 heads, and 340 million parameters.
The study uses uncased BERTBASE with sequences up to 512 tokens, adding [CLS] and [SEP] segments; the [CLS] embedding serves classification.
Different fine-tuning strategies attach a classifier to BERT representations using a linear network, nonlinear layers, a Bi-LSTM, or a CNN over transformer-layer outputs.

Experiment setup

The study evaluates BERT-based hate speech detection on three Twitter datasets, using preprocessing, stratified splits, and macro-averaged F1 for imbalanced classes.

Datasets: The experiments use Waseem, Waseem and Hovy, and Davidson Twitter datasets annotated for racism, sexism, hate, offensive, or neither content.Waseem and Hovy collected 16,914 annotated tweets; Davidson sampled 25,000 tweets from a larger crawl and labeled them through CrowdFlower.
Pre-processing: Preprocessing lowercases tweets, removes user mentions, URLs, and emoticons, standardizes elongated words, and converts hashtags to textual counterparts.These steps are intended to create clean model inputs while protecting user identities and excluding unmodeled emotional information.
Model implementation: The implementation uses pretrained BERT tokenization and WordPiece representations, with sequences padded or truncated to 64 tokens.Training uses batch size 32, three epochs, Adam with learning rate 2e-5, and cross-entropy loss.
Evaluation metric: Models are compared using macro-averaged F1-measure because the hate speech datasets contain imbalanced classes.Macro F1 summarizes precision and recall across classes using their geometric mean.

Experiment results

BERT-based models generally outperform or match existing approaches on the two datasets, while BERT + CNN performs best and remains effective with limited training data. Fine-tuning also produces tighter class-specific embeddings, but dataset language patterns contribute to errors and misclassification.

Benchmark performance: Most BERT-based fine-tuning strategies outperform or competitively match existing approaches on both Waseem and Davidson datasets.The exception is BERT + nonlinear classifier on top of BERT.
Benchmark performance: 88% F1-measure is achieved by BERTBASE + CNN on Waseem-dataset, improving 5% over Park et al. [43].On Davidson-dataset, BERTBASE + CNN reaches 92% F1-measure, while Zhang et al. reports 94% under binary classification.
Limited training data: At 0.1 of the training data, the BERT-based model reaches almost 87% F1-measure versus 72% for the Davidson baseline.The BERT-based model also has comparable performance across different training-data portions and is mostly stable beyond 0.3 on Waseem.
BERT embeddings analysis: BERT embeddings become strongly clustered by racism, sexism, and neither content in the last four layers after BERTBASE + CNN fine-tuning.Before fine-tuning, the first four layers are highly sparse, while the last four layers are somewhat more class-clustered, especially for racism samples.
Error analysis: Davidson-dataset language patterns involving African American Vernacular English are oversampled and associated with high misclassification rates.Some offensive words and slurs were labeled as hate without considering context, including language used in daily communication.

Bias mitigation module

The study identifies dataset and classifier biases in BERT-based hate-speech detection and applies sample reweighting during fine-tuning to reduce racial bias. Cross-domain evaluation shows substantial AAE–White prediction disparities that generally decrease after reweighting, although in-domain performance declines and residual bias remains.

Bias identification: The framework examines data-driven and algorithm-driven bias in BERT-based hate-speech detection using explicit n-gram correlations and cross-domain predictions.It evaluates classifiers trained on Waseem and Davidson datasets against AAE-aligned and White-aligned Twitter groups.
Re-weighting mechanism: The mitigation mechanism reweights training and validation samples to smooth phrase–class correlations, then incorporates sample weights into BERT fine-tuning loss.The method targets bias propagated through trained classifiers rather than only changing data collection, sampling, or annotation.
Bias identification: Class-specific 2-grams correlate strongly with negative labels, including race- and gender-related terms in Waseem and stereotyped terms in Davidson.These correlations appear in both training and test distributions and can transfer stereotype bias to classifiers.
Cross-domain evaluation: 10.5 times more often, the unreweighted Waseem classifier assigns AAE-aligned tweets to Racism than White-aligned tweets; reweighting reduces this disparity by 6.8 times.The reduction also applies to the Sexism class.
Cross-domain evaluation: For Davidson, Hate assignments shift from 5.8% versus 2.6% to 4.3% versus 3.1% for AAE-aligned versus White-aligned tweets after reweighting.Offensive assignments to AAE-aligned tweets fall from 36% to 19%, with reported mitigation rates of 0.85 for Hate and 0.70 for Offensive.
Trade-offs: F1-measure drops 3.7% on Waseem and 5.5% on Davidson after reweighting, while substantial racial bias persists and still favors negative labels for AAE-aligned tweets.The authors attribute the performance decline to reduced correlations between specific 2-grams and labels shared by training and test sets.

Discussion and challenges

Cross-domain evaluation exposes systematic racial bias in classifiers trained on hate-speech datasets, while reweighting reduces several disparities but leaves important sources of bias unresolved.

Cross-domain bias: Classifiers trained without reweighting assign AAE-aligned tweets to negative classes more often than SAE-aligned tweets, revealing systematic racial bias.The cross-domain evaluation covers racism, sexism, hate, and offensive predictions.
Racism predictions: Nearly 10 times as many AAE samples as SAE samples were classified as racism by the unreweighted Waseem classifier.The absolute number of racism assignments was low in both groups.
Effects of reweighting: Reweighting reduced AAE samples classified as racism and reduced AAE sexism predictions involving common AAE and feminism-related words.The sexism analysis links the reduction to lowering the effect of frequently used training n-grams.
Remaining challenges: The mechanism remains limited because it does not address gender bias, dialect and social identity together, or bias associated with pretrained BERT knowledge.The authors identify these areas as future work for more precise mitigation.
Effects of reweighting: Reweighting reduced racial bias for Davidson’s Hate and Offensive classes, where AAE-associated terms were highly correlated with negative labels.The terms “n*gga” and “b*tch” occurred thirty and five times more often in AAE-aligned than SAE-aligned samples, respectively.

Conclusion

The study finds that benchmark datasets contain label-correlated n-gram oddities that produce systematic classifier bias, and reports that reweighted BERT fine-tuning decreases racial bias.

Conclusion: Benchmark datasets contain specific n-grams highly correlated with negative classes, causing classifiers to prefer assigning some samples to those classes.The paper characterizes these patterns as oddities in hate-speech and abusive-language benchmarks.
Conclusion: Cross-domain evaluation demonstrates systematic biases in classifiers trained on the benchmark datasets.The classifiers are evaluated on groups aligned with different dialects and language varieties.
Conclusion: Reweighting the training set before fine-tuning a pretrained BERT-based classifier decreases racial bias.The authors describe this as an important step toward debiasing hate-speech and abusive-language detection systems.
Future research: Future research should jointly consider sexual and gender identities, speaker dialect and social identity, and correlated n-grams.The conclusion identifies these factors as targets for making bias alleviation more precise.

Supporting information

The supporting materials document BERT sentence-embedding visualizations and lists language features associated with AAE and SAE, including n-grams correlated with negative classes.

S1 Fig.: S1 Fig. visualizes 768-dimensional sentence embeddings from 12 BERT layers before and after fine-tuning using PCA.The visualizations cover Davidson-dataset samples and use bert-as-service to extract embeddings.
S1 Table: S1 Table lists the top 20 unigrams and 2-grams most correlated with AAE and SAE languages, together with occurrence counts.The listed phrases include “n*gga”, “b*tch”, “sh*t”, “f*ck w*t”, and “**s n*gga”.
S1 Table: Several phrases common in AAE dialects are highly correlated with Racism, Sexism, Hate, and Offensive classes in the datasets.The same feature pattern is summarized for the supporting table of AAE- and SAE-associated n-grams.