Reducing Gender Bias in Abusive Language Detection

1 Introduction

• Automatic abusive-language detection can support moderation as social media produces growing volumes of content that are difficult to moderate manually.Abusive language online can contribute to personal trauma, cyber-bullying, hate crime, and discrimination.Source: §1 · p. 1
• Supervised abusive-language classifiers may overfit to frequent neutral words, causing false positives such as labeling “You are a good woman” sexist because it contains “woman.”The introduction identifies this as false positive bias and cites Dixon et al. (2017).Source: §1 · p. 1
• Bias can be difficult to measure with usual validation and test sets because those sets may themselves be biased, motivating measurement and mitigation for real-world generalization.The authors warn that without sufficient generalization, abusive-language detection models may not be suitable for real-life situations.Source: §1 · p. 1
• The work measures gender bias in abusive-language datasets with a generated unbiased test set and proposes debiased embeddings, gender-swap augmentation, and fine-tuning with a larger corpus.It also compares the effects of different pre-trained word embeddings and model architectures on gender bias.Source: §1 · p. 1

3.1 Sexist Tweets (st)

• The sexist-tweets dataset was collected using sexism-related search terms, expert-annotated under critical race theory criteria, and restricted to sexist samples despite originally including some racist tweets.Waseem and Hovy (2016); Waseem (2016) describe sexist and racist language as subsets of abusive language.Source: §3.1 · p. 2

3.2 Abusive Tweets (abt)

• Founta et al. (2018)'s 60K-tweet corpus labels 11% Abusive, 7.5% Hateful, 22.5% Spam, and 59% None; the study merges None/Spam and Abusive/Hateful for binary classification.The corpus was annotated through systematic crowdsourcing methods, including boosted sampling and exploratory rounds.Source: §3.2 · p. 2

4.1 Methodology

• Because the original test distribution reproduces its bias, the authors create separate unbiased male and female test sets using identity-term templates from Dixon et al. (2017); paired sentences differ only in identity terms and should receive the same prediction.The template set contains 1,152 samples (576 pairs) and uses neutral and offensive nouns and adjectives.Source: §4.1 · p. 2
• The templates generated 1,152 samples (576 pairs) using common gender identity pairs and included neutral and offensive nouns and adjectives.Source: §4.1 · p. 2
• They use AUC on original and generated test sets, plus FPED and FNED to measure equality in false-positive and false-negative rates; AUC reflects accuracy, while equality differences reflect fairness.Prediction decisions use equal error rate thresholds.Source: §4.1 · p. 3

4.2 Experimental Setup

• The experiments first measure gender bias in st and abt datasets and explore three neural models used in prior abusive-language classification work, including a CNN.Source: §4.2 · p. 3

Gen. AUC

• Equality differences were larger with pre-trained embeddings when CNN or α-RNN was trained on st.The table caption reports larger false negative/positive equality differences under these conditions.Source: Table 4 · p. 3
• The α-GRU uses self-attention with a simpler mechanism used in Felbo et al. (2017), and hyperparameters were selected using the validation set.Source: Gen. AUC · p. 3

4.3 Results & Discussions

• Pre-trained embeddings improved task performance and scores on the unbiased generated test set (Gen. ROC).The passage attributes the Gen. ROC improvement to prior knowledge provided by word embeddings.Source: §4.3 · p. 3

Gen. AUC

• The models detect both discriminative classification features and unintended features that cause bias.Source: Gen. AUC · p. 4

• The section experiments with and discusses methods to reduce the gender biases identified in Section 4.3.Source: §5 · p. 4

5.3 Results & Discussion

• On GRU, combining debiased embeddings and gender swapping reduced equality differences by 98% and 89%, with only 1.5% loss of original performance.Using all three methods improved generated-unbiased-set performance and equality differences, but caused the largest decrease in original performance.Source: Table 6 · p. 4, §5.3 · p. 5

5.1 Methodology

• The three methods were debiased word embeddings, gender-swap augmentation of male and female entities, and transfer learning from a larger, less-biased corpus followed by target-corpus fine-tuning.The proposed fine-tuning approach is intended to regularize against overfitting to a small, biased dataset.Source: §5.1 · p. 4

5.2 Experimental Setup

• For bias fine-tuning, the abt dataset was selected as the less-biased source and st as the more-biased target; the model was trained on the source before target fine-tuning.The setup replaced the final softmax layer, used a slower target learning rate, and selected early stopping using each dataset’s validation set.Source: §5.2 · p. 5

5.3 Results & Discussion

• Fine-tuning with a larger, less-biased source corpus decreased equality-difference scores and greatly improved AUC on the generated unbiased test set; debiased embeddings alone were not effective, while gender swapping significantly reduced both scores.The AUC improvement indicates fewer errors on the unbiased set overall.Source: §5.3 · p. 5
• All mitigation methods involved some performance loss; fine-tuning had the largest original-test decrease, attributed partly to source-target task differences, though the decrease was under 4%.The authors suggest that modifying data or models may sometimes hinder discrimination of important unbiased features.Source: §5.3 · p. 5

6 Conclusion & Future Work

• The proposed methods reduce gender bias by 90–98%, while pre-trained embeddings, model architecture, and datasets also influence bias in abusive language detection.The authors say these reductions improve model robustness.Source: §6 · p. 5
• Mitigation can reduce classification performance, motivating methods that preserve or improve performance while reducing bias and future exploration of adversarial training (Beutel et al.; Zhang et al., 2018).The authors note that those approaches address neither latent gender and race variables in language nor directly comparable methods, but consider them worth exploring for this problem.Source: §6 · p. 5
• The authors advocate evaluating NLP systems beyond accuracy or AUC and extending their bias-measurement and correction approach to racial identities and tasks such as sentiment analysis.They frame this work as preliminary and describe ethical and practical systems as a motivation for measuring and correcting gender bias.Source: §6 · p. 5