Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function

Yusu Qian; Urwa Muaz; Ben Zhang; Jae Won Hyun

Source-linked AI summary

Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function

Yusu Qian, Urwa Muaz, Ben Zhang, Jae Won Hyun

arXiv:1905.12801v2May 30, 2019cs.CL

TL;DR

Gender bias in training data can be learned by neural language models and appear in generated text. The paper adds a loss term that equalizes male and female word probabilities, finding reduced bias without a significant perplexity increase and stronger results when combined with data augmentation.

Problem
Neural language models can learn gender bias from natural-language training data, producing biased text generation.
Source: §1 · p. 1
Method
The paper modifies the training loss to encourage equal probabilities for male and female words and steer optimization toward lower-bias minima.
Source: §1 · p. 1, §3.3 · p. 3
Results
The method reduces multiple bias metrics without a significant perplexity increase and performs best on occupation biases compared with CDA and REG.
Source: §4.1 · p. 5
Takeaways & Limitations
Combining the proposed loss with counterfactual data augmentation outperforms existing methods across the reported bias evaluation metrics.
Source: §1 · p. 1
The debiasing techniques rely on predefined gender pairs, leaving unpaired gender-associated words such as pregnant improperly treated.
Source: §5 · p. 6

Abstract

from arXiv · show

Gender bias exists in natural language datasets which neural language models tend to learn, resulting in biased text generation. In this research, we propose a debiasing approach based on the loss function modification. We introduce a new term to the loss function which attempts to equalize the probabilities of male and female words in the output. Using an array of bias evaluation metrics, we provide empirical evidence that our approach successfully mitigates gender bias in language models without increasing perplexity. In comparison to existing debiasing strategies, data augmentation, and word embedding debiasing, our method performs better in several aspects, especially in reducing gender bias in occupation words. Finally, we introduce a combination of data augmentation and our approach, and show that it outperforms existing strategies in all bias evaluation metrics.

1 Introduction

NLP models can learn gender and other social biases from training data, motivating methods that reduce bias directly during language-model training. This study introduces a loss term that equalizes probabilities for male and female word pairs and reports strong debiasing results.

NLP models can capture training-data biases and stereotypes, raising concerns about discrimination in public deployment.Source: §1 · p. 1
Language modelling supports important downstream applications such as text generation and is vulnerable to gender bias in training corpora.Source: §1 · p. 1
The proposed loss function directly encourages equal probabilities for gendered word pairs such as he and she during training.Source: §1 · p. 1
The study presents direct loss modification as its first investigation of bias alleviation in text generation and links it to end-to-end embedding debiasing.Source: §1 · p. 1
Combining the proposed method with counterfactual data augmentation achieves the best result and outperforms existing methods.Source: §1 · p. 1

2 Related Work

Prior work addresses NLP gender bias through word-embedding debiasing or data preprocessing, but both approaches have documented limitations. This paper instead targets bias in model outputs through loss-function training.

Gender-bias research broadly includes word-embedding debiasing and data debiasing through preprocessing.Source: §2 · p. 1
Projection-based embedding debiasing can leave stereotyped male and female words clustered and bias recoverable.Source: §2 · pp. 1–2
Bordia and Bowman (2019) found that standard language-model datasets contain gender bias and that trained models amplify it in generated text.Source: §2 · p. 2
REG minimizes neutral-word projections onto a gender subspace, whereas this paper minimizes output bias and adjusts the whole network accordingly.Source: §2 · p. 2
Counterfactual Data Augmentation swaps gender pairs, but doubles training data and increases language-model training time.Source: §2 · p. 2

3.1 Dataset

The experiments use a 5% random subsample of Daily Mail news articles to train the language models.

The dataset contains around 8.25 million tokens from a 5% random subsample of 219,506 Daily Mail articles.The articles cover topics including business, sports, and travel and are described as biased and sensational.Source: §3.1 · p. 2

3.2 Language Model

The language model uses pre-trained GloVe embeddings and an LSTM trained with tuned hyperparameters, dropout, gradient clipping, and early stopping.

3.2 Language Model

The model uses pre-trained 300-dimensional GloVe embeddings and an LSTM with two hidden layers of 300 units each.Random search selected the language-model hyperparameters.Source: §3.2 · p. 2
Training uses sequence length 35, learning rate 20 with annealing, dropout 0.25, and gradient clipping 0.25.The annealing schedule decays from 0.25 to 0.95.Source: §3.2 · p. 2
Models are trained for 150 epochs with batch size 48 and early stopping using a patience of 5.Source: §3.2 · p. 2

3.3 Loss Function

The model augments standard cross-entropy with a gender-bias loss that equalizes predicted probabilities for corresponding gendered words. This term steers optimization toward low-bias minima while preserving the model’s semantic output distribution.

3.3 Loss Function

The added loss term equalizes predicted probabilities for corresponding gender pairs such as woman and man.Gender pairs are selected so only gender information is neutralized, leaving distributions across unrelated semantic concepts unaffected.Source: §3.3 · p. 2
The overall objective combines cross-entropy with a gender-bias term weighted by λ over the corpus.Cross-entropy uses the ground-truth token and output softmax probabilities; λ controls the added bias penalty.Source: §3.3 · pp. 2–3
Among similar loss minima, the bias term encourages convergence toward a solution with the lowest gender bias.The paper frames this as modifying training dynamics rather than explicitly changing the embedding layer.Source: §3.3 · p. 3

3.4 Model Evaluation

The evaluation combines perplexity with complementary measures of generated-text, causal occupation, and embedding bias. These metrics assess unequal gender associations, gender-conditioned occupation probabilities, and occupation distances from gendered words.

3.4 Model Evaluation

The evaluation uses perplexity plus an array of bias metrics, with 223 gender pairs defining gendered words and all others treated as gender-neutral.This framework is intended to provide a holistic diagnosis of model behavior under debiasing.Source: §3.4 · p. 3

3.4.1 Co-occurrence Bias

Co-occurrence bias compares how frequently gender-neutral words occur with female versus male words in generated text.The score uses a sliding window of size 10 and excludes words occurring 20 or fewer times with gendered words.Source: §3.4.1 · p. 3
Conditional co-occurrence bias reduces the influence of unequal overall male and female word frequencies, while the gender ratio reports that frequency disparity directly.The paper treats general gender-frequency disparity as another form of bias.Source: §3.4.1 · p. 3

3.4.2 Causal Bias

Causal bias tests paired templates that vary gender or occupation information and measures resulting changes in occupation or gendered-word probabilities.The two template types evaluate occupation probabilities conditioned on gendered words and gendered-word probabilities conditioned on occupations.Source: §3.4.2 · pp. 3–4
Causal occupation bias uses gender-neutral occupations and separates effects conditioned on gender from effects conditioned on occupations.The latter can be more affected by the general disparity between male and female word probabilities.Source: §3.4.2 · p. 4

3.4.3 Word Embedding Bias

Embedding bias measures the difference between an occupation’s Euclidean distances to male and female words.The metric uses gender-neutral occupations, a set of gender pairs, and the word-to-vector dictionary, corresponding to bias by projection.Source: §3.4.3 · p. 4

3.5 Existing Approaches

The paper compares its loss-based debiasing with counterfactual data augmentation and embedding regularization. These baselines represent preprocessing and training-time word-embedding strategies.

3.5 Existing Approaches

Counterfactual data augmentation swaps gendered words using a bidirectional gender-pair dictionary and doubles the dataset with matched contextual distributions.The augmented corpus is used to train the language models.Source: §3.5 · p. 4
The REG baseline minimizes neutral-word projection onto the gender axis during language-model training using a tuned regularization coefficient.This implements the bias-regularization method of Bordia and Bowman (2019).Source: §3.5 · p. 4

4 Experiments

Experiments evaluate the proposed loss against baseline, CDA, and REG models using bias metrics and perplexity. The proposed loss reduces several bias measures, while combining it with CDA achieves the strongest overall result.

4.1 Results

At λ=1, the proposed loss reduces BN by 58.95%, CB|o by 100%, CB|g by 98.52%, and EBd by 98.98%.Source: §4.1 · p. 5
Increasing λ steadily decreases bias while slightly increasing perplexity, revealing a bias–perplexity trade-off.Source: §4.1 · p. 5
REG fails to achieve the best bias result on any metric but produces the best perplexity, indicating a slight regularization effect.Source: §4.1 · p. 5
The proposed loss outperforms CDA on occupation biases and embedding bias, although CDA performs slightly better on co-occurrence biases and perplexity.Source: §4.1 · p. 5
The CDA-plus-loss combination outperforms all evaluated methods on every bias measure without compromising perplexity.Source: §4.1 · p. 5

5 Conclusion and Discussion

The paper finds loss-function-based debiasing effective for neural language models and argues that end-to-end approaches extend beyond geometric embedding debiasing. Its methods depend on predefined gender pairs, motivating context-aware extensions and leaving unpaired gender-associated words inadequately treated.

Loss-function-based debiasing effectively reduces gender bias in neural language models across multiple evaluation metrics.Source: §5 · p. 5
End-to-end debiasing challenges geometric embedding debiasing as a complete solution for downstream applications.Source: §5 · p. 6
All evaluated debiasing methods rely on predefined gender pairs, limiting treatment of associated words without counterparts such as pregnant.Source: §5 · p. 6
Future work should make the loss function context-aware so it penalizes biased but not unbiased mentions of gendered words.Source: §5 · p. 6