Source-linked AI summary
Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings
Thomas Manzini, Yao Chong Lim, Yulia Tsvetkov, Alan W Black
TL;DR
Word embeddings can propagate social biases, and binary debiasing methods do not cover multiclass attributes such as race and religion. The paper generalizes debiasing to multiclass settings, introduces MAC for evaluation, and reports reduced bias with utility maintained across standard NLP tasks. It also identifies limitations in measuring class extremes and removing cluster bias.
Problem
Word embeddings inherit and amplify social biases, while existing debiasing methods were limited to binary labels despite demographic attributes often having multiple categories.
Method
The paper generalizes bias-subspace debiasing to multiclass embeddings and introduces mean average cosine similarity (MAC) to evaluate bias removal.
Results
Multiclass debiasing decreases bias, with MAC scores moving toward 1.0 and statistically significant associated P-values, while utility remains on par across POS tagging, NER, and POS chunking.
Takeaways & Limitations
The multiclass generalization robustly removes reported multiclass bias while preserving embedding utility across different NLP tasks.
Takeaways & Limitations
Bias-component removal is insufficient to eliminate multiclass cluster bias, although enlarging the bias subspace reduces correlation with the original bias.
Abstract
from arXiv · showhide
Online texts -- across genres, registers, domains, and styles -- are riddled with human stereotypes, expressed in overt or subtle ways. Word embeddings, trained on these texts, perpetuate and amplify these stereotypes, and propagate biases to machine learning models that use word embeddings as features. In this work, we propose a method to debias word embeddings in multiclass settings such as race and religion, extending the work of (Bolukbasi et al., 2016) from the binary setting, such as binary gender. Next, we propose a novel methodology for the evaluation of multiclass debiasing. We demonstrate that our multiclass debiasing is robust and maintains the efficacy in standard NLP tasks.
1 Introduction
Word embeddings inherit and can amplify social biases from training texts, while existing debiasing methods addressed only binary categories. This work extends debiasing to multiclass attributes, proposes a bias-evaluation metric, and reports preserved utility across NLP tasks.
- Word embeddings reflect and propagate social biases from training corpora, which downstream machine-learning systems can further amplify against disadvantaged groups.
- Bolukbasi et al. debiased binary gender embeddings by removing stereotype-related subspace components while preserving embedding utility.
- Gender, race, and religion can be continuous or categorical attributes with more than two categories, motivating multiclass debiasing.
- The proposed method generalizes prior debiasing to multiclass settings while preserving embedding utility.
- The study introduces a bias-evaluation metric and evaluates debiased embeddings on POS tagging, NER, and POS chunking, finding utility on par with off-the-shelf embeddings.
2 Background
Prior hard debiasing identifies a stereotype-related subspace and removes or equalizes its components, while soft debiasing learns a transformation balancing debiasing against preservation of inner products.
- Binary debiasing identifies a bias subspace and removes its components from each embedding.
- Defining word sets are centered, their vector differences are decomposed, and the most significant components form the gender subspace.
- Hard debiasing neutralizes nongendered words and equalizes gendered embeddings so neutral words are equidistant from biased words in the bias subspace.
- Neutralization removes each embedding’s bias-subspace component and normalizes the result, while equalization centers equality-set embeddings and adjusts their bias components.
- Soft debiasing: Soft debiasing learns a linear transformation that preserves biased–debiased inner products while minimizing projections of bias-neutral embeddings onto the bias subspace.
3 Methodology
The method extends word-embedding debiasing to multiclass bias by identifying a multiclass bias subspace, removing its components, and evaluating residual bias and utility.
- Multiclass bias subspace: The core contribution is identifying a bias subspace for multiclass settings, enabling prior debiasing methods to address multiple classes.
- Multiclass bias subspace: Multiclass defining sets are combined for principal component analysis, with the first k components forming the bias subspace.The number of components can be selected from PCA eigenvalues or a threshold, and defining sets may differ in size.
- Debiasing procedure: Hard Neutralize and Equalize and soft debiasing completely or partially remove bias-subspace components from embeddings.Hard debiasing removes components and equalizes defining words, whereas soft debiasing learns a projection that preserves inner products while reducing neutral-word projections.
- Bias evaluation: Mean average cosine similarity (MAC) evaluates associations between biased target words and attribute sets in a multiclass setting.The metric averages cosine similarities between each target and terms in each attribute set, with paired t-tests used to quantify debiasing effects.
- Utility evaluation: Utility is evaluated with NER, POS tagging, and POS chunking on sentences containing at least one word affected by debiasing.The evaluation also measures performance after replacing the embedding matrix and retraining the model with debiased embeddings.
4 Data
The study trains embeddings from U.S. Reddit data and evaluates gender, race, and religious bias using lexicons assembled from prior NLP and social-science studies.
- Embedding corpus: The initial word2vec embeddings are trained on approximately 56 million sentences from U.S.-collected Reddit data.The corpus contains posts and comments by native and non-native English speakers; author countries are inferred from posts, metadata, and self-identifying flairs.
- Bias lexicons: Bias lexicons are compiled for gender, race, and religion from prior NLP and social-science studies.Race includes Caucasian, African American, and Asian American vocabularies; religion includes Christian, Jewish, and Muslim labels.
- Downstream evaluation: Downstream evaluation uses the CoNLL 2003 shared-task data for NER, POS tagging, and POS chunking.These tasks assess the utility of biased and debiased embeddings.
5 Results and Discussion
The experiments find that multiclass debiasing reduces measured bias and largely preserves downstream utility, though effects vary by task and many changes are not statistically secure.
- Bias evaluation: The evaluation uses analogy construction across five word2vec spaces, intersecting the results to obtain analogies robust to embedding-space perturbations.A subset of these analogies is presented in Table 1.
- Bias removal: Multiclass debiasing moves MAC scores closer to 1.0, indicating increased cosine distance, with statistically significant associated P-values.The authors interpret this as evidence that their approach decreases bias.
- Downstream effects: Debiasing almost always decreases POS-tagging performance but consistently increases performance on NER and POS chunking.The authors suggest these models depend on bias subspaces differently.
- Utility preservation: Minor changes in Table 3 support preserving semantic utility in the multiclass setting.The paper frames this as comparable to gender debiasing, which is known to preserve utility.
- Robustness: Statistically significant MAC changes indicate that the calculated bias subspace is robust enough to support the debiasing operations.The claim concerns operational robustness rather than complete elimination of every form of bias.
6 Limitations & Future Work
The proposed multiclass debiasing has limitations in what it can resolve and where its bias lexicons generalize, while component removal does not eliminate clustered bias.
- Multiclass representation: The multiclass bias subspace captures bias-component magnitude but cannot measure the extremes of each class.Unlike binary gender debiasing, projecting multiple classes into a linear space cannot place class extremes at opposite ends.
- Cross-cultural scope: The defining and equality-set vocabularies were collected from US-specific data and may not translate to other countries or cultures.These lists identify words representing biases and words that should or should not contain them.
- Lexicon coverage: Some peer-reviewed vocabulary terms may be subjective and may not fully capture the bias subspace.This limits confidence that the selected defining and equality sets represent the full structure of bias.
- Residual bias: Removing the bias component is insufficient to remove multiclass cluster bias, although enlarging the bias subspace reduces the correlation between original bias and bias-class neighbors.The clustering concern follows prior work and is examined using appendix figures and Table 4.
7 Conclusion
The paper identifies multiclass biases in Reddit-trained word embeddings, introduces an evaluation metric, removes bias through a generalized debiasing method, and preserves utility across NLP tasks.
- Findings: The study finds multiclass biases in word embeddings trained on Reddit data.It reports this as an empirical finding about the evaluated embeddings.
- Evaluation: The authors introduce a novel metric for evaluating debiasing procedures for word embeddings.The metric is part of the paper’s proposed evaluation methodology.
- Debiasing: A generalization of existing techniques robustly removes multiclass bias.The conclusion summarizes the method’s reported debiasing outcome.
- Utility: The multiclass generalization preserves embedding utility across different NLP tasks.The conclusion states this as the final evaluation result.
A Addressing Cluster Bias
The evaluation tests whether increasing the bias-subspace dimensionality reduces cluster bias in religion-related embeddings. Although correlations decline for larger subspaces, biased neighbors remain strongly associated with professions’ original bias.
- Evaluation procedure: The procedure visualizes cluster bias by relating each profession’s number of biased neighbors to its original bias across subspace dimensionalities.It uses 100 closest neighbors for each profession and plots neighbors with positive bias against the profession’s original bias.
- Bias directions: The defining-set mean and normalized class-specific directions identify the bias directions used to rank the 500 most biased vocabulary words.For each class, the direction is the normalized difference between its embedding and the defining-set mean; words are ranked by their component in that direction.
- Findings: As subspace dimensionality increases to 2 or 3, the correlation between biased-neighbor counts and original profession bias decreases, but its magnitude remains high.Table 4 reports Pearson’s r and Spearman’s ρ for religion target classes, with all correlation coefficients having p-values < 10^-30.
- Findings: Figures 1 and 2 plot biased-neighbor counts against original bias for the target classes jew and muslim before and after debiasing.The plots compare different bias-subspace dimensionalities k for each profession.