Source-linked AI summary
Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them
Hila Gonen, Yoav Goldberg
TL;DR
The paper examines whether methods that reduce gender bias in word embeddings truly remove it. Through experiments on two debiasing methods and their biased counterparts, it finds that bias remains systematically encoded in gender-neutral words’ similarities and representations, limiting confidence in gender-neutral modeling.
Problem
Existing debiasing methods substantially reduce gender bias according to a projection-based definition, but evidence is limited regarding whether bias remains elsewhere in the embedding geometry.
Method
The paper compares HARD-DEBIASED and GN-GLOVE embeddings with corresponding biased baselines and conducts experiments probing residual bias in gender-neutral words.
Results
The experiments show that semantically related words retain gender bias in their similarities and representations after debiasing.
Takeaways & Limitations
Gender-direction debiasing is insufficient because it measures gender association without determining the full manifestation of bias.
Takeaways & Limitations
The paper did not verify whether Zhang et al.’s adversarial method produces similar results, although it expects it would because that method uses the same bias definition.
Abstract
from arXiv · showhide
Word embeddings are widely used in NLP for a vast range of tasks. It was shown that word embeddings derived from text corpora reflect gender biases in society. This phenomenon is pervasive and consistent across different word embedding models, causing serious concern. Several recent works tackle this problem, and propose methods for significantly reducing this gender bias in word embeddings, demonstrating convincing results. However, we argue that this removal is superficial. While the bias is indeed substantially reduced according to the provided bias definition, the actual effect is mostly hiding the bias, not removing it. The gender bias information is still reflected in the distances between "gender-neutralized" words in the debiased embeddings, and can be recovered from them. We present a series of experiments to support this claim, for two debiasing methods. We conclude that existing bias removal techniques are insufficient, and should not be trusted for providing gender-neutral modeling.
1 Introduction
Word embeddings reflect pervasive gender stereotypes and social associations, motivating debiasing methods that substantially reduce bias under a projection-based definition. The paper argues that this apparent reduction is superficial because bias remains recoverable from gender-neutral words’ geometry.
- Gender bias is consistent and pervasive across different word embeddings.
- Word embeddings can produce gender-stereotypical analogies, such as associating computer programmers with men and homemakers with women.
- Embedding associations between gendered names and stereotypically gendered domains include arts versus science and correlate with labor-force participation data.
- Existing post-processing and training-based methods substantially reduce projection-based gender bias without hurting word-similarity performance.
- The paper argues that these methods mostly hide rather than remove bias, which remains encoded in the geometry of gender-neutral words.
2 Gender Bias in Word Embeddings
Prior debiasing methods target gender associations through a predefined gender direction, but the paper argues that this definition misses systematic bias encoded in similarities among gender-neutral words. The authors frame this residual structure as a deeper form of bias that remains after debiasing.
- Definition and Existing Debiasing Methods: Prior work defines bias through projection on a gender direction, with larger projections indicating stronger gender association.
- Definition and Existing Debiasing Methods: Both methods substantially reduce bias under the projection-based definition, but their results depend on that specific definition.
- Definition and Existing Debiasing Methods: HARD-DEBIASED zeroes neutral words’ gender projections and equalizes their distances to explicitly gendered word pairs.
- Definition and Existing Debiasing Methods: GN-GLOVE modifies GloVe training to concentrate gender information in the last vector coordinate, which can then be excluded from representations.
- Definition and Existing Debiasing Methods: Reducing gender-direction projections is insufficient because gender bias remains in similarities and spatial groupings among gender-neutral words.
- Definition and Existing Debiasing Methods: The paper notes that earlier work mentioned indirect bias but did not quantify its extent before and after debiasing or propose methods to address it.
3 Experimental Setup
The experiments compare two debiased embedding methods with corresponding biased baselines and quantify hidden bias using original-versus-debiased representations. The evaluation also uses projection-based bias and WEAT to measure word associations.
- The study evaluates HARD-DEBIASED against pre-debiasing embeddings and GN-GLOVE against standard GloVe trained on the same corpus.
- The vocabulary is restricted to the 50,000 most frequent words and phrases after filtering punctuation, digits, uppercase letters, long words, and predefined gendered terms.
- Bias is measured by projection on the gender direction, while WEAT uses permutation tests to estimate associations between target and attribute word sets.
4 Experiments and Results
Experiments show that gender information remains recoverable after HARD-DEBIASED and GN-GLOVE debiasing, including through clustering, nearest neighbors, correlations, and classification.
- Nearest-neighbor bias: Gender-biased words remained grouped with socially related words of the same gender, motivating bias measurement through the gender composition of nearest neighbors.The neighbor-based measure compares male- or female-biased words among a target word’s k nearest neighbors with the original projection-based bias.
- Nearest-neighbor bias: 0.686 and 0.736 Pearson correlations connected neighbor-based bias with original bias for HARD-DEBIASED and GN-GLOVE, respectively.The corresponding biased-version correlations were 0.741 and 0.773, and all correlations had p-values of 0.
- Professions: 0.606 and 0.792 Pearson correlations linked professions’ original bias to male-neighbor counts after HARD-DEBIASED and GN-GLOVE debiasing.The corresponding biased-version correlations were 0.747 and 0.820; all correlations were significant with p-values < 1 × 10−30.
- Association tests: p-values remained significant across all three association experiments: 0, 0.00016, and 0.0467 for HARD-DEBIASED, versus 7.7 × 10−5, 0.00031, and 0.0064 for GN-GLOVE.The experiments tested associations between gendered names or concepts and family/career, arts/mathematics, and arts/science words.
- Classification: 88.88% and 96.53% classifier accuracy remained for HARD-DEBIASED and GN-GLOVE when predicting gender from previously biased-word vectors.The classifier generalized from 1,000 labeled words to 4,000 held-out words; the non-debiased HARD-DEBIASED baseline achieved 98.25%.
5 Discussion and Conclusion
The discussion argues that debiasing methods remove the gender direction but leave deeper, geometry-based gender information intact. This residual structure can support stereotype-based generalization and motivates broader bias measures.
- Discussion and Conclusion: Gender bias persists independently of the gender direction: semantically related words retain gendered similarities and representations.The conclusion summarizes systematic bias revealed across the preceding experiments.
- Discussion and Conclusion: Previously gender-biased words remain easy to cluster together after debiasing.This indicates that the representation preserves gender-consistent structure beyond direct gender-direction projections.
- Discussion and Conclusion: Implicitly gendered words still group with other implicitly gendered words of the same gender, similarly to non-debiased embeddings.Examples include receptionist, hairdresser, and captain.
- Discussion and Conclusion: The implicit gender of strongly biased words remains easy to predict from their vectors alone.The result supports the claim that gender information can be recovered from debiased representations.
- Discussion and Conclusion: The authors conclude that removing the gender direction is mostly superficial because corpus-learned stereotypes remain deeply embedded in representation geometry.The discussion distinguishes reducing the measured direction from removing the broader bias structure.
- Discussion and Conclusion: Such residual cues could let a resume classifier favor male candidates despite being oblivious to explicit gender markers.The experiments indicate that classifiers retain opportunities to learn and generalize gender-specific regularities after gender-direction debiasing.