Source-linked AI summary
It's All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution
Rowan Hall Maudslay, Hila Gonen, Ryan Cotterell, Simone Teufel
TL;DR
This paper compares projection-based debiasing with counterfactual data methods for reducing gender bias in word embeddings. It introduces Counterfactual Data Substitution and the Names Intervention, finding that names-based CDA/CDS variants uniquely mitigate indirect bias while preserving useful gender information better than projection-based methods.
Problem
Word embeddings retain gender bias from training corpora, motivating comparison of projection-based debiasing with counterfactual data methods.
Method
The paper empirically compares Word Embedding Debiasing and Counterfactual Data Augmentation on Wikipedia and English Gigaword, introducing Counterfactual Data Substitution and the Names Intervention.
Results
CDA variants outperform projection-based methods for non-biased gender analogies, while names-based nCDA and nCDS significantly reduce biased-word cluster purity on both corpora.
Takeaways & Limitations
Names-based counterfactual methods provide a partial solution to indirect gender bias, with nCDS outperforming other methods significantly on both corpora.
Takeaways & Limitations
All compared methods rely on predefined gender-word pairs, imposing a gender binary and largely ignoring non-binary identities.
Abstract
from arXiv · showhide
This paper treats gender bias latent in word embeddings. Previous mitigation attempts rely on the operationalisation of gender bias as a projection over a linear subspace. An alternative approach is Counterfactual Data Augmentation (CDA), in which a corpus is duplicated and augmented to remove bias, e.g. by swapping all inherently-gendered words in the copy. We perform an empirical comparison of these approaches on the English Gigaword and Wikipedia, and find that whilst both successfully reduce direct bias and perform well in tasks which quantify embedding quality, CDA variants outperform projection-based methods at the task of drawing non-biased gender analogies by an average of 19% across both corpora. We propose two improvements to CDA: Counterfactual Data Substitution (CDS), a variant of CDA in which potentially biased text is randomly substituted to avoid duplication, and the Names Intervention, a novel name-pairing technique that vastly increases the number of words being treated. CDA/S with the Names Intervention is the only approach which is able to mitigate indirect gender bias: following debiasing, previously biased words are significantly less clustered according to gender (cluster purity is reduced by 49%), thus improving on the state-of-the-art for bias mitigation.
1 Introduction
Gender bias persists in word embeddings and can distort analogies and downstream NLP tasks. The paper compares projection-based debiasing with counterfactual corpus interventions and proposes improvements aimed especially at indirect bias.
- Word embeddings retain corpus-derived gender bias, producing stereotypical analogies and affecting tasks such as coreference resolution.
- WED reduces direct bias by removing gender-direction associations, but the original structure of indirect bias remains largely intact.
- The paper introduces CDS and the Names Intervention to improve counterfactual debiasing.CDS avoids duplication, while the Names Intervention expands treatment to first names using frequency- and gender-specificity-aware pairing.
- 49% average reduction in cluster purity follows CDA/S with the Names Intervention across both corpora.This is reported for clusters of the most biased words after treatment.
2 Related Work
Prior work operationalises gender bias either as a geometric direction or through counterfactual corpus transformations. Projection-based WED can reduce direct bias, while CDA changes training data but introduces duplication-related concerns.
- Word Embedding Debiasing: The gender subspace hypothesis assumes that most embedding gender bias lies in a linear subspace B.Under this operationalisation, bias is measured as the squared norm of a word vector’s projection onto B.
- Word Embedding Debiasing: WED neutralises non-gendered words by removing the gender direction and equalises gendered pairs relative to non-gendered words.
- Word Embedding Debiasing: WED uses 10 definitional pairs, 218 gender-specific seed words, and 52 equalise pairs.
- Word Embedding Debiasing: WED yields a 68% reduction in stereotypical analogies judged by humans, but the original bias structure remains intact.
- Counterfactual Data Augmentation: CDA appends a transformed corpus copy to the original, commonly swapping words in 124 gendered pairs.
- Counterfactual Data Augmentation: Grammar-aware CDA uses coreference and part-of-speech information to preserve grammaticality during gender-word substitutions.
3 Improvements to CDA
The paper improves CDA by replacing duplication with probabilistic substitution and by explicitly pairing first names. These changes target statistical naturalness, efficiency, grammaticality, and untreated name-linked bias.
- Counterfactual Data Substitution: CDA’s corpus duplication creates unusually even word frequencies and a lower-than-expected type–token ratio.The paper notes that the resulting effect on embeddings is difficult to predict.
- Counterfactual Data Substitution: CDS applies substitutions with 0.5 probability on a per-document basis, producing a non-duplicated counterfactual corpus.Per-document processing is intended to maintain grammaticality and discourse coherence while improving naturalness and efficiency.
- The Names Intervention: The Names Intervention uses bipartite-graph matching to explicitly treat first names rather than leaving name-coreferent gender associations untreated.
- The Names Intervention: The intervention pairs names from the SSA dataset, where names vary in frequency and gender-specificity.
- The Names Intervention: Name pairing balances frequency and gender-specificity, since optimizing either property alone can distort distributions or retain gender signals.
4 Experimental Setup
The study compares eight mitigation variants on Wikipedia and English Gigaword, evaluating direct and indirect bias, embedding quality, and gender-preserving analogies. The evaluation combines standard NLP benchmarks with clustering and classification tests.
- Experimental design: The experiments compare mitigation variants including CDA, grammar-aware CDA, Names Intervention CDA, their CDS counterparts, WED40, WED70, and an unmitigated control.
- Experimental design: WED40 and WED70 use one- and two-dimensional gender subspaces, with WED70 accounting for more than 70% of variance.
- Experimental design: The study uses Wikipedia and the Annotated English Gigaword to create Word2Vec embeddings for comparison.
- Evaluation: The study also evaluates word similarity, sentiment classification, and non-biased gender analogies.The analogy task reports error percentage for completing 506 family analogies such as boy:girl :: nephew:niece.
- Evaluation: Direct bias is measured with WEAT effect size Cohen’s d and a one-sided significance test.
- Evaluation: Indirect bias is evaluated through clustering the 1000 most biased words and measuring cluster V-measure after tSNE projection and k-means.Lower cluster purity indicates that biased words are less clustered after mitigation.
5 Results
Across direct-bias, embedding-quality, analogy, and indirect-bias evaluations, CDA/S variants generally outperform WED on non-biased gender analogies and reduce biased-word clustering, while direct-bias results favor WED slightly.
- Direct bias: On Wikipedia, nWED70 outperforms every other method on direct-bias tests (p < 0.01), with bias undetectable at α = 0.1.
- Direct bias: 0.95 versus 1.39: nCDS achieves lower average d across Wikipedia direct-bias tests than the best nonnames CDA/S variants.
- Method comparison: CDS variants are numerically better than their CDA counterparts in 80% of test cases, although many differences are not significant.
- Indirect bias: 0.420: nCDS reaches this Gigaword V-measure, a 58% reduction in biased-word cluster purity; on Wikipedia, the reduction is 39%.nCDA and nCDS have significantly lower biased-word cluster purity than all other evaluated mitigation techniques on both corpora.
- Sentiment classification: WED70 significantly improves sentiment classification over the unmitigated embedding on both corpora, but the improvement never exceeds 1.1%.Nothing outperforms WED70 or the Names Intervention variants on either corpus.
- Non-biased gender analogies: CDA and CDS improve non-biased gender analogies over unmitigated embeddings, whereas the best WED variant performs worse: 27.1 vs. 9.3% on Gigaword and 18.8 vs. 8.7% on Wikipedia.The authors suggest WED removes too much gender information, while CDA/CDS reduce stereotypical associations used incorrectly in analogies.
6 Conclusion
The comparison finds that WED and CDA mitigate direct bias, while CDA improvements better preserve gender representation and reduce indirect bias. The authors also identify limitations in predefined gender pairings and binary assumptions.
- 6 Conclusion: 23% average error was observed for WED’s best variants when drawing non-biased analogies.The authors interpret this as suggesting that too much gender information was removed.
- 6 Conclusion: 49% average reduction in cluster purity followed the Names Intervention when clustering the most biased words.The intervention was the only approach reported to successfully mitigate indirect gender bias.
- 6 Conclusion: Counterfactual Data Substitution generally outperformed its CDA equivalents and was notably quicker to compute.Because Word2Vec is linear in corpus size, CDS theoretically permits multiple intervention layers without exponentially enlarging the corpus.
- 6 Conclusion: All compared methods rely on predefined gender-word lists, especially paired words, which can encode outdated meanings and exclude biologically rooted terms.Examples include manager::manageress, spelling differences, and morphology differences.
- 6 Conclusion: Strict pairings impose a gender binary, leaving non-binary identities largely ignored in the bias-mitigation literature.
- 6 Conclusion: Future work could extend the Names Intervention beyond the US-based gazetteer to names from other languages.
A Proofs for method from Bolukbasi et al. (2016)
The appendix supplies proofs for properties of the linear-algebraic debiasing method, including the equalise step’s distance guarantee. It also clarifies the relevant vector construction and orthogonality.
- A Proofs for method from Bolukbasi et al. (2016): The authors provide proofs for equations they found opaque in Bolukbasi et al. (2016).
- A Proofs for method from Bolukbasi et al. (2016): ν = μ − μB is orthogonal to both wB and μB by construction.
- A Proofs for method from Bolukbasi et al. (2016): The equalise step ensures that gendered pairs such as man–woman are equidistant to all gender-neutral words.
- A Proofs for method from Bolukbasi et al. (2016): The proof defines e and w according to Bolukbasi et al.’s construction before computing their distance.
B WEAT word sets
The WEAT evaluation uses predefined target and attribute word sets for art–maths, arts–sciences, and careers–family comparisons. The careers–family test reverses the usual target–attribute assignment.
- B WEAT word sets: The WEAT word sets test direct bias using target and attribute groups defined from Nosek et al. (2002).For careers–family, gender is represented by target words rather than attribute words.
- B WEAT word sets: Art–Maths contrasts mathematics terms with arts terms and male-associated attributes with female-associated attributes.
- B WEAT word sets: Arts–Sciences contrasts science terms with arts terms and male-associated family terms with female-associated family terms.
- B WEAT word sets: Careers–Family contrasts male first names with female first names and career terms with family terms.
C Additional Gigaword results
This appendix lists additional English Gigaword results for cluster purity, word reclassification, sentiment classification, and non-biased gender analogies. The supplied passages provide figure titles but no numerical outcomes.
- C Additional Gigaword results: Additional results are reported for the Annotated English Gigaword.
- C Additional Gigaword results: Figure 10 reports most biased cluster purity results.
- C Additional Gigaword results: Figure 11 reports reclassification results for the most biased words.
- C Additional Gigaword results: Figure 12 reports sentiment classification results.
- C Additional Gigaword results: Figure 13 reports non-biased gender analogy results.