PaperlayerView exact revision on arXiv

Source-linked AI summary

Attenuating Bias in Word Vectors

Sunipa Dev, Jeff Phillips

arXiv:1901.07656v1cs.CL

TL;DR

Word embeddings can carry and amplify social biases, creating a need for effective ways to detect and reduce them. The paper uses linear projection guided by gender-associated names and extends names to other demographic biases, showing that this approach can measure and attenuate bias while retaining embedding structure.

  • Problem

    Word embeddings can express and amplify biases from their source data, including biases relevant to applications that affect people’s lives.

  • Method

    The paper detects bias directions from word pairs or common names and reduces bias through linear projection, with variants including partial projection and name-based directions.

  • Results

    Names provide effective gender bias directions and can also detect and support debiasing of race-, nationality-, and age-associated bias in embeddings.

  • Takeaways & Limitations

    Names offer indicators for biases lacking suitable word pairs, while weighted projection can remove bias while retaining more of the embedding’s inherent structure.

  • Takeaways & Limitations

    Hard Debiasing does not debias the full vocabulary automatically, because it selects words using crowdsourcing, definitions, and extrapolation.

Abstract

from arXiv · show

Word vector representations are well developed tools for various NLP and Machine Learning tasks and are known to retain significant semantic and syntactic structure of languages. But they are prone to carrying and amplifying bias which can perpetrate discrimination in various applications. In this work, we explore new simple ways to detect the most stereotypically gendered words in an embedding and remove the bias from them. We verify how names are masked carriers of gender bias and then use that as a tool to attenuate bias in embeddings. Further, we extend this property of names to show how names can be used to detect other types of bias in the embeddings such as bias based on race, ethnicity, and age.

1 BIAS IN WORD VECTORS

Word embeddings can express and amplify social biases, motivating methods that detect and reduce bias directly in the vectors. The paper proposes simple linear projection, refined projection damping, and names as general bias indicators.

  • Motivation: Word embeddings can amplify biases inherited from training data, affecting applications such as credit assessment, crime prediction, hiring, and admissions.Reported associations include gendered professions, gendered links to arts versus mathematics, and racialized positive or negative connotations.
  • Contributions: Simple linear projection along a bias direction is presented as more effective than Hard Debiasing while being less complex and less dependent on crowdsourcing.The method projects all word vectors rather than debiasing only a selected set.
  • Contributions: Dampening the projection of words far from the projection distance can slightly improve the debiasing results.This refinement targets words that may be strongly affected by straightforward projection.
  • Experiments: Randomly flipping or swapping gendered word pairs in raw text before embedding construction does not alone eliminate bias in the resulting embeddings.The paper therefore compares corpus modification with repairing the embeddings themselves.
  • Names as bias indicators: Common gender-associated names can provide a more effective gender subspace for debiasing than gendered words such as he and she.The paper further uses names to identify bias directions associated with race, nationality, and age.

2 DATA AND NOTATIONS

The study uses word embeddings in a high-dimensional vector space and represents bias with a vector or, more generally, a multidimensional subspace.

  • Data: The default embedding is a 300-dimensional GloVe model trained on a 4.57-billion-token Wikipedia dump with the 100,000 most frequent words.The corpus and vocabulary define the study’s default experimental setting.
  • Notation: Each vocabulary word is represented by a vector in the embedding space.These vectors are the objects modified by the debiasing procedures.
  • Notation: The bias subspace is typically represented by a single unit vector vB, although the approach can generalize to multiple vectors.Using one vector simplifies the descriptions while allowing multidimensional extensions.

3 HOW TO ATTENUATE BIAS

The section develops automatic debiasing methods that construct a bias direction from equality sets, project word vectors away from it, and optionally dampen projection based on orthogonal distance. It also examines how raw-text word swapping affects gender structure and finds that complete swapping can exacerbate residual gender components.

  • Bias direction: Equality sets provide paired vectors whose stacked differences define the bias vector vB as the top singular vector of a matrix Q.The framework permits equality sets with more than two words, although the experiments primarily use word pairs.
  • Existing method: Hard Debiasing neutralizes selected words and separately adjusts equality sets, but its word selection relies on definitions, crowd sourcing, and SVM extrapolation rather than the full vocabulary.Consequently, the procedure is not fully automatic and does not debias every word vector.
  • Simple methods: A universal orthogonal projection removes every word vector’s component along vB, reducing a D-dimensional embedding to D −1 dimensions while preserving orthogonal differences.The authors report that this single projection debiases better than Hard Debiasing.
  • Simple methods: Linear projection makes gender-paired words close but not necessarily equal, preserving non-gender distinctions arising from differences in word senses and usage.For example, man retains distinctions associated with humankind, general persons, and informal expressions, whereas he and she may become nearly synonymous.
  • Partial projection: Partial projection decomposes each word into bias component β(w) and orthogonal magnitude η(w), retaining the residual while adjusting β(w) toward µ according to η(w).The default function fixes the bias term, making the procedure functionally equivalent to linear projection; variants dampen reduction as η increases.

4 THE BIAS SUBSPACE

The paper defines bias directions from gendered word pairs and uses projections onto those directions to identify stereotypically gendered words. It shows that common male and female names provide an effective alternative direction, closely approximating the gender subspace and recovering similar occupational and adjectival associations.

  • Bias direction: The gender direction vB is typically derived as the top singular vector of a matrix formed from biased word-pair vectors.The paper uses gendered pairs to construct the bias subspace and examines variance within that subspace using SVD.
  • Bias direction: The magnitude of ⟨vB, w⟩ indicates how strongly word w lies along the gender subspace, while its sign indicates female or male association.This dot-product criterion is used to rank the most gendered words in the embedding.
  • Bias direction using names: Names can replace standard gendered word pairs as an alternative basis for detecting the gender direction.Many highly gendered words identified by the standard direction are names, motivating names as a bootstrap signal.
  • Bias direction using names: The name-based direction is computed from ten common unambiguous male names and ten common unambiguous female names, whose category means define the direction.The names are selected from the top 100K words and paired randomly before the modified computation is applied.
  • Results: 0.809 dot product: on the default Wikipedia dataset, the name-based direction closely approximates the gender subspace obtained from gendered word pairs.The paper reports similarly large dot products on other datasets.
  • Results: Using names as the gender direction recovers similar occupational and adjectival classifications, including homemaker as more female and policeman as more male.The resulting gendered occupations are illustrated in Table 4.

5 QUANTIFYING BIAS

The paper introduces automatic tests for measuring bias in embeddings and evaluates debiasing methods against both bias-sensitive and general embedding-quality criteria. Linear projection generally reduces measured bias while preserving more standard structure than competing approaches.

  • Bias measures: WEAT, ECT, and EQT provide automatic tests for gender bias and the effects of neutralization without relying on crowdsourced judgments.ECT examines nearest-neighbor coherence, while EQT quantifies biased analogy behavior.
  • Bias measures: Projection raises ECT scores to 0.996 with vB and 0.943 with vB,names, compared with 0.917 for Hard Debiasing.Flipping generally improves ECT less reliably, especially for ECT using names.
  • Bias measures: Projection increases EQT from 0.128 for the original embedding to 28.3% with vB and 29.1% with vB,names.Subtraction reaches 23–27%, while Hard Debiasing reaches 0.145.
  • Bias measures: WEAT decreases from 1.623 in the original embedding to 1.219 with projection using vB,names and 1.234 using vB.Flipping lowers the score to about 1.1, while values closer to 0 are better.
  • Embedding quality: Debiasing reduces standard similarity and analogy scores, but linear projection usually causes the smallest decrease, about 1% except 5–7% on analogies.Damped approaches retain more structure on the Google Analogy test, while ordinary projection is otherwise roughly comparable.
  • Embedding quality: Damped variants f1, f2, and f3 produce ECT, EQT, and WEAT scores similar to simple projection, usually slightly worse.Their main advantage appears on the unbiased Google Analogy test, where scores closer to the original indicate more retained structure.

6 DETECTING OTHER BIAS USING NAMES

Names provide bias directions beyond gender, enabling detection and attenuation of racial, ethnic, and age-related associations in embeddings. The observed associations vary by demographic axis and are reduced after projection.

  • Method comparison: Projection using names works as well as projection using gendered word pairs for identifying and removing the gender direction.The name-derived gender subspace is presented as effective for extending the approach to other demographic biases.
  • Racial and ethnic bias: Names enable bias detection when suitable word pairs for a protected demographic direction are unavailable.The paper examines Hispanic/European-American and African-American/European-American name divisions.
  • Racial and ethnic bias: Occupation words show different associations along Hispanic–European-American, African-American–European-American, and male–female axes across embeddings and datasets.For example, footballer is more Hispanic than European-American, while maid is more neutral racially than by gender.
  • Racial and ethnic bias: Racial directions also retrieve distinct non-occupation associations, including drugs and illegal near Hispanic–European-American and slang near African-American–European-American.These associations are evaluated with WEAT before and after projection.
  • Racial and ethnic bias: Dampening removes demographic-group names, slang, and colloquial terms from the closest 10% of words.The paper links this reduction to the distinguishability of demographic characteristics in occupational or financial bias.
  • Age bias: Names from different historical periods correlate with age-related terms, revealing an age association in embeddings.The paper notes that age bias appears less clearly in professions than gender bias.

7 DISCUSSION

The discussion argues that names can serve as indicators for detecting and attenuating biases when suitable word pairs are unavailable. Weighted projection is presented as retaining embedding structure while removing bias.

  • Discussion: Names allow bias directions to be measured and embeddings to be projected away from them even when other bias indicators are unavailable.The discussion contrasts this with gender, where frequent word pairs and names both support measurement.
  • Discussion: A weighted variant of projection is reported to remove bias while retaining the inherent structure of the word embedding.The statement is presented as a conclusion of the work.

A Bias in different embeddings

The paper examines whether gender-bias patterns recur across embedding types and datasets. It reports similar patterns across the tested embeddings, including GloVe and Word2Vec resources.

  • Cross-embedding analysis: Gender-bias patterns are examined across different embeddings, datasets, and embedding mechanisms.The paper reports similar patterns across all settings shown in Figure 7.
  • Cross-embedding analysis: The evaluation uses GloVe embeddings from Wikipedia, Common Crawl, and Twitter, plus Word2Vec embeddings from Google News.The listed corpora range from 27B to 840B tokens.

B WORD EMBEDDING ASSOCIATION TEST

The Word Embedding Association Test measures preferential associations between target word groups and gendered attributes, using cosine-based association scores normalized by their standard deviation. In this setup, occupation and career/family words are tested against male/female words or names.

  • WEAT adapts the Implicit Association Test to measure human-like bias in word embeddings.It can reveal associations such as career words with male attributes and family words with female attributes.
  • The test compares two target sets against two attribute sets using cosine-based association scores.Target sets include occupation or career/family words, while attributes include gendered words or names.
  • The normalized statistic measures the relative association of target groups with the two attribute groups.Values closer to 0 indicate less preferential association between the target and attribute groups.
  • The evaluation uses career and family word lists alongside male and female names and gendered words.The male-name list includes names such as john, paul, mike, and kevin; the female-name list includes amy, joan, lisa, and sarah.

C DETECTING THE GENDER DIRECTION

The paper identifies a largely one-dimensional gender direction by analyzing variance in gender-pair differences. The dominant singular direction is used to represent gender bias, while comparisons with random constructions distinguish gender-related structure from random variation.

  • Detecting the Gender Direction: SVD of embeddings projected onto a gender-pair subspace shows a dominant first singular value nearly twice the second.The authors therefore use only the top singular vector rather than two or more vectors to define the gender subspace.
  • Detecting the Gender Direction: Gender-pair constructions retain the strongest singular-value drop, whereas random-word constructions show no significant drop.Using one gendered word with one random word produces an intermediate decline.
  • Detecting the Gender Direction: The comparisons indicate that the embedding contains roughly one important gender direction, with related directions resembling random variation.This supports modeling gender bias with a single direction.
  • Detecting the Gender Direction: The figures compare gender, adjective, racial, and singular-value patterns across several word embeddings and random baselines.The embeddings include GloVe trained on Wikipedia, Twitter, and Common Crawl, plus Word2Vec trained on Google News.
  • Detecting the Gender Direction: The gender component of a word vector is its projection onto the normalized first right singular vector.For vector w and gender direction vB, the component is ⟨w, vB⟩vB.
  • Detecting the Gender Direction: The paper contrasts its origin-passing SVD direction with PCA-based subspaces that are defined after centering the data.The PCA comparison is associated with the hard-debiasing approach, whereas this method uses SVD.

D Word Lists

The paper assembles word lists for gender, racial, ethnic, and age-related analyses, including gendered word pairs, occupations, and culturally associated names. These lists provide the vocabulary used to examine different forms of embedding bias.

  • Gendered Word Lists: The gendered vocabulary includes paired terms such as actor/actress, brother/sister, and king-related or family-role variants.The lists also contain occupational and social-role terms with masculine and feminine forms.
  • Occupation and Social-Role Lists: The occupation and social-role list contains terms spanning professions, public roles, education, healthcare, trades, and everyday work.Examples include detective, scientist, secretary, surgeon, teacher, programmer, and banker.
  • Name Lists: The gender-name lists define male and female groups using names such as john, william, mary, victoria, and sarah.These names are used as gender-associated vocabulary for the embedding analyses.
  • Race and Ethnicity Lists: The racial and ethnic lists distinguish European American, African American, and Hispanic-associated names.The groups include names such as brad, darnell, jamal, alejandro, and sofia.
  • Age Lists: The age-related lists separate names associated with aged and youth groups.Examples include ruth, horace, and edward in the aged group and taylor, jamie, and ethan in the youth group.
Loading 1901.07656v1…