Understanding Undesirable Word Embedding Associations

1 Introduction

• For implicitly matrix-factorizing models such as SGNS and GloVe, subspace projection is equivalent under certain conditions to training on an unbiased corpus without reconstruction error.The guarantee is with respect to the word pairs defining the bias subspace; vector length should be retained, and the subspace should be their span rather than a principal component.Source: §1 · p. 1
• WEAT systematically overestimates bias, can misclassify gender-neutral words when attribute-word frequencies differ, and can be manipulated to label virtually any word as biased.Contrived attribute sets can classify even a gender-neutral word such as “door” as male- or female-biased relative to another gender-neutral word.Source: §1 · pp. 1–2
• RIPA measures association using a word vector’s inner product with a relation vector derived from ordered word-pair differences, enabling comparison with corpus-expected association.The relation vector is the first principal component of the pair-difference vectors.Source: §1 · p. 2
• On average, SGNS does not make most words more gendered than in the training corpus, but it amplifies corpus associations for gender-stereotyped or gender-specific words.Individual words may be slightly more or less gendered because of reconstruction error.Source: §1 · p. 2
• An unsupervised method for identifying gender-specific words is reported to preserve gender-appropriate analogies more effectively and preclude gender-biased analogies better than the cited supervised method.The method addresses the need to leave gender-specific words undebiased when applying subspace projection.Source: §1 · p. 2
• The experiments focus on gender association, although the authors state that their claims extend to other association types.The stated experimental focus is intended to allow fair comparison with prior work.Source: §1 · p. 2

2 Related Work

• SGNS optimizes observed word-context pairs against sampled negative examples and implicitly factorizes a shifted word-context PMI matrix; GloVe implicitly factorizes log co-occurrence counts.Levy and Goldberg (2014) proved the SGNS factorization result; Pennington et al. (2014) are cited for GloVe.Source: §2 · p. 2
• A word analogy a:b::x:y holds when a + (y − x) = b; for GloVe and SGNS, exactness without reconstruction error requires coplanarity and equal shifted PMI across the word pairs.The exactness conditions are attributed to Ethayarajh et al. (2018).Source: §2 · p. 2
• WEAT uses cosine similarity to compare target-word sets’ associations with two attribute-word sets, but evidence that it measures association well is limited beyond intuitive examples.Caliskan et al. (2017) is cited for the test and its reported science/art association result.Source: §2 · p. 2
• Bolukbasi et al. (2016) removed projections onto a gender-bias subspace to preclude biased analogies, but provided no theoretical guarantee of unbiased vectors; learning neutral embeddings from scratch required model-specific objective changes.Their subspace was defined by principal components of gender-relation vectors.Source: §2 · p. 3

3 Provably Debiasing Embeddings

• Unbiasedness with respect to S means that each word outside S has equal matrix entries for the two words in every pair in S.The definition treats the entries as interchangeable without changing the embeddings; it does not distinguish socially acceptable from undesirable associations.Source: §3 · p. 3
• Under perfect reconstruction and stated geometry assumptions, projection onto the complement of the bias subspace makes the reconstructed matrix unbiased with respect to S.The result applies to embedding models that implicitly factorize a matrix, including SGNS and GloVe.Source: §3 · p. 3
• The guarantee requires the bias subspace to span the pairwise difference vectors, and word vectors should not be normalized before debiasing.With no reconstruction error, projection is equivalent to training on a perfectly unbiased corpus.Source: §3 · p. 4
• Unbiasedness is guaranteed only for the chosen word pairs; a nonexhaustive set cannot establish that all gender associations have been erased.For example, debiasing with man–woman does not establish unbiasedness for policeman–policewoman.Source: §3 · p. 4

4 The Flaws of WEAT

• Even for words unbiased with respect to an attribute pair, WEAT can report nonzero association when the two attribute words occur at different corpus frequencies.For SGNS and GloVe, equal association requires equal attribute-word frequencies; the issue is not guaranteed to be absent for less-balanced attribute sets.Source: §4 · p. 4
• With one word in each target set, WEAT's effect size is maximal in one direction regardless of the target words.The effect size is the normalized difference between mean association scores; in the single-word case, the denominator reduces to the target-score difference divided by two.Source: §4 · pp. 4–5
• Changing the attribute-word sets can reverse whether door or curtain is reported as more male-associated, with both comparisons highly statistically significant.Table 1 contrasts the masculine/feminine and woman/man attribute choices.Source: Table 1 · p. 5, §4 · p. 5
• The paper concludes that WEAT's theoretical flaws cause it to overestimate association, and that the cosine-to-bias-subspace alternative can also overestimate it.The stated flaws make WEAT inappropriate as a word-embedding association metric.Source: §4 · p. 5

5 Relational Inner Product Association

• RIPA measures association as the inner product of a word vector and a relation vector derived from ordered word-pair differences.The relation vector is the first principal component of the pairwise difference vectors.Source: §5 · p. 5
• When defining the relation vector as a single vector, the sign indicates association direction and its unit length bounds RIPA by the word-vector norm.The value is bounded between −∥w∥ and ∥w∥, enabling comparisons across relation vectors.Source: §5 · p. 5, §5.1 · p. 6

5.1 Interpreting RIPA

• Under perfect reconstruction, SGNS RIPA can be interpreted through co-occurrence statistics, and unrelated defining pairs bring the association toward zero.If an analogy holds across the defining pairs, the shifted PMI is expected to be the same for each pair.Source: §5.1 · p. 6
• RIPA is robust to the choice of defining pair when those pairs have roughly the same difference vector.For example, using king–queen rather than man–woman has negligible impact under that condition.Source: §5.1 · p. 6
• For GloVe, learned bias terms make RIPA less interpretable; equivalence to the SGNS form is conjectured if those terms approximate log counts.The log-count equivalence is attributed to Levy et al. (2015).Source: §5.1 · p. 6

5.2 Statistical Significance

• RIPA has no attached statistical-significance measure because assessing whether an association is spurious depends on training-corpus co-occurrence frequencies.Word embeddings alone do not supply the frequency information needed to assess that uncertainty.Source: §5.2 · p. 6

6 Experiments

• The experiments ask how gender association reflects the embedding model, training corpus, or noise, and how to debias biased words without prior category labels.They use SGNS embeddings trained on Wikipedia and compare RIPA in the corpus and embedding space across biased, appropriate, and neutral word groups.Source: §6 · p. 7, §6.1 · p. 7

6.3 Debiasing without Supervision

• 94.9% of gender-appropriate analogies with strength at least 0.5 are preserved, versus 16.5% with Bolukbasi et al. (2016); gender-biased analogies retained are 36.7% versus 80.0%.Both approaches use the same projection method; the reported performance difference is attributed to how gender-appropriate words are selected.Source: Figure 1 · p. 8, §6.3 · p. 9

6.2 Breaking down Gender Association

• On average, SGNS does not make gender-neutral words more gendered, but it amplifies gender association for gender-biased and gender-appropriate words.Individual words may shift because of reconstruction error; the absolute-change amplification is statistically significant for both gendered groups (p < 0.001).Source: §6.2 · p. 8

6.3 Debiasing without Supervision

• Subspace projection requires identifying gender-appropriate words to leave untouched; the proposed unsupervised rule compares a word's association with gender-defining and bias-defining relation vectors.The rule debiases a word when its absolute association with the gender-defining vector is smaller, and is reported to preserve almost all appropriate analogies while precluding most biased ones.Source: §6.3 · pp. 8–9

• Under certain conditions, subspace projection is equivalent to training an implicitly factorizing model on a corpus unbiased with respect to the bias-subspace words.Source: §7 · p. 9
• WEAT has theoretical flaws that cause it to systematically overestimate bias, and contrived attribute sets can classify virtually any word as gender-biased relative to another.Source: §7 · p. 9
• RIPA is a new measure of association in word embeddings derived using subspace projection removal.Source: §7 · p. 9
• On average, SGNS does not make most words more gendered in the embedding space than in the training corpus, but it amplifies genderedness for gender-biased or gender-specific words.Source: §7 · p. 9