Source-linked AI summary
OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings
Sunipa Dev, Tao Li, Jeff M Phillips, Vivek Srikumar
TL;DR
Word embeddings carry stereotypical associations, and projective debiasing can also erase valid information. OSCaR corrects associations through subspace operations; gender experiments report mitigation comparable to projective approaches while retaining more valid associations.
Problem
Projective methods can mitigate stereotypical bias but may also remove valid information from word embeddings.
Source: §1 · p. 1Method
OSCaR orthogonalizes and rectifies subspaces of incorrectly associated concepts, and the paper evaluates bias and information retention with NLI-based measures.
Source: §1 · p. 2, §3.3 · p. 4Results
OSCaR mitigates gender bias as well as projective approaches while retaining more valid associations.
Source: §1 · p. 2Takeaways & Limitations
The paper argues that debiasing should target specific associations for a given scenario or task rather than apply one operation uniformly.
Source: §8 · p. 9Beyond-binary gender associations are poorly represented, and quantifying their effects and attenuation is outside the paper’s scope.
Source: §2 · p. 2
Abstract
from arXiv · showhide
Language representations are known to carry stereotypical biases and, as a result, lead to biased predictions in downstream tasks. While existing methods are effective at mitigating biases by linear projection, such methods are too aggressive: they not only remove bias, but also erase valuable information from word embeddings. We develop new measures for evaluating specific information retention that demonstrate the tradeoff between bias removal and information retention. To address this challenge, we propose OSCaR (Orthogonal Subspace Correction and Rectification), a bias-mitigating method that focuses on disentangling biased associations between concepts instead of removing concepts wholesale. Our experiments on gender biases show that OSCaR is a well-balanced approach that ensures that semantic information is retained in the embeddings and bias is also effectively mitigated.
1 Introduction
Word embeddings encode stereotypical associations, while projective debiasing can also erase valid information. OSCaR targets biased associations through subspace correction, and the experiments report bias mitigation alongside greater retention of valid associations.
1 Introduction
- Existing projection approaches can leave residual bias or remove useful associations, potentially hindering tasks such as coreference resolution.The paper gives the association between pregnant, female, and mother as an example of information that can be unnecessarily altered.Source: §1 · p. 1
- Without explicit bias mitigation, a GloVe-based NLI model assigned 97% entailment to “A matriarch sold a watch” / “A woman sold a watch”; after projective debiasing, entailment fell to 16% and neutral rose to 62%.Source: §1 · p. 1
- OSCaR orthogonalizes and rectifies subspaces of incorrectly associated concepts, while leaving other embeddings unchanged or stretching them in a graded manner.The approach aims to make minimal changes that preserve desired associations.Source: §1 · p. 2
- The paper develops NLI-based tests to evaluate both bias mitigation and retention of valid information.Its motivation is that word-embedding associations can contribute to unfair downstream decisions.Source: §1 · pp. 1–2
- OSCaR mitigates gender bias as well as projective approaches while retaining more valid associations.The paper presents it as a well-balanced approach.Source: §1 · p. 2
2 Bias, Gender and NLP
This section frames gender bias as stereotypical associations in language representations and focuses the experiments on male–female relations. It also identifies beyond-binary gender associations as outside the paper’s evaluation scope.
2 Bias, Gender and NLP
- The experiments identify a gender subspace using terms associated with male and female, following common NLP practice.The paper notes that gender is more complex than this experimental framing.Source: §2 · p. 2
- The paper uses male–female relations to occupations to demonstrate disassociation between concepts.Many experiments focus on these associations because they are explicitly encoded in word representations and measured by standard metrics.Source: §2 · p. 2
- Beyond-binary gender associations are poorly represented and their quantification and attenuation are outside the paper’s scope.The paper encourages further research on potential harms arising through omission.Source: §2 · p. 2
3 Quantifying Bias and Information Retained
The evaluation combines embedding-geometry measures with NLI probes to assess stereotypical bias and retained information. It introduces tests for valid gender information alongside measures of unwanted associations.
3 Quantifying Bias and Information Retained
- The evaluation combines intrinsic embedding-geometry metrics with extrinsic measurements of downstream NLI outputs.The section organizes these measures around bias and information retention.Source: §3 · pp. 2–3
3.1 Designing Evaluation Data
- NLI probes distinguish incorrect gender associations from valid gender information by testing whether sentence-pair labels are neutral, entailed, or contradicted.Existing embedding-quality tasks do not directly assess definitionally gendered information in sentence pairs.Source: §3.1 · p. 3
3.2 Measuring Stereotypical Associations
- Stereotypical associations are measured with intrinsic WEAT and an extrinsic NLI neutrality dataset of about 1.9 million template-generated pairs.The neutrality examples are intentionally constructed to have the label neutral.Source: §3.2 · p. 3
3.3 Measures for Information Preservation
- WEAT* measures meaningful male–female associations, while SIRT uses entailment and contradiction data to evaluate retained gender information in model predictions.Higher Net Entail, Fraction Entail, Net Contradict, and Fraction Contradict scores indicate more retained information.Source: §3.3 · pp. 3–4
4 Orthogonal Subspace Correction and Rectification (OSCAR)
OSCAR corrects associations between selected concept directions through a graded rotation rather than removing an entire concept subspace. It confines adjustment to the span of those directions, preserving components outside it.
4 Orthogonal Subspace Correction and Rectification (OSCAR)
- OSCAR applies a graded rotation to orthogonalize the selected concept directions while leaving information outside their span unchanged.Points near the occupation direction move most, those near the male-vs-female direction move little, and other points receive graded adjustments.Source: §4 · pp. 4–5
- It identifies unit directions for two concepts from word lists, spans their subspace S, and adjusts only its two coordinates while fixing the other d-2 components.The running examples are male-vs-female and occupation directions.Source: §4 · p. 4
- Within S, f leaves v1 unchanged, maps v2 to an orthogonal direction, and smoothly rotates other vectors by an angle determined from their position.The angle-selection cases depend on which quadrant contains the vector.Source: §4 · pp. 4–5
- The operation is almost everywhere differentiable, enabling use with contextualized embeddings and integration into gradient-based fine-tuning.As post-processing, it has relatively low computational cost and can be adapted to task-specific subspaces.Source: §4 · p. 5
5 Experimental Methodology
The methodology compares OSCAR with projection-based debiasing methods and evaluates embeddings through NLI and other tests. It also describes contextualized-embedding experiments and notes train/test word overlap that may favor two baselines.
5 Experimental Methodology
- Comparators include Linear Projection, which removes a selected direction; Hard Debiasing equalizes selected pairs; and INLP iteratively removes residual-bias directions.Hard Debiasing is attributed to Bolukbasi et al. (2016), and INLP to Ravfogel et al. (2020).Source: §5 · pp. 5–6
- Unlike Hard Debiasing and INLP, OSCAR uses no predetermined list of words to exempt from debiasing, and its training and evaluation words are kept separate.The occupation words used to identify the subspace are disjoint from those used in WEAT and NLI tests.Source: §5 · pp. 6–7
- For contextualized embeddings, the methods are applied to RoBERTa's context-independent first layer; fine-tuning then updates the initialized subword embeddings.INLP is likewise restricted to that layer for consistency, and is called INLP⋆.Source: §5 · p. 6
- NLI models are trained on SNLI because MultiNLI showed stronger implicit gender associations, making it a confounding factor for evaluating embedding bias.With MultiNLI, over 90% of sentence pairs that should be neutral were classified incorrectly; with SNLI, the rate was under 70%.Source: §5 · p. 6
- HD and INLP training lists overlap with words in WEAT and NLI evaluation, which the authors say may give those methods an advantage.The authors allowed the overlap because the lists were recommended for the respective methods.Source: §5 · p. 6
6 Experimental Results
Across intrinsic and NLI-based evaluations, OSCaR reduces gender bias while retaining valid gender information. Its retention is strongest across the evaluated name, word, entailment, and contradiction tests, with projection methods showing losses on some measures.
Intrinsic measurement of bias
- Across intrinsic bias metrics, OSCaR substantially reduces bias and performs on par with the best methods.Its best WEAT result is on occupations, and rectifying occupations also generalizes to other tasks.Source: Table 1 · p. 7, §6 · p. 7
Extrinsic measurement of bias
- On RoBERTa, OSCaR is significantly more successful at debiasing than other methods; on GloVe, it is second best on each measure.Most methods, including OSCaR, improve neutrality without substantially sacrificing F1, except HD on GloVe.Source: Table 2 · p. 7, §6 · p. 7
Intrinsic metric of male-vs-female information preserved
- OSCaR performs at or near best in WEAT* retention tests of he–she, gendered words, and statistically male and female names.LP retains the least information for he–she and other gendered-word tests; HD’s performance on specified words does not generalize to names.Source: §6 · pp. 7–8, Table 3 · p. 8
Extrinsic metric of gendered information preserved
- OSCaR outperforms LP, HD, and INLP on all four SIRT scores for valid gender entailments and contradictions.RoBERTa retains information better after debiasing; LP, OSCaR, and HD perform similarly to baseline, while only INLP* with multiple projections noticeably drops in SIRT score.Source: Table 4 · p. 8, §6 · p. 8
7 Discussion & Conclusion
OSCaR rectifies associations by re-orthogonalizing concepts, aiming to preserve informative components while separating associations that might otherwise be lost or overlapped. The discussion presents possible extensions beyond gender and NLP, while noting that the experiments are limited to English.
7 Discussion & Conclusion
- OSCaR re-orthogonalizes concepts to dissociate associations that may otherwise be lost or overlapped in distributed representations.The method is described as computationally lightweight and sub-differentiable, allowing adaptive application without extensive retraining.Source: §7 · p. 8
- The authors envision applying OSCaR to other unwanted associations and to vectorized representations of images, graphs, or spatial data.They state that informative components of the features remain unchanged because OSCaR decouples specific associations.Source: §7 · pp. 8–9
- All experiments use English embeddings, models, and tests; extending the analysis to other languages is future work.Source: §7 · p. 8
8 Broader Impact
The broader-impact discussion argues for task- and bias-specific debiasing rather than one operation applied uniformly across embeddings.
8 Broader Impact
- Debiasing needs vary by task and word group, so continuously disassociating only the biased associations for a given scenario is advantageous.Source: §8 · p. 9
Word Embeddings Supplementary Material
The discussion says orthogonal correction can adapt to associations involving race or religion, but comprehensive evaluation of these biases is harder when fewer words support templates or tests.
A Discussion
- Orthogonal correction is easy to adapt to associations such as race-related good–bad notions or religion.Source: A Discussion · p. 11
- Comprehensive bias-reduction and information-retention evaluation is harder for these cases because fewer words are available for templates or tests.The authors leave this evaluation challenge for future exploration.Source: A Discussion · p. 11
B Comparison of Debiasing Methods
The methods differ in their word-list requirements: OSCAR uses separate subspaces, while hard debiasing and INLP rely on additional, longer lists. Those extra lists can reduce separation between evaluation and training words.
B Comparison of Debiasing Methods
- OSCAR determines male-vs-female and occupation subspaces, using two small word lists of roughly 10 words each.The other compared methods determine only the gender subspace; INLP does so iteratively.Source: B Comparison of Debiasing Methods · p. 11
- Linear projection and OSCAR need no supplementary word lists for debiasing, unlike hard debiasing and INLP.Hard debiasing uses four lists, while INLP uses a hard-coded list to generate a set of words it does not debias.Source: B Comparison of Debiasing Methods · p. 11
- Hard debiasing and INLP require lengthy supplementary word lists, which can reduce train/test separation in bias evaluations and make results better than in fairer settings.The passage notes that these lists can be cumbersome because not every meaningfully gendered word can be anticipated.Source: B Comparison of Debiasing Methods · p. 11
C Supporting Experiments
The supporting experiments favor SNLI over MultiNLI because MultiNLI expresses more gender bias. Standard similarity and analogy tests suggest embedding structure is preserved, but they cannot establish that bias is removed or valid gender associations retained.
C.1 SNLI versus MultiNLI
- MultiNLI expresses more gender bias than SNLI, so the experiments use SNLI to focus on bias in language representations rather than task data.The comparison is reported in Table S2.Source: §C.1 · p. 11, Table S2 · p. 12
C.2 Standard Tests for Word Embedding Quality
- Word-similarity and analogy tests show debiased GloVe models perform similarly to baseline GloVe, indicating that embedding structure is preserved.Table S3 reports the standard tests used for this comparison.Source: §C.2 · p. 11, Table S3 · p. 12
- These standard tests omit many sensitive gender comparisons, so they cannot establish bias removal or retention of valid gender associations.The paper treats them as preliminary information-retention checks and calls for more targeted tests.Source: §C.2 · p. 12
C.3 Male/Female Names and Debiasing
The supplement compares gender-association retention across names and reports a profession-level bias measure. It also documents the word lists and code used to determine subspaces and construct evaluation templates.
C.3 Male/Female Names and Debiasing
- The experiments use he and she and occupation terms to determine subspaces, then test retention with separate statistically gendered name sets.The name sets are drawn from Social Security data and filtered to names frequent in Wikipedia.Source: §C.3 · p. 12, Gendered Words for Subspace… · p. 13, Definitionally Male-vs-female Words… · p. 15
- Correction yields higher WEAT* associations than projection for he-she and other gender-word and name comparisons, indicating greater retention of correctly male-vs-female information.The comparisons include statistically male and female names not used to determine the subspace.Source: §C.3 · p. 12, Table S4 · p. 12
C.4 TPR-Gap-RMS
- RoBERTa and RoBERTa HD perform on par on TPR-Gap-RMS, while linear projection, iterative projection, and OSCAR perform close to each other.TPR-Gap-RMS aggregates male-vs-female bias scores over professions; lower scores indicate less bias.Source: §C.4 · pp. 12–13, Table S5 · p. 13
D.1 Code
- The supplement provides the code for template generation, correction-based debiasing, and debiasing contextual embeddings.The code is identified as available in the OSCaR repository.Source: §D.1 · p. 13
D.3 Word Lists for HD
- The hard-debiasing materials include a filtered gender-word list and a separate list of male-female pairs to equalize.The supplement presents the paired words used in the equalization operation.Source: §D.3 · pp. 13–14
D.4 Word Lists for INLP
- The INLP gender-word list contains 1,425 words and is filtered to remove words used for templates and WEAT.The source list is identified as gender_specific_full.json.Source: §D.4 · p. 14
D.5 Words Lists for Template Generation
- Template generation mostly retains the paper's word lists, but removes OSCAR occupation-subspace words from the occupation list to create a test/train split.The male-vs-female lists are also modified for SIRT entailment and contradiction templates.Source: §D.5 · p. 14