Source-linked AI summary
Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang
TL;DR
Coreference systems may rely on gender stereotypes in training data, creating a need for focused evaluation. The paper introduces WinoBias and tests three systems, finding a 21.1-point average F1 gap between pro- and anti-stereotypical cases. Gender-swapped data augmentation combined with debiased embeddings removes the demonstrated WinoBias bias without significantly affecting benchmark performance.
Problem
Coreference systems risk relying on societal stereotypes in training data, potentially affecting performance for some demographic groups.
Method
The paper introduces WinoBias, a Winograd-style benchmark using occupation-referenced people and balanced pro- versus anti-stereotypical coreference tests.
Results
21.1 F1 points is the average difference between pro-stereotypical and anti-stereotypical performance across rule-based, feature-rich, and neural systems.
Takeaways & Limitations
Gender-swapped training data combined with debiased word embeddings completely eliminates demonstrated WinoBias bias without significantly affecting overall coreference accuracy.
Takeaways & Limitations
The authors do not claim that WinoBias's template set is complete and describe it as representative of bias in existing systems.
Abstract
from arXiv · showhide
We introduce a new benchmark, WinoBias, for coreference resolution focused on gender bias. Our corpus contains Winograd-schema style sentences with entities corresponding to people referred by their occupation (e.g. the nurse, the doctor, the carpenter). We demonstrate that a rule-based, a feature-rich, and a neural coreference system all link gendered pronouns to pro-stereotypical entities with higher accuracy than anti-stereotypical entities, by an average difference of 21.1 in F1 score. Finally, we demonstrate a data-augmentation approach that, in combination with existing word-embedding debiasing techniques, removes the bias demonstrated by these systems in WinoBias without significantly affecting their performance on existing coreference benchmark datasets. Our dataset and code are available at http://winobias.org.
1 Introduction
Coreference systems risk using gender stereotypes from training data, so WinoBias tests whether they resolve pronouns differently in pro- and anti-stereotypical scenarios. The paper finds substantial bias across three system types and proposes data augmentation that removes it without significantly affecting benchmark accuracy.
- Coreference resolution identifies phrases referring to the same entity, using rule-based, feature-based, and neural approaches.
- WinoBias tests whether systems link gendered pronouns more accurately to pro-stereotypical than anti-stereotypical occupations.Its examples use occupation references and are designed so pronoun gender is irrelevant to the correct coreference decision.
- 21.1 F1 points is the average performance difference between pro-stereotypical and anti-stereotypical conditions across three systems.The systems are rule-based, feature-rich, and end-to-end neural.
- Female entities are significantly underrepresented in OntoNotes 5.0, motivating an auxiliary corpus that swaps male and female entities.The original and swapped corpora are used together for training.
- Combining gender-swapped data augmentation with debiased word embeddings completely eliminates WinoBias bias without significantly affecting overall coreference accuracy.
2 WinoBias
WinoBias is a benchmark designed to expose gender bias in coreference resolution through occupation-based, Winograd-schema-style scenarios. It separates cases requiring semantic reasoning from those using syntactic information and compares pro-stereotypical with anti-stereotypical pronoun links.
- Dataset construction: WinoBias uses 40 occupations from US Department of Labor statistics to define gender-stereotypical and anti-stereotypical roles.Occupation pairs are used to construct test-case scenarios involving people referred to by their occupations.
- Test types: Type 1 sentences require world knowledge about circumstances because they contain no syntactic cues for resolving coreference.These are prototypical WinoCoRef-style cases.
- Test types: Type 2 sentences provide syntactic information and pronoun understanding, so both semantic and syntactic cues support disambiguation.The authors expect systems to perform well on these cases.
- Evaluation: WinoBias evaluates whether models make pro-stereotyped and anti-stereotyped coreference decisions with the same accuracy across both sentence types.The dataset is split into sections where correct linking requires either stereotypical or anti-stereotypical gender associations.
- Evaluation: The template set is intended to provide representative examples of bias in existing systems rather than a complete collection of cases.This qualification limits the benchmark’s template coverage.
3 Gender Bias in Co-reference
The paper identifies gender imbalance in training resources and auxiliary resources as sources of bias in coreference systems. It proposes gender-swapped training data, debiased embeddings, and balanced gender statistics to mitigate those sources.
- Bias sources: Over 80% of entities headed by gendered pronouns in OntoNotes 5.0 are male, and male mentions are more than twice as likely to contain a job title as female mentions.These trends hold across genres.
- Mitigation strategies: Gender swapping augments training with a corpus in which male and female entities are exchanged while non-gender-revealing correlations are preserved.Training on both original and swapped corpora is intended to eliminate correlations between gender and coreference cues.
- Mitigation strategies: The swapping procedure anonymizes named entities and applies worker-derived rules that replace gendered terms with opposite-gender realizations consistently within documents.Examples include pronoun, honorific, and kinship-term substitutions.
- Implementation: The paper limits gender-swapping rules to mentions whose head word is a gender pronoun, excluding cases such as “his mother.”A head finder identifies the head word of each mention.
- Mitigation strategies: Debiased word embeddings replace GloVe vectors because coreference systems can inherit gender associations from pretrained embeddings.The paper specifically uses debiased vectors from Bolukbasi et al. (2016).
- Mitigation strategies: Gender lists are balanced by adjusting male and female counts in corpus-based statistics used by feature-rich and rule-based systems.These lists record noun-phrase frequencies in male, female, neutral, and plural contexts.
4 Results
The evaluation examines three representative coreference systems on WinoBias and related datasets, including pro/anti-stereotypical comparisons and debiasing methods.
- Three representative systems—rule-based, feature-rich, and end-to-end neural—are evaluated on WinoBias and OntoNotes.
- WinoBias evaluations separate Type-1 and Type-2 sentences into pro-stereotypical and anti-stereotypical conditions.
- Table 2 reports F1 on OntoNotes and WinoBias development sets, with significance marked for pro/anti-stereotypical differences.
- Table 3 reports F1 on OntoNotes and WinoBias test sets from single runs intended to support the development-set conclusions.
- Table 4 compares performance on the original and gender-reversed anonymized development datasets.
5 Related Work
Related work situates gender bias in coreference within broader evidence that biased associations can arise from data and appear across machine-learning systems.
- Word embeddings, vision and language models, and online news have been reported to encode or reflect sexist stereotypes.
- Imbalanced datasets can cause under-represented samples to receive less influence, motivating regularization and other fair-classifier methods.
- Concurrent work independently studies gender bias in job-title-based Winograd-style coreference data, while this work emphasizes debiasing methods.
6 Conclusion
The conclusion reports gender bias across three coreference systems and presents detection and debiasing approaches that preserve benchmark performance.
- The paper provides a method for detecting bias in coreference, a prototypical NLP task where bias may amplify societal stereotypes.
- The authors note that a concurrent dataset additionally includes gender-neutral pronouns and examples containing one job title instead of two.
- Three systems are significantly gender biased, but sufficiently strong alternative cues can allow systems to ignore that bias.
- The proposed methods aim to improve robustness to spurious gender-biased cues without significant penalties on benchmark-dataset performance.