PaperlayerView exact revision on arXiv

Source-linked AI summary

Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods

Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang

arXiv:1804.06876v1cs.CLcs.AI

TL;DR

Coreference systems may rely on gender stereotypes in training data, creating a need for focused evaluation. The paper introduces WinoBias and tests three systems, finding a 21.1-point average F1 gap between pro- and anti-stereotypical cases. Gender-swapped data augmentation combined with debiased embeddings removes the demonstrated WinoBias bias without significantly affecting benchmark performance.

  • Problem

    Coreference systems risk relying on societal stereotypes in training data, potentially affecting performance for some demographic groups.

    Source: §1 · p. 1

  • Method

    The paper introduces WinoBias, a Winograd-style benchmark using occupation-referenced people and balanced pro- versus anti-stereotypical coreference tests.

    Source: §1 · p. 1, §2 · p. 2

  • Results

    21.1 F1 points is the average difference between pro-stereotypical and anti-stereotypical performance across rule-based, feature-rich, and neural systems.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Gender-swapped training data combined with debiased word embeddings completely eliminates demonstrated WinoBias bias without significantly affecting overall coreference accuracy.

    Source: §1 · p. 2

    The authors do not claim that WinoBias's template set is complete and describe it as representative of bias in existing systems.

    Source: §2 · p. 2

Abstract

from arXiv · show

We introduce a new benchmark, WinoBias, for coreference resolution focused on gender bias. Our corpus contains Winograd-schema style sentences with entities corresponding to people referred by their occupation (e.g. the nurse, the doctor, the carpenter). We demonstrate that a rule-based, a feature-rich, and a neural coreference system all link gendered pronouns to pro-stereotypical entities with higher accuracy than anti-stereotypical entities, by an average difference of 21.1 in F1 score. Finally, we demonstrate a data-augmentation approach that, in combination with existing word-embedding debiasing techniques, removes the bias demonstrated by these systems in WinoBias without significantly affecting their performance on existing coreference benchmark datasets. Our dataset and code are available at http://winobias.org.

1 Introduction

Coreference systems risk using gender stereotypes from training data, so WinoBias tests whether they resolve pronouns differently in pro- and anti-stereotypical scenarios. The paper finds substantial bias across three system types and proposes data augmentation that removes it without significantly affecting benchmark accuracy.

  • Coreference resolution identifies phrases referring to the same entity, using rule-based, feature-based, and neural approaches.Source: §1 · p. 1
  • WinoBias tests whether systems link gendered pronouns more accurately to pro-stereotypical than anti-stereotypical occupations.Its examples use occupation references and are designed so pronoun gender is irrelevant to the correct coreference decision.Source: §1 · p. 1
  • 21.1 F1 points is the average performance difference between pro-stereotypical and anti-stereotypical conditions across three systems.The systems are rule-based, feature-rich, and end-to-end neural.Source: §1 · p. 2
  • Female entities are significantly underrepresented in OntoNotes 5.0, motivating an auxiliary corpus that swaps male and female entities.The original and swapped corpora are used together for training.Source: §1 · p. 2
  • Combining gender-swapped data augmentation with debiased word embeddings completely eliminates WinoBias bias without significantly affecting overall coreference accuracy.Source: §1 · p. 2

2 WinoBias

WinoBias is a benchmark designed to expose gender bias in coreference resolution through occupation-based, Winograd-schema-style scenarios. It separates cases requiring semantic reasoning from those using syntactic information and compares pro-stereotypical with anti-stereotypical pronoun links.

Dataset construction

  • WinoBias uses 40 occupations from US Department of Labor statistics to define gender-stereotypical and anti-stereotypical roles.Occupation pairs are used to construct test-case scenarios involving people referred to by their occupations.Source: §2 · p. 2, Table 1 · p. 2

Test types

  • Type 1 sentences require world knowledge about circumstances because they contain no syntactic cues for resolving coreference.These are prototypical WinoCoRef-style cases.Source: §2 · p. 2
  • Type 2 sentences provide syntactic information and pronoun understanding, so both semantic and syntactic cues support disambiguation.The authors expect systems to perform well on these cases.Source: §2 · p. 2

Evaluation

  • WinoBias evaluates whether models make pro-stereotyped and anti-stereotyped coreference decisions with the same accuracy across both sentence types.The dataset is split into sections where correct linking requires either stereotypical or anti-stereotypical gender associations.Source: §2 · pp. 2–3
  • The template set is intended to provide representative examples of bias in existing systems rather than a complete collection of cases.This qualification limits the benchmark’s template coverage.Source: §2 · p. 2

3 Gender Bias in Co-reference

The paper identifies gender imbalance in training resources and auxiliary resources as sources of bias in coreference systems. It proposes gender-swapped training data, debiased embeddings, and balanced gender statistics to mitigate those sources.

Bias sources

  • Over 80% of entities headed by gendered pronouns in OntoNotes 5.0 are male, and male mentions are more than twice as likely to contain a job title as female mentions.These trends hold across genres.Source: §3.1 · p. 3

Mitigation strategies

  • Gender swapping augments training with a corpus in which male and female entities are exchanged while non-gender-revealing correlations are preserved.Training on both original and swapped corpora is intended to eliminate correlations between gender and coreference cues.Source: §3.1 · p. 3
  • The swapping procedure anonymizes named entities and applies worker-derived rules that replace gendered terms with opposite-gender realizations consistently within documents.Examples include pronoun, honorific, and kinship-term substitutions.Source: §3.1 · p. 3

Implementation

  • The paper limits gender-swapping rules to mentions whose head word is a gender pronoun, excluding cases such as “his mother.”A head finder identifies the head word of each mention.Source: §3.1 · p. 3

Mitigation strategies

  • Debiased word embeddings replace GloVe vectors because coreference systems can inherit gender associations from pretrained embeddings.The paper specifically uses debiased vectors from Bolukbasi et al. (2016).Source: §3.2 · p. 3
  • Gender lists are balanced by adjusting male and female counts in corpus-based statistics used by feature-rich and rule-based systems.These lists record noun-phrase frequencies in male, female, neutral, and plural contexts.Source: §3.2 · p. 3

4 Results

The evaluation examines three representative coreference systems on WinoBias and related datasets, including pro/anti-stereotypical comparisons and debiasing methods.

  • Three representative systems—rule-based, feature-rich, and end-to-end neural—are evaluated on WinoBias and OntoNotes.Source: §4 · p. 3
  • WinoBias evaluations separate Type-1 and Type-2 sentences into pro-stereotypical and anti-stereotypical conditions.Source: §4 · p. 3
  • Table 2 reports F1 on OntoNotes and WinoBias development sets, with significance marked for pro/anti-stereotypical differences.Source: Table 2 · p. 4
  • Table 3 reports F1 on OntoNotes and WinoBias test sets from single runs intended to support the development-set conclusions.Source: Table 3 · p. 4
  • Table 4 compares performance on the original and gender-reversed anonymized development datasets.Source: Table 4 · p. 4

5 Related Work

Related work situates gender bias in coreference within broader evidence that biased associations can arise from data and appear across machine-learning systems.

  • Word embeddings, vision and language models, and online news have been reported to encode or reflect sexist stereotypes.Source: §5 · p. 5
  • Imbalanced datasets can cause under-represented samples to receive less influence, motivating regularization and other fair-classifier methods.Source: §5 · p. 5
  • Concurrent work independently studies gender bias in job-title-based Winograd-style coreference data, while this work emphasizes debiasing methods.Source: §5 · p. 5

6 Conclusion

The conclusion reports gender bias across three coreference systems and presents detection and debiasing approaches that preserve benchmark performance.

  • The paper provides a method for detecting bias in coreference, a prototypical NLP task where bias may amplify societal stereotypes.Source: §6 · p. 5
  • The authors note that a concurrent dataset additionally includes gender-neutral pronouns and examples containing one job title instead of two.Source: §6 · p. 5
  • Three systems are significantly gender biased, but sufficiently strong alternative cues can allow systems to ignore that bias.Source: §6 · p. 5
  • The proposed methods aim to improve robustness to spurious gender-biased cues without significant penalties on benchmark-dataset performance.Source: §6 · p. 5

Related papers

Loading 1804.06876v1…