PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward Gender-Inclusive Coreference Resolution

Yang Trista Cao, Hal Daumé

arXiv:1910.13913v4cs.CL

TL;DR

Coreference resolution requires inferences about people that can encode gender bias and harm trans and cis stakeholders. The paper develops a sociological and sociolinguistic framework plus two datasets to study bias in annotations and models, finding gaps in system performance and problems with treating crowd judgments as gold standards.

  • Problem

    Coreference systems infer properties of people, creating risks of systemic gender bias and harms for binary and non-binary trans and cis stakeholders.

  • Method

    The paper combines a nuanced gender taxonomy with MAP counterfactual annotation studies and GICOREF evaluations of gender-inclusive coreference data.

  • Results

    The datasets reveal significant gaps in system performance, while annotation analyses show that crowdworker judgments can be problematic as a gold standard.

  • Takeaways & Limitations

    Gender-inclusive datasets and systems may require attention to experiential expertise alongside conventional crowd annotation and evaluation practices.

  • Takeaways & Limitations

    The study is limited to English, whose lack of grammatical gender makes extensions to languages with grammatical gender non-trivial.

Abstract

from arXiv · show

Correctly resolving textual mentions of people fundamentally entails making inferences about those people. Such inferences raise the risk of systemic biases in coreference resolution systems, including biases that can harm binary and non-binary trans and cis stakeholders. To better understand such biases, we foreground nuanced conceptualizations of gender from sociology and sociolinguistics, and develop two new datasets for interrogating bias in crowd annotations and in existing coreference resolution systems. Through these studies, conducted on English text, we confirm that without acknowledging and building systems that recognize the complexity of gender, we build systems that lead to many potential harms.

1 Introduction

Coreference systems infer properties of people, creating risks of gender bias and harms for binary and non-binary trans and cis stakeholders. The paper responds with a sociolinguistic gender framework, two datasets, and analyses of annotation and model bias.

  • Motivation: Coreference resolution requires inferences about real-world entities, including people, which can produce unlicensed gender-related inferences and harms.Gender may appear explicitly in linguistic forms or implicitly through social expectations and stereotypes.
  • Motivation: Existing coreference systems have been shown to over-rely on social stereotypes when resolving HE and SHE pronouns.Earlier work primarily examined binary gender discrimination in trained systems.
  • Motivation: Folk models treating gender as binary, assigned at birth, immutable, and perfectly aligned with linguistic forms can produce exclusionary and harmful technology.The concern applies to binary and non-binary trans and cis people.
  • Contributions: The paper develops a gender framework distinguishing lexical, referential, grammatical, and social gender to analyze how bias enters NLP systems.It connects sociological and sociolinguistic conceptions of gender to gender-bias analysis in NLP.
  • Contributions: The authors construct MAP for counterfactual manipulations and GICOREF for coreference texts written by and about trans people.They use these datasets to examine bias in annotation and model-definition stages.

2 Related Work

Prior work established that coreference systems can reproduce binary gender stereotypes, while related NLP research examined gender bias across many tasks. This paper extends that discussion through a trans-inclusive perspective on gender.

  • Coreference bias: Rudinger et al. and Zhao et al. used Winograd-esque examples to test occupational gender stereotypes in coreference systems.Rudinger et al. found that humans reliably resolved the examples while systems often failed in gender-stereotypical ways.
  • Contribution: The paper builds on prior work by taking a trans-inclusive perspective on gender in coreference resolution.Its datasets and analysis address linguistic references involving binary and non-binary trans people.
  • Broader NLP context: Gender bias has also been studied in natural language inference, word embeddings, sentiment analysis, and machine translation.These related NLP studies provide broader context for examining gender bias beyond coreference resolution.

3 Linguistic & Social Gender

Gender is socially complex and contested, while linguistic gender categories do not map one-to-one onto social gender. The paper uses this distinction to analyze how gender cues shape coreference judgments and potential bias.

  • Sociological gender: Gender includes internal experience, expression, social expectations, and social recognition, with gender self-determination placing authority in each person over their identity.Trans-inclusive models also recognize that gender can extend beyond a female/male binary and vary across time or context.
  • Linguistic gender: Linguistic gender comprises distinct lexical, grammatical, and referential categories that need not correspond to one another or to social gender.Lexical gender belongs to linguistic units, grammatical gender classifies nouns by agreement, and referential gender links expressions to entities.
  • Linguistic gender: Referential gender applies when an entity’s gender or sex is linguistically realized, including through SHE, HE, ZE, EM, and singular THEY.It differs from non-referential lexical gender, which can occur without a real-world referent.
  • Implications: Because gender notions interact without one-to-one alignment, the paper urges researchers to specify which notions of gender their bias analyses use.The study applies this framework to gender bias in coreference resolution systems.
  • Social and linguistic interplay: In English, social expectations can lead listeners to infer gender from occupations or roles even though English lacks grammatical gender.For example, speakers may infer “female” from “My cousin is a librarian” through social rather than grammatical or lexical gender.

4 Bias in Human Annotation

The study tests how ablating different linguistic gender cues changes human coreference judgments in MAP, finding that pronouns and names matter most while lexical gender cues show little significant effect.

  • Dataset and ablation design: The ablation study hides pronouns, names, semantically gendered nouns, and terms of address through counterfactual substitutions.The study uses forward selection to separate syntactic cues from stereotypes and backward selection to measure each cue’s impact amid the others.
  • Dataset and ablation design: The authors construct MAP examples from Wikipedia with two possible antecedent names, without requiring the names to match in gender.This design enables testing how annotators use additional gender information when resolving a pronoun.
  • Annotation results: Removing pronouns significantly reduces annotation accuracy, indicating that gender-based inferences, especially social stereotypes, strongly influence human coreference judgments.Accuracy is essentially the same for ZERO and ¬PRO, suggesting that removing explicit binary gender from pronouns also removes the impact of other linguistic gender cues.
  • Annotation results: Names are another significant cue, whereas lexical gender cues do not significantly affect accuracy, partly because semantically gendered nouns and address terms are relatively infrequent.Semantically gendered nouns occur in 49% of examples, while terms of address occur in only 3%.
  • Annotation results: Annotators’ certainty follows the same trend as accuracy, suggesting that they generally recognize when they are unsure.The experiments collected certainty ratings alongside binary coreference judgments.

5 Bias in Model Specifications

The paper examines how gender assumptions enter coreference models through published research practices, model design, and evaluation data. It finds strong binary assumptions in NLP literature and substantial performance weaknesses on gender-inclusive references.

  • 5.2 System performance on MAP: Five existing systems are evaluated on ablated MAP data and on the new GICOREF dataset to examine model behavior under gender-inclusive conditions.
  • 5.1 Cis-normativity in published NLP papers: Only 5.6% of analyzed coreference papers explicitly model gender as inclusive of non-binary identities.
  • 5.1 Cis-normativity in published NLP papers: The literature analysis finds that most coreference papers rely on a binary “folk” theory of gender, with non-binary and binary trans identities broadly erased.
  • 5.2 System performance on MAP: System accuracy drops dramatically when referential gender is removed from pronouns, indicating heavy reliance on gender-based inferences.
  • 5.3 System behavior on gender-inclusive data: GICOREF contains naturally occurring texts with non-binary identities, varied pronouns, genderfluid people, misgendering, deadnaming, and non-heteronormative relationships.The dataset contains 95 documents from Wikipedia, LGBTQ periodicals, and fan fiction, with author annotation and adjudication.

6 Discussion and Moving Forward

The paper finds substantial gaps in coreference performance on text involving binary and non-binary trans referents, while also questioning crowd judgments as gold standards. It proposes sociologically informed analysis and datasets as a roadmap for measuring and addressing these biases, but remains constrained by English, Western-dominant data, limited naturally occurring inclusive text, and contested frameworks of gender.

  • Discussion and Moving Forward: The MAP and GICOREF datasets reveal system-performance gaps and show that crowdworker judgments can be problematic as gold standards.The authors suggest experiential experts may be needed to build more gender-inclusive datasets and systems.
  • Discussion and Moving Forward: Annotation results may depend on annotator positionality, and the dataset remains Western-dominant despite including some non-Western non-binary expressions.The study used Mechanical Turk workers and acknowledges that the authors' own positionality introduced additional bias.
  • Discussion and Moving Forward: The study is limited to English, whose lack of grammatical gender makes extensions to languages with grammatical gender non-trivial.This is identified as the primary limitation of the study and analysis.
  • Discussion and Moving Forward: Limited availability of naturally occurring gender-inclusive text led to counterfactual manipulation, which cannot be performed flawlessly.Evaluation is also constrained because systems and metrics do not explicitly account for misgendering or deadnaming.
  • Discussion and Moving Forward: Because gender is fundamentally contested as a social construct, some results may apply only under particular frameworks.The authors present the work as a roadmap for future studies and as inspiration for measuring and reducing bias.

A Examples of Possible Bias in Data Annotation

Coreference annotation can encode gender bias when annotators infer social gender from names, lexical expressions, grammatical gender, or social roles before matching those inferences to pronouns. Ambiguous or unfamiliar names and role terms make the intended referent harder to determine.

  • Possible Bias in Data Annotation: Annotators may infer social gender from names and use that inference to resolve a pronoun, favoring Mary over John in “John and Mary visited her mother.”The passage notes that unfamiliar name stereotypes, including Chinese and Indian names, can make such judgments difficult.
  • Possible Bias in Data Annotation: When both candidate names suggest the pronoun’s inferred gender, as in “Sue and Mary visited her mother,” annotation becomes more difficult.The example illustrates ambiguity created by overlapping name-based gender expectations.
  • Possible Bias in Data Annotation: The annotation process plausibly involves inferring a referent’s social gender from a name, inferring the pronoun’s referential gender, and checking whether the two match.This mechanism explains why linguistic and social gender cues can influence coreference judgments.
  • Possible Bias in Data Annotation: The same inference pattern applies to lexical gender references, grammatical gender in languages such as Arabic, and social gender references.Social-role cases are described as least clear to most annotators and are the focus of prior coreference bias studies.

B Annotation of ACL Anthology Papers

This appendix lists the authors’ annotations of ACL Anthology papers using a multi-column coding scheme. The entries record paper-level yes/no or unavailable judgments across the listed dimensions.

  • Annotation of ACL Anthology Papers: The appendix introduces a complete annotation set for the papers considered in the ACL Anthology analysis.The authors state that each paper is annotated on a specified collection of items.
  • Annotation of ACL Anthology Papers: The tabulated records assign each listed paper sequences of Y, N, or unavailable marks across the annotation dimensions.The displayed entries include papers from Sidner (1981) through later work such as Rudinger et al. (2018) and Zhao et al. (2018a).

C Example GICoref Document from Wikipedia: Dana Zzyym

The example document follows Dana Alix Zzyym, an intersex activist and former sailor who sought a non-binary gender U.S. passport. It supplies multiple mentions and pronouns referring to Zzyym, their parents, and related entities across the document.

  • Example GICoref Document: The document introduces Dana Alix Zzyym as an intersex activist and former sailor who sought a non-binary gender U.S. passport.It identifies Zzyym as the first military veteran in the United States to seek such a passport.
  • Example GICoref Document: Zzyym’s childhood, military service, and relationship to the queer community provide biographical context for the document’s coreference chains.The text states that Zzyym joined the Navy as a machinist’s mate in 1978.
  • Example GICoref Document: The document describes Zzyym’s parents hiding their intersex status and Zzyym later discovering that identity and related surgeries independently.Several pronouns in the passage refer back to Zzyym, their parents, or both.
  • Example GICoref Document: Later passages connect Zzyym to the Intersex Campaign for Equality and to litigation over recognition of an appropriate passport gender marker.A federal court enjoined the State Department from relying on a binary-only policy to withhold the requested passport.

[[Author: cornheck]]

Crona finishes final exams and discusses the prospect of staying with Stein and Marie during break. Soul encourages Crona through anxiety, self-doubt, and remorse about their past conflict.

  • Final exams: Crona completes the final exams despite discomfort with the essay portion and the amount of writing required.The essay concerns the origins, history, and importance of partnered and grouped soul resonance.
  • Break plans: After leaving the lecture hall, Crona hesitates over how to spend the seasonal break.Crona considers whether to accept Stein and Marie’s invitation to stay with them.
  • After the exam: Soul and Crona continue comparing their exam experiences, including Black Star’s poor test-taking and Maka’s rapid completion of essay questions.Soul uses these comparisons to keep the interaction light after Crona’s apology.
  • Break plans: Soul frames Stein and Marie’s offer as an extended welcome and suggests they would probably be happy to have Crona.He also argues that staying alone through the summer would be lonely and boring.
  • Emotional support: Soul’s conversation helps Crona’s confidence briefly recover, although Crona’s self-doubt remains likely to return.The exchange moves Crona from hesitation toward a more genuine smile, while Soul doubts his own ability to sustain the improvement.
  • Emotional support: Crona apologizes for nearly killing and seriously hurting Soul during their first duel, while Soul deflects sole responsibility toward Ragnarok.Soul responds with surprise before using humor to soften Crona’s remorse.
Loading 1910.13913v4…