MISGENDERED: Limits of Large Language Models in Understanding Pronouns

Motivation

• MISGENDERED targets the gap left by gender-bias research focused primarily on female and male categories.The paper links this exclusion to potential harm against non-binary individuals.Source: §1 · pp. 1–2

Framework

• The framework declares an individual’s preferred pronouns and asks a model to fill a missing pronoun in a following sentence.It evaluates masked and auto-regressive models through a unified setup.Source: §1 · p. 2

Dataset

• The dataset spans five pronoun forms, 11 pronoun sets, and 3.8 million instances built from varied names and templates.Pronoun types include binary, gender-neutral, and neo-pronouns.Source: §1 · p. 2

Results

• 7.7% average accuracy for neo-pronouns and 34.2% for gender-neutral pronouns contrast with 75.9% for binary pronouns.The evaluation covers seven model families, including masked and auto-regressive models.Source: §1 · p. 2

Analysis

• Scarce training representation and memorized associations between names and pronouns may explain poor zero-shot performance.C4 contains 220× more masculine-pronoun tokens than xe tokens, and name-gender associations affect predictions.Source: §1 · p. 2

Adaptation

• Few-shot prompting raises neo-pronoun accuracy to 64.7%, but performance plateaus with additional examples.The paper notes that downstream bias mitigation remains unclear.Source: §1 · p. 2

Evaluation setup

• The framework varies explicit versus parenthetical declarations and declares between two and five pronoun forms.These declaration conditions are represented in the evaluation design.Source: Figure 2 · p. 3

Gender and pronouns

• Binary female-male pronouns are insufficient for the diverse and changing landscape of gender, motivating evaluation of singular they and neo-pronouns.The paper distinguishes preferred pronouns from gender identity and cautions that pronouns do not determine identity.Source: §2 · p. 3

Pronoun inventory

• The pronoun inventory contains binary, gender-neutral, and neo-pronoun groups organized across five grammatical forms.Table 1 lists the pronoun groups and forms used for evaluation.Source: Table 1 · p. 4

Evaluation setup

• MISGENDERED evaluates incorrect pronoun use by giving each instance a name, preferred pronouns, and a template with one missing pronoun.The dataset uses one pronoun group per individual rather than mixed sets such as they/she.Source: §2 · p. 4

3 MISGENDERED Framework

• MISGENDERED evaluates masked and auto-regressive language models with instances that declare an individual’s pronouns before a sentence containing a missing pronoun.The framework combines these evaluation instances with a unified method for both model types.Source: §3 · p. 4

3.1 Dataset Construction

• The dataset uses manually crafted templates populated with names and single preferred pronoun groups, covering binary, gender-neutral, and selected neo-pronouns.It includes five pronoun forms and excludes cases where an individual uses multiple pronoun sets.Source: §3.1 · p. 4

3.2.1 Constrained Decoding

• The evaluation constrains predictions to pronouns of the same form, using model-specific inputs and labels before selecting the output with each model’s loss function.This procedure provides a common prediction framework for masked and auto-regressive models.Source: §3.2.1 · pp. 4–5

3.3 Experiments

• The experiments compare models from BART, T5, GPT-2, GPT-J, OPT, BLOOM, and LLaMA while varying declaration format and the number of declared pronoun forms.Declaration formats are explicit and parenthetical, with two-to-five pronoun forms declared.Source: §3.3 · p. 5, Table 3 · p. 5
• The analysis relates zero-shot behavior to pronoun-token prevalence and possible memorization of name–pronoun associations, then tests few-shot adaptation with 2 to 20 examples.In-context experiments use GPT-J-6B, OPT-6.7b, and LLaMA-7B with explicit declarations of all five forms.Source: §3.3 · pp. 5–6

4.1 Direct Prompting

• Across all zero-shot experiments, language models averaged 7.7% accuracy for neo-pronouns and 34.2% for gender-neutral pronouns, indicating poor performance on both pronoun types.Source: §4.1 · p. 6
• Explicit declarations improve neo-pronoun accuracy from 5.9% to 9.5%, whereas parenthetical declarations work better for singular they and binary pronouns.Increasing declared forms helps neo-pronouns, but minimal declarations perform best for gender-neutral pronouns.Source: §4.1 · p. 6, Table 5 · p. 6, Table 6 · p. 6
• Model size has no uniform effect: OPT’s gender-neutral accuracy rises from 21.2% at 350m parameters to 94.2% at 6.7b, while BLOOM’s neutral-pronoun performance declines with size.BLOOM’s binary-pronoun performance increases with size, while its neo-pronoun performance remains relatively stable.Source: §4.1 · pp. 6–7, Figure 3 · p. 7
• Binary pronouns reach 75.4% masculine and 76.3% feminine accuracy, while thon leads neo-pronouns at 18.1% and xe follows at 14.1%.Pronoun forms also diverge: nominative accuracy is 81.1% for binary pronouns but 3.2% for neo-pronouns, whereas possessive-independent accuracy is 10.8% for non-binary and 62.6% for binary pronouns.Source: §4.1 · p. 7, Table 7 · p. 7, Table 8 · p. 7

4.2 Explaining Direct Prompting Results

• Sparse and semantically atypical corpus occurrences of non-binary pronouns, alongside memorized name-pronoun associations, help explain poor zero-shot performance.Neo-pronouns are substantially rarer than binary pronouns in C4, Open Web Text, and Pile; unisex names perform best for neo-pronouns, while gendered names perform worst.Source: §4.2 · pp. 7–8, Table 11 · p. 8, Table 12 · p. 8

4.3 In-Context Learning

• Few-shot prompting raises LLaMA-7B neo-pronoun accuracy to 64.7% at 20 shots, but performance can plateau, decline, or vary across models and shot counts.GPT-J and OPT improve for neo-pronouns only up to six shots, while models show different trends for singular they.Source: §4.3 · pp. 8–9

5 Related Work

• Existing studies address non-binary gender bias through specialized datasets, evaluation frameworks, perplexity analyses, and modeling desiderata, but coverage remains narrow.Earlier work often focuses on only a few neo-pronouns or evaluates few models in zero-shot settings; this paper evaluates a wider range of pronouns and forms.Source: §5 · p. 9

• Current language models heavily misgender individuals using non-binary pronouns, failing to use explicitly declared neopronouns and struggling even with gender-neutral pronouns like they.Source: §6 · p. 9
• The analysis suggests that poor performance may reflect scarce neo-pronouns in pre-training data and memorized associations between names and pronouns.Source: §6 · p. 9
• Few-shot prompting improves models’ neo-pronoun accuracy to as high as 64.7%, indicating some ability to adapt to new word use.Performance plateaus with more examples, and its effectiveness for mitigating downstream bias remains unclear.Source: §6 · p. 9, Table 13 · p. 9
• The authors release the dataset, code, and demo to facilitate further research on non-binary pronoun understanding and bias mitigation.Source: §6 · p. 9
• Future work should move beyond measuring misgendering toward techniques that detect and correct it, with affected non-binary and transgender people centrally involved.Source: §6 · p. 9

• Template-based measurements can be sensitive to template choice, so the findings are not a definitive verdict on misgendering by language models.Source: Limitations · p. 10
• The study evaluates only upstream language models and does not assess downstream applications, limiting conclusions about real-world use.Source: Limitations · p. 10
• The evaluation is restricted to English, a Western conception of gender, and names and birth-assigned genders from the United States.Source: Limitations · p. 10
• The framework excludes people using multiple pronoun sets and does not cover the full expanding range of non-binary pronouns.Source: Limitations · p. 10
• Larger models were omitted because of computational-budget limitations, leaving their preferred-pronoun accuracy unevaluated.Source: Limitations · p. 10

• The framework targets one specific type of misgendering harm within a broader evaluation of stereotyping, identity erasure, dead-naming, and related harms.Source: Ethics Statement · p. 10
• The absence of measured misgendering under this paradigm is not evidence that no misgendering or other gender harms occur.Source: Ethics Statement · p. 10
• Responsible model deployment requires scrutiny that is appropriate to the context of use.Source: Ethics Statement · p. 10

• The MISGENDERED dataset is created from templates used to evaluate pronoun prediction.Source: A Templates · p. 12
• Table 14 presents the templates used in the MISGENDERED framework.Source: A Templates · p. 12

• Models select the most probable pronoun from a set sharing the same form for each evaluation instance.The setup identifies the candidate with the highest probability or lowest loss.Source: B Constrained Decoding Example · p. 12
• Model-specific input and label formatting allows the evaluation to determine each pronoun's probability or loss.Source: B Constrained Decoding Example · p. 12

• The full dataset, code base, and demo are released to facilitate further work on misgendering by language models.Source: C Data and Code · p. 12
• The constrained-decoding figure describes selecting the highest-probability, or lowest-loss, pronoun among candidates with the same form.Source: Figure 4 · p. 13
• Evaluation dataset templates are provided to encourage researchers to build upon the MISGENDERED framework.Source: Table 14 · p. 14