PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Gender Bias in Natural Language Processing

Karolina Stanczak, Isabelle Augenstein

arXiv:2112.14168v1cs.CLcs.CY

TL;DR

Gender-bias research in NLP lacks sufficiently inclusive definitions, broad language coverage, systematic model testing, and coherent evaluation practices. The paper surveys 304 papers, connects social-science and NLP definitions, reviews datasets and lexica, and compares detection and mitigation methods. It identifies four major limitations and recommends addressing them as priorities for future research.

  • Problem

    Gender-bias research uses limited definitions and coverage, while many NLP models lack accompanying bias and ethical evaluations.

  • Method

    The paper surveys 304 papers, connecting social-science gender concepts with NLP bias definitions and reviewing resources, detection methods, and mitigation methods.

  • Results

    The survey identifies four major limitations: binary gender modeling, predominantly monolingual research, insufficient model bias testing, and flawed methodologies lacking broad definitions and evaluation baselines.

  • Takeaways & Limitations

    The paper treats overcoming these limitations as crucial for further development of gender-bias research and recommends more inclusive definitions, datasets, and evaluations.

  • Takeaways & Limitations

    The surveyed field is constrained by limited definitions of gender bias, inadequate evaluation baselines and pipelines, and predominantly English or other high-resource-language coverage.

Abstract

from arXiv · show

Language can be used as a means of reproducing and enforcing harmful stereotypes and biases and has been analysed as such in numerous research. In this paper, we present a survey of 304 papers on gender bias in natural language processing. We analyse definitions of gender and its categories within social sciences and connect them to formal definitions of gender bias in NLP research. We survey lexica and datasets applied in research on gender bias and then compare and contrast approaches to detecting and mitigating gender bias. We find that research on gender bias suffers from four core limitations. 1) Most research treats gender as a binary variable neglecting its fluidity and continuity. 2) Most of the work has been conducted in monolingual setups for English or other high-resource languages. 3) Despite a myriad of papers on gender bias in NLP methods, we find that most of the newly developed algorithms do not test their models for bias and disregard possible ethical considerations of their work. 4) Finally, methodologies developed in this line of research are fundamentally flawed covering very limited definitions of gender bias and lacking evaluation baselines and pipelines. We suggest recommendations towards overcoming these limitations as a guide for future research.

1 INTRODUCTION

The survey addresses a rapidly growing but fragmented literature on gender bias in NLP by synthesizing prior work and identifying four major limitations. These concern binary treatment of gender, limited language coverage, insufficient bias testing, and flawed evaluation methodologies.

  • The survey reviews 304 papers because rapid growth has fragmented research across communities and publications often fail to engage with parallel work.
  • Most research treats gender as binary, neglecting gender’s fluidity and continuity and risking harms such as misgendering and erasure.
  • Most prior work is monolingual, focusing mainly on English and a small number of high-resource languages such as Chinese and Spanish.
  • Recently published models often omit bias and ethical evaluations, while bias testing commonly occurs only after deployment.
  • Gender-bias detection methodologies often lack baselines, fail to engage with parallel research, and use limited, incoherent definitions and evaluation metrics.

2 METHODOLOGY

The survey systematically gathered relevant gender-bias research from major NLP and related sources, retaining papers that analyse gender bias in language or provide broad methodological overviews. Publication counts were also examined to characterize the field’s growth.

  • The authors identified 304 papers by querying ACL Anthology, NeurIPS, and FAccT for gender- and bias-related keywords before June 2021.
  • The review retained papers about gender bias and excluded papers addressing other meanings of the queried terms, such as inductive or social bias.
  • The analysis found a steady increase in publications since 2015, with 83 papers in 2019 and 107 in 2020.
  • The survey also identified 40 gender-bias papers published in 2021 by the time of data collection.

3 GENDER IN SOCIETY AND LINGUISTICS

The paper distinguishes social, linguistic, and computationally relevant understandings of gender, emphasizing that gender is broader than a binary variable. It also notes that gender often must be inferred from incomplete and culturally variable data.

  • Gender may refer to self-determined identity, expression, perception, or social expectations, and differs from sex as physical and physiological characteristics.
  • Linguistic gender categories include grammatical, referential, lexical, and biosocial gender, but linguistic categories do not map neatly onto social categories.
  • Treating gender as binary oversimplifies its complexity because gender can be viewed as a broad spectrum.
  • Gender-neutral forms such as singular they, hen, and hän challenge datasets and algorithms, while relevant terms remain underrepresented in some corpora and databases.
  • Gender is often inferred from names, but name-based methods are imprecise because names vary across languages, cultures, and gender identities.

4 GENDER BIAS, SEXISM AND HARMS THEY MAKE

The paper presents gender bias and sexism as multifaceted phenomena that can arise in language, models, and social outcomes. It surveys competing definitions and links these biases to representational, allocational, and gender-gap harms.

  • Gender bias is defined as systematic unequal treatment based on gender, while sexism concerns discrimination, stereotyping, or prejudice based on sex.
  • Bias may be pre-existing, technical, emergent, reporting-based, or interpretation-based, reflecting different sources and mechanisms.
  • Sexism includes hostile and benevolent forms, with benevolent sexism remaining discriminatory despite its subjectively positive attitude.
  • Algorithmic harms include representational harms such as stereotyping and under-representation, and allocational harms involving unjust distributions of opportunities and resources.
  • NLP can reproduce or amplify cultural prejudices through data and algorithms, producing harms in downstream tasks and gender gaps in representation.
  • Gender bias in text may be structural, contextual, or both, including words or syntactic constructions that imply prejudice against a gender.

5 RESOURCES

This section surveys lexica and datasets used to measure gender bias in NLP, while highlighting their coverage and validity constraints. Resources support controlled diagnostics but remain limited in language coverage, gender representation, and ecological validity.

  • Lexica and datasets are selected according to the bias definition, NLP task, and domain being investigated.
  • Lexica: Lexicon matching is interpretable and technically simple, using genderness, sentiment, valence, arousal, and dominance measures as bias proxies.
  • Lexica: Gender ladenness quantifies perceived feminine or masculine association, with words assigned scores reflecting gender specificity and stereotypicality.The cited example assigns “bride” a value of 0.84 on a scale from -1 to 1.
  • Lexica: Lexica are limited to known words, assume stable context, require costly human annotation, and are overwhelmingly English-only.
  • Datasets: Template datasets provide controlled experiments, but may introduce artificial biases whose results do not transfer to natural environments.
  • Datasets: Most lexica and datasets are English-only, while multilingual resources often focus on high-resource languages and binary gender categories.

6 DEFINING BIAS

The paper organizes formal bias definitions into measures for detecting gender bias in language and in NLP methods.

  • Formal bias definitions are divided between detecting gender bias in natural or generated language and detecting bias in NLP methods.

6.1 Measuring Gender Bias in Language

Language-level gender bias measures quantify differences in how genders are portrayed, using co-occurrence, sentiment, and stereotype-based approaches. These measures capture selected textual patterns but can overgeneralize across heterogeneous data.

  • Language-level analysis focuses on complex lexical biases and differences in gender portrayals rather than only gender distributions in data.
  • Differences in Gender Descriptions: Point-wise mutual information measures how strongly descriptors co-occur with particular genders and is widely used to quantify differences in gender depictions.
  • Differences in Gender Descriptions: High PMI values for one gender are treated as indicators of gender bias, although word-level co-occurrences may overgeneralize across heterogeneous datasets.
  • Differences in Gender Descriptions: PMI can also compare gendered word choices beyond a binary gender distinction.
  • Stereotypical and Occupational Bias: Stereotype scores quantify how often gender-neutral words co-occur with male versus female words, with zero indicating equal frequencies.
  • Stereotypical and Occupational Bias: Occupation stereotype scores are computed by averaging stereotype scores across gender-neutral occupations in a text.

6.2 Measuring Gender Bias in Methods

The survey covers performance, association, causal, translation, and embedding-based measures for bias in NLP methods. It emphasizes that widely used approaches often capture only stereotypical bias and that some metrics have important validity limitations.

  • Performance-based measures evaluate gender handling in downstream tasks such as machine translation and coreference resolution when gold gender labels exist.
  • Performance-based Measures: Task-specific metrics include labeled attachment score for dependency parsing and BLEU for machine translation.
  • Stereotypical Associations: Association-based measures test whether systems prefer stereotypical gender-role links or associations over anti-stereotypical alternatives.
  • Language Models: Paired templates support tests of gender differences in predicted probabilities, while language-model metrics measure preference for stereotyping sentences.
  • Overall, commonly used methods are simple and interpretable but cover only stereotypical bias, while causal testing likewise leaves other harms underexplored.
  • Causal Testing: Causal testing defines bias through disparities in model outputs when gender inputs change, including evaluations over gender-neutral occupations.
  • Machine Translation: Translation gender-bias indices target male-default risks by comparing female, male, and gender-neutral pronoun outputs across systems.
  • Word Embeddings: Embedding measures include projection, relative-distance, association, and sentence-level tests, but projection-based measures may miss systematic bias and WEAT can overestimate it.

6.3 Summary

The survey finds that gender-bias definitions vary widely across algorithms and tasks, often remain implicit, and are measured inconsistently. Existing measures also predominantly represent gender as binary and can reduce bias detection to performance effects.

  • Gender-bias definitions vary dramatically across algorithms and tasks, and are often described without a formal definition.
  • Defining bias through decreased performance risks detecting bias only when it affects performance, making model improvement the operative goal.
  • Some performance measures are themselves limited evaluation benchmarks, including BLEU as a coarse and indirect machine-translation indicator.
  • Most gender-bias measures quantify only binary genders, even when a metric could analyze non-binary genders.

7 DETECTING GENDER BIAS

The survey reviews gender-bias detection across natural language, NLP methods, and downstream tasks. Across these settings, it reports stereotypical associations, male defaults, and uneven treatment, while emphasizing that research remains largely binary and English-focused.

  • The survey synthesizes research on gender bias in natural language, NLP algorithms, and downstream tasks using datasets and formal bias definitions.
  • Natural language: Studies of media and directed language report gendered differences in portrayals, topics, appearance references, relationships, agency, and perceived power.
  • Natural language: Word embeddings can reveal historical changes in gendered associations, including decreases in bias over the last 100 years.
  • Summary: The surveyed evidence remains limited because most studies focus on binary identities and English, despite applications with societal implications.
  • NLP methods: Cross-lingual embedding bias can be roughly predicted from corresponding monolingual embedding bias, linking bias patterns across representation spaces.
  • NLP methods: Research on language models reports inconsistent effects of model size and distillation on gender bias, motivating more exhaustive testing.
  • Gender-inclusive detection: Non-binary research includes aggregating gender n-tuples, gender-neutral coreference tagging, and testing generation probabilities for they.
  • Downstream tasks: Gender bias appears in downstream systems through male defaults in translation, stereotypical coreference resolution, gendered story associations, and statistically significant sentiment differences.

8 MITIGATING GENDER BIAS

The survey reviews mitigation methods based on data manipulation and algorithm adjustment, while highlighting trade-offs, scalability limits, and the need for broader testing. Data-based approaches include augmentation, gender tagging, added context, and balanced fine-tuning.

  • Mitigation cannot completely remove gender bias, and debiasing methods require additional tests on previously untested tasks because effectiveness may be unclear.The survey encourages debiasing earlier in the modelling process and notes a trade-off between reducing encoded bias and preserving downstream performance.
  • Gender-bias mitigation is motivated by concerns that dataset biases can propagate through downstream tasks and cause societal allocational harms.
  • Data Augmentation: Counterfactual data augmentation creates gender-swapped auxiliary examples and trains on them with the original data to balance gender representation.The method presents equivalent scenarios with different gendered entities, encouraging models to abstract from entities to context.
  • Data Augmentation: Fine-tuning solely on an augmented corpus successfully decreases gender bias in contextualised representations, including ELMo and monolingual BERT.The approach has also been applied to hate speech detection, but collecting gender-pair lists can be expensive and doubles training data.
  • Gender Tagging: Gender tagging supplies speaker or entity gender information through inference-time phrases, sentence-level annotations, token-level annotations, or post-processing.These approaches can address multiple gendered entities, but results for some schemes are ambiguous and they may require costly or unavailable metadata.
  • Balanced Fine-Tuning: Balanced fine-tuning transfers from a less biased dataset before adapting to a more biased target dataset, reducing potential over-fitting to the target data.A gender-balanced corpus from a different domain has been reported to mitigate bias and increase performance quality, although qualitative portrayal differences remain unaddressed.

8.2 Debiasing by Adjusting Algorithms

Algorithm-adjustment methods mitigate gender bias through projection, output constraints, adversarial learning, loss modification, and gender-neutral representation learning. Their effectiveness and applicability depend on word lists, data distributions, protected-attribute assumptions, and downstream evaluation.

  • Algorithm-adjustment methods include projection-based debiasing, constrained predictions, adversarial learning, and other modifications.
  • Projection-Based Debiasing: Projection-based embedding methods remove or equalise gender directions, while later approaches identify relevant subspaces for contextualised models and masked-language-model evaluation.Some methods apply projections only to hidden or non-contextualised layers and depend on attribute and target-word lists.
  • Projection-Based Debiasing: Hard- and soft-debiasing have been applied to embeddings and language models, but selected downstream tests found that hard-debiasing did not always produce the desired effect.
  • Projection-Based Debiasing: Projection and related debiasing methods rely on predefined gender-neutral or attribute-word lists, and errors in identifying neutral words can affect downstream performance.The literature also reports a trade-off between perplexity and gender bias, while some supposedly removed bias can remain recoverable.
  • Constraining Output: Output constraints can force model predictions to follow gender distributions estimated from a training corpus, making the method dependent on that corpus’s balance and bias.
  • Adversarial Learning: Adversarial learning trains a predictor to retain task information while fooling an adversary that predicts a protected attribute such as gender.The approach is described as unstable and applicable only when gender is protected rather than the variable of interest.
  • Other Methods: Other approaches neutralise protected attributes in selected embedding dimensions, equalise gender-word probabilities through the loss, preserve non-stereotypical gender information, or incorporate mitigation upstream.These methods may rely on predefined lists or narrow definitions of bias, and upstream mitigation is reported as less effective but more efficient than direct mitigation.

9 DISCUSSION

The survey identifies limitations in how NLP research defines gender, covers languages, tests models for bias, and measures bias. It recommends transparent and inclusive gender definitions, broader multilingual coverage, formal testing, and comprehensive evaluation.

  • Gender in NLP: The survey reports that gender labels are often assigned without explanation, raising ethical concerns and weakening the reproducibility and validity of findings.
  • Gender in NLP: Researchers commonly define gender as binary, an oversimplification that can perpetuate harms to non-binary people.The survey recommends transparent, inclusive definitions, inclusive pronouns, and evaluation on non-binary pronouns, whose downstream performance has been consistently lower.
  • Monolingual Focus: Most gender-bias research is monolingual and concentrated on English or a few high-resource languages, despite cross-linguistic and cultural variation in gender expression.
  • Need for Formal Testing: Widely used NLP models often report major task gains without studying bias alongside their release, motivating formal bias testing.
  • Limited Definitions: Gender-bias research uses incoherent evaluation metrics and commonly captures only one of the many ways bias can be expressed.The survey calls for comprehensive, multifaceted measures that engage with parallel research.

10 CONCLUSION

The survey identifies four major limitations in gender-bias research, including binary treatment of gender, limited language coverage, and insufficient bias testing and ethical consideration. It presents overcoming these limitations as crucial for further field development.

  • The survey frames overcoming these limitations as necessary for further development of gender-bias research.
  • Most research treats gender as binary, conflicting with social-science views and neglecting gender’s fluidity and continuity.
  • Research concentrates on English and other high-resource languages, disregarding low-resource languages and restricting views of gender bias in NLP.
  • Most newly developed gender-bias algorithms do not test models for bias or address ethical considerations, creating potential societal harms in deployment.
Loading 2112.14168v1…