Source-linked AI summary
Gender Bias in Machine Translation
Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, Matteo Negri, Marco Turchi
TL;DR
Gender bias in MT is an important but fragmented research area involving technical, linguistic, and societal dimensions. The paper synthesizes related disciplines, reviews existing assessments and mitigation strategies, and proposes a unified perspective. It concludes that current interventions lack a conclusive state-of-the-art solution and that gender analysis should move beyond binary assumptions.
Problem
Gender bias in MT lacks internal cohesion, while existing analyses and mitigation strategies address different aspects across languages, datasets, and contexts.
Method
The paper critically reviews conceptualizations, assessment studies, and mitigation strategies, integrating insights from related disciplines into a unified framework.
Results
Current MT debiasing interventions remain modular and aspect-specific, with no conclusive state-of-the-art method for mitigating bias.
Takeaways & Limitations
Future research should consider explainability, diverse users, and inclusive or non-binary dimensions rather than relying only on binary gender assumptions.
Takeaways & Limitations
Research on MT gender asymmetries has largely focused on occupational stereotyping and binary masculine/feminine categories.
Abstract
from arXiv · showhide
Machine translation (MT) technology has facilitated our daily tasks by providing accessible shortcuts for gathering, elaborating and communicating information. However, it can suffer from biases that harm users and society at large. As a relatively new field of inquiry, gender bias in MT still lacks internal cohesion, which advocates for a unified framework to ease future research. To this end, we: i) critically review current conceptualizations of bias in light of theoretical insights from related disciplines, ii) summarize previous analyses aimed at assessing gender bias in MT, iii) discuss the mitigating strategies proposed so far, and iv) point toward potential directions for future work.
1 Introduction
Gender bias in MT has become an important research concern because language technologies can encode social asymmetries and affect users at scale. Existing work is fragmented across languages and subtasks, motivating a comprehensive framework for understanding and assessing the problem.
- NLP systems have exhibited gender disparities, including misgendering women, overlooking women in visual contexts, and performing better with male voices.
- Commercial MT users have reported occupational stereotypes, such as translating engineers as masculine and nurses as feminine.
- Existing NLP surveys focus on monolingual applications whose assumptions and solutions may not transfer directly to other languages or cross-lingual settings.
- The paper presents a comprehensive MT review with a unified framework for bias concepts, sources, and effects, while identifying research blind spots and challenges.
2 Bias statement
The paper frames bias as a normative, human-centered issue involving harms to representation and resource allocation. It distinguishes under-representation and stereotyping as forms of representational harm relevant to MT.
- Bias can involve divergence from expected values caused by spurious cues, but assessing it also requires judging harmful behavior, how it occurs, and whom it affects.
- The framework distinguishes representational harms, which affect social-group visibility and identity, from allocational harms, which withhold opportunities or resources.
- Representational harm is further divided into under-representation and stereotyping when applied to gender bias in MT.
- Under-representation reduces women’s visibility, fails to recognize non-binary individuals, or flattens speakers’ preferred gendered expressions.
- Stereotyping propagates negative generalizations, including associating feminine forms with less prestigious occupations or attractiveness judgments.
3 Understanding Bias
Gender bias in MT must be understood through related disciplines because gender is a multifaceted issue shaped by language, translation, and implicit social assumptions.
- The paper argues that interdisciplinary perspectives are vital for analyzing how socio-cultural notions of gender interact with languages, translation, and implicit biases.
3.1 Gender and Language
Gender is expressed and interpreted differently across languages, while linguistic choices also reflect social roles, identities, and cultural assumptions. These differences shape both human and machine translation.
- Gender-related linguistic structures vary across languages, gender assignment depends on context and social assumptions, and language helps articulate personal identities.
- Linguistic gender systems: Genderless, notional-gender, and grammatical-gender languages differ in whether gender appears lexically, pronominally, or through broad morphosyntactic agreement.
- Linguistic gender systems: The same sentence can show no overt gender in Turkish but several masculine or feminine markings in Spanish, illustrating typological differences relevant to translation.
- Social interpretation: Gender bias can persist in apparently neutral forms because social gender influences how epicene occupational nouns are pronominalized.
- Translation and variation: Translation choices reflect both cross-language gender-system differences and socio-cultural factors influencing how gender is assigned.
- Translation and variation: Gender-related linguistic features have supported demographically informed NLP and personalized MT, but reducing gender to male and female imposes a limitation.
3.2 Gender Bias in MT
The paper frames gender bias in MT as arising from pre-existing, technical, and emergent sources that form a continuum. It distinguishes representational harms from technical and contextual constraints, emphasizing that data and model-side factors require scrutiny.
- Sources of bias: The paper identifies pre-existing, technical, and emergent bias as overlapping sources of gender bias in MT.These categories cover institutional and social conditions, technical decisions, and system–user interactions.
- Pre-existing bias: Historical and sociocultural conditions can enter MT data, as illustrated by Europarl sentences spoken by women comprising only 30%.The imbalance reflects unequal access to parliamentary positions and constitutes historical bias in the corpus.
- Technical bias: Technical bias includes data creation, model design, training, and testing choices that can reproduce quantitative and qualitative gender asymmetries.Balancing speaker counts alone cannot address qualitative bias in representations.
- Emergent bias: Emergent bias can arise when MT systems are deployed in contexts or demographic groups different from those considered during design.The paper links this source to risks from male-grounded products in other domains.
4 Assessing Bias
Research assessing gender bias in MT examines both stereotypical associations and preservation of gender information, using qualitative analyses, synthetic tests, natural sentences, and dedicated benchmarks. The survey emphasizes that metrics and test sets are proxies whose coverage and interpretation constrain conclusions about real-world bias.
- 4.1 MT and Gender Stereotypes: Early qualitative studies found severe English–German difficulties in recovering syntactic and semantic information needed for correct gender agreement.Subsequent commercial-system studies examined comparable phenomena in other target languages with grammatical gender.
- 4.1 MT and Gender Stereotypes: Prates et al. tested pronoun translation from 12 genderless languages into English using approximately 1,000 occupation-based constructions.Cho et al. extended a similar design to Korean–English with occupations and sentiment words.
- 4.1 MT and Gender Stereotypes: MT produces a masculine default and underestimates feminine occupation references more than occupation data alone would predict.Prates et al. compared predicted pronoun proportions with real-world employment proportions across 22 occupation categories.
- 4.1 MT and Gender Stereotypes: Natural ambiguous English sentences translated into Russian, Spanish, German, and French consistently exhibited occupational social asymmetries.Other studies found systems could ignore explicit feminine information or use socially connoted adjectives as gender cues.
- 4.2 MT and Gender Preservation: Gender-informed English–French modeling produced its largest improvement for women’s sentences, whose baseline service quality was lower than for men’s sentences.The authors attributed gains to morphological agreement and gendered lexical preferences, using manual analysis.
- 4.3 Existing Benchmarks: Standard BLEU and TER evaluations are coarse-grained and can reward biased predictions when generic test sets reproduce training-data gender imbalances.Dedicated benchmarks such as MuST-SHE balance masculine and feminine phenomena across English–French, Italian, and Spanish data.
- 4.3 Existing Benchmarks: Challenge sets quantify stereotyping and under-representation but contain limited synthetic phenomena, making them relatively easy to overfit and insufficient to prove bias absence.Natural corpora capture authentic representation and speaker-level service differences but cannot identify stereotypical propagation when all gender-marked words are treated equally.
- 4.3 Existing Benchmarks: Benchmarks remain useful for monitoring model behavior, but high scores on formalized proxies may create false security without real-world observation.The paper presents each test set and metric as an approximation of the phenomenon or ability being measured.
5 Mitigating Bias
The reviewed mitigation strategies target different gender-bias harms through data, model, inference, and output interventions, but no conclusive general solution exists. Their effectiveness and feasibility depend on the language, data, bias formulation, and available metadata.
- Mitigation approaches are mapped to the benchmarks they use, their binary or non-binary gender scope, and the representational or allocational harms they target.Representational harms include under-representation and stereotyping; allocational harm is reduced quality of service.
- Model Debiasing: Gender tagging improves morphological agreement in English-to-French translation but requires speaker-gender metadata that may be infeasible to obtain.Cross-lingual gender-agreement rules offer an alternative by identifying speaker and listener references from part-of-speech information.
- Model Debiasing: Word-level gender tags can introduce noise when sentences mention multiple participants by pushing their translations toward the same gender.A small synthetic non-binary dataset produced inconclusive neutral-translation results.
- Model Debiasing: Additional context yields a slight improvement for anaphorical gender translation in English-Spanish without requiring extra training or inference information.The authors motivate document-level exploration but call for manual and interpretability validation.
- Model Debiasing: Debiasing embeddings or fine-tuning balanced data can improve some gender-form outputs, but these interventions do not reliably address stereotyping and may require language-specific treatment.Balanced fine-tuning improves feminine-form generation overall yet is less effective on the anti-stereotypical WinoMT set; embedding debiasing also raises concerns for grammatical-gender languages.
- Debiasing through External Components: External components such as black-box injection, lattice re-scoring, and gender re-inflection avoid retraining but add integration and maintenance costs.Lattice re-scoring increases gender-form selection accuracy on WinoMT, while its gender-debiased reranking model comes at the cost of lower generic translation quality.
- Debiasing through External Components: The survey finds no conclusive state-of-the-art mitigation method, and integrating modular interventions within one MT system remains unexplored.Because gender bias covers heterogeneous phenomena across languages and corpora, the authors argue for context-aware interventions and multidisciplinary commitment beyond engineering alone.
6 Conclusion and Key Challenges
The paper organizes emerging research on gender bias in MT within a unified framework and identifies challenges for developing the field. It highlights model de-biasing, non-textual modalities, gender beyond binaries, intersectionality, and human-centered evaluation as future priorities.
- 6 Conclusion and Key Challenges: The review integrates current studies on gender bias in MT within a unified framework to critically assess the field and guide future research.The framework incorporates interdisciplinary knowledge and concludes with directions for advancing research.
- Model de-biasing: Explainability techniques may reveal spurious cues that models exploit, informing changes to data, architectures, loss functions, and optimization.The paper notes that stereotypical correlations could be identified and down-weighted during training.
- Non-textual modalities: Gender bias in non-textual translation remains largely neglected, while using vocal or visual gender cues can impose reductionist classifications and harm diverse users.Direct speech translation systems have used vocal characteristics to improve feminine translation, but physical cues such as pitch can be problematic.
- Beyond Dichotomies: Future work should move beyond the masculine/feminine binary by exploring inclusive solutions and nuanced dimensions of gender.The paper notes that progress is constrained by the absence of suitable data and points to inclusive language practices.
- Intersectionality: Future studies should account for interactions between gender and other sociodemographic categories because discrimination can arise intersectionally and may not be detectable in isolation.The paper specifically identifies race and gender as interacting identity categories.
- Human-in-the-loop: Research remains restricted to lab tests, leaving people’s experiences and deployment contexts insufficiently represented in evaluating progress.The paper presents participatory design and context-specific evaluation as important considerations for guiding the field forward.