Source-linked AI summary
Getting Gender Right in Neural Machine Translation
Eva Vanmassenhove, Christian Hardmeier, Andy Way
TL;DR
The paper addresses how machine translation loses speaker-gender information needed for grammatical agreement and other language choices. It compiles speaker-annotated corpora for 20 language pairs and adds gender tags to NMT training. Gender-aware systems significantly improve translation quality for some language pairs, especially those expressing grammatical gender, although improvements are not always consistent.
Problem
Machine translation can lose speaker-gender information, causing incorrect morphological agreement and making author traits difficult to preserve.
Method
The authors compile speaker-annotated parallel corpora for 20 language pairs and incorporate speaker-gender tags into NMT systems.
Results
Gender tagging significantly improves translation quality for some language pairs, with improvements reported for French, Italian, Portuguese, Greek, and Danish.
Takeaways & Limitations
Speaker gender can benefit NMT beyond morphological agreement because it also relates to word choices and syntactic patterns.
Takeaways & Limitations
The authors report that improvements are not always consistent and call for further manual evaluation and other integration methods.
Abstract
from arXiv · showhide
Speakers of different languages must attend to and encode strikingly different aspects of the world in order to use their language correctly (Sapir, 1921; Slobin, 1996). One such difference is related to the way gender is expressed in a language. Saying "I am happy" in English, does not encode any additional knowledge of the speaker that uttered the sentence. However, many other languages do have grammatical gender systems and so such knowledge would be encoded. In order to correctly translate such a sentence into, say, French, the inherent gender information needs to be retained/recovered. The same sentence would become either "Je suis heureux", for a male speaker or "Je suis heureuse" for a female one. Apart from morphological agreement, demographic factors (gender, age, etc.) also influence our use of language in terms of word choices or even on the level of syntactic constructions (Tannen, 1991; Pennebaker et al., 2003). We integrate gender information into NMT systems. Our contribution is two-fold: (1) the compilation of large datasets with speaker information for 20 language pairs, and (2) a simple set of experiments that incorporate gender information into NMT for multiple language pairs. Our experiments show that adding a gender feature to an NMT system significantly improves the translation quality for some language pairs.
1 Introduction
Gender is expressed through morphology, word choice, and syntax, creating information-loss problems for machine translation. The paper proposes integrating speaker gender into NMT to preserve these traits across language pairs.
- Gender author classification from text remains unresolved, and selecting informative classification features is difficult.
- Gender information is often lost in human and machine translation, producing morphologically incorrect agreement that harms fluency and adequacy.
- Most MT systems translate sentences in isolation and choose statistically likely gender forms when contextual information is unavailable.
- Author gender influences language through morphological agreement, context-based word choices, and syntactic constructions.
- The paper integrates a gender feature into NMT and conducts experiments across multiple language pairs.
2 Related Work
Prior work documented gender-related traits and explored personalized translation, but this paper focuses on speaker-informed NMT. Earlier personalized SMT experiments did not improve over baseline systems.
- Linguistic studies and author-profiling research report gender differences in language style and syntax.
- Mirkin et al. motivated personalized MT, while Rabinovich et al. treated female and male SMT as separate domains.
- Rabinovich et al.’s gender-specific phrase tables, language models, and tuning set did not improve over the baseline.
- This work presents a speaker-informed NMT system using gender information similarly to tagged approaches for controlling politeness.
3 Compilation of Datasets
The paper addresses limited speaker-annotated parallel data by releasing corpora for 20 language pairs and annotating Europarl sentences with demographic information. The compiled data show substantial gender imbalance.
- Annotated parallel datasets were scarce beyond EN–FR and EN–DE, limiting personalized MT for other language pairs.
- The authors published Creative Commons parallel corpora with speaker information for 20 language pairs.
- Europarl sentences were tagged with speaker name, gender, age, birth date, euroID, and session date using source-file and MEP metadata.
- 32%–33% of sentences were uttered by female speakers across the 10 experimental datasets.
- 67.39% of EN–FR sentences were produced by male speakers, and the 50–60 age group contributed 43.76%.
- The authors conclude that Europarl’s gender imbalance is reflected in translations produced by systems trained on the corpus.
4 Experimental Setup
The experiments compare baseline and gender-tagged NMT systems across 10 English-centered language pairs. Evaluation uses a general test set for all systems and gender-specific tests for EN–FR.
- The study selected 10 language pairs with more than 500k annotated Europarl parallel sentences.
- Each source sentence received an English-side tag identifying whether the speaker was female or male.
- For every language pair, the authors trained a baseline system and a gender-tagged system.
- All systems were evaluated on a randomly selected 2K general test set, while EN–FR also used male-only, female-only, and first-person-singular test sets.
- The NMT models used OpenNMT-py sequence-to-sequence LSTM encoder-decoders with BPE segmentation using 89,500 operations.
5 Results
Gender-informed NMT improves translation quality for several language pairs, especially on female and first-person test sets, while also influencing lexical choices beyond morphological agreement. The gains are not universal, and some gender-related agreement errors remain.
- General test set: Gender tags significantly improve BLEU for French, Italian, Portuguese, Greek, and Danish, but Spanish declines by –0.19 BLEU and Dutch, German, Finnish, and Swedish do not improve.The Danish EN–DA system gains +0.31 BLEU, while the other listed improvements are reported without values here.
- Gender-specific test sets: The largest EN–FR BLEU improvement occurs on female test data, particularly sentences containing first-person singular pronouns.The evaluation compares male, female, first-person male, and first-person female test sets.
- Agreement examples: Gender-informed translations can correct speaker-gender agreement, producing feminine forms such as “vice-présidente” and “heureuse” where the baseline selects masculine forms.These examples show improvements on French translations of female-speaker utterances.
- Remaining errors: The tagged system can still produce incorrect agreement, as both baseline and tagged systems output masculine “embarrassé” instead of the correct feminine form.The failure occurs in a French example where the reference uses “gênée” or “embarrassée.”
- Lexical choices: Gender-informed systems also select different lexical variants, and the tagged system can achieve higher BLEU when its word choices match preferred speaker usage.The paper discusses French variants such as “pense” and “crois” and gives examples where both translations are correct.
6 Conclusions and Future Work
The paper finds that speaker-gender tags can improve NMT, particularly for languages with grammatical gender agreement, while effects are inconsistent in some translations. Future work will manually evaluate translations and explore other ways to integrate speaker information.
- Speaker-gender tags can significantly improve NMT over state-of-the-art baselines, especially for languages with grammatical gender agreement.
- EN–FR improvements are not always consistent, and gender-aware translations also differ from the baseline in word choices.
- Future work includes manual evaluation of translations, alternative methods for integrating speaker information, and gender classification for more informal corpora.