PaperlayerView exact revision on arXiv

Source-linked AI summary

Mitigating Gender Bias in English to Romanian Machine Translation

Ioana Grigore, Sergiu Nisioi

arXiv:2608.08606v1cs.CLcs.AI

TL;DR

English-to-Romanian machine translation often mishandles implicit gender, so this paper combines LLM-based gender disambiguation and tag-aware Transformer translation. The pipeline improves gender correctness by over 40 percentage points relative to raw MT and reduces reliance on stereotypical defaults.

  • Problem

    English-to-Romanian MT must infer implicit English gender and realize explicit Romanian agreement, but gender bias and Romanian’s complex morphology remain insufficiently addressed.

  • Method

    The pipeline uses a fine-tuned LLM to insert inline gender tags and a Transformer trained on gender-tagged English-Romanian data to realize the intended morphology.

  • Results

    Over 40 percentage points of gender-correctness improvement over raw MT were achieved across external diagnostic benchmarks, with a substantially reduced pro/anti-stereotype performance gap.

  • Takeaways & Limitations

    The approach provides explicit controllability and compatibility with existing MT systems while mitigating gender bias in English-Romanian translation.

  • Takeaways & Limitations

    LoRA adapter tuning underperformed in this setting, suggesting that tag-conditioned Romanian morphological control may require greater adaptation capacity.

Abstract

from arXiv · show

Machine translation (MT) systems often fail to correctly translate gender, especially when converting from a gender-neutral language like English to a gendered target language such as Romanian. This bias results in translations that default to masculine forms or reinforce gender stereotypes. We propose a hybrid pipeline to mitigate this issue by combining large language model (LLM)-based gender classification with neural machine translation (NMT). Our system uses a fine-tuned LLM to detect the intended gender of target words in English sentences and insert inline gender hint tags. These tagged sentences are then passed to a Transformer model fine-tuned to generate morphologically correct Romanian translations. To support this, we introduce three novel datasets for gender disambiguation and translation. Our approach improves gender accuracy on the WinoMT and WinoGender benchmarks by over 40 percentage points compared to a baseline MT system. This is the first method to explicitly address and evaluate gender bias in English-Romanian MT using both LLM inference and tag-aware translation.

1 Introduction

The paper addresses gender bias in English-to-Romanian machine translation, where implicit English gender must be rendered with explicit Romanian grammatical agreement. It proposes a hybrid pipeline combining LLM-based gender inference, inline hints, and tag-aware NMT, supported by new datasets and evaluated on WinoMT and WinoGender.

  • Problem: English-to-Romanian MT often defaults to masculine forms or fails to resolve gender from context, producing biased or grammatically incorrect translations.The challenge arises because English frequently leaves gender implicit, whereas Romanian requires explicit grammatical gender agreement.
  • Method: A fine-tuned LLaMA model classifies target-word gender in English sentences and inserts inline gender hint tags such as <tgF>teacher</tgF>.The tagged sentences are passed to a fine-tuned neural machine translation model.
  • Method: A Transformer trained to recognize and act on the tags produces gender-aware Romanian translations.This tag-aware NMT component operationalizes the LLM-derived gender information during translation.
  • Contributions: The work releases 1 novel, high-quality corpora for gender-aware English classification and gender-controlled EN→RO translation.The corpora support both gender disambiguation and controlled translation in the proposed system.
  • Evaluation: The approach is evaluated on WinoMT and WinoGender and achieves significant gains in gender translation accuracy over baseline MT.The results are presented as evidence that combining LLM-based context understanding with targeted NMT adaptation can mitigate gender bias in low-resource language pairs.

2 Related Work

Prior research documents persistent gender bias in machine translation, especially when translating from gender-neutral English into morphologically gendered languages. Although diagnostic benchmarks, source-side tags, and LLM-based control have been explored, English–Romanian MT remains unaddressed despite Romanian’s complex gender agreement.

  • Documented MT bias: Gender bias is documented when gender-neutral source languages such as English map to targets requiring grammatical gender agreement.Early diagnostic datasets, including WinoGender and WinoMT, were introduced to evaluate this bias systematically.
  • Persistent model limitations: State-of-the-art MT systems and LLMs often default to masculine forms when gender-neutral or gender-ambiguous translations are appropriate.Multilingual benchmarks indicate difficulty using contextual cues, extended discourse, and explicit instructions to produce inclusive or neutral forms reliably.
  • Existing mitigation approaches: Prior mitigation methods include source-side gender tags and instruction-tuned LLMs that generate gender-controlled outputs through prompt engineering.LLMs can produce separate masculine and feminine translations by conditioning on contextual cues or examples.
  • English–Romanian research gap: English–Romanian MT remains unaddressed, while Romanian poses challenges through its three-gender system and complex morphological agreement.This gap distinguishes Romanian from the language pairs most commonly examined in prior gender-bias research.

3 EnGen: The English Gender Disambiguation Dataset

EnGen introduces two semi-automatically constructed, curriculum-ordered datasets for fine-tuning LLMs to infer the gender of target words from English context. Dataset 1 teaches single-entity gender classification, while Dataset 2 adds distractors and more complex contexts.

  • Dataset construction: The datasets are created semi-automatically by Romanian native speakers, GPT-generated examples, and subsequent native-speaker checking and filtering.Their two-stage curriculum increases example difficulty progressively.
  • Dataset 1: 11,472 examples comprise Dataset 1, each containing one genderable word from categories including jobs, family members, animals, roles, and proper names.Contexts contain up to three sentences and include feminine, masculine, and ambiguous cases.
  • Dataset 1: Dataset 1 balances feminine, masculine, and ambiguous examples across semantic categories and context lengths of one, two, and three sentences.Ambiguous cases include gender-neutral or non-binary references when context does not determine gender.
  • Dataset 2: 996 examples comprise Dataset 2, with unique target words paired with an ambiguous-gender distractor across one- or two-sentence contexts.Distractors vary in sentence position to increase task complexity and reduce predictability.
  • Curriculum and splits: The curriculum progresses from Dataset 1’s single-entity phrases to Dataset 2’s two-genderable-word contexts containing a target and a distractor.Both datasets use training, validation, and test splits designed to preserve class balance and avoid leakage.

4 EnRoGend: a Parallel English-Romanian Gender-Tagged Dataset

EnRoGend is a 1,974-example parallel English–Romanian dataset designed to teach machine translation gender distinctions through paired gender-tagged sentences. It covers occupations and person-related nouns, pairing tagged English sources with untagged Romanian translations.

  • Dataset structure: 1,974 examples form pairs containing two versions of the same sentence, with the target word marked for masculine and feminine gender.Each pair represents masculine and feminine variants of the same sentence.
  • Gender tagging: English sources surround the target word with either <tgM>target word</tgM> or <tgF>target word</tgF> tags.The tags explicitly encode the intended gender of the target word.
  • Parallel translations: Each tagged English source is paired with a corresponding Romanian translation containing no gender tags.The dataset separates gender hints in the English input from the Romanian translation output.
  • Coverage: The dataset contains only occupations and person-related nouns.Its stated purpose is to teach the MT system gender distinctions for these noun categories.

5 Methodology

The methodology combines entity selection, LLM-based gender classification, inline gender tags, and tag-aware English-to-Romanian Transformer translation. Curriculum training and fine-tuning experiments evaluate classification quality, translation performance, and the trade-off between adaptation efficiency and capacity.

  • Entity selection: The Entity Selector matches input words against a predefined list of gendered role nouns and occupations, using fuzzy matching for misspellings and plural forms.The selector identifies candidate targets whose gender must be inferred.
  • Gender classification: The classifier uses a 1B-parameter LLaMA 3.2 model with 4-bit NF4 quantization, mixed precision, and LoRA fine-tuning for fast deployment on limited hardware.Only LoRA parameters and the classification head are updated during training.
  • Pipeline: The end-to-end pipeline extracts candidate entities, classifies their gender with an LLM, inserts gender tags, and translates the tagged sentence into Romanian.The pipeline is designed to preserve intended gender information through analysis and generation stages.
  • Gender classification: The classification head maps the final-token hidden state to feminine, masculine, and ambiguous logits, while curriculum training progresses from single-entity Dataset 1 to multi-entity Dataset 2.The first stage teaches the core task with clear and ambiguous contexts; the second adds more complex contextual reasoning.
  • Classification results: The top Dataset 1 model reaches an F1 of 0.97 and accuracy of 97% on the test set, while the best Dataset 2 model reaches an F1 of 0.95.Dataset 2 performance ranges as low as 0.66, reflecting the difficulty of its multi-entity structure.
  • Translation model: The tag-aware Transformer uses four source-side gender tokens, and full fine-tuning achieves a COMET score of 0.846, a BLEU score of 96.92, a chrF++ score of 98.21, and a TER of 1.94.LoRA underperforms because the task requires learning both source-side tag use and Romanian morphological agreement.

6 Benchmark Evaluation

The evaluation uses external WinoMT and WinoGender benchmarks to test gender resolution and grammatical agreement in English-to-Romanian translation. It compares raw Transformer MT, the gender-aware pipeline, and zero-shot GPT-5.2, judging outputs primarily by intended gender and sentence-wide agreement.

  • Evaluation setup: The evaluation uses external benchmark datasets not included in training, avoiding data leakage despite GPT-4o’s involvement in data creation.All benchmark sentences were compared with external data, and GPT-4o was not evaluated on training examples.
  • Benchmarks: WinoMT contains 1,584 pro-stereotypical and 1,584 anti-stereotypical English sentences requiring gender inference from context.Each sentence has pro- and anti-stereotypical variants, such as pronouns aligning with or contradicting occupational stereotypes.
  • Benchmarks: WinoGender contains 720 minimal-pair sentences differing only in the pronoun he, she, or they, testing whether Romanian outputs preserve those gender distinctions.The benchmark evaluates pronoun resolution and gender agreement in the gendered target-language translation.
  • Evaluation setup: The comparison includes Raw MT, the Gender-Aware Pipeline, and zero-shot GPT-5.2 translation under default settings.Because the test sets lack gold-standard Romanian references, evaluation includes manual assessment.
  • Evaluation criteria: Translations count as correct when the intended gender is correct and agrees with the rest of the sentence, even if wording or diacritics contain errors.For example, a feminine translation using “vânzătora” instead of “vânzătoarea” is still considered correct.

7 Conclusion

The paper identifies persistent stereotypical and masculine-default gender bias in English→Romanian MT, including in GPT-5.2, and proposes a hybrid LLM–Transformer pipeline to mitigate it. The approach improves gender correctness by over 40 percentage points and contributes datasets and training insights for gender-tagged translation.

  • Bias findings: English→Romanian MT frequently defaults to masculine forms and performs substantially better on pro-stereotypical WinoMT examples.The conclusion also reports stereotype sensitivity in the standard Transformer baseline.
  • Bias findings: GPT-5.2 also shows a bias toward stereotypical translations into Romanian, despite possible exposure to WinoMT and WinoGender.This finding extends the observed bias beyond the standard Transformer baseline.
  • Proposed pipeline: Over 40 percentage points of accuracy improvement in gender correctness are achieved across external diagnostic benchmarks.The gain comes from a hybrid pipeline combining LLM-based contextual gender disambiguation with tag-aware neural machine translation.
  • Resources and training: The work introduces EnGen and EnRoGend datasets with curriculum learning, controlled gender-tagged translation, and leakage-safe splitting strategies.EnGen targets English gender disambiguation, while EnRoGend targets controlled English-to-Romanian translation with gender tags.
  • Resources and training: Full and partial MT-model fine-tuning effectively leverage gender tags, whereas the tested LoRA configurations underperform for this task.The conclusion presents this as evidence about effective tag-conditioned morphological translation training.
  • Scope: Although experiments focus on Romanian, the proposed pipeline is applicable to other morphologically gendered target languages.The conclusion explicitly frames the method as transferable beyond Romanian.

Note

This document is an unreviewed preprint; its Version of Record was published in Springer’s Lecture Notes in Computer Science.

  • Publication status: The preprint has not undergone peer review or post-submission improvements or corrections, while its Version of Record is available online in Springer’s LNCS.The published version is available at https://doi.org/10.1007/978-3-032-29532-3_11.
Loading 2608.08606v1…