Does Machine Translation Retain Media Bias?

1.1 Background and Motivation

• Media bias is a deviation from neutral reporting, expressed through devices such as framing or word choice, and belongs to associative meaning.The paper argues that readers deserve information about a text’s neutrality and its author’s positioning, not only its propositional content.Source: §1.1 · p. 1
• Whether MT retains different forms of media bias, or loses them in translation, remains unclear.MT may be used to assess the position of a writer or news outlet in a language the reader does not know.Source: §1.1 · p. 1

1.2 Scope

• The study examines whether MT retains media bias across English-to-Bulgarian and German-to-Bulgarian translation.It presents an exploratory study of bias retention and introduction, alongside the BiasTrans-BG-1.0-small diagnostic corpus.Source: §1.2 · p. 1

2.1 Work on Automatic Machine Translation of Associative Meaning

• The paper frames news-bias retention as part of the broader question of whether MT preserves nuances of associative meaning.Source: §2.1 · p. 1
• The authors were unaware of prior work studying news-bias retention during machine translation.Earlier MT studies addressed sarcasm, irony, and stance toward rumors.Source: §2.1 · p. 2

2.2 Work on Automatic News Bias Analysis.

• Prior news-bias research includes sentence-level detection, annotation, framing-resistant summarization, and models for detecting 27 bias types.The passage also cites reviews of early detection work and a model by Menzner and Leidner (2024).Source: §2.2 · p. 2

2.3 Work on Media Bias for the Bulgarian Language

• Azizov et al. (2024) described cross-lingual bias detection and included Bulgarian among the languages considered.Source: §2.3 · p. 2

3 Dataset

• The source dataset combines user-submitted examples with samples from BABE, Reuters/Fox News, and TRC2/Reuters, totaling 2,196 unique sentences.The five most frequent user-submitted sources were Tagesschau, Fox News, Bild, Die Zeit, and BBC.Source: §3 · p. 2
• The experimental sample contains 446 sentences: 233 English and 213 German, annotated with a multi-label schema.The paper uses isolated sentences rather than full articles to avoid copyright issues; its use of “sentence” can also include a cohesive union of short snippets.Source: §3 · p. 2
• Because the bias model is single-class while the dataset permits multiple labels, the model’s output was used as the gold label only when it matched a human-annotated label.This maps multi-label annotations to a single class while keeping the selected label human-approved.Source: §3 · p. 2

• English and German sources were translated into Bulgarian; German sources were also translated into English to support the human evaluator.The four systems were DeepL, Google Translate, GPT-5.5, and BgGPT.Source: §4 · pp. 2–3
• BiasScanner classifications were compared with human-annotated gold labels, and a Bulgarian native speaker checked translation correctness and bias labels.The classifier's reported Bulgarian AUC was 0.867, compared with 0.865 on English.Source: §4 · p. 3
• Reference-free xCOMET was used to evaluate translation quality, with overall µ = 0.9139 and M = 0.9632.Source: §4 · p. 3
• Estimated translation-quality scores were high: median scores were M ≥0.9577, and English-to-Bulgarian scored µ = 0.9306 versus German-to-Bulgarian µ = 0.8957.DeepL had the highest reported mean score (µ = 0.9291), followed by GPT-5.5 (0.9162), BgGPT (0.9086), and Google Translate (0.9019).Source: §4 · p. 3

5 Experimental Evaluation and Results

• The evaluation used 446 paired source sentences across five conditions and applied paired profile tests, Cochran’s Q, exact McNemar tests, and Holm correction.The sample included 243 human-validated biased and 203 validated unbiased sentences.Source: §5 · p. 3

5.1 Quantitative Evaluation

• 64%–78% of news bias was retained after translation into Bulgarian across the four MT systems.Retention was slightly higher for German than English sources, while engine variability was greater for German.Source: §5.1 · p. 4
• The global bias-type profile differed significantly between the original and each translation, but no significant differences were detected among the four translations.The global test was significant (TsumQ = 168.20, pMC < .001); all six translation-engine pairwise comparisons had pHolm = 1.00.Source: §5.1 · p. 4

5.2 Qualitative Evaluation

• Retention varied more by bias category than by engine; structural biases often retained fully, while tone-dependent categories such as Circular Reasoning (50%) and Discriminatory Bias (55%) retained less.Word Choice and Emotional Sensationalism retained around 75%; most 0% and 100% extremes came from categories with n ≤2 and are tentative. Figure 3 reports bias percentages by category, source language, and engine, with sample counts.Source: §5.2 · p. 4, Figure 3 · p. 4

6 Summary, Conclusions and Future Work

• The study compares human-assigned and machine-assigned bias labels after English→Bulgarian and German→Bulgarian translation.Source: §6 · p. 4
• Table 2 presents example English and German sentences, their Bulgarian translations, and bias labels before and after translation.Source: Table 2 · p. 5
• Most translation engines retain 64%–78% of news bias in sentences, while translation can also introduce non-neutral reporting.The authors say the findings are consistent with claims that MT does not fully retain input bias and may add bias of its own.Source: §6 · pp. 4–5
• Future work should examine more language pairs, additional translation systems, and non-propositional semantic nuances beyond news bias.Source: §6 · p. 5

7 Limitations

• The evaluation covers two language pairs and four translation engines, and uses only one news-bias model.The authors note that many more language pairs and engines exist, and that other bias models could be tried.Source: §7 · p. 5
• The source corpus needs to be larger and include sufficient biased and unbiased sentences across language pairs and both translation directions.The authors say an adequate, defensible corpus size and sentence-sampling approach remain unsettled.Source: §7 · p. 5
• Source-language bias annotations may be wrong or themselves biased.Source: §7 · p. 5
• BiasScanner’s errors have been evaluated, but no specific score has been published for Bulgarian input.Source: §7 · p. 5
• The Bulgarian human evaluation relied on one available speaker, who was also the first author.The authors suggest a larger study use several subjects and report inter-annotator agreement.Source: §7 · p. 6
• MT quality issues can affect measured bias loss or introduction, although xCOMET checks indicated high quality across the engines used.Source: §7 · p. 6
• Confounding factors and the study’s limited scope prevent strong general claims that MT does not fully retain bias or causes bias.The authors say the findings are consistent with both claims.Source: §7 · p. 6
• Observed bias-histogram shapes depend on the bias model and the translation engine; the authors expect many observations may be confirmed in larger studies.Source: §7 · p. 6

8 Ethical Considerations

• The authors frame studying MT’s effect on news-bias retention as an ethically legitimate research goal.Source: §8 · p. 6
• The method or data could also be used to study concealing or adding news bias through machine translation.Source: §8 · p. 6
• Users must register organizationally before downloading the data, a measure intended to mitigate dual-use threats.Source: §8 · p. 6
• Sentence contributors acted under informed consent; the authors report applying for an IRB waiver letter because their local process is defined for named individuals.Source: §8 · p. 6