Source-linked AI summary
TranslateGemma Technical Report
Mara Finkelstein, Isaac Caswell, Tobias Domhan, Jan-Thorsten Peter, Juraj Juraska, Parker Riley, Daniel Deutsch, Geza Kovacs, Cole Dilanni, Colin Cherry, Eleftheria Briakou, Elizabeth Nielsen, Jiaming Luo, Kat Black, Ryan Mullins, Sweta Agrawal, Wenda Xu, Erin Kats, Stephane Jaskiewicz, Markus Freitag, David Vilar
TL;DR
Open machine translation needs strong, transparent models that support reproducibility and community innovation. TranslateGemma fine-tunes Gemma 3 with supervised learning and reinforcement learning, achieving consistent gains across 55 language pairs while smaller models often match larger baselines.
Problem
Strong open machine translation models are needed to support transparency, reproducibility, and community-driven innovation.
Method
TranslateGemma combines supervised fine-tuning on diverse human and synthetic parallel data with reinforcement learning using an ensemble of reward models.
Results
Automatic evaluations across 55 language pairs show consistent gains over baseline Gemma 3 models, while smaller TranslateGemma models often match or exceed larger baselines.
Takeaways & Limitations
The released TranslateGemma models provide open-source tools for machine translation research and application development.
Takeaways & Limitations
The authors hypothesize that the 27B model benefited more from exposure to many languages during supervised fine-tuning, but lack direct experimental confirmation.
Abstract
from arXiv · showhide
We present TranslateGemma, a suite of open machine translation models based on the Gemma 3 foundation models. To enhance the inherent multilingual capabilities of Gemma 3 for the translation task, we employ a two-stage fine-tuning process. First, supervised fine-tuning is performed using a rich mixture of high-quality large-scale synthetic parallel data generated via state-of-the-art models and human-translated parallel data. This is followed by a reinforcement learning phase, where we optimize translation quality using an ensemble of reward models, including MetricX-QE and AutoMQM, targeting translation quality. We demonstrate the effectiveness of TranslateGemma with human evaluation on the WMT25 test set across 10 language pairs and with automatic evaluation on the WMT24++ benchmark across 55 language pairs. Automatic metrics show consistent and substantial gains over the baseline Gemma 3 models across all sizes. Notably, smaller TranslateGemma models often achieve performance comparable to larger baseline models, offering improved efficiency. We also show that TranslateGemma models retain strong multimodal capabilities, with enhanced performance on the Vistra image translation benchmark. The release of the open TranslateGemma models aims to provide the research community with powerful and adaptable tools for machine translation.
1. Introduction
TranslateGemma is an open Gemma 3 variant refined for machine translation through supervised fine-tuning and reinforcement learning. It shows substantial gains across 55 language pairs while retaining multimodal capabilities and improving image translation performance.
- TranslateGemma is an open variant of the Gemma 3 foundation model specifically enhanced for machine translation.
- Its two-stage refinement combines supervised fine-tuning on diverse parallel data with reinforcement learning from human and model-based feedback.The parallel data blend includes human-translated and synthetically generated texts.
- 55 language pairs show substantial gains in evaluations on the WMT25 and WMT24++ datasets.
- TranslateGemma retains Gemma 3’s multimodal capabilities, while improved text translation also benefits image translation on the Vistra corpus.
2. Training data
Training combines high-quality synthetic translations with human-generated parallel data, using filtered multi-sample generation to support both individual segments and longer texts. The SFT mixture also includes generic instruction-following data to preserve broader capabilities.
- Synthetic data generation: Synthetic data is generated from MADLAD-400 monolingual data, targeting up to 10K examples per language pair after length bucketing and source selection.The process samples 1 million source segments per language pair and uses two Gemin… samples to identify sources likely to benefit from 128-sample QE decoding.
- Synthetic data generation: For selected sources, 128 Gemini 2.5 Flash samples are filtered with MetricX 24-QE to retain the best-performing translations.Translations cover individual sentences and text blobs of up to 512 tokens.
- Human parallel data: Human-generated parallel data from SMOL and GATITOS increases diversity and script coverage for additional lower-resource languages.SMOL covers 123 languages and GATITOS covers 170.
- SFT and RL data: RL uses the same translation data as SFT, except that GATITOS and SMOL are used in SFT only.The final language proportions for both phases are reported in Figure 1.
- SFT and RL data: The SFT mixture includes 30% generic instruction-following data from the original Gemma 3 mixture to prevent translation-task overfitting and preserve generic capabilities.This data is included specifically to maintain generic instruction-following capabilities.
3. Supervised Fine-Tuning
Supervised fine-tuning starts from Gemma 3 checkpoints and combines human-generated, synthetic, and generic instruction-following data using Kauldron tooling and AdaFactor optimization.
- SFT begins from the released Gemma 3 27B, 12B, and 4B checkpoints.
- The training mixture includes human-generated parallel texts, Gemini-generated synthetic parallel data, and generic instruction-following data.
- Kauldron SFT tooling is used with the AdaFactor optimizer at a learning rate of 0.0001.
4. Reinforcement Learning
TranslateGemma applies reinforcement learning on top of the supervised fine-tuning checkpoint to further improve translation quality. The approach combines multiple reward models with token-level advantages, enabling fine-grained span-level reward signals for improved credit assignment and training efficiency.
- Reward models: RL was performed on top of the SFT checkpoint using an ensemble of metrics as reward models to boost translation quality.The supplied passage identifies reinforcement learning, the SFT checkpoint, and an ensemble of metrics as the training setup.
- Reward models: MetricX-24-XXL-QE was used as a QE reward by passing an empty reference and rescaling scores with 5.0 −score so higher values indicate better quality.MetricX produces scores from 0 (best) to 25 (worst), matching the standard MQM score range.
- Reward models: The reward ensemble also included Gemma-AutoMQM-QE, ChrF, an in-house Naturalness Autorater, and a generalist reward model.ChrF uniquely used synthetic references and was scaled by a factor of two; the Naturalness Autorater penalized non-native-sounding spans.
- Advantage computation: Token-level advantages were added to sequence-level rewards, allowing direct use of fine-grained span-level signals from AutoMQM and the Naturalness Autorater.The passage states that this improved credit assignment and training efficiency.
5. Automatic Evaluation
TranslateGemma consistently improves automatic translation quality over baseline Gemma 3 models across evaluated sizes, metrics, and 55 language pairs. These gains also transfer to image translation while largely preserving multimodal capabilities.
- WMT24++ evaluation: 23.5%: the 27B TranslateGemma model reduces average MetricX from 4.04 for baseline Gemma 3 to 3.09.The passage characterizes this reduction as indicating increased translation fidelity.
- Language-pair analysis: TranslateGemma improvements remain consistent across all 55 language pairs, including English-to-German MetricX 1.63 to 1.19 and English-to-Icelandic 8.31 to 5.69.Examples span high-resource languages such as German and low-resource languages such as Icelandic and Swahili.
- Vistra image translation: Nearly 0.5 points: the 27B model improves Vistra MetricX, while the 12B model improves by 0.25; the 12B Comet22 result is an exception.TranslateGemma retains the image-processing capabilities of base Gemma 3, whereas the smaller 4B model shows only small gains.
6. Human Evaluation
Human evaluation using MQM on 10 WMT25 language pairs generally confirms TranslateGemma’s superiority over Gemma 3, with especially strong gains for low-resource pairs. German-targeted directions are exceptions, and Japanese→English regresses because of named-entity mistranslations.
- Evaluation setup: Professional translators used MQM annotations collected with Anthea to evaluate TranslateGemma on 10 language pairs spanning 3 source languages.Documents were limited to 12 source sentences, with literary chapters split into independently evaluated chunks.
- Overall results: TranslateGemma clearly outperforms Gemma 3 for most language pairs in human evaluation, confirming the trend observed in automatic metrics.Both models are on par when the target language is German.
- Overall results: Japanese→English is the main regression, caused by named-entity mistranslations despite improvements in other error categories.The human-evaluation passage identifies named-entity mistranslation as the source of the regression.
- Low-resource languages: 1.6 points is the improvement for English→Marathi, while English→Swahili improves by 1.0 point.These gains are highlighted as especially relevant for low-resource language pairs.
- Model-size comparison: The 12B TranslateGemma model remains competitive with Gemma 3 27B, especially for high-resource languages, although TranslateGemma 27B performs better than TranslateGemma 12B.This confirms the model-size performance difference seen in automatic evaluation.
7. Conclusions
TranslateGemma is an open series of Gemma 3-based models enhanced for machine translation through supervised fine-tuning and reinforcement learning. Across languages and model sizes, the models improve translation quality, efficiency, and image translation while retaining multimodal capabilities.
- Conclusions: TranslateGemma combines supervised fine-tuning on human and synthetic parallel data with reinforcement learning using an ensemble of reward models.The models span 4B, 12B, and 27B parameters.
- Conclusions: TranslateGemma shows consistent gains over baseline Gemma 3 models on MetricX and Comet22 across 55 WMT24++ language pairs.Improvements cover both high-resource languages, such as German and Spanish, and lower-resource languages, including Icelandic and Swahili.
- Conclusions: Smaller fine-tuned models often match or exceed larger baseline models, improving the trade-off between translation quality and computational cost.This efficiency pattern is reported across various language types and model sizes.
- Conclusions: TranslateGemma retains Gemma 3’s multimodal capabilities, with Vistra improvements extending to image translation particularly for the 12B and 27B models.These gains occur without specific multimodal fine-tuning.
- Conclusions: The released TranslateGemma models provide open-source tools intended to support further machine translation research and application development.The authors encourage adoption and exploration for translation-related tasks.
Contributions
The Contributions section lists the individuals associated with the work.
- Mara Finkelstein, Isaac Caswell, Tobias Domhan, Jan-Thorsten Peter, Juraj Juraska, Parker Riley, Daniel Deutsch, and Geza Kovacs are listed.
- David Vilar and Markus Freitag are listed.
- Erin Kats and Stephane Jaskiewicz are listed.
A. Automatic metrics per language
Table 4 compares TranslateGemma (GT) models with baseline Gemma (G3) models for each WMT24++ language pair using MetricX.
- A. Automatic metrics per language: Table 4 reports MetricX performance for TranslateGemma (GT) and baseline Gemma (G3) models across each language pair in WMT24++.
B. Additional synthetic data languages · C. Full list of languages for SFT
TranslateGemma’s SFT data extends beyond WMT24++ with synthetic data for 28 additional English-centered language pairs. Its full language coverage is organized across three tables covering bidirectional English pairs, English-source pairs, and non-English pairs.
- B. Additional synthetic data languages: The additional synthetic-data list includes English-Armenian, English-Hawaiian, English-Western Frisian, and English-Corsican.These are among the language pairs explicitly enumerated in the passage.
- B. Additional synthetic data languages: It also includes English-Hmong, English-Maltese, English-Tajik, English-Samoan, English-Macedonian, and English-Mongolian.The passage continues the explicit enumeration of additional language pairs.
- B. Additional synthetic data languages: Further listed pairs include English-Galician, English-Albanian, English-Uzbek, English-Uyghur, English-Belarusian, English-Sinhala, and English-Basque.These pairs are explicitly included in the additional synthetic-data language list.
- B. Additional synthetic data languages: The remaining listed pairs include English-Haitian Creole, English-Bosnian, English-Kyrgyz, English-Kazakh, English-Khmer, English-Scottish Gaelic, English-Lao, English-Irish, English-Luxembourgish, English-Burmese, and English-Sundanese.The passage enumerates these language pairs as part of the additional synthetic-data set.
- C. Full list of languages for SFT: Table 5 covers languages paired with English in both directions, Table 6 covers languages paired with English as source language, and Table 7 covers pairs not involving English.Together, the three tables define the full language coverage of the SFT data used for TranslateGemma.