PaperlayerView exact revision on arXiv

Source-linked AI summary

A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models

Haoran Xu, Young Jin Kim, Amr Sharaf, Hany Hassan Awadalla

arXiv:2309.11674v2cs.CL

TL;DR

Moderate-sized LLMs have lagged behind conventional translation models, and prior efforts to improve their translation performance made limited gains. The paper proposes ALMA, which fine-tunes first on monolingual data and then on a small, high-quality parallel dataset. Across 10 directions, it reports more than 12 BLEU and COMET average improvement over zero-shot LLaMA-2, surpassing prior work, NLLB-54B, and GPT-3.5-text-davinci-003.

  • Problem

    Moderate-sized decoder-only LLMs lag behind supervised encoder-decoder translation models, while earlier efforts to improve their translation performance have made limited gains.

    Source: §1 · pp. 1–2

  • Method

    ALMA fine-tunes a decoder-only LLM first on non-English monolingual data and then on a small amount of high-quality parallel data.

    Source: §1 · p. 2

  • Results

    More than 12 BLEU and COMET average improvement over zero-shot LLaMA-2 across 10 directions; ALMA surpasses prior work, NLLB-54B, and GPT-3.5-text-davinci-003.

    Source: §7 · p. 9

  • Takeaways & Limitations

    The results show that LLM translation can improve without relying heavily on the abundant parallel data used by traditional translation models.

    Source: §1 · p. 2, §7 · p. 9

Abstract

from arXiv · show

Generative Large Language Models (LLMs) have achieved remarkable advancements in various NLP tasks. However, these advances have not been reflected in the translation task, especially those with moderate model sizes (i.e., 7B or 13B parameters), which still lag behind conventional supervised encoder-decoder translation models. Previous studies have attempted to improve the translation capabilities of these moderate LLMs, but their gains have been limited. In this study, we propose a novel fine-tuning approach for LLMs that is specifically designed for the translation task, eliminating the need for the abundant parallel data that traditional translation models usually depend on. Our approach consists of two fine-tuning stages: initial fine-tuning on monolingual data followed by subsequent fine-tuning on a small set of high-quality parallel data. We introduce the LLM developed through this strategy as Advanced Language Model-based trAnslator (ALMA). Based on LLaMA-2 as our underlying model, our results show that the model can achieve an average improvement of more than 12 BLEU and 12 COMET over its zero-shot performance across 10 translation directions from the WMT'21 (2 directions) and WMT'22 (8 directions) test datasets. The performance is significantly better than all prior work and even superior to the NLLB-54B model and GPT-3.5-text-davinci-003, with only 7B or 13B parameters. This method establishes the foundation for a novel training paradigm in machine translation.

1 INTRODUCTION

Moderate-sized decoder-only LLMs lag behind conventional translation systems, and earlier attempts to improve them made limited gains. The paper introduces ALMA, a two-stage fine-tuning approach, and reports stronger translation performance with less reliance on parallel data.

1 INTRODUCTION

  • Earlier studies with smaller LLMs reported gains of only 3–4 BLEU or COMET over zero-shot LLaMA on WMT’22, while a substantial gap to leading systems remained.The paper attributes these modest gains to an unsuitable training recipe.Source: §1 · p. 2
  • ALMA first fine-tunes on non-English monolingual data, then on a small amount of high-quality parallel data.The authors hypothesize that these stages respectively build multilingual knowledge and guide translation generation.Source: §1 · p. 2
  • The paper argues that LLM translation need not rely on the large parallel datasets used by traditional translation frameworks, and that extensive parallel data may harm LLM translation capabilities.Prior studies had fine-tuned LLMs using more than 300M parallel instances.Source: §1 · p. 2
  • More than 12 BLEU and COMET average improvement over zero-shot LLaMA-2 across 10 directions; ALMA surpasses prior work, NLLB-54B, and GPT-3.5-text-davinci-003.The reported gains use a two-stage fine-tuning recipe.Source: §1 · pp. 2–3
  • Only 1B monolingual tokens achieved performance comparable to NLLB-54B across 10 directions, requiring around 18 hours on 16 MI200 GPUs.The authors report that both stages are crucial for achieving large improvements.Source: §1 · p. 3

2 PRELIMINARY

The preliminary section defines translation for decoder-only models as predicting target tokens conditioned on the source sentence and a fixed prompt. It then compares zero-shot LLMs to select backbones for further study.

2.1 TASK DEFINITION

  • The training objective is causal language modeling of the target sentence given the source and prompt, with no loss computed for the prompt or source sentence.The prompt names the source and target languages; the target sequence is predicted token by token.Source: §2.1 · p. 3, Figure 2 · p. 3

2.2 A BACKBONE LLM FOR TRANSLATION

  • The zero-shot comparison includes OPT-7B, Falcon-7B, BLOOM-7B, MPT-7B, LLaMA-1-7B, and LLaMA-2-7B, alongside two GPT-3.5 variants.The evaluation focuses on 7B models because most of the compared LLM families provide a version at that size.Source: §2.2 · p. 3
  • Zero-shot evaluation covers five English-centric language pairs in both directions, using WMT’21 data for Icelandic and WMT’22 data for the other languages.The evaluation reports sacre-BLEU and COMET.Source: §2.2 · pp. 3–4
  • Among 7B models, LLaMA-2-7B leads in COMET in both directions; by BLEU, it leads into English while MPT-7B leads out of English.The authors select LLaMA-2-7B and MPT-7B for further study of parallel-data requirements.Source: §2.2 · p. 4

3 DO LLMS HAVE AN APPETITE FOR PARALLEL DATA?

The paper investigates whether decoder-only LLMs need the large parallel datasets common in conventional translation training. In an English-to-Russian fine-tuning study, results differ by model and metric, with LLaMA-2-7B showing a COMET decline as parallel data increases.

3.1 EXPERIMENTAL DESIGN

  • The experiment fine-tunes LLaMA-2-7B and MPT-7B on English→Russian using five data sizes from 10K to 20M examples and compares results with NLLB-54B.The authors chose a direction out of English into a non-Latin language because their initial investigation found larger gaps with state-of-the-art systems for those categories.Source: §3.1 · p. 4, §3.2 · p. 4

3.2 OBSERVATIONS

  • By COMET, LLaMA-2-7B peaks with 10K or 100K examples and then declines, while MPT-7B improves with more data; at 20M, both models have similar BLEU and COMET scores.The authors suggest that extensive parallel data may dilute LLaMA-2-7B’s existing knowledge.Source: §3.2 · p. 5, Figure 4 · p. 5
  • At 20M parallel examples, a model trained from scratch reached similar BLEU and COMET scores, which the authors say strengthens their hypothesis that extensive data washes out prior knowledge.The authors frame catastrophic forgetting as a hypothesis for this pattern.Source: §3.2 · p. 5
  • For LLaMA-2-7B, COMET declines as parallel data increases even as BLEU rises, a divergence the authors attribute partly to BLEU’s lexical-overlap focus and test-domain similarity.The authors argue that additional metrics are needed for a comprehensive evaluation.Source: §3.2 · p. 5
  • The results lead the authors to argue that LLaMA-2 and potentially other well-trained LLMs should not use the data-heavy training approach of earlier models.Their experiment focused on English→Russian and two 7B models.Source: §3.1 · p. 4, §3.2 · p. 5

4 A NEW TRAINING RECIPE

The proposed ALMA recipe combines monolingual-data fine-tuning with fine-tuning on a small, high-quality parallel dataset. The two stages target non-English language proficiency and translation generation, respectively.

4 A NEW TRAINING RECIPE

  • ALMA is the name for the result of a two-stage recipe: continuous monolingual-data fine-tuning followed by high-quality parallel-data fine-tuning.The approach is presented as improving translation without relying heavily on parallel data.Source: §4 · p. 5
  • The first stage fine-tunes on monolingual data in non-English task languages to improve proficiency, and also includes English data to prevent English-knowledge forgetting.The authors note full-weight fine-tuning at this stage.Source: §4 · pp. 5–6
  • The second stage uses a small, high-quality parallel dataset drawn from human-written WMT and Flores-200 development and test data.The authors explore both full-weight fine-tuning and LoRA applied to each feed-forward network’s down-projection layer.Source: §4 · p. 6

5 EXPERIMENTS

The experiments evaluate ALMA’s data, training setup, and performance against prior translation systems and stronger benchmarks. ALMA-13B-LoRA leads prior work and some large models, while remaining slightly behind GPT-3.5-T and GPT-4.

5.1 DATA

  • Training data combines OSCAR monolingual text with 58K human-written parallel examples collected from WMT’17–WMT’20 and Flores-200.The test set covers 10 directions across cs, de, is, zh, ru, and English; WMT’21 data, except Icelandic, supplies development data.Source: §5.1 · p. 6

5.2 TRAINING SETUP

  • ALMA uses LLaMA-2-7B or -13B in a many-to-many setup, with full-weight tuning in both stages or LoRA in the parallel-data stage.The LoRA variants update 0.1% of parameters; training runs use up to 20B monolingual tokens for 7B and 12B for 13B, followed by two epochs on human-written data.Source: §5.2 · p. 6

5.3 BASELINES

  • The evaluation compares ALMA with prior LLM translation studies and state-of-the-art systems, while noting differences in training data and architectures make comparisons imperfect.The reported baselines include NLLB-54B and zero-shot GPT-3.5 and GPT-4; Tables 1 and 2 cover en→xx and xx→en.Source: §5.3 · p. 7, Table 1 · p. 7, Table 2 · p. 8

5.4 RESULTS

  • ALMA-13B-LoRA outperforms prior studies, NLLB-54B, and GPT-3.5-D, while marginally underperforming GPT-3.5-T and GPT-4.For en→xx, its average COMET is 87.00 versus 86.56 for GPT-3.5-T; Icelandic performance remains strong against the compared systems.Source: §5.4 · pp. 7–8, Table 1 · p. 7, Table 2 · p. 8

6 ANALYSES

The analyses examine how monolingual-token volume and parallel-data quality affect ALMA. One billion monolingual tokens followed by human-written data yields performance comparable to NLLB-54B and GPT-3.5-D, while monolingual data and higher-quality parallel data improve results in the ablation.

6.1 HOW MUCH MONOLINGUAL DATA TO USE?

  • 1B monolingual tokens followed by human-written fine-tuning yields performance comparable to NLLB-54B and GPT-3.5-D across the 10 directions.The reported training time is about 19 hours: 18 hours for 1B tokens and one additional hour for human-written data.Source: §6.1 · p. 9

6.2 THE EFFECT OF MONOLINGUAL DATA AND PARALLEL DATA QUALITY

  • 83.98 versus 74.35 COMET for en→xx shows the gain from adding monolingual data when both models use the same Filtered parallel data.Across settings, higher-quality parallel data performs better, and the authors identify both monolingual and human-written data as important factors.Source: §6.2 · p. 9, Table 3 · p. 9

6.3 OTHER ANALYSES

  • The appendix examines human-written data volume and domain, stage-two fine-tuning versus few-shot in-context learning, and cross-lingual capabilities after stage one.Source: §6.3 · p. 9

7 CONCLUSION

The conclusion presents ALMA as a translation recipe for decoder-only LLMs that does not require as extensive a parallel-data collection as traditional translation models. With LLaMA-2, ALMA improves over zero-shot performance and surpasses prior studies, NLLB-54B, and GPT-3.5-D.

7 CONCLUSION

  • ALMA exceeds LLaMA-2 zero-shot performance by more than 12 BLEU and COMET on average across 10 directions, and surpasses prior studies, NLLB-54B, and GPT-3.5-D.The authors conclude that LLMs do not require as extensive a parallel-data collection as traditional translation models.Source: §7 · p. 9

A COMPARING LLM TRAINING OBJECTIVES FOR MACHINE TRANSLATION

With limited parallel data, causal language modeling performs better than prefix LM and mixture-of-denoisers for translation in the reported comparison. The study therefore adopts CLM as its primary objective.

A COMPARING LLM TRAINING OBJECTIVES FOR MACHINE TRANSLATION

B FULL RESULTS OF ZERO-SHOT EVALUATION

This appendix reports detailed zero-shot translation results and documents the training, monolingual sampling, and data-split settings used in the experiments.

B FULL RESULTS OF ZERO-SHOT EVALUATION

C TRAINING DETAILS

  • Training used 16 MI200 GPUs, an effective batch size of 256, and a peak learning rate of 2e-5; LLaMA-2 human-written-data fine-tuning ran for 2 epochs.MPT-7B training ran for 3 epochs.Source: C TRAINING DETAILS · p. 16

D.1 SAMPLING RATIO FOR MONOLINGUAL DATA

  • English received 1/6 of monolingual sampling probability, while the remaining 5/6 used temperature sampling with T=6 for non-English languages.Source: §D.1 · p. 16

D.2 DATA STATISTICS

  • Training data combined WMT’17–WMT’20 parallel data and Oscar monolingual data, with WMT’21 development and WMT’22 test data except for Icelandic.Icelandic had no development set.Source: §D.2 · p. 17, Table 5 · p. 17

E OFF-TARGET ISSUE FOR LLAMA-2-13B

The appendix examines LLaMA-2-13B’s off-target zero-shot outputs and mitigation strategies, reports results from tuning at 1B-token intervals, and describes the ablation design.

E OFF-TARGET ISSUE FOR LLAMA-2-13B

F NUMERIC RESULTS FOR MODELS FINE-TUNED WITH EVERY 1B TOKENS

G DETAILED RESULTS IN ABLATION STUDY

  • The ablation study examines monolingual fine-tuning and parallel-data quality; Table 9 marks omitted monolingual tuning or parallel data with crosses.Checks indicate monolingual-data fine-tuning.Source: G DETAILED RESULTS IN ABLATION STUDY · p. 19

H IS MORE HUMAN-WRITTEN PARALLEL DATA BETTER?

These analyses compare parallel-data composition and fine-tuning with in-context learning, then test how monolingual-data volume relates to cross-lingual performance.

H IS MORE HUMAN-WRITTEN PARALLEL DATA BETTER?

I PARALLEL DATA FINE-TUNING VS. IN-CONTEXT LEARNING

  • Five-shot in-context learning benefited from higher-quality examples but substantially underperformed stage 2 fine-tuning, and adding ICL after stage 2 brought no further benefit.The comparison used LLaMA-2-13B after stage 1 as its backbone.Source: Table 11 · p. 20, I PARALLEL DATA FINE-TUNING VS.… · p. 21

J CROSS-LINGUAL PROFICIENCY OF FINE-TUNED MODELS

  • Across XNLI, XStoryCloze, and XWinograd, 1B- or 20B-token monolingual tuning improved non-English performance, with more data consistently corresponding to larger gains.Evaluations covered German (XNLI only), Chinese, Russian, and English; none of the datasets covered Icelandic.Source: J CROSS-LINGUAL PROFICIENCY OF… · p. 21, Table 12 · p. 21

Related papers

Loading 2309.11674v2…