PaperlayerView exact revision on arXiv

Source-linked AI summary

Balancing Training for Multilingual Neural Machine Translation

Xinyi Wang, Yulia Tsvetkov, Graham Neubig

arXiv:2004.06748v4cs.CL

TL;DR

Multilingual MT training must balance uneven data across languages, while common heuristic sampling is sensitive to its settings and ignores language interactions. The paper learns language-level data weights from development performance and reports that MultiDDS outperforms baselines across tested settings, with flexible control over optimization objectives.

  • Problem

    Multilingual training data is imbalanced and heterogeneous, while heuristic up-sampling is sensitive to its setting and ignores interactions between languages.

    Source: §1 · p. 1

  • Method

    MultiDDS learns a scorer that weights language datasets to optimize performance on an aggregate of multilingual development sets.

    Source: §1 · p. 2

  • Results

    MultiDDS consistently outperforms various baselines across the tested language sets and translation directions, and supports different multilingual optimization objectives.

    Source: §1 · p. 2, §8 · p. 9

  • Takeaways & Limitations

    The learned weighting provides a flexible framework for prioritizing different multilingual objectives while optimizing performance across languages.

    Source: §1 · p. 2, §8 · p. 9

    Evaluation focuses on multilingual NMT; applications to other multilingual tasks remain for future work.

    Source: §1 · p. 2, §8 · p. 9

Abstract

from arXiv · show

When training multilingual machine translation (MT) models that can translate to/from multiple languages, we are faced with imbalanced training sets: some languages have much more training data than others. Standard practice is to up-sample less resourced languages to increase representation, and the degree of up-sampling has a large effect on the overall performance. In this paper, we propose a method that instead automatically learns how to weight training data through a data scorer that is optimized to maximize performance on all test languages. Experiments on two sets of languages under both one-to-many and many-to-one MT settings show our method not only consistently outperforms heuristic baselines in terms of average performance, but also offers flexible control over the performance of which languages are optimized.

1 Introduction

Multilingual training must balance languages with uneven data while managing interference from heterogeneous data. The paper proposes learning language weights from development performance, and reports that MultiDDS improves overall performance across tested settings while allowing optimization objectives to be selected.

1 Introduction

  • Uneven, heterogeneous multilingual data can cause interference that reduces performance for languages with sufficient monolingual data, especially in modest-sized models.Heuristic sampling adjustments are sensitive to their temperature setting and do not account for language similarity.Source: §1 · p. 1
  • A learned language scorer weights training data to optimize average development loss across languages, avoiding heuristic temperatures and accounting for interactions between languages.The scorer is optimized on a multilingual development set.Source: §1 · p. 2
  • MultiDDS extends DDS for multiple objectives with flexible scorer parameterization, memory efficiency, and a stabilized reward intended to improve objectives simultaneously.The stabilized reward addresses the risk of one objective overwhelming others.Source: §1 · p. 2
  • MultiDDS consistently outperforms various baselines across the tested language sets and translation directions, while supporting different multilingual optimization objectives.Experiments cover two language sets and one-to-many and many-to-one settings.Source: §1 · p. 2

2 Multilingual Training Preliminaries

The preliminaries formulate multilingual training as choosing a distribution over language-pair datasets and evaluating performance across languages. Existing fixed sampling strategies differ in how they balance dataset size and low-resource representation, and the temperature heuristic has limitations.

2 Multilingual Training Preliminaries

  • A multilingual model trains across n language-pair datasets, each with its own sentence distribution and risk, under an overall objective defined using a distribution over datasets.In practice, training samples a language according to that distribution and computes a model gradient on its data batch.Source: §2 · p. 2
  • The stated multilingual evaluation objective minimizes average development risk across languages, while assuming likelihood-based loss correlates with the test metric such as BLEU.The passage identifies this correlation as a simplifying assumption.Source: §2 · p. 2
  • Fixed sampling strategies include uniform, proportional-to-corpus-size, and temperature-based sampling, which interpolates between proportional and uniform sampling.The temperature heuristic is sensitive to its setting and ignores language similarity.Source: §2 · p. 3

3 Differentiable Data Selection

Differentiable Data Selection (DDS) uses bilevel optimization to learn training-data weights that improve performance on held-out development data. Its scorer is updated using a reward tied to the alignment between training and development gradients.

3 Differentiable Data Selection

  • DDS learns scorer parameters ψ alongside model parameters θ so that weighting training data improves the objective on held-out development data.The method frames this as bilevel optimization.Source: §3 · p. 3
  • With ψ fixed, DDS updates θ using the training objective, then iteratively updates the scorer during training.The scorer's reward is based on the estimated effect of training data on development performance.Source: §3 · p. 3
  • The data scorer is updated with a reinforcement-learning reward approximating how training data affects development performance.The reward favors training data whose gradients align with the development-data gradient.Source: §3 · p. 3

4 DDS for Multilingual Training

MultiDDS adapts DDS to multilingual training by learning a distribution over language datasets against an aggregate of language development sets. Its design targets flexible optimization across objectives and scalable training over many languages.

4 DDS for Multilingual Training

  • MultiDDS adapts DDS to optimize multilingual data usage for performance across multiple languages, with generality and scalability as design goals.The method is presented as an extension tailored for multilingual training.Source: §4 · pp. 3–4
  • MultiDDS parameterizes the sampling distribution over language datasets and optimizes it against an aggregate of development-set risks.Unlike standard DDS, it weights datasets rather than individual training examples.Source: §4 · p. 4
  • The scorer favors a training language when its gradient aligns with the gradient of the aggregated development risk across languages.Figure 1 depicts solid updates for θ and dashed updates for ψ.Source: §4 · p. 4, Figure 1 · p. 4
  • The scorer is updated periodically rather than at every training step because per-step updates are too computationally expensive for NMT.After model training for a set number of steps, the scorer is updated for all languages.Source: §4 · p. 4
  • MultiDDS estimates each language's effect by sampling a batch from its training dataset, avoiding the need to store n model gradients.The passage identifies this memory efficiency as useful when scaling to many languages.Source: §4 · p. 4

5 Stabilized Multi-objective Training

Scaling DDS to multilingual training exposed an unstable scorer reward, attributed to noisier, higher-variance multilingual development gradients. The method uses a memory-efficient approximation in place of storing a moving average of training gradients.

5 Stabilized Multi-objective Training

  • Multilingual development gradients are less consistent and have higher variance than monolingual gradients, destabilizing the scorer's reward.Source: §5 · p. 4
  • The method uses a memory-efficient reward approximation that performed as well as a moving-average approximation in preliminary experiments.The moving-average approach stores a moving average of n training gradients.Source: §5 · p. 4

25 end

The proposed stable reward averages gradient-alignment rewards calculated separately for each development set. The paper names this setting MultiDDS-S and reports lower variance than the reward in Eq. 10.

25 end

  • The stable reward averages data-to-gradient alignment rewards calculated separately for each of the n development sets.Source: §25 · p. 5
  • MultiDDS-S replaces the standard reward calculation with the stable reward and has less variance than the reward in Eq. 10.Source: §25 · p. 5

6 Experimental Evaluation

Across two language groups and both translation directions, MultiDDS-S outperforms the best heuristic baseline overall, while MultiDDS can prioritize different language objectives. The experiments also examine learned sampling distributions and show that stabilizing rewards reduces variance.

6.1 Data and Settings

  • The experiments use Related and Diverse language groups in M2O and O2M settings, with the “one” language being English.Related contains four low-resource languages and a related high-resource language for each; Diverse contains eight languages selected without regard to relatedness.Source: §6.1 · p. 5

6.2 Experiment Setup

  • The comparison includes uniform, temperature-scaled, and proportional sampling baselines, alongside MultiDDS with standard or stabilized rewards.Models use standard Transformer architectures, train for 40 epochs, and are evaluated with BLEU using sacreBLEU.Source: §6.2 · pp. 5–6

6.3 Main Results

  • MultiDDS-S delivers better overall performance than the best baseline across the translation settings, while MultiDDS alone is comparable in one setting and stronger in three.The study reports that proportional sampling is best for M2O but worst for O2M; the M2O system takes around 20 hours to converge with MultiDDS versus around 19 hours with the standard method.Source: §6.3 · p. 6, Table 1 · p. 6
  • MultiDDS-S outperforms the best baseline on more language pairs, with significant gains for five of eight languages in O2M-Related and no losses on the other three.The reported gains are larger for the Related group than for the Diverse group.Source: §6.3 · p. 6, Table 2 · p. 7

6.4 Prioritizing what to Optimize

  • Low generally improves performance on low-performing languages, while High favors high-performing languages; both priorities maintain competitive average BLEU against the baseline.The pattern is most consistent in O2M; Korean is a low-performing language despite having the most training data, and Low can prioritize it.Source: §6.4 · pp. 6–7, Table 3 · p. 7, Figure 2 · p. 7

6.5 Learned Language Distributions

  • MultiDDS gradually increases low-resource-language usage, more so in O2M than M2O, and decreases Korean usage most in the Diverse group.The authors suggest the O2M pattern may reflect the increased importance of learning language-specific decoders.Source: §6.5 · p. 7, Figure 3 · p. 8

6.6 Effect of Stablized Rewards

  • The stabilized reward has consistently less variance, and MultiDDS-S achieves higher mean and lower variance in average BLEU than MultiDDS for the Diverse group.The learned distributions fluctuate similarly, but MultiDDS makes more drastic changes than MultiDDS-S.Source: §6.6 · pp. 7–8, Table 4 · p. 8

7 Related Work

Prior work balances multilingual data with heuristic sampling and explores related language-selection, multitask-weighting, and meta-learning approaches. MultiDDS instead targets data usage for a single model supporting many languages.

7 Related Work

  • Recent multilingual work emphasizes balancing high- and low-resource languages, commonly through heuristic sampling with a temperature term.Source: §7 · p. 8
  • Wang and Neubig (2019) and Lin et al. (2019) select languages to improve one particular language, while MultiDDS targets a single model for translation across many languages.Source: §7 · p. 8
  • Zaremoodi et al. (2018) and Wang et al. (2018, 2019a) modify model architecture, whereas MultiDDS is model-agnostic and optimizes multilingual data usage.Source: §7 · p. 8
  • Kendall et al. (2018) weights task losses according to uncertainty; the paper describes its multilingual data-usage focus as somewhat orthogonal to, and less heuristic than, such loss weighting.Source: §7 · p. 8
  • MultiDDS applies meta-learning to optimize multilingual data usage for one model, unlike Gu et al. (2018), which requires a separately adapted model for each language; the authors call this the first such application.Source: §7 · p. 9

8 Conclusion

The conclusion presents MultiDDS as a language scorer for optimizing multilingual data usage and supporting performance across languages. It reports overall performance gains and flexible objective prioritization, while identifying applications and objectives for future exploration.

8 Conclusion

  • MultiDDS learns a language scorer to optimize multilingual data usage, with a more efficient multilingual DDS algorithm and a stable reward for multiple objectives.Source: §8 · p. 9
  • The method outperforms prior methods in overall performance across languages and allows users to prioritize different multilingual objectives.Source: §8 · p. 9
  • Future work may apply MultiDDS to other multilingual tasks and explore objectives beyond those considered in the paper.Source: §8 · p. 9

A.1 Effect of Step-ahead Reward

The appendix compares step-ahead reward with moving-average and baseline rewards, lists experiment hyperparameters, and provides dataset statistics and detailed BLEU tables.

A.1 Effect of Step-ahead Reward

  • The step-ahead reward is better or comparable to the moving-average reward, and both outperform the baseline in average BLEU for the related-language group.Source: Table 5 · p. 11

A.2 Hyperparameters

  • Experiments use six encoder and decoder layers, four attention heads, 512-dimensional embeddings, 1024-dimensional feedforward layers, dropout 0.3, and batches of 9600 tokens.Source: §A.2 · p. 11
  • The setup also uses label smoothing at 0.1 and scaled L2 normalization before residual connections.Source: §A.2 · p. 11

A.3 Dataset statistics

A.4 Detailed Results for All Settings

Related papers

Loading 2004.06748v4…