PaperlayerView exact revision on arXiv

Source-linked AI summary

Meta Back-translation

Hieu Pham, Xinyi Wang, Yiming Yang, Graham Neubig

arXiv:2102.07847v1cs.CLcs.LG

TL;DR

Back-translation can improve NMT, but better pseudo-parallel translation quality does not necessarily yield better forward models, and the role of data diversity and quality remains unresolved. MetaBT adapts a pre-trained backward model using the forward model’s validation performance, and the paper reports gains over strong methods in standard and multilingual settings.

  • Problem

    Pseudo-parallel translation quality does not necessarily predict forward-model performance, and the effects of data diversity and quality are not fully understood.

    Source: §1 · pp. 1–2

  • Method

    MetaBT updates a pre-trained backward model based on the forward model’s performance on held-out ground-truth parallel data.

    Source: §1 · p. 2

  • Results

    MetaBT outperforms strong methods in standard and multilingual settings, including +1.66 BLEU over sampling-based BT on WMT En-De 2014.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    MetaBT continues training the backward model on its generated pseudo-parallel data without the multiple retraining rounds required by iterative BT.

    Source: §1 · p. 2

    MetaBT’s large memory footprint currently makes it difficult to apply to larger models.

    Source: §7 · p. 9

Abstract

from arXiv · show

Back-translation is an effective strategy to improve the performance of Neural Machine Translation~(NMT) by generating pseudo-parallel data. However, several recent works have found that better translation quality of the pseudo-parallel data does not necessarily lead to better final translation models, while lower-quality but more diverse data often yields stronger results. In this paper, we propose a novel method to generate pseudo-parallel data from a pre-trained back-translation model. Our method is a meta-learning algorithm which adapts a pre-trained back-translation model so that the pseudo-parallel data it generates would train a forward-translation model to do well on a validation set. In our evaluations in both the standard datasets WMT En-De'14 and WMT En-Fr'14, as well as a multilingual translation setting, our method leads to significant improvements over strong baselines. Our code will be made available.

1 INTRODUCTION

Back-translation uses target-language monolingual data to train a forward model, but the backward model can remain limited by parallel data and the best way to generate pseudo-parallel data is unclear. MetaBT updates the backward model based on the forward model’s held-out performance, improving results over strong baselines.

1 INTRODUCTION

  • The backward model in standard BT is trained on parallel data, while iterative BT addresses this constraint through multiple, slow and expensive rounds of retraining.The backward model’s quality is consequently limited by the available parallel corpus.Source: §1 · p. 1
  • Sampling and noisy beam search can produce lower-BLEU pseudo-parallel data that trains better forward models, but diversity alone is insufficient without some quality.The effect of the generated pseudo-parallel data on forward-model performance remains unclear.Source: §1 · pp. 1–2
  • MetaBT continues updating the backward model during forward-model training, using held-out ground-truth performance to derive an end-to-end update.This lets the method adapt generated pseudo-parallel data to the forward model’s validation performance.Source: §1 · p. 2
  • MetaBT trains the backward model on its own generated pseudo-parallel data and avoids iterative BT’s multiple retraining rounds.Its updates use the forward model’s held-out performance rather than requiring an explicit account of how pseudo-parallel data affects model quality.Source: §1 · p. 2
  • +1.66 BLEU over sampling-based BT on WMT En-De 2014; MetaBT also improves low-resource translation by up to +1.20 BLEU.The low-resource gain accompanies extending the backward model’s initial parallel training set with data from slightly different languages.Source: §1 · p. 2

2 A PROBABILISTIC PERSPECTIVE OF BACK-TRANSLATION

The section frames back-translation as choosing an approximation to the source-given-target distribution, then asks how to judge and improve that choice. It motivates MetaBT by posing the choice as a bi-level optimization problem, while noting that the unrestricted problem is infeasible to solve directly.

2 A PROBABILISTIC PERSPECTIVE OF BACK-TRANSLATION

  • A probabilistic framework interprets sampling, beam search, and noisy beam search as different derivations from a pre-trained backward model’s distribution.The framework supports analysis of these generation methods and motivates MetaBT’s update rules.Source: §2 · pp. 2–3
  • The framework treats each language as a distribution over token sequences and models translation with a conditional distribution between source and target sentences.The ideal training objective minimizes expected translation loss over the joint source-target distribution.Source: §2 · pp. 2–3
  • Back-translation approximates target-language sampling with a monolingual corpus and source-given-target sampling with a distribution derived from a pre-trained backward model.Beam search, sampling, and noisy beam search correspond to different choices for that derived distribution.Source: §2 · p. 3
  • The framework asks how to compare generation methods and whether better choices can be derived from a pre-trained backward model.Practitioners commonly compare choices by the resulting forward model’s held-out validation performance.Source: §2 · p. 3
  • Choosing a pseudo-parallel generation distribution can be posed as bi-level optimization, but directly solving it is infeasible.The inner loop requires training a forward model from scratch, and the unrestricted outer-loop choice of distribution is poorly defined; the paper next restricts permissible distributions to enable gradient descent.Source: §2 · p. 3

3 META BACK-TRANSLATION

Meta Back-Translation (MetaBT) adapts a pre-trained backward model so its pseudo-parallel data trains a forward model to perform well on a meta validation set. It does this through alternating updates to the forward and backward models, using the forward model’s validation loss to guide the backward model.

3 META BACK-TRANSLATION

  • MetaBT adapts a pre-trained backward model to generate pseudo-parallel data that trains a forward model to perform well on a held-out validation set.The backward model’s parameters control the generation distribution throughout forward-model training.Source: §3 · p. 4
  • The meta validation dataset measures the forward model during training, while a separate validation set is reserved for hyper-parameter tuning and model selection.The paper denotes the meta validation dataset by DMetaDev.Source: §3 · p. 4
  • The method alternates an inner update to the forward model on sampled pseudo-parallel data with an outer update to the backward model.The forward-model step treats each sampled pseudo-parallel pair as real data; the backward-model step uses the meta validation loss and its gradient with respect to backward-model parameters.Source: §3 · p. 4
  • MetaBT computes the backward-parameter gradient through the forward-model update using the chain rule and the log-gradient trick for sampling.The implementation uses high-order AutoGrad tools and Jacobian-vector products to compute the gradient dot-product term.Source: §3 · p. 4
  • The backward-model update favors pseudo-parallel samples whose forward-model gradients align with gradients on the meta validation set.The paper interprets the gradient dot product as measuring similarity in gradient directions.Source: §3 · p. 5

4 A MULLTILINGUAL APPLICATION OF METABT

MetaBT is applied to translation settings where the language pair of interest has little parallel training data. The method uses related-language parallel data to improve the backward model, then adapts it using validation data for the target language pair.

4 A MULLTILINGUAL APPLICATION OF METABT

  • With little parallel data for the language pair of interest, its backward model can be low quality and generate pseudo-parallel data that contaminates forward-model training.The passage describes this as a disadvantage shared by back-translation approaches.Source: §4 · p. 5
  • Training the backward model on related-language and target-language parallel data increases its training data, but can make generated pseudo-source sentences resemble the related language more.The related-language data may dominate the backward model’s training set.Source: §4 · p. 5
  • MetaBT can adapt the backward model using forward-model gradients on target-language-pair validation data, moving generated pseudo-source sentences closer to the language of interest.Source: §4 · p. 5

5 EXPERIMENTS

MetaBT outperforms the reported back-translation baselines across bilingual and multilingual settings, while its analysis shows that generated data adapts to the forward model’s learning state and the meta-validation data. The experiments compare standard WMT benchmarks and multilingual translation, with analyses of data fit, vocabulary coverage, and output lengths.

5.1 DATASET AND PREPROCESSING

  • The standard setting uses WMT En-De 2014 and En-Fr 2014, while the multilingual setting translates Azerbaijani, Belarusian, Galician, and Slovak to English using related high-resource languages.The filtered WMT parallel corpora contain 4.5M En-De and 40.8M En-Fr sentence pairs; the multilingual dataset contains 58 languages paired with English.Source: §5.1 · pp. 5–6, §5 · p. 5

5.3 IMPLEMENTATION

  • Experiments use Transformer-Base models, with pre-trained backward models supplying pseudo-source sentences as the forward model trains on parallel and monolingual data.The multilingual setup also uses low-resource parallel data and English data from the related high-resource language; Transformer-Large exceeds the available 16G accelerator memory for MetaBT’s training requirements.Source: §5.3 · p. 6

5.2 BASELINES

  • MLE sampling-based back-translation is the strongest consistent baseline; DualNMT loses to MLE on all tasks except az-en and is not reported for En-Fr after numerical instability.No BT uses parallel data only, while DualNMT further trains the backward model with reinforcement learning.Source: §5.4 · p. 6, §5.2 · p. 6

5.4 RESULTS

  • MetaBT consistently outperforms all baselines, gaining up to +1.20 BLEU in low-resource multilingual translation, +1.66 BLEU on WMT En-De 14, and +0.51 BLEU over MLE on WMT En-Fr.The En-Fr gain is reported as statistically significant; Table 1 reports BLEU scores for the standard bilingual and multilingual settings.Source: §5.4 · pp. 6–7, Table 1 · p. 7
  • On WMT En-De, UNMT reaches 28.76 BLEU, near SampleBT’s 28.73, and runs about 1.4 times slower per step than MetaBT on the authors’ hardware.The authors attribute UNMT’s slower steps to its multiple rounds of back-translation, denoising, and supervised training.Source: §5.4 · p. 7

5.5 ANALYSIS

  • MetaBT adjusts pseudo-parallel difficulty with dataset scale: it generates easier-to-fit data for underfitting En-Fr and harder data to help avoid overfitting on smaller En-De.En-Fr has 40.8 million parallel sentences, compared with 4.5 million for En-De; the authors describe the observed behavior as an apparent adaptation to the forward model’s learning state.Source: §5.5 · p. 7, Figure 2 · p. 7, Figure 3 · p. 7
  • In multilingual training, MetaBT increases the share of pseudo-source words from the low-resource vocabulary, favoring data more similar to the S-T meta-validation set.The plotted vocabulary coverage rises across training bins; the meta-validation set contains low-resource S-T parallel sentences, unlike the related-language S′-T data.Source: §5.5 · p. 8, Figure 4 · p. 8
  • MLE back-translation generates more outputs with extreme length differences from references, while MetaBT suffers less from this problem.The analysis attributes the difference to MetaBT continuing to train the backward model to improve forward-model development-set performance.Source: §5.5 · p. 8, Figure 5 · p. 8

6 RELATED WORK

MetaBT is related to monolingual-data methods, iterative back-translation, and approaches that select or reweight training data. Its learned data generation also connects to prior meta-learning applications.

6 RELATED WORK

  • Related methods use source-side or target-side monolingual data to improve final translation quality.The passage names He et al. (2020) for source-side data and Sennrich et al. (2016) and Edunov et al. (2018) for target-side data.Source: §6 · p. 8
  • IterativeBT trains multiple rounds of backward and forward models and improves further, while MetaBT is cheaper but cannot improve backward-model quality as much.DualNMT jointly optimizes the backward model using indirect indicators, which the paper reports lead to weak performance in § 5.4.Source: §6 · p. 9
  • MetaBT extends methods that learn to reweight or select extra training data by learning to generate pseudo-parallel data for effective training.The paper compares this relationship to sampling from a distribution versus computing its density.Source: §6 · p. 9
  • Prior work applies meta-learning to initialization, architecture design, synthetic-image generation, and pseudo-labeling.The cited examples are Finn et al. (2017), Gu et al. (2018), Liu et al. (2019), Such et al. (2019), and Pham et al. (2020).Source: §6 · p. 9

7 LIMITATION, FUTURE WORK, AND CONCLUSION

MetaBT adjusts a back-translation model to generate data effective for forward-model training, and experiments report it outperforms strong methods in standard and multilingual settings. A large memory footprint currently makes scaling to larger models difficult.

7 LIMITATION, FUTURE WORK, AND CONCLUSION

  • MetaBT learns to adjust a back-translation model to generate data that are most effective for training the forward model.Experiments report outperformance over strong existing methods in both standard NMT and multilingual settings.Source: §7 · p. 9
  • A large memory footprint currently makes MetaBT difficult to apply to larger models.The authors expect this limitation may be lifted with better technology, such as automated model parallelism or more powerful accelerators.Source: §7 · p. 9

A.1 DERIVATION FOR THE GRADIENT OF ψ

The derivation updates the backward-model parameters to minimize the forward model’s development-set loss after an expected training update. It approximates expectations with single samples and notes that optimizer choice changes the update details.

A.1 DERIVATION FOR THE GRADIENT OF ψ

  • The Jacobian is an output-by-input matrix of partial derivatives, and the gradient vector is represented by its transpose.The derivation uses standard Jacobian notation and treats vectors as columns unless otherwise specified.Source: §A.1 · p. 12
  • The backward-model gradient is derived by differentiating development-set loss through the forward-model update after sampling a pseudo-source sentence.The forward model uses the sampled pair for an SGD update, and the backward model is then updated to minimize development loss at the resulting expected parameters.Source: §A.1 · p. 12
  • The derivation approximates the first factor with a sampled θ(t), then approximates the second expectation with a sampled pseudo-source sentence.It combines these approximations with the preceding expressions to obtain the final approximating gradient ∇ψJdev.Source: §A.1 · pp. 12–13
  • The derivation assumes vanilla SGD for the forward model, but can be adapted for Adam or LAMB with a modified update.The optimizer function h can replace the gradient update if its moving averages and momentums are independent of θ and ψ.Source: §A.1 · p. 13
  • Two expectations are approximated with single samples, which could potentially cause high variance in the gradient approximation.The authors hypothesize variance is reasonable because the pre-trained backward model’s samples concentrate around correct pseudo-source sentences, but say it is hard to measure.Source: §A.1 · p. 13

A.2 TRAINING DETAILS

The appendix lists standard and multilingual training configurations, including model architecture and repeated runs. It also describes a moving-average baseline used to stabilize backward-model optimization.

A.1 TRAINING DETAILS

  • The standard setting uses Transformer-Base, shared encoder-decoder embeddings and softmax weights, and 200,000 paired forward/backward update steps.Batch sizes are 2,048 sentences for the forward model, 1,024 for the backward model, and 512 for validation batches.Source: §A.2 · p. 13
  • The multilingual model uses 512-dimensional word embeddings, 1,024-dimensional feed-forward layers, six layers, and four attention heads per encoder and decoder.The encoder and decoder embeddings are shared.Source: §A.2 · pp. 13–14
  • MBT shows larger target-word F1 gains for words with middle frequency in the related-language data than for other frequency buckets.Figure 6 orders the target-word frequency buckets from left to right by increasing frequency in S′-T data.Source: Figure 6 · p. 14
  • Each multilingual experiment is run four times with different random seeds, and the reported value is the average.Source: §A.2 · p. 14
  • Backward-model updates use a moving-average baseline of the forward-model gradient-alignment reward, subtracted before each update.The baseline is used to stabilize training.Source: §A.2 · p. 14

A.3 EFFECT OF MBT ON MULTILINGUAL TRANSFER

MBT improves word prediction accuracy in multilingual transfer, especially for middle-frequency words in the related language. For WMT en-fr, it also yields higher development BLEU and lower perplexity than standard back-translation.

A.3 EFFECT OF MBT ON MULTILINGUAL TRANSFER

  • MBT generally improves word prediction accuracy over MLE, with the largest gains for middle-frequency words in the related language.The authors suggest high-frequency words may already be well covered by low-resource-language training data; they interpret the accuracy gains as better use of related-language data.Source: §A.3 · p. 14

A.4 TRAINING PLOTS FOR WMT EN-FR

  • MBT consistently yields higher development BLEU and lower perplexity than standard back-translation on WMT en-fr.Figure 7 presents validation perplexity and BLEU for WMT en-fr.Source: §A.4 · p. 14, Figure 7 · p. 15

Related papers

Loading 2102.07847v1…