PaperlayerView exact revision on arXiv

Source-linked AI summary

Regularizing Neural Machine Translation by Target-bidirectional Agreement

Zhirui Zhang, Shuangzhi Wu, Shujie Liu, Mu Li, Ming Zhou, Tong Xu

arXiv:1808.04064v2cs.CL

TL;DR

Autoregressive NMT can feed early generation errors back into the model, where they accumulate and harm later translation. The paper adds KL-divergence regularization between L2R and R2L models and trains them jointly as helpers for one another. It reports significant improvements over strong baselines on Chinese-English and English-German translation tasks.

  • Problem

    During inference, model-generated target tokens replace correct training histories, so early errors can mislead later translation; prior R2L reranking still uses exposure-biased L2R candidates.

    Source: Introduction · p. 1, Neural Machine Translation · p. 2

  • Method

    The method adds two KL-divergence regularization terms to L2R and R2L training objectives and jointly updates the models as helpers for each other.

    Source: Introduction · p. 2

  • Results

    The approach significantly improves results over strong baseline systems on Chinese-English and English-German translation tasks.

    Source: Introduction · p. 2, Conclusion · p. 7

  • Takeaways & Limitations

    L2R and R2L models can improve each other through agreement-based regularization and interactive joint training.

    Source: Introduction · p. 2

    The reported evaluations concern Chinese-English and English-German translation; testing on other sequence-to-sequence tasks is left for future work.

    Source: Conclusion · p. 7

Abstract

from arXiv · show

Although Neural Machine Translation (NMT) has achieved remarkable progress in the past several years, most NMT systems still suffer from a fundamental shortcoming as in other sequence generation tasks: errors made early in generation process are fed as inputs to the model and can be quickly amplified, harming subsequent sequence generation. To address this issue, we propose a novel model regularization method for NMT training, which aims to improve the agreement between translations generated by left-to-right (L2R) and right-to-left (R2L) NMT decoders. This goal is achieved by introducing two Kullback-Leibler divergence regularization terms into the NMT training objective to reduce the mismatch between output probabilities of L2R and R2L models. In addition, we also employ a joint training strategy to allow L2R and R2L models to improve each other in an interactive update process. Experimental results show that our proposed method significantly outperforms state-of-the-art baselines on Chinese-English and English-German translation tasks.

Introduction

Autoregressive NMT can amplify early generation errors, producing translations with good prefixes but bad suffixes. The paper regularizes L2R and R2L models to agree during training and reports significant gains over state-of-the-art baselines on Chinese-English and English-German tasks.

  • Autoregressive exposure bias lets early prediction errors mislead subsequent translation, producing good prefixes but bad suffixes.The issue can become more severe as sequence length increases.Source: Introduction · p. 1
  • Prior R2L reranking leaves exposure-biased L2R candidates in the n-best list, limiting room for improvement; R2L outputs can also have bad prefixes.Source: Introduction · p. 1
  • The method adds two KL divergence regularizers to L2R and R2L training objectives to reduce disagreement between their probability distributions.The models are jointly optimized as helper systems for each other through interactive updates.Source: Introduction · pp. 1–2
  • The proposed method significantly outperforms state-of-the-art baselines on Chinese-English and English-German translation tasks.Source: Introduction · p. 2

Neural Machine Translation

NMT models the conditional probability of a target translation given a source sentence using an encoder-decoder architecture. During inference, generated target tokens replace the correct histories seen in training, so errors can accumulate along the sequence.

  • NMT directly models the conditional probability P(y|x), with an encoder transforming the source into hidden vectors and a decoder generating the target.Source: Neural Machine Translation · p. 2
  • NMT architectures include recurrent networks and variants, as well as fully convolutional and self-attentional transformer models.Source: Neural Machine Translation · p. 2
  • During training, target-word histories are correct and observed, but inference feeds model-predicted tokens back as inputs, allowing mistakes to accumulate.This mismatch is called exposure bias.Source: Neural Machine Translation · p. 2

Our Approach

The approach regularizes L2R and R2L models to reduce disagreement between their output distributions, then trains them jointly as reciprocal helpers. Because exact KL gradients require summing over an exponential translation space, the method approximates them using sampled subspaces.

Notations

  • L2R predicts each target from preceding targets, whereas R2L predicts it from following targets.They are different chain decompositions of the same translation probability, but separate maximum-likelihood training does not guarantee identical output distributions.Source: Notations · p. 2, NMT Model Regularization · p. 2

NMT Model Regularization

  • Two KL-divergence terms are added to the maximum-likelihood objective to reduce disagreement between L2R and R2L output probabilities.The terms are zero when the two distributions match; λ weights the regularization terms.Source: NMT Model Regularization · pp. 2–3

Efficient Gradient Approximation

  • Exact KL gradients require summing over an exponential translation space, so the method approximates them with a sampled subspace and sampling methods for the two regularizers.For one direction, the gradient expectation is approximated by sampling pseudo sentence pairs from the helper model.Source: NMT Model Regularization · p. 3

Joint Training for Paired NMT Models

  • In joint training, each decoder serves as an auxiliary system that regularizes the other, and the process iterates to seek further improvements.Each iteration fixes one model while optimizing the other; training continues until development-set performance stops increasing.Source: Joint Training for Paired NMT Models · p. 4
  • The joint objective sums the L2R and R2L objectives, with both models first pretrained by maximum likelihood and then jointly optimized in alternating updates.The iterative training continues until development-set performance does not increase.Source: Joint Training for Paired NMT Models · p. 4

Experiments

Experiments evaluate the method on Chinese-English and English-German translation tasks using NIST and WMT settings. The reported comparisons show gains over baseline systems, including with back-translation.

Evaluation on NIST Corpora

Combining with Back-Translation Method

  • Transformer-big+BT+RT achieves the best reported WMT results on English-German and Chinese-English, including against ensemble systems.The authors also report that the method further improves results when combined with back-translation.Source: Table 3 · p. 6, Evaluation on WMT17 Corpora · p. 6

Only Parallel Corpora

  • On WMT test sets using only parallel corpora, Transformer+RT significantly outperforms Transformer+JS on both translation tasks.Both methods improve over their baselines on English-German and Chinese-English.Source: Table 3 · p. 6, Evaluation on WMT17 Corpora · p. 6

Training Process

  • Two joint-training iterations suffice for convergence in these experiments; further iterations bring no noticeable accuracy gains but require more training time.Total training time is almost twice that of original MLE training.Source: Table 4 · p. 6, Effect of Joint Training · p. 6

Combining with Back-Translation Method

  • 1.21 BLEU points on English-German and 1.5 BLEU points on Chinese-English are reported for Transformer-big+BT over Transformer-big.Adding RT to back-translation further improves results and gives the best results on all translation tasks.Source: Evaluation on WMT17 Corpora · p. 6

Case Study

  • In the translation case study, L2R produces a good prefix but bad suffix, R2L the reverse, and Transformer+RT a higher-quality translation.The authors attribute this case to agreement training penalizing bad suffixes and encouraging desirable suffixes.Source: Table 5 · p. 7, Example · p. 7

Related Work

Prior NMT work addresses exposure bias through new objectives or reinforcement learning, while target-bidirectional NMT has used inference-time reranking or decoder changes. This paper instead introduces target-bidirectional agreement regularization during training without changing the architecture or inference strategy.

Related Work

  • Target-bidirectional NMT methods use inference-time candidate reranking or a two-stage decoder, whereas this approach adds regularization during training and keeps inference speed unchanged.The cited prior approaches are Liu et al. (2016) and Zhang et al. (2018).Source: Related Work · p. 7
  • Exposure-bias approaches include task-specific training objectives and reinforcement-learning methods, including expected-loss minimization, beam-search optimization, mixture training, and actor-critic prediction.These methods are contrasted with adding regularization terms to the standard training objective.Source: Related Work · p. 7
  • The paper characterizes its regularization alternative as simple to implement, avoiding task-specific objectives or complex training strategies.It adds regularization terms to the standard training objective.Source: Related Work · p. 7

Conclusion

The paper presents a regularization approach based on agreement between left-to-right and right-to-left NMT models, with efficient approximation and joint optimization. It reports significant improvements over strong baselines on Chinese-English and English-German translation, while evaluation on other sequence-to-sequence tasks remains future work.

Conclusion

  • The approach significantly improves over strong baseline systems on Chinese-English and English-German translation tasks.The conclusion reports empirical evaluations on both tasks.Source: Conclusion · p. 7
  • Two KL divergences between left-to-right and right-to-left model distributions are added to the standard training objective as regularization terms.The method is based on agreement between the two directional NMT models.Source: Conclusion · p. 7
  • An efficient approximation algorithm enables fast training, and a joint strategy optimizes the left-to-right and right-to-left models together.Source: Conclusion · p. 7
  • Testing on other sequence-to-sequence tasks, including summarization and dialogue generation, is left for future work.Source: Conclusion · p. 7

Related papers

Loading 1808.04064v2…