Source-linked AI summary
Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets
Zhen Yang, Wei Chen, Feng Wang, Bo Xu
TL;DR
NMT objectives based on manually defined n-gram precision may miss important aspects of the data distribution and yield suboptimal sentences. The paper introduces a BLEU-reinforced conditional sequence GAN with adversarial generator-discriminator training, and reports consistent improvements over RNNSearch and Transformer baselines on English-German and Chinese-English translation.
Problem
Manually defined n-gram precision may not cover all crucial aspects of the data distribution, allowing NMT models to generate suboptimal sentences.
Method
The proposed BR-CSGAN jointly trains a generator and source-conditioned discriminator, while using sentence-level BLEU as an additional generator reward.
Results
The proposed approach consistently outperforms baselines on English-German and Chinese-English translation, including +0.81 BLEU points on Chinese-English and +0.62 on English-German with Transformer models.
Takeaways & Limitations
Combining adversarial feedback with the BLEU objective yields better translation performance than naive GAN training or BLEU-only guidance in the reported experiments.
Abstract
from arXiv · showhide
This paper proposes an approach for applying GANs to NMT. We build a conditional sequence generative adversarial net which comprises of two adversarial sub models, a generator and a discriminator. The generator aims to generate sentences which are hard to be discriminated from human-translated sentences (i.e., the golden target sentences), And the discriminator makes efforts to discriminate the machine-generated sentences from human-translated ones. The two sub models play a mini-max game and achieve the win-win situation when they reach a Nash Equilibrium. Additionally, the static sentence-level BLEU is utilized as the reinforced objective for the generator, which biases the generation towards high BLEU points. During training, both the dynamic discriminator and the static BLEU objective are employed to evaluate the generated sentences and feedback the evaluations to guide the learning of the generator. Experimental results show that the proposed model consistently outperforms the traditional RNNSearch and the newly emerged state-of-the-art Transformer on English-German and Chinese-English translation tasks.
1 Introduction
The paper identifies a limitation in manually defined NMT objectives and introduces adversarial training to generate translations that resemble human translations. It combines this approach with a sentence-level BLEU objective and evaluates it across two translation tasks and two NMT architectures.
- Traditional NMT objectives based on n-gram precision may not cover all crucial aspects of the data distribution, producing suboptimal sentences.
- The proposed conditional sequence generative adversarial net trains a generator to produce target sentences that are difficult for a discriminator to distinguish from human translations.
- The discriminator learns automatically what human translations look like instead of relying solely on a manually defined data distribution.
- Experiments cover English-German and Chinese-English translation using both RNNSearch and Transformer models, with the proposed approach consistently achieving strong results.
- Smoothed sentence-level BLEU provides a static generator objective that biases generation toward high BLEU scores and combines with adversarial training.
2 Background and Related Work
The background describes encoder-decoder NMT architectures, highlighting attention-based context construction and the Transformer’s fully attention-based design. It also situates the work within emerging applications of generative adversarial training beyond computer vision.
- RNNSearch and Transformer: RNNSearch encodes source sentences with bidirectional recurrent units and generates targets with an attention-based recurrent decoder.
- RNNSearch and Transformer: Attention computes context vectors by weighting source annotations according to their alignment with each predicted target word.
- RNNSearch and Transformer: The Transformer uses stacked encoder and decoder layers built around multi-head attention and positionwise feed-forward networks.
- RNNSearch and Transformer: Unlike recurrent or convolutional architectures, the Transformer permits greater parallelization during training.
- Related Work: Generative adversarial training originated in computer vision and had begun extending to conditional settings and NLP applications.
3 The Approach
BR-CSGAN combines a conditional generator, discriminator, and static sentence-level BLEU objective for adversarial sequence training. Policy-gradient learning uses discriminator and BLEU rewards, with Monte Carlo sampling to evaluate incomplete target sequences.
- Model overview: BR-CSGAN views sentence generation as policy-controlled actions and comprises a generator, discriminator, and BLEU objective.The generator targets human-like translations, while the discriminator distinguishes generated from human translations.
- BLEU objective: The static sentence-level BLEU objective Q rewards high BLEU points and is fused with the discriminator reward, with both ranging from zero to one.Q measures n-gram precisions for generated and ground-truth sentences.
- Generator: The generator uses the same general architecture as NMT models and is tested with RNNSearch and Transformer architectures.The method does not assume a specific generator architecture.
- Discriminator: The conditional discriminator uses a CNN and padding to map variable-length sentences to a fixed maximum length for classification.Source and target sentence representations are constructed from word embeddings and convolutional features.
- Policy-gradient training: Policy-gradient training maximizes expected end reward, using discriminator and BLEU feedback to define action values for generated target sequences.The baseline reduces reward variance, while Monte Carlo searches sample unknown suffix tokens for intermediate states.
- Training strategy: Training alternates discriminator and generator updates, after maximum-likelihood generator pretraining and discriminator pretraining on human and machine-generated data.Teacher forcing is adopted because simple policy-gradient updates were unstable in practice.
4 Experiments and Results
BR-CSGAN is evaluated on Chinese-English and English-German translation using RNNSearch and Transformer generators. It consistently improves BLEU over the baselines, including state-of-the-art Transformer systems.
- Experimental setup: Experiments evaluate BR-CSGAN on English-German and Chinese-English translation with RNNSearch and Transformer generators.The English-German setup uses WMT’14 En-De, while Chinese-English uses LDC-derived training data.
- Experimental setup: The experiments use BLEU for evaluation, with development sets selecting hyperparameters and test sets reporting translation performance.The Transformer result averages five checkpoints around the best development-set model.
- Main results: +1.83 BLEU points on Chinese-English and +1.69 BLEU points on English-German are achieved by RNNSearch with BR-CSGAN using λ=0.7.These improvements exceed the corresponding naive GAN and BLEU-only variants.
- Main results: +0.81 BLEU points on Chinese-English and +0.62 BLEU points on English-German are achieved when BR-CSGAN is applied to Transformer.The gains occur despite Transformer’s state-of-the-art baseline performance.
- Main results: BR-CSGAN consistently outperforms the baselines, naive GAN, and the model guided only by the BLEU objective.The reported comparisons cover both translation directions and both generator architectures.
5 Analysis
The analysis compares BR-CSGAN with MRT and examines how discriminator initialization and Monte Carlo sampling affect training and translation performance. Results indicate that both adversarial balance and sampling size require careful tuning.
- 5.1 Compared with MRT: BR-CSGAN is a weighted combination of naive GAN and MRT, combining dynamic discriminator rewards with static BLEU guidance.With λ=1 it corresponds to naive GAN, while λ=0 approaches MRT; reinforcement learning replaces MRT’s random-sampling risk approximation.
- 5.1 Compared with MRT: BR-CSGAN consistently outperforms MRT on Chinese-English and English-German translation tasks.
- 5.2 When to stop pre-training: Discriminator initialization strongly affects performance: accuracies of 0.6, 0.7, 0.9, and 0.95 perform badly, whereas about 0.8 is used empirically.The authors attribute this to the need for a balanced generator–discriminator relationship at the start of adversarial training.
- N NIST02 NIST03 NIST04 NIST05: Monte Carlo sample counts below 15 provide no improvement over the pretrained generator and can destabilize training.The authors attribute this to high reward variance and incorrect gradient directions.
- N NIST02 NIST03 NIST04 NIST05: Increasing N improves translation performance up to approximately N=20, after which gains are small and training becomes excessively time-consuming.
6 Conclusion and Future Work
The paper proposes BR-CSGAN, a BLEU-reinforced generative adversarial approach for improving neural machine translation. Experiments across two architectures and two translation directions report consistent improvements, while future work considers multiple discriminators and generators.
- 6 Conclusion and Future Work: BR-CSGAN applies BLEU-reinforced generative adversarial training to neural machine translation.
- 6 Conclusion and Future Work: Experiments with RNNSearch and Transformer on Chinese-English and English-German translation tasks show consistent significant improvements.
- 6 Conclusion and Future Work: Future work will investigate a multi-adversarial framework containing multiple discriminators and generators.