Source-linked AI summary
Competence-based Curriculum Learning for Neural Machine Translation
Emmanouil Antonios Platanios, Otilia Stretcu, Graham Neubig, Barnabas Poczos, Tom M. Mitchell
TL;DR
Large-scale NMT systems can be slow to train and may require specialized optimization heuristics. This paper proposes a competence-based curriculum that selects examples according to difficulty and model competence. It reports up to a 70% training-time reduction and up to 2.2 BLEU points of improvement.
Problem
Large-scale NMT systems are slow and often require specialized learning rates, large batches, and extensive heuristic tuning.
Method
The framework continuously filters training examples by estimated difficulty and the model’s current competence, while requiring only the curriculum duration as a tunable parameter.
Results
The approach reduces training time by up to 70% and improves learned-model performance by up to 2.2 BLEU points.
Takeaways & Limitations
The method makes training Transformers faster and more reliable, while having a much smaller effect on RNN training.
Takeaways & Limitations
Experiments did not cover other architectures or configurations, so the baseline architectures were selected from existing literature rather than for favorability to the method.
Abstract
from arXiv · showhide
Current state-of-the-art NMT systems use large neural networks that are not only slow to train, but also often require many heuristics and optimization tricks, such as specialized learning rate schedules and large batch sizes. This is undesirable as it requires extensive hyperparameter tuning. In this paper, we propose a curriculum learning framework for NMT that reduces training time, reduces the need for specialized heuristics or large batch sizes, and results in overall better performance. Our framework consists of a principled way of deciding which training samples are shown to the model at different times during training, based on the estimated difficulty of a sample and the current competence of the model. Filtering training samples in this manner prevents the model from getting stuck in bad local optima, making it converge faster and reach a better solution than the common approach of uniformly sampling training examples. Furthermore, the proposed method can be easily applied to existing NMT models by simply modifying their input data pipelines. We show that our framework can help improve the training time and the performance of both recurrent neural network models and Transformers, achieving up to a 70% decrease in training time, while at the same time obtaining accuracy improvements of up to 2.2 BLEU.
1 Introduction
NMT systems, especially Transformers, can be difficult and expensive to train because they rely on large-scale optimization and specialized heuristics. The paper proposes a continuous, competence-based curriculum that orders examples by difficulty and reports faster training with improved performance.
- Transformers can outperform recurrent models but often require specialized learning rates and large-batch training, making large-scale NMT difficult to optimize.
- The proposed curriculum presents easier examples before harder ones as the model becomes more competent, aiming to avoid bad local optima during training.
- The framework uses a continuous curriculum with one tunable parameter: the duration of curriculum learning.
- The method is generic and extensible, and heuristic-based approaches such as Kocmi and Bojar (2017) can be formulated as special cases.
- 70% reduction in training time is reported, while learned-model performance improves by up to 2.2 BLEU points.
2 Proposed Method
The framework filters NMT training data according to sample difficulty and the learner’s current competence, gradually expanding the usable training set. It supports multiple difficulty and competence functions while requiring only limited pipeline changes and hyperparameter tuning.
- Framework concepts: Difficulty scores rank training samples comparably, while competence c(t) determines the proportion of ranked examples available at each training step.Difficulty may depend on the learner’s state; competence is defined between 0 and 1 as the proportion of training data the learner may use.
- Design properties: The framework supports continuous curricula with a single tunable duration parameter and can be applied to existing NMT systems through small training-pipeline modifications.Once competence reaches 1, training becomes equivalent to non-curriculum training while the learner is expected to be more capable of handling difficult examples.
- Training procedure: At each step, the algorithm computes sample difficulties and their empirical CDF scores, computes model competence, and samples a batch whose difficulty is at most that competence.The filtering constrains the domain of the input distribution rather than changing the relative probability of eligible samples.
- Difficulty metrics: Sentence length and word rarity provide two example difficulty heuristics for NMT training data.Sentence length is motivated by longer sequences and error propagation; word rarity uses approximate sentence likelihood based on unigram probabilities.
- Competence functions: The generalized competence model achieves best observed performance at p = 2, while larger p values approach the performance of training without a curriculum.The paper also notes that more sophisticated competence strategies based on loss, gradients, or held-out performance are possible but are not studied.
3 Experiments
Experiments evaluate curriculum learning across RNNs and Transformers using BLEU and relative training time, with comparisons against plain training and Transformer learning-rate scheduling. The approach improves convergence speed and final performance, especially for Transformers.
- Experimental setup: Experiments use three NMT datasets and evaluate both bidirectional-LSTM RNNs and Transformers.The datasets range from a small benchmark to a large-scale corpus with millions of sentences.
- Curriculum hyperparameters: The curriculum starts with 1% of the easiest examples, while curriculum length is the only tuned hyperparameter.Curriculum length is set using the baseline steps needed to reach approximately 90% of final BLEU.
- Scope: The experiments did not include other architectures or configurations, so the baseline architectures were selected for prevalence in prior literature.This limits how broadly the findings can be generalized across model configurations.
- Results: Transformers consistently gain up to 2 BLEU and reduce training time by up to 70% with curriculum learning.RNNs also benefit, but to a lesser extent.
- Results: The square root competence model consistently outperforms the linear model, while sentence length and sentence rarity yield similar performance.These comparisons evaluate alternative competence schedules and difficulty heuristics.
- Results: RNNs converge faster than Transformers on the two small datasets, but Transformers achieve higher final test BLEU.For IWSLT-15, the curriculum approach reaches 29.81 BLEU versus a previously reported 29.03 BLEU.
- Learning-rate schedule: With the Transformer learning-rate schedule, curriculum learning obtains a better model in about 70% less training time while tuning only curriculum length.Without a schedule, curriculum methods still reached performance comparable to Plain* in about twice as many steps, whereas Plain stayed below 2.00 BLEU.
4 Related work
Prior machine-translation work explored curricula, hard-example selection, active learning, and Transformer optimization, but these approaches differ in scope and training objectives. The paper positions its method as a more principled way to address Transformer training difficulty.
- Curriculum learning: Earlier machine-translation curricula included frequency-based bilingual embedding groups and easy-to-hard sentence ordering.Some approaches did not directly follow the original curriculum-learning definition or introduced all samples during the first epoch.
- Hard-example training: Hard-example selection improved BLEU using the hardest 80% of training examples but did not speed training.The paper distinguishes this approach from curriculum learning because it resembles boosting.
- Active learning: Active-learning methods request examples using n-gram occurrence frequency, treating rare n-grams as indicators of difficult translation examples.These methods focus on soliciting data rather than ordering the full training process as a curriculum.
- Transformer optimization: Transformer studies report higher performance from carefully tuned learning-rate schedules and larger batch sizes.The paper interprets these findings as indicative of noisy gradients early in Transformer training.
5 Conclusion and Future Work
The paper presents competence-based curriculum learning for NMT, which improves existing systems while reducing training time and tuning demands. It identifies future directions in difficulty heuristics, competence metrics, and broader machine-learning applications.
- Conclusion: The framework boosts existing NMT performance while significantly reducing training time and dependence on difficult-to-tune hyperparameters.It filters training data using learner competence and example difficulty rather than relying on multiple hyperparameters or a manually designed discretized regime.
- Conclusion: It makes Transformer training faster and more reliable, whereas its effect on recurrent neural networks is much smaller.
- Future Work: Future work will explore richer difficulty heuristics, including alignment, sentence-length discrepancies, and pretrained language-model scores.These methods are proposed as possible improvements over the sentence-rarity heuristic.
- Future Work: Future work will also investigate more sophisticated competence metrics that may depend on the loss function.
- Future Work: The authors plan to test the framework in more general machine-learning tasks.