Source-linked AI summary
Let the Model Decide its Curriculum for Multitask Learning
Neeraj Varshney, Swaroop Mishra, Chitta Baral
TL;DR
Prior MTL curricula use human difficulty judgments or computationally expensive exhaustive search, motivating model-based curriculum formation. The paper introduces dataset-level and instance-level ordering and reports average gains of 4.17% and 3.15% over respective baselines across 12 datasets, concentrated on difficult instances.
Problem
Prior MTL curriculum methods rely on human difficulty judgments that may diverge from machine interpretation or on exhaustive searches that become computationally intractable.
Method
The paper uses model-computed difficulty scores to arrange training curricula at dataset-level and instance-level granularity.
Results
Across 12 datasets, instance-level and dataset-level techniques improve average performance by 4.17% and 3.15% over their respective baselines, with difficult instances contributing most.
Takeaways & Limitations
The techniques are generic for MTL and are especially effective on tasks containing difficult instances.
Takeaways & Limitations
Computing difficulty scores adds training-time computation, and noisy instances may receive high scores that can negatively affect learning when noise is substantial.
Abstract
from arXiv · showhide
Curriculum learning strategies in prior multi-task learning approaches arrange datasets in a difficulty hierarchy either based on human perception or by exhaustively searching the optimal arrangement. However, human perception of difficulty may not always correlate well with machine interpretation leading to poor performance and exhaustive search is computationally expensive. Addressing these concerns, we propose two classes of techniques to arrange training instances into a learning curriculum based on difficulty scores computed via model-based approaches. The two classes i.e Dataset-level and Instance-level differ in granularity of arrangement. Through comprehensive experiments with 12 datasets, we show that instance-level and dataset-level techniques result in strong representations as they lead to an average performance improvement of 4.17% and 3.15% over their respective baselines. Furthermore, we find that most of this improvement comes from correctly answering the difficult instances, implying a greater efficacy of our techniques on difficult tasks.
1 Introduction
Prior MTL curricula rely on human difficulty judgments or costly exhaustive ordering, so this work lets models form curricula from model-based difficulty at dataset and instance granularity.
- Model-based curricula address the mismatch between human and machine difficulty judgments and avoid exhaustive search that becomes intractable with larger datasets.
- The proposed framework arranges training at two granularities: dataset-level ordering and instance-level ordering based on model-computed difficulty.Dataset-level methods keep each dataset together, whereas instance-level methods relax dataset boundaries.
- 4.17% and 3.15% average performance gains show that instance-level and dataset-level techniques improve over their respective baselines across 12 datasets.
- The techniques can be combined with dynamic sampling and pre-finetuning to further improve multi-task learning performance.
- Difficult instances contribute most to the gains, making the techniques especially applicable to difficult real-world tasks in low-data settings.
2 Difficulty Score Computation
The paper computes instance difficulty from model confidence using cross-review or confidence averaged across training epochs, then uses these scores to characterize examples.
- 2 Difficulty Score Computation: Difficulty scores are computed with two model-based methods: cross review across separately trained meta-dataset models and average confidence across epochs.
- 2.1 Cross Review Method: 2.1 Cross Review Method splits the training data into N equal meta-datasets and scores each instance by one minus the average confidence from models trained on other splits.
- 2.2 Average Confidence Across Epochs: 2.2 Average Confidence Across Epochs computes difficulty as one minus the average prediction confidence recorded across E checkpoints of a single model.
- 2.2 Average Confidence Across Epochs: Algorithm 1 initializes a model, incrementally trains it on sampled fractions of ordered splits, and finally trains it on the full dataset.
3 Proposed Techniques
The proposed curriculum trains sequentially on difficulty-ordered splits while replaying some earlier instances, with dataset-level and instance-level methods differing in how splits are formed.
- 3 Proposed Techniques: Training data are divided into ordered difficulty splits, trained sequentially with previous-split instances replayed to reduce catastrophic forgetting, then trained jointly on the full dataset.
- Dataset-level techniques: Dataset-level techniques make each dataset a split and order datasets by their instances’ average difficulty score.
- Instance-level techniques: Figure 1 shows that difficulty-score distributions vary substantially across datasets and within datasets, motivating potentially unequal distribution-based splits.
- Instance-level techniques: Instance-level techniques ignore dataset boundaries and arrange examples solely by difficulty using uniform or score-distribution-based splits.
4 Experiments
Experiments evaluate the curriculum methods across 12 NLP datasets in a low-resource, unified question-answering formulation against batching and dataset-order baselines.
- Datasets: The study covers 12 datasets spanning inference, paraphrase identification, commonsense, question answering, dialogue, numerical reasoning, robustness, and generalization.
- Setup: Experiments cap each dataset at 5,000 training instances and reformulate tasks as span-identification question answering to share one output space.
- Implementation Details: The implementation varies replay fraction across 0, 0.2, and 0.4, uses five meta-dataset splits and five epochs, and tests three- or five-way distribution splits.
- Baseline Methods: Instance-level methods use heterogeneous random batching as their baseline, while dataset-level methods average performance across multiple dataset-order baselines.
5 Results and Analysis
The proposed curriculum techniques improve multitask performance, with the largest gains concentrated on difficult instances. Results also favor Average Confidence scoring and distribution-based instance splitting.
- Performance Improvement: 4.17% and 3.15% average improvements are achieved by instance-level and dataset-level techniques over their respective baselines.The results are consistent across datasets and outperform single-task performance in most cases.
- Performance Improvement: Average Confidence outperforms Cross Review in both performance and computation because Cross Review trains multiple models.Cross Review requires one model for each meta-dataset.
- Uniform Vs Distribution based splitting: Distribution-based splitting slightly improves over uniform splitting by grouping instances with similar difficulty scores.The authors attribute this to a superior inductive bias.
- Effect of adding instances from previous splits: Adding instances from previous splits does not improve dataset-level techniques but improves instance-level techniques through same-dataset inductive bias.Dataset-level splits contain all instances from one dataset, whereas instance-level previous splits include same-dataset instances.
- Difficulty Scores Analysis: 32% of training instances receive difficulty scores ≤0.1 with Average Confidence, compared with 37% using Cross Review.Difficulty varies widely across and within datasets; Equate and QNLI are relatively easy, while PAWS and Winogrande are relatively difficult.
- Test Set Analysis: Performance improvement is especially strong for instances with high difficulty scores, indicating greater efficacy on tasks containing difficult instances.The analysis computes improvement over the baseline for each test-instance difficulty bucket.
6 Conclusion
The paper concludes that model-interpreted difficulty can organize multitask curricula at dataset and instance granularity. Across 12 datasets, the techniques improve performance, particularly on difficult instances, and can complement other multitask methods.
- 6 Conclusion: Model-interpreted curricula improve performance by 4.17% and 3.15% across experiments with 12 datasets.The two technique classes arrange learning curricula at instance-level and dataset-level granularity.
- 6 Conclusion: Difficult instances contribute most to the improvement, implying greater efficacy on difficult tasks.The techniques can be used with dynamic sampling and pre-finetuning to further improve performance.
- A Statistics of Evaluation Datasets: The evaluation uses datasets spanning several NLU tasks, with statistics reported for the evaluation sets.The paper states that 12 datasets are used.
B Implementation Details
The implementation uses BERT-based multitask training, converts heterogeneous datasets into a common question-answering format, and computes difficulty distributions with two model-based approaches.
- B Implementation Details: The experiments use a Hugging Face BERT-base model with batch size 16 and learning rate 5e−5.Training uses a maximum sequence length of 512 and hardware-adjusted batching.
- B Implementation Details: The implementation uses N=5 for Cross Review, E=5 for Average Confidence, and three splits for reported distribution-based results.The tested frac values are 0, 0.2, and 0.4, with hyperparameters selected on development performance.
- C Dataset Examples: Datasets are transformed into Question-Answering format so one model can handle tasks with different output spaces.Table 2 illustrates context-question pairs from the considered datasets.
- D Difficulty Scores: Difficulty-score distributions are computed with Cross Review and Average Confidence approaches.Figure 3 presents the instance distributions produced by both methods.
E Results
Table 3 reports results for the proposed instance-level and dataset-level curriculum techniques.
- Table 3 reports the results of instance-level and dataset-level techniques.
F Analysis
The analysis compares random and curriculum-based instance ordering across difficulty scores, while noting that broader scheduling methods remain future work.
- F Analysis: Table 5 compares random ordering with the proposed curriculum ordering across all difficulty scores for instance-level techniques.
- G Scheduling in Multi-task Learning: Scheduling methods developed for multi-task learning, particularly in machine translation, could be adapted to further improve multi-task performance.The paper leaves these explorations for future work.
H Limitations of Computing Difficulty Scores using Model-based Techniques
Model-based difficulty scoring adds training-time computation and may be vulnerable to datasets containing many noisy instances.
- Model-based difficulty scores have also supported selective prediction, training-dynamics analysis, and efficient evaluation.
- Difficulty scoring requires additional computation during training, but it adds no computational overhead during inference.
- Wrongly annotated or noisy instances receive high difficulty scores and are deferred, which may harm learning when they are numerous.The authors expect limited impact when noisy instances are few and leave substantial-noise settings for future work.