PaperlayerView exact revision on arXiv

Source-linked AI summary

Curriculum Learning: A Survey

Petru Soviany, Radu Tudor Ionescu, Paolo Rota, Nicu Sebe

arXiv:2101.10382v3cs.LGcs.CLcs.CV

TL;DR

Existing survey coverage includes a reinforcement-learning-focused treatment that is less general than this survey. The authors organize curriculum-learning methods with a multi-perspective taxonomy and an automatically built hierarchical tree, reporting that the tree corroborates the taxonomy.

  • Problem

    A related curriculum-learning survey focuses on reinforcement learning, leaving less general coverage than the broader treatment offered here.

    Source: §1 · p. 2

  • Method

    The authors manually construct a taxonomy across five perspectives and use agglomerative clustering to build a hierarchical tree of curriculum-learning methods.

    Source: §1 · p. 2, §5 · p. 32

  • Results

    The authors report that an automatically built hierarchical tree corroborates their manually constructed taxonomy of curriculum-learning methods.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The survey identifies unsupervised learning, especially self-supervised learning, as an underexplored area for curriculum-learning research.

    Source: §6.1 · p. 35

    Curriculum learning does not always bring significant performance improvements, and other factors also influence performance.

    Source: §6.1 · p. 34

Abstract

from arXiv · show

Training machine learning models in a meaningful order, from the easy samples to the hard ones, using curriculum learning can provide performance improvements over the standard training approach based on random data shuffling, without any additional computational costs. Curriculum learning strategies have been successfully employed in all areas of machine learning, in a wide range of tasks. However, the necessity of finding a way to rank the samples from easy to hard, as well as the right pacing function for introducing more difficult data can limit the usage of the curriculum approaches. In this survey, we show how these limits have been tackled in the literature, and we present different curriculum learning instantiations for various tasks in machine learning. We construct a multi-perspective taxonomy of curriculum learning approaches by hand, considering various classification criteria. We further build a hierarchical tree of curriculum learning methods using an agglomerative clustering algorithm, linking the discovered clusters with our taxonomy. At the end, we provide some interesting directions for future work.

1 Introduction

The survey formalizes curriculum learning (CL) in a generic framework and organizes its methods through a manually constructed taxonomy corroborated by hierarchical clustering. It reviews CL’s reported benefits and argues for broader adoption.

Context and motivation

  • Neural-network research has emphasized deeper architectures, while training examples are still typically selected in random order through mini-batch stochastic gradient descent.The introduction contrasts an 8-layer CNN with 15.4% ImageNet top-5 error and a 152-layer ResNet with 3.6% top-5 error.Source: §1 · pp. 1–2
  • Curriculum learning orders easy concepts before hard ones, and cited empirical work reports faster convergence and better accuracy than random minibatch sampling across several domains.Bengio et al. (2009) formalized easy-to-hard training as curriculum learning; opposing strategies such as Hard Example Mining and anti-curriculum have also improved results under certain conditions.Source: §1 · p. 2

Contributions

  • The survey unifies existing curriculum learning methods in a generic formulation, links curricula to data, model, task, and performance measure through loss smoothing, and builds a taxonomy corroborated by hierarchical clustering.The taxonomy groups methods by data type, task, curriculum strategy, ranking criterion, and curriculum schedule; the clustering tree largely confirms it while suggesting additional perspectives.Source: §1 · p. 2

Related surveys

  • Unlike the reinforcement-learning focus of Narvekar et al. (2020) and the related survey of Wang et al. (2021), this survey proposes a generic formulation and covers nearly 200 works.It also connects its automatically built hierarchical tree with its manual taxonomy, offering a distinct perspective on curriculum methods.Source: §1 · p. 2

2 Curriculum Learning

Curriculum learning gradually changes training conditions, most often from easy to hard, to make early optimization easier. It can operate on data, model capacity, tasks, or the performance measure, using criteria to set the curriculum and schedulers to control its pace.

2 Curriculum Learning

  • Curriculum over data, model capacity, tasks, or performance measures can smooth early objectives, though continuation methods are treated as an independent field.Using easier samples or a lower-capacity model can make the objective easier to optimize initially; increasing difficulty or capacity then makes it more complex. Data- and task-level approaches may require external difficulty measures, whereas capacity-based curricula do not.Source: §2 · pp. 3–4
  • Data-level curricula rank and select examples by difficulty, while model-level curricula change model capacity without requiring a difficulty criterion.Both frameworks use a curriculum scheduler and performance measure; the scheduler may use a linear or logarithmic pace, or account for current performance in self-paced learning.Source: §2 · p. 3, Figure 1 · p. 4
  • The generic algorithm uses a curriculum criterion C at a chosen level l to modify data, model, or performance measure, with a scheduler S controlling when updates occur.The scheduler may base its timing on the training iteration or current performance, and curricula can be applied at multiple levels jointly.Source: §2 · pp. 4–5
  • Difficulty criteria can be task-specific or generated through teacher-student guidance or self-paced learning, and curricula can order multiple tasks or adapt model elements.Examples of task-specific criteria include image shape complexity, text grammar properties, and audio signal-to-noise ratio. Data-level selection methods include batching, weighting, and sampling.Source: §2 · pp. 4–5

3 Taxonomy of Curriculum Learning Methods

The survey organizes curriculum learning into seven overlapping categories and considers additional perspectives, including application domain, task, curriculum-building criteria, scheduling, and application level.

Vanilla CL and SPL

  • Vanilla CL orders samples using predefined easy-to-hard rules, whereas SPL repeatedly evaluates difficulty from model performance, so the sample order can change during training.Examples of difficulty measures include geometric complexity or sequence length for Vanilla CL, and prediction likelihood or objectness for SPL.Source: §3 · pp. 5–6

BCL and SPCL

  • BCL combines easy-to-hard ordering with balance or diversity constraints, while SPCL combines predefined criteria with learning-based metrics to set sample order.BCL constraints can promote diversity across image regions or classes; SPCL explicitly joins CL and SPL.Source: §3 · p. 6

Progressive CL

  • PCL progressively changes model capacity or task settings rather than ordering individual samples, so training moves from an easier setup toward a harder one.Examples include decreasing dropout probability during training and progressively growing GAN capacity.Source: §3 · p. 6

Teacher-student CL

  • Teacher-student CL uses an auxiliary teacher model to determine learning parameters or a policy for the student, which performs the principal task and final inference.The curriculum is implemented through a network that applies a policy to the student model.Source: §3 · p. 6

Implicit CL

  • Implicit CL produces an easy-to-hard effect as a side effect of training methodology rather than through an explicitly constructed curriculum.Examples include gradually deblurring convolutional activation maps and ranking raw examples by informativeness through unsupervised training.Source: §3 · p. 6

Category overlap

  • The seven categories are overlapping pools rather than disjoint classes, with methods combining BCL, SPL, teacher-student frameworks, and complexity-based approaches.SPCL is one prominent CL–SPL combination; BCL can also combine with SPL or teacher-student approaches.Source: §3 · p. 6

Related methodologies and curriculum criteria

  • Curricula can prioritize difficulty, uncertainty, or hard examples, and difficulty may be manually labeled or estimated from domain measures, other networks, or dedicated models.Hard-example mining emphasizes high-loss examples, anti-curriculum orders examples from hard to easy, and active learning selects by uncertainty; teacher-student and SPL approaches can use learning progress, while reinforcement-learning task order can maximize reward.Source: §3 · pp. 6–7

Scheduling and application levels

  • Curriculum methods use batching, easy-then-hard groups, constrained sampling, weighting, hard-example removal, task stages, or continuous and iterative schedules, and may operate at data, task, or model level.Batching gradually adds subsets, whereas easy-then-hard uses each subgroup distinctly without augmenting the training set.Source: §3 · p. 7

4 Applications of Curriculum Learning

The section surveys curriculum-learning applications across machine-learning domains and tasks, including computer vision, NLP, speech, medical imaging, and reinforcement learning. It describes how approaches assess difficulty and organize examples, tasks, or training mechanisms through sequencing, selection, weighting, and scheduling.

4 Applications of Curriculum Learning

  • The survey groups application papers by domain and task to help readers locate relevant work and trace curriculum methods within each area.Table 1 summarizes distinctive elements of the selected strategies.Source: §4 · p. 7

4.1 Multi-domain approaches

  • Multi-domain applications span speech recognition, computer vision, natural-language processing, medical tasks, and reinforcement-learning games.The listed examples also include robotics, graph traversal, and image generation.Source: §4.1 · p. 9
  • Curriculum difficulty signals and training mechanisms vary by task, including utterance length, learning progress, graph size, sentence features, and batching or data weighting.Other listed signals include image severity, model confidence, and task difficulty; strategies include sampling and iterative data introduction.Source: §4.1 · pp. 9, 11
  • Bengio et al. (2009) introduced curriculum learning, training from easy to difficult examples; their experiments reported faster convergence, a better local minimum, and advantages over random or hard-to-easy sampling in some settings.Their shape-recognition curriculum also outperformed standard training on difficult data, while the paper notes that ranking examples can be complex in practice.Source: §4.1 · p. 13
  • Kumar et al. (2010) introduced self-paced learning, which selects examples using model learning progress rather than a predefined external curriculum, gradually admitting harder samples until all are considered.Easiness reflects how readily the model predicts the correct output, and a gradually increased weight controls the transition.Source: §4.1 · pp. 13–14
  • Jiang et al. (2015) combined prior curriculum knowledge with learning progress in self-paced curriculum learning, while Wu et al. (2018) trained a teacher network to generate objectives and improve its teaching.SPCL updates a predefined curriculum in an SPL manner; the teacher network adjusts objectives to student progress and optimizes its own parameters during teaching.Source: §4.1 · p. 14

4.2 Computer vision

  • Computer-vision approaches estimate difficulty from human visual-search response times, object statistics, jointly trained hardness predictors, or learnable data parameters, then use the estimates to guide sampling or exclude hard examples.The human-response-time estimator was used for weakly supervised object localization and semi-supervised object classification; learnable parameters prioritized clean examples under noisy conditions.Source: §4.2 · p. 15
  • Image-classification curricula also adapt sample selection through dictionary-based difficulty, class-local loss weights, and changing pseudo-label thresholds or percentile cutoffs.The pseudo-label methods can balance class selection or let previously selected samples enter or leave training, while dictionary learning increases the number chosen over iterations.Source: §4.2 · pp. 15–16
  • Curriculum dropout outperformed standard dropout in extensive image-classification experiments by scheduling neuron-retention probability to make optimization progressively harder.The authors describe fixed dropout probability as suboptimal.Source: §4.2 · p. 16
  • For noisy web images and cross-domain adaptation, curricula use density-based clustering, transferability estimates, or discriminator loss to prioritize training examples.Choi et al. (2019) use high-density examples early to progressively improve pseudo-labels; Shu et al. (2019) estimate source-sample transferability with a domain discriminator.Source: §4.2 · pp. 16–17
  • In image classification, curriculum learning can accelerate early convergence, while teacher-based approaches score examples or adapt sample weights using student feedback.Weinshall and Cohen (2018) report faster convergence in early training; Gong et al. (2016) use teacher consensus, while Jiang et al. (2018) use a data-driven MentorNet.Source: §4.2 · pp. 17–18
  • Other curricula order tasks, unlabeled examples, or architecture-search spaces, including sequential multi-task learning, open-set semi-supervised learning, and staged architecture search.The cited approaches select task order for expected classification performance, alternate network and out-of-distribution-score updates, or expand search spaces using knowledge from smaller ones.Source: §4.2 · p. 17
  • Some methods depart from easy-to-hard selection by emphasizing historically hard or diverse examples, while others combine loss-based curricula with boosting or meta-learned weights.Zhou and Bilmes (2018) select difficult examples with diversity, and Ren et al. (2018a) use a clean validation set to guide weights.Source: §4.2 · p. 18

Object detection and localization

  • Object-detection curricula rank images or regions using size, detector-segmenter consistency, annotation-based scores, or pseudo-label confidence.For example, Li et al. (2017c) use IoU thresholds and Wang et al. (2018) classify images by mean average precision per image.Source: §4.2 · p. 19

Object segmentation

  • Semantic-segmentation curricula adapt across simpler tasks or environmental conditions, and can progressively admit lower-confidence pseudo-labels during self-training.Examples include transfers from daytime to nighttime or fogless to dense-fog domains, and increasing the number of activated pseudo-labels across rounds.Source: §4.2 · p. 20

Face recognition

  • Buyuktas et al. (2020) report that head-pose-based curriculum batching significantly improves face recognition over the random baseline.Upright frontal faces are treated as easiest, with pose estimated from yaw, pitch, and roll.Source: §4.2 · p. 20

Image generation and translation; Video processing

  • For image generation and video processing, curricula vary data difficulty, model capacity, discriminator feedback, or the samples used to retrain detectors and trackers.GAN approaches include progressively increasing image resolution or weighting discriminator feedback; video methods add easier video samples over time while removing difficult source-image samples.Source: §4.2 · p. 21

4.2 Computer vision

  • Jiang et al. (2014b) find that self-paced learning with diverse sampling outperforms standard SPL for event detection and action recognition, with experiments showing the importance of both components.A negative l2,1-norm term favors selecting samples from multiple groups rather than overfitting to easy examples from one group.Source: §4.2 · pp. 21–22
  • Wang et al. (2019b) report superiority over conventional training on two attribute-analysis datasets using dynamic sampling and loss schedulers for imbalanced data.The sampling scheduler moves from imbalanced to balanced and easy to hard; the loss scheduler adjusts classification- and metric-learning weights.Source: §4.2 · p. 22

4.3 Natural language processing

  • Zhang et al. (2018) report improved convergence time without loss of translation quality, and emphasize choosing an appropriate difficulty criterion and curriculum schedule.Their German-English translation experiments assessed difficulty using an auxiliary translation model or linguistic information and weighted data shards according to difficulty and training progress.Source: §4.3 · p. 23
  • Zhan et al. (2021) find that adding curriculum learning over meta-learning improves cross-domain translation performance on both previously seen and unseen domains.Their curriculum progresses from common domain elements to more specific ones, with commonalities and individualities computed using pretrained neural language models.Source: §4.3 · pp. 23–24
  • Subramanian et al. (2017) find curriculum ordering essential for generating long sequences with an LSTM; Zaremba and Sutskever (2014) also mix difficulty levels to expose models to harder code examples early.The code-sequence curriculum mixes random difficulty levels with an easy-to-hard progression over sequence length and nesting.Source: §4.3 · p. 25
  • Chang et al. (2021) find that joint data-and-text difficulty measures outperform measures of either alone, while curriculum learning improves output quality and convergence speed.Examples are selected according to the model’s competence at the current training step.Source: §4.3 · p. 26
  • Gong et al. (2021) report that dynamic curriculum learning for intent detection improves on both traditional training and other curriculum strategies.Their scheduler reduces easy examples and increases complex ones as training progresses.Source: §4.3 · p. 26
  • Zhao et al. (2021) report that all three teacher-student curriculum schedules improve dialogue policy learning over standard random sampling.The framework accounts for difficulty and diversity, using agent learning progress to measure difficulty and penalizing overrepetition.Source: §4.3 · p. 26
  • Jafarpour et al. (2021) combine active learning and curriculum learning for named entity recognition, selecting sentences using linguistic difficulty and informativeness scores.Difficulty metrics include seven novel measures; informativeness is measured with min-margin or max-entropy.Source: §4.3 · p. 26

4.4 Speech processing

  • Speech curricula span recognition, speaker identification and verification, emotion recognition, and spoken-language understanding, using ranked samples, staged data, or transferred knowledge.For speech recognition, the appropriate schedule can depend on the data distribution; one method uses utterance length for a one-epoch curriculum warm-up before random training.Source: §4.4 · pp. 26–27
  • Other speech approaches use teacher-student curricula, self-paced pseudo-labeling, hard-to-easy SNR ordering, or increasingly difficult tasks instead of ranking examples.The anti-curriculum based on low-to-high SNR performed better than the tested high-to-low alternative.Source: §4.4 · p. 27

4.5 Medical imaging

  • Medical imaging curricula use handcrafted or annotator-based difficulty and staged training for tasks including cancer detection, fracture classification, histopathology, and glaucoma diagnosis.Examples include progressing from lesion masks to whole mammograms, using expert judgments to set fracture-class difficulty, and training from normal to abnormal cases.Source: §4.5 · pp. 27–28
  • Burduja and Ionescu (2021) report that curriculum by input blur matches curriculum by smoothing in performance gains while significantly reducing computational complexity.The approach gradually deblurs images during unsupervised training.Source: §4.5 · pp. 28–29

4.6 Reinforcement learning

  • Reinforcement-learning curricula often operate at task level and use teacher-student frameworks; approaches also schedule distance-based starts, goals, task complexity, precision, or intermediate goals.Some methods adapt goal selection using proximity and curiosity, progressively expand object-manipulation tasks, or continuously tighten reach-accuracy requirements.Source: §4.6 · pp. 29–30
  • Eppe et al. (2019) find medium-difficulty goals work best for deep deterministic policy gradient, while sampling difficult goals more often works best with hindsight experience replay.Goal masking estimates difficulty from the learner’s past success on goals with the same mask.Source: §4.6 · p. 29
  • Qu et al. (2018) find that edge-type learning order affects heterogeneous-network node representations, while Narvekar and Stone (2019) model transfer curricula as MDPs and learn policies to solve target tasks faster.Qu et al. define reward by performance on external tasks; Narvekar and Stone represent states with potential functions accounting for sampled source tasks.Source: §4.6 · p. 31
  • Turchetta et al. (2020) target safety during learning with reset controllers selected according to student progress to optimize the agent’s final reward.The controllers activate when the agent behaves dangerously.Source: §4.6 · p. 31
  • Portelas et al. (2020b) reuse curricula from previous students to adapt teaching to new students, while Feng et al. (2020a) select task instances at the limit of solvability.Portelas et al. combine inferred progress niches with learning progress and become independent of the expert teacher once the trajectory is completed.Source: §4.6 · p. 31

4.7 Other domains

  • Zhao et al. (2015) use loss-based soft difficulty weights for matrix factorization, while Graves et al. (2016) order graph-traversal lessons by task complexity.The graph curriculum uses the number of nodes as a difficulty metric and presents lessons in a linear ascending sequence.Source: §4.7 · pp. 31–32
  • Ma et al. (2018) prove that under light conditions, the self-paced learning process converges to critical points of its implicit objective, supporting the theoretical soundness of SPL robustness analysis.Source: §4.7 · p. 32
  • Zheng et al. (2020) use easy-to-hard soft weighting to address outliers in unsupervised feature selection, while Sun and Zhou (2020) iteratively select samples and tasks from simple to difficult in multi-task learning.Sun and Zhou claim their method addresses scalability issues in limited-data scenarios.Source: §4.7 · p. 32
  • Zhang et al. (2020a) learn online multi-task curricula with a multi-armed bandit, selecting tasks by average or highest loss and reporting good zero-shot and few-shot results.Their worst-case-aware losses induce automatic curriculum learning across tasks.Source: §4.7 · p. 32

5 Hierarchical Clustering of Curriculum Learning Methods

Agglomerative clustering of curriculum-learning papers produced homogeneous clusters that broadly align with the survey’s hand-built taxonomy. The dendrogram suggests an initial division by learning paradigm, followed by divisions among supervised methods by task or application domain.

5 Hierarchical Clustering of Curriculum Learning Methods

  • The authors clustered TF-IDF vectors of article abstracts using agglomerative clustering with Ward’s linkage, chosen to perform well amid cluster noise.They removed stop words and used TF-IDF to reduce the influence of words expected to be common across the papers, such as “curriculum” and “learning.”Source: §5 · p. 32
  • The largest homogeneous cluster mainly contains self-paced methods and joins the other methods only at the dendrogram’s final stage, while the second largest contains reinforcement-learning methods.The authors relate self-paced learning’s late joining point to its development as an independent field not necessarily tied to curriculum learning.Source: §5 · p. 32, Figure 2 · p. 33
  • Reinforcement-learning curricula form a distinct cluster and typically use teacher-student models or operate over tasks, whereas curricula in other domains commonly operate over data samples.The third-largest homogeneous cluster mainly comprises domain-adaptation methods, which typically adjust a model gradually from source to target domain; a further homogeneous cluster contains speech-processing methods.Source: Figure 2 · p. 33, §5 · p. 34
  • Two heterogeneous clusters group text processing with image classification, and object detection with semantic segmentation and medical imaging, but neither yielded representative homogeneous subclusters.The authors describe the latter cluster’s tasks as connected at the broader level of computer vision.Source: Figure 2 · p. 33, §5 · p. 34
  • The dendrogram suggests first grouping curriculum-learning works by paradigm—supervised, reinforcement, or self-paced learning—and then supervised works by task or application domain, consistent with the manual taxonomy.The suggested supervised task or domain groups include image classification and text processing, speech processing, object detection and segmentation, and domain adaptation.Source: Figure 2 · p. 33, §5 · p. 34

6 Closing Remarks and Future Directions

The survey concludes that curriculum learning can help across many domains but may also impair diversity or optimization, and identifies underexplored learning paradigms and domain-specific opportunities. It recommends investigating curriculum design at the model and performance levels, in unsupervised settings, and alongside alternative optimization approaches.

6.1 Generic directions

  • Curriculum learning has improved tasks across domains, but it can also reduce sample diversity and lead to suboptimal training or failure cases.A difficulty measure that favors easy examples from a small subset of classes can reduce diversity early in training and steer the model toward a suboptimal solution.Source: §6.1 · p. 34
  • Most approaches schedule experience E or task class T, while model- and performance-level curricula remain scarce and warrant broader study.A robust difficulty measure that incorporates diversity appears to bring higher improvements than one that overlooks it; model-capacity curricula also need more efficient approaches and wider task coverage.Source: §6.1 · pp. 34–35
  • Curriculum learning is underexplored in unsupervised and self-supervised learning, where sample or task ordering may benefit optimization.An initially less diverse set of easy samples might help unsupervised optimization, while the order of self-supervised tasks may affect final performance.Source: §6.1 · p. 35
  • Curriculum learning’s interaction with SGD can constrain optimization, motivating automatic regulation and exploration of alternatives such as evolutionary search.Restricting early training to easy samples may trap SGD in a local minimum that is hard to escape, making curriculum training harder to control and potentially causing convergence failures or inferior results.Source: §6.1 · p. 35

6.2 Domain-specific directions

  • Vision transformers offer curriculum opportunities in both self-supervised pre-training and supervised fine-tuning, although curriculum has not yet been applied to them.Suggested directions include ordering pre-training tasks by complexity, using diversity-aware data difficulty in fine-tuning, and gradually unsmoothing tokens.Source: §6.2 · pp. 35–36
  • Medical imaging and signal processing invite domain-specific curricula tailored to image ambiguity, noise, or source count.For medical images, curricula could begin with fully healthy tissue or predominantly lesions before weakly labeled images containing both; signal-processing samples could be ordered by noise level or number of sources.Source: §6.2 · p. 36
  • For language transformers, a vocabulary-growth curriculum may better reflect human language learning than text-length heuristics.Future work should determine which words belong in the initial vocabulary and when to expand it.Source: §6.2 · p. 36

Related papers

Loading 2101.10382v3…