PaperlayerView exact revision on arXiv

Source-linked AI summary

Meta-Learning in Neural Networks: A Survey

Timothy Hospedales, Antreas Antoniou, Paul Micaelli, Amos Storkey

arXiv:2004.05439v2cs.LGstat.ML

TL;DR

Deep learning often depends on fixed algorithms, large datasets, and substantial computation, motivating methods that can learn how to learn. This survey formalizes contemporary neural-network meta-learning, proposes a taxonomy and reviews applications, finding broad use across settings while identifying generalization challenges. Its synthesis clarifies the field’s methods, applications, and open research problems.

  • Problem

    Conventional learning commonly relies on fixed algorithms and abundant data and computation, limiting settings with scarce data or unavailable computing resources.

  • Method

    The survey formalizes neural-network meta-learning, proposes a taxonomy of meta-representation, meta-objective, and meta-optimizer, and reviews applications and related fields.

  • Results

    The survey synthesizes meta-learning methods and applications spanning few-shot learning, reinforcement learning, unsupervised learning, hyperparameter optimization, and neural architecture search.

  • Takeaways & Limitations

    The taxonomy and application survey provide a framework for developing and customizing meta-learning methods across different purposes and help practitioners orient themselves in the field.

  • Takeaways & Limitations

    Meta-learners often generalize poorly when meta-test tasks come from distributions different from those used for training, including cross-domain settings.

Abstract

from arXiv · show

The field of meta-learning, or learning-to-learn, has seen a dramatic rise in interest in recent years. Contrary to conventional approaches to AI where tasks are solved from scratch using a fixed learning algorithm, meta-learning aims to improve the learning algorithm itself, given the experience of multiple learning episodes. This paradigm provides an opportunity to tackle many conventional challenges of deep learning, including data and computation bottlenecks, as well as generalization. This survey describes the contemporary meta-learning landscape. We first discuss definitions of meta-learning and position it with respect to related fields, such as transfer learning and hyperparameter optimization. We then propose a new taxonomy that provides a more comprehensive breakdown of the space of meta-learning methods today. We survey promising applications and successes of meta-learning such as few-shot learning and reinforcement learning. Finally, we discuss outstanding challenges and promising areas for future research.

1 INTRODUCTION

Meta-learning replaces fixed, hand-designed learning procedures with approaches that improve through experience across learning episodes or related tasks. This survey formalizes the area, proposes a taxonomy, situates it among neighboring fields, and reviews applications and scope.

  • Motivation: Conventional deep learning commonly trains models from scratch with fixed algorithms, requiring abundant data and substantial computing resources.These requirements limit applications where data is rare or expensive, or computing resources are unavailable.
  • Motivation: Meta-learning uses experience across multiple learning episodes, often from related tasks, to improve future learning performance.The approach is associated with potential data and compute efficiency.
  • Motivation: Neural-network meta-learning extends joint feature and model learning toward integrating algorithm learning.This frames meta-learning as a further step beyond hand-engineered features and deep learning’s joint representation learning.
  • Scope and applications: Meta-learning applies to both multi-task settings, where knowledge transfers across task families, and single-task settings, where repeated episodes improve one problem.Reported applications include few-shot recognition, reinforcement learning, unsupervised learning, hyperparameter optimization, and neural architecture search.
  • Scope and applications: The survey focuses on contemporary neural-network meta-learning as end-to-end learning of an explicitly defined objective, while also covering single-task learning and objectives such as robustness and compute efficiency.It distinguishes this focus from broader uses of algorithm learning.
  • Survey contribution: The paper presents an up-to-date survey that formalizes meta-learning, introduces a taxonomy based on meta-representation, meta-objective, and meta-optimizer, and reviews applications and related fields.It positions the field with respect to transfer learning and multi-task learning while discussing areas including few-shot learning, reinforcement learning, and architecture search.

2 BACKGROUND

Meta-learning improves a learning algorithm by optimizing how models are trained across tasks or repeated episodes, rather than fixing that algorithm in advance. The survey formalizes this process through task-distribution, bilevel, and feed-forward views, while recognizing both multi-task and single-task settings.

  • Definition: Meta-learning seeks to improve learning measures such as accuracy or data efficiency by learning the learning algorithm instead of keeping it fixed.This commonly involves learning across a distribution of tasks rather than training from scratch for each problem.
  • Task-Distribution View: In the task-distribution view, meta-knowledge ω is evaluated across tasks T = {D, L} and represents how to learn.Source tasks provide training and validation data, often called support and query sets, while target tasks are used during meta-testing.
  • Meta-Testing: Meta-testing applies learned meta-knowledge ω∗ to previously unseen target tasks, where it can specify initial parameters, a learning model, or an optimization strategy.Performance is evaluated on each target task’s test split after training the base model with ω∗.
  • Bilevel Optimization View: Bilevel meta-training places an outer optimization over ω around an inner task-learning optimization over θ.The outer objective evaluates models produced by the inner learner, with Lmeta and Ltask denoting the outer and inner objectives.
  • Single-Task Setting: Meta-learning does not require a task distribution: a single task can be split into training and validation data, with ω learned across multiple train-validation episodes.Different train-validation splits are usually used during meta-training in this single-task limiting case.
  • Feed-Forward Model View: Feed-forward approaches map an embedded training set to model weights, reducing new-task learning to a forward pass after optimization costs are paid during meta-training.These amortized methods vary in predictive-model complexity and support-set embedding choices, including pooling, CNNs, and RNNs.

3 TAXONOMY

The survey argues that conventional three-way taxonomies do not capture the full meta-learning landscape, and proposes three independent axes for organizing methods and applications.

  • 3.1 Previous Taxonomies: Earlier taxonomies divide methods into optimization-based, model-based, and metric-based categories.Optimization-based methods solve an inner task optimization; model-based methods embed the dataset in a feed-forward state; metric-based methods compare validation and training points.
  • 3.1 Previous Taxonomies: The conventional breakdown is insufficient for understanding connections across the wide variety of available meta-learning frameworks.
  • 3.2 Proposed Taxonomy: The proposed taxonomy organizes meta-learning along three independent axes: meta-representation, meta-optimizer, and meta-objective.These correspond respectively to what is learned, how the outer level is optimized, and why meta-learning is performed.
  • Meta-Representation (“What?”): Meta-representation specifies the meta-knowledge learned, ranging from initial model parameters to readable code for program induction.
  • Meta-Optimizer (“How?”): Meta-optimizer specifies the outer-level optimization procedure, including gradient descent, reinforcement learning, and evolutionary search.
  • Meta-Objective (“Why?”): Meta-objective specifies the goal, task distribution, and data flow, enabling customization for sample efficiency, speed, robustness, and other purposes.Examples include few-shot learning, many-shot optimization, domain-shift robustness, label-noise robustness, and adversarial robustness.

4 SURVEY: METHODOLOGIES

The survey organizes contemporary methodologies by what meta-knowledge is learned and how it is used, covering optimizers, representations, losses, architectures, modules, and hyperparameters.

  • Overview: Meta-learning methods differ in which aspects of the learning strategy are learned and which remain fixed.
  • Parameter Initialization: Parameter-initialization methods learn starting network parameters so a few gradient steps can solve new tasks while reducing few-shot overfitting.MAML is the most prominent example, but its outer optimization may involve as many parameters as the inner optimization.
  • Optimizer: Optimizer-centric methods learn an inner optimizer that maps optimization states such as parameters and task gradients to update steps.
  • Feed-Forward Models: Feed-forward models map a support set directly to classifier parameters instead of iteratively optimizing those parameters with gradients.Hypernetworks synthesize task-specific parameters from support-set embeddings in a feed-forward pass.
  • Feed-Forward Models: Amortized-inference formulations use a network to approximate Bayesian inference over task parameters and train it through validation loss across training tasks.The resulting predictive integral may be evaluated exactly, by sampling, or with a point estimate.
  • Other Methodologies: Other methodological choices learn embeddings, losses, auxiliary tasks, architectures, modules, hyperparameters, or data-augmentation strategies.These approaches support similarity-based recognition, differentiable task losses, architecture discovery, task-specific module recomposition, regularization, and validation-driven augmentation.
  • Architectures: Architecture-search methods automate expensive architecture design through evolutionary algorithms, reinforcement learning, or gradient-based representations such as DARTS.
  • Modules: Modular meta-learning recomposes task-agnostic modules in task-specific ways, connecting meta-learning with structural knowledge sharing and compositional learning.

5 APPLICATIONS

Meta-learning has broad applications, especially in computer vision and few-shot learning, while benchmark diversity and domain shift remain major challenges for evaluating generalization.

  • 5.1 Computer Vision and Graphics: Computer vision is a major application domain because few-shot learning addresses the long tail of visual concepts with limited labeled data.
  • 5.1.1 Few-Shot Learning Methods: Few-shot learning is difficult for large neural networks because small datasets can cause overfitting or non-convergence.Meta-learning approaches increasingly train powerful convolutional networks on small datasets across vision problems.
  • 5.1.1 Few-Shot Learning Methods: Few-shot classification methods have steadily improved over early approaches but remain far behind fully supervised methods.Open issues include cross-domain generalization, joint label spaces, and incremental addition of new classes.
  • 5.1 Computer Vision and Graphics: Meta-learning has also been applied to few-shot object detection, landmark prediction, object segmentation, image and video generation, and density estimation.Examples include hypernetwork-based detection and segmentation, pose estimation, novel-view synthesis, talking faces, video generation, and few-shot PixelCNN learning.
  • 5.1.2 Few-Shot Learning Benchmarks: Meta-learning benchmarks must evaluate generalization across tasks, requiring task families for meta-training and meta-testing rather than only instance-level splits.
  • 5.1.2 Few-Shot Learning Benchmarks: Standard few-shot benchmarks have narrow task distributions, making their performance non-reflective of real-world few-shot tasks.Broader evaluation should include diverse domains such as medical, satellite, agricultural, and underwater imagery.

5.2 Meta Reinforcement Learning and Robotics

Meta-reinforcement learning addresses sample inefficiency and difficult optimization by learning exploration, optimization, and other policy-related components across tasks or online within a single task. The survey covers benchmarks, robotics applications, and open challenges including compositional transfer and disentangling perception from control.

  • Methods: RL meta-learning targets sparse rewards, exploration costs, and high-variance optimization by learning components of the learning process.Applications include learning initial conditions, hyperparameters, step directions, step sizes, and policy-generating models.
  • Methods: Learning exploration policies or curiosity functions as metaknowledge improves sample efficiency by learning how to acquire data.This directly addresses action-dependent data distributions and sparse rewards.
  • Optimization: Episode-level return objectives reward both sample-efficient learning and asymptotic performance, motivating learned losses or rewards beyond sparse environmental rewards.Meta-RL commonly optimizes net return over a full episode.
  • Online meta-RL: Online meta-RL trains losses, rewards, hyperparameters, or exploration strategies alongside a base policy for a single task, without requiring task families.These methods directly improve the corresponding base learner during task learning.
  • Other Trends and Challenges: Open challenges include compositional knowledge transfer in robotics, unsupervised and continual adaptation, and separating visual generalization from fast control learning.CoinRun results indicate that standard vision techniques such as batch normalization can be a major factor in performance.
  • Benchmarks: Meta-RL benchmarks span task or domain families, but continuous-control evaluation lacks consensus and often uses narrow, home-brewed parametric tasks.Vision-control benchmarks also vary substantially in task diversity, affecting the difficulty of knowledge transfer.

5.3 Environment Learning and Sim2Real

Environment learning and Sim2Real treat simulation choices as meta-learnable parameters, using real-world performance to evaluate models trained in simulation. This reframes simulation-distribution tuning as an outer optimization problem.

  • Environment Learning and Sim2Real: Domain randomization trains robust models by simulating a distribution of domains or MDPs, but tuning that simulation distribution remains challenging.The approach has succeeded in both vision and reinforcement learning.
  • Environment Learning and Sim2Real: In Sim2Real meta-learning, inner optimization trains a simulation model while outer optimization evaluates real-world performance and updates environment parameters.The source-task distribution is parameterized by the meta-representation ω rather than supplied in advance.

5.4 Neural Architecture Search (NAS)

The survey presents neural architecture search as meta-learning over architecture specifications, while also covering Bayesian and unsupervised approaches that broaden the meta-learning design space. These methods address architecture selection, uncertainty, and ill-defined unsupervised objectives.

  • Neural Architecture Search (NAS): NAS treats architecture specification as a meta-representation: inner optimization trains the chosen network, while outer optimization searches for strong validation performance.Its design dimensions correspond to architecture search space, search strategy, and performance estimation strategy.
  • Bayesian Meta-learning: Bayesian meta-learning uses hierarchical modeling and approximate inference because exact Bayesian learning is typically intractable in this setting.Common approximations include stochastic variational inference and sampling.
  • Bayesian Meta-learning: Uncertainty over meta-learning parameters can provide prediction-uncertainty measures relevant to safety-critical applications, RL exploration, and active learning.The uncertainty concerns the meta-representation ω parameters.
  • Bayesian Meta-learning: Bayesian neural meta-learners include task-variable VAEs, Neural Processes, deep-kernel and Gaussian-process methods, and probabilistic variants of gradient-based meta-learning.The surveyed methods report competitive results and increasingly address ambiguous tasks, active learning, and RL.
  • Unsupervised Meta-Learning: Unsupervised learning interacts with meta-learning differently depending on whether it occurs in the inner or outer loop and during meta-training or meta-testing.The survey distinguishes several such combinations.
  • Unsupervised Meta-Learning: Synthetic unsupervised source tasks can meta-train supervised learners without requiring a large labeled source-task collection.Clustering and class-preserving augmentation define the synthetic tasks and meta-objective.
  • Unsupervised Meta-Learning: Meta-training an unsupervised learner against downstream supervised objectives converts an ill-defined unsupervised problem into one with a clear supervised meta-objective.Amortized clustering performs clustering in a single feed-forward pass instead of repeated iterative computation.

5.7 Continual, Online and Adaptive Learning

Continual, online, and adaptive meta-learning address task streams by balancing adaptation to new tasks with retention of prior knowledge. The survey highlights benchmark incompatibilities and domain-shift objectives as important constraints and extensions.

  • Continual Learning: Continual learning seeks forward transfer, retention without catastrophic forgetting, and learning without storing past data as tasks arrive sequentially.Deep neural networks particularly struggle with forgetting earlier information.
  • Online and Adaptive Learning: Online and adaptive learning prioritize effective adaptation to the current task stream rather than remembering all previous tasks.Approaches include online MAML and pre-trained meta-learners adapted across sequences of target tasks.
  • Benchmarks: Most continual-learning benchmarks are not readily compatible with meta-learning because they lack many explicit learning and evaluation sets.Early meta-learning-ready benchmarks mainly used Omniglot and perturbed MNIST.
  • Domain Shift: Domain shift occurs when deployment-data statistics differ from training-data statistics, motivating domain adaptation and generalization methods.These settings include supervised, unsupervised, and semi-supervised approaches.
  • Domain Generalization: Meta-learning can learn regularizers, losses, or noise augmentation to improve robustness to train-test domain shift.Domain adaptation can also use a meta-objective to optimize an unsupervised adaptation algorithm.

5.9 Hyper-parameter Optimization

Meta-learning applies to hyperparameter optimization by learning settings that improve performance across tasks or within a single task. The section also situates applications spanning language, medicine, abstract reasoning, and other domains.

  • Meta-learning can optimize hyperparameters such as regularization strength or learning rate across a task distribution or for one task.
  • MAML can be improved by learning a learning rate per layer and per optimization step.
  • Language Modelling: Few-shot language modeling has extended from one-shot missing-word completion to classification, program induction, synthesis, translation, and related tasks.
  • Meta-learning addresses data-scarce applications including medical image classification and one-shot molecular behavior prediction.
  • Meta-learning improved abstract reasoning on Raven’s Progressive Matrices by meta-learning a teacher that defines the panels’ data-generating distribution.

5.14 Systems

Meta-learning is applied across systems problems involving constrained devices, changing environments, costly annotation, noisy labels, adversarial robustness, and cold-start recommendation. These applications often formulate a learned policy, initialization, or generator as the object optimized for downstream performance.

  • Network Compression: Meta-learning supports network compression by training gradient-generator and weight-generator meta-networks for quantized networks.
  • Communications: Few-shot meta-learning enables rapid adaptation of communication codes to changing channel characteristics.
  • Active Learning: Active learning meta-optimizes an annotation query policy using validation performance after iterative learning and annotation.
  • Learning with Label Noise: Label-noise methods include transductively learning sample-wise weights to down-weight noisy samples and learning noise-robust initial conditions.
  • Recommendation Systems: Meta-learning has been applied to item and user cold-start recommendation using black-box and gradient-based methods, respectively.

6 CHALLENGES AND OPEN QUESTIONS

Open questions concern task diversity, meta-generalization, task-family availability, and the computational cost of bilevel optimization. Feed-forward meta-learners offer lower deployment cost than optimization-based alternatives, particularly on mobile devices.

  • Diverse and multi-modal task distributions: Diverse task distributions challenge existing meta-learners, partly because conflicting task gradients can hinder fitting a single strategy.
  • Diverse and multi-modal task distributions: Many frameworks assume a unimodal task distribution with one effective strategy, although different tasks may require different learning strategies.
  • Meta-generalization: Meta-generalization must address both novel tasks from the training distribution and tasks drawn from a different distribution.
  • Meta-generalization: Few-shot meta-learning can fail through memorization when meta-training tasks are solved without task-specific support-set adaptation.
  • Computation Cost & Many-shot: Bilevel optimization is expensive in time and memory because outer steps require inner steps and reverse-mode differentiation stores intermediate states.
  • Feed-forward meta-learners have a significant meta-test cost advantage over optimization-based methods, supporting deployment on mobile devices.

7 CONCLUSION

The survey clarifies meta-learning’s rapidly growing and sometimes confusing landscape through a methodological taxonomy and an application-focused review. It aims to help practitioners orient themselves and identify future research opportunities.

  • The survey organizes meta-learning methodologically through a taxonomy of meta-representation, meta-optimizer, and meta-objective.
  • It also surveys applications to clarify what meta-learning can be applied to and how the field relates to neighboring areas.
  • The authors intend the survey to help newcomers and practitioners develop and exploit meta-learning while highlighting future research opportunities.
Loading 2004.05439v2…