A Survey of Deep Meta-Learning

Mike Huisman; Jan N. van Rijn; Aske Plaat

Source-linked AI summary

A Survey of Deep Meta-Learning

Mike Huisman, Jan N. van Rijn, Aske Plaat

arXiv:2010.03522v2Oct 7, 2020cs.LGcs.AIstat.ML

TL;DR

Deep neural networks require large amounts of data and training, limiting their ability to learn new concepts quickly, while Deep Meta-Learning lacks a coherent, detailed overview. This survey addresses these gaps by unifying and explaining contemporary techniques, relating their strengths and weaknesses, and identifying current challenges and future directions.

Problem
Deep neural networks need large amounts of data and training for good performance, and Deep Meta-Learning lacks a coherent, detailed overview of its key techniques.
Method
The survey provides detailed explanations using unified notation, covers contemporary supervised- and reinforcement-learning techniques, and relates methods through their strengths, weaknesses, and key ideas.
Results
The survey organizes and compares Deep Meta-Learning approaches, including their applicability, key ideas, benchmarks, backbones, and classification mechanisms.
Takeaways & Limitations
The survey identifies network complexity as highly related to few-shot classification performance and highlights current challenges and promising future research directions.
Takeaways & Limitations
Online MAML's computational costs grow over time because all encountered data are stored.

Abstract

from arXiv · show

Deep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Meta-learning is one approach to address this issue, by enabling the network to learn how to learn. The field of Deep Meta-Learning advances at great speed, but lacks a unified, in-depth overview of current techniques. With this work, we aim to bridge this gap. After providing the reader with a theoretical foundation, we investigate and summarize key methods, which are categorized into i)~metric-, ii)~model-, and iii)~optimization-based techniques. In addition, we identify the main open challenges, such as performance evaluations on heterogeneous benchmarks, and reduction of the computational costs of meta-learning.

1 Introduction

Deep Meta-Learning addresses deep networks’ limited ability to learn new concepts quickly and provides a unified, detailed overview of the field’s contemporary techniques.

Large data and computing requirements constrain deep neural networks’ ability to learn new concepts quickly.
Deep Meta-Learning focuses on procedures that learn inductive biases for deep neural networks.
The survey fills a lack of coherent, detailed coverage by explaining contemporary techniques with unified notation.
The survey covers supervised and reinforcement learning methods, relates techniques, discusses strengths and weaknesses, and identifies challenges and future directions.
Larger feature-extraction backbones tend to achieve better 1-shot miniImageNet classification performance despite the few-shot setting.
Its taxonomy groups approaches into metric-based, model-based, and optimization-based techniques.

2 Foundation

The foundation contrasts regular learning with meta-learning, formalizes supervised and reinforcement-learning objectives, and describes task-based evaluation and related fields.

Regular learning and meta-learning: Regular learning optimizes model parameters for one data set, whereas meta-learning learns meta-knowledge across multiple tasks for faster adaptation.
Supervised meta-learning: Supervised meta-learning seeks meta-knowledge that enables a base learner to learn many different tasks well.
Reinforcement learning: Reinforcement learning trains an agent to maximize expected reward through policy-driven interactions with an environment.
Meta-reinforcement learning: Meta-reinforcement learning extends this objective across a distribution of MDPs by learning meta-knowledge.
Related fields: Unlike multi-task learning on fixed tasks, meta-learning aims to learn new previously unseen tasks quickly.
Training and evaluation: Meta-learning uses meta-train, meta-validation, and meta-test stages to train, tune, and estimate performance on unseen tasks.

3 Metric-based Meta-Learning

Metric-based meta-learning learns feature spaces or similarity functions across tasks, then predicts new inputs by comparing them with labeled support examples. These methods can be computationally efficient and effective, but their applicability and robustness are constrained by task size, task distance, and supervision.

Core idea: Metric-based techniques learn a feature space or similarity kernel across tasks, using similarity to labeled examples to predict query labels.The learned feature space is represented by network weights, while the kernel outputs similarity scores between inputs.
Core idea: In metric-based learning, new-task information is not incorporated into network weights; predictions rely on comparisons in the meta-learned feature space.This makes the inner-level adaptation non-parametric, while the embedding is learned across a distribution of tasks.
Prediction: A query input is classified by combining support-set labels according to input similarities, with the highest predicted class probability determining the output.The example uses one-hot labels and selects the class with the largest entry in pθ(Y |x, S).
Key techniques: Prototypical networks reduce pair-wise comparisons through class prototypes and outperform matching networks in 5-way, k-shot learning for k = 1, 5 on Omniglot and miniImageNet.The passage reports this result despite prototypical networks not using complex task-specific embedding functions.
Key techniques: Graph neural networks propagate labels from labeled examples to unlabeled inputs, achieve good few-shot performance, and also apply to semi-supervised and active learning.Relation networks instead use trainable similarity functions, yielding better performance than techniques with fixed similarity metrics.
Limitations: Metric-based methods are fast on small test tasks but may degrade on distant tasks, become prohibitively expensive for larger tasks, and usually require labeled examples.Attentive recurrent comparators additionally incur higher computational cost without often outperforming graph neural networks.

4 Model-based Meta-Learning

Model-based meta-learning maintains an adaptive internal representation of each task, typically updated while processing support examples sequentially. This flexibility supports broad task applicability and fast adaptation, but performance can vary with task complexity, dataset size, and task distance.

Core idea: Model-based techniques process support sets sequentially to update a stateful internal representation that captures task-specific information for later predictions.Their hidden internal dynamics and memory component motivate the description of these models as black boxes.
Core idea: Unlike metric-based methods, model-based techniques can learn internal dynamics for processing and predicting task data, not only feature spaces.The designer has greater freedom to choose the algorithm’s internal dynamics.
Memory-based methods: A memory-based example stores each support example in a row of an external memory matrix, which then serves as the internal task representation.For a new input, dot products with stored examples produce scores used to select a class.
Recurrent meta-learners: Recurrent meta-learners use LSTMs or GRUs to store dynamic task embeddings and adapt quickly to new tasks, especially in reinforcement learning.At each time step, the recurrent network receives environment variables, outputs an action, and updates its hidden state.
Recurrent meta-learners: Recurrent meta-learners perform almost as well as asymptotically optimal algorithms on simple reinforcement learning tasks but degrade when temporal dependencies span longer horizons.Improving performance on such complex tasks is identified as future research.
Memory-augmented methods: Memory-augmented neural networks combine a controller with external memory for quick task-specific adaptation and apply to both classification and regression.The controller acts as long-term memory while the external module provides short-term memory.
Other methods and limitations: SNAIL applies to supervised and reinforcement learning and performs well relative to earlier techniques, but selecting its TCBlock and DenseBlock architecture can be time-consuming.Model-based approaches overall offer flexible internal dynamics and broader applicability, yet can be outperformed in supervised settings and generalize less well to distant tasks than optimization-based techniques.

5 Optimization-based Meta-Learning

Optimization-based meta-learning explicitly targets fast task adaptation through a bi-level process: a base-learner updates within tasks while outer optimization improves performance across tasks.

Optimization-based techniques: Optimization-based techniques formulate meta-learning as bi-level optimization, combining task-specific inner updates with outer optimization across tasks.The inner procedure may use gradient descent or another optimization strategy.
LSTM optimizers: LSTM optimizers replace hand-crafted gradient updates with learned updates tailored to a task family.The LSTM learns an update policy from observed losses and optimization histories.
LSTM optimizers: LSTM optimizers can learn faster than hand-crafted optimizers, but their few-shot performance and scalability to larger base-learners remain unclear.The cited work did not apply the technique to few-shot learning or larger architectures.
LSTM meta-learners: LSTM meta-learners store base-learner parameters in the LSTM cell state, making cell-state updates equivalent to parameter updates.When the forget gate and learning-rate conditions match specific values, the update becomes equivalent to gradient descent.
LSTM meta-learners: Weight-sharing limits the number of meta-learner parameters, while second-order derivatives are ignored to reduce computational expense.The shared update rule still depends on each parameter’s history.
LSTM optimizers: Despite their mechanism for quicker optimization, LSTM optimizers can be outperformed by simpler approaches such as MAML.This comparison is reported in the survey’s summary of the technique.

5.4 Reinforcement Learning Optimizer

This section presents reinforcement-learning optimizers and MAML as optimization-based approaches, contrasting learned general optimizers with gradient-based adaptation from a learned initialization.

Reinforcement Learning Optimizer: The reinforcement-learning optimizer generates a sequence of task-specific weights from an initial parameter set and task loss.The final weight sequence element is the solution found by the optimizer.
Reinforcement Learning Optimizer: Reinforcement-learning optimizers treat optimization as a POMDP in which weights form the state, updates are actions, and the optimizer is a learned policy.Guided policy search trains the policy from features based on weights, gradients, and objective functions.
Reinforcement Learning Optimizer: These optimizers were shown to generalize across network architectures and data sets, but the base-learner was quite small.Whether the approach scales to larger architectures remains open.
MAML: MAML learns an initialization that adapts quickly to new tasks through a simple gradient-based inner optimization procedure.At test time, the model starts from the learned initialization and performs gradient updates on task data.
MAML: FOMAML performed reasonably similarly to MAML while using only first-order gradients.MAML’s higher-order derivatives are expensive in time and memory, especially for long optimization trajectories.
MAML: MAML updates its initialization using losses measured after task-specific inner updates across sampled tasks.The query-set losses are aggregated in an outer update.
MAML: MAML is simple, broadly applicable, and strong-performing, but optimizing each task and computing higher-order derivatives can be expensive.Several technical improvements target training stability, speed, flexibility, and generalization.
iMAML: iMAML reduces memory costs by approximating meta-gradients without differentiating through the optimization trajectory.It also permits non-differentiable inner optimizers and more inner gradient steps, while computational costs remain roughly comparable to MAML.

5.7 Meta-SGD

Meta-SGD extends MAML by learning both an initialization and parameter-specific learning rates, making the optimizer more expressive but increasing its hyperparameters.

Meta-SGD: Meta-SGD learns an initialization together with a learning rate for every model parameter.Its learning-rate vector provides greater flexibility than scalar-rate SGD.
Meta-SGD: The Meta-SGD objective uses support-set updates and query-set losses to maximize generalization after one update step.Because the objective is fully differentiable, Meta-SGD can be trained with standard SGD.
Meta-SGD: Meta-SGD is more expressive than MAML but requires an increased number of hyperparameters.The additional learned learning rates account for this trade-off.

5.8 Reptile

Reptile repeatedly trains on sampled tasks and moves the initialization toward the resulting task-specific weights, avoiding differentiation through the optimization trajectory.

Reptile: Reptile seeks a shared initialization by repeatedly sampling a task, training on it, and moving weights toward the trained parameters.Unlike MAML, it does not directly optimize fast learning through trajectory differentiation.
Reptile: Reptile can interpret the displacement between initial and trained weights as a gradient-like update and average directions across task batches.The initialization is moved toward the average update direction.
Reptile: Reptile’s initialization oscillates between optimal weights for different tasks when task updates are interleaved.This trajectory reflects movement toward task-specific optima.
Reptile: Reptile saves time and memory by avoiding differentiation through the optimization trajectory, though its theoretical foundation is weaker than MAML’s.Its performance may be somewhat worse than MAML’s in some settings.

5.9 Latent embedding optimization (LEO)

LEO addresses poor generalization in few-shot gradient-based meta-learning by optimizing a task-conditioned, lower-dimensional latent embedding that indirectly produces initial parameters. It improves generalization but is more complex and limited in practice to generating final-layer weights.

LEO targets gradient-based meta-learners that use few examples to optimize in a high-dimensional parameter space, potentially causing poor generalization.
LEO learns a lower-dimensional latent embedding space conditioned on tasks, indirectly producing a suitable set of initial parameters.
The workflow encodes support-set examples, forms all pairwise code combinations, processes them with a relation network, and parameterizes latent-code distributions by class.
The decoder generates task-specific initializations from latent codes, and losses from the generated weights are propagated backward to update the embedding space.
LEO optimizes in a lower-dimensional space, aiding generalization, but is more complex than MAML and experimentally generates only final-layer weights, limiting expressivity.

5.10 Online MAML (FTML)

Online MAML extends MAML to sequential online learning, using task-buffered meta-updates to perform well across presented loss functions. Its main limitation is growing computational and memory cost because encountered task data remain stored.

Online MAML extends MAML to online settings in which tasks arrive sequentially with corresponding loss functions.
The online objective is to choose parameter sequences that perform well over the task sequence, measured by cumulative regret against the best fixed parameters in hindsight.
FTML minimizes regret by setting the next parameters toward the best parameters in hindsight, using gradients from tasks sampled from the buffer.
The algorithm buffers tasks, collects data until a batch threshold, performs MetaUpdate, evaluates test loss, and records task efficiency and final performance.
Online MAML is described as robust for online learning, but computational costs grow over time because all encountered data are stored.

5.11 LLAMA

LLAMA extends MAML probabilistically by learning distributions over task-specific parameters, enabling multiple potential solutions for each task. This flexibility increases computational cost, and its Laplace approximation can be inaccurate.

LLAMA places MAML in a probabilistic framework so that a task has a distribution over task-specific parameters rather than one solution.
The method learns multiple potential task solutions by estimating a task-specific Gaussian distribution and sampling parameters from it.
LLAMA adapts parameters using support data and updates the initialization by minimizing negative log-likelihood on query data.
Its ML-LAPLACE procedure approximates the quadratic curvature matrix using K-FAC while estimating the loss used to update the initialization.
LLAMA increases computational costs, its Laplace approximation can be inaccurate, and the approach is developed only for supervised learning.

5.12 PLATIPUS

PLATIPUS extends probabilistic meta-learning by learning distributions over initializations, allowing multiple task-adapted starting points and uncertainty-aware predictions. Its increased computational cost and lack of reinforcement-learning applicability constrain its scope.

PLATIPUS learns a probability distribution over initializations rather than directly learning a distribution over task-specific parameters.
PLATIPUS learns distributions over priors from query and support data and uses a Kullback–Leibler divergence term to keep them close.
At test time, PLATIPUS samples an initialization conditioned on support data and applies gradient descent to obtain task-specific fast weights.
The method can sample multiple potential initializations for a task, reflecting its probabilistic formulation.
PLATIPUS is uncertainty-aware and can request labels for inputs about which it is unsure, but it has higher computational costs and does not apply to reinforcement learning.

5.13 Bayesian MAML (BMAML)

BMAML represents a task with multiple jointly optimized parameter particles rather than a single solution, using SVGD and a chaser loss to approximate task-specific posterior solutions. Its flexibility comes with memory and kernel-selection costs.

Bayesian MAML: BMAML jointly optimizes M parameter particles to propose multiple solutions for a task.Unlike probabilistic MAML variants that learn a distribution, BMAML maintains a discrete set of possible solutions.
Particle updates: SVGD updates each particle using similarity-weighted gradients and a repulsive force that prevents particle collapse.The approach uses an RBF kernel in the described implementation, although other kernels are theoretically possible.
Chaser loss: The chaser loss brings support-set particles closer to a leader distribution obtained by additional SVGD steps on support and query data.The leader uses s additional updates, while the chaser uses n support-set updates.
Applications: BMAML can also be applied to reinforcement learning by replacing SVGD with Stein Variational Policy Gradient.This extends the particle-based formulation beyond supervised task prediction.
Limitations: Keeping M parameter sets in memory does not scale well, and SVGD is sensitive to the selected kernel function.The survey identifies memory reduction and learning the kernel function as future-work directions.

5.14 Simple Differentiable Solvers

Simple differentiable solvers embed analytically or iteratively solved classical learners within a meta-learning pipeline. They offer computational simplicity and state-of-the-art few-shot classification, but their applicability is limited to few-shot settings.

Solver families: R2-D2 and LR-D2 use differentiable ridge-regression and logistic-regression base-learners with analytical or iterative solvers.Their shared principle is selecting simple base-learners that support efficient learning through closed-form or differentiable optimization.
R2-D2: Woodbury’s identity changes the ridge-regression computation from quadratic in embedding size to quadratic in the number of examples.The transformed expression is linear in embedding size and more manageable when the few-shot task has small n.
LR-D2: LR-D2 uses Newton’s method for logistic regression and likewise applies Woodbury’s identity because the direct matrix is quadratic in embedding size.The logistic-regression targets are binary, with y ∈ {−1, +1}^n.
R2-D2: R2-D2 computes task-specific weights from embedded support examples, then uses query loss to update the embedding and base-learner hyperparameters.The meta-learner learns α, β, γ, and the CNN embedding weights.
MetaOptNet: MetaOptNet uses linear SVMs as its base-learner and achieved state-of-the-art performance on few-shot image classification.The survey groups it with differentiable classical solvers built on learned feature extractors.
Limitations: Simple differentiable solvers are reasonably fast but limited to few-shot learning settings.The survey identifies investigating other simple base-learners as future work.

5.15 Optimization-based Techniques, in conclusion

Optimization-based meta-learning learns task-adaptation procedures or initialization parameters, with MAML serving as a central influence on many later methods. These approaches can handle wider task distributions but remain computationally expensive, motivating work on efficiency, solution quality, generalization, benchmarking, and reproducibility.

Optimization-based approaches: Optimization-based meta-learners learn optimization procedures and/or use multiple tasks to enable rapid adaptation to new tasks.This distinguishes them from ordinary base-level optimization such as gradient descent.
Historical development: The trainable LSTM optimizer preceded methods including the LSTM meta-learner and MAML in the development of optimization-based techniques.MAML simplified the LSTM meta-learner by replacing its trainable optimizer with hand-crafted gradient descent.
MAML descendants: MAML inspired methods that learn learning rates, model uncertainty, multiple initializations, lower-dimensional representations, or first-order adaptation.Examples include Meta-SGD, probabilistic MAML variants, BMAML, LEO, and Reptile.
Differentiable solvers: R2-D2, LR-D2, and MetaOptNet place ridge regression, logistic regression, or SVM classifiers on top of learned feature extractors.These methods connect optimization-based meta-learning with simple classical machine-learning solvers.
Trade-offs: Optimization-based approaches can achieve better performance on wider task distributions than model-based approaches.Their broader task-distribution advantage is accompanied by substantial computational expense from task-specific optimization or learned optimization procedures.
Open challenges: Future work is expected to reduce computational demands while improving solution quality and generalization through benchmarking and reproducibility research.The survey characterizes optimization-based meta-learning as an active research area.

6 Concluding Remarks

Deep Meta-Learning organizes methods around learning across tasks and offers a unified view of metric-, model-, and optimization-based approaches. The field still faces challenges involving fair evaluation, generalization to distant tasks, meta-overfitting, computational cost, and faster learning beyond feature reuse.

Overview: Deep Meta-Learning methods are categorized as metric-based, model-based, and optimization-based approaches relying on input similarity, task embeddings with states, and task-specific updates.
Overview: 5-shot classification typically outperforms 1-shot classification on miniImageNet, indicating that data scarcity remains a major bottleneck.
Open Challenges and Future Work: Comparisons across techniques are difficult because different methods use different backbone architectures, making methodological gains hard to separate from backbone effects.
Open Challenges and Future Work: Deep Meta-Learning can suffer from meta-overfitting when networks memorize meta-training tasks rather than generalizing to new tasks.
Open Challenges and Future Work: Meta-learning techniques are often evaluated on narrow benchmark sets, and adaptation can decline on more distant tasks; pre-training and fine-tuning may outperform them under dataset shift.
Open Challenges and Future Work: Meta-training can be computationally expensive despite rapid adaptation at deployment, while MAML has been found to rely largely on feature reuse rather than rapid learning.