Source-linked AI summary
A Concise Review of Recent Few-shot Meta-learning Methods
Xiaoxu Li, Zhuo Sun, Jing-Hao Xue, Zhanyu Ma
TL;DR
Few-shot learning remains challenging because it relies on very few labeled examples. This review formulates the setting, surveys representative methods across four technical branches, and identifies challenges and future prospects.
Problem
Method
The review formulates few-shot learning, presents a general episodic training and evaluation guide, and organizes representative methods into four technical branches.
Source: §1 · p. 1, §2.2 · p. 2, §3 · p. 2Results
The review categorizes methods as learning an initialization, generating parameters, learning an optimizer, and memory-based methods.
Source: §3 · p. 2Takeaways & Limitations
The concise taxonomy and review are intended to help researchers quickly grasp representative few-shot meta-learning methods.
Source: §1 · p. 1Uneven performance across tasks can make meta-learner knowledge uncertain for novel tasks from unseen classes, motivating more diversified meta-learners.
Source: §4 · p. 6
Abstract
from arXiv · showhide
Few-shot meta-learning has been recently reviving with expectations to mimic humanity's fast adaption to new concepts based on prior knowledge. In this short communication, we give a concise review on recent representative methods in few-shot meta-learning, which are categorized into four branches according to their technical characteristics. We conclude this review with some vital current challenges and future prospects in few-shot meta-learning.
1. Introduction
The introduction frames few-shot image classification as a challenge despite deep learning’s progress on large-scale classification, and previews a concise review of methods, training, and open challenges.
1. Introduction
- Few-shot image classification remains challenging despite major breakthroughs in large-scale image classification.The paper frames the review around this gap.Source: §1 · p. 1
- The review formulates few-shot learning and presents a pseudo-coded algorithm for general training and evaluation.Source: §1 · p. 1
- It categorizes representative methods into four branches and concludes with challenges and future prospects.Source: §1 · p. 1
2. The Framework of Few-shot Meta-learning
The framework defines few-shot learning around adaptation from a small labeled support set to query images, then outlines episodic training and evaluation using base, validation, and novel data.
2.1. Notation and definitions
- A few-shot task pairs a small labeled support set with query images from the same classes, which the classifier must distinguish.Few-shot cases have few labeled examples per class; one-shot uses one, while C-way K-shot uses K examples for each of C classes.Source: §2.1 · p. 1
- Few-shot meta-learning transfers prior knowledge from a base dataset to novel tasks, with a meta-learner and task-specific learner sharing information across tasks.Base and novel class sets are disjoint, so transferable knowledge links their tasks.Source: §2.1 · pp. 1–2
2.2. Training and evaluation of few-shot meta-learning
- The framework evaluates models through few-shot episodes, with a distinct validation dataset available for model selection.Novel-class tasks are sampled from the novel dataset, and validation accuracy can guide choices such as the number of epochs.Source: §2.2 · p. 2
- The general episodic guide represents the classifier as dependent on a support set and the base dataset, covering meta-training, pre-training, and novel-task evaluation.The review identifies this dependence as central to few-shot meta-learning.Source: §2.2 · p. 2
3. Methods of Few-shot Meta-learning
Few-shot meta-learning methods learn task-specific networks from meta-networks, with meta-learners extracting transferable prior knowledge from base data. The review groups representative methods into four branches: initialization, parameter generation, optimizer learning, and memory-based methods.
3.1. Learning an initialization
- Initialization methods either meta-train a global parameter start through support adaptation and query updates, or fine-tune base-trained parameters on novel tasks.MAML updates task-specific parameters using support-set loss, then uses query-set loss to update the global initialization; the review also describes Baseline++ fine-tuning an embedding with a new task-specific classifier.Source: §3.1 · p. 3
- TAML seeks an initialization unbiased across tasks by maximizing per-task entropy reduction or minimizing performance inequality, and reports state-of-the-art results on Onimiglot 5-way 1-shot and 5-way 5-shot classification [10].Source: §3.1 · p. 3
3.2. Generation of parameters
- Parameter-generation methods use meta-learners to predict pupil-network parameters, generate parameters for meta- and base-learners through attention, or generate classifier weights for joint base- and novel-class prediction.The weight generator takes novel-class deep features and trained base-class weights as inputs.Source: §3.2 · p. 4
- Other methods scale and shift pretrained features while sampling hard tasks, imprint novel-class classifier weights from normalized examples, or regularize base-class weights during incremental novel-class training.Source: §3.2 · pp. 4–5
3.3. Learning an optimizer
- Optimizer-learning methods either model task-specific parameter updates with LSTM cell-state evolution or perform gradient-based meta-learning in a data-dependent low-dimensional latent parameter space.LEO uses support-set loss to learn task-specific parameters and query-set loss to update the global initialization.Source: §3.3 · p. 5
3.4. Memory-based methods
- Memory-based methods include LRUA, which controls external-memory access and predicts from sequentially presented labeled examples, and SNAIL and CSNs, which use temporal convolution with attention or conditional shifts, respectively.LRUA was tested only on Omniglot; SNAIL outperformed Meta Networks on Mini-ImageNet and Omniglot, while CSNs outperformed several widely used architectures on both datasets.Source: §3.4 · p. 5
4. Some Remaining Challenges
The review identifies persistent challenges in transferring knowledge to few-label tasks, fitting meta- and task-specific learners, and handling domain shifts between base and novel data.
4. Some Remaining Challenges
- Parameter-generation methods directly generate task-specific learner parameters, but their design lacks in-depth theoretical analysis and algorithms still need to generalize from few labels.Meta-training extracts transferable or shared knowledge from an auxiliary dataset, yet training on few labeled samples remains difficult.Source: §4 · pp. 5–6
- Uneven performance across tasks can make meta-learned knowledge uncertain for novel tasks from unseen classes.The review also identifies learning suitable loss functions or network structures as alternative meta-learning emphases to explore.Source: §4 · p. 6
- Task-specific learners need feature extractors tailored to few-shot data and loss functions designed to support robustness and model performance.Existing deep feature extractors are not tailored to few-shot learning, and many loss functions are not specifically designed for few-shot scenarios.Source: §4 · p. 6
- When base and novel datasets come from quite different domains, most existing methods perform poorly because they usually assume a shared domain.The review notes that only a few domain-adaptation proposals exist for few-shot image classification.Source: §4 · p. 6
- Existing methods usually assume a single-domain base dataset, leaving multi-domain few-shot meta-learning as a research direction.The review expects transferable knowledge learned from multi-domain base data to improve generalization ability.Source: §4 · p. 6