Source-linked AI summary
Unsupervised Meta-Learning For Few-Shot Image Classification
Siavash Khodadadeh, Ladislau Bölöni, Mubarak Shah
TL;DR
Few-shot meta-learning traditionally depends on labeled tasks similar to the target, creating substantial data requirements. UMTRA instead constructs synthetic tasks from unlabeled, target-relevant data through random sampling and augmentation. On benchmark datasets, it outperforms tested unsupervised representation-learning methods and approaches supervised MAML with several orders of magnitude fewer labels, while requiring suitable class-discriminating augmentations.
Problem
Model-agnostic meta-learning requires labeled meta-training tasks, making broad task preparation costly and limiting the task types it can address.
Method
UMTRA generates synthetic few-shot classification tasks from an unlabeled dataset using random sampling, augmentation, and artificial labels.
Results
UMTRA outperforms tested unsupervised representation-learning approaches on Omniglot and Mini-Imagenet, while achieving comparable supervised-MAML accuracy with vastly fewer labels.
Takeaways & Limitations
UMTRA trades off some classification accuracy for a reduction of several orders of magnitude in the labels required compared with supervised model-agnostic meta-learning.
Takeaways & Limitations
UMTRA assumes unlabeled data comes from the target-task distribution, contains many more natural classes than the target task, and supports augmentations that preserve class membership.
Abstract
from arXiv · showhide
Few-shot or one-shot learning of classifiers requires a significant inductive bias towards the type of task to be learned. One way to acquire this is by meta-learning on tasks similar to the target task. In this paper, we propose UMTRA, an algorithm that performs unsupervised, model-agnostic meta-learning for classification tasks. The meta-learning step of UMTRA is performed on a flat collection of unlabeled images. While we assume that these images can be grouped into a diverse set of classes and are relevant to the target task, no explicit information about the classes or any labels are needed. UMTRA uses random sampling and augmentation to create synthetic training tasks for meta-learning phase. Labels are only needed at the final target task learning step, and they can be as little as one sample per class. On the Omniglot and Mini-Imagenet few-shot learning benchmarks, UMTRA outperforms every tested approach based on unsupervised learning of representations, while alternating for the best performance with the recent CACTUs algorithm. Compared to supervised model-agnostic meta-learning approaches, UMTRA trades off some classification accuracy for a reduction in the required labels of several orders of magnitude.
1 Introduction
UMTRA addresses the label requirements of supervised meta-learning by generating synthetic few-shot classification tasks from unlabeled data. It uses random sampling, augmentation, and artificial labels, and reports strong benchmark performance with far fewer labels than supervised MAML.
- Motivation: Supervised model-agnostic meta-learning requires labeled learning tasks, limiting meta-learning to task types with available supervised examples.The paper identifies labeling many tasks as costly and conceptually restrictive.
- UMTRA: UMTRA performs unsupervised, model-agnostic meta-learning for few-shot classification using an unlabeled dataset rather than a collection of labeled tasks.The unlabeled data should match the target-task distribution and contain substantially more classes than the final classifier.
- UMTRA: UMTRA creates synthetic tasks by randomly sampling unlabeled images, applying domain-specific augmentation, and assigning temporary artificial labels.The sampled images form training data, while augmented versions provide validation data for the meta-learning process.
- Results: UMTRA outperforms tested unsupervised representation-learning approaches on Omniglot and Mini-Imagenet while alternating for best performance with CACTUs.The paper also reports a substantial label-efficiency advantage over supervised MAML.
- Results: 95.43% accuracy is achieved by UMTRA on 5-way 5-shot Omniglot with 25 labels, versus 98.83% for supervised MAML with 24025 labels.This comparison illustrates the reported trade-off between accuracy and the amount of labeled data required.
2 Related Work
Related work frames meta-learning as preparation for rapid adaptation from few examples, with model-agnostic methods designed for differentiable architectures. UMTRA is positioned alongside unsupervised approaches that construct learning signals without labeled meta-training tasks.
- Meta-learning: Few-shot meta-learning prepares networks to learn target tasks from the small amount of training data available in few-shot settings.The related-work discussion also notes applications such as fast adaptation for video tracking.
- Model-agnostic approaches: Model-agnostic methods such as MAML and Reptile encode meta-learning in network weights and can be used with differentiable architectures.Their target phase uses established learning algorithms, potentially with meta-learning-specific hyperparameters.
- Unsupervised meta-learning: Unsupervised meta-learning generates tasks from unlabeled data to learn structures relevant to future supervised tasks, which need not overlap with meta-training tasks.The paper identifies CACTUs as a related unsupervised meta-learning model.
3 The UMTRA algorithm
UMTRA constructs synthetic one-shot meta-learning tasks from unlabeled data by random sampling and augmentation, replacing supervised task collections with artificial labels. Its effectiveness depends on sampling likely distinct classes and using augmentations that preserve class membership while providing separate validation examples.
- Task construction: UMTRA creates synthetic tasks from an unlabeled dataset instead of requiring a collection of supervised meta-learning tasks.The dataset should be drawn from the target-task distribution and contain many natural classes.
- Task construction: During meta-training, UMTRA samples N examples, assigns labels 1 through N, and uses one-shot tasks with K = 1.The artificial labels need only preserve class distinctions during meta-training; target learning can still use K different from 1.
- Task construction: When N ≪ c, random sampling is likely to select examples from different natural classes, supporting the artificial-label construction.For 5-way classification, the reported probability is 99.21% on Omniglot with c = 1200 and 85.23% on Mini-Imagenet with c = 64.
- Validation construction: UMTRA forms validation examples by applying an augmentation A to each training example, requiring A to preserve class membership.Examples include masking pixels or translating image pixels; the augmentation is domain-dependent and class membership cannot be directly verified without labels.
- Validation construction: Using the same data for training and validation biases the generalization estimate and produces accuracy close to training from scratch, whereas suitable augmentation can yield results comparable to supervised meta-learning.The method does not require clustering, unlike the described CACTUs approach.
4 Experiments
Experiments on Omniglot and Mini-Imagenet evaluate UMTRA’s model-agnostic unsupervised meta-learning against supervised, unsupervised-representation, scratch, and CACTUs baselines. Results show strong performance, while augmentation choice and hyperparameters materially affect outcomes.
- The authors report that suitable augmentation samples remain within the same class manifold, while augmentation choice and other hyperparameters strongly affect performance.
- UMTRA outperformed training from scratch and unsupervised representation-learning approaches on Omniglot and Mini-Imagenet.
- UMTRA outperformed CACTUs on Omniglot, while CACTUs-MAML with DeepCluster surpassed it on Mini-Imagenet 5-, 20-, and 50-shot settings.
- 83.80% accuracy with 5 labels versus supervised MAML’s 94.46% with 24005 labels in 5-way one-shot Omniglot classification.
- 95.43% accuracy with 25 labels versus supervised MAML’s 98.83% with 24025 labels in 5-way 5-shot Omniglot classification.
- Auto-augmentation produced the best tested Mini-Imagenet results and was used in subsequent experiments.
5 Conclusions
The paper concludes that UMTRA enables model-agnostic few-shot and one-shot meta-learning from unlabeled data. It sacrifices some accuracy relative to supervised meta-learning while requiring dramatically fewer labels.
- UMTRA performs unsupervised meta-learning for few-shot and one-shot classifiers without constraining classifier architecture.
- UMTRA outperformed learning-from-scratch and unsupervised representation-learning approaches on Omniglot and Mini-Imagenet.
- UMTRA alternated for the best results with CACTUs, whose clustering-based approach differs from UMTRA’s sampling and augmentation.
- UMTRA performed worse than supervised meta-learning but required 3-4 orders of magnitude less labeled data.
Supplementary Material for Unsupervised Meta-Learning for Few-Shot Image Classification
UMTRA rapidly adapts to novel image tasks from unlabeled-data meta-learning, while its video extension uses domain-specific initialization, data splits, and temporal augmentation. Across the reported experiments, UMTRA adapts faster than training from scratch, outperforms other unsupervised approaches for video, and faces a K-dependent recovery pattern on Mini-Imagenet.
- Evolution of accuracy during training: UMTRA and supervised MAML reach their accuracy plateaus quickly during target training, whereas learning from scratch requires more gradient steps.The comparison averages results across 1000 tasks on Omniglot and Mini-Imagenet.
- Evolution of accuracy during training: For Mini-Imagenet with K = 5 and K = 20, UMTRA accuracy dips after the first iteration before recovering over several iterations.The paper conjectures this reflects meta-learning with K = 1, which optimizes the network for one sample per class.
- Representation analysis: The Omniglot representation analysis compares scratch training, UMTRA, and MAML using last-hidden-layer t-SNE visualizations before and after target training.Each class has a distinct color and shape; one instance per class is used for target training, with training instances marked by larger, lighter symbols and dotted connections.
- Video Domain: In video action recognition, UMTRA is evaluated on UCF-101 after pretraining on Sports-1M and meta-learning on Kinetics, creating a domain shift between meta-training and evaluation.Kinetics supplies 400 actions, while UCF-101 contains 101 action classes; the authors describe this as resembling practical few-shot learning on a novel domain.
- Video Domain: For video, UMTRA uses temporally shifted fragments from the same clip as augmentation, and it performs better than other approaches using unsupervised data.The evaluation samples five UCF-101 classes for one-shot learning and reports both accuracy and F1-score because class sizes differ.