PaperlayerView exact revision on arXiv

Source-linked AI summary

TransMatch: A Transfer-Learning Scheme for Semi-Supervised Few-Shot Learning

Zhongjie Yu, Lin Chen, Zhongwei Cheng, Jiebo Luo

arXiv:1912.09033v2cs.LGcs.CVstat.ML

TL;DR

Few-shot learning seeks robust recognition from only a few labeled examples, while most existing approaches use episodic meta-learning. TransMatch instead combines transfer learning from labeled base classes with semi-supervised use of unlabeled novel-class data, and the paper reports state-of-the-art results on two few-shot benchmarks.

  • Problem

    Few-shot learning seeks to recognize novel classes from few labeled examples, while most existing methods use episodic meta-learning.

    Source: §1 · p. 1

  • Method

    TransMatch pretrains on labeled base classes, initializes novel-class classifier weights by imprinting, and fine-tunes with MixMatch on unlabeled data.

    Source: §1 · p. 2, §3.3 · p. 5

  • Results

    TransMatch achieves state-of-the-art results on two popular few-shot learning datasets.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The authors conclude that TransMatch effectively uses both labeled base-class data and unlabeled novel-class data.

    Source: §5 · p. 8

    Directly applying semi-supervised methods to unlabeled test data may perform poorly when the number of labeled examples is extremely small.

    Source: §2.3 · p. 4

Abstract

from arXiv · show

The successful application of deep learning to many visual recognition tasks relies heavily on the availability of a large amount of labeled data which is usually expensive to obtain. The few-shot learning problem has attracted increasing attention from researchers for building a robust model upon only a few labeled samples. Most existing works tackle this problem under the meta-learning framework by mimicking the few-shot learning task with an episodic training strategy. In this paper, we propose a new transfer-learning framework for semi-supervised few-shot learning to fully utilize the auxiliary information from labeled base-class data and unlabeled novel-class data. The framework consists of three components: 1) pre-training a feature extractor on base-class data; 2) using the feature extractor to initialize the classifier weights for the novel classes; and 3) further updating the model with a semi-supervised learning method. Under the proposed framework, we develop a novel method for semi-supervised few-shot learning called TransMatch by instantiating the three components with Imprinting and MixMatch. Extensive experiments on two popular benchmark datasets for few-shot learning, CUB-200-2011 and miniImageNet, demonstrate that our proposed method can effectively utilize the auxiliary information from labeled base-class data and unlabeled novel-class data to significantly improve the accuracy of few-shot learning task.

1. Introduction

Few-shot learning seeks to recognize novel concepts from very few labeled examples, while prior approaches commonly use episodic meta-learning. The paper proposes a transfer-learning framework that uses labeled base-class data and unlabeled novel-class data, and introduces TransMatch within it.

1. Introduction

  • Few-shot learning targets learning novel concepts from a single or a few labeled examples, against a backdrop of costly labeled-data collection.The introduction notes that deep learning with limited labeled data can overfit.Source: §1 · p. 1
  • Meta-learning approaches mimic few-shot testing with episodic training, whereas transfer-learning approaches adapt models pretrained on abundant base-class data.TransMatch is inspired by transfer learning and also uses unlabeled novel-class data.Source: §1 · p. 1
  • Earlier semi-supervised few-shot work used meta-learning approaches that incorporate unlabeled data during episodic training.The cited work reports that incorporating unlabeled data improved few-shot classification accuracy under meta-learning.Source: §1 · p. 2
  • The framework uses labeled base-class data and unlabeled novel-class data as auxiliary information for semi-supervised few-shot learning.The authors note that large-scale visual datasets can provide base-class data, while unlabeled data for a new task is relatively easy to acquire.Source: §1 · p. 2
  • TransMatch combines transfer-learning-based few-shot learning with semi-supervised learning, unlike prior semi-supervised few-shot methods based on meta-learning.The authors report state-of-the-art results on two popular few-shot benchmarks.Source: §1 · p. 2

2. Related Work

The related-work section reviews few-shot learning, semi-supervised learning, and semi-supervised few-shot learning. It distinguishes episodic meta-learning from transfer-based adaptation and motivates combining transfer learning with unlabeled-data methods.

2.1. Few-Shot Learning

  • Few-shot methods are grouped into episodic meta-learning and transfer learning, which adapts a model pretrained on base classes to novel classes.Prior transfer approaches include imprinting classifier weights from novel-class feature means and learning or predicting classifier weights.Source: §2.1 · p. 3

2.2. Semi-Supervised Learning

  • Semi-supervised learning includes consistency regularization and entropy minimization; MixMatch combines both families and is adopted in the proposed framework.The authors caution that applying MixMatch naively to few-shot learning may perform poorly, especially in 1-shot and 2-shot settings.Source: §2.2 · pp. 3–4

2.3. Semi-Supervised Few-Shot Learning

  • Prior semi-supervised few-shot methods extend prototypical networks, use label propagation, or add confident predictions through self-training.The reviewed approaches are meta-learning-based; the authors note that directly applying semi-supervised methods with extremely few labeled examples may lead to poor performance.Source: §2.3 · p. 4

3. The Proposed Framework

The framework addresses semi-supervised few-shot classification by using labeled base-class data alongside few labeled and many unlabeled novel-class examples. It proceeds from base-class pretraining through classifier-weight imprinting to semi-supervised fine-tuning.

3. The Proposed Framework

  • The task uses labeled base-class data as auxiliary information to classify novel classes from few labeled and many unlabeled examples.The base and novel classes are disjoint.Source: §3 · p. 4

3.1. Part I: Pre-train Feature Extractor

  • A feature extractor is pre-trained on many-shot base-class examples to encode information that serves as prior information for later few-shot learning.Unlike conventional meta-learning, this stage does not use episodic training to mimic the test scenario.Source: §3.1 · p. 4

3.2. Part II: Classifier Weight Imprinting

  • Novel-class classifier weights are imprinted from the mean feature vectors of few-shot examples, providing a better initialization that reduces intra-class feature variation.The imprinted classifier can perform well even without fine-tuning.Source: §3.2 · pp. 4–5
  • Novel-class prediction compares each example with classifier weights using cosine similarity and assigns the class with the highest score.The weights and feature vector are normalized onto a unit ball.Source: §3.2 · p. 5

3.3. Part III: Semi-Supervised Fine-tuning

  • When unlabeled novel-class data is available, MixMatch fine-tunes the imprinted classifier using both labeled and unlabeled examples.The framework allows other semi-supervised methods, but this work uses MixMatch.Source: §3.3 · p. 5
  • Mean predictions across augmented versions of each unlabeled example provide estimated labels, with sharpening temperature T = 0.5 in the experiments.The same data augmentation is applied to labeled examples.Source: §3.3 · p. 5
  • MixMatch concatenates and shuffles labeled and unlabeled examples before applying MixUp, and its objective includes cross-entropy and consistency-regularization losses.The consistency-regularization loss is identified with [21].Source: §3.3 · p. 5

4. Experiments

The experiments evaluate TransMatch on miniImageNet and CUB-200-2011, comparing it with few-shot and semi-supervised methods and testing how unlabeled data affects performance.

4. Experiments

  • On miniImageNet, TransMatch outperforms conventional and state-of-the-art semi-supervised few-shot methods; on CUB-200-2011, it achieves the best result against Imprinting and Max-Match.The miniImageNet comparison is reported in Table 1, and the CUB-200-2011 comparison in Table 5.Source: §4.1 · p. 6, Table 1 · p. 7, Table 5 · p. 8

4.1. Experiments on miniImageNet

  • TransMatch performs better with more unlabeled images on miniImageNet, with results beginning to saturate after 100 images in the 1-shot setting.The authors report the unlabeled-image comparisons in Table 2.Source: §4.1 · p. 6, Table 2 · p. 7
  • Within the framework, using Pseudo-Label instead of MixMatch produces worse overall performance.The authors report this comparison for 5-way miniImageNet classification with 100 unlabeled images.Source: Table 3 · p. 7, §4.1 · p. 7
  • 11.02, 4.28, 2.92, 1.73, and 1.22 are TransMatch's gains over MixMatch in the 1-, 2-, 3-, 4-, and 5-shot settings, respectively.Figure 3 compares Imprinting, MixMatch, and TransMatch with 100 unlabeled images; the authors report larger gains over MixMatch with fewer shots.Source: Figure 3 · p. 7, §4.1 · p. 7
  • With distractor classes among the unlabeled images, MixMatch degrades while TransMatch still outperforms Imprinting in all reported cases.The distractor classes are randomly selected from classes disjoint from the novel classes.Source: Table 4 · p. 8, §4.1 · p. 8

4.2. Experiments on CUB-200-2011

  • On CUB-200-2011, TransMatch achieves the best result compared with Imprinting and Max-Match; evaluations across different shot and unlabeled-image counts show better performance with more unlabeled data.Source: §4.2 · p. 8

5. Conclusion

TransMatch combines a transfer-learning framework with Imprinting and MixMatch for semi-supervised few-shot learning. Experiments show state-of-the-art results and effective use of labeled base-class and unlabeled novel-class data.

  • Experiments on two popular few-shot learning datasets show TransMatch achieves state-of-the-art results while effectively using labeled base-class and unlabeled novel-class data.Source: §5 · p. 8
  • TransMatch instantiates the proposed framework by combining Imprinting with the semi-supervised method MixMatch.Source: §5 · p. 8
  • The framework uses transfer learning to explore information from labeled base-class data and unlabeled novel-class data, rather than relying on meta-learning.Source: §5 · p. 8
Loading 1912.09033v2…