PaperlayerView exact revision on arXiv

Source-linked AI summary

Revisiting Metric Learning for Few-Shot Image Classification

Xiaomeng Li, Lequan Yu, Chi-Wing Fu, Meng Fang, Pheng-Ann Heng

arXiv:1907.03123v2cs.CV

TL;DR

Few-shot classification aims to recognize new visual concepts from only a few labeled examples, but metric-based approaches have not fully used relationships among training samples to learn embeddings. The paper extends triplet learning into an episodically trained K-tuplet network and combines its features with a nonlinear distance metric. It reports stronger results than other metric-based methods on miniImageNet and generalization to four other datasets.

  • Problem

    Few-shot classification seeks to assign query examples to new categories using only a small number of labeled support examples, and existing metric-based methods do not fully exploit relationships among input samples.

    Source: §1 · pp. 1–2

  • Method

    The deep K-tuplet Network extends triplet learning to compare an anchor with multiple negative samples and learns transferable feature embeddings for few-shot classification.

    Source: §1 · p. 2

  • Results

    The method outperforms other metric-based methods on miniImageNet and is reported to excel on CUB-200, Stanford Dogs, Stanford Cars, and Caltech-101.

    Source: §1 · p. 3, §6 · p. 17

  • Takeaways & Limitations

    A miniImageNet-trained model is reported to generalize to novel categories in four other datasets, supporting the paper's cross-dataset evaluation setting.

    Source: §1 · p. 3, §6 · p. 17

Abstract

from arXiv · show

The goal of few-shot learning is to recognize new visual concepts with just a few amount of labeled samples in each class. Recent effective metric-based few-shot approaches employ neural networks to learn a feature similarity comparison between query and support examples. However, the importance of feature embedding, i.e., exploring the relationship among training samples, is neglected. In this work, we present a simple yet powerful baseline for few-shot classification by emphasizing the importance of feature embedding. Specifically, we revisit the classical triplet network from deep metric learning, and extend it into a deep K-tuplet network for few-shot learning, utilizing the relationship among the input samples to learn a general representation learning via episode-training. Once trained, our network is able to extract discriminative features for unseen novel categories and can be seamlessly incorporated with a non-linear distance metric function to facilitate the few-shot classification. Our result on the miniImageNet benchmark outperforms other metric-based few-shot classification methods. More importantly, when evaluated on completely different datasets (Caltech-101, CUB-200, Stanford Dogs and Cars) using the model trained with miniImageNet, our method significantly outperforms prior methods, demonstrating its superior capability to generalize to unseen classes.

1. Introduction

Few-shot classification must assign unlabeled queries to new categories from only a few labeled support examples, while existing metric-based methods leave room to better use relationships among training samples. The paper extends triplet learning to a K-tuplet network and reports strong miniImageNet and cross-dataset results.

1. Introduction

  • Metric-based few-shot methods use episodic training and learned features, but the authors argue they do not fully exploit relationships among input samples.The paper identifies improving embedding transferability and generality to unseen classes as its motivation.Source: §1 · p. 2
  • The deep K-tuplet Network jointly compares an anchor with K negative samples to learn transferable, discriminative embeddings for few-shot classification.The paper also presents semi-hard mining to sample informative hard triplets, which it says can speed convergence and stabilize training.Source: §1 · p. 2
  • The framework combines the learned embedding with a learnable nonlinear distance metric and evaluates a miniImageNet-trained model on four other datasets.The reported cross-dataset results indicate generalization to novel classes across datasets.Source: §1 · p. 2
  • The paper presents a baseline focused on feature embedding, which it says prior few-shot work has neglected.Its contributions also include the K-tuplet model and a cross-domain evaluation setting.Source: §1 · pp. 2–3
  • The method outperforms other metric-based methods on miniImageNet and excels on CUB-200, Stanford Dogs, Stanford Cars, and Caltech-101 in cross-dataset evaluation.The paper frames evaluation across datasets as a new setting for assessing cross-domain generalization.Source: §1 · p. 3

2. Related Work

Related work spans early generative and inference-based approaches, meta-learners, and metric-based methods. The paper situates its approach among methods that learn embeddings and compare query and support examples, emphasizing feature embedding for unseen classes.

2. Related Work

  • Early few-shot approaches focused on generative models and inference strategies, while recent deep-learning work brought significant progress.The cited early methods assume knowledge from previously learned classes can support predictions for new classes from one or a few labels.Source: §2 · p. 3

2.1. Meta-learners for Few-Shot Learning

  • Meta-learners use prior tasks to support new-task learning, but several cited methods fine-tune or update parameters on the target task.MAML trains a basic model across tasks; Ravi and Larochelle use an LSTM to update classifier weights, while other approaches use fast parameterization or memory.Source: §2.1 · p. 3

2.2. Deep Metric Learning

  • Deep metric learning combines feature embeddings with triplet losses to reduce within-class variation and increase between-class variation.The cited methods apply metric learning to image retrieval, face recognition, and person re-identification; this paper focuses on unseen-class few-shot learning.Source: §2.2 · p. 4

2.3. Metric Learning for Few-shot Learning

  • Metric-based few-shot methods learn embeddings and similarity measures for feed-forward query classification, using approaches such as cosine or Euclidean distance and nonlinear comparators.The cited methods include matching and prototypical networks, relation networks, and episode-based training; the paper argues feature embedding has received insufficient attention.Source: §2.3 · pp. 4–5

3. Method

The method learns transferable embeddings by comparing multiple negative samples per update, then applies a nonlinear metric to compare query and support features. It also uses semi-hard mining during later training and evaluates classification by selecting the support label with the highest similarity score.

3.1. Overview

  • The framework meta-learns an embedding from training data, then compares query and support features with a nonlinear metric for few-shot classification.The predicted label is the support label with the maximum similarity score.Source: §3.1 · pp. 5–6, §3.4 · p. 8

3.2. Meta-learn Feature Embedding

  • K-tuplet loss jointly compares an anchor and positive sample against K negative samples, addressing triplet loss’s single-negative update.K=1 recovers triplet loss; experiments report the best performance at K=5, while larger K eventually increases computational burden and can reduce accuracy.Source: §3.2 · pp. 6–7, Figure 2 · p. 7

3.3. Efficient Training with Semi-hard Mining

  • Semi-hard mining selects informative triplets by checking the loss condition and is used when training starts to converge.The reported setting begins at 80 epochs; the authors say this strategy improves training efficiency and contributes to feature embedding learning.Source: §3.3 · pp. 7–8

3.4. Non-linear Distance Metric Learning

  • The nonlinear metric takes concatenated query and support embeddings and outputs similarity scores used to predict the query label.The support label with the maximum score is selected.Source: §3.4 · p. 8

3.5. Technique Details

  • For 5-way 1-shot evaluation, each query is compared with five support samples; for 5-way 5-shot, support features are summed within each class before comparison.Metric training uses multiple episodes per mini-batch to increase class-sample diversity and make training more stable.Source: §3.5 · p. 9

4. Experiments

The experiments evaluate few-shot classification on miniImageNet, test generalization on different datasets using a miniImageNet-trained model, and analyze the method’s components.

4. Experiments

  • The experiments evaluate the method on miniImageNet.This is the first evaluation described.Source: §4 · p. 9
  • The experiments test generalization by evaluating a miniImageNet-trained model directly on completely different datasets.Source: §4 · p. 9
  • The experiments analyze different components of the method.Source: §4 · p. 9

4.1. Few-shot Classification on the miniImageNet

On miniImageNet, the method achieves the highest accuracy among metric-based approaches for 5-way 1-shot and very competitive accuracy for 5-way 5-shot classification. Its learned embedding also produces competitive results with Euclidean K-NN, without the nonlinear metric.

4.1. Few-shot Classification on the miniImageNet

  • The authors use a deeper ResNet embedding network because shallow embeddings did not adequately use the method's expressive capacity and risked underfitting.They follow recent works in adopting ResNet.Source: §4.1 · p. 9
  • With Euclidean distance and K=1, the embedding still achieves competitive results, indicating discrimination on unseen novel categories.This variant removes the nonlinear metric and classifies query images using K-NN over embedded query and support images.Source: §4.1 · pp. 9–10
  • The learned embedding's 10 nearest neighbors for a miniImageNet test query preserve apparent visual similarity and facilitate accurate recognition.The neighbors are selected using Euclidean distance between embedded features.Source: §4.1 · p. 10
  • The method achieves the highest accuracy among metric-based methods on 5-way 1-shot and very competitive accuracy on 5-way 5-shot miniImageNet classification.Table 1 reports average accuracies over 600 test episodes with 95% confidence intervals.Source: §4.1 · p. 9, Table 1 · p. 10

4.2. Generalizing to Other Datasets

The authors evaluate transfer by applying a miniImageNet-trained model to novel classes in four other datasets without fine-tuning. It performs consistently better than Relation Net and Proto Net across those datasets.

4.2. Generalizing to Other Datasets

  • The evaluation targets distribution shift by testing on novel classes from Caltech-101, CUB-200, Stanford Dogs, and Stanford Cars after miniImageNet training.For each dataset, 20 test classes are randomly selected; the model is applied directly without fine-tuning.Source: §4.2 · p. 12
  • Without fine-tuning, the miniImageNet-trained model performs consistently better than Relation Net and Proto Net on four other datasets using the same network backbone.The test classes are selected separately from the training label space.Source: §4.2 · p. 12, Table 2 · p. 12
  • Caltech-101 accuracy is higher than on the other datasets, including miniImageNet, which the authors attribute to single objects and purer backgrounds.CUB-200, Stanford Dogs, and Stanford Cars have relatively complex backgrounds.Source: §4.2 · p. 12
  • Visualized 5-way 5-shot comparisons indicate that the method is discriminative for similar objects and learns more transferable features across datasets.Figure 5 shows examples from miniImageNet, CUB-200, Stanford Cars, and Stanford Dogs, with support images ranked by similarity to the test image.Source: Figure 5 · p. 11, §4.2 · p. 13

4.3. Analysis of Our Method

The analysis tests how backbone choice, the number of negatives, semi-hard mining, margin, and training-class count affect few-shot performance on miniImageNet.

4.3.1. Results with Different Network Backbones

  • ResNet improves accuracy by about 10% over AlexNet and VGG11 in both 5-way 1-shot and 5-way 5-shot settings, while ResNet50 performs worse than ResNet34.ResNet18 and ResNet34 are similar on 5-way 5-shot; ResNet34 is slightly better on 5-way 1-shot, and the authors select it as the backbone.Source: §4.3.1 · p. 13, Table 3 · p. 13

4.3.2. The Tuplet-loss with Different Negative Pairs

  • K-tuplet loss with K = 5 reaches 54.46% 1-shot and 68.15% 5-shot, versus 40.15% and 54.62% with triplet loss (K = 1).Accuracy improves as more negatives are used, then saturates; the authors attribute the gain to more stable gradients from interacting with more samples per mini-batch.Source: §4.3.2 · p. 14, Table 4 · p. 14

4.3.3. Effects of Semi-hard Mining

  • Semi-hard mining further improves miniImageNet accuracy by relative 1.6% in 1-shot and 1.0% in 5-shot evaluation.The comparison is between training without semi-hard mining for 100 epochs and training with it for the final 20 of 100 epochs.Source: §4.3.3 · p. 14, Table 5 · p. 14

4.3.4. The Analysis of Different Margin

  • A margin of 0.5 gives the best feature embedding in this task; smaller margins weaken inter-class variation learning, while larger margins may make training harder.Table 6 reports few-shot classification accuracy for different margins on miniImageNet.Source: §4.3.4 · p. 15, Table 6 · p. 15

4.3.5. The Results with More Training Classes

  • Adding ImageNet training classes that do not share labels with the test images further improves accuracy on miniImageNet.The authors suggest that the extra training classes provide more samples for learning transferable feature embeddings.Source: §4.3.5 · p. 15, Table 7 · p. 15

5. Comparison of Visualized Features

The authors visualize miniImageNet test-set features from Proto Net, Relation Net, and their method using t-SNE, comparing methods on the same samples.

5. Comparison of Visualized Features

  • For each visualization, five test classes and 200 samples per class are embedded with t-SNE, using the same samples across methods.Figures 6 and 7 show two examples.Source: §5 · p. 16, Figure 6 · p. 16, Figure 7 · p. 16
  • The authors report that their features separate the five classes better than Proto Net and Relation Net, including two challenging dog species.Figure 6 shows especially clear separation for cuirass, crate, and mixing bowl; the authors say this supports subsequent K-NN classification.Source: §5 · pp. 16–17, Figure 6 · p. 16

6. Conclusion

The conclusion presents the K-tuplet network as a way to learn transferable feature embeddings and reports stronger few-shot results on miniImageNet and four other datasets.

6. Conclusion

  • The K-tuplet network uses relationships among training samples to learn feature embeddings intended to transfer to novel classes.The authors describe the method as efficient, simple, and effective.Source: §6 · p. 17
  • The method outperforms other metric-based methods on the public benchmark and generalizes to novel categories across four other datasets.The conclusion attributes improved few-shot accuracy to the generalized feature embedding.Source: §6 · p. 17

Related papers

Loading 1907.03123v2…