Source-linked AI summary
Learning to Compare: Relation Network for Few-Shot Learning
Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H. S. Torr, Timothy M. Hospedales
TL;DR
Few-shot learning must recognize new classes from very few labeled examples, while existing approaches often rely on complex mechanisms or fine-tuning. The Relation Network learns an embedding and deep nonlinear comparison metric end-to-end, achieving state-of-the-art few-shot results and effectiveness across conventional and generalized zero-shot learning.
Problem
Few-shot learning seeks to recognize new classes from only a few labeled examples, challenging methods that depend on abundant annotations or complex inference and fine-tuning.
Method
The Relation Network episodically learns an embedding and deep nonlinear distance metric end-to-end to compare query items with support examples or class descriptions.
Results
The Relation Network achieves state-of-the-art few-shot performance and is effective for both conventional and generalized zero-shot learning.
Takeaways & Limitations
The framework provides a unified, simpler, and more efficient approach to few-shot and zero-shot learning.
Takeaways & Limitations
The architecture does not guarantee the self-similarity and symmetry properties of a formal similarity function.
Abstract
from arXiv · showhide
We present a conceptually simple, flexible, and general framework for few-shot learning, where a classifier must learn to recognise new classes given only few examples from each. Our method, called the Relation Network (RN), is trained end-to-end from scratch. During meta-learning, it learns to learn a deep distance metric to compare a small number of images within episodes, each of which is designed to simulate the few-shot setting. Once trained, a RN is able to classify images of new classes by computing relation scores between query images and the few examples of each new class without further updating the network. Besides providing improved performance on few-shot learning, our framework is easily extended to zero-shot learning. Extensive experiments on five benchmarks demonstrate that our simple approach provides a unified and effective approach for both of these two tasks.
1. Introduction
The paper addresses the limited-data challenge in few-shot learning by introducing a Relation Network that learns to compare query images with labeled examples. Its episode-based, end-to-end framework targets both few-shot and zero-shot learning while aiming to be simpler and faster than alternatives.
- Motivation: Few-shot learning seeks to recognise novel visual categories from very few labelled examples, making standard deep-learning fine-tuning difficult.Data augmentation and regularisation can alleviate overfitting but do not solve the limited-data problem.
- Method: The Relation Network uses an embedding module and a relation module to compare query images with few-shot labeled sample images and determine category matching.The method is organised as a two-branch architecture.
- Method: Embedding and relation modules are meta-learned end-to-end in episodes, using a learnable non-linear comparator instead of a fixed linear comparator.This extends episode-based strategies associated with prior one-shot-learning approaches.
- Scope: The approach is designed to encompass both few-shot and zero-shot learning in a clean framework.The introduction describes the framework as directly generalising across these tasks.
- Contributions: The authors report that the approach outperforms prior approaches while being simpler and faster because it avoids RNNs and fine-tuning.The reported evaluation covers four benchmarks and describes performance as compelling across the board.
2. Related Work
Prior few-shot learning work spans generative inference, meta-learned fine-tuning, recurrent memory, and feed-forward embedding or metric-learning approaches. The Relation Network instead performs feed-forward relation classification and extends to zero-shot learning by replacing sample images with category descriptions.
- Earlier few-shot methods commonly used generative models with complex iterative inference, while recent work generalised discriminative deep learning to sparse-data settings through meta-learning.
- Learning to Fine-Tune: MAML meta-learns neural-network weights that can be fine-tuned on few-shot problems within a small number of gradient-descent updates.
- Learning to Fine-Tune: Unlike fine-tuning strategies, the proposed approach solves target problems entirely feed-forward with no model updates, supporting low-latency or low-power applications.
- RNN Memory Based: Recurrent approaches learn target problems by iterating over examples and accumulating task knowledge in hidden activations or external memory.
- Embedding and Metric Learning Approaches: Embedding and metric-learning methods project query and sample images for feed-forward classification; prototypical and siamese networks use learned embeddings with fixed nearest-neighbour or linear classifiers.
- Zero-Shot Learning: For zero-shot learning, the framework replaces the sample image with a category description and predicts whether an image and category-embedding pair match.
3. Methodology
The methodology frames few-shot learning through episodic training with disjoint training and support/testing label spaces. The Relation Network compares embedded samples and queries, extending from few-shot image support to zero-shot semantic class embeddings.
- Problem setup: Few-shot learning uses disjoint training and support/testing label spaces, with C-way K-shot defined by K labelled examples for each of C classes.The support and testing sets share labels, while the training set uses a separate label space.
- One-shot: The Relation Network combines embedding-module feature maps for each sample-query pair and feeds them to a relation module that outputs a scalar similarity score from 0 to 1.Feature maps are concatenated in depth in this work, although other combination operators are possible.
- K-shot and objective: For K-shot learning, embeddings from all samples in each class are summed element-wise into a class feature map, so each query still produces C relation scores.The model trains these scores with mean square error loss against automatically generated {0, 1} targets.
- Zero-shot learning: Zero-shot learning replaces image support examples with semantic class embedding vectors, using a second heterogeneous embedding module before applying the relation network.The zero-shot objective function is the same as for few-shot learning.
- Architecture: The image embedding module follows four convolutional blocks, each with a 64-filter 3 × 3 convolution, batch normalization, and ReLU, with max pooling only in the first two blocks.The zero-shot architecture uses a DNN subnet such as Inception or ResNet pretrained on ImageNet.
4. Experiments
Experiments evaluate the Relation Network on few-shot classification across Omniglot and miniImagenet and on zero-shot classification across AwA and CUB. The model achieves state-of-the-art or competitive performance across the reported few-shot and conventional zero-shot settings.
- Tasks: Experiments cover few-shot classification on Omniglot and miniImagenet and zero-shot classification on AwA and CUB.All experiments are implemented in PyTorch.
- Few-shot learning settings: The few-shot models are trained end-to-end from scratch with Adam, an initial learning rate of 10−3, and no additional dataset.The learning rate is halved every 100,000 episodes.
- Omniglot: On Omniglot, the Relation Network achieves state-of-the-art performance across all reported few-shot experiments.Accuracies are averaged over 1000 randomly generated test episodes.
- Zero-shot learning: For zero-shot learning, the experiments evaluate both the old and new GBU settings, including conventional testing on unseen classes and generalised testing mixing seen and unseen classes.The new GBU setting prevents test classes from appearing in ImageNet 1K pretraining classes.
- Zero-shot learning: Under the conventional old zero-shot setting, the model achieves competitive results on AwA and state-of-the-art performance on CUB using only attribute vectors as class embeddings.The comparison includes 15 alternative approaches.
5. Why does Relation Network Work?
Relation Network works by jointly learning a deep embedding and a deep nonlinear similarity metric end-to-end, avoiding reliance on manually chosen fixed metrics. Synthetic examples show that this deep relation module can capture comparisons that Euclidean, Mahalanobis, and embedding-based alternatives fail to learn.
- 5. Why does Relation Network Work?: Relation Network jointly learns a deep embedding and deep nonlinear metric end-to-end, unlike methods using fixed metrics or shallow learned Mahalanobis metrics.This mutual tuning lets the model learn the representation and similarity function together for few-shot learning.
- 5. Why does Relation Network Work?: A flexible learned similarity function avoids manually choosing Euclidean, cosine, or Mahalanobis distance and relaxes element-wise comparison or post-embedding linear separability assumptions.Prior approaches depend critically on the embedding network because their comparison metric is fixed or limited in form.
- 5. Why does Relation Network Work?: The architecture does not guarantee formal self-similarity or symmetry, although both properties hold numerically in trained Relation Networks.This is an empirical observation rather than an architectural guarantee.
- 5. Why does Relation Network Work?: Only the full deep relation module succeeds on a synthetic matching task where Euclidean nearest-neighbour, Mahalanobis, and two-hidden-layer MLP-plus-Mahalanobis alternatives are inadequate.The example uses 2D query and sample embeddings and demonstrates a relation learnable by Relation Network but not by nonlinear embedding with metric learning.
- 5. Why does Relation Network Work?: Comparing real Omniglot query and sample embeddings remains challenging under standard nearest-neighbour and metric-learning assumptions.The paper illustrates this difficulty by projecting embeddings to 2D and contrasting matching and mismatching samples for example queries.
6. Conclusion
The Relation Network is a simple end-to-end framework that learns embeddings and a deep nonlinear distance metric through episodic training for few-shot and zero-shot learning. It is efficient, achieves state-of-the-art results, and works in both conventional and generalised zero-shot settings.
- Conclusion: The Relation Network learns embeddings and a deep nonlinear distance metric to compare query and sample items.It is trained end-to-end with episodic training to tune both components for effective few-shot learning.
- Conclusion: The method is simpler and more efficient than recent few-shot meta-learning approaches while producing state-of-the-art results.
- Conclusion: The Relation Network is effective in both conventional and generalised zero-shot learning settings.