Source-linked AI summary
Few-Shot Learning with Graph Neural Networks
Victor Garcia, Joan Bruna
TL;DR
Few-shot learning requires inference from partially labeled image collections rather than ordinary iid image-label samples. The paper formulates this problem as graph-based message passing, achieving state-of-the-art few-shot performance with fewer parameters and extending the framework to semi-supervised and active learning.
Problem
Few-shot learning asks how to learn from partially labeled image collections when ordinary supervised learning does not cover settings where learning remains possible and desirable.
Method
The paper represents image collections as graphs and uses an end-to-end graph neural network that performs task-driven message passing over trainable image similarities.
Results
The model matches state-of-the-art performance on Omniglot and Mini-Imagenet with fewer parameters and extends to semi-supervised and active learning.
Takeaways & Limitations
A graph formulation unifies few-shot, semi-supervised, and active learning under one relational message-passing framework.
Takeaways & Limitations
The formulation assumes tasks are drawn from a distribution of partially labeled image collections, with target labels associated with designated images.
Abstract
from arXiv · showhide
We propose to study the problem of few-shot learning with the prism of inference on a partially observed graphical model, constructed from a collection of input images whose label can be either observed or not. By assimilating generic message-passing inference algorithms with their neural-network counterparts, we define a graph neural network architecture that generalizes several of the recently proposed few-shot learning models. Besides providing improved numerical performance, our framework is easily extended to variants of few-shot learning, such as semi-supervised or active learning, demonstrating the ability of graph-based models to operate well on 'relational' tasks.
1 INTRODUCTION
The paper frames few-shot learning as meta-learning over collections of images and proposes a graph neural network that propagates label information through trainable similarities. The same framework extends to semi-supervised and active learning while matching state-of-the-art few-shot performance with fewer parameters.
- Motivation: Few-shot learning addresses learning from few examples by leveraging a distribution of similar tasks rather than relying only on regularization.Meta-learning treats collections of images and their label similarities as the input-output examples.
- Related approach: Existing few-shot image-classification methods learn contextual, task-specific similarities by embedding images and propagating label information toward a target.These approaches combine embedded images within each collection, including through attention mechanisms.
- Graph formulation: The proposed model casts few-shot learning as supervised interpolation on a graph whose nodes are images and whose edges are trainable similarity kernels.Its task-driven message passing architecture is trained end-to-end and captures permutation invariance within input collections.
- Extensions: The graph architecture extends naturally to semi-supervised and active learning with minimal changes in the training design.In active learning, the learner can request missing labels that are most helpful for prediction.
- Empirical scope: The model matches state-of-the-art performance on Omniglot and Mini-Imagenet tasks with fewer parameters.The paper reports numerical experiments across few-shot, semi-supervised, and active learning settings.
2 RELATED WORK
The related-work discussion situates the paper among one-shot meta-learning methods and graph neural network architectures. Prior work spans learned distances, contextual attention, adaptive optimization, and recurrent or gated message passing on graphs.
- One-shot learning: Early one-shot learning assumes that classes learned previously can support predictions for new classes when only one or a few labels are available.A hierarchical Bayesian model later reached human-level error on few-shot alphabet recognition tasks.
- Similarity-based meta-learning: Siamese networks learn pairwise distances for k-nearest-neighbors classification, while attention-based models incorporate the full sample subset when computing similarities.These methods established end-to-end learned similarity and contextual mechanisms for one-shot prediction.
- Meta-learning variants: Other meta-learners update classifier weights with an LSTM, change inductive bias through fast parameterization, or optimize models via gradient descent across tasks.These approaches learn task-level knowledge rather than only a fixed image representation.
- Graph neural networks: Graph neural networks originated as trainable recurrent message-passing systems and later incorporated untied weights and gated nonlinear updates.This line of work established neural architectures for learning on graph-structured data.
- Applications: Graph neural architectures have been applied to particle interactions, molecular fingerprints, and molecular prediction.These applications demonstrate the broader use of graph-based representations beyond few-shot image learning.
3 PROBLEM SET-UP
The paper models each task as a partially labeled collection of images with labeled, unlabeled, and target examples drawn from class-specific distributions. Few-shot, semi-supervised, and active learning arise by varying which elements are labeled, auxiliary, or queried.
- General setup: The general setup consists of input-output pairs of partially labeled image collections drawn independently from a task distribution.The model receives collections containing labeled samples, optional unlabeled samples, and target samples to classify.
- Notation: The collection contains s labeled samples, r unlabeled samples, and t target samples across K classes, with the paper focusing on t = 1.The formulation allows arbitrary s, r, t, and K, while r > 0 identifies semi-supervised and active-learning scenarios.
- Objective: The learning objective uses a model Φ(T ; Θ) = p(Y | T) together with a standard regularization objective.Training is defined over a set of task collections and their target labels.
- Few-shot learning: Few-shot learning is the special case r = 0, t = 1, and s = qK, where each label appears exactly q times.This setting is called q-shot, K-way learning and contains one image with an unknown label.
- Semi-supervised learning: Semi-supervised learning adds auxiliary unlabeled images drawn from the same class distributions as the output samples.These images can be used to improve prediction accuracy without observed labels.
- Active learning: Active learning allows the learner to request labels for auxiliary samples and evaluates whether this improves over the semi-supervised setup.The comparison considers matching a one-shot setting with more known labels when s + r = s0 and s is much smaller than s0.
4 MODEL
The model represents few-shot learning as posterior inference on a fully connected graph of labeled and unlabeled images. It learns image relationships and propagates label information through graph neural network message passing, while encompassing several existing few-shot architectures.
- Set and graph input representations: Few-shot learning is formulated as posterior inference that propagates labels from known samples to an unlabeled query image.The input collection contains both labeled and unlabeled images, and the graph is determined by these images and labels.
- Set and graph input representations: The image collection is represented by a fully connected graph whose nodes are images and whose edge similarities are learned discriminatively.The learned metric uses a neural model rather than a fixed similarity between images.
- Graph neural networks: GNN layers apply local graph operators to node features and produce updated node representations through trainable parameters and point-wise nonlinearities.The architecture can use adjacency-based operators and leaky ReLU nonlinearities, while variants include gating and multi-hop operators.
- Graph neural networks: The model learns a symmetric node-pair metric by applying an MLP to the absolute difference between node features, then normalizes the learned adjacency row-wise with a softmax.This learned adjacency is important when the input has geometric structure but its metric is unknown beforehand.
- Set and graph input representations: Known-label nodes concatenate one-hot labels with image embeddings, while unknown-label nodes use a uniform distribution over the K-simplex.This initialization explicitly represents label uncertainty for unlabeled images.
- Relationship with existing models: The GNN formulation generalizes Siamese, Prototypical, and Matching Networks by interpreting them as related message-passing or attention mechanisms.Siamese Networks correspond to a single message-passing iteration, Prototypical Networks aggregate within same-label clusters, and Matching Networks use a set representation with a different attention pattern.
5 TRAINING
The paper trains the GNN for few-shot, semi-supervised, and active learning settings using node-level label prediction. Active learning selects an informative unlabeled node, incorporates its queried label, and trains the selection mechanism end-to-end.
- Few-shot and semi-supervised learning: Training covers few-shot, semi-supervised, and active learning by adapting the GNN's node-label prediction setup.In few-shot learning, the final layer predicts the query node's label with a softmax and cross-entropy loss.
- Few-shot and semi-supervised learning: Semi-supervised learning differs by initializing unlabeled nodes with uniform label distributions while otherwise using the same training procedure.The uniform distributions represent uncertainty in the initial label fields.
- Active learning: Active learning queries one label from the unlabeled nodes after the first GNN layer using a Softmax attention mechanism.A two-layer neural network maps each unlabeled node representation to a scalar used to form the attention distribution.
- Active learning: The active-learning mechanism selects one node by retaining the maximum attention value at test time and sampling from the multinomial distribution during training.The selected attention is multiplied by the label vectors before the queried label is added to the node representation.
- Active learning: The queried label is forward propagated through the network, and the attention mechanism is trained end-to-end by backpropagating the output loss.This couples label acquisition with the rest of the GNN training process.
6 EXPERIMENTS
Experiments evaluate the graph neural network on few-shot, semi-supervised, and active learning tasks using Omniglot and Mini-Imagenet. The model matches competitive few-shot performance with fewer parameters, benefits from unlabeled samples, and improves active selection over random labeling.
- Experimental setup: The experiments use Omniglot and Mini-Imagenet in q-shot, K-way settings, with matching q-shot and K-way values during training and testing.Few-shot tasks sample K classes, q examples per class, and an additional query sample.
- Few-shot learning: Omniglot results are competitive, reach state-of-the-art performance in 5-Way and 20-Way 1-shot settings, and reduce parameters from ∼5M to ∼300K versus TCML.The GNN is slightly better than Munkhdalai & Yu (2017) in 20-Way 1-shot, while TCML is slightly better in 20-Way 5-shot.
- Few-shot learning: 66.41% versus 64.02% shows the improvement from aggregating node information with the full GNN over the metric-learning-plus-KNN baseline in 5-shot 5-Way Mini-Imagenet.The GNN uses a simple four-convolutional-layer embedding for this comparison; its parameter count is ∼400K versus ∼11M for TCML.
- Semi-supervised learning: In Omniglot semi-supervised learning, using 20% of the labels in a 5-shot setting achieves the same results as the 40% supervised setting.The authors report that the GNN extracts information from the distribution of unlabeled samples.
- Semi-supervised learning: Mini-Imagenet semi-supervised learning improves by ∼2% in both the 20% and 40% labeled settings, although gains are less significant than on Omniglot.The authors attribute the smaller improvement context to Mini-Imagenet's more complex image distribution.
- Active learning: The learned GNN-AL criterion improves Mini-Imagenet accuracy by ∼3.4% over random selection, while random labeling practically does not improve accuracy.The learned criterion selects a more informative sample than a random one; the Omniglot improvement is smaller because accuracy is nearly saturated.
7 CONCLUSIONS
The paper presents graph neural representations as a shared framework for few-shot, semi-supervised, and active learning. It identifies scaling graph models to millions of nodes as a future requirement and points to broader active-learning and reinforcement-learning extensions.
- Graph neural representations frame few-shot, semi-supervised, and active learning as supervised learning over collections whose relational structure supports neural message passing.
- Stacked node and edge features generalize the contextual similarity learning used by previous few-shot learning models.
- The graph formulation unifies few-shot, active, and semi-supervised training setups under one framework for learners operating across different label regimes.
- Scaling to millions of nodes is identified as a requirement motivating graph hierarchical and coarsening approaches.
- Future extensions include broader active-learning queries and reinforcement-learning settings involving adaptation to non-stationary environments.
APPENDIX
Figure 3 presents the GNN model configuration used for Omniglot and Mini-Imagenet, with three blue blocks and n_f=96.
- Figure 3 presents the GNN model.
- Three blue blocks are used for Omniglot and Mini-Imagenet.
- The model configuration uses n_f=96.