Source-linked AI summary
Edge-labeling Graph Neural Network for Few-shot Learning
Jongmin Kim, Taesup Kim, Sungwoong Kim, Chang D. Yoo
TL;DR
Few-shot classification must operate with limited labeled support data, while prior few-shot GNNs mainly model relationships through node-labeling. The paper introduces EGNN, which iteratively updates edge-labels to explicitly model clustering and can accommodate different class counts without retraining; it outperforms existing GNNs across supervised and semi-supervised benchmark tasks.
Problem
Few-shot classification must learn to classify queries from compact support sets whose limited labels do not fully reflect inter- and intra-class variation.
Method
EGNN alternates node and edge feature updates, uses an edge-labeling loss with episodic training, and explicitly exploits intra-cluster similarity and inter-cluster dissimilarity.
Results
EGNN significantly outperformed existing few-shot GNNs on supervised and semi-supervised image classification tasks across two benchmark datasets.
Takeaways & Limitations
The edge-labeling framework supports few-shot inference with varying numbers of classes without retraining and can be extended to transductive inference.
Takeaways & Limitations
The experiments excluded ResNet-based models for fair comparison.
Abstract
from arXiv · showhide
In this paper, we propose a novel edge-labeling graph neural network (EGNN), which adapts a deep neural network on the edge-labeling graph, for few-shot learning. The previous graph neural network (GNN) approaches in few-shot learning have been based on the node-labeling framework, which implicitly models the intra-cluster similarity and the inter-cluster dissimilarity. In contrast, the proposed EGNN learns to predict the edge-labels rather than the node-labels on the graph that enables the evolution of an explicit clustering by iteratively updating the edge-labels with direct exploitation of both intra-cluster similarity and the inter-cluster dissimilarity. It is also well suited for performing on various numbers of classes without retraining, and can be easily extended to perform a transductive inference. The parameters of the EGNN are learned by episodic training with an edge-labeling loss to obtain a well-generalizable model for unseen low-data problem. On both of the supervised and semi-supervised few-shot image classification tasks with two benchmark datasets, the proposed EGNN significantly improves the performances over the existing GNNs.
1. Introduction
The paper proposes EGNN, which predicts and iteratively updates edge-labels to explicitly exploit intra-cluster similarity and inter-cluster dissimilarity in few-shot learning. It supports varying class counts without retraining, transductive and non-transductive inference, and improves benchmark performance over existing GNNs.
- Few-shot learning targets new tasks using few labeled examples and knowledge acquired from previous experiences.
- Previous few-shot GNNs mainly use node-labeling, implicitly modeling intra-cluster similarity and inter-cluster dissimilarity.
- EGNN predicts edge-labels and iteratively updates edge features to explicitly exploit intra-cluster similarity and inter-cluster dissimilarity.Its layers alternate node-update and edge-update blocks, with final edge features producing edge-label predictions.
- EGNN is suited to various numbers of classes without retraining and supports both transductive and non-transductive inference.
- EGNN significantly improves performance over existing GNNs on supervised and semi-supervised few-shot image classification across two benchmark datasets.
2. Related works
Related work spans graph neural networks, edge-labeling and correlation clustering, representation-based few-shot classification, and meta-learners that transfer knowledge across tasks.
- Graph neural networks process graph-structured data through neighborhood aggregation, recursively transforming and aggregating neighboring node features.
- Edge-labeling and correlation-clustering methods infer relationships or partitions by exploiting graph edge information, including intra-cluster similarity and inter-cluster dissimilarity.
- Representation-based few-shot image classifiers commonly predict with nearest-neighbor similarity, using distances or learned representations.
- Meta-learners optimize model parameters or learning strategies to extract transferable knowledge across tasks for few-shot learning.
3. Method
The method defines few-shot tasks with labeled support and unlabeled query samples, then applies EGNN to jointly update node and edge features. Final edge predictions support weighted-vote classification using explicitly separated intra- and inter-class relations.
- Problem definition: Few-shot classification: Few-shot tasks contain a support set with K labeled samples per class and an unlabeled query set, forming an N-way K-shot problem.Episodic training samples tasks that mimic test-time few-shot settings and optimizes query predictions across episodes.
- Model: EGNN constructs a fully connected graph whose nodes are task samples and whose edges represent pairwise relationships with ground-truth labels derived from node labels.Node features come from a convolutional embedding network, while initial edge features encode labeled support relationships and uncertain query relationships.
- Model: Edge features separately encode normalized intra-cluster similarity and inter-cluster dissimilarity, allowing each update to use relations among the corresponding and other node pairs.The metric network computes similarity scores, with optional separate networks for similarity and dissimilarity.
- Model: Each EGNN layer alternates node-feature and edge-feature updates, using edge features to weight neighborhood aggregation before recomputing pairwise similarities and dissimilarities.The node transformation includes both intra-class aggregation from similar neighbors and inter-class aggregation from dissimilar neighbors.
- Model: After L updates, the final edge feature predicts whether two nodes share a class, and each node is classified by weighted voting over labeled support samples.The paper uses the voting formulation directly rather than first partitioning the entire graph through an optimization-based clustering procedure.
- Training: Training minimizes binary cross-entropy edge losses from all layers, combining intermediate predictions to improve gradient flow into lower layers.The EGNN parameters are trained end-to-end under episodic training.
4. Experiments
EGNN was evaluated on miniImageNet and tieredImageNet under supervised, semi-supervised, transductive, and non-transductive few-shot settings. Results and ablations examine benchmark performance, propagation depth, explicit similarity separation, cross-way flexibility, and feature evolution.
- Experimental setup: Experiments used miniImageNet and tieredImageNet, with standard 5-way 5-shot evaluation averaged over 600 randomly generated test episodes.A more challenging 10-way experiment was also conducted on miniImageNet.
- Few-shot classification: EGNN achieved the best 5-way 5-shot performance on miniImageNet under both transductive and non-transductive settings, outperforming the node-labeling GNN.EGNN with transduction also outperformed TPN on both datasets.
- Semi-supervised few-shot classification: 61.88% vs 52.45% when 20% of support samples were labeled: EGNN outperformed the previous GNN in semi-supervised miniImageNet learning.Semi-supervised learning improved performance over labeled-only learning across all tested labeling proportions, with further gains under transduction.
- Ablation studies: 67.99% →73.19%: few-shot accuracy increased when EGNN depth changed from one to two layers, reaching 76.37% with three layers.The ablation attributes the comparison to the number of EGNN layers and reports progressively better performance as depth increases.
- Ablation studies: Separate use of intra-cluster similarity and inter-cluster dissimilarity clearly improved performance across all tested EGNN layer counts.The comparison used a model with only intra-cluster aggregation versus one using both aggregations.
- Ablation studies: Cross-way experiments trained on 5-way 5-shot and tested on 10-way 5-shot, and vice versa, while the node-labeling model was not applicable because its parameters depend on the number of ways.The EGNN’s feature visualizations also showed differently labeled samples separating and equally labeled query-support samples approaching across propagation layers; edge features evolved toward ground-truth labels from 0.5 initialization.
5. Conclusion
The paper concludes that EGNN performs alternating node and edge updates to infer query associations with support clusters through explicit similarity and dissimilarity modeling. Experiments show improved performance across supervised and semi-supervised few-shot image classification tasks.
- Conclusion: EGNN alternates node-feature and edge-feature updates, obtaining edge-label predictions from the final edge features.Its parameters are trained episodically with an edge-labeling loss.
- Conclusion: EGNN explicitly models intra-cluster similarity and inter-cluster dissimilarity while inferring query associations with existing support clusters.The edge-labeling framework represents whether connected nodes belong to the same class cluster.
- Conclusion: Figure 4 compares node-feature t-SNE visualizations for GNN and EGNN across the initial embedding and three propagation layers.Rows distinguish GNN from EGNN, while columns show successive representations; marker shapes identify query and support samples, and colors identify labels.
- Conclusion: Figure 5 visualizes edge-feature propagation from the initial feature through two layers and the ground-truth edge labels in a 5-way 3-shot task.Red denotes higher edge values and blue denotes lower values; the task contains 15 support and 15 query samples.
- Conclusion: EGNN outperformed other few-shot learning algorithms on supervised and semi-supervised few-shot image classification tasks.The conclusion reports this result across both evaluation settings.