Source-linked AI summary
Meta-GNN: On Few-shot Node Classification in Graph Meta-learning
Fan Zhou, Chengtai Cao, Kunpeng Zhang, Goce Trajcevski, Ting Zhong, Ji Geng
TL;DR
Few-shot node classification on graphs is difficult because GNNs must adapt to new classes from very few labeled samples, while graph meta-learning remains underexplored. Meta-GNN trains across many similar few-shot tasks to learn GNN parameter initializations that adapt to new classes, and experiments on three benchmark datasets report improved performance and more general task adaptation.
Problem
Existing GNNs struggle with few-shot node classification because new classes require parameter relearning from very few labeled samples, while meta-learning has seen little application to irregular graph data.
Method
Meta-GNN uses episodic meta-learning across many sampled few-shot tasks to learn GNN parameter initializations that quickly adapt to new classes with few labeled samples.
Results
Meta-GNN improves few-shot node-classification performance over state-of-the-art GNNs on three benchmark datasets and adapts well to new learning tasks.
Takeaways & Limitations
The framework provides a general graph meta-learning approach that can be combined with existing GNNs for few-shot node classification.
Abstract
from arXiv · showhide
Meta-learning has received a tremendous recent attention as a possible approach for mimicking human intelligence, i.e., acquiring new knowledge and skills with little or even no demonstration. Most of the existing meta-learning methods are proposed to tackle few-shot learning problems such as image and text, in rather Euclidean domain. However, there are very few works applying meta-learning to non-Euclidean domains, and the recently proposed graph neural networks (GNNs) models do not perform effectively on graph few-shot learning problems. Towards this, we propose a novel graph meta-learning framework -- Meta-GNN -- to tackle the few-shot node classification problem in graph meta-learning settings. It obtains the prior knowledge of classifiers by training on many similar few-shot learning tasks and then classifies the nodes from new classes with only few labeled samples. Additionally, Meta-GNN is a general model that can be straightforwardly incorporated into any existing state-of-the-art GNN. Our experiments conducted on three benchmark datasets demonstrate that our proposed approach not only improves the node classification performance by a large margin on few-shot learning problems in meta-learning paradigm, but also learns a more general and flexible model for task adaption.
1 INTRODUCTION
Few-shot node classification remains difficult for GNNs because new classes require parameter relearning from very few labeled nodes. Meta-GNN addresses this gap by applying meta-learning to graph data and is designed as a general framework for adapting GNNs to unseen classes.
- Research gap: Few-shot node classification challenges GNNs because incorporating new classes requires relearning parameters from very few labeled samples.When each new class has few nodes, existing GNN performance can decline sharply.
- Research gap: Meta-learning learns transferable knowledge across many tasks and adapts to new tasks using only a few samples.The paper distinguishes meta-training on many tasks from meta-testing on a new task.
- Research gap: Meta-learning has progressed on Euclidean data such as images and text, but few studies address irregular, noisy, and relational graph data.These graph properties make direct application of existing meta-learning methods difficult.
- Proposed approach: Meta-GNN is a general graph meta-learning framework for few-shot node classification on graph data, including classes unseen during training.The framework is intended to adapt or generalize to new classes with very few samples.
- Contributions: The framework can be combined with popular GNN models and improves over state-of-the-art GNNs on three benchmark datasets.These are stated contributions of the paper's evaluation and framework design.
2 METHODOLOGY
Meta-GNN frames few-shot node classification as episodic meta-learning over graph tasks, learning parameters that adapt to new classes from a small support set. It combines GNN representations with task-level updates and evaluates adapted parameters on query nodes.
- Graph Neural Networks: The GNN jointly uses graph structure and node features through iterative neighborhood aggregation to produce node representations for downstream classification.After l layers, a node representation captures structural information from its l-hop neighbors; aggregation and combination operations define each layer.
- Meta-Training: Meta-training updates task-adapted parameters from each support set, then optimizes the shared initialization across many tasks.The framework uses one or several gradient-descent steps for each task and performs meta-optimization over the initial parameters rather than the task-specific updated parameters.
- Meta-Testing: At meta-testing, Meta-GNN adapts to a new task using its support nodes and predicts labels for the query set with the adapted parameters.The new task contains unseen classes, and performance is measured after fine-tuning on a few support samples and evaluating on the query set.
- Framework Overview: Figure 1 depicts parameter progression from random initialization through one meta-update to the parameters obtained after all meta-updates across M tasks.The black, red, and blue θ values correspond respectively to randomly initialized, once-updated, and fully meta-updated parameters.
- Task Sampling: Meta-GNN constructs meta-training tasks by sampling classes, support nodes, and query nodes from the training graph.Each task samples |C2| classes, K nodes per class for support, and P remaining nodes for the query set; this process is repeated M times.
3 EXPERIMENTS
Experiments evaluate Meta-GNN on Cora, Citeseer, and Reddit under few-shot node-classification settings, using held-out classes and very small support sets. Meta-GNN improves more strongly as labeled support decreases and performs particularly well on the challenging Reddit dataset, while Meta-SGC and Meta-GCN show dataset-dependent differences.
- Datasets and setup: Experiments use Cora, Citeseer, and Reddit with modified partitions for few-shot node classification.The source code and datasets are publicly available for reproducibility.
- Datasets and setup: Cora and Citeseer hold out two classes for meta-testing, while Reddit holds out five classes because it has more unique labels.
- Datasets and setup: Each support-set class contains only K = 1 or K = 3 samples, and Cora and Citeseer results average accuracy over 50 random support-node selections.For Reddit, the same support nodes are used for each run.
- Results: Meta-GNN's improvement over baselines on Cora and Citeseer increases as the support set becomes smaller.The reported comparison identifies fewer labeled samples as the setting with greater improvement.
- Results: On Reddit, Meta-GNN achieves greater improvement, attributed to learning node representations across more tasks and obtaining a more general task-adaptation model.
- Module comparison: Meta-SGC performs better on Cora, whereas Meta-GCN achieves slightly higher scores on Citeseer and Reddit; SGC-based modules are faster than GCN-based modules.The comparison reports no significant overall discrepancy between the two GNN modules.
4 CONCLUSIONS
The paper concludes that Meta-GNN is a generic framework for few-shot node classification that learns better GNN parameter initialization. It adapts to new tasks and unseen classes with few labeled samples, with encouraging results on three datasets.
- Meta-GNN uses meta-learning to learn better parameter initialization for GNNs in few-shot node classification.
- Meta-GNN adapts to new learning tasks and previously unseen classes with few labeled samples.
- The framework obtains encouraging results on three widely used datasets.