PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross-Domain Few-Shot Graph Classification

Kaveh Hassani

arXiv:2201.08265v1cs.LGcs.NE

TL;DR

Few-shot graph classification across domains must handle domain shifts and non-equivalent feature spaces, unlike settings that assume equivalent features. The paper introduces three benchmarks and an attention-based encoder over contextual and topological graph views; paired with metric-based meta-learning, it achieves the best average meta-test accuracy across all three benchmarks.

  • Problem

    Cross-domain few-shot graph learning must address domain shifts and, for graphs, feature spaces that may be non-equivalent, whereas standard cross-domain assumptions often require equivalent features.

    Source: Introduction · p. 1, Problem Formulation · p. 2

  • Method

    The paper introduces three benchmarks and an attention-based graph encoder that uses one contextual and two topological views to represent task-specific and task-agnostic information.

    Source: Introduction · p. 1, Encoders · p. 3

  • Results

    Metric-based meta-learning coupled with the proposed encoder achieves the best average meta-testing classification accuracy across all three benchmarks.

    Source: Introduction · p. 1, Conclusion · p. 7

  • Takeaways & Limitations

    Across all three benchmarks, meta-learning and contrastive approaches outperform naive classifiers, indicating transferable knowledge across the evaluated tasks.

    Source: Experimental Results · p. 7

    Tasks may come from distinct domains with non-equivalent feature spaces, and target tasks are seen only during meta-testing rather than training.

    Source: Problem Formulation · pp. 2–3

Abstract

from arXiv · show

We study the problem of few-shot graph classification across domains with nonequivalent feature spaces by introducing three new cross-domain benchmarks constructed from publicly available datasets. We also propose an attention-based graph encoder that uses three congruent views of graphs, one contextual and two topological views, to learn representations of task-specific information for fast adaptation, and task-agnostic information for knowledge transfer. We run exhaustive experiments to evaluate the performance of contrastive and meta-learning strategies. We show that when coupled with metric-based meta-learning frameworks, the proposed encoder achieves the best average meta-test classification accuracy across all benchmarks. The source code and data will be released here: https://github.com/kavehhassani/metagrl

Introduction

Cross-domain few-shot graph classification addresses settings where source and target tasks differ, including in feature spaces, while graph labels can be costly to obtain. The paper introduces benchmarks and a multi-view encoder; with metric-based meta-learning, the encoder achieves the best average meta-test accuracy across all three benchmarks.

Introduction

  • Prior meta-learning assumes meta-training and meta-testing tasks share a distribution, but real applications may instead provide datasets from different domains.With large source-target shifts, meta-learning has been reported to underperform pretraining and fine-tuning.Source: Introduction · p. 1
  • Graph datasets commonly differ in feature spaces as well as distribution shifts, while labeling graphs can require costly experiments or domain expertise.Molecular datasets may use different feature sets to represent molecules.Source: Introduction · p. 1
  • The paper introduces three benchmarks for cross-domain few-shot graph classification and evaluates supervised, contrastive, and meta-learning strategies.Source: Introduction · p. 1
  • The proposed encoder attends to one contextual and two topological graph views to represent task-specific information for fast adaptation and task-agnostic information for transfer.The paper presents the encoder as a task-conditioned approach to attending to different task representations.Source: Introduction · p. 1
  • Metric-based meta-learning with the proposed encoder achieves the best average meta-testing classification accuracy across all three benchmarks.The benchmarks evaluate supervised, contrastive, and meta-learning strategies.Source: Introduction · p. 1

Related Work

Related work covers graph neural networks, meta-learning for graph tasks, and domain adaptation and generalization. The paper distinguishes its setting by addressing few-shot graph classification across domains with non-equivalent feature spaces.

Related Work

  • Graph neural networks iteratively transfer, transform, and aggregate representations from topological neighbors, then summarize node representations into a graph representation.Source: Related Work · p. 2
  • Prior graph meta-learning studies address few-shot node classification, link prediction, or graph classification, but assume tasks are identically distributed; this work targets cross-domain graph classification.MetaSpecGraph uses super-classes built by clustering graphs with normalized-Laplacian spectra.Source: Related Work · p. 2
  • Domain adaptation reduces distribution shift between domains with equivalent classes, often through adversarial methods that require unlabeled target samples during training.Source: Related Work · p. 2
  • Domain generalization targets unseen domains without target-distribution knowledge during training; prior work reports that meta-learning can underperform fine-tuning or even random-weight networks on cross-domain few-shot benchmarks.Source: Related Work · p. 2
  • Domain-generalization strategies include adversarial augmentation, domain-invariant features, and learning-to-learn, while prior cross-domain few-shot methods include FWT and MetaNorm.The paper combines task-specific domain-invariant features and inductive normalization layers for graph classification.Source: Related Work · p. 2

Problem Formulation

The formulation places source tasks in meta-training and target tasks in meta-testing, with domain and task distributions that may differ. It extends cross-domain few-shot learning to graph tasks with potentially distinct domains and non-equivalent feature spaces.

Problem Formulation

  • A domain is defined by a joint distribution over feature and label spaces, and each k-way task uses support and query datasets with n and m examples per category, respectively.Source: Problem Formulation · p. 2
  • The source domain is available during meta-training, while the target domain appears only at meta-testing; task distributions differ between the phases.Learning in this cross-domain setting is described as more difficult than traditional transductive meta-learning, where source and target domains and task distributions match.Source: Problem Formulation · p. 2
  • Unlike the usual cross-domain assumption of equivalent feature spaces, graph datasets may use different feature spaces, such as molecular datasets with different node or edge attributes.Source: Problem Formulation · p. 2
  • Each task may come from its own domain, and tasks can have non-equivalent feature spaces, differing distributions, and disjoint label spaces.Tasks can also be grouped into meta-domains that capture conceptual domains such as social networks.Source: Problem Formulation · p. 3

Method

The method separates task-specific node-feature context from task-agnostic graph topology, then uses task-conditioned attention to combine their representations. Context supports fast adaptation, while topology can support transfer across tasks.

Method

  • Task-conditioned attention combines one contextual and two topological graph views, with context supporting fast adaptation and topology serving as an anchor for cross-domain transfer.The encoders and attention are jointly learned through meta-learning.Source: Method · p. 3

Augmentations

  • Contextual augmentation transforms initial node features; among heterogeneous augmentation, Deep Sets, and padding, the authors report simple padding performed best and speculate tasks lack overlapping features.Source: Augmentations · p. 3
  • Topological augmentation uses sinusoidal degree encodings and graph subsampling, plus diffusion eigenvalues; the authors report diffusion outperformed heat-kernel, Laplacian, and shortest-path alternatives.Source: Augmentations · p. 3

Encoders

  • Dedicated GIN encoders process contextual node features and degree encodings, while an MLP projects sorted diffusion eigenvalues; mean pooling yields the two graph-level GNN representations.Source: Encoders · pp. 3–4
  • A single-layer MLP and softmax produce attention scores from the three representations, which weight them into a final graph representation.Source: Encoders · p. 4
  • The attention mechanism can favor context for tasks similar to seen tasks and geometry after a drastic feature-space shift, with target domains unavailable during training.Source: Encoders · p. 4

Benchmarks

The authors introduce three fixed train/validation/test benchmarks from public graph datasets, using molecule tasks as the source meta-domain and molecule, bioinformatics, and social-network tasks as targets. The benchmark construction specifies graph filters, task splits, and few-shot support and query sets.

Benchmarks

  • Three benchmarks use molecule classification tasks as the source meta-domain and molecule, bioinformatics, and social-network tasks as targets; the first benchmark's source and target tasks differ in feature and class spaces.Source: Benchmarks · pp. 4–5
  • Graphs are filtered by degree, feature dimension, size, and connectivity; oversized graphs are reduced to 500 nodes, and multi-task datasets are split into single-task datasets without shared samples.Source: Benchmarks · p. 5
  • The benchmarks use k-shot 2-way tasks with 20 support and 50 query samples per class; tasks from the same dataset are kept in one split.Source: Benchmarks · p. 5
  • Bioinformatics and social-network datasets serve as meta-testing tasks in benchmarks two and three, while benchmark one groups tasks originating from the same dataset into one split.Source: Benchmarks · p. 5

Experimental Results

The experiments find that the proposed encoder improves metric-based meta-learning across benchmarks, while view ablations, transfer strategies, and task-conditioning tests characterize how the approach behaves. The reported gains are strongest for molecular-to-social-network transfer, where target tasks lack initial node features.

Experimental Results

  • With the encoder, the best 1-shot meta-learning methods exceed regular meta-learning by 3.28%, 4.29%, and 5.17% absolute accuracy on molecule, bioinformatics, and social-network benchmarks, respectively.RelationNet with the encoder and 20 examples is 4.46%, 6.96%, and 2.68% less accurate than fully supervised models on those benchmarks, respectively.Source: Table 2 · p. 6, Table 3 · p. 6, Experimental Results · p. 7

Effect of Views.

  • Both topological views contribute complementary performance gains, with diffusion having a slightly larger effect; the encoder performs better with metric-based than optimization-based methods.The authors speculate that diffusion’s global-information encoding provides an inductive bias.Source: Effect of Views · p. 7

Why topological properties are task-agnostic?

  • Sinusoidal degree encodings, graph subsampling, and interleaved FWT and GNN layers address cross-domain degree shifts; diffusion encodings are less sensitive to graph size and degree-distribution shifts.The approach first relies on topological signals to relax heterogeneity, then uses these techniques to control distribution shifts.Source: Why topological properties are… · p. 7

Effect of Meta-Learning Approach.

  • Without the encoder, metric-based methods are competitive with contrastive methods and show some transfer, while optimization-based methods are outperformed by independent classifiers; with the encoder, optimization-based methods improve but still trail metric-based methods.The authors suggest metric-based methods can propagate scarce labels through the representation-label inductive bias.Source: Effect of Meta-Learning Approach · p. 7

Effect of Task Encoding.

  • Conditioning attention on support-set task encodings and the top three similar tasks produces results nearly identical to leaving attention unconditioned.The task encodings were pretrained with MVGRL and stored in a memory bank for retrieval during meta-testing.Source: Effect of Task Encoding · p. 7

Conclusion

The conclusion presents three benchmarks and a multi-view attention encoder for cross-domain few-shot graph classification. It reports the best performance across the benchmarks for metric-based meta-learning paired with the encoder, while noting that optimization-based methods struggle in this setting.

Conclusion

  • The paper introduces three benchmarks for cross-domain few-shot graph classification and evaluates independent classifiers, contrastive methods, and meta-learning strategies.It also presents a multi-view graph encoder with attention-based aggregation for knowledge transfer and adaptation.Source: Conclusion · p. 7
  • Metric-based meta-learning paired with the proposed encoder achieves the best performance across all benchmarks.Source: Conclusion · p. 7
  • The conclusion reports that optimization-based meta-learning methods struggle in the cross-domain setting and identifies black-box and hybrid adaptations as future work.Source: Conclusion · p. 7

Datasets, Pre-processing, & Implementation

The experimental setup uses molecules as the meta-training domain and a k-way 2-shot task setting, alongside specified feature initialization and training infrastructure.

Datasets, Pre-processing, & Implementation

  • Because molecule datasets are frequent and most datasets are binary-classification problems, molecules serve as the meta-training domain in a k-way 2-shot setting.The original dataset statistics distinguish multi-task status, extracted task counts, and classification categories.Source: Datasets, Pre-processing, &… · p. 10
  • Datasets without initial node features use one-hot node-label features when labels exist; otherwise, each node starts with a 16-dimensional vector of ones, followed by row-wise normalization.Source: Datasets, Pre-processing, &… · p. 10
  • Experiments use PyTorch, PyTorch Geometric, and Learn2Learn on four RTX 6000 GPUs; an epoch takes about 45 seconds for optimization-based methods and 6 seconds for metric-based methods.Source: Datasets, Pre-processing, &… · p. 10

Hyper-Parameters

The experiments fix core model and training settings while selecting task, adaptation, and architecture parameters from specified candidate sets.

Hyper-Parameters

  • All experiments use 1,000 epochs, hidden dimension 256, 32-dimensional sinusoidal degree embeddings, 128 diffusion eigenvalues, and meta-batches of 16 tasks.Training uses Adam, cosine learning-rate scheduling, and early stopping with patience 30.Source: Hyper-Parameters · p. 10
  • Task steps and adaptation steps are selected from [10, 25, 50], while task learning rates use [0.001, 0.01] and adaptation rates use [0.01, 0.1].The numbers of GNN and MLP layers are selected from [1, 2, 3].Source: Hyper-Parameters · p. 10
  • Models use dropout probability 0.6 and feature-wise transforms after linear layers; the diffusion teleport probability is 0.2.MAML and MetaSGD are tested with first- and second-order optimization settings.Source: Hyper-Parameters · p. 10

Effect of Views & Encoder

The proposed encoder’s two geometric views contribute complementary performance enhancements, and it performs better with metric-based than optimization-based meta-learning methods. The section also describes how the views and graph diffusion are constructed.

Effect of Views & Encoder

  • The view comparison tests initial node features X alone, X with degree features U, X with diffusion eigenvalues Z, and all three together.Settings 2–4 add the proposed encoder to the meta-learning algorithms.Source: Effect of Views & Encoder · p. 10
  • Both geometric views contribute complementary performance enhancements, and the encoder performs better with metric-based than optimization-based meta-learning methods.The comparisons use Tables 6–8, but the supplied passages do not include their cell values.Source: Effect of Views & Encoder · p. 10

subsection

  • Graph diffusion is a weighted sum of powers of a generalized transition matrix; its weighting coefficient controls the ratio of global to local information.Personalized PageRank and the heat kernel are described as two instantiations of generalized graph diffusion.Source: subsection · p. 10
  • The supplied dataset tables cover molecule, bioinformatics, and social-network meta-domains; a separate table lists selected metric-based meta-learning hyperparameters for the 20-shot setting.Source: Table 4 · p. 11, Table 5 · p. 11, Table 6 · p. 12
  • Tables 10 and 11 report mean and standard deviation of meta-test accuracy on the bioinformatics and social-network benchmarks, respectively, after ten runs.Source: Table 10 · p. 13, Table 11 · p. 14
Loading 2201.08265v1…