Source-linked AI summary
GraphNAS: Graph Neural Architecture Search with Reinforcement Learning
Yang Gao, Hong Yang, Peng Zhang, Chuan Zhou, Yue Hu
TL;DR
GNN architecture design requires substantial manual tuning and domain knowledge. GraphNAS uses a recurrent controller and reinforcement learning to search variable-length GNN architecture descriptions. Across transductive and inductive node-classification tasks, it performs consistently better on several benchmark networks and can rival the best human-invented architectures.
Problem
Designing GNN architectures requires tuning many parameters through heavy manual work and domain knowledge.
Source: §1 · p. 1Method
GraphNAS uses reinforcement learning with a recurrent controller to search GNN architectures across sampling, aggregation, and gated functions.
Source: §1 · pp. 1–2, §3.2 · p. 3Results
Takeaways & Limitations
GraphNAS can automatically design accurate GNN architectures while reducing the manual effort involved in architecture design.
Source: §1 · p. 2, §5 · p. 6The search does not predict training parameters such as learning rates, and experiments fix child networks to two layers.
Source: §3.2 · p. 3, §4.3 · p. 5
Abstract
from arXiv · showhide
Graph Neural Networks (GNNs) have been popularly used for analyzing non-Euclidean data such as social network data and biological data. Despite their success, the design of graph neural networks requires a lot of manual work and domain knowledge. In this paper, we propose a Graph Neural Architecture Search method (GraphNAS for short) that enables automatic search of the best graph neural architecture based on reinforcement learning. Specifically, GraphNAS first uses a recurrent network to generate variable-length strings that describe the architectures of graph neural networks, and then trains the recurrent network with reinforcement learning to maximize the expected accuracy of the generated architectures on a validation data set. Extensive experimental results on node classification tasks in both transductive and inductive learning settings demonstrate that GraphNAS can achieve consistently better performance on the Cora, Citeseer, Pubmed citation network, and protein-protein interaction network. On node classification tasks, GraphNAS can design a novel network architecture that rivals the best human-invented architecture in terms of test set accuracy.
1 Introduction
GraphNAS addresses the manual, domain-intensive design of GNN architectures by using reinforcement learning to search them automatically. It targets challenges in GNN search spaces, search efficiency, and evaluation across learning settings, with strong results on real-world node-classification benchmarks.
1 Introduction
- The search addresses GNN-specific challenges involving irregular graph structure, large search spaces, slow reinforcement-learning convergence, and transductive versus inductive evaluation.These challenges distinguish graph architecture search from search for CNNs and RNNs.Source: §1 · p. 1
- GraphNAS uses reinforcement learning to automatically search graph neural architectures, reducing the manual work required for GNN design.The method is motivated by the many architecture parameters that otherwise require substantial manual tuning and domain knowledge.Source: §1 · pp. 1–2
- GraphNAS searches a space covering sampling, aggregation, and gated functions, while using parameter sharing and a more efficient architecture-search algorithm.The search space and efficiency strategy are designed to address the first two stated challenges.Source: §1 · p. 2
- Experiments on Cora, Citeseer, Pubmed, and protein-protein networks show consistently better performance in transductive and inductive node classification.The paper reports validation on real-world datasets and both learning settings.Source: §1 · p. 2
- GraphNAS designs architectures that rival the best human-invented architectures in test-set accuracy and F1 score.The authors also release PyTorch code for future comparisons.Source: §1 · p. 2
2 Related work
Prior graph architectures use neighborhood aggregation through sampling, correlation measurement, and information aggregation, while NAS has primarily targeted CNNs and RNNs. GraphNAS applies reinforcement learning to search combinations of these GNN functions instead of setting them manually.
2 Related work
- Spatial graph convolutions aggregate neighbor information and, with sampling, can process batches of nodes instead of an entire graph.This contrasts with spectral methods that typically handle the whole graph simultaneously and are harder to scale.Source: §2 · p. 2
- GNN layers commonly combine neighbor sampling, correlation measurement, and information aggregation within a neighborhood-aggregation scheme.Semi-GCN exemplifies this decomposition with first-order sampling, degree-based correlation, and aggregation.Source: §2 · p. 2
- GraphNAS uses reinforcement learning to search the layer-wise combination of these functions rather than relying on manual settings.This connects established GNN design components to automated architecture search.Source: §2 · p. 2
- Earlier reinforcement-learning NAS generates CNN and RNN architectures with a recurrent controller, while evolution-based NAS can require enormous computational time.The related-work discussion cites 3150 GPU days for one evolution-based result.Source: §2 · p. 2
3 Methods
GraphNAS formulates graph neural architecture search as reinforcement learning over a structured GNN search space, using validation accuracy to train a recurrent controller. It combines policy-gradient optimization with conditional parameter sharing to make architecture search more efficient.
3.1 Problem formulation
- GraphNAS searches for the GNN architecture that maximizes expected validation accuracy by sampling feasible architectures with a recurrent controller.The controller generates architecture descriptions, while generated GNN validation results provide its feedback.Source: §3.1 · pp. 2–3
3.2 Search Space
- The search space represents each GNN layer through feature transformation, sampling, correlation measurement, aggregation, residual, and gated functions.Architectures are emitted as variable-length token sequences covering choices such as SAM, ATT, AGG, K, DIM, I, and ACT.Source: §3.2 · p. 3
3.3 Search Algorithm
- GraphNAS trains its controller with policy gradients using converged child-model accuracy as a non-differentiable reward and a moving-average baseline to reduce variance.Shared parameters are optimized separately with stochastic gradient descent on node-classification cross-entropy loss.Source: §3.3 · pp. 3–4
3.4 Parameters Sharing
- The method shares transform weights only for specific attention–aggregation combinations rather than indiscriminately across different GNNs.This conditional strategy addresses the finding that unconditional sharing does not work immediately between different GNN architectures.Source: §3.4 · p. 4, Figure 2 · p. 4
- Shared parameters are merged after a child model receives positive reward, while rewards are computed after training child models and smoothed with a moving average.Exploration trains shared parameters on novel structures with the controller fixed, and final architecture derivation retrains only the highest-reward sampled model from scratch.Source: §3.4 · p. 4
4 Experiments
GraphNAS is evaluated on citation-network and protein-protein interaction node-classification tasks using task-appropriate baselines, search spaces, and training settings. Across transductive and inductive experiments, it finds architectures matching strong existing models and improves over random or alternative search strategies.
4.2 Baseline Methods
- The experiments use accuracy for single-label transductive classification and F1 score for multi-label inductive classification.Citation datasets use fixed two-layer searches, whereas PPI searches use three-layer architectures.Source: §4.2 · p. 5, §4.3 · p. 5, §4.4 · p. 6
4.3 Architecture on Transductive learning
- GraphNAS designs architectures that perform as well as strong existing models on Cora, Citeseer, Pubmed, and PPI node-classification tasks.The transductive results are summarized in Tables 3 and 4, while PPI results are summarized in Table 5.Source: §4.3 · p. 5, Table 3 · p. 5, §4.4 · p. 6
- GraphNAS finds better GNNs than random search during transductive architecture search, measured by the number of validation models exceeding 0.81 accuracy.Figure 3 compares the number of qualifying models found by GraphNAS and random search.Source: §4.3 · p. 5, Figure 3 · p. 5
4.4 Architecture on Inductive Learning
- On PPI, GraphNAS searches over three-layer architectures with parameter sharing and finds models that perform as well as the best existing models.The controller is trained with shared child-model parameters, and the selected model is evaluated using micro-F1 and per-epoch training time.Source: §4.4 · p. 6, Table 5 · p. 6
- GraphNAS outperforms random search, NAS-like GraphNAS, and ENAS-like GraphNAS in the PPI search-strategy comparison.Figure 5 compares validation F1-score during search, while Table 5 reports the resulting model performance.Source: §4.4 · p. 6, Figure 5 · p. 6, Table 5 · p. 6
5 Conclusions
The paper introduces GraphNAS for reinforcement-learning-based graph neural architecture search. Experiments across transductive and inductive node-classification settings show consistently better performance on citation and protein-protein interaction networks, with architectures rivaling the best human-invented models.
5 Conclusions
- GraphNAS uses reinforcement learning to design accurate GNN architectures that rival the best human-invented architectures.The method is evaluated on both transductive and inductive node-classification tasks.Source: §5 · p. 6