PaperlayerView exact revision on arXiv

Source-linked AI summary

Node-level Graph Neural Architecture Search Framework

Lintao Yanga, Sirui Lia, Yaqing Wang, Pietro Liò, Xu Shen, Baisong Liu, Chengbin Peng

arXiv:2610.09297v1cs.LGcs.AI

TL;DR

Uniform graph operations may overlook node diversity and contribute to over-smoothing, while manual architecture design can require substantial expertise and computation. N-GNAS searches for architectures for node subsets and adds a contrastive loss. The paper reports that it outperforms conventional GNAS and manually crafted GNNs across diverse graph-learning tasks.

  • Problem

    Traditional GNAS applies uniform operations despite node diversity, and architecture optimization can require substantial expertise and computational investment.

    Source: §1 · p. 1

  • Method

    N-GNAS selects GNN architectures for node subsets based on their characteristics and adds a contrastive learning loss.

    Source: §1 · pp. 1–2

  • Results

    N-GNAS outperforms conventional GNAS and manually crafted GNN methods across diverse graph-learning tasks in the reported evaluations.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Across increasing layer depths, N-GNAS performance remains stable while the reported baselines perform worse and decline, consistent with over-smoothing in those baselines.

    Source: §4.2 · p. 8

Abstract

from arXiv · show

In recent years, Graph Neural Networks (GNNs) and architecture search frameworks have gained extensive application in non-Euclidean data processing, attributable to their superior capacity in managing unstructured data. Nevertheless, traditional approaches typically apply uniform convolution operations to all nodes, regardless of their varying structural and feature characteristics, which can undermine model performance and result in over-smoothing issues as the number of layers increases. To overcome this limitation, in this work, we propose a \textbf{N}ode-Level \textbf{G}raph \textbf{N}eural \textbf{A}rchitecture \textbf{S}earch (N-GNAS) algorithm. It can automatically choose an appropriate network architecture for each subset of nodes when updating node features. N-GNAS also introduces a contrastive learning loss to separate sample features from different categories and vice versa. In experiments conducted on eight datasets for node and graph classification, our methodology outperforms current leading GNAS techniques and traditional human-designed GNNs. For example, it achieves an accuracy rate of 78.26\% on the CiteSeer dataset.

1. Introduction

GNN architecture design can require substantial expertise and computational effort, while uniform operations may fail to reflect differences among nodes. N-GNAS searches for architectures at the node-subset level and adds a contrastive loss; the paper reports stronger performance than conventional GNAS and manually designed GNNs.

1. Introduction

  • GNN architectures often fit only certain tasks and datasets, while their manual optimization typically requires substantial expertise and computational investment.Source: §1 · p. 1
  • Traditional GNAS applies uniform operations despite node diversity, and repeated operations can average node features excessively, causing over-smoothing.Source: §1 · p. 1
  • The proposed framework automatically selects GNN architectures for node subsets according to their characteristics, rather than applying uniform convolution to all nodes.Source: §1 · p. 1
  • N-GNAS outperforms conventional GNAS and manually crafted GNN methods across diverse graph-learning tasks, according to the reported evaluations.Source: §1 · p. 2
  • N-GNAS introduces a contrastive learning loss to separate features from different categories and is reported to alleviate over-smoothing in deep GNNs.Source: §1 · p. 2

2. Related Work

The related work reviews graph neural network approaches and graph architecture search, then identifies node diversity and reliance on cross-entropy as gaps addressed by this paper.

2.1. Recent Advances In Graph Neural Networks

  • Spatial GNNs aggregate local-neighborhood information, whereas spectral GNNs derive node representations using graph spectral decomposition.Source: §2.1 · p. 2

2.2. Graph Neural Architecture Search

  • GNAS spans reinforcement-learning, evolutionary, and gradient-descent approaches, with search spaces ranging from components to whole-network architectures.Source: §2.2 · p. 3
  • Prior approaches overlook node-attribute diversity and rely on cross-entropy, while N-GNAS learns node-level operations without type annotations.Source: §2.2 · p. 3

3.1. Notations

The notation section establishes the graph, node, feature, neighborhood, and matrix symbols used in the method.

3.1. Notations

  • The graph G has n nodes V, and its node-feature matrix X has dimension n by d_V, with row x_u representing node u.Source: §3.1 · p. 3
  • N(v) denotes node v's one-hop neighbors, while A and D denote the adjacency and degree matrices.Source: §3.1 · p. 3
  • The graph's structural relationships and node features are represented by adjacency matrix A and feature matrix X.Source: §3.1 · p. 3

3.2. Search Space Architecture Design

N-GNAS adapts graph processing to node characteristics through a DAG search space with node selection and GNN-operation stages; graph classification also searches for layer-level readout operations.

3.2. Search Space Architecture Design

  • Each layer uses node-selection and GNN-operation stages in a DAG, with cells as search elements and links connecting them.Layer 1 is usually linear and layer L is usually a classification layer.Source: §3.2 · p. 3

3.2.1. Stage I: Node Selection Operations.

  • Node scores combine feature, positional, and structural approaches into soft gates, routing low-scoring nodes mainly through residual connections.The scores consider node features, positions, and structures; the continuous gates keep the framework differentiable during training.Source: §3.2.1 · pp. 3–4

3.2.2. Stage II: Graph Neural Network Operations.

  • Selected nodes undergo searched GNN operations, while low-gate nodes pass through residual connections that help alleviate over-smoothing.The operation search includes widely used GNNs, each taking a graph and feature matrix and returning a feature matrix.Source: §3.2.2 · p. 4

3.2.3. Stage III: Readout Operations For Graph Classification Tasks.

  • For graph classification, each layer searches among readout operations to produce that layer’s output from its feature matrix.The search space includes GLOBAL_MEAN, GLOBAL_MAX, GLOBAL_SUM, GLOBAL_SORT, GLOBAL_ATT, and ZERO.Source: §3.2.3 · p. 5

3.3. Search Strategy Design

N-GNAS trains with cross-entropy and contrastive losses, searches its architecture using DARTS, then selects and retrains a discrete architecture.

3.3.1. Loss Function

  • The objective combines cross-entropy for training-data fit with contrastive loss intended to balance performance and alleviate over-smoothing.The contrastive loss uses sampled node features or graph features and treats samples with the same ground-truth label as positive pairs.Source: §3.3.1 · pp. 5–6

3.3.2. Search Strategy

  • DARTS continuously relaxes the discrete architecture so the DAG search space can be optimized with gradients.Candidate operations on each edge contribute through learnable weights.Source: §3.3.2 · p. 6

3.3.3. Discretization and Final Architecture

  • After search, the highest-weight operation is selected for each layer, yielding a fixed architecture retrained from scratch.The resulting discrete model retains the flexibility of continuous search while producing a compact architecture for deployment.Source: §3.3.3 · p. 6

3.4. Interpretability and Explainability

N-GNAS uses soft gates to control how much further GNN processing each node receives, making its information filtering inspectable and controllable.

3.4. Interpretability and Explainability

  • A soft gating vector learned from seven scoring perspectives indicates each node’s contribution to subsequent GNN processing.Low gate values identify nodes that require less further processing; for visualization, values above 0.5 are shown as strongly selected and values at or below 0.5 as weakly selected.Source: §3.4 · p. 6
  • Assigning low gate values to nodes that need less processing can reduce over-smoothing and enhance discrimination power.The gates provide an explicit information-filtering mechanism for controlling information propagation across deep layers.Source: §3.4 · p. 6

4. Experiments

Across node and graph classification, N-GNAS outperforms existing approaches, with reported gains on node-classification accuracy and two graph datasets. Experiments also examine layer depth, learned node selection, and search-space and loss ablations.

4.1. Experimental setting

  • The evaluation covers node and graph classification on eight datasets spanning citation networks, proteins, movie collaborations, molecular inhibitors, and review graphs.The node datasets are Cora, CiteSeer, and PubMed; the graph datasets are D&D, PROTEINS, IMDB-MULTI, COX2, and MR.Source: §4.1 · pp. 6–7
  • N-GNAS is compared with manually designed GNNs and GNAS baselines; each baseline is trained five times across datasets.The reported baseline measure is mean accuracy with a variability measure.Source: §4.1 · p. 7
  • Search uses a two-stage space with seven and five operational cells, two-layer models, and 100 epochs; the top-1 architecture is selected on seven datasets.The selected architecture is repeated five times for 400 epochs during training.Source: §4.1 · p. 7

4.2. Results and Analysis

  • N-GNAS surpasses current methods on node classification (+1.3% test accuracy) and graph classification (+0.7% on D&D; +0.6% on PROTEINS).The node-classification results cover Cora, CiteSeer, and PubMed; the graph-classification results cover D&D, PROTEINS, IMDB-MULTI, COX2, and MR.Source: Table 3 · p. 8, §4.2 · p. 8, Table 4 · p. 8
  • Across Cora and PubMed, N-GNAS performance remains stable with increasing depth while baseline performance is lower and declines as depth increases.The authors identify over-smoothing as an issue for the declining baselines.Source: §4.2 · p. 8, Figure 4 · p. 9
  • On Cora, local structural feature scoring selects higher-degree nodes in layer one, while global position scoring favors more central nodes in layer two.Figure 5 visualizes strongly selected nodes in red and weakly selected nodes in green using a 0.5 gate threshold.Source: §4.2 · p. 8, Figure 5 · p. 9
  • On Cora, the accuracy gap between N-GNAS and its variant without node selection widens from 1.58% at 1 layer to 4.19% at 5 layers.The authors interpret this as evidence that learned gating becomes increasingly critical for discriminative predictions in deeper models.Source: Table 5 · p. 9, §4.2 · p. 9
  • Random fixed architectures lower accuracy by 1.8–4.6%, while omitting contrastive loss lowers it by 1.8–2.5%.The authors attribute the first decrease to missing node-wise optimal convolution selection and the second to ignoring over-smoothing.Source: §4.2 · pp. 9–10, Table 6 · p. 10

5. Conclusion

N-GNAS combines node-wise architecture selection, a loss addressing performance and over-smoothing, and a two-phase search, with reported competitiveness across tasks.

5. Conclusion

  • The method is reported to be competitive with manually crafted GNNs across various tasks.Broad experiments across multiple graph-learning tasks are reported to demonstrate its effectiveness.Source: §5 · p. 10
  • N-GNAS integrates node-wise optimal convolution selection with graph neural network operations in a dynamic search space for the optimal operation count.The framework is described as adaptable and general.Source: §5 · p. 10
  • A novel loss function is designed to balance model-performance challenges and the over-smoothing phenomenon.Source: §5 · p. 10
  • A two-phase search approach is presented to enhance search efficiency.Source: §5 · p. 10
  • The conclusion reports broad experimental results across multiple graph-learning tasks as evidence of the method’s effectiveness.It notes that graphical illustrations reinforce widely accepted insights.Source: §5 · p. 10

CRediT authorship contribution statement

The contribution statement assigns research, implementation, resource, supervisory, administrative, and writing responsibilities across seven authors.

CRediT authorship contribution statement

Related papers

Loading 2610.09297v1…