Source-linked AI summary
Large-Scale Learnable Graph Convolutional Networks
Hongyang Gao, Zhengyang Wang, Shuiwang Ji
TL;DR
Regular CNNs struggle with graphs because neighborhoods have variable sizes and no inherent ordering. The paper introduces LGCL to rank-select neighbors into grid-like 1-D inputs for CNNs, plus sub-graph training for scale. Across transductive and inductive node classification, LGCN models consistently outperform prior methods while sub-graph training improves efficiency with negligible performance loss.
Problem
Regular CNNs require fixed, ordered receptive fields, whereas generic graph neighborhoods vary in size and lack an inherent order.
Method
LGCL selects the k largest neighboring values for each feature, transforms neighborhoods into 1-D grid-like data, and applies regular CNNs; sub-graph training supports large-scale learning.
Results
LGCN models achieve consistently better performance than prior methods in transductive and inductive settings, including margins of 1.8%, 2.7%, 0.6%, and 16% on reported benchmarks.
Takeaways & Limitations
The approach achieves state-of-the-art results on four datasets while making graph models more efficient through sub-graph training.
Takeaways & Limitations
Sub-graph training does not learn the whole-graph structure and may ignore testing-node information in transductive tasks, although experiments report negligible performance loss.
Abstract
from arXiv · showhide
Convolutional neural networks (CNNs) have achieved great success on grid-like data such as images, but face tremendous challenges in learning from more generic data such as graphs. In CNNs, the trainable local filters enable the automatic extraction of high-level features. The computation with filters requires a fixed number of ordered units in the receptive fields. However, the number of neighboring units is neither fixed nor are they ordered in generic graphs, thereby hindering the applications of convolutional operations. Here, we address these challenges by proposing the learnable graph convolutional layer (LGCL). LGCL automatically selects a fixed number of neighboring nodes for each feature based on value ranking in order to transform graph data into grid-like structures in 1-D format, thereby enabling the use of regular convolutional operations on generic graphs. To enable model training on large-scale graphs, we propose a sub-graph training method to reduce the excessive memory and computational resource requirements suffered by prior methods on graph convolutions. Our experimental results on node classification tasks in both transductive and inductive learning settings demonstrate that our methods can achieve consistently better performance on the Cora, Citeseer, Pubmed citation network, and protein-protein interaction network datasets. Our results also indicate that the proposed methods using sub-graph training strategy are more efficient as compared to prior approaches.
1 INTRODUCTION
Generic graphs are valuable for node classification, but their irregular neighborhoods make regular CNN convolutions difficult. The paper addresses this with LGCL and sub-graph training for graph learning at scale.
- Graph data supports node classification in social, citation, and biological networks, but neighboring nodes vary in number and lack a natural order.
- Prior graph convolution methods aggregate neighboring features but use filters whose sizes or weights differ across nodes and are not learned like regular CNN filters.
- LGCL transforms graphs into grid-like data so regular convolutional operations can be applied, achieving better performance in both transductive and inductive node classification.
- Sub-graph training reduces the memory and computational resources required by whole-graph training on large graphs, with negligible performance loss.
2 RELATED WORK
Prior graph convolution methods adapt convolution to irregular graphs through aggregation or attention, but these approaches differ from regular CNN filtering and can be resource-intensive. The paper instead transforms graph neighborhoods into grid-like inputs for direct CNN processing.
- Graph convolutional networks: GCNs aggregate each node’s own and neighboring features using normalized adjacency operations followed by a trainable linear transformation.
- Graph convolutional networks: GCN receptive fields vary with node degree, and their aggregation assigns the same non-trainable weight to adjacent nodes, limiting automatic feature extraction.
- Graph attention networks: GATs assign different trainable neighbor weights through attention, but attention requires extra computation between each node and all adjacent nodes.
- Learnable graph convolutional layers: LGCL selects the k largest neighboring values for each feature, forms a 1-D grid with k + 1 positions, and feeds it to a 1-D CNN.
- Proposed direction: The proposed approach performs graph-to-grid transformation inside the network rather than preprocessing, and adds sub-graph training for large-scale learning.
3 METHODS
The method converts irregular graph neighborhoods into fixed, ordered 1-D structures for regular convolution, then trains on iteratively selected sub-graphs to reduce large-graph resource demands.
- Challenges of Applying Convolutional Operations on Graph Data: Generic graphs challenge regular convolution because nodes have variable numbers of unordered neighbors.The method targets both varying neighborhood size and the absence of ranking information among neighboring nodes.
- Learnable Graph Convolutional Layers: LGCL selects k neighboring nodes by feature-wise value ranking, producing a fixed (k + 1)-position representation for each node.The central node is inserted alongside the selected neighbors; when fewer than k neighbors exist, zero padding is used.
- Learnable Graph Convolutional Layers: The ranked representations form a 1-D grid-like structure that a regular 1-D CNN maps to an updated feature vector for every node.The CNN reduces the spatial dimension from (k + 1) to 1 and outputs N × D node representations.
- Learnable Graph Convolutional Networks: LGCNs use deeper LGCL-based architectures with dense-style skip concatenation connections for graph node classification.The illustrated network applies graph embedding, stacked LGCL layers, and a fully connected prediction layer.
- Sub-Graph Training on Large-Scale Data: Sub-graph training samples initial nodes and expands their neighborhoods with BFS until reaching a target sub-graph size.Different expansion iterations may use different maximum numbers of added nodes.
4 EXPERIMENTAL STUDIES
The experiments evaluate LGCNs on node classification under transductive and inductive settings, comparing accuracy, generalization, layer choices, sub-graph efficiency, and k selection. LGCNs achieve stronger reported performance, while sub-graph training substantially improves efficiency with negligible performance loss.
- Experimental setup: The study evaluates LGCNs on node classification using transductive citation-network benchmarks and inductive PPI experiments.The transductive datasets are Cora, Citeseer, and Pubmed; the inductive dataset is PPI.
- Transductive learning: LGCN models exceed current state-of-the-art GCNs by 1.8%, 2.7%, and 0.6% on Cora, Citeseer, and Pubmed, respectively.These are node classification accuracy margins under transductive learning.
- Inductive learning: LGCN outperforms GraphSAGE-LSTM by 16% in micro-averaged F1 under inductive learning and generalizes without test-graph structure during training.The result is reported on the PPI dataset.
- LGCL versus GCN layers: Replacing LGCLs with GCN layers lowers performance, indicating that LGCL is more effective than GCN within the matched LGCNsub architecture.The comparison keeps the other settings unchanged for fairness.
- Sub-graph versus whole-graph training: Sub-graph training uses far fewer nodes than the 2,000-node maximum and incurs negligible performance loss relative to whole-graph training.Actual training sub-graphs contain 644, 442, and 354 nodes for Cora, Citeseer, and Pubmed, respectively.
- Sub-graph versus whole-graph training: Sub-graph training improves efficiency by reducing the input graph size, making large-scale LGCNs effective and efficient despite their more complex layers.The experiments specifically report advantages in training speed and reduced memory requirements.
- Hyper-parameter analysis: LGCNs achieve their best performance at k = 8 on all three transductive datasets, with performance decreasing when k becomes too large.The selected k is usually slightly larger than the datasets’ average node degree.
5 CONCLUSIONS AND FUTURE WORK
The paper concludes that LGCNs improve node classification across transductive and inductive settings, while sub-graph training improves efficiency with negligible performance loss. It identifies graph classification and graph down-sampling as important directions for future work.
- LGCN models achieve consistently better performance than prior methods under both transductive and inductive learning settings.The models achieve new state-of-the-art results on four datasets.
- Sub-graph training addresses excessive memory and computational requirements for large-scale graphs.It also provides a significant training-speed advantage with negligible performance loss.
- The proposed methods are limited mainly to node classification and cannot perform graph down-sampling like pooling operations.A layer that effectively reduces the number of nodes is needed for graph classification.
- Future work may investigate applying the methods to other data types, including text represented as graphs.