Source-linked AI summary
Adaptive Graph Convolutional Neural Networks
Ruoyu Li, Sheng Wang, Feiyun Zhu, Junzhou Huang
TL;DR
Existing graph CNNs are constrained by fixed or shared graph structures and difficulty handling diverse topology and size. The paper introduces an adaptive graph CNN that learns sample-specific residual Laplacians through shared metric learning and feature transformation. Experiments report improved predictive performance across graph-structured datasets, with reduced RMSE on molecular benchmarks.
Problem
Existing graph CNNs restrict topology, graph size, or graph degree and may use fixed graphs that are not optimal for supervised learning.
Method
AGCN learns a residual graph Laplacian for each sample using a shared distance metric and feature transformation during graph CNN training.
Results
AGCN outperformed state-of-the-art graph CNN models across various prediction tasks and reduced mean RMSE by 31%–40% on the Delaney dataset.
Takeaways & Limitations
The model accepts raw graph data with arbitrary graph structure and size while adapting graph topology to the prediction task.
Takeaways & Limitations
Training requires additional RAM to retain initial graph Laplacians, although these Laplacians are usually sparse.
Abstract
from arXiv · showhide
Graph Convolutional Neural Networks (Graph CNNs) are generalizations of classical CNNs to handle graph data such as molecular data, point could and social networks. Current filters in graph CNNs are built for fixed and shared graph structure. However, for most real data, the graph structures varies in both size and connectivity. The paper proposes a generalized and flexible graph CNN taking data of arbitrary graph structure as input. In that way a task-driven adaptive graph is learned for each graph data while training. To efficiently learn the graph, a distance metric learning is proposed. Extensive experiments on nine graph-structured datasets have demonstrated the superior performance improvement on both convergence speed and predictive accuracy.
Introduction
Graph CNNs struggle with irregular, diverse graph structures because existing kernels and fixed graphs limit topology handling. The paper proposes adaptive, task-driven graph convolutions that learn sample-specific graph structure while accepting flexible graph inputs.
- Motivation: Irregular graph data such as molecules, point clouds, and social networks lacks the regular-grid properties that support classical convolution.Graph inputs vary in neighborhood connectivity and may lie in non-Euclidean domains.
- Limitations of existing methods: Existing graph CNNs restrict graph degree, often require shared graph structure, keep graphs fixed during training, and inadequately exploit topology.Early kernels can be over-localized, while shared spectral filters require resizing that may damage graph information.
- Adaptive graph learning: AGCN assigns each sample a customized graph Laplacian, combining its initial graph with a learned residual graph for topology-specific spectral filtering.The residual graph is trained to discover task-relevant substructures missing from the intrinsic graph.
- Adaptive graph learning: A shared learned distance metric updates graph topology during prediction-network training with complexity O(d2), independent of input size.The updated residual graph is constructed from the trained metric and transformed feature space.
- Feature processing: The model transforms vertex features before convolution, connecting intra-vertex and inter-vertex information.This feature embedding is part of the proposed graph convolution design.
- Flexible inputs: The network accepts graph data with different structures and sizes, removing restrictions on graph degree.This flexibility follows from learning individual residual Laplacians and distance metrics.
Related Work
Prior graph CNNs developed spatial and spectral approaches for graph filtering, but continued to face topology and dimensionality constraints. Existing methods also struggled to learn representations from raw molecular graphs because spatial kernels were over-localized.
- Spatial graph convolution: Spatial graph convolution aggregates neighborhood features using graph adjacency, but its finite-size kernel is nonparametric and over-localized.Varying local neighborhoods make a unified spatial filter difficult without restricting graph topology.
- Spectral graph convolution: Spectral methods use graph Fourier bases and localized polynomial filters to reduce computational cost and training parameters.Chebyshev-polynomial approximations and first-order graph filters are described as key developments.
- Remaining constraints: Shared spectral kernels still require resized inputs for unified layer dimensionality, which can destroy graph-oriented information.Molecular graph coarsening may be chemically difficult to justify.
- Molecular graph learning: Neural networks for raw molecules achieved progress, but spatial-convolution constraints limited their ability to capture molecular structures.The paper positions adaptive graph convolution as addressing this limitation.
SGC-LL Layer
The SGC-LL layer learns task-adaptive graph topology by parameterizing vertex distances, constructing graph Laplacians, and applying localized spectral convolution. Its trainable metric and feature transform keep learning complexity independent of input graph size.
- SGC-LL parameterizes vertex distances so the graph Laplacian becomes trainable and unique adaptive graphs can be constructed for differently shaped inputs.
- The normalized graph Laplacian encodes vertex connectivity and degree, and its eigendecomposition supplies the graph Fourier basis for spectral filtering.
- K-localized polynomial spectral filters aggregate information from vertices within shortest-path distance d_G < K, while filter coefficients control the relative importance of farther connections.
- The adaptive metric uses a trainable transform basis W_d to compute distances, convert them into a Gaussian kernel, and normalize the result into an adjacency matrix.
- The layer applies a transform matrix and bias vector to output features, jointly modeling intra- and inter-vertex features while using parameters whose complexity is independent of graph size or degree.The parameters are {M_i, W_i, b_i}, with learning complexity O(d_i d_i−1).
- At each SGC-LL layer, the learned residual Laplacian updates the original topology, with its influence controlled by α.
AGCN Network
AGCN combines adaptive spectral graph convolution with batch normalization, graph max pooling, and graph gathering for graph-level prediction. It updates topology while preserving graph sizes to retain informative local structures.
- AGCN uses SGC-LL to learn adaptive graph topology from data and the context of the learning task.
- Graph max pooling replaces each feature at a vertex with the maximum value across that vertex and its neighbors.
- The graph gather layer sums vertex feature vectors element-wise to produce a graph representation for graph-level prediction.
- Bilateral filtering regularizes SGC-LL activations to reduce over-fitting, and batch normalization is used to accelerate training.
- Each layer combo contains one SGC-LL layer, batch normalization, and graph max pooling, while the adaptive graph is reused until the next SGC-LL layer.
- AGCN updates graph structures after each layer combo while preserving graph sizes, avoiding coarsening or feature averaging that could damage informative local molecular structures.
Batch Training of Diverse Graphs
AGCN trains directly on raw graph-structured samples with diverse topology and size rather than requiring a shared or reshaped graph structure. Initial graph Laplacians add RAM usage but are usually sparse.
- Graph convolution must handle diverse local topologies because graph kernel invariance is not guaranteed and graph resizing or reshaping can be unreasonable for molecules.
- AGCN accepts training batches containing raw graph-structured samples with different topology and size.
- Initial graph Laplacians require additional RAM and must be retained for kernel updates, although they are usually sparse.
Experiments
Experiments compare AGCN with established graph CNNs on molecular and point-cloud datasets. Across these settings, adaptive graphs and residual Laplacian learning are associated with faster convergence and improved predictive performance.
- Benchmark comparisons: AGCN was compared with graphconv, NFP, and GCN on molecular, toxicity, and point-cloud benchmarks.The experiments include regression, multi-task classification, and Sydney urban object recognition.
- Performance boosted by SGC-LL Layer: After 20 epochs, node similarities changed visibly while weighted l2 losses and mean RMSE dropped dramatically.The paper links these changes to updated adaptive graphs and reports better convergence speed and predictive accuracy for AGCN.
- Multi-task Prediction on Molecular Datasets: 31%-40% lower mean RMSE was reported on Delaney, with average reductions of 15% on Az-logD and 2∼4% on the NCI testing set.These results compare AGCN against the state-of-the-art graph CNN benchmarks.
- Multi-task Prediction on Molecular Datasets: 3% (0.03) average improvement was reported for AGCN on the 617-task Toxcast classification benchmark.The paper also reports significantly improved accuracy on both small and large multi-task datasets.
- Adaptive graph handling: AGCN accepts original molecular graphs and point sets of varying structure or size instead of requiring a shared topology or downsampling.Its adaptive graph gives each sample a customized Laplacian, while SGC-LL learns graph updates during training.
- Point Cloud Object Classification: 3∼6% higher average ROC-AUC was reported across Sydney urban object classes, with at least 10% improvement for traffic-light classification.AGCN also achieved an AUC close to 1 for building recognition, while competing networks first coarsened the graphs.
Conclusions
The paper proposes an adaptive spectral graph convolver that supports arbitrary graph structure and size, using supervised residual Laplacian learning to fit prediction tasks. Experiments across various graph-structured data found improved performance over state-of-the-art graph CNNs.
- SGC-LL is a spectral graph convolver designed to work with adaptive graphs.
- AGCN accepts graph data with arbitrary structure and size rather than requiring a shared graph structure.
- Supervised residual Laplacian training drives the model to better fit the prediction task.
- Extensive multi-task experiments on various graph-structured data showed that AGCN outperformed state-of-the-art graph CNN models across prediction tasks.