PaperlayerView exact revision on arXiv

Source-linked AI summary

Graph Neural Networks in TensorFlow and Keras with Spektral

Daniele Grattarola, Cesare Alippi

arXiv:2006.12138v1cs.LGstat.ML

TL;DR

Graph neural networks need reusable methods for learning from graph-structured data, but accessible and flexible tooling remains important for adoption. Spektral provides TensorFlow and Keras components for graph representation learning, including message passing, pooling, utilities, and datasets, and evaluates them across node classification, graph classification, and graph regression. The paper concludes that Spektral is a mature TensorFlow-ecosystem library designed for beginners and experts, while continuing development is planned for its core components.

  • Problem

    Researchers need accessible and flexible software building blocks to adopt graph neural networks across varied graph-learning tasks.

  • Method

    Spektral implements graph neural network methods, graph representations, pooling operators, utilities, and benchmark datasets as TensorFlow and Keras components.

  • Results

    Across node classification, graph classification, and graph regression experiments, implemented methods show task- and dataset-dependent performance patterns.

  • Takeaways & Limitations

    Spektral offers a streamlined TensorFlow-ecosystem library for implementing GNNs for beginners and expert practitioners.

Abstract

from arXiv · show

In this paper we present Spektral, an open-source Python library for building graph neural networks with TensorFlow and the Keras application programming interface. Spektral implements a large set of methods for deep learning on graphs, including message-passing and pooling operators, as well as utilities for processing graphs and loading popular benchmark datasets. The purpose of this library is to provide the essential building blocks for creating graph neural networks, focusing on the guiding principles of user-friendliness and quick prototyping on which Keras is based. Spektral is, therefore, suitable for absolute beginners and expert deep learning practitioners alike. In this work, we present an overview of Spektral's features and report the performance of the methods implemented by the library in scenarios of node classification, graph classification, and graph regression.

1. Introduction

Graph neural networks support learning on graph-structured data across many scientific and practical domains, but their growing adoption increases the need for accessible software. Spektral addresses this need by bringing graph-learning components into TensorFlow and Keras for both beginners and experts.

  • GNNs perform inference on graph-structured data and have been applied in physics, recommender systems, biology, chemistry, natural language processing, and abstract reasoning.
  • Message passing learns node representations from local information, while graph pooling reduces graph size to support hierarchical representations and graph-level prediction.
  • Keras reduces boilerplate and cognitive load while retaining TensorFlow compatibility, making it suitable for beginners and expert practitioners.
  • Spektral provides GNN methods as Keras layers integrated with TensorFlow models, training features, distributed execution, GPUs, TPUs, and production deployment.
  • The project provides detailed documentation, source-code examples, and release under the MIT license.

2. Library Overview

Spektral represents graphs flexibly and supplies core GNN building blocks, datasets, and utilities within the TensorFlow and Keras ecosystem. Its library overview spans message passing, pooling, global pooling, graph formats, and benchmark access.

  • Representing graphs: Spektral represents graphs with adjacency matrices, node features, and edge features while supporting Keras-compatible formats, NumPy, SciPy sparse matrices, batches, and disjoint unions.
  • Message passing: Spektral implements fifteen message-passing layers covering methods including GCN, GraphSAGE, GAT, APPNP, ARMA, ECC, and GIN.
  • Graph pooling: Graph pooling reduces node count for hierarchical graph representations, with implementations including DiffPool, MinCut, Top-K, and SAGPool.
  • Graph pooling: Six global pooling strategies reduce graph node features to a single vector, including sum, average, max, GAP, SortPool, and AWSP.
  • Datasets: The datasets module provides benchmarks for node classification, graph signal classification, graph classification, and graph regression, including citation, interaction, and graph-kernel datasets.
  • Utilities: Utilities support graph-matrix computation, data manipulation, molecular-graph processing, and construction of new GNN layers.

3. Comparison to other libraries

Spektral differs from PyTorch-based GNN libraries by targeting TensorFlow and offering comparable library features within that ecosystem. The cited comparison also identifies a flexibility limitation in another library.

  • Spektral is developed for TensorFlow, whereas PyTorch Geometric and DGL are based on PyTorch.
  • The compared library implements six message-passing layers, offers four corresponding layers in Spektral, lacks pooling layers, and uses a custom graph-data format.

4. Applications

Spektral’s experiments evaluate message passing, graph pooling, and global pooling across node classification, graph classification, and graph regression. Results vary by task: APPNP performs strongly on citation networks, MinCut generally leads graph classification, and global pooling choice substantially affects regression.

  • Experimental settings: The evaluation covers node classification and graph-level classification and regression on established benchmark datasets.
  • Node classification: APPNP consistently performs well on citation networks, outperforming other methods on Cora and Pubmed and approaching ARMA on CiteSeer.
  • Graph classification: MinCut generally achieves the best graph-classification performance, followed by DiffPool, while the Flat baseline often outperforms equivalent pooled GNNs.
  • Graph regression: Global pooling choice significantly affects graph regression: GAP performs best on Alpha and U0, max pooling on Mu, and AWSP on Homo.
  • Graph regression: Max pooling and AWSP show dataset-dependent instability because both fail on Alpha and U0.

5. Conclusion

Spektral is a Keras-based library for building graph neural networks, with state-of-the-art methods, utilities, and popular datasets. Future work will track GNN research and improve core-component performance.

  • Spektral provides message-passing and pooling layers, utilities, and popular graph datasets for building graph neural networks.
  • The library is designed to provide a streamlined user experience within the TensorFlow ecosystem.
  • Spektral is described as the most mature library for GNNs in the TensorFlow ecosystem.

A. Data modes

Spektral supports four graph-representation modes for different batching and inference settings. Their compatibility varies across message-passing and pooling layers because of structural constraints.

  • Spektral supports four graph-representation modes, called single, disjoint, batch, and mixed mode.
  • Single mode: Single mode represents one graph with its adjacency matrix and attributes, typically for node-level inference.
  • Disjoint mode: Disjoint mode batches variable-order graphs by stacking node attributes and combining adjacency matrices in a blockdiagonal matrix.
  • Batch mode: Batch mode stacks graph tensors but requires every graph in a batch to have the same number of nodes.
  • Message-passing layers support single and disjoint modes, while more than half also support mixed and batch modes; pooling support is more constrained.

B. Technical notes

Spektral is distributed as a Python package for UNIX-like platforms without proprietary dependencies. Its TensorFlow and Keras compatibility depends on the library version.

  • Spektral is distributed through PyPI as the package spektral and supports UNIX-like platforms without proprietary dependencies.
  • The library is compatible with Python 3.5 and above.
  • Starting with version 0.2, Spektral targets TensorFlow 2 and its integrated Keras implementation.
  • Version 0.1 is based on TensorFlow 1 and standalone Keras and will be maintained until TensorFlow 1 is discontinued.
  • Spektral is also largely compatible with Windows.

C. Experimental details

The experiments used the architectures and hyperparameters described in the paper and ran on a single NVIDIA Titan V GPU with 12GB of video memory.

  • All experiments were run on a single NVIDIA Titan V GPU with 12GB of video memory.

C.1. Node classification

The node-classification experiments use Cora, Citeseer, and Pubmed with random class-balanced splits and repeated evaluation across methods and datasets.

  • Training uses learning-rate, weight-decay, epoch, and patience settings taken from the original papers.
  • Each method and dataset combination is evaluated with 100 repetitions.
  • Cora, Citeseer, and Pubmed provide the node-classification datasets.The data use random splits with 20 labels per class for training, 30 for early stopping, and all remaining labels for testing.

C.2. Graph classification

The graph-classification experiments evaluate pooling layers within a GNN architecture using benchmark graph-kernel datasets and repeated random train, validation, and test splits.

  • The evaluated architecture stacks GCS layers, tested pooling layers, global sum pooling, and a dense output layer.GCS layers use 32 units, ReLU activation, and L2 regularisation on both weight matrices.
  • DiffPool and MinCut output 4 nodes at the second layer, with the target size defined relative to the dataset’s average graph order.
  • DiffPool removes the first two GCS layers because it includes an internal message-passing layer for input features.
  • Each method and dataset combination is evaluated with 10 repetitions, and models are trained to convergence.
  • The benchmark graph-kernel datasets exclude isomorphic graphs and use random splits of 80% training, 10% early stopping, and 10% testing.

C.3. Graph regression

The graph-regression experiments use ECC-based GNNs with alternative global pooling layers to predict molecular properties under repeated random dataset splits.

  • The regression architecture uses two ECC layers, a tested global pooling layer, and a dense output layer.ECC layers have 32 units and ReLU activation, with no regularisation applied to the GNN.
  • Molecular node features encode atomic numbers, while edge features encode bond types.
  • The target variables use debye units for Mu, a0^3 for Alpha, and Hartree for Homo and U0.The Bohr radius is denoted by a0.
  • Each method and dataset combination is evaluated with 5 repetitions, with models trained to convergence.
  • Each run randomly splits the molecules into 80% training, 10% early stopping, and 10% testing sets.
Loading 2006.12138v1…