Source-linked AI summary
Variational Graph Auto-Encoders
Thomas N. Kipf, Max Welling
TL;DR
Many existing approaches to unsupervised graph learning do not support node features, a gap relevant to graph link prediction. VGAE uses a GCN encoder and inner-product decoder to learn latent graph representations; experiments report that adding features significantly improves predictive performance across datasets.
Problem
Method
VGAE is an unsupervised variational framework for learning latent representations of undirected graphs, with a GCN encoder and inner-product decoder.
Source: §1 · p. 1Results
Adding input features significantly improves predictive performance across datasets, and VGAE achieves higher predictive performance on Cora and Citeseer.
Source: §2 · p. 2Takeaways & Limitations
VGAE's supported scope includes learning interpretable latent representations for undirected graphs.
Source: §1 · p. 1Future work identifies better-suited priors, more flexible generative models, and stochastic-gradient training for improved scalability.
Source: §2 · p. 2
Abstract
from arXiv · showhide
We introduce the variational graph auto-encoder (VGAE), a framework for unsupervised learning on graph-structured data based on the variational auto-encoder (VAE). This model makes use of latent variables and is capable of learning interpretable latent representations for undirected graphs. We demonstrate this model using a graph convolutional network (GCN) encoder and a simple inner product decoder. Our model achieves competitive results on a link prediction task in citation networks. In contrast to most existing models for unsupervised learning on graph-structured data and link prediction, our model can naturally incorporate node features, which significantly improves predictive performance on a number of benchmark datasets.
1 A latent variable model for graph-structured data
VGAE is a VAE-based framework for unsupervised learning that uses latent variables to learn interpretable representations of undirected graphs.
Inference model
- A two-layer GCN uses node features and adjacency to infer Gaussian latent distributions, with the mean and log-scale networks sharing first-layer weights.The graph is defined as undirected and unweighted, with diagonal entries of the adjacency matrix set to 1.Source: §1 · p. 1
Generative model
- The generative decoder models adjacency entries from pairs of latent variables using an inner-product score passed through a logistic sigmoid.Source: §1 · p. 1
Learning
- Training optimizes a variational lower bound with a standard Gaussian prior and reparameterization; for sparse graphs, positive-edge terms are reweighted.Source: §1 · pp. 1–2
- The featureless approach drops dependence on node features and replaces them with identity features in the GCN.Source: §1 · p. 2
Non-probabilistic graph auto-encoder (GAE) model
- A non-probabilistic GAE variant calculates embeddings and reconstructs the adjacency matrix.Source: §1 · p. 2
2 Experiments on link prediction
The experiments evaluate VGAE and GAE on citation-network link prediction, finding that input features improve predictive performance and that VGAE performs better on Cora and Citeseer. Both models also achieve competitive results without input features.
2 Experiments on link prediction
- The models learn embeddings for citation-link prediction from graphs with some edges removed while retaining node features, and evaluation pairs removed edges with sampled non-edges.Validation and test sets are formed from previously removed citation links and an equal number of randomly sampled unconnected node pairs.Source: §2 · p. 2
- Evaluation classifies edges versus non-edges using 5% and 10% link validation and test sets, comparing against featureless spectral clustering and DeepWalk [6].The validation set is used to optimize hyperparameters; the baselines do not support input features.Source: §2 · p. 2
- VGAE achieves higher predictive performance on both Cora and Citeseer, while adding input features significantly improves performance across datasets.The reported test metrics are AUC and average precision, with results averaged over 10 runs on fixed dataset splits.Source: Table 1 · p. 2, §2 · p. 2
- Both VGAE and GAE achieve competitive results without input features, although a Gaussian prior may mismatch an inner-product decoder that pushes embeddings away from zero.The paper identifies better-suited prior distributions as a direction for future work.Source: §2 · p. 2
- Future work will explore better-suited priors, more flexible generative models, and stochastic gradient descent to improve scalability.Source: §2 · p. 2