Source-linked AI summary
Hyperbolic Graph Neural Networks
Qi Liu, Maximilian Nickel, Douwe Kiela
TL;DR
The paper asks whether GNNs can better capture graph structure through geometrically appropriate representations. It develops a manifold-agnostic GNN framework with differentiable exponential and logarithmic maps, then compares Euclidean and hyperbolic models. Across synthetic graph classification and molecular and blockchain applications, hyperbolic methods show substantial improvements, while spherical manifolds remain outside the study’s scope.
Problem
The paper examines whether GNNs can use geometrically appropriate inductive biases to capture structural properties such as node centrality and degree distributions.
Method
The authors generalize GNNs to Riemannian manifolds using differentiable exponential and logarithmic maps, comparing Euclidean, Poincaré-ball, and Lorentz-model architectures.
Results
Hyperbolic GNNs substantially improve full-graph classification and outperform Euclidean counterparts on synthetic structural classification, molecular property prediction, and blockchain transaction-graph prediction.
Takeaways & Limitations
Hyperbolic representations capture high-level graph structure effectively, including in low-dimensional settings, and can outperform Euclidean models across several graph-learning applications.
Takeaways & Limitations
Spherical and other non-hyperbolic Riemannian manifolds are beyond the scope of this work and are left for future study.
Abstract
from arXiv · showhide
Learning from graph-structured data is an important task in machine learning and artificial intelligence, for which Graph Neural Networks (GNNs) have shown great promise. Motivated by recent advances in geometric representation learning, we propose a novel GNN architecture for learning representations on Riemannian manifolds with differentiable exponential and logarithmic maps. We develop a scalable algorithm for modeling the structural properties of graphs, comparing Euclidean and hyperbolic geometry. In our experiments, we show that hyperbolic GNNs can lead to substantial improvements on various benchmark datasets.
1 Introduction
The paper extends GNNs to Riemannian manifolds to test whether hyperbolic geometry provides better inductive biases for capturing graph structure. It compares Euclidean, Poincaré-ball, and Lorentz-model GNNs across graph-classification settings.
- Hyperbolic geometry can model hierarchical structure associated with heterogeneous degree distributions and strong clustering in complex networks.
- The proposed framework generalizes GNNs to Riemannian manifolds using differentiable exponential and logarithmic maps.
- The study compares standard Euclidean GNNs with hyperbolic GNNs operating on the Poincaré ball and Lorentz model.
- The experiments evaluate whether hyperbolic GNNs capture structural properties such as node centrality and graph degree distributions.
- Hyperbolic GNNs improve full-graph classification, synthetic structural-property modeling, molecular-property prediction, and blockchain transaction-graph prediction.
2 Related Work
Prior work established hyperbolic representations for hierarchical data, hyperbolic neural architectures, and network modeling. This paper extends those ideas to manifold-agnostic GNNs, while related concurrent work emphasizes attention and trainable curvature.
- Hyperbolic embeddings were developed to represent latent hierarchies, including symbolic data, directed graphs, and tree-like network structures.
- Hyperbolic neural and attention networks extended deep-learning methods to hyperbolic space before this work.
- This paper’s formalism applies to any Riemannian manifold with differentiable logarithmic and exponential maps, including the Lorentz model.
- Unlike the Einstein-midpoint approach, the proposed message passing generalizes across manifolds by mapping representations to and from tangent spaces.
- Concurrent work used attention-based neighborhood aggregation with trainable curvature and evaluated link prediction and node classification.
3 Hyperbolic Graph Neural Networks
The paper generalizes graph convolutional networks to Riemannian manifolds by using tangent-space transformations and differentiable logarithmic and exponential maps. It compares Euclidean, Poincaré, and Lorentz geometries and provides manifold-compatible prediction mechanisms.
- Graph Neural Networks: GNNs perform message passing by aggregating transformed representations from neighboring nodes across multiple steps.The normalized adjacency matrix includes self-loops and degree normalization before neighbor messages are aggregated and passed through an activation.
- Riemannian Graph Convolutions: The proposed framework makes graph convolutional networks agnostic to the underlying Riemannian manifold by operating through tangent spaces.Neighbor representations are mapped with a logarithmic map, linearly transformed, aggregated, and returned to the manifold with an exponential map.
- Riemannian Graph Convolutions: Applying the activation after the exponential map prevents the logarithmic and exponential maps across successive layers from cancelling and reducing the model to a Euclidean GCN.Activations applied directly on a manifold must also preserve that manifold; the Poincaré implementation uses norm-decreasing nonlinearities.
- Riemannian Manifolds: The experiments compare a canonical Euclidean model with hyperbolic models based on the Poincaré ball and Lorentz manifolds.The Poincaré ball has constant negative curvature, while the Lorentz model is used to avoid numerical instabilities from repeated distance and map computations.
- Riemannian Manifolds: The Lorentz model supports deep architectures by avoiding numerical issues that can compound through multiple sequential exponential and logarithmic maps.Its Lorentzian scalar product defines the model's hyperbolic geometry, with the manifold represented as a constrained subset of R^(n+1).
- Centroid-Based Regression and Classification: Centroid-based prediction summarizes each manifold embedding by its distances to learned centroids before applying standard Euclidean classifiers or regressors.For graph-level predictions, node-level distance summaries are average-pooled and passed to fully connected networks.
4 Experiments
The experiments compare Euclidean and hyperbolic GNNs on synthetic graph structures, molecular property prediction, and Ethereum transaction graphs. Hyperbolic models generally outperform Euclidean models, with Lorentz often stronger than Poincaré and computational overhead remaining relatively minor.
- 4.1 Synthetic Structures: Hyperbolic GNNs outperform Euclidean GNNs on synthetic graph-generation classification, especially in low-dimensional settings.The task classifies Erdős-Rényi, Barabási-Albert, and Watts-Strogatz graphs using F1 score; hyperbolic methods still perform better at 256 dimensions.
- 4.1 Synthetic Structures: Lorentz performs better than Poincaré in all but one synthetic classification setting, while differences shrink as dimensionality increases.The reported comparison attributes the low-dimensional advantage to the representational efficiency of hyperbolic methods.
- 4.2 Molecular Structures: The molecular benchmark uses ZINC because QM9 has unavailable original splits, relatively small molecules, and high result variance.ZINC contains 250k examples, with 25k randomly sampled for validation and 25k for testing; its molecules average 23 heavy atoms.
- 4.2 Molecular Structures: For molecular property prediction, the study evaluates mean absolute error for logP, QED, and SAS, comparing manifold variants and established graph baselines.The baselines include GGNN, DTNN, and MPNN, and the table metrics are scaled by 100, with lower error preferred.
- 4.2 Molecular Structures: The Lorentz model outperforms Poincaré on all molecular properties, while Euclidean models perform worse than the hyperbolic variants.The authors relate Lorentz’s advantage to improved numerical stability and report relatively minor computational overhead for non-Euclidean manifolds.
- 4.3 Blockchain Transaction Graphs: For Ethereum price-fluctuation prediction, Lorentz significantly outperforms Poincaré, which in turn outperforms the Euclidean manifold.The comparisons include a degree-selected node2vec-plus-MLP baseline and ARIMA; whale nodes also have lower average norm than the overall address population.
5 Conclusion
The paper generalizes GNNs to Riemannian manifolds and finds hyperbolic representations effective for structural graph information, molecular properties, and hierarchical transaction-graph prediction.
- The proposed method makes GNNs agnostic to the underlying Riemannian manifold.
- Hyperbolic methods better classify synthetic graphs according to their underlying generation algorithm.
- Hyperbolic methods outperform their Euclidean counterpart and state-of-the-art models on ZINC molecular property prediction.
- The architecture models blockchain transaction graphs for price-fluctuation prediction by capturing their hierarchical structure.