RiemannGFM: Learning a Graph Foundation Model from Riemannian Geometry

1 Introduction

• GNNs often require retraining across tasks or datasets, while existing graph-modeling approaches include text-focused or domain- and task-specific methods.The introduction also notes that many real graphs lack fruitful textual attributes and that sequential graph descriptions can miss structural complexity.Source: §1 · pp. 1–2
• Trees and cycles form the proposed shared structural vocabulary, with hyperbolic space aligned to trees and hyperspherical space to cycles.The introduction gives node triangles as an example of cycles.Source: §1 · p. 2
• RiemannGFM uses a product bundle coupling a Riemannian manifold for local geometry with tangent spaces for graph-structure information.Its universal Riemannian layers combine vocabulary learning and global learning modules.Source: §1 · p. 2
• The vocabulary module updates tree or cycle coordinates in hyperbolic or hyperspherical space, while the global module aggregates substructures and computes graph-level node encodings.Bundle convolution addresses incompatibility over the tangent bundle; geometric midpoints and parallel transport support graph-level encoding, followed by geometric contrastive learning across views.Source: §1 · p. 2
• The authors evaluate RiemannGFM for cross-domain transfer and few-shot learning on diverse real graphs.They present the product-bundle pretraining as a way to learn shared structural knowledge for cross-domain transferability.Source: §1 · p. 2

2 Preliminaries

• A Riemannian manifold is a smooth manifold with a metric; its points have tangent spaces, and exponential and logarithmic maps connect tangent spaces with the manifold.Parallel transport transforms between tangent spaces, while geodesics are minimum-length connecting curves.Source: §2 · p. 2
• Constant Curvature Spaces have equal curvature everywhere and admit a closed-form metric; hyperbolic, hyperspherical, and Euclidean spaces have negative, positive, and zero curvature, respectively.The passage also introduces a unified formalism for hyperbolic and hyperspherical spaces.Source: §2 · p. 2, Lorentz/Spherical Model · p. 2

Lorentz/Spherical Model.

• The Lorentzian metric is induced by a diagonal matrix whose first entry is the sign of the curvature and whose remaining entries are one.The passage gives the north pole of the space and notes that exponential and logarithmic maps exist.Source: Lorentz/Spherical Model · p. 2
• A graph foundation model is defined as a universally pretrained model whose parameters transfer across graph domains with slight treatments to generate downstream representations.The definition allows node attributes but does not require them.Source: Notations & Problem Formulation · p. 3

Structural Vocabulary and Constant Curvature Spaces (CCS).

• Trees and cycles form the structural vocabulary, modeled in hyperbolic and hyperspherical spaces, respectively.Trees and cycles with coinciding edges can form an arbitrary connected component unless it is exactly a tree.Source: Structural Vocabulary and Constant… · p. 3

3.1 Universal Riemannian Layer

• RiemannGFM samples trees and cycles, then stacks product-bundle layers whose vocabulary and global modules learn substructure coordinates and node encodings.The vocabulary module uses cross-geometry attention, while the global module aligns substructures and applies bundle convolution.Source: Overall Architecture · p. 3, §3.1 · p. 3

3.1.1 A Layer on the Product Bundle.

• The product bundle associates each node with manifold coordinates for relative substructure position and tangent-space encodings for global graph information.The framework uses hyperbolic and hyperspherical bundles for trees and cycles, respectively.Source: §3.1.1 · p. 4

3.1.2 Deriving Riemannian Operations.

• The proposed Riemannian linear operation preserves the manifold, and the arithmetic mean in Eq. (4) is its geometric midpoint.The stated guarantee covers nonzero constant curvature and weight matrices of matching or changed output dimension.Source: §3.1.2 · p. 4

3.1.3 Vocabulary Learning Module.

• Vocabulary learning updates trees bottom-up in hyperbolic space and cycles from neighboring nodes toward the center in hyperspherical space.Cross-geometry attention uses information from the other geometry, and its unidirectional structure differs from bidirectional graph-transformer attention over all nodes.Source: §3.1.3 · p. 4, Comparison to Graph Transformers · p. 4

3.1.4 Global Learning Module.

• Bundle convolution resolves tangent-space incompatibility by parallel-transporting encodings to the target point's tangent space before message passing.This combines global-structure encodings with local manifold geometry.Source: §3.1.4 · p. 5

On Stacking Multiple Layers.

• Stacking layers expands a node's receptive field, while the model learns node encodings in tangent bundles rather than only embedding nodes in a manifold.Successive layers can incorporate higher-order descendants; the approach considers structural vocabulary and global information.Source: On Stacking Multiple Layers · p. 5, Comparison to Previous Riemannian… · p. 5

3.2 Geometric Contrastive Learning

• Geometric contrastive learning uses hyperbolic and hyperspherical views without graph augmentation, contrasting tangent-space encodings through a shared tangent space.The combined objective includes both geometry directions, and the model is described as generating informative encodings for arbitrary graphs.Source: §3.2 · pp. 5–6

4.1 Experimental Setups

• The experiments use two text-attributed graphs, one mix-attributed graph, and one non-attributed graph, comparing nine baselines across node classification and link prediction.The baselines comprise vanilla GNNs, self-supervised graph-learning models, and graph foundation models; metrics are ACC and weighted F1 for classification, and AUC and AP for link prediction.Source: §4.1.1 · p. 6, §4.1.2 · p. 6

4.2.1 Cross-domain Transfer Learning Performance (RQ1).

• RiemannGFM achieves the best link-prediction results among GFMs and specialized models, and leads on node classification for graphs without textual attributes while remaining comparable on text-attributed graphs.The comparison covers Citeseer, Pubmed, GitHub, and Airport; OFA cannot run on GitHub or Airport, and GraphAny is unavailable for link prediction.Source: §4.2.1 · p. 6

4.2.2 The Connection between Structural Vocabulary and Constant Curvature Spaces (CCSs) (RQ2).

• Trees embedded in hyperbolic space and cycles in hyperspherical space give the best performance among the tested geometric placements.The ablation reports link-prediction AUC on Citeseer, Pubmed, and Airport.Source: Table 2 · p. 6, §4.2.2 · p. 7
• Cross-geometry attention consistently outperforms the single-geometry variant on node classification and link prediction across datasets.The variant uses query, key, and value vectors in the same constant-curvature space.Source: §4.2.3 · p. 7, Figure 3 · p. 7

4.2.4 Few-shot Learning Performance (RQ3).

• Transferring text-derived knowledge can produce negative transfer on mixed-attribute GitHub graphs and graphs without attributes.The passage identifies mixed attributes such as numbers and addresses in GitHub as one affected setting.Source: §4.2.4 · p. 7

4.2.5 On the Expressiveness of Structural Knowledge (RQ4).

• Without target-graph attributes, RiemannGFM’s pretrained structural encodings achieve competitive or superior link-prediction results against node2vec and OpenGraph.Node2vec is trained on the target datasets, whereas RiemannGFM uses structural knowledge learned from pretraining datasets.Source: §4.2.5 · p. 7, Figure 4 · p. 7

4.2.6 Impact of Pre-training Datasets (RQ5).

• RiemannGFM’s performance is more stable across pretraining graphs, while similar-domain pretraining can improve OpenGraph and GCOPE results.On Citeseer, OpenGraph scores 89.64% with WikiCS, versus 65.16% with Flickr and 60.16% with AComp.Source: §4.2.6 · pp. 7–8, Table 4 · p. 8
• Pretrained RiemannGFM node encodings appear more separable than GCN encodings in the Cora t-SNE visualization.The RiemannGFM visualization uses node encodings in the shared tangent space of the north pole of the Lorentz/Spherical model.Source: §4.2.7 · p. 8

Graph Neural Network & Self-supervised Graph Learning.

• Traditional GNNs pair neighborhood aggregation or transformer-like encoding with task-specific heads or losses, while self-supervised models train on graph data using specialized pretext tasks.The passage notes that graph augmentation is nontrivial and that attribute-distribution differences limit direct application of parameters across graphs.Source: Graph Neural Network & Self-supervised… · p. 8

Graph Foundation Model.

• Graph foundation-model efforts include graph prompts, multi-domain training, and LLM adaptations that translate graphs into sequences or natural-language descriptions.Prior models are described as struggling on graphs without textual attributes and as modeling graphs in Euclidean space, which can trivialize structural complexity.Source: Graph Foundation Model · p. 8
• Riemannian graph-learning work connects hyperbolic spaces with trees and hyperspherical spaces with cycles, but prior approaches are described as lacking generalization to unseen graph structures.The paper distinguishes its approach by introducing a tangent bundle for graph foundation modeling.Source: Riemannian Manifold & Graphs · p. 8

6 Conclusion

• RiemannGFM proposes a shared tree-cycle vocabulary, connected to Riemannian geometry, as a basis for graph foundation models.The conclusion describes this as a new opportunity for cross-domain transferability.Source: §6 · p. 8
• A product bundle incorporates the vocabulary’s diverse geometries, and Riemannian layers learn the shared structural knowledge on that space.The conclusion says this supports informative node encodings for arbitrary graphs.Source: §6 · p. 8
• The conclusion reports empirical superiority for RiemannGFM.Source: §6 · p. 8

A Notation Table

• Table 5 is titled “Importation Notations.”Source: Table 5 · p. 10

B.1 The Proposed Linear Operation

• For nonzero curvature, the proposed linear operation maps points from L^d1_κ to L^d2_κ for any weight matrix W.The proof concludes that the operation’s output remains in the target manifold.Source: §B.1 · p. 10

B.2 Geometric Midpoint

• The appendix proves that the weighted arithmetic mean remains on the manifold and is equivalent to the geometric midpoint.The geometric midpoint is defined using manifold points and their weights.Source: §B.2 · p. 10

B.3 Bundle Convolution

• Bundle convolution aggregates node encodings in tangent spaces, using parallel transport to address incompatibility between those spaces.The appendix describes parallel transport along a geodesic as a linear isometry.Source: §B.3 · p. 10

C Algorithm

• The appendix points to pseudocode for cross-geometry attention in Algorithm 2.Source: C Algorithm · p. 11

D Riemannian Geometry

• The paper’s constant-curvature formalism covers hyperbolic and hyperspherical spaces but excludes Euclidean space, requiring κ ≠ 0.It uses the Lorentz/Spherical model; the logarithmic map projects a manifold point to a tangent space and is the inverse of the exponential map.Source: D Riemannian Geometry · p. 11

E.1 Datasets

• The dataset descriptions cover Citeseer, Pubmed, GitHub, Airports, ogbn-arxiv, Physics, and Amazon Computers.They include citation, social, transportation, co-authorship, and co-purchase networks, with varied node features and labels.Source: §E.1 · p. 11
• Table 6 is titled “Summary of Datasets.”Source: Table 6 · p. 12

E.2 Baselines

• The listed baselines include GCN, DGI, GraphMAE2, OFA, LLaGA, OpenGraph, GCOPE, and GraphAny.Their descriptions span graph neural networks, self-supervised methods, LLM-based approaches, and graph pretraining.Source: §E.2 · p. 12

E.3 Reproducibility & Implementation Notes

• Few-shot evaluation retains up to k training instances per labeled class on a new dataset to train the classification head.The example pretrains on ogbn-Arxiv, Amazon Computers, and Coauthor Physics, then uses k Citeseer samples per class.Source: §E.3.1 · p. 12

E.3.1 On Few-shot Learning.

• The initialization procedure computes the normalized graph Laplacian and decomposes it, retaining the largest K eigenvectors for node initialization.Source: §E.3.2 · p. 12

E.3.2 Initialization and Configurations.

• Table 7 reports geometric ablation results on Citeseer, Pubmed, and Airport using AUC (%) as mean±std.Source: Table 7 · p. 12
• Table 8 reports cross-domain node-classification performance across different pre-training datasets.Source: Table 8 · p. 12
• Initialization normalizes each graph with a K-dimensional encoding z, then maps it to manifold coordinates using Exp_o.The coordinates lie on manifold L with reference point o at the north pole; mini-batch sampling follows SAGE.Source: §E.3.2 · p. 12

E.3.3 Hyperparameters.

• The model uses dropout 0.1, learning rate 0.01, 32-dimensional product-bundle factors, two Riemannian layers, and a 256-dimensional attention-map hidden layer.The implementation uses PyTorch and Adam.Source: §E.3.3 · p. 12

• Node-classification geometric ablations show a pattern similar to link prediction, aligning trees with hyperbolic space and cycles with hyperspherical space.Source: F Additional Results · p. 12
• Performance remains stable across different pre-training datasets in the reported cross-domain node-classification results.Source: F Additional Results · p. 12
• The authors interpret the stable results as demonstrating the superiority of structural vocabulary based on common tree and cycle substructures.Source: F Additional Results · p. 12