Towards Graph Foundation Models: A Transferability Perspective

• GFMs extend foundation-model ideas to graph data, supporting tasks such as node classification, link prediction, and graph classification across real-world domains.Applications include social networks, molecular graphs, and citation networks.Source: §1 · p. 1
• Domain-specific GFMs generalize across datasets and tasks within one field, exemplified by TxGNN for medical prediction and GridFM for power-grid adaptation.TxGNN addresses drug efficacy across diseases, while GridFM targets transfer across regions and countries.Source: §1 · pp. 1–2
• General-purpose GFMs seek knowledge transfer across graph domains and tasks, although truly general-purpose systems remain an open challenge.Early advances suggest increasing feasibility and potential applications in scientific and medical discovery.Source: §1 · p. 2
• Transferability includes intra-domain adaptation, cross-domain generalization, and cross-task adaptation, each addressing different graph-data or learning-objective shifts.Domain-specific GFMs emphasize intra-domain and cross-task transfer, whereas general-purpose GFMs additionally require cross-domain transfer.Source: §1 · p. 2
• The survey analyzes GFMs through backbones, knowledge acquisition, and knowledge transfer, providing a transferability-centered taxonomy distinct from architecture-, recommendation-, vocabulary-, or self-supervision-focused surveys.Its goal is to structure current progress and identify pathways toward broader GFM generalization.Source: §1 · p. 2

2.1 Domain-Specific vs. General-Purpose GFMs

• The taxonomy separates domain-specific GFMs, optimized for tasks within one field, from general-purpose GFMs, designed to transfer knowledge across multiple graph domains.GridFM illustrates the first category, while AnyGraph illustrates the second.Source: §2 · p. 2, §2.1 · p. 2

2.2 Transferability: Core Requirement for GFMs

• Intra-domain transferability handles distribution changes across datasets in one domain, cross-domain transferability moves knowledge between domains, and cross-task transferability adapts models to different objectives.These capabilities define how GFMs generalize across datasets, domains, and tasks.Source: §2.2 · p. 3
• Domain-specific GFMs primarily require intra-domain and cross-task transfer, whereas general-purpose GFMs additionally target cross-domain transfer.A power-grid model may need to adapt across cities and countries while supporting fault detection and peak-load detection.Source: §2.2 · p. 3

2.3 Unified Taxonomy of GFMs

• Table 1 presents the unified taxonomy by combining application scope with transferability capabilities.Source: §2.3 · p. 3, Table 1 · p. 3
• Domain-specific models show transfer across specialized applications, including knowledge graphs, biology, transportation, and algorithmic tasks.Examples include ULTRA, MiniMol, COATI, JMP, TxGNN, Triplet-GMPNN, TFM, and TransWorldNG.Source: §2.3 · p. 3
• General-purpose GFM research focuses on cross-domain transfer, but many systems still lack explicit intra-domain evaluation and some do not achieve cross-task transferability.OpenGraph reports zero-shot cross-domain performance, while several models remain focused on node classification across domains.Source: §2.3 · p. 3

• The backbone model provides the architecture for processing graph data and learning representations.A strong backbone must capture structural relationships, node features, and semantic information.Source: §3 · p. 3
• Knowledge acquisition builds transferable graph knowledge from data using supervised, contrastive, or generative learning techniques.Domain-specific GFMs acquire knowledge within one domain, whereas general-purpose GFMs integrate knowledge from multiple domains.Source: §3 · p. 4
• Knowledge transfer determines how effectively acquired knowledge is applied to new tasks or domains despite differing task emphases and data distributions.Source: §3 · pp. 3–4
• The survey reviews existing GFMs through these three components, emphasizing design choices and strategies for improving transferability.Source: §3 · p. 4

4 Backbone Model

• Backbones need expressiveness for complex features and topologies, flexibility across unseen structures and tasks, and scalability for large datasets.These properties jointly support transferable graph representations.Source: §4 · p. 4

4.1.1 GNN-based Models.

• GNNs use local message passing and remain dominant, but locality bias limits global-structure understanding, cross-domain flexibility, and scalability.Sampling can mitigate scalability costs but does not remove the underlying locality limitation.Source: §4.1.1 · p. 4, §4.1 · p. 4

4.1.2 Transformer-based Models.

• Transformers capture local and global dependencies through attention and are more flexible across graph types, but global attention is computationally expensive at scale.They may also be less advantageous than local aggregation on simpler tasks.Source: §4.1.2 · p. 4, §4.1 · p. 4

4.1.3 LLM-based Models.

• LLM-based models convert graphs into token sequences, offering strong expressiveness and task flexibility but limiting efficient structural transfer and large-graph scalability.Their sequential representation can capture semantic relationships while making graph processing less efficient.Source: §4.1.3 · pp. 4–5

4.2 Backbone Models of Existing GFMs

• Existing GFM backbones include GNNs for structural representations, transformers for latent or tokenized graph modeling, LLMs for language-mediated reasoning, and hybrids for structural-semantic integration.Examples include AnyGraph, GraphFM, OpenGraph, InstructGLM, GraphGPT, COATI, and UniGraph.Source: §4.2.1 · p. 5, §4.2.2 · p. 5, §4.2.3 · p. 5

4.3 Discussion

• The best backbone for GFMs remains unclear: GNNs are current strong contenders, transformers may gain importance with scale, and instruction-tuned LLMs show flexibility.Hybrid models combine complementary backbone strengths for task and domain transfer.Source: §4.3 · p. 5

• Pre-training is the key approach for acquiring meaningful graph knowledge and improving GFM transferability.The paper frames pre-training as the basis for learning knowledge that can transfer across tasks and datasets.Source: §5 · p. 5
• Cross-task pre-training uses data from varied tasks to learn task-specific patterns while reducing the task gap.Different tasks emphasize distinct graph aspects, so pre-training must support efficient learning across those differences.Source: §5 · p. 5
• Intra-domain and cross-domain pre-training must accommodate structural and feature variation across datasets to prevent negative transfer.General-purpose GFMs face variation across graph sources, while even datasets within one domain can differ substantially.Source: §5 · p. 5

5.1 Mitigating Task Gap

• Different graph tasks focus on distinct graph aspects, making task-gap mitigation essential for learning transferable task-specific knowledge.The section identifies subgraph sampling and multi-task learning as two strategies for acquiring relevant knowledge across tasks.Source: §5.1 · p. 5

5.1.1 Subgraph sampling.

• Subgraph sampling unifies node-, edge-, and graph-level tasks by converting them into graph-level operations while preserving structural details.ProG [64] applies this conversion, GCC [57] learns universal patterns through subgraph discrimination, and Prodigy [36] captures local context for in-context learning.Source: §5.1.1 · pp. 5–6
• Multi-task pre-training captures diverse patterns, including node connectivity and global graph structure, to support transfer across tasks.Triplet-GMPNN [37], MiniMol, and JMP [62] use multiple tasks or datasets to learn transferable representations.Source: §5.1.2 · p. 6

5.2 Mitigating Graph Disparity

• Graph disparity across topologies and features can cause negative transfer, so pre-training aligns datasets within a unified graph space.The paper describes alignment as strengthening connections while harmonizing both features and topologies.Source: §5.2 · p. 6

5.2.1 Aligning graph structures and features with SVD.

• SVD aligns topology and feature dimensions by mapping normalized adjacency matrices and initial features into shared lower-dimensional spaces.OpenGraph [87] and AnyGraph [86] align topology, while ProG [64] and GCOPE [105] unify feature dimensions.Source: §5.2.1 · p. 6

5.2.2 Using PLM for semantic alignment.

• Pre-trained language models encode textual node attributes and descriptions into shared semantic spaces when SVD cannot capture rich textual relationships.ZeroG [45] uses SentenceBERT [59], while GraphAlign [30], BooG [11], and OFA [47] embed textual node features.Source: §5.2.2 · p. 6
• Graph prompts use virtual nodes or learnable tokens as bridges that align domain-specific features while retaining graph-specific properties.MDGPT [95], ZeroG [45], GraphFM [44], and GCOPE [105] implement domain-linked tokens, latent tokens, or coordinators.Source: §5.2.3 · p. 6

5.3.1 Supervised learning.

• Supervised pre-training directly learns task-specific patterns from labels and can improve adaptation across graph datasets and tasks.GraphFM [44], Prodigy [36], MiniMol, and JMP [62] use labeled or multi-task supervision for transferable representations.Source: §5.3.1 · p. 6

5.3.2 Self-supervised learning.

• Supervised learning is limited by its reliance on large labeled datasets, motivating self-supervised learning for diverse unlabeled graph data.Self-supervised methods are divided into contrast-based and generation-based approaches.Source: §5.3.2 · p. 6
• Contrast-based self-supervision learns representations by contrasting positive and negative samples or aligning complementary graph modalities.Examples include GraphCL [93], DGI [73], link prediction, text–graph alignment, and 2D–3D molecular alignment.Source: §5.3.2 · pp. 6–7
• Generation-based self-supervision reconstructs masked graph data to capture structural and attribute semantics for downstream transfer.OpenGraph [87], GCOPE [105], and Unigraph [27] use masked reconstruction or graph modeling for structural learning and cross-domain alignment.Source: §5.3.2 · p. 7

• Multiple-dataset pre-training is common in GFMs to facilitate intra-domain and cross-domain transfer.Source: §5.4 · p. 7
• Intra-domain transferability receives limited attention, particularly for size variance and data distribution shifts within the same domain.Source: §5.4 · p. 7
• Supervised and contrastive learning are widely used for pre-training, while labeling costs may encourage greater use of contrastive or generative techniques.Source: §5.4 · p. 7

6 Knowledge Transfer

• Knowledge transfer adapts learned representations to downstream tasks or domains despite differences in graph structure, features, data distributions, and task requirements.Source: §6 · p. 7

6.1 Prompt-based Strategies

• Graph prompts support domain adaptation by aligning downstream graphs with pre-trained representations and task adaptation by guiding models across graph tasks.Source: §6.1 · p. 7

6.1.1 Graph prompt for domain adaptation.

• Prompt methods adjust downstream graph inputs using prompt tokens, structures, insertion patterns, virtual super nodes, or mixed domain-specific information.ProG, BooG, and MDGPT exemplify different mechanisms for improving compatibility across domains.Source: §6.1.1 · p. 7
• Cross-task prompts recast tasks, integrate examples and queries into unified graphs, or append task-specific prompt graphs to encode task semantics.OpenGraph and AnyGraph transform node classification into link prediction, while Prodigy and OFA use contextual task structures.Source: §6.1.2 · p. 7

6.2 Finetuning-based Strategies

• Vanilla fine-tuning is rarely used because it can overfit and perform unstably with limited data, motivating meta-learning, PEFT, and probing.Source: §6.2 · p. 8

6.2.1 Meta learning.

• Meta-learning acquires adaptable initializations or strategies from related meta-training tasks for efficient downstream adaptation.ProG refines graph prompts across node-, edge-, and graph-level tasks during meta-training and meta-testing.Source: §6.2.1 · p. 8

6.2.2 Parameter-efficient fine-tuning.

• Parameter-efficient fine-tuning updates only a small parameter subset, reducing computation and overfitting risk when data are scarce.UniGraph uses LoRA, while MiniMol trains a smaller MLP from molecular fingerprints produced by a pre-trained model.Source: §6.2.2 · p. 8

6.2.3 Probing.

• Probing trains lightweight networks for rapid, low-data transfer, but simple probes may underperform on complex downstream graph tasks.GraphFM and GCC use linear probes, whereas BooG, ProG, and COATI use more flexible probe architectures.Source: §6.2.3 · p. 8

Domain-Specific vs. General-Purpose GFMs

• Domain-specific GFMs provide efficient, strong intra-domain transfer through specialization but have limited adaptability across domains.Source: §7 · p. 8
• General-purpose GFMs aim to generalize across domains and tasks, but structural, feature, and data-availability differences make cross-domain transfer difficult.Source: §7 · p. 8
• A modular architecture combining domain-specific GFMs with shared representations is proposed as a possible route toward more effective general-purpose models.Source: §7 · p. 8

The Role of LLMs in GFMs

• LLM–graph model integration is presented as a direction for combining semantic reasoning and generalization with graph-structural awareness, despite current relational and scalability challenges.Source: §7 · p. 8

Standardized Benchmarking for Transferability

• Standardized benchmarks spanning domains and tasks are needed to evaluate transferability and identify factors limiting GFM progress.Source: §7 · p. 8

• The taxonomy categorizes GFMs by intra-domain, cross-domain, and cross-task transferability alongside domain-specific and general-purpose application scope.Source: §8 · p. 8
• The proposed framework comprises Backbone Models, Knowledge Acquisition, and Knowledge Transfer as key components influencing GFM transferability.Source: §8 · p. 8
• The survey structures current progress and challenges to guide development of more transferable GFMs for diverse real-world applications.Source: §8 · p. 8