PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Domain Graph Foundation Models: Robust Knowledge Transfer via Topology Alignment

Shuo Wang, Bokui Wang, Zhixiang Shen, Boyan Deng, Zhao Kang

arXiv:2502.02017v2cs.SIcs.AIcs.LG

TL;DR

Multi-domain graph models face differences in topology and structural knowledge across domains. MDGFM adaptively balances features and topology, learns aligned structure, and applies prompt learning for transfer; experiments on homophilic and heterophilic benchmarks support its effectiveness and robustness.

  • Problem

    Existing methods use fixed topologies and uniform encodings across domains, while graph structural knowledge varies substantially across domains.

    Source: §1 · p. 1

  • Method

    MDGFM adaptively balances feature and topology information, uses graph structure learning to reduce noise and align structures, and adds prompt learning for transfer.

    Source: §1 · p. 2

  • Results

    Experiments on homophilic and heterophilic benchmark datasets demonstrate the method's effectiveness, robustness, and versatility.

    Source: §1 · p. 2

  • Takeaways & Limitations

    MDGFM explicitly addresses invariant information in topology across homophilic and heterophilic domains and targets transfer to unseen domains.

    Source: §7 · p. 9

    The theoretical analysis assumes an optimal invariant graph learner exists with invariance and sufficiency properties.

    Source: §5 · p. 5

Abstract

from arXiv · show

Recent advances in CV and NLP have inspired researchers to develop general-purpose graph foundation models through pre-training across diverse domains. However, a fundamental challenge arises from the substantial differences in graph topologies across domains. Additionally, real-world graphs are often sparse and prone to noisy connections and adversarial attacks. To address these issues, we propose the Multi-Domain Graph Foundation Model (MDGFM), a unified framework that aligns and leverages cross-domain topological information to facilitate robust knowledge transfer. MDGFM bridges different domains by adaptively balancing features and topology while refining original graphs to eliminate noise and align topological structures. To further enhance knowledge transfer, we introduce an efficient prompt-tuning approach. By aligning topologies, MDGFM not only improves multi-domain pre-training but also enables robust knowledge transfer to unseen domains. Theoretical analyses provide guarantees of MDGFM's effectiveness and domain generalization capabilities. Extensive experiments on both homophilic and heterophilic graph datasets validate the robustness and efficacy of our method.

1. Introduction

The introduction identifies cross-domain topology variation and graph noise as obstacles to general-purpose graph models. It presents MDGFM as a framework for aligning structure and transferring knowledge across domains.

  • Existing methods rely on fixed topologies and uniform encoding across domains, limiting generalizability amid diverse structural knowledge.Source: §1 · p. 1
  • Cross-domain graphs differ semantically, calling for adaptive use of features and topology to learn domain-invariant knowledge.Source: §1 · p. 1
  • Real-world graphs contain unreliable, irrelevant, misleading, or missing edges, and graph learning algorithms are susceptible to adversarial attacks.Source: §1 · p. 1
  • MDGFM combines decoupled embeddings with an adaptive balance token, graph structure learning, and prompt learning for source-to-target transfer.The framework uses structure learning to reduce noise and align topology, supporting robust transfer to unseen domains.Source: §1 · p. 2
  • Experiments on homophilic and heterophilic benchmark datasets demonstrate effectiveness, robustness, and versatility, while theoretical analyses support domain generalization.Source: §1 · p. 2

2. Related work

The related work reviews graph pre-training and multi-domain generalization, noting limits in cross-domain knowledge capture and topology reliability.

2.1. Graph Pre-training

  • Graph pre-training includes pretrain-and-fine-tune and pretrain-and-prompt-tuning approaches, but most prompt methods remain limited to single-domain pre-training and tuning.Source: §2.1 · p. 2

2.2. Multi-domain Generalization

  • Multi-domain graph generalization transfers knowledge from multiple sources to unseen domains, while prior methods align topologies or features and gaps remain in topology semantics and reliability.Source: §2.2 · pp. 2–3

3. Problem Definition

The problem definition represents graphs by their nodes, edges, features, and adjacency, then specifies pre-training on visible domains and testing on an unseen target domain.

  • Each graph is represented as G = (V, E, Xori) = (A, Xori), with nodes V, edges E, node features Xori, and adjacency matrix A.Source: §3 · p. 3
  • The model is pretrained on graphs from distinct visible domains and tested on a graph from an unseen target domain.Source: §3 · p. 3

4. Methodology

MDGFM combines multi-domain pre-training with target-domain adaptation, aligning features and graph topology before transferring knowledge through dual prompts. Its components address cross-domain semantic differences and topology refinement for downstream transfer.

4. Methodology

  • MDGFM has a multi-domain pre-training module and a downstream target-domain adaptation module.Pre-training unifies feature dimensions, refines source graphs with graph structure learning, and trains an encoder with self-supervised signals; adaptation combines meta and task-specific prompts.Source: §4 · p. 3

4.1. Feature Projection into Unified Semantic Space

  • PCA projects graph features into a shared dimension, while domain-specific and shared tokens align domain characteristics and transferable patterns.Dimensional alignment addresses numerical consistency but does not by itself resolve semantic differences between domains.Source: §4.1 · p. 3

4.2. Graph Topology-aware Alignment

  • Graph structure learning refines source and unseen target graphs using semantic and topological information, rather than feature similarity alone.The refinement synchronizes more reliable topology information across domains.Source: §4.2 · p. 4
  • Contrastive pre-training uses relationships across original and refined graphs to preserve global structure and semantic consistency.The loss incorporates the refined graph structure to reduce the information gap between original and refined graphs.Source: §4.2 · p. 4

4.3. Knowledge Transfer to Downstream Domain

  • Downstream adaptation combines meta prompts for learned cross-domain knowledge with task-specific prompts for target-domain alignment.Only prompt coefficients and the specific prompt need training during adaptation, and a learnable β can attenuate meta-prompt weights when source and target graphs differ substantially.Source: §4.3 · pp. 4–5
  • For node classification, the downstream objective compares node embeddings with class embeddings computed from labeled training instances.Class embeddings are mean embeddings of the training instances belonging to each class; similarity is computed using cosine similarity.Source: §4.3 · p. 5

5. Theoretical Analysis

The theoretical analysis frames domain generalization through invariant graph learning. Under stated invariance, sufficiency, and distribution-shift conditions, the paper relates this learning to a minimal error bound and an optimal predictor.

5. Theoretical Analysis

  • The analysis uses invariant graph learning to minimize an upper error bound for domain generalization.The paper implements the invariant graph learner through graph structure learning across domains.Source: §5 · p. 5
  • The assumed optimal invariant graph learner yields domain-invariant representations while retaining sufficient task-relevant information.The assumption requires label distributions conditional on the invariant representation to agree across environments, and expresses labels through that representation with independent noise.Source: §5 · p. 5
  • Under the theorem's conditions on invariant and variant subgraphs, the analysis gives an optimal predictor under distribution shifts.The stated conditions include independence between the invariant and variant subgraphs and distributional properties for invariant learners.Source: §5 · p. 6

6. Experiments

Experiments evaluate MDGFM on few-shot node classification across unseen target domains, testing transfer performance, component contributions, sensitivity to source-domain removal, and robustness to attacks. MDGFM outperforms baselines in one-shot settings and remains stable under source-domain removal and attack scenarios.

6.1. Experimental Setups

  • The evaluation uses GCN as the backbone and tests unseen targets by holding out one dataset at a time, plus a six-source setup with Penn94 as target.The experiments address few-shot transfer, component effects, source-domain sensitivity, and robustness to attacks and deletion.Source: §6.1 · p. 6, §6 · p. 6

6.2. Cross-domain Transfer Efficacy under Few-shot Learning Conditions (RQ1)

  • MDGPT surpasses GCOPE on homophilic graphs, while GCOPE leads on heterophilic graphs; GPPT generally underperforms, and GPF has high memory requirements.The passage attributes GCOPE’s homophilic result to noise from virtual nodes and MDGPT’s heterophilic result to limited capture of cross-domain commonalities.Source: §6.2 · p. 6
  • In few-shot settings, some methods lose performance on heterophilic graphs as shot count rises, while MDGFM avoids deterioration; on Penn94, its improvement grows with K.GCOPE and MDGPT show minimal change as K increases on Penn94.Source: §6.2 · pp. 6–7
  • MDGFM exceeds the second-best model by up to 18.04% in one-shot and 8.11% in few-shot scenarios across seven datasets, outperforming all baselines on homophilic and heterophilic graphs.The experiments designate one of six datasets as target and use the other five as sources, or use all six as sources for Penn94.Source: §6.2 · pp. 6–7, §6.1 · p. 6, Table 1 · p. 7

6.3. Ablation Study (RQ2)

  • Removing the shared token yields the weakest performance on most datasets, while removing graph structure learning, topology information, or the balance token also generally lowers performance.The ablations compare w/o-sumtoken, w/o-refinedadj, w/o-topology, and w/o-balance against MDGFM.Source: §6.3 · p. 7, Figure 2 · p. 7

6.4. Domain Sensitivity (RQ3)

  • With Cornell as target, MDGFM remains strong and stable when one to three source domains are removed, whereas GCOPE and MDGPT deteriorate significantly.In some cases, removing source domains improves performance, which the authors associate with conflicts or interference among domains.Source: §6.4 · pp. 7–8, Table 4 · p. 8

6.5. Robustness Analysis (RQ4)

  • Under random edge additions or removals and Metattack, MDGFM remains stable and outperforms all baselines as attack intensity increases.The authors attribute this robustness to topology alignment.Source: Figure 3 · p. 8, §6.5 · p. 8

7. Conclusion

MDGFM unifies graphs from diverse domains in a universal semantic space by aligning features and topology. The paper reports theoretical and empirical validation, including across homophilic and heterophilic domains.

7. Conclusion

  • MDGFM unifies graphs from diverse domains in a universal semantic space through feature and topology alignment.It extracts domain-invariant knowledge from both.Source: §7 · p. 9
  • The paper validates MDGFM’s effectiveness through theoretical analysis and empirical evaluation.Source: §7 · p. 9
  • The work explicitly addresses invariant information in topology structures across homophilic and heterophilic domains.Source: §7 · p. 9
  • Dynamic graphs and large-scale heterogeneous networks are identified as potential extensions.Source: §7 · p. 9

Impact Statement

This material states that Table 5 lists frequently used notations and describes the model’s pre-training and downstream complexity. It also characterizes the model as scaling linearly with nodes and edges.

Impact Statement

  • The impact statement says the work has potential societal implications but identifies no specific concerns requiring particular emphasis.Source: Impact Statement · p. 9

A. Notations

  • Table 5 is identified as a listing of frequently used notations.Source: Table 5 · p. 12

B. Time complexity analysis

  • The pre-training pipeline processes source graphs independently, using PCA, feature-topology weighting, kNN structure learning, and an L-layer GCN.The stated costs include O(|V| · d · log d′) for PCA, O(|V| · B · d) for graph reconstruction, and GCN computation.Source: B. Time complexity analysis · p. 12
  • Downstream processing adds PCA, prompt fusion and token modulation, kNN structure learning, GCN message passing, and prototype matching against C class centroids.Source: B. Time complexity analysis · p. 12
  • The model is described as scaling linearly with the number of nodes and edges, with efficient structure refinement and modular design.Source: B. Time complexity analysis · p. 12

C. Datasets

The datasets cover citation, web-page, Wikipedia, and online social networks. In robustness experiments, MDGFM maintains robustness on homophilic targets, while the relative robustness of GCOPE and MDGPT differs between heterophilic and homophilic graphs.

C. Datasets

  • The datasets include citation networks, Cornell web pages, Wikipedia page networks, and Penn94, a Facebook university-student social network.The dataset section refers to statistics in Table 6; the supplied passage gives no table values.Source: C. Datasets · p. 13, Table 6 · p. 13

D.1. Expansion of Robustness Experiments

  • On homophilic target graphs, MDGFM maintains commendable robustness under random edge additions and deletions across domains.The experiments compare MDGFM with MDGPT and GCOPE; Figure 4 reports robustness analysis with Cora as the target domain.Source: §D.1 · p. 13
  • The relative robustness of GCOPE and MDGPT reverses by graph type: GCOPE is more robust on heterophilic graphs, while MDGPT is more robust on homophilic graphs.Source: §D.1 · p. 13

D.2. Sensitivity Analysis

MDGFM is reported to have low sensitivity to the kNN parameter across homophilic and heterophilic targets. The preferred k range differs by target graph type.

D.2. Sensitivity Analysis

  • Figure 5 examines sensitivity to k, and performance is described as low-sensitivity across homophilic and heterophilic target domains.Source: §D.2 · p. 13, Figure 5 · p. 14
  • For homophilic target graphs, larger k values such as 20 or 30 yield the best results.Source: §D.2 · p. 14
  • For heterophilic target graphs, relatively smaller k values can produce satisfactory outcomes.Source: §D.2 · p. 14

E. Experimental Details

The experiments evaluate transfer to domains held out from pre-training, while the methodology refines graph structure and uses it to shape node-level mutual-information objectives.

E. Experimental Details

  • For one-shot and few-shot classification, models are pretrained on five datasets and tested on a remaining unseen domain; Penn94 is used only as a target domain.Source: E. Experimental Details · p. 14
  • Experiments average five runs; PCA maps initial features to 50 dimensions, with k=30 for homogeneous-graph GSL and k=15 for heterophilic-graph GSL.The setup also fixes three GCN layers and a 256-dimensional hidden layer; homophilic and heterophilic training use 60 and 100 epochs, respectively.Source: E. Experimental Details · p. 14

F. Methodology Details

  • Locality-sensitive-hashing kNN retains each node’s top-k edges, sparsifying the learned graph while reducing complexity from O(|V|^2) to O(|V|B).B is the batch size of sparse kNN.Source: F. Methodology Details · p. 15
  • The graph refinement pipeline symmetrizes edge connections, applies ReLU for non-negative weights, then normalizes the adjacency matrix with self-loops.Source: F. Methodology Details · p. 15

G. Details of Loss Functions

  • Because graph-level mutual information is impractical to compute, the objective uses node-representation mutual information as a sample-based differentiable lower bound.The score critic approximates the bound using joint and marginal distributions of node representations, with same-node views as positive samples.Source: G. Details of Loss Functions · pp. 15–16
  • The pre-training objective combines identity-based positives with refined-adjacency neighbors, increasing the positive samples; it is minimized sequentially across input graphs from different domains.The cosine-similarity temperature τc is fixed at 0.2.Source: G. Details of Loss Functions · p. 16
Loading 2502.02017v2…