PaperlayerView exact revision on arXiv

Source-linked AI summary

Complementary Supervised and Self-Supervised Representations for Out-of-Distribution Graph Learning

Qingying Hao, Zikang Chen, Chuxuan Hu, Jinyuan Jia, Bo Li, Gang Wang, Carl Gunter

arXiv:2610.07628v1cs.LG

TL;DR

Graph distributions shift over time and across domains, challenging GNN generalization, and SSL representations are not explicitly optimized for downstream classification. The paper develops Co-Train and Dual-Space Retrieval to combine supervised and SSL representations at different stages. Co-Train consistently outperforms strong OOD baselines, while Dual-Space Retrieval is competitive across diverse settings.

  • Problem

    Graph OOD generalization is challenging under temporal and cross-domain shifts, and it is unclear whether SSL representations can complement supervised representations despite not being optimized for downstream classification.

    Source: §1 · pp. 1–2

  • Method

    Co-Train jointly learns and adaptively integrates supervised and SSL representations during training, while Dual-Space Retrieval fuses confidence-aware non-parametric predictions from both spaces at inference.

    Source: §1 · p. 2

  • Results

    Across multiple graph datasets and distribution shifts, Co-Train consistently outperforms strong OOD generalization baselines, while Dual-Space Retrieval achieves strong performance.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    The results show that incorporating SSL as a complementary signal can improve supervised graph OOD generalization.

    Source: §6 · p. 10

Abstract

from arXiv · show

Out-of-distribution (OOD) generalization remains challenging for graph neural networks (GNNs), as graph distributions can vary substantially across time and domains. Supervised and self-supervised graph representation learning are guided by distinct objectives and offer different perspectives on graph representations. In this work, we study whether self-supervised representations (SSL) can provide complementary signals to improve supervised OOD node classification. We develop two backbone-agnostic frameworks that exploit such information at different stages of learning and prediction. Co-Train jointly learns supervised and SSL representations and adaptively integrates them during training, while Dual-Space Retrieval performs non-parametric prediction in the two representation spaces and combines their predictions through confidence-aware fusion at inference time. The supervised and SSL encoders are separately parameterized and need not share the same GNN architecture. We evaluate multiple GNN backbones and two distinct SSL objectives, DGI and GRACE, on four graph benchmarks spanning temporal and cross-domain distribution shifts. Extensive experiments show that Co-Train consistently outperforms strong supervised OOD baselines, while Dual-Space Retrieval achieves competitive performance as a flexible non-parametric alternative. Results across different backbones and SSL objectives, together with representation analyses and ablations, demonstrate that SSL representations provide complementary information to supervised representations and can improve OOD node classification across diverse settings.

1 Introduction

Graph OOD generalization is difficult because graph distributions shift over time and across domains, while supervised representations may overfit label-correlated patterns. The paper tests whether SSL representations add complementary information and proposes two ways to combine them, reporting consistent gains for Co-Train and competitive results for Dual-Space Retrieval.

1 Introduction

  • Graph distributions shift over time and across domains, so the same-distribution assumption of most GNNs can fail and degrade performance.Examples include changing transaction patterns and structural or feature differences between social communities.Source: §1 · p. 1
  • Supervised representations may capture label-correlated patterns that are unstable under shifts, while SSL representations are not task-optimized and can retain irrelevant information.Graph augmentations may also discard task-relevant information.Source: §1 · pp. 1–2
  • Co-Train adaptively integrates separately parameterized supervised and SSL representations during training; Dual-Space Retrieval fuses confidence-weighted predictions from both spaces at inference.The encoders need not share parameters or use the same GNN architecture.Source: §1 · p. 2
  • Co-Train consistently improves over strong supervised OOD baselines across four benchmarks, while Dual-Space Retrieval achieves competitive performance as a non-parametric alternative.The benchmarks cover temporal and cross-domain shifts; retrieval costs can be reduced with class-centroid representations.Source: §1 · p. 2
  • The study examines whether SSL representations provide useful information beyond label-guided supervised representations for graph OOD generalization.Its empirical analyses find distinct signals from the two representation types, and improved OOD generalization when they are integrated.Source: §1 · p. 2

2 Related Work

Related work spans GNN architectures, supervised and self-supervised approaches to distribution shifts, and distance-based retrieval. The paper positions its contribution as studying representational complementarity under OOD and combining the spaces through training-time and inference-time fusion.

2.1 Graph Neural Networks

  • GNNs include spectral and neighborhood-based methods, with architectures such as GCN, GraphSAGE, GAT, GCNII, and GPR-GNN addressing different modeling needs.The cited work also covers scalability and trustworthiness.Source: §2.1 · p. 2

2.2 Distribution Shift Detection in Graph Neural Networks

  • Supervised OOD methods use synthetic environments, invariant predictors, or other mechanisms, but supervised representations can specialize to unstable label-correlated patterns.Methods discussed include DIR, EERM, LiSA, GRM, GNNSafe, and KnowGraph.Source: §2.2 · pp. 2–3
  • SSL methods learn without labels, and this paper distinguishes its OOD focus on representational complementarity from prior combinations of supervised and unsupervised objectives.Prior work includes graph contrastive methods and OOD-focused approaches such as MARIO.Source: §2.2 · p. 3

2.3 Distance-Based Retrieval Methods for Robust Classification

  • Distance-based retrieval is less explored for graph OOD generalization; Dual-Space Retrieval combines similarity-weighted class scores from supervised and SSL spaces through confidence-aware fusion.The framework provides inference-time prediction without an additional classifier.Source: §2.3 · p. 3

3 Preliminaries

The preliminaries define graph distribution shifts and describe supervised and self-supervised node representations. They outline label-guided GNN classification alongside DGI and GRACE objectives and the use of frozen SSL encoders for downstream classification.

3.1 OOD in Graph Structured Data

  • Graph OOD generalization here concerns naturally occurring shifts across domains or time, where training and test distributions differ in features, structure, labels, or their relationships.The passage expresses distribution shift as P_train(X, Y) ≠ P_test(X, Y).Source: §3.1 · p. 3

3.2 Supervised Graph Representation Learning.

  • Supervised graph representation learning uses node labels to learn task-discriminative representations, with a GNN encoder applying neighborhood message passing and a classification head predicting labels.The encoder and head are jointly optimized using cross-entropy loss.Source: §3.2 · p. 3

3.3 Self-Supervised Graph Representation Learning.

  • DGI maximizes local-node/global-summary agreement while distinguishing corrupted views, whereas GRACE contrasts each node's representations across augmented views against other nodes.These are local–global and node-level cross-view contrastive objectives, respectively.Source: §3.3 · pp. 3–4
  • For downstream classification, the SSL encoder is frozen and a trainable linear classifier is learned on labeled training nodes.The frozen encoder supplies the node representations used by the classifier.Source: §3.3 · p. 4

4 Methodology

The methodology uses self-supervised representations as complementary signals through two backbone-agnostic approaches: Co-Train integrates them during training, while Dual-Space Retrieval combines predictions at inference. Both use independently parameterized supervised and SSL encoders, which need not share a GNN architecture.

4.1 Co-Train

  • The frameworks support different GNN architectures across branches, and Co-Train tests both DGI and GRACE as distinct SSL objectives.DGI represents local–global learning, while GRACE represents node-level cross-view contrastive learning.Source: §4.1 · p. 4, §4 · p. 4
  • Co-Train jointly optimizes supervised and SSL encoders, using a learned gate to adaptively add the SSL representation to the supervised base and an alignment objective to encourage compatibility.The SSL loss and alignment loss contributions are controlled separately.Source: §4.1 · p. 4

4.2 Dual-Space Retrieval

  • For retrieval, each branch can use a class-centroid memory bank, and graph-aware feature smoothing averages node representations with neighbors before centroid construction.The smoothing is intended to reduce local representation variation and yield more representative class centroids.Source: §4.2 · p. 5
  • Dual-Space Retrieval performs independent non-parametric prediction in supervised and SSL spaces, then weights their scores by branch confidence, emphasizing the supervised prediction when hard-label disagreement exceeds 0.5.The disagreement case uses α_sup = 1 and a small fixed α_ssl, such as 0.01.Source: §4.2 · p. 5

Datasets.

Evaluation spans temporal and cross-domain graph shifts: Co-Train consistently improves over strong supervised OOD baselines, while Dual-Space Retrieval remains competitive. Ablations and representation analyses support complementary signals from supervised and self-supervised learning.

Datasets.

  • The four benchmarks cover temporal shifts on Elliptic and OGB-Arxiv and cross-domain shifts on Twitch and FB-100, with multiple backbones and DGI or GRACE evaluations.The experiments report test performance averaged over 10 runs.Source: Datasets · pp. 5–7

5.2 Temporal Generalization Performance

  • On Elliptic, CoTrain-GRACE with GCNII averages 79.79% F1 versus 75.57% for MARIO, with Co-Train gains sustained across temporal periods.Dual-Space Retrieval is also competitive: with GraphSAGE, Retrieval-GRACE averages 75.24% F1 versus the strongest baseline's 75.21%.Source: Table 1 · p. 6, §5.2 · p. 7
  • On OGB-Arxiv, Co-Train improves over LiSA across all three temporal splits, while feature-smoothed Retrieval* exceeds the strongest baseline by up to 15.62 percentage points.The best reported Retrieval* accuracies with GraphSAGE are 52.01%, 54.15%, and 56.66% across the three splits.Source: §5.2 · p. 7, Table 2 · p. 8

5.3 Cross-domain Generalization Performance

  • On Twitch, Co-Train improves average ROC-AUC over the strongest baseline by 2.01–2.68 points across the reported GCN and GCNII settings; Retrieval remains competitive.Retrieval-DGI averages 59.01% ROC-AUC with GCN and 59.60% with GCNII.Source: §5.3 · p. 7, Table 3 · p. 8
  • On FB-100, Retrieval-DGI exceeds the strongest baseline on Penn and Brown, while Co-Train remains competitive across source-graph combinations.On Texas, CoTrain-DGI achieves 56.89% under WUSTL+BRD+CMU, with the reported improvements more modest.Source: §5.3 · pp. 7–8, Table 4 · p. 9

5.4 Ablation Study

  • On OGB-Arxiv, Co-Train beats either individual objective across all three temporal splits; on Twitch, DGI dual-space retrieval beats either individual space across all five domains, gaining up to 2.89 percentage points.Co-Train’s gains over the stronger branch were 2.58, 3.16, and 3.27 points with DGI, and 2.40, 2.50, and 2.96 points with GRACE, across the 2014–2016, 2016–2018, and 2018–2020 periods, respectively. With GRACE on Twitch, combined prediction improves over both individual spaces on ES, FR, and PTBR and is comparable on RU and TW.Source: §5.4 · p. 8

5.5 Additional Analysis

  • On OGB-Arxiv, Co-Train has the highest class-separation score with both SSL objectives and yields more compact classes with clearer boundaries; it is also the fastest method compared.Centroid-based Dual-Space Retrieval has runtime comparable to ERM, and all compared methods fit within GPU memory.Source: §5.5 · pp. 9–10, Figure 5 · p. 9

6 Conclusion

The paper examines how self-supervised representations can complement supervised learning for graph OOD generalization. Co-Train and Dual-Space Retrieval achieve strong performance across datasets and shifts, with Co-Train consistently outperforming strong OOD baselines.

6 Conclusion

  • Co-Train consistently outperforms strong OOD generalization baselines across multiple graph datasets and distribution shifts.Source: §6 · p. 10
  • Both Co-Train and Dual-Space Retrieval achieve strong performance across multiple graph datasets and distribution shifts.Source: §6 · p. 10
  • Ablation and representation analyses indicate that SSL provides complementary information and improves representation separation.The paper concludes that incorporating SSL as a complementary signal can improve supervised graph OOD generalization.Source: §6 · p. 10

A Datasets

The evaluation uses four graph datasets spanning temporal and cross-domain shifts. On OGB-Arxiv, the visualization and quantitative analysis indicate clearer class structure after Co-Train integrates supervised and SSL representations.

A Datasets

  • Four datasets cover temporal and cross-domain shifts, with tasks including illicit-transaction detection, paper subject prediction, streamer-language classification, and user-gender prediction.Elliptic and OGB-Arxiv evaluate temporal generalization; Twitch-Explicit and Facebook-100 use networks from different regions or universities for cross-domain evaluation.Source: A Datasets · pp. 10–11

B.1 Visualization of Learned Representations.

  • Co-Train-DGI achieves the highest representation-separation score on OGB-Arxiv, with t-SNE showing clearer class structure after integration.The analysis uses test-node representations from the 2018–2020 temporal split and compares supervised, SSL, and Co-Train representations.Source: §B.1 · p. 11, Figure 7 · p. 11

B.2 Neighbor-Based Retrieval Results on OGB-Arxiv.

On OGB-Arxiv, neighbor retrieval can be too local for the 40-class task, motivating class-centroid retrieval. Feature smoothing increases neighborhood label purity, and the feature-smoothed Retrieval* variant substantially improves over centroid-based Retrieval.

B.2 Neighbor-Based Retrieval Results on OGB-Arxiv.

  • With K=30, DGI and GRACE neighborhoods average about 11 unique classes, with label purity of 23.31% and 21.52%, respectively.Source: §B.2 · p. 11, Table 8 · p. 12
  • At K=30, the ground-truth class appears in Top-1 for 35.14% and 28.97% of validation nodes, and in Top-3 for 56.22% and 49.59%, for DGI and GRACE, respectively.Increasing K to 100 expands neighborhoods to about 19 classes, but purity remains low and Top-1 performance changes little.Source: §B.2 · p. 11
  • Class-centroid retrieval is used on OGB-Arxiv because local neighborhoods cover only some classes, limiting voting against classes outside the neighborhood.Centroids explicitly represent every class and enable prediction over the full class space.Source: §B.2 · p. 11

B.3 Feature Smoothing for Centroid-Based Retrieval on OGB-Arxiv.

  • Feature smoothing increases training-node label purity, and Retrieval* substantially improves over corresponding centroid-based Retrieval results.The comparison uses GAT as the supervised backbone and GraphSAGE as the SSL encoder.Source: §B.3 · p. 12, Table 9 · p. 12

B.4 Efficiency and Memory Usage Comparison.

Co-Train is the fastest method in the OGB-Arxiv runtime comparison, while Dual-Space Retrieval is also lightweight. Retrieval performance remains stable across the tested SSL fusion weights and retrieval sizes.

B.4 Efficiency and Memory Usage Comparison.

  • Dual-Space Retrieval requires 69.16 seconds and its feature-smoothed variant 96.52 seconds; both are substantially faster than MARIO and EERM.The retrieval runtime is relatively low on OGB-Arxiv because class-centroid retrieval uses only a small number of representatives.Source: §B.4 · p. 12
  • Co-Train requires 43.75 seconds, compared with 70.83 seconds for ERM, 50.29 seconds for LiSA, 206.19 seconds for MARIO, and more than two hours for EERM.It is the most efficient method in the comparison despite its strong OOD performance.Source: §B.4 · p. 12, Table 10 · p. 13

C.1 Sensitivity to the SSL Fusion Weight for Dual-Space Retrieval.

  • With SSL fusion weights from 0 to 0.1, 2018–2020 accuracy ranges from 44.92% to 45.25%, and performance is similar across all three temporal splits.The experiments use a small fixed SSL weight because the branches have prediction disagreement greater than 0.5; the paper sets α_ssl=0.01 for OGB-Arxiv.Source: Table 11 · p. 13, §C.1 · p. 13

C.2 Sensitivity to the Retrieval Size 𝐾.

  • Dual-Space Retrieval performs comparably across centroid retrieval sizes K∈{10, 20, 30, 40} and remains stable across neighbor sizes K∈{30, 100, 200, 300, 500}.The neighbor-size comparison covers nine Elliptic test splits; K=40 retrieves all OGB-Arxiv class centroids.Source: Table 12 · p. 13, §C.2 · p. 13, Table 13 · p. 13

C.3 Sensitivity to the SSL Encoder Architecture.

The SSL encoder can be selected independently of the supervised backbone; across Elliptic and OGB-Arxiv, results suggest Co-Train is not highly sensitive to that choice.

C.3 Sensitivity to the SSL Encoder Architecture.

  • The supervised and SSL backbones are selected independently because the encoders are separately parameterized without parameter sharing.Validation performance selects GCN for Elliptic and GraphSAGE for OGB-Arxiv; Dual-Space Retrieval uses the same selected configurations.Source: §C.3 · p. 13
  • Co-Train is not highly sensitive to SSL encoder architecture, with only small differences across Elliptic and OGB-Arxiv.On Elliptic, the encoders are comparable across nine test splits; on OGB-Arxiv, GCN is slightly better on all three test periods despite GraphSAGE being selected on validation performance.Source: §C.3 · p. 13
  • 0.41, 0.37, and 0.58 percentage points: GCN performs slightly better than GraphSAGE on the three OGB-Arxiv test periods.GraphSAGE was selected based on validation performance.Source: §C.3 · p. 13

D Additional Experimental Results

Additional results show Co-Train leading on most Twitch target domains, while FB-100 outcomes vary across training-domain combinations, with different methods achieving the best results.

D Additional Experimental Results

  • 1.51, 0.06, 2.55, and 0.72 percentage points: Co-Train exceeds the strongest existing baseline on Twitch target domains ES, FR, PTBR, and RU.It achieves the best performance on four of five target domains; Dual-Space Retrieval is less competitive under the GAT backbone, suggesting greater sensitivity to the learned representation space.Source: D Additional Experimental Results · p. 14
  • 56.89% accuracy: CoTrain-DGI achieves the best overall result under WUSTL+BRD+CMU, exceeding the strongest baseline by 0.26 percentage points.The comparison is on FB-100 with Texas as the test domain, using GCN for both the supervised backbone and SSL encoder.Source: D Additional Experimental Results · pp. 13–14, Table 16 · p. 14
  • 56.11% and 56.08% accuracy: GRACE achieves the best results under JHU+CIT+AMH and BIN+DUK+PRI, respectively.Across the three FB-100 training-domain combinations, performance varies more noticeably.Source: D Additional Experimental Results · p. 14
Loading 2610.07628v1…