Source-linked AI summary
Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning
Xiao Wang, Nian Liu, Hui Han, Chuan Shi
TL;DR
HGNNs often require scarce labels, motivating self-supervised learning for heterogeneous information networks. HeCo addresses this with cross-view contrastive learning over network-schema and meta-path views, and the paper reports superior performance across diverse evaluations.
Problem
Most HGNNs use semi-supervised learning even though accurate node labels can be scarce or expensive to obtain in real-world HINs.
Method
HeCo contrasts network-schema and meta-path views, using view masking and two extensions that generate higher-quality negative samples.
Results
HeCo generally outperforms state-of-the-art methods, including a semi-supervised method, across four public datasets and multiple evaluation settings.
Takeaways & Limitations
Cross-view contrastive learning enables the two HIN views to mutually supervise and collaboratively learn node embeddings capturing local and high-order structures.
Abstract
from arXiv · showhide
Heterogeneous graph neural networks (HGNNs) as an emerging technique have shown superior capacity of dealing with heterogeneous information network (HIN). However, most HGNNs follow a semi-supervised learning manner, which notably limits their wide use in reality since labels are usually scarce in real applications. Recently, contrastive learning, a self-supervised method, becomes one of the most exciting learning paradigms and shows great potential when there are no labels. In this paper, we study the problem of self-supervised HGNNs and propose a novel co-contrastive learning mechanism for HGNNs, named HeCo. Different from traditional contrastive learning which only focuses on contrasting positive and negative samples, HeCo employs cross-viewcontrastive mechanism. Specifically, two views of a HIN (network schema and meta-path views) are proposed to learn node embeddings, so as to capture both of local and high-order structures simultaneously. Then the cross-view contrastive learning, as well as a view mask mechanism, is proposed, which is able to extract the positive and negative embeddings from two views. This enables the two views to collaboratively supervise each other and finally learn high-level node embeddings. Moreover, two extensions of HeCo are designed to generate harder negative samples with high quality, which further boosts the performance of HeCo. Extensive experiments conducted on a variety of real-world networks show the superior performance of the proposed methods over the state-of-the-arts.
1 INTRODUCTION
HGNNs typically depend on scarce node labels, motivating self-supervised learning for heterogeneous information networks. HeCo addresses this gap with cross-view contrastive learning over network-schema and meta-path views, enhanced by masking and harder negatives.
- Motivation: HGNNs commonly use semi-supervised learning, but obtaining accurate labels can be challenging or expensive in real-world settings.Gene labeling is given as an example requiring substantial molecular-biology expertise.
- Motivation: Contrastive learning derives supervision from positive and negative samples, enabling discriminative embeddings without labels.The passage contrasts its broad use in vision and language with limited investigation for HINs.
- Design challenges: HIN contrastive learning must handle multiple node types, relations, and semantically distinct meta-paths rather than relying on one view.The paper identifies heterogeneous cross-view contrast as a central design problem.
- Design challenges: Network schema and meta-path views jointly cover local direct connections and high-order structures in a HIN.The paper argues that selecting views with both structural scopes is necessary.
- Proposed approach: HeCo uses cross-view contrastive learning between network-schema and meta-path views, with view masking to diversify them and extract higher-level factors.The two views collaboratively supervise node-embedding learning.
- Contributions: HeCo adds two extensions for higher-quality negative samples and outperforms state-of-the-art methods, including a semi-supervised method, across four public datasets.The reported experiments evaluate the method under diverse settings.
2 RELATED WORK
Related work covers heterogeneous graph neural networks and graph contrastive learning, while identifying a lack of cross-view contrastive methods for HINs. Figure 1 illustrates the underlying HIN, meta-path, and network-schema concepts.
- Heterogeneous Graph Neural Network: HGNN research includes attention, meta-path, connection-identification, and web-scale approaches, while unsupervised heterogeneous learning remains a related direction.Examples include HAN, MAGNN, GTN, and HGT.
- Contrastive Learning: Graph contrastive methods learn representations by contrasting positive and negative pairs, with graph examples including DGI and GMI.The passage situates these methods within broader contrastive-learning research.
- Illustration: Figure 1 presents a toy ACM HIN alongside illustrations of its meta-path and network-schema representations.The figure’s conceptual comparison is between direct type-level structure and composite relational paths.
- Contrastive Learning: Prior HIN approaches contrast original and corrupted networks within individual meta-path views, whereas cross-view contrast for capturing high-level factors is identified as lacking.This distinction motivates HeCo’s cross-view design.
3 PRELIMINARY
The preliminary section defines HINs, network schemas, and meta-paths. Together, these concepts distinguish heterogeneous node and relation types, direct local structure, and composite high-order semantics.
- Heterogeneous Information Network: An HIN is a typed network G = (V, E, A, R, 𝜙, 𝜑) with node and edge sets, type mappings, and multiple object or link types.The definition requires |A + R| > 2.
- Heterogeneous Information Network: In the toy example, authors, papers, and subjects are node types connected by write and belong-to relations.Authors write papers, and papers belong to subjects.
- Network Schema: A network schema T_G = (A, R) is a directed meta-template over object types, with relations as its edges.It abstracts the type-level organization of the HIN.
- Network Schema: Network schema describes direct connections between different node types and therefore represents local structure.The example schema records that papers are written by authors and belong to subjects.
- Meta-path: A meta-path is a typed path whose composite relation is R = R_1 ◦ R_2 ◦ · · · ◦ R_l between endpoint node types.The notation abbreviates the sequence of node types and relations.
- Meta-path: Meta-paths such as PAP and PSP encode semantic similarity through multiple relations and are treated as high-order structure.PAP links papers sharing an author, while PSP links papers sharing a subject.
4 THE PROPOSED MODEL: HeCo
HeCo learns heterogeneous node embeddings from complementary network-schema and meta-path views, then uses masked cross-view contrastive learning so the views supervise each other. It also extends negative-sample generation to provide harder negatives.
- Dual-view encoding: HeCo encodes each node through network-schema and meta-path views to capture local and high-order HIN structures.The network-schema view aggregates direct neighbors, while the meta-path view uses meta-path-specific GCNs and semantic attention.
- Node feature transformation: Type-specific mappings project heterogeneous node features into a common latent space before view-specific encoding.Each node uses a mapping matrix determined by its node type; the transformation also applies activation and bias.
- Network schema view: Hierarchical attention aggregates schema neighbors at node and type levels, while random neighbor sampling equalizes information and increases embedding diversity.When neighborhoods exceed a threshold, neighbors are sampled; otherwise, selection is repeated to maintain a fixed amount of aggregated information.
- Meta-path view: Meta-path-specific encoders produce multiple semantic embeddings, which semantic-level attention fuses into the meta-path-view representation.Each meta-path represents a semantic similarity, and its attention weight reflects its importance.
- View mask mechanism: The view mask hides self-information in schema encoding and intermediate-node information in meta-path encoding, making the two representations complementary.The resulting correlated but complementary embeddings can mutually supervise training.
- Collaborative contrastive optimization: Cross-view contrastive optimization uses projected embeddings, multiple positives linked by meta-paths, and remaining nodes as negatives.The objective alternates view roles: one view supplies the target embedding while the other supplies positive and negative embeddings; two losses are balanced by λ.
- Model extensions: HeCo extensions generate additional or harder negatives, including Gaussian samples in the GAN-based HeCo_GAN variant.The extensions are motivated by the importance of harder negative samples for contrastive learning.
5 EXPERIMENTS
Experiments across classification, clustering, visualization, ablations, collaborative attention, extensions, and hyper-parameter analyses consistently support HeCo’s effectiveness and the value of cross-view contrastive learning.
- Node Classification: HeCo generally outperforms all baselines across datasets, label splits, and evaluation metrics, including the semi-supervised HAN method.Node embeddings are evaluated with a linear classifier using Macro-F1, Micro-F1, and AUC.
- Node Clustering: HeCo achieves the best clustering results on all datasets, improving ACM NMI by about 10% and ARI by about 20%.Clustering uses K-means with NMI and ARI, averaged over 10 runs.
- Visualization: Silhouette scores for ACM visualizations rise from 0.0292 in (a) to 0.3642 in (d), with HeCo showing clearer class boundaries than comparison methods.The visualization compares Mp2vec, DGI, DMGI, and HeCo using t-SNE, where colors represent labels.
- Variant Analysis: HeCo consistently outperforms its single-view variants, while HeCo_mp remains competitive and HeCo_sc performs worse than the other methods.The comparison uses ACM and DBLP with 40 labeled nodes per class.
- Collaborative Trend Analysis: Attention trends in ACM and AMiner are collaborative and consistent across network-schema and meta-path views during training.ACM emphasizes type A and meta-path PAP, while AMiner emphasizes type R and meta-path PRP.
- Extension and Hyper-parameter Analysis: HeCo_GAN improves results with a clear margin over HeCo and DMGI, while HeCo_MU is second-best in most cases.The extensions generate harder or more numerous high-quality negative samples; positive thresholds perform best at 7 for ACM and 15 for AMiner.
6 CONCLUSION
HeCo uses network schema and meta-path views for cross-view contrastive learning, with mutual supervision and harder negative samples. Extensive experiments and collaborative view trends verify its effectiveness.
- HeCo captures local and high-order structures through network schema and meta-path views, which mutually supervise node-embedding learning.
- A view mask mechanism and two HeCo extensions make contrastive learning harder by generating higher-quality negative samples.
- Collaboratively changing trends between the two views, together with extensive experiments, verify HeCo's effectiveness.
A SUPPLEMENT
The supplement provides reproducibility resources, including baseline and dataset websites plus detailed implementation settings.
- The supplement provides websites for all baselines and datasets to support reproducibility.
- Implementation details are included in the supplement.
- Detailed hyper-parameter values are also provided.
A.1 Baselines
The supplement lists publicly available implementations of the baselines used in the study.
- Publicly available baseline implementations can be found at the listed URLs.
- The supplement directs readers to online resources for reproducing baseline experiments.
- Baseline code access is provided through external URLs.
A.2 Datasets
The datasets used in the experiments are publicly accessible through listed online resources, including ACM, DBLP, Freebase, and AMiner.
- Dataset access is provided through publicly listed URLs.
- The dataset resources support access to the experimental data used in the study.
- The listed datasets include ACM, DBLP, Freebase, and AMiner.
A.3 Implementation Details
HeCo’s implementation uses specified learning, sampling, feature-dropout, and attention-dropout parameters, with neighbor-threshold sampling applied only to paper nodes in DBLP.
- Table 5 specifies HeCo’s learning rate, neighbor-sampling threshold, projected-feature dropout, and attention dropout parameters.The sampling threshold is denoted sample_num = T_Φm.
- For DBLP’s network schema, the author node connects only with paper nodes, so the neighbor threshold is set only for type P.
B DETAILS OF HeCo_GAN
HeCo_GAN first warms up HeCo, then alternates discriminator and generator training to produce higher-quality negative samples that are added to HeCo training.
- Training process: HeCo_GAN first trains HeCo alone for K0 epochs to warm up the embeddings used by the discriminator and generator.
- Training process: The discriminator is trained for KD epochs to distinguish positive cross-view samples from generator-produced negative samples.Under the network-schema view, it identifies positive meta-path-view samples and generated negatives; the meta-path-view objective is analogous.
- Training process: The generator is trained for KG epochs while the discriminator is frozen, improving generated samples by fooling the discriminator.
- Negative-sample generation: These discriminator and generator steps alternate for IDG times before the resulting negative samples are combined with original negatives.
- Negative-sample generation: The extended negative-sample set is fed into HeCo for KH epochs, and the HeCo_GAN training processes continue iteratively until convergence.