Source-linked AI summary
Cluster-guided Contrastive Graph Clustering Network
Xihong Yang, Yue Liu, Sihang Zhou, Siwei Wang, Wenxuan Tu, Qun Zheng, Xinwang Liu, Liming Fang, En Zhu
TL;DR
Existing contrastive deep graph clustering methods are limited by augmentation-dependent positive samples and unreliable negative samples that underuse clustering information. CCGC uses un-shared Siamese encoders and high-confidence clustering results to construct more discriminative pairs, and experiments on six datasets demonstrate its effectiveness against state-of-the-art competitors.
Problem
Existing contrastive graph clustering methods can produce semantically drifted or indiscriminative positive samples and unreliable negative samples by underusing clustering information.
Method
CCGC uses parameter-unshared Siamese encoders and high-confidence clustering pseudo-labels to construct same-cluster positives and different-cluster-center negatives for contrastive learning.
Results
CCGC outperforms most compared methods on six datasets and four metrics, exceeding the runner-up on EAT by 5.06% ACC, 9.92% NMI, 4.46% ARI, and 4.34% F1.
Takeaways & Limitations
Mining high-confidence clustering information improves the discriminative capability and reliability of the positive and negative samples used by CCGC.
Takeaways & Limitations
Many existing deep graph clustering methods remain highly dependent on good initial cluster centers and manual trial-and-error pre-training.
Abstract
from arXiv · showhide
Benefiting from the intrinsic supervision information exploitation capability, contrastive learning has achieved promising performance in the field of deep graph clustering recently. However, we observe that two drawbacks of the positive and negative sample construction mechanisms limit the performance of existing algorithms from further improvement. 1) The quality of positive samples heavily depends on the carefully designed data augmentations, while inappropriate data augmentations would easily lead to the semantic drift and indiscriminative positive samples. 2) The constructed negative samples are not reliable for ignoring important clustering information. To solve these problems, we propose a Cluster-guided Contrastive deep Graph Clustering network (CCGC) by mining the intrinsic supervision information in the high-confidence clustering results. Specifically, instead of conducting complex node or edge perturbation, we construct two views of the graph by designing special Siamese encoders whose weights are not shared between the sibling sub-networks. Then, guided by the high-confidence clustering information, we carefully select and construct the positive samples from the same high-confidence cluster in two views. Moreover, to construct semantic meaningful negative sample pairs, we regard the centers of different high-confidence clusters as negative samples, thus improving the discriminative capability and reliability of the constructed sample pairs. Lastly, we design an objective function to pull close the samples from the same cluster while pushing away those from other clusters by maximizing and minimizing the cross-view cosine similarity between positive and negative samples. Extensive experimental results on six datasets demonstrate the effectiveness of CCGC compared with the existing state-of-the-art algorithms.
Introduction
Deep graph clustering is a challenging unsupervised task, and contrastive methods face limitations in constructing reliable positive and negative sample pairs. CCGC addresses these issues by using un-shared Siamese encoders and high-confidence clustering information.
- Deep graph clustering is a fundamental yet challenging unsupervised graph-learning task.
- Existing contrastive methods depend on graph augmentations that can cause semantic drift and indiscriminative positive samples, while their negative samples may neglect clustering information.
- CCGC constructs two node views with parameter-unshared Siamese encoders, avoiding complex structure- and attribute-level augmentations.
- High-confidence clustering pseudo-labels guide positive pairs from the same cluster, while different high-confidence cluster centers serve as reliable negative samples.
- CCGC uses an objective that maximizes cross-view cosine similarity for same-cluster samples and minimizes it for samples from different clusters.
- Extensive experiments on six benchmark datasets demonstrate CCGC’s effectiveness against existing state-of-the-art deep graph clustering competitors.
Related Work
Deep graph clustering methods include generative, adversarial, and contrastive approaches, each addressing representation learning and clustering through different mechanisms. Contrastive methods avoid some initialization difficulties but remain dependent on view construction and sample-pair quality, which CCGC targets with cluster-guided processing.
- Deep graph clustering methods are commonly grouped into generative, adversarial, and contrastive categories.
- Generative and adversarial methods often depend on good initial cluster centers, requiring manual trial-and-error pre-training.
- Contrastive methods replace clustering-guided losses with contrastive losses, thereby avoiding trial-and-error pre-training.
- Data augmentation: Existing contrastive approaches construct graph views through augmentations such as diffusion, edge perturbation, and feature masking.
- Positive and negative sample pair construction: Contrastive methods pull positive samples together and push negative samples apart, making sample-pair quality central to performance.
- Positive and negative sample pair construction: Existing methods use same-node pairs, shuffled features, decorrelation constraints, or randomly selected different-cluster samples, often relying on a well-pre-trained model for positive-pair selection.
Method
CCGC constructs semantically distinct node views with un-shared Siamese encoders, then uses high-confidence clustering information to build discriminative positive and reliable negative samples. Its joint objective pulls same-cluster samples together and pushes different-cluster centers apart.
- Parameter Un-shared Siamese Encoders: CCGC creates two node views with same-architecture Siamese encoders whose learnable parameters are not shared.The approach avoids complex graph augmentations and is intended to avoid semantic drift caused by inappropriate augmentations.
- Parameter Un-shared Siamese Encoders: A Laplacian filter first aggregates neighborhood information to produce a smoothed attribute matrix before MLP encoding.The filter uses the symmetric normalized graph Laplacian and a layer-number parameter t.
- Cluster-guided Contrastive Learning: CCGC performs K-means on fused embeddings and selects top-τ high-confidence samples with clustering pseudo labels for subsequent sample construction.The confidence score is based on each node embedding and the center of its assigned cluster.
- Cluster-guided Contrastive Learning: DPS constructs positive pairs from corresponding high-confidence samples grouped within the same cluster across the two views.This uses high-confidence pseudo labels as supervisory information to improve positive-sample discriminative capacity.
- Cluster-guided Contrastive Learning: RNS treats centers of different high-confidence clusters as negative samples, enhancing reliability and reducing false-negative possibilities.The centers are calculated separately in the two views using an averaging operation.
- Objective Function: The total loss combines positive-sample MSE and negative-sample cosine-similarity objectives as L = Lpos + αLneg.The positive objective pulls normalized cross-view samples together, while the negative objective pushes different high-confidence centers apart; α controls the trade-off.
Experiments
Experiments evaluate CCGC on six benchmark datasets through clustering comparisons, ablations, sensitivity analyses, and embedding visualizations. The results report stronger performance, essential sample-construction strategies, effective un-shared encoders, robust α sensitivity, and a useful τ range.
- Performance Comparison: CCGC outperforms most competing methods across six datasets and four clustering metrics, exceeding the runner-up on EAT by 5.06% ACC, 9.92% NMI, 4.46% ARI, and 4.34% F1.The comparison includes classical and contrastive deep graph clustering baselines.
- Ablation Studies: Removing either DPS or RNS decreases performance, indicating that both proposed sample-construction strategies contribute essentially to CCGC.The experiments also report improved positive- and negative-pair quality relative to regular construction.
- Ablation Studies: Un-shared Siamese encoders outperform alternatives using edge dropping, edge adding, graph diffusion, or feature masking, which may cause semantic drift.The alternatives randomly modify 20% of edges or features, while diffusion uses a 0.20 teleportation rate.
- Hyper-parameter Analysis: CCGC performs promisingly when τ ∈[50%, 70%]; below 50% few positive samples limit discrimination, whereas above 70% over-confident pseudo labels can cause confirmation bias.This sensitivity analysis covers six datasets.
- Hyper-parameter Analysis: Performance changes little as α varies, demonstrating insensitivity to α; the method is also reported insensitive to the Laplacian-filter layer number t.The α analysis is shown in Figure 4, while evidence for t is referenced in the appendix.
- Visualization Analysis: On CORA and AMAP, CCGC’s learned embeddings better reveal intrinsic clustering structure than those of six baselines in t-SNE visualizations.Figure 3 presents the two-dimensional visualizations, with CORA in the first row and AMAP in the second.
Conclusion
CCGC improves positive and negative sample quality by using un-shared-parameter Siamese encoders and high-confidence clustering pseudo labels. Experiments on six datasets demonstrate its effectiveness.
- CCGC constructs two graph views with un-shared-parameter Siamese encoders to avoid semantic drift from inappropriate graph augmentations.The method avoids complex structure- and attribute-level data augmentations.
- High-confidence clustering pseudo labels guide positive and negative sample construction, improving sample discriminative capability and reliability.Positive samples come from the same cluster, while negative samples use centers of different high-confidence clusters.
- Extensive experiments on six datasets demonstrate the effectiveness of CCGC.