Source-linked AI summary
Deep Graph Clustering via Dual Correlation Reduction
Yue Liu, Wenxuan Tu, Sihang Zhou, Xinwang Liu, Linxuan Song, Xihong Yang, En Zhu
TL;DR
Existing GCN-based graph clustering methods can suffer representation collapse, limiting the discriminative capability of node representations. DCRN uses siamese encoding with dual sample- and feature-level correlation reduction, and experiments on six benchmark datasets demonstrate its superiority over state-of-the-art competitors.
Problem
Existing GCN-based clustering methods can map nodes from different categories to similar representations, producing indiscriminative embeddings and limited clustering performance.
Method
DCRN uses a siamese network and forces cross-view sample- and feature-correlation matrices toward identity matrices, with propagation regularization for long-distance information.
Results
Experiments on six benchmark datasets demonstrate DCRN’s superiority over existing state-of-the-art deep graph clustering competitors.
Takeaways & Limitations
Dual correlation reduction filters redundant cross-view information while preserving discriminative features, helping DCRN avoid representation collapse for clustering.
Abstract
from arXiv · showhide
Deep graph clustering, which aims to reveal the underlying graph structure and divide the nodes into different groups, has attracted intensive attention in recent years. However, we observe that, in the process of node encoding, existing methods suffer from representation collapse which tends to map all data into the same representation. Consequently, the discriminative capability of the node representation is limited, leading to unsatisfied clustering performance. To address this issue, we propose a novel self-supervised deep graph clustering method termed Dual Correlation Reduction Network (DCRN) by reducing information correlation in a dual manner. Specifically, in our method, we first design a siamese network to encode samples. Then by forcing the cross-view sample correlation matrix and cross-view feature correlation matrix to approximate two identity matrices, respectively, we reduce the information correlation in the dual-level, thus improving the discriminative capability of the resulting features. Moreover, in order to alleviate representation collapse caused by over-smoothing in GCN, we introduce a propagation regularization term to enable the network to gain long-distance information with the shallow network structure. Extensive experimental results on six benchmark datasets demonstrate the effectiveness of the proposed DCRN against the existing state-of-the-art methods.
Introduction
Deep graph clustering seeks to learn node representations that divide unlabeled graph nodes into groups, but existing GCN-based methods can suffer representation collapse. DCRN addresses this problem through dual information-correlation reduction.
- Deep graph clustering trains neural networks to learn representations that divide graph nodes into groups without human annotations.
- Existing GCN-based clustering methods may map nodes from different categories to similar representations, limiting discriminability and clustering performance.
- DCRN is a self-supervised deep graph clustering method that reduces information correlation at two levels to avoid representation collapse.
- Its dual mechanism makes cross-view sample- and feature-correlation matrices approximate identity matrices, promoting discriminative and noise-robust representations.
- Experiments on six benchmark datasets demonstrate DCRN’s superiority over existing state-of-the-art deep graph clustering competitors.
Related Work
Graph clustering methods use graph structure and node attributes, while self-supervised representation learning methods address representation collapse through mechanisms such as contrastive pairs, momentum encoders, stop-gradient, and redundancy reduction.
- GAE and VGAE combine node attributes with graph structure through graph encoders and reconstruct graph structure using an inner-product decoder.
- Later attributed-graph clustering methods improve early approaches with graph attention, Laplacian sharpening, or generative adversarial learning.
- Over-smoothing remains insufficiently addressed in several graph clustering methods, affecting clustering performance.
- Self-supervised methods mitigate representation collapse using negative-pair mechanisms, cluster replacement, momentum encoders, stop-gradient, or redundant-information reduction.
Dual Correlation Reduction Network
DCRN uses graph distortions and dual information correlation reduction to learn discriminative node representations for clustering. Its sample- and feature-level objectives, together with propagation regularization, address representation collapse and over-smoothing.
- Architecture: DCRN combines a graph distortion module with a dual information correlation reduction module in a siamese graph-encoding framework.The distortion module creates two graph views using feature corruption and edge perturbation.
- Dual Information Correlation Reduction: The DICR mechanism filters redundant information while preserving discriminative latent features, helping alleviate representation collapse for clustering.The resulting combined embedding is used for K-means clustering.
- Sample-level Correlation Reduction: Sample-level correlation reduction makes the cross-view sample correlation matrix approximate an identity matrix, aligning corresponding nodes and separating different nodes.Diagonal entries are encouraged toward 1, while off-diagonal entries are minimized toward 0.
- Feature-level Correlation Reduction: Feature-level correlation reduction makes the cross-view feature correlation matrix approximate an identity matrix so corresponding feature dimensions agree while others are separated.The two-view node embeddings are projected into cluster-level embeddings before feature correlations are reduced.
- Propagated Regularization: Propagation regularization uses Jensen-Shannon divergence between the latent embedding and its propagated version to capture long-distance information with a shallow network.This is intended to alleviate over-smoothing as propagated information moves deeper through the framework.
- Objective Function: The overall objective combines DICR, joint attribute-and-structure reconstruction, and Kullback–Leibler clustering losses.The clustering loss aligns soft assignment and target distributions over clustering-oriented node embeddings.
Expriments
Experiments on six benchmark datasets evaluate DCRN against established clustering methods, analyze its components, and examine visualization and hyper-parameter behavior. DCRN consistently achieves the strongest reported clustering performance, while dual correlation reduction and propagated regularization improve the backbone.
- Performance Comparison: DCRN consistently outperforms all compared methods across four clustering metrics on six benchmark datasets.The evaluated metrics are ACC, NMI, ARI, and F1.
- Performance Comparison: On DBLP, DCRN exceeds DFCN by 3.66%, 5.25%, 6.60%, and 3.58% for ACC, NMI, ARI, and F1, respectively.
- Ablation Studies: Baseline-P improves DBLP performance by about 0.5% to 1.0% across four metrics, indicating benefits from propagated regularization.The ablation text associates this component with improved generalization capacity and alleviation of over-smoothing.
- Ablation Studies: Baseline-D exceeds the DBLP baseline by 3.63%, 5.25%, 6.48%, and 3.56% for ACC, NMI, ARI, and F1, while Baseline-P-D performs best among variants.These results are attributed to enhanced discriminative capacity from the DICR mechanism and the combination of both components.
- Ablation Studies: Sample-level correlation reduction consistently improves over the baseline across datasets, whereas feature-level reduction performs worse on DBLP and CORAFULL.Baseline-S improves DBLP accuracy by 3.60%; the combined Baseline-F-S uses both correlation-reduction types to make embeddings more discriminative.
- Visualization and Hyper-parameter Analysis: DCRN produces clearer t-SNE cluster structure on DBLP and ACM and is insensitive to K, generally performing well when K equals the number of clusters C.Accuracy initially increases with K and then generally remains high with slight variation.
Conclusion
DCRN reduces information correlation at both the sample and feature levels to preserve discriminative representations and avoid representation collapse. Experiments on six benchmarks demonstrate its superiority.
- DCRN reduces information correlation at both sample and feature levels through a dual information correlation reduction mechanism.The mechanism filters redundant information from latent variables across two views while preserving more discriminative features.
- The mechanism helps avoid representation collapse, supporting better clustering.
- Experimental results on six benchmark datasets demonstrate DCRN's superiority.