Source-linked AI summary
Attention-driven Graph Clustering Network
Zhihao Peng, Hui Liu, Yuheng Jia, Junhui Hou
TL;DR
Existing graph clustering methods lack flexible fusion of node attributes and topological features and overlook multi-scale information. AGCN addresses these gaps with attention-based heterogeneity-wise and scale-wise fusion in a unified unsupervised framework, and experiments validate its superiority over state-of-the-art methods, especially on low-quality graphs.
Problem
Existing methods lack adaptive fusion between node attribute and topological graph features and neglect multi-scale information across layers for clustering.
Method
AGCN uses attention-based heterogeneity-wise fusion, scale-wise fusion, and unified unsupervised training to jointly learn features and cluster assignments.
Results
AGCN consistently outperforms state-of-the-art methods on benchmark datasets, with superiority especially demonstrated for low-quality graphs.
Takeaways & Limitations
Adaptive fusion of heterogeneous and multi-scale features provides a flexible basis for directly producing clustering results from graph data.
Abstract
from arXiv · showhide
The combination of the traditional convolutional network (i.e., an auto-encoder) and the graph convolutional network has attracted much attention in clustering, in which the auto-encoder extracts the node attribute feature and the graph convolutional network captures the topological graph feature. However, the existing works (i) lack a flexible combination mechanism to adaptively fuse those two kinds of features for learning the discriminative representation and (ii) overlook the multi-scale information embedded at different layers for subsequent cluster assignment, leading to inferior clustering results. To this end, we propose a novel deep clustering method named Attention-driven Graph Clustering Network (AGCN). Specifically, AGCN exploits a heterogeneity-wise fusion module to dynamically fuse the node attribute feature and the topological graph feature. Moreover, AGCN develops a scale-wise fusion module to adaptively aggregate the multi-scale features embedded at different layers. Based on a unified optimization framework, AGCN can jointly perform feature learning and cluster assignment in an unsupervised fashion. Compared with the existing deep clustering methods, our method is more flexible and effective since it comprehensively considers the numerous and discriminative information embedded in the network and directly produces the clustering results. Extensive quantitative and qualitative results on commonly used benchmark datasets validate that our AGCN consistently outperforms state-of-the-art methods.
1 INTRODUCTION
AGCN addresses limitations in deep graph clustering by adaptively fusing node attributes with topological features and aggregating information across layers. Its unified unsupervised framework directly produces clustering results and is evaluated against state-of-the-art methods.
- Deep clustering methods often extract node attribute features with auto-encoders while overlooking topological graph information.
- Graph convolutional networks complement auto-encoders by learning graph structure features from topological information.
- AGCN introduces heterogeneity-wise and scale-wise attention modules to dynamically fuse GCN and AE features and aggregate multi-scale representations.
- AGCN-H merges GCN and AE features from the same layer, while AGCN-S concatenates multi-scale features from different layers.
- A unified learning framework trains AGCN without supervision and directly produces clustering assignments.
2 RELATED WORK
Prior clustering methods use either auto-encoder or graph-based representations, but existing approaches inadequately balance attribute and topological features and neglect multi-scale information. These omissions limit comprehensive graph-structure exploitation and clustering performance.
- Auto-encoder methods such as DEC and IDEC learn deep representations through unsupervised feature learning and clustering objectives.
- Graph convolutional clustering methods introduce graph structure information through models including GAE, VGAE, DAEGC, and ARGA.
- Existing approaches may naively assign equal importance to topological graph and node attribute features.
- They also neglect multi-scale information embedded in different layers, weakening comprehensive exploitation of graph structure.
3 PROPOSED METHOD
AGCN combines auto-encoder and graph-convolutional representations through attention-based heterogeneity-wise and scale-wise fusion, then jointly learns representations and cluster assignments in an unsupervised framework.
- AGCN-H: AGCN-H dynamically fuses GCN and AE features from the same layer using learned attention coefficients and weighted feature fusion.The coefficients measure the importance of the GCN and AE features before producing the fused representation for the next GCN layer.
- Auto-encoder module: The auto-encoder extracts latent representations by minimizing reconstruction loss between raw and reconstructed data.The encoder and decoder produce layer-wise representations, with the final decoder output reconstructing the input.
- AGCN-S: AGCN-S adaptively combines multi-scale features from different layers because those features provide different levels of semantic description.It learns scale-specific attention coefficients and applies weighted fusion after aggregating features through concatenation.
- AGCN-S: The scale-wise module uses learned attention coefficients to weight features before concatenating them for subsequent prediction.The coefficients are normalized with softmax and ℓ2 normalization, and the fused representation is passed to the final prediction layer.
- Training process: AGCN trains feature learning and cluster assignment jointly by aligning the combined-feature and AE-feature distributions with an auxiliary target distribution.Student’s t-distribution provides soft assignments, while the auxiliary target distribution is introduced to avoid trivial solutions during KL-divergence optimization.
- Training process: The unified framework directly produces clustering results from the learned prediction representation.The network minimizes reconstruction and alignment losses, and the predicted cluster label is inferred directly from the output representation.
4 EXPERIMENTS
Experiments evaluate AGCN on six benchmark datasets using standard clustering metrics, ablations, sensitivity analysis, and visualization. AGCN generally achieves the strongest clustering performance, while its heterogeneity-wise and scale-wise fusion modules improve representations and clustering results.
- Datasets and evaluation: Experiments use six benchmark datasets spanning image, record, text, and graph data, with KNN graphs generated for non-graph datasets.Evaluation uses ACC, NMI, ARI, and macro F1, where larger values indicate better clustering.
- Clustering results: AGCN obtains the best clustering performance in most comparisons, improving over the second-best method on HHAR and DBLP across ACC, NMI, ARI, and F1.On HHAR, the average improvements are 3.85% ACC, 2.54% NMI, 4.23% ARI, and 5.42% F1; on DBLP, they are 5.21%, 0.18%, 3.34%, and 5.09%, respectively.
- Clustering results: AGCN’s gains are attributed to adaptively fusing GCN and AE features, combining multi-scale features, and constructing soft assignments with abundant discriminative information.The method outperforms alternatives across all six datasets, although its improvement is smaller on ACM.
- Ablation study: The AGCN-H ablation improves performance, supporting attention-based heterogeneity-wise fusion and dynamic weighting of GCN and AE features.The ablation compares models without and with AGCN-H for each dataset.
- Ablation study: Multi-scale fusion improves clustering in most cases, while attention-based scale-wise weighting achieves the best performance and mitigates HHAR’s over-smoothing effects.HHAR is the exception for unweighted multi-scale fusion because some middle-layer features suffer from over-smoothing.
- Parameter analysis: AGCN is not sensitive to the number of neighbors k′ on the non-graph datasets USPS, HHAR, and Reuters.The parameter analysis evaluates k′ through clustering results on these three datasets.
- Visualization: On HHAR, AGCN’s learned representation shows the strongest cluster separability in t-SNE visualizations, with same-class samples grouped and different groups separated.The visualization compares AGCN with the best-performing competing methods.
5 CONCLUSION
AGCN combines dynamic heterogeneity-wise fusion with multi-scale feature fusion through two attention-based modules, and experiments validate its superiority over state-of-the-art methods, especially for low-quality graphs.
- AGCN simultaneously uses dynamic fusion and multi-scale feature fusion for deep clustering.