PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Fusion Clustering Network

Wenxuan Tu, Sihang Zhou, Xinwang Liu, Xifeng Guo, Zhiping Cai, En zhu, Jieren Cheng

arXiv:2012.09600v1cs.LG

TL;DR

Deep clustering needs better ways to interactively combine graph structure and node attributes and to construct reliable target distributions from both. DFCN uses SAIF, fused-space target estimation, and triplet self-supervision, and experiments on six benchmark datasets show consistent state-of-the-art performance.

  • Problem

    Existing deep clustering methods lack dynamic fusion of graph structure and node attributes and seldom use both sources to generate target distributions.

  • Method

    DFCN uses the SAIF module to fuse AE and IGAE representations, constructs target distributions in fused space with Student’s t-distribution, and applies triplet self-supervision.

  • Results

    DFCN consistently outperforms state-of-the-art baseline methods on six benchmark datasets.

  • Takeaways & Limitations

    Dynamic cross-modality fusion and triplet self-supervision jointly support consensus, discriminative representations and precise network training guidance.

Abstract

from arXiv · show

Deep clustering is a fundamental yet challenging task for data analysis. Recently we witness a strong tendency of combining autoencoder and graph neural networks to exploit structure information for clustering performance enhancement. However, we observe that existing literature 1) lacks a dynamic fusion mechanism to selectively integrate and refine the information of graph structure and node attributes for consensus representation learning; 2) fails to extract information from both sides for robust target distribution (i.e., "groundtruth" soft labels) generation. To tackle the above issues, we propose a Deep Fusion Clustering Network (DFCN). Specifically, in our network, an interdependency learning-based Structure and Attribute Information Fusion (SAIF) module is proposed to explicitly merge the representations learned by an autoencoder and a graph autoencoder for consensus representation learning. Also, a reliable target distribution generation measure and a triplet self-supervision strategy, which facilitate cross-modality information exploitation, are designed for network training. Extensive experiments on six benchmark datasets have demonstrated that the proposed DFCN consistently outperforms the state-of-the-art deep clustering methods.

Introduction

DFCN addresses insufficient interaction between graph structure and node attributes in deep clustering by dynamically fusing AE and GAE representations. It also combines both information sources for target-distribution construction and unified triplet self-supervision.

  • Research Gap: Existing methods often align or concatenate graph and attribute information without dynamic cross-modality interaction.This limits information merging and obstructs negotiation between the two sources.
  • Proposed Method: DFCN introduces the SAIF module to process AE and GAE embeddings at local and global levels for consensus representation learning.The method is designed to finely process attribute and structure information extracted from both sub-networks.
  • Proposed Method: DFCN constructs the target distribution by estimating sample-to-center similarity in the fused latent space with Student’s t-distribution.The resulting soft assignments represent the distribution of all samples.
  • Proposed Method: A triplet self-supervision mechanism uses the target distribution to guide AE, GAE, and the fusion component simultaneously.This integrates the learning of the three parts in a unified system.
  • Contributions: The paper develops a symmetric improved graph autoencoder and reports experiments on six public benchmark datasets.The authors report that DFCN consistently outperforms state-of-the-art methods with a preferable margin.

Related Work

Attributed graph clustering methods combine graph structure with node attributes, while target-distribution methods provide soft-label guidance for discriminative representation learning. DFCN contributes a fusion approach that merges complementary information from its sub-networks.

  • Attributed Graph Clustering: GCN-based clustering methods jointly learn graph structure and node attributes by aggregating neighborhood representations.GAE and VGAE integrate graph structure into node attributes through iterative neighborhood aggregation.
  • Attributed Graph Clustering: DFCN merges complementary information from two sub-networks to reach a consensus and learn more discriminative representations.The fusion is presented as combining information from the AE and IGAE components.
  • Target Distribution Learning: Self-optimizing deep clustering methods generate target distributions as soft-label guidance for discriminative representation learning.DEC is described as defining a target distribution with Student’s t-distribution after encoder pretraining.

The Proposed Method

DFCN combines an autoencoder and improved graph autoencoder through a dynamic SAIF module to learn consensus representations and generate clustering guidance from both information sources. Its training jointly reconstructs inputs and aligns AE, graph, and fused assignments with an iteratively generated target distribution.

  • DFCN comprises an autoencoder, improved graph autoencoder, fusion module, and optimization targets for clustering.
  • Fusion-based Autoencoders: The improved graph autoencoder minimizes a hybrid loss that reconstructs both the weighted attribute matrix and adjacency matrix.The coefficient γ balances the two reconstruction losses.
  • Cross-modality Dynamic Fusion Mechanism: SAIF dynamically fuses AE and IGAE embeddings using learnable weighting, local message passing, global self-correlation, and a skip connection.The learnable coefficient α selectively determines the importance of the two information sources, while β scales the global component.
  • Triplet Self-supervised Strategy: The target distribution is generated from Student’s t-kernel soft assignments in the fused embedding space and used to increase assignment confidence.The target distribution is updated iteratively during training.
  • Triplet Self-supervised Strategy: Triplet clustering loss simultaneously aligns AE, IGAE, and fused soft assignment distributions with the target distribution without human guidance.The overall objective combines AE and IGAE reconstruction losses with clustering loss.
  • Optimization: The optimization procedure initializes clustering centers with K-means, alternates representation and distribution updates, and obtains final results by K-means on the fused embedding.

Benchmark Datasets

DFCN is evaluated on six public datasets spanning graph and non-graph settings, with heat-kernel affinity matrices constructed when unavailable.

  • The evaluation covers six public datasets: ACM, DBLP, CITE, USPS, HHAR, and REUT.ACM, DBLP, and CITE are graph datasets; USPS, HHAR, and REUT are non-graph datasets.

Experiment Setup

The experiments use staged DFCN training, established clustering metrics, and dataset-specific optimization settings with early stopping and fixed balanced hyper-parameters.

  • Compared methods use reported settings or original code and data, while ablations use Adam and stop when validation loss plateaus.The implementation also adopts SDCN preprocessing and testing for the proposed method.
  • Table 3 reports clustering performance on six datasets as mean±std, marking the best and runner-up values in red and blue.The color coding distinguishes the top two reported results.
  • The learning rate varies by dataset, batch size is 256, early stopping is used, and γ and λ are fixed to 0.1 and 10.The nearest-neighbor count is set to 5 for all non-graph datasets.
  • The experiments evaluate clustering with ACC, NMI, ARI, and macro F1 using Kuhn-Munkres label matching.These metrics assess agreement after finding the best mapping between cluster IDs and class IDs.

Comparison with the State-of-the-art Methods

Across six datasets, DFCN is compared with ten baselines and generally performs best, while ablations support its reconstruction, fusion, and self-supervision designs.

  • The comparison includes K-means, autoencoder-based methods, and graph-based methods among ten state-of-the-art baselines.The study frames these baselines as representatives of classic shallow, autoencoder, and graph autoencoder clustering approaches.
  • GAE-Lw consistently outperforms GAE-La, while IGAE improves over adjacency-only reconstruction across six datasets.These ablations indicate that reconstructing both weighted attributes and adjacency exploits more comprehensive information.
  • DFCN selectively integrates graph structure and node attributes for consensus representation learning, improving over methods that use only one information source.AE, DEC, and IDEC primarily exploit node attributes, whereas DFCN combines complementary structure and attribute information.
  • Target-distribution ablations show that AE or IGAE alone is not consistently superior, supporting the use of both-source information.Table 4 compares variants using only AE or only IGAE against the combined approach.
  • DFCN improves clustering performance over GAE, VGAE, ARGA, and DAEGC by combining AE representations with graph structure in a fusion module.The comparison identifies under-utilized data information and possible over-smoothing as limitations of these GCN-based methods.
  • On DBLP, DFCN improves over SDCN by 7.9% ACC, 4.2% NMI, 7.8% ARI, and 8.0% F1.The paper reports better results than SDCN and SDCNQ in the majority of cases, especially on HHAR, DBLP, and CITE.
  • DFCN sensitivity is examined by varying λ across six datasets.The supplied figure identifies λ as the parameter varied in the sensitivity analysis.

Ablation Studies

Ablation studies show that IGAE, SAIF, triplet self-supervision, and two-source target-distribution learning each contribute to DFCN's clustering performance.

  • Effectiveness of IGAE: IGAE consistently outperforms adjacency-only GAE-L a on six datasets, indicating that its reconstruction measure exploits more comprehensive information.The resulting latent embedding inherits more properties from the graph's attribute space.
  • Analysis of the SAIF Module: Baseline-C improves over the baseline by about 0.5% to 5.0% across datasets through local- and global-level structure–attribute exploration.This supports consensus latent representation learning for clustering.
  • Analysis of the SAIF Module: Baseline-C-T consistently outperforms Baseline-C-S because triplet self-supervision provides more reliable guidance across AE, IGAE, and the fusion component.The strategy enables these parts to benefit from each other during training.
  • Visualization: The visualization compares raw data, a baseline, and DFCN across six datasets in separate rows.The DFCN row is the final row in each dataset visualization.
  • Influence of Exploiting Both-source Information: DFCN consistently outperforms single-source variants by combining DNN- and GCN-based representations for more reliable and robust target distributions.Either AE or IGAE alone performs better on different datasets, while their combination is consistently stronger.

Analysis of Hyper-parameter λ

DFCN's reconstruction–clustering trade-off parameter λ affects clustering performance but remains stable across a wide range of values.

  • Analysis of Hyper-parameter λ: λ varies from 0.01 to 100, and DFCN remains stable across a wide range while generally performing well.The parameter controls the trade-off between reconstruction and clustering in Eq.(13).

Visualization of Clustering Results

t-SNE visualization of the learned clustering embedding shows that DFCN better reveals intrinsic clustering structure among the data.

  • Visualization: The t-SNE visualization uses the learned clustering embedding eZ to represent clustering results in two dimensions.The comparison is made across six datasets using raw data, baseline, and DFCN rows.
  • Visualization: DFCN can better reveal the intrinsic clustering structure among data in the visualized embedding.This qualitative result is reported for the six-dataset visualization.

Conclusion

DFCN combines graph structure and node attributes through SAIF, triplet self-supervision, and IGAE to improve clustering representations and target-distribution guidance. Experiments on six benchmark datasets show consistent outperformance over state-of-the-art baselines.

  • Conclusion: DFCN uses SAIF to dynamically fuse graph structure and node attributes through cross-modality fusion and triplet self-supervision.The method encodes consensus and discriminative information from both modalities.
  • Conclusion: The fused information constructs a robust target distribution that provides precise guidance for network training.IGAE further assists the method's generalization capability.
  • Conclusion: Experiments on six benchmark datasets show that DFCN consistently outperforms state-of-the-art baseline methods.This is the paper's reported overall experimental conclusion.
  • Conclusion: Future work will adapt DFCN to multi-view graph clustering and incomplete multi-view graph clustering.These applications define the stated extension scope.
Loading 2012.09600v1…