PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-view Contrastive Graph Clustering

Erlin Pan, Zhao Kang

arXiv:2110.11842v1cs.LGcs.AIcs.CVcs.SI

TL;DR

Multi-view attributed graph clustering must integrate graph structure and node attributes despite noisy or incomplete original graphs. MCGC filters graph signals, learns an adaptively weighted consensus graph, and regularizes it with graph-level contrastive learning. On benchmark datasets, it achieves state-of-the-art performance and outperforms existing deep learning-based methods, while potentially requiring substantial memory on very large graphs.

  • Problem

    Existing multi-view clustering methods commonly focus on multiple graphs or multi-view attributes, while original graphs may be noisy or incomplete.

  • Method

    MCGC combines graph filtering for smoothed node representations with adaptive consensus graph learning and graph-level contrastive regularization.

  • Results

    MCGC achieves state-of-the-art performance against shallow and deep methods on five benchmark datasets, with contrastive loss improving performance across evaluated datasets.

  • Takeaways & Limitations

    The results show that a shallow approach can outperform deep learning methods while exploiting both attribute content and graph structure information.

  • Takeaways & Limitations

    The approach could require substantial memory when the data contain too many nodes.

Abstract

from arXiv · show

With the explosive growth of information technology, multi-view graph data have become increasingly prevalent and valuable. Most existing multi-view clustering techniques either focus on the scenario of multiple graphs or multi-view attributes. In this paper, we propose a generic framework to cluster multi-view attributed graph data. Specifically, inspired by the success of contrastive learning, we propose multi-view contrastive graph clustering (MCGC) method to learn a consensus graph since the original graph could be noisy or incomplete and is not directly applicable. Our method composes of two key steps: we first filter out the undesirable high-frequency noise while preserving the graph geometric features via graph filtering and obtain a smooth representation of nodes; we then learn a consensus graph regularized by graph contrastive loss. Results on several benchmark datasets show the superiority of our method with respect to state-of-the-art approaches. In particular, our simple approach outperforms existing deep learning-based methods.

1 Introduction

MCGC is introduced as a generic framework for clustering multi-view attributed graphs by combining graph filtering, consensus graph learning, and graph-level contrastive regularization. It is reported to achieve state-of-the-art performance on five benchmark datasets, including against deep methods.

  • 1 Introduction: Existing approaches often emphasize multiple graphs or multi-view attributes, while related methods may omit feature information or use only one selected informative view.The introduction identifies limitations in multiplex embedding and one-view graph autoencoder approaches.
  • 1 Introduction: MCGC introduces a graph-level contrastive loss that draws similar nodes together and pushes dissimilar nodes apart.The contrastive objective is proposed to improve the quality of the learned graph.
  • 1 Introduction: MCGC targets multilayer graphs with multi-view attributes through graph filtering, graph learning, and graph contrastive components.The framework is designed as a generic clustering approach for multi-view attributed graph data.
  • 1 Introduction: Its graph filtering component obtains a smoothed representation, while graph learning generates a consensus graph with adaptive view weighting.These components address representation smoothing and integration across views.
  • 1 Introduction: MCGC achieves state-of-the-art performance compared with shallow and deep methods on five benchmark datasets.The reported evaluation covers five benchmark datasets and includes comparisons with both method categories.

2 Related Work

Related work spans multi-view clustering, network embedding, and contrastive learning. Prior graph methods differ in whether they integrate structure, attributes, latent representations, or contrastive objectives.

  • 2 Related Work: Multi-view clustering methods include joint low-rank sparse subspace learning, cross-view graph matching, and shared low-rank transition modeling.These approaches impose different forms of cross-view affinity or transition structure.
  • 2 Related Work: Contrastive learning maximizes similarity between positive pairs and distance between negative pairs, with objectives including triplet, NCE, and NT-Xent losses.The related-work discussion presents contrastive learning as a family of unsupervised learning objectives.
  • 2 Related Work: Deep robust clustering applies contrastive loss by converting mutual-information maximization into a contrastive-learning objective.This connects contrastive formulations with unsupervised clustering improvements.

3 Methodology

MCGC filters multi-view graph signals to obtain smooth node representations, learns an adaptively weighted consensus graph, and regularizes it with graph-level contrastive learning.

  • 3.1 Notation: MCGC represents multi-view graph data with nodes, view-specific edges, and feature matrices, then derives normalized adjacency and Laplacian matrices for each view.
  • 3.2 Graph Filtering: Graph filtering obtains smooth representations by suppressing high-frequency noise while preserving graph geometric features.The filtered representation is computed using an m-th order graph filter, H = (I − sL)^mX.
  • 3.3 Graph Learning: MCGC learns a consensus graph S from the smoothed representations because real-world graphs may be noisy or incomplete.The graph-learning objective uses self-expression reconstruction with regularization and combines views through adaptive weighting factors λv.
  • 3.4 Graph Contrastive Regularizer: Graph-level contrastive regularization uses each node and its k-nearest neighbors as positive pairs, drawing neighbors close and non-neighbors apart in S.Unlike standard graph contrastive methods that perturb nodes or edges, MCGC applies the regularizer to the graph matrix rather than node features.
  • 3. Methodology: MCGC integrates multi-view attributes and multiple structural graphs while constructing the consensus graph from smooth signals instead of raw data.
  • 3.5 Optimization: The optimization alternates updates of S and λv, using Adam for S and initializing it with the solution of Eq. (5) before clustering on the symmetrized graph C.The procedure repeats until convergence and then computes C = (|S| + |S|^⊤) / 2.

4 Experiments

MCGC is evaluated on five benchmark datasets using standard clustering metrics and comparisons with single-view, multi-view, and deep learning methods. Across the reported experiments, it outperforms the compared approaches, while dataset tables summarize the evaluation settings.

  • 4.1 Datasets and Metrics: MCGC is evaluated on ACM, DBLP, IMDB, Amazon photos, and Amazon computers.ACM, DBLP, and IMDB are described as attributed networks with multiple relationship graphs; the Amazon datasets are also included in the benchmark evaluation.
  • 4.1 Datasets and Metrics: The evaluation reports Accuracy (ACC), normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and F1 score.
  • 4.2 Compared Methods: MCGC is compared with single-view methods LINE and GAE, alongside multi-view methods PMNE, RMSC, and SwMC.The comparisons include methods using structural information, attribute features, or both.
  • 4.3 Results: MCGC improves ACC over GAE by more than 9%, 4%, and 19% on ACM, DBLP, and IMDB, respectively.Against PMNE, ACC, NMI, ARI, and F1 improve by 16%, 20%, 20%, and 12% on average.
  • 4.3 Results: The results are presented in Table 2 for ACM, DBLP, and IMDB and in Table 3 for Amazon photos and Amazon computers.Table 3 uses a dash to denote an out-of-memory problem.

5 Ablation Study

Ablation studies show that contrastive loss, multi-view integration, and graph filtering each contribute to MCGC's clustering performance. The full model benefits from contrastive regularization, complementary views, and filtered representations.

  • 5.1 The Effect of Contrastive Loss: Removing contrastive loss causes performance to fall precipitously across all datasets.The contrastive regularizer pulls neighbors into the same cluster and decreases intra-cluster variance.
  • 5.1 The Effect of Contrastive Loss: 16%, 8%, 5%, and 12% ACC improvements are reported on DBLP, ACM, IMDB, and Amazon, respectively, for MCGC with contrastive regularization.Other clustering metrics also improve significantly with the contrastive regularizer.
  • 5.2 The Effect of Multi-View Learning: The best performance is achieved when all views are incorporated on both ACM and Amazon photos.Performance varies substantially across individual views, supporting adaptive view weighting.
  • 5.3 The Effect of Graph Filtering: Removing graph filtering lowers ACC by 0.8%, 1.3%, and 0.8% on ACM, DBLP, and IMDB, respectively.MCGC also outperforms MCGC- on most other metrics.

6 Parameter Analysis

Parameter analyses examine graph-filtering parameters m and s, the trade-off parameter α, and objective-function evolution. MCGC performs reasonably with small m and s, is not sensitive to α, and converges quickly.

  • Graph-filtering parameters: MCGC achieves reasonable performance for small graph-filtering parameters m and s on ACM.The analysis varies m from 1 to 5 and s from 0.01 to 10.
  • Graph-filtering parameters: m = 2 and s = 0.5 are selected for all experiments.These settings are chosen based on the ACM sensitivity analysis.
  • Trade-off parameter α: MCGC is not sensitive to α across ACM, DBLP, and IMDB.The trade-off parameter is tuned over α = [10^-3, 0.1, 1, 10, 10^2, 10^3].
  • Convergence: The objective function shows that MCGC converges quickly.Objective variation is plotted in Figure 3.

7 Conclusion

MCGC learns a consensus graph for multi-view attributed graphs by combining graph filtering with contrastive regularization. It reports superior performance, while requiring caution about memory usage on very large graphs.

  • MCGC exploits both attribute content and graph structure to learn a consensus graph for multi-view graph clustering.
  • Graph filtering removes noisy components, while contrastive regularization further enhances the learned graph.
  • MCGC achieves superior performance on multi-view attributed graph datasets and reports state-of-the-art results against shallow and deep methods on five benchmarks.
  • The learned graph has size N × N, which can require substantial memory when datasets contain many nodes.
Loading 2110.11842v1…