Source-linked AI summary
Deep Multiview Clustering by Contrasting Cluster Assignments
Jie Chen, Hua Mao, Wai Lok Woo, Xi Peng
TL;DR
Existing deep multiview clustering methods still face difficulty learning representations invariant across views. CVCL addresses this by pretraining view-specific autoencoders and then contrasting cluster assignments during fine-tuning; experiments and theory support its effectiveness and alignment strategy.
Problem
Existing deep MVC methods leave learning invariant representations across multiple views as an intractable problem.
Method
CVCL uses two-stage training with deep autoencoder pretraining and cluster-level cross-view contrastive learning during fine-tuning.
Results
CVCL outperforms several state-of-the-art approaches in extensive experiments on multiview clustering datasets.
Takeaways & Limitations
Contrasting cluster assignments produces more discriminative cluster assignments and provides a theoretical explanation for soft assignment alignment.
Takeaways & Limitations
Strictly aligning encoders across multiple views is generally intractable, and cosine similarity can be inaccurate across different domains such as text and images.
Abstract
from arXiv · showhide
Multiview clustering (MVC) aims to reveal the underlying structure of multiview data by categorizing data samples into clusters. Deep learning-based methods exhibit strong feature learning capabilities on large-scale datasets. For most existing deep MVC methods, exploring the invariant representations of multiple views is still an intractable problem. In this paper, we propose a cross-view contrastive learning (CVCL) method that learns view-invariant representations and produces clustering results by contrasting the cluster assignments among multiple views. Specifically, we first employ deep autoencoders to extract view-dependent features in the pretraining stage. Then, a cluster-level CVCL strategy is presented to explore consistent semantic label information among the multiple views in the fine-tuning stage. Thus, the proposed CVCL method is able to produce more discriminative cluster assignments by virtue of this learning strategy. Moreover, we provide a theoretical analysis of soft cluster assignment alignment. Extensive experimental results obtained on several datasets demonstrate that the proposed CVCL method outperforms several state-of-the-art approaches.
1. Introduction
Multiview clustering seeks meaningful groups from complementary, semantically shared views, but existing deep methods still leave view-invariant representation learning unresolved. CVCL addresses this gap by contrasting cluster assignments and reports effectiveness across seven datasets.
- Multiview clustering groups samples using distinct feature information shared across multiple, complementary views.
- Deep MVC methods improve representation learning through view-specific encoders, but invariant representations across views remain an open question.
- CVCL contrasts cluster assignments across views to capture consistent semantic label information and learn view-invariant representations.
- The model uses a two-stage training scheme to learn view-invariant representations in an end-to-end manner.
- A theoretical analysis explains soft cluster assignment alignment, while experiments on seven multiview datasets demonstrate CVCL's effectiveness.
2. Related Work
Related work spans shallow and deep multiview clustering, autoencoder-based methods, and contrastive learning. CVCL applies contrastive learning to semantic cluster assignments to explore consistency across views.
- Traditional MVC methods include subspace learning, nonnegative matrix factorization, graph learning, and multiple-kernel approaches.
- Deep autoencoder-based MVC methods learn consensus representations by minimizing reconstruction error across multiple views.
- Contrastive learning maximizes similarity between positive pairs and minimizes similarity between negative pairs in latent feature space.
- CVCL applies contrastive learning to semantic labels and uses cluster assignments to explore consistency across multiple views.
3.1. Proposed Statement
The proposed statement formalizes multiview data as multiple views containing instances, assumes K semantic clusters, and frames clustering as assigning samples to those clusters.
- Multiview data are represented as X^(v) for v=1 through n_v, with each view containing N instances.
- Each instance is d_v-dimensional, and K denotes the number of clusters into which the samples are categorized.
3.2. Network Architecture
The CVCL network combines view-specific autoencoders with cross-view contrastive learning to produce semantic labels from raw multiview instances.
- The architecture contains view-specific autoencoder and cross-view contrastive learning modules.
- The autoencoder module learns clustering-friendly features separately for the multiple views under unsupervised representation learning.
3.3. Cluster-Level CVCL
The cluster-level CVCL module maps semantic features to cluster-assignment probabilities and contrasts assignments across views. It aligns assignments for the same sample and separates assignments associated with different clusters while discouraging cluster collapse.
- Two linear layers followed by softmax transform semantic features into cluster-assignment probabilities for each view.
- The semantic label of each instance is determined from the largest cluster-assignment probability.
- Cluster assignments for corresponding instances share semantic information because they describe the same sample across views.
- The method uses positive and negative assignment pairs across views to enforce assignment-level discrimination.
- The contrastive objective maximizes similarity between intracluster assignments and minimizes similarity between intercluster assignments across views.
- A cross-view consistency regularizer prevents all instances from being assigned to one cluster and encourages populated clusters across views.
3.4. Two-Stage Training Scheme
CVCL uses a two-stage training scheme: deep autoencoders first initialize view-specific representations, then the full network is fine-tuned for multiview clustering using its joint objectives.
- Training first pretrains a deep autoencoder and then fine-tunes the whole network for multiview clustering.
- Each view has a view-specific encoder and decoder, with encoders learning embedded representations and decoders reconstructing the inputs.
- The deep autoencoder uses at least four layers per encoder or decoder and ReLU activations.
- The pretraining stage minimizes reconstruction loss to initialize network parameters.
- The complete objective combines reconstruction, cross-view contrastive, and cross-view consistency losses.
- Adaptive momentum-based minibatch gradient descent optimizes the network across the two training stages, producing final clustering labels.
3.5. Theoretical Analysis
The analysis establishes lower bounds for the contrastive losses and characterizes strict alignment as the condition under which these bounds can be achieved. It also explains why cluster-assignment alignment is preferable to instance-feature alignment, while noting strict encoder alignment is generally intractable.
- Lower-bound analysis: Under the stated assumption, L_a has a lower bound, so minimizing L_fine is theoretically guaranteed to obtain a lower bound.The assumption requires q^(v)_ij > c for all clusters j.
- Strict alignment: Strict alignment produces one positive and four negative cluster-assignment pairs for each cluster across two views.This construction is used to analyze the realizability of similarity alignment.
- Strict alignment: Theorem 2 states that L_c is minimized when the representations are strictly aligned across every pair of views.Strict alignment is defined through conditions on positive and negative cluster-assignment pairs.
- Limitations: Designing an encoder that is strictly aligned across multiple views is intractable in the general case.Cosine similarity can also be inaccurate across different domains, such as text and image pairs, and may have insufficient alignment sensitivity.
- View invariance: The proposed cluster assignments are theoretically invariant to all view types, and identical assignment matrices can represent corresponding views.The matrix construction uses a transpose when the arrangement assumption is violated.
- Complexity analysis: The feedforward computation and fine-tuning losses are analyzed in terms of minibatch size, hidden-layer width, views, feature dimensions, clusters, and iterations.The reconstruction, contrastive, and consistency losses have separate complexity terms, and total complexity depends on the maximum iteration count.
4. Experiments
Experiments evaluate CVCL on seven public multiview datasets against state-of-the-art methods using ACC, NMI, and purity. CVCL achieves the best clustering results across datasets, while ablations show that its complete loss and two-stage training are important.
- Experimental setup: CVCL is evaluated on seven publicly available multiview datasets against several state-of-the-art MVC and contrastive-learning methods.The comparisons include EE-IMVC, ASR, DSIMVC, DCP, DSMVC, and MFL.
- Evaluation metrics: ACC, NMI, and purity are used to evaluate clustering performance, with larger values indicating better results.ACC measures the best matching between predicted and ground-truth assignments.
- Network architecture and parameter settings: The network uses view-specific autoencoders, an extra linear layer, and a softmax layer, with tunable feature dimensions and loss weights.The encoders and decoders contain three to five hidden layers, and parameters are selected by grid search.
- Performance evaluation: CVCL achieves the best clustering results on all datasets and significantly outperforms the other contrastive-learning methods.Improvements over the second-best method on MSRC-v1 are approximately 4.29% for ACC, 8.47% for NMI, and 4.29% for purity.
- Performance evaluation: Contrastive-learning methods often improve over traditional methods on large-scale datasets, with CVCL benefiting from cluster-level alignment of soft assignments.The paper attributes CVCL’s stronger representations to contrasting assignments across views.
4.3. Ablation Studies
The ablation studies show that CVCL performs best when both its two-stage training scheme and all loss components are retained. Parameter sensitivity and convergence analyses further examine the method’s behavior across datasets.
- Loss-component ablations: CVCL achieves its best clustering performance when all loss terms and the two-stage training scheme are used.The comparison includes variants without pretraining and without the regularization term La.
- Loss-component ablations: 5.24%, 18.81% and 15.24% improvements in ACC, NMI and purity, respectively, are obtained over CVCLfine-tuning on MSRC-v1.The reported gains show the strongest benefit of pretraining on this dataset.
- Loss-component ablations: 0.08%, 0.22% and 0.15% ACC improvements over CVCLfine-tuning are obtained on BDGP, MNIST-USPS and Fashion, respectively.The passage reports narrower gaps on datasets with substantially more samples.
- Loss-component ablations: Ignoring La causes clustering performance to dramatically decline on most datasets.The ablation indicates that the regularization term is important within the overall reconstruction loss.
- Parameter sensitivity: ACC and NMI are evaluated under different α and β combinations, with performance reported across four and two representative datasets, respectively.The sensitivity analysis uses α and β values chosen from {0.001, 0.005, 0.01, 0.05}.
- Convergence analysis: The loss in Eq. (12) drops sharply during the first few iterations and then decreases slowly until convergence, while Eq. (13) shows a similar trend on most datasets.Eq. (13) fluctuates slightly after the first few iterations on some datasets.
5. Conclusion
The paper proposes CVCL to learn view-invariant representations for multiview clustering through cluster-level contrastive learning. Its theoretical analysis, experiments, and ablations support the usefulness of the cluster-level strategy and the overall reconstruction-loss components.
- Conclusion: CVCL learns view-invariant representations by exploring consistent semantic label information across multiple views.The method uses a cluster-level CVCL strategy during two successive stages.
- Conclusion: Theoretical analysis of soft cluster assignment alignment indicates the importance of cluster-level learning in CVCL.The conclusion also reports extensive experiments and ablation studies on multiview clustering datasets.
- Conclusion: Experiments and ablation studies validate the model’s reported superiority and the effectiveness of each overall reconstruction-loss component.The conclusion frames these evaluations as evidence for CVCL’s design.