Multi-level Feature Learning for Contrastive Multi-view Clustering

Jie Xu; Huayi Tang; Yazhou Ren; Liang Peng; Xiaofeng Zhu; Lifang He

Source-linked AI summary

Multi-level Feature Learning for Contrastive Multi-view Clustering

Jie Xu, Huayi Tang, Yazhou Ren, Liang Peng, Xiaofeng Zhu, Lifang He

arXiv:2106.11193v2Jun 21, 2021cs.LGcs.CV

TL;DR

Multi-view clustering must reconcile learning shared semantics with reconstructing view-private information. MFLVC separates these objectives across fusion-free, multi-level feature spaces and reports state-of-the-art clustering performance, while noting sensitivity to intrinsic data bias, especially dirty samples.

Problem
Existing multi-view clustering methods can fuse misleading view-private information or impose reconstruction and consistency objectives on the same features despite their conflicting goals.
Method
MFLVC learns low-level features, high-level features, and semantic labels per view without direct fusion, assigning reconstruction to low-level features and contrastive consistency to higher-level features and labels.
Results
MFLVC achieves state-of-the-art clustering effectiveness, outperforming the best comparison method CoMVC on Fashion by about 14% in ACC.
Takeaways & Limitations
Separating feature levels lets MFLVC explore common semantics while reducing the adverse influence of view-private information.
Takeaways & Limitations
The trained model might be affected by intrinsic data bias, especially with dirty samples.

Abstract

from arXiv · show

Multi-view clustering can explore common semantics from multiple views and has attracted increasing attention. However, existing works punish multiple objectives in the same feature space, where they ignore the conflict between learning consistent common semantics and reconstructing inconsistent view-private information. In this paper, we propose a new framework of multi-level feature learning for contrastive multi-view clustering to address the aforementioned issue. Our method learns different levels of features from the raw features, including low-level features, high-level features, and semantic labels/features in a fusion-free manner, so that it can effectively achieve the reconstruction objective and the consistency objectives in different feature spaces. Specifically, the reconstruction objective is conducted on the low-level features. Two consistency objectives based on contrastive learning are conducted on the high-level features and the semantic labels, respectively. They make the high-level features effectively explore the common semantics and the semantic labels achieve the multi-view clustering. As a result, the proposed framework can reduce the adverse influence of view-private information. Extensive experiments on public datasets demonstrate that our method achieves state-of-the-art clustering effectiveness.

1. Introduction

Multi-view clustering must separate common semantics from view-private information while reconciling reconstruction and consistency objectives. MFLVC addresses this conflict through fusion-free multi-level feature learning and reports state-of-the-art clustering effectiveness.

Motivation: Multi-view data combines common semantics across views with view-private information that can mislead common-semantic learning.Feature fusion may allow meaningless private information to dominate clustering.
Motivation: Existing methods may impose reconstruction and consistency objectives on the same features, creating conflicting demands.Consistency favors common semantics, whereas reconstruction preserves view-private information.
Method: MFLVC learns low-level features, high-level features, and semantic labels from each view without direct feature fusion.An autoencoder produces low-level features, followed by shared multilayer perceptrons for higher-level features and labels.
Method: The reconstruction objective uses low-level features, while contrastive consistency objectives use high-level features and semantic labels.The high-level features represent common semantics, and semantic labels support multi-view clustering.
Contributions: MFLVC simultaneously applies multi-view contrastive learning to high-level features and semantic labels.The framework is designed to explore common semantics and improve semantic-label quality.
Evaluation: Extensive experiments report state-of-the-art clustering effectiveness, with robustness to hyper-parameter settings and detailed ablations.The experiments examine loss components and contrastive-learning structures.

2. Related Work

Multi-view clustering research includes subspace, matrix-factorization, graph-based, and deep-learning approaches. Contrastive learning has also been adapted to multi-view representation learning and clustering.

Multi-view clustering: Subspace methods learn a common subspace representation for multiple views.The literature describes diversity-induced mechanisms for multi-view subspace clustering.
Multi-view clustering: Matrix-factorization methods use shared clustering indicator structures and are formally related to relaxed K-means.These methods formulate constrained matrix-factorization problems for multiple views.
Multi-view clustering: Graph-based methods construct graph structures to preserve adjacency relationships among samples.Deep MVC methods instead use neural networks to discover latent cluster patterns.
Contrastive learning: Contrastive learning maximizes similarities between positive pairs while minimizing similarities between negative pairs in a feature space.Recent work applies this paradigm to multi-view coding, representation learning, and clustering.

3. Method

MFLVC separates reconstruction and consistency objectives across low-level, high-level, and semantic-label spaces in a fusion-free multi-view clustering framework. Contrastive learning aligns high-level features and cluster assignments while preserving low-level representations for reconstruction.

Multi-level feature learning: MFLVC represents each view with low-level features Z_m, high-level features H_m, and semantic labels Q_m.An autoencoder produces low-level features from raw inputs, while additional mappings produce high-level features and cluster assignments.
Fusion-free design: The framework avoids direct feature fusion and assigns reconstruction and consistency losses to separate feature spaces, reducing interference from view-private information.The method learns high-level features and cluster assignments without directly fusing view representations.
Low-level reconstruction: The reconstruction objective operates on low-level features to preserve representation ability and avoid model collapse.The reconstruction loss compares each view's input with its decoder output.
High-level consistency: High-level features are learned with multi-view contrastive learning so corresponding samples across views become consistent and capture common semantics.Positive feature pairs are formed across views, while remaining feature pairs serve as negatives; the resulting representations have dense within-cluster structure.
Semantic-label consistency: Semantic labels are produced by a shared label MLP and aligned across views with a second contrastive objective.The label contrastive objective treats corresponding cluster labels across views as positive pairs and uses a regularizer to prevent single-cluster assignment.
Joint optimization: MFLVC combines reconstruction loss L_Z with high-level and label consistency losses L_H and L_Q without requiring loss-balancing weights.The separate learning structure prevents the reconstruction loss gradient from influencing the mappings used for high-level features and labels.

4. Experiments

Experiments evaluate MFLVC on five public datasets against traditional and deep baselines using ACC, NMI, and PUR. MFLVC generally achieves the strongest clustering performance, including robustness across varying numbers of Caltech views and clearer high-level feature structure.

Experimental Setup: The experiments use five public datasets, including MNIST-USPS, BDGP, CCV, Fashion, and Caltech-derived datasets.Caltech supplies four evaluation settings with two through five views.
Experimental Setup: Performance is compared against four traditional methods and four deep methods using ACC, NMI, and PUR averaged over 10 runs.The baselines include RMSL, MVC-LFA, COMIC, IMVTSC, CDIMC-net, EAMC, SiMVC, and CoMVC.
Results: MFLVC achieves the best results on all metrics across four datasets and improves ACC on Fashion by about 14% over CoMVC’s 85%.The authors attribute this to separating reconstruction and consistency objectives across feature spaces while avoiding feature fusion.
Results: MFLVC mostly achieves the best performance across Caltech settings with different view counts, while some baselines degrade as views increase.For example, RMSL, COMIC, and EAMC reach about 35%, 53%, and 31% ACC on Caltech-5V, versus 59%, 63%, and 35% on Caltech-4V.
Result Analysis: During training, low-level features retain sample diversity whereas high-level features form denser clusters with better low-dimensional manifolds.The visualization is based on the MNIST view of MNIST-USPS.
Result Analysis: Positive feature-pair and label-pair similarities rise while negative feature-pair similarities fall, supporting common-semantic learning and clustering consistency.The loss analysis also reports increasing clustering effectiveness as LZ, LH, LQ, and LP decrease.

5. Model Analysis

The analyses show that MFLVC’s separated feature levels support reconstruction, common-semantic learning, clustering consistency, convergence, and low sensitivity to hyper-parameter choices.

Multi-level feature learning: Low-level features retain sample diversity through reconstruction, while high-level features develop clearer cluster structures during training.The MNIST-USPS visualization reports clearer low- and high-level cluster structures, with low-level clusters remaining less dense because reconstruction preserves diversity.
Multi-level feature learning: Positive feature-pair similarity rises while negative-pair similarity falls, indicating that high-level features approach common semantics across views.The reported behavior is attributed to the feature MLP filtering view-private information and producing denser high-level features.
Multi-level feature learning: Positive label-pair similarity also rises, indicating that semantic-label clustering consistency is achieved.
Convergence analysis: Clustering effectiveness increases as the loss values decrease, indicating that MFLVC has good convergence properties.The analysis states that LZ, LH, LQ, and LP are convex objectives.
Parameter sensitivity analysis: Mean NMI remains insensitive to λ1 and λ2 across 10 independent runs, supporting robustness to loss-component weighting.The model sets λ1 = 1.0 and λ2 = 1.0 for all used datasets; the supplied analysis also reports insensitivity to τF and τL.
Ablation studies: The loss ablation assigns distinct roles to LQ, LZ, LH, and LP: clustering consistency, reconstruction, high-level feature learning, and label refinement.The complete loss combines these components, while alternative contrastive structures are evaluated separately.

6. Conclusion

The paper concludes that MFLVC learns low-level features, high-level features, and semantic labels in a fusion-free manner to capture common semantics while reducing view-private influence. It reports state-of-the-art performance on five public datasets and identifies downstream uses alongside sensitivity to intrinsic data bias.

Conclusion: MFLVC learns low-level features, high-level features, and semantic labels in a fusion-free manner for multi-view clustering.
Conclusion: The framework is intended to learn common semantics across views while reducing the adverse influence of view-private information.
Conclusion: Extensive experiments on five public datasets demonstrate state-of-the-art performance.
Broader impacts: The learned high-level feature extractor and label predictor can support feature compression, unsupervised labeling, and cross-modal retrieval.
Broader impacts: The trained model might be affected by intrinsic data bias, especially with dirty samples.The paper presents extending the framework to other application scenarios as future work.