PaperlayerView exact revision on arXiv

Source-linked AI summary

Self-Supervised Discriminative Feature Learning for Deep Multi-View Clustering

Jie Xu, Yazhou Ren, Huayi Tang, Zhimeng Yang, Lili Pan, Yang Yang, Xiaorong Pu, Philip S. Yu, Lifang He

arXiv:2103.15069v3cs.LGcs.CV

TL;DR

Multi-view clustering must exploit complementary views without being undermined by views with unclear clustering structures. SDMVC independently learns view embeddings, combines them into global features, and uses pseudo-labels with a unified target distribution for discriminative learning. Experiments across different multi-view datasets report superior performance over recent state-of-the-art methods.

  • Problem

    Existing multi-view clustering methods insufficiently address the negative impact of views with unclear clustering structures.

  • Method

    SDMVC independently learns each view’s embedded features, concatenates them into global features, and iteratively uses a unified target distribution for self-supervised discriminative learning.

  • Results

    SDMVC achieves the best quantitative performance across all evaluated datasets, with statistically significant improvements under a 5% t-test threshold.

  • Takeaways & Limitations

    SDMVC simultaneously promotes consistent cluster assignments across views while preserving feature diversity and reducing the impact of unclear views.

  • Takeaways & Limitations

    The framework assumes that view diversity can make consistent predictions unreasonable for all examples, and defines aligned examples accordingly.

Abstract

from arXiv · show

Multi-view clustering is an important research topic due to its capability to utilize complementary information from multiple views. However, there are few methods to consider the negative impact caused by certain views with unclear clustering structures, resulting in poor multi-view clustering performance. To address this drawback, we propose self-supervised discriminative feature learning for deep multi-view clustering (SDMVC). Concretely, deep autoencoders are applied to learn embedded features for each view independently. To leverage the multi-view complementary information, we concatenate all views' embedded features to form the global features, which can overcome the negative impact of some views' unclear clustering structures. In a self-supervised manner, pseudo-labels are obtained to build a unified target distribution to perform multi-view discriminative feature learning. During this process, global discriminative information can be mined to supervise all views to learn more discriminative features, which in turn are used to update the target distribution. Besides, this unified target distribution can make SDMVC learn consistent cluster assignments, which accomplishes the clustering consistency of multiple views while preserving their features' diversity. Experiments on various types of multi-view datasets show that SDMVC outperforms 14 competitors including classic and state-of-the-art methods. The code is available at https://github.com/SubmissionsIn/SDMVC.

1 INTRODUCTION

Multi-view clustering exploits complementary information across views, but existing methods can struggle with unclear view-specific structures and feature-diversity trade-offs. SDMVC addresses these issues through self-supervised discriminative feature learning with a unified target distribution.

  • Motivation: Multi-view clustering leverages complementary information and comprehensive characteristics from multiple views or modalities.Examples include multiple object mappings, visual plus textual features, and SIFT plus LBP.
  • Motivation: Existing methods may use many dataset-sensitive hyper-parameters and enforce view consistency in embedded feature space, reducing feature diversity.Unclear clustering structures in some views can also negatively affect multi-view clustering performance.
  • Proposed method: SDMVC independently learns embedded features with autoencoders, concatenates them into global features, and constructs pseudo-labels and a unified target distribution.The unified target distribution supports self-supervised multi-view discriminative feature learning.
  • Proposed method: Global discriminative information supervises all views to learn more discriminative features while preserving view-specific diversity and complementary information.Independent view training allows features to retain their distinct characteristics.
  • Contributions: The framework theoretically analyzes a generalization bound showing expected clustering risk is bounded by empirical training risk with high probability.The result concerns the proposed loss function.
  • Contributions: Experiments on different dataset types report superior performance over recent state-of-the-art methods, with linear complexity in data size and fewer hyper-parameters.The experiments are described as demonstrating superior performance across datasets.

2 RELATED WORK

Related work spans deep autoencoder clustering, subspace and graph-based multi-view methods, and self-supervised approaches. SDMVC introduces self-supervised multi-view discriminative feature learning within deep multi-view clustering.

  • Deep clustering: Deep autoencoder clustering methods learn embedded features and cluster assignments, with IDEC adding reconstruction regularization to prevent embedded-space collapse.DEC jointly learns cluster assignments and autoencoder features; IDEC trades off clustering and reconstruction.
  • Multi-view clustering: Subspace-based multi-view methods model shared latent structure, while other approaches learn joint representations or mine common structure and cluster assignments.These methods include self-representation, encoding, and joint-embedding strategies.
  • Multi-view clustering: Graph-based approaches integrate graph or connection-structure information to learn multi-view representations.Graph autoencoders have also been introduced for this purpose.
  • Deep multi-view clustering: Deep multi-view methods use autoencoders, diversity regularization, adversarial learning, and related techniques to model multi-view information.The cited approaches target self-representation, higher-order relations, or intrinsic embedded structure.
  • Self-supervised learning: Self-supervised multi-view clustering had been combined with subspace clustering, but the cited approach depends on eigenvalue decomposition and has cubic data-size complexity.This distinguishes the prior self-supervised approach from SDMVC’s stated linear complexity.
  • Novelty: SDMVC is presented as the first work to propose a self-supervised multi-view discriminative feature learning framework.The framework targets consistent predictions across views while retaining view diversity.

3 THE PROPOSED METHOD

SDMVC independently learns view-specific embeddings, then uses concatenated global features to build a unified target distribution that guides discriminative feature learning and consistent cluster assignments across views.

  • View-specific representation learning: Deep autoencoders independently transform each view into low-dimensional embedded features while preserving view-specific characteristics for complementary information.Each view has its own autoencoder and clustering layer.
  • Global pseudo-label construction: Global features concatenate all views’ embeddings so discriminative information from clearer views can reduce the influence of unclear clustering structures.K-means on global features produces robust pseudo soft assignments when one view is ambiguous.
  • Self-supervised discriminative learning: A unified target distribution is derived from pseudo soft assignments and used in every view’s KL-divergence clustering loss.High-probability pseudo-assignment components are enhanced to increase discriminability.
  • Self-supervised discriminative learning: Each view’s total loss combines clustering and reconstruction terms, with reconstruction regularizing embeddings and clustering loss increasing their discriminability.The trade-off coefficient γ balances the two loss components.
  • Iterative refinement: Iterative optimization uses improved view features to update the target distribution, progressively strengthening discriminability and clarifying clustering structures.The framework particularly guides views whose initial features are less discriminative.
  • Generalization bound: Under boundedness assumptions, the expected clustering risk is bounded by empirical risk and sample-dependent complexity terms with high probability.The bound depends on constants associated with K, M, and γ.
  • Consistent multi-view clustering: Minimizing multiple KL divergences against one unified target makes cluster assignments across views tend toward consistency while independent optimization preserves feature diversity.The paper states this consistency result theoretically and links the design to reduced negative influence from unclear views.

4 EXPERIMENTAL SETUP

The experiments evaluate SDMVC on several public multi-view datasets against traditional, shallow, deep, single-view, and multi-view clustering baselines using standard clustering metrics and fixed implementation settings.

  • Optimization procedure: Training pre-trains each view’s autoencoder, initializes centroids with K-means, repeatedly updates the target distribution, fine-tunes autoencoders, and stops near 90% aligned rate.Optimization uses mini-batch gradient descent and backpropagation.
  • Datasets: Experiments use Caltech101-20, MNIST-USPS, Fashion-MV, and BDGP, covering image, handwritten-digit, fashion-product, visual, and textual multi-view data.The datasets vary in categories, views, sample counts, and feature modalities.
  • Compared methods: SDMVC is compared with classic and recent methods spanning K-means, spectral clustering, deep embedded clustering, matrix-factorization, graph-based, adversarial, and collaborative multi-view approaches.The comparison includes both single-view and multi-view baselines, as well as shallow and deep models.
  • Implementation settings: Fully connected autoencoders process vector views, while convolutional autoencoders process image views; all embedded features have dimensionality 10.The setup uses ReLU activations, Adam with default learning rate 0.001, batch size 256, and input scaling to [0, 1].
  • Computational environment: The experiments run on a Windows PC equipped with a GeForce RTX 2060 GPU, Intel i5-9400F CPU, and 16 GB RAM.The reported hardware specifies the computational environment.
  • Evaluation protocol: The evaluation reports ACC, NMI, and ARI averaged over 10 runs, where larger values indicate better clustering performance.These metrics are used for quantitative comparison.

5.1 Results on Real Data

SDMVC achieves the best quantitative clustering performance across all evaluated datasets and is more robust across datasets than several competing methods.

  • SDMVC achieves the best performance on the quantitative metrics across all datasets, with statistically significant improvements at the 5% t-test level.The comparison includes convolutional and fully connected autoencoder settings.
  • SDMVC improves existing methods by a large margin on Caltech101-20.
  • Some competitors perform well on individual datasets but lack robustness across multiple datasets.DAMC performs well on BDGP but poorly elsewhere, while SiMVC and CoMVC show the reverse pattern across several datasets.
  • On BDGP, EAMC and SiMVC underperform single-view methods DEC and IDEC.The passage attributes this limitation to large discriminability gaps and unclear clustering structures among views.

5.2 Ablation Study

The ablation and visualization analyses support SDMVC’s unified target distribution and self-supervised, view-independent learning design for improving individual-view clustering while preserving multi-view structure.

  • 5.2.1 Improvements Compared to Individual Views: SDMVC improves clustering over IDEC by about 15% on MNIST-USPS, 30% on Fashion-MV, 3% on BDGP, and 30% on Caltech101-20.The improvement also applies to views with the worst individual clustering performance.
  • 5.2.1 Improvements Compared to Individual Views: SDMVC improves even the worst-performing views, indicating that it mitigates negative effects from low-discriminative views and mines complementary information across views.
  • 5.2.2 Multi-View Discriminative Feature Learning: After pre-training, embedded features have low discriminability and centroids poorly reflect true cluster structures.This initial state corresponds to low NMI and low Aligned Rate.
  • 5.2.2 Multi-View Discriminative Feature Learning: Global features initially improve clustering by exploiting discriminative information from both views before view-specific feature learning.
  • 5.2.2 Multi-View Discriminative Feature Learning: During subsequent learning, embedded-view clustering structures become clearer and centroids separate as NMI and Aligned Rates increase.
  • 5.2.3 Unified Target Distribution (UTD) and Self-Supervised Manner (SSM): Removing the unified target distribution produces unsatisfactory results, while removing the self-supervised manner limits improvement over IDEC.The variants support necessary contributions from both components.

5.3 Model Analysis

SDMVC’s convergence and sensitivity analyses examine alignment, KL-divergence loss, the trade-off coefficient, and target-distribution update frequency to characterize training behavior and settings.

  • 5.3.1 Convergence Analysis: Aligned Rate and clustering performance are positively correlated during training.
  • 5.3.1 Convergence Analysis: Each target-distribution update increases KL-divergence loss because newly generated pseudo soft assignments have stronger discriminability.Subsequent consistency training improves the Aligned Rate across views.
  • 5.3.1 Convergence Analysis: The objectives in Eq. (3) and Eq. (12) are convex, supporting good convergence after each target-distribution update.
  • 5.3.1 Convergence Analysis: A high Aligned Rate such as 90% is recommended as an unsupervised stopping criterion for multi-view clustering consistency.
  • 5.3.2 Parameter Analysis: The clustering–reconstruction trade-off coefficient γ is robust in [10^-1, 10^1], and γ = 0.1 is used for all experiments.Too-large γ can degenerate feature diversity, whereas too-small γ may prevent meaningful clusters.
  • 5.3.3 Batch Analysis: Updating the unified target distribution too frequently harms performance, while larger batch settings are steadier; all datasets use 1000 batches between updates.

6 CONCLUSION

SDMVC is proposed as a self-supervised discriminative feature-learning framework for deep multi-view clustering. It uses global information to improve view representations and consistency, while reporting theoretical and empirical support for its performance and scalability.

  • SDMVC uses global discriminative information to establish a unified target distribution for learning more discriminative features and consistent predictions across views.
  • The framework is designed to reduce the negative clustering impact of views with unclear clustering structures.
  • Experiments on different types of multi-view datasets report superior performance over recent state-of-the-art multi-view clustering methods.
  • The paper theoretically analyzes SDMVC's generalization bound.
  • SDMVC has linear complexity in data size, and its feature-learning paradigm may extend to semi-supervised clustering and classification.
  • The authors identify the stability of autoencoders' unsupervised pre-training as a bottleneck for autoencoder-based multi-view clustering.

APPENDIX

The appendix establishes a generalization bound under boundedness assumptions and derives the bound using expectation estimates, concentration inequalities, and constants depending on model-related quantities.

  • Theorem 1 assumes uniformly bounded inputs, representations, and parameters, with norms constrained to the interval [0, M].
  • The bound holds with probability 1 −δ for every hypothesis h in H, relating empirical risk to expected risk.
  • McDiarmid's inequality is used after comparing sample sets that differ by one point, yielding a concentration step for the empirical risk.
  • The proof bounds the expectation term using the Khintchine-Kahane inequality and Rademacher random variables.
  • The resulting upper bound contains constants c1 and c2 that depend on K, M, and γ.
Loading 2103.15069v3…