PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Incomplete Multi-view Clustering with Cross-view Partial Sample and Prototype Alignment

Jiaqi Jin, Siwei Wang, Zhibin Dong, Xinwang Liu, En Zhu

arXiv:2303.15689v2cs.LGcs.CV

TL;DR

Incomplete multi-view clustering must work with partially available samples, and existing contrastive methods may impose rigid cross-view representations while prototype sets become shifted. CPSPAN aligns pair-observed samples and prototypes across views. Experiments report that it outperforms conventional and deep IMVC state-of-the-art methods.

  • Problem

    Incomplete views challenge multi-view clustering, and contrastive deep methods may enforce uniform cross-view representations while incomplete data can bias cluster prototypes.

    Source: §1 · p. 1

  • Method

    CPSPAN uses pair-observed data to guide cross-view sample correspondences and aligns prototype sets across views.

    Source: §1 · p. 2

  • Results

    Experiments demonstrate CPSPAN’s superiority over conventional and deep IMVC state-of-the-art methods.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    The paper presents sample and prototype alignment as a framework for deep incomplete multi-view clustering.

    Source: §1 · p. 2, §5 · p. 8

    The approach treats IMVC as a partially aligned setting in which pair-observed data provides instance-alignment supervision.

    Source: §1 · p. 2

Abstract

from arXiv · show

The success of existing multi-view clustering relies on the assumption of sample integrity across multiple views. However, in real-world scenarios, samples of multi-view are partially available due to data corruption or sensor failure, which leads to incomplete multi-view clustering study (IMVC). Although several attempts have been proposed to address IMVC, they suffer from the following drawbacks: i) Existing methods mainly adopt cross-view contrastive learning forcing the representations of each sample across views to be exactly the same, which might ignore view discrepancy and flexibility in representations; ii) Due to the absence of non-observed samples across multiple views, the obtained prototypes of clusters might be unaligned and biased, leading to incorrect fusion. To address the above issues, we propose a Cross-view Partial Sample and Prototype Alignment Network (CPSPAN) for Deep Incomplete Multi-view Clustering. Firstly, unlike existing contrastive-based methods, we adopt pair-observed data alignment as 'proxy supervised signals' to guide instance-to-instance correspondence construction among views. Then, regarding of the shifted prototypes in IMVC, we further propose a prototype alignment module to achieve incomplete distribution calibration across views. Extensive experimental results showcase the effectiveness of our proposed modules, attaining noteworthy performance improvements when compared to existing IMVC competitors on benchmark datasets.

1. Introduction

Incomplete views challenge multi-view clustering, while contrastive deep methods may impose overly rigid cross-view representations and overlook shifted cluster prototypes. CPSPAN uses pair-observed sample alignment and prototype alignment; experiments report superiority over conventional and deep state-of-the-art methods.

1. Introduction

  • Existing multi-view clustering assumes fully available data, but missing views can disrupt sample alignment and limit the mining of complementary and consistent information.Conventional methods also rely heavily on raw-feature quality.Source: §1 · p. 1
  • Contrastive deep methods may enforce uniform representations across views, overlooking view flexibility and the structural correspondence that incomplete multi-view clustering seeks.The paper frames the task as discovering structural correspondence rather than rigidly enforcing uniform representations.Source: §1 · p. 1
  • Incomplete data can shift and bias cluster prototypes across views, producing incorrect correspondences and motivating prototype matching.The paper terms this the Prototype-Shifted Problem (PSP).Source: §1 · p. 2, Figure 1 · p. 2
  • CPSPAN uses pair-observed samples to construct cross-view correspondences and aligns prototype sets to address shifted prototypes.The prototype alignment module solves for correspondences between prototype sets, while structural relationship transfer supports missing-item imputation before fusion and clustering.Source: §1 · p. 2
  • CPSPAN’s experiments demonstrate the effectiveness of its alignment modules and superiority over conventional and deep state-of-the-art methods.The introduction reports these comparisons without giving benchmark-specific values.Source: §1 · p. 2

2. Related Work

Related work reviews conventional incomplete multi-view clustering and its deep-learning extensions. It groups conventional approaches by matrix factorization, kernel learning, and graph learning, and describes several families of deep methods.

2. Related Work

  • The related-work section surveys recent progress in incomplete and deep incomplete multi-view clustering.Source: §2 · p. 2

2.1. Incomplete Multi-view Clustering

  • Conventional IMVC methods use matrix factorization, kernel learning, or graph learning, but rely on raw-feature quality and have limits in learning clustering-friendly representations of complex data.Pair-observed instances are those complete in both views i and j.Source: §2.1 · pp. 2–3, Figure 2 · p. 3

2.2. Deep Incomplete Multi-view Clustering

  • Deep IMVC methods include autoencoder-, GAN-, GCN-, and contrastive-based approaches; the paper says contrastive methods can reduce representation flexibility and diversity and lose complementary information.The cited representative contrastive work is.Source: §2.2 · p. 3

3. Proposed Method

CPSPAN jointly learns incomplete-view representations, aligns pair-observed samples and shifted prototypes, then fills missing embeddings before clustering. Its modules are trained together, with their losses combined in a shared objective.

3.1. Representation Learning Module and Prototype Set Generation

  • Each view uses an autoencoder trained only on complete instances, projecting features to a shared dimension for cross-view prototype comparison.Prototype sets are generated from the view-specific embeddings and used in the subsequent alignment modules.Source: §3.1 · p. 3

3.2. Cross-view Partial Sample Alignment

  • CPSA uses cosine similarity to maximize matching for pair-observed embeddings without forcing matches to other samples to zero, allowing more flexible representations.The authors state that this avoids stretching samples that originally belong to the same cluster and better suits structural-information mining.Source: §3.2 · p. 4

3.3. Shifted Prototype Alignment

  • SPA optimizes pairwise prototype correspondence as an integer linear program and uses a differentiable Hungarian-algorithm substitution to align prototype sets.The module is intended to calibrate prototype relationships across views and address prototype shifts.Source: §3.3 · p. 5

3.4. Structure Embedding Imputation Strategy

  • After training, structure embedding imputation fills a missing feature with the embedding of its nearest neighbor from another view.For a missing feature in view 1, the method finds the nearest neighbor to its embedding in view 2 and transfers that neighbor's embedding in view 1.Source: §3.4 · p. 5

3.5. The Objective Function

  • The shared objective combines within-view reconstruction, cross-view partial-sample alignment, and shifted-prototype alignment losses, with both balance coefficients set to 0.001.The complete and imputed embeddings are concatenated before k-means produces the final clustering.Source: Figure 3 · p. 4, §3.5 · p. 5

4. Experiments

CPSPAN is evaluated on five benchmark datasets under four missing rates, with comparisons to seven incomplete multi-view clustering methods. The reported experiments assess its benchmark performance, alignment modules, imputation stability, parameter sensitivity, and clustering visualizations.

4.1. Experimental Setting

  • Experiments cover Caltech101-7, HandWritten, ALOI-100, YouTubeFace10, and EMNIST at missing rates 0.1, 0.3, 0.5, and 0.7, using ACC, NMI, and F-measure.Source: §4.1 · pp. 5–6

4.3. Flexibility Validation of Cross-view Partial Sample Alignment

  • CPSPAN outperforms the contrastive-loss variant at every missing rate on Caltech101-7 and HandWritten.Figure 5 contrasts CPSPAN’s flexible features with features learned using contrastive loss.Source: Figure 5 · p. 6, §4.3 · p. 6

4.2. Comparison with State-of-The-Arts

  • At a 0.7 missing rate, CPSPAN’s ACC exceeds the second-highest method by 7.9%, 5.9%, 1.3%, 7.3%, and 15.4% across the five datasets, respectively.Source: §4.2 · p. 6
  • CPSPAN significantly outperforms other incomplete multi-view clustering algorithms across all datasets and metrics.Table 2 covers five datasets and varying missing rates.Source: §4.2 · p. 6, Table 2 · p. 7

4.4. Stability Analysis of Imputation Strategy

  • Varying the nearest-neighbor index from 1 to 100 reduces performance by less than 5% on both datasets.Source: §4.4 · p. 7

4.5. Parameter Sensitivity Analysis

  • On Caltech101-7, β has a greater impact than α in the sensitivity analysis.Source: §4.5 · p. 7

4.6. Ablation Study on Modules

  • Both cross-view instance alignment and shifted prototype alignment significantly contribute to performance on Caltech101-7 and HandWritten.The ablations are reported in Tables 3 and 4, respectively, using ACC.Source: §4.6 · p. 8, Table 3 · p. 8, Table 4 · p. 8

4.7. Visualization

  • At a missing rate of 0.5, final embeddings show more compact same-cluster instances and greater separation between different clusters; prototypes are re-estimated and calibrated after alignment.Source: §4.7 · p. 8, Figure 6 · p. 8

5. Conclusion

The conclusion presents CPSPAN as a framework for incomplete multi-view data that aligns partially observed samples and cluster prototypes across views. Experiments are reported to show superiority over conventional and deep IMVC state-of-the-art methods.

  • CPSPAN is proposed as a cross-view partial sample and prototype alignment framework for incomplete multi-view data.Source: §5 · p. 8
  • Pair-observed data alignment guides reconstruction of correspondence between samples.Source: §5 · p. 8
  • A shifted prototype alignment module calibrates prototype sets across views to address the prototype-shifted problem.Source: §5 · p. 8
  • Experiments demonstrate CPSPAN’s superiority to conventional and deep IMVC state-of-the-art methods.Source: §5 · p. 8
Loading 2303.15689v2…