PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Partial Multi-View Clustering

Qianqian Wang, Zhengming Ding, Zhiqiang Tao, Quanxue Gao, Yun Fu

arXiv:2003.13088v1cs.CV

TL;DR

Partial multi-view clustering is difficult because conventional methods assume complete views for every instance. GP-MVC learns shared representations, adaptively fuses views, and uses view-specific GANs to generate missing data; experiments report better clustering performance than competing methods.

  • Problem

    Conventional multi-view clustering assumes complete views for every instance, whereas real data can contain missing views caused by collection and equipment problems.

  • Method

    GP-MVC learns a consistent shared subspace for clustering and missing-view generation, using weighted adaptive fusion and view-specific generative adversarial networks.

  • Results

    Experiments on several multi-view datasets report better clustering performance for GP-MVC than several state-of-the-art methods.

  • Takeaways & Limitations

    GP-MVC jointly uses generated complementary views and shared clustering structure to address incomplete multi-view clustering.

Abstract

from arXiv · show

Nowadays, with the rapid development of data collection sources and feature extraction methods, multi-view data are getting easy to obtain and have received increasing research attention in recent years, among which, multi-view clustering (MVC) forms a mainstream research direction and is widely used in data analysis. However, existing MVC methods mainly assume that each sample appears in all the views, without considering the incomplete view case due to data corruption, sensor failure, equipment malfunction, etc. In this study, we design and build a generative partial multi-view clustering model, named as GP-MVC, to address the incomplete multi-view problem by explicitly generating the data of missing views. The main idea of GP-MVC lies at two-fold. First, multi-view encoder networks are trained to learn common low-dimensional representations, followed by a clustering layer to capture the consistent cluster structure across multiple views. Second, view-specific generative adversarial networks are developed to generate the missing data of one view conditioning on the shared representation given by other views. These two steps could be promoted mutually, where learning common representations facilitates data imputation and the generated data could further explores the view consistency. Moreover, an weighted adaptive fusion scheme is implemented to exploit the complementary information among different views. Experimental results on four benchmark datasets are provided to show the effectiveness of the proposed GP-MVC over the state-of-the-art methods.

I. INTRODUCTION

The paper addresses partial multi-view clustering, where missing views violate the completeness assumption of conventional MVC methods. It proposes GP-MVC, which learns shared representations, generates missing-view data, and integrates complementary information for clustering.

  • Motivation: Traditional MVC methods assume every instance has complete observations across all views, but missing views can degrade their performance.Incomplete views arise from environmental issues, noise, obstacles, and collection, transmission, or storage equipment malfunctions.
  • Related limitations: Existing PMVC methods may use zero or mean imputation, producing data unlike real observations and weakening consistent clustering.Other approaches establish shared latent subspaces or compensate missing latent representations.
  • Proposed approach: View-specific generators predict missing data from latent representations of other views, while discriminators distinguish generated samples from real samples.Adversarial training is used to explore consistent information across views.
  • Proposed approach: GP-MVC uses multi-view encoders to learn a consistent shared subspace that supports both clustering and missing-view generation.The model combines encoder networks, a weighted adaptive fusion layer, a clustering layer, and view-specific generative adversarial networks.
  • Proposed approach: Weighted adaptive fusion exploits complementary information across views through learnable weights, while clustering is integrated into the generation process.The authors report that generated complementary views can improve clustering performance.
  • Results: Experiments on several multi-view datasets report superior clustering performance for GP-MVC compared with several state-of-the-art methods.The paper describes extensive experiments and comparisons with existing methods.

A. Multi-View Clustering

The paper reviews multi-view and partial multi-view clustering methods, emphasizing their shared-structure objectives and computational or modeling limitations for incomplete data.

  • Multi-View Clustering: Multi-view clustering methods learn shared structures across views and use fused information to obtain cluster assignments.Spectral-based approaches commonly learn shared similarity matrices before spectral clustering.
  • Partial Multi-View Clustering: Some partial multi-view methods complete missing kernel matrices using a complete view or shared-instance alignment.One approach requires at least one complete view containing all instances.
  • Limitations: Kernel and NMF-based PMVC methods predict missing data or representations but can require substantial computation.NMF-based methods may need inverse operations and therefore cannot be applied to large-scale data.
  • Limitations: Existing PMVC methods mainly learn a shared latent space for clustering without also learning a space suitable for generating missing views.This limitation motivates a model that connects clustering representations with missing-view generation.

B. Framework

The GP-MVC framework encodes each view, adaptively fuses latent representations, refines their clustering structure, and uses view-specific adversarial networks to generate missing views.

  • Framework: GP-MVC contains encoders, a weighted adaptive fusion layer, a deep embedding clustering layer, generators, and discriminators.For V views, it uses V encoders, one fusion layer, one clustering layer, V generators, and V discriminators.
  • Encoder Network: Each encoder maps an original view into a low-dimensional latent representation, with partially shared parameters across views.The shared parameters are intended to capture common multi-view structure.
  • Generator Network: Each view-specific generator takes a latent representation from another view and outputs a generated sample for its target view.The generator can also function as the decoder in the auto-encoder structure.
  • Discriminator Network: Each discriminator judges whether a target-view sample is real or generated and feeds its judgment back to the corresponding generator.Adversarial training continues until the generator can produce samples that the discriminator cannot distinguish from real ones.
  • Weighted Adaptive Fusion: The fusion layer combines the view-specific latent spaces into a common representation using learnable view weights.The common representation is computed through a fusion function parameterized by β = [β1, . . . , βV].
  • Deep Embedded Clustering: The deep embedding clustering layer refines the common representation toward a more compact target distribution and produces clustering results.The encoder is refined to make the learned representation resemble the target distribution.

C. Objective Function

GP-MVC optimizes four complementary terms: auto-encoder reconstruction, adversarial training, weighted adaptive fusion, and KL clustering. The auto-encoder reconstructs inputs from latent representations, while adversarial training addresses degradation caused by unpaired data.

  • Objective components: GP-MVC’s objective function combines auto-encoder, adversarial training, weighted adaptive fusion, and KL clustering losses.These four terms jointly define the model objective.
  • Auto-encoder loss: The auto-encoder loss minimizes the squared Frobenius-norm reconstruction error between generated and input samples.The encoder produces a latent representation, and the generator acts as its decoder.
  • Paired versus partial data: When input data are fully paired, auto-encoder loss alone can train the encoder and generator networks.This sufficiency does not hold in the partial multi-view setting.
  • Paired versus partial data: Unpaired data greatly degrade encoder and generator performance, motivating adversarial training in addition to auto-encoder loss.The adversarial term is introduced to address the lack of paired data.
  • Adversarial training: A GAN uses a generator to produce samples from noise and a discriminator to distinguish generated samples from real data.The discriminator outputs a value indicating whether an input is real or fake.

2) Adversarial Training Loss:

GP-MVC uses cycle-GAN-based adversarial training to generate one view from another despite abundant unpaired data. GAN loss enforces realism, while cycle consistency constrains translations to preserve the desired correspondence.

  • Adversarial training: Cycle GAN is used because partial multi-view data contain abundant unpaired samples and insufficient paired samples.The model trains two GAN mappings and adds cycle-consistency loss to adversarial loss.
  • Cross-view generation: The mapping Gv∘Ew transforms a sample from view w into a generated sample in view v, while the reverse mapping translates back.The discriminator compares generated and real samples in the target view.
  • Adversarial training: GAN loss makes generated samples resemble real samples, whereas cycle consistency prevents arbitrary permutations within the target distribution.Cycle consistency helps map an input to a specific paired output rather than any plausible target sample.
  • Cycle consistency: The multi-view cycle-consistency loss minimizes the ℓ1-norm reconstruction error between the final translated sample and the input.Combining GAN and cycle-consistency losses promotes a desired cross-view output.

3) Weighted Adaptive Fusion Loss:

After encoding each view, GP-MVC fuses the latent subspaces with a weighted adaptive function to extract shared information for clustering. The resulting common representation supports confidence-aware KL-based clustering refinement.

  • Weighted adaptive fusion: GP-MVC fuses the latent representations from all views before clustering.The latent spaces are combined into a common representation Z.
  • Weighted adaptive fusion: The fusion function h(·) may concatenate or sum the view-specific latent representations.This defines how multiple latent subspaces are combined.
  • Weighted adaptive fusion: The adaptive fusion extracts public identification information beneficial to clustering from the multi-modal primitive space.The encoder outputs Z^(v), while β parameterizes the learnable fusion function.
  • KL clustering loss: The clustering layer assigns common-representation points to centroids using a Student’s t-distribution kernel.The resulting probabilities are soft cluster assignments.
  • KL clustering loss: Minimizing KL divergence between the original and target distributions sharpens assignments and emphasizes high-confidence data points.The procedure concentrates samples from the same class and seeks a more common representation.

5) Overall objective:

The overall GP-MVC procedure initializes encoders, generators, discriminators, and clustering components, then alternates representation learning, adversarial completion, and clustering updates. It finally clusters the completed common representation.

  • Overall objective: The GP-MVC objective integrates auto-encoder, adversarial training, weighted adaptive fusion, and KL clustering losses.The coefficients λ1, λ2, and λ3 control the impact of objective terms.
  • Training procedure: Algorithm 1 first trains the encoder and generator on paired data using the auto-encoder objective.It initializes the encoder, clustering layer, generator, and discriminator before iterative updates.
  • Training procedure: The second training stage updates the generator and discriminator on all data and generates missing samples.This stage uses the adversarial objective for view completion.
  • Training procedure: The procedure computes a common representation and updates clustering centroids during training.The common representation is recomputed after model updates.
  • Final clustering: After iterative training, GP-MVC performs clustering on the common representation.The final clustering input is the fused representation produced by the preceding stages.

D. Implementation

GP-MVC is trained in three stages: paired data initialize encoders and generators, all data train generator–discriminator networks, and the completed dataset supports joint refinement.

  • Training procedure: Step 1 trains encoders and generators on paired data using an auto-encoder loss and obtains clustering centroids.The centroids help assign generated missing-view samples to the appropriate group.
  • Training procedure: Step 2 trains view-specific generators and discriminators on all data using a multi-view cycle GAN.Paired data enter the encoders directly, while unpaired data are used to generate missing views.
  • Training procedure: Generated missing-view samples are saved, and a new common representation is computed on the completed database.This completes the data before the final joint training stage.
  • Training procedure: Step 3 retrains the model on the completed dataset while updating centroids, the common representation, and generated missing-view samples each epoch.The completed inputs include original and generated views.
  • Training procedure: Algorithm 1 summarizes the overall training procedure, which alternates initialization, missing-view generation, and refinement.The procedure is explicitly presented as the model’s training algorithm.

IV. EXPERIMENTAL ANALYSIS

The experiments evaluate GP-MVC on four multi-view databases spanning visual, textual, image, and feature-based representations, with comparisons against state-of-the-art PMVC and MVC methods.

  • Experimental datasets: Experiments compare GP-MVC with state-of-the-art partial multi-view clustering and multi-view clustering methods on four databases.The evaluation is designed to test performance across both PMVC and MVC baselines.
  • Experimental datasets: BDGP contains 2,500 drosophila-embryo images represented by 1,750-dimensional visual and 79-dimensional textual vectors.All data are used to evaluate clustering on the two features.
  • Experimental datasets: Sampled MNIST uses 4,000 randomly selected images with original black-and-white and corresponding edge views.The sampled database is used because comparison on the full large-scale database is difficult.
  • Experimental datasets: Handwritten numerals contains 2,000 images from 10 digit classes and six feature types.The features include shape, profile, transform, pixel, and rotationally invariant descriptors.
  • Experimental datasets: NUS-WIDE contributes 269,648 images from 12 selected animal concepts with three low-level feature types.The extracted features are color correlogram, wavelet texture, and block-wise color moment.

2) Baseline Methods and Evaluate Metrics:

The study compares GP-MVC with PMVC and MVC baselines under varying completeness ratios, using standard clustering metrics and repeated evaluations; GP-MVC performs best across the reported cases.

  • Baselines and metrics: Baselines include IMG, PVC, PVC-GAN, ConSC, RMSC, and other multi-view clustering methods.The comparisons cover both partial multi-view and conventional multi-view clustering approaches.
  • Baselines and metrics: Impartial ratios range from 0.1 to 0.9 at 0.2 intervals, and MVC methods receive average-feature imputations for missing instances.The evaluation uses Accuracy, NMI, and Purity as clustering validation metrics.
  • Evaluation protocol: The experiments randomly create missing data at impartial ratios 0.1, 0.3, 0.5, 0.7, and 0.9, repeating each setting 10 times.Average accuracy and standard deviation are reported across four databases, with NMI and Purity also presented.
  • Results: GP-MVC achieves better clustering performance than the other methods across all reported cases.The evaluation results are summarized in Tables I–IV and Figure 4.
  • Results: PMVC methods are superior in most databases, especially at larger partial ratios, while some MVC methods are sensitive to missing data or noise.The results indicate that missing-view data negatively affect MVC effectiveness.
  • Results: GP-MVC outperforms the tested methods, and its weighted adaptive fusion improves over PVC-GAN in the reported comparisons.The authors associate this advantage with consistent per-view clustering structures and more effective missing-data generation.
  • Evaluation scope: PVC, IMG, and PVC-GAN are evaluated using only the first two views on HW and NUS because those methods support two-view databases.The reported results indicate that GP-MVC can be applied to multi-view databases with more than two views.

C. Model Discussion

The ablation studies show that combining all objective-function terms gives the strongest clustering performance, while generated missing-view data improve representation learning. Visual and large-scale experiments further indicate that GP-MVC can generate missing data and operate on the whole MNIST database.

  • Ablation Study: The full objective function achieves the best clustering performance across the tested impartial ratios.The complete model adds adaptive fusion and KL-clustering losses to auto-encoder and adversarial training losses.
  • Ablation Study: Adding adversarial training to auto-encoder loss improves clustering accuracy.The study attributes this improvement to generated missing-view samples promoting a better clustering structure.
  • Ablation Study: Adaptive fusion and KL-clustering losses further boost performance after adversarial training is added.The reported explanation is that better common representations help generate more realistic missing-view samples and improve clustering again.
  • Missing Data Generation: GP-MVC generates missing MNIST-view images that become more realistic as the number of paired data increases.The generated images are shown under impartial ratios 0.1, 0.5, and 0.9.
  • Large-Scale Partial Multi-View Clustering: On the whole 60000 × 2-image MNIST database, other methods ran out of memory, while GP-MVC produced clustering results.The authors therefore report only GP-MVC's clustering performance for this large-scale setting.
  • Model Discussion: The paper concludes that GP-MVC fills incomplete views through a GAN-based common subspace while learning a clustering structure simultaneously.The approach also uses complementary incomplete-view information to study a consistent common structure.
Loading 2003.13088v1…