PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Partial Multi-View Learning

Changqing Zhang, Yajie Cui, Zongbo Han, Joey Tianyi Zhou, Huazhu Fu, Qinghua Hu

arXiv:2011.06170v1cs.LG

TL;DR

Partial multi-view learning must model complex cross-view correlations despite arbitrary missing-view patterns. CPM-Nets learns complete and structured latent representations using degradation and adversarial imputation strategies, and experiments report improvements across representation learning, classification, and data imputation.

  • Problem

    Partial multi-view learning must handle complex correlations and arbitrary missing-view patterns, whereas conventional methods assume complete views for every sample.

  • Method

    CPM-Nets learns a unified, complete, versatile, and structured latent representation using degradation, adversarial imputation, and a nonparametric classification loss.

  • Results

    CPM-Nets improves unified representation, classification, and data imputation over current state-of-the-art methods across diverse multi-view data.

  • Takeaways & Limitations

    The learned common representation supports joint use of all samples and views under arbitrary missing-view patterns, while clustering-like structure makes the classifier interpretable.

  • Takeaways & Limitations

    The completeness guarantee assumes that different views are conditionally independent given the shared multi-view representation.

Abstract

from arXiv · show

Although multi-view learning has made signifificant progress over the past few decades, it is still challenging due to the diffificulty in modeling complex correlations among different views, especially under the context of view missing. To address the challenge, we propose a novel framework termed Cross Partial Multi-View Networks (CPM-Nets), which aims to fully and flflexibly take advantage of multiple partial views. We fifirst provide a formal defifinition of completeness and versatility for multi-view representation and then theoretically prove the versatility of the learned latent representations. For completeness, the task of learning latent multi-view representation is specififically translated to a degradation process by mimicking data transmission, such that the optimal tradeoff between consistency and complementarity across different views can be implicitly achieved. Equipped with adversarial strategy, our model stably imputes missing views, encoding information from all views for each sample to be encoded into latent representation to further enhance the completeness. Furthermore, a nonparametric classifification loss is introduced to produce structured representations and prevent overfifitting, which endows the algorithm with promising generalization under view-missing cases. Extensive experimental results validate the effectiveness of our algorithm over existing state of the arts for classifification, representation learning and data imputation.

1 INTRODUCTION

CPM-Nets addresses partial multi-view learning by jointly learning complete, structured latent representations while flexibly handling arbitrary missing-view patterns. The framework combines degradation, adversarial imputation, and nonparametric classification strategies, with experiments reporting improvements in representation learning, classification, and imputation.

  • Motivation: Partial multi-view learning is difficult because complex cross-view correlations and missing views undermine conventional methods that assume every sample has all views.Existing approaches include completing missing views, grouping samples by available sources, or learning shared spaces, but these strategies face limitations under blockwise missingness and complex patterns.
  • Proposed framework: CPM-Nets learns a unified latent representation that jointly considers completeness and structure for partial multi-view data.The representation is designed to encode information from different views and support arbitrary view-missing patterns.
  • Representation and classification: The clustering-like classification schema improves latent separability, while the latent representation is theoretically and empirically shown to be complete and versatile.The classification design introduces structure intended to support generalization in partial-view settings.
  • Proposed framework: A degradation strategy flexibly encodes observed-view information, while adversarial training stabilizes missing-view imputation and improves the learned representation.The method jointly exploits samples and views regardless of their missing-view patterns.

2 CROSS PARTIAL MULTI-VIEW NETWORKS

CPM-Nets learn a unified latent representation for samples with arbitrary missing-view patterns by jointly modeling completeness, versatility, class structure, and training–test consistency.

  • Problem: Partial multi-view classification handles training and test samples with arbitrary combinations of available views.The framework targets comparability across samples despite differing view-missing patterns.
  • Completeness: CPM-Nets define completeness by requiring each available observation to be reconstructible from a shared latent representation.The reconstruction objective uses availability indicators and view-specific reconstruction networks.
  • Completeness: The shared representation is intended to encode information from available views into a common space, making samples with different missing patterns comparable.Representations are formed from available views and decoded through view-specific mappings.
  • Versatility: A versatile representation can reproduce mappings defined on any individual view, and Proposition 2.1 states that a solution to the reconstruction objective has this property.The proof composes each view reconstruction mapping with the individual-view mapping.
  • Theory: Theoretical analysis establishes an upper bound linking reconstruction error to versatility error under a Lipschitz-continuous mapping, with the result generalized intuitively to missing-view cases.The exact versatility guarantee is stated for the ideal setting, while practical cases minimize the associated error.
  • Structured representation: CPM-Nets jointly optimize reconstruction and a clustering-like classification loss, balancing multi-view completeness against class-structured separability.The coefficient λ controls the balance between completeness and class-label structure.

3 ENHANCED COMPLETENESS WITH ADVERSARIAL STRATEGY

CPM-Nets use adversarial learning to impute missing views and improve latent representations. Missing data and latent representations are optimized alternately, with reconstruction supervised by observed data.

  • CPM-Nets introduce adversarial learning to promote missing-view imputation and latent representation learning.
  • The generator produces missing data that are iteratively updated to approximate the distribution of observed data.
  • The reconstruction loss is supervised by observed data, while missing data and latent representations are jointly optimized.
  • Missing data and latent representations are updated alternately so each can improve the other.
  • Unlike standard GANs, CPM-Nets optimize latent representations as inputs, use sample-to-sample supervision, and employ multiple discriminators.

4.1 Experimental Settings

The experiments evaluate CPM-Nets on multiple multi-view datasets against feature concatenation, correlation-based, representation-learning, and metric-learning baselines. Parameters are selected by cross-validation, and results are averaged over repeated runs.

  • Experiments cover Handwritten, Animal, CUB, and additional multi-view datasets described in the study.
  • Compared methods include feature concatenation, CCA-based methods, autoencoder approaches, and metric-learning baselines.
  • Classification performance is compared across different missing rates η.
  • All methods use five-fold cross-validation for parameter tuning.
  • CPM-Nets tune latent dimensionality K from {64, 128, 256} and λ from {0.1, 1, 10}.
  • Each method runs 10 times, with average values and standard deviations reported.

4.2 Network Architectures and Parameter Settings

The study uses fully connected networks whose depth, latent dimensionality, hidden-layer sizes, output dimensions, and learning rates vary by dataset. Most datasets use three-layer architectures, with exceptions for CUB and Animal.

  • All datasets use fully connected networks with sigmoid activation and ℓ2 regularization of 0.001.
  • Handwritten: Handwritten uses three-layer networks with K = 64, M = 200, six view-specific output dimensions, and learning rate 0.001.
  • CUB: CUB uses two-layer networks with K = 128, output dimensions 1024 and 300, and learning rate 0.01.
  • Animal: Animal uses four-layer networks with K = 256, hidden sizes 512 and 1024, output dimension 4096 for both views, and learning rate 0.001.
  • Football and Politics: Football and Politics use three-layer networks with K = 256 and K = 128 respectively, both using learning rate 0.01.
  • 3Source-complete: 3Source-complete uses three-layer fully connected networks, while its listed view-specific dimensions accompany the complete-data setting.
  • ADNI and 3Source-partial: ADNI and 3Source-partial use three-layer networks with K = 128; their hidden sizes are 50 and 128 respectively.

4.3 Supervised Experimental Results

The experiments evaluate CPM-Nets across missing-view rates, representation structure, fine-tuning, imputation, and parameter settings. Results indicate robust performance under missing views, improved structured representations, and benefits from fine-tuning.

  • Representation learning: CPM-Nets is evaluated against state-of-the-art representation-learning methods under diverse random view-missing patterns and missing rates.Missing views are randomly selected while ensuring at least one view remains available for each sample.
  • Representation learning: As the missing rate increases, CPM-Nets experiences smaller performance degradation than competing methods; on Handwritten, the decline is less than 5% from η = 0.0 to η = 0.3.On CUB, CPM-Nets achieves 89.48% accuracy versus LMNN’s 86.27% without missing views, with the gap increasing as missingness rises.
  • Imputation comparison: With η = 0.5, CPM-Nets outperforms average-value and CRA-based completion comparisons across the reported multi-view datasets.It exceeds the second performer by 2.5%, 10.2%, 36.7%, and 6.6% on CUB, 3Sources-complete, Football, and Politics, respectively.
  • Representation visualization: CPM-Nets produces representations that better reveal class structure, with label information making clusters more compact and class margins clearer.The visualization uses Handwritten data with missing rate η = 0.5 and distinguishes unsupervised from supervised settings.
  • Fine-tuning evaluation: Fine-tuning improves performance over the without-fine-tuning procedure for most tested cases on Handwritten and CUB.The evaluation addresses the gap between label-dependent training and label-unavailable testing.

4.4 Unsupervised Experimental Results

The experiments evaluate CPM-GAN for missing-view imputation and latent-representation clustering across varying and naturally partial view patterns. CPM-GAN generally outperforms comparison methods, with adversarial training especially beneficial as missingness increases.

  • Imputation Performance: CPM-GAN outperforms all comparative methods across missing rates on all six datasets.Imputation is evaluated using NRMSE, and performance declines for every method as the missing-view ratio increases.
  • Imputation Performance: CPM-GAN consistently outperforms CPM-without-GAN, supporting the adversarial strategy for missing-view imputation.The reported advantage is observed across all datasets and is linked to improved latent representations.
  • Clustering Performance: At missing rate η = 0.5, CPM-GAN improves average clustering performance by 3.36% in ACC and 5.11% in NMI.These averages are reported for the comparison in Fig. 8.
  • Clustering Performance: CPM-GAN and CPM-without-GAN achieve promising ACC and NMI compared with baselines, while baseline performance degenerates under high missing rates.CPM-GAN is especially stronger than CPM-without-GAN at high missing rates, although the latter can slightly lead at low rates on Football or CUB.
  • Clustering Performance: CPM-Nets outperform comparison methods on naturally partial multi-view datasets in both ACC and NMI.The result is reported in Fig. 9 and is presented as evidence of complementary-information encoding and adversarial robustness.
  • Comparison with VIGAN: CPM-Nets outperform VIGAN in Accuracy and NMI on CUB and Animal despite VIGAN using 10% complete data for encoder training.The comparison is conducted on two-view datasets suitable for VIGAN.

5 CONCLUSIONS

The paper concludes that CPM-Nets learn complete, versatile representations from partial multi-view data and remain flexible across arbitrary missing-view patterns. Experiments indicate robustness to heavy and complex view missingness.

  • 5 CONCLUSIONS: CPM-Nets jointly exploit all samples and views while handling arbitrary view-missing patterns.The shared representation supports data with an arbitrary number of views and complex missing-view configurations.
  • 5 CONCLUSIONS: The learned representation is designed to capture complex correlations among multiple views and remain complete and versatile.A clustering-like classification loss structures the representation and makes the classifier interpretable.
  • 5 CONCLUSIONS: Empirical results show that CPM-Nets are relatively robust to heavy and complex view-missing data.
Loading 2011.06170v1…