PaperlayerView exact revision on arXiv

Source-linked AI summary

Kernelized Multiview Subspace Analysis by Self-weighted Learning

Huibing Wang, Yang Wang, Zhao Zhang, Xianping Fu, Zhuo Li, Mingliang Xu, Meng Wang

arXiv:1911.10357v3cs.LGcs.MM

TL;DR

Many multiview dimension-reduction methods focus on graph correlations and overlook the information in data representations, while a general framework remains lacking. KMSA performs multiview subspace learning in kernel space, learns view weights, and regularizes views toward consensus; experiments across six datasets report superiority over other multiview methods.

  • Problem

    Many multiview dimension-reduction methods focus on graph correlations and overlook data representations, and existing approaches do not provide a general framework for many DR methods.

    Source: §I · p. 2

  • Method

    KMSA extends dimension reduction to multiview kernel space and combines contribution-based view weighting with co-regularization to minimize divergence between views.

    Source: §I · p. 2

  • Results

    Experiments on six datasets for multiview classification and image retrieval found KMSA superior to other multiview-based methods.

    Source: §IV · p. 11

  • Takeaways & Limitations

    KMSA is presented as a generalized multiview graph-embedding framework that exploits data representations while learning view weights and consensus.

    Source: §IV · p. 11

Abstract

from arXiv · show

With the popularity of multimedia technology, information is always represented or transmitted from multiple views. Most of the existing algorithms are graph-based ones to learn the complex structures within multiview data but overlooked the information within data representations. Furthermore, many existing works treat multiple views discriminatively by introducing some hyperparameters, which is undesirable in practice. To this end, abundant multiview based methods have been proposed for dimension reduction. However, there are still no research to leverage the existing work into a unified framework. To address this issue, in this paper, we propose a general framework for multiview data dimension reduction, named Kernelized Multiview Subspace Analysis (KMSA). It directly handles the multi-view feature representation in the kernel space, which provides a feasible channel for direct manipulations on multiview data with different dimensions. Meanwhile, compared with those graph-based methods, KMSA can fully exploit information from multiview data with nothing to lose. Furthermore, since different views have different influences on KMSA, we propose a self-weighted strategy to treat different views discriminatively according to their contributions. A co-regularized term is proposed to promote the mutual learning from multi-views. KMSA combines self-weighted learning with the co-regularized term to learn appropriate weights for all views. We also discuss the influence of the parameters in KMSA regarding the weights of multi-views. We evaluate our proposed framework on 6 multiview datasets for classification and image retrieval. The experimental results validate the advantages of our proposed method.

I. INTRODUCTION

KMSA addresses the lack of a general multiview dimension-reduction framework that uses data representations, not only graphs. It combines kernel-space subspace learning with contribution-based view weights and co-regularization so views can learn from one another.

I. INTRODUCTION

  • Most multiview dimension-reduction methods rely on graphs and overlook data representations, while existing methods that use original data do not provide a general framework for many DR approaches.The stated goal is to construct low-dimensional representations from multiple views while achieving ideal performance.Source: §I · p. 2
  • KMSA extends dimension reduction to multiview kernel space, accommodating views with different feature dimensions while weighting views by their contributions and minimizing divergence between them.The framework combines self-weighted learning with a co-regularized term to promote multiview consensus.Source: §I · p. 2
  • The authors state that most eigen-decomposition-based DR methods can be extended to multiview versions through KMSA.Source: §I · p. 3
  • The introduction reports that KMSA achieves state-of-the-art performance.Source: §I · p. 3

II. KERNEL-BASED MULTIVIEW EMBEDDING WITH SELF-WEIGHTED LEARNING

  • KMSA assigns view weights according to their contributions because views can affect the objective differently, and some may contribute negatively to the final representations.The weight regularizer encourages all views to participate; larger r makes weights more similar, while views with larger trace values receive smaller weights.Source: 2) Self-weighted Learning of the… · p. 4
  • Because separately learned representations do not learn from other views, KMSA adds a co-regularized term that minimizes divergence between pairs of view coefficient matrices.The coefficient matrices encode samples, and minimizing their divergence encourages views to learn from one another and bridge the gap between them.Source: 2) Self-weighted Learning of the… · p. 4, 3) Minimize the Divergence between… · p. 4
  • KMSA weights pairwise co-regularization using the learned importance of both views; it uses one parameter η to control the term’s influence rather than m parameters.The authors state that a larger |η| reduces the co-regularized term’s influence, and that the weights reflect both each view’s graph and cross-view correlations.Source: 4) Overall Objective Function · p. 5

A. Optimization Process for KMSA

KMSA uses alternating optimization, updating each view’s subspace while holding other variables fixed, then updating the view weights. The subspaces are obtained by generalized eigen-decomposition, and the parameter r shapes how concentrated the learned weights are.

A. Optimization Process for KMSA

  • The algorithm alternates between updating each U_v and updating α, then calculates the low-dimensional representations.It initializes U_v and α_v = 1/m and repeats the updates in Algorithm 1.Source: A. Optimization Process for KMSA · p. 5
  • Generalized eigen-decomposition updates each view’s subspace using the eigenvectors for the smallest d eigenvalues.The update fixes all variables but U_v and applies the constraint (U_v)^T M_v U_v = I.Source: A. Optimization Process for KMSA · p. 5
  • The objective values decrease with iterations and tend to stabilize after approximately 10–12 iterations.The figure tracks objective-function values against iteration count.Source: Figure 3 · p. 6
  • As r approaches 1, only the view with the smallest trace has a nonzero weight; as r tends to infinity, weights approach 1/m.The paper states that r > 1 directly influences α_v.Source: A. Optimization Process for KMSA · p. 6

B. Convergence of KMSA

The paper establishes convergence by showing that the objective is bounded and decreases monotonically during optimization. In experiments, objective values stabilize after roughly 10–12 iterations.

B. Convergence of KMSA

  • The objective in Eq. 8 is bounded, and the proposed optimization algorithm monotonically decreases its value at each step.The paper concludes convergence from the bounded monotone convergence theorem.Source: B. Convergence of KMSA · pp. 6–7
  • The α update uses gradient descent to make G smaller.This statement is part of the paper’s explanation of the optimization sequence.Source: B. Convergence of KMSA · p. 6
  • Objective values for KMSA-LDA and KMSA-PCA decrease and stabilize after 10–12 iterations on Corel1K, Caltech101, and ORL.The paper presents these iteration curves in Fig. 3 as empirical support for convergence.Source: B. Convergence of KMSA · p. 7

C. Extension of Various DR Algorithms by KMSA

KMSA extends several dimensional-reduction algorithms to multiview data by specifying how to set their matrices S_v and B_v.

C. Extension of Various DR Algorithms by KMSA

III. EXPERIMENTS

The experiments evaluate KMSA on image retrieval and multiview classification using six datasets. The authors state that the experiments reveal strong performance, but the supplied passages give no comparative results.

III. EXPERIMENTS

  • The evaluation covers image retrieval on Corel1K, Corel5K, and Holidays, and classification on Caltech101, ORL, and 3Sources.Together, these are the six datasets named for the experiments.Source: §III · p. 7
  • The authors report that the experiments reveal excellent performance of the proposed methods.The passage provides no metric or specific comparison supporting this general statement.Source: §III · p. 7
  • The experiments are organized around image retrieval and multiview data classification.The paper introduces datasets and comparison methods before presenting the experiments.Source: §III · p. 7

A. Datasets and Comparison Methods

The experiments cover image retrieval and multiview classification, using several datasets, feature views, and comparison methods to evaluate KMSA.

A. Datasets and Comparison Methods

  • Image retrieval uses Corel1K, Corel5K, and Holidays; classification uses Caltech101, ORL, and 3Sources.The experiments compare multiview subspace-learning methods across these tasks.Source: A. Datasets and Comparison Methods · p. 7
  • On Corel1K, KMSA-PCA outperforms other unsupervised multiview methods, while KMSA-LDA is best in most situations and MvDA does better for few retrieved images.The figure reports average retrieval results over 20 repetitions.Source: Figure 5 · p. 8
  • On Corel5K, KMSA achieves the best retrieval performance in most situations; PCAFC performs worst, while CCA and Co-Regu also perform well.The figure reports average retrieval results over 20 repetitions.Source: Figure 6 · p. 8
  • Comparisons include MDcR, MSE, PCAFC, GMA, CCA, and MvDA; GMA represents a multiview extension of PCA, while PCAFC applies PCA after concatenating views.KMSA uses a Gaussian kernel with κ = 0.1, η = −1, and r = 3.Source: A. Datasets and Comparison Methods · p. 8
  • The datasets use image descriptors such as MSD, Gist, and HOG, face features such as grayscale intensity, LBP, and EDH, and three news sources as 3Sources views.BBC, Reuters, and the Guardian are the three 3Sources views.Source: Table III · p. 8

B. Image Retrieval

Across image-retrieval experiments, KMSA-PCA and KMSA-LDA frequently rank among the strongest methods, with outcomes varying by dataset and retrieval setting.

B. Image Retrieval

  • On Corel1K, KMSA-PCA outperforms other unsupervised multiview methods, and KMSA-LDA outperforms MvDA.PCAFC performs poorly because it is essentially a single-view method, despite concatenating the views.Source: B. Image Retrieval · p. 9
  • On Corel5K, KMSA-LDA outperforms the other methods in most situations, and KMSA-PCA is a stronger unsupervised method.The authors attribute KMSA-LDA's advantage over KMSA-PCA to its use of label information.Source: B. Image Retrieval · p. 9
  • For Holidays retrieval, each class contributes one query image and two gallery images, and all methods use 50-dimensional representations.The experiment averages results over 20 repetitions.Source: B. Image Retrieval · p. 9
  • KMSA-PCA and KMSA-LDA achieve the best results in most situations on Holidays, while MvDA and Co-Regu also perform well.PCAFC performs worst; the table reports average precision, recall, mAP, and F1 over 20 repetitions.Source: B. Image Retrieval · p. 9, Table IV · p. 9

C. Classification of Multiview Data

Classification experiments evaluate KMSA on Caltech101, ORL, and 3Sources, with results generally favoring KMSA variants across the reported settings.

C. Classification of Multiview Data

  • Caltech101 and ORL experiments use 30% or 50% of samples for training, vary representation dimensions, and evaluate with 1NN.Caltech101 uses MSD, Gist, and HOG views; ORL uses grayscale intensity, LBP, and EDH.Source: C. Classification of Multiview Data · pp. 9–10
  • On Caltech101 and ORL, KMSA-LDA is better than MvDA, while KMSA-PCA is the strongest unsupervised multiview method; all methods improve as dimensions increase.KMSA's weights are learned according to view importance, and its co-regularized term encourages views to learn from each other.Source: C. Classification of Multiview Data · p. 10
  • For 3Sources, 30% or 50% of samples are randomly selected for training; the benchmark has 3 views, and methods construct 30-dimensional representations for 1NN classification of testing samples.Source: C. Classification of Multiview Data · p. 10
  • On 3Sources, KMSA-LDA is better than MvDA and KMSA-PCA outperforms other unsupervised methods in most situations.The experiment uses 30-dimensional representations and 1NN classification.Source: C. Classification of Multiview Data · p. 10, Figure 8 · p. 10

D. Discussions on KMSA

The discussion tests the roles of learned view weights and kernelization, and visualizes how KMSA transforms sample distributions.

D. Discussions on KMSA

  • Learned self-weights improve KMSA over fixing every view weight to 1/m, and KMSA-PCA exceeds KCCA and KPCA.The comparison uses the Holidays dataset; KCCA and KPCA are kernelized with Gaussian kernels.Source: D. Discussions on KMSA · p. 10, Table VI · p. 11
  • The t-SNE visualization shows disordered original Caltech101 features, while KMSA-PCA and KMSA-LDA produce better visualized representations.KMSA-LDA's 30-dimensional representation separates samples from the first view into many clusters.Source: Figure 9 · p. 11

E. Visualization of KMSA

The visualization examines first-view features from Caltech101 after multiview projection, showing improved distributions with KMSA-PCA and KMSA-LDA, especially class clustering with KMSA-LDA.

E. Visualization of KMSA

  • KMSA-PCA and KMSA-LDA project the original first-view features into 30 dimensions using information from all views before visualization.The authors then use t-SNE to embed the learned features into two dimensions on Caltech101.Source: E. Visualization of KMSA · p. 11
  • The original data distributions are disordered for most samples, while KMSA-PCA and KMSA-LDA achieve better performances.Source: E. Visualization of KMSA · p. 11
  • After KMSA-LDA, first-view samples separate into many clusters, which the authors say is helpful for many applications.Source: E. Visualization of KMSA · p. 11

IV. CONCLUSIONS

The conclusion presents KMSA as a kernel-space framework for multiview graph embedding that combines co-regularization and self-weighting. Experiments on six datasets support its superiority to other multiview methods for classification and image retrieval.

IV. CONCLUSIONS

  • KMSA handles multiview data in kernel space to exploit data representations across views.Source: §IV · p. 11
  • KMSA combines a co-regularized term to minimize divergence among views with self-weighting to learn view weights.Source: §IV · p. 11
  • Experiments on six datasets for classification and image retrieval found KMSA superior to other multiview-based methods.Source: §IV · p. 11
Loading 1911.10357v3…