PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Multi-View Clustering

Guoqing Chao, Shiliang Sun, Jinbo Bi

arXiv:1712.06246v2cs.LGstat.ML

TL;DR

Multi-view clustering needs methods that integrate multiple feature views while finding consistent groups, but its rapidly developing literature lacked a consolidating survey. This paper reviews view-combination strategies, proposes a taxonomy, relates MVC to neighboring paradigms, surveys applications, and identifies open problems. It distinguishes generative and discriminative methods and highlights convex multi-view mixture formulations, consensus-oriented spectral methods, and unresolved issues including incomplete views, local minima, and mixed data types.

  • Problem

    Rapidly developed multi-view clustering lacked a survey that summarized and analyzed its progress, strategies, applications, relationships, and open problems.

  • Method

    The paper reviews multi-view combination strategies, proposes a taxonomy separating generative and discriminative methods, and discusses related learning paradigms and applications.

  • Results

    The survey organizes discriminative methods into five view-integration classes and reports that generative methods have developed less sufficiently than discriminative methods.

  • Takeaways & Limitations

    The paper identifies incomplete views, NMF local minima, and mixed data types as challenging directions requiring further investigation.

Abstract

from arXiv · show

With advances in information acquisition technologies, multi-view data become ubiquitous. Multi-view learning has thus become more and more popular in machine learning and data mining fields. Multi-view unsupervised or semi-supervised learning, such as co-training, co-regularization has gained considerable attention. Although recently, multi-view clustering (MVC) methods have been developed rapidly, there has not been a survey to summarize and analyze the current progress. Therefore, this paper reviews the common strategies for combining multiple views of data and based on this summary we propose a novel taxonomy of the MVC approaches. We further discuss the relationships between MVC and multi-view representation, ensemble clustering, multi-task clustering, multi-view supervised and semi-supervised learning. Several representative real-world applications are elaborated. To promote future development of MVC, we envision several open problems that may require further investigation and thorough examination.

I. INTRODUCTION

Multi-view clustering integrates multiple feature sets to identify consistent groupings, addressing limitations of single-view methods. This survey defines MVC, organizes its approaches, and situates them among related learning paradigms.

  • Single-view clustering can fail when views contain complementary or differing cluster structures, because feature concatenation may overweight one view.
  • Multi-view clustering combines available feature information to place similar subjects together, separate dissimilar subjects, and seek consistent clusterings across views.
  • The survey divides MVC into generative and discriminative approaches, further organizing discriminative methods by how they combine multi-view information.
  • MVC performs sample partitioning without labels, unlike multi-view representation learning and supervised or semi-supervised multi-view learning.
  • MVC has applications across computer vision, natural language processing, social multimedia, bioinformatics, and health informatics.
  • The paper reviews generative methods based on mixture models and expectation maximization, including convex mixture models extended to multiple views.

2) Multi-View Clustering Based on Mixture Models or EM Algorithm:

Multi-view mixture-model methods extend mixture and convex mixture models by coordinating view-specific distributions toward a common clustering. The reviewed multi-view CMM formulation is convex, while a weighted variant addresses unequal view importance.

  • Multi-view EM methods execute view-specific E and M steps and exchange posterior information between views during optimization.
  • The multi-view CMM assigns each view its own mixture distribution while sharing component priors to pursue a common clustering across views.
  • The algorithm minimizes the sum of KL divergences between each view’s empirical distribution and its mixture distribution.
  • The optimized multi-view CMM objective is convex, so a global minimum can be found.
  • The prior q_j measures how likely instance j is to be an exemplar when all views are considered.
  • Because the basic objective weights all views equally, a weighted multi-view CMM variant was proposed to account for differing view importance.

III. DISCRIMINATIVE APPROACHES

Discriminative MVC methods directly optimize clustering objectives rather than first modeling the data, and the survey organizes them by how multiple views are combined. Spectral clustering constructs graph-based embeddings and clusters their rows.

  • The survey categorizes discriminative MVC methods into five classes according to their multi-view combination strategy.
  • Spectral clustering: Spectral MVC methods seek common clustering results by assuming views share the same or similar eigenvector matrix.
  • Spectral clustering: Spectral clustering represents similarities with graph edges and can handle arbitrarily shaped clusters through graph-Laplacian optimization.
  • Spectral clustering: The standard spectral pipeline constructs W, computes a normalized Laplacian, extracts the top K eigenvectors into U, normalizes rows, and applies k-means.

2) Co-Training Multi-View Spectral Clustering:

Co-training and co-regularization encourage agreement among multi-view spectral clusterings by exchanging graph information or aligning view-specific eigenvector embeddings. Their variants differ in whether agreement is enforced pairwise or through a consensus embedding.

  • Co-Training: Co-training multi-view spectral clustering uses one view’s graph-Laplacian eigenvectors to cluster samples and modify the other view’s graph Laplacian.
  • Co-Training: Similarity vectors are projected onto the largest K eigenvector directions to retain discriminative clustering information and discard confusing within-cluster details.
  • Co-Training: The co-training algorithm iteratively updates each view’s graph similarities and Laplacians, recomputes K eigenvectors, and assigns clusters using normalized embeddings.
  • Co-Regularization: Pairwise co-regularization measures disagreement between every pair of view embeddings and balances spectral objectives against disagreement with λ.
  • Co-Regularization: Centroid-based co-regularization regularizes each view’s eigenvectors toward a common consensus eigenvector matrix.
  • Co-Regularization: A potential drawback is that noisy views can affect the optimal eigenvectors in centroid-based multi-view clustering because all views contribute to them.
  • Co-Regularization: Directly solving a nonnegative common cluster-indicator matrix is described as more robust to k-means initialization than first clustering eigenvectors.

4) Others:

The survey connects MVC with subspace-based methods that exploit self-expressiveness and transform learned representations into graphs for spectral clustering. Multi-view variants seek consistency across view-specific coefficient matrices.

  • Subspace clustering: Subspace clustering targets low-dimensional structure when data are sampled from multiple subspaces.
  • Subspace clustering: Subspace clustering uses self-expressiveness, representing each sample as a linear combination of a few other samples.
  • Subspace clustering: The representation matrix Z contains coefficients expressing each data point from other points, while E represents noise.
  • Subspace clustering: The constraints Z(i, i) = 0 and ZT1 = 1 prevent self-representation and encode membership in a union of affine subspaces.
  • Subspace clustering: After learning Z, the method forms W = |Z| + |ZT|, constructs a graph Laplacian, and applies spectral clustering.
  • Multi-view integration: Multi-view subspace clustering obtains a representation from each view and encourages pairwise coefficient matrices to be similar for consistent clustering.

2) Multi-View Subspace Clustering:

Multi-view subspace clustering combines view-specific self-expressive representations while controlling sparsity, noise, and consistency. Related NMF methods instead combine nonnegative factors through shared or consensus indicator matrices.

  • Multi-view subspace clustering: Pairwise l1 co-regularization encourages coefficient matrices from different views to agree while alleviating noise.
  • Multi-view subspace clustering: The l1 norm promotes sparse solutions, and zero diagonal constraints prevent trivial self-representation.
  • Multi-view subspace clustering: Related multi-view methods combine shared coefficient matrices with multi-graph regularization, low-rank and sparse representations, and sparse-noise modeling.
  • Multi-view NMF: NMF approximates data using nonnegative factors U and V, with K set to the desired reduced dimension or number of clusters.
  • Multi-view NMF: Nonnegative constraints support part-based representations, and NMF has been applied successfully to cluster analysis.
  • Multi-view NMF: Multi-view NMF methods enforce either a common indicator matrix or view-specific indicators pulled toward a consensus matrix.
  • Multi-view NMF: Normalizing each view-dependent indicator matrix keeps it on a comparable scale, after which cluster labels are obtained by argmax over the consensus matrix.

3) Multi-View K-Means:

Multi-view k-means methods use a common cluster-indicator matrix across views, sometimes jointly learning view weights, feature selections, or consensus structures to handle heterogeneous multi-view clustering concerns.

  • Multi-View K-Means: The k-means formulation can be expressed through NMF using an indicator matrix H, whose centroid matrix G contains cluster centroids.The approach is motivated partly by k-means avoiding the expensive eigen-decomposition required by some alternatives for large-scale clustering.
  • Multi-View K-Means: Multi-view k-means adopts a common indicator matrix across views and learns view weights so important views receive larger weights.The weight-distribution parameter controls how weights are assigned across views.
  • Multi-View K-Means: Multi-view subspace clustering merges subspace representation and spectral clustering while enforcing a common indicator matrix across views.Unlike methods enforcing a common coefficient matrix, this approach uses a common indicator matrix for different views.
  • Multi-View K-Means: Feature-selection formulations combine a common cluster-indicator matrix with view-level and individual-feature regularization across clusters.Group l1 regularization evaluates whole-view feature importance, while the l2,1 norm selects individual features important across clusters.
  • Multi-View K-Means: Matrix factorization can reconcile individual-view partitions by decomposing them into meta-cluster contributions and instance assignments.The resulting factors represent how individual groupings contribute to the final multi-view clustering and how instances assign to meta-clusters.
  • Multi-View K-Means: Additional common-indicator approaches introduce missing-entry weights, self-paced handling of sample and view complexity, or ensemble-based consensus clustering.These variants target missing values, local minima, or consensus formation across views.

D. Direct Combination (Mainly Multi-Kernel Based MultiView Clustering)

Direct combination methods construct a kernel for each view and combine kernels for clustering, with linear, nonlinear, and data-dependent weighting strategies.

  • Direct Combination: A common direct-combination strategy defines one kernel per view and combines them through a convex combination.This approach treats kernels as view-specific similarity representations for clustering.
  • Direct Combination: Kernel functions provide inner products in an implicit feature space, enabling nonlinear relationships to be handled by linear learning algorithms.In RKHS settings, kernels can also serve as non-Euclidean similarity measures for spectral clustering and kernel k-means.
  • Direct Combination: Multi-kernel multi-view clustering combines kernels derived from separate views, extending multi-kernel learning from single-view clustering.The survey identifies multi-kernel learning as a central component of this class of MVC methods.
  • Direct Combination: Linear kernel combinations include unweighted sums and weighted sums with nonnegative view weights summing to one.A hyperparameter controls the distribution of the kernel weights.
  • Direct Combination: Nonlinear combinations use operations such as multiplication, power, and exponentiation, whereas data-dependent combinations assign weights per instance.Instance-specific weights can model local data distributions and learn combination rules for different regions.
  • Direct Combination: Kernel k-means and spectral clustering are kernel-based methods used to optimize intra-cluster variance.Both provide clustering foundations for combining view-specific kernels.

2) Kernel K-Means and Spectral Clustering:

Kernel k-means and spectral clustering extend naturally to multiple views by combining view-specific kernel matrices, while relaxing binary assignments enables an eigenvector-based formulation.

  • Kernel K-Means and Spectral Clustering: Kernel k-means represents clusters with a binary indicator matrix H and uses RKHS feature mappings to formulate intra-cluster variance minimization.The matrix H is also called the cluster assignment matrix.
  • Kernel K-Means and Spectral Clustering: The kernel k-means objective can be rewritten in matrix-vector form using the kernel matrix, cluster-size normalization, and the indicator representation.This reformulation supports the subsequent relaxation of the binary assignment constraint.
  • Kernel K-Means and Spectral Clustering: Relaxing H from binary to arbitrary real values makes the optimization easier and yields a formulation involving U = HL.The relaxation also treats Tr(K) as constant.
  • Kernel K-Means and Spectral Clustering: The constraint H^T H = L^-1 induces orthogonality on U, so the optimum is obtained from the top K eigenvectors of the kernel matrix.This relaxed formulation is characterized as a generalized spectral-clustering optimization.
  • Kernel K-Means and Spectral Clustering: The relaxed kernel k-means formulation is equivalent to the spectral-clustering formulation when the kernel matrix is a normalized Gram matrix.The equivalence depends on the specified normalized Gram-matrix form.
  • Kernel K-Means and Spectral Clustering: For m view-specific kernels, weighted kernel combinations with nonnegative weights summing to one support multi-view kernel k-means and multi-view spectral clustering.The exponent p controls the combination form, with p ≥ 1.
  • Kernel K-Means and Spectral Clustering: Direct feature combination and two-level weighting methods also assign view and variable weights, while fuzzy clustering has been extended to multi-view data.Direct combination is more suitable when views share data types; heterogeneous views such as genetic information and clinical symptoms are harder to combine directly.

1) CCA and KCCA:

CCA seeks projections of paired views that maximize correlation, while KCCA applies the kernel trick to maximize correlation between nonlinear projected variables and solves an eigen-problem.

  • CCA and KCCA: CCA finds projection vectors for two zero-mean datasets that maximize the correlation between their projected representations.The two datasets may have different feature dimensions, represented by Sx and Sy.
  • CCA and KCCA: CCA’s correlation objective is invariant to independently scaling the projection vectors, enabling an equivalent constrained formulation.The correlation uses the cross-covariance matrix Cxy.
  • CCA and KCCA: Lagrange multipliers reduce the CCA problem to an eigen-problem; when Cyy is invertible, wy is expressed from wx and covariance matrices.Eigenvectors are obtained successively for eigenvalues ordered from large to small.
  • CCA and KCCA: The canonical-correlation problem can be reformulated as distance minimization and generalized to simultaneously identify p projection pairs.Wx and Wy collect the projection vectors for the two datasets.
  • CCA and KCCA: The matrix formulation of the canonical-correlation optimization uses an identity matrix and orthogonality constraints among projection components.The formulation is presented as a matrix optimization and an equivalent transformed objective.
  • CCA and KCCA: KCCA uses the kernel trick to maximize correlation between two nonlinear projected variables.It extends the correlation-maximization objective beyond linear projections.
  • CCA and KCCA: Unlike linear CCA, which solves a covariance-matrix eigen-decomposition, KCCA solves an eigen-problem involving kernel matrices.The displayed KCCA formulation uses the paired kernel matrices Kx and Ky.

2) CCA Based Multi-View Clustering:

The survey organizes multi-view clustering methods by how they combine information across views, including CCA-based approaches and shared structures. It also compares MVC with related multi-view and clustering paradigms, while noting key trade-offs among method families.

  • CCA Based Multi-View Clustering: CCA-based methods project multiple views into correlated lower-dimensional spaces before clustering or learn cross-view projections using paired data.CCA can require weaker separation conditions under conditional view uncorrelatedness; variants use class or cluster correspondences when labels are available.
  • CCA Based Multi-View Clustering: Incomplete-view methods complete kernels, learn consensus pattern matrices, or connect partially observed views through a common indicator matrix.These strategies address settings with at least one complete view as well as settings where no view is complete.
  • Taxonomy and Shared Structures: MVC methods commonly combine views by sharing a similar structure, such as similarity, eigenvector, coefficient, or indicator matrices.The taxonomy also includes shared indicator vectors, cross-view clustering consistency, diversity terms, and disagreement minimization.
  • Taxonomy and Shared Structures: Generative MVC handles missing values naturally and can obtain global solutions in some convex models, but assumptions may be false and optimization can be time consuming.Discriminative methods exploit consensus information, support arbitrary-shaped clusters through common eigenvectors, and include subspace and k-means or nonnegative-factorization approaches.
  • Relationships to Related Topics: Sequentially learning a multi-view representation and then clustering can perform poorly because the representation step is unaware of the final clustering objective.The survey identifies simultaneous integration of clustering and multi-view representation learning as an open direction, especially for deep methods.
  • Relationships to Related Topics: MVC overlaps with ensemble and multi-task clustering when multiple view-specific clusterings are fused into one consensus result.It differs from multi-view supervised and semi-supervised learning because those settings learn classifiers from labeled or partly labeled data.

V. APPLICATIONS

MVC is applied across computer vision, language, social multimedia, and health informatics. These applications combine complementary views such as image descriptors, languages, event media, and clinical measurements for clustering tasks.

  • Computer Vision: Computer-vision applications use multiple image feature types, including CENTRIST, ColorMoment, HOG, LBP, and SIFT, for categorization and motion segmentation.The survey describes sparse subspace, consensus, ensemble, and nonnegative-factorization approaches for these tasks.
  • Natural Language Processing: Multilingual document categorization treats each language as a view and applies co-training or co-regularization ideas to multi-view clustering.The cited comparison on multilingual data reports superiority of these two approaches, without giving a quantitative value in the supplied passage.
  • Social Multimedia: Social-event detection combines pictures with textual features such as tags and titles, using multi-view spectral, density-based, or graph-based clustering methods.The examples include concerts and NBA games as representative social events.
  • Health Informatics: Health-informatics applications use multiple clinical views to refine diagnostic classification, analyze treatment outcomes, and combine information across measurements.The survey illustrates vital signs, urine drug screens, and craving measures as three views for heroin-dependence patients.

A. Large Scale Problem (size and dimension)

Multi-view clustering faces unresolved challenges involving scale, dimensionality, incomplete views, initialization, optimization, deep learning, mixed data types, and multiple valid clusterings. The survey organizes existing methods and identifies these issues as future directions.

  • Large scale data: Existing multi-view clustering methods often handle only small datasets, while massive samples make graph construction and eigen-decomposition expensive.The survey highlights extending spectral clustering effectively to large-scale multi-view settings as an open problem.
  • High dimensionality: High-dimensional multi-view data with few samples create over-fitting challenges that current feature-selection, dimensionality-reduction, and deep-learning approaches do not convincingly resolve.The paper points to bioinformatics data combining millions of genetic features with clinical phenotypes as an example.
  • Incomplete views: Incomplete views are common, but existing methods rely on constraints such as a complete view or restricted missingness and cannot fully handle arbitrary missing values.Missingness may be random or non-random, and simple replacement or imputation does not account for these differences.
  • Optimization: K-means-based MVC remains sensitive to initialization, while many NMF-based methods are vulnerable to local optima, especially with missing values and outliers.The survey notes self-paced learning and convex clustering as possible ways to alleviate local-minimum problems.
  • Deep learning: Multi-view deep clustering remains underdeveloped, with few truly deep clustering works and an open need to extend single-view methods to multi-view settings.Existing deep approaches commonly learn a multi-view representation first and then apply a regular clustering method.
  • Data types and outputs: Mixed symbolic, ordinal, numerical, and categorical data require integration methods that preserve information lost by converting all features to categorical values.The paper also identifies discovering multiple reasonable clustering solutions as an unresolved direction for real-world applications.
Loading 1712.06246v2…