Source-linked AI summary
Doubly Aligned Incomplete Multi-view Clustering
Menglei Hu, Songcan Chen
TL;DR
Incomplete multi-view clustering must use views with missing instances despite prior methods commonly assuming complete views. DAIMC combines weighted semi-NMF with instance and basis-matrix alignment, and experiments on four real-world datasets demonstrate its effectiveness.
Problem
Incomplete multi-view clustering must integrate views with missing instances, a setting in which traditional methods cannot be directly used.
Method
DAIMC uses weighted semi-NMF with view-specific weights, instance alignment, and L2,1-Norm regularized regression to align basis matrices.
Results
Experiments on four real-world multi-view datasets demonstrate DAIMC's effectiveness and advantages.
Takeaways & Limitations
DAIMC is designed to extend incomplete multi-view clustering beyond two views while reducing the influence of missing instances on clustering performance.
Takeaways & Limitations
The method fixes the shared basis representation to a K-dimensional identity matrix, and the paper identifies large-scale data as future work.
Abstract
from arXiv · showhide
Nowadays, multi-view clustering has attracted more and more attention. To date, almost all the previous studies assume that views are complete. However, in reality, it is often the case that each view may contain some missing instances. Such incompleteness makes it impossible to directly use traditional multi-view clustering methods. In this paper, we propose a Doubly Aligned Incomplete Multi-view Clustering algorithm (DAIMC) based on weighted semi-nonnegative matrix factorization (semi-NMF). Specifically, on the one hand, DAIMC utilizes the given instance alignment information to learn a common latent feature matrix for all the views. On the other hand, DAIMC establishes a consensus basis matrix with the help of $L_{2,1}$-Norm regularized regression for reducing the influence of missing instances. Consequently, compared with existing methods, besides inheriting the strength of semi-NMF with ability to handle negative entries, DAIMC has two unique advantages: 1) solving the incomplete view problem by introducing a respective weight matrix for each view, making it able to easily adapt to the case with more than two views; 2) reducing the influence of view incompleteness on clustering by enforcing the basis matrices of individual views being aligned with the help of regression. Experiments on four real-world datasets demonstrate its advantages.
1 Introduction
Multi-view clustering exploits shared and complementary information across views, but incomplete views violate the completeness assumption and complicate clustering. DAIMC addresses this through instance alignment, view-specific weighting, and basis-matrix alignment.
- Multi-view data contain views that can share consistency and complementary information useful for clustering and other learning tasks.
- Incomplete views arise when instances are missing because of camera failures, occlusion, or unavailable document translations.
- Traditional instance-imputation methods fail when incompleteness removes whole rows or columns from the data.
- PVC and IMG use instance alignment to learn shared representations, but they are limited to two-view incomplete clustering.
- DAIMC combines semi-NMF with L2,1-Norm regularized regression to align instances and basis matrices while learning a common latent feature matrix.
- DAIMC assigns zero weights to missing instances and one weights to present instances, enabling extension beyond two incomplete views and reducing missing-view influence.
2 Related Work
This section introduces semi-NMF as a latent factor learning extension of NMF and formulates incomplete multi-view clustering through instance-presence indicators. It also describes iterative optimization and the goal of clustering all instances from incomplete views.
- Semi-NMF: Semi-NMF approximates an input data matrix X with the product of a basis matrix U and a nonnegative latent feature matrix V.U contains basis factors, while V contains latent features.
- Semi-NMF: The semi-NMF objective is biconvex, so iterative updates seek a locally optimal solution rather than guaranteeing the global minimum.The procedure alternately updates U while fixing V and updates V while fixing U.
- Semi-NMF: The V-update uses the positive component of X^T U within the iterative semi-NMF procedure.The supplied update fragment identifies the positive-part operation used in the V update.
- Incomplete multi-view clustering: For incomplete multi-view clustering, an indicator matrix M records whether each instance is present in each view.Each row of M represents instance presence for the corresponding view.
- Incomplete multi-view clustering: The task integrates incomplete views to cluster N instances into C clusters, whereas complete views produce an all-one presence matrix.An incomplete view has missing data columns, reflected by fewer than N present instances.
3 Proposed Approach
DAIMC jointly models incomplete multi-view data with weighted semi-NMF, a shared latent feature matrix, and regression-based alignment of view-specific basis matrices. Alternating optimization produces the factorization and clustering representation while handling missing instances through view-specific weights.
- Weighted Semi-NMF: Each view uses a diagonal weight matrix that assigns zero weight to missing instances and one weight to observed instances.Missing-instance reconstruction losses are therefore ignored during factorization.
- Weighted Semi-NMF: Different views retain distinct basis matrices but share a common latent feature matrix V, yielding a common representation for incomplete-view instances.This shared latent space incorporates cross-view consistency while preserving view-specific bases.
- Basis Alignment: Regression aligns individual basis matrices to a shared low-dimensional representation, while L2,1 regularization makes the regression coefficients row-sparse.The row sparsity performs feature selection during alignment; the shared representation is set to the identity matrix I_K.
- Optimization: An alternating procedure updates each U^(i), B^(i), and V, normalizes the factors, and applies K-means to V for the final clustering.The optimization alternates convex subproblems and can at least find a locally optimal solution.
4 Experiments and Analysis
Experiments on four real-world multi-view datasets evaluate DAIMC under incomplete-view settings using NMI and clustering precision. DAIMC generally outperforms competing methods, with particularly large gains on Digit and strong results across varying view counts and incomplete rates.
- Experimental setup: Experiments use four real-world multi-view datasets, reporting NMI and clustering precision under incomplete rates from 0 to 0.5.Complete datasets are made incomplete by randomly removing instances; 3Sources is naturally incomplete.
- Experimental results: DAIMC improves Wikipedia performance by about 8.65% in NMI across different incomplete rates.For incomplete rates from 0 to 0.1, it also outperforms all other methods by about 3.53% in clustering precision.
- Experimental results: 60.78% in NMI and 64.67% in clustering precision are the gains reported for DAIMC on Digit at incomplete rates 0.4 and 0.5, respectively.The authors associate these results with Digit containing five views and DAIMC exploiting information across them.
- Experimental results: DAIMC raises Flowers performance by about 10.29% in NMI and 20.37% in clustering precision.The authors note that basis-matrix alignment helps DAIMC perform well despite a negative-role view in the Flowers dataset.
- Experimental results: On 3Sources, DAIMC outperforms all other methods in both NMI and clustering precision across two-view combinations and the whole dataset.The comparison includes various subsets of the naturally incomplete dataset.
- Additional analysis: DAIMC converges after about 30 iterations on Digit with incomplete rate 0.4 and hyper-parameters α=1e1 and β=1e-1.A separate view-number study reports better results as the number of available views increases; a hyper-parameter study finds relatively good performance at α=1e1 and β={1e-1,1e0,1e1}.
5 Conclusion
The paper proposes DAIMC for incomplete multi-view clustering by using instance alignment and simultaneously aligning different basis matrices. Experiments on four real-world multi-view datasets demonstrate the method’s effectiveness, while future work targets large-scale data through online and incremental learning.
- Conclusion: DAIMC addresses incomplete multi-view clustering by using instance alignment and simultaneously enforcing alignment among different basis matrices.The conclusion identifies these two alignment mechanisms as the method’s central design.
- Conclusion: Experiments on four real-world multi-view datasets demonstrate the effectiveness of DAIMC.The conclusion reports this result without specifying an individual dataset or metric.
- Conclusion: Future work will consider large-scale data by introducing online learning and incremental learning strategies into the model.This is the stated future research direction.