Source-linked AI summary
Iterative Views Agreement: An Iterative Low-Rank based Structured Optimization Method to Multi-View Spectral Clustering
Yang Wang, Wenjie Zhang, Lin Wu, Xuemin Lin, Meng Fang, Shirui Pan
TL;DR
Multi-view spectral clustering must integrate complementary views because any individual view may be insufficient, while existing LRR methods overlook distinct local manifolds and impose a rigid shared representation. The paper proposes iterative, graph-regularized, view-specific low-rank optimization with mutual structural consensus, achieving superior clustering performance on real-world multi-view datasets.
Problem
Existing LRR-based clustering overlooks distinct local manifold structures and imposes a common representation that may not flexibly preserve heterogeneous views.
Method
The method combines view-specific graph-Laplacian regularization and separate low-rank constraints with iterative mutual structural consensus across views.
Results
The method outperforms competing approaches across most ACC and NMI evaluations, including 86.39% ACC on UCI digits versus 43.81% for MFMSC and 51.74% for MAASC.
Takeaways & Limitations
Iterative agreement preserves view-dependent nonlinear manifold structure while producing more effective consensus clustering, including under noise corruptions.
Takeaways & Limitations
Adapting the iterative technique to cross-view data sources with shared latent semantics remains future work.
Abstract
from arXiv · showhide
Multi-view spectral clustering, which aims at yielding an agreement or consensus data objects grouping across multi-views with their graph laplacian matrices, is a fundamental clustering problem. Among the existing methods, Low-Rank Representation (LRR) based method is quite superior in terms of its effectiveness, intuitiveness and robustness to noise corruptions. However, it aggressively tries to learn a common low-dimensional subspace for multi-view data, while inattentively ignoring the local manifold structure in each view, which is critically important to the spectral clustering; worse still, the low-rank minimization is enforced to achieve the data correlation consensus among all views, failing to flexibly preserve the local manifold structure for each view. In this paper, 1) we propose a multi-graph laplacian regularized LRR with each graph laplacian corresponding to one view to characterize its local manifold structure. 2) Instead of directly enforcing the low-rank minimization among all views for correlation consensus, we separately impose low-rank constraint on each view, coupled with a mutual structural consensus constraint, where it is able to not only well preserve the local manifold structure but also serve as a constraint for that from other views, which iteratively makes the views more agreeable. Extensive experiments on real-world multi-view data sets demonstrate its superiority.
1 Introduction
Multi-view spectral clustering integrates complementary feature views to seek consensus clusters, but existing LRR approaches can miss view-specific manifold structure by enforcing one shared representation. The paper addresses this with graph-regularized, view-specific low-rank representations coupled through iterative agreement.
- Multi-view spectral clustering combines complementary representations because any individual view may be insufficient for effective learning.
- LRR-based methods model each view as Xi = XiZ + Ei, using shared sample correlations Z and sparse errors Ei for noise robustness.Z represents self-expressive linear correlations shared across views, while Ei models view-specific noise.
- LRR can overlook distinct manifold structures in each view and impose a common representation that cannot flexibly preserve heterogeneous local structure.
- The method couples LRR with a graph Laplacian for each view to characterize view-dependent nonlinear local similarity.
- It separately learns view representations and iteratively uses each representation to regulate the others, progressively increasing their agreement.
- The proposed objective is solved with Linearized Alternating Direction Method with Adaptive Penalty (LADMAP).
2 Iterative Low-Rank based Structured Optimization Method to Multi-view Spectral Clustering
The method preserves each view’s local manifold through graph-Laplacian and sparse-neighborhood regularization while learning low-rank representations and enforcing cross-view agreement. An augmented-Lagrangian optimization alternately updates view-specific variables and handles noise and structural constraints.
- For each view, the method preserves the original data’s nonlinear manifold within the space spanned by its low-rank sparse representation Zi.
- The similarity matrix Wi uses a Gaussian kernel, and the graph Laplacian is Li = Di − Wi.Wi encodes pairwise similarity in the original feature space; s-nearest-neighbor constraints determine when similarities are retained.
- The objective combines low-rank representation, sparse errors, graph regularization, and agreement among representations from all views.The constraints include Xi = XiZi + Ei and nonnegative Zi, which encodes data correlations through neighboring objects.
- The graph regularizer Tr(Zi^T LiZi) characterizes local manifold structure, while ||Zi||1 extracts each object’s local sparse neighborhood.
- LADMAP solves the low-rank optimization by alternately updating variables through an augmented-Lagrangian formulation.An auxiliary Gi is introduced, and the variables Zi, Ei, and Gi are updated with other variables fixed.
3 Optimization Strategy
The optimization alternates updates for the model variables, using linearized smooth terms, proximal regularization, and singular value thresholding for low-rank representations. Each view’s representation is influenced by the others, and the converged representations are combined for spectral clustering.
- Updating Zi: The Zi subproblem is linearized with a proximal term because the original smooth objective has no closed-form solution.The replacement is valid when ξ exceeds a bound involving the graph Laplacian’s largest eigenvalue and the data matrix norm.
- Updating Zi: Singular value thresholding solves the resulting low-rank update through singular value decomposition and soft thresholding.The threshold operator shrinks singular values and sets sufficiently small values to zero.
- Updating Zi: Each Zi update is explicitly influenced by the other views’ representations, which alternately constrain one another toward agreement.The optimization alternatively updates variables while fixing the remaining variables until convergence.
- Clustering with Zi: After convergence, normalized view-specific representations are symmetrized into similarity matrices, aggregated across views, and used for spectral clustering.Small entries below threshold τ are set to zero before constructing the final similarity matrix.
4 Experiments
Experiments evaluate the method on benchmark multi-view data sets using ACC and NMI, parameter studies, and comparisons with established baselines. The results show gains over fusion, common-subspace, co-regularized, and robust low-rank approaches, including under feature noise.
- Experimental settings: Experiments use UCI digits, AwA, and NUS benchmark data sets, with ACC and NMI reported in Tables 2 and 3.Competitors follow their published experimental settings; results are averaged over 10 runs.
- Parameter study: Increasing either β or λ3 generally improves ACC and NMI until an optimal parameter pair, after which both measures slightly decline.The study varies λ3 and β over [0.001,10] on AwA and transfers the setting to other data sets.
- Comparisons: The method outperforms RLRR on nearly most ACC and NMI comparisons, supporting the added multi-graph regularization and iterative views agreement scheme.RLRR’s common low-rank representation is described as less flexible for preserving heterogeneous local manifold structures.
- Comparisons: 86.39% ACC on UCI digits versus 43.81% for MFMSC and 51.74% for MAASC demonstrates the method’s margin over view-fusion baselines.The paper reports similar improvements on AwA and NUS.
- Comparisons: The method exceeds CCAMSC, which uses a common low-dimensional subspace, by encoding local graph structures through iterative views agreement.The improvement is reported for both ACC and NMI.
- Robustness: 41.02% accuracy on noisy NUS data exceeds 33.63% for CoMVSC and 34.25% for Co-training.Noise experiments corrupt 20% of feature elements in each view with a uniform distribution over [5,-5].
5 Conclusions
The paper proposes an iterative structured low-rank optimization method that preserves view-specific manifolds while achieving multi-view agreement. Extensive real-world experiments validate its reported advantages, and future work targets cross-view data sources sharing latent semantics.
- Conclusions: The proposed method combines view-specific local manifold encoding with iterative multi-view agreement in low-rank optimization.Its superiorities are validated on real-world multi-view data sets.
- Future work: A future direction is adapting the iterative technique to cross-view data sources that correspond to the same latent semantics.The authors aim to learn projections into a common latent space for those sources.