Source-linked AI summary
Self-Supervised Convolutional Subspace Clustering Network
Junjian Zhang, Chun-Guang Li, Chong You, Xianbiao Qi, Honggang Zhang, Jun Guo, Zhouchen Lin
TL;DR
Subspace clustering is limited by visual data that do not fit raw linear-subspace models, while ConvNet feature learning typically lacks labels in this setting. S2ConvSCN jointly learns features, self-expression, and segmentation with dual spectral-clustering supervision, and experiments on benchmark datasets validate its effectiveness; prior related work can require usually unknown subspace dimensions.
Problem
Subspace clustering may not fit practical visual data, while ConvNet feature learning usually requires labeled images unavailable in subspace-clustering applications.
Method
S2ConvSCN jointly integrates convolutional feature extraction, self-expression-based affinity learning, and spectral clustering, using clustering outputs to supervise feature learning and self-expression.
Results
Experiments on benchmark datasets validate the effectiveness of S2ConvSCN and demonstrate superior performance.
Takeaways & Limitations
The framework jointly refines feature representation, self-expression, and data segmentation through feedback from spectral clustering.
Takeaways & Limitations
A related prior framework requires the dimension of each subspace, which is usually unknown.
Abstract
from arXiv · showhide
Subspace clustering methods based on data self-expression have become very popular for learning from data that lie in a union of low-dimensional linear subspaces. However, the applicability of subspace clustering has been limited because practical visual data in raw form do not necessarily lie in such linear subspaces. On the other hand, while Convolutional Neural Network (ConvNet) has been demonstrated to be a powerful tool for extracting discriminative features from visual data, training such a ConvNet usually requires a large amount of labeled data, which are unavailable in subspace clustering applications. To achieve simultaneous feature learning and subspace clustering, we propose an end-to-end trainable framework, called Self-Supervised Convolutional Subspace Clustering Network (S$^2$ConvSCN), that combines a ConvNet module (for feature learning), a self-expression module (for subspace clustering) and a spectral clustering module (for self-supervision) into a joint optimization framework. Particularly, we introduce a dual self-supervision that exploits the output of spectral clustering to supervise the training of the feature learning module (via a classification loss) and the self-expression module (via a spectral clustering loss). Our experiments on four benchmark datasets show the effectiveness of the dual self-supervision and demonstrate superior performance of our proposed approach.
1. Introduction
Subspace clustering segments data drawn from unions of low-dimensional subspaces, but raw visual data may violate this model and ConvNet training typically requires unavailable labels. S2ConvSCN addresses both issues through joint feature learning, self-expression, spectral clustering, and dual self-supervision.
- Motivation: Subspace clustering segments data whose columns come from a union of low-dimensional subspaces into their corresponding groups.Examples include face images, rigid-object trajectories, and handwritten digits.
- Motivation: Raw practical visual data can violate the linear-subspace model because pose, expression, and alignment variations alter face images.
- Motivation: ConvNets learn useful visual representations, but training them generally requires many labeled images that subspace-clustering applications often lack.
- Proposed framework: S2ConvSCN jointly learns feature representations, self-expression coefficients, and data segmentation in an end-to-end trainable framework.
- Dual self-supervision: Spectral-clustering outputs supervise self-expression through an alignment loss and feature learning through a classification loss.The two supervision paths use current clustering labels to refine the affinity representation and feature extractor.
- Results: Experiments on four benchmark datasets demonstrate superior performance for the proposed approach.
2. Related Work
Prior subspace-clustering methods learn affinities and then segment data, while feature-space approaches use latent or manually learned representations. S2ConvSCN integrates convolutional feature extraction with self-expression and spectral-clustering feedback to self-supervise both components.
- Subspace Clustering in Original Space: Spectral-clustering methods typically learn a data affinity matrix first and then apply spectral clustering to obtain the segmentation.
- Subspace Clustering in Original Space: Self-expression models represent each data point as a linear combination of other points, using regularization to obtain informative affinities.
- Subspace Clustering in Feature Space: Feature-space methods use latent spaces from kernels or matrix decomposition, or explicit spaces from handcrafted or learned features.
- Subspace Clustering in Feature Space: Some deep feature-space methods combine autoencoders or convolutional networks with self-expression models for subspace clustering.
- Limitations of prior work: A prior subspace-specific generator and discriminator requires the dimension of each subspace, which is usually unknown.
- Proposed framework: S2ConvSCN combines convolutional feature extraction and self-expression-based affinity learning, using spectral-clustering feedback to supervise both.
3. Our Proposal: Self-Supervised Convolutional Subspace Clustering Network
S2ConvSCN jointly learns convolutional features, self-expression coefficients, and data segmentation, using spectral-clustering outputs to supervise both feature learning and self-expression. Its formulation combines reconstruction, self-expression, spectral-clustering, and classification-related objectives, with pseudo-label alignment and staged training.
- Network Formulation: S2ConvSCN combines feature extraction, self-expression, and self-supervision modules in an end-to-end network.The framework is trained by jointly optimizing the corresponding costs.
- Feature Extraction Module: Convolutional feature maps are vectorized and concatenated into representations used by the subsequent modules.The top-layer feature maps are transformed into a representation vector, while a decoder reconstructs the input image.
- Self-Expression Module: The self-expression module learns coefficients that reconstruct feature representations while regularization and optional zero diagonal constraints prevent trivial solutions.Its loss includes reconstruction and regularization terms, with λ controlling the tradeoff.
- Self-Supervision for Self-Expression: Spectral clustering converts self-expression coefficients into an affinity matrix and segmentation, then uses that segmentation to constrain coefficients within clusters.The weighted objective penalizes coefficients connecting points assigned to different groups.
- Self-Supervision for Feature Learning: Spectral-clustering labels supervise feature learning through a classification layer whose loss combines cross-entropy with center loss.The center-loss term compresses intra-cluster variation, and τ controls the tradeoff.
- Training S2ConvSCN: Because spectral-clustering outputs are pseudo-labels with arbitrary cluster ordering, successive labels are aligned using the Hungarian algorithm before supervision.The paper calls the resulting feedback to both modules a dual self-supervision mechanism.
4. Experimental Evaluations
Experiments evaluate S2ConvSCN across face and object datasets, comparing network variants, dual self-supervision, and training behavior. The method achieves strong clustering performance, while ablations and convergence analyses examine the contribution and dynamics of its components.
- Datasets and settings: Experiments cover Extended Yale B, ORL, COIL20, and COIL100, using dataset-specific convolutional architectures and training settings.Extended Yale B experiments vary the number of subjects, while ORL, COIL20, and COIL100 use down-sampled images and specified network configurations.
- Extended Yale B: S2ConvSCN-ℓ1 and S2ConvSCN-ℓ2 yield the lowest clustering errors on Extended Yale B across n ∈ {10, 15, 20, 25, 30, 35, 38} among listed baselines.DASC reports a slightly better 1.44% clustering error for n = 38.
- Ablation study: Dual self-supervision further reduces clustering errors beyond either spectral-clustering or classification self-supervision alone.The classification-loss module is more effective than the spectral-clustering-loss module when used individually, while their combination performs best.
- ORL: On ORL, the proposed approaches yield the best results despite the dataset’s nonlinear variations and small sample size per subject.ORL includes changes in lighting, facial expression, and facial details, making it more challenging for subspace clustering.
- COIL20 and COIL100: On COIL20 and COIL100, S2ConvSCN-ℓ1 and S2ConvSCN-ℓ2 significantly reduce clustering errors.The authors attribute this result to the dual self-supervision components’ use of information from spectral-clustering outputs.
- Convergence behavior: During Extended Yale B training with n = 10, the total loss, selected component losses, and clustering error decrease rapidly before becoming nearly flat.The analysis notes that increasing self-expression-related costs can reflect shrinking extracted features and increasing absolute coefficients when feature normalization is absent.
5. Conclusion
S2ConvSCN jointly learns visual features and subspace structure in an end-to-end framework, using spectral clustering for dual self-supervision. This integration improves the convolutional module and self-expression model, achieving superior benchmark performance.
- S2ConvSCN integrates convolutional feature extraction, self-expression affinity learning, and spectral clustering into one joint optimization framework.
- Dual self-supervision uses spectral-clustering output to improve convolutional-module training and refine the self-expression model.
- Experiments on benchmark datasets validate the approach’s effectiveness and demonstrate superior performance.