A Survey on Soft Subspace Clustering

• High-dimensional clustering can miss clusters associated with different feature subsets, where same-cluster neighbors may be far apart in some dimensions.Subspace clustering seeks clusters in different subspaces or in a subspace of the original data space.Source: §1 · p. 2
• Hard subspace clustering seeks exact subspaces, whereas soft subspace clustering weights dimensions by their contribution to each cluster.The survey divides SSC into conventional (CSSC), independent (ISSC), and extended (XSSC) methods.Source: §1 · pp. 2–3
• The survey aims to profile existing SSC methods and support the development of more effective clustering technologies.It reviews the categories and their characteristics and discusses potential future development.Source: §1 · pp. 2–3

• CSSC separates subspace identification from clustering or learns feature weights while clustering.The survey calls these approaches separated and coupled feature weighting, respectively.Source: §2 · p. 4, Table 3 · p. 5
• ISSC methods use K-means, fuzzy C-means, or probability mixture models with fuzzy, entropy, or other weighting mechanisms.Source: §2 · pp. 4–5, Table 3 · p. 5
• XSSC is divided into eight subcategories based on strategies for enhancing CSSC and ISSC methods.These include between-class separation, evolutionary learning, new metrics, ensemble and multi-view learning, imbalanced clusters, transformed-space subspace extraction, and other approaches.Source: §2 · pp. 4–5, Table 3 · p. 5

3 CSSC

• CSSC separates feature weighting from clustering or learns weights during clustering; Table 4 lists nine representative CSSC algorithms.In separated weighting, weights are determined before clustering; in coupled weighting, they are learned during clustering.Source: §3 · p. 5, Table 4 · p. 6

3.1 Separated Feature Weighting Algorithms (1) C-K-means

• C-K-means selects the partition with the minimum generalized Fisher ratio across feasible weight groups, but those groups may exclude optimal weights and are impractical to define for high-dimensional data.For each feasible weight group, k-means generates a partition and its Q value is calculated.Source: §3.1 · p. 7

(2) OVW-UAT

• OVW-UAT optimizes variable weights for hierarchical clustering through objective functions, but its hierarchical methods are computationally complex and the approach is not scalable to large datasets.Its k-means extension can optimize over an objective function rather than only within predefined weight groups, as in C-K-means.Source: (2) OVW-UAT · p. 7

(3) WFCM

• WFCM estimates weights before clustering and then uses weighted Euclidean distance in place of Euclidean distance in FCM; the weights cannot be further optimized during clustering.The weights are estimated using an objective function and gradient descent.Source: (3) WFCM · p. 8

(1) SYNCLUS, FWSA, and FWFKM

• SYNCLUS, FWSA, and FWFKM alternate between partitioning data and learning feature weights with separate objective functions, complicating convergence analysis; SYNCLUS is also time-consuming and unsuitable for large datasets.FWFKM iteratively applies fuzzy k-means and supervised ReliefF to update feature weights.Source: (1) SYNCLUS, FWSA, and FWFKM · pp. 9–10, (2) W-k-means · p. 10

(2) W-k-means

• W-k-means updates feature weights from within-cluster distance variance and uses them in the next membership step, jointly optimizing partition and weights with a common objective whose convergence can be analyzed rigorously.Its converged weights can identify important features, and features that may be noise can be removed in future analysis.Source: (2) W-k-means · p. 10

(3) MWLA and MA-DDC-FW

• MWLA uses a Gaussian mixture model with separate weights for mixture components and features, while MA-DDC-FW models discrete features as finite mixtures and learns relevance weights probabilistically.MA-DDC-FW uses Bayesian and information-theoretic approaches through stochastic complexity; its feasibility is demonstrated for visual-concept recognition and text clustering.Source: (3) MWLA and MA-DDC-FW · pp. 10–11

3.3 Simulation on the Toy dataset

• On Toy-D, W-k-means assigned larger weights to important features, but its shared weight vector could not describe feature importance in each cluster.Table 6 reports ten-run means and standard deviations for RI and NMI.Source: §3.3 · p. 12, Figure 2 · p. 13, Table 6 · p. 13

4 ISSC

• ISSC distinguishes itself from CSSC by associating each cluster with an independent subspace and weight vector.Reviewed approaches include fuzzy and entropy weighting, among other mechanisms.Source: §4 · pp. 13–14

(1) AWA, FWKM, and FSC

• AWA can identify cluster-relevant features but fails when an attribute has zero standard deviation; FWKM and FSC add constants to address this weakness.FSC's constant is set manually, whereas FWKM uses a predefined formulation.Source: (1) AWA, FWKM, and FSC · p. 15

(2) EWKM and LAC

• EWKM and LAC use entropy weighting; LAC accounts for cluster size, unlike EWKM, and large γ in EWKM yields equal feature weights.The reviewed alternative LAC objective is reported to perform worse than the entropy-based LAC approach.Source: (2) EWKM and LAC · pp. 16–17

(1) AWFCM

• AWFCM is presented as the first fuzzy-weighting ISSC algorithm, and improved versions such as EFWSSC have been proposed.AWFCM is described as a soft-partition version of AWA that is more adaptive to noisy data.Source: (1) AWFCM · pp. 17–18

(2) SCAD

• SCAD-2 and AWFCM are characterized as fuzzy-weighting SSC algorithms of the same kind, while SCAD-2 is more general than SCAD-1.The comparison is based on their objective functions.Source: (2) SCAD · p. 18

4.3 Probability Mixture Model based ISSC

• FCM/K-means-based ISSC algorithms do not account for data distribution, motivating mixture-model approaches in this subsection.The review introduces two representative mixture-model algorithms.Source: §4.3 · p. 18

(1) FPC/MPC

• FPC/MPC models projected dimensions with Gaussian mixtures; its entropy term controls cluster memberships rather than feature weights.The review expects FPC to adapt better to data distributions, while noting mixture-model ISSC is more complicated than FCM/K-means approaches.Source: (1) FPC/MPC · pp. 18–19

(2) EWMM

• EWMM uses entropy to control feature weights, unlike FPC/MPC, whose entropy term controls memberships; EWMM is described as more complicated and not popular.The review also states that mixture-model FPC algorithms can adapt better to data distributions.Source: (2) EWMM · p. 20

4.4 ISSC based on Other Models

• COSA is an entropy-based subspace-clustering algorithm, and a cited shortcoming is that it may not scale to large datasets.Source: §4.4 · pp. 20–21

4.5 Simulation on the Toy Dataset

• FSC identifies important features in Toy-D clusters and shows better clustering performance than W-k-means on Toy-D.The reported feature-identification advantage over W-k-means is supported by Table 8, and Table 9 compares clustering performance.Source: §4.5 · p. 21, Table 8 · p. 21, Figure 3 · p. 21

5 XSSC

• XSSC extends CSSC or ISSC with new mechanisms for general or specific purposes, including multi-view data and imbalanced clusters.The survey identifies seven listed subcategories and an additional “others” category not included in the table.Source: §5 · p. 22, Table 10 · p. 22

5.1 Between-cluster Separation

• Because most SSC algorithms optimize within-cluster compactness without between-cluster information, newer methods combine separation and compactness to enhance clustering performance.IEWKM, ESSC, and EFWSSC are identified as representative algorithms.Source: §5.1 · p. 23

(1) IEWKM

• IEWKM incorporates between-cluster information, but its objective is difficult to optimize and its cluster-center learning-rule derivation is described as insufficiently rigorous.Source: (1) IEWKM · p. 23

(2) ESSC and EFWSSC

• ESSC jointly minimizes weighted within-cluster compactness and maximizes between-cluster separation; EFWSSC uses fuzzy weighting, and FSC is a special case of EFWSSC.ESSC adds manually set parameters whose appropriate values remain an open problem; ESSC and EFWSSC are described as mathematically more rigorous than IEWKM.Source: (2) ESSC and EFWSSC · pp. 23–24

5.2 Evolutionary Learning

• Evolutionary learning is introduced to address initialization sensitivity and local optimization in iterative SSC methods.Source: §5.2 · p. 24

(1) Coevolutionary SSC

• Coevolutionary SSC methods come close to outperforming K-means-like algorithms, but require more CPU time than hill-climbing methods.The reviewed methods use either an AWC distance-based cost function or a partitioning-quality measure as fitness.Source: (1) Coevolutionary SSC · p. 25

(2) PSOVW

• PSOVW transforms constrained variable weighting into bound-constrained optimization and is reported to improve clustering performance and reduce dependence on initial centers.Its PSO parameters require manual setting, and its random optimization can yield unstable results across runs.Source: (2) PSOVW · p. 25

(3) MOEA-SSC

• MOEA-SSC uses multiple objectives, including within-cluster dispersion and information about weight entropy and cluster separation, but run-to-run solution stability is not guaranteed.Like other evolutionary methods discussed, it is described as less sensitive to initialization.Source: (3) MOEA-SSC · p. 26

(1) Minkowski Metric

• MWK-Means replaces Euclidean distance with the Minkowski metric, whose parameter p must be chosen for the clustering task; at p = 2, AWA is a special case.The passage frames selecting p as task-dependent.Source: (1) Minkowski Metric · pp. 26–27

(2) Kernel Metric

• Kernel-distance SSC is described as more adaptable to data distributions than Euclidean-based SSC, but selecting the kernel and its parameters remains a critical challenge.The survey says further study is required to resolve this selection problem.Source: (2) Kernel Metric · pp. 27–28

5.4 Ensemble Learning

• WSPA fuses LAC runs with different parameter settings, but its computational complexity is high and the number of runs to combine remains open.The survey presents ensemble learning as a response to parameter-sensitive clustering.Source: §5.4 · p. 28
• MOEA-PCE generally produces higher-quality projective consensus clustering than EM-PCE, while the multi-objective method has high computational complexity.The passage describes EM-PCE as more efficient, although it mistakenly repeats EM-PCE when naming the more efficient algorithm.Source: §5.4 · p. 28

5.5 Multi-view Learning

• SSC methods for multi-view data are introduced because traditional SSC algorithms are not suitable for that setting.The section presents TW-k-means and FG-k-means as representative multi-view approaches.Source: §5.5 · p. 28, (1) TW-k-means · p. 28

(1) TW-k-means

• TW-k-means adjusts feature importance within views first, then adjusts each view's influence, but its co-learning only adaptively adjusts view objective functions.More advanced co-learning, such as co-learning fuzzy partition matrices across views, is expected to further improve performance.Source: (1) TW-k-means · p. 29

(2) FG-k-means

• FG-k-means extends TW-k-means from CSSC to ISSC by allowing feature weights to vary by cluster, while retaining a simple objective-function-based co-learning mechanism.The survey says more effective multi-view SSC requires advanced co-learning mechanisms.Source: (2) FG-k-means · pp. 29–30

5.6 Imbalanced Clusters

• WLAC applies elite selection of weighted clusters to address the degradation of classic SSC on imbalanced datasets; cluster diameter balances the influence of imbalanced clusters.Choosing the WLAC parameter γ2 remains challenging.Source: §5.6 · p. 30

5.7 Subspace Extraction in the Transformed Feature Space

• RKM generally performs better than FKM when most features reflect the clustering structure and/or features are standardized before analysis.Both methods use PCA to form a low-dimensional subspace; when Q = D, both reduce to classic K-means.Source: §5.7 · pp. 30–31

(1) Categorical Data

• Several reviewed SSC variants handle categorical or mixed categorical-numeric data, but convergence analysis for these algorithms remains challenging.The survey names IEWKM, modified EWKM, and probabilistic-distance SSC among the approaches.Source: (1) Categorical Data · p. 31

(2) Data Reliability

• Reliability-based SSC can be applied across various clustering algorithms, and a gene-expression evaluation reports improvements over baseline and important soft and crisp subspace-clustering methods.The survey also notes work to improve efficiency for large datasets.Source: (2) Data Reliability · pp. 31–32

5.9 Simulation on the Toy Dataset

• ESSC assigns larger weights to important Toy-D features and outperforms W-k-means and FSC in clustering performance on that dataset.The survey also reports that ESSC identifies Toy-D's important feature space better than FSC.Source: §5.9 · p. 32, Figure 4 · p. 32, Table 12 · p. 33

6 Model Complexity

• ISSC algorithms are usually more computationally complex than CSSC algorithms but less complex than XSSC algorithms.The comparison concerns model complexity from iterative-rule-based learning.Source: §6 · p. 33, Table 13 · p. 33

7 Experimental Validation

• The study compares W-k-means, EWKM, and ESSC on five real-world cancer gene-expression datasets, running each algorithm 10 times and reporting means and standard deviations for RI and NMI.Features are normalized to [0, 1] before the experiments.Source: §7 · pp. 34–35, Table 14 · p. 34
• ESSC improves clustering performance beyond EWKM, while EWKM outperforms W-k-means on the gene-expression experiments.The comparisons are reported across the experimental results summarized in the RI and NMI tables.Source: §7 · p. 35, Table 16 · p. 35, Table 17 · p. 36

8 Conclusions

• The survey systematically classifies SSC algorithms into CSSC, ISSC, and XSSC, including their subcategories.The categories and subcategories are reviewed and discussed in detail.Source: §8 · p. 36
• The survey provides an overall picture of SSC research and insights into future advancements.Source: §8 · p. 36
• The authors suggest that this overview can help researchers study unresolved SSC problems and propose more adaptive algorithms.Source: §8 · p. 36