Source-linked AI summary
ClustOfVar: An R Package for the Clustering of Variables
M. Chavent, V. Kuentz, B. Liquet, L. Saracco
TL;DR
ClustOfVar addresses the limited availability of variable-clustering methods for qualitative and mixed data. It provides PCAMIX-based hierarchical and k-means type algorithms whose homogeneity criterion summarizes cluster variables with synthetic quantitative variables, alongside bootstrap stability assessment. The package offers tools for dimension reduction, recoding, and selecting a suitable number of clusters, while large variable sets and high missingness remain practical boundaries.
Problem
Methods specifically devoted to clustering qualitative or mixed-type variables were limited, although variable clustering can support dimension reduction and variable selection.
Method
ClustOfVar uses PCAMIX-based hierarchical and k-means type clustering, synthetic cluster variables, and bootstrap assessment of partition stability.
Results
The package provides hierarchical and k-means type algorithms for clustering quantitative and/or qualitative variables and tools for visualizing links and redundancy.
Takeaways & Limitations
Cluster synthetic variables can support dimension reduction and recoding, while bootstrap stability can help determine a suitable number of clusters.
Takeaways & Limitations
Computational time becomes relatively long with very large numbers of variables, and the simple missing-data imputation may perform poorly when missingness is too large.
Abstract
from arXiv · showhide
Clustering of variables is as a way to arrange variables into homogeneous clusters, i.e., groups of variables which are strongly related to each other and thus bring the same information. These approaches can then be useful for dimension reduction and variable selection. Several specific methods have been developed for the clustering of numerical variables. However concerning qualitative variables or mixtures of quantitative and qualitative variables, far fewer methods have been proposed. The R package ClustOfVar was specifically developed for this purpose. The homogeneity criterion of a cluster is defined as the sum of correlation ratios (for qualitative variables) and squared correlations (for quantitative variables) to a synthetic quantitative variable, summarizing "as good as possible" the variables in the cluster. This synthetic variable is the first principal component obtained with the PCAMIX method. Two algorithms for the clustering of variables are proposed: iterative relocation algorithm and ascendant hierarchical clustering. We also propose a bootstrap approach in order to determine suitable numbers of clusters. We illustrate the methodologies and the associated package on small datasets.
1. Introduction
ClustOfVar addresses the limited availability of variable-clustering methods for qualitative and mixed data by providing R methods based on PCAMIX. It offers hierarchical and k-means type clustering, synthetic variables, and bootstrap-based stability assessment.
- Motivation: Variable clustering groups strongly related variables that bring the same information, supporting variable selection and synthetic-variable construction.For quantitative clusters, the first principal component can serve as the synthetic variable.
- Related methods: Existing variable-specific approaches such as VARCLUS, CLV, and Diametrical Clustering work only with quantitative data and were not implemented in R.
- Package contribution: ClustOfVar provides R methods for clustering quantitative, qualitative, or mixed sets of variables.The package also allows missing data through simple replacements tailored to quantitative and qualitative variables.
- Package contribution: The package offers hierarchical clustering and k-means type partitioning algorithms based on PCAMIX, which includes ordinary PCA and MCA as special cases.
- Package contribution: Bootstrap assessment evaluates partition stability and helps determine a suitable number of clusters.
2. The homogeneity criterion
The homogeneity criterion measures how well variables in a cluster relate to a central synthetic quantitative variable. PCAMIX supplies that variable as the cluster’s first principal component.
- Synthetic variable: For each cluster, PCAMIX produces a central synthetic quantitative variable that is the first principal component of the cluster’s variables.
- Cluster homogeneity: The homogeneity of a cluster measures adequacy between its variables and the central synthetic variable.
- Cluster homogeneity: Quantitative variables contribute squared Pearson correlations, while qualitative variables contribute correlation ratios to the homogeneity criterion.Squared correlations measure links independently of relationship sign; correlation ratios measure variance explained by qualitative categories.
- Cluster homogeneity: Homogeneity is highest when quantitative variables are correlated or anti-correlated with the synthetic variable and qualitative correlation ratios equal 1.This corresponds to variables in the cluster bringing the same information.
- Partition criterion: Partition homogeneity is the sum of the homogeneities of its clusters.
3. The clustering algorithms
ClustOfVar provides hierarchical and partitioning algorithms that optimize cluster homogeneity, plus bootstrap evaluation of partition stability for choosing the number of clusters.
- Objective: The clustering objective is to partition quantitative and/or qualitative variables into strongly related groups while maximizing the homogeneity function H.
- Hierarchical clustering: Hierarchical clustering builds nested partitions by repeatedly merging the two clusters with the smallest homogeneity loss.The resulting hierarchy is implemented in hclustvar and can be cut into a selected number of clusters with cutreevar.
- Partitioning algorithm: The partitioning algorithm uses squared canonical correlation as a similarity measure between variables of any type.For quantitative–quantitative pairs it equals squared Pearson correlation, and for qualitative–quantitative pairs it equals the correlation ratio.
- Partitioning algorithm: kmeansvar iteratively represents clusters with PCAMIX synthetic variables, reallocates variables to the closest cluster, and stops when assignments stabilize or an iteration limit is reached.Multiple random initializations can be used, retaining the partition with the highest H; the resulting optimum is local.
- Stability assessment: Bootstrap stability is measured by the mean adjusted Rand index across dendrograms generated from resampled observations.Plotting this criterion against cluster number can help select a suitable number of clusters.
4. Illustration on simple examples
ClustOfVar is illustrated on quantitative and mixed datasets, showing how hierarchical clustering, stability assessment, and partitioning support variable grouping and synthetic-variable construction.
- Datasets: The package is illustrated on a 10-variable decathlon dataset and a mixed wine dataset containing 31 variables.The decathlon data contain 41 athletes; the wine data contain 21 French wines, with two qualitative and 29 quantitative variables.
- Quantitative data: For the decathlon data, aggregation levels suggest 3 or 5 clusters, while bootstrap stability clearly suggests 5 clusters and dispersion suggests 3 or 5.Stability is assessed with 40 bootstrap samples using adjusted Rand indices.
- Quantitative data: The decathlon hierarchy is cut into 3 clusters because PCA applied within each cluster yields only one eigenvalue greater than 1.The resulting partition and associated cluster objects provide memberships, within-cluster sum of squares, cohesion gain, sizes, and scores.
- Synthetic variables: Each cluster’s synthetic variable is the first PCAMIX component, with squared loadings representing squared correlations for quantitative variables.For example, the squared correlation between “100m” and the first cluster’s synthetic variable is 0.68.
- Mixed data: The six synthetic variables form a 21×6 quantitative matrix that can replace the original 21×31 mixed matrix for recoding.The package also provides a k-means partition into 6 clusters.
- Algorithm comparison: Hierarchical clustering generally produces better-quality partitions than k-means, whereas k-means is faster for datasets with many variables.For one partition, cohesion gain is 56.84 for hierarchical clustering versus 51.02 for k-means.
5. Concluding remarks
ClustOfVar provides variable-clustering algorithms for quantitative, qualitative, and mixed data, with tools for dimension reduction, recoding, and redundancy visualization. Its main practical boundaries are computational cost for very large variable sets and simple missing-data imputation that may be inadequate when missingness is substantial.
- ClustOfVar provides hierarchical and k-means type algorithms for clustering quantitative, qualitative, and mixed variables.
- The package supports dimension reduction and recoding of qualitative or mixed data matrices into quantitative data matrices.Its synthetic cluster variables can be correlated, unlike principal components, which are uncorrelated by construction.
- The package includes tools for visualizing links between variables and redundancy in a dataset.
- For very large numbers of variables, computational time becomes relatively long, motivating future parallel versions of key functions.
- Although missing data are supported, the simple imputation method may perform poorly when the proportion of missing data is too large.The authors recommend using a dedicated missing-data imputation package beforehand in that case.
Affiliation:
The supplied publication metadata identifies the outlet as the Journal of Statistical Software and also references the American Statistical Association. The issue and submission fields are shown as placeholders.
- The paper is associated with the Journal of Statistical Software.
- The metadata references the American Statistical Association as the publisher.
- The volume, issue, submission date, and acceptance date fields are represented by placeholders.