Source-linked AI summary
ClusterGAN : Latent Space Clustering in Generative Adversarial Networks
Sudipto Mukherjee, Himanshu Asnani, Eugene Lin, Sreeram Kannan
TL;DR
GAN back-projection does not generally preserve cluster structure in the latent space. ClusterGAN addresses this with discrete-continuous latent variables, an inverse network, and clustering-specific joint training; the paper reports superior clustering performance while retaining cross-category interpolation. Its CIFAR-10 results also show that unsupervised modes may capture visual commonalities rather than dataset labels.
Problem
Back-projecting data into traditional Gaussian or uniform GAN latent spaces usually fails for clustering because their distance geometry does not reflect inherent data clusters.
Method
ClusterGAN combines discrete-continuous latent mixtures, an inverse-mapping network, and a jointly trained clustering-specific loss to structure the latent space.
Results
ClusterGAN comparisons across varied datasets demonstrate superior clustering performance and preserve interpolation across classes, even though the discriminator is never exposed to interpolated samples.
Takeaways & Limitations
GANs can be adapted for latent-space clustering while retaining smooth interpolation across categories.
Takeaways & Limitations
On CIFAR-10, unsupervised modes may follow visual commonalities rather than labels; the authors suggest mutual-information objectives as future work.
Abstract
from arXiv · showhide
Generative Adversarial networks (GANs) have obtained remarkable success in many unsupervised learning tasks and unarguably, clustering is an important unsupervised learning problem. While one can potentially exploit the latent-space back-projection in GANs to cluster, we demonstrate that the cluster structure is not retained in the GAN latent space. In this paper, we propose ClusterGAN as a new mechanism for clustering using GANs. By sampling latent variables from a mixture of one-hot encoded variables and continuous latent variables, coupled with an inverse network (which projects the data to the latent space) trained jointly with a clustering specific loss, we are able to achieve clustering in the latent space. Our results show a remarkable phenomenon that GANs can preserve latent space interpolation across categories, even though the discriminator is never exposed to such vectors. We compare our results with various clustering baselines and demonstrate superior performance on both synthetic and real datasets.
1 Introduction
ClusterGAN addresses the failure of traditional GAN latent priors to preserve cluster geometry by combining discrete-continuous latent variables with an inverse network and clustering-specific training. The resulting framework targets latent-space clustering while retaining interpolation across categories and outperforming clustering baselines.
- ClusterGAN: ClusterGAN uses discrete and continuous latent variables to create separated, non-smooth geometry that supports clustering in the latent space.Small continuous-noise variance keeps one-hot-defined clusters separated.
- ClusterGAN: An explicit inverse-mapping network is trained jointly with the GAN using a clustering-specific loss so projected distances reflect latent-variable distances.The encoder maps data into the latent space while jointly optimizing the GAN objective.
- Results: ClusterGAN is evaluated against GAN-based methods and clustering baselines across varied datasets, with the paper reporting superior clustering performance.The introduction frames the evaluation as a comparison with methods including InfoGAN and multiple clustering baselines.
- Results: ClusterGAN retains smooth interpolation across different classes encoded by one-hot latent variables, despite the discriminator never seeing such interpolated samples.The paper presents this as a notable property of the learned latent space.
2 Discrete-Continuous Prior
Traditional GAN latent spaces can preserve information yet scatter classes continuously, undermining clustering through back-projection. ClusterGAN addresses this with separated discrete-continuous priors and decoding procedures that support clustering while retaining interpolation.
- Motivation: Back-projecting data into a traditional Gaussian or uniform latent space does not yield clusters because the recovered distribution follows the prior.The latent space may contain full information, but its continuously scattered geometry conflicts with clustering.
- Prior design: A discrete-continuous prior combines Gaussian variables zn with one-hot categorical variables zc, whose separated components represent K clusters.ClusterGAN samples z=(zn,zc), with zc=ek and k uniformly selected among K components.
- Modified decoding: ClusterGAN decodes data by fixing each one-hot component, optimizing only the continuous variables, and using K restarts to select the lowest-loss embedding.The decoding objective uses reconstruction loss with a regularizer on zn, while zc remains fixed during each restart.
- Prior design: For mixture-of-Gaussians data, a linear generator cannot recover the mixture from continuous variables alone but can generate it using discrete-continuous variables.The construction uses G(zn,zc)=zn+Azc, where the one-hot component selects the Gaussian mean.
- Empirical behavior: In synthetic experiments, GANs with traditional priors failed to cluster in latent space, whereas one-hot components generated samples from specific data classes without additional loss terms.The mixture-of-10-Gaussians experiment compared a Gaussian prior with one-hot encoded normal vectors; the traditional configurations did not cluster.
- Interpolation: Interpolating between one-hot components while fixing zn produced smooth transitions across digits and Fashion-MNIST classes, despite those interpolated vectors being unseen by the discriminator.ClusterGAN therefore combines separated latent modes with smooth cross-class interpolation.
3 ClusterGAN
ClusterGAN augments GAN training with an encoder and clustering-specific latent-code recovery, preserving both discrete and continuous structure. Joint cycle consistency makes data and latent reconstructions close.
- 3 ClusterGAN: ClusterGAN introduces an encoder E:X→Z and trains it jointly with the generator to enforce precise latent-vector recovery.The approach adds a clustering-specific loss to the minimax objective.
- 3 ClusterGAN: Cross-entropy and latent-distance regularization separately control preservation of the discrete and continuous portions of the latent code.The coefficients βn and βc determine their relative importance.
- 3 ClusterGAN: 0.038 was the mean square L2-distance for x−G(E(x)), while 0.0004 was the distance for z−E(G(z)) on a held-out MNIST test batch.Both cycle losses decreased during training, although only one was directly optimized.
4 Experiments
Experiments evaluate ClusterGAN on synthetic, image, time-series, biological, and object datasets against GAN-based and standard clustering baselines. The reported evaluations cover clustering quality, generated-sample quality, robustness to cluster-count choice, and scalability.
- Datasets: The synthetic benchmark generates 2,500 points per component from a four-component Gaussian mixture in a 2D latent space, then applies a nonlinear transformation to obtain X.The transformation uses sigmoid layers with W∈R10×2 and U∈R100×10.
- Baselines and evaluation: ClusterGAN is compared with GAN back-projection, InfoGAN, discriminator features, NMF, agglomerative clustering, spectral clustering, and DCN.Clustering is performed without exposing algorithms to true labels; metrics include NMI, ARI, and purity (ACC).
- Evaluation: ClusterGAN achieved good clustering without compromising image sample quality, with generated-image quality assessed using FID alongside clustering metrics.Table 2 reports FID, for which lower distance is better.
- Robustness: Overestimating MNIST’s cluster count did not severely hurt ClusterGAN, whereas underestimating it did.The true number of clusters was provided to all algorithms in the main comparisons.
- Scalability: ClusterGAN obtained good clusters on Coil-20 with K=20 and Coil-100 with K=100 despite few thousand training points and difficult GAN training.Distinct modes generated distinct 3D objects together with rotations.
5 Discussion and Future Work
The discussion frames ClusterGAN as a GAN architecture for latent-space clustering based on discrete-continuous priors and reports applicability across varied datasets. It identifies data-driven priors and sparse generative structures as future directions.
- 5 Discussion and Future Work: ClusterGAN replaces traditional latent priors with discrete-continuous mixtures and enables clustering in the GAN latent space.The paper presents comparisons across varied datasets as evidence that GANs can be adapted for clustering.
- 5 Discussion and Future Work: Future work includes better data-driven latent-space priors and improved results for sparse generative structures such as compressed sensing.
6 Supplementary Material
Additional results show that unsupervised ClusterGAN modes can organize CIFAR-10 images by visually separating features rather than dataset labels. The supplementary settings use standard optimizer and architecture choices across datasets.
- 6.1 Additional Results: CIFAR-10 modes separate images by features such as blue or predominantly white backgrounds, which may not correspond to dataset labels.Blue backgrounds occur across airplane, bird, and ship categories.
- 6.1 Additional Results: CIFAR-10’s intra-class variability leaves no unsupervised pressure to preserve higher-order label semantics.The paper suggests maximizing mutual information between an intermediate convolutional layer and generated images as future work.
- Training details: All datasets use Adam with η=1e-04, β1=0.5, β2=0.9, five discriminator updates per generator update, and WGAN-GP gradient penalty 10.The dimension of zc equals the number of dataset classes.
Synthetic Data
Synthetic-data experiments used batch size 64 with dataset-specific continuous latent dimensions and LReLU activations. The βn setting differed between MNIST and Fashion-MNIST, while βc was fixed across both.
- MNIST used batch size 64, a 6-dimensional z_n, LReLU leak 0.2, and β_n = 10, β_c = 10.
- For Fashion-MNIST, z_n = 40, β_n = 0, and β_c = 10, while the remaining architecture was unchanged.
Pendigits
The Pendigits configuration used a 5-dimensional continuous latent variable with batch size 64, LReLU leak 0.2, and βn and βc both set to 10.
- Pendigits used batch size 64, z_n of 5 dimensions, LReLU leak 0.2, β_n = 10, and β_c = 10.
Coil-20, Coil-100 and CIFAR-10
The experiments used dataset-specific batch sizes and continuous latent dimensions, common βn and βc settings, and tailored baseline implementations for comparison.
- Coil-20 used batch size 64, Coil-100 used batch size 512, and CIFAR-10 used batch size 64, with z_n of 20, 20, and 50 dimensions, respectively.
- All datasets used β_n = 10, β_c = 10, and LReLU activation with leak 0.2.
- InfoGAN used the authors’ implementation for MNIST and Fashion-MNIST, while other datasets added a Q network; GAN baselines reused ClusterGAN generator and discriminator hyperparameters.
6.3 Reporting Clustering Performance
Clustering performance was evaluated through repeated runs with validation-based model selection, while training remained unsupervised on the training split. Final metrics were reported on held-out test data.
- Five runs were used to select the best model by validation purity for fair comparison with prior reported best accuracies.
- Datasets were split into Train, Validation, and Test portions, and GANs were trained unsupervised using only the Train split.
- Table 1 reports metrics on the Test split for the saved model, whose full-dataset metrics were usually nearly identical or slightly better.
6.4 Generated Modes
The section presents generated samples from distinct modes for digits, fashion items, and CIFAR-10 categories.
- Generated digits are shown from distinct modes.
- Generated fashion items are shown from distinct modes.
- Generated CIFAR-10 categories are shown from distinct modes.