Source-linked AI summary
Exploring Unlabeled Faces for Novel Attribute Discovery
Hyojin Bahng, Sunghyo Chung, Seungjoo Yoo, Jaegul Choo
TL;DR
Existing facial image translation methods depend on labeled images and can generalize poorly across distributions, while facial attributes are costly and incomplete to annotate. The paper introduces XploreGAN, which discovers attributes from unlabeled faces using pre-trained feature guidance and transfers cluster-shared attributes with ASIN. Experiments report high-quality, identity-preserving translations that are as good as or sometimes better than state-of-the-art labeled-data methods.
Problem
Existing image translation systems require many labeled images, while facial annotation is costly, incomplete, and vulnerable to dataset bias across distributions.
Method
XploreGAN clusters pre-trained feature representations of unlabeled faces into attribute groups and uses ASIN to transfer their shared attributes during translation.
Results
XploreGAN produces high-quality translations and preserves identity as well as, or sometimes better than, most state-of-the-art methods trained with attribute labels.
Takeaways & Limitations
Unlabeled facial data can support unpaired and unlabeled multi-domain translation through feature-guided attribute discovery and cluster-based normalization.
Abstract
from arXiv · showhide
Despite remarkable success in unpaired image-to-image translation, existing systems still require a large amount of labeled images. This is a bottleneck for their real-world applications; in practice, a model trained on labeled CelebA dataset does not work well for test images from a different distribution -- greatly limiting their application to unlabeled images of a much larger quantity. In this paper, we attempt to alleviate this necessity for labeled data in the facial image translation domain. We aim to explore the degree to which you can discover novel attributes from unlabeled faces and perform high-quality translation. To this end, we use prior knowledge about the visual world as guidance to discover novel attributes and transfer them via a novel normalization method. Experiments show that our method trained on unlabeled data produces high-quality translations, preserves identity, and be perceptually realistic as good as, or better than, state-of-the-art methods trained on labeled data.
1. Introduction
The paper targets facial image translation without large labeled datasets, motivated by dataset bias and the costly, incomplete annotation of facial attributes. It proposes discovering attributes from unlabeled faces and transferring them with XploreGAN.
- Motivation: Existing translation systems still require many labeled images, and models trained on labeled CelebA data may generalize poorly to different test distributions.CelebA-trained models are biased toward Western celebrity faces, limiting real-world use.
- Motivation: Facial translation requires extensive annotation, while benchmark labels remain incomplete for the diversity of possible attributes.CelebA annotates 40 attributes for 202,599 images but includes only pale skin among possible skin colors.
- Approach: XploreGAN explores discovering novel attributes directly from unlabeled faces to move toward unpaired and unlabeled multi-domain translation.The paper frames attribute discovery as making attributes emerge from data rather than relying on predefined labels.
- Approach: The method uses pre-trained CNN features to group unlabeled faces and transfers cluster-shared attributes through attribute summary instance normalization.ASIN summarizes a common attribute within an image group instead of transferring entangled attributes from one style image.
2. Proposed Method
The proposed method clusters pre-trained feature representations to obtain pseudo-labels for unlabeled facial attributes, then conditions translation on cluster statistics through ASIN. Its objectives combine target-cluster classification, reconstruction, latent consistency, and adversarial realism while preserving content.
- Clustering stage: Pre-trained feature spaces guide k-means clustering of unlabeled images, whose assignments become pseudo-labels and whose statistics condition the generator.The clustering uses feature vectors, producing assignments C, centroids µ, and standard deviations σ.
- Attribute summary instance normalization: ASIN maps cluster statistics to normalization parameters, transferring attributes shared by an image group rather than entangled properties from a single style image.The approach can use centroid and variance information to summarize common cluster attributes.
- Objective function: Cluster classification trains the discriminator and generator to associate real and generated images with their original or target pseudo-domains.Unlike prior multi-domain methods, the domains are cluster assignments rather than pre-given attribute labels.
- Objective function: Reconstruction and latent losses encourage content preservation while allowing domain-related details to change.The latent loss matches real and translated images in encoder feature space, supporting perceptual similarity despite pixel-level differences.
- Objective function: Adversarial training uses a Wasserstein GAN objective with gradient penalty to make generated images indistinguishable from real images.The generator receives the input image and target cluster statistics, while the discriminator distinguishes generated from real images.
- Implementation: The implementation extracts and reduces convolutional features for clustering, then uses an encoder-decoder generator with ASIN, residual blocks, noise, and stochastic variation.BagNet-17 is used for FFHQ and CelebA, while SIN-pre-trained ResNet-50 is used for EmotioNet.
3. Experiments
Experiments evaluate XploreGAN on unlabeled face datasets against supervised translation baselines, testing attribute transfer, identity preservation, realism, and clustering choices. XploreGAN produces high-quality translations across datasets and performs comparably to or better than labeled-data methods on reported evaluations.
- Style normalization: ASIN transfers a cluster’s common attribute while preserving other content attributes, avoiding AdaIN’s tendency to transfer entangled properties from one style image.AdaIN is instance-conditioned, whereas ASIN uses group statistics; each XploreGAN result is generated from one cluster’s statistics.
- Qualitative evaluation: XploreGAN generates high-quality translations on FFHQ, CelebA, and EmotioNet, matching the quality of models trained with attribute labels.The method discovers attributes including hair color, ethnicity, age, and facial expressions from unlabeled images.
- Quantitative evaluation: XploreGAN outperforms baseline models on almost all five CelebA attribute-translation classification evaluations despite using unlabeled training data.The attribute classifier achieves 95.8% average accuracy on real test images.
- Quantitative evaluation: XploreGAN preserves input identity as well as or better than most label-trained baselines, whereas MUNIT and DRIT’s strong attribute transfer reduces identity preservation.Identity is evaluated with face-verification accuracy on real–translated image pairs.
- Quantitative evaluation: A user study with 32 participants finds XploreGAN as realistic as supervised models across diverse attributes.The study includes 120 comparisons spanning six attributes, with random single outputs selected for multimodal baselines.
- Clustering analysis: Texture-biased BagNets expose cues such as skin color, age, hair, and lighting, while SIN-trained ResNets emphasize expressions, gestures, and viewpoints.The experiments use BagNet-17 for CelebA and SIN-pretrained ResNet-50 for EmotioNet.
4. Related Work
Related work spans GAN-based and unpaired multi-domain translation, but these systems generally depend on labeled data. This paper instead uses clustering to discover attributes in unlabeled images and normalization techniques to transfer them.
- Existing image-to-image translation methods include GANs, unpaired translation, and multi-domain models that generate diverse or multimodal outputs.Conditional GANs can use class, text, domain, input-image, or color information; CycleGAN, DRIT, and MUNIT address unpaired or multimodal translation.
- Most existing translation models call their setting unsupervised when data are unpaired, yet still require large amounts of labeled images.The paper defines unsupervised translation as requiring both unpaired and unlabeled data, a setting it reports as previously unaddressed.
- Clustering has been used to discover unknown object classes, whereas this work applies it to discover attributes within unlabeled images.The paper notes that attributes are more complicated because a single image can contain multiple attributes.
- The paper positions its contribution as image translation using newly discovered attributes from unlabeled data rather than predefined attribute labels.This reframes clustering from object-class discovery toward attribute discovery for translation.
- Normalization methods such as instance, conditional instance, and adaptive instance normalization support style-transfer systems by manipulating feature statistics.These methods differ in whether statistics come from each sample or whether learned parameters are conditioned on style.
5. Conclusion
The paper presents an unpaired and unlabeled facial image translation method that discovers attributes from pretrained feature spaces and transfers them through ASIN. It reports high-quality results comparable to or better than most state-of-the-art methods.
- The method groups unseen unlabeled images using pretrained feature spaces and uses ASIN to summarize cluster attributes for high-quality translation.Cluster assignments serve as pseudo-labels, while cluster statistics condition the generator's normalization layer.
- The model produces results as good as, or sometimes better than, most state-of-the-art methods.