Source-linked AI summary
Fashion-Gen: The Generative Fashion Dataset and Challenge
Negar Rostamzadeh, Seyedarian Hosseini, Thomas Boquet, Wojciech Stokowiec, Ying Zhang, Christian Jauvin, Chris Pal
TL;DR
Text-to-image fashion research lacks large, high-quality datasets designed for detailed conditional generation. The paper introduces Fashion-Gen with expert descriptions and multi-angle high-definition images, evaluates high-resolution and text-conditioned generation baselines, and releases a challenge, while noting limitations of Inception Score evaluation.
Problem
Text-conditioned image generation remains lower quality, and existing datasets were not specifically collected for text-to-image synthesis.
Method
The paper introduces a large fashion dataset with detailed expert descriptions, multi-angle images, and baseline models for high-resolution and text-conditioned generation.
Results
The paper reports high-resolution P-GAN images with Inception Scores close to original images and empirical improvements from pre-trained text encoders, especially a fixed bi-LSTM encoder.
Takeaways & Limitations
The dataset and challenge provide a shared basis for developing and comparing conditional fashion image generators.
Takeaways & Limitations
Inception Scores can differ substantially across implementations, so the challenge also includes human evaluation.
Abstract
from arXiv · showhide
We introduce a new dataset of 293,008 high definition (1360 x 1360 pixels) fashion images paired with item descriptions provided by professional stylists. Each item is photographed from a variety of angles. We provide baseline results on 1) high-resolution image generation, and 2) image generation conditioned on the given text descriptions. We invite the community to improve upon these baselines. In this paper, we also outline the details of a challenge that we are launching based upon this dataset.
1. Introduction
The paper targets text-to-image generation for fashion design and introduces a large, expert-described dataset to support this task. It also reports baseline generation experiments and launches a related challenge.
- Fashion applications include style matching, recommendation, trend prediction, virtual try-on, and clothing classification.
- The paper explores translating verbal descriptions of clothing and accessories into matching images for fashion designers.
- The dataset contains almost 300k high-definition images with detailed descriptions from professional designers and standardized multi-angle photography.
- The paper provides dataset statistics, comparisons with existing datasets, a text-to-image challenge, and baseline results using progressive GANs and StackGAN variants.
2. Related Work
Prior generative models produce realistic images, but text-conditioned generation remains limited, partly because existing datasets were not designed specifically for text-to-image synthesis. The paper situates its dataset and challenge within this gap.
- Generative adversarial networks have been applied to super-resolution, video generation, inpainting, image translation, and text-to-image synthesis.
- Conditional generation and translation remain lower quality than unconditional realistic image generation, motivating improved datasets.
- The paper presents its challenge as an effort to stimulate research on conditional generative models.
- Existing text-to-image datasets were generally created for other purposes, such as classification or captioning, rather than specifically for synthesis.
- Prior datasets include bird, flower, celebrity, captioning, and fashion collections with different image counts, attributes, and annotation schemes.
3. Our Fashion Dataset
Fashion-Gen is a large, high-resolution fashion dataset combining expert descriptions, standardized multi-angle images, fine-grained categories, and stylistic metadata. It is designed to support text-to-image synthesis and related fashion modeling.
- 293,008 images are split into 260,480 training, 32,528 validation, and 32,528 test images.
- The dataset provides full HD images captured under consistent studio conditions.
- Each fashion item is photographed from 1 to 6 angles depending on its category, forming a large-scale multi-angle collection.
- Items are organized into 48 main categories and 121 fine-grained categories.
- Each item has paragraph-length captions written by professional designers.
- Metadata includes stylist-recommended matched items, season, designer, brand, and colors extracted from descriptions.
4. Our Challenge
The challenge uses the Fashion dataset for text-to-image synthesis and evaluates submissions with both automated Inception scores and human judgments. Human evaluation is included because Inception scores do not measure text–image correlation and can vary across implementations.
- The challenge applies the Fashion dataset to text-to-image synthesis and encourages participants to use additional information such as pose or category.The paper describes this as the first challenge on this task and provides a comparison framework.
- Different implementations trained on the same dataset can produce significantly different Inception scores.
- Submissions receive automated Inception scores through a platform that runs containerized code on an unreleased test set.The platform displays scores, maintains a leaderboard, and evaluates submissions on an Amazon Web Services cloud instance.
- Human evaluation determines the competition winner because Inception scores omit text–image correlation and have other documented issues.Evaluators receive a text and five generated images from each submission, and scores are aggregated across evaluators.
5. Experiments with the Dataset
The experiments assess Fashion-Gen with progressive GANs for high-resolution generation and StackGAN architectures for text-conditioned synthesis. Results show coherent high-resolution images, strong P-GAN scores relative to originals, and a trade-off between StackGAN-v1 scores and StackGAN-v2 visual quality, while pretrained fixed bi-LSTM embeddings perform best overall.
- Experiments: The experiments cover high-resolution image generation with progressive GANs and text-to-image synthesis with StackGAN-v1 and StackGAN-v2.
- Generating high-resolution images using P-GANs: Progressive GANs grow the generator and discriminator symmetrically from low resolution, adding fine-grained detail at each stage; the experiments generate 1024 × 1024 images.
- Generating high-resolution images using P-GANs: Generated P-GAN images exhibit global coherence across varied poses, colors, categories, accessory textures, and fashion-design characteristics.
- Generating high-resolution images using P-GANs: The P-GAN images’ Inception score is very close to that of the original images after generated images are downsampled to 256 × 256.
- Text-to-Image synthesis:: StackGAN-v1 sketches a 64×64 image from text and noise, then StackGAN-v2 refines it into a 256 × 256 image using the first-stage output and text embeddings.
- Text-to-Image synthesis:: Pretrained fixed bi-LSTM text embeddings produce better visual and quantitative results overall, while encoder pretraining improves correspondence between descriptions and generated images.
- Text-to-Image synthesis:: StackGAN-v1 has a better score, whereas StackGAN-v2 has better image quality but experienced significant mode collapse.
6. Conclusion
The paper introduces a Fashion-themed text-to-image dataset with expert annotations and evaluates both high-resolution generation and text-conditioned image generation. It releases the dataset through a challenge to encourage further research in conditional generative models.
- The dataset combines high-quality fashion images with extensive annotations provided by fashion experts.It supports research on fashion image generation and text-conditioned synthesis.
- The authors evaluate high-resolution image generation without textual input and image generation conditioned on product descriptions.They use StackGAN-v1 and StackGAN-v2 with various text encoders for the conditional experiments.
- The dataset is released through a challenge intended to stimulate further research on conditional generative models.The challenge provides submission instructions and an API that computes an inception score trained on the Fashion dataset.
- Challenge winners are selected based on image quality judged by human evaluators.