Source-linked AI summary
DeSIGN: Design Inspiration from Generative Networks
Othman Sbai, Mohamed Elhoseiny, Antoine Bordes, Yann LeCun, Camille Couprie
TL;DR
The paper asks whether AI can generate original, compelling fashion designs while respecting clothing constraints. It develops shape- and texture-aware GANs with creativity losses and evaluates them using automatic metrics and human studies. More than 60% of generated designs were judged human-created and original, and creativity-loss models were especially well received.
Problem
The paper asks how to generate original fashion designs that remain realistic and wearable, beyond models that tend to reproduce training examples.
Source: §1 · pp. 1–2Method
The authors investigate GAN architectures and creativity losses for shape- and texture-aware fashion generation, evaluating outputs with adapted automatic metrics and human studies.
Source: §1 · pp. 2–3Results
More than 60% of generated designs were judged human-created and original; creativity-loss models, especially MCE, were perceived as most designer-like and most liked overall.
Source: §1 · p. 3, §4.4 · p. 14Takeaways & Limitations
The authors conclude that their generated designs could make AI an efficient inspirational assistant.
Source: §1 · p. 3The authors seek better image resolution and identify automatic evaluation of generations and GAN stability as open research problems.
Source: §5 · p. 17
Abstract
from arXiv · showhide
Can an algorithm create original and compelling fashion designs to serve as an inspirational assistant? To help answer this question, we design and investigate different image generation models associated with different loss functions to boost creativity in fashion generation. The dimensions of our explorations include: (i) different Generative Adversarial Networks architectures that start from noise vectors to generate fashion items, (ii) novel loss functions that encourage novelty, inspired from Sharma-Mittal divergence, a generalized mutual information measure for the widely used relative entropies such as Kullback-Leibler, and (iii) a generation process following the key elements of fashion design (disentangling shape and texture components). A key challenge of this study is the evaluation of generated designs and the retrieval of best ones, hence we put together an evaluation protocol associating automatic metrics and human experimental studies that we hope will help ease future research. We show that our proposed creativity criterion yield better overall appreciation than the one employed in Creative Adversarial Networks. In the end, about 61% of our images are thought to be created by human designers rather than by a computer while also being considered original per our human subject experiments, and our proposed loss scores the highest compared to existing losses in both novelty and likability.
1. Introduction
The paper explores fashion generation as a test of whether AI can produce original designs while respecting the realism and wearability constraints of clothing. It develops shape- and texture-aware GANs and creativity losses, then evaluates their outputs with automatic metrics and human studies.
1. Introduction
- The paper motivates creativity as a balance: increasing originality can raise appreciation until novelty becomes too great to understand or find appealing.It describes this inverted-U relation between novelty and likability as the Wundt curve.Source: §1 · pp. 1–2
- The study introduces fashion GANs that encourage deviation from existing shapes, textures, or both, and frames creativity through fashion’s design elements.Its contributions include a creativity loss conditioned on texture and shape, a multi-class cross-entropy criterion linked to Sharma-Mittal divergence, adapted automatic evaluation metrics, and a stochastic shape-conditioned StyleGAN.Source: §1 · pp. 2–3, Table 1 · p. 3
- Fashion is a demanding creativity domain because original garments must remain wearable and satisfy clothing standards such as dimensions.The paper contrasts this balance with Creative Adversarial Networks, which encourage deviation from known art styles but do not judge how enhanced creativity mixes with realism and standards (Elgammal et al. (2017)).Source: §1 · p. 2
- The evaluation jointly assesses novelty, likeness, and realism using adapted automatic metrics and human-subject experiments.The authors also examine correlations between the automatic measures and human evaluation.Source: §1 · p. 3
- More than 60% of generated designs were judged human-created while also being considered original; the best models generated realistic 512 × 512 images.The authors suggest such designs could make AI an efficient inspirational assistant.Source: §1 · p. 3
2. Related work
Related work traces progress in GAN image generation and describes prior methods for controlling clothing appearance. The paper distinguishes its focus on creative generation from approaches that condition outputs on existing images or other inputs.
2. Related work
- GAN image generation progressed from 32 × 32 images to higher resolutions through cascaded convolutional networks, deeper architectures, auxiliary class labels, text inputs, and progressive growing.The cited examples include 128 × 128 images with auxiliary classifiers, 256 × 256 photo-realistic images with text inputs, and 1024 × 1024 results with progressive growing.Source: §2 · pp. 3–4
- Prior clothing-generation approaches applied styles to existing images, started from binary edge images, matched input texture patches, or conditioned full-body outputs on text, color, or pose.The cited approaches include neural style transfer (Johnson et al., 2016), edge-conditioned handbag generation (Isola et al., 2017), texture-patch guidance (Xian et al., 2018), and full-body generation (Zhu et al., 2017b; Lassner et al., 2017).Source: §2 · p. 4
- This paper instead explores generative-model creativity using random vectors or shape masks as inputs, leaving freedom to explore GAN generation.The related-work passage contrasts this focus with methods whose outputs are controlled by input images, texture patches, text, color, or pose.Source: §2 · p. 4
3. Models: architectures and losses
The section explores GAN architectures and creativity losses for fashion generation, including a mask-conditioned model that separates garment shape from texture. Its losses encourage generated items to deviate from existing shape or texture classes.
3.1 GANs
- The generator learns to produce examples classified as real while the discriminator learns to distinguish real samples from generated ones.The models draw latent vectors from a normal distribution; the stated vector dimension is 100.Source: §3.1 · p. 4, §3 · p. 4
3.2 GANs with classification loss
- Shape and texture classification branches are added to the discriminator, with classification loss applied to one or both branches.The authors say these labels improve on the plain model and stabilize training at larger resolutions.Source: §3.2 · pp. 4–5
3.3 Creativity losses
- MCE makes generated samples harder to classify by comparing discriminator predictions with a uniform target; it uses class information jointly and requires less compute than CAN.The authors describe MCE as equivalent to a KL loss between the uniform distribution and the softmax output.Source: §3.3 · pp. 5–6
- The authors generalize the creativity loss to Sharma–Mittal divergence, of which MCE/KL, Tsallis, Rényi, and Bhattacharyya losses are special cases.They report that KL and Bhattacharyya worked best among the tested losses and focus on them in experiments.Source: §3.3 · pp. 6–7
3.4 Network architectures
- The authors compare modified DCGAN, unconditional StackGAN, and mask-conditioned StyleGAN architectures; their adapted StackGAN did not perform well for this application.The StackGAN adaptation replaces nearest-neighbor upsampling with transposed convolutions.Source: §3.4 · p. 7
- StyleGAN takes a segmented garment mask and style noise as inputs; an added L1 reconstruction loss preserves the mask at zero noise, encouraging noise to encode texture variation.The mask is obtained using the graph-based random walker algorithm because simple thresholding does not extract accurate masks.Source: §3.4 · pp. 7–8, Figure 3 · p. 8
4. Experiments
The experiments compare fashion datasets, automatic metrics, model outputs, and human judgments to identify the strongest generations.
4.1 Datasets
- The experiments use white-background RTW and handbag images, with fivefold augmentation by random scaling and translation.The RTW set has 4,157 images; the bag set has 5,783 images selected for white backgrounds.Source: §4.1 · pp. 8–9
4.2 Automatic evaluation metrics
- The ten automatic metrics adapt classifier-based shape and texture scores and AM scores, and add nearest-neighbor distance, classifier confusion, darkness, average intensity, and skewness.Nearest-neighbor distance uses k=10 and 512-dimensional ResNet-18 features; a high value can indicate novelty or artifacts.Source: §4.2 · pp. 9–10
4.3 Automatic evaluation results
- On RTW, DCGAN with MCE and Style CAN with texture creativity scored highly on AM texture and NN distance; on handbags, DCGAN with MCE texture or shape-and-texture led overall.High NN distance may indicate either creativity or a higher failure rate; complex-print differences were amplified on RTW.Source: §4.3 · pp. 11–12, Table 2 · p. 12
4.4 Human evaluation results
- The main evaluation used 800 images per model, each rated by five people; a lab study of 500 generations and 300 down-sampled RTW images found 20% judged real and 21.5% of originals judged generated.The generations were randomly selected from five best models; about 45 participants rated 20 images each on average.Source: §4.4 · pp. 13–14
- More than 75% likability for the best shape-creativity model versus 64% for classical GANs; MCE shape and texture also outperform the DCGAN baseline.The lowest-NN-distance set reached an average overall score of 4.16 for its best model.Source: Figure 7 · p. 14, §4.4 · p. 14
- MCE and Sharma-Mittal models balanced novelty and likability, while the classical GAN was most novel but less appreciated; Figure 8 plots likability against NN-distance novelty.Novelty was difficult for raters to judge directly, so the diagram uses average distance from training images.Source: Figure 8 · p. 15, §4.4 · p. 15
- Classifier confidence, inception texture score, and average intensity correlated most with overall likability; skewness tracked shape and texture complexity, while darkness best correlated with real appearance.The PCA’s first two components were mostly explained by darkness and average intensity, although ratings were mixed across projections.Source: Table 3 · p. 15, §4.4 · pp. 15–16, Figure 10 · p. 16
5. Conclusion
The paper conditions GANs on fashion shape and texture, extends creativity losses, and introduces mask-conditioned shape control. Its contributions achieve the best human-evaluation results, while higher resolution, automatic evaluation, and GAN stability remain open issues.
5. Conclusion
- Fashion GANs conditioned on shape and texture use creativity criteria to move beyond training-set reconstruction, while a Sharma-Mittal generalization of CAN produces more popular results.The proposed StyleGAN takes an input mask to control shape while leaving the item’s interior open to creative variation.Source: §5 · p. 16
- The paper’s contributions yield its best results in human evaluation, and its models accurately generate 512 × 512 images.The authors seek better resolution and identify automatic generation evaluation and GAN stability as open research problems.Source: §5 · p. 17
6. Appendix: Network architectures
The appendix details two 256 × 256 generation architectures: an unconditional, two-stage StackGAN and a StyleGAN combining mask and style-noise inputs.
6.1 Unconditional StackGAN
- The unconditional StackGAN uses a noise-driven DCGAN low-resolution stage followed by a high-resolution generator that upsamples a 64×64 image to 256×256.The high-resolution generator adapts a fast style-transfer model and uses convolutions for downsampling and transposed convolutions for upsampling.Source: §6.1 · pp. 19–20, Figure 11 · p. 20, Table 4 · p. 20
6.2 StyleGAN architecture
- StyleGAN upsamples style noise and downsamples an input mask, then concatenates the resulting tensors and applies convolutional layers to generate a 256 × 256 image.The architecture combines mask and style-noise inputs; its convolutional layers use batch normalization except for the last layer.Source: §6.2 · p. 20, Table 5 · p. 21