Source-linked AI summary
Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lee, Jiwon Kim
TL;DR
The paper addresses how to discover relations between different data domains when paired examples are unavailable or costly to obtain. It introduces DiscoGAN, which couples GANs and reconstruction losses to learn bidirectional mappings from unpaired datasets. The model supports cross-domain translation with transferred styles and preserved attributes, although its formulation assumes bijective relations and baseline reconstruction variants can still suffer mode collapse.
Problem
Cross-domain image generation commonly relies on explicitly paired data, although supervision is costly and corresponding images may be unavailable or ambiguous.
Method
DiscoGAN couples two GAN-based mappings with reconstruction losses so unpaired, unlabeled domains are constrained in both directions.
Results
DiscoGAN discovers relations across toy and real-world domains, translates images while preserving non-target attributes, and transfers style between visually different domains.
Takeaways & Limitations
The approach can generate high-quality cross-domain images and discover shared properties or styles without explicit pair labels.
Takeaways & Limitations
The formulation constrains relations to bijective mappings, while a reconstruction-loss baseline remains vulnerable to mode collapse because it lacks the reverse-direction constraint.
Abstract
from arXiv · showhide
While humans easily recognize relations between data from different domains without any supervision, learning to automatically discover them is in general very challenging and needs many ground-truth pairs that illustrate the relations. To avoid costly pairing, we address the task of discovering cross-domain relations given unpaired data. We propose a method based on generative adversarial networks that learns to discover relations between different domains (DiscoGAN). Using the discovered relations, our proposed network successfully transfers style from one domain to another while preserving key attributes such as orientation and face identity. Source code for official implementation is publicly available https://github.com/SKTBrain/DiscoGAN
1. Introduction
The paper frames cross-domain relation discovery as conditional image generation without paired supervision and introduces DiscoGAN to learn bidirectional mappings from unpaired datasets. Experiments report robust relation discovery, attribute-preserving translation, and style transfer across varied domains.
- Cross-domain relation discovery is reformulated as generating an image in one domain conditioned on an image from another.
- Paired training data are often required, but labels are labor intensive and image pairing can be ambiguous or unavailable.
- DiscoGAN learns cross-domain mappings from two independently collected image sets without explicit pair labels or pre-training.
- The model couples two GANs with reconstruction losses to constrain mappings in both directions and encourage one-to-one coverage between domains.
- Toy and real-world experiments found DiscoGAN more robust to mode collapse than two baselines while translating faces, cars, chairs, edges, and photos.
- Translations changed attributes such as hair color, gender, and orientation while maintaining other components, and transferred fashion style between handbags and shoes.
2. Model
The paper formulates cross-domain relation discovery as learning bidirectional mappings between unpaired image domains. DiscoGAN couples two GAN-based translation models with reconstruction constraints so the learned relation can be bijective and preserve domain correspondences.
- 2.1. Formulation: Cross-domain relations are modeled as functions GAB and GBA between domains, constrained to be inverse mappings that establish a one-to-one correspondence.The range of each mapping must remain within its target domain.
- 2.1. Formulation: The task combines reconstruction constraints with adversarial losses so translated samples remain reconstructable and appear to belong to the target domain.The reconstruction terms minimize distances after two successive translations, while GAN losses enforce target-domain membership.
- 2.3. GAN with a Reconstruction Loss: A standard GAN with one reverse generator and reconstruction loss remains one-directional, yielding an injection rather than a guaranteed bijection.The two relaxed conditions do not constrain the mapping from domain B back to domain A.
- 2.3. GAN with a Reconstruction Loss: Adding reconstruction loss alone does not resolve mode collapse: multiple source modes can map to one target mode, with reconstruction producing a symmetric oscillation.The simplified one-dimensional illustration contrasts ideal mode matching with failures of standard GAN and GAN-with-reconstruction models.
- 2.4. Our Proposed Model: Discovery GAN: DiscoGAN couples two models trained simultaneously, shares generator parameters, reconstructs inputs from both domains, and uses separate discriminators for domains A and B.Its two reconstruction losses, LCONSTA and LCONSTB, complement the adversarial losses for both translation directions.
- 2.4. Our Proposed Model: Discovery GAN: With two GAN losses and two reconstruction losses, DiscoGAN achieves a bijective mapping that can discover the paper’s defined cross-domain relation.The total generator and discriminator objectives sum the corresponding losses across the two partial models.
3. Experiments
Toy and real-image experiments show that DiscoGAN discovers cross-domain relations from unpaired data while reducing mode collapse and preserving or transforming relevant visual attributes.
- 3.1. Toy Experiment: The toy-domain comparison found less mode collapse with reconstruction loss than with standard GAN, but residual overlap remained.The standard GAN mapped multiple source modes to the same target mode, while the reconstruction-loss variant reduced but did not eliminate this behavior.
- 3.1. Toy Experiment: In a synthetic experiment, DiscoGAN translated five source modes into distinct regions covering all ten target modes, unlike the baseline variants.Standard GAN and reconstruction-loss GAN mappings overlapped modes and failed to cover every target mode, whereas DiscoGAN produced non-overlapping regions and fooled the target discriminator.
- 3.2.1. CAR TO CAR, FACE TO FACE: In car-to-car translation, DiscoGAN produced strongly correlated input and output azimuths, with translations preserving either the same or the opposite azimuth range.The opposite-range behavior corresponds to mirror-image translations around 0°.
- 3.2.1. CAR TO CAR, FACE TO FACE: For face translation, DiscoGAN generated outputs spanning the opposite rotation range, while both comparison models showed limited rotation variation and mode collapse.Input faces ranged from -90° to +90°, and DiscoGAN produced outputs from +90° to -90°.
- 3.2.2. FACE CONVERSION: On face attributes, DiscoGAN changed targeted properties such as gender, hair color, and eyeglasses while generally preserving facial features and background.The experiments also applied gender and hair-color conversions sequentially and repeatedly.
- 3.2.3. CHAIR TO CAR, CAR TO FACE: Across chair-to-car and car-to-face translation, DiscoGAN matched input orientation while preserving domain-specific visual characteristics.The model also generated realistic handbag and shoe photos from edge images and discovered fashion style between visually different handbag and shoe domains.
4. Related Work
Prior work developed GAN architectures, conditioning strategies, coupled models, attribute controls, and image-to-image translation methods for generating or relating images across domains.
- GAN foundations: GANs use a generator and discriminator trained simultaneously in a two-player minimax game.The generator matches real-data distributions, while the discriminator distinguishes real from generated samples.
- Conditional generation: Conditional GAN methods generate images using additional information such as class labels, characteristic codes, or conditional features.Examples include specified MNIST classes and object characteristics such as color and viewpoint.
- Multiple GANs: Coupled GANs learn joint image distributions across two domains without pairwise data by coupling generators and discriminators through weight sharing.Stacked GANs instead arrange two GANs sequentially, with the first generating low-resolution images from text descriptions.
- Attribute control: Other approaches disentangle controllable image factors or condition generation on explicit attribute vectors.These methods use target codes, attribute predictors, and latent encoders to control specific image properties.
- Image translation: Image-to-image translation methods convert images between domains using paired data or pretrained recognition features as conditional inputs.Examples include black-and-white-to-color translation and face-to-emoji translation.
- Training stability: GAN research also addressed mode collapse and training instability through geometric-metric and mode regularizers.These regularizers modify the general GAN objective.
5. Conclusion
The paper presents DiscoGAN, a GAN-based method for discovering relations between very different domains without explicit pair labels. It generates high-quality images with transferred style and identifies mixed-modality handling as a future direction.
- Conclusion: DiscoGAN discovers cross-domain relations between datasets from very different domains without explicit pair labels.The method is presented as a generative adversarial network for learning these relations.
- Conclusion: DiscoGAN generates high-quality images with transferred style.The conclusion reports this capability as demonstrated by the paper.
- Future direction: Handling mixed modalities such as text and image is identified as a possible future direction.This extends the stated scope beyond the paper’s demonstrated cross-domain image settings.