Source-linked AI summary
Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping
Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, Kun Zhang, Dacheng Tao
TL;DR
Unpaired domain mapping must infer meaningful correspondences from marginal distributions, making suitable constraints necessary. GcGAN uses paired original/transformed inputs and geometry-consistent translators for one-sided mapping. The method produces competitive and sometimes better translations than state-of-the-art methods, with reported gains over CycleGAN and DistanceGAN in quantitative evaluations.
Problem
Unpaired marginal distributions permit many joint distributions, so meaningful correspondences between individual inputs and outputs cannot be guaranteed without additional constraints.
Method
GcGAN applies a predefined geometric transformation to create transformed domains and co-regularizes translators by requiring corresponding outputs to preserve that transformation.
Results
GcGAN achieves competitive and sometimes better translations than state-of-the-art methods, with both variants outperforming DistanceGAN and CycleGAN by about 6%–7% in classification accuracy.
Takeaways & Limitations
Geometry consistency enables one-sided unsupervised domain mapping while reducing semantic distortions and addressing mode collapse in standard GANs.
Takeaways & Limitations
Preferable results for specific translation tasks may require heavily tuning λ.
Abstract
from arXiv · showhide
Unsupervised domain mapping aims to learn a function to translate domain X to Y by a function GXY in the absence of paired examples. Finding the optimal GXY without paired data is an ill-posed problem, so appropriate constraints are required to obtain reasonable solutions. One of the most prominent constraints is cycle consistency, which enforces the translated image by GXY to be translated back to the input image by an inverse mapping GYX. While cycle consistency requires the simultaneous training of GXY and GY X, recent studies have shown that one-sided domain mapping can be achieved by preserving pairwise distances between images. Although cycle consistency and distance preservation successfully constrain the solution space, they overlook the special properties that simple geometric transformations do not change the semantic structure of images. Based on this special property, we develop a geometry-consistent generative adversarial network (GcGAN), which enables one-sided unsupervised domain mapping. GcGAN takes the original image and its counterpart image transformed by a predefined geometric transformation as inputs and generates two images in the new domain coupled with the corresponding geometry-consistency constraint. The geometry-consistency constraint reduces the space of possible solutions while keep the correct solutions in the search space. Quantitative and qualitative comparisons with the baseline (GAN alone) and the state-of-the-art methods including CycleGAN and DistanceGAN demonstrate the effectiveness of our method.
1 Introduction
Unsupervised domain mapping is difficult because unpaired marginal distributions do not determine meaningful input-output correspondences. GcGAN addresses this by enforcing consistency under predefined geometric transformations, enabling one-sided mapping while reducing semantic distortions.
- Motivation: Unsupervised domain mapping is practical with unpaired data but lacks meaningful input-output correspondences without additional constraints.Paired examples can be expensive or infeasible to obtain, whereas unpaired image sets are often convenient to collect.
- Existing Constraints: Cycle consistency jointly trains GXY and GYX, whereas distance preservation can enable one-sided mapping.Cycle consistency reconstructs inputs through inverse translators; DistanceGAN preserves distances between images within domains.
- Motivation: Existing constraints overlook that simple global geometric transformations preserve image semantic structure.The paper identifies this property as applying to transformations without shape deformation, such as rotation.
- GcGAN: GcGAN uses geometry consistency to reduce the solution search space while retaining the correct solutions.The constraint compares translations of original and geometrically transformed images.
- GcGAN: GcGAN trains related translators so transformed inputs produce correspondingly transformed outputs, allowing GXY to be trained independently from GYX.The constraint is expressed as f(GXY(x)) ≈ G̃X̃Y(f(x)) and also uses the inverse transformation.
2 Related Work
Domain mapping has been studied with supervised paired data and unsupervised unpaired data, using adversarial constraints and structural assumptions to make translation feasible. CycleGAN, DistanceGAN, and GcGAN represent different unsupervised constraint strategies.
- Unsupervised Domain Mapping: GcGAN differs by comparing translations of original and geometrically transformed domains through geometry consistency.Figure 2 contrasts this constraint with CycleGAN’s reconstruction and DistanceGAN’s distance-based constraint.
- Generative Adversarial Networks: GANs use adversarial constraints to make generated images indistinguishable from real images in the target domain.The formulation applies this adversarial constraint to learn translators between two individual domains.
- Domain Mapping: Domain mapping includes tasks such as scene parsing and colorization and can use paired or unpaired inputs.The unpaired setting is more practical but more difficult because only unaligned examples are available.
- Supervised Domain Mapping: Supervised methods use meaningful cross-domain pairs, but many applications cannot provide aligned images because data preparation is expensive.Conditional GANs distinguish real pairs (x,y) from generated pairs (x,GXY(x)).
- Unsupervised Domain Mapping: Unsupervised approaches have included weight sharing, cycle consistency, and distance preservation to constrain mappings without training pairs.CoGAN, CycleGAN, DiscoGAN, DualGAN, and DistanceGAN exemplify these strategies.
3 Preliminaries
The paper formulates unpaired translation as learning a target-distributed mapping that changes style without distorting semantic structure. GcGAN augments the original and target domains with transformed versions and co-regularizes their translators.
- Problem Setup: Training uses unpaired examples drawn from the marginal distributions PX and PY of domains X and Y.The examples are represented as separate sets from the two domains rather than aligned pairs.
- Problem Setup: The desired mapping satisfies PGXY(X) ≈ PY while changing image style without distorting semantic structures.These are the two stated goals for unsupervised style transfer.
- Geometry-Consistent Framework: A predefined transformation f(·) creates transformed domains X̃ and Ỹ from X and Y.The framework applies the transformation to examples in both domains.
- Geometry-Consistent Framework: GcGAN learns GXY and an additional translator G̃X̃Y while enforcing f(GXY(x)) ≈ G̃X̃Y(x̃), where x̃ = f(x).The two translators are regularized to preserve the same geometric relationship between original and transformed inputs.
4 Proposed Method
The method reviews cycle, distance, and adversarial constraints before introducing geometry consistency for one-sided unsupervised domain mapping. GcGAN applies shared-parameter translators to original and geometrically transformed images, combining adversarial and geometry-consistency losses.
- Existing constraints: Cycle consistency reconstructs inputs through jointly learned inverse translators, whereas distance preservation enables one-sided unsupervised domain mapping.CycleGAN uses bidirectional reconstruction; DistanceGAN preserves within-domain image distances instead.
- GcGAN framework: The qualitative comparison contrasts GAN alone with GcGAN on Cityscapes parsing-image and Google Maps map-aerial translation.The figure reports mode collapse for GAN alone and more detailed translated images for GcGAN.
- Geometry-consistency constraint: GcGAN feeds an image and its transformed counterpart into related translators, requiring their outputs to obey the same geometric transformation.The loss includes both forward and inverse transformation terms, comparing translated transformed images with transformed translations.
- Implementation: The original and transformed translators share the same architecture and all parameters, using vertical flipping or 90° clockwise rotation as transformations.The implementation follows CycleGAN’s generator and discriminator architectures and uses λ = 20.0 to weight adversarial and geometry losses.
- Full objective: The geometry-consistency loss is combined with adversarial losses for the original and transformed domains in the full GcGAN objective.The transformed-domain adversarial loss has the same form as the original-domain adversarial loss.
- Ablation studies: Table 1 reports Cityscapes parsing scores for GcGAN and variants including separate parameters, mixed transformations, and an added cycle constraint.The table caption defines the variants used in the ablation study.
5 Experiments
Experiments compare GcGAN with GAN alone, CycleGAN, and DistanceGAN across quantitative and qualitative domain-mapping tasks. GcGAN generally improves over GAN alone, achieves competitive or better results than prior methods, and reduces mode collapse and semantic distortions.
- Quantitative Analysis: GcGAN outperforms GAN alone by a large margin on Cityscapes parsing metrics.For image →parsing, GcGAN scores 32.6% versus CycleGAN's 32.0%; for parsing →image, it scores 29.0% ∼29.5% versus DistanceGAN's 27.7%.
- Ablation Studies: Ablations indicate that geometry consistency filters candidate solutions with mode collapse or undesired shape deformation and is compatible with cycle consistency.The GcGAN-rot + Cycle variant shows consistent improvement, while adding more preserved transformations does not further improve filtering.
- Quantitative Analysis: GcGAN improves pixel accuracy over GAN alone by 18.0% ∼21.9% for Aerial photo →Map.The authors attribute the improvement to fake maps containing more details, while one-sided GcGAN remains competitive with two-sided CycleGAN.
- Qualitative Evaluation: Across qualitative applications, geometry consistency provides an effective remedy for GAN-alone mode collapse and supports more impressive translations.The evaluated applications include Horse →Zebra, Monet →Photo, Synthetic ⇌Real, Summer ⇌Winter, Photo ⇌Artistic Painting, and Day ⇌Night.
6 Conclusion
The paper proposes GcGAN, which enforces geometry consistency through predefined geometric transformations for one-sided unsupervised domain mapping. Experiments show competitive or sometimes better translations than DistanceGAN and CycleGAN, and the constraint is compatible with other unsupervised constraints.
- Conclusion: GcGAN enforces a predefined geometric transformation as a geometry-consistency constraint for one-sided unsupervised domain mapping.The constraint preserves scene geometry while co-regularizing translation networks on original and transformed images.
- Conclusion: GcGAN achieves competitive and sometimes better translations than DistanceGAN and CycleGAN across qualitative and quantitative evaluations.The model is evaluated in various applications against GAN alone and prior state-of-the-art methods.
- Conclusion: The geometry-consistency constraint is compatible with other well-studied unsupervised constraints.The conclusion specifically identifies compatibility as a property of the proposed constraint.
Network Architecture
The experiments use CycleGAN-style generators and discriminators, with task-specific architectures and geometric transformations across several domain-mapping applications. Qualitative comparisons cover diverse translation settings and include explicit failure cases.
- Network Architecture: The reported generator and discriminator architectures use abbreviations for channels, kernels, strides, deconvolutional and convolutional layers, and residual blocks.The architecture is documented for 256 × 256 experiments, with a separate network architecture listed for SVHN →MNIST.
- Applications: Cityscapes comparisons use GcGAN-rot, while Google Maps comparisons report competitive Map →Aerial photo translations against CycleGAN.The Cityscapes figure concerns Parsing ⇌Image; the Google Maps figure concerns Aerial photo ⇌Map.
- Applications: For SVHN →MNIST, the correct translations are about 24, 26, and 35 for CycleGAN, DistanceGAN, and GcGAN, respectively.The qualitative comparison uses results for all three methods, with GcGAN having the highest reported count.
- Applications: For Horse →Zebra, GcGAN is slightly better for some images but does not generally outperform CycleGAN.Other qualitative applications include Monet →Photo, Synthetic ⇌Real, Summer ⇌Winter, Photo →Artist Painting, and Day ⇌Night.
- Failure Cases: The failure-case figure states that GcGAN cannot guarantee reasonable translations for all cases and calls for additional assumptions and constraints.This marks a scope boundary for the qualitative claims about GcGAN's translation quality.