Source-linked AI summary
DLOW: Domain Flow for Adaptation and Generalization
Rui Gong, Wen Li, Yuhua Chen, Luc Van Gool
TL;DR
Domain shift leaves models trained on source data mismatched to target distributions. DLOW generates controllable intermediate-domain images with a domainness variable, improving cross-domain segmentation and producing styles unseen during training.
Problem
Models trained on source distributions can suffer performance drops on mismatched target distributions, motivating adaptation and generalization across domains.
Method
DLOW extends image translation with a continuous domainness variable that controls source-image translation through intermediate domains between source and target.
Results
DLOW improves pixel-level domain adaptation for cross-domain semantic segmentation and generates new, unseen styles for image translation on benchmark datasets.
Takeaways & Limitations
Intermediate-domain images can ease domain adaptation, while controllable flows support style generalization across multiple target domains.
Takeaways & Limitations
The proposed flow assumes a domain manifold and seeks a shortest geodesic path with proportional source-to-intermediate distance.
Abstract
from arXiv · showhide
In this work, we present a domain flow generation(DLOW) model to bridge two different domains by generating a continuous sequence of intermediate domains flowing from one domain to the other. The benefits of our DLOW model are two-fold. First, it is able to transfer source images into different styles in the intermediate domains. The transferred images smoothly bridge the gap between source and target domains, thus easing the domain adaptation task. Second, when multiple target domains are provided for training, our DLOW model is also able to generate new styles of images that are unseen in the training data. We implement our DLOW model based on CycleGAN. A domainness variable is introduced to guide the model to generate the desired intermediate domain images. In the inference phase, a flow of various styles of images can be obtained by varying the domainness variable. We demonstrate the effectiveness of our model for both cross-domain semantic segmentation and the style generalization tasks on benchmark datasets. Our implementation is available at https://github.com/ETHRuiGong/DLOW.
1. Introduction
DLOW generates a continuous flow of intermediate domains between source and target, supporting domain adaptation and style generalization beyond fixed one-to-one translation.
- Domain adaptation transfers models from label-rich source domains to label-scarce target domains, while image translation transfers source images into target styles.
- DLOW translates source images into arbitrary intermediate domains, producing a sequence that characterizes the distribution shift toward the target.
- Intermediate-domain images bridge source–target distribution gaps and can ease domain adaptation, improving target-domain performance and generalization to unseen datasets.
- Unlike deterministic one-to-one translation, DLOW combines multiple target styles into customizable intermediate styles controlled by an input vector.
- DLOW extends CycleGAN with a domainness variable that affects output style through conditional instance normalization and balances discriminator weights across domains.
2. Related Work
Related work addresses domain shift through image translation, feature adaptation, domain generalization, and intermediate-domain modeling. DLOW differs by directly translating images along a controllable pixel-level flow between domains.
- Image to Image Translation: Image-to-image translation maps images between domains, evolving from paired-image methods to unpaired approaches such as CycleGAN, DiscoGAN, and UNIT.
- Image to Image Translation: Multimodal translation methods model intra-domain diversity or use discrete domain labels, whereas DLOW targets intermediate domains and derives domainness from distribution distance.
- Domain Adaptation and Generalization: Domain adaptation uses labeled source data to perform well on unlabeled target data, while domain generalization uses multiple labeled sources for unseen targets.
- Domain Adaptation and Generalization: Prior work models intermediate domains as subspaces or covariance manifolds; DLOW instead translates images directly at the pixel level.
- Domain Adaptation and Generalization: Semantic-segmentation adaptation commonly uses feature-level adversarial training or target-style transfer, while DLOW transfers source images through intermediate domains and augments feature-level methods.
3. Domain Flow Generation
DLOW models domain shift as a continuous flow of intermediate image domains between source and target, rather than translating only to the target. Built on CycleGAN, it uses domainness to control translation and supports adaptation and style generalization.
- Problem: Domain adaptation addresses performance degradation caused by applying models trained on a label-rich source distribution to a label-scarce target distribution.
- Model: DLOW augments CycleGAN with domainness input to the generator and uses two discriminators to regulate relatedness to the source and target domains.The adversarial losses measure distances between the intermediate distribution and each endpoint distribution.
- Intermediate domains: DLOW represents intermediate domains M(z) with domainness z∈[0,1], where z=0 is the source, z=1 is the target, and varying z produces a sequence between them.
- Intermediate domains: The model seeks a shortest domain path while keeping each intermediate domain’s distance from the source proportional to its position between source and target.
- Applications: For adaptation, translated images sampled across the domain flow replace the source dataset, producing diverse training data for cross-domain semantic segmentation.The translated dataset spreads along the flow from source to target and is used to learn domain-invariant models.
- Applications: With multiple target domains, DLOW can generate mixtures of target styles and unseen intermediate styles by varying the domainness input.The mixture can be customized using an input vector encoding relatedness to different domains.
4. Experiments
Experiments evaluate DLOW for cross-domain semantic segmentation and style generalization. Intermediate-domain images gradually bridge GTA5 and Cityscapes, improve adaptation, generalize to unseen domains, and support mixtures of painting styles.
- Cross-Domain Semantic Segmentation: The translated GTA5 dataset preserves the original image count while randomly drifting styles from synthetic to real for segmentation training.Each source image is processed with a random domainness variable.
- Intermediate Domain Images: DLOW generates GTA5-to-Cityscapes images whose styles gradually shift from synthetic to realistic as domainness increases from 0 to 1.The source image is fixed while the domainness variable is varied during inference.
- Cross-Domain Semantic Segmentation: 40.7% to 42.3%: intermediate-domain images improve pixel-level adaptation over DLOW(z = 1), while DLOW(z = 1) is comparable to CycADA-pixel at 40.7% versus 41.0%.The metric is semantic-segmentation performance reported in Table 1.
- Cross-Domain Semantic Segmentation: 42.3% to 44.8%: intermediate-domain images improve AdaptSegNet by 2.5% under GTA5-to-Cityscapes domain adaptation.The same translated images also yield significantly better results than the original model on KITTI, WildDash, and BDD100K.
- Style Generalization: With four target painting domains, varying the domainness vector produces corresponding artworks and new styles related to different painting styles without a test-time reference image.The target domains are Monet, Van Gogh, Ukiyo-e, and Cezanne.
- Style Generalization: In the user study, more users preferred DLOW translations than FadNet and MUNIT for both style transfer and style generalization.The study compared Van Gogh transfer and mixed Van Gogh–Ukiyo-e generalization.
5. Conclusion
The conclusion presents DLOW as a domain-flow model controlled by a domainness variable or vector. Experiments support its benefits for cross-domain segmentation and generating unseen image styles.
- Conclusion: DLOW transfers images into intermediate domains controlled by a domainness variable or vector.The conclusion frames this mechanism as bridging different domains.
- Conclusion: User preferences favored DLOW translations over FadNet and MUNIT for both style transfer and style generalization.This comparison is reported in the user-preference results.
- Conclusion: For cross-domain semantic segmentation, DLOW improves pixel-level adaptation when translated intermediate-domain images are used as training data.The conclusion reports this as the first demonstrated benefit.
- Conclusion: DLOW exhibits style generalization by transferring images into styles unseen during training.The conclusion states that extensive benchmark experiments verify the model’s effectiveness.
6. Supplementary
The supplementary material adds domain adaptation, domain generalization, style generalization, network-structure, and qualitative-comparison results for DLOW.
- Comparison with brightness-adjusted images: Brightness adjustment alone retains game-style textures, whereas DLOW also modifies subtle textures to mimic Cityscapes.The comparison highlights high-contrast road textures and a red curb remaining after brightness adjustment.
- Additional domain adaptation and generalization results: On SYNTHIA-to-Cityscapes segmentation, DLOW improves adaptation from 41.6% to 42.8% with intermediate-domain images.DLOW(z = 1) reaches 41.6%, while adding intermediate-domain images raises performance to 42.8%.
- Additional domain adaptation and generalization results: Combined with AdaptSegNet, DLOW improves adaptation from 45.7% to 47.1% compared with the retrained AdaptSegNet model.The table reports mIoU over 13 categories for adaptation and generalization scenarios.
- Network structure and additional examples: The supplementary material also documents the four-target-domain network structure and provides additional style-generalization examples.The network structure covers both S → T and T → S directions.
- Additional style-generalization results: With four target styles, DLOW produces mixed styles between neighboring domains and a central mixture of all four styles unseen during training.The target styles are Monet, Van Gogh, Cezanne, and Ukiyo-e; one unified model handles them.
- Qualitative comparison for style transfer and generalization: In qualitative comparisons, DLOW captures brushwork and lines in Van Gogh transfer and combines color and stroke across Van Gogh and Ukiyo-e styles.MUNIT mainly changes dominant colors, while FadNet fails to translate the photo to painting in the described comparison.