PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Style Mining for One-Shot Unsupervised Domain Adaptation

Yawei Luo, Ping Liu, Tao Guan, Junqing Yu, Yi Yang

arXiv:2004.06042v1cs.CVcs.LGeess.IV

TL;DR

One-shot UDA must adapt using only one unlabeled target sample, a setting in which conventional approaches can be prone to failure. ASM adversarially couples style transfer and the task model to mine stylized samples, and the paper reports state-of-the-art performance in one-shot classification and segmentation.

  • Problem

    One-shot UDA asks how to adapt with only one unlabeled target sample, while many prior approaches assume more target data and may fail in data-scarce settings.

    Source: §3.1 · p. 3

  • Method

    ASM iteratively mines harder stylized images around the lone target sample by updating a style generator and task model through adversarial feedback.

    Source: §1 · p. 2

  • Results

    ASM achieves state-of-the-art performance compared with other domain-adaptation approaches in one-shot classification and segmentation.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    ASM searches for new stylized samples beyond the single target example to help the task model adapt to an almost unseen target domain.

    Source: §5 · p. 8

    The OSUDA training setting provides source data with labels but only one unlabeled target datum.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

We aim at the problem named One-Shot Unsupervised Domain Adaptation. Unlike traditional Unsupervised Domain Adaptation, it assumes that only one unlabeled target sample can be available when learning to adapt. This setting is realistic but more challenging, in which conventional adaptation approaches are prone to failure due to the scarce of unlabeled target data. To this end, we propose a novel Adversarial Style Mining approach, which combines the style transfer module and task-specific module into an adversarial manner. Specifically, the style transfer module iteratively searches for harder stylized images around the one-shot target sample according to the current learning state, leading the task model to explore the potential styles that are difficult to solve in the almost unseen target domain, thus boosting the adaptation performance in a data-scarce scenario. The adversarial learning framework makes the style transfer module and task-specific module benefit each other during the competition. Extensive experiments on both cross-domain classification and segmentation benchmarks verify that ASM achieves state-of-the-art adaptation performance under the challenging one-shot setting.

1. Introduction

One-shot UDA is challenging because target data may be scarce, while conventional adaptation and style-transfer approaches can fail or overfit. ASM couples style generation with task-model feedback to mine harder stylized samples, and the paper reports superior performance across one-shot classification and segmentation.

1. Introduction

  • The paper motivates domain adaptation by the distribution shift between real and simulated data, which complicates transferring source-trained models to target tasks.Simulated data can provide automatic annotations, but the domains differ in distribution.Source: §1 · p. 1
  • With only one unlabeled target sample, conventional adaptation strategies are prone to failure, and style-transfer modules may overfit or miss the target style distribution.Sequential, decoupled style transfer and adaptation also lack dynamic classifier feedback, so generated samples may be too easy or too hard for the model.Source: §1 · pp. 1–2
  • ASM iteratively updates the target-initialized style vector and task model in an adversarial regime, mining harder stylized images for the model's current learning state.The model trains on generated stylized images and returns feedback to adjust the style vector.Source: §1 · p. 2
  • RAIN makes style search differentiable, enabling end-to-end search with gradient back-propagation.It is proposed as a key style-transfer component for ASM.Source: §1 · p. 2
  • ASM consistently achieves superior performance over previous UDA and one-shot style-transfer approaches on cross-domain classification and semantic segmentation in one-shot settings.The introduction reports these comparisons across both task types.Source: §1 · p. 2

2. Related Work

Prior domain-adaptation work addresses distribution gaps through feature, image, or output alignment, while style-transfer work changes low-level appearance and can serve as data augmentation. RAIN builds on AdaIN with a latent VAE so styles can be sampled and searched differentiably.

2.1. Domain Adaptation

  • Domain-adaptation approaches include latent-feature alignment, image translation, joint image-and-feature alignment, output-space alignment, entropy minimization, and confident target-label training.The cited examples include adversarial matching of marginal distributions and output-space alignment for semantic segmentation.Source: §2.1 · p. 2

2.2. Style Transfer

  • Style transfer alters low-level appearance while preserving semantic content; prior work includes AdaIN, GAN-based methods, one-shot techniques, and style-transfer data augmentation.Prior augmentation approaches generally treat style transfer as a separate module without interaction with the downstream task [21].Source: §2.2 · p. 2
  • RAIN encodes AdaIN style statistics—feature mean and variation—into a latent distribution, then samples styles or perturbs a sampled vector for differentiable search.This enables end-to-end style search through gradient back-propagation.Source: Figure 2 · p. 3

3. Method

ASM uses the sole unlabeled target image as an anchor for generating stylized source examples, then uses feedback between the generator and task model to mine harder styles.

3.1. Problem Settings and Overall Idea

  • With source images and labels but only one unlabeled target sample, ASM treats that sample’s style as an anchor for generating varied styles to train the task model.Slight noise may not avoid overfitting, while purely random styles may be unrealistic or too far from the target distribution.Source: §3.1 · p. 3

3.2. Random AdaIN

  • RAIN combines AdaIN with a style VAE that encodes style statistics into a Gaussian distribution and decodes samples to reconstruct style vectors.AdaIN aligns content features with a selected style image’s channel-wise mean and standard deviation.Source: §3.2 · pp. 3–4

3.3. Adversarial Style Mining

  • With pretrained RAIN held fixed, ASM initializes ε from the target-derived distribution and perturbs it along the task-loss gradient to find harder styles for M.The updated task model learns from these more arduous samples, with the aim of improving performance on unseen target-domain samples.Source: §3.3 · p. 4, Figure 3 · p. 5

3.4. Cost Function

  • M combines task loss on source or stylized source data with a consistency loss that keeps features similar across same-content images with different styles.The consistency term operates on second-last-layer features and encourages semantic consistency across a mini-batch.Source: §3.4 · p. 4

4. Experiments

Experiments evaluate ASM on cross-domain classification and segmentation, with one-shot target data as the central setting. ASM achieves strong reported results, while analysis examines its style search, training behavior, and sampling strategy.

4.1. Datasets and Evaluation Protocols

  • Classification experiments use M→S, U→S, and M→U with accuracy, while segmentation evaluates SYNTHIA or GTA5→Cityscapes with IoU.Cityscapes is the one-shot target domain for segmentation; the classification setup uses all source data for M→S and U→S, and 2,000 MNIST images for M→U.Source: §4.1 · p. 5

4.2. Implementation Details

  • Training first fits RAIN with source and style images, then fixes RAIN while training the task model within ASM.Classification uses ResNet-18; segmentation uses a ResNet-101-based DeepLab-v2 segmentor.Source: §4.2 · pp. 5–6, §4.1 · p. 5

4.3. Image Classification

  • 46.3% and 40.3% classification accuracy on M→S and U→S, respectively, are state-of-the-art under one-shot UDA; style-transfer methods fall short on M→U.The paper attributes the M→U limitation to content rather than style differences.Source: §4.3 · p. 6, Figure 4 · p. 6

4.4. Semantic Segmentation

  • In conventional GTA5→Cityscapes UDA, ASM reaches 45.5% mIoU, on par with other methods and slightly above its one-shot result.Other conventional UDA strategies improve the source-only baseline by at least 5% mIoU.Source: §4.4 · p. 6, Table 2 · p. 7
  • 44.5% mIoU sets a new benchmark for one-shot GTA5→Cityscapes segmentation, outperforming sequential style-transfer methods such as OST and CycleGAN.The comparison reports an advantage for ASM's adversarial scheme over sequential style transfer and segmentation.Source: §4.4 · p. 7, Table 2 · p. 7
  • 45.5%→44.5% mIoU as target data shrinks from conventional UDA to one-shot UDA, versus approximately 44.0%→37.0% for competing methods.The same qualitative observation is reported for SYNTHIA→Cityscapes.Source: §4.4 · p. 7, Table 2 · p. 7, Table 3 · p. 7

4.5. Analysis of the proposed method

  • ASM's loss rises within each mining iteration but converges to a small value by training's end, while mined styles explore unseen styles around the anchor.The authors interpret the local and global loss behavior as evidence of an effective and stable adversarial course.Source: §4.5 · pp. 7–8, Figure 7 · p. 8
  • ASM's sampling strategy outperforms anchored and random sampling by around 2% mIoU on GTA5→Cityscapes.Anchored sampling yields similar images near the target sample, while random sampling can produce styles unhelpful for adaptation.Source: §4.5 · p. 8, Table 4 · p. 8, Figure 6 · p. 8

5. Conclusion

ASM addresses UDA with scarce target data by pairing style transfer and a task model adversarially, iteratively searching for stylized samples to support adaptation to the almost unseen target domain. The paper reports state-of-the-art one-shot results across cross-domain classification and segmentation.

  • ASM combines style transfer and a task model adversarially, iteratively searching for stylized samples to help the task model adapt to the almost unseen target domain.Source: §5 · p. 8
  • ASM yields state-of-the-art performance compared with other domain adaptation approaches in the one-shot scenario.Source: §5 · p. 9
  • The reported experiments cover both cross-domain classification and segmentation tasks.Source: §5 · p. 9
Loading 2004.06042v1…