PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Feature Augmentation for Unsupervised Domain Adaptation

Riccardo Volpi, Pietro Morerio, Silvio Savarese, Vittorio Murino

arXiv:1711.08561v2cs.CV

TL;DR

Unsupervised domain adaptation must classify unlabeled target samples despite domain shift, while prior GAN approaches primarily aligned target and source features. The paper introduces domain-invariant feature extraction with conditional GAN-based feature augmentation, achieving higher accuracies and generally state-of-the-art-comparable or superior benchmark performance, subject to dependence on the source representation.

  • Problem

    Unsupervised domain adaptation seeks to classify unlabeled target samples despite domain shift, while feature-space augmentation has received less attention than image-space augmentation.

  • Method

    The method trains a domain-invariant extractor and a conditional GAN feature generator that produces labeled features from noise and label codes.

  • Results

    The approaches lead to higher target-data accuracies and are comparable or superior to current state-of-the-art methods except on a single benchmark.

  • Takeaways & Limitations

    Feature augmentation and domain-invariance are both valuable approaches for unsupervised domain adaptation, and the learned feature extractors can also be used on source data.

  • Takeaways & Limitations

    Performance depends strongly on the source-trained representation because aligning source and target features does not guarantee correct target-space regions.

Abstract

from arXiv · show

Recent works showed that Generative Adversarial Networks (GANs) can be successfully applied in unsupervised domain adaptation, where, given a labeled source dataset and an unlabeled target dataset, the goal is to train powerful classifiers for the target samples. In particular, it was shown that a GAN objective function can be used to learn target features indistinguishable from the source ones. In this work, we extend this framework by (i) forcing the learned feature extractor to be domain-invariant, and (ii) training it through data augmentation in the feature space, namely performing feature augmentation. While data augmentation in the image space is a well established technique in deep learning, feature augmentation has not yet received the same level of attention. We accomplish it by means of a feature generator trained by playing the GAN minimax game against source features. Results show that both enforcing domain-invariance and performing feature augmentation lead to superior or comparable performance to state-of-the-art results in several unsupervised domain adaptation benchmarks.

1. Introduction

The paper extends GAN-based unsupervised domain adaptation by enforcing domain-invariant features and augmenting data in feature space. Across digit and cross-modal object benchmarks, the method achieves results superior or comparable to current state of the art in most tasks.

  • Unsupervised domain adaptation uses labeled source and unlabeled target datasets separated by domain shift, aiming to classify target samples correctly.
  • The method extends GAN-based adaptation by training a single domain-invariant feature extractor and performing conditional feature augmentation.The feature generator produces labeled feature vectors conditioned on desired classes.
  • Feature augmentation introduces GAN-based data augmentation directly in feature space rather than image space.
  • The proposed method combines feature augmentation with source/target feature domain-invariance for unsupervised domain adaptation.
  • On cross-dataset digit and cross-modal object classification benchmarks, results are superior or comparable to current state of the art in most addressed tasks.

2. Related work

The related work spans GANs, adversarial feature-based domain adaptation, and image-to-image translation. The paper distinguishes its approach by generating class-conditioned features and training a shared encoder for source and target samples.

  • Generative adversarial networks: GANs learn realistic samples through a minimax game between a generator and discriminator, while CGANs condition generation on desired classes.
  • Feature augmentation: The proposed generator creates features from noise vectors and label codes, playing the adversarial game against features from a pre-trained model instead of images.
  • Unsupervised domain adaptation: Prior domain-adaptation methods pursue domain-invariant representations using adversarial training, distribution matching, shared/private representations, or separate source and target encoders.
  • Image-to-image translation: Image-to-image translation methods address adaptation by generating target-style images while preserving content or satisfying task-specific constraints.
  • Domain-invariant encoder: The domain-invariant encoder works for both source and target samples, avoiding catastrophic forgetting and eliminating the need for two different encoders.

3. Model

The model is trained in stages: first a source classifier and feature extractor, then a conditional feature generator, and finally a domain-invariant encoder initialized from the source model. The resulting shared representation supports inference with the source classifier.

  • Step 0: Step 0 trains a source feature extractor and softmax classifier using source samples and cross-entropy loss.The source model supplies a reference feature space and classifier.
  • Step 1: Step 1 trains a feature generator with a conditional GAN to produce feature samples resembling source features.Least Squares GANs are used because they provided more stable training in these minimax games.
  • Feature generation: The generator takes concatenated noise and one-hot label codes and outputs feature vectors from the desired class.Noise is sampled uniformly from [−1, 1] throughout the work.
  • Step 2: Step 2 trains the domain-invariant encoder through a minimax game after initialization with weights optimized in Step 0.
  • Inference: Because the encoder is trained on source and target domains, it maps both into a common feature space and can be combined with the Step 0 classifier for inference.

4. Datasets

The evaluation uses benchmark source/target dataset splits spanning handwritten digits, synthetic-to-real adaptation, and RGB-to-depth modality adaptation. Protocols specify dataset sizes, preprocessing, and test-set choices for each task.

  • Evaluation benchmarks: The study evaluates its approach on several public source/target benchmark splits for domain adaptation.These benchmarks include MNIST↔USPS, SVHN→MNIST, SYN DIGITS→SVHN, and NYUD RGB→D.
  • MNIST ↔USPS: MNIST↔USPS uses two protocols, with P1 sampling 2,000 MNIST and 1,800 USPS images, while P2 uses 50,000 MNIST training images and specified USPS train, test, and validation splits.P1 tests both adaptation directions; P2 tests only MNIST→USPS.
  • MNIST ↔USPS: For MNIST↔USPS, USPS digits are resized to 28 × 28 pixels to match MNIST image size in both protocols.
  • SVHN →MNIST: SVHN→MNIST follows the standard protocol using both datasets’ whole training sets, with testing on MNIST’s test set.MNIST images are resized to 32 × 32 pixels, SVHN is converted to grayscale, and SVHN’s extra set is excluded.
  • SYN DIGITS →SVHN: SYN DIGITS→SVHN evaluates synthetic-to-real adaptation using 500,000 labeled synthetic images and the SVHN test set.SYN DIGITS contains the same classes as SVHN.
  • NYUD (RGB →D): NYUD RGB→D is a modality adaptation task with 2,186 labeled RGB source images and 2,401 unlabeled HHA-encoded depth target images.The source and target images come from different original-dataset splits so the same instance is not seen in both domains.

5. Experiments

The experiments evaluate feature generation, ablations of domain-invariance and feature augmentation, and comparisons with competing unsupervised domain adaptation methods. Generated features preserve class structure and are quantitatively reliable, while domain-invariance and feature augmentation generally improve target accuracy, with dataset-specific exceptions and limitations.

  • Experimental setup: The evaluation covers feature generation, an ablation study, and comparisons with competing algorithms on unsupervised domain adaptation benchmarks.The experiments also assess the variability and classification reliability of generated features.
  • Generating features: Real and generated features appear indistinguishable in t-SNE plots, while class structure is preserved across the considered datasets.The plots compare real versus generated features and separately visualize class assignments for MNIST, SVHN, SYN, and USPS.
  • Generating features: Generated features are quantitatively reliable for all considered datasets when classified with the classifier trained on the original samples.Reliability is measured by the classification accuracy of generated features according to their given labels.
  • Feature augmentation: S generates substantially more activation patterns than appear in training features, including approximately 10^6 patterns for SVHN versus 69,625 observed patterns.For NYUD, only 19 activation patterns occur among 2,186 source samples, while the generator can enrich the feature set broadly.
  • Ablation study: Domain-invariance generally improves performance over LS-ADDA, and feature augmentation adds a further increment, although LS-ADDA does not converge on NYUD.The MNIST → USPS (P1) experiment is an exception to the general improvement trend.
  • Comparisons with other methods: The approach achieves accuracies comparable to or higher than current state-of-the-art methods on several benchmarks, but LS-ADDA performs best on USPS → MNIST.On SVHN → MNIST, comparison with UNIT is complicated by UNIT’s use of an extra SVHN set; on MNIST → USPS (P2), the method performs better or comparably to tested alternatives.
  • Source-data evaluation: The shared encoder performs well on source samples, enabling one encoder for both source and target data, though source performance is worst on SVHN.The authors attribute the SVHN result most likely to the large difference between source and target domains.
  • Limitations: Domain-indistinguishable features do not guarantee correct target-region mapping, so performance depends strongly on the quality of the source-trained representation.If the source representation is poor, the resulting adaptation can be sub-optimal.

6. Conclusions and future work

The paper proposes domain-invariance and GAN-based feature augmentation for unsupervised domain adaptation, and reports improved target-data accuracy. Across benchmarks, the approach is comparable or superior to current state-of-the-art methods, while future work will test more complex problems and other frameworks.

  • The authors propose domain-invariance and feature-space data augmentation through GANs as two techniques for unsupervised domain adaptation.
  • Both domain-invariance and feature augmentation lead to higher target-data accuracies across the evaluated benchmarks.
  • The proposed approach achieves comparable or superior performance to current state-of-the-art methods, except on one benchmark.
  • Future work: The authors plan to evaluate the approach on more complex adaptation problems and investigate feature augmentation in other frameworks.

A. Architectures

The experiments use encoder, classifier, discriminator, and feature-generator architectures tailored to the digit and NYUD settings. Most digit encoders follow standard unsupervised domain adaptation architectures, while NYUD uses a modified pretrained VGG-16 protocol.

  • Digit architectures: Digit-dataset encoders follow standard architectures commonly used in unsupervised domain adaptation.
  • Digit architectures: Figures specify ES and EI architectures for MNIST ↔USPS, SVHN →MNIST, and SYN →SVHN experiments.
  • Shared architectures: Figures provide the S and D1 architectures used across all experiments.
  • D2 architectures: D2 has separate architectures for SVHN →MNIST, SYN →SVHN, MNIST ↔USPS, and NYUD (RGB →D).
  • NYUD architecture: For NYUD, ES and EI use a pretrained VGG-16 cut at fc7, reduced to 128 dimensions, with tanh activations and a 19-dimensional softmax classifier.
  • NYUD architecture: In NYUD, D2 uses two or three hidden layers to stabilize the minimax game against EI.

B. Hyperparameters

The reported hyperparameters were selected to achieve convergence of the GAN minimax games without using target labels for cross-validation.

  • Hyperparameters were set to reach convergence of the GAN minimax games, with no cross-validation using target labels.

B.1. Digits

All training steps use batches of 64 samples, with step-specific learning rates that vary across the experiments.

  • Each training step uses a batch size of 64 samples.
  • Learning rates are 3 · 10^-4 for Step 0, 1 · 10^-4 for Step 1, and 3 · 10^-5 for Step 2, except MNIST ↔USPS uses 3 · 10^-6.

B.2. NYUD

The NYUD experiments use the paper’s specified training protocol and architecture components for the classification and feature-generation modules.

  • Training protocol: Step 0 fine-tunes an ImageNet-pretrained VGG-16 for 20,000 iterations, using batch size 32 because of hardware limitations.The learning rates are 10−4 for Step 0, 10−5 for Step 1, and 10−7 for Step 2.
  • Architectures: The classification architectures use C ◦ES and C ◦EI, with dropout values indicated for the MNIST ↔USPS and SVHN →MNIST experiments.The classification module C is a fully connected layer followed by softmax.
  • Architectures: The architecture of S indicates the size of generated features, while D1 indicates the size of its hidden layer.These architectures are used for the feature generator S and discriminator D1.
  • Architectures: D2 uses one architecture in NYUD and MNIST ↔USPS experiments and another architecture in all other experiments.The figure distinguishes the two D2 architecture settings by experiment group.
Loading 1711.08561v2…