PaperlayerView exact revision on arXiv

Source-linked AI summary

Drop to Adapt: Learning Discriminative Features for Unsupervised Domain Adaptation

Seungmin Lee, Dongwan Kim, Namil Kim, Seong-Gyun Jeong

arXiv:1910.05562v1cs.CV

TL;DR

Unsupervised domain adaptation must transfer knowledge from labeled source data to an unlabeled target domain despite domain shift and class-label-blind feature alignment. DTA applies adversarial dropout to enforce clustered target features, achieving consistent gains across image classification and semantic segmentation.

  • Problem

    Domain adversarial methods align source and target features without considering class labels, which can yield suboptimal task representations.

  • Method

    DTA uses adversarial dropout on feature extractors and classifiers to enforce the cluster assumption in unsupervised domain adaptation.

  • Results

    DTA consistently improves over source-only and state-of-the-art results across image classification and semantic segmentation experiments.

  • Takeaways & Limitations

    Adversarial dropout provides a unified, straightforward adaptation framework that is effective across architectures and domain-adaptation tasks.

Abstract

from arXiv · show

Recent works on domain adaptation exploit adversarial training to obtain domain-invariant feature representations from the joint learning of feature extractor and domain discriminator networks. However, domain adversarial methods render suboptimal performances since they attempt to match the distributions among the domains without considering the task at hand. We propose Drop to Adapt (DTA), which leverages adversarial dropout to learn strongly discriminative features by enforcing the cluster assumption. Accordingly, we design objective functions to support robust domain adaptation. We demonstrate efficacy of the proposed method on various experiments and achieve consistent improvements in both image classification and semantic segmentation tasks. Our source code is available at https://github.com/postBG/DTA.pytorch.

1. Introduction

Deep neural networks require large labeled datasets, but annotation is costly, time-consuming, and sometimes impractical. Synthetic data offers an alternative because labels can be generated efficiently across many scenarios.

  • Large-scale datasets enable strong DNN performance across visual recognition tasks.
  • Annotating training data is expensive and time-consuming.
  • Some rare or dangerous events lack appropriate real-world data collection opportunities.
  • Synthetic simulation environments can provide effortless labels across a wide range of scenarios.

CODE42.ai

The paper addresses unsupervised domain adaptation under domain shift, where source-trained models perform poorly on differently characterized target domains. DTA uses adversarial dropout to enforce the cluster assumption and supports adaptation across classification and segmentation settings.

  • Domain shift causes performance drops when target-domain characteristics differ from the source domain.
  • Domain adversarial training may align source and target features without considering class labels, producing suboptimal representations.
  • DTA uses adversarial dropout to push decision boundaries away from dense target features and enforce the cluster assumption.
  • The framework is built for unsupervised domain adaptation, supports convolutional and fully connected layers, and extends to semantic segmentation.
  • DTA achieves competitive image-classification results across varied benchmarks and adapts from simulation to real-world environments for segmentation.

2. Related Work

Prior domain-adaptation methods reduce discrepancy through image translation, feature alignment, or decision-boundary objectives. DTA is positioned as a simpler alternative to minimax approaches, applying adversarial dropout to both classifiers and feature extractors.

  • Image-translation methods minimize discrepancies between domains at the image level.
  • Feature-alignment methods match source and target feature distributions, often using a domain discriminator to produce domain-invariant features.
  • Because domain classifiers cannot consider class labels, aligned features can remain suboptimal for classification.
  • MCD and ADR use minimax training to push target features away from decision boundaries under the cluster assumption.
  • DTA replaces classifier discrepancy maximization with adversarial dropout and also applies it to the feature extractor.
  • Channel-wise adversarial dropout removes entire feature maps rather than individual neurons.

3. Proposed Method

DTA uses adversarial dropout to enforce the cluster assumption, making target predictions consistent under targeted perturbations. Its unified objective combines task-specific learning with adaptation, entropy minimization, and VAT, while supporting both fully connected and convolutional layers.

  • Unsupervised Domain Adaptation: DTA models UDA with labelled source data, unlabelled target data, a feature extractor, and a classifier whose layers can receive dropout masks.The network is h(x; mf, mc) = c(f(x; mf); mc).
  • Adversarial Dropout: Adversarial dropout generates divergent predictions from random and adversarial masks, then minimizes their divergence to push decision boundaries away from target features.The adversarial mask is minimally modified while maximizing output divergence; training subsequently minimizes that divergence.
  • Adversarial Dropout: EAdD drops individual units, whereas CAdD drops entire feature maps to accommodate the spatial correlation of convolutional activations.CAdD constrains each channel mask to all zeros or all ones and estimates impact per activation map.
  • Objective Functions: The unified loss is a weighted sum of source task loss, DTA adaptation loss, target entropy minimization, and VAT objectives.λ1, λ2, and λ3 control the relative importance of the adaptation, entropy, and VAT terms.
  • Drop to Adapt: Applying adversarial dropout to the classifier identifies volatile regions near the decision boundary, and minimizing classifier divergence encourages features to avoid them.Applying it to the feature extractor instead pushes the decision boundary away from target features that cross it under dropout.
  • Objective Functions: Entropy minimization penalizes target samples near decision boundaries, while VAT perturbs target inputs and complements DTA’s network-parameter perturbations.DTA regularizes through adversarial dropout, whereas VAT regularizes through input perturbations.

4. Experimental Results

DTA is evaluated across small and large domain-adaptation benchmarks spanning image classification and semantic segmentation. It consistently improves over source-only models, with strong results on both synthetic-to-real and small-dataset shifts.

  • Experimental scope: Experiments cover small and large domain-adaptation benchmarks for image classification and semantic segmentation.The evaluation uses digits, object-recognition, VisDA-2017 classification, and GTA-to-Cityscapes segmentation settings.
  • Small datasets: DTA approaches target-only performance on SVHN →MNIST despite the datasets’ drastic distributional shift.SVHN contains colored street-number images, whereas MNIST contains binary handwritten digits.
  • Small datasets: DTA achieves accuracy close to fully supervised target-domain learning in both MNIST ↔USPS directions and exceeds direct USPS training when adapting from MNIST.The authors attribute the USPS result to its relatively small training set.
  • Small datasets: DTA improves over source-only performance across all small-dataset configurations and beats recent state-of-the-art results in four of five configurations.The summary compares the method with source-only models and recent methods across the evaluated domain configurations.
  • Large datasets: 30.7% improvement in average accuracy over source-only on VisDA-2017 classification with ResNet-101; DTA also exceeds MCD and ADR by 9.6% and 6.7%.DTA achieves the best per-class performance except on truck, where it trails ADR by 0.2%.
  • Large datasets: DTA improves VisDA-2017 semantic-segmentation mIoU over source-only and competing methods, producing relatively clean and accurate predictions.The segmentation evaluation uses GTA5 as source, Cityscapes as target, and mIoU over 19 classes.

5. Discussion

Ablation studies on VisDA-2017 examine DTA’s modules across ResNet-50 and ResNet-101, while qualitative segmentation results evaluate adaptation from GTA to Cityscapes. The components generally improve source-only performance and show complementary effects.

  • Ablation studies: Ablations on VisDA-2017 evaluate VAT, fDTA, and cDTA individually and in combination using ResNet-50 and ResNet-101.The experiments assess both module importance and generality across backbone capacities.
  • Ablation studies: All three modules improve performance over a source-only model, with category-specific contributions across the two backbones.In ResNet-101, fDTA strongly affects knife, whereas VAT significantly boosts skakteboard.
  • Semantic segmentation: The adaptation method is presented qualitatively on the GTA →Cityscapes semantic segmentation task using a modified FCN with ResNet-50.Figure 5 compares results before and after adaptation with DTA.
  • Ablation studies: The complementary effects of VAT and the proposed feature-space perturbations support DTA as a general regularization technique across model capacities.The paper reports that the observed trends are mostly maintained for both backbone models.

6. Conclusion

The conclusion presents DTA as a simple method for unsupervised domain adaptation under large domain shifts, using adversarial dropout to enforce the cluster assumption. Experiments across datasets and tasks report improvements over source-only and state-of-the-art results.

  • Conclusion: DTA is presented as a simple yet effective method for unsupervised domain adaptation despite large domain shifts.This is the paper’s stated overall conclusion.
  • Conclusion: EAdD and CAdD are adversarial dropout modules designed to enforce the cluster assumption on the target domain.The proposed methods are described as easily integrated into existing deep learning architectures.
  • Conclusion: Extensive experiments on small and large datasets report significant improvements over source-only and state-of-the-art results across image classification and semantic segmentation.The conclusion states that improvements were achieved on both domain adaptation tasks.
  • Appendices: The appendices provide an approximation of channel-wise adversarial dropout, implementation details, and additional GradCAM visualizations.These materials are organized in Appendices A, B, and C, respectively.

Appendix A. Approximation of Channel-wise Adversarial Dropout

The appendix approximates channel-wise adversarial dropout by representing masks as vectors, measuring output divergence under different masks, and using activation impacts to choose constrained dropout changes. A 0/1 Knapsack problem enforces the channel-wise constraint.

  • Mask representation: The dropout mask is vectorized into v ∈ R^CHW, with v0 and vs representing the vectorized forms of m0 and ms.The elements of each activation map are referenced through index sets πi.
  • Divergence approximation: The divergence d(x, v; vs) measures the difference between outputs produced with different dropout masks.It is defined through a divergence D applied to h(x; vs) and h(x; v).
  • Divergence approximation: d(x, v; vs) is approximated with a first-order Taylor expansion around v0 using the Jacobian J = ∇v d(x, v; vs).The approximation assumes differentiability with respect to v.
  • Impact values: Each Jacobian element represents an activation’s impact on the divergence, enabling systematic modification of v to maximize that divergence.Individual element changes are restricted by the channel-wise dropout constraint.
  • Constrained optimization: The resulting impact values are used to solve a 0/1 Knapsack problem while preserving the channel-wise dropout constraints.This selects constrained dropout changes rather than modifying every vector element independently.

Training with DTA Loss

DTA training stabilizes its loss by gradually increasing adversarial dropout perturbation magnitudes. A shared ramp-up period controls perturbations for both proposed dropout components.

  • Ramp-up training: A ramp-up factor is applied to the DTA loss to stabilize training.The method increases perturbation magnitudes rather than directly modulating λ1.
  • Ramp-up training: The perturbation magnitudes δe and δc determine how many hidden units are eliminated and are gradually increased during training.This regulates the consistency term and trains robustness to varying adversarial-dropout levels.
  • Ramp-up schedule: β(t) denotes the ramp-up factor at epoch t, while Tr specifies the ramp-up period.The maximum perturbation level is denoted by ¯δ.
  • Ramp-up schedule: The same ramp-up period Tr is applied to both δe and δc.Both perturbation magnitudes therefore follow a shared schedule in practice.

Hyperparameters

The experiments use task-specific training schedules and predefined hyperparameter search sets. Additional GradCAM visualizations illustrate the effects of adversarial dropout.

  • Hyperparameter search: The hyperparameter search samples a very small subset of target-domain training labels and restricts each objective function to predefined values.The listed sets include λ1 = {2}, λ2 = {0, 0.01, 0.02}, λ3 = {0, 0.1, 0.2}, δe = {0, 0.1}, δc = {0, 0.01, 0.02, 0.05}, and ϵ = {0, 3.5, 15}.
  • Training schedules: Small-dataset experiments ran for 90 epochs with Adam, starting at learning rate 0.001 and decaying it by 0.1 every 30 epochs.
  • Training schedules: VisDA-2017 classification used ResNet-50 and ResNet-101 for 20 epochs with SGD, momentum 0.9, and learning rate 0.001 decayed by 0.1 after epoch 10.
  • Training schedules: GTA5-to-Cityscapes segmentation ran for 5 epochs with SGD and momentum 0.9, set δ̄e to 0, used maximum δ̄c from the start, and disabled VAT.The task-specific objective was dominant early in training, while the VAT objective was turned off because it hindered segmentation learning.
  • Visualization: Additional GradCAM visualizations were provided to highlight the effects of adversarial dropout.
Loading 1910.05562v1…