PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsupervised Domain Adaptation with Similarity Learning

Pedro O. Pinheiro

arXiv:1711.08995v2cs.CV

TL;DR

Unsupervised domain adaptation must transfer a classifier from labeled source data to an unlabeled target domain despite distribution shift. SimNet jointly learns domain-invariant features and category prototypes for similarity-based classification, and reports state-of-the-art performance across multiple adaptation scenarios.

  • Problem

    Unsupervised domain adaptation seeks to classify an unlabeled target domain using labeled source data despite different data distributions, while target labels may be expensive to acquire.

  • Method

    SimNet jointly learns domain-confused image embeddings, one prototype per category, and a pairwise similarity function for end-to-end nearest-prototype classification.

  • Results

    SimNet achieves new state-of-the-art performance in multiple unsupervised domain adaptation scenarios and is reported as more robust than the commonly used classifier.

  • Takeaways & Limitations

    Similarity learning together with feature learning can outperform the standard fully connected classifier in domain adaptation.

  • Takeaways & Limitations

    The paper focuses on classification and leaves scaling the similarity-based classifier to larger numbers of categories and training samples for future work.

Abstract

from arXiv · show

The objective of unsupervised domain adaptation is to leverage features from a labeled source domain and learn a classifier for an unlabeled target domain, with a similar but different data distribution. Most deep learning approaches to domain adaptation consist of two steps: (i) learn features that preserve a low risk on labeled samples (source domain) and (ii) make the features from both domains to be as indistinguishable as possible, so that a classifier trained on the source can also be applied on the target domain. In general, the classifiers in step (i) consist of fully-connected layers applied directly on the indistinguishable features learned in (ii). In this paper, we propose a different way to do the classification, using similarity learning. The proposed method learns a pairwise similarity function in which classification can be performed by computing similarity between prototype representations of each category. The domain-invariant features and the categorical prototype representations are learned jointly and in an end-to-end fashion. At inference time, images from the target domain are compared to the prototypes and the label associated with the one that best matches the image is outputed. The approach is simple, scalable and effective. We show that our model achieves state-of-the-art performance in different unsupervised domain adaptation scenarios.

1. Introduction

Unsupervised domain adaptation seeks domain-invariant representations that transfer from labeled source data to unlabeled target data despite domain shift. SimNet replaces the usual fully connected classifier with jointly learned similarity-based prototypes while retaining domain confusion.

  • Domain shift makes direct feature transfer unreliable, while acquiring labeled target data for fine-tuning can be expensive.
  • Unsupervised domain adaptation transfers labels from a labeled source domain to an unlabeled target domain with a similar but different distribution.
  • Most approaches learn source-discriminative features and make source and target representations indistinguishable, typically using adversarial training or distribution matching.
  • SimNet compares each image with one prototype per category and assigns the label of the best-matching prototype.
  • The image embeddings, prototype representations, and pairwise similarity function are learned jointly with domain-confusion training in an end-to-end system.
  • SimNet is reported to be more robust to domain shift, outperform the commonly used classifier, and achieve state-of-the-art results across Digits, Office-31, and VisDA.

2. Related Works

Prior unsupervised domain adaptation methods align source and target distributions through reweighting, feature transformation, discrepancy losses, adversarial objectives, or generative modeling. SimNet instead changes the classifier to similarity learning while using domain confusion for adaptation.

  • Non-deep adaptation methods mainly use sample re-weighting or feature-space transformation to match source and target distributions.
  • Deep adaptation methods commonly combine supervised classification with discrepancy-based or adversarial domain-confusion losses.
  • Adversarial approaches formulate adaptation as a minimax game that learns source-discriminative and domain-indiscriminate features.
  • Other approaches incorporate generative modeling, including domain-specific GANs or source-conditioned image generation.
  • Most reviewed methods use fully connected classifiers, whereas SimNet uses similarity learning and retains RevGrad-style domain confusion.

3. Method

SimNet combines adversarial domain confusion with a similarity-based classifier that compares image embeddings against learned category prototypes. Input and prototype representations, along with their similarity function, are trained jointly so target images can be classified by their best-matching prototype.

  • Method: SimNet combines domain-confusion features with a prototype-based classifier, training both components jointly and end-to-end.The domain-confusion component makes source and target features indistinguishable, while the classifier uses one prototype per category.
  • Similarity-Based Classifier: Each category prototype is computed from source examples, while input images from either domain are mapped to learned feature representations.The prototype represents a category’s variations, and source labels identify the examples used to compute it.
  • Similarity-Based Classifier: The classifier learns a bilinear similarity function between an image embedding and each prototype, then uses the highest-scoring prototype for prediction.The similarity operator is trainable and unconstrained, rather than required to be positive or symmetric.
  • Similarity-Based Classifier: A soft-orthogonality regularizer encourages prototypes to encode different aspects of each category.During training, prototypes are approximated using randomly selected examples for each class.
  • Training and Inference: Domain confusion uses a discriminator and Reverse Gradient training to make source and target feature distributions difficult to distinguish.The resulting shared representation is intended to support applying a classifier trained on source features to target features.
  • Training and Inference: At inference, stored prototypes are compared with each target test image, and the label of the best-matching prototype is returned.The low-rank similarity parametrization also enables efficient computation and storage of the prototype-side representation.

4. Experimental Results

SimNet is evaluated across digits, Office-31, and VisDA domain shifts using standard unsupervised adaptation protocols. It achieves competitive or state-of-the-art results, with additional analyses examining robustness, scaling, and model variants.

  • Experimental Setup: SimNet is evaluated on three domain-adaptation benchmarks: Digits, Office-31, and synthetic-to-real VisDA.The experiments use labeled source and unlabeled target images, with classification accuracy as the evaluation metric.
  • Digits Results: SimNet achieves competitive results across all three digits domain shifts, although PixelDA performs especially well when the target generation process matches its learned pixel transformation.The evaluated shifts are USPS→MNIST, MNIST→USPS, and MNIST→MNIST-M.
  • Office-31 Results: SimNet beats the previous state of the art in most Office-31 transfer settings and is similar in the remaining settings under a shared ResNet-50 architecture.Office-31 evaluation covers all six domain-pair transfer tasks.
  • VisDA Results: SimNet provides a 10% performance boost over the matched RevGrad-ours baseline, while larger backbones and the prototype regularizer further improve performance.Sharing f and g weights hurts performance, and the regularizer encourages orthogonal prototypes to ease similarity computation.
  • VisDA Results: 68.66% average per-class accuracy is achieved by SimNet on VisDA-test without additional inference tricks.VisDA targets synthetic-to-real adaptation and contains 12 categories across more than 280K combined images.
  • VisDA Results: Feature visualizations show that adaptation makes source and target representations more domain invariant, while confusion matrices compare source-only, SimNet, and target-trained oracle models.These analyses are reported for the VisDA synthetic-to-real setting.

5. Conclusion

The paper proposes SimNet, a similarity-based classifier for unsupervised domain adaptation, and reports strong performance across multiple adaptation scenarios. It also identifies broader modalities, discrepancy methods, and scaling as future directions.

  • SimNet combines similarity-based classification with feature learning for unsupervised domain adaptation.
  • The method achieves new state-of-the-art performance in multiple unsupervised adaptation scenarios.
  • Future work includes semantic segmentation, object detection, alternative discrepancy-reduction methods, and scaling to more categories and training samples.
  • Figure 5 shows adaptation bringing source and target feature distributions closer while displaying category prototypes.
Loading 1711.08995v2…