PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-source Domain Adaptation in the Deep Learning Era: A Systematic Survey

Sicheng Zhao, Bo Li, Colorado Reed, Pengfei Xu, Kurt Keutzer

arXiv:2002.12169v1cs.LGcs.CVstat.ML

TL;DR

Using labeled data from multiple differently distributed source domains raises the question of how to adapt effectively when simply combining sources can underperform the best single source. This survey defines and classifies deep-learning MDA strategies, summarizes evaluation datasets, and compares representative methods, while identifying open questions about strategy selection.

  • Problem

    When labeled data come from differently distributed sources, simply combining them can perform worse than using the best single source, motivating effective multi-source adaptation strategies.

    Source: §1 · p. 2

  • Method

    The survey classifies deep-learning MDA approaches, summarizes commonly used evaluation datasets, and technically and experimentally compares representative methods.

    Source: §1 · p. 2, §5 · p. 6

  • Results

    The survey organizes deep-learning MDA methods into categories and compares representative methods technically and experimentally.

    Source: §5 · p. 6

  • Takeaways & Limitations

    The survey focuses on unsupervised, homogeneous, closed-set MDA with one target domain.

    Source: §5 · p. 6

    Determining which MDA strategies work best for which problem types remains an open question.

    Source: §5 · p. 6

Abstract

from arXiv · show

In many practical applications, it is often difficult and expensive to obtain enough large-scale labeled data to train deep neural networks to their full capability. Therefore, transferring the learned knowledge from a separate, labeled source domain to an unlabeled or sparsely labeled target domain becomes an appealing alternative. However, direct transfer often results in significant performance decay due to domain shift. Domain adaptation (DA) addresses this problem by minimizing the impact of domain shift between the source and target domains. Multi-source domain adaptation (MDA) is a powerful extension in which the labeled data may be collected from multiple sources with different distributions. Due to the success of DA methods and the prevalence of multi-source data, MDA has attracted increasing attention in both academia and industry. In this survey, we define various MDA strategies and summarize available datasets for evaluation. We also compare modern MDA methods in the deep learning era, including latent space transformation and intermediate domain generation. Finally, we discuss future research directions for MDA.

1 Background and Motivation

Because labeled data can be costly or unavailable, transferring knowledge from labeled sources to unlabeled or sparsely labeled targets is attractive, but domain shift can undermine direct transfer.

1 Background and Motivation

  • Direct transfer shows substantial losses: digit-classification accuracy falls from 96.0% to 52.3%, while synthetic-to-real segmentation mIoU falls from 62.6% to 21.7%.The examples are MNIST-to-MNIST-M digit recognition and GTA-to-Cityscapes segmentation.Source: §1 · p. 1, Figure 1 · p. 1
  • Deep SDA combines a source-task model with source–target alignment, using discrepancy, adversarial generation, adversarial discrimination, or reconstruction strategies.Domain shift refers to differences in the joint probability distributions of observed data and labels across domains.Source: §1 · p. 2
  • 71.3% versus 70.8% accuracy for digit adaptation, and 38.7% versus 37.3% mIoU for segmentation, show source-combined DA can underperform the best single source.The survey argues that differing distributions among sources can make combined data interfere during learning, motivating MDA to leverage all available sources.Source: §1 · p. 2, Figure 2 · p. 2
  • Early MDA used shallow latent spaces or combined classifiers; this survey reviews deep MDA approaches, compares their similarities and differences, and outlines future research directions.The survey notes that MDA emphasis has shifted from shallow methods toward deep learning architectures.Source: §1 · p. 2

2 Problem Definition

The section defines MDA as adaptation from multiple source domains to one target domain and classifies the problem by data supervision, feature dimensions, label-set relations, and target-domain availability.

  • MDA involves multiple source domains and one target domain, with data drawn from source and target distributions.Source: §2 · p. 2
  • MDA is strongly supervised when every source sample is labeled and weakly supervised otherwise; target-label availability also distinguishes semi-supervised MDA.Source: §2 · pp. 2–3
  • MDA is homogeneous when source and target feature dimensions match, and heterogeneous otherwise.Source: §2 · p. 2
  • Label-set strategies include closed set (all sets equal), open set (a source-target intersection is a proper subset of the target set), partial (the target set is a proper subset of a source set), and universal MDA (no prior label-set knowledge).Source: §2 · pp. 2–3
  • Adapting simultaneously to multiple target domains is multi-target MDA, while target data unavailable during training defines multi-source domain generalization or zero-shot MDA.Source: §2 · p. 3

3 Datasets

The section surveys commonly used computer-vision and NLP datasets whose domains differ in style and data conditions. It also summarizes how domains are assigned as sources and targets for evaluation.

Digit recognition

  • Digits-five comprises five digit-image domains: MNIST, MNIST-M, SVHN, Synthetic Digits, and USPS.Typically, 25,000 training and 9,000 test images are sampled for MNIST, MNIST-M, SVHN, and Synthetic Digits; all 9,298 USPS images are selected.Source: §3 · p. 3

Object classification

  • Object-classification benchmarks range from Office-31’s three office domains to Office-Caltech’s 10 shared categories and Office-Home’s four everyday-object domains.Office-31 has 4,110 images; Office-Caltech combines its categories with Caltech-256, while Office-Home has about 15,500 images across 65 categories.Source: §3 · p. 3
  • ImageCLEF, PACS, and DomainNet provide further object-classification benchmarks, with DomainNet the largest DA dataset to date for this task.ImageCLEF has 600 images per domain; PACS has 9,991 images across four domains, and DomainNet has about 600K images across six domains and 345 categories.Source: §3 · p. 3

Sentiment classification of images

  • SentiImage supports binary image-sentiment classification across four social and artistic domains, ranging from 603 to 23,308 images.The domains are Flickr and Instagram, ArtPhoto, Twitter I, and Twitter II.Source: §3 · p. 3

Vehicle counting

  • WebCamT provides 60,000 annotated frames, with eight cameras at different intersections used as domains, each containing more than 2,000 labeled images.The videos have low resolution, low frame rate, and high occlusion.Source: §3 · p. 3

Scene segmentation

  • Sim2RealSeg combines synthetic GTA and SYNTHIA data with real Cityscapes and BDDS data for segmentation.GTA has 24,966 images with 19 classes, Cityscapes has 5,000 annotated images across 19 classes, BDDS has 10,000 frames, and the SYNTHIA subset uses 16 common Cityscapes classes for MDA.Source: §3 · pp. 3–4

Natural language processing

  • NLP benchmarks include binary-sentiment product and movie reviews, plus SANCL’s part-of-speech tagging across five web domains.Amazon Reviews uses 2,000 labeled examples per source; Media Reviews commonly uses five domains, and SANCL uses 750 training sentences per source.Source: §3 · p. 4

Evaluation protocol

  • Evaluation usually treats one domain as the target and the rest as sources, with task-specific exceptions for WebCamT, Sim2RealSeg, and SANCL.WebCamT randomly selects two target domains; Sim2RealSeg commonly adapts from synthetic GTA and SYNTHIA to real Cityscapes and BDDS, while SANCL uses N, R, and A as targets, and E and W as targets.Source: §3 · p. 4

4 Deep Multi-source Domain Adaptation

The survey focuses on deep MDA with one unlabeled target, fully labeled sources, shared data space, and common label sets. It groups methods by whether they align latent spaces or generate adapted intermediate domains.

4 Deep Multi-source Domain Adaptation

  • The surveyed deep MDA setting assumes one available unlabeled target, fully labeled sources, the same data space, and identical label sets; task models use source task loss alongside domain alignment.The survey notes that much existing work centers on these unsupervised, homogeneous, closed-set, strongly supervised conditions.Source: §4 · p. 4

4.1 Latent Space Transformation

  • Discrepancy methods align features by minimizing distances [Guo et al., 2018; Zhu et al., 2019; Hoffman et al., 2018a; Rakshit et al., 2019; Peng et al., 2019], whereas adversarial methods confuse a discriminator [Xu et al., 2018; Zhao et al., 2018a; Li et al., 2018; Wang et al., 2019; Zhao et al., 2020].A mixture of distances is proposed because individual measures have pathological cases; Wasserstein distance can provide more stable gradients when source and target distributions do not overlap [Zhao et al., 2020].Source: §4.1 · p. 5
  • Latent-space methods vary in domain pairing, feature-extractor sharing, and classifier design, including pairwise source-target alignment [Xu et al., 2018; Guo et al., 2018; Zhao et al., 2018a; Hoffman et al., 2018a; Zhu et al., 2019; Zhao et al., 2020; Guo et al., 2020].Some align every source and target pair; unshared extractors can improve alignment but increase parameters [Rakshit et al., 2019; Zhao et al., 2020], while classifier discrepancy or compound classifiers address prediction misalignment.Source: §4.1 · p. 5
  • Target predictions from multiple source classifiers can be uniformly or non-uniformly weighted using source-selection measures, while adaptive selection of relevant training samples remains open [Zhao et al., 2020].Methods include uniform weights, perplexity, PoS, relative error, and Wasserstein distance [Zhao et al., 2020]; Zhao et al. use a manually selected distance threshold to distill samples.Source: §4.1 · p. 6

4.2 Intermediate Domain Generation

  • Intermediate-domain generation addresses the limits of feature-level alignment for fine-grained predictions by enabling pixel-level alignment, typically with GANs.Methods use CoGAN, CycleGAN [Zhao et al., 2019a], or a VAE latent space [Lin et al., 2020]; semantic consistency can help preserve labels [Lin et al., 2020].Source: §4.2 · p. 6
  • Intermediate-domain methods align generated domains with the target and vary between one classifier per adapted domain and a classifier on an aggregated or combined domain [Zhao et al., 2020; Lin et al., 2020].MADAN aggregates adapted domains with sub-domain aggregation and cross-domain cycle discriminators [Zhao et al., 2019a]; feature- and pixel-level alignment are often jointly optimized with GAN loss.Source: §4.2 · p. 6

5 Conclusion and Future Directions

The survey reviews recent deep-learning-era MDA, focusing on the unsupervised, homogeneous, closed-set, one-target setting, and defines strategies, summarizes evaluation datasets, and compares representative methods.

Specific MDA strategy implementation

  • MDA strategies should be tailored to problem requirements; which strategies suit which problems and how much labeled target data is needed remain open questions.The survey also identifies how to incorporate small amounts of labeled target data as an unresolved issue.Source: §5 · p. 6

Multi-modal MDA

  • MDA needs techniques to fuse labeled source data across modalities, including settings with varied modalities across sources and partially labeled multi-modal sources.Source: §5 · p. 6

Incremental and online MDA

  • Incremental and online MDA algorithms remain largely unexplored and could support deployed models as new source or target data becomes available.Source: §5 · p. 6

Related papers

Loading 2002.12169v1…