PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Visual Domain Adaptation: A Survey

Mei Wang, Weihong Deng

arXiv:1802.03601v4cs.CV

TL;DR

The paper addresses limited labeled data and the domain shifts that complicate transferring knowledge across visual tasks and domains. It surveys deep domain adaptation by organizing scenarios and loss-based approaches, reviewing applications, and identifying limitations and future directions. Its supported conclusion is that deep DA uses deep architectures to enhance domain adaptation and has achieved success, while important homogeneous-domain and shared-label-space assumptions remain.

  • Problem

    Deep domain adaptation addresses scarce labeled data and distribution shifts between source and target domains, while prior surveys had focused mainly on shallow or broader transfer-learning settings.

  • Method

    The survey develops a taxonomy of deep DA scenarios, categorizes approaches by training loss, studies multi-step adaptation, and reviews computer vision applications.

  • Results

    The survey concludes that deep DA can enhance domain adaptation through deep architectures and has achieved success in visual categorization tasks.

  • Takeaways & Limitations

    Deep DA provides a surveyed framework for comparing adaptation settings, methods, and applications across computer vision tasks.

  • Takeaways & Limitations

    Most existing deep DA assumes identical source and target feature spaces, and many methods assume a shared label space across domains.

Abstract

from arXiv · show

Deep domain adaption has emerged as a new learning technique to address the lack of massive amounts of labeled data. Compared to conventional methods, which learn shared feature subspaces or reuse important source instances with shallow representations, deep domain adaption methods leverage deep networks to learn more transferable representations by embedding domain adaptation in the pipeline of deep learning. There have been comprehensive surveys for shallow domain adaption, but few timely reviews the emerging deep learning based methods. In this paper, we provide a comprehensive survey of deep domain adaptation methods for computer vision applications with four major contributions. First, we present a taxonomy of different deep domain adaption scenarios according to the properties of data that define how two domains are diverged. Second, we summarize deep domain adaption approaches into several categories based on training loss, and analyze and compare briefly the state-of-the-art methods under these categories. Third, we overview the computer vision applications that go beyond image classification, such as face recognition, semantic segmentation and object detection. Fourth, some potential deficiencies of current methods and several future directions are highlighted.

I. INTRODUCTION

Deep domain adaptation addresses scarce labeled data by transferring knowledge across domains despite distribution shifts, using deep networks to learn more transferable representations. This survey organizes deep DA scenarios and methods, and reviews their computer vision applications.

  • Motivation: Collecting and annotating datasets for every task and domain is expensive and time-consuming, while sufficient training data may be unavailable.Auxiliary labeled data from other domains or tasks can help when current-task data are scarce.
  • Motivation: Domain shifts caused by illumination, pose, and image quality can degrade performance when models transfer between domains.Domain adaptation uses labeled data from relevant source domains to execute tasks in a target domain.
  • Deep domain adaptation: Deep networks learn high-level abstractions through multiple nonlinear transformations and can produce more transferable representations for visual categorization.The survey discusses architectures including convolutional neural networks, deep belief networks, and stacked autoencoders.
  • Survey scope: The survey presents a taxonomy of deep DA scenarios based on how data properties create domain divergence and categorizes approaches by classification, discrepancy, and adversarial losses.It also studies multi-step adaptation mechanisms and compares approaches across different domain adaptation settings.
  • Applications: The survey covers applications including image classification, face recognition, style translation, object detection, semantic segmentation, and person re-identification.Its structure moves from notation and adaptation settings to approaches, applications, conclusions, and future work.

B. Different Settings of Domain Adaptation

Domain adaptation settings are distinguished by whether source and target domains differ in distributions or feature spaces, by target-label availability, and by whether transfer occurs directly or through intermediate domains. Deep DA further organizes one-step approaches by their training criteria and uses deep networks to learn transferable representations.

  • Domain adaptation assumes identical tasks while allowing domain divergence through distribution shifts or feature-space differences.
  • Homogeneous DA uses identical feature spaces and dimensions, whereas heterogeneous DA allows nonequivalent feature spaces and differing dimensions.
  • Supervised, semi-supervised, and unsupervised DA differ according to whether labeled and unlabeled target-domain data are available during training.
  • Multi-step DA transfers knowledge through intermediate bridge domains when source and target domains have little overlap for effective one-step adaptation.
  • Deep DA embeds adaptation into deep representation learning so features become semantically meaningful and domain invariant.
  • One-step deep DA approaches are categorized by discrepancy, adversarial, and classification losses, with discrepancy methods using class, statistic, architecture, or geometric criteria.

B. Categorization of Multi-Step Domain Adaptation

Multi-step domain adaptation inserts intermediate domains between source and target to reduce information loss, with transfer organized into hand-crafted, instance-based, or representation-based mechanisms. The survey also distinguishes adaptation settings and training criteria, including class, discrepancy, and adversarial losses.

  • Multi-Step Domain Adaptation: Multi-step domain adaptation selects intermediate domains more related to the source and target, then transfers knowledge through successive one-step adaptations.Its key challenge is selecting and using intermediate domains while minimizing information loss.
  • Hand-Crafted: Hand-crafted adaptation uses user-selected intermediate domains based on experience.
  • Instance-Based: Instance-based adaptation composes intermediate domains by selecting parts of data from auxiliary datasets.
  • Representation-Based: Representation-based adaptation transfers knowledge through frozen previously trained networks and their intermediate representations.
  • Adaptation Settings: The survey focuses on supervised and non-supervised domain adaptation, while treating semi-supervised adaptation as a combination of both.Supervised adaptation uses labeled target data; non-supervised adaptation uses unlabeled target data.
  • Training Criteria: Class-criterion approaches use target class labels to fine-tune a source-pretrained network, whereas adversarial approaches encourage representations that confuse domain discrimination.Soft-label loss and domain-confusion loss are examples of these training strategies.
  • Discrepancy-Based Approaches: Discrepancy-based adaptation trains source and target networks with classification and distribution-alignment objectives, using fine-tuning or frozen early layers depending on data and domain similarity.MMD measures source-target distance; DDC adapts one layer, while DAN uses multiple adaptation layers and kernels.

2) Adversarial-Based Approaches:

Adversarial-based domain adaptation trains models to make source and target representations indistinguishable, using either generative transformations or non-generative discriminators. The survey organizes these methods by architectural and loss choices, including GAN-based synthesis, domain confusion, gradient reversal, and alternative discrepancy measures.

  • Overview: Adversarial domain adaptation aims to ensure that networks cannot distinguish between source and target domains.The survey categorizes adversarial methods into generative and non-generative subsettings.
  • Non-Generative Models: Adversarial methods also include unified architectural variants, selective class-wise discriminators for partial transfer, Wasserstein-distance estimation, and classifier-discrepancy objectives.Selective adversarial networks filter outlier source classes and align distributions in the shared label space.
  • Generative Models: Generative approaches create synthetic target data with annotations so the target model can be trained as if domain adaptation were unnecessary.GANs learn the source-to-target transformation without supervision.
  • Generative Models: CoGAN uses paired source- and target-domain GANs with shared weights to learn a domain-invariant feature space without correspondence supervision.Early generator layers and late discriminator layers are tied across the two GANs.
  • Generative Models: Other generative methods translate source knowledge into target-like images while preserving annotations through domain and real/fake discrimination.These methods use discriminators to enforce content invariance and target-domain visual similarity.
  • Non-Generative Models: Non-generative methods learn domain-invariant representations by confusing domain classifiers, as in DANN's gradient reversal architecture.DANN combines source label prediction with domain-confusion objectives using shared feature extraction layers and two classifiers.

3) Reconstruction-Based Approaches:

Reconstruction-based approaches learn transferable representations by reconstructing source or target data, often alongside classification or adversarial objectives. They include shared encoder-decoder models, dual-domain reconstruction, and cycle-consistent image translation.

  • Reconstruction-based domain adaptation uses source or target reconstruction as an auxiliary task to create a shared representation.
  • Encoder-Decoder Reconstruction: Encoder-decoder methods use a shared encoder to learn domain-invariant features while reconstruction losses preserve domain-specific information.
  • Encoder-Decoder Reconstruction: DRCN combines a shared CNN encoder with source-label classification and unsupervised target reconstruction for cross-domain object recognition.
  • Encoder-Decoder Reconstruction: DRCN balances source classification and reconstruction through a hyperparameter λ, using cross-entropy classification loss and squared reconstruction loss.
  • Encoder-Decoder Reconstruction: Domain separation networks partition representations into shared and domain-private components, reconstruct inputs from both, and classify using the shared representation.
  • Adversarial Reconstruction: Cycle GAN translates between unpaired image domains with two generators and two discriminators, combining adversarial realism with reconstruction consistency.

4) Hybrid Approaches:

Hybrid approaches combine multiple adaptation objectives or criteria within a single method. Examples integrate domain confusion, soft labels, distribution statistics, architecture changes, or reconstruction-based representation separation.

  • Hybrid methods combine complementary adaptation objectives, including domain confusion with soft labels and MMD with classifier adaptation through residual functions.
  • Other combinations add class-specific auxiliary weights to MMD or integrate encoder-decoder reconstruction with private and shared representation separation.

B. Heterogeneous Domain Adaptation

Heterogeneous domain adaptation addresses domains with nonequivalent feature spaces, often caused by different sensory devices or media. Existing deep methods remain limited and frequently resemble homogeneous adaptation approaches.

  • Heterogeneous domain adaptation involves source and target feature spaces that differ, with potentially different feature dimensions.
  • Shallow heterogeneous adaptation commonly uses symmetric or asymmetric transformations to project domains into a common subspace or otherwise align them.
  • Deep heterogeneous adaptation has received limited attention, and specialized effective methods have not yet been proposed.

1) Discrepancy-Based Approach:

Discrepancy-based methods are constrained when source and target feature dimensions differ, because shared-layer strategies assume compatible inputs. Existing heterogeneous approaches therefore use resizing when possible or specialized multimodal networks when it is not.

  • Discrepancy-based methods generally share or reuse early layers, limiting source and target inputs to the same feature dimension.
  • When heterogeneous images can be resized to matching dimensions, class and statistic criteria remain applicable, including CNN-based transfer from RGB to depth data.
  • When modalities cannot be resized into matching dimensions, discrepancy-based methods fail without additional processing.
  • Weakly shared DTNs and transfer neural trees use separate modality-specific streams with limited shared layers to learn domain-invariant representations.

2) Adversarial-Based Approach:

Generative models support heterogeneous domain adaptation by generating target-domain data while transferring source-domain information. The surveyed examples use GAN-based objectives for image synthesis and cross-modal translation.

  • Generative models generate heterogeneous target data while transferring information from the source domain.
  • A compound loss transfers unlabeled face photos to emoji images using multiclass GAN, regularizing, and f-constancy components.
  • Text-conditioned GANs generate bird and flower images from text features encoded by a hybrid character-level convolutional-recurrent network.

3) Reconstruction-Based Approach:

Adversarial reconstruction addresses heterogeneous domain adaptation by learning bidirectional mappings between photos and sketches. CycleGAN, DualGAN, and DiscoGAN use two generators for these translations, while a later method targets facial artifacts.

  • Adversarial reconstruction uses two generators to translate sketches from photos and photos from sketches.
  • CycleGAN, DualGAN, and DiscoGAN apply this bidirectional reconstruction strategy to photo-sketch translation.
  • A multi-adversarial network builds on CycleGAN to reduce facial photo-sketch synthesis artifacts using multiresolution feature maps in the generator.

V. MULTI-STEP DOMAIN ADAPTATION

Multi-step domain adaptation connects distant source and target domains through one or more intermediate domains. The surveyed methods select or construct intermediates using predefined choices, data mixtures, reconstruction errors, or gradual domain replacement.

  • Intermediate-domain selection is problem specific, and different problems may require different strategies.
  • When source images and target text differ, annotated images can be crawled in advance as an intermediate domain.
  • Nighttime light intensity information serves as an intermediate domain for transferring daytime satellite-image knowledge to poverty prediction.
  • When many intermediate domains are available, automatic selection criteria can construct intermediates by mixing parts of source and target data.
  • DDTL relates long-distance domains through multiple intermediate domains by selecting unlabeled data that minimizes reconstruction errors across source, intermediate, and target instances.
  • The DLID model constructs intermediate domains by gradually replacing source samples with target samples, while progressive networks transfer representations through lateral connections across domain-specific networks.

VI. APPLICATION OF DEEP DOMAIN ADAPTATION

Deep domain adaptation has been applied across classification, recognition, detection, style translation, and semantic segmentation. The surveyed applications address domain shifts using adversarial alignment, reconstruction, weak labels, classifier adaptation, and image-space feature projection.

  • Application scope: Deep domain adaptation is applied to image classification, object recognition, face recognition, object detection, style translation, and related vision tasks.
  • Image classification: Image-classification comparisons include Office-31 evaluations across Amazon, DSLR, and Webcam domains, alongside supervised comparisons with DANN and DLID.
  • Image classification: Cross-domain handwritten-digit recognition evaluates adversarial methods including DANN, CoGAN, and ADDA with VGG-16 as the baseline.
  • Face recognition: BAE addresses face-recognition shifts across view angle, ethnicity, and imaging sensor by shifting source samples toward the target domain and using sparse reconstruction.
  • Object detection: Object detection requires substantial bounding-box labeling, so adaptation can target classifiers when the window-selection mechanism is treated as domain independent.
  • Object detection: LSDA uses limited target bounding-box labels and massive weak labels by adapting a target classification layer from a pretrained source model.
  • Semantic segmentation: Semantic-segmentation methods transfer weak class or object-location labels through encoder-decoder and attention architectures.
  • Semantic segmentation: Unsupervised segmentation combines global adversarial domain alignment with class-aware constrained multiple-instance loss for spatial-layout transfer, while another method projects features into image space for discrimination.

E. Image-to-Image Translation

Deep domain adaptation uses deep networks to address domain divergence across visual tasks, including image-to-image translation and applications beyond classification. The survey organizes methods by domain setting, transfer distance, and training strategy while identifying limitations in current coverage.

  • Heterogeneous adaptation: Image-to-image translation addresses heterogeneous adaptation when source and target images have different feature spaces.The survey identifies this setting as requiring translation between image domains before adaptation.
  • Translation mechanisms: Paired-image translation methods use conditional generative networks to learn mappings, while other approaches address unpaired image domains.Examples include pix2pix for paired images and CoGAN or CycleGAN for unpaired settings.
  • Translation mechanisms: Statistical distribution matching can fine-tune deep networks using combined content and style losses to preserve image content while changing its style.The described loss is a linear combination of content and style losses.
  • Open challenges: Current methods commonly assume shared feature and label spaces, leaving heterogeneous domains and differing category sets as open challenges.The survey highlights both the predominance of homogeneous adaptation and the shared-label-space assumption.
  • Applications: Deep domain adaptation also covers image classification, face recognition, style translation, object detection, semantic segmentation, and person re-identification.The survey notes that relatively few studies address tasks beyond classification and recognition.
Loading 1802.03601v4…