PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Visual Domain Adaptation: A Survey

Mei Wang, Weihong Deng

arXiv:1802.03601v4cs.CV

TL;DR

The paper addresses limited labeled data and the domain shifts that complicate transferring knowledge across visual tasks and domains. It surveys deep domain adaptation by organizing scenarios and loss-based approaches, reviewing applications, and identifying limitations and future directions. Its supported conclusion is that deep DA uses deep architectures to enhance domain adaptation and has achieved success, while important homogeneous-domain and shared-label-space assumptions remain.

  • Problem

    Deep domain adaptation addresses scarce labeled data and distribution shifts between source and target domains, while prior surveys had focused mainly on shallow or broader transfer-learning settings.

    Source: §I · p. 1

  • Method

    The survey develops a taxonomy of deep DA scenarios, categorizes approaches by training loss, studies multi-step adaptation, and reviews computer vision applications.

    Source: §I · pp. 1–2, A. Categorization of One-Step Domain… · p. 3

  • Results

    The survey concludes that deep DA can enhance domain adaptation through deep architectures and has achieved success in visual categorization tasks.

    Source: §III · p. 3, §VII · p. 16

  • Takeaways & Limitations

    Deep DA provides a surveyed framework for comparing adaptation settings, methods, and applications across computer vision tasks.

    Source: §I · pp. 1–2

    Most existing deep DA assumes identical source and target feature spaces, and many methods assume a shared label space across domains.

    Source: §VII · p. 17

Abstract

from arXiv · show

Deep domain adaption has emerged as a new learning technique to address the lack of massive amounts of labeled data. Compared to conventional methods, which learn shared feature subspaces or reuse important source instances with shallow representations, deep domain adaption methods leverage deep networks to learn more transferable representations by embedding domain adaptation in the pipeline of deep learning. There have been comprehensive surveys for shallow domain adaption, but few timely reviews the emerging deep learning based methods. In this paper, we provide a comprehensive survey of deep domain adaptation methods for computer vision applications with four major contributions. First, we present a taxonomy of different deep domain adaption scenarios according to the properties of data that define how two domains are diverged. Second, we summarize deep domain adaption approaches into several categories based on training loss, and analyze and compare briefly the state-of-the-art methods under these categories. Third, we overview the computer vision applications that go beyond image classification, such as face recognition, semantic segmentation and object detection. Fourth, some potential deficiencies of current methods and several future directions are highlighted.

I. INTRODUCTION

Deep domain adaptation addresses scarce labeled data by transferring knowledge across domains despite distribution shifts, using deep networks to learn more transferable representations. This survey organizes deep DA scenarios and methods, and reviews their computer vision applications.

Motivation

  • Collecting and annotating datasets for every task and domain is expensive and time-consuming, while sufficient training data may be unavailable.Auxiliary labeled data from other domains or tasks can help when current-task data are scarce.Source: §I · p. 1
  • Domain shifts caused by illumination, pose, and image quality can degrade performance when models transfer between domains.Domain adaptation uses labeled data from relevant source domains to execute tasks in a target domain.Source: §I · p. 1

Deep domain adaptation

  • Deep networks learn high-level abstractions through multiple nonlinear transformations and can produce more transferable representations for visual categorization.The survey discusses architectures including convolutional neural networks, deep belief networks, and stacked autoencoders.Source: §I · p. 1

Survey scope

  • The survey presents a taxonomy of deep DA scenarios based on how data properties create domain divergence and categorizes approaches by classification, discrepancy, and adversarial losses.It also studies multi-step adaptation mechanisms and compares approaches across different domain adaptation settings.Source: §I · pp. 1–2

Applications

  • The survey covers applications including image classification, face recognition, style translation, object detection, semantic segmentation, and person re-identification.Its structure moves from notation and adaptation settings to approaches, applications, conclusions, and future work.Source: §I · p. 2

B. Different Settings of Domain Adaptation

Domain adaptation settings are distinguished by whether source and target domains differ in distributions or feature spaces, by target-label availability, and by whether transfer occurs directly or through intermediate domains. Deep DA further organizes one-step approaches by their training criteria and uses deep networks to learn transferable representations.

B. Categorization of Multi-Step Domain Adaptation

Multi-step domain adaptation inserts intermediate domains between source and target to reduce information loss, with transfer organized into hand-crafted, instance-based, or representation-based mechanisms. The survey also distinguishes adaptation settings and training criteria, including class, discrepancy, and adversarial losses.

Multi-Step Domain Adaptation

  • Multi-step domain adaptation selects intermediate domains more related to the source and target, then transfers knowledge through successive one-step adaptations.Its key challenge is selecting and using intermediate domains while minimizing information loss.Source: B. Categorization of Multi-Step Domain… · p. 5

Hand-Crafted

Instance-Based

Representation-Based

Adaptation Settings

  • The survey focuses on supervised and non-supervised domain adaptation, while treating semi-supervised adaptation as a combination of both.Supervised adaptation uses labeled target data; non-supervised adaptation uses unlabeled target data.Source: §IV · p. 5

Training Criteria

  • Class-criterion approaches use target class labels to fine-tune a source-pretrained network, whereas adversarial approaches encourage representations that confuse domain discrimination.Soft-label loss and domain-confusion loss are examples of these training strategies.Source: 1) Discrepancy-Based Approaches · pp. 5–7

Discrepancy-Based Approaches

  • Discrepancy-based adaptation trains source and target networks with classification and distribution-alignment objectives, using fine-tuning or frozen early layers depending on data and domain similarity.MMD measures source-target distance; DDC adapts one layer, while DAN uses multiple adaptation layers and kernels.Source: 1) Discrepancy-Based Approaches · pp. 5–7

2) Adversarial-Based Approaches:

Adversarial-based domain adaptation trains models to make source and target representations indistinguishable, using either generative transformations or non-generative discriminators. The survey organizes these methods by architectural and loss choices, including GAN-based synthesis, domain confusion, gradient reversal, and alternative discrepancy measures.

Overview

  • Adversarial domain adaptation aims to ensure that networks cannot distinguish between source and target domains.The survey categorizes adversarial methods into generative and non-generative subsettings.Source: 2) Adversarial-Based Approaches · p. 9

Non-Generative Models

  • Adversarial methods also include unified architectural variants, selective class-wise discriminators for partial transfer, Wasserstein-distance estimation, and classifier-discrepancy objectives.Selective adversarial networks filter outlier source classes and align distributions in the shared label space.Source: Figure 8 · p. 9, 2) Adversarial-Based Approaches · p. 10, Figure 12 · p. 10

Generative Models

  • Generative approaches create synthetic target data with annotations so the target model can be trained as if domain adaptation were unnecessary.GANs learn the source-to-target transformation without supervision.Source: 2) Adversarial-Based Approaches · p. 9
  • CoGAN uses paired source- and target-domain GANs with shared weights to learn a domain-invariant feature space without correspondence supervision.Early generator layers and late discriminator layers are tied across the two GANs.Source: 2) Adversarial-Based Approaches · p. 9, Figure 9 · p. 9
  • Other generative methods translate source knowledge into target-like images while preserving annotations through domain and real/fake discrimination.These methods use discriminators to enforce content invariance and target-domain visual similarity.Source: 2) Adversarial-Based Approaches · p. 9

Non-Generative Models

  • Non-generative methods learn domain-invariant representations by confusing domain classifiers, as in DANN's gradient reversal architecture.DANN combines source label prediction with domain-confusion objectives using shared feature extraction layers and two classifiers.Source: 2) Adversarial-Based Approaches · p. 10, Figure 11 · p. 10

3) Reconstruction-Based Approaches:

Reconstruction-based approaches learn transferable representations by reconstructing source or target data, often alongside classification or adversarial objectives. They include shared encoder-decoder models, dual-domain reconstruction, and cycle-consistent image translation.

Encoder-Decoder Reconstruction

Adversarial Reconstruction

4) Hybrid Approaches:

Hybrid approaches combine multiple adaptation objectives or criteria within a single method. Examples integrate domain confusion, soft labels, distribution statistics, architecture changes, or reconstruction-based representation separation.

  • Hybrid methods combine complementary adaptation objectives, including domain confusion with soft labels and MMD with classifier adaptation through residual functions.Source: 4) Hybrid Approaches · p. 12
  • Other combinations add class-specific auxiliary weights to MMD or integrate encoder-decoder reconstruction with private and shared representation separation.Source: 4) Hybrid Approaches · p. 12

B. Heterogeneous Domain Adaptation

Heterogeneous domain adaptation addresses domains with nonequivalent feature spaces, often caused by different sensory devices or media. Existing deep methods remain limited and frequently resemble homogeneous adaptation approaches.

1) Discrepancy-Based Approach:

Discrepancy-based methods are constrained when source and target feature dimensions differ, because shared-layer strategies assume compatible inputs. Existing heterogeneous approaches therefore use resizing when possible or specialized multimodal networks when it is not.

2) Adversarial-Based Approach:

Generative models support heterogeneous domain adaptation by generating target-domain data while transferring source-domain information. The surveyed examples use GAN-based objectives for image synthesis and cross-modal translation.

3) Reconstruction-Based Approach:

Adversarial reconstruction addresses heterogeneous domain adaptation by learning bidirectional mappings between photos and sketches. CycleGAN, DualGAN, and DiscoGAN use two generators for these translations, while a later method targets facial artifacts.

V. MULTI-STEP DOMAIN ADAPTATION

Multi-step domain adaptation connects distant source and target domains through one or more intermediate domains. The surveyed methods select or construct intermediates using predefined choices, data mixtures, reconstruction errors, or gradual domain replacement.

  • Intermediate-domain selection is problem specific, and different problems may require different strategies.Source: §V · p. 13
  • When source images and target text differ, annotated images can be crawled in advance as an intermediate domain.Source: A. Hand-Crafted Approaches · p. 13
  • Nighttime light intensity information serves as an intermediate domain for transferring daytime satellite-image knowledge to poverty prediction.Source: A. Hand-Crafted Approaches · p. 13
  • When many intermediate domains are available, automatic selection criteria can construct intermediates by mixing parts of source and target data.Source: B. Instance-Based Approaches · p. 13
  • DDTL relates long-distance domains through multiple intermediate domains by selecting unlabeled data that minimizes reconstruction errors across source, intermediate, and target instances.Source: B. Instance-Based Approaches · p. 13
  • The DLID model constructs intermediate domains by gradually replacing source samples with target samples, while progressive networks transfer representations through lateral connections across domain-specific networks.Source: B. Instance-Based Approaches · p. 13, C. Representation-Based Approaches · p. 13

VI. APPLICATION OF DEEP DOMAIN ADAPTATION

Deep domain adaptation has been applied across classification, recognition, detection, style translation, and semantic segmentation. The surveyed applications address domain shifts using adversarial alignment, reconstruction, weak labels, classifier adaptation, and image-space feature projection.

Application scope

  • Deep domain adaptation is applied to image classification, object recognition, face recognition, object detection, style translation, and related vision tasks.Source: §VI · p. 14

Image classification

  • Image-classification comparisons include Office-31 evaluations across Amazon, DSLR, and Webcam domains, alongside supervised comparisons with DANN and DLID.Source: A. Image Classification · p. 14
  • Cross-domain handwritten-digit recognition evaluates adversarial methods including DANN, CoGAN, and ADDA with VGG-16 as the baseline.Source: A. Image Classification · p. 14

Face recognition

  • BAE addresses face-recognition shifts across view angle, ethnicity, and imaging sensor by shifting source samples toward the target domain and using sparse reconstruction.Source: B. Face Recognition · p. 14

Object detection

  • Object detection requires substantial bounding-box labeling, so adaptation can target classifiers when the window-selection mechanism is treated as domain independent.Source: C. Object Detection · pp. 14–15
  • LSDA uses limited target bounding-box labels and massive weak labels by adapting a target classification layer from a pretrained source model.Source: C. Object Detection · p. 15

Semantic segmentation

  • Semantic-segmentation methods transfer weak class or object-location labels through encoder-decoder and attention architectures.Source: D. Semantic Segmentation · p. 15
  • Unsupervised segmentation combines global adversarial domain alignment with class-aware constrained multiple-instance loss for spatial-layout transfer, while another method projects features into image space for discrimination.Source: D. Semantic Segmentation · pp. 15–16

E. Image-to-Image Translation

Deep domain adaptation uses deep networks to address domain divergence across visual tasks, including image-to-image translation and applications beyond classification. The survey organizes methods by domain setting, transfer distance, and training strategy while identifying limitations in current coverage.

Heterogeneous adaptation

  • Image-to-image translation addresses heterogeneous adaptation when source and target images have different feature spaces.The survey identifies this setting as requiring translation between image domains before adaptation.Source: E. Image-to-Image Translation · p. 16

Translation mechanisms

  • Paired-image translation methods use conditional generative networks to learn mappings, while other approaches address unpaired image domains.Examples include pix2pix for paired images and CoGAN or CycleGAN for unpaired settings.Source: E. Image-to-Image Translation · p. 16
  • Statistical distribution matching can fine-tune deep networks using combined content and style losses to preserve image content while changing its style.The described loss is a linear combination of content and style losses.Source: E. Image-to-Image Translation · p. 16

Open challenges

  • Current methods commonly assume shared feature and label spaces, leaving heterogeneous domains and differing category sets as open challenges.The survey highlights both the predominance of homogeneous adaptation and the shared-label-space assumption.Source: §VII · p. 17

Applications

  • Deep domain adaptation also covers image classification, face recognition, style translation, object detection, semantic segmentation, and person re-identification.The survey notes that relatively few studies address tasks beyond classification and recognition.Source: §VII · p. 17

Related papers

Loading 1802.03601v4…