Source-linked AI summary
Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives
Xiaofeng Liu, Chaehwa Yoo, Fangxu Xing, Hyejin Oh, Georges El Fakhri, Je-Won Kang, Jonghye Woo
TL;DR
Deep neural networks need extensive labeled data and often face performance degradation when source-trained models encounter shifted, unlabeled target domains. This review systematically surveys deep UDA methods and applications, explaining their theoretical basis and connections to related tasks. It concludes by identifying current deficiencies and promising directions for deploying models across unseen target domains.
Problem
Deep learning assumes abundant labeled data and i.i.d. training and testing distributions, yet source-trained models can degrade on shifted target domains where labeling is expensive.
Method
The paper systematically reviews and compares deep UDA approaches, applications, theoretical foundations, and connections with related tasks using labeled source and unlabeled target data.
Results
The review provides a comprehensive summary of recent deep UDA approaches, their merits and demerits, successful applications, and current challenges.
Takeaways & Limitations
The identified challenges and future directions concern the deployment of models trained in source domains into unseen and new target domains.
Takeaways & Limitations
The review mainly focuses on covariate-shift alignment, while alignment of other shifts and their combinations remains a future research direction.
Abstract
from arXiv · showhide
Deep learning has become the method of choice to tackle real-world problems in different domains, partly because of its ability to learn from data and achieve impressive performance on a wide range of applications. However, its success usually relies on two assumptions: (i) vast troves of labeled datasets are required for accurate model fitting, and (ii) training and testing data are independent and identically distributed. Its performance on unseen target domains, thus, is not guaranteed, especially when encountering out-of-distribution data at the adaptation stage. The performance drop on data in a target domain is a critical problem in deploying deep neural networks that are successfully trained on data in a source domain. Unsupervised domain adaptation (UDA) is proposed to counter this, by leveraging both labeled source domain data and unlabeled target domain data to carry out various tasks in the target domain. UDA has yielded promising results on natural image processing, video analysis, natural language processing, time-series data analysis, medical image analysis, etc. In this review, as a rapidly evolving topic, we provide a systematic comparison of its methods and applications. In addition, the connection of UDA with its closely related tasks, e.g., domain generalization and out-of-distribution detection, has also been discussed. Furthermore, deficiencies in current methods and possible promising directions are highlighted.
1. Introduction
Deep learning depends on abundant labeled data and aligned training and testing distributions, but domain shift can severely degrade performance on unlabeled target domains. This review surveys deep UDA methods, applications, and connections to related tasks.
- Supervised deep learning relies on vast labeled datasets and the i.i.d. assumption between training and testing data.
- Domain shift between source and target distributions can severely degrade a trained model’s performance on out-of-distribution target data.The paper illustrates this concern with disease diagnosis across populations.
- UDA transfers knowledge from labeled source data to unseen, heterogeneous, and unlabeled target domains while mitigating domain shifts.
- Domain adaptation is treated as a special case of transfer learning in which labeled data are available only in the source domain.
- The review provides broad coverage and comparison of deep UDA models, algorithms, methods, and applications, addressing gaps in earlier reviews.It covers areas including natural images, video, NLP, time series, medical imaging, climate, and geosciences.
2. Overview
UDA learns from labeled source and unlabeled target samples to improve target-domain generalization despite differing distributions. Its theory frames target error through source loss, domain divergence, and conditional-distribution discrepancy, while practice mainly studies covariate shift.
- UDA uses an i.i.d. labeled source dataset and an i.i.d. unlabeled target dataset to improve a model’s target-domain generalization.The target data are sampled from the target marginal distribution, and both datasets are used during learning.
- UDA methods aim to minimize divergence between source and target domains to lower an upper bound on target-domain generalization error.The bound also includes source loss and a conditional-label discrepancy term described as negligible.
- Domain shifts include covariate shift in p(x), conditional shift in p(x|y), label shift in class proportions, and concept shift in labeling rules.
- The review mainly focuses on covariate-shift alignment because it is the most commonly studied setting in UDA.It discusses alignment of other shifts and their combinations as future research directions.
3. Methodology
Deep UDA methods primarily reduce source–target discrepancy through latent feature alignment, adaptive adversarial divergence estimation, or data-level domain transformation. The review also covers class-conditioned, graph-based, and combined approaches.
- Mainstream UDA methods include statistic-divergence alignment, adversarial training, generative domain mapping, normalization alignment, ensemble methods, and self-training.
- Statistic Divergence Alignment: Domain-invariant representation methods minimize source–target discrepancy in a latent feature space using measures such as MMD, CORAL, CDD, Wasserstein distance, and graph matching.
- Statistic Divergence Alignment: MMD compares sample means under a characteristic RKHS function, while CORAL aligns second-order feature statistics across domains.
- Statistic Divergence Alignment: CDD incorporates class labels into MMD, enlarging cross-class divergence and minimizing within-class divergence while estimating missing target labels through clustering.
- Adversarial Learning: Adversarial UDA learns feature-level domain divergence with a discriminator while training a classifier on labeled source data.
- Adversarial Learning: DANN uses a gradient-reversal domain discriminator, whereas ADDA initializes a target model from source training before adversarial adaptation.
- Adversarial Learning: Wasserstein-based discriminators are proposed as alternatives to JS divergence, particularly for large discrepancies between distributions.
3.3 Normalization Statistics
Normalization-statistics methods adapt models by addressing domain-specific batch statistics, while other approaches transform target data or use self-training and uncertainty calibration. These methods face constraints from statistic expressiveness and noisy pseudo-labels.
- Normalization Statistics: Batch normalization supports faster training, smoother optimization, and stable convergence through domain-sensitive batch statistics.
- Normalization Statistics: AdaBN adapts a trained model by replacing source-domain batch-normalization statistics with target-domain statistics while fixing other parameters and weights.
- Normalization Statistics: Mean and variance are domain-specific, but forcing them to match can reduce network expressiveness; high-order statistics may become shareable after alignment.
- Domain Style Transfer: Data-level methods render target-style data from source inputs and train classifiers or segmentation networks on the generated data, sometimes jointly with GANs.
- Domain Style Transfer: Cycle-based image translation can distort local structures, so MIND reconstruction loss is used to preserve modality-independent neighborhood information.
- Self-training: Self-training alternates pseudo-label creation and model training, but noisy target pseudo-labels require confidence selection, soft labels, or additional supervision.
- Self-training: Self-training extends to generative tasks by using Bayesian uncertainty masks to control confident pseudo-labels for continuous pixel values.
3.6 Self-supervision
Self-supervised UDA introduces objectives computable from unlabeled data, while related methods exploit clustering, task-specific representations, model selection, and combinations of complementary alignment strategies.
- Self-supervision: Self-supervised UDA uses unlabeled data to define pretext tasks such as context prediction, image rotation, reconstruction, or jigsaw prediction.
- Self-supervision: Some methods combine source classification with target reconstruction, while others use in-domain and across-domain self-supervision to reduce source-label requirements.
- Clustering and Decision Boundaries: Clustering-based UDA places same-class target samples near one another and seeks decision boundaries in low-density regions.
- Clustering and Decision Boundaries: Virtual adversarial domain adaptation and teacher-based iterative refinement use this clustering assumption to improve target decision boundaries.
- Task-discriminative Adaptation: Adversarial dropout compares two dropout networks as a discriminator to generate target-discriminative features, with channel dropout extending the approach to convolutional layers.
- Task-discriminative Adaptation: Other methods use model selection, class-wise or fine-grained prototypes, dual mixup, domain randomization, graph reasoning, or self-semantic contours.
- Combined Methods: Feature-level adversarial alignment, image-level generative mapping, batch-normalization alignment, entropy minimization, and prototype normalization can be combined sequentially or jointly.
4. Applications
UDA has been applied across diverse application areas, including visual, language, temporal, medical, climate, and geoscience problems. Some applications use general UDA principles, while others exploit dataset-specific characteristics.
- UDA has been applied to image perception and understanding, video analysis, natural language processing, time-series analysis, medical imaging, climate, and geoscience.
- Application-specific methods complement general UDA principles by exploiting the characteristics of the relevant training and testing datasets.
4.1 Image Analysis
Image analysis is the most explored UDA application area, spanning classification, detection, segmentation, synthesis, and related tasks across varied domain shifts. The reviewed methods include adversarial alignment, image-level generation, pseudo-label self-training, and specialized approaches for ordinal labels.
- Natural image analysis is the most explored UDA area, covering classification, detection, segmentation, generation, and captioning tasks.
- Image classification studies address object and face recognition under shifts in pose, illumination, expression, age, ethnicity, and imaging modality.
- Common benchmarks include Office-31, VisDA17, and DomainNet, which contains ∼0.6M images with 345 sub-classes from 24 meta-classes.
- Ordinal classification first induces a non-trivial ordinal distribution before projecting data into a latent space, using recursively conditional Gaussian modeling.
- Object detection UDA uses adversarial feature alignment, image-level generative mapping, and pseudo-label self-training for progressive adaptation.
- Segmentation provides pixel-wise object delineation, making low-density and prototype-based UDA methods difficult to apply at scale.
4.2 Medical Image Analysis
Medical image analysis is a major UDA application because clinical imaging involves sensitive data and diverse modalities. Reviewed work covers segmentation, classification, diagnosis, generation, and source-free adaptation, though some studies remain proof-of-concept evaluations.
- Medical imaging uses diverse modalities, including CT, MRI, X-ray, and optical coherence tomography, each posing distinct challenges.
- UDA has been applied to medical image segmentation, classification, generation, and diagnosis across several imaging modalities.
- Self-ensembling improved model generalization in a small MRI-based segmentation evaluation, which served as a proof-of-concept rather than a real medical application.
- Source-free UDA adapts pre-trained segmentation models without source-domain data during adaptation to address sensitive patient information.
- Medical image translation and synthesis evaluations include brain MRI datasets from three IXI centers and paired cine and tagged tongue MRI data from two clinical sites.
4.3 Video Analysis
Video UDA addresses the difficulty of collecting and annotating large volumes of temporally rich video data. The reviewed approaches transfer from images, align domains across spatial and temporal dimensions, and support recognition, segmentation, restoration, and other tasks.
- Video annotation is labor-intensive and time-consuming because useful models require spatiotemporal information across many frames.
- Video UDA covers action recognition, person re-identification, action segmentation, captioning, quality assessment, and artifact reduction.
- Image-to-video adaptation transfers knowledge from large-scale image datasets to video analysis and addresses visual-quality differences between still images and video frames.
- TA3N aligns source and target domains spatio-temporally by encoding features with an attention mechanism to address unseen temporal dynamics.
- Video UDA also includes translation, self-learning for simulated-to-real segmentation, face recognition, person re-identification, and video captioning.
- Evaluation datasets include UCF-Olympic, UCF-HMDBsmall, UCF-HMDBfull, Kinetics-Gameplay, Kinetics, NEC-DRONE, DIV2K, BSD68, and Set12.
4.4 Natural Language Processing
NLP UDA responds to the cost of annotating large language datasets and is especially developed for sentiment analysis. Methods include adversarial training, pre-training, and adaptation across diverse language tasks and datasets.
- NLP UDA is motivated partly by the costly and demanding annotation of large volumes of language data.
- Sentiment analysis is the most explored NLP application, with early work using domain-adversarial neural networks and gradient reversal layers.
- Adversarial training can be more effective when source and target language datasets differ in both language and content.
- NLP UDA has been applied to dependency parsing, POS tagging, relation extraction, trigger identification, language identification, and political data classification.
- Transformer-based adaptation commonly combines large-scale source pre-training with target-domain or task-specific pre-training and fine-tuning.
- Image captioning UDA combines visual encoders with text generators and can use adversarial training on paired source and unpaired target data.
- Evaluation spans sentiment classification across English, Chinese, and Arabic, plus dependency parsing and relation extraction datasets.
4.5 Time Series Data Analysis
Time-series UDA addresses distribution discrepancies across datasets and subjects while modeling temporal dependencies that visual-data methods may neglect. Applications span medical signals, recognition, prediction, anomaly detection, fault diagnosis, and geoscience data, but methods designed for natural images can be ineffective for temporally structured satellite-image series.
- Applications and benchmarks: UDA has been applied to sleep and arrhythmia classification, motor imagery, emotion and speech recognition, inertial tracking, maneuver prediction, anomaly detection, fault diagnosis, and lifetime prediction.Sleep-signal benchmarks use MASS as source and Sleep-EDF variants as targets; DEAP and DREAMER benchmark emotion recognition.
- Medical time-series data: UDA targets distribution discrepancies across medical datasets and subjects arising from demographic variation.Examples include age, sex, and illness-related differences in medical data.
- Medical time-series data: Adversarial training and self-training with domain, subject, and stage discriminators preserve local sleep-stage structures.The approach is illustrated for sleep-signal adaptation.
- Temporal modeling: Time-series UDA methods incorporate recurrent architectures to capture temporal dependencies that visual-data methods may neglect.R-DANN uses LSTM networks, while VRADA uses a variational recurrent neural network.
- Climate science and geosciences: In geoscience, UDA addresses shifts across location and time, including satellite sensing, seismic-image segmentation, crop classification, and corn-yield prediction.Natural-image UDA methods were ineffective for SITS classification because of its temporal nature, motivating explicit temporal-shift modeling.
5. Promising Directions
The review identifies promising UDA directions spanning realistic domain shifts, privacy-preserving adaptation, continuously changing environments, foundation models, and links to related adaptation settings. It also notes that current UDA performance remains unsatisfactory in many cases and that semi-supervised alternatives can use limited target labels.
- 5.1 Realistic Shift Assumption: Current UDA methods mainly address covariate shift, while conditional, label, and concept shifts remain comparatively underexplored.The review argues that realistic applications require assumptions tailored to the task and may need joint consideration of conditional and covariate shifts.
- 5.2 Partial/Open-set Domain Adaptation: Partial UDA is a label-shift case with zero-probability target classes, whereas open-set and universal UDA address novel target classes.The category mismatch in partial UDA can cause negative transfer; the review proposes broader label-shift methods and novel-class discovery.
- 5.3 Source-free domain adaptation: Source-free UDA can reduce privacy and intellectual-property concerns by sharing only a pretrained source model instead of large labeled source datasets.Future frameworks should also support low-cost mobile training, continuously changing environments, and avoidance of catastrophic forgetting.
- 5.4 Continuous and test time adaptation: Existing UDA typically models discrete stationary domains, motivating lifelong adaptation and datasets representing smoothly evolving distribution shifts.Mixup or interpolation could hallucinate intermediate data between substantially different domains.
- 5.5 Adaptation in Foundation Model Era: Foundation models may improve robustness to covariate and conditional shifts, but label and concept shifts remain challenging without target-domain data.The review highlights label-shift UDA and investigation of foundation-model generality as important directions.
- 5.6 Semi-supervised Domain Adaptation: When UDA performance is unsatisfactory, semi-supervised domain adaptation can incorporate labeled source data, labeled target data, and unlabeled target data.Semi-supervised domain adaptation remains largely underexplored for object detection and image generation.
6. CONCLUSION
The review systematically examines deep learning-based UDA approaches, their applications, merits, and demerits. It also identifies current challenges as promising directions for future research.
- The paper systematically reviews deep learning-based unsupervised domain adaptation approaches.
- It summarizes recent deep UDA approaches together with their merits and demerits.
- The review covers several successful applications of deep UDA methods.
- It identifies challenges in current deep UDA approaches as promising future research directions.