Source-linked AI summary
Learning Disentangled Semantic Representation for Domain Adaptation
Ruichu Cai, Zijian Li, Pengfei Wei, Jie Qiao, Kun Zhang, Zhifeng Hao
TL;DR
Domain adaptation struggles to learn domain-invariant representations when semantic and domain information are entangled. DSR models independent semantic and domain latent variables with a variational auto-encoder and dual adversarial network, and achieves state-of-the-art performance on several benchmark datasets.
Problem
Domain adaptation must extract domain-invariant semantic information despite entangled semantic and domain information in the feature space.
Method
DSR reconstructs independent semantic and domain latent variables with a variational auto-encoder and disentangles them using a dual adversarial network.
Results
DSR achieves state-of-the-art performance on several domain adaptation benchmark datasets.
Takeaways & Limitations
Disentangled semantic latent variables provide the representation used to adapt and predict labels across domains.
Takeaways & Limitations
The approach assumes that semantic and domain factors are independent.
Abstract
from arXiv · showhide
Domain adaptation is an important but challenging task. Most of the existing domain adaptation methods struggle to extract the domain-invariant representation on the feature space with entangling domain information and semantic information. Different from previous efforts on the entangled feature space, we aim to extract the domain invariant semantic information in the latent disentangled semantic representation (DSR) of the data. In DSR, we assume the data generation process is controlled by two independent sets of variables, i.e., the semantic latent variables and the domain latent variables. Under the above assumption, we employ a variational auto-encoder to reconstruct the semantic latent variables and domain latent variables behind the data. We further devise a dual adversarial network to disentangle these two sets of reconstructed latent variables. The disentangled semantic latent variables are finally adapted across the domains. Experimental studies testify that our model yields state-of-the-art performance on several domain adaptation benchmark datasets.
1 Introduction
Unsupervised domain adaptation must transfer labeled source knowledge to an unlabeled target despite domain shift. The paper attributes false alignment to entangled semantic and domain information and proposes DSR to recover disentangled latent variables for semantic prediction.
- Unsupervised domain adaptation transfers labeled source data to an unlabeled target domain under domain shift.
- Entangled semantic and domain information can distort feature representations, causing false alignment between semantically different samples.The toy example describes a Peppa Pig appearing near a hair drier when residual domain information remains.
- DSR assumes independent semantic and domain latent variables and reconstructs their disentangled latent space.
- A variational auto-encoder and dual adversarial network disentangle reconstructed semantic and domain variables, then use semantic variables to predict target labels.
- DSR outperforms state-of-the-art unsupervised domain adaptation methods on standard benchmark datasets.
2 Related Work
Prior unsupervised domain adaptation work aligns deep features or semantic representations across domains. Semantic alignment methods often depend on target pseudo labels, whose uncertainty can accumulate errors.
- Deep feature learning methods seek domain-invariant representations by aligning feature distributions across domains.Methods use discrepancy measures such as maximum mean discrepancy or domain-confusion losses.
- Semantic alignment methods reduce cross-domain variation among samples with the same label, often using labeled source and pseudo-labeled target data.
- Target pseudo-label uncertainty can cause error accumulation in semantic alignment methods.The paper positions variational-autoencoder and adversarial-learning techniques as its alternative for extracting domain-invariant semantic representations.
3 Disentangled Semantic Representation Model
DSR models data with independent semantic and domain latent variables, reconstructs them with a VAE, and disentangles them through dual adversarial learning for domain adaptation. Its analysis links disentanglement and reduced target-domain error under the independence assumption.
- 3 Disentangled Semantic Representation Model: DSR assumes independent semantic variables zy and domain variables zd, then reconstructs these factors from features using a VAE-like architecture.The framework encodes G(x) into zy and zd and reconstructs the feature from both latent variables.
- 3 Disentangled Semantic Representation Model: DSR uses dual adversarial modules to retain label information in zy while removing domain information, and to retain domain information in zd while removing semantic information.The semantic module uses label and domain classifiers with gradient reversal, while the domain module uses maximum entropy for target-domain label predictions.
- 3 Disentangled Semantic Representation Model: The reconstruction and disentanglement components are combined into a total training objective with fixed hyperparameters β=1 and γ=1.The model is trained on the source domain before adapting the trained classifier to target domains.
- 3 Disentangled Semantic Representation Model: Under independent semantic and domain factors, Theorem 1 states that disentangling the representation is helpful and might be necessary for obtaining lower classification error.The analysis decomposes source and target errors using the independence property between zy and zd.
- 3 Disentangled Semantic Representation Model: Theorem 2 connects lower source-domain classification error with a tighter upper bound on target-domain error for the learned representation.The bound is formulated using source and target errors, representation-induced distributions, and the errors of a shared hypothesis.
4 Experiments and Results
Experiments evaluate DSR on Office-31 and Office-Home, then examine feature alignment and the dual adversarial module. DSR generally outperforms competing methods, especially on challenging transfers, while showing dataset-dependent weaknesses.
- Benchmark Results: DSR outperforms other baselines on most Office-31 and Office-Home transfer tasks.The Office-31 comparison is reported in Table 1, and the Office-Home comparison in Table 2.
- Office-31 Result: DSR substantially improves hard Office-31 transfers such as D→A and W→A, while remaining comparable on simpler A→W and D→W tasks.
- Office-Home Result: DSR performs especially well when Art or Clipart is the source domain, but performs slightly worse than CDAN-M when Real World is the source.The authors attribute the Real World weakness to ambiguous samples whose semantic distinctions are difficult to disentangle.
- The Study of the Disentangled Semantic Representation: On Ar→Cl, t-SNE visualizations show DSR achieving the best alignment, whereas DANN and MSTN contain many falsely aligned samples.The comparison uses deep features extracted by the three methods, with source and target samples visualized separately.
- Ablation Study of the Dual Adversarial Learning: Removing domain adversarial learning reduces performance even with δ = 2, indicating that the module helps separate semantic information into z_y and domain information into z_d.The ablated model is called DSR WD.
5 Conclusion
The paper presents a disentangled semantic representation model for unsupervised domain adaptation. It recovers a latent space with a variational autoencoder and disentangles representations through dual adversarial learning.
- DSR is a disentangled semantic representation model designed for unsupervised domain adaptation.
- The approach recovers a latent space with a variational autoencoder and disentangles representations using dual adversarial learning.
- The method extracts disentangled semantic representations on the recovered latent space according to the data-generation causal model.