Source-linked AI summary
Contrastive Adaptation Network for Unsupervised Domain Adaptation
Guoliang Kang, Lu Jiang, Yi Yang, Alexander G Hauptmann
TL;DR
UDA must predict on an unlabeled target domain, while class-agnostic alignment can misalign classes and harm target generalization. CAN introduces class-aware Contrastive Domain Discrepancy, alternating target-label clustering and feature adaptation during end-to-end training. It achieves the best-published Office-31 result and competitive VisDA-2017 performance.
Problem
UDA lacks target-domain labels, while class-agnostic discrepancy minimization may align different classes and yield less discriminative target decision boundaries.
Method
CAN optimizes CDD by minimizing intra-class and maximizing inter-class domain discrepancy, alternating target-label clustering with feature adaptation.
Results
CAN achieves the best-published result on Office-31 and competitive performance against state-of-the-art methods on VisDA-2017.
Takeaways & Limitations
Explicitly modeling intra-class and inter-class discrepancy enables class-aware alignment through end-to-end mini-batch training.
Takeaways & Limitations
CDD requires unknown target labels to be estimated and can be hindered when mini-batches lack both source and target samples for a class.
Abstract
from arXiv · showhide
Unsupervised Domain Adaptation (UDA) makes predictions for the target domain data while manual annotations are only available in the source domain. Previous methods minimize the domain discrepancy neglecting the class information, which may lead to misalignment and poor generalization performance. To address this issue, this paper proposes Contrastive Adaptation Network (CAN) optimizing a new metric which explicitly models the intra-class domain discrepancy and the inter-class domain discrepancy. We design an alternating update strategy for training CAN in an end-to-end manner. Experiments on two real-world benchmarks Office-31 and VisDA-2017 demonstrate that CAN performs favorably against the state-of-the-art methods and produces more discriminative features.
1. Introduction
UDA uses labeled source data and unlabeled target data, but class-agnostic discrepancy minimization can misalign classes and weaken target generalization. CAN addresses this with class-aware Contrastive Domain Discrepancy and alternating end-to-end training, achieving strong benchmark performance.
- Motivation: UDA addresses target-domain prediction when labels are available only for source data, where obtaining target annotations can be difficult or expensive.The absence of in-domain labeled data limits the application of data-fitting models to real-world problems.
- Problem: Class-agnostic domain discrepancy methods may align samples from different classes, producing sub-optimal solutions and less discriminative target decision boundaries.Such solutions can fit source data well while generalizing poorly to the target domain.
- Approach: CDD minimizes intra-class domain discrepancy while maximizing inter-class discrepancy to perform class-aware alignment across domains.This draws same-class source and target samples together and pushes different-class samples apart.
- Approach: CAN facilitates CDD optimization by alternating target-label-hypothesis estimation through clustering with feature adaptation during end-to-end training.Ambiguous target data and classes are zeroed out when estimating CDD.
- Results: The method achieves the best-published result on Office-31 and competitive performance against state-of-the-art methods on VisDA-2017.The paper also reports ablation studies evaluating key framework components.
2. Related Work
Related UDA work commonly seeks domain-invariant representations by minimizing discrepancy between source and target domains, including MMD- and JMMD-based approaches.
- Class-agnostic domain alignment: Class-agnostic UDA methods minimize source-target domain discrepancy to learn domain-invariant representations.Examples include domain confusion, DAN with MMD, and JAN with Joint MMD.
3. Methodology
CAN performs class-aware unsupervised domain adaptation by measuring conditional discrepancies and alternating target-label clustering with feature adaptation. Its objective combines contrastive domain alignment with supervised source classification, while filtering ambiguous target assignments and adapting domain-specific features.
- Contrastive Domain Discrepancy: CDD minimizes intra-class domain discrepancy and maximizes inter-class discrepancy to compact same-class features and separate different-class features.For c1 = c2, the metric measures intra-class discrepancy; for c1 ≠ c2, it measures inter-class discrepancy.
- Contrastive Domain Discrepancy: CAN estimates conditional discrepancies across source and target domains using class labels for source samples and estimated labels for target samples.The target-label hypothesis is required to compute class-specific discrepancy masks and is obtained during training.
- Overall Objective: CAN incorporates CDD over domain-specific fully connected layers into an objective combining the discrepancy penalty with source cross-entropy loss.The penalty weight is β, and minimizing the objective performs class-aware domain alignment.
- Alternative Optimization: CAN alternates spherical K-means clustering of target representations with back-propagation that updates features using CDD and source cross-entropy.Clustering updates the target-label hypothesis with fixed features; back-propagation then adapts features using the updated labels.
- Target Label Hypothesis: Target clusters use the number of underlying classes, initialize each center from the corresponding source center, and represent samples with first task-specific-layer activations.Cosine dissimilarity is used for distances between feature-space points.
- Sampling and Filtering: Ambiguous target samples and classes are discarded from CDD estimation, while class-aware sampling supports efficient mini-batch optimization.The method also independently samples labeled source data for the cross-entropy loss and CDD estimation.
4. Experiments
CAN achieves strong domain-adaptation performance across Office-31 and VisDA-2017 while producing more discriminative target features. Ablations support the roles of contrastive discrepancy, alternative optimization, and class-aware sampling.
- Comparison with the state-of-the-art: CAN outperforms baseline methods across all six Office-31 tasks, improving average accuracy over JAN by 6.3% and over MADA by 5.4%.The evaluation uses classification accuracy on Office-31 with ResNet-50.
- Feature visualization: t-SNE visualizations on Office-31 W →A show CAN learning greater intra-class compactness and larger inter-class margins than JAN.The visualized representations use input activations from the last fully connected layer.
- Comparison with the state-of-the-art: 87.2% mean accuracy on VisDA-2017 validation exceeds self-ensembling’s 84.3% by 2.9%.The comparison covers accuracy over 12 classes on the validation set.
- Comparison with the state-of-the-art: 87.4% accuracy on the VisDA-2017 test set is comparable to the second-place leaderboard result of 87.7%.This result uses a single ResNet-101 model without ensemble or additional data augmentation.
- Ablation studies: Adding inter-class discrepancy improves adaptation performance over intra-class discrepancy alone.The authors attribute the benefit to reducing source overfitting when intra-class discrepancy cannot be completely eliminated.
- Ablation studies: CAN’s alternating optimization and class-aware sampling are validated through ablations, while CDD alone remains somewhat robust to pseudo-label noise.The ablations compare variants that remove alternative optimization or class-aware sampling, and a variant without alternative optimization still improves over class-agnostic methods.
- Ablation studies: Updating pseudo-labels during training improves over fixed pseudo-label training, but CAN performs better by explicitly modeling class-aware domain discrepancy.The fixed pseudo-label model matches the initial clustering accuracy, whereas the updating variant is better but remains below CAN.
- Ablation studies: CAN’s accuracy is generally less sensitive to β, increasing over a broad range before declining as the balance weight becomes too large.The reported examples are Office-31 tasks A →D and D →A, with similar trends on other tasks.
5. Conclusion
The paper proposes Contrastive Adaptation Network for class-aware alignment in UDA, explicitly modeling intra-class and inter-class domain discrepancies through end-to-end mini-batch training.
- CAN explicitly models and optimizes intra-class and inter-class domain discrepancies for class-aware UDA alignment.