Source-linked AI summary
Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation
Chao Chen, Zhihong Chen, Boyuan Jiang, Xinyu Jin
TL;DR
Existing deep domain adaptation methods mainly reduce distribution discrepancy, but residual domain shift can leave target samples near class boundaries vulnerable to misclassification. The paper proposes JDDA, combining domain alignment with instance-based and center-based discriminative feature learning. Experiments report stronger transfer performance across difficult and easy tasks, while the method assumes labeled source data and unlabeled target data.
Problem
Domain alignment reduces but does not remove domain shift, leaving edge-of-cluster or far-from-center target samples vulnerable to source-trained hyperplane errors.
Method
JDDA jointly performs domain alignment and instance-based and center-based discriminative feature learning to improve intra-class compactness and inter-class separability.
Results
JDDA outperforms comparison methods on most Office-31 transfer tasks, performs best overall, and improves difficult transfers while remaining comparable on easy transfers.
Takeaways & Limitations
Learning discriminative features in the shared space can further mitigate domain shift and benefit final classification and transfer performance.
Takeaways & Limitations
The approach is evaluated under unsupervised adaptation with labeled source data and unlabeled target data.
Abstract
from arXiv · showhide
Recently, considerable effort has been devoted to deep domain adaptation in computer vision and machine learning communities. However, most of existing work only concentrates on learning shared feature representation by minimizing the distribution discrepancy across different domains. Due to the fact that all the domain alignment approaches can only reduce, but not remove the domain shift. Target domain samples distributed near the edge of the clusters, or far from their corresponding class centers are easily to be misclassified by the hyperplane learned from the source domain. To alleviate this issue, we propose to joint domain alignment and discriminative feature learning, which could benefit both domain alignment and final classification. Specifically, an instance-based discriminative feature learning method and a center-based discriminative feature learning method are proposed, both of which guarantee the domain invariant features with better intra-class compactness and inter-class separability. Extensive experiments show that learning the discriminative features in the shared feature space can significantly boost the performance of deep domain adaptation methods.
Introduction
Domain alignment reduces but does not remove domain shift, leaving edge-of-cluster target samples vulnerable to source-trained hyperplane errors. JDDA addresses this by combining alignment with discriminative feature learning.
- Domain alignment can reduce, but not remove, domain discrepancy, leaving some target samples near cluster edges or far from class centers vulnerable to misclassification.
- Figure 1 contrasts source-only adaptation, alignment-only adaptation, and joint alignment with discriminative feature learning using source-trained hyperplanes and domain-invariant features.
- JDDA jointly learns domain alignment and discriminative deep features for unsupervised domain adaptation.
- The proposed framework uses instance-based and center-based discriminative learning strategies to improve intra-class compactness and inter-class separability.
Related Work
Deep domain adaptation methods include shared-weight or related-parameter two-stream CNNs, discrepancy-based alignment, reconstruction, domain separation, selective learning, and adversarial adaptation.
- Two-stream CNN approaches represent source and target models with either shared weights or related but non-shared parameters.
- Discrepancy-based methods add losses that minimize source-target distribution differences in a shared feature space, using metrics such as MMD, multi-kernel MMD, and CORAL.
- Reconstruction-based methods jointly learn shared representations for source label prediction and reconstruction of unlabeled target samples.
- Other approaches use domain-private subspaces, gradually select useful unlabeled data from intermediate domains, or optimize adversarial objectives.
Our Approach
JDDA combines shared-weight two-stream domain alignment with discriminative feature learning at the bottleneck layer. Its instance-based and center-based losses improve intra-class compactness and inter-class separability while remaining compatible with mini-batch SGD.
- Architecture: The shared-weight two-stream CNN processes source and target data separately, applying domain discrepancy and discriminative losses in the bottleneck layer.The source stream uses labeled data and the target stream uses unlabeled data.
- Domain Alignment: CORAL aligns source and target bottleneck-feature covariances to reduce domain discrepancy.The alignment is performed on mini-batches during training.
- Instance-Based Discriminative Learning: Instance-based learning pulls same-class feature pairs within margin m1 and pushes different-class pairs beyond margin m2.The loss uses pairwise bottleneck-feature distances and balances intra-class compactness against inter-class separability with α.
- Center-Based Discriminative Learning: The global class center is initialized from the first batch and updated with subsequent mini-batches using a learning rate γ.This modification approximates centers that would otherwise require averaging features over the full training set.
- Center-Based Discriminative Learning: Center-based learning penalizes feature-to-class-center distances while enforcing large margins between centers of different categories.Unlike Center Loss, the proposed objective includes both intra-class compactness and inter-class separation.
- Efficiency: Center-based learning is computationally cheaper than instance-based learning and is expected to converge faster because it uses global information each iteration.With mini-batch SGD, the reported complexities are O(bc + c2) for center-based learning and O(b2) for instance-based learning.
Experiments
The approach is evaluated against state-of-the-art deep domain adaptation methods on two image-classification adaptation datasets. The evaluation includes Office-31 and a large-scale digital recognition dataset.
- Evaluation compares the approach with several state-of-the-art deep domain adaptation methods.
- Office-31 is one of the two image classification adaptation datasets used for evaluation.
- The second evaluation dataset is a large-scale digital recognition dataset.
Setup
The experiments use Office-31 and a five-domain digital recognition benchmark, evaluating six and four cross-domain transfer tasks respectively. Comparisons cover methods that learn domain-invariant feature representations.
- Office-31: Office-31 contains 4,110 images in 31 classes across Amazon, Webcam, and DSLR domains.
- Office-31: The Office-31 evaluation covers six transfers: A→W, W→A, W→D, D→W, A→D, and D→A.
- Digital recognition: The digital recognition benchmark contains SVHN, MNIST, MNIST-M, USPS, and synthetic digits, all with 10 digit classes.
- Digital recognition: The digital recognition evaluation uses SVHN→MNIST, MNIST→MNIST-M, MNIST→USPS, and synthetic digits→MNIST.
- Compared methods: Comparisons include DDC, DAN, DANN, CMD, ADDA, and CORAL, which learn domain-invariant feature representations.
Implementation Details
Implementation uses domain-specific architectures and training settings tailored to the two benchmarks. Office-31 uses partial ResNet fine-tuning, while digital recognition uses a modified LeNet.
- Model and data: Office-31 experiments fine-tune ImageNet-pretrained ResNet representations from the pool5 layer under unsupervised adaptation.
- Model and data: Digital recognition experiments use a modified LeNet to verify the approach.
- Optimization: All methods are implemented in TensorFlow and trained with the Adam optimizer.
- Optimization: For Office-31, only fully connected layers and the final ResNet block are updated because of the dataset’s small sample size.
- Optimization: Training uses batches of 256 samples, learning rate η = 10−4, global class center learning rate γ = 0.5, and margins m1 = 0 and m2 = 100.
Result and Discussion
JDDA outperforms comparison methods on most Office-31 transfers and all digital-recognition tasks, with especially large gains on difficult domain shifts. The authors attribute this to discriminative learning that promotes compact classes and separated class boundaries.
- Office-31 results: JDDA outperforms all comparison methods on most Office-31 transfer tasks.
- Office-31 results: JDDA improves substantially on difficult Office-31 transfers A→W and W→A while remaining comparable on easier D→W and W→D.
- Office-31 discussion: JDDA performs best and sets new state-of-the-art results in the Office-31 comparison.
- Digital recognition results: JDDA outperforms all comparison methods on every digital-recognition transfer task, with huge margins on SVHN→MNIST and MNIST→MNIST-M.
- Digital recognition discussion: The results suggest that the discriminative loss is effective for large-scale domain adaptation.
Analysis
Discriminative loss produces more compact and separated features while improving target-domain alignment and convergence. Its benefits depend on balancing domain alignment with discriminative learning.
- Feature Visualization: Features with the proposed discriminative loss become more compact and well separated than features learned without it.The comparison uses 2D features from 2,000 randomly selected source-domain samples in SVHN.
- Feature Visualization: JDDA features show larger inter-class distances and better category alignment across source and target domains than CORAL features on SVHN→MNIST.Category-based t-SNE shows fewer points in inter-class gaps, while domain-based t-SNE shows better category alignment between domains.
- Convergence Performance: Incorporating discriminative loss achieves much better target-domain convergence performance on SVHN→MNIST and MNIST→MNIST-M.JDDA generally converges quickly and stably to the lowest test error, while JDDA-C converges fastest because it uses global cluster information.
- Parameter Sensitivity: Average accuracy increases and then decreases as λ2 increases, forming a bell-shaped curve on SVHN→MNIST.The result indicates that transfer performance depends on a proper trade-off between domain alignment and discriminative feature learning.
- Parameter Sensitivity: Better convergence performance is obtained when λC2 is appropriately increased.The paper relates this to keeping feature-alignment speed synchronized with changes in source-domain features under discriminative loss.
Conclusion
The paper jointly learns domain-aligned and discriminative features for unsupervised deep domain adaptation. Instance-based and center-based strategies improve compactness and separability, benefiting alignment and classification.
- Conclusion: JDDA jointly performs domain alignment and discriminative feature learning for unsupervised deep domain adaptation.The paper proposes instance-based and center-based discriminative learning strategies.
- Conclusion: The proposed methods enforce better intra-class compactness and inter-class separability in the shared feature space.Better clustering can facilitate domain alignment, while larger margins can reduce target-domain misclassification near cluster edges or centers.
- Conclusion: The paper identifies future work on further mitigating domain shift in the aligned feature space using other constraints for domain-invariant features.This is stated as a future research direction rather than an evaluated component of JDDA.