Source-linked AI summary
Generalized Source-free Domain Adaptation
Shiqi Yang, Yaxing Wang, Joost van de Weijer, Luis Herranz, Shangling Jui
TL;DR
The paper addresses source-free adaptation methods that may forget source-domain performance while adapting to unlabeled target data. It proposes G-SFDA with LSC and SDA, reporting 85.4% target performance on VisDA while maintaining good source performance.
Problem
Existing SFDA methods adapt using only a source-pretrained model and target data but do not consider preserving performance on both source and target domains.
Method
G-SFDA combines LSC, which clusters target features with nearest neighbors, and SDA, which activates domain-specific channels and regularizes gradients during adaptation.
Results
85.4% target performance on VisDA is reported as state of the art, while the method maintains good source performance and works across single or multiple target domains.
Takeaways & Limitations
The learned model can perform well on both source and target domains after source-free adaptation, including in continual multi-domain settings.
Takeaways & Limitations
SDA prevents forgetting only in the last two layers and does not prevent all source-domain forgetting.
Abstract
from arXiv · showhide
Domain adaptation (DA) aims to transfer the knowledge learned from a source domain to an unlabeled target domain. Some recent works tackle source-free domain adaptation (SFDA) where only a source pre-trained model is available for adaptation to the target domain. However, those methods do not consider keeping source performance which is of high practical value in real world applications. In this paper, we propose a new domain adaptation paradigm called Generalized Source-free Domain Adaptation (G-SFDA), where the learned model needs to perform well on both the target and source domains, with only access to current unlabeled target data during adaptation. First, we propose local structure clustering (LSC), aiming to cluster the target features with its semantically similar neighbors, which successfully adapts the model to the target domain in the absence of source data. Second, we propose sparse domain attention (SDA), it produces a binary domain specific attention to activate different feature channels for different domains, meanwhile the domain attention will be utilized to regularize the gradient during adaptation to keep source information. In the experiments, for target performance our method is on par with or better than existing DA and SFDA methods, specifically it achieves state-of-the-art performance (85.4%) on VisDA, and our method works well for all domains after adapting to single or multiple target domains. Code is available in https://github.com/Albert0147/G-SFDA.
1. Introduction
The paper introduces G-SFDA for adapting a source-pretrained model using only unlabeled target data while preserving performance on both source and target domains. It combines LSC for target adaptation with SDA for retaining source information.
- Existing SFDA methods focus on target performance by fine-tuning the source model, which can cause forgetting on old domains.
- 85.4% target performance on VisDA is reported as state of the art, while the method maintains good source performance.
- G-SFDA adapts a source-pretrained model to target domains without source data while maintaining source-domain performance.
- LSC clusters each target feature with its nearest neighbors to exploit local semantic structure during source-free adaptation.
- SDA activates different feature channels for different domains and regularizes backpropagation gradients to preserve source-domain information.
2. Related Works
Related work spans conventional domain adaptation, source-free adaptation, and continual domain adaptation. These lines of work use distribution alignment, classifier or feature strategies, and forgetting-mitigation mechanisms.
- Domain Adaptation: Conventional domain adaptation methods include moment matching, adversarial learning, conditional training, cluster-regularized adaptation, and category-level feature alignment.
- Source-free Domain Adaptation: Source-free domain adaptation methods operate without source data, including approaches for universal, open-set, multi-source, and close-set adaptation.
- Continual Domain Adaptation: Continual domain adaptation addresses sequential domain changes using sample replay, domain relation graphs, or domain-incremental learning.
3. Methods
The method combines local structure clustering to adapt using unlabeled target data with sparse domain attention to preserve source information during source-free adaptation. It further supports unified, domain-ID-aware, and continual adaptation settings.
- Problem Setting and Notations: The source model is divided into a feature extractor f and a classifier g, with target adaptation relying on target features and predictions.The classifier contains one fully connected layer, and the network output is p(x) = g(f(x)).
- Local Structure Clustering: LSC finds cosine-similar nearest neighbors in a target feature bank and encourages consistent predictions while regularizing toward a balanced class distribution.A score bank stores softmax predictions, and the bank is updated with current mini-batch entries.
- Sparse Domain Attention: SDA activates different feature channels for each domain and uses the source attention to regularize gradients during target adaptation.The source and target attention vectors are near-binary, differentiable masks; source attention is trained beforehand and fixed during target adaptation.
- Unified Training: Unified training pretrains with both attentions, then adapts using target attention, target-only data, LSC, and SDA gradient regularization.The unified procedure builds the feature bank from target-attended features so irrelevant channels are excluded.
- Continual Source-free Domain Adaptation: Continual source-free adaptation regularizes gradients with an attention initialized from source attention and formed by retaining prior domain attentions while excluding the current one.The resulting regularization is intended to prevent training on one target domain from influencing others.
4. Experiments
Experiments evaluate G-SFDA for target adaptation, source retention, ablations, feature structure, and continual adaptation across VisDA and Office-Home. The method achieves strong target performance while reducing, but not eliminating, source forgetting.
- Experimental setup: G-SFDA is evaluated in both conventional DA/SFDA settings, which focus on target accuracy, and the G-SFDA setting, which evaluates source and target performance.The experiments use VisDA and Office-Home, with domain-aware and domain-agnostic evaluation in G-SFDA.
- Generalized Source-free Domain Adaptation: G-SFDA improves the H value over SHOT by 8.8% on Office-Home and 4.6% on VisDA, mainly through stronger source performance.Relative to the source model, performance still drops by 2.1% on Office-Home and 9.2% on VisDA, indicating remaining forgetting.
- Number of nearest neighbors K: The method is robust to K except at K=1, where a single nearest neighbor may be noisy in dense feature regions.The analysis varies K over {1, 5, 10, 15, 20, 30} on VisDA.
- Ablation study: Removing SDA causes large source-performance degradation and also lowers target accuracy by 10.4 on VisDA and 0.6 on Office-Home.The reported ablation shows SDA supports both source retention and target performance.
- Feature visualization and continual adaptation: After adaptation, source feature clusters remain structured while target features become more structured; shared channels cluster across domains and specific channels separate them.Continual adaptation works for all domains, and adapting to one target can improve performance on an unseen target domain.
5. Conclusion
The paper introduces G-SFDA to adapt a source-pretrained model using unlabeled target data while retaining source performance. LSC exploits local target structure, and SDA separates domain channels while regularizing adaptation; experiments support the method’s efficacy.
- G-SFDA adapts a source-pretrained model to target domains without source data while aiming to preserve performance on both source and target domains.
- LSC clusters target features with semantically close neighbors, while SDA activates domain-specific feature channels and regularizes target-training gradients.
- Experiments testify to the efficacy of the proposed method for source-free adaptation and source-information maintenance.