Source-linked AI summary
Source-free Domain Adaptation via Avatar Prototype Generation and Adaptation
Zhen Qiu, Yifan Zhang, Hongbin Lin, Shuaicheng Niu, Yanxia Liu, Qing Du, Mingkui Tan
TL;DR
Source-free UDA must adapt a source model using only unlabeled target data, without access to source data, while domain shifts can make pseudo-labels unreliable. CPGA mines source-model knowledge to generate class prototypes and adapt target features through robust contrastive alignment. Experiments on three benchmark datasets demonstrate its effectiveness and superiority.
Problem
Source-free UDA adapts a well-trained source model to an unlabeled target domain without source data, but missing source data and target labels make adaptation challenging.
Method
CPGA generates source-class avatar prototypes through contrastive learning and aligns pseudo-labeled target data to corresponding prototypes using robust contrastive prototype adaptation.
Results
Experiments on three domain adaptation benchmark datasets demonstrate the effectiveness and superiority of CPGA.
Takeaways & Limitations
CPGA adapts a source-trained model to an unlabeled target domain without access to source data.
Takeaways & Limitations
The setting assumes access only to a well-trained source model and unlabeled target data, with source and target domains sharing the same label space.
Abstract
from arXiv · showhide
We study a practical domain adaptation task, called source-free unsupervised domain adaptation (UDA) problem, in which we cannot access source domain data due to data privacy issues but only a pre-trained source model and unlabeled target data are available. This task, however, is very difficult due to one key challenge: the lack of source data and target domain labels makes model adaptation very challenging. To address this, we propose to mine the hidden knowledge in the source model and exploit it to generate source avatar prototypes (i.e., representative features for each source class) as well as target pseudo labels for domain alignment. To this end, we propose a Contrastive Prototype Generation and Adaptation (CPGA) method. Specifically, CPGA consists of two stages: (1) prototype generation: by exploring the classification boundary information of the source model, we train a prototype generator to generate avatar prototypes via contrastive learning. (2) prototype adaptation: based on the generated source prototypes and target pseudo labels, we develop a new robust contrastive prototype adaptation strategy to align each pseudo-labeled target data to the corresponding source prototypes. Extensive experiments on three UDA benchmark datasets demonstrate the effectiveness and superiority of the proposed method.
1 Introduction
Source-free UDA adapts a trained source model to unlabeled target data without source data, but domain shifts and missing labels make adaptation difficult. CPGA addresses this by generating source-class prototypes and aligning pseudo-labeled target features, with experiments on three benchmark datasets demonstrating effectiveness and superiority.
- Source-free UDA adapts a well-trained source model to an unlabeled target domain without using source data, motivated by privacy constraints.
- Missing source data and target labels hinder source-distribution estimation and target-class exploitation, making source-free UDA challenging.
- CPGA mines hidden source-model knowledge through contrastive prototype generation and prototype adaptation for source-free UDA.The method generates source-class feature prototypes and aligns pseudo-labeled target data with same-class prototypes.
- Generated avatar prototypes are representative features that are intra-class compact and inter-class separated, supporting source-free UDA and conventional UDA methods.
- CPGA's robust adaptation strategy aligns pseudo-labeled target data to corresponding source prototypes while alleviating pseudo-label noise.
- Extensive experiments on three domain adaptation benchmark datasets demonstrate the effectiveness and superiority of CPGA.
2 Related Work
Prior source-free UDA methods refine source models through pseudo-labeling or target-style image generation, but domain shifts can make pseudo-labels noisy. CPGA instead generates source feature prototypes and uses confidence reweighting and regularization to reduce negative transfer from noisy pseudo-labels.
- Source-free UDA adapts a source model to an unlabeled target domain without using source data.
- Existing methods refine source models through pseudo-labeling or target-style image generation.
- CPGA generates source feature prototypes for each class instead of directly generating images.
- CPGA uses confidence reweighting and regularization to alleviate negative transfer caused by noisy pseudo-labels.
3 Proposed Method
CPGA adapts a source model to an unlabeled target domain without source data by generating source-class avatar prototypes and aligning pseudo-labeled target features to them. Its two stages combine contrastive prototype generation with robust prototype adaptation that addresses noisy pseudo labels.
- Problem Definition: CPGA assumes access only to a well-trained source model and unlabeled target data, with the goal of adapting the model to the target domain.
- Prototype Generation: Prototype generation trains a class-conditional generator under the source classifier to produce avatar feature prototypes for each source class.The generator uses class information hidden in the classifier and generates p=Gg(y, z) from a label and uniform noise.
- Prototype Generation: Adding a contrastive loss to cross entropy makes generated prototypes more intra-class compact and inter-class separated.The contrastive objective samples same-category positive pairs and different-category negative pairs; in high-dimensional spaces, different-class prototypes can become approximately orthometric.
- Pseudo-Label Generation: Target pseudo labels are generated by computing class centroids from classifier probabilities and assigning each target feature the nearest centroid label.Centroids are updated during training, and pseudo labels are recomputed each epoch.
- Prototype Adaptation: Prototype adaptation aligns pseudo-labeled target features with corresponding source prototypes using weighted contrastive learning, early learning regularization, and neighborhood clustering.Confidence weighting emphasizes reliable samples, while the early-learning regularizer uses prediction momentum to reduce overfitting to noisy pseudo labels.
4 Experiments
Experiments evaluate CPGA on three benchmark datasets against source-only, source-based UDA, and source-free UDA baselines, with ablations of its prototype-generation and prototype-adaptation components. CPGA achieves strong benchmark performance, while contrastive prototype generation, robust adaptation losses, and selected hyperparameters contribute to its results.
- Datasets and baselines: Experiments use Office-31, VisDA, and Office-Home, comparing CPGA with source-only, source-based UDA, and source-free UDA baselines.Office-31 contains three domains and 31 categories; VisDA is a 12-class synthesis-to-real task; Office-Home contains four domains and 65 categories.
- Ablation studies: Adding the contrastive prototype-generation loss separates inter-class features, compacts intra-class features, and improves VisDA accuracy from 85.0% to 86.0%.The reported distances use cosine distance, and accuracy is measured after 400 training epochs.
- Ablation studies: The confidence-weighted prototype-adaptation loss performs better than conventional contrastive loss, while adding the other two losses yields the best performance on VisDA.The additional losses are L_elr and L_nc, combined with L_wcon.
- Ablation studies: VisDA performance is best at λ = 5 and η = 0.05, and the results indicate that CPGA is not sensitive to these hyperparameters.The study varies λ over [1, 3, 5, 7, 9] and η over [0.001, 0.005, 0.01, 0.05, 0.1].
5 Conclusions
The conclusion presents CPGA as a source-free UDA method that generates class-level avatar prototypes and adapts the source model using pseudo-labeled target data. Experiments verify its effectiveness and superiority.
- Conclusions: CPGA generates avatar feature prototypes for each class through contrastive learning and uses them to adapt the source model without source data.Prototype adaptation pulls pseudo-labeled target data toward corresponding source prototypes.
Appendix
The appendix provides the inference algorithm, implementation details, and additional experimental results.
- Appendix: The appendix contains the inference scheme, further implementation details, and additional experimental results.
A. Inference Details of CPGA
During inference, CPGA extracts a feature from each target image and applies the classifier to produce its prediction.
- Inference procedure: Inference maps an input image x to its target prediction by computing Cy(Ge(x)) with the feature extractor Ge and classifier Cy.The procedure first extracts Ge(x), then feeds that feature into Cy.
B. More Implementation Details
The generator uses class labels and random noise to produce feature prototypes, while the second stage generates class prototypes each epoch for class-wise domain alignment.
- Generator Architecture: The generator maps class labels through an embedding layer before combining them element-wise with input noise.The input noise follows z∼U(0, 1), and the label is represented as y∈R^K.
- Generator Architecture: The generator architecture comprises an embedding layer, two fully connected layers, and two deconvolution layers.The output dimensions are detailed in Table 7.
- Training Settings: Stage one trains the generator by optimizing Lce+Lpcon.The batch size is 128, and SGD uses a learning rate of 0.001.
- Training Settings: During stage two, feature prototypes for K classes are generated in each epoch to achieve class-wise domain alignment.Hyper-parameters are selected using an unsupervised reverse validation strategy.
C. More Experimental Results
Additional experiments compare CPGA with conventional and source-free UDA methods, examine hyper-parameter settings, and assess optimization and testing robustness. The reported results show competitive benchmark performance and resistance to pseudo-label noise.
- State-of-the-art Comparisons: CPGA outperforms conventional UDA methods on Office-Home despite those methods using source data.It also achieves competitive performance against source-free methods SHOT and BAIT.
- Hyper-parameter Influence: β=0.9 and λ=5 produce the best performance on VisDA among the tested hyper-parameter settings.The study evaluates β∈[0.5, 0.7, 0.9, 0.99] and λ∈[3, 5, 7].
- Optimization Curves: CPGA converges well in total loss and accuracy during training, while its validation curve indicates no suffering from pseudo-label noise.Figure 3 presents optimization curves on Office-31(A→W).
- Robustness Comparisons: Compared with BAIT, CPGA is reported as more robust to mistaken certain-versus-uncertain divisions and pseudo-label noise.Figure 4 presents testing curves for CPGA and BAIT on VisDA.
- Benchmark Results: The supplementary tables report classification accuracies for Office-Home, VisDA, and Office-31 benchmarks.The tables use ResNet-50 for Office-Home and Office-31, and ResNet-101 for VisDA.