Source-linked AI summary
Unsupervised Multi-source Domain Adaptation Without Access to Source Data
Sk Miraj Ahmed, Dripta S. Raychaudhuri, Sujoy Paul, Samet Oymak, Amit K. Roy-Chowdhury
TL;DR
The paper addresses unsupervised adaptation from multiple source models without source data, where selecting or combining sources is challenging. It proposes weighted adaptation using information maximization and pseudo-labeling, and reports gains over adapted-source baselines in many benchmark settings.
Problem
Existing source-free adaptation methods handle one source model and assume an oracle supplies the most correlated source, leaving multi-source selection and negative-transfer prevention unresolved.
Method
The method combines target soft labels from multiple source models with learned weights, using information maximization and pseudo-labeling to jointly adapt encoders and source weights.
Results
Across Office-Home tasks, the method achieves a mean accuracy increase of 2% over the respective best adapted source models.
Takeaways & Limitations
The algorithm can automatically combine multiple source models without source data and can reject poor sources during target adaptation.
Takeaways & Limitations
Inference time is O(m) for m source models, although distillation can produce a single target model to reduce this overhead.
Abstract
from arXiv · showhide
Unsupervised Domain Adaptation (UDA) aims to learn a predictor model for an unlabeled domain by transferring knowledge from a separate labeled source domain. However, most of these conventional UDA approaches make the strong assumption of having access to the source data during training, which may not be very practical due to privacy, security and storage concerns. A recent line of work addressed this problem and proposed an algorithm that transfers knowledge to the unlabeled target domain from a single source model without requiring access to the source data. However, for adaptation purposes, if there are multiple trained source models available to choose from, this method has to go through adapting each and every model individually, to check for the best source. Thus, we ask the question: can we find the optimal combination of source models, with no source data and without target labels, whose performance is no worse than the single best source? To answer this, we propose a novel and efficient algorithm which automatically combines the source models with suitable weights in such a way that it performs at least as good as the best source model. We provide intuitive theoretical insights to justify our claim. Furthermore, extensive experiments are conducted on several benchmark datasets to show the effectiveness of our algorithm, where in most cases, our method not only reaches best source accuracy but also outperforms it.
1 University of California, Riverside, 2 Google Research
The passage lists five email identifiers associated with the section.
- The listed identifiers use the ucr.edu email domain.
- Five email handles are provided: sahme047, drayc001, spaul003, oymak, and amitrc.
- The identifiers include both standard and department-style account forms, such as sahme047@ and oymak@ece.
1. Introduction
The paper addresses unsupervised adaptation when source data is unavailable and multiple source models differ in their relevance to an unlabeled target. It proposes combining these models while limiting negative transfer, using unsupervised objectives and theoretical analysis.
- Deep models often struggle with visual distribution shifts, motivating adaptation for practical deployment.
- Conventional UDA methods require source data during adaptation, creating privacy, security, and storage concerns.
- Single-source source-free adaptation assumes an oracle provides the most correlated source model.
- The proposed setting adapts a bag of source models while combining useful knowledge and preventing negative transfer.
- DECISION uses pseudo-labeling and information maximization, with theoretical insights supporting performance at least as good as the best available source.
2. Related works
The related work spans standard UDA, hypothesis transfer, and multi-source adaptation, while positioning this paper as source-free and unsupervised for multiple source models.
- UDA research includes feature-space methods such as moment matching and adversarial learning, alongside pixel-space adaptation.
- The comparison framework evaluates adaptation settings by attributes, and the proposed setting satisfies the listed holistic criteria.
- Hypothesis transfer learning transfers source hypotheses without source data but assumes labeled target data.
- Unsupervised hypothesis transfer removes target labels for single-source adaptation, whereas this paper extends the setting to multiple source models.
3. Methodology
The method jointly adapts multiple source feature encoders and learns source weights using only unlabeled target data, while keeping source classifiers fixed. Information maximization and pseudo-labeling guide the resulting target predictor.
- Problem setting: The target model is learned from source models and unlabeled target samples without access to source datasets.
- Overall Framework: Source models are decomposed into feature extractors and classifiers, with poor or irrelevant source models intended to be automatically rejected.
- Overall Framework: Weights α_i form a probability distribution over sources, and higher values indicate greater transferability to the target.
- Weighted Information Maximization: The framework jointly adapts source feature maps and source weights using unlabeled target instances while fixing source classifiers.
- Weighted Information Maximization: Information maximization combines conditional-entropy minimization with label-distribution entropy maximization to encourage confident and diverse predictions.
- Weighted Pseudo-labeling: Because information maximization can reinforce wrong clusters through confirmation bias, the method uses DeepCluster-inspired self-supervised pseudo-labeling.
- Weighted Pseudo-labeling: Pseudo-labels are obtained by assigning target samples to nearest combined cluster centroids, then used in a cross-entropy objective.
4. Theoretical Insights
The paper motivates a weighted combination of source predictors and analyzes when it can perform at least as well as the best individual source. Its optimization jointly updates source feature encoders and source weights using unlabeled target data, under distributional assumptions supporting the theoretical guarantee.
- Theoretical motivation: The framework seeks source weights whose combined target predictor can perform better than or equal to the best source model under intuitive source–target distribution assumptions.The theoretical construction uses a convex combination of source predictors.
- Assumptions: The guarantee assumes a convex loss and a target distribution that is exactly a convex mixture of the source distributions.The mixture weights are nonnegative and sum to one.
- Guarantee: Under these assumptions, the target predictor’s pseudo-label-induced and supervised losses are each no greater than the loss of the best source.The claim concerns the expected loss over the target distribution and uses the convex-combination predictor.
- Guarantee: The inequality becomes strict when all mixture coefficients are positive and at least one source is strictly better than the target-best source on its own distribution.This condition is presented as a natural mild condition for strict improvement.
- Optimization: The algorithm jointly fine-tunes source feature extractors and learns input-agnostic source weights from unlabeled target data, combining pseudo-labeling with information maximization.The optimization updates both the encoders and the source weights; the implementation computes pseudo-labels and target-batch embeddings during training.
- Assumptions: The theoretical link between the algorithm and the guarantee relies on treating source distributions as uniform over the target support, with scaling factors absorbed into the learned weights.Under this uniformity assumption, the optimization is intended to return a favorable combination satisfying the lemma’s bounds.
5. Experiments
Experiments evaluate source-data-free multi-source adaptation across digit and object-recognition benchmarks, ablations, learned weights, and inference-time distillation. The method generally matches or exceeds the best adapted source while reducing negative transfer and outperforming ensemble and other baselines.
- Experimental setup: The experiments cover digit recognition, Office, Office-Home, and Office-Caltech benchmarks, with domains alternately designated as targets and source data discarded after source-model training.Baselines include SHOT variants, source-data-using methods, and uniform ensembling.
- Object recognition: 5.2% average improvement over SHOT-Ens is achieved on Office, while the method prevents Amazon-induced negative transfer for Webcam and exceeds the best adapted source by 3.5%.It performs at least as well as the best adapted source on all Office tasks.
- Object recognition: 2% mean accuracy improvement over the best adapted source is obtained across Office-Home tasks, where the method outperforms all baselines.Office-Caltech results follow the same trend, with 98% mean accuracy across tasks and performance above all baselines.
- Weight analysis: Learned source weights outperform uniform weighting and indicate which source model performs better on the target, supporting their use as a model-selection proxy.Figure 3 reports a positive correlation between learned weights and unadapted source-model performance.
- Inference efficiency: Inference costs O(m) for m source models, so knowledge distillation combines weighted adapted-model predictions to train a single target model.The distilled model is intended to reduce inference overhead when many source models are used.
6. Conclusions and Future Work
The paper presents source-data-free unsupervised multi-source adaptation that combines source models and validates the approach across domain-adaptation benchmarks. It identifies data augmentation and unsupervised selection of source-model subsets as future directions.
- Conclusions and future work: The proposed UDA algorithm combines multiple source models without source data, with theoretical intuitions and benchmark experiments supporting its effectiveness.The authors report future work on data augmentation and automatically selecting an optimal source-model subset.
7. Proof of Lemma 1
Under the stated convexity and mixture assumptions, the convex combination of optimal source predictors yields a target predictor whose supervised loss is no worse than that of the best individual source model. The inequality becomes strict when all combination weights are positive and a source-wise strict inequality holds.
- The proof assumes convex loss in the predictor and a target distribution equal to a nonnegative mixture of source distributions.
- The target predictor is constructed as a convex combination of the optimal source predictors.
- The pseudo-labeling and supervised losses of this target predictor are each no greater than the loss of the best source predictor.
- With strictly positive combination weights, the target-loss bound is strict when one source model is strictly worse than another on that source distribution.
8. Detailed steps of combination rule under source distribution uniformity assumption
The passage directs readers to the discussion after Lemma 1 in the main paper for reference.
- The detailed combination-rule discussion is located after Lemma 1 in the main paper.
- The passage provides a cross-reference rather than describing the combination rule itself.
- Readers are directed to the main paper for the relevant discussion.
9. Additional Experiments
Additional experiments show that learned weights track unadapted source quality, the method can resist poor source models, and performance remains strong across diverse datasets and after distillation.
- Higher unadapted source accuracy receives higher weight, making learned weights a proxy for source-model selection.
- λ = 0.3 gives the best performance in the Office experiments when varying the pseudo-labeling-loss weight.
- 93% accuracy remains close to the 94% best-source result despite the 21.3% USPS outlier, while removing USPS yields 94.5%.
- On DomainNet, the method consistently outperforms SHOT-best except for infograph, although its average is slightly below SHOT-Ens.
- Distillation preserves performance across object-recognition datasets despite compressing the models into a single target model.