Source-linked AI summary
Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning
Ziyi Zhang, Weikai Chen, Hui Cheng, Zhen Li, Siyuan Li, Liang Lin, Guanbin Li
TL;DR
Source-free domain adaptation seeks to adapt a source-trained model to unlabeled target data, but existing global and local approaches face noisy-label and source-bias concerns. DaC divides target samples by source-model confidence and applies tailored adaptive contrastive learning with memory-bank MMD alignment. It reports state-of-the-art results on VisDA, Office-Home, and DomainNet.
Problem
Source-free domain adaptation adapts a source-trained model without source data, while global pseudo-labeling and local-neighborhood approaches have different limitations.
Source: §1 · pp. 1–2Method
DaC splits target samples by source-model confidence, using class-level contrastive learning for source-like samples, local structure for target-specific samples, and memory-bank MMD to align them.
Source: §1 · p. 2Results
DaC reports state-of-the-art performance on VisDA, Office-Home, and DomainNet.
Source: §1 · p. 3, §6 · p. 10Takeaways & Limitations
DaC’s confidence-based split combines global class adaptation and local consistency within one framework while aligning the two target-data groups.
Source: §3 · p. 4, §6 · p. 10The theoretical analysis assumes disjoint target class-conditional distributions and a target-domain expansion property.
Source: §3 · pp. 3–4
Abstract
from arXiv · showhide
We investigate a practical domain adaptation task, called source-free domain adaptation (SFUDA), where the source-pretrained model is adapted to the target domain without access to the source data. Existing techniques mainly leverage self-supervised pseudo labeling to achieve class-wise global alignment [1] or rely on local structure extraction that encourages feature consistency among neighborhoods [2]. While impressive progress has been made, both lines of methods have their own drawbacks - the "global" approach is sensitive to noisy labels while the "local" counterpart suffers from source bias. In this paper, we present Divide and Contrast (DaC), a new paradigm for SFUDA that strives to connect the good ends of both worlds while bypassing their limitations. Based on the prediction confidence of the source model, DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals under an adaptive contrastive learning framework. Specifically, the source-like samples are utilized for learning global class clustering thanks to their relatively clean labels. The more noisy target-specific data are harnessed at the instance level for learning the intrinsic local structures. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch. Extensive experiments on VisDA, Office-Home, and the more challenging DomainNet have verified the superior performance of DaC over current state-of-the-art approaches. The code is available at https://github.com/ZyeZhang/DaC.git.
1 Introduction
DaC adapts a source model without source data by dividing target samples according to prediction confidence and assigning them different learning goals. It combines global class structure with local consistency and aligns the two groups, reporting state-of-the-art performance on three challenging datasets.
1 Introduction
- Global pseudo-labeling can suffer from noisy labels, while neighborhood consistency can inherit source bias and form spurious local clusters.These limitations motivate combining global and local structure rather than relying on either alone.Source: §1 · p. 2
- Figure 1 contrasts CPGA’s global clusters and NRC’s local consistency with DaC’s reported combination of clear global clusters and strong local consistency.The UMAP visualizations use target features trained for 60 epochs on six randomly selected VisDA classes.Source: Figure 1 · p. 2
- DaC divides target data into source-like and target-specific groups using source-model confidence, then applies tailored adaptive contrastive learning.High-confidence samples support robust class centroids; for low-confidence samples, the method uses local structure instead of noisy class-level supervision.Source: §1 · p. 2
- A memory-bank MMD loss aligns source-like and target-specific samples to reduce distribution mismatch and mini-batch instability.The memory bank contains representations of all target samples and is momentum-updated during training.Source: §1 · p. 2
- DaC reports new state-of-the-art performance on VisDA, Office-Home, and DomainNet.The framework also uses memory-bank-based MMD to align source-like and target-specific samples.Source: §1 · pp. 2–3
2 Related Work
Prior domain-adaptation work uses distribution alignment, adversarial training, and contrastive learning, while SFUDA methods use target-style generation or information from the source model. The reviewed approaches differ in whether they exploit class-level or neighborhood structure.
Unsupervised Domain Adaptation
- Conventional UDA methods address domain discrepancy with MMD, adversarial training, or contrastive learning, but source-data-dependent methods cannot be applied directly when source data are absent.MMD matches high-order distribution moments; contrastive methods target global class structure or use memory banks for class- and instance-level information.Source: §2 · p. 3
Source-free Unsupervised Domain Adaptation
Contrastive Learning
- Instance-level contrastive learning generalizes to downstream tasks but does not perform well in SFUDA, which requires correct inter-class affinity estimates on unlabeled target data.Source: §2 · p. 3
3 Preliminaries and Analysis
The analysis partitions target samples by source-model confidence and relates target risk to pseudo-label fitting, local consistency, and discrepancy between the groups. Its assumptions include disjoint class-conditional target distributions and an expansion property.
Preliminary
- The SFUDA setup seeks a model with low target-domain task risk using unlabeled target samples and a source-trained model, with source and target sharing a label space.The model maps inputs through a feature extractor and classifier to class predictions.Source: §3 · p. 3
Assumptions
- The analysis assumes disjoint target class-conditional distributions and introduces an expansion assumption defined through neighborhoods under suitable input transformations.The expansion condition requires neighborhoods of sufficiently large sets to contain additional probability mass.Source: §3 · pp. 3–4
Theoretical analysis
- DaC selects source-like samples whose source-classifier prediction confidence exceeds a threshold and treats the remaining target data as target-specific.Under the stated Lipschitz condition, Claim 3.1 says a threshold exists for which the source-like set has zero consistency error; the remark also reports empirically greater prediction robustness there.Source: §3 · p. 4
- Theorem 3.2 bounds target risk using pseudo-label disagreement, local consistency error, and the H-divergence between source-like and target-specific samples.The authors state that the bound holds under Claim 3.1’s condition and the constant-expansion assumption.Source: §3 · p. 4
- The training objective combines pseudo-label fitting, adaptive contrastive learning for source-like and target-specific samples, and LEMMD alignment between the groups.Source: §3 · p. 4
4 Divide and Contrast for Source-free Unsupervised Domain Adaptation
DaC adapts a source-trained model to unlabeled target data through pseudo-label self-training, adaptive contrastive learning, and distribution alignment. It divides target samples into source-like and target-specific groups and assigns each a tailored learning strategy.
4 Divide and Contrast for Source-free Unsupervised Domain Adaptation
- DaC combines pseudo-label self-training, adaptive contrastive learning, and distribution alignment for source-free adaptation.The method's three parts are summarized as self-training by pseudo-labels, adaptive contrastive learning, and distribution alignment.Source: §4 · p. 4
4.1 Data Segmentation and Self-Training
- Weak and strong augmentations support pseudo-label self-training, while a threshold dynamically divides target data into source-like samples and target-specific outliers.The target-specific set is the remainder after source-like samples are selected.Source: §4.1 · p. 5
4.2 Adaptive Contrastive Learning
- A momentum-updated memory bank stores target representations and supplies class centroids and target-specific features for adaptive contrastive learning.The loss uses a positive prototype and negative pairs for each anchor.Source: §4.2 · p. 6, §4.2.1 · p. 6
4.2.1 Prototype Generation
- Source-like anchors use their class centroid as the positive prototype, while target-specific anchors use local structure from strong views and nearest memory-bank features.The two groups therefore receive class-wise and instance-level adaptation, respectively.Source: §4.2.1 · p. 6
4.2.2 Memory Bank
- The memory bank is initialized from source-model features and supplies target features; source-like sets begin with each class's top 5% predictions and update during training.Class centroids are computed as means of source-like features.Source: §4.2.2 · pp. 6–7
4.3 Distribution Alignment
- Memory bank-based MMD aligns source-like and target-specific distributions to reduce mini-batch estimation bias, with a non-negative exponential form also proposed.The exponential-MMD loss can be computed from memory-bank results without additional computing costs.Source: §4.3 · p. 7
5 Experiments
Experiments evaluate DaC on Office-Home, VisDA, and DomainNet, with ablations examining its contrastive strategies and components. DaC leads the reported benchmark comparisons, while the ablations support the value of dividing target samples and combining the framework’s components.
5.1 Experimental Setup
- Experiments cover Office-Home, VisDA, and seven single-source DomainNet scenarios using four domains and 126 classes, with results averaged over three runs.The experiments use ResNet-50 for Office-Home, ResNet-101 for VisDA, and ResNet-34 for DomainNet.Source: §5.1 · pp. 7–8
5.2 Comparison with State-of-the-arts
- DaC reaches 87.3% versus CPGA’s 86.0% average accuracy on VisDA and 68.3 versus the best source-free baseline’s 65.1 on DomainNet, outperforming all compared methods on both benchmarks.Office-Home results show a similar observation.Source: §5.2 · p. 8, Table 2 · p. 9, Table 3 · p. 9
- DaC achieves fast convergence and stable high classification performance, while CPGA has false predictions within clusters and NRC’s local clusters are affected by source bias.The VisDA accuracy curve covers 60 epochs; CPGA’s curve is recorded from epoch 18 because its early accuracy is low.Source: §5.2 · p. 8, Figure 3 · p. 8
- DaC++ outperforms SHOT++ by more than 1 percent and comes close to target-supervised performance without target annotations.DaC++ uses a two-stage extension, with its second stage matching SHOT++ for a fair comparison.Source: §5.2 · p. 8, Table 1 · p. 8
5.3 Ablation Analysis
- Without dividing target data, both class-level Scheme-S and instance-level Scheme-T improve self-training, but Scheme-S is vulnerable to noisy pseudo-labels and Scheme-T does not transfer robust class-level classification to target outliers.DaC outperforms all alternatives by a large margin.Source: §5.3 · p. 9, Table 5 · p. 10
- Each tested component helps accuracy; the adaptive contrastive framework contributes the most, while removing distribution alignment lowers average accuracy to 86.5%.Strong augmentation and local structure enhance local-neighborhood information; the exponential memory-bank MMD outperforms the linear form and is stable in batch training.Source: §5.3 · pp. 9–10, Table 6 · p. 10
6 Conclusion
DaC divides target data by source-model prediction confidence and applies tailored adaptive contrastive learning to source-like and target-specific samples. A memory bank-based MMD loss transfers knowledge between the groups; extending DaC to other domain-adaptation tasks is identified as future work.
6 Conclusion
- DaC divides target data by source-model confidence and applies tailored adaptive contrastive learning to source-like and target-specific samples.It also uses a memory bank-based MMD loss to transfer knowledge from source-like to target-specific data.Source: §6 · p. 10
- The framework combines class-wise learning on source-like samples with local consistency learning on target-specific data.The paper describes this as inheriting advantages of global class-wise and local neighbor-consistency approaches while sidestepping their limitations.Source: §6 · p. 10
- Extending DaC to semi-supervised SFUDA or source-free open-set DA is identified as an interesting future direction.The conclusion characterizes the achieved performance as promising.Source: §6 · p. 10
A Theoretical Details
The appendix analyzes DaC’s confidence-based division of target data into source-like samples and target-specific outliers. Under stated regularity and expansion assumptions, it establishes source-like prediction consistency and an upper bound on target task error.
A Theoretical Details
- Under an Lh-Lipschitz condition and a confidence threshold τ, the source-like set DS is consistency robust, with RDS(h) = 0.The analysis defines local neighborhoods through overlapping augmentation neighborhoods B(x), with B(x) formed from augmentations within radius r.Source: A Theoretical Details · p. 14
- The target-error bound assumes (q, γ)-constant expansion and relates expected target error to disagreement with the pseudo-labeler, source pseudo-label error, q, and consistency error.The analysis assumes disjoint supports for class-conditional target distributions and considers source-like samples selected by confidence above τ.Source: A Theoretical Details · pp. 14–15
- For source-like samples, the analysis assumes both lower task error and lower pseudo-labeler disagreement than on the full target distribution.It partitions source-like classification mistakes into three disjoint subsets according to whether the model and pseudo-labeler agree and whether the pseudo-labeler is correct.Source: A Theoretical Details · p. 15
- The theorem’s proof bounds errors on source-like subpopulations and combines them with consistency error to obtain a whole-target error bound.The conclusion of the derivation identifies λ as a constant with respect to expansion constant q and the task risk of the ideal optimal model.Source: A Theoretical Details · pp. 16–17
B.1 Implementation Details
Experiments use SGD with momentum and weight decay across VisDA, Office-Home, and DomainNet, with benchmark-specific learning rates and training lengths.
B.1 Implementation Details
- Experiments use SGD with momentum 0.9, weight decay 0.0005, and batch size 64 across VisDA, Office-Home, and DomainNet.Training uses four Nvidia Geforce GTX 1080Ti graphic cards.Source: §B.1 · p. 17
- Initial learning rates are 5e-4 for VisDA, 2e-2 for Office-Home, and 1e-2 for DomainNet.The learning rate is reduced tenfold after 40 epochs on VisDA and 15 epochs on Office-Home and DomainNet.Source: §B.1 · p. 17
- Training lasts 60 epochs on VisDA and 30 epochs on Office-Home and DomainNet, using scheduler η = η0(1 + 15p)3/4.The training-process parameter p ranges from 0 to 1.Source: §B.1 · p. 17
- The confidence threshold τc is 0.95 for all experiments, and the reported performance is not sensitive to its choice.For VisDA and DomainNet, α = 0.5, β = 0.5, K = 5; for most Office-Home transfers, α = 0.7, β = 0.3, K = 3.Source: §B.1 · p. 17
B.2 Baseline Methods on DomainNet
The DomainNet comparison includes source-present and source-free methods; baseline results come from specified sources or official implementations, with learning rates selected by cross-validation.
B.2 Baseline Methods on DomainNet
- The comparison covers source-present and source-free domain-adaptation methods on a DomainNet subset used as a single-source benchmark.MME and CDAN results are copied from [34], while the other methods are implemented using their official codes.Source: §B.2 · pp. 17–18
- Baseline learning rates are selected by five-fold cross-validation, and each method uses its own training scheduler.The supplied table caption identifies a sensitivity analysis of τc, but gives no result values.Source: §B.2 · p. 18, Table 7 · p. 18
C Algorithm for DaC
DaC uses pseudo-label self-training before dividing target data into source-like and target-specific groups for representation learning. Adaptive contrastive learning uses local and global information, while distribution alignment reduces mismatch between the groups.
C Algorithm for DaC
- DaC performs pseudo-label self-training, then divides target samples into source-like and target-specific groups for representation learning, followed by distribution alignment.Source: C Algorithm for DaC · p. 18
- The target model starts from source parameters, initializes a feature memory bank, obtains pseudo-labels, and draws labeled target minibatches for training.Source: C Algorithm for DaC · p. 18
- For each minibatch, DaC augments inputs, updates memory features and group assignments, then computes self-training, contrastive, and distribution-alignment losses.Source: C Algorithm for DaC · p. 18
- DaC combines contrastive, self-training, and distribution-alignment losses with weights α and β, updates φ by SGD, and outputs gs ◦φ.Source: C Algorithm for DaC · p. 18