PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross-domain Contrastive Learning for Unsupervised Domain Adaptation

Rui Wang, Zuxuan Wu, Zejia Weng, Jingjing Chen, Guo-Jun Qi, Yu-Gang Jiang

arXiv:2106.05528v2cs.CVcs.AIcs.LG

TL;DR

UDA must transfer models from labeled source domains to unlabeled targets with shifted distributions. CDCL aligns cross-domain features by class using pseudo labels and source prototypes, achieving state-of-the-art results on Office-31 and VisDA while also supporting source-data-free adaptation.

  • Problem

    UDA seeks robust target-domain predictions despite distribution shifts and the absence of target labels, but contrastive learning lacks naturally defined cross-domain positive pairs.

  • Method

    CDCL forms cross-domain positive and negative pairs from shared class information, generates target pseudo labels with prototype-initialized clustering, and substitutes source prototypes when source data are unavailable.

  • Results

    CDCL achieves state-of-the-art performance on Office-31 and VisDA; on VisDA it reaches 88.6% mean accuracy, while the data-free setting reaches 87.5%.

  • Takeaways & Limitations

    The framework supports effective domain alignment in both conventional UDA and source-data-free UDA settings.

Abstract

from arXiv · show

Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a fully-labeled source domain to a different unlabeled target domain. Most existing UDA methods learn domain-invariant feature representations by minimizing feature distances across domains. In this work, we build upon contrastive self-supervised learning to align features so as to reduce the domain discrepancy between training and testing sets. Exploring the same set of categories shared by both domains, we introduce a simple yet effective framework CDCL, for domain alignment. In particular, given an anchor image from one domain, we minimize its distances to cross-domain samples from the same class relative to those from different categories. Since target labels are unavailable, we use a clustering-based approach with carefully initialized centers to produce pseudo labels. In addition, we demonstrate that CDCL is a general framework and can be adapted to the data-free setting, where the source data are unavailable during training, with minimal modification. We conduct experiments on two widely used domain adaptation benchmarks, i.e., Office-31 and VisDA-2017, for image classification tasks, and demonstrate that CDCL achieves state-of-the-art performance on both datasets.

I. INTRODUCTION

The paper asks whether contrastive learning can align domains in UDA despite the lack of naturally paired cross-domain views. CDCL uses shared class structure and pseudo labels to form cross-domain contrastive pairs, extending to source-data-free adaptation.

  • Motivation: UDA reduces training–testing distribution discrepancy by aligning labeled source and unlabeled target features.Prior approaches use metrics including MMD, covariance, and adversarial losses.
  • Motivation: Contrastive learning motivates pulling related representations together while separating unrelated samples, but cross-domain positive pairs are not naturally available.Unlike self-supervised learning, domain adaptation lacks two augmented views of the same image as an obvious positive pair.
  • CDCL: CDCL forms positive pairs from cross-domain samples sharing a class and negative pairs from different categories, using target pseudo labels.The method initializes k-means centers with source class prototypes and minimizes positive-pair distances relative to negative-pair distances.
  • CDCL: CDCL supports source-data-free UDA by replacing unavailable source samples with prototypical features derived from a source-trained model.This modification preserves the cross-domain alignment framework without requiring source data during training.
  • Results: CDCL achieves state-of-the-art performance on the Office-31 and VisDA benchmarks and produces domain-invariant features without source data.The experiments include ablations assessing the effectiveness of different components.

III. METHODOLOGY

UDA adapts a model from a fully labeled source domain to an unlabeled target domain sharing the same predefined categories. The model combines feature encoding with classification and is trained on both domains.

  • Problem Setup: UDA assumes labeled source samples and unlabeled target images whose distributions differ but whose categories are shared.The source dataset contains image–label pairs, while target labels remain unknown during training.
  • Problem Setup: The adaptation model predicts target labels using a feature encoder followed by a classifier.The encoder maps inputs to d-dimensional features, and the classifier maps features to M category outputs.
  • Objective: The paper’s goal is to align source and target feature distributions through contrastive self-supervised learning.CDCL forms positive and negative pairs across domains to learn domain-invariant features and also targets data-free scenarios.

A. Contrastive Learning with InfoNCE

CDCL adapts contrastive learning by forming positive and negative pairs across domains using shared class information and target pseudo labels. Its bidirectional loss is combined with source classification.

  • A. Contrastive Learning with InfoNCE: InfoNCE minimizes the distance of a positive pair relative to all other pairs using normalized feature representations and a temperature parameter.Positive and negative samples are defined relative to an anchor feature.
  • B. Cross-domain Contrastive Learning: CDCL assumes same-class samples should be close and different-class samples far apart regardless of domain.This class-conditioned assumption provides the basis for cross-domain feature alignment.
  • B. Cross-domain Contrastive Learning: For a target anchor, CDCL uses source samples with the same pseudo-labeled class as positives and other source samples as negatives.Summing over all same-category source samples in a mini-batch can reduce sampling variance.
  • B. Cross-domain Contrastive Learning: The cross-domain loss can also use source anchors, and bidirectional alignment uses anchors from both domains.The final objective combines cross-domain contrastive loss with source-domain cross-entropy, weighted by a trade-off parameter.
  • Implementation: CDCL’s standard-UDA procedure initializes the encoder, clusters target data using source class prototypes, computes the loss, and updates the model iteratively.The data-free procedure freezes the classifier and replaces source samples with source-derived prototypes.

C. Pseudo Labels for the Target Domain

Because target labels are unavailable, CDCL generates pseudo labels with spherical k-means initialized by source class prototypes. Ambiguous target assignments are removed to reduce label noise.

  • Pseudo-label generation: CDCL uses k-means clustering to generate target pseudo labels for cross-domain contrastive pair formation.The number of clusters is set to the number of predefined classes.
  • Pseudo-label generation: Source class prototypes initialize the target clustering centers to preserve correspondence with predefined category semantics.The prototypes are computed as source-sample centroids for each category.
  • Pseudo-label generation: Spherical k-means assigns target samples using cosine similarity between target features and cluster centers.The clustering operates on target-domain features after prototype-based initialization.
  • Pseudo-label refinement: Samples whose feature-to-assigned-center cosine similarity falls below threshold d are removed as ambiguous pseudo-label assignments.This filtering step is intended to reduce noise in target pseudo labels.

D. Source Data-free UDA

Source data-free UDA adapts a source-trained model using only unlabeled target data because source samples are unavailable. CDCL replaces source samples and class prototypes with classifier weights while estimating target labels through clustering.

  • D. Source Data-free UDA: Source data-free UDA provides a source-trained model but no source data, requiring adaptation with only unlabeled target data.This setting addresses source-data unavailability caused by corruption or privacy concerns.
  • D. Source Data-free UDA: Without source samples, discrepancy-based and adversarial UDA methods cannot measure source-target statistics or train source-target discriminators.
  • D. Source Data-free UDA: CDCL replaces source samples with classifier weights, treating normalized class weight vectors from the source model as source class prototypes.The classifier layer is frozen during target adaptation to preserve these prototypes.
  • D. Source Data-free UDA: Target labels are estimated with clustering, while sample-based class prototypes are replaced by class weights.
  • D. Source Data-free UDA: The source data-free objective omits cross-entropy because source samples are unavailable for supervised training.

A. Datasets and Compared Approaches

The study evaluates UDA on Office-31 and VisDA-2017, comparing source-only training with established domain-adaptation methods in standard and source data-free settings.

  • A. Datasets and Compared Approaches: VisDA-2017 contains 12 classes, 152,397 synthetic source images, and 55,388 real-world target images for synthesis-to-real adaptation.
  • A. Datasets and Compared Approaches: Office-31 contains 4,110 images across Amazon, DSLR, and Webcam domains, with 31 categories and six pairwise adaptation tasks.
  • A. Datasets and Compared Approaches: Compared approaches include adversarial, discrepancy-based, discriminative-feature, feature-norm, and sliced-Wasserstein methods.The compared methods include DANN, DAN, JAN, ADR, SAFN, SWD, and MMAN.
  • A. Datasets and Compared Approaches: SHOT and ModelAdapt are identified as source data-free UDA methods, while CDCL is implemented in both standard and source data-free settings.

B. Implementation Details

Implementation uses ImageNet-pretrained ResNet encoders, domain-specific batch normalization, scheduled SGD training, and benchmark tables reporting accuracy for VisDA-2017 and Office-31.

  • B. Implementation Details: Standard UDA uses ImageNet-pretrained ResNet-50 for Office-31 and ResNet-101 for VisDA-2017, with domain-specific batch-normalization layers.Other network parameters are shared across domains, and the final fully connected layer is task-specific.
  • B. Implementation Details: Training uses mini-batch SGD with momentum 0.9 and a scheduled learning rate η=η0·(1+10·p)^−b.The initial learning rates are 1e−3 for pretrained convolutional layers and 1e−2 for newly added layers; b is 0.75 for Office-31 and 2.25 for VisDA-2017.
  • B. Implementation Details: Table I reports Accuracy(%) on VisDA-2017 for unsupervised domain adaptation using ResNet-101.The dagger marks methods developed under the source data-free UDA setting.
  • B. Implementation Details: Table II reports Accuracy(%) on Office-31 for unsupervised domain adaptation using ResNet-50.The dagger marks methods developed under the source data-free UDA setting.
  • B. Implementation Details: For source data-free experiments, the source dataset is split 0.9/0.1 into training and validation sets to select the source pretrained model.

C. Main Results

CDCL achieves strong results on VisDA-2017 and Office-31 in both conventional and source data-free UDA settings. Its data-free performance remains competitive despite a small decrease from conventional UDA.

  • C. Main Results: 88.6% mean accuracy on VisDA-2017 exceeds all compared state-of-the-art approaches and improves the source-only baseline by 26.2%.
  • C. Main Results: 1.4% absolute point over CAN and 1.0% over JCL on VisDA-2017 demonstrate CDCL’s leading benchmark performance.
  • C. Main Results: 87.5% mean accuracy in the source data-free setting surpasses SHOT by 4.6% point.
  • C. Main Results: The source data-free result is 0.9% worse than conventional UDA, while still surpassing many standard-UDA methods.
  • C. Main Results: On Office-31, CDCL reaches 90.6% mean accuracy across six conventional UDA tasks and 89.3% in the data-free setting.The conventional result is on par with the best literature results, while the data-free result is comparable to ModelAdapt.

D. Ablation Studies and Discussions

The ablations show that cross-domain pairing and bidirectional anchors are important for CDCL, while its learned representations align domains and preserve class separation. CDCL is also relatively insensitive to its temperature and weighting hyper-parameters.

  • Positive and Negative Pairs: Cross-domain alignment outperforms in-domain alignment, while bidirectional anchors outperform using anchors from only one domain.Mixing both domains performs worse than CDCL, possibly because jointly modeling intra-class and inter-class information is challenging.
  • The impact of hyper-parameters: CDCL is not sensitive to its hyper-parameters, with stable accuracy around τ = 0.05 and λ = 1.6.For λ > 1.4, the accuracy gap is smaller than 0.2 on VisDA-2017 standard UDA.
  • Feature Visualization: Before alignment, source and target features form separate clusters; after CDCL alignment, domains mix while different classes remain separated.The t-SNE visualization uses triangles and circles for source and target samples, with colors denoting classes.
  • Learned Feature Distance: As training progresses, retrieved cross-domain samples shift from shape-similar but differently categorized images toward samples from the anchor’s class.This indicates that CDCL gradually pulls same-class features closer.

V. CONCLUSION

The conclusion presents CDCL as a contrastive framework for aligning cross-domain, class-conditioned features in both standard and source data-free UDA. Experiments on two benchmarks report state-of-the-art performance, while clustering supplies target pseudo labels and classifier weights support the data-free adaptation.

  • V. CONCLUSION: CDCL aligns each image with same-class samples from the other domain relative to cross-domain samples from different categories.Target pseudo labels are generated using clustering because target labels are unavailable.
  • V. CONCLUSION: CDCL applies to both standard UDA and source data-free settings, using classifier weights as class prototypes in the latter.The source data-free adaptation requires only an adaptation of the framework.
  • V. CONCLUSION: CDCL achieves state-of-the-art performance on the Office-31 and VisDA domain adaptation benchmarks.The conclusion reports extensive experiments on two widely used benchmarks.
Loading 2106.05528v2…