PaperlayerView exact revision on arXiv

Source-linked AI summary

Universal Source-Free Domain Adaptation

Jogendra Nath Kundu, Naveen Venkat, Rahul M, R. Venkatesh Babu

arXiv:2004.04393v1cs.CVcs.LG

TL;DR

Domain adaptation methods often require a known source–target label relationship and source data during deployment, limiting their practicality. The paper proposes a two-stage Universal Source-Free Domain Adaptation framework using generative negative modeling and Source Similarity Metric weighting, and reports superior performance with lower deployment complexity.

  • Problem

    Existing unsupervised DA methods commonly require source–target label-set knowledge and source samples during deployment, conflicting with universal and source-free adaptation scenarios.

  • Method

    The framework first trains a generative classifier with labeled synthetic negative samples, then performs source-free deployment adaptation using the instance-level Source Similarity Metric.

  • Results

    The method achieves state-of-the-art results on several tasks, including 64.72% accuracy versus 13.43% for UAN*, while using fewer parameters and less deployment training time.

  • Takeaways & Limitations

    Universal Source-Free Domain Adaptation can address unknown label-set relationships without source samples during adaptation and is suitable for real-time deployment.

  • Takeaways & Limitations

    Generative memory-network solutions are not scalable to large source datasets because they add parameters and training difficulties.

Abstract

from arXiv · show

There is a strong incentive to develop versatile learning techniques that can transfer the knowledge of class-separability from a labeled source domain to an unlabeled target domain in the presence of a domain-shift. Existing domain adaptation (DA) approaches are not equipped for practical DA scenarios as a result of their reliance on the knowledge of source-target label-set relationship (e.g. Closed-set, Open-set or Partial DA). Furthermore, almost all prior unsupervised DA works require coexistence of source and target samples even during deployment, making them unsuitable for real-time adaptation. Devoid of such impractical assumptions, we propose a novel two-stage learning process. 1) In the Procurement stage, we aim to equip the model for future source-free deployment, assuming no prior knowledge of the upcoming category-gap and domain-shift. To achieve this, we enhance the model's ability to reject out-of-source distribution samples by leveraging the available source data, in a novel generative classifier framework. 2) In the Deployment stage, the goal is to design a unified adaptation algorithm capable of operating across a wide range of category-gaps, with no access to the previously seen source samples. To this end, in contrast to the usage of complex adversarial training regimes, we define a simple yet effective source-free adaptation objective by utilizing a novel instance-level weighting mechanism, named as Source Similarity Metric (SSM). A thorough evaluation shows the practical usability of the proposed learning framework with superior DA performance even over state-of-the-art source-dependent approaches.

1. Introduction

The paper targets domain adaptation that remains source-free during deployment and does not assume a known source–target label-set relationship. It proposes synthetic negative samples and a unified framework for handling varied category gaps and domain shifts.

  • Motivation: Existing DA methods commonly assume shared, partial, or otherwise known source–target label-set relationships, requiring separate solutions for different category gaps.Closed-set DA assumes Cs = Ct, while Partial DA assumes Ct ⊂ Cs; other settings relax these assumptions differently.
  • Motivation: Prior unsupervised DA methods generally require source and target samples to coexist during deployment, which is impractical when source data is unavailable for privacy, proprietary, or real-time reasons.Adversarial discriminative approaches depend on source samples to characterize the source distribution.
  • Proposed direction: The proposed framework uses synthetic negative samples formed from source categories to approximate characteristics of novel target categories and improve rejection of out-of-source samples.Composite samples are intended to provide a better proxy for unknown target categories than unrelated negative datasets.
  • Contribution: The paper introduces Universal Source-Free Domain Adaptation, addressing unknown category gaps without source access during adaptation.The framework is presented as applicable across a variety of unknown label-set relationships.

2. Related work

Prior domain adaptation research is organized around assumptions about source–target label-set relationships and commonly uses distribution matching or adversarial strategies. Universal DA removes the need to know that relationship but, in prior work, still assumes access to source and target samples during adaptation.

  • Closed-set DA: Closed-set DA methods minimize latent-space domain gaps using statistical distances or adversarial distribution matching, with some methods generating target-like images through GANs.These approaches are reviewed as one major division based on label-set assumptions.
  • Universal DA: Universal DA requires no prior knowledge of the source–target label-set relationship but traditionally retains access to both source and target samples during adaptation.This distinguishes Universal DA from settings that assume a specific category-gap while preserving a source-dependent deployment assumption.
  • Universal DA: The paper positions its setting as extending Universal DA to source-free adaptation, where previously seen source samples are unavailable during deployment.The proposed framework is described as addressing Universal Source-Free Domain Adaptation.

3. Proposed approach

The proposed framework prepares a model for source-free adaptation without knowing the future category gap or domain shift, then adapts using unlabeled target data while preserving source knowledge. Procurement uses simulated negatives and generative modeling; Deployment uses domain-specific features and SSM-based weighting.

  • Overview: The two-stage process separates model preparation with labeled source data from source-free adaptation using unlabeled target samples.The source dataset is inaccessible during Deployment.
  • Procurement stage: Procurement must place source clusters suitably for unknown category gaps and enable rejection of out-of-distribution samples.These requirements address the inability to modify source-cluster placement using target samples during source-free adaptation.
  • Procurement stage: Simulated negative samples are created by randomly compositing local regions from images belonging to different positive source classes.The resulting samples are expected to lie between source clusters and yield |Cn| = |Cs|C2 new class labels.
  • Procurement stage: The generative source classifier combines a backbone, feature extractor, and classifier with source-class priors whose loss enforces intra-class compactness and inter-class separability.The classifier operates over positive and simulated negative categories, while generative reconstruction regularizes the representation.
  • Deployment stage: Deployment initializes a domain-specific target feature extractor from the trained source extractor and trains it while retaining the source-cluster placement.Only the target feature extractor is adapted, and the generative classifier supplies the positive-versus-negative structure.
  • Deployment stage: The Source Similarity Metric weights each target instance toward positive or negative source clusters according to its similarity to positive source categories.The adaptation objective moves higher-SSM samples toward positive clusters and lower-SSM samples toward negative clusters.
  • Deployment stage: Separate positive- and negative-class probability vectors support entropy minimization for distinguishing target-private samples from the full target dataset.This entropy treatment complements the SSM-weighted adaptation objective.

4. Experiments

Experiments evaluate USFDA across datasets, label-set relationships, source-free metrics, and ablations. The results show strong performance, practical computational advantages, and generalization across unknown category-gaps, while excessive negative classes can cause under-fitting.

  • Comparison against prior arts: USFDA achieves state-of-the-art results in a source-free setting across several domain-adaptation tasks.The evaluation compares USFDA with UAN and other prior methods across multiple datasets.
  • Comparison against prior arts: USFDA achieves much higher Tunk than UAN* on evaluated datasets, improving detection of target-private classes.Tunk measures the proportion of target-private samples classified as unknown, complementing Tavg under Open-set class imbalance.
  • Comparison against prior arts: Deployment uses 3.5M parameters and 44s of training, compared with UAN’s 26.7M parameters and 450s in a consistent setting.Procurement uses 11.1M parameters and 380s; the deployment-stage advantage supports real-time adaptation.
  • Ablative analysis: The SSM separates target-private from target-shared samples, attributed to the generative Procurement stage distinguishing marginally more negative target-private samples.Figure 5A analyzes SSM histograms at Procurement iterations 100 and 500.
  • Sensitivity analysis: USFDA maintains reasonably low sensitivity to fixed hyperparameters, treating α = 0.2 and β = 0.1 as global constants while |Cn| remains adjustable.The analysis reports complementary regularization effects and reasonably low sensitivity across these hyperparameters.
  • Generalization across category-gap: Varying private classes across source and target label spaces shows USFDA’s superiority in practical category-gap scenarios near the diagonal.The comparison fixes |Cs ∪Ct| = 31 and reports Tavg across the label-set relationship grid.
  • DA in absence of shared categories: With no shared categories, USFDA obtains 64.72% one-shot recognition accuracy versus 13.43% for UAN*.This setting uses Office-31 A→D with |Cs| = 15 and |Ct| = 16, allowing one labeled target sample per target category.
  • Dependency on the simulated negative dataset: Increasing the number of simulated negative classes can make the model under-fit positive source categories because of over-regularization.Accuracy decreases beyond a certain limit as |Cn| increases.

5. Conclusion

The paper introduces Universal Source-Free Domain Adaptation without assuming a source–target label-set relationship. Its two-stage design emphasizes robust procurement and SSM-based deployment adaptation.

  • The framework targets practical domain adaptation without assuming the relationship between source and target label sets.
  • The Procurement stage is crucial because it learns general class-separability knowledge while improving robustness to out-of-distribution samples.
  • Deployment-stage success is attributed to learning objectives that effectively use the Source Similarity Metric.

Supplementary: Universal Source-Free Domain Adaptation

The supplementary material covers implementation details and additional analyses for Universal Source-Free Domain Adaptation, including stage descriptions, complexity, label-set variation, and visualizations.

  • Implementation Details: Implementation details cover the Procurement and Deployment stages.
  • Additional Results: Additional results examine pretraining, complexity, varying label-set relationships, sensitivity, closed-set adaptation, source accuracy, incremental classification, and feature visualization.
  • The supplementary material also includes computing-resource specifications and code references.

1. Notations

The paper summarizes its notation in a dedicated notation table.

  • A notation table summarizes the symbols used throughout the paper.
  • The notation summary is presented in Table 1.
  • The section provides a reference for interpreting the paper’s notation.

2. Implementation Details

The implementation uses separate Procurement and Deployment designs. Procurement generates negative examples and classifiers for future shifts, while Deployment adapts the feature extractor using source-similarity weighting without the decoder.

  • Procurement Stage: The classifier architecture permits changing the number of negative classes after Procurement for later adaptation.This design anticipates an additional domain shift after an earlier adaptation.
  • Procurement Stage: The method maintains separate positive and negative classifiers and concatenates their logits before applying softmax.This supports retraining with a different number of negative classes after Deployment.
  • Procurement Stage: Negative examples are generated by compositing regions from images belonging to different positive source classes.Quadratic-spline masks splice alternate image regions to create four composites per image pair.
  • Procurement Stage: The Procurement loss promotes intra-class compactness and inter-class separability, while negative samples are excluded because they approximate future private classes.
  • Procurement Stage: Training samples equal numbers of positive and negative images, producing an effective batch size of 64.Each iteration samples 32 positive and 32 negative images.
  • Deployment Stage: During Deployment, only the feature extractor is trainable, and SSM uses confidence on source classes after softmax over positive and negative categories.The Procurement decoder is unavailable during Deployment.

3. Additional Results

Additional experiments show that the proposed method remains effective with unrelated pretraining and improves target-unknown detection while reducing adaptation cost. It also achieves state-of-the-art performance without source data during adaptation.

  • Additional Results: With Places-pretrained ResNet-50 on Office-31 A→D, the method outperforms source-dependent approaches.This evaluates applicability when initialization uses an unrelated dataset rather than ImageNet.
  • Additional Results: The method uses fewer trainable parameters than UAN by freezing the backbone and omitting an adversarial discriminator.The reduced parameterization follows from the frozen backbone and discriminator-free design.
  • Additional Results: 44 sec versus 280 sec in UAN for Office-31 A→D adaptation demonstrates substantially shorter training time.The comparison uses batch size 32.
  • Additional Results: USFDA significantly improves target-unknown accuracy over UAN on most evaluated label-set settings.The improvement is attributed to the Procurement stage's outlier-class detection capability.

3.4. Sensitivity Analysis

The experiments examine sensitivity to negative-class count, weighting, and batch-size choices, while also comparing the method in unsupervised closed-set adaptation. Source accuracy remains stable under the partially generative framework.

  • Sensitivity Analysis: The method fixes α = 0.2, β = 0.1, |Cn| = |Cs|C2, and b+ve/b−ve = 1 across experiments.These hyperparameters are treated as global constants, with sensitivity analyzed in Figure 2.
  • Sensitivity Analysis: Figure 2 evaluates adaptation sensitivity to |Cn|, α, and batch-size ratio across varied datasets.The negative-class analysis varies |Cs| or |Ct| while fixing the remaining quantities; the batch-size analysis fixes b+ve + b−ve = 64.
  • Sensitivity Analysis: The method is additionally evaluated against closed-set DA methods DAN, ADDA, CDAN, and UAN in the source-free closed-set scenario.DAN, ADDA, and CDAN assume shared source-target label spaces, while other methods require source retraining during adaptation.
  • Sensitivity Analysis: Source-sample accuracy does not drop as a result of the partially generative framework.The observation is reported for the experiments associated with Figure 5C.

3.7. Incremental one-shot classification

One-shot nearest-neighbor evaluation tests whether class separability transfers to target data using one labeled sample per target category. The proposed framework produces substantially better target classification and structured feature clusters than UAN*.

  • Incremental one-shot classification: The evaluation uses one labeled target sample from each target category for nearest-neighbor classification.Predicted labels are assigned by minimizing feature-space distance to the corresponding target exemplars.
  • Incremental one-shot classification: 64.72% accuracy versus 13.43% for UAN* shows stronger one-shot target classification by the proposed framework.Higher accuracy indicates clustering in the intermediate feature representation.
  • Incremental one-shot classification: The t-SNE visualization uses intermediate features for both target and source samples to inspect their embedding structure.Target embeddings are computed as ut = Ft ◦ M(xt), while source embeddings are computed as us = Fs ◦ M(xs).
  • Feature space visualization: Source categories form compact clusters, while target-private clusters separate from source-shared and source-private clusters.The paper attributes these patterns to the Procurement stage and SSM weighting in Deployment.

4. Miscellaneous

The Procurement and Deployment stages use a machine with an Intel Core i7-7700K CPU, 32 GB RAM, and an NVIDIA GeForce GTX 1080Ti GPU. The implementation uses Python 2.7, PyTorch 1.0.0, and CUDA v8.0.61.

  • Miscellaneous: Both learning stages were run with an Intel Core i7-7700K, 32 GB RAM, and an NVIDIA GeForce GTX 1080Ti.The GPU has 11 GB of memory.
  • Miscellaneous: The model was trained in Python 2.7 with PyTorch 1.0.0 and CUDA v8.0.61.
Loading 2004.04393v1…