Source-linked AI summary
Optimal Transport for Multi-source Domain Adaptation under Target Shift
Ievgen Redko, Nicolas Courty, Rémi Flamary, Devis Tuia
TL;DR
The paper studies multi-source domain adaptation under target shift, where domains share output classes but have different class proportions, a largely understudied practical setting. It proposes JCPOT, an optimal-transport method that jointly estimates unlabeled target class probabilities and aligns multiple distributions, and reports superiority over state-of-the-art methods on synthetic and satellite-image segmentation data.
Problem
Target shift changes class proportions between source and target domains, but this widely occurring setting has received little attention in domain adaptation.
Method
JCPOT jointly estimates target class proportions and optimal transport couplings, extending the approach to multiple sources with Wasserstein barycenters.
Results
JCPOT reports superior performance over state-of-the-art methods on synthetic data and satellite-image segmentation experiments.
Takeaways & Limitations
The method provides multi-source adaptation while explicitly correcting class-proportion mismatch under target shift.
Takeaways & Limitations
JCPOT assumes target shift and requires source data distributions to satisfy the paper's target-shift characterization; source class proportions must be reweighted without observed target labels.
Abstract
from arXiv · showhide
In this paper, we propose to tackle the problem of reducing discrepancies between multiple domains referred to as multi-source domain adaptation and consider it under the target shift assumption: in all domains we aim to solve a classification problem with the same output classes, but with labels' proportions differing across them. This problem, generally ignored in the vast majority papers on domain adaptation papers, is nevertheless critical in real-world applications, and we theoretically show its impact on the adaptation success. To address this issue, we design a method based on optimal transport, a theory that has been successfully used to tackle adaptation problems in machine learning. Our method performs multi-source adaptation and target shift correction simultaneously by learning the class probabilities of the unlabeled target sample and the coupling allowing to align two (or more) probability distributions. Experiments on both synthetic and real-world data related to satellite image segmentation task show the superiority of the proposed method over the state-of-the-art.
1 INTRODUCTION
The paper addresses target shift in domain adaptation, where source and target class proportions differ, a setting that is widely occurring but understudied. It proposes JCPOT, which jointly estimates target class proportions and aligns multiple domains using optimal transport.
- Motivation: Most domain adaptation methods address covariate shift, while relatively few explicitly handle changes caused by output-distribution differences.
- Target shift: Target shift occurs when source and target domains differ in class proportions, also known as prior probability shift.
- Optimal transport: Optimal transport can learn a transformation from a source probability distribution to a target distribution while preserving the original marginals.
- Proposed approach: JCPOT corrects target shift by jointly estimating class proportions and transportation plans, extending the approach to multiple source domains.
- Problem with existing adaptation: Under class-proportion imbalance, conventional optimal-transport adaptation can move mass between instances from different classes.
- Reported contribution: The paper reports that JCPOT handles multi-source adaptation under changing output distributions and improves performance as the number of source domains increases.
2 OPTIMAL TRANSPORT
Optimal transport finds a minimum-cost coupling that maps one probability measure to another while satisfying marginal constraints. The paper describes its discrete and regularized formulations and its use for domain adaptation.
- Core formulation: Optimal transport searches for a plan that moves one probability measure onto another at minimum cost.
- Core formulation: The Kantorovich formulation represents transport as a coupling, or joint probability distribution, with prescribed source and target marginals.
- Discrete optimal transport: In the discrete setting, the coupling is a matrix whose entries represent transport mass between empirical source and target bins.
- Discrete optimal transport: The cost matrix records pairwise transport costs, while the coupling matrix has dimensions that grow quadratically with sample size.
- Entropic regularization: Entropy regularization smooths the coupling and enables faster solution through successive projections over marginal constraints.
- Application to domain adaptation: For domain adaptation, optimal transport estimates a source transformation minimizing average displacement relative to the target while preserving topology.
3 DOMAIN ADAPTATION UNDER THE TARGET SHIFT
The section formalizes target shift in multi-source domain adaptation and shows that class-proportion mismatch affects adaptation error. It then motivates estimating target proportions by Wasserstein-distance minimization and proves the resulting solution recovers the target proportions under stated conditions.
- Problem formulation: The multi-source objective is to learn a hypothesis on source domains that performs well on the target domain.Source errors are combined using weights α_k, which reflect the relative proximity of source distributions to the target.
- Theoretical analysis: The generalization bound identifies combined source error, discrepancy between source and target distributions, and a residual term as factors governing target performance.The proposition applies to any fixed α and hypothesis h under a convex loss.
- Theoretical analysis: Reweighting source class distributions can minimize the bound’s discrepancy term when source and target class proportions match, but the target proportions are unknown.The paper therefore estimates them rather than assuming prior access to the target proportions.
- Proportion estimation: Minimizing Wasserstein distances between reweighted sources and the target yields the target class proportions under the proposition’s identifiability assumption.The result extends to multiple sources by minimizing the sum of Wasserstein distances, with empirical convergence characterized through Wasserstein concentration inequalities.
4 JOINT CLASS PROPORTION AND OPTIMAL TRANSPORT (JCPOT)
JCPOT jointly estimates target class proportions and optimal transport plans by reweighting source instances to compensate for class-proportion discrepancies. It supports barycentric source adaptation and label propagation for classifying unlabeled target samples.
- JCPOT method: JCPOT jointly finds optimal transportation plans and target class proportions under the target-shift assumption.Its central operation is reweighting source instances to compensate for differing source and target class proportions.
- Data and weighting: The method represents multiple source domains as labeled instances with class-based masses and a target domain whose sample labels are unknown.Source measures are decomposed by class so their weights can be adjusted to match target proportions.
- Optimization: JCPOT estimates target proportions by solving a constrained Wasserstein barycenter problem that simultaneously enforces source-class and target-marginal constraints.The resulting optimization jointly handles the coupling matrices and class proportions, with domain weights accounting for relative source importance.
- Target classification: Barycentric mapping transports source instances toward target-aligned positions, which can then train a classifier for target prediction.The method extends the optimal-transport mapping to multiple sources and uses transported source samples for classification.
- Target classification: Label propagation uses transport matrices to estimate each target instance’s class proportions from the source-class mass arriving at it.The paper names this alternative procedure JCPOT-LP and describes it as a linear combination of weak classifiers from the source domains.
- Optimization: The Iterative Bregman projections algorithm updates the couplings and class proportions until convergence and can compute domain-specific coupling updates in parallel.The algorithm returns the estimated proportions h and coupling matrices for all source domains.
5 EXPERIMENTAL RESULTS
Experiments compare JCPOT with target-shift and OT-based baselines on synthetic multi-source adaptation and Zurich Summer remote-sensing classification. JCPOT-LP generally achieves the strongest results, including a reported 10% improvement over no adaptation on Zurich data.
- JCPOT is compared with betaEM, betaKMM, MDA Causal, and OTDA variants using label propagation or point transformation.
- Synthetic Data: Synthetic experiments vary the number of source domains from 2 to 20, with 20 sources containing 500 instances each and a target containing 400 instances.
- Synthetic Data: JCPOT-LP provides the best synthetic results except with 20 source domains, where MDA Causal slightly surpasses it.
- Synthetic Data: Target-shift methods perform better than OTDA, supporting algorithms that explicitly account for shifting class proportions.
- Synthetic Data: JCPOT gives comparable or better class-proportion estimates than Scott et al. in most simulated cases and also produces couplings for domain alignment.
- Real-World Data From Remote Sensing Application: On Zurich Summer, JCPOT-LP significantly improves classification accuracy over baselines, with an increase reaching 10% over no adaptation and performance above the Target only baseline.
6 CONCLUSIONS
The paper concludes that JCPOT addresses the understudied target-shift setting by jointly estimating target class proportions and aligning multiple domains through optimal transport. Experiments on synthetic and real-world data report strong performance and reliable proportion estimation.
- JCPOT targets domain adaptation when source and target distributions differ through class-proportion changes.
- The method jointly estimates unlabeled-target class proportions and aligns domains using optimal transport, extending to multiple sources through Wasserstein barycenters.
- Synthetic and real-world experiments report that JCPOT outperforms current state-of-the-art methods and estimates target proportions computationally attractively and reliably.