PaperlayerView exact revision on arXiv

Source-linked AI summary

Conformal Risk Minimization for Semi-Supervised Domain Adaptation via Optimal Transport

Manos Giannopoulos, Yi Shen, Michael M. Zavlanos

arXiv:2608.23153v1cs.LG

TL;DR

The paper addresses efficient uncertainty quantification for SSDA when target labels are scarce and post-hoc conformal prediction can produce overly large sets. It proposes CP-JDOT, which integrates CRM into SSDA training using OT-derived pseudolabels, and reports smaller average set sizes than CP-agnostic baselines while preserving coverage and supporting domain-specific constraints.

  • Problem

    SSDA methods focus primarily on point-prediction accuracy, while post-hoc conformal prediction can yield large sets and CRM requires labeled data scarce in the target domain.

  • Method

    CP-JDOT jointly trains domain adaptation and conformal objectives, using a small labeled target set for differentiable quantiles and OT to generate pseudolabels for unlabeled targets.

  • Results

    CP-JDOT consistently achieves smaller downstream average set sizes than CP-agnostic SSDA methods with post-hoc conformal wrappers while preserving coverage guarantees.

  • Takeaways & Limitations

    The framework enables compact, coverage-valid prediction sets and supports domain-specific constraints such as penalizing mutually contradictory diagnoses.

  • Takeaways & Limitations

    Full conditional coverage is generally impossible in finite-sample, distribution-free settings without including the entire label space in prediction sets.

Abstract

from arXiv · show

In high-stakes healthcare applications, machine learning models are frequently trained on data from one patient population and deployed on another, creating a distribution shift that degrades both accuracy and reliability. Semi-Supervised Domain Adaptation (SSDA) addresses this by leveraging labeled data from some source domain to improve model performance on a target domain where labels are scarce. However, existing SSDA methods optimize primarily for point-prediction accuracy and offer no principled uncertainty quantification --- a prerequisite for clinical trust. Conformal Prediction (CP) can address this limitation by providing prediction sets with rigorous, distribution-free coverage guarantees. However, applying CP post-hoc to a pre-trained model can yield prohibitively large prediction sets, as SSDA pre-training methods do not account for the nonconformity score geometry that determines conformal set size. Conformal Risk Minimization (CRM) has been used to resolve this issue in the fully supervised setting by integrating the CP objective directly into model training, but it requires a large labeled dataset to compute nonconformity thresholds during training, precisely the data that is scarce in the SSDA regime. We propose an end-to-end framework that integrates CRM into the SSDA training objective, enabling effective CRM in the limited-labeled-target-data regime. The key idea is to utilize Optimal Transport (OT) to generate pseudolabels for unlabeled target instances, providing the additional training signal needed by CRM to operate using only a small labeled target set. This results in a model jointly optimized for domain invariance and conformal efficiency, producing prediction sets that are compact, coverage-valid, and support domain-specific constraints such as excluding mutually contradictory diagnoses in skin lesion classification.

1 Introduction

The paper introduces CP-JDOT, an end-to-end SSDA framework that integrates conformal optimization into training when labeled target data are scarce. It uses OT-derived pseudolabels to jointly improve domain alignment and conformal efficiency while preserving coverage and supporting domain-specific set constraints.

  • Motivation: SSDA leverages labeled source data and limited labeled target data, but high-stakes deployment also requires calibrated uncertainty beyond point-prediction accuracy.Conformal Prediction provides distribution-free, finite-sample uncertainty guarantees through prediction sets.
  • Approach: The training procedure computes a differentiable target nonconformity quantile, then optimizes domain and conformal alignment losses using pseudolabeled unlabeled target data.The conformal alignment component is derived from prediction-set sizes.
  • Motivation: Post-hoc conformal prediction can produce clinically impractical large sets because accuracy-focused pre-training ignores the nonconformity geometry governing set size.Optimized post-hoc calibration can improve set size only modestly.
  • Approach: CP-JDOT integrates a CRM objective into Joint Domain Adaptation with OT, using OT-derived pseudolabels to address scarce target labels.The framework calibrates a conformal predictor in a globally aligned feature space.
  • Contributions: CP-JDOT supports domain-specific penalties that can reduce coverage confusion by discouraging mutually exclusive high-level classes from appearing together in prediction sets.The resulting sets are intended to be statistically valid and semantically meaningful.
  • Contributions: Empirically, CP-JDOT consistently outperforms CP-agnostic SSDA methods with post-hoc conformal wrappers on downstream average set size while preserving coverage guarantees.The paper frames this as a rigorous empirical comparison against state-of-the-art baselines.

2 Preliminaries

The preliminaries define conformal coverage and split conformal prediction, then motivate CRM as training for smaller calibrated prediction sets. They also identify the labeled-data challenge that CRM faces in SSDA.

  • Conformal Prediction: Conformal prediction wraps a probabilistic classifier to produce sets intended to contain the true label with a desired coverage probability.Coverage is the probability that Cθ(X) contains Y.
  • Coverage Constraints: Full conditional coverage requires the guarantee for every input, but is generally impossible distribution-free with finite samples without trivial full-label prediction sets.Marginal coverage instead guarantees coverage on average over the feature distribution.
  • Coverage Constraints: Marginal coverage is attainable with standard conformal methods but may under-cover particular regions or subpopulations of the input space.Its guarantee is therefore weaker than full conditional coverage.
  • Conformal Predictors: Split Conformal Prediction uses a held-out calibration set to compute a nonconformity-score threshold and then construct prediction sets for new inputs.The threshold and predicted probabilities both affect the resulting sets.
  • Conformal Predictors: HPS thresholds predicted probabilities, whereas APS uses cumulative sums of sorted class probabilities; both calibrate a quantile to ensure marginal coverage.The calibration quantile is based on scores at the true labels.
  • Efficiency: Because coverage is guaranteed regardless of pre-training, inefficiency—the average prediction-set cardinality—is the standard comparison metric.For a test set, cardinality is represented by |Cθ(x_i)|.
  • Conformal Risk Minimization: CRM augments ERM with a conformal alignment loss that minimizes expected smoothed set cardinality while trading off classification and conformal objectives.The smoothed indicator enables optimization of set membership.
  • Conformal Risk Minimization: CRM in SSDA must estimate changing conformal quantiles and classification losses from few labeled target samples while bridging source and target domains.This is the central data-use challenge addressed by the proposed method.

3 Related Work

Prior work addresses domain transfer, uncertainty quantification, and conformal efficiency separately, but this paper combines CRM with SSDA using OT-based supervision for unlabeled target data.

  • Optimal Transport establishes cross-domain sample correspondences and can transfer source-label information to unlabeled target observations.
  • SSDA uses limited labeled target data alongside labeled source data to adapt predictions across domains.
  • CP-JDOT incorporates CRM into SSDA training and uses OT-based adaptation to supervise unlabeled target data, comparing against post-hoc conformalized SSDA methods.
  • Conformal Prediction provides finite-sample prediction sets with marginal coverage guarantees under exchangeability.
  • CRM pre-trains models to learn better nonconformity scores, reducing downstream prediction-set inefficiency after post-hoc calibration.

4 Problem Definition

The paper formulates target-domain learning as CRM under domain shift, seeking small conformal prediction sets while maintaining coverage and adapting with limited target labels.

  • The task uses labeled source data and a small labeled target subset drawn from different joint distributions over a shared input and label space.
  • The predictive model is wrapped by a post-hoc conformalizing function to produce target-domain prediction sets.
  • CRM aims to minimize expected target prediction-set cardinality while maintaining the desired coverage level.
  • The DA-CRM objective combines classification, domain-adaptation, and conformal-alignment losses.
  • The classification, adaptation, and conformal terms respectively promote predictive accuracy, cross-domain adaptation, and smaller resulting prediction sets.

5 Method

CP-JDOT extends DeepJDOT with CRM-aware losses, using differentiable conformal thresholds and OT couplings to jointly align domains, transfer labels, and improve conformal efficiency.

  • 5.1 Joint Domain Adaptation with Optimal Transport: DeepJDOT decomposes the model into an embedding function and classifier optimized jointly for domain adaptation.
  • 5.2 Incorporating CRM with CP-JDOT: The framework’s stated goal is more efficient downstream conformal sets than standard SSDA followed by post-hoc calibration.
  • 5.2 Incorporating CRM with CP-JDOT: CP-JDOT computes a differentiable (1 −α)-quantile from labeled target nonconformity scores and uses it as the training threshold.
  • 5.2 Incorporating CRM with CP-JDOT: The training loop alternates between solving an OT coupling problem for minibatches and updating the embedding and classifier by backpropagation.
  • 5.2 Incorporating CRM with CP-JDOT: The CRM-aware classification loss penalizes exclusion of the OT-transported class from a target conformal set using a differentiable sigmoid surrogate.
  • 5.1 Joint Domain Adaptation with Optimal Transport: The OT coupling aligns source and unlabeled target samples while transferring source-label information to target instances.
  • 5.2 Incorporating CRM with CP-JDOT: The total objective combines source ERM, CRM-aware cross-domain classification, OT-based alignment, and a penalty that reduces unlabeled-target prediction-set size.
  • 5.3 Encoding Domain Knowledge into Lcls: For structural constraints, a control loss preserves the transported class while penalizing inclusion of specified conflicting classes.

6 Results

CP-JDOT is evaluated against post-hoc conformalized SSDA baselines on Office-Home and skin-lesion transfer tasks, targeting smaller and structurally more useful prediction sets. It consistently reduces set size, while customized losses reduce clinically conflicting sets with a modest size tradeoff.

  • Average Set Size Reduction: CP-JDOT achieves the smallest average set size across all 12 Office-Home transfer directions, outperforming the best baseline by approximately 5–10% on average.The improvement is strongest in easier directions and remains consistent in harder transfers.
  • Average Set Size Reduction: CP-JDOT improves conformal set size despite achieving up to 10% lower top-1 accuracy than the strongest SSDA baselines.This supports a mismatch between conventional accuracy optimization and downstream conformal set-size efficiency.
  • Average Set Size Reduction: Skin-lesion results improve over baselines on four of five target datasets, with gains of up to 12% outside SONIC.SONIC is an exception because all algorithms can achieve accuracy close to or above 1 − α under the available labeled-target setting.
  • Loss Function Customization for Set Control: Lcontrol encodes the constraint that prediction sets should not simultaneously include benign and malignant classes.The goal is to reduce conflicting diagnostic signals even when this slightly increases cardinality.
  • Loss Function Customization for Set Control: The Lcontrol ablation reduces conflicting-set rates from 12%, 28%, 38%, 2%, and 30% to 4%, 6%, 9%, 0.5%, and 9% across PH2, MSK, Derm7pt, SONIC, and UDA.The reduction is 60–75% across datasets for less than a 10% increase in set size.

7 Conclusion

The paper concludes that CP-JDOT jointly optimizes domain adaptation and conformal objectives instead of treating uncertainty quantification as post-processing. Its framework supports efficient, statistically calibrated, and application-aligned prediction sets.

  • Conclusion: CP-JDOT integrates conformal objectives directly into domain-adaptation training so representations and prediction scores reflect efficient prediction-set geometry.The framework also supports customizable losses for domain-specific structure.
  • Conclusion: Customizable training losses allow CP-JDOT to incorporate domain-specific structure into prediction sets beyond reducing set size.The stated scope includes prediction sets that are statistically calibrated, efficient, and aligned with application-specific requirements.

A Geometric Optimal Transport information

CP-JDOT uses label-guided Geometric Optimal Transport as a replacement for Euclidean transport cost. GOT captures within-domain and cross-domain geometry while incorporating source-label class structure.

  • Transport Cost: GOT replaces DeepJDOT’s squared Euclidean transport cost with a diffusion-based, label-guided geometric cost.The source-label information encourages the diffusion geometry to respect source-domain class structure.
  • Affinity and Diffusion Construction: GOT constructs source, target, and cross-domain affinity matrices and normalizes them into diffusion operators.The operators are combined into a source-to-target diffusion operator.
  • Affinity and Diffusion Construction: Each source–target diffusion connectivity value captures geometry within both domains and their relationship.The transport cost is obtained as the negative logarithm of the diffusion-based transport probability.
  • Implementation: The label-guided GOT construction is used as a drop-in replacement for the Euclidean transport cost in DeepJDOT.Remaining affinity, normalization, and diffusion details follow Maman & Talmon (2025).

B Experimental Details

The experiments use fixed hardware and software, pretrained ResNet backbones, and SGD-based training configurations for Office-Home and skin-lesion transfer tasks.

  • Reproduction Environment: All reproduction experiments use an NVIDIA TITAN RTX GPU, an Intel Core i9-9980XE CPU, PyTorch, and CUDA 12.2.The same hardware and software environment is used throughout.
  • Office-Home: Office-Home uses a pretrained ResNet-34 backbone, balanced source minibatches, SGD at learning rate 0.001, batch size 65, and 10,000 iterations.Both 1-shot and 3-shot labeled-target settings are evaluated.
  • Skin Lesion: Skin-lesion experiments use a pretrained ResNet-18 backbone and SGD with learning rate 0.001.The configuration also specifies OT, diffusion-kernel, and unbalanced-Sinkhorn hyperparameters.

C.1 Accuracy Tables

This section reports Top-1 Accuracy tables for Office-Home under 1-shot and 3-shot transfer, and for Skin Lesion Classification under 10-shot transfer.

  • Table 4 reports Top-1 Accuracy on Office-Home using a ResNet-34 backbone in the 1-shot setting.
  • Table 5 reports Top-1 Accuracy on Office-Home using a ResNet-34 backbone in the 3-shot setting.
  • Table 6 reports Top-1 Accuracy on Skin Lesion Classification using a ResNet-18 backbone in the 10-shot setting.
Loading 2608.23153v1…