PaperlayerView exact revision on arXiv

Source-linked AI summary

WhiteCon: Semi-Supervised Domain Adaptation Regression Through Whitening Transform and Dual Consistency

Se Jin Sim, Seoung Bum Kim

arXiv:2609.34078v1cs.CVcs.AI

TL;DR

Regression-focused semi-supervised domain adaptation remains relatively unexplored, especially when target labels are limited. WhiteCon combines domain-specific whitening with dual consistency regularization, and achieves state-of-the-art performance on benchmark datasets. Its reported approach uses whitening to reduce parameter variance under OLS assumptions and aligns feature variances across augmentations for robustness.

  • Problem

    Semi-supervised domain adaptation for continuous-output regression remains relatively unexplored, despite practical settings with limited labeled target data.

    Source: §1 · pp. 1–2

  • Method

    WhiteCon combines domain-specific whitening, which transforms feature covariance to an identity matrix, with dual consistency regularization across predictions and augmented features.

    Source: §1 · p. 3

  • Results

    WhiteCon achieves state-of-the-art performance on multiple benchmark datasets under SSDAR settings.

    Source: §1 · p. 3, §5 · p. 13

  • Takeaways & Limitations

    The paper concludes that WhiteCon is effective for SSDAR, reducing domain discrepancies through whitening and dual consistency regularization.

    Source: §5 · p. 13

    The OLS motivation is analyzed with a single linear-layer regressor, so its stated parameter-variance rationale is bounded to that setup.

    Source: §3.2 · p. 6

Abstract

from arXiv · show

Domain adaptation is crucial for addressing distributional shifts that degrade model performance across domains. While most existing research has centered on classification, semi-supervised domain adaptation regression (SSDAR) for continuous-output tasks remains largely unexplored, particularly in practical scenarios with limited labeled target data. To address this gap, we propose semi-supervised domain adaptation regression through whitening transform and dual consistency (WhiteCon), which combines domain-specific whitening transform (DWT) and dual consistency regularization to enhance training stability and domain adaptation. DWT reduces the variance of the model parameters by transforming the feature covariance matrix into an identity matrix, thus stabilizing training under ordinary least squares assumptions. In addition, variance consistency regularization, as part of dual consistency regularization, aligns the variances of weak, strong, and mixup-augmented features to improve resilience against augmentation-induced perturbations. Empirical evaluations on various benchmark datasets under SSDAR settings demonstrate that the proposed WhiteCon achieves state-of-the-art performance compared to existing methods, effectively addressing domain shifts in regression tasks. The code for WhiteCon is available at https://github.com/sejin-sim/WhiteCon.

1 Introduction

WhiteCon addresses the limited development of regression-focused domain adaptation for continuous outputs when target labels are scarce. It combines domain-specific whitening and dual consistency regularization, and reports state-of-the-art results on benchmark datasets.

1 Introduction

  • Most domain-adaptation research targets classification, whose class-probability methods are poorly suited to continuous regression outputs.Source: §1 · p. 2
  • SSDAR combines labeled source data, unlabeled target data, and a small amount of labeled target data, which can improve performance when available.Source: §1 · p. 2
  • WhiteCon combines domain-specific whitening transform (DWT) with dual consistency regularization to address regression-specific domain shifts.Source: §1 · p. 3
  • DWT transforms the feature covariance matrix into an identity matrix, reducing regression-parameter variance under ordinary least squares assumptions.Variance consistency regularization aligns feature variances across weak, strong, and mixup augmentations to improve robustness.Source: §1 · p. 3
  • WhiteCon achieves state-of-the-art performance on various benchmark datasets, with evaluations including ablations, parameter-variance analysis, labeled-target proportions, and feature visualizations.Source: §1 · p. 3

2 Related work

Prior domain-adaptation work largely developed classification methods, while regression methods use different alignment strategies and carry distinct assumptions. Existing SSDAR approaches include moment alignment with graph Laplacians and invariant representations with domain-specific predictors.

2.1 Domain Adaptation for Classification

  • Classification adaptation uses moment matching and adversarial alignment, while consistency regularization aligns predictions across augmented views; its effectiveness for regression remains unproven.Source: §2.1 · p. 4
  • Classification SSDA methods often rely on softmax class-probability entropy, limiting direct application to continuous-output regression.Source: §2.1 · p. 4

2.2 Domain Adaptation for Regression

  • Regression adaptation includes importance weighting and invariant representations; UDAR methods also use subspace or inverse-Gram alignment, with noted hyperparameter and modeling assumptions.RSD preserves feature scale, while DARE-GRAM requires a low-rank hyperparameter; other approaches assume infinite-width networks or depend on dictionary size and initialization.Source: §2.2 · pp. 4–5
  • DeepDAR aligns distributions with MMD and graph Laplacians, while LIRR uses domain-specific predictors and adversarial learning; the cited work notes limits in connecting alignment to regression outputs or parameter stability.Source: §2.2 · p. 5

3 Proposed Methods

WhiteCon combines domain-specific whitening with dual consistency regularization for semi-supervised domain adaptation regression. It uses whitening to reduce regressor parameter variance under OLS assumptions and consistency losses to align predictions and feature variances across target-data augmentations.

3.1 Preliminaries

  • DWT transforms the feature covariance matrix into the identity, removing feature correlations and aligning source and target distributions.The method applies batch whitening to the extracted features.Source: §3.1 · p. 5

3.2 Motivations

  • Under OLS assumptions, whitening reduces regressor parameter variance, which can help improve regression performance.The regressor is constructed as a single linear layer to clarify the OLS formulation.Source: §3.2 · p. 6

3.3 Overview of the Proposed Method

  • WhiteCon combines DWT with prediction and feature-variance consistency losses, applying weak, strong, and mixup augmentations to unlabeled target samples.DWT is applied in the feature extractor; the losses regularize predictions and features across augmentations.Source: §3.3 · p. 7

3.4 Dual Consistency Regularization

  • Prediction consistency uses weak-augmentation predictions as pseudo-labels for strong and mixup predictions, while feature consistency aligns their variances with weak-feature variance.Because regression outputs are continuous, threshold-based pseudo-label refinement is challenging; variance alignment is presented as additional consistency beyond prediction consistency.Source: §3.4 · p. 8

4 Experiments

Across three benchmark datasets, WhiteCon achieved the best reported overall performance. Experiments used limited labeled target data and analyzed component contributions, parameter variance, label availability, and feature alignment.

4.1 Datasets

  • The three datasets cover head-pose estimation (BIWI, QMUL) and object-position estimation (MPI3D), with 5% of target data labeled for SSDAR training.Source: §4.1 · p. 9

4.2 Experimental Setup

  • Comparisons included supervised, UDA, UDAR, SSDAR, and full-target baselines; experiments used ResNet50, AdamW, and 50 training epochs.Methods that could not be implemented under the same experimental conditions were excluded.Source: §4.2 · pp. 9–10

4.4 Analyses

  • Across BIWI, QMUL, and MPI3D, removing the prediction-consistency component caused the largest degradation overall, while DWT and variance consistency made complementary contributions.Source: §4.4 · p. 10, Table 3 · p. 11

4.3 Results

4.4 Analyses

  • For QMUL F→M, WhiteCon consistently outperformed comparison methods with 5%, 10%, 20%, and 30% labeled target data; at 30%, it exceeded Full_T trained with 100%.The reported scores were 0.91 for WhiteCon at 30% and 0.88 for Full_T.Source: Figure 5 · p. 12, §4.4 · p. 13

5 Conclusion

WhiteCon combines DWT and dual consistency regularization for SSDAR and achieved state-of-the-art performance on multiple benchmarks. The conclusion identifies time-series and tabular data as future directions.

  • WhiteCon combines DWT and dual consistency regularization to reduce domain discrepancies in SSDAR.Source: §5 · p. 13
  • Across multiple benchmark datasets, WhiteCon achieved state-of-the-art performance.Source: §5 · p. 13
  • Future work will extend WhiteCon to time-series and tabular data, requiring modality-specific augmentation and adaptation strategies.Source: §5 · p. 13

A-1. Feature Scale.

The appendix tests whether domain-adaptation regression performance depends on matching supervised learning’s feature scale. Its reported comparisons do not support feature-scale similarity as a strict requirement.

  • RSD’s feature scale of 20.91 was closest to S+T’s 19.76, while WhiteCon achieved the highest overall performance.Source: A-1. Feature Scale · p. 16
  • MMD had the second-highest performance and the second-largest feature scale, at 26.42.Source: A-1. Feature Scale · p. 16
  • The comparison suggests that matching S+T’s feature scale is not a strict requirement for high domain-adaptation regression performance.Source: A-1. Feature Scale · p. 16

A-2. Datasets Settings

The appendix describes benchmark data splits and labeled-target proportions, alongside WhiteCon’s hyperparameter search. Optimal hyperparameters were selected using validation performance.

  • The three benchmarks are BIWI and QMUL for head-pose estimation and MPI3D for object-position estimation; labeled-target proportions included 5%, 10%, 20%, and 30%.Source: A-2. Datasets Settings · p. 17
  • Optimal values in the methods’ hyperparameter searches were selected based on validation performance.Source: Table A2 · p. 18
  • WhiteCon’s hyperparameter search plots MAE against λ1 and λ2 across three benchmark datasets.Source: Figure A2 · p. 18

A-3. Hyperparameter Search

The authors tuned comparison-method hyperparameters for fair evaluation and selected WhiteCon settings by validation performance. The search covered two regularization weights, with MAE-optimal settings reported for three benchmarks.

A-3. Hyperparameter Search

  • The search examined λ1 and λ2 values from {0.1, 0.2, 0.3, 0.4, 0.5}.Fig. A2 shows WhiteCon's hyperparameter search results.Source: A-3. Hyperparameter Search · p. 18
  • All compared methods underwent extensive hyperparameter optimization, with optimal values selected using validation performance.Table A2 lists the ranges examined for each method.Source: A-3. Hyperparameter Search · p. 18
  • WhiteCon's optimal (λ1, λ2) settings were (0.1, 0.1) for BIWI, (0.3, 0.5) for QMUL, and (0.1, 0.1) for MPI3D.The settings were selected based on MAE performance.Source: A-3. Hyperparameter Search · p. 19

A-4. Augmentation

The benchmark images were resized to 224 × 224 pixels, and the methods used weakly augmented images. The augmentation setup used angle-preserving weak transforms, FixMatch-style strong augmentation, and mixup combinations of weakly and strongly augmented samples.

A-4. Augmentation

  • All benchmark images were resized to 224 × 224 pixels.Source: A-4. Augmentation · p. 19
  • Weak augmentation used random cropping and Gaussian blur while excluding angle-altering transformations such as flips.The weak-augmentation guidelines followed Hu et al..Source: A-4. Augmentation · p. 19
  • Strong augmentation followed the strategy used in FixMatch.Source: A-4. Augmentation · p. 19
  • Mixup augmentation linearly combined weakly and strongly augmented samples.The passage points to Section 3.4 for this description.Source: A-4. Augmentation · p. 19

Related papers

Loading 2609.34078v1…