Source-linked AI summary
Correlation Alignment for Unsupervised Domain Adaptation
Baochen Sun, Jiashi Feng, Kate Saenko
TL;DR
Domain adaptation must transfer knowledge across changing source and target distributions when target labels are unavailable or limited. CORAL aligns second-order statistics with simple linear or deep transformations, and the resulting methods perform strongly across benchmark domain shifts, while relying on assumptions about transferable statistics and class means.
Problem
Domain adaptation seeks to transfer knowledge across changing source and target distributions, especially when labeled target examples are limited and unlabeled target data are more available.
Method
CORAL aligns source and target second-order statistics without target labels, using linear source transformations, equivalent classifier-weight transformations, and differentiable deep-network correlation losses.
Results
CORAL achieves the best average performance across 12 domain shifts, CORAL-LDA outperforms LDA by a large margin, and Deep CORAL achieves better average performance than CORAL and six baselines.
Takeaways & Limitations
The approach provides simple, efficient unsupervised adaptation for shallow classifiers and deep neural networks using unlabeled target distributions.
Takeaways & Limitations
The LDA adaptation assumes that the difference between positive and negative means is the same in source and target, and dissimilar domain statistics can significantly hurt performance.
Abstract
from arXiv · showhide
In this chapter, we present CORrelation ALignment (CORAL), a simple yet effective method for unsupervised domain adaptation. CORAL minimizes domain shift by aligning the second-order statistics of source and target distributions, without requiring any target labels. In contrast to subspace manifold methods, it aligns the original feature distributions of the source and target domains, rather than the bases of lower-dimensional subspaces. It is also much simpler than other distribution matching methods. CORAL performs remarkably well in extensive evaluations on standard benchmark datasets. We first describe a solution that applies a linear transformation to source features to align them with target features before classifier training. For linear classifiers, we propose to equivalently apply CORAL to the classifier weights, leading to added efficiency when the number of classifiers is small but the number and dimensionality of target examples are very high. The resulting CORAL Linear Discriminant Analysis (CORAL-LDA) outperforms LDA by a large margin on standard domain adaptation benchmarks. Finally, we extend CORAL to learn a nonlinear transformation that aligns correlations of layer activations in deep neural networks (DNNs). The resulting Deep CORAL approach works seamlessly with DNNs and achieves state-of-the-art performance on standard benchmark datasets. Our code is available at:~\url{https://github.com/VisionLearningGroup/CORAL}
1 Introduction
CORAL addresses unsupervised domain adaptation by aligning source and target feature distributions using second-order statistics and unlabeled target data. It offers linear transformations, classifier-weight adaptations, and deep-network extensions for practical domain transfer.
- Unsupervised adaptation: Unsupervised adaptation is emphasized because unlabeled target examples are typically more abundant and easier to obtain than annotated examples.This setting is motivated by applications such as adapting pedestrian detectors to changing visual appearances.
- CORAL motivation: CORAL minimizes differences between source and target second-order statistics by transferring target feature correlations to transformed source features.The method uses correlations to capture domain structure while requiring no target labels.
- Linear CORAL: The linear solution re-colors whitened source features with target covariance, then trains a classifier on the transformed source data.Its main computations are covariance estimation and a whitening/re-coloring transformation.
- Linear classifiers: For linear classifiers, applying CORAL to classifier weights instead of examples improves efficiency when target data are numerous and high-dimensional.The equivalence avoids transforming every target example during classification.
- Extensions and results: CORAL-LDA outperforms standard LDA by a large margin, while Deep CORAL extends the approach to nonlinear, end-to-end neural-network adaptation.Deep CORAL uses a differentiable CORAL loss for deep neural networks.
2 Linear Correlation Alignment
Linear CORAL aligns source and target covariance statistics through a transformation that whitens source features and re-colors them using target covariance. Regularized whitening provides a stable efficient implementation, while the transformed representation can support direct classifier-weight adaptation.
- Formulation: CORAL minimizes covariance distance between transformed source features and target features using a linear transformation and squared Frobenius norm.The transformed source covariance is compared directly with the target covariance.
- Analytical solution: When source covariance rank is sufficient, the analytical solution chooses the transformed source covariance to equal the target covariance.Low-rank covariances require a general solution based on the minimum of the source and target ranks.
- Analytical solution: The low-rank solution retains the target covariance components associated with the largest min(rCS,rCT) singular values and vectors.The result follows from the stated SVD-based lemma and theorem.
- Algorithm: Traditional whitening adds diagonal regularization λ before applying inverse-square-root whitening and square-root coloring operations.This avoids expensive SVD steps and makes covariance matrices full rank.
- Algorithm: The implementation is stable across λ ∈ {0, 0.001, 0.01, 0.1, 1}, with λ = 1 used in the reported experiments.At λ = 0, the analytical solution in Equation (2) is used.
- Classifier application: Transforming source features into target space lets classifiers use source labels while operating on features aligned with the target domain.The alternative of transforming target data into source space performed worse in the cited experiments.
- Classifier application: For linear classifiers, CORAL can equivalently transform classifier parameters, reducing computation when few classifiers operate on many high-dimensional target examples.The approach is applied to LDA, whose weights already depend on data covariance.
3 CORAL Linear Discriminant Analysis
CORAL-LDA adapts LDA-based detection to domain shift by using target statistics to decorrelate target features, while retaining an efficient linear-classifier formulation. The method addresses the failure of mismatched covariance statistics and supports unsupervised or semi-supervised adaptation.
- CORAL-LDA: CORAL-LDA applies CORAL to multiple linear classifiers, using LDA as an efficient example for object detection.It is designed for cross-domain classification and detection settings.
- LDA background: LDA models class-conditional feature distributions with normal densities sharing a covariance matrix, producing a linear classifier.The shared covariance assumption enables inverse-covariance decorrelation of features.
- Domain mismatch: Reusing global covariance and mean statistics across categories can remove discriminative information when source and target domains have different statistics.Figure 4 illustrates the effect of decorrelating with mismatched-domain statistics.
- CORAL adaptation: CORAL adapts the classifier by decorrelating source features with source covariance, then using target covariance to properly decorrelate target inputs before scoring.This produces a target-aware transformation rather than applying the source whitening directly to target data.
- Assumption: The method assumes that the difference between positive and negative class means is the same in source and target domains.The covariance can be estimated from unlabeled target data, but the positive-class mean cannot.
- Practical scope: Dissimilar domain statistics can significantly hurt performance, although either source or target statistics may work when the other domain lacks suitable background data.CORAL-LDA also supports semi-supervised adaptation by combining source-positive and target-positive templates.
4 Deep CORAL
Deep CORAL incorporates a differentiable CORAL loss into deep neural networks to learn nonlinear, end-to-end domain adaptation. It aligns source and target activation covariances while balancing target adaptation against source classification accuracy.
- Deep CORAL extends CORAL to deep neural networks through a differentiable loss that enables end-to-end adaptation and nonlinear transformation learning.
- Deep CORAL can be integrated into different network layers or architectures; the example applies it to AlexNet’s fc8 layer.
- The CORAL loss measures the distance between source and target feature covariance matrices, aligning their second-order activation statistics.
- Deep CORAL uses batch covariances with shared network parameters between source and target networks, although other settings are possible.
- Joint training combines classification and CORAL losses so learned features remain discriminative while generalizing to the target domain.
- The combined objective weights CORAL losses by λ_i, trading off domain adaptation against classification accuracy on the source domain.
5 Experiments
Experiments evaluate CORAL and Deep CORAL for unsupervised adaptation across object-recognition and detection benchmarks. CORAL achieves the best average performance across 12 shallow-feature domain shifts, while Deep CORAL achieves the best average performance among methods evaluated on six deep-feature shifts.
- Object Recognition with Shallow Features: CORAL is evaluated on Office-Caltech10 object recognition using SURF features across all 12 source-to-target domain shifts.The protocol uses four domains and 800-bin bag-of-words SURF histograms.
- Object Recognition with Shallow Features: CORAL achieves the best average performance across all 12 shallow-feature domain shifts, outperforming the no-adaptation baseline and five published methods.The comparison includes SVMA, DAM, GFK, SA, TCA, and NA.
- Object Recognition with Deep Features: Deep CORAL is evaluated on the standard Office dataset across all six shifts using unlabeled target data and a CORAL loss applied to the last classification layer.The last fully connected layer is used as the general case for deep classifier architectures.
- Object Recognition with Deep Features: Deep CORAL achieves better average performance than CORAL and six baseline methods, attaining the highest accuracy in 3 of 6 shifts.For the remaining three shifts, its margin over the best baseline is ≤0.7.
- Domain Adaptation Equilibrium: On A→W, adding CORAL loss improves target-domain performance while maintaining strong source accuracy and constraining source-target CORAL distance during fine-tuning.Without CORAL loss, the distance becomes ≥100 times larger, consistent with source overfitting; with the loss, classification and CORAL losses reach an approximate equilibrium.
6 Conclusion
CORAL is a simple, efficient unsupervised domain-adaptation method that aligns second-order statistics without target labels, with linear and deep variants. Its stated limitation is that it captures only second-order statistics and may not preserve higher-order statistics.
- CORAL aligns source and target distributions by minimizing differences between their second-order statistics without requiring target labels.
- Linear CORAL transforms source features before classifier training, while an equivalent weight transformation improves efficiency for linear classifiers.
- CORAL-LDA improves classification accuracy over standard LDA on several domain-adaptation benchmarks.
- Deep CORAL aligns correlations of neural-network layer activations and supports end-to-end adaptation in arbitrary network architectures.
- CORAL captures second-order statistics only and may not preserve higher-order statistics.