Source-linked AI summary
Deep CORAL: Correlation Alignment for Deep Domain Adaptation
Baochen Sun, Kate Saenko
TL;DR
Deep neural networks can lose accuracy when training and test distributions differ, motivating unsupervised adaptation to unlabeled target domains. Deep CORAL aligns source and target activation correlations through a differentiable deep-network loss and achieves better average Office benchmark performance than CORAL and six baselines.
Problem
Unsupervised domain adaptation seeks to transfer knowledge from labeled source domains to unlabeled target domains under domain shift.
Method
Deep CORAL adds a differentiable loss that minimizes differences between source and target correlations in deep feature activations.
Results
Deep CORAL achieves better average performance than CORAL and six baselines on the Office benchmark, attaining the highest accuracy on 3 of 6 shifts.
Takeaways & Limitations
Deep CORAL provides end-to-end unsupervised adaptation that integrates with deep networks and can be applied across layers or architectures.
Abstract
from arXiv · showhide
Deep neural networks are able to learn powerful representations from large quantities of labeled input data, however they cannot always generalize well across changes in input distributions. Domain adaptation algorithms have been proposed to compensate for the degradation in performance due to domain shift. In this paper, we address the case when the target domain is unlabeled, requiring unsupervised adaptation. CORAL is a "frustratingly easy" unsupervised domain adaptation method that aligns the second-order statistics of the source and target distributions with a linear transformation. Here, we extend CORAL to learn a nonlinear transformation that aligns correlations of layer activations in deep neural networks (Deep CORAL). Experiments on standard benchmark datasets show state-of-the-art performance.
1 Introduction
The paper addresses domain shift, which persists despite deep CNN features’ partial invariance to low-level cues. It extends CORAL into deep networks with a differentiable correlation-alignment loss and reports state-of-the-art benchmark performance.
- Motivation: Training and test data often violate the i.i.d. assumption because distributions change over time and space.Deep CNN features may be invariant to low-level cues yet remain susceptible to domain shift.
- Method: Deep CORAL incorporates CORAL into deep networks through a differentiable loss minimizing differences between source and target correlations.The method is described as the CORAL loss.
- Contribution: Deep CORAL learns a more powerful nonlinear transformation that works seamlessly with deep CNNs.The method was evaluated on standard benchmark datasets.
- Evaluation: The evaluation shows state-of-the-art performance on standard benchmark datasets.This result is reported for the proposed Deep CORAL approach.
2 Related Work
Prior unsupervised adaptation methods re-weight losses or align source and target subspaces, while recent deep approaches use architecture changes, MMD, or domain-confusion losses. Deep CORAL instead aligns learned feature covariances with a CORAL loss and is designed for simpler, seamless integration across layers and architectures.
- Related Work: Earlier methods re-weight training losses toward the test distribution or transform lower-dimensional manifolds to bring source and target subspaces closer.Re-weighting methods often assume selection bias and may not apply to more general domain shifts.
- Related Work: Adaptive deep methods include DLID’s joint source-target CNN with two adaptation layers, DDC’s single-layer linear-kernel MMD, DAN’s multi-layer multi-kernel MMD, and ReverseGrad’s binary domain classifier.These methods represent distinct architectural and distribution-matching strategies for unsupervised adaptation.
- Related Work: Deep CORAL adds a loss that minimizes cross-domain feature-covariance differences, analogous to MMD with a polynomial kernel.The approach is more powerful than DDC’s sample-mean alignment, simpler to optimize than DAN and ReverseGrad, and seamlessly integrates across layers or architectures.
3 Deep CORAL
Deep CORAL addresses unsupervised domain adaptation by combining generic pretrained deep features with labeled source data while encouraging features that work well on the unlabeled target domain. It aligns source and target activation covariances and balances this adaptation objective against source classification through a weighted joint loss.
- Unsupervised domain adaptation: Deep CORAL targets unlabeled-target adaptation by initializing from a generic pretrained network and fine-tuning with labeled source data.The method also seeks final learned features that work well on the target domain.
- CORAL loss: The CORAL loss measures the distance between source and target feature covariances, aligning their second-order statistics.The features are d-dimensional deep-layer activations of input examples.
- CORAL loss: Deep CORAL computes batch covariances while sharing network parameters between the source and target networks.The covariance-based loss is differentiated with respect to input features using the chain rule.
- Joint training objective: The training objective combines classification and CORAL losses so features remain discriminative for source classification while becoming invariant across domains.Classification loss alone can overfit the source domain, whereas CORAL loss alone can produce degenerate features.
- Joint training objective: λ weights the trade-off between domain adaptation and source classification accuracy, with the two losses expected to reach an equilibrium during training.The objective includes t CORAL loss layers in a deep network.
4 Experiments
Experiments on the Office benchmark evaluate Deep CORAL across all six unlabeled domain shifts and show improved target-domain performance over established baselines. Applying CORAL loss to the final classification layer improves target accuracy while preserving strong source accuracy, with the two training losses becoming comparable after several hundred iterations.
- Dataset and protocol: Deep CORAL is evaluated on the 31-category Office dataset spanning Amazon, DSLR, and Webcam image domains.The benchmark contains three domains and supports six source-to-target shifts.
- Dataset and protocol: Experiments use all labeled source data and unlabeled target data across all 6 domain shifts.Each experiment designates one domain as source and another as target.
- Implementation: CORAL loss is applied to the last classification layer, a general choice because many deep classifiers contain a fully connected classification layer.The paper states that applying CORAL loss to other layers or architectures should be straightforward.
- Results: Deep CORAL achieves better average performance than CORAL and 6 other baselines, ranks highest on 3 out of 6 shifts, and trails the best baseline by at most 0.7 on the remaining shifts.The comparison includes CNN, GFK, SA, TCA, CORAL, DDC, and DAN.
- Results: Adding CORAL loss produces much better target-domain performance while maintaining strong source-domain classification accuracy.This comparison is shown for the A→W shift with and without CORAL loss.
- Results: After a few hundred training iterations, the classification loss and CORAL loss become approximately equal.Initially, CORAL loss is very small and classification loss is very large because the final fully connected layer is randomly initialized with N(0, 0.005).
5 Conclusion
The paper extends CORAL to end-to-end unsupervised domain adaptation in deep neural networks, achieving state-of-the-art performance while integrating seamlessly across layers and architectures.
- 5 Conclusion: Deep CORAL extends CORAL to perform end-to-end adaptation in deep neural networks.It remains an unsupervised domain adaptation method.
- 5 Conclusion: Experiments on standard benchmark datasets show state-of-the-art performance.
- 5 Conclusion: Deep CORAL works seamlessly with deep networks and integrates easily into different layers or network architectures.