Source-linked AI summary
Unsupervised Domain Adaptation using Feature-Whitening and Consensus Loss
Subhankar Roy, Aliaksandr Siarohin, Enver Sangineto, Samuel Rota Bulo, Nicu Sebe, Elisa Ricci
TL;DR
Domain shift makes models trained on one dataset perform poorly on novel domains, motivating unsupervised domain adaptation without annotated target data. The paper combines domain-specific feature whitening with the Min-Entropy Consensus loss to align distributions and exploit unlabeled target samples. Across digit-classification and object-recognition benchmarks, the approach reports significant gains and state-of-the-art performance.
Problem
Domain shift causes predictors trained on one dataset to perform poorly on novel domains, while UDA lacks annotated target data during training.
Method
The framework uses domain-specific whitening of intermediate features and a Min-Entropy Consensus loss that combines entropy and consistency objectives for unlabeled target data.
Results
The approach reports significant performance gains and state-of-the-art results across digit-classification and object-recognition UDA benchmarks.
Takeaways & Limitations
DWT aligns intermediate source and target features, while MEC exploits target data without requiring confidence-threshold hyper-parameters.
Abstract
from arXiv · showhide
A classifier trained on a dataset seldom works on other datasets obtained under different conditions due to domain shift. This problem is commonly addressed by domain adaptation methods. In this work we introduce a novel deep learning framework which unifies different paradigms in unsupervised domain adaptation. Specifically, we propose domain alignment layers which implement feature whitening for the purpose of matching source and target feature distributions. Additionally, we leverage the unlabeled target data by proposing the Min-Entropy Consensus loss, which regularizes training while avoiding the adoption of many user-defined hyper-parameters. We report results on publicly available datasets, considering both digit classification and object recognition tasks. We show that, in most of our experiments, our approach improves upon previous methods, setting new state-of-the-art performances.
1. Introduction
The paper addresses unsupervised domain adaptation by combining intermediate-feature whitening with a Min-Entropy Consensus loss. The resulting framework aligns source and target distributions, exploits unlabeled target data, and achieves strong performance across several benchmarks.
- Domain shift causes predictors trained on one dataset to perform poorly on novel domains, while UDA learns from labeled source data without annotated target samples.Unlabeled target data can still be used during training.
- DWT aligns source and target distributions by whitening intermediate features with domain-specific covariance matrices and projecting them into a common spherical distribution.Unlike BN-based alignment, DWT accounts for inter-feature correlations rather than only feature standardization.
- MEC combines entropy and consistency objectives to encourage coherent predictions for perturbed target samples while using those predictions as pseudo-labels.The loss is designed to avoid confidence-threshold hyper-parameters that are difficult to tune with unlabeled target data.
- The proposed DWT and MEC components are integrated into different network architectures and evaluated on digit and object-recognition UDA benchmarks.The reported benchmarks include MNIST, USPS, SVHN, CIFAR-10, STL10, and Office-Home.
- The approach reports a significant performance boost and state-of-the-art results across the evaluated UDA benchmarks.
2. Related Work
Related UDA methods reduce domain discrepancy through distribution alignment, domain-invariant representations, image translation, entropy minimization, or prediction consistency. The paper situates its approach among these paradigms and highlights limitations of confidence thresholding and standardization-based alignment.
- UDA methods differ mainly in how they reduce discrepancies between source and target feature distributions.The related-work taxonomy includes statistical alignment, domain-invariant representations, GAN-based image transformation, entropy minimization, and consistency enforcement.
- Distribution-alignment methods model domains using first- or second-order statistics, while other approaches embed BN-based alignment layers directly in deep networks.Correlation alignment uses covariance differences, whereas BN-based methods use feature standardization.
- Domain-invariant representation methods use mechanisms such as gradient reversal, domain-confusion losses, or associative embeddings to reduce domain sensitivity.
- GAN-based UDA methods transform target images toward the source domain but have limited capabilities for complex images.
- Entropy minimization uses high-confidence predictions on unlabeled samples as pseudo-labels, while consistency methods enforce agreement between predictions for perturbed target inputs.
- Consistency losses can become uninformative for uniform predictions, motivating confidence thresholding that introduces additional user-defined and dataset-specific hyper-parameters.
3. Method
The method combines domain-specific feature whitening with Min-Entropy Consensus loss to adapt classifiers using labeled source data and unlabeled target data. Whitening aligns intermediate source and target features, while MEC encourages coherent, peaked predictions without confidence thresholds.
- Preliminaries: The framework learns from labeled source and unlabeled target batches to address unsupervised domain adaptation.The source batch supports cross-entropy training, while target batches provide perturbed examples for the unsupervised loss.
- Domain-specific Whitening Transform: Domain-specific Whitening Transform replaces batch standardization with whitening based on domain-specific covariance statistics.The transform is applied to intermediate activations from separate source and target batches.
- Domain-specific Whitening Transform: DWT projects source and target activations into a common spherical distribution by whitening their intermediate features.The whitening statistics are computed separately for source and target batches and used for the corresponding activations.
- Implementation Details: Feature grouping trades full whitening for better-conditioned covariance estimates when feature dimension is large and batch size is small.With group size g = 1, the whitening matrices become diagonal and recover feature standardization.
- Min-Entropy Consensus Loss: Min-Entropy Consensus compares predictions for two perturbed versions of each target sample, selects their maximally agreeing class as a pseudo-label, and avoids confidence thresholds.Unlike consistency losses that can become near zero for uniform posteriors, MEC also encourages peaked predictions.
- Min-Entropy Consensus Loss: The final objective is a weighted sum of the source cross-entropy loss and the target MEC loss.The target batches contain duplicate image pairs differing only through image perturbations, while network weights are shared.
4. Experiments
The experiments evaluate the method across small digit and object benchmarks and the large-scale Office-Home dataset, using ablations and comparisons with state-of-the-art UDA methods. Results show benefits from DWT feature whitening and MEC, including strong performance across several domain-transfer settings.
- Datasets: Experiments cover MNIST↔USPS, MNIST↔SVHN, CIFAR-10↔STL, and the four-domain Office-Home benchmark.Office-Home contains 15,500 images across Art, Clipart, Product, and Real World domains, each with 65 categories.
- Experimental design: The study combines ablation analyses of DWT and MEC with comparisons against previous UDA methods on digits, CIFAR-10↔STL, and Office-Home.The experiments separately assess feature alignment through DWT and target-data exploitation through MEC.
- DWT ablation: Accuracy remains below 90% with group size g = 1, increases monotonically through g = 4, then decreases at larger group size in the SVHN→MNIST ablation.The authors attribute the final decline probably to poorly estimated 8 × 8 covariance matrices, while increasing the number of DWT layers improves accuracy.
- MEC ablation: MEC matches Self-Ensembling in MNIST↔USPS and achieves higher accuracy in SVHN→MNIST while avoiding confidence-threshold hyperparameters.Self-Ensembling performs well with confidence threshold 0.936 but degrades when the threshold is 0, especially for SVHN→MNIST.
- State-of-the-art comparisons: DWT outperforms previous non-augmentation UDA methods in the three digit settings, although it is less effective for MNIST→SVHN under the stronger domain shift.GAN-based methods are more effective in MNIST→SVHN, while DWT-MEC is better than the reimplemented Self-Ensembling method in SVHN→MNIST.
5. Conclusions
The paper addresses unsupervised domain adaptation with domain-specific feature whitening and Min-Entropy Consensus loss, achieving state-of-the-art performance on digit and object-recognition benchmarks.
- Domain-specific whitening aligns source and target distributions at intermediate feature levels and increases loss-landscape smoothness.
- The Min-Entropy Consensus loss better exploits unlabeled target data during training.
- Both components integrate readily into standard convolutional neural networks.
- Experiments show state-of-the-art performance on digit categorization and object-recognition tasks.