Source-linked AI summary
Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation
Muhammad Ghifary, W. Bastiaan Kleijn, Mengjie Zhang, David Balduzzi, Wen Li
TL;DR
Domain adaptation must recognize target-domain objects without labeled target examples, despite source-target distribution differences. DRCN shares an encoder between supervised source classification and unsupervised target reconstruction, and it improves performance across cross-domain recognition tasks while producing target-like reconstructions.
Problem
Domain adaptation seeks robust recognition when labeled source data and target data differ, but target labels are unavailable and real-world accuracy remains limited.
Method
DRCN jointly learns supervised source-label prediction and unsupervised target-data reconstruction through a shared encoding representation.
Results
Across varied cross-domain object recognition tasks, DRCN outperforms the state-of-the-art domain adaptation algorithm by up to ∼8% accuracy.
Takeaways & Limitations
Reconstructed source images resemble target images, indicating that DRCN learns domain correspondence while combining target structure with source classification.
Takeaways & Limitations
The comparison between DRCN variants does not establish that adding unlabeled source data degrades performance because model capacity may explain the difference.
Abstract
from arXiv · showhide
In this paper, we propose a novel unsupervised domain adaptation algorithm based on deep learning for visual object recognition. Specifically, we design a new model called Deep Reconstruction-Classification Network (DRCN), which jointly learns a shared encoding representation for two tasks: i) supervised classification of labeled source data, and ii) unsupervised reconstruction of unlabeled target data.In this way, the learnt representation not only preserves discriminability, but also encodes useful information from the target domain. Our new DRCN model can be optimized by using backpropagation similarly as the standard neural networks. We evaluate the performance of DRCN on a series of cross-domain object recognition tasks, where DRCN provides a considerable improvement (up to ~8% in accuracy) over the prior state-of-the-art algorithms. Interestingly, we also observe that the reconstruction pipeline of DRCN transforms images from the source domain into images whose appearance resembles the target dataset. This suggests that DRCN's performance is due to constructing a single composite representation that encodes information about both the structure of target images and the classification of source images. Finally, we provide a formal analysis to justify the algorithm's objective in domain adaptation context.
1 Introduction
Unsupervised domain adaptation addresses dataset bias when labeled source data and unlabeled target data differ. DRCN jointly learns source classification and target reconstruction through a shared representation, improving cross-domain recognition.
- Dataset bias arises when labeled training instances come from a source domain but test instances come from a related, different target domain.
- Unlabeled target data can reduce manual labeling needs and provide auxiliary training information for generalization on the target domain.
- DRCN learns a representation by supervised source-label learning and unsupervised target-data learning as related tasks in a multitask framework.
- DRCN is a deep model for unsupervised domain adaptation designed to provide scalable feature learning for visual recognition.
- Up to ∼8% accuracy gap separates DRCN from the state-of-the-art domain adaptation algorithm across cross-domain object recognition tasks.
- Reconstructed source images resemble target-domain appearances, while probabilistic analysis relates the objective to semi-supervised learning and target-only reconstruction.
2 Related Work
Related work frames unsupervised domain adaptation as a broad dataset-bias problem spanning several fields and feature-learning strategies. Prior deep methods use pretraining, domain-discrepancy constraints, or domain classification, whereas DRCN learns target-image structure directly.
- Domain adaptation is also discussed as class imbalance, covariate shift, sample selection bias, transfer learning, and dataset bias.
- Unsupervised domain adaptation assumes target labels are unavailable, and many object-recognition approaches were designed for small datasets and SURF-based features.
- Deep adaptation methods have used denoising-autoencoder features and pretrained convolutional networks for domain-adaptive visual recognition.
- DDC and DAN fine-tune pretrained networks using maximum mean discrepancy, with DAN adapting more layers and outperforming DDC.
- ReverseGrad treats domain invariance as binary classification by minimizing label loss while maximizing domain-classification loss through gradient reversal.
- Unlike ReverseGrad, DRCN seeks a good label classifier while simultaneously learning the structure of target images.
3 Deep Reconstruction-Classification Networks
DRCN is a convolutional multitask architecture that shares an encoder between source-label prediction and target-data reconstruction. Its objective combines classification and reconstruction losses, optimized by alternating stochastic-gradient updates.
- DRCN addresses unsupervised domain adaptation by learning from labeled source data and unlabeled target data.
- The model seeks a feature mapping that reduces source-target discrepancy while supporting target-domain labeling.
- DRCN jointly learns source label prediction and target data reconstruction through two convolutional pipelines with a shared encoding representation.
- The label pipeline maps inputs through an encoder and feature-labeling function, while the reconstruction pipeline maps them through the encoder and decoder.
- The shared encoder parameters support both pipelines, while labeling and decoding parameters remain separate.
- The learning objective is a convex combination of supervised classification and unsupervised reconstruction losses controlled by λ.
- Training alternates source classification and target reconstruction updates using stochastic gradient descent, with RMSprop and dropout used in implementation.
- Data augmentation and denoising generate transformed or noisy training inputs to improve generalization and feature invariance.
4 Experiments and Results
DRCN is evaluated across cross-domain recognition benchmarks and the Office dataset, using shared experimental architectures and standard comparisons. It generally outperforms competing adaptation and pretraining methods, while reconstruction analyses support target-domain adaptation.
- Evaluation scope: Experiments cover five cross-domain benchmarks and three dataset pairs, including MNIST–USPS, SVHN–MNIST, and CIFAR–STL.The evaluation includes both large-scale deep-learning benchmarks and the Office dataset.
- Architecture and learning setup: DRCN uses a shared convolutional encoder with separate classification and reconstruction pipelines, trained with RMSprop and a validation-selected control penalty.The encoder comprises convolutional, pooling, and fully connected layers; the decoder reverses the encoder configuration.
- Cross-domain recognition: DRCN performs best on all but one cross-domain task and exceeds ReverseGrad by approximately 8% on sv →mn.On mn →sv, it improves over ReverseGrad by approximately 5%, although the gap to the target-only ground truth remains large.
- Comparison with pretraining: DRCN convincingly outperforms SDAsh, SCAE, and SCAEt, supporting simultaneous reconstruction-classification training over greedy-layer pretraining.The comparison targets pretraining-finetuning strategies in unsupervised domain adaptation.
- Reconstruction-data ablation: Using only unlabeled target images for reconstruction outperforms variants using source images alone or source and target images together.DRCN consistently beats DRCNs and DRCNst, with the variants closely matching it only on mn→us.
- Visual reconstruction analysis: For sv →mn, reconstructed SVHN images resemble MNIST digits while preserving selected digit content, unlike reconstructions from comparison settings.The reconstruction selects the middle digit and ignores remaining digits; this cross-reconstruction appearance does not occur for mn →sv.
- Office dataset: On the Office dataset, DRCN is usually the best or second-best method and performs best when the target dataset contains relatively many images.These experiments follow the standard protocol using labeled source data and unlabeled target data.
5 Analysis
The analysis gives a probabilistic interpretation of DRCN and relates its objective to semi-supervised learning on the target domain. It argues that target-only reconstruction training can be sufficient, while noting assumptions and architectural qualifications.
- The analysis relates optimizing DRCN’s objective to a semi-supervised learning problem on the target domain.
- Target-only unsupervised reconstruction training is argued to be sufficient, so adding unlabeled source data may not further improve adaptation.
- DRCN’s learning algorithm can be interpreted probabilistically by modeling inputs as Gaussian and labels conditionally with a multinomial logistic-regression model.
- The objective combines supervised source label prediction with unsupervised reconstruction of noisy target inputs.
- Under covariate shift and related conditions, the DRCN estimator can approximately correspond to the semi-supervised objective.
- The analysis does not imply that unlabeled source data degrades performance, because differences may arise from model capacity and architecture choice.
6 Conclusions
The paper concludes that DRCN performs multitask learning with shared encoding for source label prediction and target reconstruction. Its results, visualizations, and analysis support domain correspondence and the target-only reconstruction strategy.
- DRCN alternates source label prediction and target data reconstruction through a shared encoding representation.
- DRCN improves performance on some cross-domain recognition tasks over the state-of-the-art model and standard pretraining-finetuning.
- Reconstructed source images resemble target images, indicating that DRCN learns domain correspondence.
- The theoretical analysis relates DRCN to semi-supervised learning and supports using only unlabeled target data for reconstruction.
Supplemental Material
The supplemental material contains additional experimental results, including source-image reconstructions for MNIST→USPS and USPS→MNIST.
- The supplemental document provides experimental results omitted from the main manuscript because of space limitations.
- Figure 3 presents original input pixels and reconstructed MNIST source images for MNIST→USPS.
- Figure 4 presents original input pixels and reconstructed USPS source images for USPS→MNIST.
Data Reconstruction
Across MNIST→USPS, USPS→MNIST, and SVHN→MNIST, DRCN’s reconstructed source images resemble the target-domain style.
- DRCN reconstructs source images in a style resembling the target images across the reported cross-domain settings.The supplemental material reports this trend for MNIST→USPS and USPS→MNIST, consistent with the SVHN→MNIST result.
Training Progress
During SVHN →MNIST training, DRCN trades lower source accuracy for higher target accuracy than ConvNet, indicating reduced source-domain overfitting.
- Training Progress: DRCN produces higher target accuracy but lower source accuracy than ConvNet on the SVHN →MNIST task.The comparison is made during training and is described as the clearest indication of reduced overfitting.
t-SNE visualization.
The t-SNE visualization compares last-layer feature clouds from source and target domains, using their overlap to assess domain invariance.
- t-SNE visualization.: Figure 6 visualizes DRCN’s last hidden-layer activations with t-SNE for MNIST →USPS and SVHN →MNIST.Red points represent source features and gray points represent target features.
- t-SNE visualization.: Greater overlap between source and target feature clouds indicates stronger domain invariance.The visualization compares DRCN with the standard ConvNet.