Source-linked AI summary
Domain Generalization for Object Recognition with Multi-task Autoencoders
Muhammad Ghifary, W. Bastiaan Kleijn, Mengjie Zhang, David Balduzzi
TL;DR
Domain generalization seeks to transfer knowledge from multiple labeled source domains to unseen target domains despite dataset bias. The paper introduces MTAE, which reconstructs analogous views across domains to learn robust features, and reports state-of-the-art performance across cross-domain recognition tasks. Its applicability is constrained by data requirements and limited shared classes in the evaluated datasets.
Problem
Domain generalization seeks object-recognition features that transfer from multiple source domains to unseen target domains despite dataset bias.
Method
MTAE treats natural inter-domain variability as corruption and jointly reconstructs analogs across source domains to learn invariant features for classification.
Results
MTAE and denoising MTAE achieve state-of-the-art performance across cross-domain object-recognition evaluations on unseen target domains.
Takeaways & Limitations
Naturally occurring transformations can serve as a training signal for learning domain-invariant object-recognition features.
Takeaways & Limitations
MTAE requires category-level corresponding instances across domains, while VLCS evaluation shares only five classes and averages below 70% accuracy for the best model.
Abstract
from arXiv · showhide
The problem of domain generalization is to take knowledge acquired from a number of related domains where training data is available, and to then successfully apply it to previously unseen domains. We propose a new feature learning algorithm, Multi-Task Autoencoder (MTAE), that provides good generalization performance for cross-domain object recognition. Our algorithm extends the standard denoising autoencoder framework by substituting artificially induced corruption with naturally occurring inter-domain variability in the appearance of objects. Instead of reconstructing images from noisy versions, MTAE learns to transform the original image into analogs in multiple related domains. It thereby learns features that are robust to variations across domains. The learnt features are then used as inputs to a classifier. We evaluated the performance of the algorithm on benchmark image recognition datasets, where the task is to learn features from multiple datasets and to then predict the image label from unseen datasets. We found that (denoising) MTAE outperforms alternative autoencoder-based models as well as the current state-of-the-art algorithms for domain generalization.
1. Introduction
Domain generalization addresses poor transfer across dataset domains caused by biases such as background, viewpoint, and illumination. The paper introduces Multi-Task Autoencoder (MTAE), which uses naturally occurring inter-domain transformations to learn features robust across domains.
- Dataset bias from backgrounds, camera viewpoints, and illumination often causes poor cross-dataset generalization.
- Domain generalization learns from multiple source domains and tests on previously unseen target domains.
- MTAE learns unbiased object features through a multi-task reconstruction strategy.
- Denoising MTAE learns representations robust to real-world image variability and outperforms prior state-of-the-art domain-generalization methods.
- MTAE replaces artificial corruption with transformations such as viewpoint rotation, object-size dilation, and lighting shifts.
2. Related work
Related work on domain generalization includes bias-aware classifiers, metric learning, and regularized exemplar-SVMs. Unlike domain adaptation, domain generalization cannot use target-domain information during training, and feature learning had not previously been applied to this setting according to the paper.
- Undo-Bias encodes dataset-specific biases in feature space and encourages dataset-specific weights to resemble global weights.
- Unbiased Metric Learning learns a less biased distance metric from weakly labeled web images for object recognition.
- LRE-SVM extends exemplar-SVM with a nuclear-norm regularizer that captures likelihoods across positive samples.
- Domain adaptation differs because unlabeled target samples are available during training, whereas domain generalization has no target-domain information.
- The paper identifies no prior feature-learning work for the more difficult domain-generalization problem.
3. The Proposed Approach
MTAE extends autoencoders into a multi-domain reconstruction model with shared input-hidden weights and domain-specific output weights. It learns robust features by reconstructing related-domain views, optionally with denoising, while addressing unequal category counts through random selection.
- Feature use: The learned encoder features can be fixed as inputs to supervised algorithms or used to initialize a neural network for fine-tuning.This connects unsupervised feature learning to the subsequent discriminative modeling stage.
- Multi-task Autoencoder architecture: MTAE uses multiple output layers, one per domain, with shared input-hidden parameters and domain-specific hidden-output parameters.Its architecture is an autoencoder whose outputs correspond to domains rather than class labels.
- Training strategy: The training strategy treats naturally occurring domain transformations as corruption, learning invariance to changes such as viewpoint, scale, and lighting.Unlike standard denoising autoencoders, MTAE uses inter-domain variability rather than only artificially added noise.
- Reconstruction tasks: MTAE performs M × M reconstruction tasks across M source domains, including self-domain and between-domain reconstructions.Self-domain reconstruction matches standard autoencoder reconstruction, while between-domain tasks transform examples across domains.
- Denoising extension: D-MTAE adds a denoising criterion by replacing each input with a corrupted counterpart during MTAE training.The resulting model is named the Denoising Multi-task Autoencoder.
- Handling unbalanced samples: MTAE requires category-level correspondences across domains, while RAND-SEL balances unequal category counts for between-domain reconstruction.The procedure preserves category-level correspondence while selecting balanced samples.
4. Experiments and Results
Experiments evaluate MTAE and D-MTAE on synthetic transformed views and real-world cross-domain recognition benchmarks. Across these settings, D-MTAE generally performs best, while MTAE consistently outperforms single-task autoencoder alternatives.
- MNIST results: D-MTAE performs best on average and in 9 of 11 individual MNIST-r and MNIST-s cross-domain cases, while MTAE ranks second on average.The results indicate that multi-task feature learning improves discriminative features for unseen object views, and denoising further benefits domain generalization.
- ETH-80 results: D-MTAE and MTAE are the best and second-best models on ETH80-p and ETH80-y, with average accuracies of 87.85% and 87.50% on ETH80-p and 97% and 96.50% on ETH80-y.These results show the same ranking across both pose-based ETH-80 benchmarks.
- Qualitative analysis: MTAE and D-MTAE learn filters capturing rotation across MNIST-r views, whereas AE and DAE filters primarily describe digit contents.D-MTAE filters additionally describe object contents more distinctively, combining transformation and content properties.
- VLCS results: D-MTAE outperforms other algorithms on three of four VLCS cross-domain cases and on average, while MTAE ranks second on average.All algorithms reduce dataset bias after learning from multiple source domains, and Caltech-101 is the easiest dataset to recognize.
- Office+Caltech results: D-MTAE+1HNN performs best in two of four Office+Caltech cross-domain cases and ranks second in the remaining cases, exceeding prior state-of-the-art LRE-SVM on average.This extends the strongest results to another real-world benchmark.
5. Conclusions
The paper concludes that MTAE learns domain-invariant features through multi-task reconstruction across domains, achieving state-of-the-art performance on unseen target domains. However, dataset bias remains unresolved, and further work is needed for unbalanced samples and deeper architectures.
- MTAE and Denoising MTAE reconstruct cross-domain analogs to learn shared, domain-invariant features for object recognition.
- State-of-the-art performance was achieved when predicting object labels from unseen target domains across comprehensive cross-domain evaluations.
- Further study should investigate whether stacking MTAEs improves performance and develop better procedures for unbalanced samples.
- On VLCS, the best model achieved average accuracies below 70%, with only five classes shared across datasets.
- The authors identify larger datasets as necessary for further progress in domain generalization.