Source-linked AI summary
Multi-Representation Adaptation Network for Cross-domain Image Classification
Yongchun Zhu, Fuzhen Zhuang, Jindong Wang, Jingwu Chen, Zhiping Shi, Wenjuan Wu, Qing He
TL;DR
Limited labels make cross-domain image classification difficult, motivating adaptation from labeled source data to less-labeled target domains. MRAN addresses this gap by aligning multiple representations extracted with IAM and extending MMD for adaptation. Across three benchmark datasets, it reports strong performance against state-of-the-art competitors.
Problem
Obtaining sufficient manually labeled data for every target domain is difficult, while single-representation adaptation may align only partial image information.
Method
MRAN uses IAM to extract and align multiple representations across domains, and extends MMD to measure conditional-distribution discrepancy.
Results
MRAN achieves remarkable performance compared with state-of-the-art competitors on three benchmark datasets.
Takeaways & Limitations
MRAN learns multiple domain-invariant representations and can be extended to most feed-forward models with efficient back-propagation training.
Takeaways & Limitations
The formulation assumes source and target domains are sampled from different probability distributions, P ≠ Q, with unlabeled target examples.
Abstract
from arXiv · showhide
In image classification, it is often expensive and time-consuming to acquire sufficient labels. To solve this problem, domain adaptation often provides an attractive option given a large amount of labeled data from a similar nature but different domain. Existing approaches mainly align the distributions of representations extracted by a single structure and the representations may only contain partial information, e.g., only contain part of the saturation, brightness, and hue information. Along this line, we propose Multi-Representation Adaptation which can dramatically improve the classification accuracy for cross-domain image classification and specially aims to align the distributions of multiple representations extracted by a hybrid structure named Inception Adaptation Module (IAM). Based on this, we present Multi-Representation Adaptation Network (MRAN) to accomplish the cross-domain image classification task via multi-representation alignment which can capture the information from different aspects. In addition, we extend Maximum Mean Discrepancy (MMD) to compute the adaptation loss. Our approach can be easily implemented by extending most feed-forward models with IAM, and the network can be trained efficiently via back-propagation. Experiments conducted on three benchmark image datasets demonstrate the effectiveness of MRAN. The code has been available at https://github.com/easezyc/deep-transfer-learning.
1 Introduction
Image classification needs substantial labeled data, but collecting labels for every new domain is often impractical. The paper addresses this domain-adaptation challenge by aligning multiple representations rather than relying on a single representation.
- Motivation: Domain adaptation transfers feature representations from a labeled source domain to a target domain with limited or no labels.It is motivated by the difficulty and expense of obtaining enough manually labeled data for every new scenario.
- Prior approaches: Deep domain adaptation methods typically extract domain-invariant representations by optimizing measures of domain shift.Examples include maximum mean discrepancy, correlation distances, and adversarial domain-discrepancy objectives.
- Problem: CNN-based feature extraction may lose important information, leaving representations with only partial saturation, brightness, or hue information.The paper illustrates this issue by comparing an original image with transformed forms that retain different partial information.
- Problem: Single-representation adaptation aligns source and target distributions using only the partial information captured by one structure.The paper therefore argues that multiple representations should be considered to obtain a more comprehensive understanding of objects.
- Proposed approach: MRAN aligns multiple domain-specific representations across domains using an Inception Adaptation Module and an extended conditional MMD adaptation loss.IAM extracts multiple representations, while the extended MMD measures discrepancy between conditional distributions in deep networks.
- Evaluation: Experiments on three benchmark datasets report that MRAN achieves remarkable performance compared with state-of-the-art competitors.The paper also presents IAM as compatible with most feed-forward models and trainable by standard back-propagation.
2 Related Work
Related work spans image classification, domain adaptation, and multi-view learning. The paper distinguishes its multi-representation approach from methods that use multiple predefined feature views.
- Image Classification: Image classification research includes parametric and non-parametric classifiers, while deep neural networks have recently achieved strong performance.Examples discussed include maximum likelihood, linear discriminant analysis, attention-based scene classification, and low-rank representation methods.
- Domain Adaptation: Earlier shallow domain-adaptation methods reweight training data or transform lower-dimensional manifolds to reduce source-target differences.These approaches address the visual dataset-bias problem through data or subspace adjustments.
- Domain Adaptation: Deep networks are used for domain adaptation because they learn abstract representations and transferable factors underlying different populations.The related work reports significant performance gains from applying deep neural networks to domain adaptation.
- Domain Adaptation: Deep transfer networks mainly minimize differences between source and target distributions during feature learning.They are broadly divided into embedding methods, which add adaptation layers, and adversarial methods, which use a domain discriminator.
- Multi-view Learning: Multi-view learning uses data represented by multiple distinct feature sets or views.This setting differs from multi-representation learning, which extracts multiple representations from a single data view using a hybrid structure.
3 Multi-Representation Adaptation Networks
MRAN performs unsupervised domain adaptation by extracting and aligning multiple representations with IAM, using CMMD to reduce class-conditional discrepancies between source and target domains. It combines this adaptation objective with classification training and can extend feed-forward models by replacing global average pooling.
- Inception Adaptation Module: IAM replaces global average pooling with multiple substructures that extract distinct representations from low-pixel images.The resulting representations are concatenated before classification.
- Classification: The classifier receives concatenated representations, recombines them through a fully connected layer, and outputs predicted labels through a softmax layer.The network is expressed as y = s([(h1 ◦g)(X); . . . ; (hnr ◦g)(X)]).
- Implementation: IAM can be implemented in most feed-forward models by replacing the final average pooling layer.The paper describes this as an implementation path for extending existing networks with multi-representation adaptation.
- Conditional Maximum Mean Discrepancy: CMMD extends MMD to measure discrepancies between class-conditional source and target distributions using source labels and target pseudo-labels.Minimizing CMMD draws the conditional distributions closer, with iterative improvement of target labeling quality expected during optimization.
- MRAN: MRAN aligns multiple domain-specific representations extracted by IAM across source and target domains in an end-to-end model.The network uses multi-representation alignment for unsupervised domain adaptation.
- Training objective: MRAN optimizes classification loss together with CMMD-based adaptation loss, weighted by a positive trade-off parameter λ.The implementation based on ResNet replaces global average pooling by IAM and adapts task-specific layers by minimizing both losses.
4 Experiments
Experiments evaluate MRAN on ImageCLEF-DA, Office-31, and Office-Home against established domain-adaptation baselines and variants. Results and analyses indicate that conditional multi-representation alignment improves transfer performance, produces clearer target separation, and adds little iteration time.
- Experimental Setup: MRAN is evaluated on three benchmark datasets using all six transfer tasks where specified, against classical and deep domain-adaptation baselines.The benchmarks are ImageCLEF-DA, Office-31, and Office-Home; Office-31 uses six domain-transfer directions, while ImageCLEF-DA uses six combinations.
- Main Results: MRAN (CMMD+IAM) outperforms comparison methods on most transfer tasks and substantially improves accuracy on ImageCLEF-DA.The authors attribute the result to incorporating CMMD and IAM for learning better transferable representations.
- Ablation Comparisons: MRAN (CMMD) performs better than DAN, while MRAN (CMMD+IAM) substantially outperforms MRAN (CMMD), supporting conditional and multi-representation alignment.The comparison isolates conditional-distribution alignment from marginal alignment, and multiple representations from a single representation.
- Feature Visualization: Combined MRAN representations discriminate target categories more clearly than DAN representations, while individual IAM substructures yield different distributions and clustering errors.The t-SNE analysis indicates that different neural structures extract different information and that the combined representation improves separation.
- Distribution Analysis: The combined representations have smaller proxy A-distance than CNN, DAN, and individual MRAN representations on A →W and W →D.Proxy A-distance is used with source risk to bound target risk in the reported domain-adaptation analysis.
- Sensitivity and Efficiency: Accuracy varies bell-shaped with λ, with about 0.5 providing a proper trade-off for transfer performance; IAM and CMMD add only 0.025s and 0.014s per iteration, respectively.The reported overhead is described as small or reasonable despite the added adaptation components.
5 Conclusion
The paper proposes MRAN to align multiple representations across source and target domains, using IAM and conditional MMD to improve transferability.
- MRAN aligns distributions of multiple representations to learn domain-invariant features containing more image information.
- IAM is a hybrid neural structure that extracts multiple representations from images and can be adapted to different networks.
- The framework extends marginal MMD to conditional MMD and incorporates it into the adaptation model.
- Experiments on three datasets demonstrate the effectiveness of the proposed model.