Source-linked AI summary
Deep Domain Confusion: Maximizing for Domain Invariance
Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, Trevor Darrell
TL;DR
Dataset bias and limited labeled target data make adapting deep visual representations across domains difficult. The paper introduces a CNN adaptation layer with an MMD-based domain confusion loss and uses domain confusion to select architecture choices. The method achieves state-of-the-art performance on the standard visual domain adaptation benchmark, including 96% target accuracy for minor pose, resolution, and lighting changes.
Problem
Dataset bias and problematic fine-tuning with small labeled target datasets limit adaptation of deep representations to new visual domains.
Method
The method jointly optimizes classification and domain invariance using an adaptation layer, an MMD-based domain confusion loss, and domain-confusion-guided architecture selection.
Results
The full method achieves state-of-the-art performance on the standard visual domain adaptation benchmark and reaches 96% target-domain accuracy for minor pose, resolution, and lighting changes.
Takeaways & Limitations
Incorporating domain confusion into discriminative representation learning effectively produces representations that are useful for classification and invariant to domain shifts.
Takeaways & Limitations
Prior deep-feature methods had no way to select a representation from the architecture, while fixed-representation optimization could not directly adapt the representation.
Abstract
from arXiv · showhide
Recent reports suggest that a generic supervised deep CNN model trained on a large-scale dataset reduces, but does not remove, dataset bias on a standard benchmark. Fine-tuning deep models in a new domain can require a significant amount of data, which for many applications is simply not available. We propose a new CNN architecture which introduces an adaptation layer and an additional domain confusion loss, to learn a representation that is both semantically meaningful and domain invariant. We additionally show that a domain confusion metric can be used for model selection to determine the dimension of an adaptation layer and the best position for the layer in the CNN architecture. Our proposed adaptation method offers empirical performance which exceeds previously published results on a standard benchmark visual domain adaptation task.
UC Berkeley, EECS & ICSI
The authors are affiliated with UC Berkeley, EECS & ICSI.
- Trevor Darrell is affiliated with UC Berkeley, EECS & ICSI.
- The paper lists UC Berkeley, EECS & ICSI as an affiliation.
- The listed contact addresses use the eecs.berkeley.edu domain.
1. Introduction
The paper addresses dataset bias and limited target-domain data by learning deep representations that jointly support classification and domain invariance. Its CNN uses an adaptation layer and domain confusion, achieving strong benchmark performance across supervised and unsupervised adaptation settings.
- Dataset bias causes source-trained classifiers to transfer imperfectly across differing target distributions, while existing adaptation methods were largely shallow.Supervised fine-tuning on small labeled target sets is also problematic.
- The objective combines classification error with a domain confusion loss that makes source and target representations more similar.
- The proposed CNN adds an adaptation layer and MMD-based domain confusion loss to jointly optimize classification and domain invariance.The metric also selects adaptation-layer dimensions and placement within a pretrained CNN.
- The architecture supports supervised adaptation with limited target labels and unsupervised adaptation without labeled target training data.
- 96% accuracy is achieved on the target domain for minor pose, resolution, and lighting changes.The result is reported on the Office benchmark and is presented as evidence of invariance to these biases.
2. Related work
Related work addresses visual domain adaptation through feature transformations, invariant representations, and deep visual features, but prior approaches face limitations in depth or unsupervised applicability.
- Visual domain adaptation methods commonly align source and target representations through feature-space transformations or classifier regularization.
- Deep representations reduce the effects of resolution and lighting shifts, strengthening their usefulness for visual domain adaptation.
- Source-trained classifiers may fail on target domains when dataset bias causes distributions to differ.
- Siamese networks learn invariant representations but require labels for every training instance, limiting clear extension to unsupervised settings.
- Multimodal invariant-representation methods primarily use generative architectures rather than the full representational capacity of supervised CNNs.
- Earlier joint source-target CNN training was limited to two layers and was outperformed by deeper models pretrained on large auxiliary datasets.
- A shallow denoising-autoencoder approach learned domain invariance but lacked the strong semantic representation associated with supervised deep CNN classification objectives.
3. Training CNN-based domain invariant representations
The paper introduces a CNN architecture that jointly learns semantically meaningful, domain-invariant representations by combining classification with domain confusion. An adaptation layer and MMD guide representation selection and are trained using source and target data.
- The proposed CNN adds an adaptation layer and MMD-based domain confusion loss to jointly optimize classification and domain invariance.The architecture is designed to learn a representation with both strong semantic separation and reduced domain discrepancy.
- Direct source-only training can overfit the source distribution, reducing target-domain performance when the source and target distributions differ.The method addresses this setting by minimizing source-target distance while training a classifier on labeled source data.
- The joint loss combines classification loss on labeled data with λMMD^2(XS, XT), where λ controls the strength of domain confusion.Classification promotes class discrimination, while MMD encourages source and target representations to become similar.
- MMD selects both the adaptation layer’s network position and its dimension by minimizing source-target distance.Layer placement is selected from a fixed representation, while width is chosen through a grid search over fine-tuned networks.
- The architecture uses shared-weight source and target CNNs, applying classification loss to labeled examples and domain confusion loss to all source and target data.A fork after the adaptation layer separates the classifier branch from the MMD computation branch.
- MMD and target accuracy seem inversely related across representation layers, supporting MMD as a criterion for selecting an adaptation layer.The figure compares maximum mean discrepancy between source and target with target-domain test accuracy.
4. Evaluation
The evaluation tests adaptation on the Office benchmark across supervised and unsupervised transfer, using MMD to select representations and adaptation-layer dimensionality. MMD-regularized fine-tuning improves transfer performance, limits source-domain overfitting, and produces more domain-invariant embeddings.
- Evaluation setup: The method is evaluated on the Office dataset across Amazon, DSLR, and Webcam domains, using five random splits for each of three transfer tasks.Both supervised and unsupervised scenarios are compared with six recently published methods.
- Representation selection: MMD ranks pretrained representations by domain invariance, selecting fc7 as the best bridge layer and fc6 as the worst for Amazon→Webcam.The adaptation layer is placed after fc7 in the remaining experiments.
- Adaptation-layer dimension: The dimensionality search trains adaptation layers from 64 to 4096 dimensions and selects the width minimizing source-target MMD.For the reported experiment, MMD selects 256 dimensions; this avoids the extreme widths where performance suffers, although it is not the test-accuracy maximum.
- Fine-tuning protocol: The supervised and unsupervised evaluations use the same fine-tuning architecture, but only supervised adaptation supplies labeled target examples to the classifier.The MMD regularizer uses all source and target data in both settings because it does not require labels.
- Regularization and representations: Regularization slows initial learning but prevents source-data overfitting and yields higher final test accuracy than unregularized fine-tuning.The learned representation also forms tighter class clusters while mixing Amazon and Webcam examples within clusters, unlike the pretrained fc7 embedding.
- Benchmark results: MMD-regularized fine-tuning dramatically outperforms competing methods in both supervised and unsupervised settings on the Office benchmark.The paper reports a 3.4% multiclass-accuracy improvement over DeCAF for supervised Amazon→Webcam transfer and a 5.5% improvement for the corresponding unsupervised task.
5. Conclusion
The paper develops domain-invariant representations by combining classification with domain confusion, and provides optimization strategies ranging from representation selection to end-to-end CNN training. Its full MMD-based method achieves state-of-the-art benchmark performance and shows that domain confusion supports both classification utility and invariance to domain shifts.
- The proposed objective learns domain-invariant representations for classification by adding a domain confusion term.
- The full method uses MMD to select architecture depth and width and as a regularizer during fine-tuning.
- The full method achieves state-of-the-art performance on the standard visual domain adaptation benchmark, beating previous methods by a considerable margin.
- The experiments indicate that domain confusion can produce representations useful for classification while invariant to domain shifts.