Source-linked AI summary
Efficient Learning of Domain-invariant Image Representations
Judy Hoffman, Erik Rodner, Jeff Donahue, Trevor Darrell, Kate Saenko
TL;DR
Domain mismatch makes image representations and linear classifiers less reliable when feature distributions change across domains. MMDT jointly learns a target-to-source transformation and classifier parameters using classification loss, combining multi-class adaptation, heterogeneous feature handling, and scalability. Experiments show higher accuracy and computational advantages over previous methods, including generalization to novel target categories; the linear formulation leaves kernelized nonlinear transforms for future work.
Problem
Changing image feature distributions across domains can substantially degrade classifiers, motivating domain-invariant representations for linear vision models.
Method
MMDT jointly learns an asymmetric target-to-source feature transform and classifier parameters using the same classification loss across available target-labeled categories.
Results
MMDT outperforms previous domain-adaptation methods in classification accuracy, generalizes to novel target categories, and provides a scalable framework for large datasets.
Takeaways & Limitations
A shared category-independent transform enables adaptation to categories without target labels and to domains with nonequivalent feature dimensionalities.
Takeaways & Limitations
The paper focuses on linear transforms for speed and scalability, leaving kernelized nonlinear transforms for future work.
Abstract
from arXiv · showhide
We present an algorithm that learns representations which explicitly compensate for domain mismatch and which can be efficiently realized as linear classifiers. Specifically, we form a linear transformation that maps features from the target (test) domain to the source (training) domain as part of training the classifier. We optimize both the transformation and classifier parameters jointly, and introduce an efficient cost function based on misclassification loss. Our method combines several features previously unavailable in a single algorithm: multi-class adaptation through representation learning, ability to map across heterogeneous feature spaces, and scalability to large datasets. We present experiments on several image datasets that demonstrate improved accuracy and computational advantages compared to previous approaches.
1 Introduction
MMDT learns a shared target-to-source feature transformation jointly with linear classifiers, addressing domain mismatch while supporting multi-class adaptation, heterogeneous features, and scalable optimization. Experiments report higher multi-class accuracy and computational advantages over previous approaches.
- Motivation: Domain shifts caused by sensors, noise, pose, video source, and collection bias can degrade image-classifier performance.The paper motivates adaptive representations for efficient linear classifiers in vision applications.
- Method: MMDT jointly learns an asymmetric transform W and classifier parameters using classification loss across categories with available target labels.The transform maps target features x to Wx, aligning them with the source domain.
- Results: MMDT outperforms competing methods in multi-class accuracy, including previous kernelized methods, while operating in linear feature space.The contribution statement reports this result without specifying a numerical accuracy value.
- Capabilities: MMDT adapts unlabeled categories and heterogeneous source and target feature spaces through a shared category-independent representation.The same learned shift can transfer from labeled categories to categories without target labels.
- Efficiency: The optimization scales to large datasets because its constraints grow linearly with training points and it can be optimized in linear feature space.The final iterative solution can also use standard quadratic-programming packages.
2 Related Work
Domain adaptation addresses changing feature distributions while labels remain fixed, whereas multi-task learning changes labels with a shared feature distribution. Prior vision methods include parameter adaptation, feature transforms, common latent spaces, and geodesic-flow kernels.
- Definitions: Domain adaptation varies p(X) across domains while keeping output labels Y the same; multi-task learning instead varies labels while retaining one feature distribution.The paper frames its setting as multi-task learning across domains, where both p(X) and Y can change.
- Prior Methods: Vision adaptation methods include combined source-target SVMs, transductive SVMs, feature replication, Adaptive SVM, PMT-SVM, and Domain Transfer SVM.These approaches adapt classifiers or features in different ways for image domain adaptation.
- Transform-Based Adaptation: Transform-based methods learn feature-space perturbations, including ARC-t similarity alignment, HFA common latent spaces, and GFK geodesic-flow kernels.These methods differ from class-specific parameter perturbations by transforming the feature space.
3 Max-Margin Domain Transforms
MMDT jointly learns a shared target-to-source feature transformation and multi-class max-margin classifiers, using alternating convex subproblems to adapt domains efficiently.
- Model formulation: MMDT jointly learns affine class-separating hyperplanes and a shared transformation W that maps target points into the source domain.The transformation is learned so projected target points fall on the correct side of each source hyperplane.
- Model formulation: The multi-class objective pools constraints across K categories while regularizing the transformation and classifier parameters.The formulation extends the binary problem by summing regularizers over θk and pooling constraints for all categories.
- Optimization: Alternating minimization updates W and (θ, b), making each fixed-parameter subproblem convex and solvable with standard SVM or quadratic-programming solvers.The target points are projected using the current W for the classifier subproblem, while the transformation subproblem is convex.
- Optimization: Each alternating stage cannot increase the global cost J(W, θ, b), so the iterative algorithm converges to a local optimum.The guarantee is convergence to a local optimum, not necessarily a global solution.
- Relation to existing work: MMDT requires K·nT constraints instead of ARC-t’s nS·nT, scaling to much larger source domains in K-category problems.Its max-margin transformation is also learned jointly with the classifier and directly optimizes the classification objective.
- Relation to existing work: Unlike HFA and GFK, MMDT supports representations generalized across novel target categories and source-target domains with different initial dimensions.HFA learns a separate transformation for each binary category, while GFK cannot handle heterogeneous initial dimensions or directly optimize classification.
4 Experiments on Image Datasets
Experiments on Office, Caltech256, and Bing evaluate MMDT across standard adaptation, heterogeneous feature dimensions, novel categories, and dataset scaling. MMDT generally achieves strong or improved multiclass accuracy while retaining computational advantages, including linear constraint scaling.
- Standard Domain Adaptation: MMDT achieved the highest multiclass accuracy on 9 of 12 domain shifts and was competitive on the remaining 3.Its advantage was especially pronounced when source and target domains were initially very different.
- Standard Domain Adaptation: MMDT’s linear method outperformed all baselines on average, although the baselines learned nonlinear transformations.The comparison covered the Office and Caltech256 adaptation benchmarks.
- Asymmetric Transform: MMDT effectively learned target representations with different feature dimensions and optimized the final classification objective.The asymmetric-transform experiment used a 600-dimensional representation for the dslr target domain and compared MMDT with svmt, arc-t, and hfa.
- Generalizing to Novel Categories: MMDT outperformed baselines for amazon→dslr and provided adaptive benefit over svms for webcam→dslr when target labels covered only the first 15 categories.This experiment tests generalization to target categories without labeled examples.
- Scaling to Larger Datasets: MMDT produced considerable multiclass-accuracy gains over baselines on Bing and was faster than all but gfk.Both MMDT and arc-t scaled approximately linearly with the number of target training points, whereas arc-t’s constraint count is quadratic in the number of training points.
5 Conclusion
The paper’s feature-learning technique combines multi-task domain adaptation with direct classifier-parameter adaptation. Experiments show effectiveness, generalization to novel target categories, asymmetric transformations, scalability, and higher accuracy, while the reported work focuses on linear transforms.
- MMDT combines feature-transform methods’ multi-task adaptation with the performance benefits of directly adapting classifier parameters.
- Experiments on two standard image-adaptation benchmarks show that MMDT can outperform previous methods and generalize to novel target categories at test time.
- MMDT can learn asymmetric transformations within a framework that is scalable to larger datasets.
- The framework achieves higher classification accuracy than previous approaches while validating computational efficiency and effectiveness.
- The presented experiments focus on linear transforms for speed and scalability, although the method can also be kernelized for nonlinear transforms.