PaperlayerView exact revision on arXiv

Source-linked AI summary

Domain Separation Networks

Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, Dumitru Erhan

arXiv:1608.06019v1cs.CV

TL;DR

Domain adaptation is needed because inexpensive synthetic training data often transfers poorly to realistic images, while existing methods overlook domain-specific characteristics. DSN partitions representations into private and shared components, using reconstruction and separation objectives alongside domain adaptation losses. Across unsupervised adaptation scenarios, DSN with DANN outperforms the compared methods and yields interpretable private and shared visualizations.

  • Problem

    Synthetic data provides automatically labeled examples but models trained on it often fail to generalize to realistic target images, while existing approaches overlook domain-specific characteristics.

  • Method

    Domain Separation Networks jointly learn private domain-specific and shared domain-invariant representations using independence, reconstruction, orthogonality, and domain-similarity objectives.

  • Results

    DSN with DANN outperforms the other evaluated methods across all unsupervised domain adaptation scenarios and produces visualizations of private and shared representations.

  • Takeaways & Limitations

    Separating private from shared information supports domain adaptation while making the adaptation process visually interpretable.

  • Takeaways & Limitations

    The approach assumes source and target domains mainly differ in low-level image statistics, with similar high-level parameters and the same label space.

Abstract

from arXiv · show

The cost of large scale data collection and annotation often makes the application of machine learning algorithms to new tasks or datasets prohibitively expensive. One approach circumventing this cost is training models on synthetic data where annotations are provided automatically. Despite their appeal, such models often fail to generalize from synthetic to real images, necessitating domain adaptation algorithms to manipulate these models before they can be successfully applied. Existing approaches focus either on mapping representations from one domain to the other, or on learning to extract features that are invariant to the domain from which they were extracted. However, by focusing only on creating a mapping or shared representation between the two domains, they ignore the individual characteristics of each domain. We suggest that explicitly modeling what is unique to each domain can improve a model's ability to extract domain-invariant features. Inspired by work on private-shared component analysis, we explicitly learn to extract image representations that are partitioned into two subspaces: one component which is private to each domain and one which is shared across domains. Our model is trained not only to perform the task we care about in the source domain, but also to use the partitioned representation to reconstruct the images from both domains. Our novel architecture results in a model that outperforms the state-of-the-art on a range of unsupervised domain adaptation scenarios and additionally produces visualizations of the private and shared representations enabling interpretation of the domain adaptation process.

1 Introduction

Large-scale labeled datasets are costly, while models trained on synthetic data often fail to generalize to realistic domains. The paper studies domain-invariant representations for transferring knowledge from labeled source data to unlabeled target data under specified domain assumptions.

  • Collecting and annotating large-scale datasets is expensive and time-consuming, motivating inexpensive synthetic alternatives.
  • Synthetic datasets can provide abundant automatically labeled data, but models trained on them often generalize poorly to realistic domains.
  • The target setting transfers knowledge from a labeled source domain to an unlabeled target domain whose training and testing distributions differ.
  • The method targets object classification and pose estimation with foreground objects in source and target images.
  • The domains are assumed to differ mainly in low-level image statistics while sharing similar high-level parameter distributions and the same label space.
  • Domain Separation Networks learn domain-invariant representations by separating domain-specific properties into private subspaces and common information into a shared subspace.

2 Related Work

Prior unsupervised domain adaptation methods learn domain-invariant features through adversarial or discrepancy-based objectives, or transform one domain's representations toward the other. These approaches include DANN, MMD-based methods, and CORAL.

  • Domain adaptation theory bounds target-domain classifier performance using source-specific error and a classifier's ability to distinguish source from target domains.
  • DANN jointly trains a source task classifier and a domain classifier, minimizing task loss while making learned features less informative about domain identity.
  • MMD-based approaches replace adversarial domain-loss maximization with feature-distribution matching between source and target samples.
  • Transformation-based methods adapt domains by learning mappings while keeping feature extraction fixed, including CORAL's covariance-based source-feature recoloring.

3 Method

Domain Separation Networks learn representations split into domain-private and domain-shared components for unsupervised adaptation, combining task supervision, reconstruction, separation, and cross-domain similarity objectives. The classifier uses the shared representation, while private and shared components jointly reconstruct inputs.

  • Architecture: DSNs use shared-weight and domain-specific private encoders to capture common and domain-specific image components, respectively.A shared decoder reconstructs each input from the private and shared representations.
  • Representation separation: DSNs jointly model private and shared components so shared features are less contaminated by domain-unique aspects and can better generalize across domains.The method is motivated by the limitation that shared representations can include noise correlated with the shared distribution.
  • Architecture: The classifier predicts task labels from the shared representation, while the decoder reconstructs inputs from the sum of shared and private representations.Inference is defined as ˆx = D(Ec(x) + Ep(x)) and ˆy = G(Ec(x)).
  • Training objective: The objective combines task, reconstruction, difference, and similarity losses, with α, β, and γ controlling their interaction.The classification loss uses labeled source data because the target domain is unlabeled.
  • Training objective: Reconstruction uses a scale-invariant mean squared error on both domains, emphasizing object shape over absolute color or intensity.The loss penalizes differences between pairs of pixels rather than errors caused solely by scaling.
  • Training objective: The difference loss encourages orthogonality between private and shared representations, while the similarity loss makes shared representations similar across domains.These constraints encourage the two encoders to capture different aspects while aligning the shared representation.
  • Similarity losses: Similarity can be learned adversarially with a gradient reversal layer or through Maximum Mean Discrepancy between source and target shared encodings.The MMD implementation uses a linear combination of multiple RBF kernels.

4 Evaluation

The evaluation tests DSN across synthetic-to-real classification and pose-estimation settings, under assumptions about domain differences and label spaces. DSN with DANN performs best across the evaluated unsupervised adaptation scenarios, while ablations and visualizations examine the roles of its losses and representations.

  • Datasets and adaptation scenarios: The evaluation covers MNIST, MNIST-M, GTSRB, SVHN, and LINEMOD, including object classification, instance recognition, and 3D pose estimation.The scenarios include synthetic-to-real and real-to-real adaptations, with LINEMOD evaluating both classification and pose estimation.
  • Evaluation protocol: Table 1 reports mean classification accuracy across adaptation scenarios, with Source-only and Target-only models providing non-adapted reference points.The table also replicates experiments from prior work and marks the best result in each task in bold.
  • Results: The DSN with DANN outperforms the other evaluated methods in every unsupervised domain adaptation scenario.The model improves upon both MMD regularization and DANN, while DANN provides a stronger similarity loss than MMD in these experiments.
  • Ablations: Removing the difference or reconstruction losses produces consistently worse classification results.Table 3 compares the complete model with versions lacking the soft orthogonality constraint or using regular rather than scale-invariant MSE.
  • Representation visualizations: Reconstructions combine shared and private representations, while separate decoding visualizes domain-shared and domain-private content in MNIST-M and LINEMOD.The shared representations look similar across source and target domains and support classification and pose estimation in the LINEMOD scenario.

5 Conclusion

The model explicitly separates representations private to each domain from those shared between source and target, improving unsupervised domain adaptation across synthetic-to-real scenarios.

  • The model separates representations private to each domain from representations shared between source and target domains.

A Correlation Regularization

Correlation Regularization adapts feature hierarchies by aligning correlation matrices, and the DSN with CorReg outperforms both CORAL and CorReg.

  • Correlation Alignment maps source representations to target representations by matching second-order statistics.
  • Correlation Regularization aligns correlation matrices while adapting convolutional features, unlike CORAL's fixed feature extraction process.
  • Correlation Regularization is theoretically as powerful as an MMD loss with a second-order polynomial kernel and outperforms CORAL across the datasets.
  • The DSN with CorReg performs better than both CORAL and CorReg.

B Office Dataset Criticism

Office and Caltech–256 contain substantial low- and high-level domain variation, including label pollution and inconsistent object poses, complicating visual domain adaptation.

  • Office and Caltech–256 vary in camera and background texture as well as classes, object identities, and geometric configurations.
  • The backpack class includes 92 Amazon samples, 12 DSLR samples, 29 Webcam samples, and 151 Caltech samples with dataset-specific inconsistencies.
  • Label pollution includes non-motorcycle objects in the motorcycle class and two laptops in the backpack class.
  • Some classes contain the object in only one pose, leading some methods to pretrain on ImageNet before domain adaptation.

C Domain Separation

The paper visualizes reconstructions from shared and private representations across several adaptation scenarios, comparing source and target domains and showing how their information differs.

  • Figure 3 organizes backpack examples by domain: Amazon, DSLR, Webcam, and Caltech.
  • The shared representations show similarities across domains, while private representations exclude some shared information.
  • Figure 4 compares original images with full, shared-only, and private-only reconstructions across four domain adaptation scenarios.
  • Each reconstruction block places target-domain examples in the top row and source-domain examples in the bottom row.

D Network Topologies and Optimal Parameters

The appendix specifies the network topologies and hyperparameter ranges used across the domain adaptation experiments, supporting reproducibility.

  • Network topologies: Different network topologies were used for the domain adaptation scenarios and are presented in Figures 5–8.
  • Optimal parameters: CORAL uses an optimized SVM penalty of 1e−4 for three scenarios and 1e−3 for SVHN to MNIST.
  • Optimal parameters: MMD uses 19 RBF kernels with equal η weights, while its learning rate is between [0.01, 0.015] and γ ∈[0.1, 0.3].
  • Optimal parameters: DANN uses learning rate [0.01, 0.015] and γ ∈[0.15, 0.25], whereas DSN variants use an initial rate of 0.01 with method-specific α, β, and γ ranges.
  • Optimal parameters: All cases use exponential learning-rate decay of 0.95, with additional GTSRB-specific α ∈[0.01, 0.015] and DSN w/ CorReg γ ∈[20, 100].
  • Network topologies: Figures 6–8 provide the network topologies for Synth SVHN to SVHN, SVHN to MNIST, Synth Signs to GTSRB, and Synthetic Objects to Linemod experiments.
Loading 1608.06019v1…