PaperlayerView exact revision on arXiv

Source-linked AI summary

Correlation-aware Adversarial Domain Adaptation and Generalization

Mohammad Mahfujur Rahman, Clinton Fookes, Mahsa Baktashmotlagh, Sridha Sridharan

arXiv:1911.12983v1cs.CVcs.LG

TL;DR

The paper addresses domain shift in domain adaptation and the harder domain-generalization setting, where target data are unseen during training. It proposes correlation-aware adversarial learning for both settings and reports improved state-of-the-art performance on benchmark datasets. The paper also notes scope constraints from limited data in one evaluated dataset and a remaining discrepancy issue in the domain-generalization approach.

  • Problem

    Domain adaptation and domain generalization face differing source and target distributions; adversarial methods are rarely used for domain generalization and do not consider correlation alignment.

  • Method

    The proposed framework combines adversarial learning with correlation alignment for unsupervised domain adaptation and domain generalization.

  • Results

    The proposed model yields state-of-the-art results on benchmark domain adaptation and domain generalization experiments.

  • Takeaways & Limitations

    Combining adversarial learning with correlation alignment improves the approach's domain-discrepancy reduction capability.

  • Takeaways & Limitations

    The approach still suffers from discrepancy among domains, and ImageCLEF-DA has only 600 images per domain, which may be insufficient for training.

Abstract

from arXiv · show

Domain adaptation (DA) and domain generalization (DG) have emerged as a solution to the domain shift problem where the distribution of the source and target data is different. The task of DG is more challenging than DA as the target data is totally unseen during the training phase in DG scenarios. The current state-of-the-art employs adversarial techniques, however, these are rarely considered for the DG problem. Furthermore, these approaches do not consider correlation alignment which has been proven highly beneficial for minimizing domain discrepancy. In this paper, we propose a correlation-aware adversarial DA and DG framework where the features of the source and target data are minimized using correlation alignment along with adversarial learning. Incorporating the correlation alignment module along with adversarial learning helps to achieve a more domain agnostic model due to the improved ability to reduce domain discrepancy with unlabeled target data more effectively. Experiments on benchmark datasets serve as evidence that our proposed method yields improved state-of-the-art performance.

1. Introduction

Domain adaptation and generalization address differing source and target distributions, with generalization operating without target data during training. The paper combines correlation alignment and adversarial learning for both settings and reports state-of-the-art results across five benchmark datasets.

  • Domain adaptation uses target data during training, whereas domain generalization does not and must generalize to an unknown target domain.
  • Adversarial adaptation extracts features and trains a domain classifier to distinguish source from target data, with a gradient reversal layer reducing domain disparity.
  • The proposed framework jointly uses correlation alignment and adversarial learning to minimize source-target domain disparity.
  • Using both strategies further brings same-class features together, separates different-class features in adaptation, and reduces source-domain discrepancies in generalization.
  • The method is evaluated under unsupervised adaptation and generalization settings on five benchmark datasets.
  • The framework achieves state-of-the-art results and the best average image-classification accuracies across the reported evaluations.

2. Related Work

Prior work reduces domain discrepancy through distribution matching, adversarial learning, generative models, normalization, and source-domain alignment. This paper builds on adversarial adaptation by using separate feature extractors, correlation alignment, and a unified architecture for DA and DG.

  • Distribution matching: Distribution-matching methods reduce domain discrepancy using metrics such as MMD, CORAL, and CMD.
  • Adversarial adaptation: Adversarial methods train a discriminator to classify source or target features while the feature extractor learns to fool it.
  • Generative approaches: GAN-based approaches adapt either joint distributions or representations, with pixel-space and feature-space methods forming distinct variants.
  • Domain generalization: Domain generalization methods use source domains without target data, including aggregation, invariant transformations, classifier adjustment, auto-encoding, and adversarial approaches.
  • Relation to prior work: The proposed method uses two feature extractors, adversarial discrepancy reduction, and correlation alignment between the final layers of two CNN streams.
  • Relation to prior work: The framework extends to domain generalization and remains unified for both DA and DG settings without changing its architecture.

3. Proposed Method

The proposed framework addresses unsupervised domain adaptation and generalization by combining adversarial learning with correlation alignment to reduce domain discrepancy. It uses separate source and target feature extractors, aligns second-order feature statistics, and extends the same strategy to unseen target domains.

  • Domain adaptation: Unsupervised domain adaptation uses labeled source data and unlabeled target data during training.The model learns a classifier intended to classify target samples despite unavailable target labels.
  • Adversarial learning: The adversarial component learns domain-invariant, class-discriminative features through source and target extractors and a domain discriminator.The discriminator predicts whether extracted features originate from the source or target domain, while the extractors minimize domain discrepancy.
  • Correlation-aware framework: The framework combines adversarial learning with correlation alignment to reduce discrepancy between source and target domains.The alignment module operates on source and target activation features using second-order statistics.
  • Correlation alignment: The correlation alignment loss minimizes the squared Frobenius distance between source and target covariance matrices.The covariance matrices are computed from the source and target feature activations, with a tunable hyperparameter weighting the alignment term.
  • Reported effects: The authors report that correlation alignment further reduces domain discrepancy and produces a more domain-agnostic model for target classification.For domain generalization, the model is described as reducing source-domain shifts and being applicable to unseen target domains.
  • Domain generalization: For domain generalization, the framework discards target data during training and minimizes discrepancy among available source domains.The same architecture is applied to unknown target data, with source data split and aggregated into training and validation streams.
  • Domain generalization: Splitting and aggregating source data across two CNN streams reduces domain discrepancy but does not remove domain mismatch completely.The remaining mismatch is observed between data passed through the two streams.

4. Experiments

The paper evaluates its approach on five benchmark datasets spanning domain adaptation and domain generalization, using AlexNet-based feature extraction with adversarial and correlation-alignment modules.

  • The proposed approach is evaluated on five commonly used datasets for domain adaptation and domain generalization.
  • The evaluation covers Office-31, Office-Home, ImageCLEF-DA, Office-Caltech, and PACS, whose domains include camera, product, artistic, sketch, and image-recognition settings.
  • The experiments use AlexNet, with a 256-unit bottleneck layer added after fc7 for safer transfer learning.
  • The network uses convolutional layers and fully connected layers for both source and target streams, with fcB feeding the discriminator and correlation alignment applied between fc8 layers.
  • A gradient reversal layer encourages similar source and target feature distributions, producing domain-invariant features for the domain classifier.

4.3. Experimental Setup

The experiments evaluate the proposed approach on benchmark datasets under unsupervised domain adaptation and domain generalization protocols, using labeled source data and either unlabeled or unseen target data.

  • Implementation: Experiments use Caffe and average results from three runs on high-performance computing infrastructure.Image-classification experiments use batch size 128, learning rate 0.001, momentum 0.9, and weight decay 5 × 10^-4.
  • Domain adaptation protocol: Unsupervised domain adaptation uses labeled source data and unlabeled target data.The evaluated transfer tasks include Office-31, Office-Home, and ImageCLEF-DA datasets.
  • Domain generalization protocol: Domain generalization evaluates transfer to target domains that are totally unseen during training.The evaluated datasets include Office-31, Office-Caltech, and PACS.
  • Baselines: The study compares the proposed method with deep adaptation, adversarial, correlation-alignment, and domain-generalization baselines.Compared methods include DANN, deep-CORAL, DAN, DRCN, RTN, JAN, ADDA, MADA, DICA, DSN, DBADG, CIDDG, Undo-Bias, UML, and MTAE.

4.5. Hyper-parameter tuning

Hyper-parameter tuning selects two weight-balance parameters for the proposed method and then reuses the selected values across transfer tasks.

  • Tuning procedure: The method tunes two weight-balance hyper-parameters, σ and γ, for unsupervised domain adaptation and generalization experiments.The tuning procedure applies different parameter values to transfer-task experiments.
  • Selected values: γ = σ = 0.1 provides the best results on the A →W transfer task.The selected values are then used for all other transfer tasks.

4.6. Results and Discussion

Results across benchmark datasets show strong performance for the proposed architecture in both domain adaptation and domain generalization, with correlation alignment complementing adversarial learning.

  • Office-Home: 48.19% average accuracy on Office-Home outperforms DANN by 3.25% and matches JAN’s reported 48.2% average accuracy.The method beats other state-of-the-art methods except JAN.
  • Office-31 domain adaptation: Deep domain adaptation methods outperform standard deep learning methods on Office-31, indicating that standard networks do not eliminate domain shift.DANN uses an additional domain classifier and outperforms standard deep networks by about 4%.
  • Office-31 domain adaptation: Adding correlation alignment to adversarial learning improves average accuracy by 5% over baseline adversarial methods on Office-31.The evaluated transfer tasks are A →W, D →W, D →A, W →A, W →D, and A →D.
  • Office-Home: The proposed method achieves state-of-the-art performance on six Office-Home transfer tasks, while JAN leads the other six.The proposed tasks are A →P, C →R, P →A, P →R, R →A, and R →P.
  • ImageCLEF-DA: 80.2% average accuracy on ImageCLEF-DA exceeds the prior best 79.8% achieved by MADA, establishing new state-of-the-art performance.The method outperforms existing approaches on four transfer tasks: P →I, I →C, C →I, and C →P.
  • ImageCLEF-DA: The method remains effective when source and target domains share few categories and outperforms DANN on all ImageCLEF-DA transfer tasks.The reported average accuracy across the different tasks is 78.3%.
  • Office-31 domain generalization: For domain generalization, the method achieves state-of-the-art performance on two Office-31 tasks and 81.0% average accuracy across three tasks.The relevant tasks include A, W →D; A, D →W; and D, W →A, with target data unavailable during training.
  • PACS: On PACS, the proposed approach achieves state-of-the-art performance on two tasks and 71.98% average accuracy across four transfer tasks.The target data is unavailable during training in these domain-generalization experiments.

4.7. Visualization

The proposed model produces embeddings that separate categories while mixing domains more effectively than standard deep learning and single-loss variants. The visualization also suggests stronger same-class alignment between source and target features.

  • Visualization: t-SNE embeddings are used to visualize images from Amazon and Webcam in the Office-31 Amazon →Webcam transfer task.The comparison includes standard AlexNet, the model without adversarial loss, the model without discrepancy loss, and the full model.
  • Visualization: The proposed model separates categories while mixing domains more effectively than standard DNN and single-loss variants.The comparison covers models using only discrepancy loss or only adversarial loss.
  • Visualization: The visualization suggests that the proposed method better aligns source and target features from the same class.

4.8. Convergence Performance

The method converges at a speed analogous to DANN while maintaining remarkably better accuracy throughout convergence. Recognition accuracy also increases as more unlabeled target data are included in training for adaptation and generalization tasks.

  • Convergence: The convergence comparison evaluates test accuracy over 50,000 iterations for the proposed method and DANN on A →D and A →W transfer tasks.The graphs use iterations on the x-axis and percentage accuracy on the y-axis.
  • Convergence: The proposed model has analogous convergence speed to DANN with remarkably better accuracy throughout the convergence procedure.
  • Target-data size: As unlabeled target-data size expands, image-recognition accuracy increases for the A →W adaptation and W, D, C →A generalization tasks.The experiments use Office-31 for adaptation and Office-Caltech for generalization, varying class data access from the top 20% to 100%.
  • Target-data size: More unlabeled target data during training can produce a more transferable classifier for the target domain.

4.10. Sensitivity of Embedding Dimension

The approach is evaluated across domain adaptation and generalization settings, including embedding-dimension sensitivity, correlation alignment, component ablations, discriminator choices, and initialization. Results indicate stable performance near dimension 256 and improved performance when correlation alignment is combined with adversarial learning.

  • Embedding dimension: Embedding dimension 256 is reported as a setting under which the approach is not sensitive to the chosen feature-space dimension.
  • Correlation alignment: Adding correlation alignment boosts recognition performance by 2.1% for domain adaptation and 1.89% for domain generalization.The case study reports these gains after using correlation alignment to further reduce domain discrepancy.
  • Component ablation: Ablation results show better performance from combining correlation alignment with adversarial learning than from using either component alone across transfer tasks.The combined approach is reported to improve discrimination and generalization ability.
  • Discriminator and initialization: Using another discriminator to align Fc8 features performs worse than the proposed correlation-alignment method.
  • Discriminator and initialization: Initializing with a pre-trained source model improves image-recognition accuracy by about 0.4% to 0.5%.The comparison reports this improvement across domain adaptation and generalization tasks.

4.13. Discussion

Across the discussion, deep domain adaptation and generalization methods outperform standard deep-learning approaches, while the combined adversarial and correlation-alignment approach improves model generalization. Accuracy also rises with more unlabeled target data and remains stable across embedding dimensions.

  • Discussion: Deep domain adaptation and generalization methods outperform standard deep-learning approaches.
  • Discussion: Adversarial learning with correlation alignment improves the model's generalization capability.
  • Discussion: As target-data size increases, image-classification accuracy also increases, and more unlabeled target data can yield a more transferable target-domain classifier.
  • Discussion: Image-recognition accuracy remains almost at the same level across embedding dimensions and is slightly higher in the reported comparison.
  • Discussion: Using a pre-trained source model as initialization improves image-recognition accuracy by 0.4% to 0.5% on domain adaptation and generalization tasks.

5. Conclusion

The paper proposes correlation-aware adversarial domain adaptation and generalization to minimize domain discrepancy using correlation alignment alongside adversarial learning. Experiments across datasets verify the method’s effectiveness for both adaptation and generalization.

  • The framework combines correlation alignment with adversarial learning for domain adaptation and domain generalization scenarios.
  • The proposed method minimizes domain discrepancy during training with correlation alignment and adversarial learning.
  • The model addresses domain generalization without changing the basic network architecture.
  • Experiments on different datasets verify the effectiveness of the proposed method for domain adaptation and domain generalization.
Loading 1911.12983v1…