PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Residual Correction Network for Partial Domain Adaptation

Shuang Li, Chi Harold Liu, Qiuxia Lin, Qi Wen, Limin Su, Gao Huang, Zhengming Ding

arXiv:2004.04914v1cs.CV

TL;DR

Partial domain adaptation must transfer knowledge from a large labeled source to a smaller unlabeled target whose label space is a subset of the source. DRCN combines a residual correction block with weighted class-wise alignment, and experiments report superior performance across partial, traditional, and fine-grained recognition tasks.

  • Problem

    Partial domain adaptation addresses transfer from a labeled source to an unlabeled target with a smaller label space, unlike methods assuming identical label spaces.

  • Method

    DRCN plugs a residual correction block into the source network and uses target output probabilities for weighted class-wise alignment of relevant source subclasses.

  • Results

    DRCN outperforms competitive deep domain adaptation methods across partial, traditional, and fine-grained cross-domain visual recognition.

  • Takeaways & Limitations

    The approach extends partial-domain adaptation to traditional and fine-grained cross-domain visual recognition settings.

  • Takeaways & Limitations

    The setting assumes labeled source data and unlabeled target data, with the target label space contained within the source label space.

Abstract

from arXiv · show

Deep domain adaptation methods have achieved appealing performance by learning transferable representations from a well-labeled source domain to a different but related unlabeled target domain. Most existing works assume source and target data share the identical label space, which is often difficult to be satisfied in many real-world applications. With the emergence of big data, there is a more practical scenario called partial domain adaptation, where we are always accessible to a more large-scale source domain while working on a relative small-scale target domain. In this case, the conventional domain adaptation assumption should be relaxed, and the target label space tends to be a subset of the source label space. Intuitively, reinforcing the positive effects of the most relevant source subclasses and reducing the negative impacts of irrelevant source subclasses are of vital importance to address partial domain adaptation challenge. This paper proposes an efficiently-implemented Deep Residual Correction Network (DRCN) by plugging one residual block into the source network along with the task-specific feature layer, which effectively enhances the adaptation from source to target and explicitly weakens the influence from the irrelevant source classes. Specifically, the plugged residual block, which consists of several fully-connected layers, could deepen basic network and boost its feature representation capability correspondingly. Moreover, we design a weighted class-wise domain alignment loss to couple two domains by matching the feature distributions of shared classes between source and target. Comprehensive experiments on partial, traditional and fine-grained cross-domain visual recognition demonstrate that DRCN is superior to the competitive deep domain adaptation approaches.

1 INTRODUCTION

The paper frames partial domain adaptation as transferring knowledge from a large labeled source to a smaller unlabeled target whose label space may be only a subset of the source. DRCN addresses this mismatch by weighting relevant classes and adding residual correction for domain alignment.

  • Partial domain adaptation involves a large-scale source domain subsuming the classes of a small-scale unlabeled target domain.
  • Aligning entire source and target domains can mix irrelevant source subclasses with target data and degrade target classification.
  • DRCN also aligns domain-wise knowledge when classifier predictions are poor, supplementing class-wise transfer with general information.
  • DRCN plugs one residual block into a unified source network to capture feature differences and mitigate cross-domain discrepancy.
  • Weighted class-wise alignment uses target output probabilities to assign larger weights to source subclasses most relevant to target data.
  • Experiments across partial, traditional, and fine-grained visual recognition show DRCN outperforming competitive deep domain adaptation methods.

2 RELATED WORK

Related work includes instance reweighting, domain-invariant feature learning, moment matching, and adversarial adaptation, but most methods assume identical label spaces. DRCN instead targets partial adaptation by selecting relevant source classes and correcting domain discrepancies.

  • Existing domain adaptation methods commonly reweight source instances or learn domain-invariant features to reduce cross-domain distribution differences.
  • Many deep adaptation methods assume identical source and target label spaces, an assumption that is too strict for real-world applications.
  • Identical-label-space methods can cause negative transfer when only partial relevant knowledge should move from a large source to a small unlabeled target.
  • Prior partial adaptation methods reduce irrelevant-class effects through selective adversarial matching or down-weighting outlier source classes.
  • DRCN differs by adding one residual correction block and using target output probabilities for weighted class-wise feature alignment.

3 THE PROPOSED ALGORITHM: DRCN

This section introduces the proposed Deep Residual Correction Network and presents its background and preliminary knowledge before detailing the method.

  • The section introduces the proposed Deep Residual Correction Network in detail.
  • Background knowledge is presented before the proposed algorithmic details.
  • Preliminary concepts are presented before the proposed algorithmic details.

3.1 Preliminary

Partial domain adaptation transfers relevant knowledge from a labeled source to an unlabeled target when the target label space is a subset of the source. The domains share feature space but have different feature distributions.

  • Partial domain adaptation provides labeled source data and unlabeled target data for transferring relevant knowledge.
  • The target label space is a subset of the source label space, with fewer target classes than source classes.
  • Source and target domains share feature space but have different feature distributions because of domain shift.
  • The objective is to reduce partial distribution discrepancy while transferring the most relevant source discriminative knowledge to the target.

3.2 Motivation

Partial transfer learning must identify which source knowledge is relevant because the target label space is only a subset of the source label space. DRCN motivates combining selective class alignment with direct correction of task-specific feature discrepancies.

  • Partial domain adaptation transfers from a large source label space to a smaller target label space with unknown target categories.
  • Aligning entire source and target distributions can let irrelevant source classes negatively affect domain adaptation.
  • Effective adaptation therefore requires identifying relevant source classes and aligning their conditional distributions precisely.
  • Because nonlinear source-network layers can amplify cross-domain differences, DRCN corrects feature discrepancies at the task-specific layer with a residual correction block.

3.3 Deep Residual Correction Network

DRCN adapts a pretrained source network by correcting target task-specific features and weighting class-wise alignment according to target predictions. Its objective combines source classification, domain adaptation, and weighted class-wise matching, while also supporting traditional adaptation settings.

  • Maximum Mean Discrepancy: MMD measures source-target distribution similarity, and minimizing an improved MMD metric supports feature and task adaptation.
  • Residual Correction Block: DRCN uses an added residual block to learn target feature discrepancies after the task-specific source layer.The modified target representation is Ft(xt) = Fs(xt) + ∆Fs(xt), while source samples use the original network path.
  • Weighted Class-wise Alignment: DRCN estimates target class proportions from averaged source-classifier soft labels to identify relevant source subclasses.Small target-prediction weights indicate source classes likely irrelevant to the target domain.
  • Weighted Class-wise Alignment: Weighted class-wise matching emphasizes shared classes with larger weights and reduces distraction from irrelevant source classes.The method uses a probabilistic class-wise multiple-kernel MMD to incorporate prior target category distributions.
  • Overall Formulation of DRCN: The overall DRCN formulation jointly adapts general and task-level knowledge while aligning shared-class feature distributions with larger importance scores.Soft-label and hard-label variants are defined, and equal class weights recover traditional domain adaptation settings.

4 EXPERIMENT

The experiments evaluate DRCN across four cross-domain object-recognition benchmarks spanning partial transfer, traditional domain adaptation, and large-scale or synthetic-to-real settings. Comparisons include shallow baselines and twelve traditional or partial deep adaptation methods.

  • Datasets: Evaluation uses Office-Home, Office-31, VisDA2017, and ImageNet-Caltech for cross-domain object recognition.These datasets cover artistic, clip-art, product, real-world, office-camera, synthetic, photorealistic, and large-scale image domains.
  • Datasets: Office-Home provides 12 partial-transfer tasks across four domains, with the first 25 alphabetical classes selected as target categories.
  • Datasets: VisDA2017 compares synthetic 2D renderings with photorealistic images across 12 common categories.
  • Datasets: ImageNet-Caltech supplies two partial-transfer tasks between ImageNet-1K and Caltech-256 using 84 common classes.
  • Compared Methods: DRCN is compared with shallow GFK and TCA baselines plus twelve traditional or partial deep domain adaptation methods.

4.2 Setup

The setup uses labeled source data and unlabeled target data, averages accuracy over three random experiments, and fine-tunes ImageNet-pretrained ResNet-50 models. Office-Home results are reported for partial-transfer tasks.

  • Protocol: Source data are labeled whereas target data are unlabeled under the standard unsupervised domain adaptation protocol.
  • Evaluation: Table 2 reports accuracy percentages on Office-Home partial-transfer tasks using ResNet-50.
  • Protocol: Reported accuracy averages each cross-domain task over three random experiments.
  • Implementation: DRCN and comparison methods are fine-tuned from PyTorch-provided ImageNet-pretrained ResNet-50 models.The classifier learning rate is ten times larger than other layers, while the residual layer uses one tenth their learning rate.

4.3 Results and Analysis

DRCN is evaluated across partial-transfer benchmarks and generally outperforms comparison methods, while label strategy effectiveness varies with dataset scale. Its gains are attributed to identifying relevant source classes and learning transferable features.

  • DRCN outperforms comparison methods on most Office-Home tasks and achieves the highest accuracy on 7 of 12 tasks.
  • At least 3.2% improvement over SAN is reported for DRCN’s average accuracy on Office-Home.
  • DRCN achieves 95.9% average accuracy with hard labels and 94.3% with soft labels on Office-31.
  • DRCN substantially improves accuracy on larger-scale VisDA2017 and ImageNet-Caltech tasks, although PADA reaches 76.5% versus DRCN’s 74.2% on R→S.
  • DRCN exceeds PADA by at least 3.7% on the S→R task.
  • Soft-label DRCN tends to work better on large-scale datasets, whereas hard-label DRCN tends to work better on small-scale datasets.
  • Across small- and large-scale datasets, DRCN achieves significant improvements on most tasks by identifying relevant source classes and learning transferable features.

4.4 Extension: Traditional Transfer Learning Tasks

The paper extends DRCN to traditional domain adaptation by adjusting its alignment settings, evaluates it on Office-Home and Office-31, and applies it to fine-grained Web→GSV recognition.

  • Extension: Traditional Transfer Learning Tasks: Setting α larger and class-wise weights w = 1 generalizes DRCN to traditional domain adaptation scenarios.
  • Traditional Transfer Results: On traditional Office-Home tasks, DRCN outperforms other methods on most tasks and remains competitive with CDAN+E.
  • Traditional Transfer Results: On traditional Office-31 tasks, soft-label DRCN reaches 87.2% average accuracy, hard-label DRCN reaches 87.0%, and the best increase over CDAN is 0.7%.
  • Traditional Transfer Results: Soft-label DRCN is slightly better than hard-label DRCN in traditional transfer scenarios because target prediction distributions provide more discriminative knowledge than one-hot pseudo-labels.
  • Cross-domain Fine-grained Recognition in the Wild: Fine-grained recognition distinguishes objects within the same category, while cross-domain recognition addresses sparse annotations in new real-world scenarios.
  • Cross-domain Fine-grained Recognition in the Wild: The Web→GSV task addresses large viewpoint, occlusion, and resolution discrepancies between high-resolution Web images and Google Street View images.
  • Cross-domain Fine-grained Recognition in the Wild: DRCN outperforms comparison methods on the Web→GSV fine-grained recognition task.

4.5 Empirical Analysis

The empirical analyses show that DRCN progressively aligns source and target features, identifies relevant source classes, and benefits from its residual correction block, weighted class-wise alignment, and balanced parameters. Across the evaluated settings, these components support stronger domain adaptation and competitive performance.

  • Feature correction process: DRCN progressively makes source and target representations similar, with samples becoming nearly indistinguishable by the end of training in both traditional and partial adaptation.The t-SNE analyses cover task A →W in Office-31.
  • Feature correction process: DRCN produces more cohesive shared-class clusters than IWAN, SAN, PADA, and DRCN without the residual correction block.The residual correction block further mitigates feature discrepancy and corrects misaligned class centers.
  • Variants of DRCN: DRCN with class-wise and domain-wise transfer generally outperforms PADA, while removing class-wise alignment reduces accuracy by over 1% on Office-31 and 7% on Office-Home.The ablations also show that fixed class weights can trigger negative transfer and that removing the residual correction block reduces performance by over 1% on Office-31 and more than 2% on Office-Home.
  • Learned class weights: DRCN assigns substantially larger weights to shared classes than irrelevant classes, with all irrelevant-class weights below 0.1 for A →W.The learned weights use the target output distribution to identify source subclasses relevant to the target domain.
  • Parameter sensitivity: Parameter effects depend on the transfer setting: increasing α hurts Office-31 but helps Office-Home, whereas increasing β helps Office-31 but can slightly hurt Office-Home.The observations support choosing α and β through a task-dependent trade-off.
  • Overall empirical findings: With reasonable α and β values, DRCN performs stably and outperforms other competitors, while its features achieve the smallest proxy A-distance among four methods.The smaller proxy A-distance indicates features that are easier to transfer.

5 CONCLUSION

The paper introduces DRCN for partial domain adaptation, where target classes form a subset of source classes, by combining weighted class-wise matching with residual feature correction. Extensive experiments report that DRCN outperforms competitive deep domain adaptation methods in partial and traditional scenarios.

  • Method: DRCN addresses partial domain adaptation by using target output probabilities to weight source classes and align target data with the most relevant source subclasses.The network also transfers general and task-level knowledge through a residual correction block inserted into the source network.
  • Scope: DRCN extends to traditional and fine-grained cross-domain visual recognition through an efficiently implemented residual correction architecture.The paper states that the architecture can be generalized to new deep networks.
  • Results: DRCN outperforms several competitive deep domain adaptation approaches by a large margin in both partial and traditional domain adaptation scenarios.The conclusion bases this claim on extensive experiments across several standard cross-domain datasets.
Loading 2004.04914v1…