PaperlayerView exact revision on arXiv

Source-linked AI summary

Bi-Classifier Determinacy Maximization for Unsupervised Domain Adaptation

Shuang Li, Fangrui Lv, Binhui Xie, Chi Harold Liu, Jian Liang, Chen Qin

arXiv:2012.06995v1cs.CV

TL;DR

Unsupervised domain adaptation transfers knowledge from labeled source data to an unlabeled target domain, where bi-classifier methods may neglect prediction determinacy. BCDM uses CDD to encourage diverse yet consistent, determined target predictions. The paper reports that it outperforms state-of-the-art methods across image-classification and semantic-segmentation UDA scenarios.

  • Problem

    Bi-classifier UDA methods can neglect target classifier determinacy, while domain adaptation must transfer knowledge from labeled source data to an unlabeled target domain.

    Source: Introduction · p. 1

  • Method

    BCDM defines CDD from the class relevance of distinct target predictions and adversarially optimizes it to promote diverse, consistent, and determined outputs.

    Source: Introduction · pp. 1–2

  • Results

    BCDM outperforms state-of-the-art UDA methods across image-classification and semantic-segmentation scenarios.

    Source: Conclusion · pp. 8–9

  • Takeaways & Limitations

    The paper concludes that classifier determinacy and representation discriminability are relevant to transferability in unsupervised domain adaptation.

    Source: Conclusion · p. 8

    The generalization bound treats the ideal joint-hypothesis error as rather small when the hypothesis space is sufficiently large.

    Source: Generalization Bound of BCDM · p. 5

Abstract

from arXiv · show

Unsupervised domain adaptation challenges the problem of transferring knowledge from a well-labelled source domain to an unlabelled target domain. Recently,adversarial learning with bi-classifier has been proven effective in pushing cross-domain distributions close. Prior approaches typically leverage the disagreement between bi-classifier to learn transferable representations, however, they often neglect the classifier determinacy in the target domain, which could result in a lack of feature discriminability. In this paper, we present a simple yet effective method, namely Bi-Classifier Determinacy Maximization(BCDM), to tackle this problem. Motivated by the observation that target samples cannot always be separated distinctly by the decision boundary, here in the proposed BCDM, we design a novel classifier determinacy disparity (CDD) metric, which formulates classifier discrepancy as the class relevance of distinct target predictions and implicitly introduces constraint on the target feature discriminability. To this end, the BCDM can generate discriminative representations by encouraging target predictive outputs to be consistent and determined, meanwhile, preserve the diversity of predictions in an adversarial manner. Furthermore, the properties of CDD as well as the theoretical guarantees of BCDM's generalization bound are both elaborated. Extensive experiments show that BCDM compares favorably against the existing state-of-the-art domain adaptation methods.

Introduction

Unsupervised domain adaptation seeks to transfer knowledge from labeled source data to an unlabeled target domain, but bi-classifier methods can neglect target prediction determinacy. BCDM introduces CDD to promote diverse classifier predictions and discriminative, consistent, determined target representations; experiments report gains across several UDA settings.

Introduction

  • Prior bi-classifier discrepancy can ignore class relevance, allowing uncertain target predictions that may impair feature discriminability.The paper gives the example of identical predictions [0.34, 0.33, 0.33] despite a near-zero ℓ1 discrepancy.Source: Introduction · p. 1
  • On VisDA-2017, BCDM produces more target predictions with maximum probability above 0.9 than MCD, whose predictions are more uncertain.The figure caption also reports fewer MCD target samples on the prediction-pair diagonal.Source: Figure 1 · p. 2
  • BCDM adversarially optimizes CDD to enhance classifier determinacy and prediction diversity, generating target features with consistent, determined predictions.The classifiers first maximize CDD to encourage disagreement across classes; the generator then minimizes it.Source: Introduction · p. 2
  • Experiments report significant improvements over state-of-the-art UDA methods across toy, image-classification, and synthetic-to-real semantic-segmentation settings.The paper lists a toy dataset, four cross-domain image-classification benchmarks, and a synthetic-to-real semantic-segmentation setup.Source: Introduction · p. 2

Related Work

The related work covers discrepancy-based and adversarial approaches to domain adaptation, including methods that use two task-specific classifiers. It positions BCDM as a bi-classifier approach that incorporates class relevance and classifier determinacy into its discrepancy design.

Related Work

  • Discrepancy-based methods such as DAN and JAN align domains by minimizing MMD-based metrics.DAN reduces the domain gap with MMD, while JAN uses modified MMD versions to align joint distributions.Source: Related Work · p. 2
  • Adversarial UDA methods learn indistinguishable source and target representations using either a domain discriminator or two task-specific classifiers.The two-classifier paradigm treats the classifiers as a domain discriminator.Source: Related Work · p. 2
  • MCD’s ℓ1 discrepancy compares predictions at matching class positions, while SWD adds computation and hyperparameter-tuning demands.STAR instead models infinitely many classifiers; CLAN weights pixels differently in semantic segmentation without explicitly incorporating classes.Source: Related Work · p. 2
  • BCDM addresses neglected classifier determinacy by measuring discrepancy through the relevance of different class predictions, and experiments report gains over competing baselines.Prior methods may allow decision boundaries through high-density regions and class misalignment.Source: Related Work · p. 2

Bi-Classifier Determinacy Maximization

BCDM uses CDD to measure disagreement across class predictions while accounting for their relevance and determinacy. Its adversarial training alternates classifier-side CDD maximization with generator-side minimization, and the paper develops a generalization bound based on determinacy-disparity discrepancy.

Classifier Determinacy Disparity

  • CDD uses cross-class probability products to measure prediction discrepancy, capturing class relevance that same-position ℓ1 comparison overlooks.The paper describes each matrix entry as the product of the two classifiers’ probabilities for the corresponding classes.Source: Classifier Determinacy Disparity · p. 3
  • CDD reaches zero only when the two predictions are identical and fully determined, such as matching one-hot outputs.The paper states that this property guarantees feature discriminability.Source: Classifier Determinacy Disparity · p. 3

Bi-Classifier Determinacy Maximization

  • BCDM first trains the whole network with source supervision, then uses target-domain CDD in an adversarial classifier–generator update.The classifiers maximize CDD; the generator minimizes it while the classifiers are fixed.Source: Bi-Classifier Determinacy Maximization · pp. 3–4
  • Classifier-side CDD maximization encourages diverse class predictions, while generator-side minimization seeks consistent, high-certainty target predictions.The paper links this minimization to pushing target distributions away from the decision boundary and generating discriminative representations.Source: Bi-Classifier Determinacy Maximization · p. 4

Generalization Bound of BCDM

  • The Determinacy Disparity Discrepancy measures distribution differences using CDD and depends on the hypothesis space and a specific classifier.The paper states that this dependence makes it easier to optimize than H∆H-distance.Source: Generalization Bound of BCDM · p. 4
  • The paper derives a target-domain generalization bound and states that BCDM training minimizes its major terms to improve target-domain generalization.The account identifies source error minimization followed by a min-max process for the determinacy-disparity difference.Source: Generalization Bound of BCDM · p. 5

Experiment

BCDM is evaluated across image-classification and semantic-segmentation adaptation benchmarks, with additional analyses of its decision boundaries and feature representations. The reported results favor BCDM across the benchmark settings and show improved target alignment in the analyses.

Dataset and Setup / Implementation Details

  • The evaluation covers four domain-adaptation datasets and two segmentation datasets, using classification accuracy and IoU or mIoU for the respective tasks.Image classification uses pretrained ResNet-50/101; semantic segmentation uses DeepLabv2 with ResNet-101, and training schedules differ by task.Source: Dataset and Setup · p. 5, Implementation Details · pp. 5–6

Experimental Results

  • On DomainNet, BCDM significantly outperforms comparison methods in final mean accuracy; on VisDA-2017, it outperforms other approaches on 7 of 12 tasks and gains 31.0% over source-only ResNet-101.It also gains 11.5% over MCD and 7.0% over SWD on VisDA-2017.Source: Experimental Results · p. 6, Table 1 · p. 6, Table 2 · p. 7
  • On Office-31 and ImageCLEF, BCDM achieves the best average accuracies of 89.0% and 89.3%; on GTA5 →Cityscapes, it exceeds NoAdaptation by 10.0%.The segmentation comparison reports mIoU on the Cityscapes validation set.Source: Experimental Results · p. 6, Table 3 · p. 7, Table 4 · p. 7

Insight Analysis

  • On rotated two-moons target data, BCDM achieves the best target matching among the compared boundaries and classifies both classes correctly.The target distribution is generated by rotating the source distribution by 30 degrees.Source: Insight Analysis · pp. 6, 8, Figure 4 · p. 8
  • As an additive module to STAR, CDD performs best against ℓ1 and SWD and improves the baseline by up to 1.4%.Source: Insight Analysis · p. 8, Table 5 · p. 8
  • For task A →W, BCDM has the smallest proxy A-distance among compared approaches and reaches comparable convergence with smaller test errors.Source: Insight Analysis · p. 8
  • The feature SVD analysis shows BCDM retains high singular values while reducing the gap between the largest and remaining values; t-SNE shows clearer class boundaries than MCD.The authors interpret the SVD pattern as indicating that dimensions associated with smaller singular values positively influence classification.Source: Insight Analysis · p. 8

Conclusion

The paper presents BCDM and CDD as a way to connect classifier determinacy and representation discriminability with transferability. It reports favorable results across image classification and semantic segmentation UDA scenarios.

  • BCDM outperforms state-of-the-art methods across a wide range of UDA scenarios, including image classification and semantic segmentation.Source: Conclusion · pp. 8–9
  • The paper introduces CDD to evaluate discrepancy across distinct target predictions and uses it to adversarially optimize BCDM.Source: Conclusion · p. 8
  • The paper investigates representation discriminability and classifier determinacy as factors that imply greater transferability in UDA.Source: Conclusion · p. 8

Ethics Statement

The authors describe automated annotation for unseen domains as having potential benefits and risks. They identify annotation-cost savings alongside possible job displacement, misleading outputs, and reliance on data biases.

  • Automated annotation for unseen domains may save annotation time and cost and improve efficiency.Source: Ethics Statement · p. 9
  • The authors note that more classification or object-detection jobs for rare or variable conditions may be canceled.Source: Ethics Statement · p. 9
  • System failures could lead people to believe classification is unbiased when it may not be, and the work leverages data biases.Source: Ethics Statement · p. 9

Supplementary Materials

The supplementary material characterizes CDD and the induced distribution disparity as pseudometrics, then gives a target-error bound that includes empirical source error, empirical source–target disparity, and a joint-error term.

Properties of CDD

Generalization Bounds with CDD

  • With probability at least 1 − 3δ, target error is bounded by empirical source error, empirical source–target disparity, and λ, the minimum combined source and target errors.Source: Generalization Bounds with CDD · p. 12
Loading 2012.06995v1…