PaperlayerView exact revision on arXiv

Source-linked AI summary

Two-phase Pseudo Label Densification for Self-training based Domain Adaptation

Inkyu Shin, Sanghyun Woo, Fei Pan, InSo Kweon

arXiv:2012.04828v1cs.CV

TL;DR

Self-training for UDA can produce sparse target pseudo-labels, while lowering selection thresholds risks noisy predictions. TPLD gradually densifies labels through voting and easy-hard classification, with bootstrapping in the self-training loss. Combined with CRST, it reports new state-of-the-art results on two standard UDA benchmarks.

  • Problem

    Self-training selects confident target predictions as pseudo-labels, leaving sparse labels; simply lowering the threshold risks accumulating and propagating early noisy predictions.

    Source: §1 · p. 2

  • Method

    TPLD uses sliding-window voting, confidence-based easy-hard classification with full pseudo-labels for easy samples and adversarial alignment for hard ones, and bootstrapping in the self-training loss.

    Source: §1 · p. 2

  • Results

    Combined with CRST, TPLD achieves new state-of-the-art results on two standard UDA benchmarks.

    Source: §6 · p. 14

  • Takeaways & Limitations

    TPLD is reported to integrate with existing self-training approaches and improve their performance.

    Source: §1 · pp. 2–3

    The setting assumes labeled source-domain data and uses target-domain data without target labels.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

Recently, deep self-training approaches emerged as a powerful solution to the unsupervised domain adaptation. The self-training scheme involves iterative processing of target data; it generates target pseudo labels and retrains the network. However, since only the confident predictions are taken as pseudo labels, existing self-training approaches inevitably produce sparse pseudo labels in practice. We see this is critical because the resulting insufficient training-signals lead to a suboptimal, error-prone model. In order to tackle this problem, we propose a novel Two-phase Pseudo Label Densification framework, referred to as TPLD. In the first phase, we use sliding window voting to propagate the confident predictions, utilizing intrinsic spatial-correlations in the images. In the second phase, we perform a confidence-based easy-hard classification. For the easy samples, we now employ their full pseudo labels. For the hard ones, we instead adopt adversarial learning to enforce hard-to-easy feature alignment. To ease the training process and avoid noisy predictions, we introduce the bootstrapping mechanism to the original self-training loss. We show the proposed TPLD can be easily integrated into existing self-training based approaches and improves the performance significantly. Combined with the recently proposed CRST self-training framework, we achieve new state-of-the-art results on two standard UDA benchmarks.

1 Introduction

Self-training for UDA can leave sparse pseudo labels, while simply lowering the selection threshold risks accumulating noisy predictions. TPLD gradually densifies labels in two phases and reports new state-of-the-art results on two standard UDA benchmarks.

1 Introduction

  • Sparse pseudo labels omit meaningful target training signals, while lowering the selection threshold can introduce early noisy predictions that accumulate and propagate errors.The authors argue that these effects can move the final model away from the optimal solution.Source: §1 · p. 2
  • TPLD propagates confident predictions with sliding-window voting, then classifies images as easy or hard using an image-level confidence score.Easy samples use full pseudo labels; hard samples receive adversarial learning for hard-to-easy feature alignment.Source: §1 · p. 2
  • The authors describe pseudo-label densification as formally defined and explored for self-training-based domain adaptation for the first time, to their knowledge.Source: §1 · p. 2
  • The paper introduces bootstrapping into the self-training loss to address noisy labels during both phases.Source: §1 · pp. 2–3
  • TPLD achieves new state-of-the-art results on two standard UDA benchmarks and is applied to existing self-training approaches.The paper reports extensive ablations and application to various self-training approaches.Source: §1 · p. 3

2 Related works

Related work contrasts adversarial alignment with self-training approaches that use target predictions as pseudo-labels. The paper identifies sparse pseudo-labels as a limitation and proposes gradual densification with a loss aimed at noisy labels.

2 Related works

  • Adversarial UDA aligns source and target feature distributions while retaining source-task performance, whereas self-training uses target pseudo-labels to learn from target data.Source: §2 · p. 3, Self-training · p. 3
  • CBST adds class balancing and spatial priors, while CRST adds loss regularization to discourage overconfident predictions.Source: Self-training · p. 3
  • The paper characterizes sparse pseudo-labels as a fundamental self-training limitation and proposes gradual densification plus a loss for noisy labels.It reports that these proposals improve strong state-of-the-art baselines with healthy margins.Source: Self-training · p. 3

3 Preliminaries

The preliminaries set out the labeled-source, unlabeled-target UDA setting and review self-training with pseudo-label generation and network retraining. They also describe bootstrapping loss as a means of handling noisy predictions.

3.1 Problem Setting

  • The UDA setting provides labeled source images and labels, but only images from the target domain.Source: §3.1 · p. 4

3.2 Self-training for UDA

  • The reviewed self-training objective combines source-label training, target pseudo-label training, and confidence regularization; its first two terms match CBST and the added regularizer gives CRST.Source: §3.2 · p. 4
  • Self-training alternates pseudo-label generation and network retraining, selecting a predicted class when its probability exceeds a class-specific threshold and otherwise assigning a zero vector.The selection parameter is initially kept low and gradually increased across rounds to avoid early unconfident predictions.Source: §3.2 · pp. 4–5
  • The paper characterizes sparse pseudo-labels as limiting representations and potentially producing a suboptimal model, motivating gradual densification while avoiding noisy predictions.It states that TPLD can be applied to existing self-training frameworks and consistently boost performance.Source: §3.2 · p. 5

3.3 Noisy label handling

  • Bootstrapping loss is a weighted sum of standard cross-entropy and self-entropy, and is applied to self-training to handle noisy predictions.The paper says it encourages correct pseudo-target predictions and confidence in those predictions.Source: §3.3 · p. 5

4 Method

TPLD densifies target pseudo labels in two phases: sliding-window voting first propagates confident predictions, then image-level easy-hard classification guides full-label training and hard-to-easy adversarial alignment. Bootstrapped self-training loss is used in both phases.

4.1 1st phase: Voting based Densification

  • Sliding-window voting pools neighboring confident predictions for the top two competing classes, combines them with the original values, and assigns a pseudo label when the threshold is passed.The process is repeated across the image; the method uses three iterations with a 57 × 57 window.Source: §4.1 · p. 6, Figure 2 · p. 7
  • Voting iterations make sparse pseudo labels denser but can introduce noisy predictions, motivating a second phase for full-pseudo-label training.Bootstrapping is introduced to train under noisy pseudo labels, and both phases use the proposed bootstrapped self-training loss.Source: §4 · p. 5, §4.1 · p. 7, §4.2 · p. 8

4.2 2nd phase: Easy-Hard Classification based Densification

  • An image-level confidence score classifies target images as easy or hard, with the top q portion selected as easy and q initially set to 30%.The score averages per-class confident-prediction counts with a class-threshold adjustment intended to encourage sampling images with rare classes.Source: §4.2 · p. 8
  • Easy samples use full pseudo-label predictions with bootstrapping loss, while hard samples use adversarial learning to align their features toward easy samples.The intra-domain discriminator distinguishes easy from hard samples using target weighted self-information maps.Source: §4.2 · p. 8, Hard sample training · p. 9

5 Experiments

TPLD improves results on GTA5→Cityscapes and SYNTHIA→Cityscapes and consistently improves existing self-training approaches. Ablations examine its two densification phases, confidence scoring, hyperparameters, and training losses.

5.1 Dataset

  • Evaluation adapts models from GTA5 or SYNTHIA to Cityscapes and measures performance on the Cityscapes validation set.GTA5 contains 24,966 synthetic images and SYNTHIA contains 9,400.Source: §5.1 · p. 10

5.2 Implementation details

  • The CRST-MRKLD experiments use VGG-16 or ResNet-101 backbones, DeepLab-v2 or DeepLab-v3, and nine training rounds split six for phase one and three for phase two.Models are ImageNet-pretrained and fine-tuned on source images.Source: §5.2 · p. 10

5.3 Main Results

5.4 Ablation study

  • Lowering CRST's selection threshold yields 47.0→47.1 mIoU at p=0.4 or 47.0→45.7 at p=0.6, while TPLD reaches 47.0→51.2 mIoU.Removing voting changes performance from 51.2 to 38.1 mIoU, removing easy-hard classification changes it from 51.2 to 49.5, and easy-hard-first yields 49.1 versus 51.2 mIoU for voting-first.Source: §5.4 · p. 11, Framework design choices · p. 12
  • Weighting confidence by 1 λk raises mIoU from 50.5 to 51.2 and R-mIoU from 33.7 to 35.1.The stated rationale is to oversample images containing rare classes rather than favor images composed of frequent classes.Source: Framework design choices · p. 12

5.6 Loss function analysis

  • Bootstrapping raises mIoU from 47.00 to 48.47, voting to 49.52, easy-sample training to 50.11, and hard-sample training to 51.20.The paper reports that hard-sample training improves category-level feature alignment and prediction cleanliness.Source: Table 5 · p. 13, §5.6 · pp. 13–14, Figure 6 · p. 14

5.5 Parameter analysis

  • Parameter analysis selects a 57-pixel voting field, three voting iterations, α=0.7, q=0.3, and γ=2; setting q=1 degrades performance.q controls the portion of target images treated as easy samples; γ calibrates prediction values for full pseudo-label generation.Source: §5.5 · p. 13

6 Conclusions

Self-training for UDA suffers from sparse pseudo-labels; the paper proposes two-phase pseudo-label densification and reports new state-of-the-art results with CRST on two UDA benchmarks.

6 Conclusions

  • Self-training methods for UDA suffer from sparse pseudo-labels during training.The paper identifies this as the problem motivating its approach.Source: §6 · p. 14
  • The paper presents a two-phase pseudo-label densification method for self-training.The method is referred to as TPLD elsewhere in the paper context.Source: §6 · p. 14
  • Combined with CRST, the method achieves new state-of-the-art results on UDA benchmarks.The conclusion describes the benchmarks as plural.Source: §6 · p. 14

Related papers

Loading 2012.04828v1…