PaperlayerView exact revision on arXiv

Source-linked AI summary

Confidence Regularized Self-Training

Yang Zou, Zhiding Yu, Xiaofeng Liu, B. V. K. Vijaya Kumar, Jinsong Wang

arXiv:1908.09822v3cs.CVcs.LGcs.MMcs.RO

TL;DR

Domain adaptation suffers when cross-domain differences degrade a source model and hard pseudo-labels introduce overconfident mistakes or propagated errors. CRST jointly optimizes continuous pseudo-labels and network parameters with label or model confidence regularization, and experiments across image classification and semantic segmentation report state-of-the-art performance.

  • Problem

    Cross-domain differences reduce model performance, while noisy hard pseudo-labels can produce overconfident mistakes and propagated errors during self-training.

  • Method

    CRST formulates self-training as alternating optimization over continuous pseudo-labels and network parameters, using label regularization or model output-smoothing regularization.

  • Results

    CRST achieves state-of-the-art performance across image-classification and semantic-segmentation domain-adaptation tasks.

  • Takeaways & Limitations

    Confidence regularization acts as a safety measure against infinite entropy minimization and degraded performance in self-training.

Abstract

from arXiv · show

Recent advances in domain adaptation show that deep self-training presents a powerful means for unsupervised domain adaptation. These methods often involve an iterative process of predicting on target domain and then taking the confident predictions as pseudo-labels for retraining. However, since pseudo-labels can be noisy, self-training can put overconfident label belief on wrong classes, leading to deviated solutions with propagated errors. To address the problem, we propose a confidence regularized self-training (CRST) framework, formulated as regularized self-training. Our method treats pseudo-labels as continuous latent variables jointly optimized via alternating optimization. We propose two types of confidence regularization: label regularization (LR) and model regularization (MR). CRST-LR generates soft pseudo-labels while CRST-MR encourages the smoothness on network output. Extensive experiments on image classification and semantic segmentation show that CRSTs outperform their non-regularized counterpart with state-of-the-art performance. The code and models of this work are available at https://github.com/yzou2/CRST.

1. Introduction

Unsupervised domain adaptation uses unlabeled target data to address performance loss under domain shifts, while deep self-training risks propagating errors from overconfident pseudo-labels. CRST addresses this through confidence regularization, with label- and model-regularized variants evaluated across adaptation tasks.

  • Motivation: Unsupervised domain adaptation adapts source models to target domains using unlabeled target data when cross-domain differences reduce performance.The paper motivates UDA with examples including synthetic-data training and robotics or autonomous-driving simulation.
  • Motivation: Deep self-training selects high-confidence target predictions as hard pseudo-labels and retrains networks iteratively.CBST formulates this process as unified loss minimization with pseudo-labels.
  • Problem: Hard pseudo-labels can encode overconfident mistakes and propagated errors, especially when semantic labels are ambiguous.Forcing confidence in only one class can hurt learning in under-determined UDA settings.
  • Proposed approach: CRST prevents infinite entropy minimization by redistributing confidence through soft pseudo-labels or directly smoothing network outputs.These mechanisms are intended to attenuate misleading effects from incorrect or ambiguous supervision.
  • Contributions: The framework introduces label-regularized and model-regularized self-training and evaluates multiple regularizers on image-classification and semantic-segmentation adaptation tasks.The study includes image classification on VisDA17 and Office-31 and semantic segmentation from GTA5 or SYNTHIA to Cityscapes.

2. Related works

Related work situates the paper at the intersection of deep self-training and unsupervised domain adaptation. These approaches differ in whether they learn flexible embeddings or reduce cross-domain feature-distribution differences.

  • Self-training: Self-training is widely studied in semi-supervised learning and was revitalized by deep neural networks.Deep self-training learns embeddings, providing greater flexibility for domain alignment than classifier-level adaptation.
  • Domain adaptation: Unsupervised domain adaptation commonly learns domain-invariant embeddings by minimizing differences between source and target feature distributions.Related criteria include MMD, CORAL, sliced Wasserstein discrepancy, and adversarial learning at input, feature, or output levels.

3. Continuous class-balanced self-training

Continuous CBST relaxes discrete pseudo-labels to a continuous feasible space and alternates pseudo-label generation with network retraining. Confidence thresholds still govern class-balanced target-sample selection.

  • Continuous formulation: CBST jointly learns network parameters and discrete pseudo-labels, which are either one-hot vectors or all-zero vectors.The continuous formulation replaces this discrete pseudo-label space with a probability simplex together with the zero vector.
  • Continuous formulation: Continuous CBST relaxes pseudo-label variables from one-hot vectors to continuous probability vectors in the probability simplex.The feasible set is the union of the zero vector and the probability simplex ΔK−1.
  • Alternating optimization: One self-training round alternates pseudo-label generation with network retraining while fixing network weights or pseudo-labels in the respective step.Pseudo-label generation and network retraining are solved in alternating optimization steps.
  • Alternating optimization: Pseudo-label generation simultaneously performs pseudo-label learning and target-sample selection.The network is retrained using typical gradient-based methods after pseudo-labels are fixed.
  • Class-balanced selection: A target sample is selected when its most probable class exceeds the corresponding class threshold; otherwise it is not selected.Selected samples receive the argmax class as their one-hot pseudo-label.
  • Class-balanced selection: Class-balanced thresholds select the most confident p portion of predictions for each class across the target set.The sample confidence is defined as the maximum output softmax probability.

4. Confidence regularized self-training

CRST extends self-training with confidence regularization, alternating pseudo-label generation and network retraining. It includes label regularization with soft pseudo-labels and model regularization with output smoothing.

  • General CRST: CRST alternates pseudo-label generation with network retraining while adding a confidence regularizer weighted by α.The regularizer can depend on pseudo-labels or network outputs.
  • Label regularization: CRST-LR optimizes continuous pseudo-labels and uses label regularization to prefer smoother selections over sparse pseudo-labels.Its pseudo-label generation is solved in two stages: optimize within the probability simplex, then choose between that solution and zero.
  • Label regularization: LRENT is a negative-entropy label regularizer whose pseudo-label solution is listed with the proposed regularizers in Table 1.The table also reports corresponding pseudo-label solutions or gradients with respect to softmax logits.
  • Model regularization: CRST-MR retains hard pseudo-label generation but regularizes network retraining with cross-entropy plus an output-smoothing term.Its model regularizer operates on the network softmax output probabilities.
  • Model regularization: Three CRST-MR regularizers are based on L2, negative entropy, and KLD between the uniform distribution and softmax output.Table 1 provides their gradients with respect to softmax logits; H(p) denotes entropy.

5. Theoretical properties

The paper gives CRST an expectation-maximization interpretation and establishes convergence under conditions. It also connects LRENT and model regularization to familiar softmax and smoothing formulations.

  • Probabilistic interpretation: CRST is modeled as a regularized classification maximum-likelihood problem optimized through classification expectation maximization.The paper states this as Proposition 1 and refers to an appendix proof.
  • Convergence: Given predetermined λk, CRST is convergent under certain conditions.This result is stated as Proposition 2, with the proof deferred to the appendix.
  • Soft pseudo-label connection: For α →∞, temperature-scaled softmax approaches a uniform distribution, while α = 1 recovers ordinary softmax probabilities.The temperature controls distribution smoothness and uncertainty.
  • Soft pseudo-label connection: For α →0, temperature-scaled softmax collapses to a sparse one-hot vector concentrated on the class with the largest original softmax probability.This describes the low-temperature limit of the distribution.
  • Soft pseudo-label connection: When all λk are equal, LRENT's soft pseudo-label exactly matches softmax with temperature.The paper further characterizes LRENT as a generalized temperature-scaled softmax when λk vary.
  • Regularizer connections: KLD model confidence regularization is equivalent to uniformly smoothing pseudo-labels by ϵ = (Kα − α)/(K + Kα).Here α is the regularizer weight.
  • Regularizer connections: The reverse KLD regularizer DKL(p(xt)||u) is equivalent to the entropy model regularizer −H(p(xt)).The uniform distribution is denoted by u.

6. Experiments

Experiments across image classification and semantic segmentation compare CRST variants with CBST and other methods, while analyses examine sensitivity, confidence behavior, computational costs, and regularizer properties.

  • Experimental setup: CRST variants are evaluated on VisDA17, Office-31, GTA5 →Cityscapes, and SYNTHIA →Cityscapes using standard adaptation protocols and matched backbones.VisDA17 and Office-31 cover image classification; Cityscapes tasks cover semantic segmentation.
  • Image classification: MRKLD and LRENT outperform non-regularized CBST on VisDA17, while their combination further outperforms single regularizers and other recent methods.The combined method also exceeds certain methods using stronger ResNet-152 backbones.
  • Image classification: All CRSTs outperform CBST on Office-31, and MRKLD+LRENT achieves comparable or better performance than other recent methods.The comparison uses the same ResNet-50 backbone.
  • Semantic segmentation: MRKLD achieves the best result on GTA5 →Cityscapes with DeepLabv2 and outperforms previous state-of-the-art; all CRSTs outperform CBST with Wide ResNet-38.With spatial priors and multi-scale testing, the system reaches state-of-the-art performance.
  • Semantic segmentation: On SYNTHIA →Cityscapes, CRSTs perform on par with or better than CBST, while MRKLD remains the best regularizer and outperforms previous state-of-the-art.The result is reported in the semantic-segmentation experiments.
  • Analysis and practical considerations: CRSTs are not sensitive to the starting or incremental pseudo-label portion, nor to α within certain intervals.The regularizer weight α balances self-training loss against confidence regularization; model regularizers add negligible gradient-computation cost, whereas label regularizers require dataset-level soft-label storage.
  • Analysis and practical considerations: Confidence regularization lowers confidence for both true and false positives, but improves false-positive discrimination through higher TP/FP confidence ratios on almost all classes.This comparison is made between CBST and MRKLD/LRENT on GTA5 →Cityscapes after the first self-training round.
  • Regularizer properties: LR preserves inter-class confidence ranking, whereas MRs tend to equalize negative-class confidences; MRKLD’s minimizer uniformly smooths negative classes.LRENT preserves class ordering, while MRKLD does not; combining MR and LR can improve performance at the cost of additional hyperparameter tuning.

8. Conclusions

The conclusion presents CRST as regularized self-training with model and label regularizers, supported by theoretical analysis, broad experiments, and practical guidance for future UDA/SSL research.

  • 8. Conclusions: CRST formulates self-training as regularized loss minimization and includes model and label regularization with proposed confidence regularizers.The paper also analyzes probabilistic properties and connections to softmax with temperature.
  • 8. Conclusions: Comprehensive experiments demonstrate CRST effectiveness with state-of-the-art performance, while the paper discusses regularizer trade-offs and practical choices.The authors suggest that regularization designs may provide useful inductive biases for UDA and SSL.

A. Derivation of soft pseudo-label in LRENT

The LRENT derivation frames soft pseudo-label learning as a regularized optimization problem and connects CRST's alternating procedure to classification expectation maximization.

  • LRENT formulates soft pseudo-label learning as an entropy-regularized optimization problem.
  • The resulting CRST self-training algorithm is an instance of classification expectation maximization.
  • CRST alternates posterior estimation, pseudo-label optimization, and network-weight optimization in an E-step, C-step, and M-step.

B.2. Proof of Proposition 2

The proof establishes convergence of alternating CRST optimization by showing that both pseudo-label learning and network retraining decrease their respective objectives under stated conditions.

  • Pseudo-label learning is globally minimizing and makes the self-training objective non-increasing.The argument assumes α ≥ 0 and convexity of the regularizer with respect to w and ˆy_t.
  • The framework also covers continuous CBST as the special case α = 0.
  • With a proper learning rate, gradient descent makes the network-retraining loss decrease monotonically.Mini-batch gradient descent may not strictly guarantee monotonic decrease, though it will almost certainly converge to a lower loss in practice.
  • Because the self-training loss is lower bounded, alternating pseudo-label learning and network retraining are convergent.

C. Additional details on experiments

Additional experiments examine optimization behavior, feature alignment, class confusions, prediction confidence, and qualitative outputs across VisDA17 and GTA5 →Cityscapes.

  • MRKLD+LRENT gives the best VisDA17 performance and consistently improves over the CBST baseline.Mean accuracy is averaged over five runs, and the proposed methods are generally stable with slight fluctuations after 10 epochs.
  • Both CBST and MRKLD+LRENT improve class-wise feature alignment over the source model, with MRKLD+LRENT showing slightly more accurate alignment.
  • MRKLD+LRENT produces more diagonalized confusion matrices and fewer mistakes than the source model, reducing confusions such as person versus horse and motor versus bike.
  • Confidence regularization produces softer target-domain softmax distributions by reducing highly confident probability entries.The distributions are evaluated with a ResNet-38 backbone on GTA5 →Cityscapes.
  • Figures 10 and 11 compare segmentation predictions and pseudo-label maps for GTA5 →Cityscapes models on sampled Cityscapes images.Figure 10 uses validation images for predictions, while Figure 11 uses training images at the beginning of the second self-training round.
Loading 1908.09822v3…