Source-linked AI summary
Instance Adaptive Self-Training for Unsupervised Domain Adaptation
Ke Mei, Chuang Zhu, Jiaqi Zou, Shanghang Zhang
TL;DR
UDA addresses performance drops caused by divergence between labeled source and unlabeled target data, while existing methods struggle to balance scalability and performance. IAST uses instance-adaptive pseudo-label selection and region-guided regularization for semantic segmentation UDA, achieving improved benchmark performance and easy integration with other UDA methods.
Problem
UDA must address performance degradation caused by divergence between labeled source and unlabeled target domains, while existing methods struggle to balance scalability and performance.
Method
IAST combines an instance adaptive selector for pseudo-label generation with region-guided regularization for pseudo-label and non-pseudo-label regions.
Results
IAST reaches 51.5% mIoU in ablation experiments, while combining it with multi-scale testing reaches 52.2% mIoU.
Takeaways & Limitations
IAST can be applied to other UDA methods with almost no additional cost and can also extend to semi-supervised semantic segmentation.
Takeaways & Limitations
The UDA setting assumes labeled source images and semantic masks but only unlabeled target images.
Abstract
from arXiv · showhide
The divergence between labeled training data and unlabeled testing data is a significant challenge for recent deep learning models. Unsupervised domain adaptation (UDA) attempts to solve such a problem. Recent works show that self-training is a powerful approach to UDA. However, existing methods have difficulty in balancing scalability and performance. In this paper, we propose an instance adaptive self-training framework for UDA on the task of semantic segmentation. To effectively improve the quality of pseudo-labels, we develop a novel pseudo-label generation strategy with an instance adaptive selector. Besides, we propose the region-guided regularization to smooth the pseudo-label region and sharpen the non-pseudo-label region. Our method is so concise and efficient that it is easy to be generalized to other unsupervised domain adaptation methods. Experiments on 'GTA5 to Cityscapes' and 'SYNTHIA to Cityscapes' demonstrate the superior performance of our approach compared with the state-of-the-art methods.
1 Introduction
Unsupervised domain adaptation addresses performance drops caused by source–target domain shifts, but existing approaches trade off performance against scalability. IAST targets this gap with adaptive pseudo-label selection and region-guided regularization.
- Motivation: Domain shifts between labeled source and unlabeled target data can cause significant target-domain performance drops, motivating UDA.UDA transfers knowledge from labeled source data to improve generalization on the unlabeled target domain.
- Motivation: AT + ST reaches 49.0 on GTA5 to Cityscapes, compared with 47.8 for ST and 43.7 for AT.Mixed methods gain +4.2 and +5.6 after adding self-training, but generally introduce coupling between submodules.
- Motivation: Existing self-training can discard learnable information because confidence-based selection favors easy classes and misses hard-image content.CBST pseudo-labels concentrate on roads while ignoring pedestrians and trucks.
- Proposed Framework: IAST is an instance adaptive self-training framework that improves pseudo-label diversity through an instance adaptive selector.The framework is designed for semantic segmentation UDA and is shown in Fig. 2.
- Proposed Framework: Region-guided regularization smooths prediction in pseudo-label regions and sharpens prediction in non-pseudo-label regions.The method is intended to reduce the impact of noisy pseudo-labels during self-training.
- Contributions: The framework is designed to be applied easily to other non-self-training UDA methods and extended to semi-supervised semantic segmentation.The authors describe it as having no model or special-structure dependency.
2 Related Works
Related work covers adversarial training, self-training, and regularization for UDA. It highlights self-training’s bias toward easy classes and prior regularization limited to pseudo-label regions.
- Adversarial Training: Adversarial training methods reduce domain gaps by aligning source and target feature distributions with adversarial losses.Recent approaches commonly add a domain discriminator to minimize domain adversarial loss.
- Self-Training: Self-training iteratively combines labeled source data with generated target pseudo-labels to align source and target domains.These schemes are also commonly used in semi-supervised learning.
- Self-Training: Directly selecting high-confidence pseudo-labels biases models toward easy classes and can harm transfer for hard classes.Class-balanced self-training was proposed to address this problem.
- Regularization: Regularization aims to reduce testing error and improve generalization, but prior UDA regularization was applied only to pseudo-label regions.Examples of general regularization include weight decay and label smoothing.
3 Preliminary
UDA transfers knowledge from labeled source data to unlabeled target data for semantic segmentation, where self-training uses generated target pseudo-labels alongside source supervision. Adversarial training instead aligns source and target feature distributions through a discriminator.
- UDA setup: UDA trains semantic segmentation models with labeled source data while adapting them to an unlabeled target domain.The target domain lacks ground-truth labels, so the task uses source supervision and target-domain adaptation.
- Self-training: Self-training treats target images as an unlabeled dataset and uses generated pseudo-labels as approximate target ground truth.Pseudo-labels are generated before training, used in cross-entropy optimization, and periodically updated.
- Self-training: Pseudo-labels are one-hot class assignments or all-zero vectors, depending on whether a target prediction satisfies the confidence criterion.The supplied passages describe these labels as approximate target ground truth.
- Adversarial training: Adversarial training adds a discriminator that distinguishes source and target output-space feature distributions while the segmentation model learns to confuse them.The discriminator and segmentation model are optimized alternately using source cross-entropy and an adversarial loss.
4 Proposed Method
IAST combines instance-adaptive pseudo-label selection with region-guided regularization in a three-phase self-training framework. It uses adaptive thresholds and regularizes confident and ignored regions differently to reduce pseudo-label noise and improve target-domain training.
- Framework: IAST combines an instance adaptive selector with region-guided regularization to improve pseudo-label generation and target-domain self-training.The framework is designed as a concise component that can be applied to other UDA methods.
- Framework phases: IAST training has three phases: warm-up initialization, instance-adaptive pseudo-label generation, and self-training on target data.The warm-up model also serves as the initial pseudo-label generator.
- Instance adaptive selector: Compared with constant or class-balanced thresholds, IAS adaptively adjusts each class threshold according to the current instance.The thresholding illustration contrasts one shared threshold, class-balanced thresholds, and instance-adaptive thresholds.
- Pseudo-label generation: The pseudo-label generator retains a pixel when its predicted class is the maximum-probability class and its probability exceeds the class threshold.Otherwise, the corresponding target label can be represented as all-zero or ignored.
- Instance adaptive selector: The selector maintains historical and current thresholds, combines global and local information, and updates thresholds with an exponentially weighted moving average.For each instance and class, a local threshold is obtained from sorted confidence probabilities before the historical threshold is updated.
- Hard-class weight decay: Hard-class weight decay reduces pseudo-label proportions for hard classes more strongly, while affecting easy classes more weakly.The method uses γ as a decay-control parameter; larger historical thresholds for easy classes make the adjustment weaker.
- Region-guided regularization: Region-guided regularization smooths confident pseudo-label regions with KLD minimization and sharpens ignored regions through entropy minimization.These terms address noise in supervised pseudo-label regions and extract useful features from ignored regions without direct supervision.
- Region-guided regularization: Entropy minimization sharpens ignored-region predictions and can promote learning useful features without supervised signals.The ignored region has no direct supervision because it contains pixels excluded from pseudo-label supervision.
5 Experiment
Experiments show that instance-adaptive self-training improves pseudo-label diversity and quality, reaches strong semantic-segmentation UDA performance, and transfers to other UDA and semi-supervised settings.
- Discussion and Ablation Study: IAST combines global and local information, producing more diverse pseudo-labels than the class-balanced method, especially for hard classes.With α = 0.2 and β = 0.9, the method achieves its best performance.
- Discussion and Ablation Study: As γ increases, pseudo-label proportions decrease sharply for hard classes while remaining relatively stable for easy classes, reducing noise in pseudo-labels.The authors select γ = 8 as a trade-off between pseudo-label proportion and quality.
- Discussion and Ablation Study: 51.5% mIoU is achieved after adding the IAST modules, while multi-scale testing raises the combined result to 52.2% mIoU.Self-training alone provides a 1.3% performance gain, followed by steady improvements from IAS, R_i, and R_c.
- Experimental Results: 52.2% mIoU is the best result on GTA5 to Cityscapes, exceeding AdaptSegNet and SIBAN by 9.6% mIoU and MRKLD by 4.8% mIoU.BLF is the second-best method at 48.5% mIoU.
- Experimental Results: 49.8% mIoU and 57.0% mIoU* are the best results on SYNTHIA to Cityscapes and are significantly higher than recent state-of-the-art methods.The study reports both 13-class mIoU* and 16-class mIoU because the domain gap is larger in this transfer setting.
- Experimental Results: IAST improves two adversarial-training UDA methods and also extends to semi-supervised semantic segmentation.The framework is applied to AdaptSeg and AdvEnt without special structure or model dependencies.
6 Conclusions
The paper proposes IAST for semantic-segmentation UDA and reports significant performance improvements over popular UDA methods. Its lack of model or special-structure dependencies supports application to other UDA methods and semi-supervised segmentation.
- Conclusions: IAST is an instance adaptive self-training framework for semantic segmentation UDA that improves performance over other popular UDA methods.The conclusion presents IAST as the paper’s central contribution and performance result.
- Conclusions: Because IAST has no model or special-structure dependency, it can be applied to other UDA methods with almost no additional cost.The conclusion also reports state-of-the-art performance when applied to semi-supervised semantic segmentation.