PaperlayerView exact revision on arXiv

Source-linked AI summary

Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images

Lefei Zhang, Meng Lan, Jing Zhang, Dacheng Tao

arXiv:2108.12611v1cs.CV

TL;DR

Road segmentation models trained on labeled source imagery can suffer domain shift when applied to unlabeled target imagery. RoadDA addresses this with two-stage adversarial adaptation and self-training, and outperforms the compared UDA methods on the reported benchmark settings.

  • Problem

    Domain shift limits the generalization of road-segmentation models trained on labeled source data when applied to unlabeled target imagery.

  • Method

    RoadDA combines GAN-based inter-domain adaptation, feature pyramid fusion, and iterative adversarial self-training using confidence-ranked target pseudo labels.

  • Results

    RoadDA outperforms the compared UDA methods across two benchmark settings, achieving 74.92% IoU and 85.81% F1 on DeepGlobe → CNDS.

  • Takeaways & Limitations

    The reported results support RoadDA as a promising approach for road segmentation on unlabeled target domains.

Abstract

from arXiv · show

Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. Deep neural networks have advanced this field by leveraging the power of large-scale labeled data, which, however, are extremely expensive and time-consuming to acquire. One solution is to use cheap available data to train a model and deploy it to directly process the data from a specific application domain. Nevertheless, the well-known domain shift (DS) issue prevents the trained model from generalizing well on the target domain. In this paper, we propose a novel stagewise domain adaptation model called RoadDA to address the DS issue in this field. In the first stage, RoadDA adapts the target domain features to align with the source ones via generative adversarial networks (GAN) based inter-domain adaptation. Specifically, a feature pyramid fusion module is devised to avoid information loss of long and thin roads and learn discriminative and robust features. Besides, to address the intra-domain discrepancy in the target domain, in the second stage, we propose an adversarial self-training method. We generate the pseudo labels of target domain using the trained generator and divide it to labeled easy split and unlabeled hard split based on the road confidence scores. The features of hard split are adapted to align with the easy ones using adversarial learning and the intra-domain adaptation process is repeated to progressively improve the segmentation performance. Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.

I. INTRODUCTION

Road segmentation supports practical applications, but models trained on labeled source data can degrade substantially on unlabeled target imagery because of domain shift. RoadDA addresses this problem with stagewise inter-domain adaptation and adversarial self-training.

  • Remote-sensing road segmentation supports vehicle navigation, urban planning, and disaster assistance.
  • Public road datasets and digital maps provide labeled source data, but direct transfer to target images can cause a significant performance drop.
  • Domain shift arises from differences such as road surfaces and rural or urban background areas between training and test images.
  • RoadDA uses inter-domain adaptation followed by adversarial self-training to address source–target and target intra-domain discrepancies.
  • The framework includes a feature pyramid fusion module and progressively updates target pseudo labels during intra-domain adaptation.

A. Supervised Road Segmentation

Prior work established deep learning and UDA approaches for remote-sensing segmentation and classification, while RoadDA combines feature fusion with adversarial self-training for unlabeled target road data.

  • Supervised Road Segmentation: Road-segmentation research evolved from pixel-level classification toward deep CNNs with end-to-end feature learning.
  • Unsupervised Domain Adaptation: UDA uses labeled data from a relevant source domain to perform a similar task in an unlabeled target domain while reducing domain shift.
  • Unsupervised Domain Adaptation: RoadDA combines intra-domain adaptation with self-training to further improve segmentation on unlabeled target data.
  • Unsupervised Domain Adaptation: RoadDA is organized into inter-domain adaptation and adversarial self-training stages.

B. Inter-domain Adaptation

RoadDA performs inter-domain adaptation with a GAN that trains a segmentation generator on labeled source data and aligns target prediction distributions with source predictions.

  • The GAN takes labeled source data and unlabeled target data, with a segmentation model serving as generator G_inter.
  • The discriminator classifies source and target prediction domains, while the generator is trained to fool it.
  • The generator combines supervised source-domain cross-entropy with adversarial target-domain training.
  • Adversarial training forces target predictions toward a source-like distribution, reducing inter-domain discrepancy.
  • During inference, only the adapted segmentation generator is used for road segmentation.

1) Feature Pyramid Fusion Module:

The feature pyramid fusion module addresses spatial information loss in long and thin roads by combining hierarchical ResNet-101 features at a downsampling rate of 8.

  • Feature Pyramid Fusion Module: ResNet-101 extracts hierarchical features C1–C5 at downsampling rates 2, 4, 8, 16, and 32.
  • Feature Pyramid Fusion Module: High downsampling rates reduce feature size and computation but can lose spatial information from long and thin roads.
  • Feature Pyramid Fusion Module: The module upsamples deeper features and fuses them with shallower features to enhance representation at downsampling rate 8.

C. Adversarial Self-training

Adversarial self-training addresses target-domain variation after inter-domain adaptation by dividing target images according to prediction confidence and adapting between easy and hard subsets.

  • Target-domain discrepancies may arise from differing illumination conditions and background context, motivating an adversarial self-training stage.
  • Intra-domain division: The adapted model predicts target segmentation maps, then a quality estimator uses road-pixel confidence to separate easy and hard target subsets.Road pixels are emphasized because background pixels may bias confidence estimation.
  • Intra-domain division: Images are ranked by mean confidence over predicted road pixels, with λ determining the high-confidence easy split and unlabeled hard split sizes.The split sizes are |Xte| = λ |Xt| and |Xth| = (1 − λ) |Xt|.

2) Intra-domain Adaptation:

Intra-domain adaptation aligns the target hard split with the easy split at the output level using adversarial learning.

  • A GAN architecture aligns the hard target split with the easy split to reduce intra-domain discrepancy.
  • The intra-domain generator receives easy and hard images, while the discriminator predicts their domain labels.
  • The discriminator minimizes binary cross-entropy classification loss, while the generator minimizes segmentation and adversarial losses.The losses are alternatively optimized during training.

3) Self-training:

RoadDA repeatedly updates target pseudo labels after intra-domain adaptation, using the improved generator to refine segmentation until performance saturates.

  • After intra-domain adaptation, the improved generator updates target pseudo labels and repeats division and adaptation to progressively improve segmentation.
  • Stage1: Inter-domain adaptation: The training algorithm begins with inter-domain adaptation using labeled source data and unlabeled target data.The inter-domain generator and discriminator are trained in alternating stages.
  • Stage2: Adversarial self-training: Adversarial self-training initializes the intra-domain generator from the inter-domain generator, generates pseudo labels, and divides target data into easy and hard splits.The easy-split ratio is controlled by λ.
  • Stage2: Adversarial self-training: The easy-hard division and adversarial training repeat until the intra-domain generator reaches performance saturation.

D. Implementation Details

The implementation uses an ImageNet-pretrained ResNet-101 generator, a fully convolutional discriminator, and separate optimization procedures for generators and discriminators.

  • The generator uses ImageNet-pretrained ResNet-101, with fused FPFM features upsampled to produce the final segmentation probability map.
  • The discriminator is a fully convolutional network with five convolution layers using 4 × 4 kernels and stride 2.Leaky ReLU with slope 0.2 follows each convolution layer except the last.
  • Generators use SGD with momentum 0.9 and weight decay 10^-4, while discriminators use Adam with momentum values 0.9 and 0.99.Initial learning rates are 4 × 10^-4 for generators and 1 × 10^-4 for discriminators, with polynomial decay power 0.9.

IV. EXPERIMENTS

The experiments evaluate RoadDA for unsupervised road segmentation across datasets with differing resolutions and domains, using preprocessing, augmentation, and four quantitative metrics.

  • Experimental setting: The UDA experiments use Roadtracer and DeepGlobe as source domains and CasNet as the unlabeled target domain.
  • Datasets: DeepGlobe contains 8,570 RGB images at 0.5 m/pixel, while Roadtracer provides 300 high-resolution satellite-image samples at 0.6 m/pixel.
  • Datasets: The CasNet images are at least 600×600 pixels, use 1.2 m/pixel resolution, and contain complex backgrounds and diversified road shapes.
  • Preprocessing: GPU and sample-size constraints motivate cropping and augmentation before training.The preprocessing pipeline crops 20 patches of 512 × 512 pixels per image, filters patches with fewer than 4,000 road pixels, and augments CNDS samples by flipping and 90° rotations.
  • Evaluation metrics: Performance is measured with IoU, completeness, correctness, and F1 score, where larger values indicate better performance.F1 is defined as the harmonic average of completeness and correctness.

D. Comparative Methods

The comparison evaluates RoadDA against source-only, target-only, and several domain-adaptation methods. Across two source-to-CNDS settings, RoadDA achieves the strongest reported quantitative and visual results.

  • Compared methods: The comparison includes AdaptSegNet, ADVENT, BDL, and IntraDA alongside source-only and target-only baselines.
  • DeepGlobe → CNDS: 74.92% IoU and 85.81% F1 score are achieved by RoadDA for DeepGlobe → CNDS, the best results among the reported UDA methods.RoadDA is 15.52% higher than ADVENT in IoU in this setting.
  • Visual comparison: RoadDA obtains the best visual segmentation results on representative CNDS images and captures most road contours across complex backgrounds.
  • RTDS → CNDS: 61.76% IoU and 77.48% F1 score are achieved by RoadDA for RTDS → CNDS, outperforming all reported UDA comparison methods.RoadDA improves IoU by 15.8% over source-only, 7.32% over ADVENT, and 5.7% over IntraDA.

1) Ablation study:

Ablation studies show that RoadDA benefits from multi-level feature fusion, adversarial self-training, repeated adaptation, and output-space domain alignment. Performance tends to saturate after three or four self-training iterations.

  • Feature pyramid fusion: 74.92% IoU is achieved by RoadDA with the proposed feature pyramid fusion setting, outperforming the evaluated feature-fusion variants.Using only the 8× feature reaches 74.17% IoU, indicating the value of higher-resolution spatial information.
  • Adversarial self-training: 13.18% IoU is gained when intra-domain adaptation and self-training are combined over the vanilla baseline.Inter-domain adaptation alone achieves 61.74% IoU and 81.13% F1 score; intra-domain adaptation and self-training contribute 11.9% and 12.34% IoU improvements, respectively.
  • Intra-domain division: RoadDA achieves its best performance when the easy and hard target splits use λ = 0.7.λ controls the number and distribution of samples assigned to the easy and hard splits.
  • Iteration for self-training: Three or four self-training iterations may provide a practical choice because IoU gradually increases and tends to saturate after that point.The iteration analysis considers two adaptation settings and balances performance with training time.
  • Domain adaptation space: 51.42% IoU is obtained with feature-level adaptation, 23.5% lower than RoadDA using output-space adaptation.The authors attribute the difference to the suitability of structured prediction space for binary road segmentation with complex remote-sensing structures.
  • Training dynamics: The segmentation and discriminator losses decrease toward convergence, while adversarial loss increases until the two domains reach a balanced state.The associated t-SNE analysis compares source-only features with one-stage and two-stage RoadDA features.

V. CONCLUSION

RoadDA is a two-stage unsupervised domain adaptation framework for remote-sensing road segmentation. It combines feature-pyramid-based inter-domain adaptation with adversarial self-training and outperforms state-of-the-art domain-adaptation methods on two benchmark settings.

  • Conclusion: RoadDA uses inter-domain adaptation with a feature pyramid fusion module, followed by adversarial self-training to reduce target-domain discrepancy.The second stage mines easy target samples, assigns pseudo labels, and uses them to guide intra-domain adaptation.
  • Conclusion: RoadDA outperforms state-of-the-art domain adaptation methods on two benchmark settings for remote-sensing road segmentation.The authors describe this result as demonstrating promising application potential in real-world scenarios.
Loading 2108.12611v1…