PaperlayerView exact revision on arXiv

Source-linked AI summary

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation

Junjue Wang, Zhuo Zheng, Ailong Ma, Xiaoyan Lu, Yanfei Zhong

arXiv:2110.08733v6cs.CV

TL;DR

Existing HSR land-cover datasets largely overlook transferability, limiting evaluation across differing urban and rural landscapes. LoveDA introduces a real-image dataset for semantic segmentation and UDA, benchmarks both task families, and studies targeted strategies for its domain challenges. The dataset and experiments expose multi-scale objects, complex backgrounds, and inconsistent class distributions as central challenges for transferable land-cover mapping.

  • Problem

    Existing HSR land-cover datasets mainly promote semantic representation learning while ignoring model transferability across differing domains.

  • Method

    The paper constructs LoveDA from spatially independent, professionally annotated 0.3 m urban and rural imagery across three Chinese cities and evaluates semantic segmentation and UDA methods.

  • Results

    Benchmarking and exploratory studies reveal multi-scale objects, complex background samples, and inconsistent class distributions as major challenges in LoveDA.

  • Takeaways & Limitations

    LoveDA provides a common real-image benchmark for studying both land-cover segmentation and transferability between urban and rural domains.

Abstract

from arXiv · show

Deep learning approaches have shown promising results in remote sensing high spatial resolution (HSR) land-cover mapping. However, urban and rural scenes can show completely different geographical landscapes, and the inadequate generalizability of these algorithms hinders city-level or national-level mapping. Most of the existing HSR land-cover datasets mainly promote the research of learning semantic representation, thereby ignoring the model transferability. In this paper, we introduce the Land-cOVEr Domain Adaptive semantic segmentation (LoveDA) dataset to advance semantic and transferable learning. The LoveDA dataset contains 5987 HSR images with 166768 annotated objects from three different cities. Compared to the existing datasets, the LoveDA dataset encompasses two domains (urban and rural), which brings considerable challenges due to the: 1) multi-scale objects; 2) complex background samples; and 3) inconsistent class distributions. The LoveDA dataset is suitable for both land-cover semantic segmentation and unsupervised domain adaptation (UDA) tasks. Accordingly, we benchmarked the LoveDA dataset on eleven semantic segmentation methods and eight UDA methods. Some exploratory studies including multi-scale architectures and strategies, additional background supervision, and pseudo-label analysis were also carried out to address these challenges. The code and data are available at https://github.com/Junjue-Wang/LoveDA.

1 Introduction

LoveDA addresses limited transferability in existing HSR land-cover datasets by introducing a real-image dataset spanning urban and rural domains for semantic segmentation and UDA. Its domain variation creates multi-scale, complex-background, and class-distribution challenges, motivating broader benchmarking and exploratory studies.

  • Existing HSR land-cover datasets promote semantic representation learning but largely ignore model transferability.
  • LoveDA introduces an HSR dataset for both land-cover semantic segmentation and unsupervised domain adaptation.
  • Real urban and rural images from 18 scenes across three Chinese cities create multi-scale object variation and complex background samples.
  • Urban and rural scenes have inconsistent class distributions, with urban areas containing more artificial objects and rural areas more natural objects.
  • The study evaluates semantic segmentation and UDA methods and conducts exploratory analyses of multi-scale strategies, background supervision, and pseudo-labels.

2 Related Work

Prior HSR land-cover datasets advanced semantic segmentation, while limited shared categories, inconsistent annotations, and limited coverage constrained UDA research. Related UDA methods generally use adversarial training or self-training, but remote-sensing progress remains limited by dataset constraints.

  • Existing HSR land-cover datasets primarily support semantic segmentation accuracy rather than transferability.
  • Combining public datasets for UDA can suffer from insufficient common categories and inconsistent annotations.
  • UDA methods transfer models from source to target domains by learning domain-invariant features or adapting distributions.
  • Adversarial training: Adversarial training uses a feature extractor and discriminator, with segmentation adaptation performed in structured output space.
  • Self-training: Self-training alternates target pseudo-label generation with model fine-tuning and emphasizes improving pseudo-label quality.
  • UDA in remote sensing: Remote-sensing UDA research has studied adversarial and self-training methods, but insufficient shared categories and inconsistent annotation granularity limit progress.

3 Dataset Description

LoveDA is built from high-resolution imagery across urban and rural areas in three Chinese cities, with spatially independent splits and professional annotation. Its statistics and scene comparisons expose object-scale variation, complex backgrounds, and strong urban–rural class-distribution differences.

  • 3.1 Image Distribution and Division: 0.3 m imagery from Nanjing, Changzhou, and Wuhan covers 536.15 km^2 and is divided into non-overlapping 1024 × 1024 images.
  • 3.1 Image Distribution and Division: The dataset contains 18 areas—nine densely populated urban areas and nine rural areas selected from undeveloped districts.
  • 3.1 Image Distribution and Division: Training, validation, and test sets were split spatially independently to reduce nearby-image dependence.
  • 3.1 Image Distribution and Division: Professional annotators and strict inspection procedures were used in the annotation pipeline.
  • 3.2 Statistics for LoveDA: LoveDA has the largest labeled-pixel and land-cover-object counts among the compared public HSR datasets, while background contains the most pixels and causes false alarms.
  • 3.3 Differences Between Urban and Rural Scenes: Urban areas contain more buildings and roads, whereas rural areas contain more agricultural land, producing inconsistent class distributions.
  • 3.3 Differences Between Urban and Rural Scenes: Rural buildings are mostly small and long-tailed, while urban buildings show greater size variance, requiring multi-scale capture capabilities.

4 Experiments

The experiments evaluate semantic segmentation and unsupervised domain adaptation on LoveDA, examining multi-scale processing, background supervision, and pseudo-labeling under urban–rural domain differences. Results show that multi-scale designs and self-training are particularly useful for the dataset’s scale variation and inconsistent class distributions.

  • Semantic Segmentation: Eleven semantic segmentation networks were evaluated on the LoveDA Test set using intersection over union (IoU).
  • Multi-scale architectures and strategies: HRNet outperforms the other methods because its architecture repeatedly exchanges features across different scales.UNet++ also outperforms UNet through nested cross-scale connections, while cross-layer fusion outperforms in-module fusion among several architectures.
  • Multi-scale architectures and strategies: Multi-scale augmentation during training significantly improves different methods, and multi-scale inference further improves all methods through averaged rescaled outputs.Training scales were {0.5, 0.75, 1.0, 1.25, 1.5, 1.75}.
  • Additional background supervision: Additional binary cross-entropy background supervision improves background accuracy and overall performance, while combining it with Dice loss performs well.The total loss combines the original cross-entropy, binary cross-entropy, and Dice losses with modulation factors; Tables 4 and 5 report background precision, recall, and F1-score.
  • Visualization: Semantic segmentation remains difficult for small buildings and scattered trees, while forest and agricultural classes are easily confused because of similar spectra.Building edges are also hard to extract accurately because of disorderly arrangement and varied scales.
  • Unsupervised Domain Adaptation: Across UDA benchmarks, self-training methods outperform adversarial training because pseudo-label generation addresses inconsistent class distributions, although road exhibits negative transfer in Urban →Rural experiments.The domain shift affects class groups differently: artificial classes drop more in Rural →Urban, while Urban →Rural shows the opposite pattern.

5 Conclusion

LoveDA addresses limited cross-scene generalization by providing a domain-adaptive dataset that exposes three challenges and evaluates methods designed to address them.

  • LoveDA targets the limited generalization of deep learning between urban and rural scenes in land-cover mapping.
  • The dataset presents multi-scale objects, complex background samples, and inconsistent class distributions as challenges for large-scale remote sensing mapping.
  • State-of-the-art methods were evaluated on LoveDA to reveal these challenges.
  • Exploratory studies examined multi-scale architectures and strategies, additional background supervision, and pseudo-label analysis as alternative responses.

6 Broader Impact

The work releases an open dataset and benchmarks two tasks to support further research on remote sensing land-cover segmentation and transferability.

  • The work offers a free and open dataset for advancing land-cover semantic segmentation in remote sensing.
  • It provides two benchmarked tasks with three considerable challenges for researchers to build on.
  • The authors identify developing generalizable models for large-scale mapping as a potential positive societal impact.
  • The authors do not foresee negative societal impacts from this work.

A.1 Annotation Procedure and Data Division

LoveDA defines seven common land-cover classes and uses professional remote sensing annotators with an annotation pipeline based on prior work.

  • Seven land-cover types comprise buildings, road, water, forest, agriculture, and background classes.
  • Professional remote sensing annotators labeled the images using ArcGIS geo-spatial software.
  • The annotation process adopted a comprehensive pipeline referring to prior work.
  • Table 8 presents the division of the LoveDA dataset.

A.2 Top Performances Compared with Other Datasets

LoveDA is presented as more challenging than other land-cover datasets, with top-performing methods achieving lower performance on it.

  • The advanced HRNet method achieved the lowest performance on LoveDA among the compared datasets.
  • Table 9 compares top performances on LoveDA with results reported for other datasets.

A.3 Instance Differences Between Urban and Rural Areas

Urban and rural areas differ in their instance composition, producing domain-specific class imbalances. Urban scenes contain more buildings, whereas rural scenes contain more agricultural-land instances.

  • Urban areas contain more building instances and fewer agricultural-land instances than rural areas.
  • Rural areas contain more instances of agricultural land than urban areas.
  • These instance-level differences highlight inconsistent class distributions between the urban and rural domains.

A.4 Implementation Details

The experiments reimplemented baseline segmentation and UDA methods with standardized settings, repeated UDA evaluations, and analyses of uncertainty, normalization statistics, and large-scale mapping. Results indicate greater variability for self-training methods, similar BN statistics across domains, and remaining room for tailored UDA methods.

  • Model implementation: The implementation used modified ResNet50 and ASPP segmentation models with output stride 16, plus a five-layer convolutional discriminator for UDA.The discriminator used 4 × 4 kernels, stride 2, Leaky ReLU activations, and channel counts {64, 128, 256, 512, 1}.
  • Experimental setup: All UDA methods were run five times with random seeds to improve result reproducibility.The experiments included error-bar visualization for the UDA results.
  • Error analysis: Adversarial training methods show smaller error fluctuations than self-training methods.The paper attributes the greater randomness of self-training to alternating pseudo-label assignment and updates.
  • Batch normalization analysis: Oracle source and target models have similar BN mean and variance statistics, indicating that the domain gap does not lie in batch normalization.The paper contrasts this observation with TransNorm, whose target BN statistics differ substantially from the Oracle target model.
  • Large-scale visualization: CBST produces better large-scale mapping results than the baseline, while still leaving substantial room for improvement.Figure 10 presents large-scale visualizations for the Rural →Urban UDA test set, including the baseline on the Wujin area.
Loading 2110.08733v6…