Source-linked AI summary
Land-Cover Classification with High-Resolution Remote Sensing Images Using Transferable Deep Models
Xin-Yi Tong, Gui-Song Xia, Qikai Lu, Huanfeng Shen, Shengyang Li, Shucheng You, Liangpei Zhang
TL;DR
Heterogeneous HRRS images are difficult to classify accurately because high-resolution detail and acquisition differences limit transfer from annotated data. The paper adapts a pretrained CNN to unlabeled targets through confidence-based pseudo-labeling, source-sample retrieval, fine-tuning, and hybrid segmentation, reporting promising multi-source results and a 150-image dataset. The method is challenged by domain heterogeneity and confusion among some fine categories.
Problem
High-resolution detail and heterogeneous acquisition conditions make accurate land-cover classification difficult, while target images lack labels for straightforward model adaptation.
Method
A pretrained CNN assigns high-confidence pseudo-labels to target patches, retrieves related source samples for fine-tuning, and combines patch-wise classification with hierarchical segmentation.
Results
The method achieves the highest reported GE-WH performance, with Kappa 0.924 and OA 94.56%, while experiments across multi-source HRRS images show promising classification performance.
Takeaways & Limitations
The approach enables automatic classification of unlabeled multi-source HRRS images, and GID provides 150 high-resolution GF-2 images for CNN training.
Takeaways & Limitations
Domain heterogeneity from differing resolutions, geographic morphology, and seasonal conditions remains a challenge, and fine categories such as water bodies or farmland and meadow can be confused.
Abstract
from arXiv · showhide
In recent years, large amount of high spatial-resolution remote sensing (HRRS) images are available for land-cover mapping. However, due to the complex information brought by the increased spatial resolution and the data disturbances caused by different conditions of image acquisition, it is often difficult to find an efficient method for achieving accurate land-cover classification with high-resolution and heterogeneous remote sensing images. In this paper, we propose a scheme to apply deep model obtained from labeled land-cover dataset to classify unlabeled HRRS images. The main idea is to rely on deep neural networks for presenting the contextual information contained in different types of land-covers and propose a pseudo-labeling and sample selection scheme for improving the transferability of deep models. More precisely, a deep Convolutional Neural Networks is first pre-trained with a well-annotated land-cover dataset, referred to as the source data. Then, given a target image with no labels, the pre-trained CNN model is utilized to classify the image in a patch-wise manner. The patches with high confidence are assigned with pseudo-labels and employed as the queries to retrieve related samples from the source data. The pseudo-labels confirmed with the retrieved results are regarded as supervised information for fine-tuning the pre-trained deep model. To obtain a pixel-wise land-cover classification with the target image, we rely on the fine-tuned CNN and develop a hybrid classification by combining patch-wise classification and hierarchical segmentation. In addition, we create a large-scale land-cover dataset containing 150 Gaofen-2 satellite images for CNN pre-training. Experiments on multi-source HRRS images show encouraging results and demonstrate the applicability of the proposed scheme to land-cover classification.
1 Introduction
HRRS imagery offers detailed land-surface information but makes cross-domain land-cover classification difficult because spatial complexity and acquisition differences reduce model transferability. The paper addresses this with transferable deep models, pseudo-labeling, hybrid classification, and the GID dataset.
- Motivation: HRRS images provide detailed land-surface information, but object geometry, structure, imaging distortions, scale variation, and illumination changes complicate classification.These factors can reduce separability between land-cover classes and weaken models across acquisition conditions.
- Challenges: Deep models trained on annotated images may transfer poorly across sensors or locations, while obtaining target-domain annotations is labor-intensive and time-consuming.The paper identifies inadequate model transferability and insufficient labeled target samples as central problems.
- Challenges: Existing land-cover datasets often cover limited and geographically similar areas, restricting variation and potentially limiting model generalization.The paper motivates a larger, geographically broader annotated dataset for HRRS classification.
- Proposed approach: The proposed scheme adapts deep models to unlabeled multi-source HRRS images by mining samples with CNN contextual features and integrating hierarchical segmentation with patch-wise classification.The method uses pseudo-labels and related source samples to support fine-tuning and combines category and boundary information.
- Evaluation: Experiments across Gaofen-2, Gaofen-1, Jilin-1, Ziyuan-3, Sentinel-2A, and Google Earth data report promising results for the proposed scheme.The experiments are intended to demonstrate effectiveness across multi-source HRRS imagery.
- Dataset contribution: GID contains 150 high-resolution Gaofen-2 images covering more than 50,000 km2 in China for land-cover model pre-training.The paper presents GID as a large-scale, well-annotated HRRS land-cover dataset.
2 Related work
Related work progresses from spectral and spatial feature methods to deep models and transfer learning for land-cover classification. The paper is situated among approaches that learn image representations and exploit unlabeled target-domain structure.
- Land-cover classification: Spectral methods interpret individual pixels but are strongly affected by intra-class spectral variation and noise.This motivates incorporating spatial and contextual information for land-cover description.
- Land-cover classification: Deep networks adaptively learn discriminative image features, offering an advantage over methods based only on spectral or spectral-spatial descriptors.Related approaches include CNN-based and hybrid spatial-spectral classifiers.
- Land-cover classification: Object-based CNN methods incorporate deep learning into object-based analysis to improve object boundaries in high-resolution urban scenes.This line of work addresses uncertainty in object boundary partitioning.
- Transfer learning: Transfer learning adapts models from a source domain to a related target domain when ground-truth samples are insufficient for training a high-performance classifier.Remote-sensing methods include supervised approaches using source and target data and semi-supervised approaches using unlabeled target samples.
- Transfer learning: CNNs trained on large natural-image or remote-sensing datasets have been transferred through feature extraction or network fine-tuning.The paper builds on deep models’ capacity to represent high-level data abstractions.
3 Methodology
The methodology transfers a CNN trained on labeled source imagery to unlabeled target HRRS images through pseudo-labeling, relevant-sample retrieval, and target-specific fine-tuning. Final classification combines patch-wise predictions with multi-scale context and hierarchical segmentation for pixel-level mapping.
- Framework: The framework pre-trains a ResNet-50 on labeled source data, partitions each unlabeled target image into patches, and fine-tunes the model for that target image.ResNet-50 is selected as a balance between classification performance, parameter count, and computational efficiency.
- Pseudo-label assignment: High-confidence target patches receive pseudo-labels, while low-confidence patches are discarded using a probability threshold σ.The maximum class probability determines both the predicted category and whether a patch is retained.
- Relevant sample retrieval: Relevant sample retrieval validates pseudo-labels by comparing target queries with source samples using deep-feature similarity and source-domain labels.Retrieved source samples confirm a pseudo-label when the top δ results share its category; confirmed samples form the target-specific fine-tuning set.
- Hybrid classification: The final classifier combines patch-wise categorization with hierarchical segmentation and majority voting to preserve both land-cover labels and object boundaries.Selective search generates multi-scale regions whose labels are refined using majority voting.
- Patch-wise classification: Multi-scale patches around each reference pixel are resized and processed by ResNet-50 to aggregate contextual information for classification.The method samples patches at several spatial scales before obtaining scale-specific softmax probabilities.
4 GID: a well-annotated dataset for land-cover classification
GID is a large, hierarchically organized land-cover dataset built from widely distributed, multi-temporal Gaofen-2 imagery and designed to support CNN pre-training and transfer across heterogeneous HRRS data. It combines pixel-level large-scale annotations with fine-grained patches and includes diverse geographic, spectral, and morphological conditions.
- Dataset composition: GID contains 150 pixel-level annotated GF-2 images and 30,000 multi-scale image patches organized into large-scale and fine land-cover classification sets.The dataset’s two parts form a hierarchical classification system.
- Pre-training and transfer evaluation: GID is used to pre-train CNNs for remote sensing and is evaluated alongside images from GF-1, JL-1, ZY-3, Sentinel-2A, and Google Earth data.The additional sources vary in sensors, spatial resolutions, spectral bands, and geographic locations, supporting transferability evaluation.
- Land-cover types: The large-scale set includes five major categories, while the fine set contains 15 sub-categories with 2,000 training patches per class.The five major categories are built-up, farmland, forest, meadow, and water; the fine categories further subdivide land-cover types.
- Dataset properties: GID spans more than 50,000 km2 across over 60 Chinese cities, providing geographically distributed imagery for representing ground objects in different areas.The geographic distribution is illustrated by the training and validation image locations in Figure 5.
- Dataset properties: The dataset includes multi-temporal imagery from the same or overlapping locations, capturing seasonal differences in the spectral responses of ground objects.These temporal examples include imagery from Xiantao, Wuhan, Nanchang, and other locations.
5 Experimental results
Experiments evaluate transfer across same-sensor and multi-source HRRS images, showing that PT-GID and target-domain fine-tuning outperform conventional methods. The method also handles large-scale Google Earth data without target labels, though performance depends on spectral compatibility.
- Gaofen-2 experiments: PT-GID achieves Kappa/OA of 0.924/96.28% on 5 classes and 0.605/70.04% on 15 classes, outperforming the comparison methods.RF+Fusion, the strongest comparison, reaches 0.641/78.45% and 0.237/33.70% on the respective class settings.
- Multi-source experiments: FT-Utg achieves the best OA for both 5- and 15-class settings across all tested target images from different sensors.The results support selecting relevant target-domain samples and fine-tuning the CNN to improve transferability.
- Multi-source experiments: When source and target spectral responses differ substantially, source-domain supervision becomes unreliable for the target domain.The paper contrasts this limitation with gains when spectral responses are more similar.
- Multi-source experiments: Fine-tuning reduces confusion between urban and rural residential classes in JL-1(2), indicating that selected samples capture target-domain distributions.The comparison methods confuse these classes, whereas FT-Utg correctly classifies urban residential areas in the illustrated result.
- Google Earth platform data: FT-Utg reaches Kappa/OA of 0.924/94.56% on the large-scale GE-WH experiment, exceeding PT-GID by 0.205/14.24%.The target image is only partially annotated for evaluation, while the classification itself uses no target labels.
6 Sensitivity Analysis
Sensitivity experiments examine patch scale, segmentation scale, segmentation method, and transfer-learning parameters. Multi-scale patches and suitable parameter choices improve accuracy, while excessive segmentation or redundant samples can reduce efficiency or detail.
- Multi-scale information fusion: Multi-scale information fusion achieves Kappa/OA of 0.924/96.28%, exceeding the best single-scale result.The paper attributes this to ground objects exhibiting contextual variation across scales.
- Multi-scale information fusion: A 56 × 56 patch is optimal among the tested single scales because larger patches can lose object details.All pixels within each non-overlapping patch receive the same label, making oversized patches especially problematic.
- Segmentation analysis: Mean OA first increases and then decreases with segmentation scale, with the best result at an initial scale of 400.Over-segmentation introduces noise, whereas under-segmentation removes map detail.
- Segmentation analysis: Selective search and multi-resolution segmentation perform comparably, with selective search slightly higher overall Kappa and OA.Selective search performs better for farmland and water, while multi-resolution segmentation performs better for built-up, forest, and meadow.
- Transfer-learning parameters: The pseudo-label threshold σ performs best at 0.8 for 5 classes and 0.7 for 15 classes.Lower thresholds can introduce unreliable pseudo-labels, while higher thresholds reduce the number of selected samples.
- Transfer-learning parameters: The sample-number parameter µ performs best at 4000 for 5 classes and 1500 for 15 classes, while larger values can introduce redundancy and reduce efficiency.The fine-class setting contains fewer samples per subclass, affecting the preferred selection size.
- Transfer-learning parameters: The source dataset’s high sample diversity supports discriminative pre-training, but the method does not explicitly remove redundancy from selected target samples.The authors identify redundancy handling as a direction for future research.
7 Discussion
The discussion frames the method as an automatic way to transfer deep models across heterogeneous HRRS images using structural and spatial information. It also identifies persistent confusion among fine categories and the need for more reliable supervision in multi-temporal analysis.
- Discussion: High-resolution imagery supports detailed mapping, but difficult annotation limits classifier adaptability for large-scale practical use.The discussion motivates robust and transferable classification for heterogeneous areas.
- Discussion: The method links intrinsic object characteristics across acquisition conditions through high-level deep features because object structure and spatial relationships are treated as stable.It requires no target-domain annotations and improves reliability through feature-similarity constraints.
- Discussion: Fine categories such as river, lake, pond, farmland, and meadow remain seriously confused in the confusion matrices.The paper attributes these errors to similarities in spectral, texture, and structural information, as well as diverse appearances across seasons.
- Discussion: Multi-temporal analysis may distinguish difficult categories, but the current method cannot guarantee sufficiently accurate pseudo-labels for supervised multi-temporal feature learning.The authors identify multi-supervised multi-temporal analysis as future work.
8 Conclusion
The paper concludes with an automatic transfer algorithm for multi-source HRRS classification, combining target-sample selection, multi-scale context, and hybrid segmentation. It also contributes the 150-image GID dataset and reports encouraging results across multiple sensors and platforms.
- Conclusion: The algorithm automatically selects target-domain training samples from contextual information extracted by a deep model, avoiding new manual annotation or algorithm adjustment for multi-source images.This design is intended for applying the method across heterogeneous target imagery.
- Conclusion: Multi-scale contextual information characterizes object spatial distributions and strengthens transferability across images with different resolutions.The conclusion treats scale diversity as a central component of the classification scheme.
- Conclusion: Combining patch-wise classification with hierarchical segmentation yields category and boundary information simultaneously.The hybrid design connects semantic classification with spatial delineation.
- Conclusion: GID contains 150 high-resolution GF-2 images and is constructed to represent land-cover-category distributions for CNN training.Experiments also cover GF-1, JL-1, ZY-3, Sentinel-2A, and Google Earth data.