Source-linked AI summary
Feature Pyramid Network for Multi-Class Land Segmentation
Selim S. Seferbekov, Vladimir I. Iglovikov, Alexander V. Buslaev, Alexey A. Shvets
TL;DR
Multi-class land segmentation from satellite imagery remains challenging, while high-resolution optical imagery offers broad coverage for monitoring Earth’s surface. The paper develops an FPN-based model with a pretrained ResNet50 encoder and reports a public leaderboard score of 0.493 for the land-cover classification challenge.
Problem
Multi-class land segmentation is needed to categorize satellite imagery for monitoring Earth’s surface and supporting applications such as urban planning and disaster prevention.
Source: §1 · pp. 1–2Method
Results
0.493 public leaderboard score was achieved by the model on the land-cover classification challenge.
Source: §5 · p. 4Takeaways & Limitations
The FPN-based approach provides a practical multi-class land-segmentation model evaluated on the public leaderboard.
Source: §1 · p. 1, §5 · p. 4The training labels are imperfect because annotating multi-class segmentation masks is costly, and small objects are not consistently annotated.
Source: §2 · p. 2
Abstract
from arXiv · showhide
Semantic segmentation is in-demand in satellite imagery processing. Because of the complex environment, automatic categorization and segmentation of land cover is a challenging problem. Solving it can help to overcome many obstacles in urban planning, environmental engineering or natural landscape monitoring. In this paper, we propose an approach for automatic multi-class land segmentation based on a fully convolutional neural network of feature pyramid network (FPN) family. This network is consisted of pre-trained on ImageNet Resnet50 encoder and neatly developed decoder. Based on validation results, leaderboard score and our own experience this network shows reliable results for the DEEPGLOBE - CVPR 2018 land cover classification sub-challenge. Moreover, this network moderately uses memory that allows using GTX 1080 or 1080 TI video cards to perform whole training and makes pretty fast predictions.
1. Introduction
High-resolution optical satellite imagery supports broad land-surface monitoring, while fully convolutional methods enable dense pixel-wise segmentation. The paper applies an FPN-based approach to multi-class land segmentation in high-resolution satellite imagery.
Motivation
- High-resolution optical satellite imagery is widely available and can support land-surface monitoring and topographic databases.Its 0.5–1.0 m geometric resolution is described as sufficient for land-cover categorization.Source: §1 · p. 1
Prior approaches
- Fully convolutional networks replace fully connected layers with convolutions to produce spatial feature maps and dense pixel-wise outputs.The maps are upsampled, allowing end-to-end segmentation with input images of arbitrary sizes.Source: §1 · p. 1
Proposed approach
- FPN uses bottom-up and top-down pathways with lateral connections to construct feature pyramids at multiple scales.The paper selects a ResNet50 encoder pre-trained on ImageNet for the bottom-up pathway.Source: §1 · p. 1
- The proposed network targets multi-class segmentation of high-resolution satellite imagery.The work generalizes ideas from prior convolutional approaches to this remote-sensing setting.Source: §1 · pp. 1–2
2. Dataset
The challenge dataset contains high-resolution RGB satellite images paired with seven-class color-coded land-cover masks. Its labels have annotation limitations, and validation images require submitted predictions for evaluation.
Dataset composition
- The training data contains 803 RGB satellite images, each sized 2448x2448 pixels at 50 cm resolution.The imagery was collected by DigitalGlobe satellites, and each training image has a paired land-cover mask.Source: §2 · p. 2
Annotations
- Each mask encodes seven land-cover classes using RGB color coding.The supplied passage begins listing classes including urban and agriculture land.Source: §2 · p. 2
- Mask values may not be pure 0 and 255, so threshold 128 is recommended for binarization.This affects preparation of the color-coded annotations.Source: §2 · p. 2
Evaluation
- The labels are imperfect because multi-class mask annotation is costly, and small objects are intentionally left unannotated.The validation set contains 171 images without masks whose predictions are uploaded to the DeepGlobe website.Source: §2 · p. 2
3. Model
The model addresses multi-scale land-object segmentation with an FPN built from a pre-trained ResNet50 encoder and a custom decoder. It merges feature maps across scales before producing seven-channel full-resolution outputs.
Motivation
- FPN is used because segmenting objects at different scales, especially small objects, is challenging.The general FPN architecture is presented as the model’s solution to this problem.Source: §3 · p. 2
Encoder
- The bottom-up pathway uses pre-trained ResNet50 modules for feature extraction, reducing spatial resolution while increasing hierarchical representation.Outputs from convolution modules C_i are passed to the top-down pathway.Source: §3 · p. 2
Decoder
- The top-down pathway upsamples features, combines them element-wise with lateral bottom-up features, and applies convolutions to form multi-scale feature maps.Nearest-neighbor upsampling by 2 is used between successive top-down stages.Source: §3 · p. 2
Output
- The decoder concatenates four same-resolution feature maps, applies convolution, normalization, activation, and dropout, then reduces channels to seven classes.The final seven-channel map is upsampled to the original image size using bilinear interpolation.Source: §3 · p. 2
4. Training
Training combines one-hot masks, IoU-based evaluation, categorical cross entropy, and a generalized loss, with augmentation and dropout used to address limited and imperfect data. Predictions use whole-image inference and test-time augmentation.
Objective
- Masks are converted to seven-channel one-hot images, and performance is evaluated with the Jaccard index, also called IoU.The metric measures similarity between sets represented by masks and predictions.Source: §4 · p. 2
- The training objective combines categorical cross entropy with an intersection-based term weighted by α and β.The selected values α = 1 and β = 0.5 provided the best performance in validation.Source: §4 · p. 3
Augmentation
- Training uses spatial and color augmentation, including scale transformations from 0.6−1.4, 30-degree rotations, random 448x448 crops, and color changes.A quarter of the dataset is held out for validation.Source: §4 · p. 3
Regularization
- Spatial dropout with p = 0.5 is used to reduce over-fitting, while training and validation IoU become close to 0.55.The reported standard deviation is around 0.13 between classes and 0.03 between iterations.Source: §4 · p. 3
Inference
- Whole 2448x2448 images are padded by 8 pixels, and test-time augmentation averages four predictions from 90-degree rotations.Padding makes the image side divisible by 32 and helps prevent bottom-up pathway artifacts.Source: §4 · p. 3
5. Conclusions
The authors developed a multi-class land segmentation system using FPN with a pretrained ResNet50 encoder and a tailored loss function. The model achieved a best public leaderboard score of 0.493, while addressing over-fitting with spatial dropout and test-time augmentation.
- The system uses an FPN with an ImageNet-pretrained ResNet50 in the bottom-up pathway and a designed loss function.Source: §5 · p. 4
- Inaccurate class labeling is identified as the main difficulty in the multi-class segmentation problem.Source: §5 · p. 4
- Strong spatial dropout on the final layer and test-time augmentation were used to prevent over-fitting.Source: §5 · p. 4
- 0.493 was the model's best public leaderboard score.Source: §5 · p. 4