Source-linked AI summary
Continental-Scale Building Detection from High Resolution Satellite Imagery
Wojciech Sirko, Sergii Kashubin, Marvin Ritter, Abigail Annkah, Yasser Salah Eddine Bouchareb, Yann Dauphin, Daniel Keysers, Maxim Neumann, Moustapha Cisse, John Quinn
TL;DR
The paper asks how building-detection methods can generalise across Africa, where footprints matter but prior evidence has limited geographic scope. It develops a U-Net-based training and inference pipeline using varied supervision and evaluated it across diverse contexts, producing an open dataset of 516M detected footprints.
Problem
Prior building-detection evidence has limited geographical scope, while building footprints are valuable for mapping, population estimation, planning, humanitarian response, and environmental science.
Method
The study develops a U-Net-based continent-wide pipeline using 50 cm imagery, with architectural, loss, regularization, pre-training, self-training, and post-processing experiments.
Results
The pipeline was used to generate the Open Buildings dataset of 516M building footprints across Africa.
Takeaways & Limitations
The resulting pipeline obtains good results across challenging rural and urban contexts and supports Africa-wide building-footprint mapping.
Takeaways & Limitations
Performance was weaker in desert terrain, while urban scenes showed a tendency to split large buildings into separate instances.
Abstract
from arXiv · showhide
Identifying the locations and footprints of buildings is vital for many practical and scientific purposes. Such information can be particularly useful in developing regions where alternative data sources may be scarce. In this work, we describe a model training pipeline for detecting buildings across the entire continent of Africa, using 50 cm satellite imagery. Starting with the U-Net model, widely used in satellite image analysis, we study variations in architecture, loss functions, regularization, pre-training, self-training and post-processing that increase instance segmentation performance. Experiments were carried out using a dataset of 100k satellite images across Africa containing 1.75M manually labelled building instances, and further datasets for pre-training and self-training. We report novel methods for improving performance of building detection with this type of model, including the use of mixup (mAP +0.12) and self-training with soft KL loss (mAP +0.06). The resulting pipeline obtains good results even on a wide variety of challenging rural and urban contexts, and was used to create the Open Buildings dataset of 516M Africa-wide detected footprints.
1 Introduction
The paper addresses continent-wide building-footprint detection in challenging African settings, where building information is valuable but existing methods have limited geographic scope. It develops and evaluates a U-Net-based pipeline with architectural, training, and post-processing variations.
- Building footprints support mapping, population estimation, urban planning, humanitarian response, and environmental science, especially where censuses are infrequent or settlements change rapidly.
- Africa-wide detection is challenging because vegetation and geological features resemble buildings, contiguous structures lack clear boundaries, and small or naturally constructed buildings can be difficult to see.
- Existing building-detection studies have largely focused on particular cities or countries, leaving open how well methods generalise across wider developing regions.
- The study tests U-Net pipeline variations spanning architecture, loss functions, regularization, pre-training, self-training, pre-processing, and post-processing.
- The resulting pipeline was used to generate an open dataset containing 516M building footprints across Africa.
2 Related work
Satellite building detection commonly uses bottom-up semantic segmentation because dense, small structures challenge bounding-box-based instance methods. However, prior evidence has been geographically limited, motivating continent-scale evaluation at 50 cm resolution.
- Dense and small buildings challenge bounding-box regression with non-maximum suppression because suppression can remove true detections and reduce recall.
- Satellite building detection commonly classifies pixels as building or non-building, then thresholds predictions and extracts connected components as instances.
- U-Net-based methods dominated leading entries in the CVPR DeepGlobe and SpaceNet building-detection challenges.
- Prior datasets and competitions covered limited geographies, with some imagery at 4m resolution that primarily supports detection of larger buildings.
3 Datasets
The study assembles continent-wide datasets with different supervision levels for supervised learning, pre-training, self-training, and evaluation. Sampling and annotation procedures are designed to represent diverse African environments and difficult building cases.
- The datasets include fully instance-labelled images, class-labelled imagery for pre-training, unlabelled imagery for self-training, and sparsely labelled evaluation data.
- Supervised learning and evaluation data: The training data contain 99,902 RGB images across Africa, with broad coverage and higher concentrations in East and West Africa.
- Supervised learning and evaluation data: Evaluation regions were selected to mix rural, medium-density, urban, informal-settlement, and refugee-facility contexts rather than mostly empty areas.
- Supervised learning and evaluation data: Annotation addresses near-undetectable small buildings and contiguous structures whose individual boundaries cannot be determined, using a dense building class for the latter.
- Pre-training data: Pre-training uses geospatial classification labels derived from S2 cells, while self-training uses a filtered unlabelled dataset reduced from 100M to 8.7M images.
- Additional evaluation data: The sparse evaluation dataset contains 0.9M building-centred images and can estimate precision but not recall because of its sampling design.
4 Model
The model uses U-Net for pixel-wise building segmentation, then converts predictions into instances through thresholding and connected components. Its architecture uses a ResNet-50-v2 encoder and a more complex residual decoder designed for precise building delineation.
- Model pipeline: The pipeline classifies each input pixel as building or non-building, then extracts instances by thresholding confidence maps and finding connected components.This converts semantic segmentation outputs into detected building instances.
- Encoder: The U-Net encoder is based on ResNet-50-v2; deeper ResNet-v2-101 and ResNet-v2-152 encoders did not improve accuracy in preliminary experiments.
- Decoder: The modified decoder applies batch normalization, ReLU, convolutions, a residual connection, and upconvolution.This differs from the simpler decoder blocks used in standard U-Net and TernausNet-v2.
- Decoder: Increasing decoder complexity can improve performance for building detection, where precise pixel-wise annotations of small objects are important.
5 Loss functions
The loss design combines pixel-weighted cross entropy with Focal Tversky Loss and emphasizes boundaries between nearby building instances. A Gaussian edge-convolution scheme replaces a slower distance-transform formulation while improving final performance and enabling on-the-fly computation.
- Loss functions: Cross entropy uses pixel weights to control the importance of each pixel, with model confidence and binary ground-truth labels defined per pixel.
- Loss functions: Mixing cross entropy with Dice loss was previously effective, while informal experiments suggested further improvement from Focal Tversky Loss.
- Loss functions: Focal Tversky Loss uses β to trade off false positives and false negatives, while γ adjusts the relative importance of easy and hard examples.
- Loss functions: The combined loss uses α = 0.5, β = 0.99, γ = 0.25, and ϵ = 10^-6 for numerical stability.
- Weighting: When all pixels are equally weighted, emphasizing pixels near edges of nearby or touching instances helps distinguish building instances, while distant background pixels can be down-weighted.
- Weighting: The original distance weighting uses distances to the nearest and second-nearest instances, but is slow and requires precomputed weights, limiting data augmentation.
- Weighting: The alternative method constructs an edge image and convolves it with a Gaussian kernel of length scale σ, then scales the result by c.
- Weighting: σ = 3 and c = 200 produced better final building-detection performance and were efficient enough for on-the-fly computation during training.
6 Regularization
The pipeline uses standard image augmentations and adapts mixup for segmentation to regularize training while preserving separate pixel-weighted supervision.
- Standard augmentations regularize training with random crops, flips, rotations, and color modifications.Color augmentations helped generalize to over- and under-exposed images and low-visibility atmospheric conditions.
- Mixup combines two randomly selected images using a weighted average during segmentation training.The mixup ratio coefficient λ lies in [0, 1).
- The model computes cross-entropy loss against both corresponding label sets and combines the resulting losses.Labels are not averaged, unlike the original mixup specification.
- Mixup is applied only to the cross-entropy term, not Focal Tversky loss, with λ = 0.05 in the experiments.
7 Pre-training
The authors investigate domain-specific pre-training for satellite building detection, but ultimately use ImageNet initialization because it performs effectively in their experiments.
- The study evaluates satellite-specific pre-training tasks for initializing U-Net encoder weights instead of relying only on ImageNet.The tasks include predictions based on satellite datasets and geographic location.
- Pre-training with the three satellite tasks alone produced poor building-detection performance.The authors inspected initial ResNet filters and explored alternative pre-training variations.
- ImageNet pre-training was effective and was used in the final detection model.Fine-tuning with nighttime luminance increased average mAP, though not significantly.
- The considered pre-training schemes were classification tasks, leaving decoder initialization for segmentation as a possible improvement.
8 Self-training
Self-training leverages additional unlabeled satellite imagery, using soft teacher labels and focal KL divergence to improve the building-detection model and address specific supervised-data errors.
- Self-training uses a much larger unlabeled satellite-image collection than the labeled dataset, with augmentations similar to supervised training.
- The best model uses multiple self-training iterations with soft teacher labels and a Kullback–Leibler loss having focal γ = 0.25.Hard labels, γ ≥1, larger students, and stochastic depth did not improve performance in informal experiments.
- Self-training with random black-mask augmentation produced a student that detected buildings adjacent to black image regions.The supervised teacher had failed in those regions because of incorrectly labeled supervised data.
- Morphological erosion shrinks touching building instances by one pixel so post-processing can separate them.A 3 × 3 kernel is applied to labeled instances during preprocessing.
- Dense building labels were remapped to normal building labels, while treating dense buildings as unknown performed equivalently.
10 Post-processing
Post-processing ensembles confidence masks, applies multi-scale augmentation, extracts connected components as instances, and evaluates how these choices affect detection across contexts.
- 10 Post-processing: Ensembling averages confidence masks from multiple models, while test-time augmentation averages masks across image scales.The described scales include 1, 512 × 448, and 576 × 448.
- 10 Post-processing: Thresholded semantic-segmentation predictions are converted into building instances by finding 4-connected components.Each instance receives the average confidence score of its component.
- 10 Post-processing: Post-processing approximates the inverse of preprocessing erosion by dilating each instance with the same 3 × 3 kernel.
- 11 Evaluation: Distance weighting contributed most to detection performance, followed by mixup and ImageNet pre-training.The ablation changes one configuration setting at a time and measures the performance drop relative to baseline.
- 11 Evaluation: Cross-entropy-only training differed from the baseline by -0.005 mAP, a difference not statistically significant across replicas.
- 11 Evaluation: Self-training increased mAP by 0.057 on average and consistently increased precision at high recall across rural, urban, town, and displaced categories.
- 11 Evaluation: Low-scoring detections reflected rural label errors, urban splitting of large buildings, and poorer performance in desert terrain.
- 11 Evaluation: ImageNet pre-training, optionally fine-tuned with nighttime luminance, appeared effective for encoder initialization.
12 Generation of the Open Buildings dataset
The pipeline generated building-footprint detections across much of Africa and used regional evaluation to guide confidence-based filtering. Performance varied by region because of sparse labels, labeling errors, and instance-splitting errors on large buildings.
- Continental inference: 19.4M km2 of available high-resolution imagery, covering 64% of Africa’s land surface, was processed with the best model.Inference used imagery from different timestamps and resolutions.
- Post-processing: 36B building detections remained after inference, contouring, and deduplication.The contouring algorithm produced angular polygons and realigned nearby groups, although the passage truncates the subsequent dataset totals.
- Regional variation: Regional precision-recall performance was reduced in some areas by sparse populations, missing labels, and splitting large buildings into multiple instances.The cited examples are Sierra Leone–Tuelo, Mozambique–Macia, and Egypt–Cairo.
- Regional variation: Figure 11 provides example imagery for the categories evaluated in the regional analyses.The imagery is credited to Maxar Technologies.
- Confidence filtering: Regional confidence thresholds were estimated to target approximately 90% precision, with unknown effects on recall.Thresholds were computed for each S2 cell bucket using an additional evaluation dataset reweighted to match Open Buildings density.
13 Conclusion
The paper concludes that a U-Net-based instance-segmentation pipeline can detect buildings across Africa, while identifying broader architectural and imaging directions for future improvement. The resulting dataset requires regional confidence filtering and contains errors associated with contouring, deduplication, and source-image alignment.
- Conclusion: The pipeline was used to detect buildings across the entire continent of Africa from satellite imagery.The authors state that methods including self-training, mixup, and alternative distance weighting could also apply to other instance-segmentation architectures.
- Future directions: Future improvements include multimodal imagery and architectures that explicitly find instances rather than casting detection as semantic segmentation.The paper gives Sentinel imagery as an example of an additional modality.
- Evaluation: Table 3 compares mean average precision across pre-training schemes used to initialize the U-Net encoder.The supplied caption identifies the comparison but does not report the numerical outcomes.
- Dataset quality: Spatially varying filters were used to obtain estimated 90% precision across the full dataset.The figure caption identifies geographic variation in filtering thresholds.
- Dataset quality: Final-dataset errors included contouring and deduplication artifacts, confidence-dependent detections, and shifts caused by image misalignment.Orthorectification errors in source imagery were also identified as a cause of footprint-generation errors.
- Implications: Improved built-environment mapping can support practical and scientific applications as high-resolution overhead imagery becomes more widely available.The stated application scope includes the uses described in the paper’s motivation.