Source-linked AI summary
Building Disaster Damage Assessment in Satellite Imagery with Multi-Temporal Fusion
Ethan Weber, Hassan Kané
TL;DR
Satellite-image damage assessment can take hours of analyst work, although timely damage information matters for disaster response. The paper evaluates modeling and processing choices on xBD and reports improved results over baseline models.
Problem
Human analysts may take hours to identify damaged zones in satellite imagery, while responders need damage locations and severity for resource deployment.
Source: §1 · p. 1Method
The approach processes pre- and post-disaster images with shared weights, fuses their features before semantic segmentation, and evaluates damage-assessment and preprocessing choices.
Source: §1 · p. 1, §3.2 · p. 4Results
0.738 overall F1 on the xBD test set, with localization F1 of 0.835 and damage F1 of 0.697; the model outperformed xBD baselines and ranked second in xView2 Track 3 Evaluation Only.
Source: §5 · p. 5Takeaways & Limitations
The work releases its competition code for replication.
Source: §4.3 · p. 5, §5 · p. 5Inverse-proportional class weighting lets training continue longer before collapse into all-no-building predictions, but does not eliminate the issue.
Source: §4.3 · p. 5
Abstract
from arXiv · showhide
Automatic change detection and disaster damage assessment are currently procedures requiring a huge amount of labor and manual work by satellite imagery analysts. In the occurrences of natural disasters, timely change detection can save lives. In this work, we report findings on problem framing, data processing and training procedures which are specifically helpful for the task of building damage assessment using the newly released xBD dataset. Our insights lead to substantial improvement over the xBD baseline models, and we score among top results on the xView2 challenge leaderboard. We release our code used for the competition.
1 INTRODUCTION
The paper addresses the time analysts spend identifying damage in satellite imagery and examines modeling and processing choices for building damage assessment with xBD.
1 INTRODUCTION
- Damage locations and severity help emergency responders deploy resources in disaster-affected areas.Satellite-image analysis by human analysts can take hours, motivating the use of AI to accelerate the workflow.Source: §1 · p. 1
- Convolutional neural networks have achieved human-level performance on several computer-vision tasks, including object recognition and image segmentation.The cited work describes these techniques as relevant to satellite-image disaster damage assessment.Source: §1 · p. 1
- Shared-weight pre- and post-disaster image streams, feature fusion before segmentation, smaller crops, and inverse-frequency class weighting substantially improve performance over baseline models.The work also characterizes mono-temporal versus multi-temporal settings.Source: §1 · p. 1
2 RELATED WORK
Prior work applied machine learning to building damage in imagery, while dataset availability and consistency shaped how the task could be framed and compared.
2.1 COMPUTER VISION TECHNIQUES FOR BUILDING DAMAGE ASSESSMENT
- Prior satellite damage studies often framed detection as binary pixel classification (Xu et al. (2019); Cooner et al. (2016); Ji et al. (2018); Duarte et al. (2018)); CNNs also addressed registration and ground-level damage imagery (Yang et al. (2018); Nia & Mori (2017)).Reported examples include a 40% error rate for a feed-forward neural network, 78.6% overall accuracy for collapsed-building identification, and 94.4% accuracy combining drone and satellite imagery.Source: §2.1 · p. 2
2.2 DATASETS FOR DAMAGE ASSESSMENT
- Before xBD, satellite damage datasets were often limited to single disaster types and lacked common assessment criteria, complicating model comparison and agreement on problem framing.The xview dataset was a large, diverse object-detection dataset, but it did not focus on building damage assessment.Source: §2.2 · p. 2
- The xBD dataset spans 19 disasters and 22,068 images, with 850,736 building polygons; its imagery has 0.3 m spatial resolution and four damage-annotation levels.Images are 1024 by 1024 pixels, and the imagery covers 45,361.79 km2.Source: §2.2 · pp. 2–3
3 MODEL
The model combines building localization and damage assessment, using pre- and post-disaster imagery and data-processing choices selected for the xBD task.
3 MODEL
- The task combines building localization with per-pixel damage classification; the proposed framing uses one network and treats damage levels 1–4 as buildings.The xBD baselines instead use separate networks for the two tasks.Source: §3 · p. 3, §3.1 · p. 3
3.2 DATA PROCESSING TECHNIQUE
- The selected configuration uses four 512 x 512 image quadrants, shared-weight pre- and post-disaster streams fused before segmentation, joint prediction, and class-weighted cross-entropy.The image quadrants address buildings that were often too small for accurate boundary prediction in 1024 by 1024 images.Source: Table 1 · p. 3, §3 · p. 3, §3.2 · p. 4
3.1 BASELINE ARCHITECTURE
- The model uses a Mask R-CNN backbone with an FPN and semantic-segmentation head, rather than separate baseline networks for localization and damage assessment.Semantic segmentation was selected because buildings are often too small for instance segmentation; the baseline damage model uses ordinal cross-entropy.Source: §3.1 · p. 3
3 MODEL
- The final model outperforms the xBD baseline by a large margin on all metrics in Table 2.The supplied table caption gives no metric values.Source: Table 2 · p. 4
4 EXPERIMENTS
The experiments compare xBD localization and damage-assessment approaches, then use the results to guide a multi-temporal model and training choices. The ablations favor semantic segmentation, image crops, and separately processed pre/post images with fused features.
4.1 XBD DATASET
- xBD contains 30-cm-per-pixel, 1024 x 1024 satellite imagery across 16 geographic locations, with over 850,000 building annotations spanning more than 45,000 km2.The dataset covers disasters including earthquakes, floods, volcanic eruptions, wildfires, and wind events, with imagery from before and after disasters.Source: §4.1 · p. 4
4 EXPERIMENTS
- Experiments compare instance and semantic segmentation, image crops, joint localization and damage prediction, and inverse-frequency class-loss weighting.The xView2 metrics are localization F1 for building detection, damage F1 for assessment, and overall F1 weighted 30% localization and 70% damage.Source: §4.2 · p. 4, §4 · p. 4
4.2 ABLATION STUDY
- Multi-temporal input with joint prediction increased all metrics; class-specific weighting raised damage and overall F1 while slightly lowering localization F1.The weighting emphasizes building-damage classification; 73.6% of polygons have no damage, and most image pixels are “no building.”Source: §4.2 · p. 5
- Switching from instance to semantic segmentation improved localization by 0.114, and four 512 x 512 crops per image added 0.016 localization F1.The authors attribute the semantic-segmentation gain to building-box predictions on small buildings.Source: §4.2 · pp. 4–5
- The best temporal design processes pre- and post-disaster images separately, then concatenates their features before the segmentation head.Simple six-channel image concatenation and image subtraction performed poorly; the final head uses 512 stacked FPN features to predict five classes.Source: §4.2 · p. 5
4.3 TRAINING
- With ImageNet-pretrained weights, training takes roughly six hours to converge; training too long can collapse predictions to all “no building.”Inverse-proportional class weighting lets training continue longer before collapse, but does not eliminate the issue.Source: §4.3 · p. 5
5 RESULTS
The final model outperforms xBD baselines on the holdout set and achieves competitive xView2 leaderboard results. Its reported scores include localization F1 of 0.835, damage F1 of 0.697, and overall F1 of 0.738.
5 RESULTS
- 0.738 overall F1, combining localization F1 of 0.835 and damage F1 of 0.697, was achieved on the xBD test set in the xView2 competition.The authors report outperforming the xBD baselines by a large margin on the holdout set.Source: §5 · p. 5
- The reported holdout comparison is against the xBD baseline, while leaderboard results are described as competitive.Source: §5 · p. 5
6 CONCLUSION
The conclusion attributes performance to image quadrants, a shared pre/post-image architecture with feature fusion, Mask R-CNN with FPN, and class-frequency-weighted loss. It identifies alternative damage-scale losses, feature-fusion strategies, and disaster-type inputs as future directions.
6 CONCLUSION
- The authors attribute performance to image quadrants, a shared architecture that fuses pre- and post-disaster features, Mask R-CNN with FPN, and inverse-frequency class weighting.Source: §6 · p. 5
- Future work could test ordinal cross-entropy to penalize damage-scale errors differently and explore other stages for combining pre- and post-disaster features.Source: §6 · p. 5
- The authors also identify incorporating disaster type, such as flood or fire, as a possible direction for building-damage prediction.Source: §6 · p. 5