PaperlayerView exact revision on arXiv

Source-linked AI summary

RescueNet: Joint Building Segmentation and Damage Assessment from Satellite Imagery

Rohit Gupta, Mubarak Shah

arXiv:2004.07312v1cs.CVeess.IV

TL;DR

Satellite imagery can support building-damage assessment, but existing approaches use separate building-detection and damage-classification stages that are not end-to-end trainable. RescueNet jointly segments buildings and classifies damage with a localization-aware loss; it significantly outperforms the baseline, especially on damage classification, and generalizes across regions and disaster types.

  • Problem

    Prior building-damage methods separate building detection and damage classification, preventing end-to-end training and multi-task supervision.

    Source: §I · p. 1, C. Building Damage Assessment · p. 2

  • Method

    RescueNet jointly segments buildings and classifies damage using multi-temporal imagery and a localization-aware loss combining building segmentation with foreground-only damage classification.

    Source: §III · p. 2, C. Change Detection Head · p. 3, D. Loss Functions · p. 4

  • Results

    RescueNet significantly outperforms the baseline, especially on damage classification, and generalizes across geographic regions and disaster types.

    Source: §I · p. 2, §VI · p. 6

  • Takeaways & Limitations

    The authors identify resolving separate building instances as future work to improve building localization and damage assessment.

    Source: §VI · p. 6

Abstract

from arXiv · show

Accurate and fine-grained information about the extent of damage to buildings is essential for directing Humanitarian Aid and Disaster Response (HADR) operations in the immediate aftermath of any natural calamity. In recent years, satellite and UAV (drone) imagery has been used for this purpose, sometimes aided by computer vision algorithms. Existing Computer Vision approaches for building damage assessment typically rely on a two stage approach, consisting of building detection using an object detection model, followed by damage assessment through classification of the detected building tiles. These multi-stage methods are not end-to-end trainable, and suffer from poor overall results. We propose RescueNet, a unified model that can simultaneously segment buildings and assess the damage levels to individual buildings and can be trained end-toend. In order to to model the composite nature of this problem, we propose a novel localization aware loss function, which consists of a Binary Cross Entropy loss for building segmentation, and a foreground only selective Categorical Cross-Entropy loss for damage classification, and show significant improvement over the widely used Cross-Entropy loss. RescueNet is tested on the large scale and diverse xBD dataset and achieves significantly better building segmentation and damage classification performance than previous methods and achieves generalization across varied geographical regions and disaster types.

I. INTRODUCTION

Rapid disaster response depends on timely information about building damage, but ground assessments can be risky and experts may take weeks to map affected areas. RescueNet jointly segments buildings and classifies their damage levels, and is reported to outperform the baseline, especially on damage classification.

I. INTRODUCTION

  • Ground assessments can be risky or impossible, while expert analysis of aerial and satellite imagery can take large teams weeks to map affected areas.Automated imagery analysis has also been explored for detecting damaged buildings and other disaster-related features.Source: §I · p. 1
  • Existing damaged-building methods detect buildings first and classify their damage second, preventing end-to-end training and limiting multi-task supervision.The paper identifies separate stages as a limitation of prior methods.Source: §I · p. 1
  • RescueNet jointly segments buildings and assesses damage using multi-scale temporal features, and significantly improves over the baseline, especially in damage classification.The evaluation uses xBD, with disaster imagery from varied events and 19 locations worldwide.Source: §I · pp. 1–2

II. RELATED WORK

Prior work spans building-footprint segmentation and change detection, while satellite-based building-damage assessment has received limited attention. The paper identifies separate-stage damage methods as a gap that joint segmentation and assessment aims to address.

A. Building Footprint Segmentation

  • Building-footprint work uses datasets including DeepGlobe and SpaceNet, with methods such as U-Net variants, ensembles, and composite-loss models.Other datasets mentioned include BingHuts and ISPRS 2D Semantic Labelling.Source: A. Building Footprint Segmentation · p. 2

B. Change Detection

  • Remote-sensing change detection has used corrected-image pixel differences, CNNs, Siamese networks, iterative refinement, and recurrent networks.The cited approaches include –.Source: B. Change Detection · p. 2

C. Building Damage Assessment

  • Building-damage assessment has received limited attention, and prior methods separate building detection from damage classification, precluding multi-task supervision.RescueNet explores a joint segmentation and damage-assessment framework to address this gap.Source: C. Building Damage Assessment · p. 2

III. RESCUENET MODEL

RescueNet is a unified model for building segmentation and damage classification, adapting semantic-segmentation designs to multi-temporal imagery and change detection.

III. RESCUENET MODEL

  • RescueNet adapts DeepLabv3 and DeepLabv3+ semantic-segmentation designs for building segmentation and adds change detection across multi-temporal images.Its architecture groups the feature extractor, segmentation head, and change-detection head into a unified model.Source: §III · p. 2, Figure 2 · p. 3

A. Extracting Image Features

  • A dilated ResNet50 preserves features at one-eighth input resolution, while ASPP combines image pooling and dilation rates 12, 24, and 36 for multi-scale features.The model diagram also describes the backbone as producing features at 1/8th input image resolution.Source: A. Extracting Image Features · p. 2, Figure 2 · p. 3

B. Segmentation Head

The segmentation head predicts building pixels independently in pre- and post-disaster images; the post-disaster segmentation is used for damage classification.

B. Segmentation Head

  • The segmentation head upsamples pixel predictions to full resolution and independently segments pre- and post-disaster images; damage classification processes only post-disaster imagery.The encoder-decoder option adds skip connections from lower-level backbone features, unlike the simple convolution-block design.Source: B. Segmentation Head · p. 3

C. Change Detection Head

The change detection head uses differences between pre- and post-disaster features to predict damage classes. The loss combines building localization with damage classification on foreground pixels, with Dice loss additionally used for crisper building boundaries.

C. Change Detection Head

  • The change detection head classifies pixels into four damage categories using multiscale feature differences from pre- and post-disaster images.It is a three-layer convolution block, and background predictions are ignored during training and inference.Source: C. Change Detection Head · p. 3

D. Loss Functions

  • Localization-aware loss combines binary cross-entropy for building segmentation with foreground-only categorical cross-entropy for damage classification.The damage-classification loss sums over damage classes; Dice loss is also used for building segmentation to predict crisper boundaries.Source: D. Loss Functions · p. 4

IV. DATASET

The study benchmarks RescueNet on xBD, a public dataset of pre- and post-disaster satellite imagery with building and damage annotations. Training uses Tier 1 and Tier 3 data, with validation drawn from Tier 1.

IV. DATASET

  • xBD provides georegistered 1024 × 1024 pre- and post-disaster image pairs, building polygons, and four-class damage labels across 19 disaster events.The authors train on the xBD training split because test annotations were not publicly available.Source: §IV · p. 4
  • Models are trained on Tier 1 and Tier 3 training data and evaluated on a stratified validation subset of about 10% of Tier 1.The validation sampling is stratified across disaster events; the split sizes are identified in Table I.Source: §IV · p. 4, Table I · p. 4

V. RESULTS

The results section reports qualitative validation examples and compares RescueNet with a baseline. In the examples, RescueNet appears to improve damage predictions, while residual errors occur for very large or closely spaced buildings.

A. Qualitative Results

  • In diverse validation examples, RescueNet appears to predict damage levels better than the baseline, whose false positives and false negatives are eliminated in the examples.The examples span varied building densities and damage patterns; these are qualitative results from a small sample.Source: A. Qualitative Results · p. 4
  • Flooding affects neighboring buildings to varying extents, whereas tsunamis produce contiguous regions of destruction.The observed damage patterns vary across disaster types.Source: A. Qualitative Results · p. 4
  • Remaining errors include failure to treat extremely large buildings as one unit and difficulty separating very close neighboring buildings.The first issue is attributed to the damage classification head and the second to the building segmentation head.Source: A. Qualitative Results · p. 4

B. Metrics

The XView2 Challenge metric combines building segmentation and damage-classification F1 scores, with greater weight on damage classification. The section also describes ablations of loss functions, segmentation heads, and the change detection head.

B. Metrics

  • The XView2 Challenge score weights building segmentation F1 at 0.3 and the harmonic mean of class-wise damage F1 scores at 0.7.The metric penalizes overfitting to overrepresented classes, and xBD is heavily skewed.Source: B. Metrics · p. 4
  • Figure 3 compares pre- and post-disaster images, ground truth, baseline predictions, and RescueNet predictions across diverse validation examples.Mask colors distinguish background and four building-damage categories.Source: Figure 3 · p. 5

C. Ablation Studies

  • Ablations examine different loss functions, segmentation heads, and whether the change detection head is included.The supplied passage describes the ablation dimensions but does not report their outcomes.Source: C. Ablation Studies · p. 6

1) Losses:

The ablations report gains from the localization-aware loss, the encoder-decoder segmentation head, and the change-detection head. The loss comparison also finds a smaller additional boost from Dice Loss.

1) Losses:

  • Localization-aware loss substantially improves over standard cross-entropy, with Dice Loss providing a smaller further boost.Table II presents the loss ablations.Source: 1) Losses · p. 6, Table II · p. 6

2) Segmentation Head Architecture:

  • The Encoder-Decoder segmentation head boosts performance over the simple segmentation head.The Encoder-Decoder design adds skip connections from lower-level backbone features.Source: 2) Segmentation Head Architecture · p. 6

3) Change Detection Head:

D. Generalization

The evaluation tests transfer from Tier 1 training data to Tier 3 data from different disasters, then compares RescueNet with a baseline. Transfer is stronger for building segmentation than damage assessment, while intermediate damage levels are classified an order of magnitude better than by the baseline.

D. Generalization

  • Tier 1 training to Tier 3 testing indicates generalization on building segmentation, but significant degradation on damage assessment.Tier 3 contains a different set of disasters; Table III reports the generalization experiments.Source: D. Generalization · p. 6, Table III · p. 6

E. Comparison

  • The baseline comparison retrains its component models on the authors’ dataset split because the original Tier 1 test set is unavailable.The retraining uses code provided by the baseline authors and the same hyperparameters.Source: E. Comparison · p. 6
  • RescueNet correctly classifies intermediate damage levels an order of magnitude better than the baseline.The overall-score comparison is presented in Table IV, and classwise damage-classification F1 scores in Table VI.Source: E. Comparison · p. 6, Table VI · p. 6, Table IV · p. 6
  • The retrained baseline component results closely reproduce those reported in the original paper.Table V presents the component-network results.Source: E. Comparison · p. 6, Table V · p. 6

VI. CONCLUSION

RescueNet is a unified model for segmenting buildings and assessing disaster damage from multi-temporal satellite imagery. Its multiheaded architecture and hierarchical loss support the composite task; the paper reports improved performance over the baseline and identifies separate building-instance resolution as future work.

VI. CONCLUSION

  • RescueNet jointly segments buildings and assesses their disaster damage using multi-temporal satellite imagery.The model uses a multiheaded architecture and hierarchical loss, with ablations examining the loss, segmentation head, and change-detection head.Source: §VI · p. 6
  • The authors report generalization across geographic regions and disaster types while significantly outperforming the existing baseline.Future work will focus on resolving separate building instances to improve building localization and damage assessment.Source: §VI · p. 6

Related papers

Loading 2004.07312v1…