Source-linked AI summary
RescueNet: A High Resolution UAV Semantic Segmentation Benchmark Dataset for Natural Disaster Damage Assessment
Maryam Rahnemoonfar, Tashnim Chowdhury, Robin Murphy
TL;DR
Post-disaster scene understanding lacks comprehensive pixel-level benchmarks that cover more than a limited set of classes. RescueNet is a high-resolution UAV dataset with detailed classification and segmentation annotations, and tested models show its utility for disaster-scene analysis and transfer learning.
Problem
Post-disaster scene understanding lacks datasets with comprehensive pixel-level annotations, despite roads, vehicles, trees, and debris contributing to fuller scene descriptions and more accurate damage assessment.
Source: Background & Summary · pp. 1–2Method
RescueNet is a high-resolution UAV dataset of Hurricane Michael imagery with pixel-level annotations for 10 classes and both classification and semantic-segmentation labels.
Source: Background & Summary · p. 2Results
Attention UNet performs best among the tested methods, PSPNet outperforms DeepLabv3+ among non-attention methods, a heavier Segmenter backbone performs better, and transfer learning improves FloodNet segmentation.
Source: Experimental Result Analysis · pp. 8–9Takeaways & Limitations
RescueNet supports extracting scene information and detecting objects after disasters, and its learned weights can improve segmentation accuracy on another dataset.
Source: Experimental Result Analysis · p. 9Building-damage definitions are adapted to aerial top views, which provide only a view from above, using input from first responders.
Source: Annotation for Semantic Segmentation · p. 4
Abstract
from arXiv · showhide
Recent advancements in computer vision and deep learning techniques have facilitated notable progress in scene understanding, thereby assisting rescue teams in achieving precise damage assessment. In this paper, we present RescueNet, a meticulously curated high-resolution post-disaster dataset that includes detailed classification and semantic segmentation annotations. This dataset aims to facilitate comprehensive scene understanding in the aftermath of natural disasters. RescueNet comprises post-disaster images collected after Hurricane Michael, obtained using Unmanned Aerial Vehicles (UAVs) from multiple impacted regions. The uniqueness of RescueNet lies in its provision of high-resolution post-disaster imagery, accompanied by comprehensive annotations for each image. Unlike existing datasets that offer annotations limited to specific scene elements such as buildings, RescueNet provides pixel-level annotations for all classes, including buildings, roads, pools, trees, and more. Furthermore, we evaluate the utility of the dataset by implementing state-of-the-art segmentation models on RescueNet, demonstrating its value in enhancing existing methodologies for natural disaster damage assessment.
Background & Summary
RescueNet addresses the lack of comprehensive pixel-level post-disaster scene benchmarks with high-resolution UAV imagery and labels spanning scene elements and damage severity. The paper also evaluates existing segmentation methods on this dataset.
Background & Summary
- Prior disaster datasets include ground-level image-classification collections, satellite and aerial building-damage datasets, AIDER classification, and FloodNet’s nine-class UAV segmentation.Source: Background & Summary · pp. 1–2
- Post-disaster scene understanding lacks comprehensive pixel-level benchmarks, although roads, vehicles, trees, and debris contribute to a fuller scene account and more accurate damage assessment.Source: Background & Summary · pp. 1–2
- FloodNet and RescueNet are the only datasets identified as offering comprehensive semantic segmentation for high-resolution UAV imagery.Source: Background & Summary · p. 2
- RescueNet provides high-resolution UAV imagery and annotations for 10 classes, including affected and non-affected areas following Hurricane Michael.Source: Background & Summary · p. 2
- Buildings receive four severity labels, while roads are classified as Road-Clear or Road-Blocked.Source: Background & Summary · p. 2
- The paper evaluates existing semantic segmentation methods on RescueNet to demonstrate its utility for natural-disaster damage-assessment research.Source: Background & Summary · p. 2
Methods
RescueNet uses responder-collected Hurricane Michael UAV imagery and pairs pixel-level scene and building-damage annotations with image-level neighborhood damage labels. The paper also describes iterative annotation review and conversion of JSON annotations into segmentation masks and classification labels.
Data Collection
- 80 flights collected sUAV imagery during the response phase in Mexico Beach and other Hurricane Michael-affected areas; the dataset excludes videos.The collection took place October 11–14, 2018, using small UAVs operated on behalf of Florida’s emergency response team.Source: Data Collection · p. 3
Annotation for Semantic Segmentation
- Pixel-level annotations cover buildings, roads, trees, water, vehicles, and pools, with four building-damage levels and aerial-view definitions adapted using first-responder input.Roads are labeled clear or blocked depending on whether obstacles cover them.Source: Annotation for Semantic Segmentation · p. 4
Annotation for Image Classification
- Each image receives a neighborhood-level label of superficial, medium, or major damage based on damage across man-made and natural structures.Major damage applies when an image contains a completely destroyed building or about 50% of its area is debris-covered.Source: Annotation for Image Classification · p. 5
Quality Control
- A two-step annotation process assigns each image to an annotator and then a reviewer, with discrepancies returned for correction until the reviewer is satisfied.Pixel-level annotation takes approximately one hour per image.Source: Quality Control · p. 5
Generation of Segmentation Mask and Classification Label
- JSON polygon annotations generate segmentation masks by assigning class values to pixels, while neighborhood tags supply image-classification labels.The segmentation scheme assigns values 1–10 to water, four building-damage classes, vehicles, clear and blocked roads, trees, and pools; background is 0.Source: Generation of Segmentation Mask and… · p. 6
Data Records
RescueNet records semantic segmentation, image-level damage classifications, and training, validation, and test splits for Hurricane Michael UAV imagery. Its building annotations distinguish four damage levels, while image labels use three neighborhood damage categories.
Semantic Segmentation
- 4,011 no-damage, 3,119 minor-damage, 1,693 major-damage, and 2,080 total-destruction building polygons accompany pixel labels for scene classes.The dataset was collected using UAVs in areas affected by Hurricane Michael.Source: Semantic Segmentation · p. 6
Image Classification
- 1,389 images are labeled Superficial Damage, 1,906 Medium Damage, and 1,199 Major Damage.These are image-level classifications in addition to pixel-level semantic segmentation labels.Source: Image Classification · p. 6
Dataset Splits
- 3,595 training, 449 validation, and 450 test images follow an 80/10/10 allocation within each damage category.The categories used to balance the splits are superficial, medium, and major damage.Source: Dataset Splits · p. 7
Technical Validation
RescueNet is used to evaluate semantic segmentation models on pixel-level post-disaster imagery, with IoU as the evaluation metric. Attention UNet leads the reported non-attention comparisons, while RescueNet pretraining also improves segmentation results on FloodNet.
Problem Details
- RescueNet contains pixel-level annotations for 10 object classes, with detailed damage levels for some classes, including buildings and roads.Source: Problem Details · p. 7
Selected Semantic Segmentation Models
- The evaluated models span pyramid pooling, atrous encoder-decoder, attention-gated U-Net, and transformer-based segmentation designs.The paper describes PSPNet, DeepLabv3+, Attention U-Net, and Segmenter.Source: Selected Semantic Segmentation Models · p. 7
Training Details
- Training uses 713 × 713 images, a ResNet-101 backbone for PSPNet and DeepLabv3+, and scaling, flipping, shuffling, and rotation augmentation.The reported base learning rate is 0.001.Source: Training Details · p. 7
Evaluation Metric
- IoU measures the fraction of pixels shared by the target and predicted masks out of the pixels present across both masks.Source: Evaluation Metric · p. 8
Experimental Result Analysis
- Attention UNet29 performs best against non-attention methods, PSPNet26 outperforms DeepLabv3+27, and Segmenter performs better with ViT-Small than ViT-Tiny.Table 5 reports per-class results on the RescueNet testing set.Source: Experimental Result Analysis · pp. 8–9, Table 5 · p. 8
- RescueNet pretraining improves DeepLabv3+ and PSPNet segmentation performance on FloodNet.Models are first trained on RescueNet, then their learned weights are used to train on FloodNet; Table 6 reports per-class FloodNet test results.Source: Table 6 · p. 8, Experimental Result Analysis · p. 9
Usage Notes
RescueNet’s labels are intended to support damage assessment and practical response planning. Its road and building annotations address decisions about rescue effort allocation and route selection.
- Building damage classes are intended to help first responders and planners allocate rescue efforts.Source: Usage Notes · p. 9
- Road labels distinguish clear from blocked routes, with blocked roads indicating floodwater or debris from damaged buildings or trees.The paper links accurate segmentation of both road classes to finding efficient routes for rescue teams.Source: Usage Notes · p. 9
- The authors propose combining RescueNet with other datasets and using its semantic labels in semi-supervised or self-supervised disaster assessment.Source: Usage Notes · p. 9