Source-linked AI summary
FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understanding
Maryam Rahnemoonfar, Tashnim Chowdhury, Argho Sarkar, Debvrat Varshney, Masoud Yari, Robin Murphy
TL;DR
Post-disaster assessment lacks suitable high-resolution, multi-task aerial datasets, especially for disaster-focused VQA. FloodNet addresses this gap with low-altitude UAV imagery from Hurricane Harvey, annotations for three vision tasks, and baseline evaluations; the paper also identifies persistent difficulties with small objects and flooded structures.
Problem
Existing disaster datasets are mainly low-resolution satellite or noisy social-media imagery, while suitable datasets for disaster-damage VQA are absent.
Method
FloodNet provides low-altitude, high-resolution UAV imagery annotated for classification, semantic segmentation, and VQA, with baseline models evaluated across these tasks.
Results
FloodNet supports comparative analysis of image classification, semantic segmentation, and VQA, with ResNet, PSPNet, and MFB with co-attention respectively reported as strongest among the evaluated models.
Takeaways & Limitations
High-resolution UAV imagery offers a dataset basis for post-disaster computer vision across classification, segmentation, and visual question answering.
Takeaways & Limitations
FloodNet remains challenging for detecting small vehicles and pools and for distinguishing flooded buildings and roads from their non-flooded counterparts.
Abstract
from arXiv · showhide
Visual scene understanding is the core task in making any crucial decision in any computer vision system. Although popular computer vision datasets like Cityscapes, MS-COCO, PASCAL provide good benchmarks for several tasks (e.g. image classification, segmentation, object detection), these datasets are hardly suitable for post disaster damage assessments. On the other hand, existing natural disaster datasets include mainly satellite imagery which have low spatial resolution and a high revisit period. Therefore, they do not have a scope to provide quick and efficient damage assessment tasks. Unmanned Aerial Vehicle(UAV) can effortlessly access difficult places during any disaster and collect high resolution imagery that is required for aforementioned tasks of computer vision. To address these issues we present a high resolution UAV imagery, FloodNet, captured after the hurricane Harvey. This dataset demonstrates the post flooded damages of the affected areas. The images are labeled pixel-wise for semantic segmentation task and questions are produced for the task of visual question answering. FloodNet poses several challenges including detection of flooded roads and buildings and distinguishing between natural water and flooded water. With the advancement of deep learning algorithms, we can analyze the impact of any disaster which can make a precise understanding of the affected areas. In this paper, we compare and contrast the performances of baseline methods for image classification, semantic segmentation, and visual question answering on our dataset.
1. Introduction
FloodNet addresses the shortage of suitable aerial imagery for post-disaster scene understanding by introducing a high-resolution UAV dataset and evaluating classification, segmentation, and VQA.
- 1. Introduction: Existing aerial datasets mainly cover classification or semantic segmentation of a few classes and are collected under normal conditions.They do not address the distinctive challenges of post-disaster damage assessment.
- 1. Introduction: Satellite disaster datasets are often low-resolution, high-altitude, noisy, costly to collect, and unsuitable for scalable deep-learning training, while social-media imagery is noisy and not scalable.FloodNet instead provides low-altitude, high-resolution images intended to clarify post-disaster scenes.
- 1. Introduction: FloodNet introduces high-resolution UAV imagery for post-disaster damage assessment, addressing scarce and tedious aerial-image annotation.The dataset is intended for computer vision analysis of affected areas after flooding.
- 1. Introduction: The dataset supports image classification, semantic segmentation, and visual question answering, with pixel-wise annotations and manually produced questions.The overview figure presents these three task types.
- 1. Introduction: The paper compares baseline methods across the three computer vision tasks on FloodNet.The paper is organized around dataset description, experimental setups, result analysis, and conclusions.
2. Related Works
Prior disaster datasets and methods largely rely on ground or satellite imagery and address limited object classes, while FloodNet combines high-resolution UAV imagery with classification, segmentation, and disaster-focused VQA.
- 2.1. Datasets: Natural-disaster datasets include non-imaging data and imaging data from ground-level, satellite, and aerial sources.The related-work discussion organizes imaging datasets by capture position.
- 2.1. Datasets: Existing disaster imagery is commonly satellite-based, whereas social-media imagery can lack geolocation tags and sufficient scale for deep-learning training.Some UAV datasets exist but lack low-altitude, high-resolution imagery.
- 2.1. Datasets: FloodNet is presented as the only high-resolution UAV dataset collected after a hurricane that includes classification, semantic segmentation, and VQA.The comparison is summarized in Table 1.
- 2.2.1. Image Classification: Prior classification networks perform well on everyday images but are described as insufficient for aerial or spaceborne sensor datasets.The related work reviews their use in post-disaster detection, including social-media imagery and avalanche or fire applications.
- 2.2.2 Semantic segmentation: Semantic-segmentation research includes pyramid-pooling, encoder-decoder, and local-context models, while disaster applications often target specific classes rather than complete post-disaster scenes.FloodNet applies ENet, PSPNet, and DeepLabv3+ to its dataset.
- 2.2.3 Visual Question Answering: Disaster-focused VQA datasets were unavailable, so FloodNet provides UAV imagery for building and testing VQA algorithms and evaluates two baseline models.The baselines are Stacked Attention and MFB with Co-Attention.
3. The FloodNet Dataset
FloodNet is a high-resolution sUAV dataset collected after Hurricane Harvey for post-disaster damage assessment. It supports classification, semantic segmentation, and VQA through imagery and manually designed questions about flooded conditions.
- Dataset Collection: Imagery collected at 200 feet AGL during Hurricane Harvey captures post-flood damage and object attributes such as flooded or non-flooded states.Flights used DJI Mavic Pro quadcopters between August 30 and September 04, 2017, including response-phase imagery collected by emergency responders.
- Annotation Tasks: FloodNet defines three tasks aligned with response needs: classification of affected neighborhoods, segmentation of flooded buildings and roads, and VQA about object conditions.The dataset was designed with input from response and rescue teams.
- VQA Dataset: Approximately 11,000 manually created question-image pairs support VQA about buildings, roads, and the entire image.Each image has an average of 3.5 questions.
- VQA Dataset: VQA questions comprise Simple Counting, Complex Counting, and Condition Recognition groups.Simple Counting asks about object frequency, whereas Complex Counting targets flooded or non-flooded building attributes.
- VQA Dataset: Condition Recognition covers road or whole-image condition and includes yes/no questions such as whether a road is flooded.Questions begin with “Is” or “What”.
- VQA Dataset: Counting answers are most frequently ‘4, 3, 2, 1’, while ‘non-flooded, yes’ are common Condition Recognition answers.Images can contain both flooded and non-flooded buildings or roads, producing combined condition answers for some queries.
4. Experiments
FloodNet is evaluated through image classification, semantic segmentation, and visual question answering, revealing task-specific strengths and difficulties across baseline methods.
- Experimental Tasks: Three tasks evaluate FloodNet: classifying flooded versus non-flooded images, segmenting flooded regions and objects, and answering questions about the imagery.The study uses image classification, semantic segmentation, and VQA to assess flood-detection usability.
- Image Classification: ResNet achieves the highest test-set classification performance, outperforming Xception and InceptionNet despite their stronger ImageNet accuracy.The authors associate ResNet’s result with its residual architecture.
- Cross-task Interpretation: The classification findings indicate that high accuracy on everyday ImageNet images does not ensure effective feature detection in complex aerial disaster scenes.The paper motivates designing architectures specifically for urban-disaster imagery.
- Semantic Segmentation: PSPNet performs best across segmentation classes, while vehicles and pools are the most difficult objects to detect.Flooded buildings and roads are also challenging because their labels depend heavily on contextual distinctions.
- Semantic Segmentation: Flooded buildings and roads remain difficult for all segmentation models because distinguishing flooded from non-flooded objects depends on their respective contexts.This challenge persists even for DeepLabv3+ and PSPNet, which collect global contextual information.
- Visual Question Answering: MFB with co-attention outperforms the other VQA methods across question types, while counting questions are more challenging than condition recognition.Small objects make counting difficult, whereas whole-image and road conditions occupy larger visible regions.
5. Discussion and Conclusion
The paper presents FloodNet as a high-resolution, low-altitude UAV dataset for post-disaster assessment and evaluates it across three computer vision tasks. The discussion emphasizes both its broad annotation scope and persistent challenges involving small objects, building damage, and flooded-object context.
- Contribution: FloodNet provides high-resolution, low-altitude UAV imagery annotated for image classification, semantic segmentation, and visual question answering.The paper also describes the dataset collection procedure, features, and statistics.
- Contribution: FloodNet supports in-depth analysis of image classification, semantic segmentation, and visual question answering for post-natural-disaster damage assessment.These are the three computer vision tasks evaluated in the paper.
- Challenges: Vehicles and pools are difficult to detect because they are small, while flooded buildings and roads remain challenging for segmentation.Top-view UAV imagery makes estimating building damage difficult, and road classification depends on contextual and current-state cues.
- Challenges: Current state-of-the-art models lack good performance on FloodNet when distinguishing flooded from non-flooded roads and buildings.The paper identifies corresponding object context as central to this distinction.