Source-linked AI summary
Deep-Learning-Based Aerial Image Classification for Emergency Response Applications Using Unmanned Aerial Vehicles
Christos Kyrkou, Theocharis Theocharides
TL;DR
UAVs can provide aerial monitoring in emergency settings, but onboard deep-learning classification must contend with limited embedded computing resources. This paper introduces the AIDER dataset and ERNet; it reports near-state-of-the-art accuracy with faster embedded inference and substantially less memory than existing models.
Problem
Deep-learning CNNs can classify images, but their inference speed and model size challenge onboard UAV processing where embedded resources and connectivity may be limited.
Source: §1 · pp. 1–2Method
The paper constructs the AIDER emergency-response aerial-image dataset and develops ERNet, a lightweight CNN for low-cost, low-power devices.
Source: §1 · p. 2Results
Up to 3× faster on an embedded platform, ERNet requires two orders of magnitude less memory and provides similar accuracy to existing models.
Source: §5 · p. 9Takeaways & Limitations
Abstract
from arXiv · showhide
Unmanned Aerial Vehicles (UAVs), equipped with camera sensors can facilitate enhanced situational awareness for many emergency response and disaster management applications since they are capable of operating in remote and difficult to access areas. In addition, by utilizing an embedded platform and deep learning UAVs can autonomously monitor a disaster stricken area, analyze the image in real-time and alert in the presence of various calamities such as collapsed buildings, flood, or fire in order to faster mitigate their effects on the environment and on human population. To this end, this paper focuses on the automated aerial scene classification of disaster events from on-board a UAV. Specifically, a dedicated Aerial Image Database for Emergency Response (AIDER) applications is introduced and a comparative analysis of existing approaches is performed. Through this analysis a lightweight convolutional neural network (CNN) architecture is developed, capable of running efficiently on an embedded platform achieving ~3x higher performance compared to existing models with minimal memory requirements with less than 2% accuracy drop compared to the state-of-the-art. These preliminary results provide a solid basis for further experimentation towards real-time aerial image classification for emergency response applications using UAVs.
1 Introduction
The paper targets real-time aerial hazard classification on UAVs despite embedded hardware constraints. It introduces the AIDER dataset and ERNet, a compact CNN designed for onboard use.
1 Introduction
- Onboard processing is challenging because disaster settings may limit connectivity while UAV hardware constrains computing power and memory.Source: §1 · pp. 1–2
- The work constructs a dedicated emergency-response aerial-image database and develops ERNet for low-cost, low-power devices.Source: §1 · p. 2
- ∼90% accuracy and 3× faster inference on an embedded platform are reported for the proposed approach.The authors describe the accuracy as near state of the art.Source: §1 · p. 2
2 Background and Related Work
The background reviews CNN architectures and prior disaster-image classifiers, then identifies the need for models suited to onboard UAV processing. Existing approaches often rely on desktop-class systems or classify a single event.
2.1 Convolutional Neural Network Architectures
- VGG16 uses stacked 3 × 3 convolutions but has high memory demands, while ResNet uses residual learning and MobileNet uses separable convolutions.The review presents these architectural ideas as relevant to efficient aerial-scene classification.Source: §2.1 · p. 2
2.2 Image classification for emergency response and disaster management
- Prior disaster-image studies include fire classifiers, avalanche-debris detection, and aerial-photo classification, using varied datasets and computing platforms.Reported examples range from cloud or desktop processing to GPU-based fire recognition.Source: §2.2 · p. 3
- The reviewed approaches primarily adapt pretrained networks for single events and use desktop-class systems, while connectivity and embedded-computing limits motivate onboard processing.Source: §2.2 · p. 3
3 Deep Learning for Aerial Disaster-Event Classification
The section develops an aerial-disaster classification pipeline for embedded UAV platforms, pairing a dedicated dataset with transfer-learning and custom-CNN approaches. It also describes data augmentation, network design choices, and balanced-batch training.
3.1 Dataset Collection
- AIDER was constructed because the authors found no widely used public emergency-response dataset, with images for four disaster events and a Normal class.The collected classes are Fire/Smoke, Flood, Collapsed Building/Rubble, Traffic Accidents, and Normal.Source: §3.1 · p. 3
- The dataset combines images from multiple sources and UAV collection, includes varied viewpoints and conditions, and is augmented with geometric and appearance transformations.The transformations are intended to increase variability and combat overfitting; the caption identifies Figure 1 as examples from the augmented database.Source: §3.1 · pp. 3–4, Figure 1 · p. 4
3.2 CNNs for Aerial Disaster Classification
- The comparison explores pretrained transfer-learning networks and custom networks trained from scratch to examine accuracy-performance trade-offs for embedded use.Source: §3.2 · p. 4
3.2.1 Transfer Learning Networks
- Transfer learning freezes pretrained feature extractors and adds a classifier, using global average pooling before dense layers to reduce parameters and resource demands.The evaluated pretrained networks include VGG16, ResNet50, and MobileNet.Source: §3.2.1 · pp. 4–5
3.2.2 Custom Networks
- Custom-network design combines a small first layer, early downsampling, and a capped pyramid-shaped architecture with fully convolutional classification and regularization.The design also examines network depth and residual connections as accuracy-computation trade-offs.Source: §3.2.2 · pp. 5–6
- The network variants progress from baseNet to SCNet and SCFCNet, then ERNet adds residual connections; the configurations vary convolution types and classifier structure.Source: §3.2.2 · p. 6
3.3 Training
- Training uses shared evaluation conditions and a 60:20:20 train-validation-test split, with equal class representation in each batch to address Normal-class imbalance.The networks use resized inputs, with 240 × 240 pixels as the default where applicable.Source: §3.3 · p. 6
4 Experimental Evaluation and Results
The evaluation compares model accuracy and processing speed, examines ERNet’s decision cues, and tests ERNet on an embedded UAV platform. ERNet combines near-state-of-the-art accuracy with high throughput and a small memory footprint.
4.1 Performance Metrics
- Frame-rate measures images processed per second, while average accuracy (A) averages accuracy across classes to avoid bias from uneven class distributions.Frame-rate is inversely proportional to the time needed to process one image frame.Source: §4.1 · p. 7
4.2 Overall Performance, Analysis and Comparison
- VGG16 reaches 91.9% A at 2 FPS, while MobileNet reaches 20 FPS and 88.5% average accuracy at 224 × 224 resolution.The pretrained networks each require over 10MB, which the authors identify as potentially prohibitive for embedded on-chip storage.Source: §4.2 · p. 7, Table 1 · p. 7
- 90% average accuracy, over 50 FPS on a CPU platform, and ~300KB memory make final ERNet close to pretrained approaches in accuracy with a compact footprint.The paper presents ERNet’s speed and memory requirements as suitable for real-time UAV applications and on-chip storage on low-power platforms.Source: §4.2 · p. 7
4.3 Evaluation of Learning
- Grad-CAM visualizations of correctly classified scenes indicate that ERNet uses class-specific cues, including building damage, fire glow, water, and road or train features.The evaluation uses the heat maps to inspect image regions influencing classification decisions.Source: §4.3 · p. 8, Figure 3 · p. 8
4.4 Embedded Platform Results
- On the Odroid-XU4 aboard a DJI Matrice 100, ERNet achieves ~9 FPS, versus at most ~3 FPS for other state-of-the-art models that have lower accuracy.The authors note that quantization and bit reduction could further improve CPU-platform frame-rates.Source: §4.4 · pp. 8–9, Figure 4 · p. 9
5 Conclusions and Future Work
The paper introduces a dedicated aerial-image dataset for emergency response and develops ERNet, a small, efficient CNN for disaster-event classification on UAVs. It reports similar accuracy to existing models with up to 3× faster embedded performance and identifies several directions for further work.
5 Conclusions and Future Work
- The work introduces an aerial emergency-response dataset and ERNet, which is up to 3× faster on an embedded platform, uses two orders of magnitude less memory, and provides similar accuracy to existing models.The authors characterize these efforts as first steps toward automated real-time disaster-event classification onboard a UAV.Source: §5 · p. 9
- Future work includes expanding the dataset and investigating atrous convolutions, multi-frame processing, and different color spaces.The stated aims include adding images and classes and improving existing models and techniques.Source: §5 · p. 9
- Combining ERNet with people and vehicle detection or infrared imagery is proposed as a route to more enhanced situational awareness for emergency response.Source: §5 · p. 9