Source-linked AI summary
An Autonomous Drone for Search and Rescue in Forests using Airborne Optical Sectioning
D. C. Schedl, I. Kurmi, O. Bimber
TL;DR
Autonomous SAR drones must locate people in densely occluded forests despite constrained sampling and remote-area connectivity. The paper combines airborne optical sectioning with onboard classification and adaptive path planning; 1D sampling performs as well as 2D sampling with ten times as many samples. The prototype supports real-time operation and intermittent minimal-bandwidth communication, but tests were constrained by visual-line-of-sight rules and short battery life.
Problem
Fully autonomous SAR drones need efficient person classification and path planning for densely occluded forests and remote areas with limited network coverage.
Method
The prototype uses airborne optical sectioning, one-dimensional synthetic-aperture sampling, onboard real-time processing, and adaptive potential-field path planning.
Results
1D synthetic-aperture imaging performs equally well as 2D imaging using ten times as many samples despite GPS errors.
Takeaways & Limitations
Onboard processing supports autonomous SAR searches with intermittent minimal-bandwidth connections by transmitting classification results for detections.
Takeaways & Limitations
Visual-line-of-sight regulations and 15-20 minute battery life restricted test sites to 0.3-4.0 hectares.
Abstract
from arXiv · showhide
Drones will play an essential role in human-machine teaming in future search and rescue (SAR) missions. We present a first prototype that finds people fully autonomously in densely occluded forests. In the course of 17 field experiments conducted over various forest types and under different flying conditions, our drone found 38 out of 42 hidden persons; average precision was 86% for predefined flight paths, while adaptive path planning (where potential findings are double-checked) increased confidence by 15%. Image processing, classification, and dynamic flight-path adaptation are computed onboard in real-time and while flying. Our finding that deep-learning-based person classification is unaffected by sparse and error-prone sampling within one-dimensional synthetic apertures allows flights to be shortened and reduces recording requirements to one-tenth of the number of images needed for sampling using two-dimensional synthetic apertures. The goal of our adaptive path planning is to find people as reliably and quickly as possible, which is essential in time-critical applications, such as SAR. Our drone enables SAR operations in remote areas without stable network coverage, as it transmits to the rescue team only classification results that indicate detections and can thus operate with intermittent minimal-bandwidth connections (e.g., by satellite). Once received, these results can be visually enhanced for interpretation on remote mobile devices.
Introduction
The paper develops autonomous drone capabilities for SAR in occluded forests using Airborne Optical Sectioning and onboard adaptive processing. Its contributions target efficient one-dimensional sampling and dynamic flight-path adaptation.
- Airborne Optical Sectioning: Airborne Optical Sectioning combines images captured from aircraft poses to suppress occlusion and reveal targets hidden in forests.The method computationally integrates images sampled over large synthetic apertures.
- Contributions: The paper shows that person classification remains effective with one-dimensional synthetic apertures and imprecise GPS and IMU pose measurements.This addresses a central limitation of conventional two-dimensional aperture sampling and precise pose estimation.
- Contributions: 92-93% average precision was obtained for imprecise one-dimensional sampling of around 30 images and precise two-dimensional sampling of around 300 images.The comparable classification scores support onboard processing with mobile processors.
- Contributions: Adaptive online sampling dynamically changes the flight path using classification confidences to re-sample regions containing weak detection hints.This enables the drone to make search decisions during flight.
Results
The experiments evaluate one-dimensional synthetic-aperture classification and autonomous search under predefined and adaptive flight paths. Results show efficient occlusion handling, real-time operation, and dynamic prioritization of likely search cells.
- 1D Synthetic Aperture Classification: 10× fewer images are required for one-dimensional than two-dimensional synthetic-aperture sampling over the same area.The example uses approximately 30 rather than 300 images, with trade-offs including less efficient occlusion removal and defocus.
- 1D Synthetic Aperture Classification: A small sampling count is sufficient because visibility improvement in integral images flattens beyond a relatively low limit.The reported statistical model supports practical use of synthetic-aperture imaging in time-critical SAR applications.
- Predefined Search: Eight new test flights covered approximately 6 ha while processing ran in real time onboard the drone along predefined waypoints.The drone classified integral images after scanning 30 m segments with one sample per meter.
- Adaptive Search: Adaptive search selected the next 30 m × 30 m cell using initial cell probabilities, current position, and potential-field evaluation.The approach dynamically computes the flight path to search quickly and reliably.
- Adaptive Search: Weak detections with confidence of at least 5% triggered re-sampling of the corresponding cell.Scanned-cell probabilities were set to zero to avoid revisiting cells, and search stopped after confirmation, a maximum flight time, or full coverage.
Discussion
The discussion connects one-dimensional synthetic-aperture imaging with autonomous, real-time SAR operation under limited connectivity. It also identifies visual-interpretation constraints and field-test boundaries.
- Implications: 1D synthetic-aperture imaging performs as well as 2D imaging using ten times as many samples despite GPS errors.Image integration remains necessary for effective classification.
- Implications: Onboard processing enables transmission of detection classifications over intermittent minimal-bandwidth connections instead of requiring stable high-speed coverage.Received integral images can be visually enhanced on remote mobile devices, although GPS-related misregistration complicates direct interpretation.
- Limitations: 15-20 minutes of battery life and visual-line-of-sight regulations restricted test sites to 0.3-4.0 hectares.The authors identify these constraints as limitations of the prototype and field experiments.
Materials and Methods
The prototype integrates thermal imaging, onboard computing, communications, and vision processing, while terrain-aware image integration uses digital elevation models. Adaptive target-cell planning runs onboard from probability maps and detection results.
- Prototype: The octocopter carried a thermal camera, Raspberry Pi system-on-chip computer, LTE communication hardware, and a vision processing unit.These components supported sensing, onboard processing, communications, and classification.
- Image Processing: Integral images were computed using a digital elevation model rather than assuming a planar ground surface.Test-site terrain was converted into triangle meshes for image integration.
- Path Planning: Adaptive target-cell selection used potential-field equations evaluated from the drone position, probability map, and previous detection result.Planning the next target cell required 1 ms of computation on the onboard system-on-chip.
Figures and Tables
Figure 1 presents the autonomous drone platform and shows how airborne optical sectioning supports remote SAR operations.
- The drone transmits an integral image with the classification result and location to a rescue team’s mobile device after a positive finding.
A B C
The figures and tables compare one- and two-dimensional synthetic-aperture imaging and summarize predefined and adaptive forest searches. They show how integral images, flight paths, and confidence changes are represented across experiments.
- Synthetic-aperture imaging: Figure 2 contrasts occluded single thermal images with focused integral images from dense 2D sampling and defocused images from 1D sampling with GPS/IMU pose estimation.
- Synthetic-aperture imaging: Figure 3 shows classification performance and modeled visibility increasing as the number of integrated sample images N increases.
- Synthetic-aperture imaging: Table 1 reports 1D synthetic-aperture results across path lengths N using ground-truth labels, true positives, false positives, and average precision.
- Predefined search: Figure 4 depicts a predefined F16 flight divided into 30 m segments, with integral images and person classifications mapped onto satellite and elevation views.
- Predefined search: Table 2 summarizes predefined-search flights by coordinates, date, path length, forest type, average precision, persons present, correct findings, and false alerts.
- Adaptive search: Figure 5 illustrates adaptive paths based on potential fields, including scanned cells, fast unsampled segments, and re-sampling that confirms or rejects detections.
- Adaptive search: Table 3 reports adaptive-search path information and confidence changes between initial sampling and re-sampling for correct and incorrect detections.
SUPPLEMENTARY MATERIALS
The supplementary materials provide videos, test-site imagery, experiment visualizations, registration-enhancement results, and the fitted average-precision curve.
- Videos: Movie S1 documents the autonomous drone and a sample test site.
- Videos: Movie S2 illustrates improved classification performance as the number of 1D synthetic-aperture samples N increases across four test flights.
- Videos: Movie S3 visualizes predefined flight F16, including its field of view, camera positions, integral images, classification results, and person locations.
- Videos: Movie S4 shows adaptive paths under different probability maps, distinguishing scanned segments from fast flight segments without sampling.
- Test sites: Figure S1 provides satellite and RGB imagery for broadleaf, mixed, conifer, and second mixed forest test sites.
- Experiment results: Figure S2 presents integral images and classification results for predefined-search flights, including a false classification in F12.
- Experiment results: Figure S3 shows additional adaptive-search detections and their confirmation through re-sampled integral images.
- Image enhancement: Figure S4 compares real-time GPS/IMU integral images with automatically registration-enhanced regions of interest for correct detections.