Source-linked AI summary
An Autonomous Drone for Search and Rescue in Forests using Airborne Optical Sectioning
D. C. Schedl, I. Kurmi, O. Bimber
TL;DR
Autonomous search in occluded forests raises whether person classification can remain effective with 1D sampling and imprecise pose estimates. The paper presents a drone using 1D synthetic-aperture imaging and onboard adaptive search, and reports similar 92–93% average precision from imprecise 1D sampling of around 30 images and precise 2D sampling of around 300.
Problem
The paper addresses whether person classification under forest occlusion remains effective with 1D synthetic-aperture sampling and imprecise GPS/IMU pose estimates for autonomous SAR.
Source: Introduction · p. 3Method
The prototype uses 1D synthetic-aperture imaging and computes person classification and adaptive flight-path changes onboard in real time.
Source: Results · p. 5, Discussion · p. 6Results
92–93% average precision scores were similar for imprecise 1D sampling with around 30 images and precise 2D sampling with around 300 images.
Source: Introduction · p. 3, Discussion · p. 6Takeaways & Limitations
The relatively low limits on aperture size and sampling rate make airborne optical sectioning practical for time-critical applications such as SAR.
Source: Results · p. 5Testing was limited to 0.3–4.0-hectare sites because visual-line-of-sight rules and 15–20-minute battery life constrained flights.
Source: Discussion · p. 7
Abstract
from arXiv · showhide
Drones will play an essential role in human-machine teaming in future search and rescue (SAR) missions. We present a first prototype that finds people fully autonomously in densely occluded forests. In the course of 17 field experiments conducted over various forest types and under different flying conditions, our drone found 38 out of 42 hidden persons; average precision was 86% for predefined flight paths, while adaptive path planning (where potential findings are double-checked) increased confidence by 15%. Image processing, classification, and dynamic flight-path adaptation are computed onboard in real-time and while flying. Our finding that deep-learning-based person classification is unaffected by sparse and error-prone sampling within one-dimensional synthetic apertures allows flights to be shortened and reduces recording requirements to one-tenth of the number of images needed for sampling using two-dimensional synthetic apertures. The goal of our adaptive path planning is to find people as reliably and quickly as possible, which is essential in time-critical applications, such as SAR. Our drone enables SAR operations in remote areas without stable network coverage, as it transmits to the rescue team only classification results that indicate detections and can thus operate with intermittent minimal-bandwidth connections (e.g., by satellite). Once received, these results can be visually enhanced for interpretation on remote mobile devices.
Affiliations … Introduction
The study presents an autonomous drone that uses airborne optical sectioning and adaptive sampling to find people in occluded forests, addressing time and imaging constraints in search and rescue.
Introduction
- Civil emergency services increasingly use UAVs because they are more flexible and cheaper to acquire, maintain, and operate than manned aircraft, though technical and legal obstacles remain.Applications include search and rescue, firefighting, policing, traffic control, and disaster management.Source: Introduction · p. 2
- UAV path planning addresses energy, speed, payload, and environmental constraints through global and local optimization methods, with adaptive approaches also studied for obstacle avoidance, tracking, and simulated indoor rescue.Prior work spans single, cooperative, and non-cooperative UAVs and methods including sampling, graph-based, evolutionary, mathematical, and machine-learning approaches.Source: Introduction · p. 2
- Synthetic-aperture sensing combines small-aperture measurements to approximate wide-aperture imaging, addressing occlusion from conventional cameras, while camera-array apertures remain constrained by physical array size.Prior applications include radar, radio astronomy, interferometric microscopy, LiDAR, and visible-light imaging.Source: Introduction · pp. 2–3
- Airborne Optical Sectioning computationally integrates images captured across a synthetic aperture to suppress occlusion and reveal targets hidden in individual recordings.The method projects ground-surface points into sampling perspectives and averages the corresponding pixel values; it can also use thermal or near-infrared wavelengths.Source: Introduction · p. 3
- Prior SAR work found that integrating images before classification is more effective for partially occluded people, but fixed two-dimensional sampling and offline processing made the approach slow and unable to adapt to local occlusion.Computer-vision pose estimation could take hours even on high-end GPUs.Source: Introduction · p. 3
- Across 17 field experiments, the prototype found 38 of 42 hidden persons; predefined paths yielded 86% average precision, while adaptive planning improved confidence by 15%.Predefined paths found 88% of people with one false alert; adaptive planning found 100% with no false alert.Source: Introduction · p. 3
- Imprecise one-dimensional sampling with around 30 images and precise two-dimensional sampling with around 300 images yielded similar average precision scores of 92-93%.The comparison came from resampling previous test flights; one-dimensional sampling used imprecise GPS and IMU measurements for pose estimation.Source: Introduction · p. 3
- Adaptive online sampling uses potential fields and classifier confidences to change the flight path and resample regions when even weak signals suggest a hidden person.The drone makes these sampling decisions during flight.Source: Introduction · p. 3
Results
The results show that one-dimensional synthetic-aperture sampling can retain high classification performance with far fewer images, while predefined and adaptive searches detect people and use resampling to validate potential detections.
Predefined Search
- 30 of 34 people were found in predefined searches, with 86% average precision and one incorrect detection.The 10% confidence threshold was selected to minimize false positives; the lower average precision than in the earlier experiments reflects compass errors in fully automatic flights.Source: Results · p. 5
Adaptive Search
- Resampling increased true-positive confidence by around 15% and decreased false-positive confidence by around 16% across nine adaptive-search flights.The drone selected cells using a potential-field plan informed by rescue-team probabilities, then orthogonally resampled detections; without resampling, the 10% threshold would have produced two missed people and two erroneous detections in four of nine flights.Source: Results · pp. 5–6
Discussion
The discussion finds that sparse 1D synthetic-aperture imaging can classify people as effectively as 2D imaging with ten times as many samples, while noting that image integration remains necessary. Onboard processing enables detection-only transmission over minimal-bandwidth links, but field constraints and technical limitations remain.
Discussion
- 1D synthetic-aperture imaging performs equally well as 2D imaging with 10 times the samples, despite GPS errors, but image integration remains necessary for effective classification.This finding motivated the development of a fully autonomous SAR drone that processes imagery onboard in real time.Source: Discussion · p. 6
- Onboard processing allows the drone to transmit only detection results over intermittent minimal-bandwidth connections, while remote-device enhancement supports visual interpretation.GPS pose-error misregistration makes transmitted integral detection images difficult to interpret; post-processing can improve registration, but whole-image optimization is too slow and error-prone, and is not needed for automatic classification.Source: Discussion · pp. 6–7
- Field trials were limited by visual-line-of-sight rules, 15–20-minute battery life, and 0.3–4.0-hectare test sites; remaining compass errors reduced classification rate from 93% to 86%.The 1-image/second thermal camera constrained sampled flight to 1 m/s and classification updates to 30 m intervals; faster cameras could enable higher speeds and per-image updates. Potential-field path planning is not globally optimal, motivating investigation of larger planning neighborhoods, multi-drone search, and improvements to occlusion removal and classification.Source: Discussion · p. 7
- The authors identify professional combustion-engine drones, improved autonomous-flight regulations, and applications beyond SAR as future directions.They note that professional drones can fly up to 6 hours and carry payloads up to 75 kg, and identify wildlife observation, fire-hot-spot discovery, surveillance, and border control as potential applications.Source: Discussion · p. 7
Materials and Methods
The study implemented an onboard thermal-imaging drone pipeline for detecting people in forest search areas, using digital elevation models to form integral images and supporting both predefined and adaptive flight paths.
Materials and Methods
- The octocopter processed thermal images onboard, forming integral images on a Raspberry Pi and detecting people with YOLOv4-tiny on a VPU in 84 ms per integral image.Preprocessing 30 images took around 748 ms and integration took 90 ms; detections were sent to a remote mobile device via LTE.Source: Materials and Methods · p. 7
- Integral images used a digital elevation model rather than assuming planar ground, with a triangle mesh generated from freely available GeoTIFF elevation data.The central position of each synthetic aperture was used to compute integral images, where visible people were labeled.Source: Materials and Methods · p. 8
- Training data were expanded to 490 augmentations per synthetic aperture by varying image rotation, DEM altitude, and the number of samples.The detector classified one person class in 512 px × 512 px integral images; weights were selected by highest AP at 25% IoU.Source: Materials and Methods · p. 8
- Experiments used separate training, validation, and test flights, with detections evaluated using AP, persons found, and persons incorrectly found under specified confidence and overlap thresholds.For Tab. 2, the drone detected after each 30 m segment; a person counted as found if a detection at 10% confidence or higher overlapped the ground-truth label, using a 1% IoU threshold.Source: Materials and Methods · p. 8
- Adaptive-path experiments evaluated potential-field equations onboard to choose the next target cell from the drone position, probability map, and previous detection result.Planning the next target cell took 1 ms.Source: Materials and Methods · p. 8
Figures and Tables
Figure 1 depicts the drone setup used in the experiments and the basic principle of airborne optical sectioning (AOS).
Figures and Tables
- Figure 1 shows the camera drone and recording and processing equipment, with positive findings transmitted as an integral image containing the classification result and location.The image is sent to a rescue-team mobile device.Source: Figure 1 · p. 13
- Figure 1 also illustrates the basic principle of airborne optical sectioning (AOS).Supplementary Movie S1 shows drone footage and the sample test site.Source: Figure 1 · p. 13
A B C
The section compares two- and one-dimensional synthetic-aperture imaging and describes classification and search experiments using 1D sampling, predefined paths, and adaptive resampling. It reports that classification performance and modeled visibility improve as more sample images are integrated.
A B C
- Dense 2D synthetic-aperture sampling removed strong occlusion, while 1D sampling with GPS/IMU pose estimation produced defocused integral images.Computer-vision pose estimation produced well-defined person shapes in the densely sampled 2D images.Source: Figure 2 · p. 13
- Classification performance increased across all test flights, and modeled visibility improved, as the number of integrated sample images N increased.The table reports 1D synthetic-aperture results for varying path lengths; F9 was an empty forest, and fewer segments were resampled for F2 than for the other flights.Source: Figure 3 · p. 13, Table 1 · p. 14
- Predefined-path experiments evaluated flights in different forests using average precision and person-specific counts of people present, correctly found, and incorrectly detected.Paths were divided into 30 m segments plus residue; overlapping fields of view could label a person multiple times, so results report person-specific counts rather than label-specific counts.Source: Figure 4 · p. 14, Table 2 · p. 15
- Adaptive search used potential fields to plan sampled and fast unsampled segments, then resampled potential detections to confirm classifications and assess confidence changes.The table defines confidence change from initial sampling to resampling and final confidence after resampling; the figure illustrates one correct and one incorrect confirmed classification, as well as an incorrect detection in an empty scene.Source: Figure 5 · p. 15, Table 3 · p. 16
SUPPLEMENTARY MATERIALS
Supplementary materials illustrate the drone, test sites, classification behavior, and predefined and adaptive searches, and provide additional AP-fit and image-registration results.
Prototype and test sites
- Movie S1 shows the prototype at sample test site D, while Figure S1 documents broadleaf, mixed, and conifer sites and their associated test flights.Flights F14 and F15 covered both conifer and broadleaf sites.Source: SUPPLEMENTARY MATERIALS · p. 17, Figure S1 · p. 17
Classification performance
- Classification performance improves as sample count N increases across four test flights: broadleaf, conifer, mixed forest, and empty broadleaf forest.The video distinguishes ground-truth labels, true positives, and false positives.Source: SUPPLEMENTARY MATERIALS · p. 17
Predefined search
- Predefined-search visuals show flight paths, integral-image classifications, and detected people, with one false classification marked in F12.The supplementary figures cover Tab. 2 flights except F16, which is shown in Fig. 4C.Source: SUPPLEMENTARY MATERIALS · p. 17, Figure S2 · p. 18
Adaptive search
- Adaptive-search materials show computed paths across varied probability fields, including sampled and fast unsampled flight segments and detection-resampling cases.The supplementary figure covers Tab. 3 cases except F20 and F21, shown in Fig. 5B,C.Source: SUPPLEMENTARY MATERIALS · p. 17, Figure S3 · p. 19
Offline registration enhancement
- Despite substantial misregistrations in real-time integral images, person classification succeeded, and offline registration enhancement improved visual image quality.Processing took approximately 2–3 minutes on a standard PC with an Nvidia GTX 1060 GPU, using 30–39 integrated samples and a 60 px × 60 px region of interest.Source: Figure S4 · p. 20
Average Precision Curve
- The AP-fit curve is hyperbolic, with a = 0.943, b = 0.516, and mean-squared error 0.0005.The parameters were computed by least-squares optimization, with positive residuals penalized.Source: SUPPLEMENTARY MATERIALS · p. 21