Key Findings for Small UAS for Post-Disaster Damage Assessment from the CURSE'26 Exercise

I. INTRODUCTION

• CURSE’26 integrated CLARKE building and road damage assessment into an operational-style exercise linking sUAS imagery to a centralized Remote Sensing Cell.Teams collected and processed imagery locally before transmitting imagery and derived products to the Tallahassee cell.Source: §I · p. 1
• CLARKE had been deployed during Hurricanes Debby and Helene and was included in CURSE’26 as a tool for producing initial damage reports within minutes.It assesses building damage using the Joint Damage Scale and road damage using a novel labeling schema.Source: §I · p. 1
• CURSE’26 examined the sUAS data-to-decision pipeline, gathered user feedback, and explored whether exercises realistically test sUAS and machine learning for sUAS imagery.The exercise also confirmed problems encountered during prior hurricane deployments and identified new challenges and research needs.Source: §I · pp. 1–2

II. PRIOR WORK

• Prior operational deployments integrated sUAS damage assessment into data-to-decision workflows only ad hoc and across few orthomosaics, motivating a larger-scale pipeline testbed.CLARKE used four U.S. orthomosaics, while another system used four from Valencia, Spain.Source: §II · p. 2

A. Automated Damage Assessment

• Wireless connectivity can be severely degraded or unavailable after disasters, while bursts of high-resolution sUAS imagery can overwhelm cellular and satellite links.During Hurricane Ian, data managers transported hard drives to intact emergency operations centers after uplinks were overwhelmed.Source: A. Automated Damage Assessment · p. 2

B. CLARKE

• Prior CLARKE work identified limited testing on unseen orthomosaics, geographic transfer, bandwidth constraints, and pilot error as evaluation issues.Operational deployments evaluated the first three; pilot error was outside feasible data collection.Source: B. CLARKE · p. 2
• CLARKE had been tested on four unseen orthomosaics and transferred weakly to Pennsylvania imagery, likely because its training data lacked mountainous regions.The Ian workflow’s wireless limitations remained relevant when CLARKE replaced an expert classifier.Source: B. CLARKE · p. 2

III. OVERVIEW OF SUAS DAMAGE ASSESSMENT AT CURSE’26

• The overview covers both the trained data-to-decision workflow and nine damage-assessment missions conducted in two states by seven teams.The missions are compared with Debby and Helene by geography, scale, land use, and damage type.Source: §III · p. 2

A. Pipeline from sUAS to Emergency Managers

• Teams conduct nadir aerial surveys, build orthomosaics, and provide them with building polygons and road lines for model inference.Orthomosaics are uploaded wirelessly or delivered on external drives or SD cards.Source: A. Pipeline from sUAS to Emergency… · p. 2
• The data manager applies CLARKE to label known building polygons and roads, packages results in formats for multiple users, and checks outputs before distribution.Review addresses GPS-related spatial misalignment and possible transfer to new geographic settings; CLARKE does not correct that misalignment, though manual alignment is possible.Source: A. Pipeline from sUAS to Emergency… · pp. 2–3
• The final pipeline step sends data products to emergency managers at the Remote Sensing Cell, where the three groups generally work in silos.Emergency managers expect products to meet their expectations for correctness.Source: A. Pipeline from sUAS to Emergency… · p. 3

B. sUAS Damage Assessment Missions

• Seven missions were conducted at different Florida sites and two in Missouri, while average orthomosaic area was 0.354 km2.Mission sites ranged from zero buildings to 149 buildings, and average road length was 5.61 km.Source: B. sUAS Damage Assessment Missions · p. 3
• Training missions generally targeted industrial parks, urban facilities, and underpopulated rural areas rather than residential neighborhoods.The exercise offered evaluation across land uses while avoiding public concern over safety or privacy.Source: B. sUAS Damage Assessment Missions · p. 3
• Only 31 of 320 buildings showed damage (9.7%), and estimated physical road damage was approximately 5%.The road baseline primarily reflected roads obscured by vegetation and construction.Source: B. sUAS Damage Assessment Missions · p. 3
• 209 seconds (3.5 minutes) was the average inference time on a desktop with an NVIDIA GeForce RTX 4090 and Intel Core Ultra 9 285K.The authors expect similar inference times on laptops with similar NVIDIA GPUs, though direct comparison with Debby and Helene was not possible.Source: B. sUAS Damage Assessment Missions · p. 4

• BDA functioned well across all eight building missions in F1, transfer, and lack of manual spatial-alignment adjustments.Source: §IV · p. 4
• The exercise did not use actual disaster imagery, so performance across the complete collection of JDS classes could not be evaluated.Source: §IV · p. 4
• Three potentially significant failure modes were highlighted, particularly for commercial and larger buildings; pilots can mitigate only poor illumination by flying near noon when operational tempo permits.Source: §IV · p. 4

A. Performance

• 0.834 Micro F1 was achieved on the CURSE’26 orthomosaics, although incomplete, highly unbalanced class distributions prevented full evaluation of prediction correctness.Source: A. Performance · p. 4
• The evaluation used micro-average F1 because the labels were heavily imbalanced toward “no damage”; prior CRASAR-U-DROIDs evaluations used Macro F1.Source: A. Performance · p. 4
• BDA showed positive transfer to Missouri, with no obvious or persistent failure across the imagery and no manual spatial alignment of building polygons required.Source: A. Performance · p. 4

B. Failure Modes

• Observed failures included incorrect building footprints, possible under-representation of large structures in training data, and illumination-related misclassification.Source: B. Failure Modes · p. 4, Figure 4 · p. 4
• Table II frames road results against an expectation that every mission report at least 95% of roads as “no damage.”Source: Table II · p. 5
• Illumination-related errors involved early-morning or late-afternoon imagery rather than the desired midday lighting.Source: B. Failure Modes · p. 5

V. ROAD DAMAGE ASSESSMENT

• During CURSE’26, the Road Damage Assessment component labeled 35.3% of road length across nine orthomosaics as damaged, against an expected baseline of approximately 5%; one orthomosaic showed nearly 80% “total damage”.Source: §V · p. 5
• The RDA component was not ready for operational deployment based on its qualitative performance during CURSE’26.Source: §V · p. 5

A. Performance

• Performance was poor across all orthomosaics, which contained 50.52 km of roads in total; none had the approximately 95% “no damage” expected from manual inspection.Source: A. Performance · p. 5

B. Failure Modes

• Spatial misalignment was the most significant RDA failure source, alongside unfavorable illumination or road color and environmental obscurations.The authors identify shadows, tar roads, and trees as examples of the latter factors.Source: B. Failure Modes · p. 5
• Only illumination is directly addressable by pilots; midday or overcast flights may help, while alignment could be improved in post-processing and obscurations cannot be directly fixed.The authors suggest an alignment model and better management of prediction uncertainty for obscurations.Source: B. Failure Modes · p. 5
• The classifier expects road-line centerlines to coincide with road centers; misalignment, especially near medians, can direct it onto adjacent terrain and produce false damage labels.The operational system uses a priori public road maps that may be non-uniformly aligned with imagery.Source: B. Failure Modes · p. 5
• Fig. 5 shows shadows triggering false total-damage labels on intact surfaces and dark road colors receiving total-damage labels.The figure also illustrates misclassification involving road medians and spatial misalignment.Source: B. Failure Modes · p. 5, Figure 5 · p. 5

VI. DISCUSSION

• CURSE’26 highlights exercise-evaluation challenges, the need for edge computing even under ideal conditions, and ethical concerns for pilots and data managers.These are the three broader deployment insights named in the discussion.Source: §VI · p. 5

A. Role of Exercises in Evaluating sUAS and ML

• The exercise tested CLARKE’s operational workflow realistically, but not its inference on typical disaster damage.Its imagery contained no disaster damage, only 31 of 320 buildings showed any damage, and missions often targeted commercial rather than residential or rural areas.Source: A. Role of Exercises in Evaluating… · p. 6
• An orthomosaic with no damage is a useful edge case for exercises, though such imagery is absent from the CRASAR-U-DROIDs dataset.The authors note that operational deployments usually target areas expected to have damage.Source: A. Role of Exercises in Evaluating… · p. 6

B. Need for Edge Computing

• Uploads took on the order of 1 hour, compared with about 3.5 minutes for inference, even under ideal communications conditions.The authors expect the biggest decision-speed gain to come from running inference at the edge.Source: B. Need for Edge Computing · p. 6

C. Responsible Innovation

• BDA requires a data manager skilled in machine learning to check for accuracy degradation across disasters, geographies, and other out-of-distribution situations.The authors say oversight requires more than a casual glance at the output.Source: C. Responsible Innovation · p. 6
• Using CLARKE without oversight may constitute negligent misrepresentation and expose developers to legal liability for harms.Pilots and data managers are responsible for product quality, although downstream use and users may be unknown.Source: C. Responsible Innovation · p. 6

VII. FINDINGS

• sUAS teams can improve CLARKE damage assessment by flying at midday or in overcast conditions to minimize shadows, when response tempo permits.Illumination noticeably affects damage labeling, but operational tempo may prevent teams from choosing when to fly.Source: §VII · p. 6
• CLARKE BDA performed well in F1, transfer, and without manual spatial-alignment adjustments; it showed positive transfer to Missouri missions 8 and 9.CURSE’26 increased the set of never-before-seen orthomosaics from 4 to 13; the 3.5-minute average inference time was consistent with prior 5–7-minute times on CRASAR-U-DROIDs orthomosaics.Source: §VII · pp. 6–7
• The RDA component is not mature: it mislabeled undamaged roads, with nearly 80% of roads mislabeled in one case, apparently due to sensitivity to spatial alignment errors.The primary culprit appears to be non-uniform alignment errors between road lines and orthomosaics.Source: §VII · p. 7
• Exercises are useful operational tests, but their value for evaluating machine learning is less clear because CURSE’26 imagery differed from past deployments.The imagery generally lacked damage and included more commercial buildings and urban areas; it may be realistic as sUAS deployment scope expands.Source: §VII · p. 7
• Field teams need edge computing to overcome wireless upload delays when transferring imagery and products to a base of operations or emergency operations center.CURSE’26 did not record quantitative upload times.Source: §VII · p. 7
• Responsible disaster-response use requires human oversight and output verification to prevent misunderstandings of accuracy and propagation of faulty products that could cause harm.The authors identify potential negligent misrepresentation and meaningful harms from faulty data products entering operational workflows.Source: §VII · p. 7
• The authors plan to improve RDA and explore automatic uncertainty quantification so the system can flag out-of-distribution situations.CLARKE BDA is positioned for use during Florida’s 2026 hurricane cycle and will continue to be refined.Source: §VII · p. 7