BRIGHT: A globally distributed multimodal building damage assessment dataset with very-high-resolution for all-weather disaster response

Hongruixuan Chen; Jian Song; Olivier Dietrich; Clifford Broni-Bediako; Weihao Xuan; Junjue Wang; Xinlei Shao; Yimin Wei; Junshi Xia; Cuiling Lan; Konrad Schindler; Naoto Yokoya

Source-linked AI summary

BRIGHT: A globally distributed multimodal building damage assessment dataset with very-high-resolution for all-weather disaster response

Hongruixuan Chen, Jian Song, Olivier Dietrich, Clifford Broni-Bediako, Weihao Xuan, Junjue Wang, Xinlei Shao, Yimin Wei, Junshi Xia, Cuiling Lan, Konrad Schindler, Naoto Yokoya

arXiv:2501.06019v4Jan 10, 2025cs.CVcs.AIeess.IVeess.SP

TL;DR

Multimodal building damage assessment lacks suitable benchmarks for robust all-weather disaster response. BRIGHT addresses this gap with a globally distributed, open-access dataset and broad model evaluations, where decoupled models outperform direct-prediction models on benchmark performance.

Problem
Robust multimodal building damage assessment is constrained by the lack of suitable benchmarks for evaluating generalization across unseen disaster events and locations.
Method
BRIGHT provides an open-access, globally distributed multimodal dataset and evaluates supervised models across standard and cross-event transfer settings.
Results
Decoupled models outperform direct-prediction models, with ChangeMamba achieving 90.90% mIoU and 72.70% OA.
Takeaways & Limitations
BRIGHT supports benchmarking multimodal building damage assessment under standard and cross-event conditions for AI-based disaster response research.
Takeaways & Limitations
BRIGHT has regional and sample imbalance, including substantially more Turkey earthquake tiles than Hawaii wildfire tiles, which may affect model generalizability.

Abstract

from arXiv · show

Disaster events occur around the world and cause significant damage to human life and property. Earth observation (EO) data enables rapid and comprehensive building damage assessment (BDA), an essential capability in the aftermath of a disaster to reduce human casualties and to inform disaster relief efforts. Recent research focuses on the development of AI models to achieve accurate mapping of unseen disaster events, mostly using optical EO data. However, solutions based on optical data are limited to clear skies and daylight hours, preventing a prompt response to disasters. Integrating multimodal (MM) EO data, particularly the combination of optical and SAR imagery, makes it possible to provide all-weather, day-and-night disaster responses. Despite this potential, the development of robust multimodal AI models has been constrained by the lack of suitable benchmark datasets. In this paper, we present a BDA dataset using veRy-hIGH-resoluTion optical and SAR imagery (BRIGHT) to support AI-based all-weather disaster response. To the best of our knowledge, BRIGHT is the first open-access, globally distributed, event-diverse MM dataset specifically curated to support AI-based disaster response. It covers five types of natural disasters and two types of man-made disasters across 14 regions worldwide, with a particular focus on developing countries where external assistance is most needed. The optical and SAR imagery in BRIGHT, with a spatial resolution between 0.3-1 meters, provides detailed representations of individual buildings, making it ideal for precise BDA. In our experiments, we have tested seven advanced AI models trained with our BRIGHT to validate the transferability and robustness. The dataset and code are available at https://github.com/ChenHongruixuan/BRIGHT. BRIGHT also serves as the official dataset for the 2025 IEEE GRSS Data Fusion Contest.

1 Introduction

Building damage assessment must become faster and more generalizable for disaster response, but optical-only methods are constrained by daylight and cloud conditions while existing SAR methods lack large-scale benchmarks. BRIGHT addresses this gap with an open, globally distributed, multimodal very-high-resolution dataset and benchmarks for robust AI-based damage mapping.

Motivation: Disasters cause substantial human, material, economic, and environmental losses, motivating rapid building damage assessment for emergency response.UNDRR reported that natural disasters affected approximately 4.4 billion people between 1998 and 2017.
EO modalities: Optical imagery is intuitive for damage assessment but requires solar illumination and cloud-free conditions, whereas SAR can operate in adverse weather.This difference makes multimodal optical-SAR approaches promising for all-weather, day-and-night response.
Automated assessment: Manual interpretation is time-consuming, labor-intensive, and dependent on trained professionals, prompting machine-learning and deep-learning methods for automated mapping.Earlier models often used labels from a single disaster event and area, limiting broader applicability.
Research gap: Existing SAR damage-mapping methods remain focused on local regions and single events because large-scale benchmark datasets are scarce, leaving cross-event generalization largely unknown.SAR-specific challenges include speckle noise, occlusion, geometric distortions, and oblique viewing angles.
BRIGHT contribution: BRIGHT is presented as the first publicly available multimodal building damage dataset combining pre-event optical and post-event SAR imagery at sub-meter resolution with geographic and disaster diversity.Its very-high-resolution imagery supports individual-building assessment and targeted rescue operations, especially in developing countries.
Evaluation and analysis: The paper benchmarks contemporary models on BRIGHT and analyzes cross-event generalization, pre-event optical data, and performance gaps among post-event modalities.The dataset also supports UDA, SSL, UMCD, and UMIM research.

2 Dataset Description

BRIGHT comprises globally distributed multimodal optical–SAR data from 14 disaster events, with standardized building-damage labels and 4,246 cropped image pairs. Its design captures geographic and inter-event diversity while addressing registration, class imbalance, and cross-event generalization challenges.

Dataset coverage: BRIGHT samples 14 disaster events worldwide, primarily in developing countries where international disaster-response assistance is more critical.The selected events occurred since 2020, reflecting the availability of Capella Space and Umbra satellite data.
Data sources: The dataset combines optical imagery mainly from Maxar’s Open Data program with Capella Space and Umbra SAR imagery using Spotlight and Stripmap modes.Spotlight was preferred when available because it provides higher spatial resolution, whereas Stripmap offers greater coverage.
Annotation: BRIGHT labels each building with manually annotated polygons and standardized Intact, Damaged, or Destroyed post-disaster attributes.Expert inspections and external damage products support annotation quality, while diverse source terminology is harmonized through rule-based mapping.
Dataset statistics: 4,246 multimodal image pairs are included after cropping the EO data into 1024×1024-pixel tiles.The dataset exhibits geographical and inter-event feature diversity, with approximately 7:1 background-to-building pixel imbalance and about 6.5% of building pixels in one damage category.
Evaluation protocols: Cross-event evaluation includes zero-shot transfer using 13 events for training and validation and one unseen event for testing, plus one-shot adaptation with one training pair and one validation pair from the target event.These setups assess generalization to unseen disaster types and locations under realistic labeling constraints.

3 Methodology

The methodology formulates building damage assessment as a bi-temporal task and evaluates both direct semantic segmentation and decoupled localization–classification approaches. It compares supervised, transfer-learning, and unsupervised multimodal methods using tailored losses, training settings, and standard assessment metrics.

Task formulation: The experiments compare direct damage-map segmentation with task decoupling into building localization and damage classification, whose outputs are combined by Ydam = Yloc ⊙Yclf.The direct approach predicts damage from image pairs, whereas the decoupled approach first separates buildings from background and then classifies damage levels.
Task formulation: Building damage assessment uses pre- and post-event EO images to generate a damage map, aligning the task with specialized one-to-many semantic change detection.The bi-temporal formulation identifies changes between two time points while assigning multiple damage levels.
Model evaluation: Seven advanced networks are evaluated, including UNet, DeepLabV3+, SiamAttnUNet, SiamCRNN, ChangeOS, DamageFormer, and ChangeMamba.The architectures include two computer-vision segmentation networks and five state-of-the-art EO networks.
Learning strategies: BRIGHT supports zero-shot UDA, one-shot SSL, UMCD, and UMIM experiments for cross-event adaptation, annotation-free multimodal change detection, and automatic multimodal registration.UDA transfers knowledge without labeled target-event samples, SSL uses limited labeled and remaining unlabeled samples, UMCD exploits modality-independent change relationships, and UMIM learns modality-independent matching features.
Training and evaluation: Supervised models combine cross-entropy and Lovasz softmax losses, while evaluation uses OA, F1, and mIoU, with F1 for subtasks and OA/mIoU for overall damage-map quality.Lovasz softmax addresses imbalance among non-building, building, and damage-level pixels; models use AdamW with a learning rate of 1e−4, weight decay of 5e−3, 50,000 iterations, and batch size 16.

4 Results and Analysis

ChangeMamba and DamageFormer achieve the strongest supervised performance, while optical–SAR fusion improves assessment and supports SAR-based response when optical imagery is unavailable. However, performance declines on subtle damage and unseen events, although adaptation methods provide modest gains under limited supervision.

Overall performance: ChangeMamba achieves the best overall test-set performance, reaching 96.22% OA, 67.63% mIoU, and F1 scores of 90.90% and 72.70%.DamageFormer follows with 67.09% mIoU and 96.13% OA.
Event-level performance: 52.26% event-level mIoU is achieved by DamageFormer, slightly exceeding ChangeMamba’s 51.39% and demonstrating strong cross-event robustness.DamageFormer performs particularly well on Beirut-EP-2020, Marshall-WF-2021, and Derna-FL-2023, although performance varies across events.
Limitations: Average IoU for “Damaged” buildings remains below 20% in wildfire events and falls to around 5% in volcano events, reflecting SAR’s difficulty distinguishing partial damage.Single-polarization SAR lacks fine-grained information when structural integrity is partly preserved and backscatter signals are ambiguous; optical–SAR performs better for large-scale surface disruption than subtle or localized damage.
Modality analysis: Pre-event optical imagery raises UNet’s “Damaged” IoU from 35.83% with SAR only to 44.83% with Optical-SAR, while “Destroyed” IoU increases from 55.35% to 55.42%.The results indicate that pre-event optical imagery enriches features beyond building localization.
Modality analysis: 70.79% mIoU is reached by DamageFormer with Optical+SAR fusion, compared with 69.76% for optical-only and 65.56% for SAR-only.Fusion consistently yields the best results across tested models, while SAR-only performance remains reasonably close to optical-only performance.
Cross-event generalization: Zero-shot average mIoU remains below 40%, compared with typically 48% to 52% under the fully supervised split, while ChangeMamba reaches 39.38% zero-shot and 43.23% one-shot mIoU.Under domain adaptation, FDA improves the source-only 35.60% mIoU baseline by +0.69%, and Mean Teacher improves the one-shot 38.82% baseline by +1.18%.

5 Discussion

BRIGHT advances multimodal EO research for disaster response while facing limitations from data availability, registration and labeling errors, regional imbalance, and restricted modality and temporal coverage. Its versatility supports cross-event transfer, multiple EO challenges, and future foundation-model development, with additional modalities identified for future enhancement.

Limitations: BRIGHT’s composition is constrained by scarce open-access very-high-resolution SAR imagery over disaster-affected regions.These practical data-availability constraints shape the dataset’s coverage and composition.
Limitations: Minor optical–SAR registration errors may persist despite manual alignment and expert cross-checking.SAR imagery can be distorted or stretched in some areas, contributing to alignment uncertainty.
Limitations: Manual building polygons and visually interpreted damage labels may contain minor errors that slightly affect model performance.Boundary inaccuracies arise from complex building shapes and variable image resolution.
Limitations: Regional label imbalance ranges from 1,114 Turkey-Earthquake-2023 tiles to 65 Hawaii-Wildfire-2023 tiles, motivating event-level evaluation.Events with more tiles and buildings can become more dominant during training and evaluation.
Future directions: BRIGHT supports UDA, SSL, UMCD, and UMIM research and offers data diversity for EO foundation models, while future versions could add fully polarimetric SAR and LiDAR.Fully polarimetric SAR could improve damage characterization, whereas LiDAR could provide precise 3D structural information.

7 Conclusions … Appendix B: Feature visualization

BRIGHT is introduced as an open-access, globally distributed multimodal building-damage dataset spanning 14 disaster events, with sub-meter optical and SAR imagery and experiments validating its utility. The paper also describes continued dataset expansion and feature visualization using DINOv2, reference maps, t-SNE, and Grad-CAM.

7 Conclusions: BRIGHT covers 14 natural and human-made disaster events with pre-event optical and post-event SAR imagery at sub-meter spatial resolution.The authors benchmarked state-of-the-art supervised learning models and describe BRIGHT as an ongoing project.
7 Conclusions: The dataset will be expanded with new disaster events and refined existing data to improve practical disaster-response and research utility.The stated target applications span regional, national, and international levels.
Appendix A: Details of disaster events: The event collection includes explosions in Beirut and Bata, volcanic eruptions in the Democratic Republic of the Congo, Rwanda, and La Palma, and the 2021 Haiti earthquake.The reported events caused extensive deaths, injuries, displacement, housing loss, or structural destruction.
Appendix A: Details of disaster events: The dataset also represents the 2021 Colorado wildfire and the armed conflict in Ukraine, documenting substantial building destruction and displacement.Ukraine’s reported infrastructure damages reached $170 billion by November 2024, while the Marshall Fire destroyed 1,084 structures.
Appendix A: Details of disaster events: Later events include the 2023 Turkey earthquake, Myanmar cyclone, Hawaii wildfire, Morocco earthquake, Libya flood, and Mexico hurricanes.These events are described through casualties, affected populations, damaged buildings, housing losses, or economic impacts.
A14 Earthquake in Noto, 2024: The collection concludes with the 2024 Noto earthquake in Japan, which damaged 193,529 structures across nine prefectures.The event also caused deaths, injuries, evacuations, tsunami impacts, and substantial estimated economic damage.
Appendix B: Feature visualization: For building feature visualization, DINOv2 extracts high-dimensional features, reference maps remove background features, and t-SNE visualizes the resulting representations.The procedure is applied to buildings from different disaster events in Figure 5-(c).
Appendix B: Feature visualization: Grad-CAM is applied to visualize the spatial focus of models during building damage mapping.The method is described as gradient-based and complements the feature visualization procedure.

Appendix C: Manual registration and estimating registration errors

BRIGHT’s manual registration aligns SAR images to optical references using expert-selected control points and high-quality interpolation. Because registration errors cannot be directly ground-truthed, the appendix estimates them through multimodal feature matching and filtered pixel-distance errors.

Manual registration: Manual registration used QGIS Georeferencer, Thin Plate Spline transformation, and Lanczos resampling with 6×6 kernels to align SAR images to optical references.EO experts manually selected control points on disaster scenes.
Error estimation: Registration errors remain after expert cross-validation and cannot be directly measured using ground-truth control points, motivating a proxy estimation method.The appendix states that multiple rounds of meticulous registration and cross-validation cannot completely eliminate errors.
Manual registration: Figure C1 shows manually selected registration control points for several events, with zoomed regions and event-specific control-point counts.The figure includes Bata-Explosion-2020, La Palma-Volcano-2020, and Noto-Earthquake-2024 SAR data sources.
Error estimation: The proxy method detects feature points, extracts modality-independent descriptors, matches optical-SAR features, and computes pixel distances between registered point pairs.The procedure uses many points, such as 3000 per image tile, and validated multimodal registration descriptors.
Error estimation: Matched points with RMSE greater than 20 pixels are excluded as outliers because large errors were expected to have been removed through expert corrections and cross-checking.The threshold improves the reliability of the registration-error estimate.

Appendix D: Details of benchmark deep learning models

Appendix D specifies the architectures and implementation choices for the benchmark deep learning models. The models use convolutional, siamese, recurrent, transformer, and Mamba-based encoders adapted for multimodal building damage mapping.

UNet: UNet uses five encoder blocks with two 3×3 convolutions and two batch-normalization layers each, with channels 64, 128, 256, 512, and 1024.Its decoder has four convolutional blocks whose channels correspond to encoder skip-connected features.
DeepLabV3+: DeepLabV3+ uses ImageNet-pretrained ResNet-50, with its input channel modified to predict damage maps from stacked optical and SAR images.The encoder is adapted for direct multimodal prediction.
Siamese and recurrent models: SiamAttnUNet follows its original literature, while SiamCRNN uses pseudo-siamese ResNet-18 and four ConvLSTM decoder layers with 3×3 kernels and hidden dimension 128.The SiamCRNN decoder uses four ConvLSTM layers.
ChangeOS: ChangeOS uses ImageNet-pretrained ResNet-18 modified into a pseudo-siamese encoder for pre-event and post-event images with different modalities.The pseudo-siamese structure extracts features from the two image inputs.
Transformer and Mamba models: DamageFormer uses pseudo-siamese Swin-Transformer-Tiny, while the ChangeMamba experiments use MambaBDA-Tiny with a pseudo-siamese encoder for different-modality pre-event and post-event images.DamageFormer replaces the pure-siamese MixFormer used in its original literature.

Appendix E: Details of post-processing methods

Appendix E describes three post-processing methods for refining raw deep-learning predictions: test-time augmentation, object-based majority voting, and model ensembling. These methods improve prediction stability, spatial coherence, robustness, and generalization through input transformations, object-level voting, and multi-model logit averaging.

Test-time augmentation: Test-time augmentation aggregates predictions from rotated and flipped versions of each input to reduce sensitivity to spatial variations and improve prediction stability.The applied rotations are 90°, 180°, and 270°, together with horizontal and vertical flipping.
Object-based majority voting: Object-based majority voting assigns each building a label using a weighted vote across the entire object rather than independent pixel-wise predictions.The weighting follows Zheng et al. (2021) and is determined by the inverse of class pro…
Model ensembling: Model ensembling combines ChangeOS, DamageFormer, and ChangeMamba by averaging their logits before applying softmax to reduce uncertainty and improve generalization.Each model independently generates a damage proxy map, while architectural diversity helps smooth individual model biases.

Appendix F: Details of unsupervised domain adaptation methods

Appendix F evaluates representative unsupervised domain adaptation methods for cross-event zero-shot disaster assessment under an unlabeled-target setting. All methods use DeepLabV3+ and standardized training and evaluation procedures, with method-specific details retained where necessary.

Evaluated UDA methods: The evaluated methods comprise AdaptSeg, AdvEnt, CLAN, PyCDA, and FDA, representing adversarial, entropy-based, category-level, curriculum, and frequency-mixing adaptation strategies.They were originally designed for semantic segmentation and are evaluated in the cross-event zero-shot setting.
Shared architecture: DeepLabV3+ serves as the shared backbone, preserving compatibility with official implementations and providing a consistent comparison basis.The choice avoids extensive code modification because many implementations are tightly coupled with DeepLab’s structure.
Zero-shot adaptation setting: Each test disaster event is the target domain, using only its unlabeled optical-SAR image pairs for adaptation and no labeled target data for training or model selection.Model selection relies solely on source-domain feedback, reflecting emergency-response conditions where imagery is available without ground-truth annotations.
Method-specific implementation: Hyperparameters follow the original papers, while FDA mixes frequency-transferred and original source images 1:1 and PyCDA adapts after 30,000 source-training iterations.PyCDA then applies pseudo-labeling and uncertainty-aware refinement on the target domain.
Evaluation protocol: All methods share preprocessing and training schedules unless original methods specify otherwise, and final target-domain performance is reported using mIoU.The standardized pipeline includes settings such as learning rate and batch size.

Appendix G: Details of semi-supervised learning methods

The appendix evaluates representative semi-supervised learning methods for one-shot cross-event transfer in multimodal EO-based building damage assessment. The setup uses 13 labeled source events, one labeled target-event sample, and remaining unlabeled target samples for adaptation.

Methods: The evaluated methods are Mean Teacher, CCT, GCT, and CPS, adapted from natural-image segmentation to multimodal EO-based building damage assessment.Mean Teacher uses EMA teacher weights; CCT uses perturbed views and multiple decoder branches; GCT adds pre-trained-model guidance; CPS uses mutually generated pseudo-labels.
Implementation and evaluation: All methods use DeepLabV3+ as the backbone, and final target-event performance is evaluated using mIoU.The shared backbone preserves consistency with the original implementations and enables fair comparison.
Experimental setup: 13 labeled disaster events provide training and validation data, while one labeled sample and remaining unlabeled samples from the target event support one-shot SSL adaptation.The target-event configuration is intended to simulate a realistic one-shot adaptation scenario.
Training procedure: Each model is first trained for 10,000 iterations with supervised cross-entropy and Lovász-Softmax losses before applying its semi-supervised objective to unlabeled data.The supervised stage initializes a stable representation before semi-supervised training begins.

Appendix H: Unsupervised multimodal change detection: methods and evaluation protocol · H1 Evaluated UMCD methods · H2 A more practical evaluation protocol on BRIGHT

Appendix H evaluates unsupervised multimodal change-detection methods on BRIGHT and proposes a more deployment-realistic protocol using unseen scenes. It also highlights the challenges of transferring land-cover change methods to disaster damage detection, especially for damaged buildings with weak signals.

H1 Evaluated UMCD methods: BRIGHT evaluation covers recent unsupervised multimodal change-detection methods that detect binary changes from paired multimodal imagery without manual labels.
H1 Evaluated UMCD methods: Random guessing assigns each pixel a 50% probability of being changed or unchanged, providing a reference performance floor.
H1 Evaluated UMCD methods: IRG-McS matches superpixel-graph structural relationships across modalities and uses Markov cosegmentation to generate the final change map.
H1 Evaluated UMCD methods: SR-GCAE, FD-MCD, and AOSG model multimodal structure through graph representations, spectral convolution with adaptive fusion, or optimized patch-level structural graphs.These methods use graph convolutional autoencoding, Fourier-domain analysis, and iterative fusion of self-change and cross-domain differences, respectively.
H1 Evaluated UMCD methods: AGSCC enforces structural-graph similarity across domains through cycle-consistent image regression, adaptive graphs, and multiple regularization terms.
H1 Evaluated UMCD methods: AEKAN uses a superpixel-based Siamese AutoEncoder with Kolmogorov–Arnold Networks, reconstruction loss, and hierarchical consistency loss to extract shared multimodal features.
H2 A more practical evaluation protocol on BRIGHT: The proposed protocol separates representation learning, hyperparameter tuning, and evaluation across scenes to test generalization to unseen deployment data.Common UMCD protocols reuse individual image pairs for all three stages, creating overlap between tuning and evaluation scenes.
H2 A more practical evaluation protocol on BRIGHT: Directly applying land-cover change methods to disaster damage detection is difficult because damaged buildings may lack strong spatial or spectral signals in unsupervised settings.The methods were originally developed for semantic transitions such as vegetation, water, or built-up areas.

Appendix I: Details of unsupervised multimodal image matching methods

The appendix groups unsupervised multimodal image-matching methods into feature-based and area-based strategies, then evaluates registration on a downsampled Noto-Earthquake-2024 image pair using manually validated control points. The control-point proxy measures average spatial offset before and after registration as an interpretable alignment estimate without ground-truth correspondence maps.

Method categories: UMIM algorithms are categorized into feature-based and area-based methods according to their matching strategies.Feature-based methods detect sparse keypoints in both modalities, whereas area-based methods begin with optical-image keypoints.
Feature-based methods: Feature-based methods LNIFT and SRIF compute modality-invariant descriptors from independently detected optical and SAR keypoints, then match them globally by descriptor similarity.Both methods use publicly available code.
Area-based methods: Area-based methods FLSS and HOPC match optical-image keypoints by sliding a template across a local SAR search region using patch similarity.Experiments used 120×120-pixel templates and 200×200-pixel search windows, with open-source implementations.
Evaluation setup: Registration was performed on the full Noto-Earthquake-2024 EO image pair after downsampling both images to half resolution for computational feasibility.The downsampling addressed the large spatial size of the input images.
Evaluation metric: A control-point proxy evaluates average spatial offset before and after registration using manually selected, EO-expert-validated stable cross-modal features.The metric offers a practical alignment estimate without ground-truth correspondence maps, though it is not a substitute for full correspondence maps.