Source-linked AI summary
Embedded Bi-Temporal Building Damage Assessment for On-Board Data Reduction
Thomas Goudemant, Benjamin Francesconi, Marjorie Bellizzi, Adrien Dorise
TL;DR
Satellite-based damage assessment is constrained by communication capacity and ground-processing latency, while residual image misregistration complicates bi-temporal comparison. The paper compresses pre-disaster references and produces on-board object-level products, adding latent-space alignment; the approach reduces data exchange and supports the full robust pipeline on Jetson, but not Versal DPU.
Problem
Ground-centric bi-temporal assessment requires full-image downlink and offline processing, consuming communication resources and delaying actionable products.
Source: §1 · p. 1Method
A shared siamese detector compares ground-encoded pre-disaster latents with post-disaster features, using estimated global offsets to realign features before fusion.
Source: §3.2 · p. 3, §6 · p. 8Results
The pipeline cuts pre-disaster uplink volume by about two orders of magnitude while retaining detection performance, and alignment outperforms shift augmentation and cross-attention under strong de-registration.
Source: §6 · p. 8Takeaways & Limitations
The embedded GPU supports the full robust pipeline, whereas the Versal DPU supports the core detector and compression but not the long-range robustness operators.
Source: §5 · p. 7The alignment module’s grid-sampling output diverges from the reference PyTorch model on Jetson and must be resolved before production use.
Source: §4.4 · p. 6
Abstract
from arXiv · showhide
Rapid assessment of building damage after natural disasters is essential to support emergency response. Earth Observation satellites can acquire relevant imagery shortly after an event, but exploitation is limited by uplink and downlink capacity and by ground-processing latency. We address this with a bi-temporal building damage assessment pipeline built on a siamese detector derived from YOLOX, designed to compress information at both ends of the ground/space link. On the ground, pre-disaster reference images are encoded into a compact latent space -- compressed by up to a factor of 64 -- and uplinked to the satellite. On board, this reference is compared with a fresh post-disaster acquisition so that the downlink carries only actionable object-level products, bounding boxes and damage classes, instead of full scenes. This cuts the data exchanged in both directions, while on xBD the strongly compressed reference still preserves most of the detection performance. Because on-board acquisitions suffer from residual pre/post co-registration errors, we introduce a latent-space shift estimation and correction module that regresses the global offset from the coarse feature level and realigns the post-disaster features before fusion. It substantially improves robustness to de-registration -- especially under large shifts, where fusion-only variants collapse -- while also raising nominal accuracy and remaining compatible with the strongest compression. We finally port the pipeline to two embedded targets, a Xilinx Versal VCK190 and an NVIDIA Jetson AGX Orin, and report hardware performance (latency, throughput, power efficiency). The core detector and its compression port cleanly to both, but the operators needed for long-range robustness survive only on the Jetson GPU, whereas the Versal DPU does not.
1 Introduction
Bi-temporal damage assessment can support emergency response, but ground-centric pipelines require full-image downlink and offline processing. This paper extends prior work with explicit latent-space alignment and evaluates deployment on two embedded accelerators.
1 Introduction
- Ground-centric bi-temporal pipelines require full-image downlink and offline processing, delaying actionable products and consuming communication resources over areas of limited operational interest.On-board AI can instead prioritize data by transmitting actionable information rather than complete scenes.Source: §1 · p. 1
- Prior work left large pre/post de-registration robustness and compatibility with actual embedded targets unresolved.Its robustness degraded before shifts reached the model’s effective receptive field.Source: §1 · p. 1
- The paper introduces latent-space offset estimation and correction, realigning post-disaster features before fusion to improve de-registration robustness and nominal accuracy while retaining ×64 compression.It also ports the pipeline to a Versal VCK190 and Jetson AGX Orin; the full robust operators run only on the Jetson GPU.Source: §1 · pp. 1–2
- The contributions include a robustness analysis comparing shift augmentation, attention, portable attention, and alignment, plus an embedded deployment study of algorithmic and hardware performance.The study characterizes which robustness mechanisms survive on each target.Source: §1 · p. 2
2 Related work
Prior work motivates bi-temporal, object-level building assessment for coherent, compact outputs, while identifying a gap in combining that reasoning with embedded deployment. The system concept illustrates how compact references and actionable products fit the ground/on-board workflow.
2 Related work
- Object-level assessment localises each building and assigns one damage state, producing more coherent, compact outputs suited to downlinking actionable products.Dense pixel labels can be semantically inconsistent when damage affects only part of a structure.Source: §2 · p. 2
- Self-attention can retrieve context from displaced locations, but spatial softmax is poorly supported by fixed-function edge accelerators; convolutional attention remains local.Its fixed receptive field cannot fetch information from a displaced location.Source: §2 · p. 2
- Prior embedded EO efforts rarely combine bi-temporal reasoning, object-level modelling, and embedded deployment constraints; this paper focuses on latent-space alignment and portability.The system concept encodes pre-disaster references into compressed latents for uplink and downlinks object-level products instead of full scenes.Source: §2 · p. 2, Figure 1 · p. 2
3 Method
The method separates ground encoding of pre-disaster references from on-board comparison with new imagery, producing compact damage products. A shared YOLOX-derived detector and explicit latent alignment address compression and residual misregistration, with deployment shaped by each accelerator’s operator support.
3.1 Ground/on-board system concept
- The ground segment encodes selected pre-disaster images into compact multi-scale latents for uplink; on board, these are compared with new imagery to produce building boxes, damage classes, and optionally damaged-region crops.The split reduces both uplink and downlink volume; bandwidth, memory, and residual misregistration constrain the design.Source: §3.1 · p. 2
3.2 Base detection architecture
- A shared encoder processes pre- and post-disaster images independently, then concatenates their features and element-wise difference before the FPN; the same encoder supports ground/on-board reuse.Unlike early fusion’s six-channel input, siamese processing separates feature extraction before comparison.Source: §3.2 · p. 3
- Channel compression reduces the pre-disaster latent before uplink, with a symmetric on-board projection restoring its dimensionality; at r=64, the latent is 57 kB.At r=128, D3 has one latent channel, which is insufficient for the alignment module.Source: §3.2 · p. 3
3.3 Robustness to pre/post de-registration
- Residual misregistration mainly harms damage classification, so the method compares shift augmentation, global cross-attention, and a local convolutional spatial-difference gate alongside explicit alignment.Cross-attention can retrieve shifted features but its HW × HW softmax is unsupported by the Versal DPU; the portable gate has a strictly local receptive field.Source: §3.3 · p. 3
- The proposed module estimates a global offset by correlating pre/post D3 features, then bilinearly shifts post-disaster D3/D4/D5 features before fusion.It searches offsets up to 416 px, using D3’s geometric detail and a soft-argmax offset estimate.Source: §3.3 · p. 3
3.4 Embedded targets and portability
- The Versal DPU supports the siamese detector, compression, and portable gate, but not cross-attention or explicit alignment; the Jetson GPU supports those robustness operators.The module uses vectorized correlation and grid sampling; the targets are a fixed-function Versal DPU and a TensorRT-programmed Jetson AGX Orin GPU.Source: §3.3 · pp. 3–4, §3.4 · p. 4
4 Experiments and results
On xBD, explicit latent alignment improves damage detection under large shifts while preserving strong performance with compressed references. Embedded deployment results show that the robustness and hardware trade-offs differ across the Jetson and Versal targets.
4.1 Dataset and protocol
- xBD evaluation uses 4,665 standard-split pre/post pairs, 60/20/20% train/validation/test, and synthetic post-image shifts scored on common spatial support.The 1024×1024 orthorectified RGB pairs have four building-damage levels; detection is evaluated with precision, recall, F1, and mAP@0.5.Source: §4.1 · p. 4
4.2 Nominal detection performance
- 57.8% mAP@0.5 is the best nominal result; combining alignment with r=64 costs ≈1 pp while shrinking the uplinked latent by two orders of magnitude.Siamese processing improves over early fusion, while cross-attention and the portable spatial-difference gate leave nominal accuracy essentially unchanged.Source: Table 2 · p. 4, §4.2 · p. 4
4.3 Robustness to de-registration
- 49.4% mAP@0.5 at 400 px, versus 26.6% for shift augmentation and 43.0% for cross-attention; ×64 compression retains 47.0%.The proposed module remains nearly flat beyond the 150 px augmentation training range; the ×64 result preserves this shifted performance.Source: §4.3 · p. 4, Figure 3 · p. 5
4.4 Embedded deployment: algorithmic impact
- Quantisation is near-neutral for most deployed models, but the Jetson alignment model loses 3.24 pp in FP16 because of a grid-sampling porting issue.The issue is attributed to divergent warp output, not numerical precision, and remains unresolved for production use of that model on Jetson.Source: §4.4 · pp. 5–6, Table 3 · p. 6
4.5 Embedded deployment: hardware performance
- 85 vs. 41 Mpx/s, but 29.8 vs. 12.5 W: VCK190 is faster on the plain detector, while Jetson is more efficient (288 vs. 172 GFLOP/s/W).On Jetson, enabling alignment reduces efficiency from 288 to 181 GFLOP/s/W and raises latency from 25.6 to 38.8 ms; the DPU does not provide the long-range robustness.Source: Table 4 · p. 6, §4.5 · p. 6
5 Discussion
The pipeline reduces data on both links, but robust correction of large de-registration depends on operators unavailable on the Versal DPU. The evaluation uses calibrated, ortho-rectified image pairs and leaves several real-acquisition factors untested.
5 Discussion
- For a 100 km2 urban area at 0.8 m/px, ×64 compression reduces pre-disaster references from about 450 MB to under 10 MB, while downlinking only compact damage products.At measured throughput of about 27–85 Mpx/s, inference takes a few seconds, leaving the downlink—not compute—as the bottleneck.Source: §5 · p. 7
- The alignment module maintains flat mAP across the operational shift range and the best nominal accuracy, but its grid-sampling operator is absent from the Versal DPU.Classical cross-attention is also non-portable, while the DPU-compatible alternatives do not recover long-range robustness.Source: §5 · p. 7
- A fixed-function DPU still lacks an on-board correction solution for large de-registration, whereas the Jetson embedded GPU supports the full robust pipeline.The proposed alignment relies on grid sampling, unavailable on the Versal DPU.Source: §5 · p. 7
- The evaluation uses L1C-like ortho-rectified, calibrated, coarsely co-registered RGB pairs; sensor calibration, scale, rotation, cross-sensor pairs, and raw acquisitions remain unaddressed.xBD includes off-nadir pixel drift and variation across nineteen events, but the factors are not evaluated separately.Source: §5 · p. 7
- The shift module is a latent-space registration step that could be extended to scale and rotation; the authors are exploring alternative alignment operators for the DPU.A larger ground-side encoder and multi-temporal or multimodal inputs are also proposed as future directions.Source: §5 · pp. 7–8
6 Conclusion
The paper presents an on-board data-reduction architecture for bi-temporal building damage assessment, combining compressed pre-disaster references with compact damage products. It reports reduced uplink volume, improved robustness to de-registration, and deployment on two embedded accelerators, with long-range robustness limited to the Jetson GPU.
6 Conclusion
- The architecture compresses pre-disaster information on the ground and extracts compact damage products on board.It builds on the paper's previous work [9].Source: §6 · p. 8
- Up to ×64 compression cuts uplink data volume by about two orders of magnitude with no notable loss in detection.The compressed reference is intended to make pre-disaster data practical to transmit.Source: §6 · p. 8
- Latent-space shift estimation and correction regresses the pre/post offset and realigns features before fusion, outperforming shift augmentation and cross-attention under strong de-registration.The module removes the need for accurate co-registration.Source: §6 · p. 8
- The core detector and compression port to both the Versal VCK190 and Jetson AGX Orin, but long-range robustness operators are non-portable on the fixed-function DPU.The Jetson GPU runs the full pipeline; extending shift estimation to scale and rotation is proposed as a direction for on-board registration.Source: §6 · p. 8