GeBDA: Building Damage Assessment as Text-Based Sequence Prediction

• Rapid building damage mapping supports disaster response and recovery, while manual inspection is prohibitively slow and unscalable.Source: §1 · p. 1
• GeoVLMs struggle with dense BDA localization because autoregressive decoders must generate long sequences of precise coordinates and may drift or hallucinate.Source: §1 · p. 1
• Existing GeoVLM approaches simplify BDA or delegate localization, leaving joint single-pass localization and grading of every building undemonstrated.Source: §1 · p. 1
• GeBDA fine-tunes a Gemma model to directly produce building locations and damage classes from bi-temporal images and a task-specific prompt.Source: §1 · p. 1
• Preliminary results suggest GeBDA is competitive in both localization and classification using optical images.Source: §1 · p. 2

• Specialist BDA networks process pre- and post-disaster image pairs to output dense masks that localize and grade buildings.Source: §2.1 · p. 2
• Generalist VLM localization commonly uses external decoders, plain-text bounding-box serialization, or dedicated coordinate tokens.Source: §2.2 · p. 2
• Sequence-based localization preserves end-to-end generation, but autoregressive formulations share a critical failure mode in dense environments.Source: §2.2 · p. 2
• Existing GeoVLMs simplify full BDA through the dense-localization bottleneck rather than jointly localizing and grading every building.Source: §2.3 · p. 2
• GeBDA’s formulation transforms building polygons into quantized, serialized bounding-box strings with damage labels and predicts one target sequence.Source: Figure 1 · p. 3

• BDA is formulated as instance-based prediction rather than pixel-level classification to match the autoregressive next-token operation of VLMs.Source: §3.1 · p. 3

3.1 Problem Formulation

• Given pre- and post-disaster images and a prompt, the model generates a discrete sequence representing all scene buildings after converting polygons to boxes.Source: §3.1 · p. 3
• Coordinates are uniformly quantized to a 1000 × 1000 grid before serialization into text alongside each building’s damage class.Source: §3.1 · p. 3
• The target sequence concatenates all building strings, tokenizes them with the native tokenizer, and appends an <EOS> token.Source: §3.1 · p. 4
• Plain-text coordinates preserve Gemma’s pretrained numerical representation and implicit geometric priors, but use tokens less efficiently.Source: §3.1 · p. 4

Training and Inference

• GeBDA combines a Transformer backbone with a SigLIP 2 vision encoder, freezes the vision encoder, and fine-tunes the remaining model end-to-end.Source: §3.2 · p. 4, §3.3 · p. 4

Evaluation

• Localization matches predicted and ground-truth boxes with Hungarian assignment at IoU 0.5, while classification computes F1 only on correctly matched instances.Source: §3.3 · p. 4
• Predictions are rasterized into dense masks for comparison with pixel-based models, while an Oracle quantifies the geometric ceiling from box representation.Source: §3.3 · p. 4

Datasets

• GeBDA is trained and evaluated on xBD and a subset of Bright with available instance labels, excluding unclassified buildings and those smaller than 16 pixels.xBD contains intact, minor-damage, major-damage, and destroyed classes; Bright contains intact, damaged, and destroyed classes.Source: §4.1 · pp. 4–5

Experimental setup

• Each 1024×1024 image is cropped into 512×512 patches, limiting sequence length while enabling up to 200 buildings per patch.The configuration uses up to 19 tokens per building and a vision-encoder token budget of 280.Source: §4.2 · p. 5

Localization

• Predicted and ground-truth building counts show near-perfect correlation, although dense patches can produce clusters of tiny false-positive boxes and ambiguous boundaries can lead to multiple boxes for one building.These artifacts help explain why qualitative localization appears stronger than instance-based metrics.Source: §4.3 · p. 5

Damage classification

• Fcls1 drops from 76.62 on xBD to 48.99 on Bright, which the authors attribute to the modality gap in Bright’s post-event SAR imagery.The frozen RGB-pretrained vision encoder struggles to extract robust discriminative features from the SAR data.Source: §4.3 · pp. 5–6

Cross-dataset training

• Joint training on xBD and Bright yields no improvement overall except a +22.6% increase in Bright Fcls1.The authors hypothesize that xBD transfers modality-agnostic priors and grounding in shared optical pre-event imagery to Bright.Source: §4.3 · p. 6

• GeBDA frames BDA as autoregressive sequence prediction and achieves strong localization and promising damage classification with post-event optical images.The localization capability is tempered by the bounding-box representation.Source: §5 · p. 8
• A frozen vision encoder fails to extract discriminative features from post-event SAR imagery, motivating vision-encoder fine-tuning.Source: §5 · p. 8
• Dense patches can trigger coordinate-generation loops, likely because training caps sequences at 200 buildings, while heterogeneous tiles can produce a single predicted damage class.Source: §5 · p. 8
• Plain-text bounding-box coordinates are token-inefficient, and fully supervised fine-tuning limits backbone scaling; larger models and LoRA are suggested alternatives.The authors identify Gemma4-26B with a 500M-parameter vision encoder as a potentially beneficial but currently impractical scale-up.Source: §5 · p. 8
• Future work will investigate reinforcement learning for spatial metrics such as bounding-box IoU and test generalization across disasters, whose splits currently share disaster events.Source: §5 · p. 8