Decoding the Disaster: Multi-Task Geospatial Reasoning with Vision-Language Models and Crowdsourced Imagery for Disaster Mapping

I. INTRODUCTION

• Crowdsourced images can complement remote sensing with street-level detail, yet unreliable coordinates and changing disaster scenes make geolocalization difficult.Satellite and aerial imagery may miss fine-grained damage; disasters can also change the appearance of scenes and obscure persistent matching cues.Source: §I · p. 1
• VLM-based spatial reasoning for interpretable geolocalization and damage assessment remains largely underexplored despite models’ ability to interpret heterogeneous evidence.The motivation also notes that high-quality post-event annotations are often scarce, delayed, and unevenly distributed.Source: §I · pp. 1–2
• GRDisaster establishes a human-validated VGI–SVI–RSI benchmark and unifies cross-view matching, interpretable spatial validation, and damage-severity classification.Its cross-view validation uses appearance, spatial-layout, and landmark evidence, with interpretable rationales.Source: §I · p. 2

A. GeoAI and multimodal RS in disaster response

• GeoAI and multimodal disaster-response work draws on complementary satellite, aerial, street-view, and crowdsourced observations, but noisy, incomplete, misaligned data and missing metadata complicate integration.Overhead imagery can provide broad coverage while missing fine-grained street-level and context-dependent damage conditions.Source: A. GeoAI and multimodal RS in disaster… · pp. 2–3

B. VLMs and geospatial reasoning

• VLMs have progressed from transferable image–text representations to geospatial reasoning, yet large spatial extents, multi-scale patterns, and disaster-related occlusion challenge their use in remote sensing.Remote-sensing-specific models and datasets address domain transfer, while disaster scenes add damage-induced appearance changes and unreliable geographic metadata.Source: B. VLMs and geospatial reasoning · p. 3

C. Crowdsourcing and multi-task disaster mapping

• Crowdsourced imagery can reveal local damage and accessibility, but uneven coverage, uncertain quality, duplication, and unreliable geolocation constrain its use in disaster response.Multi-task mapping links geolocation recovery, damage assessment, scene interpretation, and evidence validation because the same image may support several tasks.Source: C. Crowdsourcing and multi-task… · pp. 3–4

A. Overview

• GRDisaster connects candidate retrieval, VLM-based spatial validation, and damage-severity classification in a unified workflow.Validation uses appearance, spatial layout, and landmark evidence; severity assessment uses six visually observable damage indicators and returns scores and a rationale.Source: A. Overview · p. 4, Figure 2 · p. 4

B. Deterministic and probabilistic cross-view geolocalization

• Deterministic geolocalization ranks georeferenced references, while probabilistic Riemannian flow-matching estimates a conditional geographic distribution from visual features.The probabilistic method samples plausible locations on the spherical Earth; a more concentrated distribution indicates stronger geographic evidence and lower ambiguity.Source: B. Deterministic and probabilistic… · p. 5
• Evaluation compares coordinate errors, distance-threshold accuracy, and retrieval Recall@K to assess methods that rank candidates or estimate coordinates.Retrieval settings include VGI→RSI, VGI→SVI, and multi-view configurations.Source: B. Deterministic and probabilistic… · pp. 5–6

1) Design of cross-view geolocalization reasoning indicators:

• Experts organized recurring cross-view cues into appearance, spatial-layout, and landmark dimensions across five semantic classes, yielding 30 extensible indicators.Repeated reviews were used to identify missing cues and improve coverage; the indicators support assessing spatial consistency between VGI and candidate references.Source: 1) Design of cross-view… · p. 6
• A structured prompt specifies the matching task, output format, scoring scheme, and indicators, including a three-level score for indicator-level reasoning quality.The 30 indicators guide evaluation of VGI–SVI–RSI candidate pairs.Source: 1) Design of cross-view… · p. 6
• Matching excludes disaster-related damage evidence and uses a containment criterion, requiring support from part of the reference pair rather than complete one-to-one visual correspondence.This design reflects viewpoint differences and incomplete visual overlap while prioritizing relatively stable shared characteristics.Source: 1) Design of cross-view… · p. 6

2) Reasoning and validating geolocalization results:

• Each spatial indicator receives a 0–2 score based on whether the reasoning is visually supported across the VGI image and its SVI–RSI reference pair.A score of 1 captures partial support or ambiguous evidence; 0 includes absent, contradictory, spatially inconsistent, or hallucinated reasoning.Source: 2) Reasoning and validating… · p. 6
• Annotation and evaluation are zero-shot, with Gemini-3.6-Flash generating annotations and two experts independently reviewing a stratified 20% sample.Experts assess whether matching decisions and reasoning are grounded in VGI, SVI, and RSI evidence; disagreements are discussed.Source: 2) Reasoning and validating… · pp. 6–7
• Qwen3.6-27B, GPT-4o, and InternVL3-8B are evaluated on visual evidence in each VGI image using agreement rate and quadratic weighted kappa.The same evaluation framework applies to cross-view reasoning and damage-severity assessment.Source: 2) Reasoning and validating… · p. 7
• Quadratic weighted kappa compares observed and expected agreement across score categories, penalizing larger disagreements more than adjacent-category disagreements.The observed matrix counts VLM and human score pairings across evaluated samples; expected agreement is estimated from marginal distributions.Source: 2) Reasoning and validating… · p. 7

D. VLM-based disaster damage reasoning and classification

• Six visual indicators support low, medium, or high damage-severity classification: water impact, structures, roads, vegetation, infrastructure, and debris or scene disruption.The indicators support both indicator-level assessment and an overall severity classification.Source: D. VLM-based disaster damage reasoning… · p. 7
• In zero-shot evaluation, Qwen3.6-27B, GPT-4o, and InternVL3-8B score individual indicators and overall severity and generate reasoning from each VGI image.Gemini-3.6-Flash serves as the annotation judge; two experts independently verify 20% of samples stratified by predicted severity and resolve disagreements through discussion.Source: D. VLM-based disaster damage reasoning… · pp. 7–8
• Agreement rate and quadratic weighted kappa assess consistency between VLM predictions and human-verified severity labels while accounting for ordinal category relationships.Experts check whether scores, severity levels, and explanations are supported by visible image evidence.Source: D. VLM-based disaster damage reasoning… · p. 8

A. Dataset overview and experimental settings

• The final dataset contains 26,340 images organized into 8,780 validated VGI–SVI–RSI triplets after duplicate removal and manual filtering.The Task 1 split assigns 7,024 groups to training and 1,756 to test, with no geolocation shared across subsets.Source: A. Dataset overview and experimental… · p. 8

B. Cross-view geolocalization performance

• DINOv3-L raises VGI→RSI R@1 from 6.32% to 26.59% and reaches 29.04% R@1 for VGI→SVI, while SVI→RSI reaches 89.01% R@1.VGI→RSI R@1% is 66.00%; VGI→SVI R@1% is 64.64%.Source: B. Cross-view geolocalization… · pp. 8–9
• DINOv3-L retrieval with Sinkhorn alignment reaches 45.96% accuracy within 1 km, versus 0.00% for RFM DINOv2-L and 0.06% for RFM StreetCLIP.The retrieval model also has a 27.29 km median error and 98.12% accuracy within 2,500 km.Source: Table III · p. 9, B. Cross-view geolocalization… · p. 10
• VGI-based retrieval remains a bottleneck: DINOv3-L reaches 26.59% R@1 for VGI→RSI and 29.04% for VGI→SVI, versus 89.01% for SVI→RSI.The paper attributes the gap to disaster-driven appearance changes, unconstrained viewpoints, occlusion, limited fields of view, and temporal differences.Source: B. Cross-view geolocalization… · p. 9
• For DINOv3-L, VGI–SVI–RSI fusion raises R@1 to 39.12%, and Sinkhorn alignment raises it further to 43.85%.R@1% increases from 74.66% to 77.11% with Sinkhorn alignment.Source: B. Cross-view geolocalization… · pp. 9–10

C. Cross-view spatial reasoning and validation results

• Gemini annotations had indicator-level agreement of 0.938 and 0.962 with Experts A and B, with κw values of 0.953 and 0.960, under a permissive evidence-support criterion.All expert-validation agreement and κw results exceeded 0.8 after expert quality control.Source: C. Cross-view spatial reasoning and… · p. 10
• GPT-4o achieved 69.00% matching accuracy and the strongest aggregate spatial reasoning, with 0.646 indicator-level agreement and κw of 0.489.Qwen3.6-27B and InternVL3-8B reached matching accuracy of 65.15% and 64.20%, respectively; all results were zero-shot without task-specific training.Source: C. Cross-view spatial reasoning and… · p. 10
• GPT-4o performed strongly on building indicators and scene functional type, while Qwen3.6-27B excelled on scene consistency and relational cues; InternVL3-8B was lower across most semantic groups.Qwen3.6-27B's large-facility agreement of 0.589 coincided with κw of 0.112, and InternVL3-8B's overall scene-consistency agreement and κw were 0.407 and 0.186.Source: C. Cross-view spatial reasoning and… · p. 10
• Structural and relational cues were generally the most reliable, while vegetation, terrain, waterbodies, coastlines, and regional backgrounds were more sensitive to change, occlusion, and scale differences.GPT-4o was more balanced across cue types; Qwen3.6-27B remained competitive on relational and scene-level cues.Source: C. Cross-view spatial reasoning and… · p. 10

D. VLM-based damage reasoning and classification results

• Gemini achieved indicator-level agreement of 0.950 with both experts, and overall severity agreement of 0.915 and 0.949, supporting annotation use after expert quality control.Indicator agreement ranged from 0.934 to 0.974; directional underestimation and overestimation rates were limited.Source: D. VLM-based damage reasoning and… · pp. 10, 12
• GPT-4o and Qwen3.6-27B had similar aggregate indicator results, but GPT-4o led overall severity agreement at 0.739 while Qwen3.6-27B led on debris scenes at 0.663.For overall severity, GPT-4o's κw was 0.688 versus Qwen3.6-27B's 0.630; InternVL3-8B reached 0.338 agreement and 0.226 κw.Source: D. VLM-based damage reasoning and… · pp. 12–13
• Error patterns differed by model: InternVL3-8B overestimated severity at a rate of 0.620, while GPT-4o had underestimation and overestimation rates of 0.155 and 0.106.Remaining errors mainly involved overlapping damage indicators, unclear functional impacts, or severity distinctions requiring evidence beyond the image.Source: D. VLM-based damage reasoning and… · p. 13
• Models generally recognized dominant damage in flooded and structural-collapse scenes, but disagreed more on secondary indicators and functional impacts.For example, they differed on building, vegetation, infrastructure, road accessibility, and facility damage assessments.Source: D. VLM-based damage reasoning and… · pp. 13–14

• Retrieval better suits fine-grained geolocalization when gallery coverage is adequate, while probabilistic models represent plausible regions and uncertainty when cues are incomplete.Source: §V · p. 14
• Structural and relational cues generally provide stronger geolocation evidence than vegetation, terrain, waterbodies, or regional backgrounds, which are more sensitive to context and disaster disturbance.Source: §V · p. 14
• Figure 7 compares three VLMs across six damage dimensions, with circle colors indicating accuracy.Source: Figure 7 · p. 15
• Unlike studies focused on satellite change detection and building-level classification, GRDisaster extends reasoning to multiple forms of damage at a geolocated place.Source: §V · pp. 14–15
• Models generally recognize dominant damage patterns, but secondary effects, functionality, accessibility, and adjacent severity levels are harder to assess; calibration and expert verification remain necessary.Source: §V · p. 15
• Deterministic retrieval provides precise reference-based anchoring, probabilistic geolocalization represents spatial uncertainty, and VLM reasoning examines correspondence and severity evidence.Source: §V · p. 15
• Outputs are structured decision support rather than autonomous conclusions, especially with outdated references, uneven gallery coverage, occlusion, or scenes outside PhotoMappers’ represented regions and disaster types.Source: §V · p. 15
• Evaluation covers three model versions, while broader model families, regions, event categories, image sources, reference densities, and temporal gaps remain needed to assess generalization.Source: §V · pp. 15–16

• Foundation-model features, complementary cross-view information, and optional re-ranking improve geographic matching.Source: §VI · p. 16
• GRDisaster combines deterministic and probabilistic cross-view geolocalization with explainable VLM reasoning on 8,780 human-validated VGI–SVI–RSI triplets comprising 26,340 images.Source: §VI · p. 16
• Two indicator systems support interpretable spatial validation and damage assessment, and expert verification confirms the reliability of generated annotations.Source: §VI · p. 16
• Tests with Qwen3.6-27B, GPT-4o, and InternVL3-8B demonstrate meaningful zero-shot reasoning, but generalizability is limited by reference coverage, individual-image evidence, and evaluated disasters and models.Source: §VI · p. 16