Source-linked AI summary
FloodVLM-Pix: Language-Guided Pixel-Level Flood Understanding
Chengyang Zhang, Zhihao Wang, Zhiyu Long, Xueqian Wang, Quanming Yao, Wei Wu, Gang Li
TL;DR
Flood research has emphasized segmentation, while language-level flood severity and evolution remain less addressed, and existing VLMs can misread large floods or struggle with SAR and image disturbances. The paper builds FloodVL-Pix and introduces FloodVLM-Pix, which combines flood priors with multi-temporal pixel decoding. Experiments report stronger flood-description and segmentation performance than state-of-the-art methods.
Problem
Flood segmentation lacks language-level abstraction for severity and evolution, while existing VLMs can misidentify large flooded areas, struggle with SAR, and encounter image disturbances.
Source: §1 · pp. 1–2Method
The paper constructs a multimodal, multi-temporal benchmark with pixel labels and descriptions, then combines flood priors for VLM understanding with a multi-temporal pixel decoder for segmentation.
Source: §1 · p. 2, §3 · p. 3, §4.2 · p. 4Results
FloodVLM-Pix outperforms state-of-the-art methods in both flood description and pixel-level segmentation, and its 4B model achieves state-of-the-art performance on all reported description metrics.
Source: §5.3 · p. 7, §6 · p. 8Takeaways & Limitations
The system provides pixel-level inundation maps and language-level severity and evolution reports for flood assessment, evacuation, and relief logistics.
Source: §1 · p. 2
Abstract
from arXiv · showhide
As climate change intensifies, floods are becoming more frequent and destructive, making accurate flood understanding an urgent priority. However, existing vision-language models perform poorly on flood-related tasks, suffering from attentional bias and inaccurate descriptions of flood magnitude. To bridge this gap, we construct the first large-scale remote sensing flood multi-modal benchmark that supports language-level flood understanding and pixel-level flood segmentation. The benchmark contains 131,200 flood image patches and corresponding 443,100 textual descriptions for flood severity and evolution. Based on this benchmark, we propose FloodVLM-Pix, a language-guided pixel-level flood understanding model that includes a flood prior module and a multi-temporal flood pixel decoder to achieve accurate flood understanding. The flood prior module automatically distills sub-region flood spatial priors and inputs priors into FloodVLM-Pix, solving the attentional bias problem in flood language understanding. The flood pixel decoder introduces a novel general-specific feature fusion mechanism to address performance degradation caused by interference (e.g., clouds or mountain shadows) during flood segmentation. Experiments on FloodVL-Pix demonstrate that FloodVLM-Pix improves both flood description quality and pixel-level flood segmentation performance compared with state-of-the-art VLMs and flood segmentation models. Our code and data will be released at https://github.com/Haomiaoyanmeng/FloodPix.
1. Introduction
Flood understanding from satellite imagery matters for disaster response, but segmentation alone misses the semantic and temporal abstraction needed to describe flood severity and progression. The paper introduces a multimodal benchmark and a model combining flood priors with pixel-level decoding to address gaps in language understanding and segmentation.
1. Introduction
- Flood segmentation localizes inundation but lacks the semantic and temporal abstraction needed to characterize severity, progression, or impact for damage assessment and emergency response.Multi-temporal language analysis translates visual evidence into descriptions that people can interpret rapidly.Source: §1 · p. 1
- Existing VLMs can misidentify extensive flooding as background, struggle with SAR imagery, and face disturbances such as clouds, mountain shadows, agriculture, and wet regions.Floods can cover over 80% of a patch, while many VLMs are predominantly trained on optical data.Source: §1 · pp. 1–2, Figure 1 · p. 1, Figure 2 · p. 2
- FloodVL-Pix is a multimodal, multi-temporal benchmark for language-level flood understanding and pixel-level segmentation; FloodVLM-Pix combines flood priors with a multi-temporal pixel decoder.The prior module addresses attentional bias, while the decoder fuses general shared and per-patch features to address disturbance-related segmentation degradation.Source: §1 · p. 2
- FloodVLM-Pix provides pixel-level inundation maps and language-level severity and evolution reports for flood assessment, evacuation, and relief logistics.The paper also describes accessible flood descriptions and severity information for citizens in flood-prone regions.Source: §1 · p. 2
2. Related Work
Earlier flood research largely emphasizes segmentation, while remote-sensing VLMs and benchmarks leave gaps in flood-specific language understanding, pixel-level outputs, or multi-temporal analysis. The paper positions its work against these complementary limitations.
2.1. Flood Model/Benchmark
- Flood-oriented remote-sensing models and benchmarks mostly focus on segmentation, and the cited methods do not consider language-level understanding of flood severity and evolution.Table 1 compares flood-oriented design, language support, pixel-level labels, and multi-temporal analysis.Source: §2.1 · p. 2, Table 1 · p. 3
2.2. Remote Sensing VLM/Benchmark
- Remote-sensing VLMs support language understanding, but prior pixel-level VLM frameworks lack multi-temporal understanding and existing remote-sensing VLMs are not flood-specific.The passage notes that non-flood-specific models exhibit attentional bias when interpreting flood imagery.Source: §2.2 · pp. 2–3
3. Benchmark
FloodVL-Pix combines optical and SAR imagery with pixel labels and flood descriptions across multiple dates. The benchmark is designed to support large-flood understanding and flood-dynamics analysis, and the paper outlines a model that uses priors for language understanding and a pixel decoder for segmentation.
3. Benchmark
- FloodVL-Pix contains 131,200 image patches with pixel-level flood labels and 443,100 textual descriptions, built from global flood scenes with 2–13 co-registered temporal patches per region.The benchmark combines SAR and optical imagery; patches are 256 × 256 to align with common segmentation benchmarks, reduce transformer computation and memory, and support finer-grained descriptions.Source: §3 · p. 3
- The model overview combines a prior module for language-level flood cues, a VLM backbone that produces descriptions, and a pixel decoder that fuses multi-temporal features for segmentation.The vision encoder and projector convert image patches into tokens for LLM understanding.Source: Figure 4 · p. 4
- FloodVL-Pix includes 106,500 SAR and 24,700 optical patches; 11,300 patches have more than 70% flood coverage, and Asia accounts for 59.5% of the data.The temporal patches per region support analysis of flood dynamics and evolution.Source: §3 · pp. 3–4
4. Model
FloodVLM-Pix combines language-level flood priors with a pixel decoder for multi-temporal flood segmentation. Its prior module targets VLM attentional bias, while general-specific feature fusion is designed to handle image disturbances and preserve fine-grained flood detail.
4.1. Overview of FloodVLM-Pix
- FloodVLM-Pix combines a prior module that supplies language-level flood priors, a VLM backbone that produces flood descriptions, and a pixel decoder for pixel-level flood segmentation.The VLM takes flood image patches, flood-oriented questions, and priors as input.Source: §4.1 · p. 4
4.2. Prior Module and VLM Backbone
- Sub-region flood ratios are verbalized as priors to recalibrate VLM attention toward floods and improve severity- and evolution-aware descriptions.The image patch is divided into five sub-regions, and the prior module estimates flood and non-flood proportions for each.Source: §4.2 · pp. 4–5
4.3. Pixel Decoder
- Separate optical and SAR encoders feed general-specific fusion, which combines regional cross-modal, cross-temporal context with image-specific features for flood segmentation.The general features provide robust context under disturbances, while specific features preserve fine-grained flood texture.Source: §4.3 · pp. 5–6
5. Experiment
The experiments compare FloodVLM-Pix with general-domain and remote-sensing VLMs and flood-segmentation models, then test its prior module and pixel-decoder design. FloodVLM-Pix achieves stronger flood-description and segmentation performance, while ablations examine the contributions of its components.
5.1. Implementation Details
- The prior module uses LSNet-S, while Qwen3-VL provides the vision encoder, projector, and LLM; the pixel decoder uses a ConvNeXt backbone.The vision encoder is frozen, the LLM uses LoRA fine-tuning, and the projection layer is fully fine-tuned.Source: §5.1 · p. 6
5.3. Flood Description and Segmentation Results
- FloodVLM-Pix-4B achieves state-of-the-art performance on all reported flood-description metrics, outperforming direct-inference and fine-tuned VLM baselines.The comparison covers general-domain and remote-sensing VLMs; the authors attribute the gains to the flood prior module.Source: Table 2 · p. 6, §5.3 · p. 7
5.2. Baselines
- The evaluation compares language understanding against general-domain and remote-sensing VLMs, and segmentation against reproduced SOTA methods and an LSNet baseline.Language baselines include Qwen3-VL variants, GeoPixel-7B, and TeoChat-7B; segmentation baselines include FTransUNet, MFGF-UNet, MFNet, SMAGNet, and LSNet.Source: §5.2 · p. 6
5.3. Flood Description and Segmentation Results
- FloodVLM-Pix-2B/4B achieve higher overall segmentation performance on OA, F1, and IoU; the proposed method also yields fewer missed detections in the visual results.The decoder adaptively fuses general flood-context features with single-image-specific features.Source: Table 3 · p. 7, §5.3 · p. 7, Figure 6 · p. 7
5.4. Ablation Experiments
- 31.1% average CLAIR improvement follows fine-tuning without the prior module; adding the prior yields a further 6.9% average gain.For flood ratios above 80%, fine-tuning improves CLAIR by 36.3%, and adding the prior improves it by 8.0%.Source: §5.4 · p. 8, Figure 7 · p. 8
- Layer normalization, channel-wise concatenation, the general-specific fusion module, and Qwen vision features each support decoder segmentation performance.Removing layer normalization destabilizes optimization; replacing concatenation with addition loses information.Source: §5.4 · p. 8, Table 4 · p. 8
6. Conclusion
FloodVL-Pix supports multi-modal, multi-temporal flood understanding at language and pixel levels. FloodVLM-Pix combines flood priors with a multi-temporal pixel decoder, and the experiments report improvements over SOTA methods in flood description and segmentation.
6. Conclusion
- FloodVL-Pix is presented as the first benchmark for language- and pixel-level, multi-modal, multi-temporal flood understanding.FloodVLM-Pix combines a flood prior module with a multi-temporal flood pixel decoder; experiments report that it outperforms SOTA methods in flood description and segmentation.Source: §6 · p. 8