PaperlayerView exact revision on arXiv

Source-linked AI summary

Accuracy and efficiency of compact foundation models for surface-water mapping during floods

Parastoo Farajpoor, Mohammadreza Narimani

arXiv:2610.34904v1eess.IVcs.CVcs.LGstat.AP

TL;DR

Flood-water mapping can support exposure assessment and observation planning, but the value of pretrained Earth-observation models depends on their alternatives. This study compares compact Prithvi-Tiny with supervised baselines and evaluates accuracy, transfer conditions, reliability, and computation. Pretraining improves the tiny transformer over random initialization, while U-Net remains competitive and Tiny uses less measured computation.

  • Problem

    The value of pretrained Earth-observation models for surface-water mapping depends on how they compare with task-specific alternatives and on what the imagery can reveal.

    Source: §1 · p. 1

  • Method

    The study evaluates Prithvi-Tiny against U-Net, spectral-index baselines, a randomly initialized matching transformer, and a larger Prithvi model across label amounts, event exclusions, reliability, and inference cost.

    Source: §1 · p. 2

  • Results

    Pretraining improved surface-water segmentation over random initialization, but Prithvi-Tiny did not consistently outperform U-Net, and its measured computational demand was lower.

    Source: §5 · p. 17

  • Takeaways & Limitations

    Model choice should weigh accuracy, computation, and similarity between the intended setting and the evaluated events, rather than rely on pretraining alone.

    Source: §5 · p. 17

    Protocol B is not fully event-blind for hyperparameter selection because learning rates were chosen using the full Protocol A validation pool.

    Source: §2.3 · p. 3

Abstract

from arXiv · show

Pretrained Earth observation models may reduce the effort needed to map surface water, but their value depends on the alternative against which they are compared. We evaluated Prithvi-EO-2.0-Tiny using the 446 hand-labeled Sentinel-2 chips from 11 flood events in Sen1Floods11. A fine-tuned model, a frozen-encoder model, and an identical randomly initialized architecture were compared with a similarly sized U-Net, a larger Prithvi model, and spectral-index thresholds. Experiments varied the number of training chips, excluded events from model fitting, and measured computational cost and prediction reliability. With 252 training chips, U-Net achieved a water intersection-over-union (IoU) of 0.856 on the official test set, compared with 0.833 for fine-tuned Prithvi-Tiny. Pretraining improved the tiny transformer over random initialization by a reported 0.046 IoU with 25 training chips and 0.023 with 252 chips, but did not reverse the U-Net ranking. Across 11 event-exclusion evaluations, mean IoU was 0.768 for fine-tuned Prithvi-Tiny and 0.777 for U-Net; the paired interval included zero. Fine-tuned Prithvi-Tiny processed a 512x512-pixel chip in 16.1 ms on a GPU and 258 ms on a laptop CPU, compared with 30.6 and 665 ms for U-Net. Errors were concentrated near water boundaries, and high predictive entropy identified many, but not all, erroneous pixels. These results support a specific role for compact pretrained models: lower measured computational demand with competitive accuracy, rather than consistent superiority. Event-dependent performance, shared hyperparameter selection, and the distinction between observed water and newly inundated land delimit the conclusions.

1. Introduction

Flood-water maps can inform exposure and observation priorities, but their usefulness depends on accuracy, feasibility, and clarity about what imagery cannot show. This study separates the effects of pretraining, architecture, and computational cost while testing performance under different evaluation conditions.

1. Introduction

  • Flood-water maps can inform exposure, further-observation priorities, and hydrological-model assessment when field observations are difficult to obtain.A useful mapping method must be accurate enough for its purpose, feasible to run, and explicit about imagery limits.Source: §1 · p. 1
  • Optical imagery can miss inundation hidden by clouds or vegetation, and water-land mixtures complicate classification at boundaries.A map of visible water is not automatically a map of all flooding on the ground.Source: §1 · p. 1
  • Success on one benchmark does not settle transferability to other landscapes, water conditions, or labeling conventions.Sen1Floods11 includes multiple flood events and a geographically separate Bolivia evaluation set.Source: §1 · p. 1
  • Pretrained representations may help with small labeled sets, but improvement over random initialization and superiority over a task-specific alternative are separate questions.The passage states that these questions require different controls.Source: §1 · p. 1
  • Geospatial foundation models do not consistently outperform supervised baselines across tasks and data regimes.Prior flood studies also examine sensor-dependent performance and hybrids combining pretrained features with convolutional detail.Source: §1 · p. 2
  • Nearby samples can share landscape and acquisition conditions, so spatially dependent validation may make transferability appear greater than it is.Event exclusion helps expose this issue, but interpretation also depends on where hyperparameters were selected.Source: §1 · p. 2
  • The study compares compact Prithvi-Tiny with a similarly sized U-Net, spectral-index baselines, a randomly initialized transformer, and a larger Prithvi model.It asks whether pretraining helps the same architecture and whether that translates into an advantage over U-Net with fewer labels.Source: §1 · p. 2

2. Materials and methods

The study evaluates six-band neural and spectral methods on hand-labeled Sen1Floods11 Sentinel-2 chips, using seen-event and event-excluded protocols. It varies labeled training chips and assesses accuracy, calibration, uncertainty, and device-specific inference cost.

2.1. Flood-event imagery and reference labels

  • 446 hand-labeled 512×512 Sentinel-2 chips from 11 events form the dataset; reference water includes permanent and event-observed water but does not by itself identify newly inundated land.Unlabeled and out-of-swath pixels are excluded, and 60,454 labeled pixels without band signal are excluded for every method.Source: §2.1 · p. 2

2.2. Input representation and preprocessing

  • Inputs use six Sentinel-2 bands, including B8A, clipped and scaled before the networks standardize each band using the relevant training-pool statistics.The Level-1C inputs differ from Prithvi’s HLS pretraining data; reduced-label experiments retain the full training-image pool for normalization.Source: §2.2 · p. 2

2.3. Evaluation design and label fractions

  • Protocol A uses 252 training, 89 validation, and 90 seen-event test chips, plus 15 Bolivia holdout chips; Protocol B excludes each target event from fixed training and validation pools.Protocol B reuses learning rates selected on the full Protocol A validation pool, so it is not fully nested or event-blind; Bolivia is cleaner with respect to this shared tuning step.Source: §2.3 · pp. 2–3

2.4. Spectral and neural comparison methods

  • The comparisons include NDWI and MNDWI thresholds, a six-channel U-Net, pretrained Tiny variants, and a larger Prithvi reference model.Tiny variants are fine-tuned, frozen-encoder, or randomly initialized; the complete Tiny network has about 7.79 million parameters, while the larger reference has 91.97 million and is evaluated only in Protocol A at 10% and 100% labels.Source: §2.4 · p. 6

2.5. Training and inference

  • Training uses random crops, spatial flips and rotations, AdamW, validation-based checkpoint selection, early stopping, and a common maximum budget of 1,500 steps.Fine-tuned Prithvi uses separate learning rates for new layers and encoder; full-chip neural inference averages overlapping-window probabilities and thresholds at p > 0.5.Source: §2.5 · pp. 6–7

2.6. Accuracy, calibration, and statistical uncertainty

  • Water IoU is the primary statistic, accompanied by F1, precision, recall, calibration, entropy-based error ranking, boundary analysis, and permanent-water-stratum recall.Intervals use chip- or event-level cluster bootstrap procedures; entropy is evaluated as an error-ranking score, not an error probability.Source: §2.6 · p. 7

2.7. Computational measurements

  • Computational measurements time full-chip inference on a Tesla T4 GPU and an Intel Core i9-12900H laptop CPU, while excluding data acquisition, disk access, preprocessing, and export.Reported timings are device- and implementation-specific; the table reports seen-event test performance with all training labels.Source: §2.7 · pp. 7–8, Table 3 · p. 8

3. Results

On the official seen-event test, U-Net led water IoU, while pretraining improved Tiny over random initialization without surpassing U-Net. Tiny had lower measured latency than U-Net, alongside event-dependent performance and errors concentrated near water boundaries.

3.1. Accuracy on seen events and the Bolivia holdout

  • U-Net achieved 0.856 water IoU on the official seen-event test, versus 0.833 for fine-tuned Tiny; Tiny’s paired difference from U-Net was -0.024 IoU.Fine-tuned Tiny exceeded fitted MNDWI by 0.046 and the identical randomly initialized architecture by 0.023 IoU.Source: §3.1 · p. 8, Figure 5 · p. 11

3.2. Sensitivity to the number of fine-tuning chips

  • With 25 training chips, fine-tuned Tiny reached 0.814 IoU versus 0.768 for random initialization, a paired gain of 0.046; U-Net remained highest at every label fraction.U-Net’s 25-chip IoU of 0.831 was numerically close to Tiny’s 252-chip value of 0.833, not an equivalence test.Source: §3.2 · p. 8, Table 5 · p. 10, Figure 6 · p. 12

3.3. Computational trade-offs

  • Fine-tuned Tiny processed a full chip in 16.1 ms on the T4 and 258 ms on CPU, versus 30.6 ms and 665 ms for U-Net.The models had similar parameter counts, while spectral-index baselines remained faster on CPU; Tiny was neither the most accurate network nor the least expensive method overall.Source: §3.3 · pp. 9–10, Figure 7 · p. 13, Table 6 · p. 13

3.4. Event-excluded evaluation

  • Across 11 event-excluded evaluations, mean IoU was 0.777 for U-Net and 0.768 for fine-tuned Tiny; their paired interval included zero.Tiny exceeded random initialization in all 11 target events by an average 0.043 IoU, while event-to-event differences were larger than the average network difference.Source: §3.4 · pp. 10–11, Figure 8A · p. 10, Table 7 · p. 14

3.5. Reliability, water boundaries, and difficult

  • Calibration rankings varied by evaluation: Protocol B mean ECE was 0.009 for fine-tuned Tiny and 0.030 for U-Net, unlike the Bolivia comparison.On Bolivia, mean ECE was 0.025 for fine-tuned Tiny and 0.015 for U-Net.Source: §3.5 · pp. 11–12
  • Boundary-adjacent pixels were 7.1% of valid test pixels but accounted for 68–76% of neural-model errors; high entropy ranked errors well but did not rule out low-entropy failures.Fine-tuned Tiny’s boundary error rate was 22.0% versus 17.8% for U-Net; entropy AUROC was 0.92–0.94 on seen-event test chips and 0.91–0.95 on Bolivia.Source: §3.5 · pp. 12, 14
  • Neural recall was 0.985–0.993 for JRC-permanent reference water and 0.800–0.854 for other reference water, but this is not a measured penalty for newly flooded land.The study notes that interpreting the difference as newly inundated land requires an independent pre-event reference.Source: §3.5 · p. 12

4. Discussion

Compact Prithvi pretraining improved the tiny transformer over random initialization but did not establish general superiority over U-Net. Its measured efficiency advantage is qualified by event variability, shared tuning resources, and the fact that the target is observed surface water rather than verified new inundation.

4.1. What pretraining did, and did not, provide

  • Fine-tuned Tiny exceeded its random-initialization control, but U-Net retained higher seen-event mean IoU at every label fraction and a slightly higher event-average value.The largest pretrained-versus-random gain occurred with 25 training chips; the comparison estimates the pretrained adaptation strategy under different selected learning rates and encoder-rate policies, not an initial-weights-only effect.Source: §4.1 · p. 14
  • Similar interior error rates but different boundary rates are consistent with a local-detail limitation, although they do not establish patch tokenization as its cause.A 16-pixel transformer patch covers roughly 160 m north–south at the supplied resolution, while U-Net carries spatial detail through skip connections.Source: §4.1 · p. 14

4.2. Label savings must be defined against the actual information budget

  • The reduced-chip results show useful segmentation in this benchmark, not development using only 25 labeled chips, because validation labels, full-label learning-rate selection, and full-pool normalization remained available.Subset selection and optimization seeds were coupled, and no method was reliably more transferable merely because its seen-event score changed little with fewer chips.Source: §4.2 · pp. 14, 16

4.3. Geographic transfer is a property of the evaluation as well as the model

  • Event-level performance varies, and Protocol B’s shared learning-rate selection means it is not fully nested event-exclusion validation.The unweighted event mean gives small and large targets equal weight, while “other reference water” is not independently verified newly inundated land.Source: Figure 8 · p. 15, §4.3 · p. 16

4.4. A practical efficiency advantage, not an operational guarantee

  • Fine-tuned Tiny reduced GPU latency by about 47% and CPU latency by about 61% relative to U-Net, while losing a reported 0.024 seen-event IoU.These measurements exclude cloud-free image acquisition, transfer, scene reconstruction, and validation, and do not measure energy or carbon emissions.Source: §4.4 · p. 16

4.5. Reliability maps support review but do not replace reference evidence

  • Entropy concentrated many errors into relatively few pixels and can prioritize inspection, but AUROC ranking does not establish reliable error probabilities at a chosen threshold.Hand-labeled masks are not error-free physical truth; cloud obstruction, hidden water, and boundary ambiguity affect evaluation, while bootstrap intervals do not account for annotation uncertainty.Source: §4.5 · p. 16

4.6. Limits and next tests

  • The evidence is bounded by 11 events, uneven chip representation, restricted update budgets, and single-date inputs; the study supports a computational trade-off, not a universally best model.The authors identify nested event-level selection, repeated exclusion seeds, independently verified pre-event water, and optical-radar or multitemporal inputs as targeted next tests.Source: §4.6 · pp. 16–17

5. Conclusions

Compact Prithvi pretraining improved segmentation over random initialization, especially with fewer fine-tuning chips, but did not consistently outperform U-Net. Its principal advantage was lower measured computational demand, within the scope of observed reference surface water and the evaluated settings.

  • Compact Prithvi pretraining improved segmentation over the same transformer with random initialization, especially with fewer fine-tuning chips.U-Net led on the seen-event test set, while the event-average difference was unresolved by the reported interval.Source: §5 · p. 17
  • Fine-tuned Tiny’s principal advantage was lower measured computational demand, with approximately half the GPU latency and two-fifths the CPU latency of U-Net under the tested implementation.Source: §5 · p. 17
  • The experiments map reference surface water during floods, not independently verified new inundation, and shared hyperparameter selection limits claims about wholly unseen-event performance.Event-specific errors, boundary behavior, calibration, and the similarity of intended settings to evaluated events matter when choosing a model.Source: §5 · p. 17

Data availability statement

The study reports public data and model sources and makes derived analysis materials openly available.

Ethics statement

The study used geospatial and environmental observations and involved no human participants, identifiable personal information, or animal experiments.

  • The study involved no human participants, identifiable personal information, or experimental work with animals.Source: Ethics statement · p. 17

Generative AI statement

The authors used ChatGPT for language refinement and visualization improvements, then reviewed and edited all AI-generated revisions.

  • ChatGPT was used to improve grammar, refine sentence structure, and enhance visualizations; the authors reviewed and edited all AI-generated revisions.Source: Generative AI statement · p. 17

Abbreviations

The paper defines abbreviations for its metrics, models, datasets, and water indices.

  • AUROC means area under the receiver-operating-characteristic curve, ECE expected calibration error, and IoU intersection-over-union.Source: Abbreviations · p. 17
  • GFM means geospatial foundation model, HLS Harmonized Landsat Sentinel-2, JRC Joint Research Centre, and ViT vision transformer.Source: Abbreviations · p. 17
  • LOEO means leave-one-event-out, MNDWI modified normalized difference water index, NDWI normalized difference water index, TOA top-of-atmosphere, and UPerNet Unified Perceptual Parsing Network.Source: Abbreviations · p. 17

Related papers

Loading 2610.34904v1…