PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Sensor Mapping of Vulnerable Urban Settlements Using SAR, Multispectral, and Hyperspectral Imagery: A Case Study in Córdoba, Argentina

Luigi Russo, Anabella Ferral, Silvia Liberata Ullo, Paolo Gamba

arXiv:2608.28680v1cs.CV

TL;DR

Informal-settlement mapping matters for urban planning and monitoring, but satellite imagery alone cannot capture settlements’ full socio-economic and institutional context, while official inventories can be costly and irregular to update. This study develops a multi-sensor deep-learning framework combining PlanetScope multispectral, COSMO-SkyMed SAR, and PRISMA hyperspectral data for slum-likelihood mapping in Córdoba. LF+HS achieves the best overall trade-off, and the resulting maps also help interpret broader urban vulnerability patterns.

  • Problem

    Informal-settlement mapping is important for urban planning and monitoring, but satellite imagery cannot capture their socio-economic and institutional context, while official inventories may be expensive to update and miss recent changes.

  • Method

    The study evaluates a patch-level multi-sensor deep-learning framework combining PlanetScope multispectral imagery, COSMO-SkyMed SAR backscatter, and PCA-reduced PRISMA hyperspectral descriptors against the ReNaBaP inventory.

  • Results

    LF+HS achieved the best overall trade-off among precision, F1-score, accuracy, and Cohen’s kappa, while PRISMA supplied complementary spectral information alongside higher-resolution MS and SAR features.

  • Takeaways & Limitations

    Multi-sensor Earth-observation fusion can support mapping ReNaBaP settlements and interpreting broader urban vulnerability patterns beyond officially mapped settlement boundaries.

  • Takeaways & Limitations

    PRISMA’s native 30 m spatial resolution is too coarse to resolve narrow streets, individual roofs, and other fine-scale settlement morphology.

Abstract

from arXiv · show

Informal settlements represent a major urban challenge in rapidly expanding cities, yet their identification from Earth Observation (EO) data remains difficult because of their heterogeneous appearance and incomplete official inventories. This work presents a multi-sensor deep learning (DL) framework for slum-likelihood mapping in Córdoba, Argentina, integrating high-resolution PlanetScope multispectral (MS) imagery, COSMO-SkyMed (CSK) Synthetic Aperture Radar (SAR) data, and medium-resolution PRISMA hyperspectral (HS) observations. The problem is formulated as a patch-level classification task using the official Registro Nacional de Barrios Populares (ReNaBaP) inventory as reference, and the models are evaluated through four geographically partitioned folds. SAR-only and MS-only baselines, their configurations with PRISMA HS support, and early fusion (EF), middle fusion (MF), and late fusion (LF) strategies are systematically compared. Results show that LF+HS provides the best overall balance between classification performance and spatial selectivity, while PRISMA contributes complementary spectral information alongside the higher-resolution MS and SAR representations. Beyond the standard evaluation against ReNaBaP, an external municipal vulnerability layer is used to interpret detections outside the official polygons, showing that several apparent false positives overlap broader vulnerable urban areas. Thermal analysis further shows that ReNaBaP settlements exhibit significantly higher surface temperatures than their immediate surroundings during a heatwave event, indicating localised surface-heat amplification. Taken together, these results suggest that multi-sensor EO fusion can support both the mapping of ReNaBaP settlements and the interpretation of broader urban vulnerability patterns.

I. INTRODUCTION

Informal-settlement mapping matters for urban planning and vulnerability monitoring, but settlement appearance and official labels vary across contexts. This study addresses these challenges with a locally evaluated, multi-sensor framework for Córdoba.

  • Motivation: Informal settlements are associated with inadequate housing, limited infrastructure, insecure tenure, environmental exposure, overcrowding, and reduced access to basic services.Their identification supports SDG 11 monitoring, urban planning, risk reduction, and resource allocation.
  • Motivation: Satellite imagery complements censuses and inventories by providing repeatable observations of building density, roof materials, street layout, vegetation, and urban morphology.These indicators are useful but vary with local urban context and are not universal.
  • Related work: Deep learning has replaced handcrafted descriptors with learned representations and improved informal-settlement detection in very-high-resolution imagery.Transfer learning, segmentation, multiscale architectures, and change detection have expanded the available approaches.
  • Research gap: Systematic joint use of high-resolution multispectral, SAR, and hyperspectral data remains limited, particularly for disentangling modality contributions and fusion strategies.Evaluation against incomplete or outdated official inventories can also misclassify detections outside registered settlements.
  • Contribution: The framework combines PlanetScope multispectral imagery, COSMO-SkyMed SAR, and PCA-reduced PRISMA hyperspectral descriptors for patch-level classification against ReNaBaP.It compares single-sensor baselines, hyperspectral-supported configurations, and early, middle, and late SAR–MS fusion under spatial out-of-fold evaluation.
  • Contribution: External CBA vulnerability data and Landsat-8 thermal analysis extend evaluation beyond official settlement polygons.The contextual assessment does not treat CBA as an additional settlement reference inventory.

II. STUDY AREA, DATA AND DATASET CONSTRUCTION

The study uses Córdoba’s heterogeneous urban area to construct an aligned dataset from multispectral, SAR, hyperspectral, reference-settlement, vulnerability, and thermal data. ReNaBaP supplies official settlement polygons, while CBA provides broader contextual vulnerability information.

  • Study area: Córdoba is Argentina’s second-largest city, covering about 576 km2 and containing consolidated, industrial, residential, corridor, and expanding peri-urban areas.Its heterogeneous urban fabric supports analysis across varied morphological and infrastructural conditions.
  • EO data: PlanetScope provides eight surface-reflectance bands at approximately 3 m resolution, while COSMO-SkyMed supplies HH-polarized SAR information on roughness, geometry, and structure.PRISMA contributes 230 VNIR–SWIR bands at 30 m native resolution.
  • Dataset construction: PlanetScope defines the high-resolution reference grid, and COSMO-SkyMed and PlanetScope are reprojected and resampled onto aligned 3 m grids.PRISMA remains at native 30 m resolution before conversion into compact spectral descriptors.
  • Thermal interpretation: Landsat-8 OLI/TIRS surface-temperature data acquired on 3 January 2022 are used exclusively to compare ReNaBaP settlements with their surroundings during a heatwave.Thermal data are excluded from model training, threshold selection, and city-wide inference.
  • Reference data: ReNaBaP provides georeferenced polygons for officially recognised barrios populares in Córdoba.The inventory is used as settlement reference information but is not considered exhaustive urban-vulnerability coverage.
  • Contextual data: The CBA layer assigns 186 neighbourhood units to high- or low-vulnerability classes, including 122 high-vulnerability and 64 low-vulnerability units.It is used only to interpret model outputs, especially detections outside ReNaBaP polygons.

C. Multi-Sensor Dataset Construction

The dataset construction aligns high-resolution PlanetScope and COSMO-SkyMed inputs with native-resolution PRISMA subsets, compresses hyperspectral data into compact descriptors, and derives binary labels from ReNaBaP coverage. Invalid samples and ambiguous boundary patches are removed before generating the multi-sensor training dataset.

  • Spatial alignment: PlanetScope and COSMO-SkyMed rasters are aligned on a common 3 m grid, while corresponding PRISMA subsets retain native 30 m resolution.The same geographic footprint links the high-resolution patches, hyperspectral subsets, and label windows.
  • Spatial alignment: Each retained sample contains an eight-channel PlanetScope patch and a corresponding single-channel COSMO-SkyMed patch extracted with identical windows.Patches use 224 × 224 pixels with a 179-pixel stride, preserving geographic footprints for labels and map reconstruction.
  • Quality control: Samples with invalid or missing data in either high-resolution modality are discarded before model training and inference.The resulting dataset combines aligned PS, CSK, and PRISMA-derived inputs with binary ReNaBaP labels.
  • PRISMA compression: PRISMA cubes are reduced from 230 spectral bands to K = 16 principal components before spatial averaging and hyperspectral encoding.PCA is fitted separately within each out-of-fold training split and then applied to training and held-out subsets.
  • Patch labelling: ReNaBaP-positive patches require at least 10% covered pixels, negative patches require 0% overlap, and intermediate cases are discarded as ambiguous.This thresholding strategy limits boundary-related label noise by retaining patches with clear assignments.

4) Spatial Cross-Validation Split:

The study uses four geographically partitioned folds to reduce spatial bias and leakage, holding out each sector for evaluation while training on the remaining sectors. This out-of-fold design supports patch-level assessment and city-wide slum-likelihood mapping.

  • Spatial partitioning: Four geographically partitioned sectors define the spatial cross-validation folds, replacing random patch-level splitting.Each fold corresponds to a distinct sector of the Córdoba analysis extent.
  • Fold distribution: 1534 samples are distributed approximately evenly across the four folds: 383, 403, 385, and 363 patches.The corresponding dataset shares are 24.97%, 26.27%, 25.10%, and 23.66%.
  • Out-of-fold evaluation: Each iteration trains on three folds and evaluates on the remaining geographically held-out fold for every modality and fusion strategy.The shared fold structure enables comparable out-of-fold predictions across experimental configurations.
  • Prediction task: The binary local-scene classifier maps each georeferenced patch to a scalar logit converted into a slum-likelihood score.Scores support both patch-level evaluation and city-wide mapping.

A. Learning Framework

The learning framework adapts ResNet-18 models to SAR, multispectral, and fused inputs, producing patch-level slum-likelihood scores through modality-specific feature extraction and classification.

  • ResNet-18 backbones are adapted to the channel dimensionality of SAR, MS, and EF inputs.The input convolution uses one channel for SAR, eight for MS, and nine for EF.
  • Modality-specific encoders extract spatial feature maps that are reduced to compact vectors through global average pooling.The pooled vector represents each input modality before single- or multi-modality fusion.
  • The final representation is processed by dropout and a fully connected classification head that outputs one scalar logit per patch.The classification head is denoted by g(·), and the output logit for patch i is oi.
  • A sigmoid activation converts the output logit into the slum-likelihood score used for inference and mapping.
  • Weighted binary cross-entropy gives greater weight to the positive class to address class imbalance.The positive-class weight is computed separately for each out-of-fold iteration using only the training folds.

B. PRISMA HS Embedding

PRISMA hyperspectral data are converted into a compact, material-sensitive spectral embedding associated with each high-resolution patch rather than treated as another spatial backbone.

  • PRISMA information is incorporated as a compact spectral embedding appended to the SAR, MS, or fused spatial representation before classification.
  • The PCA-reduced PRISMA subset has K = 16 components and is spatially averaged over its native-resolution support.This produces a patch-level descriptor summarising the mean hyperspectral response over the corresponding PlanetScope and COSMO-SkyMed footprint.
  • Two 1D convolutional layers and a fully connected projection transform the averaged descriptor into a 128-dimensional hyperspectral embedding.
  • The embedding supplies material-sensitive spectral information while preserving a single high-resolution spatial backbone in SAR+HS and MS+HS configurations.These variants quantify PRISMA’s contribution relative to SAR-only and MS-only single-sensor baselines.

D. Multi-Sensor Fusion Strategies

The study compares early, middle, and late strategies for integrating SAR and multispectral representations, while adding PRISMA only after the SAR–MS fusion stage.

  • Early Fusion: EF concatenates CSK and PlanetScope inputs before processing them with a single ResNet-18 encoder.This allows SAR backscatter and MS reflectance to interact from the first convolutional layers.
  • In +HS configurations, the PRISMA embedding is appended to the final representation and is excluded from the EF, MF, or LF integration stage.
  • Middle Fusion: MF processes SAR and PlanetScope inputs with independent encoders, concatenates their feature maps, and projects the result through a 1×1 convolution.The fused representation is then reduced with global average pooling while retaining spatial alignment.
  • The workflow includes patch-level learning, model-training ablations, spatial out-of-fold inference, and city-wide slum-likelihood mapping.Subsequent steps examine detections outside official polygons, vulnerability discrimination, and thermal interpretation.

3) Late Fusion:

Late fusion keeps SAR and multispectral representations separate until feature-vector concatenation, then optionally appends the PRISMA embedding before classification and mapping.

  • Late Fusion: LF encodes SAR and PlanetScope inputs independently with separate ResNet-18 encoders up to the feature-vector level.
  • Late Fusion: The SAR–MS spatial representation is formed by concatenating the two modality-specific feature vectors, with the HS embedding appended when PRISMA is used.
  • Late Fusion: LF is described as the most conservative strategy because it delays interaction between modalities with different physical meanings and statistical properties.
  • Four-fold spatial cross-validation uses three folds for training and one held-out fold for evaluation, with out-of-fold scores aggregated for patch evaluation and city-wide maps.
  • The results analysis compares patch-level performance, city-wide maps, and detections outside the ReNaBaP inventory using contextual vulnerability information.

A. Patch-Level Classification Performance

Across four-fold spatial out-of-fold evaluation, PlanetScope MS outperforms SAR alone, while fusion strategies trade recall against precision. LF+HS delivers the strongest overall classification balance and the most spatially selective city-wide maps.

  • Evaluation setup: The four-fold spatial OOF comparison evaluates SAR-only, MS-only, HS-supported single-backbone models, and EF, MF, and LF fusion configurations.AUROC uses continuous scores, while thresholded metrics use a pooled OOF threshold maximising F1.
  • Baseline comparison: MS-only reaches an F1-score of 0.4672 and Cohen’s kappa of 0.4427, exceeding SAR-only values of 0.4088 and 0.3817.SAR-only precision is 0.3590, indicating less specific backscatter responses without MS context.
  • Hyperspectral support: Adding HS to the MS baseline increases F1-score from 0.4672 to 0.4776 and AUROC from 0.8634 to 0.8837, but improvements are not consistent across configurations.The reported effect is attributed to complementary surface-material information whose usefulness depends on fusion design.
  • Fusion trade-offs: EF+HS reaches the highest recall at 0.6949 but has the lowest precision at 0.2828, whereas MF+HS becomes conservative with recall of 0.3051.Without HS, MF achieves F1-score 0.4768 and AUROC 0.8880.
  • Fusion trade-offs: LF+HS obtains the highest precision of 0.5455, F1-score of 0.5760, accuracy of 0.9654, and Cohen’s kappa of 0.5581.Its recall is 0.6102, lower than EF, but its higher precision indicates stronger false-positive control.
  • City-wide mapping: LF+HS produces selective hotspots concentrated around ReNaBaP settlements and visually consistent vulnerable patterns, while EF maps show widespread scattered activations.Correctly detected examples include Barrio Villa La Tela and Barrio 12 de Septiembre; high scores also occur outside ReNaBaP in Barrio El Chingolo and Barrio José Ignacio Díaz 2A Secc.

C. Contextual Assessment of ReNaBaP-Based False Positives Using the CBA Vulnerability Layer

The CBA layer provides external context for interpreting LF+HS detections outside ReNaBaP without changing the formal classification labels. These detections overlap broader vulnerable areas, and neighbourhood-level scores moderately discriminate municipal vulnerability classes.

  • Interpretation boundary: Areas outside ReNaBaP are formally false positives under the adopted inventory, but this designation does not imply an absence of broader urban vulnerability.The CBA layer is used for contextual interpretation rather than reassignment of true positives or metric recomputation.
  • Spatial-overlap method: The LF+HS map is binarised on the PS grid, and predicted positive locations outside the rasterised ReNaBaP inventory define the analysed false-positive mask.CBA high- and low-vulnerability polygons are rasterised onto the same grid for overlap analysis.
  • Interpretation boundary: The overlap proportion can represent mapped false-positive pixels or area, but the values derive from spatially aggregated patch predictions rather than independent pixel-wise classifications.This is a scope limitation of interpreting the reconstructed map.
  • Spatial overlap: Approximately 60% of ReNaBaP-based false-positive area falls within neighbourhoods represented in the CBA layer.This overlap supports the interpretation that some apparent false positives align with broader municipal vulnerability patterns.
  • Vulnerability discrimination: High-vulnerability CBA neighbourhoods show higher maximum LF+HS scores than low-vulnerability neighbourhoods, though low-vulnerability areas contain some high-score outliers.Maximum scores are extracted per neighbourhood from the LF+HS OOF map.
  • Vulnerability discrimination: The neighbourhood-level ROC analysis yields an AUC of 0.702, indicating moderate discrimination between high- and low-vulnerability CBA classes.The curve lies above the random baseline, but separation is not perfect.

E. Thermal Interpretation During a Heatwave Event

The study compares surface temperatures inside ReNaBaP settlements with nearby 250 m buffers during a Córdoba heatwave. Settlements were significantly warmer, but the thermal analysis remains exploratory and separate from LF+HS validation.

  • Thermal comparison: The comparison used a Landsat 8 surface-temperature product acquired on 03 January 2022 during a summer heatwave.Temperatures were compared locally within each settlement and its surrounding buffer.
  • Thermal comparison: 42.60°C inside ReNaBaP settlements versus 41.96°C in surrounding buffers yielded an average difference of about +0.63°C.The difference was statistically significant under a Welch two-sample t-test (t = 22, p < 2.2 × 10^-16).
  • Interpretation: The thermal analysis was restricted to ReNaBaP polygons and surrounding buffers, so it does not validate LF+HS detections outside the official inventory.It should be interpreted as complementary information about ReNaBaP settlements rather than evidence that all model-derived hotspots have enhanced thermal exposure.
  • Interpretation: The observed difference suggests localised surface-heat amplification associated with ReNaBaP settlements during the analysed heatwave event.The paper notes that dense built-up conditions, roofing and paving materials, limited vegetation, and reduced evaporative cooling may be related, without establishing specific causes.
  • Limitations: Thermal interpretation is limited by Landsat-8 TIRS resolution, a single heatwave acquisition, and temporal mismatch with the PS and CSK observations.The paper therefore frames the comparison as exploratory rather than a comprehensive thermal-vulnerability assessment.
  • Limitations: ReNaBaP is suitable for supervised evaluation but is not a complete representation of all urban vulnerability in Córdoba.Administrative recognition, update frequency, data-collection practices, and inclusion criteria can influence official inventory coverage.

V. CONCLUSIONS

The study develops a multi-sensor deep-learning framework for slum-likelihood mapping in Córdoba and interprets its outputs using municipal vulnerability and thermal information. The results support both official-settlement mapping and broader vulnerability interpretation, while motivating transferability and richer reference data.

  • Contributions: The framework integrates PlanetScope multispectral, COSMO-SkyMed SAR, and PRISMA hyperspectral observations, evaluated with four geographically partitioned folds.The maps were additionally interpreted using the CBA vulnerability layer and a separate thermal analysis.
  • Results: PlanetScope multispectral imagery achieved the strongest single-modality performance, while multi-sensor fusion supported slum-likelihood mapping.The conclusion contrasts the single-modality result with the broader multimodal framework.
  • Results: Additional high-slum-likelihood areas outside ReNaBaP were partly spatially consistent with vulnerable urban contexts identified by the municipal CBA layer.The comparison extends interpretation beyond officially mapped settlement boundaries without replacing ReNaBaP-based evaluation.
  • Results: ReNaBaP settlements had significantly higher surface temperatures than their immediate surroundings during the analysed heatwave event.This thermal result provides complementary physical interpretation but was restricted to ReNaBaP polygons.
  • Implications: The paper concludes that multi-sensor EO fusion can support mapping ReNaBaP settlements and interpreting broader urban vulnerability patterns.Future work should test transferability, use uncertainty-aware modelling, and integrate more heterogeneous institutional and community-level reference inventories.
Loading 2608.28680v1…