Source-linked AI summary
Multi-Label Proportion Learning for Sea-Ice Type Prediction
Samira Alkaee Taleghan, Younghyun Koo, Andrew P. Barrett, Farnoush Banaei-Kashani
TL;DR
Ice charts provide polygon-level labels, whereas standard supervised sea-ice models require patch-level labels and may replace mixed compositions with dominant classes. The paper instead uses weakly supervised MIL and multi-label proportion learning, extending the framework with AMSR2 and ERA5 auxiliary regularization. The multimodal late-fusion model with both auxiliary regularizers reports an overall MAE of 0.194 and mean ice-class F1 of 77.4%.
Problem
Ice charts provide polygon-level annotations while deep-learning models require pixel- or patch-level labels, and dominant-class conversion discards partial concentrations and introduces noise in mixed polygons.
Source: §1 · p. 1Method
The framework uses binary MIL to separate water from ice, then MLPL to predict polygon-level ice-type proportions, with a multimodal extension integrating SAR, AMSR2, and ERA5.
Source: §1 · p. 2Results
0.194 overall MAE and 77.4% mean ice-class F1 are reported for the late-fusion model with both auxiliary regularizers.
Source: §5 · p. 10Takeaways & Limitations
The framework predicts multi-label ice-type proportions directly from WMO egg-code polygon labels without patch-level annotation.
Source: §5 · p. 10Because patch-level confidence or uncertainty labels are unavailable, total Dirichlet evidence is treated only as a relative confidence proxy, with calibration evaluated separately.
Source: §3.4 · p. 5
Abstract
from arXiv · showhide
Sea-ice type prediction is important for climate monitoring, maritime navigation, and decision-making in polar regions. The main source of label data for this task is the ice chart, produced manually by ice analysts who interpret satellite imagery to delineate ice zones into polygons. Although ice charts are valuable, their production is labor-intensive and expensive, motivating recent efforts to automate the process using deep learning. However, deep learning models require patch-level (or pixel-level) label data for training, while ice charts provide only polygon-level annotations. As a workaround, supervised approaches often create approximate patch-level labels from polygon-level ice chart labels by assigning each sample the dominant ice type of its parent polygon. This approach enables supervised training but creates an ill-posed learning problem with intrinsically approximate solution. In this paper, we redefine sea-ice type prediction as a weakly supervised multi-label proportion learning problem to be able to directly use the polygon-level ice chart labels and avoid unnecessary label approximation for improved prediction accuracy. To address this problem, we propose a two-module framework where first Multiple Instance Learning (MIL) is used for water--ice classification, and then a multi-label proportion learning (MLPL) is introduced for ice-type composition prediction. We further extend this framework with a multimodal model that integrates SAR imagery with AMSR2 brightness temperatures and ERA5 reanalysis data through modality-guided auxiliary regularization. Evaluated on the AI4Arctic dataset, the SAR-only model reduces MAE by 14.5\% and more than doubles mean ice-class F1 over the best supervised baseline. The multimodal model further reduces MAE by 21.5\% and raises mean F1 by 41.2\% over the SAR-only model, and by 52.7\% over the supervised multimodal baseline.
1 Introduction
Ice charts provide valuable but labor-intensive polygon-level annotations, while standard deep-learning training requires patch-level labels. The paper preserves polygon-level ice-type proportions through a two-module weakly supervised framework, with multimodal regularization as an extension.
1 Introduction
- Ice charts are manually produced by analysts and are labor-intensive and expensive, motivating deep-learning automation for climate monitoring, modeling, navigation, and polar decision-making.SAR supports chart interpretation with high-resolution observations through clouds and polar darkness.Source: §1 · p. 1
- Ice charts record aggregate sea-ice composition for polygons, but standard supervised models need pixel- or patch-level labels and often reduce mixtures to a dominant class.Dominant-class conversion discards partial concentrations and introduces noise in mixed polygons.Source: §1 · pp. 1–2
- The framework treats each polygon as a bag of unlabeled patches, using binary MIL for water–ice classification and MLPL to predict multi-label ice-type proportions.Evidential Dirichlet aggregation lets patches contribute variable evidence rather than equal softmax mass.Source: §1 · p. 2
- The multimodal extension integrates SAR with AMSR2 brightness temperatures and ERA5 reanalysis through auxiliary regularization, and the paper also investigates bag-size calibration and uncertainty-weighted Dirichlet training.The auxiliary signals support AMSR2 ice-concentration consistency and ERA5 environmental compatibility.Source: §1 · p. 2
2 Related Work
Prior sea-ice classification work largely uses supervised labels, while MIL and LLP provide related weak-supervision approaches. The paper situates its contribution at the intersection of polygon-level learning and multi-label proportions.
2.1 Sea Ice Type Classification
- Supervised sea-ice classification advances use CNNs and common benchmarks, but often reduce mixed polygons to single labels rather than learning polygon-level proportions.The review describes CNN applications to ice–water and ice-type classification and IceBench’s standardized benchmark.Source: §2.1 · p. 2
2.2 Multi Instance Learning
- MIL assigns labels to bags rather than individual instances; deep variants aggregate instance features, while multi-label extensions commonly model label presence rather than proportions.Attention-based methods include global self-attention such as TransMIL and local-context approaches such as CAMIL [16].Source: §2.2 · pp. 2–3
2.3 Learning with Label Proportion
- LLP trains from bag-level class proportions, but common formulations assume mutually exclusive classes; the reviewed sea-ice MIPL-Ice work did not fully develop the multi-instance proportion setting considered here.Existing LLP methods include Batch Averager [3], which matches average instance predictions to known proportions using KL divergence.Source: §2.3 · p. 3
3 Methodology
The method mirrors charting with two weakly supervised modules: binary MIL separates water from ice, then MLPL estimates ice-type proportions; multimodal extensions add environmental inputs and training-time regularizers.
3 Methodology
- The two-module framework first separates open water from ice-covered regions with binary MIL, then estimates ice-type composition with multi-label proportion learning.Source: §3 · p. 3
3.1 Problem Formulation
- Each annotated polygon is a variable-size bag of image patches, supervised by its polygon-level label proportions while individual patch classes remain latent.Ice charts record up to three co-occurring ice types and their partial concentrations.Source: §3.1 · pp. 3–4
3.2 Ice-Water Classification Module
- The water–ice module uses binary MIL on coarsened SAR patches because the water–ice boundary is a coarse, visually dominant signal.Finer spatial detail is reserved for the subsequent ice-type proportion module.Source: §3.2 · p. 4
3.3 Inter-module Patch Selection
- Attention-guided clustering selects a compact patch subset, allocating more selection slots to relevant clusters while retaining visual diversity.PCA-reduced embeddings are clustered with k-means; attention scores determine cluster relevance, and centroid proximity selects patches within clusters.Source: §3.3 · p. 4
3.4 Sea Ice Type Proportion Prediction Module
- The proportion module predicts patch-level proportions and aggregates non-normalized evidence into a bag-level Dirichlet mean trained against polygon concentrations.Its objective regularizes total evidence, but α_i0 is only a relative confidence proxy and is not assumed calibrated.Source: §3.4 · pp. 4–5
3.5 Multi-Modal Extensions
- The multimodal extension combines SAR with AMSR2 and ERA5 context, using auxiliary losses for ice-concentration consistency and environmental compatibility during training.At inference, predictions come from the proportion head alone.Source: §3.5 · pp. 5–6
4 Experimental Evaluation
The evaluation uses polygon-level egg-code data from AI4Arctic to compare supervised patch classifiers, weakly supervised proportion learners, and SAR and multimodal variants. The late-fusion model with auxiliary regularizers achieves the best reported multimodal scores, and the proposed Dirichlet model remains strongest on scene-disjoint testing.
4.1 Dataset and Preprocessing
- The AI4Arctic data comprise 513 Sentinel-1 scenes from 2018–2021; 78.2% of retained polygons contain multiple ice types, and the multimodal model uses an unchanged training set.The dataset retains polygon-level egg-code concentrations; the most frequent class is Medium/Thick FY (53.8% of bags), while Glacier is rarest (12.7%).Source: §4.1 · pp. 6–7, Figure 3 · p. 7
4.2 Baselines
- Baselines include IceBench patch classifiers trained on each polygon’s dominant ice type, weakly supervised proportion learners, and controlled softmax and entmax LLP variants.Weakly supervised baselines use the same selected bags and KL-based proportion supervision.Source: §4.2 · p. 7
4.3 Experimental Setup
- The two modules use separate ResNet-50 backbones; the water–ice module uses attention-top-k MIL, while the proportion module trains on attention-selected patches.Module 1 uses k=10; Module 2 uses the top 25% of patches.Source: §4.3 · p. 7
4.4 Evaluation Metrics
- Water–ice classification is assessed with bag-level accuracy, precision, recall, F1, and AUROC, while proportion prediction uses per-class MAE and class-wise presence F1.Presence is determined by thresholding predicted proportions at 0.1; reported averages are unweighted across ice classes.Source: §4.4 · p. 8
4.5 Results
- 0.247 MAE and 54.8 F1 give the SAR-only Dirichlet head the best aggregate results among the compared methods, outperforming supervised and other proportion-learning baselines.It has the lowest MAE for C2, C3, and C6, and the highest F1 for every class except C5.Source: §4.5 · p. 8, Table 4 · p. 8, Table 5 · p. 8
- 0.194 overall MAE and 77.4 average F1 make LF + Aux-Cons the best multimodal model, versus DenseNet-121’s 0.307 MAE and 50.7 average F1; its advantage persists on scene-disjoint testing.It also achieves the lowest MAE for C0, C1, C4, and C6, and matches the best result for C5.Source: §4.5 · pp. 8–9, Table 6 · p. 9, Table 7 · p. 9
- Combining SIC consistency and environmental compatibility gives the best multimodal ablation, with 0.194 MAE and 77.4 mean ice-class F1.SIC consistency improves thinner classes C0–C2, while environmental compatibility gains concentrate on C4 and C6.Source: §4.5 · p. 9, Table 10 · p. 9
- All patches give 0.222 MAE, while uncertainty weighting lowers MAE to 0.213 and raises mean F1 to 63.8; selecting 25% of patches speeds training about 3.6× but worsens SAR-only MAE.The 25% patch budget reduces computation; uncertainty weighting is evaluated among the SAR-only component ablations.Source: §4.5 · pp. 9–10
5 Conclusions and Future Work
The late-fusion model with both auxiliary regularizers achieves the best results, while future work targets uncertainty calibration, joint learning, and spatial modeling.
5 Conclusions and Future Work
- Overall MAE of 0.194 and mean ice-class F1 of 77.4% are achieved by the late-fusion model with both auxiliary regularizers.Source: §5 · p. 10
- The two-module framework predicts water versus ice, then estimates multi-label ice-type proportions from polygon-level labels.The SAR-only Dirichlet model uses evidential aggregation, and adding multimodal auxiliary context further improves performance.Source: §5 · p. 10
- Future work proposes calibrating Dirichlet evidence for uncertainty estimates and analyst review, active learning, shared-encoder joint learning, and graph-based modeling of neighboring patches.Source: §5 · p. 10