Source-linked AI summary
Onboard Marine Anomaly Detection on $\Phi$sat-2: From Simulation-Based Development to In-Orbit Demonstration
Clotilde Szywala, Thomas Goudemant, Marjorie Bellizzi, Benjamin Francesconi, Adrien Girard
TL;DR
Onboard Earth Observation AI may reduce latency and bandwidth use, but developing it is constrained by limited representative imagery and onboard resources. This paper develops and deploys a modular marine anomaly pipeline on Φsat-2, training first on simulated imagery and then adapting it with real in-orbit data. After retraining, all three processing steps achieved F1-scores above 94% on both datasets, while the deployment experience underscores the distinction between pre-flight risk reduction and scientific validation.
Problem
Onboard AI must work within resource and integration constraints, while representative mission imagery is unavailable before launch and complete onboard-chain studies with real in-orbit validation are scarce.
Source: §1 · p. 1Method
The paper deploys a modular pipeline that filters sea patches, encodes them with self-supervised features, detects deviations from normal sea state, and optionally characterizes selected anomalies.
Source: §1 · p. 1, OBPDC 2026 · p. 2Results
F1-scores above 94% were achieved for all three processing steps on both datasets after retraining, exceeding simulated-data validation performance.
Source: §5.2 · p. 6Takeaways & Limitations
Simulated-data validation supports pre-flight risk reduction but is not final scientific validation; reliable scientific assessment requires representative in-orbit data.
Source: §6.1 · p. 7Independent block training limits global optimization, so end-to-end performance may be lower than performance measured for each block.
Source: §6.1 · p. 7
Abstract
from arXiv · showhide
Onboard Artificial Intelligence can improve responsiveness and bandwidth efficiency of Earth Observation systems by processing data directly on the satellite. This paper presents the experience gained from the development, onboard integration, and post-launch adaptation of a lightweight marine anomaly detection pipeline deployed on the European Space Agency's $\Phi$sat-2 mission. The application combines sea segmentation, self-supervised feature encoding of marine regions, generic anomaly detection based on deviations from a normal sea state, and optional characterization of selected anomaly types. Before launch, the pipeline was trained and validated on simulated $\Phi$sat-2 imagery to assess algorithmic performance and compatibility with resource-constrained onboard hardware. After integration and functional validation in the mission environment, early experiments on real $\Phi$sat-2 acquisitions revealed a significant mismatch between simulated and in-orbit data. The pipeline was therefore retrained on real Level-1 imagery using an improved annotation strategy to better handle ambiguous marine regions, substantially enhancing performance. Beyond demonstrating the onboard feasibility of the application, the $\Phi$sat-2 experience highlights the importance of robust annotation strategies and sensor-aware design, and shows that simulation-based development is valuable for pre-flight risk reduction, while reliable scientific validation requires representative in-orbit data and should be clearly distinguished from functional validation.
1. INTRODUCTION
Onboard Earth Observation AI promises faster, bandwidth-efficient processing but faces hardware, integration, and representative-data constraints. This paper presents a lightweight marine anomaly pipeline and its deployment and validation experience on Φsat-2.
1. INTRODUCTION
- Representative training data are scarce: mission imagery is unavailable before launch, and rare events may remain poorly represented and difficult to annotate after deployment.Models are therefore often developed with mission-representative simulated data and reassessed or adapted using real in-orbit imagery.Source: §1 · p. 1
- Marine anomaly methods often target predefined phenomena and may be sensor- and data-dependent, while few studies address the complete onboard chain and real in-orbit validation.Unsupervised and self-supervised detection offer a more generic, less label-dependent alternative.Source: §1 · p. 1
- The pipeline models normal sea state and detects deviations, with optional characterization of known phenomena using compact self-supervised representations, spectral features, and classical machine learning.Its design balances genericity, data frugality, and compatibility with constrained onboard hardware.Source: §1 · p. 1
- The paper documents Φsat-2 development and pre-flight validation on simulated imagery, onboard integration, comparison with early in-orbit acquisitions, and retraining on real data.Φsat-2 is the deployment with the most advanced integration and validation stage among the paper’s demonstration contexts.Source: §1 · p. 1
2. PIPELINE ARCHITECTURE AND DESIGN CHOICES
The pipeline builds on an earlier architecture and uses five sequential stages in a modular design.
2. PIPELINE ARCHITECTURE AND DESIGN CHOICES
- The pipeline follows a modular, five-stage architecture introduced in and refined in.The section refers to Fig. 1 for the architecture.Source: §2 · p. 1
OBPDC 2026
The onboard pipeline processes image patches through sequential sea filtering, feature encoding, anomaly detection, and anomaly characterization, then recombines them for visualization.
OBPDC 2026
- The Tiler splits the input image into patches, and sea patches are filtered before further processing.The sea/no-sea stage uses spectral features to retain patches with enough sea pixels.Source: OBPDC 2026 · p. 2, Figure 1 · p. 2
- Retained sea patches are encoded into a latent space, where a Gaussian mixture model scores deviations from learned normal sea conditions.The encoder is a lightweight convolutional model trained with SimCLR; anomaly detection does not require explicit examples of each anomaly type.Source: OBPDC 2026 · p. 2, Figure 1 · p. 2
- Patches exceeding an anomaly-score threshold can undergo per-pixel characterization as sea, oil, algae, or sediments, and the Mosaic'er recombines patches for visualization.Characterization uses three parallel NLSVMs.Source: OBPDC 2026 · p. 2, Figure 1 · p. 2
3. DEVELOPMENT AND VALIDATION ON SIMULATED DATA
Pre-flight development used simulated Φsat-2 imagery to train and assess a modular marine anomaly-detection pipeline and its hardware feasibility. In orbit, the pipeline executed correctly, but early real-data experiments exposed a simulation-to-flight mismatch that limited scientific assessment.
3. DEVELOPMENT AND VALIDATION ON SIMULATED DATA
- The pre-flight phase used simulated data to train and validate models, assess hardware feasibility, and establish a baseline for later in-orbit reassessment.Pipeline stages were evaluated independently on ground, assuming preceding stages operated perfectly.Source: §3 · p. 2, §3.2 · p. 3
3.1.1. Simulated Dataset Construction and Overview
- Simulated L1C datasets used Sentinel-2 imagery converted to TOA reflectance and included manually annotated marine-normality and anomaly scenes.Annotations covered sea/no-sea masks and land, sea, cloud, and anomaly masks; diffuse anomaly boundaries and imagery-only identification were difficult.Source: §3.1.1 · p. 2
3.1.2. Training Configuration
- Training combined supervised NLSVMs, a SimCLR self-supervised encoder, and a weakly supervised GMM modeling normal sea patches.The encoder mapped 32 × 32-pixel patches into a 16-dimensional latent space.Source: §3.1.2 · p. 2
3.2. Algorithmic Performance
- Sea/no-sea segmentation achieved an F1-score of almost 94%; algae and sediment detection F1-scores were above 60%, with characterization recalls above 87%.Oil spill detection precision and recall were significantly lower, and characterization recall was below 70%.Source: §3.2 · p. 3
3.3. Hardware Performance
- For a typical 4,096 × 4,096-pixel acquisition, estimated processing time was about five minutes, with the encoder the main computational bottleneck.The NLSVMs and GMM had limited impact on processing time; several acquisitions per day were possible depending on onboard processing windows.Source: §3.3 · p. 3, Table 2 · p. 4
4.1. Onboard Deployment and Functional Validation
- The pipeline ran to completion in the mission environment and was successfully executed onboard Φsat-2 across several in-orbit acquisitions.Mission-environment anomaly scores differed from ground-reference scores by a mean absolute error of 6 × 10^-4; characterization disagreement was 0.04%.Source: §4.1 · p. 4, Figure 3 · p. 4
4.2. Early Insights from First Φsat-2 Data
- Real Φsat-2 acquisitions produced many anomaly false positives despite satisfactory sea segmentation, revealing a mismatch with simulated training data that required retraining before scientific assessment.The real L1 products lacked TOA-reflectance conversion; Sentinel-2-based simulation also could not fully reproduce Φsat-2’s spatial and spectral characteristics.Source: §4.2 · pp. 4–5
5. ADAPTATION TO REAL MISSION DATA
After the first in-orbit acquisitions exposed a mismatch with simulated imagery, the pipeline was retrained on real Φsat-2 data using revised annotations. The results were strong, though limited data and a more tolerant evaluation protocol qualify their interpretation.
5. ADAPTATION TO REAL MISSION DATA
- Retraining replaced initially considered fine-tuning after in-orbit data revealed a substantial mismatch with simulated imagery.Lessons about annotation uncertainty informed both real-dataset construction and training to reduce evaluation bias.Source: §5 · p. 5
5.1.1. Real Dataset Construction and Annotation
- The real dataset comprised 17 normal-condition images and 7 images with sediment or algae anomalies, with oil spill samples excluded.Ambiguous labels captured uncertain boundaries and mixed phenomena; spectral-index masks were refined by image-processing experts.Source: §5.1.1 · p. 5
5.1.2. Pipeline Retraining
- Ambiguous pixels were excluded from anomaly-detection training and scoring, while characterization used multi-hot compatible-class targets instead of forcing single labels.The oil spill output was disabled because the real dataset contained no oil samples.Source: §5.1.2 · pp. 5–6
5.2. Updated Algorithmic Performance
- F1-scores above 94% for all three processing steps on both datasets, exceeding simulated-data validation results.Stages were assessed independently assuming perfect operation of preceding stages; the dataset was small, and ambiguous labels made evaluation more tolerant than strict single-label assessment.Source: §5.2 · p. 6, Table 3 · p. 6
- End-to-end outputs coherently represented sea-to-sediment transitions, while test-image algae predictions could be false positives or unannotated real patterns.This first real-data iteration improved performance but remains an initial operational validation requiring a more diverse dataset.Source: §5.2 · p. 6
6. LESSONS LEARNED AND PERSPECTIVES
The Φsat-2 deployment provided end-to-end experience adapting a lightweight marine anomaly pipeline from simulation to onboard validation. It highlighted limits of pre-flight evidence, uncertain annotations, sensor capabilities, and independently trained modules, alongside priorities for further in-orbit work.
6.1. Lessons Learned from the Φsat-2 Deployment
- Simulated-data validation reduced pre-flight risk but could not guarantee scientific performance because the simulations did not fully reproduce the sensor and processing chain.Functional validation checks execution and interfaces; scientific validation assesses detection relevance, robustness to real scenes, and generalization.Source: §6.1 · p. 7
- Ambiguous labels better represent diffuse or mixed phenomena, while Φsat-2’s eight-band VNIR configuration limits oil/sea separability.The paper identifies annotation strategy as affecting ground-truth quality, training and evaluation robustness, and measured performance.Source: §6.1 · p. 7
- Independent block training limits global optimization: errors can propagate, and end-to-end performance may be lower than stage-wise measurements.The paper suggests jointly trained multi-block models as a possible alternative while retaining modularity for diagnosis and targeted updates.Source: §6.1 · p. 7
6.2. Future Work and Perspectives
- Next, validate the retrained pipeline scientifically on new in-orbit acquisitions, then expand dataset coverage across conditions, regions, and anomaly types.Future work also includes a parallel expert model for water-quality parameters and possible coordinated observations across missions.Source: §6.2 · p. 7