Source-linked AI summary
On-Board Anomaly Detection for Efficient Marine Environmental Monitoring
Thomas Goudemant, Clotilde Szywala, Benjamin Francesconi, Michelle Aubrun, Yves Bobichon, Marjorie Bellizzi, Adrien Girard
TL;DR
Marine-threat monitoring must address methods that are difficult to adapt across missions and costly to run on constrained satellite hardware. This paper combines a self-supervised encoder with anomaly scoring for onboard processing; experiments report effective detection of algal blooms and sediments, but difficulty with oil spills.
Problem
Threat-specific and supervised approaches can be difficult to adapt across missions, generalize poorly, require large labeled datasets, or exceed constrained satellite computation.
Source: §1 · p. 1Method
A self-supervised encoder compresses multispectral patches into a latent space, where a GMM detects deviations from normal sea conditions.
Source: §2.2.2 · p. 3Results
Experiments showed effective detection of algal blooms and sediments, but oil spills were challenging because of overlap in the latent space.
Source: §5 · p. 7Takeaways & Limitations
Prioritizing anomalous or high-priority data for transmission can reduce data volume and support early alerting and low-bandwidth monitoring.
Source: §5 · p. 6Oil-spill detection is challenging because some spills are insufficiently separated in the latent space.
Source: §5 · p. 6
Abstract
from arXiv · showhide
Marine ecosystems are impacted by various threats such as oil spills, algal blooms, and sediment floods, which disrupt habitats, wildlife, and human activities. Advances in satellite imagery and Artificial Intelligence (AI) have enhanced our capabilities for early detection and mitigation of such hazards. In this paper, we propose a marine event detection pipeline for Earth observation satellites equipped with multi- or hyperspectral sensors. Our approach includes a self-supervised neural network encoder that compresses satellite images into a reduced latent space, enabling efficient onboard processing. A machine learning anomaly detection model identifies deviations from normal sea patterns to detect environmental anomalies. We compare its performance against traditional algorithms such as Isolation Forest, One-Class Support Vector Machine and Local Outlier Factors. Our lightweight, resource-efficient pipeline is optimized for deployment on satellites with limited computational resources, ranging from embedded CPUs to AI hardware accelerators. By prioritizing the transmission of critical information, our solution enhances system responsiveness and optimizes satellite communication bandwidth. Demonstrated through current integration across multiple missions, including European Space Agency's (ESA) Phisat-2 mission and Microsoft/Thales Alenia Space IMAGIN-e mission, our pipeline aims to improve marine environmental monitoring by providing timely alerts and efficient data reduction.
1. INTRODUCTION
Marine monitoring needs to detect threats across missions and sensors, but existing approaches face adaptation, generalization, annotation, and onboard-compute challenges. The paper proposes a self-supervised, compressed representation with generic anomaly detection for resource-limited satellite processing.
Research gap
- Threat-specific methods can be difficult to adapt across missions or regions, while traditional models may generalize poorly and supervised deep learning requires large labeled datasets.Self-supervised and reconstruction-based approaches may generalize better, but remain computationally heavy for constrained satellite hardware.Source: §1 · p. 1
Contribution
- The proposed self-supervised encoder compresses imagery for generic anomaly detection, reducing annotation needs and supporting adaptation across missions and sensors.Anomalies are elements that deviate from a normality represented by maritime images of unaffected ecosystems.Source: §1 · p. 1
Deployment context
- The solution is designed for small satellites and CubeSats, including Φsat-2 and IMAGIN-e, with their distinct multispectral and hyperspectral sensor and processor configurations.Φsat-2 uses a 5-meter, 8-band camera and Myriad 2 accelerator; IMAGIN-e uses a 50-meter, 50-band hyperspectral camera and a 16-core Cortex-A72 processor.Source: §1 · p. 1
Operational aim
- Onboard processing prioritizes critical information for transmission, aiming to reduce response time and optimize satellite bandwidth.The paper reports selection for Φsat-2 and IMAGIN-e and discusses potential deployment on Kanyini and CogniSAT-6.Source: §1 · pp. 1–2
2. METHODS
The methods combine patch-based processing, sea segmentation, latent-space anomaly detection, and pixel-level anomaly characterization. Lightweight NLSVM models handle segmentation tasks, while a self-supervised encoder and GMM score deviations from normal sea conditions.
2.1. Architecture Description
- The architecture divides images into 32 × 32 patches, filters non-sea patches, detects and characterizes anomalies, then recombines outputs for visualization and operational use.The pipeline includes sea/no-sea segmentation, latent encoding and anomaly scoring, followed by pixel-level segmentation of oil spills, algal blooms, sediments, and sea.Source: §2.1 · p. 2, Figure 1 · p. 2
2.2.1. Sea Segmentation and Anomaly Characterization
- NLSVMs perform binary sea/no-sea segmentation and pixel-level classification into oil, algae, sediments, and sea, using mission-selected spectral bands and features.Three parallel NLSVMs jointly classify pixels; the models are designed for fast inference on embedded CPUs.Source: §2.2.1 · pp. 2–3
2.2.2. Anomaly Detection
- A SimCLR-trained encoder compresses each multispectral patch into 16 latent values, which a GMM scores for deviation from learned normal sea patterns.The encoder has four convolutional layers and about one million parameters; the GMM models normal sea conditions using several dozen Gaussian components.Source: §2.2.2 · p. 3
3. DATABASE
The database combines simulated mission-matched normal marine scenes with manually annotated anomaly imagery. ESA-provided simulators generate sensor-representative images, and Level 1C imagery is used for training and evaluation.
3. DATABASE
- The pipeline uses separate mission-matched datasets for marine normality and anomaly training needs.Normality data support encoder and GMM training, while anomaly data support evaluation and anomaly characterization.Source: §3 · p. 3, §3.1.1 · p. 3, §3.1.2 · p. 3
3.1.1. Marine Normality Database
- The marine normality database contains about 60 simulated scenes with manually labeled sea/land masks, spanning Φsat-2 and IMAGIN-e image sizes and spectral bands.The encoder is trained unsupervised on these scenes; sea patches train the GMM and sea/land masks train the sea/no-sea NLSVM.Source: §3.1.1 · p. 3
3.1.2. Anomaly Database
- The anomaly database contains 31 simulated, manually annotated images covering oil spills, algal blooms, sediments, and plastic.Diffuse anomalies sometimes caused false positives or negatives during annotation; the dataset supports evaluation and multi-class NLSVM training.Source: §3.1.2 · p. 3
3.2. Φsat-2 and IMAGIN-e Simulators
- ESA-provided EO-learn simulators use Sentinel-2 Level 1C data and mission sensor characteristics to generate representative Φsat-2 and IMAGIN-e imagery.The study exclusively uses Level 1C images to avoid band misregistration and because Level 1C preprocessing is available onboard both missions.Source: §3.2 · p. 4
4. EXPERIMENTS
Experiments assess anomaly detection on simulated Φsat-2 and IMAGIN-e images and evaluate hardware latency and throughput. The pipeline detects algae and sediment anomalies but struggles with oil spills.
4.1. Settings of Model Trainings
- Training uses normalized temperature cross-entropy for the encoder, a 16-dimensional representation of each patch, and a GMM fitted to normal sea patches.The encoder input patches are 32 × 32 pixels, with 7 IMAGIN-e bands or 8 Φsat-2 bands.Source: §4.1 · p. 4
4.2. Algorithmic Results
- The algorithmic evaluation proceeds from representative anomaly-image predictions to latent-space analysis and then detector comparisons by anomaly type.The image results cover oil spills, algal blooms, and sediments from both missions.Source: §4.2 · p. 4
4.2.1. Overall Performance of the Pipeline
- Sea segmentation closely matches ground truth and anomaly characterization is accurate for most types, but oil-spill detection is difficult and false positives reflect incomplete annotations.The figure compares input images, model reconstructions, and prediction errors against ground truth.Source: §4.2.1 · p. 4, Figure 2 · p. 5
4.2.2. Latent Space Analysis
- Algae and sediment patches separate clearly from sea in the encoder’s latent space, while oil-spill patches remain closer to sea.The three main PCA axes explain over 70% of the variance; the figure shows labeled patch classes and anomaly types.Source: §4.2.2 · p. 4, Figure 3 · p. 5
4.2.3. Evaluation of Different Anomaly Detection Methods
- The evaluation compares classical anomaly detectors using precision, recall, and F1-score, and reports that performance varies by anomaly type.Table 1 covers algal blooms, sediments, and oil spills; the supplied passages do not report its cell values.Source: §4.2.3 · p. 4, Table 1 · p. 5
4.3. Hardware Results
- Three times on Myriad™ 2 and nine times on Myriad™ X: acceleration boosts encoder throughput, though the encoder remains the pipeline bottleneck.The GMM runs about ten times faster than the NLSVMs on encoded patches.Source: §4.3 · p. 6
5. DISCUSSION
The discussion reports efficient onboard anomaly detection and data reduction for marine monitoring, alongside a difficult oil-spill case and a need for in-orbit validation.
5. DISCUSSION
- Transmitting only anomalous or high-priority data optimizes bandwidth and responsiveness, while latent representations could support advanced ground-based analysis.Source: §5 · p. 6
- 152 patches/s are required for the hypothetical LEO configuration, and MyriadX achieves that rate for real-time onboard analysis.Source: §5 · p. 6, Table 2 · p. 6, Table 3 · p. 7
- Real in-orbit deployments on Φsat-2 and IMAGIN-e are proposed as a way to validate operational effectiveness under diverse environmental conditions.Source: §5 · p. 6
- The authors frame selective transmission as a scalable approach for future Earth-observation missions and efficient responses to environmental threats.Source: §5 · p. 6
- Experiments detected algal blooms and sediments, but oil-spill detection remained challenging because of latent-space overlap; representative mission hardware was suitable, especially with accelerators.Source: §5 · p. 7
- Each 8,192-value input patch is compressed to 16 latent values, achieving a compression factor of over 500 and reducing data flow.Source: §5 · pp. 6–7