Source-linked AI summary
MariSat: A Maritime Dataset for Instance Segmentation of Objects in Satellite and Aerial Images
Amir Abbes, Ines Harrabi, Lucas Justin Yirepoa Kinda, Rim Trabelsi, Adnane Cabani, Fatma Abdelkefi
TL;DR
Maritime surveillance needs richly annotated overhead imagery because existing sensing and vision approaches have limitations, especially for small vessels and cluttered scenes. MariSat provides a pixel-level, eight-class dataset built through semi-automatic annotation and manual quality control, then benchmarks segmentation and detection models with class-wise results. The experiments show strong overall performance alongside persistent difficulty for small, under-represented vessel classes and a dataset scope limited by size, imbalance, annotation residuals, and one imagery source.
Problem
Richly annotated maritime imagery for instance segmentation is scarce, while AIS, radar, and traditional vision methods have limitations in detecting or classifying small vessels in cluttered scenes.
Method
MariSat combines SAM 3 text-promptable pre-annotation, geometric and colorimetric filtering, and manual refinement and quality control in CVAT to create pixel-level masks for eight vessel classes.
Results
SAM 3 achieves the strongest overall precision and segmentation quality, while YOLO11l improves recall and mAP50:95 over YOLO11n but jet-ski remains the hardest class to detect.
Takeaways & Limitations
MariSat supports benchmarking both instance segmentation and object detection on diverse port and coastal scenes using a shared eight-class annotation set.
Takeaways & Limitations
The dataset contains only 1260 images, is imbalanced for classes such as jet-skis and sailboats, may retain minor annotation inaccuracies, and comes from a single imagery source.
Abstract
from arXiv · showhide
Automated maritime surveillance from satellite and aerial imagery requires large, precisely annotated datasets, which remain scarce for the instance-segmentation task, particularly for small vessels in cluttered port environments. We present MariSat, a new benchmark dataset of 1260 aerial and satellite images covering diverse port and coastal scenes, annotated at the pixel level for eight maritime object classes (sailboat, yacht, jet-ski, fishing boat, cruise ship, military vessel, tugboat and cargo ship). The dataset was produced through a semi-automatic annotation pipeline combining the textpromptable segmentation model SAM 3 with a cascade of geometric and colorimetric post-processing filters, followed by a manual correction and quality-control pass performed with the CVAT annotation platform. We describe the image-collection methodology, the annotation and correction process, and the resulting data organization. We also report class-wise statistics for the training, validation, and test splits. MariSat has already been used to fine-tune and benchmark segmentation and detection models (SAM 3 and YOLO11) for real-time maritime monitoring. We report detailed quantitative and per-class results for both tasks. The MariSat dataset is publicly available on GitHub : https://github.com/amirabbes/P2M-Maritime-Segmentation
1 Introduction
MariSat addresses the scarcity of richly annotated maritime imagery needed to detect and precisely delineate diverse vessels in cluttered overhead scenes. It supports both multi-class instance segmentation and object detection across eight maritime categories.
- More than 90% of world trade travels by sea, making automated monitoring of coasts and ports strategically important for maritime safety and traffic management.
- AIS can miss small or non-cooperative craft, while radar has difficulty classifying vessel types and degrades in adverse weather.
- Traditional computer-vision methods struggle with vessel-shape and texture diversity, wave foam, and water glare.
- Deep-learning approaches using attention mechanisms improve maritime detection by capturing vessel-context relationships and suppressing complex background noise.
- Richly annotated, class-diverse maritime instance-segmentation data are scarce because pixel-level labeling is costly, especially for small objects in high-altitude imagery.
- MariSat targets multi-class instance segmentation and bounding-box detection for the same eight vessel categories, with detection labels derived from segmentation masks.
2 Dataset Construction
MariSat was constructed from 1,260 aerial and satellite images using automated OpenAerialMap collection, SAM 3 pre-annotation, filtering, manual CVAT correction, and batch-level split rebalancing. The resulting pixel-level dataset supports eight maritime classes and both segmentation and detection benchmarks.
- 2.1 Image collection: OpenAerialMap imagery yielded 1,260 images spanning geographically diverse port and coastal scenes and all target maritime classes.The collection used an API-based traversal of image tiles from operator-specified geographic starting points and directions.
- 2.2 Semi-automatic annotation with SAM 3: SAM 3 generated instance masks and predicted classes from eight free-text prompts, providing semi-automatic pre-annotations for each raw image.The prompts corresponded to the eight target object classes listed in Table 1.
- 2.3 Post-processing filters: Four cascaded filters removed overlapping, oversized, non-elongated, or non-maritime masks before manual review.The filters comprised NMS, a 30%-area limit, a length-to-width threshold of 1.2, and HSV-based water-region rejection.
- 2.4 Manual correction with CVAT: CVAT quality control removed residual false positives, added missed small or occluded vessels, and refined imprecise mask contours.Filtered annotations were exported in XML and imported into CVAT for systematic manual refinement.
- 2.5 Dataset organization: Before rebalancing, fishing boat, yacht, and cargo ship were over-represented relative to sailboat, jet-ski, and cruise ship, with several splits deviating from 70%/15%/15%.The initial distribution was reported across training, validation, and test subsets.
- 2.5 Dataset organization: Batch-level reassignment moved split proportions closer to 70/15/15 while preserving image–batch integrity and leaving total images per split unchanged.Sailboat validation share increased from 1% to 17%, while tugboat validation share changed from 21% to 18%.
- 2.5 Dataset organization: MariSat provides pixel-accurate masks for eight classes and supports instance segmentation directly plus object detection through bounding boxes derived from those masks.The dataset was used to fine-tune and benchmark SAM 3 and YOLO11 for the two tasks.
3 Experiments and Results
The experiments fine-tune SAM 3 for instance segmentation and YOLO11 variants for object detection on MariSat, evaluating overall and per-class test performance. SAM 3 provides the strongest overall precision and segmentation quality, while YOLO11l improves recall and mAP50:95 over YOLO11n, especially for smaller classes.
- Experimental setup: SAM 3 was fine-tuned for instance segmentation, while YOLO11n and YOLO11l were fine-tuned for detection using bounding boxes derived from segmentation polygons.SAM 3 evaluation groups sailboat, jet-ski and fishing boat into a small-vessel category; YOLO11 uses the original eight-class taxonomy.
- Overall results: SAM 3 achieves the strongest overall precision and segmentation quality on the held-out test set.The result is consistent with SAM 3’s text-promptable design aligning with the semi-automatic annotation pipeline.
- Overall results: +0.124 recall and +0.063 mAP50:95 distinguish YOLO11l from YOLO11n, alongside a precision trade-off.YOLO11n is more conservative, whereas YOLO11l detects more true vessels at the cost of more false positives.
- Per-class results: 0.689 recall is the lowest reported value for the SAM 3 small-vessel category, whose errors commonly involve cargo ship and tugboat.The confusion matrix also reports 23 cargo-ship and 14 tugboat false positives.
- Per-class results: 0.063 and 0.099 mAP50:95 make jet-ski the hardest YOLO11 class for the nano and large variants, respectively.Jet-ski is both the smallest and one of the least represented classes; YOLO11n records perfect precision but zero recall for it.
- Per-class results: YOLO11l raises sailboat recall from 0.235 to 0.585 and fishing boat recall from 0.164 to 0.441 relative to YOLO11n.The gains come with a moderate precision cost and only partially compensate for class imbalance.
4 Conclusion
MariSat is a 1260-image benchmark with pixel-level instance masks for eight maritime object classes, built through semi-automatic annotation and organized for segmentation and detection benchmarks.
- Dataset contribution: MariSat contains 1260 aerial and satellite images with pixel-level instance masks for eight maritime object classes.The dataset covers diverse port and coastal scenes and includes training, validation and test splits.
- Dataset contribution: The dataset combines SAM 3 text-promptable pre-annotation, geometric and colorimetric filters, and manual CVAT correction.Baseline experiments with SAM 3 and YOLO11 demonstrate the dataset’s usefulness.
5 Ethics Statement
The dataset uses freely licensed OpenAerialMap imagery and depicts vessels and port infrastructure rather than individuals.
- Data ethics: All source imagery comes from OpenAerialMap, an open, freely licensed catalogue of satellite imagery.The dataset is intended for research on maritime object detection and segmentation.
- Data ethics: The images contain vessels and port infrastructure, with no personally identifiable information captured.
6 Limitations
MariSat’s scope is constrained by its limited size, class imbalance, annotation uncertainty, and reliance on a single imagery source.
- Dataset limitations: 1260 images and class imbalance, particularly for jet-skis and sailboats, constrain performance on under-represented classes.The limitation passage states that this imbalance negatively affects detection performance.
- Dataset limitations: Minor annotation inaccuracies may remain despite refinement with CVAT.
- Dataset limitations: Using imagery from only OpenAerialMap may limit generalization to other sensors and resolutions.
- Availability: The dataset and annotations are publicly available for reproducible research and future maritime-vision development.