Source-linked AI summary
AI-Based On-Board Maritime Object Detection for Earth Observation Payload Data Reduction on Versal Embedded Hardware
Thomas Goudemant, Aurélien Bobey, Omar Hlimi, Marjorie Bellizzi
TL;DR
Very-high-resolution satellite data exceed available storage and downlink capacity, while vessels occupy only a small fraction of maritime scenes. The paper trains and deploys an onboard vessel detector and uses its measured errors to estimate several downlink-selection modes. On dense harbor and coastal scenes, tiles retain nearly all vessels for under a third of the volume, while crops retain four fifths for 3% of the volume.
Problem
Very-high-resolution satellite data volumes grow faster than downlink capacity, although maritime vessels occupy only a small fraction of scenes.
Source: INTRODUCTION · p. 1Method
The paper trains a detector on controlled maritime data, deploys it on embedded hardware, and estimates downlink volumes for selection modes using measured detection errors.
Source: INTRODUCTION · p. 1, Axis 3: Data Reduction by Onboard… · p. 4Results
On dense scenes, crops show four fifths of vessels for 3% of volume, while tiles keep nearly all vessels for under a third of volume.
Source: Axis 3: Data Reduction · p. 7Takeaways & Limitations
The mode and confidence threshold set the trade-off between transmitted volume and vessels potentially lost, with processing energy weighed against transmission energy and storage avoided.
Source: DISCUSSION · p. 7The study covers one application and DPU configuration, uses corrected ground products, extrapolates scene-level figures from test-tile error rates, and evaluates dense scenes rather than open sea.
Source: DISCUSSION · p. 7
Abstract
from arXiv · showhide
Very-high-resolution Earth-observation satellites acquire more data than they can store and downlink, while in maritime surveillance the vessels cover a tiny fraction of each scene. We study onboard vessel detection as a way to select what is downlinked, which reduces the data according to its content rather than coding every pixel; it is complementary to conventional onboard compression. The work follows three axes. (i) Data and algorithm: a controlled dataset is generated from 68 annotated Maxar scenes with 43 vessel classes, and a YOLOX-S detector is trained on it. (ii) Embedded deployment: the detector is quantized and deployed on the DPU of a Versal VC1902, with a limited loss of detection quality and a processing time of a few seconds per scene. (iii) Data reduction: we propose several downlink modes, from metadata only (box, class and score of each detection) to image crops around vessels, tiles holding detections, or the whole scene with a degraded background, and estimate from the measured detection errors the trade-off each offers between the vessels kept and the volume downlinked. On our dense harbor and coastal scenes, tiles keep 98% of the vessels with 29% of the scene volume, and crops 83% with 3%.
INTRODUCTION
Very-high-resolution satellites face a mismatch between growing data volumes and limited storage and downlink capacity, while vessels occupy little of maritime scenes. This work quantifies how onboard vessel detection can guide content-based downlink selection alongside conventional compression.
INTRODUCTION
- Data volumes grow faster than downlink capacity, and maritime scenes devote most pixels to open sea rather than vessels.Mass memory must buffer acquired data until a ground station is in view.Source: INTRODUCTION · p. 1
- Onboard vessel detection can select boxes, crops, or more-compressed backgrounds, complementing conventional compression that reduces bits per pixel without identifying useful content.Boxes are compact metadata and can define crops or tiles; object-level selection is finer than existing cloud screening.Source: INTRODUCTION · p. 1
- Prior work addressed onboard detection quality and embedded feasibility, but rarely quantified the data reduction allowed by detections and their errors on real scenes.The paper ties reduction estimates to detector performance on controlled data and target hardware.Source: INTRODUCTION · p. 1
METHOD
The paper treats vessel detection as one stage in a broader onboard chain from image acquisition to downlink. Its scope is the tile-by-tile AI stage and the selection driven by its outputs, not onboard image correction or tiling.
METHOD
- The full chain corrects raw imagery, tiles scenes, detects vessels tile by tile, mosaics detections, then selects what is stored and downlinked.The AI stage outputs boxes, classes, and scores for scene-level selection.Source: METHOD · p. 2
- The study uses corrected ground products already cut into 1024 × 1024 tiles, so onboard corrections and tiling are outside its scope.Tiling and mosaicking are described in a companion paper.Source: METHOD · p. 2
- The method covers three axes: data and detector, embedded deployment, and using detector outputs to reduce downlinked data..Source: METHOD · p. 2
Axis 1: Data and Algorithm
The data-and-algorithm axis builds a controlled, traceable tile dataset from annotated Maxar scenes and trains YOLOX-S for vessel detection. Tile construction controls empty-background prevalence, class balance, and spatial splitting.
Axis 1: Data and Algorithm
- 68 annotated Maxar scenes with 43 vessel classes are converted to 8-bit RGB and cut on a regular 1024 × 1024 grid.The scenes cover harbors, dense coastal traffic, and canals at 30–70 cm ground sampling distance.Source: Axis 1: Data and Algorithm · p. 2
- Every vessel-containing grid tile is kept, while 30% of empty tiles are randomly retained to limit background dominance.Extra vessel-centered tiles add views of rare classes and whole vessels without one tile per vessel.Source: Axis 1: Data and Algorithm · p. 2
- Neighboring tile blocks are split 80/10/10 across training, validation, and test, though centered tiles can place some vessels in two subsets.Vessel identifiers link each vessel to its tiles, scene, and coordinates; the inverse record lists vessels per tile.Source: Axis 1: Data and Algorithm · p. 2
- YOLOX-S is trained in floating point for 100 epochs, with validation tiles for quantization calibration and test scores measured on grid tiles.Detection is evaluated separately from classification using IoU ≥0.5 matching; data reduction uses a lower confidence threshold than the deployed 0.30 threshold.Source: Axis 1: Data and Algorithm · pp. 2–3
Axis 2: Embedded Deployment
The detector is quantized for int8 inference and deployed on a Versal VC1902 DPU at one compiled operating point. The measured pipeline runs preprocessing and postprocessing on Arm cores and inference on the DPU.
Axis 2: Embedded Deployment
- The VCK190 board uses a Versal VC1902, and Vitis AI post-training quantization calibrates the model on validation tiles before compiling one DPU operating point.The DPUCVDX8G C32B6 configuration processes batches of six 1024 × 1024 tiles.Source: Axis 2: Embedded Deployment · p. 3
- The C++ pipeline scales tiles on Arm cores, runs inference on the DPU, then decodes boxes, thresholds confidence, and applies per-class NMS on the CPU.Tile reads from board storage are excluded because tiles would arrive as a stream from the tiler.Source: Axis 2: Embedded Deployment · p. 3
- Measurements cover per-batch stage latency, derived throughput, and board power, separating static from processing-related dynamic energy.Power is sampled every 2 ms; the measurements yield energy per tile and per scene.Source: Axis 2: Embedded Deployment · p. 3
Axis 3: Data Reduction by Onboard Filtering
The paper compares four detection-driven downlink modes with the full scene: metadata, crops, tiles containing detections, and selective coding. It estimates their volume and vessel-loss trade-offs from measured detector errors, which affect each mode differently.
Downlink modes
- Four modes retain different content: metadata sends detection records only, crops add vessel images, tiles retain any grid tile with a detection, and selective coding keeps the scene with a compressed background.The comparison uses the full scene before onboard compression as its reference.Source: Axis 3: Data Reduction by Onboard… · pp. 3–4
- Figure 3 illustrates the full-scene reference, metadata, crops with a 32-pixel margin, detection-holding tiles, and selective coding on 2 × 2 grid tiles.Source: Figure 3 · p. 4
- The modes are compared with the full scene before onboard compression, and may be combined across downlinks with different rates and availability.The paper quantifies product volume, not the link schedule.Source: Axis 3: Data Reduction by Onboard… · p. 4
Error-dependent volume estimates
- Missed vessels are lost in metadata and crops, may survive in a tile with a detected neighbor, and remain compressed in selective-coding backgrounds.False alarms add a record or crop, but add a tile only when they fall on an empty tile; the volume estimate uses threshold 0.10 rather than the deployed 0.30.Source: Axis 3: Data Reduction by Onboard… · p. 4
- The estimated volume and vessel losses are applied to all 68 scenes, and the results depend strongly on scene density; the evaluated harbor and coastal scenes are a nominal dense case.The paper characterizes its figures as conservative relative to open-sea acquisitions, where vessels are sparser and reduction would be larger.Source: Axis 3: Data Reduction by Onboard… · p. 4
Axis 1: Data and Algorithm
The controlled dataset contains a strongly imbalanced vessel-class distribution, and the float detector performs better at detection than at vessel-type classification. Reported mAP covers the ten most frequent classes, while detection scores are reported at threshold 0.30.
Dataset
- 13 967 tiles: the generated dataset retains two fifths of empty tiles, while four classes account for 80% of vessels and 33 classes each account for less than 1%.The test set roughly follows the overall class distribution; rare classes may not appear in every subset under the spatial split.Source: Axis 1: Data and Algorithm · p. 4, Table 2 · p. 5
Evaluation
- 91% of test vessels belong to the ten most frequent classes used to average mAP; the test set contains 894 tiles and 1 596 vessels.The Table 3 caption specifies IoU ≥ 0.5; AP and mAP are threshold-independent, while other scores are reported at 0.30.Source: Table 3 · p. 5, Axis 1: Data and Algorithm · p. 5
Hardware evaluation
- Table 4 reports hardware performance at the selected operating point on the VCK190, using batches of six 1024 × 1024 tiles.Its caption distinguishes the board-at-rest static value from the dynamic increase during processing; no table values are supplied here.Source: Table 4 · p. 5
Detection performance
- 86% AP and 85% F1: the float detector reaches these test-set detection scores, while one detected vessel in seven receives the wrong type.F1 is maximal near threshold 0.30 and changes by only 2 to 4 points between 0.2 and 0.5; the threshold trades missed vessels against false alarms.Source: Axis 1: Data and Algorithm · p. 5, Table 3 · p. 5
Axis 2: Embedded Deployment
Quantization preserves most detection performance but shifts confidence downward, while threshold adjustment recovers much of the recall loss. The deployed chain processes an average scene in a few seconds, with CPU preprocessing and DPU inference sharing the time.
Detection performance after deployment
- AP drops from 86 to 81 % and F1 from 85 to 81 % after quantization, while lowering the threshold to 0.10 recovers most missed vessels at the cost of more false alarms.The quantized model's lower confidence scores shift its operating point; recall is lower at every threshold.Source: Axis 2: Embedded Deployment · p. 5, Figure 5 · p. 6
Hardware performance
- About 17 ms per tile yields an average-scene processing time of a few seconds on the target hardware.CPU preprocessing and DPU inference take roughly equal time; post-processing is negligible.Source: Axis 2: Embedded Deployment · pp. 5–6
- Static consumption dominates energy: processing adds only about a seventh to power at rest, and board static power is representative within about 2 %.Input tiles read from board storage take longer than processing them.Source: Axis 2: Embedded Deployment · p. 6
Axis 3: Data Reduction
The detector's missed vessels and false alarms affect crops and tiles differently, shaping each mode's trade-off between downlinked volume and vessels retained. On dense scenes, tiles lose fewer vessels than crops, while false alarms add volume to both modes.
Missed vessels and false alarms, extrapolated to the scenes
- 17 % of vessels are lost with crops, but tile mode loses 2 % of vessels when 7 % of tiles are lost in the dense scenes.A missed vessel is lost in tile mode only when no other vessel in its tile is detected; isolated open-sea vessels would be lost about as often as with crops.Source: Axis 3: Data Reduction · p. 6
- False alarms add volume in both image modes: a fifth of downlinked tiles and over a quarter of crop volume contain only false alarms.False-alarm crops retain full margins, while neighboring vessel crops can overlap and share margins.Source: Axis 3: Data Reduction · p. 6, Table 7 · p. 7
Volume and vessels kept per mode
- Crops show images of four fifths of vessels for 3 % of volume, while tiles keep nearly all vessels for under a third of volume.At threshold 0.10, false alarms account for 21 to 28 % of the volume of the first three modes.Source: Axis 3: Data Reduction · p. 7, Table 7 · p. 7
- Raising the threshold to 0.30 saves about a fifth of tile or crop volume, but increases vessel losses more for crops than tiles.The threshold moves each mode along its volume-versus-vessels-kept trade-off curve.Source: Axis 3: Data Reduction · p. 7, Figure 6 · p. 7
DISCUSSION
The value of onboard selection depends on mission energy and acquisition conditions, and the study's estimates are bounded by its data and hardware scope. The authors identify raw-data integration, open-sea scenes, and link-aware orchestration as next steps.
DISCUSSION
- The gain is largest when detection runs before storage, as in the tile-by-tile pipeline; energy savings depend on acquisition and downlink frequency.Because energy is mostly static, faster processing and powering the processor down between acquisitions may save more than lowering dynamic power.Source: DISCUSSION · p. 7
- The study covers one application and one DPU configuration, uses corrected rather than raw data, and extrapolates scene-level results from test-tile error rates.Dense scenes do not represent open sea, selective-coding ratios are representative rather than codec-measured, and downlink orchestration is not studied.Source: DISCUSSION · p. 7
- Next steps include integrating detection into the raw-data onboard chain, evaluating open-sea scenes, and orchestrating modes over links of different rates.The threshold could adapt to scene density, and open-vocabulary detectors could make selection follow user requests rather than a fixed class list.Source: DISCUSSION · p. 8
CONCLUSION
The paper connects controlled-data training, quantized onboard deployment, and downlink-mode estimates based on the deployed detector's errors. On dense harbor and coastal scenes, crops sharply reduce volume while losing vessels, whereas tiles retain nearly all vessels at a larger volume share.
CONCLUSION
- On dense harbor and coastal scenes, crops divide volume by about 35 while keeping four vessels in five, and tiles divide it by about 3.5 while keeping almost all vessels.The authors argue that onboard reduction should be judged from deployed-model errors and how each mode absorbs them, not detection scores alone.Source: CONCLUSION · p. 8