Source-linked AI summary
Robotic Detection of Marine Litter Using Deep Visual Detection Models
Michael Fulton, Jungseok Hong, Md Jahidul Islam, Junaed Sattar
TL;DR
Marine debris threatens aquatic ecosystems, but underwater visual detection is difficult across changing environments and highly varied, degraded objects. The paper builds and annotates a realistic open-water dataset, evaluates four deep-learning detectors and YOLOv2 training variants across three devices, and finds that real-time detection is plausible, with performance especially encouraging on the Jetson TX2. Dataset representativeness and data scarcity for complex classes remain important limitations.
Problem
Underwater trash detection is difficult because environmental changes and debris diversity can make objects difficult or impossible to detect, while effective removal requires scalable capabilities.
Method
The paper constructs a publicly available underwater-debris dataset, trains four object detectors plus YOLOv2 variants, and evaluates them on challenging unseen data across three devices.
Results
Real-time deep-learning visual detection of marine debris is plausibly achievable, with especially encouraging observed performance on the NVIDIA Jetson TX2.
Takeaways & Limitations
Detectors similar to those evaluated could potentially find underwater trash with high enough accuracy to be useful in an AUV application.
Takeaways & Limitations
Representative dataset construction is difficult, and classes with few training images relative to their complexity can perform poorly.
Abstract
from arXiv · showhide
Trash deposits in aquatic environments have a destructive effect on marine ecosystems and pose a long-term economic and environmental threat. Autonomous underwater vehicles (AUVs) could very well contribute to the solution of this problem by finding and eventually removing trash. This paper evaluates a number of deep-learning algorithms preforming the task of visually detecting trash in realistic underwater environments, with the eventual goal of exploration, mapping, and extraction of such debris by using AUVs. A large and publicly-available dataset of actual debris in open-water locations is annotated for training a number of convolutional neural network architectures for object detection. The trained networks are then evaluated on a set of images from other portions of that dataset, providing insight into approaches for developing the detection capabilities of an AUV for underwater trash removal. In addition, the evaluation is performed on three different platforms of varying processing power, which serves to assess these algorithms' fitness for real-time applications.
1 Introduction
Marine debris threatens aquatic ecosystems, while underwater visual detection is difficult because appearance varies with lighting, turbidity, degradation, and object type. The paper evaluates deep-learning detectors, training techniques, and multi-device performance toward near-real-time AUV deployment.
- Marine debris kills and injures aquatic life, chokes ecosystems, contaminates water, and remains difficult to remove at scale.Limited recycling impact and manpower requirements motivate robotic approaches.
- The paper asks whether deep-learning visual detection of underwater trash is plausible in real time and how current methods perform.Detection is presented as the first capability needed for AUV-based debris operations.
- Underwater debris detection is challenging because lighting, turbidity, degradation, occlusion, and extreme object variety alter or obscure appearance.Plastic debris alone includes many forms, including bottles, bags, and containers.
- The study evaluates four state-of-the-art object detection algorithms for underwater trash detection.It compares their accuracy and performance using several metrics.
- The study also evaluates different training techniques for YOLOv2 and tests models on three devices spanning offline, high-powered robotic, and low-powered platforms.The device comparison gauges readiness for near-real-time robotic deployment.
- The paper contributes a unique marine-debris dataset for training deep visual detection models.
2 Related Work
Related work spans marine-debris monitoring, underwater robotics, sensing, and terrestrial trash removal. Prior underwater deep-learning detection demonstrated promise but relied on water-tank data, motivating evaluation with realistic open-water imagery.
- Underwater robots have been studied for environmental surveillance, mapping, navigation, localization, and detection of biological events.
- Prior marine-debris research examined open-ocean locations, dispersal patterns, floating debris, derelict fishing gear, and post-tsunami surface removal.Challenges include debris that is small, partially submerged, or buried in the seabed.
- LIDAR has been used to find and map trash on beaches, while autonomous trash detection and pickup has also been investigated in terrestrial environments.The cited indoor terrestrial system uses ultrasonic sensors rather than underwater vision.
- Forward-looking sonar imagery paired with a deep CNN achieved approximately 80 per cent accuracy for underwater debris detection.That evaluation used an in-house water-tank dataset created by placing common debris objects in a tank.
- Existing ROV-based efforts include manually operated debris removal and proposed floating cleanup vehicles, but the cited FRED platform is not an UAV.
- A deep-learned underwater trash detector requires a large annotated dataset covering diverse environments and varied appearances.Many existing datasets are not annotated for deep-learning purposes.
3 Data Source and Training Set Construction
The study constructs an underwater plastic-debris training set from publicly available real-world video and uses a compact three-class labeling scheme. It also defines a challenging, non-overlapping test set for evaluation.
- Data source: The dataset uses real-world J-EDI videos varying in quality, depth, objects, cameras, decay, occlusion, overgrowth, water clarity, and lighting.These variations are intended to conform more closely to real-world conditions than internally generated datasets.
- Dataset construction: Training data came from debris videos recorded between 2000 and 2017, with videos containing plastic selected for the study.
- Dataset construction: Over 240,000 frames were sampled at three frames per second, yielding a final training set of 5,720 annotated 480x320 images.
- Data model: The data model contains three classes: Plastic for all plastic materials, ROV for intentionally placed man-made objects, and Bio for natural biological material.
- Data model: The reduced class count targets all plastic debris while improving inference speed and generality and preventing confusion with non-trash objects.
- Data model: Alternative models with timestamp labels or multiple plastic-object classes achieved lower accuracy across all networks.The alternatives were rejected because of lower accuracy, insufficient data for some classes, or both.
- Test set: The test set contains 820 annotated images from previously unused object videos, with at least three videos and 20 images per class or object type.Images were selected across varied and intentionally challenging environments.
4 NETWORK ARCHITECTURES
The study compares four popular underwater object-detection architectures, each balancing detection accuracy, localization quality, and runtime speed differently.
- YOLOv2: YOLOv2 improves YOLO using batch normalization, higher-resolution inputs, anchor boxes, and the faster Darknet-19 backbone.Darknet-19 requires about one sixth of VGG-16’s floating-point operations for one image pass.
- Tiny-YOLO: Tiny-YOLO reduces convolutional layers and filters to achieve faster inference while retaining similar mAP values.Its reduced computational demands suit field-robotics applications with limited processing power.
- Faster R-CNN: Faster R-CNN uses a Region Proposal Network to make object detection end-to-end trainable.The implementation replaces the original VGG-16 feature extractor with Inception v2.
- SSD: SSD performs localization and classification in one forward pass, adding convolutional layers to its base network; this implementation uses MobileNet v2.The selected architectures differ in accuracy and runtime speed, which motivates their comparison.
- Comparison framework: Table 1 reports detection metrics in mAP, IoU, and AP, while Table 2 reports runtime performance in frames per second.Together, the tables frame the comparison around detection quality and processing speed.
5 TRAINING
The networks were trained on a common marine-debris image set using pretrained weights, with YOLO receiving an additional transfer-learning treatment.
- Training setup: Each network used 5,720 training images, with images resized to 416x416 for YOLOv2 and Tiny-YOLO.Training followed community-recommended methods and generally fine-tuned pretrained object-detection weights.
- Transfer learning: YOLOv2 was also trained with transfer learning by freezing all but its last several layers.This strategy updated only the final layers while retaining pretrained basic image features in earlier layers.
6 EVALUATION
Evaluation combines detection-quality metrics with runtime measurements across GPU, embedded-GPU, and CPU platforms to assess deployment-relevant performance.
- Detection metrics: mAP averages precision across recall values, while precision and recall use true-positive, false-positive, and false-negative counts.These metrics quantify detection performance across different recall levels.
- Detection metrics: IoU measures how well predicted bounding boxes fit object locations by dividing intersection area by union area.The intersection and union are taken between the true and predicted bounding boxes.
- Hardware evaluation: Runtime is measured in frames per second on an NVIDIA 1080 GPU, Jetson TX2 embedded GPU, and Intel i3-6100U CPU.The platforms approximate offline processing, a powerful robotic platform, and a lower-power robotic platform.
- YOLO training evaluation: Table 3 reports detection metrics for different YOLO training methods.The table supports comparison of training approaches within YOLO.
- Hardware evaluation: mAP, IoU, and recall can be calculated identically on every device, so hardware differences affect runtime rather than detection metrics.This separates model accuracy from platform-dependent processing speed.
7 Results
The evaluated detectors exhibit a speed–accuracy trade-off, while bounding-box overlap remains similar across architectures. YOLOv2 balances accuracy and speed, whereas Tiny-YOLO is especially promising on realistic AUV hardware.
- YOLOv2 and Tiny-YOLO have lower mAP than Faster R-CNN and SSD, while Faster R-CNN and SSD require more processing time.
- All four architectures have similar IoU values, so none is clearly superior in bounding-box accuracy.
- Faster R-CNN offers the highest accuracy but falters in inference time, while YOLOv2 provides a balance of accuracy and speed.
- SSD provides the best inference times on CPU, whereas Tiny-YOLO significantly outpaces the other algorithms on the Jetson TX2.
- Transfer learning greatly increases YOLO accuracy for the bio class, while leaving earlier layers unchanged slightly reduces plastic accuracy and increases bio accuracy.
- The evaluation finds underwater marine-debris detection plausibly feasible in real time, particularly on the 15-watt Jetson TX2.
8 FUTURE WORK
Future work focuses on addressing limited and insufficiently representative underwater-trash data while advancing AUV capabilities toward autonomous debris removal and multi-robot collaboration.
- The authors plan to test similar algorithms on datasets collected from their own real-world observations of marine debris.
- Because sufficient coverage is difficult to collect, planned approaches include transfer learning, progressive neural networks, and generative data synthesis.
- The broader goal is an AUV that clears marine environments with little to no human help through navigation, exploration, and manipulation strategies.
- The authors are interested in multi-robot collaboration between aerial and aquatic robots for solving marine-trash removal.
9 Conclusion
The paper evaluates four deep-learning object detectors on a publicly available underwater-debris dataset using challenging unseen test objects and three processing platforms. The results support visual trash detection as plausibly real-time, including as an initial capability for an AUV designed to remove aquatic debris.
- The study evaluates YOLOv2, Tiny-YOLO, Faster R-CNN, and SSD for marine-debris detection using a single trash class.
- The dataset was built from publicly available data, and models were tested on challenging objects not seen during training.
- Evaluations across three devices measured mAP, IoU, and runtime speeds for all model configurations.
- The authors conclude that visual deep-learning models can plausibly detect underwater trash in real time.