Source-linked AI summary
RoIMix: Proposal-Fusion among Multiple Images for Underwater Object Detection
Wei-Hong Lin, Jia-Xing Zhong, Shan Liu, Thomas Li, Ge Li
TL;DR
Underwater object detection must handle low-quality imagery and densely overlapping, occluded, and blurred creatures, while existing augmentation methods do not directly address these conditions. RoIMix fuses proposals from multiple images to generate training samples for region-based detectors. It improves mAP on URPC and Pascal VOC and exhibits greater training stability and robustness.
Problem
Underwater images contain low contrast, texture distortion, uneven illumination, overlap, occlusion, and sediment-induced blur that existing augmentation methods have not adequately addressed.
Method
RoIMix performs proposal-level fusion by combining RoIs from multiple images to generate virtual training proposals for region-based detectors.
Results
RoIMix improves performance by 1.18% mAP on URPC and 0.8% mAP on Pascal VOC, while showing greater stability and robustness.
Takeaways & Limitations
Mixing proposals across images provides diverse training samples that simulate overlapping, occluded, and blurred objects for underwater detection.
Abstract
from arXiv · showhide
Generic object detection algorithms have proven their excellent performance in recent years. However, object detection on underwater datasets is still less explored. In contrast to generic datasets, underwater images usually have color shift and low contrast; sediment would cause blurring in underwater images. In addition, underwater creatures often appear closely to each other on images due to their living habits. To address these issues, our work investigates augmentation policies to simulate overlapping, occluded and blurred objects, and we construct a model capable of achieving better generalization. We propose an augmentation method called RoIMix, which characterizes interactions among images. Proposals extracted from different images are mixed together. Previous data augmentation methods operate on a single image while we apply RoIMix to multiple images to create enhanced samples as training data. Experiments show that our proposed method improves the performance of region-based object detectors on both Pascal VOC and URPC datasets.
1. INTRODUCTION
Underwater detection is challenged by low-quality imagery and densely packed creatures, while existing augmentation methods do not adequately simulate overlap, occlusion, and blur. RoIMix addresses this gap by fusing proposals from multiple images for region-based detection.
- Underwater images exhibit low contrast, texture distortion, uneven illumination, dense creatures, overlap, occlusion, and sediment-induced blur.
- Existing augmentation methods have not adequately researched overlapping, occluded, and blurred underwater objects.
- RoIMix mixes proposals from multiple images to simulate overlap, occlusion, and blur for improved detector generalization.
- Unlike single-image augmentation, RoIMix performs proposal-level fusion to avoid the proposal misalignment caused by image-level fusion.
- RoIMix was introduced for region-based detectors and reported performance improvements on both Pascal VOC and URPC.
2. RELATED WORK
Prior work established broad augmentation strategies for classification and detection, while two-stage detector variants addressed architectural limitations. RoIMix builds on these lines by targeting proposal-level interactions in region-based detectors.
- 2.1. Data Augmentation: Common classification augmentations include rotation, translation, flipping, image mixing, regional dropout, and patch-based CutMix.
- 2.1. Data Augmentation: Object detectors commonly use photometric distortion, image mirroring, and multi-scale training, while existing mixing methods were not specifically designed for detectors.
- 2.2. Faster R-CNN and its variants: Faster R-CNN combines a feature-extraction head, region proposal network, and RoI classifier in a two-stage detection architecture.
- 2.2. Faster R-CNN and its variants: R-FCN shares computation during classification, while FPN combines hierarchical features to improve predictions for small objects.
- 2.2. Faster R-CNN and its variants: RoIMix is described as potentially applicable to two-stage detector variants beyond Faster R-CNN.
3. METHODOLOGY
RoIMix augments region-based detection by mixing proposals from multiple images between the RPN and classifier, creating virtual RoIs that simulate overlap, occlusion, and blur. It uses VRM-style vicinal samples to support more robust training while leaving evaluation unchanged.
- Architecture: RoIMix operates between the RPN and RoI classifier, mixing randomly generated proposals into Mixed RoIs for localization and classification.The architecture retains the head network, RPN, and classifier while inserting RoIMix between proposal generation and classification.
- Proposal mixing: Two RoIs from multiple images are resized and combined with a random mixing ratio to generate a virtual training proposal.The mixing ratio is sampled from a beta distribution, and the second proposal is resized to match the first.
- Proposal mixing: The larger mixing weight is assigned to the first RoI so its label can supervise the mixed proposal.RoIMix mixes proposals without labels, uses y_i as the mixed label, and affects training but not evaluation.
- Augmented samples: Mixed virtual RoIs simulate overlapping, occluded, and blurred objects, replacing original proposals during training under the original loss.The method is illustrated by combining scallop and sea-urchin RoIs into an occlusion-like proposal.
- Learning rationale: RoIMix is framed as linear interpolation between proposals that follows VRM, generating changing vicinal data across epochs to smooth decision boundaries and improve robustness.The discussion contrasts this with ERM, which fits the training data too closely and can overfit.
4. EXPERIMENT
Experiments evaluate RoIMix on URPC and Pascal VOC, including ablations, qualitative detection, training stability, and robustness to noise and blur. RoIMix outperforms baseline and variants, with reported gains in detection performance, stability, and robustness.
- URPC 2018: URPC 2018 experiments use 2,901 trainval images, 800 test images, four categories, ResNet-101, 128 RoI features, and mAP evaluation.The experiments use Faster R-CNN default hyper-parameters.
- Ablation study: Mixing RoIs among multiple images achieves 0.41% mAP higher than mixing proposals on a single image.The ablation compares RoI mixing with Ground Truth mixing and Single RoIMix.
- Detection comparison: RoIMix successfully detects vague and overlapping holothurians and an incomplete scallop that the baseline misses.The qualitative comparison contains three marked cases where the proposed method succeeds and the baseline fails.
- Pascal VOC: RoIMix achieves 0.8% higher performance than the baseline on Pascal VOC and outperforms its variants.The evaluation uses the VOC 2007 test set trained on VOC 2007 and 2012 trainval data.
- Stability: RoIMix reaches a maximum 2.04% margin over baseline during training and maintains better mAP curves across both datasets.The baseline begins to overfit after its peak, while RoIMix remains more stable as training epochs increase.
- Robustness: RoIMix shows a maximum 9.05% mAP gap over baseline across five artificial noise types and improves Gaussian-blurred test performance by 0.7% mAP.The noise tests include Gaussian, Poisson, salt, pepper, and salt-and-pepper noise; the blur experiment uses Gaussian Blur.
5. CONCLUSION
RoIMix uses proposal-level fusion across multiple images to generate diverse training samples for underwater object detection. It simulates overlapping, occluded, and blurred objects and improves detection performance on URPC and Pascal VOC.
- RoIMix performs proposal-level fusion among multiple images to generate diverse training samples.The method was designed for underwater object detection.
- RoIMix simulates overlapping, occluded, and blurred objects so models can learn to detect underwater creatures.
- 1.18% mAP on URPC and 0.8% mAP on Pascal VOC were reported improvements from RoIMix.The method also exhibited more stability and robustness and was used in a first-prize URPC2019 solution.