PaperlayerView exact revision on arXiv

Source-linked AI summary

Boosting R-CNN: Reweighting R-CNN Samples by RPN's Error for Underwater Object Detection

Pinhao Song, Pengteng Li, Linhui Dai, Tao Wang, Zhan Chen

arXiv:2206.13728v3cs.CV

TL;DR

Underwater object detectors struggle with vague objects produced by blur, low contrast, occlusion, and mimicry. Boosting R-CNN combines uncertainty-aware proposals, probabilistic score inference, and prior-error-based reweighting. It reports robust performance on underwater and generic object detection datasets, including state-of-the-art results on underwater data.

  • Problem

    Underwater imaging conditions create vague objects, while generic detectors often fail to distinguish their boundaries from the background or other objects.

  • Method

    Boosting R-CNN uses RetinaRPN for objectness, IoU, localization, and prior uncertainty; probabilistic inference combines first-stage priors with R-CNN scores; boosting reweighting emphasizes prior-miscalculated samples.

  • Results

    Boosting R-CNN achieves state-of-the-art performance on two underwater datasets and favorable performance on Pascal VOC and MS COCO, while maintaining relatively high inference speed.

  • Takeaways & Limitations

    The detector is designed to remain robust to vague underwater objects by using first-stage uncertainty to guide proposal scoring and second-stage learning.

Abstract

from arXiv · show

Complicated underwater environments bring new challenges to object detection, such as unbalanced light conditions, low contrast, occlusion, and mimicry of aquatic organisms. Under these circumstances, the objects captured by the underwater camera will become vague, and the generic detectors often fail on these vague objects. This work aims to solve the problem from two perspectives: uncertainty modeling and hard example mining. We propose a two-stage underwater detector named boosting R-CNN, which comprises three key components. First, a new region proposal network named RetinaRPN is proposed, which provides high-quality proposals and considers objectness and IoU prediction for uncertainty to model the object prior probability. Second, the probabilistic inference pipeline is introduced to combine the first-stage prior uncertainty and the second-stage classification score to model the final detection score. Finally, we propose a new hard example mining method named boosting reweighting. Specifically, when the region proposal network miscalculates the object prior probability for a sample, boosting reweighting will increase the classification loss of the sample in the R-CNN head during training, while reducing the loss of easy samples with accurately estimated priors. Thus, a robust detection head in the second stage can be obtained. During the inference stage, the R-CNN has the capability to rectify the error of the first stage to improve the performance. Comprehensive experiments on two underwater datasets and two generic object detection datasets demonstrate the effectiveness and robustness of our method.

1. Introduction

Underwater object detection is important for AUV visual tasks but faces vague objects caused by difficult imaging and scene conditions. Boosting R-CNN addresses these challenges through uncertainty modeling and hard example mining, achieving strong performance across underwater and generic benchmarks.

  • Underwater object detection supports AUV tasks such as marine organism capturing.
  • Unbalanced lighting, low contrast, occlusion, and aquatic organisms’ camouflage make underwater object boundaries difficult to distinguish.
  • Existing detectors can produce false over-confident predictions because standard supervision and cross-entropy loss do not accurately reflect object vagueness.
  • Boosting R-CNN combines RetinaRPN, probabilistic inference, and boosting reweighting to model uncertainty and emphasize hard examples.
  • Experiments on two underwater and two generic detection datasets report state-of-the-art underwater performance, relatively high inference speed, and favorable generic detection performance.

2. Related Work

Related work spans two-stage and one-stage detectors, hard example mining, and underwater-specific detection methods. Boosting R-CNN differs by defining example hardness through RetinaRPN’s estimated prior probability and reweighting samples accordingly.

  • Two-stage detectors generate candidate boxes with an RPN before an R-CNN head performs classification and regression, whereas one-stage detectors predict boxes and classes directly.
  • Some methods combine one-stage efficiency with two-stage components, such as anchor refinement or deformable convolutions that imitate RoIAlign.
  • Hard example mining emphasizes difficult samples using strategies including highest-loss selection, IoU-based sampling, focal loss, and IoU-based ranking.
  • Boosting R-CNN uses a soft reweighting mechanism that amplifies hard-example loss and shrinks easy-example loss based on RetinaRPN priors.
  • Underwater detection research addresses AUV needs with augmentation, feature extraction, sample reweighting, and prediction-refinement methods.

3. Boosting R-CNN

Boosting R-CNN combines uncertainty-aware proposal scoring with boosting-based RoI reweighting in a two-stage detector. RetinaRPN models objectness and IoU, probabilistic inference combines proposal priors with classification, and boosting reweighting emphasizes samples whose priors are miscalculated.

  • 3.1. Overview: Boosting R-CNN uses RetinaRPN, probabilistic inference, and boosting reweighting as its three key components.The backbone and feature-fusion neck extract image features before RetinaRPN generates proposals with prior probabilities.
  • 3.3. RetinaRPN: RetinaRPN predicts objectness, box localization, and IoU to produce proposals with uncertainty-aware object priors.Its objectness branch uses focal loss, while IoU prediction reflects proposal overlap and occlusion-related quality.
  • 3.3. RetinaRPN: Fast IoU loss adds IoU-based weighting to reduce outlier vulnerability and focus RetinaRPN on samples with moderate regression accuracy.The weighted term becomes small for low-quality samples with high regression loss, while retaining fast convergence.
  • 3.4. Probabilistic Inference Pipeline: The probabilistic inference pipeline combines first-stage prior probability with second-stage conditional classification probability to form the final detection score.In implementation, the final score is the square root of the product of RetinaRPN’s prior and the R-CNN classification score.
  • 3.5. Boosting Reweighting: The probabilistic pipeline can fail when a high-quality proposal receives a low first-stage prior, motivating boosting reweighting for vague underwater objects.Because the final score multiplies prior and classification score, a mistaken low prior can suppress a high-confidence prediction.
  • 3.5. Boosting Reweighting: Boosting reweighting increases classification loss for hard samples with miscalculated priors and decreases it for easy samples with accurately estimated priors.This soft sampling strategy trains the R-CNN head to rectify errors from the first stage; unlike OHEM, it uses one feedforward pass.

4.1. Datasets

The experiments use four object-detection datasets: two underwater benchmarks and two generic detection benchmarks, with dataset-specific categories, splits, resolutions, and evaluation protocols.

  • The evaluation covers four datasets: UTDAC2020 and Brackish for underwater detection, plus Pascal VOC and MS COCO for generic object detection.The paper states that experiments on four challenging datasets validate generalization performance.
  • UTDAC2020 contains 5,168 training images and 1,293 validation images across four underwater classes and four image resolutions.Evaluation follows the COCO-style metric.
  • Brackish contains 9,967 training, 1,467 validation, and 1,468 test images across six classes, with 25,613 annotations.The evaluation uses COCO-style AP[0.5:0.95:0.05] and Pascal VOC-style AP50 metrics.
  • Pascal VOC contains 20 categories; training uses the VOC2007 and VOC2012 trainval sets, while evaluation uses the VOC2007 test set.The VOC2007 portion has 9,963 annotated images, and VOC2012 has 11,540 trainval images.
  • MS COCO contains 80 categories, with 118k training images, 5k validation images, and test-dev evaluation without provided test annotations.

4.2. Implementation Details

The implementation uses two training recipes and specifies optimization, augmentation, inference, and loss-balancing settings for the experiments.

  • The default recipe uses 1x training for 12 epochs with SGD, weight decay 0.0001, momentum 0.9, and an initial learning rate of 0.005.The learning rate drops by 10× at epochs 8 and 11, with no extra augmentation beyond horizontal flipping.
  • The fast IoU loss uses η as a hyper-parameter controlling the IoU-weighted term, with η=0 removing that term.
  • The second recipe uses 3x training for 36 epochs with crop and multi-scale augmentation, AdamW, an initial learning rate of 0.0001, and weight decay 0.05.The learning rate drops by 10× at epochs 24 and 33.
  • The method is trained on one NVIDIA GTX 1080Ti GPU, and inference retains at most 256 proposal boxes in the second stage to improve speed.
  • The loss-balancing parameters λ_obj-rpn, λ_loc-rpn, λ_iou-rpn, λ_cls, and λ_reg are set to 1, 2, 1, 2, and 2, respectively.

4.3. Comparisons with Other State-of-the-art Methods

Boosting R-CNN achieves strong results across underwater and generic detection benchmarks, while maintaining competitive inference speed on UTDAC2020.

  • UTDAC2020: 48.5% AP in single-scale UTDAC2020 training exceeds DetectoRS, PAA, and CenterNet2, while multi-scale training reaches 51.4% AP versus CenterNet2’s 48.9% AP.The paper reports new state-of-the-art performance on UTDAC2020.
  • UTDAC2020: 13.5 FPS on UTDAC2020 exceeds most two-stage detectors, including Faster R-CNN at 11.6 FPS, but remains below CenterNet2 at 14.2 FPS.
  • UTDAC2020: Anchor-based one-stage methods generally outperform anchor-free methods in the underwater comparisons, which the authors relate to anchors providing boundary priors for vague organisms.
  • Brackish: 82.0% AP and 97.4% AP50 on Brackish are the highest reported values, exceeding DetectoRS by 0.4% on both metrics for the stated comparison.
  • Pascal VOC: 81.9% mAP in single-scale Pascal VOC training exceeds the listed Faster R-CNN, Cascade R-CNN, DSSD513, and CenterNet2 results, while multi-scale training reaches 83.0% mAP.
  • MS COCO: On MS COCO test-dev, Boosting R-CNN achieves 44.4% AP with ResNet50 and 50.7% AP with Res2Net101-DCN, exceeding GFLV2’s 50.6% AP with the same backbone.

4.4. Ablation Study

Ablations show that RetinaRPN, probabilistic inference, boosting reweighting, and PAFPN each contribute to performance on UTDAC2020, while assignment and loss choices materially affect results.

  • Overall ablation: 48.5% AP is achieved with the full configuration using PAFPN, compared with 44.5% AP for the default Faster R-CNN.RetinaRPN, probabilistic inference, boosting reweighting, and PAFPN raise performance incrementally.
  • Overall ablation: 47.9% AP results from adding probabilistic inference after RetinaRPN raises performance from 46.9% AP.The pipeline incorporates first-stage prior uncertainty into prediction.
  • Hard example mining: 48.3% AP is obtained with boosting reweighting, exceeding OHEM at 45.1% AP when used alone and improving the probabilistic configuration from 47.9% AP.The comparison evaluates alternative hard example mining methods in the R-CNN head.
  • RetinaRPN: 47.5% AP with IoU prediction improves to 47.9% AP with the proposed fast IoU loss, outperforming L1, GIoU, CIoU, and focal EIoU losses.The reported alternatives achieve 47.5%, 47.6%, 47.6%, and 47.7% AP, respectively.
  • RetinaRPN: 48.5% AP is obtained when the fast IoU-loss hyper-parameter η is set to 2; overly large or small values reduce the desired filtering behavior.Large η emphasizes easy high-IoU samples, while small η weakens outlier filtering.
  • Hard example mining: 48.3% AP is achieved when the boosting-reweighting parameter ω is set to 0.5, with normalization improving performance and shifting the optimum.PAFPN is not used in this experiment.
  • Anchor assignment: The (0.5, 0.5) anchor-assignment setting performs best, whereas ATSS, PAA, and OTA decrease performance when used as RPN assignment strategies.The paper attributes this decrease to over-confident priors reducing recall.

4.5. Qualitative Comparisons

Qualitative comparisons evaluate challenging underwater conditions using matched, unmatched, and missed-ground-truth boxes, showing how Boosting R-CNN handles difficult cases and corrects weak first-stage priors.

  • Comparison protocol: Figure 6 marks matched predictions in orange, unmatched predictions in blue, and undetected ground truths in red.More blue boxes indicate lower precision, while more red boxes indicate lower recall.
  • Challenging conditions: Boosting R-CNN detects all ground truths under blurring and low contrast with the highest precision, represented by only one unmatched prediction.The comparison uses top-k predictions, where k equals the number of ground-truth boxes.
  • Challenging conditions: Under unbalanced lighting, Boosting R-CNN detects the central echinus missed by ATSS, PAA, and DetectoRS.The third row of Figure 6 represents the unbalanced-light condition.
  • Variant analysis: The variant visualization compares replacing second-stage scores with priors, removing probabilistic inference, and using the full Boosting R-CNN.These variants isolate the roles of first-stage priors and the probabilistic inference pipeline.
  • Variant analysis: When RetinaRPN assigns too small a prior, R-CNN correction increases the second-stage score so the missed ground truth can be detected.This illustrates the intended error-correction role of the second stage.

5. Conclusion

The paper concludes that Boosting R-CNN addresses underwater blur, low contrast, occlusion, and mimicry through uncertainty modeling and hard example mining, with strong underwater and competitive generic-detection results.

  • Method: Boosting R-CNN combines RetinaRPN, probabilistic inference, and boosting reweighting in a two-stage underwater detector.RetinaRPN estimates objectness, IoU, localization, and object prior probability; the other components use and correct these priors.
  • Results: The detector outperforms other state-of-the-art detectors on two public underwater datasets and achieves competitive performance on two public generic object detection datasets.The conclusion presents the generic-detection results as evidence of generalization.
Loading 2206.13728v3…