PaperlayerView exact revision on arXiv

Source-linked AI summary

A New Dataset, Poisson GAN and AquaNet for Underwater Object Grabbing

Chongwei Liu, Zhihui Wang, Shijie Wang, Tao Tang, Yulong Tao, Caifei Yang, Haojie Li, Xing Liu, Xin Fan

arXiv:2003.01446v2cs.CV

TL;DR

Underwater robot picking is hindered by limited benchmark data, class imbalance, small objects, and constraints on onboard detection efficiency. The paper introduces a real-farm dataset, Poisson GAN augmentation, and AquaNet; it reports improved detection accuracy, including a 22.7% mAP50 gain for Poisson GAN over Baseline.

  • Problem

    Underwater robot picking lacks adequate benchmark data and must address class imbalance, small objects, and limited onboard computing resources.

    Source: §I · pp. 1–2

  • Method

    The paper builds UDD and uses Poisson GAN to create augmented and pre-training datasets, alongside AquaNet with MFF and MBP components for underwater detection.

    Source: §I · p. 2

  • Results

    22.7% on mAP50: Poisson GAN improves detection accuracy compared with Baseline.

    Source: 5) Comparison With Other Data… · p. 11

  • Takeaways & Limitations

    The paper provides a real-farm benchmark and associated augmented datasets and detector designs for underwater robot-grabbing research.

    Source: §I · p. 2, §VII · pp. 11–12

Abstract

from arXiv · show

To boost the object grabbing capability of underwater robots for open-sea farming, we propose a new dataset (UDD) consisting of three categories (seacucumber, seaurchin, and scallop) with 2,227 images. To the best of our knowledge, it is the first 4K HD dataset collected in a real open-sea farm. We also propose a novel Poisson-blending Generative Adversarial Network (Poisson GAN) and an efficient object detection network (AquaNet) to address two common issues within related datasets: the class-imbalance problem and the problem of mass small object, respectively. Specifically, Poisson GAN combines Poisson blending into its generator and employs a new loss called Dual Restriction loss (DR loss), which supervises both implicit space features and image-level features during training to generate more realistic images. By utilizing Poisson GAN, objects of minority class like seacucumber or scallop could be added into an image naturally and annotated automatically, which could increase the loss of minority classes during training detectors to eliminate the class-imbalance problem; AquaNet is a high-efficiency detector to address the problem of detecting mass small objects from cloudy underwater pictures. Within it, we design two efficient components: a depth-wise-convolution-based Multi-scale Contextual Features Fusion (MFF) block and a Multi-scale Blursampling (MBP) module to reduce the parameters of the network to 1.3 million. Both two components could provide multi-scale features of small objects under a short backbone configuration without any loss of accuracy. In addition, we construct a large-scale augmented dataset (AUDD) and a pre-training dataset via Poisson GAN from UDD. Extensive experiments show the effectiveness of the proposed Poisson GAN, AquaNet, UDD, AUDD, and pre-training dataset.

I. INTRODUCTION

The paper addresses limited underwater benchmarks, class imbalance, small objects, and onboard efficiency constraints with a real-farm dataset, Poisson GAN augmentation, and AquaNet. It also introduces augmented and pre-training datasets for underwater robot grabbing.

I. INTRODUCTION

  • Underwater detection faces limited benchmark data, class imbalance, many small objects, and onboard compute constraints that require balancing accuracy and speed.Existing contest datasets lack adequate image quality or acquisition environments and do not provide testing sets or comparable results.Source: §I · pp. 1–2
  • UDD contains 2,227 4K images of three creature categories collected on a real open-sea farm, and Poisson GAN expands it into AUDD and a pre-training dataset.The pre-training dataset contains 590,000 images; training on AUDD with its parameters improves detectors over ImageNet-pretrained or randomly initialized models on UDD.Source: §I · p. 2
  • Poisson GAN combines Poisson blending with a learning phase and Dual Restriction loss to vary object number, position, and size while generating realistic images.The loss supervises implicit-space and image-level features, and the generated objects can be annotated automatically.Source: §I · p. 2
  • AquaNet targets mass small objects in cloudy underwater images using the MFF block and MBP module to fuse multi-scale features efficiently.The introduction describes both components as lightweight and says they do not reduce accuracy.Source: §I · p. 2
  • UDD’s object-proportion distribution reflects many small objects because creature breeding quantities differ across the sea farm.The paper identifies this distribution as part of the detection challenge.Source: §I · p. 2, Figure 3 · p. 3

II. RELATED WORK

The related-work section reviews underwater detection datasets, GAN-based augmentation, and underwater object detection. It situates the work in the context of the URPC competition and its target-recognition task.

II. RELATED WORK

  • The related-work review covers background and datasets, GAN-based augmentation, and underwater object detection.The section organizes these topics as the relevant fields for the paper.Source: §II · p. 3

A. Background and Related Datasets

  • URPC2017 established a target-recognition track focused on detecting seacucumber, seaurchin, and scallop, alongside fixed-point and autonomous grasping tracks.The competition aimed to advance underwater agile-robot theory, technology, and industry.Source: A. Background and Related Datasets · p. 3
  • The authors identify two available underwater datasets before introducing their own dataset work.The supplied passage names the number of available datasets but does not describe them individually.Source: A. Background and Related Datasets · p. 3

1) URPC2017:

The paper describes limitations of URPC datasets and GAN augmentation for detection, then motivates a specialized underwater detector and presents UDD as a real-farm benchmark. UDD contains three creature categories and is described as the first 4K dataset collected in a real open-sea farm.

1) URPC2017:

  • URPC2017 contains 17,655 training and 985 test images, but its images come from six artificial-environment videos and are highly similar.The videos were sampled at 10-frame intervals.Source: 1) URPC2017 · p. 3

2) URPC2018:

  • URPC2018 has 2,901 training images, but its test set is unavailable and its image quality is described as unsatisfactory.Some images came from artificial underwater settings, and the creature counts indicate class imbalance.Source: 2) URPC2018 · p. 3

B. Data Augmentation with GAN

  • GAN augmentation methods can transform image styles, but the cited approaches may not change object number, size, or position, which matters for detection.CycleGAN augmentation may also overfit and affect model accuracy.Source: B. Data Augmentation with GAN · p. 3

C. Underwater Object Detection

  • Two-stage detectors can be computationally demanding, while one-stage methods are faster; underwater sea-farm detection still needs to balance speed and accuracy for robot platforms.The paper introduces AquaNet as a detector designed for this setting.Source: C. Underwater Object Detection · p. 3

III. UNDERWATER DETECTION DATASET

  • UDD contains 2,227 images of seacucumber, seaurchin, and scallop and is described as the first 4K dataset collected in a real open-sea farm.The authors present it as close to the real robot-grabbing environment and note possible use in culling and growth monitoring.Source: §III · p. 4

A. Dataset Construction

UDD was collected in a real underwater picking environment and contains 2,227 images. Its objects are highly imbalanced and predominantly small, and the paper describes Poisson GAN as a way to address insufficient data and imbalance.

A. Dataset Construction

  • UDD contains 2,227 pictures, with 1,827 for training and 400 for testing; the test set uses one picture in every seven of the total sequence.Images were collected at two underwater locations near Zhangzidao using a 4K camera operated by divers or robots.Source: A. Dataset Construction · p. 4

1) The number of instances:

  • UDD contains 15,022 objects: 1,148 seacucumbers, 13,592 seaurchins, and 282 scallops.The passage identifies this class-count disparity as a serious class-imbalance issue.Source: 1) The number of instances · p. 4

2) Instance size distribution:

  • UDD has the highest resolution (4K) and lowest instance percent (0.47%) among the compared datasets.The comparison table reports resolution, average instances per image, and average instance percentage.Source: 2) Instance size distribution · p. 4, Table I · p. 4
  • More than 90% of UDD instances occupy less than 1.654% of an image, and the average object size is 44 × 28 pixels.UDD averages 10 instances per image; the passage characterizes it as containing almost exclusively small instances.Source: 2) Instance size distribution · p. 5

IV. POISSON GAN

  • Poisson GAN is proposed to address insufficient data and class imbalance in UDD, unlike cited GAN-based augmentation methods mainly intended for classification.The passage presents Poisson GAN as a method for underwater object detection.Source: §IV · p. 5

A. Network Architecture

Poisson GAN combines Poisson blending and a learned image-refinement phase, with paired images designed to isolate blend-boundary differences. Its losses combine feature- and image-space supervision with adversarial training.

1) Generator:

  • The generator first changes object number, size, or position through Poisson blending, then learns a mapping to improve image realism.Objects are cropped into a category-based set and blended into backgrounds, placing them near objects of the same category.Source: 1) Generator · p. 5

B. Image Pairs Used to Train Poisson GAN

  • Training pairs use a real image and a fake image that differ mainly in the fusion-part edges, so the learning phase can learn boundary elimination.The fake image is made by pasting objects from clone images into locations of same-category objects in the real image.Source: B. Image Pairs Used to Train Poisson GAN · p. 5

C. Loss Function

  • DR loss combines content and region losses to supervise features and pixels, while adversarial loss distinguishes real and fake training pairs.Content loss uses VGG16 feature maps; region loss compares output and real images within the embedded-object mask.Source: C. Loss Function · pp. 5–6

2) Discriminator:

  • The discriminator uses a WGAN critic loss with gradient and drift penalties, with gradients penalized along lines between real and generator distributions.The discriminator is built from ResNeXt blocks and convolution layers and is configured to process 512 × 512 inputs.Source: 2) Discriminator · p. 5, C. Loss Function · p. 6

D. Expanded Dataset

The paper constructs Poisson GAN’s object set from cropped instances of all three UDD categories, then fuses those objects into background images.

D. Expanded Dataset

  • The object set uses 1,000 seaurchins, 150 seacucumbers, and 35 scallops, which are fused into background images through Poisson GAN.Source: D. Expanded Dataset · p. 6

1) AUDD dataset:

AUDD is generated from UDD training images using repeated copies and probabilistic paste operations. The resulting dataset contains 18,661 images with class counts reported for all three categories.

1) AUDD dataset:

  • AUDD contains 18,661 images, including 18,350 seacucumbers, 101,422 seaurchins, and 9,624 scallops.The passage describes the resulting images as nearly realistic and a supplement to UDD.Source: 1) AUDD dataset · p. 6
  • Each UDD training image is copied into 11 images, with one original added to AUDD and the other 10 subjected to paste operations.Source: 1) AUDD dataset · p. 6
  • For the 10 processed copies, the number of paste operations is 0, 1, 2, or 3 with probabilities 0.1, 0.35, 0.35, and 0.2.Each paste operation uses Poisson blending 0, 1, or 2 times with probabilities 0.3, 0.5, and 0.2.Source: 1) AUDD dataset · p. 6

2) Pre-training dataset:

The section describes a roughly 589,080-image pre-training dataset and AquaNet components intended to support robust, efficient detection of small objects in cloudy underwater images.

2) Pre-training dataset:

  • The pre-training dataset combines cropped underwater instances with backgrounds and groups images into hard, normal, and easy classes by object number or size.The classes contain about 8–10 tiny objects, 48 medium objects, or 2–3 large objects, respectively; each has about 197,000 images.Source: 2) Pre-training dataset · pp. 6–7

E. Discussion

  • Poisson GAN may also augment clustered pseudo-label images in unsupervised person re-identification to mitigate pseudo-label noise and enrich target-domain data.The proposed use is to train Poisson GAN in the source domain or the joint source and target domain.Source: E. Discussion · p. 7

V. AQUANET

  • AquaNet targets efficient detection of many small objects in cloudy underwater images using two modules.The section identifies robustness as important for detecting small objects in these images.Source: §V · p. 7

A. Multi-scale Blursampling

  • MBP applies stride-1 MaxPooling, splits channels into groups, and downsamples each group with an independent stride-2 BlurPool before concatenation.The experiments use three BlurPool groups; the filters use normalized Gaussian kernels of different sizes.Source: A. Multi-scale Blursampling · p. 7

B. Multi-scale Contextual Features Fusion

  • MFF expands channels, processes groups through sequential depth-wise filters of different kernel sizes, and fuses their outputs to aggregate multi-scale features.It also uses residual connections between the input and output and between expanded feature maps.Source: B. Multi-scale Contextual Features… · p. 7

C. Network Architecture

AquaNet uses a compact CenterNet-based architecture with MFF and MBP, and its ablations report gains from multi-scale blursampling and the MFF design. The discussion frames the modules as a way to balance detection accuracy and efficiency for underwater scenes.

C. Network Architecture

  • AquaNet adapts CenterNet and builds its backbone from two 3×3 convolution layers and eight MFF blocks with MBP to avoid losing or distorting small-object features in a deep backbone.The head predicts object centers, width and height, and center offsets at one-quarter input resolution.Source: C. Network Architecture · pp. 7–8

D. Discussion

  • MFF and MBP are designed to integrate multi-scale features while accounting for both efficiency and accuracy in detecting many small objects in cloudy underwater images.The discussion contrasts this aim with prior RNN-like pooling and fusion strategies –.Source: D. Discussion · p. 8

1) Experimental Settings:

  • The UDD ablations train all models from scratch for 1,600 epochs, using 512×512 inputs and random flip, scaling, and cropping augmentation.AquaNet experiments use Lookahead with Adam and an initial learning rate of 2.3×10−5.Source: 2) Ablation Studies · p. 8, 1) Experimental Settings · p. 8

2) Ablation Studies:

  • 5.5% higher accuracy than MaxPooling is reported for MBP, which also outperforms MaxBlurPool variants with a single kernel size.The authors attribute the improvement to anti-aliasing and multi-scale processing.Source: 2) Ablation Studies · p. 8, Table III · p. 8
  • Other GAN comparisons show that generated images may alter or remove objects, while the transformations do not resolve class imbalance.Fig. 8 compares Poisson GAN with CycleGAN [57], StarGAN [58], UGATIT [59], and MUNIT [60].Source: Figure 8 · p. 9
  • 4.6% accuracy improvement over the original MobileNetV2 block is reported for MFF with different kernels and skip connections.The comparisons include settings corresponding to MobileNetV2, MixNet, and the standard MFF block.Source: Table IV · p. 8, 2) Ablation Studies · p. 9

3) Comparison With Other Methods on UDD:

On UDD, AquaNet reports the highest scratch-trained mAP50 among the compared methods and competitive pretrained accuracy, alongside the fastest reported speeds on the tested platforms. The authors note that minority-class performance remains limited, motivating AUDD.

3) Comparison With Other Methods on UDD:

  • The study reports comparison results against public real-time methods and lightweight backbones, using UDD training data and scratch or pre-trained initializations.The speed evaluation uses TITAN XP and JETSON AGX XAVIER, with FPS defined from inference and post-processing time.Source: 3) Comparison With Other Methods on UDD · p. 9

Accuracy study

  • 55.3% mAP50 is reported for pretrained AquaNet, while scratch-trained AquaNet achieves the highest mAP50 among the compared methods.The reported mAP values for pretrained and scratch initialization are 55.3 and 47.4, respectively, despite nearly equal final training losses.Source: 3) Comparison With Other Methods on UDD · p. 9

Speed study

  • 5.6/9.1/10.0 FPS on JETSON AGX XAVIER modes 10W/15W/30W and 48 FPS on TITAN XP are reported for AquaNet.The authors attribute the TITAN XP speed advantage over ShuffleNet V2 partly to AquaNet's fewer blocks and narrower channels; speeds are nearly the same on XAVIER.Source: 3) Comparison With Other Methods on UDD · pp. 9–10

3) Comparison With Other Methods on UDD:

  • AquaNet and the other methods still fail to retain higher performance on seacucumber and scallop because those minority classes are insufficiently represented in training.The authors construct AUDD with Poisson GAN to address this issue.Source: 3) Comparison With Other Methods on UDD · p. 10

B. Poisson GAN

The experiments assess Poisson GAN through detector performance on UDD and URPC2018, YOLOv3 initialization on AUDD, and comparisons with image-to-image GANs. The reported results favor Poisson GAN augmentation and its pre-training data for these evaluations.

1) Experimental Settings:

  • Poisson GAN training uses 20,000 image pairs with random cropping and horizontal flipping to avoid mode collapse.The model uses 512×512 inputs, Adam, and batch size 4 on one TITAN XP GPU.Source: 1) Experimental Settings · p. 10

2) Image Quality Study:

  • The image-quality study evaluates whether augmentation improves detector performance, rather than judging image quality solely by human visual appearance.For URPC2018, 414 images were selected from its 2,901-image training set as a test set; the test set was otherwise unavailable.Source: 2) Image Quality Study · p. 10
  • Datasets produced by Poisson blending improve detector performance over the original datasets, and the learning phase further improves results on both UDD and URPC2018.Removing any of DR loss's content, region, or adversarial losses results in worse image quality.Source: 2) Image Quality Study · p. 10, Table VI · p. 10

3) Pre-training Dataset:

  • 80.5% mAP50 is reported for YOLOv3 with the Poisson GAN pre-trained model, 11.7% above scratch training and 9.0% above ImageNet initialization.YOLOv3 is trained on AUDD and evaluated on the UDD test set.Source: 3) Pre-training Dataset · p. 10, Table V · p. 10

4) Comparison With Image-to-Image Translation GANs:

  • 30.22 dB PSNR is reported for Poisson GAN, the highest score among the compared GANs on 5,000 generated images.The other methods may alter or remove objects, and their transformations do not resolve class imbalance; Poisson GAN changes object positions while preserving objects and underwater textures.Source: 4) Comparison With Image-to-Image… · p. 11, Table VII · p. 11

5) Comparison With Other Data Augmentation Methods:

Poisson GAN outperforms the reported augmentation baselines, and the conclusion frames it alongside UDD and AquaNet for underwater robot grabbing.

5) Comparison With Other Data Augmentation Methods:

  • The comparison trains AquaNet on AUDD for Poisson GAN and on UDD for the other augmentation methods, then tests all models on UDD.The evaluated alternatives include GridMask, RErase, Cutout, HaS, and Mixup.Source: 5) Comparison With Other Data… · p. 11
  • 22.7% on mAP50 improvement over Baseline is reported for Poisson GAN.Table VIII compares augmentation methods without test augmentation or with flip testing.Source: Table VIII · p. 11, 5) Comparison With Other Data… · p. 11
  • Mixup improves accuracy by 2.5% in the flip test, while information-dropping methods perform worse than Baseline.The authors report a remaining large performance gap between Mixup and Poisson GAN.Source: 5) Comparison With Other Data… · p. 11

VII. CONCLUSIONS

  • The conclusion presents UDD, Poisson GAN-generated datasets, and AquaNet with MFF and MBP as contributions for underwater robot grabbing.The authors report comprehensive experiments to verify the proposed schemes.Source: §VII · pp. 11–12
Loading 2003.01446v2…