Source-linked AI summary
A Realistic Fish-Habitat Dataset to Evaluate Algorithms for Underwater Visual Analysis
Alzayat Saleh, Issam H. Laradji, Dmitry A. Konovalov, Michael Bradley, David Vazquez, Marcus Sheaves
TL;DR
Existing fish datasets inadequately represent complex underwater habitats and mainly support classification, limiting comprehensive visual analysis. DeepFish introduces a large-scale, multi-task benchmark with diverse in-situ imagery and expanded annotations, and experiments show strong performance from ImageNet-pretrained models while leaving room for further improvement.
Problem
Existing fish datasets are small, constrained, and focused largely on classification, inadequately capturing the variability and complexity of real-world underwater habitats.
Method
DeepFish combines approximately 40 thousand images from 20 tropical Australian marine habitats with point-level and per-pixel annotations for classification, counting, localization, and segmentation.
Results
ImageNet-pretrained models achieved near-perfect classification and an MAE of 0.38 for counting, compared with 1.30 for randomly initialized counting models.
Takeaways & Limitations
DeepFish provides a challenging benchmark and baseline results for developing and evaluating models that analyze remote underwater fish habitats.
Takeaways & Limitations
Future work will add benchmarks and annotations for weak supervision, active learning, and few-shot learning with minimal human-provided labels.
Abstract
from arXiv · showhide
Visual analysis of complex fish habitats is an important step towards sustainable fisheries for human consumption and environmental protection. Deep Learning methods have shown great promise for scene analysis when trained on large-scale datasets. However, current datasets for fish analysis tend to focus on the classification task within constrained, plain environments which do not capture the complexity of underwater fish habitats. To address this limitation, we present DeepFish as a benchmark suite with a large-scale dataset to train and test methods for several computer vision tasks. The dataset consists of approximately 40 thousand images collected underwater from 20 \green{habitats in the} marine-environments of tropical Australia. The dataset originally contained only classification labels. Thus, we collected point-level and segmentation labels to have a more comprehensive fish analysis benchmark. These labels enable models to learn to automatically monitor fish count, identify their locations, and estimate their sizes. Our experiments provide an in-depth analysis of the dataset characteristics, and the performance evaluation of several state-of-the-art approaches based on our benchmark. Although models pre-trained on ImageNet have successfully performed on this benchmark, there is still room for improvement. Therefore, this benchmark serves as a testbed to motivate further development in this challenging domain of underwater computer vision. Code is available at: https://github.com/alzayats/DeepFish
1 Introduction
DeepFish addresses the limited realism and task coverage of existing fish datasets with a large-scale benchmark built from diverse underwater habitats. It adds annotations and baseline evaluations for more comprehensive fish analysis.
- Motivation: Monitoring fish in natural habitats supports sustainable fisheries, ecosystem health, and more efficient environmental assessment.Automated comprehensive monitoring could reduce labour costs and increase efficiency.
- Problem: Existing fish datasets are small and fail to capture challenging underwater conditions, including visual similarity, occlusion, and habitat variability.These limitations can inhibit progress in comprehensive visual understanding of underwater environments.
- Dataset: DeepFish contains approximately 40 thousand high-resolution images from 20 marine habitats in tropical Australia.The images represent diverse coastal and nearshore benthic habitats.
- Benchmark: DeepFish extends classification labels with point-level and semantic segmentation annotations for counting, localization, and fish size and shape analysis.The expanded labels support multiple perspectives on fish-habitat analysis.
- Contributions: The benchmark captures greater habitat complexity and diversity than previous fish datasets while providing reference results and pretrained-model evidence for future methods.The dataset and code were made public to encourage progress in underwater fish-habitat analysis.
2 Dataset
DeepFish adapts a large, diverse in-situ underwater dataset into a benchmark covering classification, counting, localization, and segmentation. It adds annotations and habitat-aware splits to support comprehensive evaluation against existing fish datasets.
- Dataset: DeepFish repurposes approximately 40 thousand images from diverse underwater fish habitats as a machine-learning benchmark.The original dataset captured high variability but was not designed to evaluate machine-learning methods.
- Data collection: Images were collected in 20 remote coastal marine habitats using cameras lowered to the seabed, allowing low-disturbance recording of natural fish communities.The vessel maintained a distance of 100m during recording, and cameras operated during daylight hours.
- Additional annotations: The original fish-presence classification labels were supplemented with point-level and semantic segmentation labels for counting, localization, and segmentation.These annotations also support analyses of fish dynamics, sizes, shapes, and weight.
- Additional annotations: Point-level annotations cover 3200 images, while per-pixel segmentation labels cover 620 images.Point annotations record fish-centroid coordinates; segmentation masks distinguish fish pixels from background.
- Dataset splits: The benchmark defines FishClf, FishLoc, and FishSeg sub-datasets for classification, counting/localization, and segmentation, respectively.Each sub-dataset is divided into training, validation, and test sets.
- Dataset splits: Splits are constructed per habitat with 50% training, 20% validation, and 30% testing while balancing background and foreground images.The resulting datasets contain 19,883/7,953/11,930 FishClf images, 1,600/640/960 FishLoc images, and 310/124/186 FishSeg images.
- Comparison to other datasets: Compared with QUT, Rockfish, and Fish4Knowledge, DeepFish provides greater size, annotation richness, scene complexity, and task coverage.DeepFish addresses classification, counting, localization, and segmentation, whereas the other datasets address fewer tasks.
3 Methods and Experiments
The benchmark evaluates standard deep learning methods across classification, counting, localization, and segmentation, using ResNet-50 backbones with task-specific output modules. Experiments compare ImageNet-pretrained and randomly initialized models using task-appropriate metrics and labels.
- DeepFish evaluates classification, counting, localization, and segmentation with standard deep learning methods.
- ResNet-50 extracts image features, followed by either a feed-forward network for scalar outputs or an upsampling path for per-pixel outputs.
- Classification: Classification identifies foreground images containing fish versus background images without fish, using accuracy on FishClf.
- Counting: 0.38 MAE was achieved by the ImageNet-pretrained counting model, compared with 1.30 for random initialization.The pretrained model averaged 0.38 fish miscounts per image, while the randomly initialized model averaged 1.30.
- Localization: Localization uses GAME because MAE alone can reward correct counts at incorrect positions; increasing L makes the metric more restrictive.GAME divides each image into 4^L non-overlapping regions and sums regional MAE scores, with GAME(0) equivalent to MAE.
- Localization: The localization model uses a ResNet-50 backbone, three-layer FCN8 upsampling path, LCFCN training, and thresholded connected components for object locations and counts.The upsampling path produces a per-pixel probability map; thresholding at 0.5 yields blobs whose connected components represent object instances.
- Segmentation: Segmentation labels every pixel as fish or not fish, using focal loss to address foreground-background imbalance and IoU for evaluation.The pretrained segmentation model achieved superior quantitative and qualitative results compared with random initialization.
4 Conclusions and Perspectives
DeepFish is introduced as a large-scale benchmark for developing and evaluating models that analyze diverse, complex underwater fish habitats. The authors identify future expansion toward additional benchmarks and annotations, including settings requiring minimal human labeling.
- 4 Conclusions and Perspectives: DeepFish provides diverse images, point-level and per-pixel annotations, and evaluation setups for classification, counting, localization, and segmentation.The benchmark is intended to support efficient analysis of remote underwater fish habitats.
- 4 Conclusions and Perspectives: ImageNet-pretrained deep learning methods demonstrate efficacy on the benchmark and provide baselines for evaluating new models.These baselines are intended to support comparison in this problem setup.
- 4 Conclusions and Perspectives: Future work will add benchmarks and annotations for additional fish-analysis use cases.Planned challenges include weak supervision, active learning, and few-shot learning with minimal human effort for label collection.
Additional Information
The authors report no competing interests and state that the work received animal-ethics approval and relevant fisheries and Great Barrier Reef Marine Park permits.
- Additional Information: The authors declare no competing interests.
- Additional Information: The study was approved by the JCU Animal Ethics Committee and conducted under specified fisheries and Great Barrier Reef Marine Park permits.The cited protocol is A2258; the permits are DAFF #168652 and GBRMP #CMES63.