Source-linked AI summary
Underwater Fish Detection using Deep Learning for Water Power Applications
Wenwei Xu, Shari Matzner
TL;DR
Manual analysis of underwater video is time-consuming, and fish detection at water-power sites poses challenges in difficult conditions. The paper trains YOLO using annotated videos from three sites and tests cross-site generalization. The model failed to recognize fish in a dataset withheld from training, leading the authors to conclude that new datasets require training for correct detection.
Problem
Manual underwater-video analysis takes time, while challenging MHK and hydropower environments have received little attention in fish-detection research.
Source: §I · p. 1Method
The authors contribute three manually annotated underwater fish-video sources and apply YOLO to evaluate fish detection and use at another site.
Source: §I · p. 2Results
The model trained on Wells Dam and Voith Hydro did not generalize to the held-out Igiugig dataset and could not correctly detect fish there.
Source: B. Experiment II: Reserve Igiugig… · p. 5Takeaways & Limitations
The authors conclude that a model applied to a different video dataset needs training on that new dataset to correctly detect fish.
Source: B. Experiment II: Reserve Igiugig… · p. 5Detection accuracy is often limited by video quality in challenging real-world underwater environments.
Source: §V · p. 5
Abstract
from arXiv · showhide
Clean energy from oceans and rivers is becoming a reality with the development of new technologies like tidal and instream turbines that generate electricity from naturally flowing water. These new technologies are being monitored for effects on fish and other wildlife using underwater video. Methods for automated analysis of underwater video are needed to lower the costs of analysis and improve accuracy. A deep learning model, YOLO, was trained to recognize fish in underwater video using three very different datasets recorded at real-world water power sites. Training and testing with examples from all three datasets resulted in a mean average precision (mAP) score of 0.5392. To test how well a model could generalize to new datasets, the model was trained using examples from only two of the datasets and then tested on examples from all three datasets. The resulting model could not recognize fish in the dataset that was not part of the training set. The mAP scores on the other two datasets that were included in the training set were higher than the scores achieved by the model trained on all three datasets. These results indicate that different methods are needed in order to produce a trained model that can generalize to new data sets such as those encountered in real world applications.
I. INTRODUCTION
Underwater video can support environmental monitoring, but manual review is time-consuming and challenging conditions at water-power sites have received little attention. The paper contributes three annotated datasets and evaluates YOLO for fish detection and cross-site use.
I. INTRODUCTION
- Because human analysis of underwater video takes time, some automation is needed to process large video datasets efficiently and provide information for permitting and adaptive management decisions.Source: §I · p. 1
- Prior work largely focused on coral fish in favorable conditions, with few studies addressing the turbid, high-velocity environments around MHK energy.The authors note that deep-learning applications to MHK and hydropower projects had not previously been reported to their knowledge.Source: §I · pp. 1–2
- The paper contributes manually annotated underwater fish videos from three MHK and hydropower sources.It also applies YOLO to challenging underwater videos and evaluates whether a trained model can detect fish at another site.Source: §I · p. 2
II. MHK AND HYDROPOWER UNDERWATER FISH VIDEO DATASETS
The paper assembles annotated underwater fish video from three water-power sites. Such varied data are needed to develop and evaluate robust algorithms, but MHK-specific data are scarce and expensive to produce.
II. MHK AND HYDROPOWER UNDERWATER FISH VIDEO DATASETS
- Automated filtering is needed because expert review is labor-intensive and partial sampling is unsuitable for long-term monitoring or rare events.Optical video can record animal occurrence, shape, color, position, and movement for environmental monitoring.Source: §II · p. 2
- Robust algorithm development and evaluation require animal-present and empty footage from varied locations, cameras, and species, with some frames annotated using fish bounding boxes.MHK-specific video data are not readily available and are expensive to produce.Source: §II · p. 2
- The three datasets come from distinct sites: a tidal-turbine test in Scotland, fish-passage monitoring at Wells Dam, and an instream-turbine pilot in Alaska.Selected video from each dataset was annotated using a purpose-built script.Source: §II · p. 3
A. YOLO
YOLO treats object detection as regression, directly predicting bounding boxes and class-confidence scores. The paper implements YOLO v3 for fish detection in underwater video.
A. YOLO
- YOLO directly outputs bounding-box coordinates and class-confidence scores, unlike region-proposal methods that classify proposed image regions.The paper describes YOLO as a more straightforward and faster object-detection approach than region-proposal methods.Source: A. YOLO · p. 3
- The authors adapted a TensorFlow and Keras YOLO v3 implementation to train on underwater fish-video frames and report detection statistics.The modifications include visualizing ground-truth and predicted boxes and outputting false positives, true positives, precision, and recall in CSV format.Source: A. YOLO · p. 3
B. Transfer Learning
The paper initializes YOLO v3 with pretrained weights and fine-tunes it for underwater fish detection. Training used fish-containing frames and stopped early when overfitting began.
B. Transfer Learning
- Transfer learning initializes a model for a new task with parameters learned on another task; the authors used YOLO v3 weights that detect many classes but not fish.YOLO v3 has 106 convolutional layers.Source: B. Transfer Learning · p. 3
A. Experiment I: Train and Evaluate on All Datasets
YOLO was trained on a combined training set from three datasets and evaluated on their test sets. Performance varied across datasets, with low visibility and difficult-to-label fish among the reported challenges.
A. Experiment I: Train and Evaluate on All Datasets
- Training combined 54,516 annotated frames from all three sources; roughly 80% of each dataset was assigned to training and 20% to testing.The model was stopped at 20 epochs after beginning to overfit, with 10% of the test set used for validation.Source: A. Experiment I: Train and Evaluate on… · pp. 3–4
- Wells Dam reached 0.5575 mAP, Voith Hydro 0.5474 mAP, and Igiugig 0.4507 mAP.The results are reported for individual-source test sets.Source: A. Experiment I: Train and Evaluate on… · p. 4, Table II · p. 5
- Wells Dam’s high resolution and full color did not prevent reduced mAP from partial fish at the ladder-window edge, which were difficult for annotators to label.These hard cases could outnumber fish passing the white plate.Source: A. Experiment I: Train and Evaluate on… · p. 4
- Voith Hydro’s low-resolution grayscale footage made single-frame detection difficult, though the model detected most fish in a school passing the devices.The paper notes that fish features were often indistinct and suggests multiple-frame training could improve accuracy.Source: A. Experiment I: Train and Evaluate on… · p. 4
- Igiugig’s fast flow, bubbles, debris, and sparse fish frames constrained training; a shifted device angle also produced false detections throughout a test video.Only 3% of frames contained fish, and about 1,000 frames entered training; the authors note that fish-free frames may help generalization.Source: A. Experiment I: Train and Evaluate on… · pp. 4–5
- Examples include blurry small fish, overlapping fish, partially visible bodies, and a flashlight-reflecting salmon missed in turbulent water.The figure distinguishes human annotations in green from model predictions in red.Source: Figure 2 · p. 4
B. Experiment II: Reserve Igiugig Dataset to Test
Experiment II tested whether a YOLO model trained on Wells Dam and Voith Hydro data could generalize to unseen Igiugig data. It failed on that held-out set; Igiugig-only retraining improved its score but did not match Experiment I.
B. Experiment II: Reserve Igiugig Dataset to Test
- 0.0055 mAP on Igiugig showed that training on Wells Dam and Voith Hydro did not generalize to the unseen dataset.The model was initialized with YOLOv3 weights.Source: B. Experiment II: Reserve Igiugig… · p. 5
- 0.3856 mAP on Igiugig after 20 epochs of Igiug-only training remained below Experiment I’s 0.4507.Fig. 4 shows the before-and-after precision-recall curves; before Igiug training, the model detected 30% of fish with low precision.Source: B. Experiment II: Reserve Igiugig… · p. 5, Figure 4 · p. 5
- Igiugig’s camera angles and environmental settings varied, and many test frames differed from training frames; the authors link this variation to weaker testing after overfitting.They argue that more diverse underwater-video training data is important for more generalized models.Source: B. Experiment II: Reserve Igiugig… · p. 5
V. CONCLUSION AND FUTURE WORK
The paper contributes manually annotated fish videos from three MHK energy projects and evaluates YOLO on them. Results underscore dataset and video-quality challenges, while the authors identify several approaches for future improvement.
V. CONCLUSION AND FUTURE WORK
- The study contributes frame-wise, manually annotated underwater fish videos from three MHK energy projects and evaluates YOLO on these datasets.The authors report that detection accuracy is often limited by video quality.Source: §V · p. 5
- The authors identify high-quality cameras as a need for future underwater energy projects to assist ecological assessment.Source: §V · pp. 5–6
- Differences in background, fish angle and size, resolution, and frame rate coincided with failure to recognize fish in a completely new dataset.The model also performed better on two datasets when it was not trained on the third.Source: §V · p. 6
- Future approaches include image augmentation, ensembles of detection models, and using fish motion extracted from multiple frames.The authors suggest these methods may improve generalization, accuracy, or detection of vague fish.Source: §V · p. 6