PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep learning for smart fish farming: applications, opportunities and challenges

Xinting Yang, Song Zhang, Jintao Liu, Qinfeng Gao, Shuanglin Dong, Chao Zhou

arXiv:2004.11848v2cs.CVcs.LGeess.IVstat.ML

TL;DR

Smart fish farming must address high-dimensional, nonlinear, and massive data, while tracking fish in schools remains challenging. This paper comprehensively reviews deep learning applications in smart fish farming and reports improved identification accuracy over stated comparison methods, alongside performance comparable to human experts for fish-size estimation.

  • Problem

    Smart fish farming faces challenges from high-dimensional, nonlinear, and massive data, while tracking individuals in fish schools is difficult because of complex nonrigid motion.

  • Method

    The paper provides a comprehensive review of deep learning and its applications in smart fish farming, outlining aquaculture applications and their latest advances.

  • Results

    CNN identification accuracy improved by 15% and 10% compared with SVM and Softmax, while R-CNN improved accuracy by at least 16% over a Gaussian method.

  • Takeaways & Limitations

    The review organizes deep learning applications in aquaculture to support understanding of current advances in smart fish farming.

  • Takeaways & Limitations

    Annotating sufficiently large numbers of images is time-consuming and laborious, and tracking individuals in fish schools remains challenging.

Abstract

from arXiv · show

With the rapid emergence of deep learning (DL) technology, it has been successfully used in various fields including aquaculture. This change can create new opportunities and a series of challenges for information and data processing in smart fish farming. This paper focuses on the applications of DL in aquaculture, including live fish identification, species classification, behavioral analysis, feeding decision-making, size or biomass estimation, water quality prediction. In addition, the technical details of DL methods applied to smart fish farming are also analyzed, including data, algorithms, computing power, and performance. The results of this review show that the most significant contribution of DL is the ability to automatically extract features. However, challenges still exist; DL is still in an era of weak artificial intelligence. A large number of labeled data are needed for training, which has become a bottleneck restricting further DL applications in aquaculture. Nevertheless, DL still offers breakthroughs in the handling of complex data in aquaculture. In brief, our purpose is to provide researchers and practitioners with a better understanding of the current state of the art of DL in aquaculture, which can provide strong support for the implementation of smart fish farming.

1. Introduction

Smart fish farming integrates modern information technologies to collect and analyze complex data for quantitative, real-time decision-making. This review examines how deep learning supports aquaculture applications while identifying data and intelligence challenges.

  • Smart fish farming integrates IoT, big data, cloud computing, artificial intelligence, and related technologies to promote efficient resource use and sustainable aquaculture.
  • Real-time data collection and quantitative decision-making are central functions of smart fish farming.
  • Aquaculture data are high-dimensional, nonlinear, massive, multisource, heterogeneous, and available in text, image, and audio formats.
  • Traditional machine learning relies heavily on manually designed features, making suitable feature selection difficult for specific tasks.
  • Deep learning automatically extracts complex features and provides analytical tools for revealing, quantifying, and understanding big data.
  • The paper comprehensively reviews deep learning applications, technical details, challenges, and future trends in smart fish farming.

2. Concepts of deep learning

Deep learning is a machine-learning approach that learns hierarchical representations through multiple neural-network layers. Its end-to-end structure supports complex nonlinear modeling and large-scale data analysis, but practical use remains vulnerable to overfitting and adversarial perturbations.

  • Terms and definitions of deep learning: Deep learning is a branch of machine learning and a representation-learning algorithm based on artificial neural networks.
  • Terms and definitions of deep learning: Deep learning decomposes complex mappings into nested simpler mappings that extract increasingly abstract features from input data.
  • Learning tasks and models: Convolutional neural networks excel at image processing, whereas recurrent neural networks process sequential data effectively.
  • Learning tasks and models: Generative adversarial networks use mutual learning between generative and discriminative models to produce outputs.
  • Learning tasks and models: In deep learning, feature learning and model construction are integrated through end-to-end optimization rather than performed as separate modules.
  • Learning tasks and models: Hierarchical architectures facilitate nonlinear modeling and large-scale data analysis, including classification and prediction.
  • Deep-learning systems can hallucinate, overfit, or produce substantially different outputs after imperceptible input perturbations.

3. Applications of deep learning in smart fish farming

The review organizes deep-learning applications in smart fish farming into six categories, with live fish identification and species classification receiving the most attention. Across these applications, learned visual features support recognition, monitoring, estimation, and management, while data quality and complex underwater conditions remain important constraints.

  • Applications overview: The review covers 41 papers across six applications: live fish identification, species classification, behavioral analysis, feeding decisions, size or biomass estimation, and water quality prediction.Most papers were published from 2016 onward, including 15 in 2019.
  • Applications overview: Deep learning extracts multilevel features that combine low-level image information with higher-level semantic representations for recognizing and detecting aquaculture targets.In CNNs, earlier layers learn low-level features while later layers learn high-level features.
  • Live fish identification: CNN-based live fish identification improved accuracy by 15% over SVM and 10% over Softmax on a public real-image dataset.R-CNN also improved accuracy by at least 16% over a Gaussian mixture model on the FCS dataset.
  • Species classification: Species-classification models achieved accuracies above 90% on the LifeCLEF 14 and LifeCLEF 15 benchmark fish datasets and can provide alternatives to manual classification.DL models can learn species characteristics that are less sensitive to environmental changes and variations.
  • Size or biomass estimation: Machine vision combined with deep learning can support fish counting and more accurate size or biomass estimation in smart fish farming.These estimates can provide references for feeding or harvest decisions.

N MSE，MCV Mean CV: 8.89%:

Deep learning is applied to feeding decisions and water-quality prediction because aquaculture outcomes depend on complex biological and environmental factors. Image-based feeding systems and spatiotemporal water-quality models show practical promise, but broader inputs, longer horizons, and greater robustness remain needed.

  • Feeding decisions: Unreasonable, insufficient, or excessive feeding can reduce production efficiency, impair fish growth, lower feed-conversion efficiency, and pollute the environment.Optimizing feeding is therefore associated with potential economic benefits.
  • Feeding decisions: Machine vision can assess feeding behavior and terminate feeding at more appropriate times, reducing unnecessary labor and improving fish welfare.Feeding intensity can also be graded to guide production practices.
  • Feeding decisions: A dual-flow model combining a 3D-CNN and an RNN captured spatial and temporal information and recognized feeding and nonfeeding behaviors better than either individual model.The 3D-CNN captured spatial information, whereas the RNN captured temporal sequence information.
  • Feeding decisions: Image-only feeding analysis is insufficient because many factors affect fish feeding; environmental measurements and physiological data should also be incorporated.The review identifies physiological, nutritional, environmental, and husbandry factors as relevant influences.
  • Water quality prediction: CNN- and DBN-based models have predicted intensive-culture water-quality parameters with accuracy and stability sufficient to meet actual production needs.Dissolved oxygen prediction is important because control measures have delayed effects.
  • Water quality prediction: Current water-quality methods generally perform well only for short-term predictions, while interacting parameters and uncertainty limit long-term modeling and generalizability.Reducing uncertainty effects is identified as an important development direction.
  • Water quality prediction: Attention-based RNN models represented spatiotemporal relationships effectively and outperformed other methods for both short- and long-term dissolved-oxygen predictions.Long-term prediction requires extracting relationships between water quality and external factors.

4. Technical details and overall performance

The reviewed aquaculture studies rely mainly on image inputs and CNN-based methods, with transfer learning used in 9 of 41 papers. Reported results generally favor DL models over traditional approaches, while dataset construction and model boundaries remain concerns.

  • Data: Annotated datasets are critical for DL performance, but dataset construction is often difficult because of quantity and quality issues.Preprocessing and augmentation commonly adjust image size, highlight regions of interest, subtract backgrounds, denoise images, or generate additional labeled samples.
  • Data: Data augmentation generates labeled samples through operations such as rotation, translation, and transposition, helping improve learning when image datasets are small.Pretrained models are also used to reduce the need for large volumes of annotated data, although negative transfer is a limitation.
  • Models and frameworks: CNNs remain the most popular models, while the reviewed studies also use GANs, RNNs, LSTMs, DBNs, YOLO, SVM, and Softmax classifiers.Transfer learning appears in 9/41 reviewed papers and adapts pretrained models from related tasks or datasets to specific aquaculture problems.
  • Model inputs: Images are the dominant model input, appearing in 34 studies (83%), although some studies use fish audio or water-quality data.Inputs include public datasets such as ImageNet, Fish4Knowledge, and Croatian and QUT fish datasets, as well as field-collected or Internet-sourced data.
  • Model outputs: The reviewed outputs cover live-fish recognition, size or biomass estimation, feeding intensity, water-quality prediction, and behavioral analysis, but task boundaries are often vague.The studies include 13 live-fish recognition tasks, 7 size or biomass estimations, 2 feeding-intensity quantifications, 6 water-quality predictions, and 5 behavior analyses.

5. Discussion

DL’s central advantage in smart fish farming is automatic feature extraction, which supports robust recognition and prediction across challenging conditions and data types. Its adoption remains constrained by data, computing, interpretability, cost, and model-scope limitations.

  • Advantages of deep learning: DL automatically learns essential features, improving recognition stability for irregular targets in complex environments without relying on manually selected features.The reviewed discussion highlights robustness under low light and high noise and handling of occlusion in behavioral analysis.
  • Advantages of deep learning: After training, DL models can perform testing efficiently, including approximately 1 ms for classifying each fish image in one study.Training may require substantially more time and computing power than SVM or random forest methods.
  • Disadvantages and limitations of deep learning: DL remains weak AI and is used as a black box in smart fish farming, with low interpretability and limited ability to learn beyond specific datasets.Unbalanced training data can cause models to ignore important features.
  • Disadvantages and limitations of deep learning: Large, diverse, accurately annotated datasets remain a major bottleneck because many fish types and growth stages require extensive data collection and training.Expert annotation can be time-consuming and error-prone, especially for challenging fish-species identification tasks.
  • Disadvantages and limitations of deep learning: Smart fish farming often lacks representative public datasets, forcing researchers to build custom image sets that may take hours or days of work.Data augmentation helps, but complex high-precision multiclass problems require more diversified training data.
  • Disadvantages and limitations of deep learning: DL deployment has substantial hardware and financial costs because sensor collection is expensive and common CPUs may not meet model requirements.GPUs and TPUs are described as mainstream sources of suitable computing power, while high costs can hinder farmer investment without expected results.
  • Future directions: Future work is expected to expand DL applications, improve learning from small samples, and use spatiotemporal models such as LSTMs or other RNNs.Potential applications include disease diagnosis, quality safety control, traceability, growth estimation, and water-demand assessment.

6. Conclusion

This review examines deep-learning applications in smart fish farming across six application categories and analyzes their data, algorithms, and performance. It identifies automatic feature extraction as DL’s greatest contribution, while emphasizing labeled-data requirements as a major bottleneck.

  • The review covers six DL application categories: live fish identification, species classification, behavioral analysis, feeding decisions, size or biomass estimation, and water quality prediction.
  • Reported methods are analyzed through their data, algorithms, and performance, including comparisons with traditional manually extracted features.
  • Automatic feature extraction is identified as DL’s greatest contribution, alongside high-precision processing results.
  • DL remains in a weak-AI stage and requires large amounts of labeled training data, creating a bottleneck for further smart-farming applications.
  • Despite these challenges, DL offers breakthroughs in processing text, images, video, sound, and other aquaculture data.
  • The review aims to improve researchers’ and practitioners’ understanding of DL applications and facilitate its use for practical aquaculture problems.

Appendix A: Public dataset containing fish

The appendix catalogs public fish datasets spanning underwater images, videos, species labels, annotations, and the broader ImageNet image database. These resources differ substantially in scale, modality, and species coverage.

  • The underwater live-fish dataset contains 27,370 verified fish images organized into 23 clusters.
  • The LCF-14 fish dataset includes approximately 1,000 videos, labels for approximately 20,000 detected fish, and 10 fish species.
  • The LCF-15 dataset contains 93 underwater videos covering 15 species and provides 9,000 annotations.
  • ImageNet is organized according to the WordNet hierarchy, with each node associated with hundreds or thousands of images and an average of over 500 images per node.
Loading 2004.11848v2…