PaperlayerView exact revision on arXiv

Source-linked AI summary

Applications of Artificial Neural Networks in Microorganism Image Analysis: A Comprehensive Review from Conventional Multilayer Perceptron to Popular Convolutional Neural Network and Potential Visual Transformer

Jinghua Zhang, Chen Li, Yimin Yin, Jiawei Zhang, Marcin Grzegorzek

arXiv:2108.00358v3cs.CVcs.AI

TL;DR

Conventional microorganism analysis is laborious and subjective, while image-based analysis must handle difficult visual characteristics and varied tasks. This review synthesizes classical and deep ANN methods across microorganism-image applications and discusses their methodological patterns and future directions. It identifies feature engineering and dataset availability as distinguishing issues between classical and deep approaches, while highlighting ViT as a potential classification method and persistent limitations in augmentation and limited-data settings.

  • Problem

    Conventional microorganism analysis is laborious and time-consuming, while automatic image analysis must address varied applications, insignificant features, under-segmentation, and diverse tasks.

  • Method

    The paper comprehensively reviews classical and deep ANN applications in microorganism image analysis across multiple tasks and discusses representative networks, methodology, and potential directions.

  • Results

    Deep MIA mainly uses CNNs that process images directly and automatically extract effective features, whereas classical MIA usually relies on preprocessing, feature extraction, and classification.

  • Takeaways & Limitations

    The review identifies Vision Transformer as a potential microorganism-image classification method and emphasizes developing open-access datasets and practical ANN-based MIA algorithms.

  • Takeaways & Limitations

    Current few-shot and domain-adaptation methods may alleviate insufficient-data limitations but still face data-acquisition challenges and a performance gap versus conventional supervised deep learning.

Abstract

from arXiv · show

Microorganisms are widely distributed in the human daily living environment. They play an essential role in environmental pollution control, disease prevention and treatment, and food and drug production. The analysis of microorganisms is essential for making full use of different microorganisms. The conventional analysis methods are laborious and time-consuming. Therefore, the automatic image analysis based on artificial neural networks is introduced to optimize it. However, the automatic microorganism image analysis faces many challenges, such as the requirement of a robust algorithm caused by various application occasions, insignificant features and easy under-segmentation caused by the image characteristic, and various analysis tasks. Therefore, we conduct this review to comprehensively discuss the characteristics of microorganism image analysis based on artificial neural networks. In this review, the background and motivation are introduced first. Then, the development of artificial neural networks and representative networks are presented. After that, the papers related to microorganism image analysis based on classical and deep neural networks are reviewed from the perspectives of different tasks. In the end, the methodology analysis and potential direction are discussed.

1 Introduction

Microorganism analysis supports environmental, medical, food, and industrial applications, but conventional methods—especially manual morphological analysis—are laborious, time-consuming, and subjective. This review examines how classical and deep artificial neural networks address microorganism image-analysis tasks amid noisy images, weak features, segmentation difficulties, and varied application requirements.

  • 1 Introduction: Microorganisms can benefit environmental management, disease prevention and treatment, and food and drug production, while others cause disease.Examples include nutrient provision, sludge digestion, nitrogen fixation, tuberculosis, and other harmful microorganisms.
  • 1 Introduction: Traditional microorganism analysis comprises chemical, physical, molecular biological, and morphological methods with different accuracy, cost, equipment, time, and expertise requirements.Manual morphological analysis is direct and cost-effective but remains laborious, tedious, time-consuming, and dependent on the biologist’s experience, workload, and mood.
  • 1 Introduction: Microorganism images challenge automatic analysis through noisy backgrounds, filopodia-related under-segmentation, illumination-dependent color changes, and inconspicuous texture.These characteristics motivate flexible and robust algorithms for broad microorganism-image application occasions.
  • 1 Introduction: AI can automate laborious image work, extract information rapidly, reduce subjective differences in microorganism analysis, and potentially improve efficiency.Machine learning has been applied to environmental microorganism segmentation, Herpesvirus detection, and Tuberculosis Bacilli classification.
  • 1 Introduction: Classical MIA workflows typically require preprocessing, segmentation, feature extraction, classifier design, and evaluation, whereas CNN-based deep networks can learn representations directly from images.Classical methods and neural networks therefore depend heavily on feature engineering, while deep CNNs reduce that dependence.
  • 1 Introduction: The review surveys ANN development and microorganism-image applications across classification, segmentation, detection, counting, feature extraction, enhancement, and augmentation.It extends prior reviews by comprehensively analyzing ANN history, representative methods, summary statistics, and potential directions across 96 papers from 1992 to 2020.

2 Artificial Neural Networks

ANNs evolved from early perceptrons and classical multilayer networks into deep CNN architectures that support major microorganism image-analysis tasks. This progression addressed training and feature-extraction limitations while producing specialized networks for classification, segmentation, and detection.

  • 2.1 Evolution of Artificial Neural Networks: ANN development passed through three stages, beginning with the 1943 M-P neuron and perceptron, followed by renewed interest after Hopfield networks and BP, and a later deep-learning rise.The perceptron’s inability to solve XOR contributed to an early research trough; BP enabled multilayer training, while improved computing supported the later resurgence.
  • 2.2 Representative Artificial Neural Networks: Early MIA systems commonly used a feature-engineering-plus-classifier workflow, whereas CNN-based systems learned representations directly from images.Classical systems extracted features using prior experience or domain knowledge before classifier training; CNNs reduced reliance on manual feature engineering.
  • 2.2 Representative Artificial Neural Networks: In microorganism image analysis, popular CNN families align with task types: AlexNet, VGGNet, ResNet, and Inception for classification; U-Net for segmentation; and YOLO for detection.The review presents these networks as representative architectures used across microorganism image-analysis tasks.
  • 2.2.1 Multilayer Perception: MLP typically contains input, hidden, and output layers, and BP enables it to adjust connection weights layer by layer.Before BP, early MLP training used random input-to-hidden weights and error correction only between hidden and output layers.
  • 2.2 Representative Artificial Neural Networks: AlexNet, VGGNet, GoogLeNet, and ResNet progressively increased CNN depth while introducing ReLU, stacked 3 × 3 filters, Inception, and identity mapping.AlexNet won ILSVRC 2012; VGGNet placed second and GoogLeNet won ILSVRC 2014. ResNet used identity mapping to address gradient vanishing or explosion during deeper training.
  • 2.2 Representative Artificial Neural Networks: U-Net uses a symmetrical contracting and expansive architecture for microscopic image segmentation, while YOLO divides an image into an S × S grid for object detection.U-Net emphasizes data augmentation and end-to-end recovery of shallow information; YOLO predicts bounding boxes and confidence scores for objects assigned to grid cells.

3 Classical Neural Network for Microorganism Image Analysis

The review organizes classical-neural-network research in microorganism image analysis by task. Classification dominates this literature and is analyzed according to the networks used, while other tasks are discussed independently because fewer papers address them.

  • 3 Classical Neural Network for Microorganism Image Analysis: Classical-neural-network studies are grouped by microorganism image-analysis task, with classification works analyzed according to their employed networks.Tasks with limited numbers of papers are reviewed independently rather than through the same network-based organization.

3.1 Classification Task

Classical neural-network classification studies commonly use preprocessing and engineered features before MLP, while RBF often performs better in comparisons. The reviewed approaches span varied preprocessing, architectures, feature-selection strategies, and microorganism categories.

  • MLP-based classification: Around 30 papers use MLP-based methods for microorganism image classification, with variation in preprocessing, feature extraction, and network structure.
  • MLP-based classification: Most MLP preprocessing methods segment microorganism objects, while others reduce noise and highlight objects; some studies omit preprocessing or do not report it.
  • MLP-based classification: Shape is the most frequently used feature type in MLP classification, but directly feeding extracted features to classifiers is described as inefficient for discovering effective features.
  • MLP-based classification: Most MLP classification studies use a three-layer BP structure, while deeper networks are avoided because more parameters may cause overfitting with insufficient data.
  • RBF-based classification: More specific feature screening is used in some RBF studies, and the review states that RBF outperforms MLP in most comparisons, with exceptions when MLP configurations are more extensively optimized.
  • RBF-based classification: RBF outperforms alternative classifiers in several bacterial classification studies, while some spiral-bacterial tasks achieve 100% accuracy with RBF, 3σ, fuzzy, or neuro-fuzzy classifiers.
  • Other classical ANNs: In one algae-identification system, segmentation, shape extraction, pigment-signature determination, and SOM classification achieve 98.6% accuracy across 23 categories and 53869 images.

3.2 Segmentation Task

The reviewed segmentation examples apply neural networks within computer-aided microorganism or disease-image workflows, including SOM-based monitoring, HMLP-based tissue-slide segmentation, and ANN-assisted tuberculosis diagnosis.

  • SOM-based segmentation and monitoring: A prototype vision system monitors morphological changes of Aureobasidium pullulans during fermentation through preprocessing, segmentation, intermediate processing, feature extraction, and classification, using a SOM network.
  • HMLP-based segmentation: An HMLP method segments Ziehl-Neelsen tissue-slide images using hue and saturation components from each pixel in a 3 × 3 kernel.
  • ANN-assisted diagnosis: A tuberculosis-assistance workflow combines image acquisition, segmentation, and post-processing, using SVM and a three-layer feed-forward neural network with pixel color features.

3.3 Counting Task

A feature-engineered MLP can enhance microscope images and support microorganism enumeration by producing an output for every original-image pixel. Performance is strong on high-contrast images and remains quantified for lower-quality substrates.

  • MLP-based counting: 93%-98% enumeration accuracy is reported for adhering bacteria on metal or silicone rubber substrata in lower-quality images, while 98% of yeast cells on silicone rubber are counted correctly after ANN enhancement.

3.4 Feature Extraction Task

The reviewed feature-extraction approach uses a biologically inspired PCNN to generate entropy-sequence features before classification of bacteria images.

  • PCNN-based feature extraction: A PCNN-based bacteria classification workflow crops images to regions of interest, generates entropy-sequence features, and then performs classification.

3.5 Summary

Classical neural-network studies in microorganism image analysis are constrained by limited public data, reliance on feature engineering, and shallow architectures. Classification dominates these applications, with results summarized across tasks and methods.

  • Private datasets limit development of general microorganism image-analysis algorithms and make comparative evaluation difficult.
  • Classical neural-network workflows usually require preprocessing, feature extraction, and classification, making feature engineering important to performance.
  • Shallow architectures are commonly used because deeper networks introduce many connections, parameters, and training difficulties.
  • Classification is the predominant task among microorganism image-analysis studies using classical neural networks.
  • Table 3 summarizes years, tasks, references, datasets, species, category counts, methods, and results for classical-neural-network studies.

4 Deep Neural Network for Microorganism Image Analysis

Deep neural networks support microorganism image-analysis tasks including classification, segmentation, feature extraction, detection, counting, and augmentation, with classification receiving the most extensive coverage. CNNs reduce reliance on feature engineering, while studies use architectures ranging from shallow CNNs to transfer learning and complex hybrid networks.

  • Deep-neural-network microorganism image analysis covers classification, segmentation, feature extraction, detection, counting, and data augmentation.
  • CNN-based workflows usually require little or no preprocessing and feature extraction because convolutional filters learn image representations.
  • Approximately 96% of coccoliths from 11 Pleistocene taxa were recognized during routine work by a simple CNN system.
  • 97.14% patch-level segmentation accuracy and 62.10% classification accuracy were achieved in a bacteria-analysis framework combining CDBN, SVM, and CNN.
  • U-Net and 1D CNN in hyperspectral bacteria analysis achieved average accuracy of 96%, average mIOU of 88%, and classification accuracy of 90%.
  • TB classification using CNNs with HOG and SURF features achieved 99.5% accuracy, 94.7% sensitivity, and 99% specificity.
  • Recent CNN studies use deeper and more complex structures, transfer learning for imbalanced data, and sometimes feature engineering for further performance improvement.
  • Xception-based bacteria classification achieved 97.5% accuracy after ImageNet pretraining and fine-tuning on experimental data.

4.2 Segmentation Task

Segmentation is a major deep-neural-network task in microorganism image analysis, with U-Net widely used and other architectures applied to viruses, yeast cells, and tuberculosis bacilli. Studies emphasize annotation efficiency, multimodal microscopy, and segmentation for downstream counting.

  • U-Net is the most popular deep-neural-network architecture for microorganism image segmentation.
  • A minimally annotated U-Net achieved better performance than U-Net while requiring less than one-third of its memory.
  • Pixel-level segmentation with transfer-learning-based VGG-16 patch classification achieved Dice 87.13%, Jaccard 79.74%, recall 87.12%, accuracy 96.91%, and VOE 20.26%.
  • SegNet segmentation of multimodal fluorescent yeast-cell microscopy images achieved a mIOU of 71.72%.
  • A CNN and mosaic-image approach generated 5000 images from positive and negative patches to segment tuberculosis bacilli for counting.

4.3 Detection Task

Deep neural networks are used for microorganism detection through staged detection-classification systems and integrated detection-counting frameworks. Applications include malaria parasites, cyanobacteria, and fungi, with reported performance varying by task and species.

  • A two-stage malaria-parasite system achieved 59% accuracy for detection and 98% accuracy for second-stage target classification.
  • Fast RCNN detected and classified cyanobacteria before CNN-based counting, forming the workflow shown in Fig. 17.
  • The cyanobacteria framework classified five species with accuracies of 0.929, 0.973, 0.829, 0.890, and 0.890, while CNN counting achieved R2 of 0.85.
  • A CNN-based fungus dataset contained 40800 images across five fungus classes and one dirt class, with detection accuracy of 94.8%.

4.4 Feature Extraction Task

Deep neural networks are used for microorganism image feature extraction, often combining pretrained or CNN-derived features with conventional classifiers. Reviewed approaches include hybrid CNN methods, transfer learning, and feature extraction across multiple datasets.

  • Deep neural networks extract microorganism image features for subsequent classification and related analysis tasks.
  • A hybrid plankton algorithm trains a three-hidden-layer CNN, then combines hidden-layer features with RF or SVM classifiers.
  • 84% classification accuracy is achieved by DeepSea pretrained on ISIIS and used with SVM on a small private dataset.
  • Around 96% accuracy is reported for bacteria classification using fine-tuned Inception-V3 features with SVM.
  • 99.5% accuracy is obtained by CNN classification with an Inception-V3 feature extractor for a five-category plankton dataset.
  • Densenet201+Polar+SVM achieves 97.989% accuracy and 97.986% recall on in situ plankton data, and 94.91% accuracy and 94.76% recall on CIFAR-10.

4.5 Data Augmentation Task

Data augmentation addresses limited microorganism image datasets, including through an enhanced GAN framework. Directional variation remains a challenge for generated microorganism images.

  • Environmental microorganism datasets often contain limited data, motivating GAN-based image augmentation.
  • The enhanced GAN framework uses segmentation ground-truth images to generate microorganism images with more uniform directions.
  • Deep neural network MIA includes data augmentation alongside classification, segmentation, detection, counting, and feature extraction.
  • The reviewed deep-neural-network works are summarized by task, reference, dataset, species, category number, method, and results in Table 4.

5 Methodology Analysis and Potential Direction

The review compares classical and deep neural network workflows, then identifies dataset limitations and potential directions involving method fusion, transfer learning, data augmentation, few-shot learning, domain adaptation, and transformers.

  • Classical Neural Networks: Classical neural-network MIA works are analyzed across classification, segmentation, counting, and feature extraction tasks, with emphasis on preprocessing, feature extraction, and network design.
  • Classical Neural Networks: MLP and RBF are the most widely used classical neural networks in the reviewed MIA studies.
  • Classical Neural Networks: Classical workflows typically rely on engineered features, making performance dependent on feature extraction and selection and limiting direct transfer across tasks.
  • Deep Neural Networks: Deep neural networks support classification, segmentation, detection, counting, feature extraction, and data augmentation, primarily using CNNs that extract features directly from images.
  • Deep Neural Networks: CNNs are the most widely used deep networks; among public networks, VGGNet, Inception, U-Net, AlexNet, and ResNet are the five most widely used.
  • Potential Directions: Transfer learning is used for limited training data and high-level feature extraction, with more than 15 reviewed papers involving the technology.
  • Potential Directions: GAN augmentation can be limited by object direction and by the availability of segmentation ground-truth images, while few-shot learning and domain adaptation still show a performance gap.

6 Conclusion and Future Work

The review synthesizes 95 papers on classical and deep neural networks for microorganism image analysis and discusses methodology and future directions. It omits a dataset summary because many studies use private data, while noting newly released open-access datasets.

  • The review collects and analyzes 95 papers, covering ANN development, representative networks, and microorganism image-analysis tasks.
  • A dataset summary is omitted because most related studies use private datasets, leaving no unified standard dataset for evaluating different MIA methods.
  • The authors report open-access datasets including EMDS-6, EMDS-7, and a SARS-CoV-2 microscopic image dataset, and identify further dataset and algorithm development as significant future work.
Loading 2108.00358v3…