Effects of model architecture and learning strategies on deep learning-based recognition of activated sludge microscopic images and comparison with quantitative image analysis

• QIA typically binarized images and characterized flocs with morphological parameters, but its processing was laborious and operator-dependent.Source: §1 · p. 2
• Prior work had not systematically evaluated transformer-based models or self-supervised pretraining, and had rarely tested downsampling strategies or compared deep learning with QIA experimentally.Source: §1 · p. 2
• The study classified microscopic images from three activated-sludge sample types using deep learning and QIA to examine training conditions and compare analytical performance.Source: §1 · pp. 2–3
• Deep learning outperformed QIA in classification accuracy and required fewer images, while transformer architectures and alternative pretraining methods were effective.The study also found that overly small images reduced performance and maintaining field of view was more effective than maintaining resolution.Source: §1 · p. 3

• Samples came from three process lines across two wastewater treatment plants in Japan.Source: §2.1 · p. 3
• Collected samples were stored under refrigerated conditions until microscopic image acquisition.Source: §2.1 · p. 3
• The sample overview is presented in Table 1.Source: §2.1 · p. 3, Table 1 · p. 3

• Microscopic images were classified by sample, with classification accuracy used to assess performance.Source: §2.2 · p. 3
• The benchmark first evaluates model size, architecture, pretraining, image size, and resizing strategy for deep learning.Source: §2.2 · p. 3
• The benchmark then compares deep learning accuracy with QIA accuracy.Source: §2.2 · p. 3

2.3.1 Image acquisition

• Images were acquired with a flow-cell microscope and automatic stage, generally across 25 fields of view per sample.The sample settled for approximately one minute before imaging.Source: §2.3.1 · p. 4, Figure 1 · p. 4

2.3.2 Software and workstation

• The implementation used Python with PyTorch and related packages for deep learning, while image processing and morphology calculations used MATLAB and other analyses used R.Source: §2.3.2 · p. 4

2.3.3 Models

• Eight pretrained models included ConvNeXt CNNs and ViT transformers, with variants spanning model sizes and pretraining strategies.The study retrained pretrained models rather than training from scratch.Source: §2.3.3 · pp. 4–5, Table 2 · p. 5

2.3.4 Training

• The eight models were trained under multiple conditions, producing 48 models with different input sizes and layer-unfreezing strategies.Training used 201–207 images per sample, with 100–107 images per sample for evaluation.Source: §2.3.4 · pp. 5–6

2.3.5 Evaluation

• Evaluation compared field-of-view preservation, by resizing a central 1216 × 1216 pixel region, with resolution preservation, by cropping directly to the model input size.Other resizing-strategy analyses used field-of-view preservation.Source: §2.3.5 · p. 6

2.4.1 Image acquisition

• QIA images were captured at 10× magnification, with 50 images taken from each slide.A 50 µL sample aliquot was placed under a 24 mm × 24 mm coverslip so all flocs could be in focus.Source: §2.4.1 · p. 6

2.4.2 Software

• MATLAB 2024b was used for image processing and morphological calculations; other analyses used R 4.4.2 and tidymodels, bonsai, and DALEX.Source: §2.4.2 · p. 6

2.4.3 Image binarization and calculation of morphological parameters

• The binarization procedure generated and subtracted a background image, then processed floc boundaries and cores separately before combining the binary images.Boundary regions used the difference between eroded and dilated images; core regions used CLAHE enhancement before thresholding.Source: §2.4.3 · pp. 6–7, Figure 2 · p. 7
• After binarization, each floc was characterized by area, perimeter, form factor, convexity, solidity, aspect ratio, compactness, roundness, eccentricity, and extent.The parameters were computed with MATLAB’s regionprops function and defined according to prior studies.Source: §2.4.3 · p. 7

2.4.4 Floc classification based on morphological parameters using machine learning

• LightGBM classified samples from randomly selected flocs’ morphological measurements, with 10–100 flocs per instance and three independent runs per floc count.Approximately 200 images per sample were used for training and 100 for evaluation; each floc count had 2,000 training and 50 evaluation instances per sample.Source: §2.4.4 · pp. 7–8

2.5 Frequency analysis of images

• Two microscopic images per sample were randomly selected from the deep learning training dataset for frequency analysis.Source: §2.5 · p. 8
• Power spectra were calculated for the selected images; implementation details were provided in Text S1.Source: §2.5 · p. 8
• The frequency-analysis implementation used Python 3.13.12 with PyTorch, torchvision, numpy, matplotlib, and opencv-python.Source: §2.5 · p. 8

3.1 Model architecture, model size, training strategy, image size

• Accuracy was largely independent of architecture and model size, while all-layer fine-tuning yielded 0.98–1.00 versus 0.71–0.96 with only FC layers unfrozen.The results suggest ViT is effective for activated-sludge microscopy, and increasing model size does not necessarily improve accuracy.Source: §3.1 · pp. 8, 10, Figure 3 · p. 9
• Accuracy increased from 256 to 672 px, but showed no significant gain from 672 to 896 px, indicating a sufficient-size plateau.The 256-versus-672 px difference was significant (p = 0.00013); the 672-versus-896 px difference was not (p = 0.46).Source: Figure 3 · p. 9, §3.1 · p. 10

3.2 Pretraining methods

• DINOv3 and supervised ImageNet pretraining showed no significant accuracy difference, suggesting both were effective in this task.The paired test reported p = 0.050 and Cohen’s d = 0.24.Source: §3.2 · p. 10, Figure 4 · p. 11

3.3 Field of view vs resolution

• Maintaining field of view produced higher accuracy than maintaining resolution in most settings, except several already near 1.0.The authors offer as one possible explanation that global floc morphology mattered more than texture in this benchmark.Source: §3.3 · pp. 10, 12, Figure 5 · p. 12

3.4 Comparison of deep learning and quantitative image analysis

• Deep learning exceeded 0.9 accuracy, while QIA peaked at 0.873 with 90 flocs and needed more than 60 flocs per instance to exceed 0.8.Those more than 60 flocs corresponded to approximately 16 images; the authors favor deep learning when high predictive performance is the primary objective, although interpretability was not evaluated.Source: §3.4 · p. 13, §3.5.1 · p. 13, Figure 6 · p. 14

3.5.1 Limitation of our research

• The comparison used accuracy without assessing interpretability, and three-sample classification may not generalize to practical property prediction or local-feature tasks.The authors suggest results may differ for tasks relying primarily on local features, such as identifying bacterial species from floc texture.Source: §3.5.1 · pp. 13–15

3.5.2 Suggestion of our research

• The authors recommend exploring ViTs and non-ImageNet pretraining because these approaches performed comparably to CNNs and supervised ImageNet models in this benchmark.Their results align with broader developments in transformer architectures and self-supervised foundation models.Source: §3.5.2 · p. 15
• The authors advise using sufficiently large inputs and preserving field of view, rather than maximizing image size or retaining original resolution.Frequency analysis found that activated-sludge images contained predominantly low-spatial-frequency information.Source: §3.3 · p. 13, §3.5.2 · p. 15

• Deep learning outperformed QIA in accuracy.Source: §4 · p. 16
• Transformer-based architectures and alternative pretraining methods were effective, while increasing model size did not necessarily improve performance.Source: §4 · p. 16
• Images that were too small reduced performance, but increasing image size beyond a certain level did not further improve it; maintaining field of view mattered more than maintaining resolution.Source: §4 · p. 16
• Image information was concentrated primarily in low-frequency components, suggesting that further increases in sensor pixel counts alone may not substantially improve analysis performance.Source: §4 · p. 16

• Fig. S1 shows the learning-rate schedule across training epochs, with different rates for all-layer versus FC-layer-only unfreezing.Source: Figure S1 · p. 18
• Fig. S2 presents the floc size distribution.Source: Figure S2 · p. 18
• Fig. S3 displays the original image, its center-cropped grayscale version, and the shifted power spectrum, where low-frequency components are centered.Source: Figure S3 · p. 19

• The code defines a Fourier transform that shifts the 2D FFT and returns the squared magnitude as the power spectrum.Source: Appendix C · pp. 20–21
• Image preprocessing converts input images to RGB and grayscale, then applies tensor conversion and a center crop of 1216 pixels.Source: Appendix C · pp. 20–21
• The plotting routine displays each RGB image, its preprocessed grayscale image, and its grayscale power spectrum.Source: Appendix C · p. 21
• The script randomly selects two images from each of folders S1, S2, and S3, processes them, and saves the resulting figures.Source: Appendix C · pp. 21–22