PaperlayerView exact revision on arXiv

Source-linked AI summary

An Analysis on Ensemble Learning optimized Medical Image Classification with Deep Convolutional Neural Networks

Dominik Müller, Iñaki Soto-Rey, Frank Kramer

arXiv:2201.11440v2cs.CVcs.AIcs.LG

TL;DR

The benefits and relative value of ensemble strategies in deep-learning medical image classification remain insufficiently analyzed. This paper compares Augmenting, Stacking, and Bagging in a reproducible pipeline, finding the largest performance gain with Stacking and a ranking of Stacking, Bagging, then Augmenting. It also reports that simple pooling can match or outperform more complex pooling and notes a small-dataset overfitting caveat for Augmenting.

  • Problem

    Which ensemble strategies benefit deep-learning medical image classification, and to what extent, remain open questions despite their use in high-performing pipelines.

    Source: §1 · pp. 1–2

  • Method

    The study compares Augmenting, Stacking, and Bagging against a baseline in a reproducible pipeline across medical imaging datasets.

    Source: §1 · p. 2, §5 · p. 12

  • Results

    Stacking ranked highest for performance gain, followed by Bagging and Augmenting; simple pooling functions were equally strong or often better than more complex ones.

    Source: §4 · p. 11

  • Takeaways & Limitations

    The authors conclude that ensemble learning can boost classification performance, with Augmenting also applicable to single-model pipelines when models are not overfitting.

    Source: §4 · p. 11, §5 · p. 12

    Augmenting may adversely affect performance on small datasets when models have a high risk of overfitting.

    Source: §4 · p. 12

Abstract

from arXiv · show

Novel and high-performance medical image classification pipelines are heavily utilizing ensemble learning strategies. The idea of ensemble learning is to assemble diverse models or multiple predictions and, thus, boost prediction performance. However, it is still an open question to what extent as well as which ensemble learning strategies are beneficial in deep learning based medical image classification pipelines. In this work, we proposed a reproducible medical image classification pipeline for analyzing the performance impact of the following ensemble learning techniques: Augmenting, Stacking, and Bagging. The pipeline consists of state-of-the-art preprocessing and image augmentation methods as well as 9 deep convolution neural network architectures. It was applied on four popular medical imaging datasets with varying complexity. Furthermore, 12 pooling functions for combining multiple predictions were analyzed, ranging from simple statistical functions like unweighted averaging up to more complex learning-based functions like support vector machines. Our results revealed that Stacking achieved the largest performance gain of up to 13% F1-score increase. Augmenting showed consistent improvement capabilities by up to 4% and is also applicable to single model based pipelines. Cross-validation based Bagging demonstrated significant performance gain close to Stacking, which resulted in an F1-score increase up to +11%. Furthermore, we demonstrated that simple statistical pooling functions are equal or often even better than more complex pooling functions. We concluded that the integration of ensemble learning techniques is a powerful method for any medical image classification pipeline to improve robustness and boost performance.

1. Introduction

Ensemble learning combines models or predictions to improve medical image classification, but its benefits and the most useful strategies in deep-learning pipelines remain insufficiently analyzed. This study proposes a reproducible comparison of three techniques across diverse medical imaging datasets.

1. Introduction

  • Medical image classification labels a whole image with a predefined class, such as a diagnosis or condition, for potential clinical decision support.The stated goals include improving diagnosis reliability or automating time-consuming processes.Source: §1 · p. 1
  • Ensemble learning combines fitted neural-network models to improve prediction performance, and prior work has applied it to improve medical image classification performance and robustness.The paper describes small medical imaging datasets as a reason these techniques are popular for efficient training-data use.Source: §1 · p. 1
  • Which ensemble strategies benefit deep-learning medical image classification, and to what extent, remain open questions.The impact of ensemble strategies in deep-learning classification had not yet been adequately analyzed in the literature.Source: §1 · pp. 1–2
  • The study compares Augmenting, Stacking, and Bagging with a baseline to assess performance across medical imaging datasets and identify beneficial or unfavorable effects.Its stated aim is to provide a broader analysis of ensemble strategies in deep-learning-based medical image classification.Source: §1 · p. 2

2. Methods and Materials

The pipeline compares Augmenting, Stacking, and Bagging with baseline models across four medical imaging datasets. It uses standardized sampling and preprocessing, diverse pretrained convolutional architectures, multiple pooling methods, and class-wise evaluation metrics.

2.1 Datasets

  • Four datasets span histology, X-ray, dermoscopy, and ophthalmoscopy, with varied sample sizes, modalities, feature types, and noise.The CHMNIST, COVID, ISIC, and DRD datasets include image patches, radiographs, skin-lesion images, and fundus images, respectively.Source: §2.1 · pp. 2–3

2.2 Sampling and Preprocessing

  • Samples were split into training (65%), validation (10%), ensemble-training (10%), and testing (15%) sets; Bagging instead used five-fold cross-validation.For Bagging, each fold used 60% of a dataset for training and 15% for validation.Source: §2.2 · p. 3
  • The pipeline used online flips, rotations, and color adjustments, squared padding, resizing, and ImageNet-based Z-Score intensity normalization.Most architectures used 224x224 inputs; EfficientNetB4 used 380x380, while InceptionResNetV2 and Xception used 299x299.Source: §2.2 · pp. 3–4

2.3 Deep Learning Models

  • Nine diverse architectures were pretrained on ImageNet, then trained with a frozen classification head and fine-tuned using weighted Focal loss and early stopping.The architectures included eight named models and a custom four-layer Vanilla network; early stopping followed 15 epochs without validation-loss improvement.Source: §2.3 · p. 4

2.4 Ensemble Learning Techniques

  • The study compared baseline models with Augmenting, Stacking, and five-fold Bagging; Boosting was excluded because it was not feasible given the training-time increase.Augmenting combined predictions from 15 randomly rotated or mirrored images per sample using an unweighted mean; Stacking pooled predictions from diverse architectures, and Bagging pooled predictions from five models.Source: §2.4 · pp. 4–5

2.5 Pooling Functions

  • Bagging and Stacking predictions were combined with 12 pooling functions, while Augmenting used only the unweighted mean.The pooling options included statistical methods and learning-based classifiers such as Decision Trees, Gaussian Processes, Logistic Regression, Naïve Bayes, Support Vector Machines, and k-Nearest Neighbors.Source: §2.5 · pp. 5–6

2.6 Evaluation

  • Evaluation used unweighted class-wise averages of Accuracy, F1-score, Sensitivity, False Positive Rate, and AUC/ROC.The supplementary material includes additional metrics such as Top-1/3-Error and Specificity.Source: §2.6 · p. 6

3. Results

Across four datasets, the ensemble methods produced varied F1-score gains over Baseline, with Stacking strongest and Bagging close behind.

3. Results

  • The analysis took 1,215 hours, with Bagging using 1,002 hours (82.5%); training stopped after an average of 51 epochs without observed overfitting.Augmenting and Stacking added little training time because they used Baseline models.Source: §3 · p. 6

3.1 Baseline

3.2 Augmenting

  • Augmenting changed best-possible F1-score versus Baseline by 0% CHMNIST, -1% COVID, +3% ISIC, and +4% DRD.Architecture rankings remained similar to Baseline, and class-wise Accuracy variance was equivalent.Source: §3.2 · pp. 7–8, Table 3 · p. 8, Figure 4 · p. 8

3.3 Stacking

  • Stacking raised median F1-score over Baseline by +1% CHMNIST, +2% COVID, +13% ISIC, and +12% DRD.Best-pooling choices varied by dataset; Decision Tree performed worst, while precision improved especially in ISIC and DRD.Source: §3.3 · pp. 8–9, Table 4 · p. 9, Figure 5 · p. 9

3.4 Bagging

  • Bagging raised F1-score over Baseline by +1% CHMNIST, +2% COVID, +11% ISIC, and +9% DRD.This used five-fold EfficientNetB4 models; Mean, Majority Voting, Gaussian Process, and Logistic Regression had the highest average performance, while ROC precision fell notably on CHMNIST and COVID.Source: §3.4 · pp. 9–10, Table 5 · p. 10, Figure 6 · p. 10

4. Discussion

Stacking had the largest reported performance gain, followed by Bagging and Augmenting. The discussion also distinguishes pooling behavior by metric and notes dataset-size and sampling conditions that can limit ensemble gains.

4. Discussion

  • For F1-score, Mean and Soft Majority Vote matched or exceeded complex pooling, while complex pooling scored higher on Accuracy.The authors relate this difference to class-weighted model training versus separate training of complex pooling functions.Source: §4 · p. 11
  • Augmenting improved performance in nearly all experiments by reducing overfitting bias, but increased image variability could hurt small, high-risk datasets.The discussion also notes that its gains depend on the augmentation methods and medical context.Source: §4 · pp. 11–12
  • Stacking ranked highest in performance gain, followed by Bagging and Augmenting.Figure 7 summarizes the experiments; it compares maximum Accuracy and F1-score distributions and performance impacts against the Baseline.Source: Figure 7 · p. 11, §4 · p. 11
  • Cross-validation Bagging improved large and complex datasets, but could reduce performance on small datasets when folds excluded rare morphological cases.The analysis used 5-fold cross-validation; the authors identify fold number and sampling size as questions for further research.Source: §4 · p. 12

5. Conclusions

The paper concludes that Stacking yielded the largest gains in its medical image classification pipeline, while Augmenting and Bagging also helped under stated conditions. It identifies further study of Bagging and explainability as open directions.

5. Conclusions

  • Stacking achieved the largest performance gain; Augmenting improved non-overfitting models, while Bagging’s gains depended on sufficient feature representation in every fold.The conclusion ranks Stacking above Bagging and Augmenting overall.Source: §5 · p. 12
  • Mean and Majority Voting were equal or often better than Support Vector Machines for pooling predictions.Source: §5 · p. 12
  • Further research is planned on cross-validation fold counts, deep-learning Boosting, and explainable AI for multi-model medical image classification.The conclusion says explainable AI for these pipelines remains an open research field.Source: §5 · pp. 12–13

Appendix

The paper makes its implementation and study data available through public repositories.

Appendix

  • The Python implementation is available under GPL-3.0, and the study data are available in a Zenodo repository.Source: Appendix · p. 13

Related papers

Loading 2201.11440v2…