PaperlayerView exact revision on arXiv

Source-linked AI summary

Balanced-MixUp for Highly Imbalanced Medical Image Classification

Adrian Galdran, Gustavo Carneiro, Miguel A. González Ballester

arXiv:2109.09850v1cs.CV

TL;DR

Rare classes are often underrepresented in medical image datasets, and severe imbalance can reduce minority-class performance. Balanced-MixUp combines instance-based and class-based sampling through MixUp, and the paper reports improved performance over other imbalance approaches across its considered cases.

  • Problem

    Rare disease examples are difficult to collect, and severe class imbalance can let majority classes dominate training and reduce minority-class performance.

    Source: §1 · p. 1

  • Method

    Balanced-MixUp pairs instance-based and class-based samples and mixes their images and labels to create synthetic training examples.

    Source: §1 · p. 2, §2.3 · p. 4

  • Results

    Balanced-MixUp outperformed conventional and recent imbalance approaches across the considered cases; on GI classification, it improved MCC for both CNNs and balanced accuracy and macro-F1 across α values.

    Source: §1 · p. 2, §3.3 · p. 8

  • Takeaways & Limitations

    The authors present Balanced-MixUp as an easy-to-implement supplementary regularizer, with extension beyond image classification left for future research.

    Source: §3.2 · p. 7, §4 · p. 8

    The DR results are not state of the art: specialized attention architectures and semi-supervised methods with segmentation branches reach higher quad-κ values.

    Source: §3.2 · pp. 6–7

Abstract

from arXiv · show

Highly imbalanced datasets are ubiquitous in medical image classification problems. In such problems, it is often the case that rare classes associated to less prevalent diseases are severely under-represented in labeled databases, typically resulting in poor performance of machine learning algorithms due to overfitting in the learning process. In this paper, we propose a novel mechanism for sampling training data based on the popular MixUp regularization technique, which we refer to as Balanced-MixUp. In short, Balanced-MixUp simultaneously performs regular (i.e., instance-based) and balanced (i.e., class-based) sampling of the training data. The resulting two sets of samples are then mixed-up to create a more balanced training distribution from which a neural network can effectively learn without incurring in heavily under-fitting the minority classes. We experiment with a highly imbalanced dataset of retinal images (55K samples, 5 classes) and a long-tail dataset of gastro-intestinal video frames (10K images, 23 classes), using two CNNs of varying representation capabilities. Experimental results demonstrate that applying Balanced-MixUp outperforms other conventional sampling schemes and loss functions specifically designed to deal with imbalanced data. Code is released at https://github.com/agaldran/balanced_mixup .

1 Introduction

Medical image datasets can underrepresent rare diseases, while real-world clinical class distributions are often long-tailed; this imbalance can hurt minority-class performance. The paper introduces Balanced-MixUp and reports improved performance over other imbalance approaches across its considered cases.

1 Introduction

  • Rare diseases are difficult to label at scale, so medical models may train on class distributions more balanced than real clinical scenarios.Under severe imbalance, majority classes can dominate training and reduce minority-class performance.Source: §1 · p. 1
  • Unlike ordinary MixUp, which combines samples without regard to class, Balanced-MixUp deliberately mixes minority-category examples with examples from other classes.The method combines modified sampling strategies with MixUp's synthetic data manipulation.Source: §1 · p. 2
  • Related approaches include ReMix's minority-label mixing, MixBoost's active selection of samples, and SMOTE's same-class nearest-neighbor combinations.The paper notes that SMOTE lacks MixUp's regularizing benefits.Source: §1 · p. 2
  • Balanced-MixUp improved performance over conventional and recent imbalance approaches across the considered cases, using retinal DR grading and long-tailed gastrointestinal image classification.The two tasks cover highly imbalanced and long-tailed settings, respectively.Source: §1 · p. 2, Figure 1 · p. 2

2 Methodology

The methodology combines MixUp regularization with modified sampling for imbalanced datasets. Balanced-MixUp pairs instance-based and class-based draws to form synthetic training examples.

2 Methodology

  • The methodology introduces the combined approach as a way to pair MixUp regularization with modified sampling for imbalanced datasets.This frames the two components developed in the following subsections.Source: §2 · p. 3, §2.3 · p. 4

2.1 MixUp Regularization

  • MixUp regularizes neural networks by forming convex combinations of training images and their labels, with λ sampled from Beta(α, α).The paper describes typical α values of 0.1–0.4 and reports that mixed-data training can improve generalization and calibration.Source: §2.1 · p. 3

2.2 Training Data Sampling Strategies

  • For class sampling, q=1 follows class frequencies, q=0 samples classes uniformly, and q=1/2 gives square-root sampling.Repeatedly oversampling minority examples can overfit them, motivating alternatives to purely class-based sampling.Source: §2.2 · p. 3

2.3 Balanced-MixUp

  • Balanced-MixUp combines an instance-based draw with a class-based draw, then mixes their images and labels to add synthetic examples in minority-sparse regions.Its λ is drawn from Beta(α, 1); increasing α gives minority examples greater weight, increasing balance while risking overfitting.Source: §2.3 · p. 4

2.4 Training Details

  • CNNs are trained with cross-entropy and SGD (batch size 8, learning rate 0.01), standard augmentation, cyclic learning-rate annealing, validation-based model selection, and flipped-image prediction averaging.The best validation-performing model is retained.Source: §2.4 · p. 5

3 Experimental Analysis

The experiments test two CNNs on imbalanced retinal and gastrointestinal image classification, comparing Balanced-MixUp with sampling and loss-function alternatives. Balanced-MixUp leads the reported diabetic-retinopathy results and improves performance across the reported gastrointestinal metrics, with outcomes varying by architecture and metric.

3.1 Experimental Details

  • The DR experiments use Eyepacs, with five grades, approximately 35,000 training examples, and approximately 55,000 test images; Messidor-2 provides external testing.The GI experiments use Hyper-Kvasir, with 10,662 images and 23 classes, evaluated by stratified five-fold cross-validation.Source: §3.1 · p. 5
  • The primary DR metric is quadratic-weighted kappa; GI uses MCC, with balanced accuracy and macro-F1 also reported to assess minority-class performance.DR results also include MCC and Kendall-τ.Source: §3.1 · p. 6
  • Comparisons include instance-, class-, and square-root-frequency sampling, Focal Loss, and Class-Balanced loss, using MobileNet V2 and ResNext50.Balanced-MixUp is evaluated at α values 0.1, 0.2, and 0.3.Source: §3.1 · p. 6

3.2 Discussion on the Numerical Results on DR grading

  • DR results are not state of the art: specialized attention architectures and semi-supervised methods with segmentation branches achieve higher quad-κ.The authors describe their results as aligned with other recent techniques and report Eyepacs-to-Messidor-2 testing without retraining as evidence of generalization.Source: §3.2 · pp. 6–7
  • For DR grading, Balanced-MixUp α=0.2 leads all measures with ResNext50, while α=0.1 leads MobileNet V2; all α values beat alternatives.Table 1 compares two CNN architectures on the Eyepacs test set.Source: §3.2 · p. 6, Table 1 · p. 7

3.3 Discussion on the Numerical Results on GI image classification

  • GI results are medians from stratified five-fold cross-validation on a more imbalanced, 23-category task than DR.Table 3 reports these Hyper-Kvasir results.Source: §3.3 · p. 7, Table 3 · p. 8
  • For GI classification, Balanced-MixUp improves MCC with both architectures; MobileNet performs best at α=0.1, while ResNeXt performs better at α=0.2.For balanced accuracy and macro-F1, smaller architectures benefit almost all methods, and Balanced-MixUp improves performance for every tested α.Source: §3.3 · p. 8

4 Conclusion

Imbalanced data is common in medical image analysis and makes learning in these regimes a challenge of clinical significance. Balanced-MixUp combines MixUp with modified sampling strategies and delivers consistent performance improvements over other popular techniques.

4 Conclusion

  • Balanced-MixUp combines MixUp regularization with modified training-data sampling strategies to address heavily imbalanced distributions.The paper describes the method as easy to implement.Source: §4 · p. 8
  • The method delivers consistent performance improvements compared with other popular techniques for handling imbalance.Source: §4 · p. 8
  • Imbalanced data occurs in almost any medical image analysis task, making learning in such regimes a challenge of clinical significance.Source: §4 · p. 8
  • Extending Balanced-MixUp beyond image classification is identified as a promising future research direction.Source: §4 · p. 8

A.1 Numerical Results on DR grading

Eyepacs test-set labels and predictions are bootstrapped in a stratified manner to derive confidence intervals. Tables 4 and 5 report mean and standard deviation for performance comparisons.

A.1 Numerical Results on DR grading

  • n=1000 stratified bootstrap samples of Eyepacs test labels and model predictions are used to derive confidence intervals.Stratification is according to the presence of DR grades.Source: §A.1 · p. 11
  • Tables 4 and 5 compare performance on Eyepacs, with results summarized by mean and standard deviation.Source: Table 4 · p. 11, Table 5 · p. 11, §A.1 · p. 11

A.2 Numerical Results on GI image classification

On GI image classification, MobileNet with Balanced-MixUp at α = 0.3 attains the best reported median, minimum, and maximum results for Balanced Accuracy or Macro-F1. Results use stratified five-fold cross-validation and are reported across folds to show dispersion.

A.2 Numerical Results on GI image classification

  • Results come from stratified five-fold cross-validation and are reported as minimum, median, and maximum performance across folds.The median was the performance reported in the paper; the additional extrema show fold-to-fold dispersion.Source: §A.2 · p. 12
  • MobileNet with Balanced-MixUp (α = 0.3) attains the best overall median, minimum, and maximum results for Balanced Accuracy or Macro-F1.CB-loss with ResNeXt50 also performs well.Source: Table 6 · p. 12, Table 7 · p. 12, §A.2 · p. 12
  • GI results are noisier than DR-grading results because the minority classes contain few instances.Source: §A.2 · p. 12
Loading 2109.09850v1…