Multi-Label Retinal Disease Classification using Transformers

I. INTRODUCTION

• Multi-label classification addresses the need to detect multiple retinal diseases simultaneously, unlike the single-disease focus of most prior work.The challenge includes diseases with different spatial extents, from localized conditions to those manifesting across the retina.Source: §I · p. 2
• Existing datasets may focus on single diseases, have few samples or pathologies, and show class imbalance, motivating dataset combination and imbalance-handling techniques.Early detection may support diagnosis before symptoms and help prevent total vision loss through timely treatment.Source: §I · pp. 1–2
• The MuReD dataset combines publicly available fundus datasets and is preprocessed to preserve image quality and provide diverse diseases with sufficient samples per label.The stated contributions describe 20 disease classes and automatic quality scoring based on image sharpness and brightness.Source: §I · p. 2
• The proposed system uses a transformer-based model optimized through experiments on architecture, hyperparameters, and techniques to partly address MuReD class imbalance.The introduction presents this as the first use of a transformer-based model for detecting and classifying multiple retinal diseases.Source: §I · p. 2

A. CNN methods

• Cen et al. report AUC 0.99, while Ju et al. report mAP 64.14% and 64.69% on their 100K- and 1 million-image datasets, respectively.For ODIR, He et al. and Li et al. report AUC 93%; Gour et al. and Wang et al. report AUC 84.93% and 74%, respectively.Source: A. CNN methods · p. 2

B. Class Imbalance

• Class-imbalance approaches include resampling, classifier adaptation, ensembles, and cost-sensitive methods, with effectiveness depending on the application.Resampling modifies dataset representation, whereas cost-sensitive methods adjust loss to increase the cost of misclassifying minority classes.Source: B. Class Imbalance · p. 3
• The authors use resampling and cost-sensitive methods, excluding ensembles because of the 20-label setting and classifier adaptation because of model complexity.The selected methods avoid training several models or building a more complex, specialized training pipeline.Source: B. Class Imbalance · p. 3

C. Fundus image datasets

• The surveyed public fundus datasets were developed for different tasks and include screening, disease-specific, and multi-disease collections.Examples described include DRIVE, STARE, CHASE-DB, Messidor, e-ophtha, Kaggle-EyePacs, ARIA, and RFMiD.Source: C. Fundus image datasets · p. 3
• The reviewed multi-label datasets have sparse representation for underrepresented diseases, long-tail class imbalance, and no guaranteed image-quality cleaning.For half of the underrepresented diseases, datasets contain at most 20 images, with some having only one.Source: C. Fundus image datasets · p. 4
• The dataset review identifies low sample counts and large differences in representation between overrepresented and underrepresented disease labels as shared concerns.The authors note that these data conditions reduce confidence in classifying diseases with very few examples.Source: C. Fundus image datasets · p. 4

III. DATASET

• The composite dataset was built from public sources to cover a wide variety of diseases, adequate per-class samples, and a minimum degree of image quality.Image quality varies with disease and external factors, which can introduce noise, blurriness, and artifacts.Source: §III · p. 4
• MuReD combines ARIA, STARE, and the RFMiD training set, initially yielding 2,451 samples, 52 disease labels, NORMAL, and OTHER.Only RFMiD's training set was included to avoid too much bias given the smaller ARIA and STARE datasets.Source: §III · p. 4

A. Dataset Cleaning

• Labels with fewer than 30 samples were dropped as prediction classes, and their samples were assigned to OTHER, leaving 20 prediction classes.The 30-sample threshold was selected through simulation studies.Source: A. Dataset Cleaning · p. 4
• The blur metric used Canny edge detection and neighboring-pixel differences; images below 0.058 were removed, including the bottom 10% by score.A higher blur-metric score indicates greater sharpness; 90% of manually identified low-quality images matched those with the lowest metric scores.Source: A. Dataset Cleaning · pp. 4–5
• 2,208 samples across 20 disease labels remained after the cleaning procedure.The final dataset size and label count are reported after cleaning.Source: A. Dataset Cleaning · p. 5

B. Comparison with publicly available datasets

• The comparison uses meanIR for average label imbalance and CVIR for variation in imbalance across labels; higher CVIR indicates greater variation.Both metrics were proposed specifically for measuring imbalance in multi-label datasets.Source: B. Comparison with publicly available… · p. 5
• MuReD reduced meanIR by 5.59 (44%) and CVIR by 0.21 (23%) compared with the best values achieved by the available datasets.meanIR measures the average imbalance relative to the majority label; CVIR measures differences in imbalance across labels.Source: B. Comparison with publicly available… · p. 5

A. Data Preprocessing

• FOV extraction removes the black background by thresholding horizontal and vertical scans of the red channel at 6% of each scan's maximum intensity.The scans locate the retinal region of interest from brightness changes.Source: A. Data Preprocessing · p. 5

B. Multi-Label Classification Model

• C-Tran combines CNN visual features with masked label embeddings in a transformer encoder, whose output feeds independent label-specific prediction networks.The architecture is designed for multi-label tasks and can model dependencies between features and labels.Source: B. Multi-Label Classification Model · pp. 5, 7
• C-Tran pairs each learned label embedding with a state embedding representing unknown, negative, or positive status.The state values are 0, -1, and +1, respectively.Source: B. Multi-Label Classification Model · p. 6
• The transformer uses self-attention to weight relationships among visual-feature and label embeddings, then updates each embedding using a weighted sum.A nonlinear ReLU is applied after the update.Source: B. Multi-Label Classification Model · p. 6
• Label-mask training hides a random 25%–100% of labels and predicts the unknown labels, exposing the model to varied known-label combinations.The remaining 0%–75% of labels are marked known, whether positive or negative.Source: B. Multi-Label Classification Model · p. 7
• At inference, all label embeddings are masked as unknown, so the model makes regular predictions without prior or partial label information.The method uses label-mask training but no prior knowledge for inference.Source: B. Multi-Label Classification Model · p. 7

A. Metrics

• The RIADD challenge metric gives equal importance to detecting disease presence and correctly classifying disease, averaging F1, mAP, and AUC across disease labels while excluding NORMAL.These averages are named ML F1, ML mAP, and ML AUC, respectively.Source: A. Metrics · p. 7
• Training used random image augmentations configured from the RIADD challenge winner’s setup, with their descriptions and parameters listed in Table III.The listed transforms include cutout, median blur, Gaussian noise, and hue, saturation, and value changes.Source: B. Determination of optimal model… · p. 8, Table III · p. 8, A. Metrics · p. 8
• DenseNet161 performs best among the compared CNN feature extractors and is used as the C-Tran backbone in subsequent experiments.Table IV compares results for different CNN architectures as feature extractors.Source: 1) Backbone Selection · p. 8, Table IV · p. 8

2) Class Imbalance:

• Increasing the oversampling percentage did not outperform the 10% baseline, and LP ROS at 10% was selected as the best resampling strategy.The investigation tested resampling percentages for ML ROS and LP ROS; Table VII covers LP ROS percentages.Source: 2) Class Imbalance · p. 9, Table VII · p. 9, Table VI · p. 9
• BCE and PolyLoss performed similarly, but PolyLoss was selected because it performed better on the other calculated metrics.The loss-function comparison is reported in Table VIII.Source: 2) Class Imbalance · p. 9, Table VIII · p. 9
• LP ROS at a 10% resampling rate combined with Polynomial Loss was the best strategy tested for alleviating class-imbalance effects on model performance.This configuration was used in subsequent experimentation.Source: 2) Class Imbalance · p. 9
• Most available techniques for alleviating class imbalance in multi-label datasets either did not help or only marginally improved results.The authors conclude that more methods and new ideas are needed in this field.Source: 2) Class Imbalance · p. 9

3) Hyperparameter Optimization:

• Increasing image size did not yield better results than the standard 384 × 384 size; the authors hypothesize that differing resolutions across combined datasets may explain this.Source: 3) Hyperparameter Optimization · p. 9
• Increasing batch size from 16 to 32 produced a slight performance gain.Table X compares results across batch sizes.Source: 3) Hyperparameter Optimization · p. 10, Table X · p. 10
• The best-performing MuReD configuration used C-Tran with Adam, a 10^-5 learning rate, Polynomial Loss, DenseNet161, 10% LP ROS, and 384 × 384 × 3 input images.This was the overall configuration selected for comparison with other fundus multi-label classification approaches.Source: 3) Hyperparameter Optimization · p. 10

• The comparison focused on methods applicable to single-image classification, excluding some ODIR methods that rely on paired-eye, patient-level diagnostics.Some other approaches were difficult to reproduce because they used private data or lacked available code; unspecified published hyperparameters were selected through trial and error.Source: C. Comparison with alternative… · p. 10
• C-Tran outperformed previous CNN-based approaches by a considerable margin on the MuReD dataset.The comparison included reproducible single-image approaches and the RIADD challenge winner, reproduced as accurately as possible for evaluation on MuReD.Source: C. Comparison with alternative… · p. 10
• Most per-class AUC scores exceeded 90%, while OTHER and ODP had lower AUC scores and HTR and ASR had no F1 results at the 0.5 confidence threshold.OTHER covers diseases outside the original label set; ODP has subtle optic-disc color changes.Source: C. Comparison with alternative… · p. 10, Table XII · p. 11

D. Class Activation Maps

• CAM heat maps showed different image zones used to classify each disease in the sample, supporting the authors’ interpretation that the model learned disease-associated features.The maps were generated with the Pytorch GradCam library; CAMs use a weighted sum of visual patterns, upsampled to image size and passed through a softmax or sigmoid output function.Source: D. Class Activation Maps · p. 11

VI. CONCLUSIONS

• MuReD combines three publicly available datasets and contains 2,208 samples across 20 classes after quality cleaning and a minimum-sample-per-class criterion.Source: §VI · p. 11
• The proposed transformer-based pipeline was optimized through experiments and achieved superior results to previously proposed techniques on the same task.Source: §VI · p. 11
• Future work will seek better ways to address MuReD’s class imbalance and explore C-Tran in other medical-imaging multilabel problems.The conclusion also envisages applying transformer architectures to a wider variety of medical tasks.Source: §VI · p. 11