PaperlayerView exact revision on arXiv

Source-linked AI summary

Domain-Specific Self-Supervised Representation Learning for Retinal Fundus Classification

Bekzat Nurlanbekova, Fung Fung Ting

arXiv:2608.26686v1cs.CVcs.LG

TL;DR

Annotated medical images are scarce and costly to produce, motivating self-supervised representation learning for retinal fundus classification. The paper compares SimSiam and SimCLR with retinal-specific augmentations and resource-constrained training, finding that learned representations support competitive downstream performance, including improvements on selected tasks.

  • Problem

    Medical image annotation remains scarce, costly, and time-consuming, limiting supervised learning despite its strong benchmark performance.

  • Method

    The study pretrains SimSiam and SimCLR on unlabeled fundus images, using retinal-specific augmentation strategies and evaluating representations through linear evaluation and fine-tuning.

  • Results

    SimSiam exceeded the supervised baseline on RetinaMNIST with 62.08% accuracy and 76.06% QWK, while self-supervised models showed higher DR sensitivity after domain-specific augmentation.

  • Takeaways & Limitations

    Lightweight self-supervised frameworks can learn transferable retinal representations under limited data and computational resources.

Abstract

from arXiv · show

Despite the growing number of public datasets, annotated medical images remain scarce. Supervised learning methods achieve strong performance on many benchmarks, however require large amounts of labeled data, which are costly and time-consuming to obtain in the medical domain. To address this limitation, contrastive self-supervised learning (SSL) has emerged as a promising alternative for learning useful representations from unlabeled data. In this work, we investigate two SSL frameworks, SimSiam and SimCLR, for retinal disease classification from fundus images. We focus on understanding how augmentation strategies and training parameters influence representation learning under resource-constrained settings. Given limited data and computational capacity, we explore the feasibility of training SSL models with small batch sizes incorporated with retinal-specific augmentation techniques. Through a series of experiments, we assess the quality of learned representations via linear evaluation and fine-tuning across downstream tasks, including multi-disease classification and diabetic retinopathy grading. Our results show that tailoring augmentation strategies to the characteristics of retinal images plays a critical role in improving performance. Even under constrained settings, lightweight SSL frameworks can learn transferable representations that reduce dependence on large annotated datasets and achieve competitive results.

I. INTRODUCTION

The paper addresses the scarcity and cost of labeled medical images by evaluating domain-specific self-supervised learning for retinal disease classification under limited data and computational resources.

  • Supervised medical image learning depends on large labeled datasets whose annotation is costly, time-consuming, and requires specialized expertise.
  • Self-supervised learning learns transferable image representations from unlabeled data for downstream medical tasks.
  • The study compares SimSiam and SimCLR for retinal disease classification under limited labeled data and strict resource constraints.
  • It evaluates whether retina-specific Graham’s preprocessing can mitigate performance loss during short-duration pretraining compared with standard augmentations.
  • The experiments investigate whether lightweight self-supervised models learn robust representations that generalize across diverse downstream retinal disease tasks.
  • Representation quality is assessed through linear evaluation and full fine-tuning on labeled data for downstream clinical utility.

II. RELATED WORK

Related work establishes strong but potentially limited transfer from natural-image supervision and motivates dataset- and task-specific augmentation for retinal self-supervised learning.

  • SimCLR and SimSiam have shown promising results across medical imaging applications, including histopathology, dermatology, chest X-ray, and CT-based diagnosis.
  • Supervised models pretrained on natural images can perform well on individual retinal datasets but degrade across differing acquisition conditions.
  • Contrastive learning performance depends strongly on augmentation design, with optimal configurations varying by dataset and task.
  • Retinal preprocessing methods such as min-pooling are incorporated into augmentation pipelines to enhance and preserve clinically relevant structures.

III. METHODOLOGY

The methodology compares contrastive SimCLR with negative-pair-free SimSiam and evaluates representations through a resource-conscious self-supervised learning pipeline.

  • The section describes datasets, training setup, augmentation strategies, and evaluation protocol.
  • SimCLR learns representations by contrasting augmented positive pairs of the same image against negative samples from the mini-batch using NT-Xent loss.
  • For a batch of N images, each image produces two augmented samples, and each positive pair is contrasted against 2N - 2 negatives.
  • SimSiam processes two augmented views with a shared encoder, architectural asymmetry, a predictor, and stop-gradient rather than negative sampling.

B. Data Augmentation Strategy

Retinal-specific augmentation must preserve diagnostically relevant structures, while Graham’s preprocessing improves lesion and vessel clarity by reducing illumination variation.

  • Excessive color jittering, noise injection, blurring, or random cropping can obscure or remove clinically important retinal features.
  • Graham’s preprocessing improves the clarity of lesion areas and blood vessels in raw fundus images.
  • The preprocessing is incorporated into the SimSiam and SimCLR pretraining pipeline as a domain-specific augmentation strategy.
  • The method subtracts a Gaussian-smoothed image to remove low-frequency illumination variation, then rescales and shifts intensity by γ.

C. Datasets and Evaluation Metrics

The study uses three public retinal datasets for self-supervised pretraining and downstream classification, with metrics selected for multi-label and ordered disease-severity tasks.

  • Three publicly available retinal fundus image datasets were used in the study.
  • FISSL provides approximately 48K unlabeled fundus images from four public sources for SSL pretraining.
  • EDID contains 5335 color fundus photographs for multi-label classification, split into training, validation, and test sets at 70%, 10%, and 20%.
  • RetinaMNIST contains 1,600 fundus images annotated across five diabetic-retinopathy stages, with 1080 training, 120 validation, and 400 test samples.
  • Downstream evaluation uses accuracy and macro F1-score, with quadratic weighted kappa additionally used for ordered RetinaMNIST severity labels.
  • QWK ranges from -1 to 1, while accuracy and F1-score range from 0 to 1; higher values indicate better performance.

D. Training Setup

Experiments use a ResNet-18 backbone to compare SimSiam and SimCLR under standardized pretraining, linear evaluation, fine-tuning, and visualization procedures.

  • Both SimSiam and SimCLR use a ResNet-18 backbone and train for 200 epochs with a 10-epoch linear warm-up.
  • During linear probing, the backbone remains frozen and only the final fully connected layer is optimized for 100 epochs.
  • Fine-tuning uses an initial learning rate of 0.001, cosine annealing, five warm-up epochs, SGD momentum of 0.9, and weight decay of 1e-4.
  • Grad-CAM++ visualization maps are extracted from layer 4 of ResNet-18 after resizing inputs to 224 × 224 pixels and normalizing them to [0, 1].
  • All experiments were conducted on NVIDIA T4 and NVIDIA A40 GPUs.

IV. RESULTS AND DISCUSSION

The results section examines how SimSiam compares with SimCLR on fundus photography and how augmentation design and training duration affect downstream performance.

  • The experiments compare negative-pair-free SimSiam with contrastive SimCLR on fundus photography.
  • The analysis evaluates augmentation design and training duration as factors affecting downstream performance.

A. Effect of training duration and batch size

Training duration and batch size affect SimSiam more strongly than SimCLR under standard augmentations. Lower SSL loss does not necessarily improve downstream accuracy, which can saturate or decline with prolonged pretraining.

  • SimSiam is more sensitive than SimCLR to batch size and training duration.The comparison was conducted under standard augmentations.
  • SimSiam accuracy declines after 100 epochs with batch size 256 on both EDID and RetinaMNIST.SimCLR remains stable across both tested batch sizes and training durations.
  • Continued decreases in training loss do not necessarily translate into improved downstream accuracy.Pretraining beyond early convergence provides limited benefit, and later linear-evaluation performance can saturate or decline.
  • The SimSiam loss–accuracy discrepancy suggests representation diversity may decrease over time under some configurations.The passage also links standard augmentations to difficulty constructing sufficiently informative positive pairs for retinal images.

B. Effect of augmentation strategies

The experiments compare standard, grayscale-enhanced, and retina-specific augmentation strategies for SimSiam and SimCLR. Adding grayscale and Ben Graham’s preprocessing improves representation quality, with especially substantial gains for SimSiam across both datasets.

  • Augmentation strategies: The standard pipeline combines horizontal flipping, rotation, resized cropping, color jitter, and Gaussian blur.Random grayscale is added in a second setup before incorporating Ben Graham’s preprocessing in the domain-specific pipeline.
  • Augmentation strategies: At 50 epochs and batch size 256, linear evaluation compares SimSiam and SimCLR on EDID and RetinaMNIST.Both models use grayscale and Ben Graham’s preprocessing in addition to standard augmentation.
  • Augmentation strategies: Grayscale combined with Ben Graham’s preprocessing consistently improves SimCLR performance.The comparison is reported through linear evaluation results.
  • Augmentation strategies: Domain-specific augmentation produces substantial SimSiam improvements across both datasets.This setup integrates grayscale and Ben Graham’s preprocessing with standard augmentation.

C. Downstream performance

Domain-specific pretraining yields competitive downstream performance, but outcomes vary by dataset and pathology. SimSiam benefits strongly on RetinaMNIST and shows a marginal EDID accuracy lead, while excluding glaucoma substantially raises both models’ EDID performance.

  • Downstream performance: SimSiam reaches 62.08% accuracy and 76.06% QWK on RetinaMNIST, exceeding the supervised baseline.SimCLR has competitive QWK but slightly lower raw accuracy than the supervised baseline.
  • Downstream performance: On EDID, SimSiam has a marginal accuracy lead over the supervised baseline, whereas SimCLR degrades more noticeably.The passage attributes performance variation across datasets to their specific pathological characteristics.
  • Downstream performance: Removing glaucoma increases SimSiam EDID performance from 75.88% to 88.75% and SimCLR performance from 73.81% to 87.49%.The subset analysis reports significant increases across all models after glaucoma cases are excluded.
  • Downstream performance: SimCLR and SimSiam achieve DR sensitivities of 93.38% and 92.05%, respectively, versus 88.74% for the supervised baseline.These results come from the EDID subset excluding glaucoma.
  • Downstream performance: Grad-CAM comparisons cover optic disc edema, diabetic retinopathy, and glaucoma across raw, ImageNet-supervised, SimSiam, and SimCLR models.The supplied figure description specifies the visualization conditions but not a reported outcome.

D. Discussion

The discussion emphasizes that augmentation effectiveness depends on alignment with dataset-specific pathological biomarkers. Ben Graham’s preprocessing helps with vascular and lesion-contrast features but may bias performance for heterogeneous or non-vascular diseases such as glaucoma.

  • Discussion: Augmentation effectiveness depends heavily on alignment with dataset-specific pathological biomarkers.Domain-specific transformations can enhance task-relevant features while introducing bias in heterogeneous settings.
  • Discussion: Ben Graham’s preprocessing is especially effective for pathologies sharing vascular structures and lesion contrast.Both SimSiam and SimCLR benefited from retina-specific pretraining.
  • Discussion: The preprocessing may suppress biomarkers or introduce bias against non-vascular diseases such as glaucoma.This establishes a scope boundary for domain-specific augmentation benefits.
  • Discussion: Table II compares domain-specific SimSiam and SimCLR fine-tuning with a supervised baseline on RetinaMNIST and EDID.The supplied table caption specifies the comparison and evaluation datasets but no numerical outcomes.
  • Discussion: Table III reports DR identification sensitivity on an EDID subset excluding glaucoma.The discussion reports higher sensitivities for both SSL models than the supervised baseline on this subset.
Loading 2608.26686v1…