Source-linked AI summary
G1020: A Benchmark Retinal Fundus Image Dataset for Computer-Aided Glaucoma Detection
Muhammad Naseer Bajwa, Gur Amrit Pal Singh, Wolfgang Neumeier, Muhammad Imran Malik, Andreas Dengel, Sheraz Ahmed
TL;DR
Public fundus datasets for glaucoma detection are often small and collected under restrictive imaging conditions, limiting their representation of routine practice. The paper presents G1020 with clinical and image annotations and evaluates deep-learning methods, with initial evaluations highlighting challenges for practical computer-aided diagnosis.
Problem
Method
The authors present a realistic-condition fundus dataset with clinical diagnoses, optic-disc and cup annotations, bounding boxes, cup-to-disc ratios, and quadrant rim measurements.
Source: §I · pp. 1–2, §III · p. 3Results
Initial evaluations of deep-learning methods for optic-disc and cup segmentation and glaucoma classification highlight challenges in developing a practical computer-aided diagnosis system.
Source: §V · p. 6Takeaways & Limitations
G1020 is presented as a dataset that closely represents practical clinical fundus imaging and does not enforce strict image-inclusion criteria.
Source: §V · p. 6
Abstract
from arXiv · showhide
Scarcity of large publicly available retinal fundus image datasets for automated glaucoma detection has been the bottleneck for successful application of artificial intelligence towards practical Computer-Aided Diagnosis (CAD). A few small datasets that are available for research community usually suffer from impractical image capturing conditions and stringent inclusion criteria. These shortcomings in already limited choice of existing datasets make it challenging to mature a CAD system so that it can perform in real-world environment. In this paper we present a large publicly available retinal fundus image dataset for glaucoma classification called G1020. The dataset is curated by conforming to standard practices in routine ophthalmology and it is expected to serve as standard benchmark dataset for glaucoma detection. This database consists of 1020 high resolution colour fundus images and provides ground truth annotations for glaucoma diagnosis, optic disc and optic cup segmentation, vertical cup-to-disc ratio, size of neuroretinal rim in inferior, superior, nasal and temporal quadrants, and bounding box location for optic disc. We also report baseline results by conducting extensive experiments for automated glaucoma diagnosis and segmentation of optic disc and optic cup.
I. INTRODUCTION
The introduction argues that small, constrained fundus datasets may not reflect routine screening and clinical imaging. It presents G1020 as a realistic-condition dataset with annotations for glaucoma detection and optic-disc and cup analysis.
I. INTRODUCTION
- Small public datasets often impose imaging constraints or exclude artifacts, potentially limiting their relevance to routine screening and clinical workflows.Fundus imaging is described as the most common and inexpensive modality for large-scale retinal-disease screening.Source: §I · p. 1
- G1020 is presented as a publicly available dataset for optic-disc and optic-cup segmentation and glaucoma detection, collected without many imaging constraints.The authors describe it as fairly representative of real-world fundus imaging practices.Source: §I · p. 1
- Its annotations include optic-disc and cup segmentation, disc bounding boxes, vertical cup-to-disc ratio, quadrant neuroretinal-rim sizes, and clinical diagnoses.The rim measurements are used to assess whether the ISNT rule is followed; the dataset also includes diagnoses of other ocular disorders.Source: §I · p. 2
II. RELATED WORK
The related-work section surveys large publicly available fundus datasets for glaucoma detection and optic-disc and cup segmentation, then reviews contemporary segmentation and classification studies.
II. RELATED WORK
- The section first surveys publicly available fundus datasets, then reviews contemporary work on segmentation and classification using those and other datasets.Source: §II · p. 2
1) ORIGA:
The section describes ORIGA, RIM-ONE, and REGA as differing in size, labels, and segmentation annotations. Their provided labels and annotations vary across datasets.
1) ORIGA:
- ORIGA contains 650 images—168 glaucoma and 482 healthy—with class labels, optic-disc and cup contours, and cup-to-disc ratios.Source: 1) ORIGA · p. 2
2) RIM-ONE:
- RIM-ONE contains 169 high-resolution images from three Spanish hospitals, labeled across five diagnostic categories and annotated for optic-disc segmentation.Source: 2) RIM-ONE · p. 2
3) RIGA:
- REGA contains 750 images sourced from Messidor and two Saudi clinics, with optic-disc and cup boundaries but no glaucoma diagnosis.Source: 3) RIGA · p. 2
4) REFUGE:
The section describes a 1,200-image public challenge dataset, ACRIMA, and several approaches to optic-disc and cup segmentation. Their annotations, class balance, and reported segmentation findings differ.
4) REFUGE:
- RIGA has 1,200 images, optic-disc and cup segmentation ground truth, and clinical glaucoma labels, but only 120 glaucoma images.Source: 4) REFUGE · p. 2
5) ACRIMA:
- ACRIMA has 705 centered-disc images, including 396 glaucoma and 309 normal cases, without optic-disc or cup segmentation annotations.The passage notes its relatively balanced class proportions as suitable for training deep-learning classifiers.Source: 5) ACRIMA · p. 2
B. Optic Disc and Optic Cup Segmentation
- Almazroa et al. reported 83.9% accuracy for optic-disc area and centroid marking using RIGA.Al-Bander et al. reported demographic, camera, and disease invariance, with state-of-the-art results on two datasets and competitive results on two others without training on them. Fu et al. modified faster R-CNN for joint optic-disc and cup segmentation and reported state-of-the-art results on ORIGA and SCES.Source: B. Optic Disc and Optic Cup Segmentation · p. 2
C. Glaucoma Classification
Prior glaucoma-classification studies used private RFIs or smaller public datasets and reported varying performance across neural-network and hybrid approaches.
C. Glaucoma Classification
- 95.6% accuracy, 95.5% sensitivity, and 95.7% specificity were reported by Raghavendra et al. using 1,426 private RFIs and an 18-layer DNN.The study trained and tested on private images.Source: C. Glaucoma Classification · p. 2
- 92.9% accuracy and 98.6% AUC were reported by Li et al. for referable GON detection using around 40,000 RFIs.They defined GON as vertical CDR greater than 0.7 and reported 95.6% sensitivity and 92.0% specificity.Source: C. Glaucoma Classification · p. 2
- 88.2% accuracy, 85% sensitivity, and 90.8% specificity were reported by Al-Bander et al. using DNN-extracted features and an SVM on 455 RIM-ONE v2 images.The study used a hybrid feature-extraction and classification approach.Source: C. Glaucoma Classification · p. 2
III. DATASET DESCRIPTION
G1020 comprises retinal fundus images collected retrospectively in routine clinical practice, with clinical diagnoses and expert-verified optic-structure annotations.
III. DATASET DESCRIPTION
- G1020 images were collected at a private Kaiserslautern clinical practice from 2005 to 2017 without imposed optic-disc or macula centering constraints.Images were captured with a 45-degree field of view after dilation; released fundus regions range from 1944×2108 to 2426×3007 pixels.Source: §III · p. 2
- The optic discs occupy a wider spatial area in G1020 than in ORIGA, making segmentation post-processing more challenging.The G1020 images also preserve fundus regions with varied dimensions after truncating the black background.Source: §III · p. 2, Figure 1 · p. 3
- The dataset contains 1,020 images from 432 patients: 296 glaucomatous images from 110 patients and 724 healthy images from 322 patients.No patient contributed images to both diagnostic classes.Source: §III · p. 3
- An expert marked optic-disc and optic-cup boundaries and disc bounding boxes, with annotations verified or corrected by an ophthalmologist with over 25 years of clinical experience.Vertical CDR and neuroretinal rim sizes in four quadrants were calculated from the ground-truth boundaries.Source: §III · p. 3
- Visible optic cups are absent in 60 glaucomatous images and 170 healthy images.Fig. 2 illustrates the optic-cup, optic-disc, and bounding-box annotations.Source: Figure 2 · p. 3, §III · p. 4
IV. EXPERIMENTS AND EVALUATION RESULTS
The evaluation tested Mask R-CNN segmentation on G1020 and ORIGA-trained models on G1020, reporting detection counts and parameter-error evaluation procedures.
IV. EXPERIMENTS AND EVALUATION RESULTS
- Mask R-CNN with a ResNet-50 backbone was trained separately for optic-disc and optic-cup segmentation, using either an 80/20 G1020 split or ORIGA training followed by G1020 evaluation.The evaluation considered IoU thresholds greater than 0.4, 0.5, or 0.6; non-maximum suppression retained the highest-probability contour.Source: §IV · p. 4
- With G1020 training and testing, only one image had an IoU below all three correctness criteria, and precision, recall, and F-1 were identical across them.That image had IoU 0.2689; the second-lowest IoU was 0.6429, and the single error produced one false positive and one false negative.Source: §IV · p. 4
- 786 of 791 cups and 1,005 of 1,020 discs were detected when Mask R-CNN was trained on ORIGA and tested on G1020.Precision and recall differed across the three IoU criteria for this experiment.Source: §IV · pp. 4–5
- Mean absolute percentage error was calculated for vertical CDR and four-quadrant neuroretinal rim measurements using correctly detected discs and cups at IoU greater than 0.5.Table II reports these parameter errors and their standard deviations, but the supplied passage gives no values.Source: §IV · p. 5, Table II · p. 5
A. Classification of Glaucoma
The evaluated classifiers performed well on ORIGA but struggled on G1020, which the authors associate with differences in dataset collection conditions.
A. Classification of Glaucoma
- Both classifiers performed with high precision and recall on ORIGA but struggled on G1020, a difference the authors associate with dataset collection practices.The authors note that ORIGA and many public datasets impose imaging and image-selection constraints that may limit representativeness of routine imaging.Source: A. Classification of Glaucoma · p. 5
B. Segmentation of OD and OC
The reported classifier performed competitively on ORIGA but degraded seriously on G1020, where glaucoma and healthy image embeddings overlapped substantially. The analysis compares the datasets using PCA visualisations and fold-based ROC/AUC results.
B. Segmentation of OD and OC
- The dataset comparison visualised image embeddings after PCA dimensionality reduction on a 2D plane, with glaucoma and healthy images shown in blue and red.Source: B. Segmentation of OD and OC · p. 5, Figure 4 · p. 5
- Glaucoma and healthy embeddings were fairly separable on ORIGA but overlapped substantially on G1020.The embeddings came from the classifier’s final convolutional layer.Source: B. Segmentation of OD and OC · p. 6
- The classifier achieved competitive AUC on ORIGA relative to Bajwa et al. (0.874) and Fu et al. (0.851), but suffered serious performance degradation on G1020.Fig. 5 reports individual-fold and mean ROC/AUC results for six G1020 folds and five ORIGA folds.Source: B. Segmentation of OD and OC · p. 6
V. CONCLUSION
Existing glaucoma fundus datasets are often small and collected under controlled conditions, while G1020 is presented as a publicly available dataset reflecting practical clinical imaging. The authors report that initial evaluations expose challenges for practical CAD and establish a baseline for future comparisons.
V. CONCLUSION
- Most existing glaucoma fundus datasets contain only a few hundred images and are collected in controlled environments, often excluding images with retinal artefacts [14].Source: §V · p. 6
- G1020 is a large publicly available dataset designed to reflect routine clinical fundus imaging without strict image inclusion criteria, and its initial evaluations highlight challenges for practical CAD.The authors state that their results set a baseline for future comparisons.Source: §V · p. 6
- Fig. 5 reports ROC and AUC for six-fold G1020 and five-fold ORIGA evaluations.Source: Figure 5 · p. 6