Source-linked AI summary
A Benchmark of Ocular Disease Intelligent Recognition: One Shot for Multi-disease Detection
Ning Li, Tao Li, Chunyu Hu, Kai Wang, Hong Kang
TL;DR
Fundus screening needs scalable multi-disease recognition because manual diagnosis is time-consuming and many existing systems focus on single diseases. The paper releases and benchmarks a binocular dataset with eight disease annotations, finding that network scaling alone is insufficient and that integrated feature or model strategies are needed. The dataset remains limited by image quantity and severe category imbalance.
Problem
Existing ocular-disease systems and datasets often focus on single diseases, although clinical fundus screening commonly involves multiple diseases.
Method
The paper releases a dataset of 10,000 binocular fundus images from 5,000 patients with annotations for eight diseases and benchmarks deep neural networks on it.
Results
Simply increasing neural-network depth alone cannot improve performance, whereas increasing width and integrating multiple deep networks can help classification performance.
Takeaways & Limitations
The dataset provides a benchmark and supports further research on clinical multi-disease fundus classification.
Takeaways & Limitations
The dataset has only 10,000 images and serious category imbalance, so more fundus images are needed for safer, more accurate clinical generalization.
Abstract
from arXiv · showhide
In ophthalmology, early fundus screening is an economic and effective way to prevent blindness caused by ophthalmic diseases. Clinically, due to the lack of medical resources, manual diagnosis is time-consuming and may delay the condition. With the development of deep learning, some researches on ophthalmic diseases have achieved good results, however, most of them are just based on one disease. During fundus screening, ophthalmologists usually give diagnoses of multi-disease on binocular fundus image, so we release a dataset with 8 diseases to meet the real medical scene, which contains 10,000 fundus images from both eyes of 5,000 patients. We did some benchmark experiments on it through some state-of-the-art deep neural networks. We found simply increasing the scale of network cannot bring good results for multi-disease classification, and a well-structured feature fusion method combines characteristics of multi-disease is needed. Through this work, we hope to advance the research of related fields.
1 Introduction
Fundus diseases can cause irreversible blindness, yet large-scale screening is difficult because manual diagnosis is time-consuming and depends on limited expertise. The paper addresses the limits of single-disease systems by releasing a binocular, multi-disease dataset and benchmarking classification networks.
- Fundus diseases are serious public health problems because they can cause irreversible blindness, making early detection important.
- Manual fundus screening is time-consuming and depends heavily on ophthalmologists’ experience, hindering large-scale screening.
- Accurate automated recognition is difficult because lesions can be tiny, low-contrast, irregularly shaped, and variable across cameras.
- Many existing models and datasets focus on only one ophthalmic disease, despite clinical needs for comprehensive multi-disease screening.
- The released dataset contains 5,000 binocular image pairs, 10,000 images, and annotations for eight diseases, supporting multi-disease classification research.
- The work benchmarks nine deep neural networks on the dataset to establish a reference for clinical multi-disease classification research.
2 Related Work
Prior ocular-disease recognition research spans OCT and fundus images, but public datasets and multi-disease coverage remain limited. Existing systems often target single diseases or require redesigned architectures and handcrafted features, motivating a broader fundus benchmark.
- OCT supports detailed retinal examination and has been used for segmentation, detection, and classification, but public OCT datasets are scarce.
- Compared with OCT, color fundus images require less demanding acquisition equipment and are easier to obtain for clinical research.
- Prior fundus studies include image-quality classification, cataract grading, multi-disease recognition, and multilabel classification on several datasets.
- Single-disease systems may require structural redesign for other diseases, and handcrafted features increase operational complexity and reliance on prior knowledge.
- Many existing fundus datasets are too small or disease-specific, while available multi-disease datasets contain relatively few images and disease types.
3 Our Dataset
OIA-ODIR is a large binocular, multi-disease fundus dataset designed for clinical-style ocular disease recognition. It combines eight disease annotations, patient metadata, multi-center images, expert annotation, and separate training, validation, and on-site testing sets, while exhibiting serious class imbalance.
- Image Collection and Annotation: Images were collected from clinical databases spanning 487 hospitals in 26 Chinese provinces and cover abnormalities across the macula, optic cup, optic disc, blood vessels, and fundus background.The source database contained more than 1.6 million images before dataset selection.
- Image Collection and Annotation: OIA-ODIR contains 10,000 fundus images from 5,000 patients, with images from both eyes and annotations for eight disease categories.The categories are normal, DR, glaucoma, cataract, AMD, hypertension, myopia, and other diseases.
- Image Collection and Annotation: Three ophthalmologists with more than two years’ experience and three with more than ten years’ experience completed annotation and arbitration over approximately ten months.The annotation process followed stated standards and procedures, with multiple annotators recording results before resolving disagreements.
- Features of Dataset: The dataset addresses a gap in which few public resources support detecting multiple ophthalmic diseases in one eye or across binocular fundus images.The authors position OIA-ODIR as a large-scale binocular multi-disease dataset intended to better reflect clinical diagnosis.
- Features of Dataset: OIA-ODIR provides binocular images together with age, gender, and ophthalmologists’ diagnostic keywords, enabling multi-disease classification and related diagnostic or prediction studies.The dataset is intended to support patient-level multi-label classification using both eyes, while preserving each eye’s characteristics.
- Split of Dataset: The dataset is split into training, off-site test, and on-site test sets, but category frequencies are seriously imbalanced, with hypertension images below one tenth the number of normal images.The on-site test set is used to evaluate deep-network generalization, while the imbalance complicates multi-label disease recognition.
4 Multi-Disease Classification
The study benchmarks nine deep networks for binocular multi-disease fundus classification using OIA-ODIR, evaluating feature fusion and error patterns. Results indicate that network scaling alone is insufficient, while simple concatenation underperforms and more structured fusion is needed.
- Experimental setup: Nine deep convolutional networks were evaluated on the Off-site and On-site testsets to establish benchmark performance on OIA-ODIR.The models include Vgg-16, ResNet variants, ResNeXt-50, SE-ResNet-50, SE-ResNeXt-50, Inception-v4, Densenet, and CaffeNet.
- Network structure: The network takes paired left- and right-eye fundus images, extracts their features, fuses them, and feeds the fused representation into eight classifiers.This binocular design reflects patient-level multi-disease annotation and classification.
- Evaluation: Performance was evaluated with Kappa, F1, AUC, and Final-score, whose value is the mean of the other three metrics.The experiments also report Off-site and On-site results separately.
- Experiment analysis: Vgg-16 outperformed deeper convolutional networks, indicating that simply increasing network depth did not improve this multi-disease task.The authors relate wider architectures and attention mechanisms to potentially useful feature combinations under some conditions.
- Feature fusion: Element-wise sum and multiplication produced similar results across models, whereas simple feature concatenation performed worse and motivated more structured fusion.The study therefore identifies feature-fusion design, rather than network scale alone, as an important direction.
- Error analysis: Misclassification analysis identified image-quality problems, interference among multiple diseases, and insufficiently visible local features as recurring challenges.The cited examples include lighting and lens stains, confusion among abnormalities, and the need for detailed optic-disc or macular features.
5 Discussion
The dataset supports multi-disease screening and related clinical research, while posing challenges from binocular inputs, disease-specific features, interacting diseases, and substantial image diversity. Its main limitation is that 10,000 images may be insufficient for reliable clinical generalization.
- Challenges: Binocular fundus-image learning must account for the spatial and structural correlation between the two eyes.
- Challenges: Different diseases require feature extraction suited to their local or global characteristics, and interacting diseases are difficult to identify correctly.Examples include optic cup-to-disc size for glaucoma and fundus-structure clarity for cataracts.
- Challenges: Wide image sources create substantial intra-class variation in color, lighting, lens conditions, and the heterogeneous Other category.
- Potential Applications: The dataset can support multi-disease screening, age or gender correlation studies, fundus-image captioning, and multimodal disease-identification models.Diagnostic keywords enable captioning, while graph neural networks could integrate unstructured data.
- Deficiency: 10,000 images may not meet clinical application needs, and more fundus images are needed to improve model generalization and diagnostic safety.
6 Conclusions
The paper releases a multi-disease fundus-image benchmark and evaluates existing deep learning models on it. Results indicate that network depth alone does not improve performance, whereas increased width and integrating multiple networks can help.
- 10,000 images from the left and right eyes of 5,000 patients form a multi-disease benchmark dataset with rich diversity.
- Increasing neural-network depth alone cannot improve performance, while increasing width can bring certain improvement.
- Integrating multiple deep networks is helpful for improving classification performance.