Source-linked AI summary
Towards Adversarial Retinal Image Synthesis
Pedro Costa, Adrian Galdran, Maria Inês Meyer, Michael David Abràmoff, Meindert Niemeijer, Ana Maria Mendonça, Aurélio Campilho
TL;DR
High-quality medical image synthesis remains challenging, and clinical annotations are often costly to obtain. This paper learns to generate retinal images from vessel trees using adversarial image-to-image translation. Generated images share vascular structure with their inputs but differ in global appearance, while quality scores indicate similar ISC quality and lower Qv quality than real images.
Problem
High-quality medical image generation remains challenging, while clinical annotations are often costly to obtain.
Source: §1 · p. 1Method
The method segments retinal images into vessel trees and uses adversarial image-to-image translation to learn a mapping from trees to retinal images.
Source: §1 · p. 2, §2.2 · p. 4Results
Synthetic images shared vessel structure with their inputs but differed in color, tone, and illumination; ISC quality differences were not significant (p = 0.2188), while real images scored better on Qv (p < 0.05).
Source: §3 · pp. 5–6Takeaways & Limitations
The authors demonstrate feasibility at 512 × 512 resolution and attribute this resolution to retinal images’ repetitive geometry and recurring high-level structures.
Source: §4 · p. 6Image generation depends on a pre-existing vessel tree, and weaknesses in the segmentation algorithm can carry into the synthesized image.
Source: §4 · p. 6
Abstract
from arXiv · showhide
Synthesizing images of the eye fundus is a challenging task that has been previously approached by formulating complex models of the anatomy of the eye. New images can then be generated by sampling a suitable parameter space. In this work, we propose a method that learns to synthesize eye fundus images directly from data. For that, we pair true eye fundus images with their respective vessel trees, by means of a vessel segmentation technique. These pairs are then used to learn a mapping from a binary vessel tree to a new retinal image. For this purpose, we use a recent image-to-image translation technique, based on the idea of adversarial learning. Experimental results show that the original and the generated images are visually different in terms of their global appearance, in spite of sharing the same vessel tree. Additionally, a quantitative quality analysis of the synthetic retinal images confirms that the produced images retain a high proportion of the true image set quality.
1 Introduction
High-quality synthetic medical imagery remains difficult to produce, while clinical annotations are often costly and large datasets are rarely available. This work learns to generate retinal images from vessel trees, constraining the synthesis task to retinal content not specified by the tree.
1 Introduction
- High-quality medical image generation remains an unsolved challenge, and clinical annotations are often costly to obtain.The introduction frames synthetic medical imagery as appealing because large training datasets are rarely available.Source: §1 · p. 1
- Earlier medical-image methods used anatomical models, which could produce realistic images for validation, training, and other applications.The paper contrasts these top-down models with directly learning relevant information from data.Source: §1 · pp. 1–2
- Adversarial learning had advanced natural-image synthesis, but its application to medical imaging remained incipient amid limited data and difficulty controlling outputs.Source: §1 · p. 2
- The proposed method learns to generate plausible retinal images from binary vessel trees, leaving other content such as the optical disk and background texture to synthesize.The paper presents this as a constrained alternative to generating images from scratch.Source: §1 · p. 2
2 Adversarial Retinal Image Synthesis
The method learns a mapping from vessel trees to retinal images using adversarial training with an L1 term, and obtains paired training examples by segmenting retinal images. The implementation trains the segmenter and generator on distinct datasets and uses 512 × 512 image pairs.
2.1 Adversarial Translation from Vessel Trees to Retinal Images
- A vessel tree can correspond to multiple acceptable retinal images, while naively minimizing L2 distance can average them and lose detail.The method combines adversarial learning with a global L1 loss: the L1 term controls low-frequency information, while the adversarial loss promotes sharp results.Source: §2.1 · p. 3
2.2 Obtaining Training Data
- Because a large manually annotated set of vessel trees and images was unavailable, the authors applied U-Net segmentation to create paired training data.U-Net uses downsampling and upsampling paths with concatenated corresponding features; its final convolution and sigmoid classify pixels as vessel or non-vessel.Source: §2.2 · pp. 3–4
2.3 Implementation
- The U-Net was trained on DRIVE patches and achieved a 0.9755 AUC on DRIVE’s test set; it was then applied to cropped, downscaled Messidor images.Some cross-dataset segmentations were imperfect, and the authors retained only pairs whose images had diabetic retinopathy grades 0, 1, or 2.Source: §2.3 · p. 4
- The adversarial dataset comprised 946 Messidor image pairs, split into 614 training, 155 validation, and 177 test pairs.The generator was modified for 512 × 512 images, while the discriminator classified overlapping 63 × 63 patches; generator and discriminator training alternated until their losses stabilized.Source: §2.3 · p. 4
3 Experimental Evaluation
The generated images retained shared vascular geometry with the real images but differed in color, tone, and illumination. Quality assessments were mixed across metrics, and six artifact-bearing cases were excluded from the ISC analysis.
3 Experimental Evaluation
- Generated images shared global geometric characteristics with their source images but differed in color, tone, and illumination.The model applied appearance information learned from the training set to the input vessel trees.Source: §3 · p. 5
- 6 of 177 test images showed artifacts when defocused inputs produced vessel trees with too little information for the generator.The other seven illustrated examples behaved as expected when their segmented vessel trees were approximately correct.Source: §3 · p. 5
- Because undercomplete-tree artifacts could artificially raise ISC scores, artifact-bearing cases and their real counterparts were removed from ISC analysis.The passage reports that this ISC issue did not arise with Qv.Source: §3 · p. 5
- p = 0.2188 for ISC: synthetic images scored slightly higher without significance; Qv favored real images significantly (p < 0.05).The ISC analysis used a reduced test set of 171 images, while Qv used all 177.Source: §3 · p. 6, Table 1 · p. 6
4 Conclusions and Future Work
The results support the feasibility of synthesizing retinal images from paired vessel trees and images, at 512 × 512 resolution. The method depends on a pre-existing vessel tree, so segmentation weaknesses can carry into its output.
4 Conclusions and Future Work
- The method demonstrates the feasibility of learning retinal-image synthesis from paired vessel trees and images, producing 512 × 512 images.The authors attribute the achievable resolution to retinal images' repetitive geometry and recurring high-level structures that guide texture and background generation.Source: §4 · p. 6
- Generating a new image requires a pre-existing vessel tree.This is identified as the method's main limitation.Source: §4 · p. 6
- When the input tree comes from segmentation, weaknesses in that algorithm can be inherited by the synthesized image.The authors state that they are working to overcome these challenges.Source: §4 · p. 6
A.1 Image Quality Metrics
The section describes two complementary no-reference quality metrics: Qv emphasizes vessel pixels, while ISC assesses broader retinal structures. Undercomplete-tree artifacts can distort ISC scores, and the authors note that anatomical plausibility may need purpose-designed metrics.
A.1 Image Quality Metrics
- Qv combines vesselness and local anisotropy around vessel pixels to estimate retinal image quality.It derives vesselness from multiscale green-channel Hessian information and weights it with local anisotropy.Source: §A.1 · p. 7
- ISC uses learned image features and clustered intensity distributions to assess whether retinal structures have appropriate pixel proportions.Its expert-labeled training set contained 1000 images, and its five clusters represent vessels, optic disk, macula, and two transition regions.Source: §A.1 · p. 7
- ISC covers image regions beyond Qv's vessel focus, but undercomplete-tree artifacts can raise ISC scores artificially.The authors observed this effect in six of 177 test images and excluded those cases from ISC statistical experiments; Qv was unaffected.Source: §A.1 · p. 7, Figure 5 · p. 7
- The authors consider current metrics reasonably suitable for visual quality, while noting that anatomical plausibility may benefit from metrics designed for that purpose.They suggest such metrics may combine local and global aspects of existing quality-assessment approaches.Source: §A.1 · p. 7
A.2 Supplementary Results
The supplementary results show generated retinal images alongside real counterparts and their vessel trees, with Qv and ISC scores included for comparison.
A.2 Supplementary Results
- A random sample of model outputs is shown alongside their real counterparts.Source: §A.2 · p. 8
- Each block aligns a segmented vessel tree with its corresponding real image and synthetic image, with Qv and ISC scores below.Source: Figure 6 · p. 8
- Figure 7 presents further results; the displayed images are 512 × 512 pixels.Source: §A.2 · p. 9, Figure 7 · p. 9