Source-linked AI summary
Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning
Ryan Poplin, Avinash V. Varadarajan, Katy Blumer, Yun Liu, Michael V. McConnell, Greg S. Corrado, Lily Peng, Dale R. Webster
TL;DR
Cardiovascular risk stratification can be hindered when relevant parameters are unavailable. This paper uses deep learning on retinal fundus photographs to predict multiple cardiovascular risk factors, including age, gender, and systolic blood pressure, suggesting retinal images may support direct cardiovascular-risk prediction.
Problem
Cardiovascular risk stratification is important, but some parameters needed for it may be unavailable.
Method
The authors developed deep learning models using retinal fundus images from large patient datasets and validated them on independent datasets.
Results
Deep learning from retinal fundus images alone predicted multiple cardiovascular risk factors, including age, gender, and systolic blood pressure.
Takeaways & Limitations
The findings provide preliminary support that retinal images may enable direct cardiovascular-risk prediction.
Takeaways & Limitations
The authors note that the dataset size was relatively small for deep learning.
Abstract
from arXiv · showhide
Traditionally, medical discoveries are made by observing associations and then designing experiments to test these hypotheses. However, observing and quantifying associations in images can be difficult because of the wide variety of features, patterns, colors, values, shapes in real data. In this paper, we use deep learning, a machine learning technique that learns its own features, to discover new knowledge from retinal fundus images. Using models trained on data from 284,335 patients, and validated on two independent datasets of 12,026 and 999 patients, we predict cardiovascular risk factors not previously thought to be present or quantifiable in retinal images, such as such as age (within 3.26 years), gender (0.97 AUC), smoking status (0.71 AUC), HbA1c (within 1.39%), systolic blood pressure (within 11.23mmHg) as well as major adverse cardiac events (0.70 AUC). We further show that our models used distinct aspects of the anatomy to generate each prediction, such as the optic disc or blood vessels, opening avenues of further research.
INTRODUCTION
Cardiovascular risk stratification is important but often depends on incompletely available clinical variables. This work proposes extracting and quantifying cardiovascular risk factors from quickly obtainable, noninvasive retinal images using deep learning.
- Motivation: Risk calculators use clinical variables such as age, gender, smoking status, blood pressure, BMI, glucose, and cholesterol to identify cardiovascular risk.These variables may be unavailable in practice, limiting individual risk assessment.
- Motivation: Less than 30% of patients had the data required to calculate 10-year risk scores in a PINNACLE registry study, largely because cholesterol values were missing.BMI can substitute for lipids in preliminary cardiovascular assessment.
- Motivation: Retinal fundus images offer a quick, cheap, and noninvasive source of cardiovascular signals because retinal vessels and disease manifestations can reflect systemic cardiovascular health.Candidate retinal features include vessel caliber, bifurcation, and tortuosity, although their clinical utility requires further study.
- Contribution: The study demonstrates extraction and quantification of multiple cardiovascular risk factors from retinal images using deep learning.Deep learning learns predictive features from examples rather than requiring hand-engineered features.
Results
Deep learning models trained on 284,335 retinal fundus images and validated in two independent cohorts predicted cardiovascular risk factors, ethnicity, and 5-year major adverse cardiac events. Attention analyses linked predictions to distinct retinal regions, including blood vessels, perivascular surroundings, and the optic disc.
- Datasets: 284,335 patients contributed training images, with validation on 12,026 UK Biobank and 999 EyePACS patients.Training used 48,101 UK Biobank and 236,234 EyePACS patients.
- Risk-factor prediction: 3.42 (3.23, 3.61) MAE predicted age in EyePACS-2K versus baseline: 8.48 (8.07, 8.90).Predicted and actual age showed a fairly linear relationship consistent across both datasets.
- Risk-factor prediction: 0.60 (95% CI: 0.58-0.63) and 0.75 (0.70-0.79) kappa scores measured ethnicity prediction in UK Biobank and EyePACS-2K, respectively.Retinal images also predicted systolic blood pressure better than baseline, with predictions leveling off above approximately 150 mmHg.
- Anatomical signals: Blood vessels were highlighted for age, smoking, and SBP; perivascular surroundings for HbA1c; and the optic disc for gender.Attention was non-specific for diastolic blood pressure and BMI, suggesting more diffusely distributed signals.
Discussion
Deep learning of retinal fundus images predicted multiple cardiovascular risk factors and showed preliminary potential for direct cardiovascular risk prediction. Attention maps and validation across two datasets support vascular mechanisms and possible generalization, while limited dataset size and clinical validation constrain confidence.
- Deep learning of retinal fundus images alone can predict multiple cardiovascular risk factors, including age, gender, and systolic blood pressure.
- Because these factors are core components of cardiovascular risk calculators, the model may potentially predict cardiovascular risk directly, supported by preliminary MACE results.
- Attention maps indicate that the neural network uses retinal vascular regions to predict several variables associated with cardiovascular risk.
- Consistent results across two separate validation datasets suggest that the predictions may generalize and indicate pathological phenomena for further study.
- The relatively small dataset, wide confidence intervals, unavailable risk factors across datasets, and need for additional validation limit confidence in the models.
- Larger datasets and more clinical validation could clarify whether retinal images augment or replace markers such as lipid panels and improve cardiovascular risk stratification.
METHODS · Study participants
The study combined UK Biobank and EyePACS retinal imaging datasets, developed Inception-v3 models for multiple cardiovascular risk-factor predictions, and evaluated them using task-specific metrics. Statistical significance was assessed by bootstrap confidence intervals, while soft-attention models generated explanatory heatmaps.
- Model development: The investigators trained Inception-v3 models to predict age, gender, smoking status, BMI, systolic and diastolic blood pressure, and HbA1c.Binary and continuous risk factors were modeled separately to keep loss functions on consistent scales.
- Model development: 10 neural network models were averaged through ensembling, while early stopping used tuning subsets comprising 10% of UK Biobank and 2.1% of EyePACS.Training and evaluation used TensorFlow.
- Evaluating the algorithm: Mean absolute error evaluated continuous predictions, AUC evaluated gender and smoking-status classification, and Cohen’s kappa evaluated multiclass classification.The metric was selected according to prediction type.
- Statistical Analysis: 2,000 bootstrap resamples of each validation set produced performance distributions, with the 2.5 and 97.5 percentiles reported as 95% confidence intervals.Sampling was performed with replacement from N validation-set patients.
- Mapping Models’ Attention: Soft-attention models with fewer parameters than Inception-v3 were used to generate attention heatmaps explaining predictions.These models were not used for the best-performing results.
- Mapping Models’ Attention: Separate soft-attention models were trained for each prediction using the same training data and early-stopping criteria as Inception-v3.Each model had an identical architecture.
HbA1c BMI
The model’s retinal predictions were examined using green soft-attention heatmaps, with doctor agreement on highlighted features quantified across risk factors. The figure presents these visualizations for a UK Biobank retinal image, while Table 4 describes the ophthalmologist assessment procedure.
- Attention visualization: Green soft-attention heatmaps indicate the retinal areas used by the neural network for each prediction.The figure shows a sample image in color and black-and-white with the heatmap overlaid.
- Attention visualization: 100 heatmaps were generated for each risk factor and evaluated by 3 ophthalmologists for highlighted retinal features.The images were shuffled into a set of 800, and ophthalmologists were not told what the heatmaps were intended to explain.
- Attention visualization: The percentage of heatmaps receiving doctor agreement for each highlighted feature was quantified, with values above 75% bolded.The quantitative analysis of highlighted features is reported in Table 4.