PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning in Cardiology

Paschalis Bizopoulos, Dimitrios Koutsouris

arXiv:1902.11122v5cs.CVcs.AIcs.LG

TL;DR

Cardiology produces large, difficult-to-use datasets, while rule-based systems struggle with complex tasks and scaling. This review surveys deep-learning applications across structured data, signals, and imaging, organizing evidence by architecture and modality. The reviewed literature reports promising performance but also highlights data, interpretability, and scalability constraints before clinical use.

  • Problem

    Cardiovascular data are increasingly abundant and complex, while conventional workflows and rule-based expert systems are inefficient for difficult tasks and large-scale data use.

  • Method

    The review searches and curates cardiology deep-learning studies across structured data, signals, and imaging, organizing them by neural-network architecture and training-data modality.

  • Results

    Deep-learning applications span cardiology tasks, with reported results including heart-failure AUCs of 0.777 for a 12-month window and 0.883 for an 18-month window, and fundus-photography results exceeding 70% AUC for several predictions.

  • Takeaways & Limitations

    The review identifies deep learning as a promising basis for cardiology applications, while emphasizing interpretable models and large datasets as important for successful clinical application.

  • Takeaways & Limitations

    Deep learning in medicine is constrained by costly expert labeling, class imbalance, data-quality and standardization problems, and limited annotated datasets.

Abstract

from arXiv · show

The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction and intervention. Deep learning is a representation learning method that consists of layers that transform the data non-linearly, thus, revealing hierarchical relationships and structures. In this review we survey deep learning application papers that use structured data, signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.

I. INTRODUCTION

Cardiology generates increasingly large and complex datasets that traditional clinical workflows and rule-based systems struggle to use efficiently. This review introduces deep learning and surveys its cardiology applications across structured data, signals, and imaging.

  • Clinical motivation: 30% of deaths in the United States in 2014 were attributed to cardiovascular diseases, while conventional diagnosis relies on subjective interpretation of patient data and literature.The cited passage also reports 45% of deaths in Europe and €210 billion in annual European Union costs.
  • Clinical motivation: Rule-based expert systems are difficult to scale because they require substantial feature engineering and domain knowledge to achieve adequate accuracy.Their limitations become more consequential as medical technologies produce larger quantities of data.
  • Deep learning foundations: Deep learning uses multiple stacked layers to learn hierarchical abstractions from data and has been applied to tasks including image classification, segmentation, language processing, speech recognition, and genomics.Its emergence is linked to increased GPU computational power and the availability of big data.
  • Review scope and method: The review searches cardiology and deep-learning terminology across Google Scholar, PubMed, and Scopus, then organizes eligible studies by architecture and data modality.It covers structured data, signals, and imaging related to heart and vessel structures, excluding papers without architectural information, duplicate methods, or preliminary work.
  • Deep learning foundations: Neural networks learn parameters such as weights and biases by minimizing a task-specific cost function, with separate training, validation, and test datasets used for fitting, selection, and generalization evaluation.Backpropagation computes cost-function gradients, while the final activation depends on the research question.
  • Evaluation: Cross entropy, AUC, and Dice coefficient are examples of task-dependent cost or evaluation measures used for classification and segmentation problems.The review explains that the chosen function depends on the task being solved.

B. Architectures overview

The review outlines several neural-network architectures with different roles in feature extraction, temporal processing, reconstruction, and function approximation. CNNs exploit local relationships, whereas RNN variants retain information across time.

  • Fully Connected Networks: Fully Connected Networks stack perceptrons across width and depth, but are not computationally efficient for fitting complex functions despite universal approximation results.The passage contrasts their theoretical expressiveness with their computational cost.
  • Convolutional Neural Networks: CNNs use convolutional layers for hierarchical feature extraction and fully connected layers for classification or regression.Convolutional layers exploit local relationships and generate feature maps.
  • Autoencoders: Autoencoders learn to reconstruct their inputs through encoding and decoding stages that compress and restore data representations.Stacked denoised autoencoders reconstruct clean inputs from artificially corrupted versions.
  • Recurrent Neural Networks: RNNs process inputs using feedback loops and internal state, while LSTMs address vanishing gradients and GRUs provide a simpler alternative to LSTMs.These mechanisms support retaining information over extended time.

III. DEEP LEARNING USING STRUCTURED DATA

Deep learning has been applied to structured cardiology data, especially EHRs, using recurrent, autoencoder, and belief-network architectures for diagnosis and prediction. Applications remain limited by scarce public databases and the restricted individual-level information in structured records.

  • Structured cardiology data mainly consist of organized EHR fields, with applications summarized in a dedicated review table.
  • RNNs, including LSTM and GRU models, predict perioperative outcomes and cardiovascular conditions from temporal clinical records.A GRU achieved AUC 0.777 for a 12-month heart-failure window and 0.883 for an 18-month window.
  • Deep learning models outperformed shallow-learning comparators across several structured-data prediction tasks, including mortality and ICD-9-code prediction.Reported AUCs included 0.873 for in-hospital mortality, 0.871 for short-term mortality, 0.87 for long-term mortality, and 0.777 for ICD-9-code prediction.
  • Autoencoders and stacked autoencoders support cardiovascular risk analysis and acute coronary syndrome prediction using demographic, clinical, environmental, and EHR variables.
  • DBNs have been used to predict hypertension, hyperlipidemia, myocardial infarction, and angina pectoris from demographic and lifestyle variables, outperforming several baseline models.
  • Structured-data applications are relatively few because public databases are scarce, limiting evaluation and architecture comparison; these records are more suited to epidemiologic studies than individual cardiology.

IV. DEEP LEARNING USING SIGNALS

Deep learning applications use ECG, PCG, and other time-series signals for arrhythmia and cardiac-condition tasks, commonly combining denoising, beat segmentation, and neural feature extraction. CNNs, autoencoders, DBNs, and RNNs have produced strong reported results across public databases.

  • Cardiology signal modalities include ECGs, PCGs, oscillometric data, and wearable data, with deep learning useful when annotations are noisy or handcrafted modeling is difficult.
  • ECG is non-invasive and widely available, making it especially useful for arrhythmia detection across public databases.
  • CNN, autoencoder, DBN, and RNN approaches have been applied to arrhythmia classification using preprocessing pipelines tailored to ECG noise and heartbeat structure.
  • 97.5% overall accuracy was achieved by an SDAE-based classifier for four arrhythmias, while a DBN-based method achieved 99.5% accuracy on five MITDB classes.
  • Reported results vary substantially in metric and task definition, including accuracy, AUC, precision, signal-to-noise ratio, and noise-resilience settings.

3) AF detection:

Deep learning methods for atrial-fibrillation detection use CNNs and recurrent architectures applied to heart-rate, spectrogram, RR-interval, and ECG representations. Reported studies generally show improved or comparable performance against prior methods and baselines.

  • A multi-scale CNN using instantaneous heart-rate sequences achieved better AF-detection accuracy than previous methods.
  • CNN comparisons found slightly better AF-detection accuracy when stationary wavelet transforms were used instead of short-term Fourier transforms.
  • An LSTM-based network classified segmented RR intervals and achieved accuracy indicating generalizability across MITDB and NSRDB validation data.
  • AF-related signal studies also include ECG beat classification, myocardial-infarction detection, and fetal-QRS identification using CNNs, autoencoders, and wavelet-based features.

5) Other tasks with private databases:

Deep learning has also been applied to abnormal ECG referral, ventricular-contraction classification, stress detection, and phonocardiogram-based heart-sound tasks. These studies frequently transform signals into spectrograms and use CNNs for classification.

  • RBM-based models classified abnormal ECGs and supported referral decisions, outperforming KNN, SVM, extreme-learning-machine, and expert-system comparators in accuracy and specificity.
  • Premature-ventricular-contraction and stress-detection studies trained CNN, Inception, and other models on waveform images or one-dimensional signals.
  • Across ECG applications, common pipelines use denoising, R-peak detection, beat segmentation, and neural feature extraction, often after converting signals to images.
  • The PHY16 heart-sound challenge contained 3,126 phonocardiograms from five databases, with recordings lasting 5 to 120 seconds.
  • Most PCG methods convert signals into spectrograms, while CNNs classify the resulting representations for normal-versus-abnormal heart sounds.
  • 79.5% overall accuracy was achieved by a four-layer 1D CNN using segmented PCG signals in the official PHY16 phase.
  • PCG studies also use MFCC clustering and DBNs for S1 and S2 recognition, with reported performance better than KNN, Gaussian-mixture, logistic-regression, and SVM methods.
  • Oscillometric signals support blood-pressure estimation and Korotkoff-sound analysis through DBN ensembles and CNN-based spectrogram processing.

2) Data from wearable devices:

Wearable cardiology data support deep-learning applications for atrial-fibrillation detection and cardiovascular prediction. MRI-based cardiac imaging applications use diverse CNN, u-net, recurrent, atlas-based, and regression methods for segmentation and related tasks.

  • Data from wearable devices: 91.8% accuracy was achieved for atrial-fibrillation detection using wrist-worn ECG, PPG, and accelerometry with a five-layer CNN.The model combined short-window CNN outputs with beat-to-beat variability and signal-quality features.
  • Data from wearable devices: Heart response to physical activity was identified as a salient biomarker for predicting disease onset in a semi-supervised multi-task LSTM study of 14,011 app users.The study targeted diabetes, high cholesterol, high blood pressure, and sleep apnoea.
  • MRI applications: MRI cardiac applications include ventricular segmentation, whole-heart segmentation, anatomical detection, frame identification, infarction detection, and ventricular quantification.The reviewed approaches combine CNNs and u-nets with RNNs, autoencoders, atlas methods, regression, and other components.
  • MRI applications: MRI studies report CNN-based localization and segmentation results across datasets including YUDB, SUN09, STA11, DS16, AC17, HVS16, and MM17.Reported examples include 98.66% for LV localization, 92% and 96% for segmentation tasks, and 90%, 85% for multi-planar whole-heart results.
  • MRI applications: MRI methods also address atrial fibrosis, missing apical and basal slices, end-diastolic and end-systolic frame detection, myocardial infarction detection, and full LV quantification.The reviewed methods use multi-atlas propagation, SSAE, CNN-RNN models, Fast R-CNN, LSTM, SAE, and Bayesian frameworks.

2) LV/RV segmentation:

Deep-learning methods for LV/RV segmentation use semi-supervised CNNs, FCNs, u-net variants, regression frameworks, and architectures with skip connections. Studies emphasize accuracy, efficient inference, and comparisons with expert or established methods.

  • LV/RV segmentation: The ACDC dataset contains MRI images from 150 patients across five clinical groups for LV/RV and myocardium segmentation.The groups include normal cases, previous myocardial infarction, dilated cardiomyopathy, hypertrophic cardiomyopathy, and abnormal RV.
  • LV/RV segmentation: A multi-resolution conv-deconv grid architecture outperformed vanilla conv-deconv and u-net by an average of 5% in Dice.The model included cardiac center-of-mass regression, shape-prior registration, and a cardiac-anatomy-specific loss.
  • LV/RV segmentation: Semi-supervised segmentation improved performance by incorporating unlabeled data alongside labeled data.The architecture adapted VGG-16 and DeepLab and refined the final segmentation with a conditional random field.
  • LV/RV segmentation: FCNs, skip-connected architectures, and u-net variants were applied to ventricular segmentation, with FastVentricle designed for faster, lower-memory inference.One reported u-net variant performed better than an expert cardiologist, especially for RV segmentation.
  • LV/RV segmentation: A regression segmentation framework represented ventricular boundaries with points predicted from DAISY features using a DBN.The framework evaluated the regression model using alternative image features including GIST and pyramid histograms.

3) Whole heart segmentation:

Whole-heart segmentation studies use CNN and FCN architectures on small cardiac MRI and multimodal MRI/CT datasets. The reviewed methods combine dilation, deep supervision, localization, multi-plane processing, transfer learning, and hybrid losses.

  • Whole heart segmentation: The HVSMR dataset contains MRI images from 20 patients for whole-heart segmentation.A ten-layer dilated CNN segmented myocardium and blood pools in axial, sagittal, and coronal slices, while deep supervision addressed vanishing gradients and training efficiency.
  • Whole heart segmentation: The MM17 dataset contains 120 multimodal cardiac MRI/CT images for whole-heart localization and segmentation.A localization CNN identified the heart bounding-box center before a segmentation CNN focused on the corresponding region.
  • Whole heart segmentation: FCN-based methods used multi-plane processing, 3D operators, transfer learning, deep supervision, and hybrid losses to delineate cardiac structures.The hybrid loss was designed to balance classes and preserve boundary details.

4) Other tasks:

Beyond segmentation, MRI deep learning addresses anatomical and functional cardiac measurements, while fundus imaging supports retinal-vessel segmentation. The reviewed methods combine specialized preprocessing, multimodal architectures, and representation-learning strategies.

  • MRI tasks: MRI methods detect atrial fibrosis, missing apical and basal slices, and end-diastolic and end-systolic frames.Reported approaches include atlas propagation with SSAE, paired CNNs, and CNN-RNN temporal regression networks.
  • MRI tasks: A multi-stage MRI model detected myocardial-infarction area, position, and shape, while another CNN-LSTM-Bayesian framework performed full LV quantification.The latter used CNN cardiac representations, parallel LSTMs for temporal dynamics, multitask relationships, and softmax classification.
  • MRI tasks: A variational autoencoder identified hypertrophic cardiomyopathy in a balanced multi-center dataset of 1,365 patients and visualized pathology-specific remodeling patterns.The visualization and quantification of learned patterns increased model interpretability in the reported study.
  • Overall MRI findings: MRI architectures often depend on preprocessing, handcrafted features, active contours, level sets, or other non-differentiable components rather than being fully end-to-end.The review identifies end-to-end models and more efficient architectures as a main direction for future work, even if short-term accuracy is lower.
  • Overall MRI findings: For whole-heart segmentation, slice-by-slice 2D processing was beneficial because of large slice thickness, while network architecture had a minor effect.This finding came from comparisons of state-of-the-art 2D and 3D CNN architectures and their modifications.
  • Fundus imaging: Fundus-vessel segmentation methods use CNNs, autoencoders, u-nets, CRFs, Gabor filters, residual autoencoders, deep supervision, and transfer learning.PixelBNN was reported as more than eight times faster than previous state-of-the-art methods at test time despite image resizing.

2) Microaneurysm and hemorrhage detection:

Deep learning was applied to retinal vascular analysis, including microaneurysm detection, vessel classification, junction identification, and cardiovascular risk prediction. CNN-based methods dominated, while augmentation effects varied by task.

  • A three-layer CNN with dropout and maxout achieved state-of-the-art results for microaneurysm detection on ROC and DIA datasets.
  • A four-layer CNN classified retinal vessel pixels as arteries or veins before constructing a graph of the vascular network.
  • Rotational and scaling data augmentation did not improve vessel-classification accuracy, possibly because interpolation altered pixel intensities.
  • ResNet18 models identified bifurcation or crossing patches and then distinguished the type of vessel junction.
  • Poplin et al. used Inception v3 with soft attention to predict cardiovascular risk factors and major cardiac events, with most results exceeding 70% AUC.
  • CNNs were especially common for vessel segmentation, while conditional random fields were used by some studies for postprocessing refinement.

C. Computerized tomography

Deep learning applications in CT span coronary calcium assessment, cardiac and vascular segmentation, anatomical localization, and stenosis-related classification. Reported results include strong correlations and segmentation performance, but the review notes inconsistent evaluation across imaging studies.

  • CT applications included coronary calcium scoring, cardiac-area localization and segmentation, and detection of obstructive artery disease.
  • Three concurrent CNNs classified orthogonal cardiac patches to distinguish coronary calcium from other high-intensity lesions after heart bounding-box detection.
  • The review did not report results for some imaging modalities because research questions and metrics were highly variable, while most datasets were private.
  • CT methods achieved Pearson correlations of 0.932 for Agatston-score regression, 0.95 for calcium quantification, and 0.983 for coronary-lesion segmentation and classification.
  • An FCN refined with an active shape model achieved a Dice of 93% for left-atrial segmentation in 3D CT volumes.
  • CNN combinations localized cardiac regions in 2D chest-CT slices and aggregated plane-specific predictions into 3D bounding boxes.
  • A two-stage approach used an initial shallow network to screen voxels, followed by a deep network combining learned image and Haar-wavelet features for carotid bifurcation detection.
  • Echocardiography applications within the reviewed imaging set included LV segmentation, viewpoint classification, quality assessment, and disease or severity recognition.

E. Optical coherence tomography

OCT deep learning methods addressed tissue characterization, vessel-lumen segmentation, and fibroatheroma identification, using autoencoders and CNNs. The review also situates OCT within imaging studies with heterogeneous questions, metrics, and database availability.

  • OCT provides high-resolution, reproducible intravascular images and quantitative measurements of coronary geometry.
  • A distribution-preserving autoencoder modeled multi-scale OCT data distributions and improved plaque-versus-normal pixel detection over three pretrained autoencoder baselines.
  • A four-layer linear-regression CNN segmented vessel lumen in polar coordinates with 40.6 ms/image computational efficiency.
  • Among AlexNet, GoogleNet, VGG-16, and VGG-19 features, VGG-19 performed best for identifying OCT images containing fibroatheroma.
  • Across the broader imaging literature, reported research questions and evaluation metrics were highly inconsistent, and limited public databases constrained testing by groups without clinical partners.

VI. DISCUSSION AND FUTURE DIRECTIONS

The review identifies data quality, interpretability, and scalability as major barriers to clinical deep learning, while recommending simpler, better-validated models and underused architectures as future directions. Clinical adoption requires interpretable models and large datasets.

  • Data limitations: Deep learning in cardiology is constrained by costly annotation, class imbalance, heterogeneous data quality, and limited evaluation on large multi-center datasets.These problems are especially acute for medical data and can limit generalizability beyond selective clinical datasets.
  • Interpretability: Deep networks remain difficult to interpret because nonlinear features depend on complex interactions, creating barriers to evidence-based clinical use and raising legal and ethical concerns.The review links this black-box character to resistance from medical experts and restrictions on clinical deployment.
  • Validation and model design: Researchers should prioritize simpler end-to-end, non-ensemble models and validate and compare existing cardiovascular models rather than continually developing new ones.The review accepts that simpler models may report lower accuracy when this improves clinical applicability, while emphasizing external validation and methodological scrutiny.
  • Interpretability tools: Attention networks and saliency maps offer established approaches for making cardiology models more interpretable.The review reports applications in medical-history prediction, ECG beat classification, and cardiovascular disease prediction using fundus images.
  • Future architectures: Capsule networks could address data and interpretability challenges but currently incur higher computational costs than CNNs and have not yet been used with cardiology data.Their layers capture feature pose and require less training data than CNNs, but routing by agreement limits wider use.
  • Clinical translation: Successful clinical application ultimately depends on interpretable models and big datasets, despite the field’s movement toward increasingly automated cardiology.The review notes that a complete theoretical understanding of deep learning is still unavailable and that its strengths and limitations require critical examination.
Loading 1902.11122v5…