Source-linked AI summary
Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks
Pranav Rajpurkar, Awni Y. Hannun, Masoumeh Haghpanahi, Codie Bourn, Andrew Y. Ng
TL;DR
The paper addresses inaccurate automated arrhythmia detection from single-lead ECGs despite the ability of experts to interpret such recordings. It trains a deep convolutional sequence-to-sequence model on a large annotated dataset and finds that it exceeds average cardiologist performance in recall and precision. The supported scope excludes several arrhythmias and other heart diseases, particularly those that may require multiple-lead ECGs.
Problem
Computerized ECG interpretation has high error rates, while arrhythmia detection remains challenging because ECG morphology varies across patients and signals contain noise.
Method
The paper trains a 34-layer convolutional neural network to map single-lead ECG sample sequences to sequences of rhythm annotations using end-to-end supervision.
Results
The model exceeds individual expert performance in both recall (sensitivity) and precision (positive predictive value) on a cardiologist-annotated test set.
Takeaways & Limitations
Large annotated data and a deep convolutional network support arrhythmia detection from single-lead ECG records beyond the reported average cardiologist performance.
Takeaways & Limitations
The system does not detect several arrhythmias and non-arrhythmic heart diseases, some of which may be difficult or impossible to detect from a single-lead ECG.
Abstract
from arXiv · showhide
We develop an algorithm which exceeds the performance of board certified cardiologists in detecting a wide range of heart arrhythmias from electrocardiograms recorded with a single-lead wearable monitor. We build a dataset with more than 500 times the number of unique patients than previously studied corpora. On this dataset, we train a 34-layer convolutional neural network which maps a sequence of ECG samples to a sequence of rhythm classes. Committees of board-certified cardiologists annotate a gold standard test set on which we compare the performance of our model to that of 6 other individual cardiologists. We exceed the average cardiologist performance in both recall (sensitivity) and precision (positive predictive value).
1. Introduction
The paper targets difficult arrhythmia detection from single-lead ECGs, where computerized interpretation has high error rates despite expert interpretability. It combines a deep convolutional model with a substantially larger annotated dataset and evaluates performance against cardiologists.
- The model maps ECG sample sequences to arrhythmia annotation sequences using a deep convolutional network.
- A single well-placed ECG lead can usually let an expert determine arrhythmias, but computerized interpretation has reported substantial errors.
- Arrhythmia detection from ECGs is challenging because wave morphology varies across patients and signals can contain noise.
- The dataset contains about 30,000 unique patients, making it 500 times larger than other datasets of its kind.
- The model exceeds individual expert performance in both recall (sensitivity) and precision (positive predictive value) on a test set annotated by cardiologists.
2. Model
The model formulates ECG arrhythmia detection as sequence-to-sequence prediction, using a deep convolutional architecture to produce rhythm labels over time. Its design combines convolutional layers, residual connections, normalization, and softmax outputs.
- Problem Formulation: The task maps an ECG signal X to a sequence of rhythm labels r, with each output label corresponding to an input segment.
- Problem Formulation: Training optimizes a cross-entropy objective based on the probability assigned to each target output label.
- Model: The network takes raw ECG time-series inputs and outputs a sequence of label predictions at approximately one-second intervals.
- Model: The architecture uses 33 convolutional layers followed by a fully connected layer and a softmax.
- Training: Shortcut connections make optimization tractable in the deep network by allowing information to propagate through its layers.
3. Data
The study builds a patient-separated ECG dataset with expert segment annotations and evaluates the model on a cardiologist-consensus test set. It covers 14 output classes, including noise, sinus rhythm, and 12 arrhythmias.
- Training Data: The dataset contains 64,121 ECG records from 29,163 patients, sampled at 200 Hz using a single-lead Zio Patch monitor.
- Training Data: Each 30-second training record can contain multiple rhythms and is segmented into 14 expert-labeled rhythm classes.
- Data Splits: Training, validation, and test splits contain no patient overlap.
- Test Set: The test set comprises 336 records from 328 unique patients with consensus labels from committees of three board-certified cardiologists.
- Test Set: Six additional cardiologists independently annotate each test record to enable comparison between the model and individual cardiologists.
- Rhythm Classes: The dataset includes clinically important distinctions such as Mobitz I versus Mobitz II AV block, which can be difficult to distinguish.
4. Results
The study evaluates the model and cardiologists against committee annotations using sequence- and set-level metrics. The model outperforms average cardiologist performance across key aggregate and rhythm-specific measures.
- Evaluation Metrics: The evaluation uses cardiologist committee annotations as ground truth for sequence-level and set-level F1 metrics.Sequence-level F1 measures overlap between predicted and ground-truth labels over time, whereas set-level F1 compares unique arrhythmias in each record without penalizing timing misalignment.
- Human Benchmark: Cardiologist performance is estimated by averaging individual F1 scores from six cardiologists using committee labels as ground truth.The test records have committee labels and separate individual labels, enabling comparison with individual cardiologist performance.
- Aggregate Results: The model outperforms the average cardiologist score on both sequence and set F1 metrics.
- Rhythm-Specific Results: The model noticeably outperforms cardiologists on AV Block rhythms, including Wenckebach, AVB Type2, and complete heart block.The authors highlight the importance of distinguishing the more severe Mobitz II and complete heart block from usually benign Wenckebach.
- Aggregate Results: The model outperforms the cardiologist average in aggregate precision and recall.Aggregate cardiologist scores are computed as the mean of individual cardiologist scores.
5. Analysis
The model’s errors often involve rhythm classes with similar ECG expressions or ambiguous boundaries. The analysis identifies specific clinically and morphologically related confusions, including EAR versus sinus rhythm.
- Error Patterns: Many arrhythmias are confused with sinus rhythm, partly because their exact onset and offset can be ambiguous.
- Morphological Confusions: Wenckebach and AVB Type2 are often confused because their ECG morphologies are generally similar.
- Morphological Confusions: SVT and AFIB are often confused with AFL because all three are atrial arrhythmias.
- Rate-Related Confusions: IVR is sometimes mistaken for VT because the rhythms differ only in heart rate and are difficult to distinguish near 100 beats per minute.
- Subtle Features: EAR is commonly confused with sinus rhythm because its distinguishing irregular P wave can be subtle when low-amplitude or obscured by noise.
6. Related Work
Earlier ECG automation used engineered features, probabilistic models, and neural networks, but existing approaches and datasets had important scope limitations. This work addresses dataset scale and multi-arrhythmia coverage.
- Feature Engineering: R-peak-based features support coarse rhythm classification, but they are insufficient for distinguishing most arrhythmias without atrial and QRS morphology information.
- Prior Models: Hidden Markov models and artificial neural networks have been applied to beat detection and beat classification.
- Prior Models: Earlier neural-network results could distinguish atrial fibrillation from sinus rhythm accurately on MIT-BIH but did not generalize to noisier single-lead recordings or multiple arrhythmias.
- Dataset Limitations: Common ECG benchmarks contain relatively few unique patients and are too small for fine-grained arrhythmia classification.The MIT-BIH arrhythmia database consists of 48 half-hour ECG strips, while other benchmarks have single-digit hundreds or fewer unique patients.
- Dataset Scale: The developed dataset contains 29,163 unique patients and 14 classes, with hundreds of unique examples for the rarest arrhythmias.
- Deep Learning Context: Deep neural networks have often approached or exceeded human agreement rates when large annotated datasets are available, including in healthcare applications.
7. Conclusion
The model exceeds cardiologist performance in detecting a wide range of arrhythmias from single-lead ECG records. Its performance is attributed to a large annotated dataset and a deep convolutional network, while broader clinical coverage remains future work.
- The model exceeds cardiologist performance in detecting a wide range of heart arrhythmias from single-lead ECG records.
- A large annotated dataset and a very deep convolutional network map ECG sample sequences to arrhythmia annotation sequences.
- The current system does not detect some arrhythmias and other heart diseases, including ventricular flutter or fibrillation and myocardial infarction.Some conditions may be difficult or impossible to detect from a single-lead ECG but can often be seen on multiple-lead ECGs.
- High-accuracy ECG diagnosis could save clinicians time, reduce misdiagnoses, and broaden access to ECG-based diagnosis where cardiologists are difficult to access.
Appendix
The appendix lists the rhythm classes classified by the model and reports training-plus-validation and test patient counts for each class. The listed classes include sinus rhythm, atrial fibrillation and flutter, conduction blocks, and ventricular or ectopic rhythms.
- The table includes atrial fibrillation and atrial flutter.
- The table includes second-degree AV block Type 2, complete heart block, ventricular bigeminy, ectopic atrial rhythm, and idioventricular rhythm.
- The listed classes include sinus rhythm and junctional rhythm.
- The appendix table lists rhythm label names, descriptive class names, examples, and patient counts for training-plus-validation and test sets.