Automatic diagnosis of the 12-lead ECG using a deep neural network

1.1 Model specification and training

• The DNN used a residual-network architecture adapted to one-dimensional signals and was trained on 2,322,513 ECG records to detect six abnormalities.The dataset was split into 98% training and 2% validation; the best validation results used roughly one quarter the layers and parameters of the network in.Source: §1.1 · pp. 2, 4

1.2 Testing and perfomance evaluation

• F1 scores above 80% and specificity indexes over 99% were obtained across the ECG abnormalities; the DNN matched or outperformed residents and students on F1 for all abnormalities.The test set contained 827 tracings from distinct patients, and Table 2 compares DNN scores with average performance of cardiology residents, emergency residents, and medical students.Source: §1.2 · p. 4, Table 2 · p. 5
• Across 10 random initializations, micro average precision ranged from 0.946 to 0.961; the analysis used the realization with mAP = 0.951 and thresholds maximizing F1 for each abnormality.The selected realization was immediately above the median mAP among the 10 executions.Source: §1.2 · p. 4
• No statistically significant performance difference was indicated between the DNN and residents or students for most classes.The comparison used bootstrap distributions and the McNemar test.Source: §1.2 · pp. 4–5
• Alternative 90%-5%-5% splits showed no statistically significant difference from the original DNN, except for the chronological split, which was affected by changes over time in telehealth-center operation.The alternative splits were stratified randomly, by patient, or in chronological order.Source: §1.2 · p. 5

2 Discussion

• End-to-end learning classifies raw ECG signals while learning features, unlike classical methods that extract features before classification.The discussion also situates neural ECG classification in feature-based, end-to-end [33], and hybrid approaches.Source: §2 · p. 5
• The dataset comprises over 2 million short, in-clinic 12-lead exams, spanning rhythm disorders and conduction disturbances unlike prior small beat-labeled or limited-lead datasets.The dataset contains 7-to-10-second tracings and exam-level annotations for abnormalities including AF, SB, ST, 1dAVb, RBBB, and LBBB.Source: §2 · pp. 5–6
• Most DNN errors involved borderline interval thresholds; the model appeared more robust to noise than trainees, while conceptual errors were more common among emergency residents and students.Attention errors occurred at similar ratios for cardiology residents, emergency residents, and medical students.Source: §2 · p. 6
• Emergency residents performed worse than medical students on many abnormalities, which the authors suggest may reflect less frequent ECG interpretation and students’ fresher training.The proposed explanation distinguishes emergency residents from cardiology residents, who interpret ECGs daily.Source: §2 · pp. 6–7
• The study cannot establish statistically significant superiority over clinicians, covers only six abnormalities, and does not replace specialist interpretation of complex or borderline exams.The authors attribute uncertainty to infrequent classes and call for experienced specialist input in difficult cases.Source: §2 · p. 7
• Future work should test broader and more complex diagnoses, normal ECG recognition, and real-time performance under specialist review; TNMG could support evaluation and wider access to ECG care.TNMG performs more than 2,000 ECGs daily, and the authors envision automation reserving cardiologists for complex cases and improving access in low- and middle-income countries.Source: §2 · pp. 6–7
• The end-to-end DNN accurately recognized six abnormalities in clinical S12L-ECG exams, with performance at least as good as medical residents and students.The authors describe the work as a proof of concept and note that expert review remains necessary for complex and borderline cases.Source: §2 · p. 7

3.1 Dataset acquisition

• TNMG, a public telehealth system serving 811 of Minas Gerais’s 853 municipalities, supplied the study’s ECGs.Since September 2017, TNMG also provided telediagnostic services to other states in the Amazonian and Northeast regions.Source: §3.1 · p. 7
• Mostly recorded in primary care, ECG exams lasted 7–10 seconds and were sampled at 300–600 Hz before electronic upload for cardiologist review.Exams used TEB ECGPC or Micromed ErgoPC 13 tele-electrocardiographs; the in-house software also uploaded patients’ clinical histories.Source: §3.1 · p. 7
• Since December 2017, Uni-G release 28.5 automatically identified waves, calculated ECG measurements, analyzed rhythm, and provided diagnostic interpretations.It also supplied Minnesota codes, a standard ECG classification used in epidemiological studies.Source: §3.1 · p. 7
• The CODE study standardized and consolidated the stored clinical information, digital tracings, and cardiologist reports; automatic measurements were shown to cardiologists from April 2018.Previously stored ECGs were also analyzed by Uni-G so measurements and automated diagnoses were available for all exams in the database.Source: §3.1 · pp. 7–8
• Training and validation used data from patients older than 16 years recorded in 2010–2016, while testing used exams from April–September 2018.Source: §3.1 · p. 8

3.2 Labelling training data from text report

• Training and validation labels were extracted from cardiologist reports by preprocessing text, applying LAC to n-grams, then using rule-based class disambiguation.LAC was trained on a 2,800-sample dictionary created from real diagnosis reports.Source: §3.2 · p. 8
• Macro F1 scores were 0.729 for 1dAVb, 0.849 for RBBB, and 0.838 for LBBB when report classification was tested against cardiologist labels.A certified cardiologist manually labeled 4,557 reports by choosing from prespecified classes after reading the free text.Source: §3.2 · p. 8
• Macro F1 scores were 0.991 for SB, 0.993 for AF, and 0.974 for ST in the manually labeled report-classification test.Source: §3.2 · p. 8

3.3 Training and validation set annotation

• Labels combined expert report diagnoses with Uni-G statements, Minnesota codes, and automatic measurements; abnormalities were accepted when the doctor and either automatic classifier agreed.A diagnosis was rejected when only one automatic classifier indicated it and both the doctor and the other classifier disagreed.Source: §3.3 · p. 8
• Sensitivity analysis guided acceptance of remaining diagnoses: all medical RBBB, d1AVb, SB, and ST diagnoses were accepted, while AF required doctor confirmation and NN-interval standard deviation above 646.The rule accepted 26,033 RBBB, 13,645 d1AVb, 12,200 SB, 14,604 ST, and 14,604 AF diagnoses.Source: §3.3 · p. 8
• ST diagnoses below 100 heartbeats per minute and SB diagnoses above 50 were rejected.This removed 8,376 medical and 2 automatic ST diagnoses, and 7,361 medical and 16,427 automatic SB diagnoses.Source: §3.3 · p. 8
• LBBB or RBBB diagnoses with QRS duration below 115 ms and 1dAVb diagnoses with PR duration below 190 ms were rejected.The rules removed 9,313 medical RBBB and 8,260 medical LBBB diagnoses, plus 3,987 automatic 1dAVb diagnoses.Source: §3.3 · p. 8
• Fewer than 3% of exams were estimated to be false positives introduced by the acceptance rules, according to sensitivity analysis.The sensitivity analysis reviewed 100 exams per abnormality.Source: §3.3 · p. 8
• The 34,512 unresolved exams were manually reviewed by medical students under certified-cardiologist supervision; prior reports and automatic measurements were used only to establish training and validation ground truth.Manual review through the Telehealth ECG diagnostic system took several months.Source: §3.3 · p. 9

3.4 Test set annotation

• Kappa agreement was 0.741 for 1dAVb, 0.955 for RBBB, 0.964 for LBBB, 0.844 for SB, 0.831 for AF, and 0.902 for ST; disagreements went to a third senior specialist.When the two cardiologists agreed, their common diagnosis was ground truth; classification followed American Heart Association standardization.Source: §3.4 · p. 9
• Upgraded TNMG software presented Uni-G measurements and predefined abnormality classes, allowing direct label coding without extracting diagnoses from textual reports.Training and validation labels, in contrast, were extracted from textual reports.Source: §3.4 · p. 9

3.5 Neural network architecture and training

• Each 7–10-second ECG is resampled to 400 Hz and zero-padded to 4096 samples per lead before entering the network.Source: §3.5 · p. 9
• The network uses an initial convolutional layer, four two-convolution residual blocks, and a sigmoid dense output for non-mutually-exclusive classes.Batch normalization and ReLU follow each convolution, with dropout after the nonlinearity; the residual design adapts residual networks using a skip-connection modification.Source: §3.5 · p. 9
• Convolutional filters have length 16; the first layer and residual block use 64 filters, increasing by 64 every second block, while each block subsamples by a factor of 4.Max pooling and 1x1 convolutions in skip connections match dimensions between the main and skip branches.Source: §3.5 · p. 9
• The model minimizes average cross-entropy with Adam at learning rate 0.001, reducing it tenfold after seven consecutive epochs without validation-loss improvement.Weights are initialized as in and biases to zero; training runs for 50 epochs, retaining the model with the best validation results.Source: §3.5 · pp. 9–10

3.6 Hyperparameter tuning

• About 30 iterations of training, validation, and expert-guided manual revision produced the final configuration, with architecture and hyperparameters tuned alongside dataset improvements.Earlier iterations used slightly different dataset versions, and expert knowledge informed how their results guided subsequent choices.Source: §3.6 · p. 10
• The search varied residual-network depth, kernel size, batch size, learning rate, optimizer, activation, dropout, and learning-rate plateau settings.Options were 2, 4, 8, or 16 residual blocks; kernel sizes 8, 16, or 32; batch sizes 16, 32, or 64; learning rates 0.01, 0.001, or 0.0001; SGD or ADAM; ReLU or ELU; dropout rates 0, 0.5, or 0.8; and 5–10 epochs without improvement with learning-rate reductions of 0.1–0.5. They also tried vectorcardiogram linear dimensionality reduction, an LSTM before convolutional layers, residual networks without the preactivation architecture proposed in, VGG, and swapping activation and batch-normalization order.Source: §3.6 · p. 10

• Precision-recall curves assessed rhythm-class discrimination, and subsequent analyses used thresholds maximizing F1, chosen for robustness to class imbalance.The curves relate precision (PPV) to recall (sensitivity); they were considered more informative than ROC plots for imbalanced classes.Source: §3.7 · p. 10
• For each class, scores were bootstrapped across 1,000 test-set resamples; McNemar tests compared misclassifications, and kappa measured inter-rater agreement.Precision, recall, specificity, F1, and confusion matrices were computed for the DNN, medical residents, and students.Source: §3.7 · p. 10
• An experienced cardiologist reviewed all misclassified exams, which were categorized as measurement, noise, unexplained, conceptual, or attention errors, with categories depending on evaluator.Unexplained errors applied only to the DNN, while conceptual and attention errors applied only to medical residents and students.Source: §3.7 · p. 10
• Alternative 90%-5%-5% splits of 2,322,513 records—random, date-ordered, or patient-stratified—were evaluated on the original test set and additional last-5% splits.F1 distributions used 1,000 bootstrap samples for the original test set and 200 for additional splits; boxplots appeared in supplementary material.Source: §3.7 · p. 10

• The test dataset and model weights are openly downloadable, while training data require individual requests and agreements for non-commercial research; source data for Supplementary Figures 1 and 2 are provided.Source: Data availability · p. 10

Code availability

• The code for training and evaluating the DNN model and generating figures and tables is available on GitHub.Repository: https://github.com/antonior92/automatic-ecg-diagnosis.Source: Code availability · p. 10

Research ethics statement

• The study complied with relevant ethical regulations and was approved by the Universidade Federal de Minas Gerais Research Ethics Committee.Approval protocol: 68496317.7.0000.5149.Source: Research ethics statement · p. 11

Author contribution statement

• The team shared responsibility for study design, model development, dataset preparation, analysis, and manuscript writing.A.L.R. conceived and led the project; A.H.R. conducted statistical analysis and generated figures and tables, while G.M.M.P. performed error analysis. Other authors handled preprocessing, annotation, semi-supervised label extraction, the dataset interface and database, data interpretation, and critical manuscript revision.Source: Author contribution statement · p. 11

Competing interesting statement

• The authors declared no competing interests.Source: Competing interesting statement · p. 11

Supplementary Table 1

• Supplementary Table 1 reports test-set false positives, false negatives, true positives, and true negatives for each abnormality.Source: Table 1 · p. 15

Supplementary Table 2

• Kappa coefficients compare test-set agreement between the DNN, residents, and students, and between the DNN and certified cardiologists.The table defines 1 as complete agreement and 0 as agreement no greater than chance.Source: Table 2 · p. 15

Supplementary Figure 1

• Bootstrapped test-set precision, recall, specificity, and F1-score distributions are shown for the DNN, medical residents, and students.The empirical distributions use 1,000 bootstrap samples; boxplots display medians, interquartile ranges, whiskers, and outliers.Source: Figure 1 · p. 16

Supplementary Table 3

• McNemar-test p-values compare test-set misclassification between the DNN, medical residents, and students two at a time.Statistical significance is indicated at the 0.05 level.Source: Table 3 · p. 16

Supplementary Figure 2

• 86% of 100 supposedly wrong RBBB cases and 83% of 100 supposedly wrong LBBB cases were judged correct by a certified cardiologist.The cases came from the secondary test split collected from October 2016 to June 2017, whose annotations were less accurate; date-ordered splits differed significantly for RBBB and LBBB, and the caption attributes this to lower annotation quality during staffing changes.Source: Figure 2 · p. 17

Supplementary Figure 3

• Most DNN errors for SB and ST occurred near the consensus heart-rate thresholds of 50 bpm and 100 bpm, respectively.Uni-G heart-rate measurements did not always agree with the cardiologists’ classifications, including some measurements above 100 bpm not classified as ST.Source: Figure 3 · p. 17