PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Label 12-Lead ECG Classification on the PTB-XL Dataset: A Comparative Evaluation of Deep Learning Architectures and Heterogeneous Ensemble Approaches

Yunus Emre Mert, Ece Akdoğan, Hüseyin Üvet

arXiv:2609.12803v1eess.SP

TL;DR

ECG classification involves open directions in long-sequence modeling, fine-grained time-frequency features, and large-scale pre-trained models. This study compares five deep-learning architectures and heterogeneous ensembles on PTB-XL; stacking leads the reported primary macro metrics, while single-model ECGFounder is the strongest individual model.

  • Problem

    Long-sequence modeling, fine-grained time-frequency feature capture, and use of large-scale pre-trained models remain open areas in ECG classification.

    Source: §I · p. 1

  • Method

    The study evaluates five ECG architectures and heterogeneous ensemble learning on PTB-XL, including fine-tuning pre-trained ECGFounder.

    Source: §I · p. 1, 5) ECGFounder (Fine-Tuning) · p. 3

  • Results

    Stacking achieved Macro AUROC 0.936, Macro AUPRC 0.836, and Macro F1 0.763, while ECGFounder stood out among individual models.

    Source: A. Comparison of Model Performances · p. 4, §IV · p. 5

  • Takeaways & Limitations

    Integrating diverse architectures' complementary predictions is reported to contribute to robust performance against class imbalance.

    Source: A. Comparison of Model Performances · p. 4

    Evaluation used only PTB-XL and a single seed, so generalizability to noisier, diverse-center real-world ECGs needs validation on independent datasets.

    Source: C. Limitations · p. 5

Abstract

from arXiv · show

This study aimed to compare the performance of different deep learning architectures and heterogeneous ensemble learning approaches for multi-label 12-lead ECG classification on the PTB-XL dataset. Five different models, namely 1D-ResNet18, Bidirectional Mamba, xLSTM, CWT-ViT-KAN, and the pre-trained ECGFounder, were evaluated. Utilizing the recommended split structure of the PTB-XL dataset, folds 1-8 were allocated as the training set, fold 9 as the validation set, and fold 10 as the independent test set. In addition to the individual models, three different ensemble approaches were investigated: Equal-Weight Soft Voting, Validation-Weighted Soft Voting, and stacking. Among the individual models, the highest performance was achieved by ECGFounder, with a Macro AUROC of 0.930 and a Macro AUPRC of 0.823. For the ensemble models, the highest values in the primary macro performance metrics were obtained by the stacking approach, achieving a Macro AUROC of 0.936, a Macro AUPRC of 0.836, and a Macro F1 of 0.763. The highest subset accuracy of 0.630 was achieved using the Validation-Weighted Soft Voting method. The findings indicate that heterogeneous ensemble models, which combine different representation learning approaches, can provide additional performance improvements over individual models in multi-label ECG classification.

I. INTRODUCT˙ION

ECG interpretation is clinically important but requires expertise and can vary, motivating machine-learning approaches. This study evaluates five architectures and heterogeneous ensembles against open directions in ECG sequence modeling, time-frequency feature capture, and pre-trained models.

I. INTRODUCT˙ION

  • ECGs are non-invasive, but accurate interpretation requires clinical expertise and is susceptible to human error and variability.The introduction frames early diagnosis as important because cardiovascular diseases rank among the leading causes of death worldwide.Source: §I · p. 1
  • Deep learning approaches have been used in medical studies, and current models are described as demonstrating cardiologist-level performance in ECG interpretation.Source: §I · p. 1
  • Long-sequence modeling, fine-grained time-frequency feature capture, and large-scale pre-trained models remain open areas for further development in ECG classification.The passage notes successful results with standard CNN- and LSTM-based models while identifying these directions as still developing.Source: §I · p. 1
  • The study evaluates 1D-ResNet18, Bidirectional Mamba, xLSTM, CWT-ViT-KAN, and pre-trained ECGFounder on PTB-XL, alongside heterogeneous ensemble learning.The stated objective is to compare model performance and investigate the potential contribution of heterogeneous ensembles.Source: §I · p. 1

II. MATERIALS AND METHODS

The study uses PTB-XL clinical 12-lead ECG records and applies a shared split structure across models. The data are represented at different sampling rates for ECGFounder and the other individual architectures.

A. Dataset and Preprocessing

  • 21,388 records remained for analysis after records without the five diagnostic superclasses were excluded from the 21,799-record, 18,869-patient PTB-XL dataset.The retained records include 11,111 from male patients and 10,277 from female patients.Source: A. Dataset and Preprocessing · pp. 1–2
  • Figure 1 presents the cohort's age and gender distribution, while Figure 2 presents frequencies for the five primary diagnostic superclasses.Figure 2 notes that class frequencies may sum to more than the number of unique records because ECG records can have multiple labels.Source: Figure 2 · p. 2, Figure 1 · p. 2
  • ECGFounder used native 500 Hz, 12 × 5000 inputs, while the other four models used 100 Hz, 12 × 1000 inputs; all models shared the same splits.Both input formats represent 10-second, 12-lead records.Source: A. Dataset and Preprocessing · p. 2

B. Label Generation

Labels are generated as five-class multi-hot vectors from diagnostic SCP-ECG statements. The architectures include a 1D residual baseline and a bidirectional state-space model for ECG signals.

B. Label Generation

  • Diagnostic SCP-ECG statements are mapped to NORM, MI, STTC, CD, and HYP, then encoded as five-dimensional multi-hot vectors.Form- and rhythm-related codes are excluded, repeated diagnostic classes are deduplicated, and records without a diagnostic superclass are removed.Source: B. Label Generation · p. 2

C. Deep Learning Architectures Utilized

1) 1D-ResNet18 (Baseline Model):

  • 1D-ResNet18 adapts standard ResNet18 with 1D convolutions as a low-cost baseline for local ECG morphology, including QRS complexes and wave transitions.Residual shortcut connections support gradient propagation and mitigate performance degradation as network depth increases.Source: 1) 1D-ResNet18 (Baseline Model) · p. 2

2) Bidirectional Mamba (Selective State-Space Models):

  • Bidirectional Mamba scans ECG signals forward and backward, integrating both passes to model long sequences with linear time complexity.Its selective mechanism makes state-transition parameters functions of the input rather than keeping them constant.Source: 2) Bidirectional Mamba (Selective… · p. 2

3) xLSTM (sLSTM / mLSTM):

The study pairs recurrent memory cells to represent short- and long-term ECG patterns, combines wavelet scalograms with KAN and ViT processing, and fine-tunes a pre-trained ECG foundation model.

3) xLSTM (sLSTM / mLSTM):

  • xLSTM uses exponential gating and sequential sLSTM and mLSTM cells to process short-term morphological details and longer-term rhythm patterns.The sLSTM has scalar memory, while the mLSTM has fully parallelizable matrix memory.Source: 3) xLSTM (sLSTM / mLSTM) · pp. 2–3

4) CWT-ViT-KAN:

  • CWT-ViT-KAN converts ECG leads into CWT scalograms, processes them with learnable wavelet functions, then uses ViT patches and self-attention to model global relationships.The design aims to jointly represent fine- and coarse-scale time-frequency features and long-range dependencies.Source: 4) CWT-ViT-KAN · p. 3

5) ECGFounder (Fine-Tuning):

  • ECGFounder is a pre-trained convolutional foundation model fine-tuned on PTB-XL's native 500 Hz records for the target classification task.Its pre-trained weights come from over 10 million expert-labeled Harvard-Emory ECG records spanning 150 diagnostic labels.Source: 5) ECGFounder (Fine-Tuning) · p. 3

D. Heterogeneous Ensemble Learning Strategy

The study combines five architecturally distinct ECG models using three ensemble strategies, with model selection, weighting, and threshold choices based on validation data. The patient-separated PTB-XL split reserves fold 10 for independent testing.

D. Heterogeneous Ensemble Learning Strategy

  • Combining models with distinct inductive biases and representation mechanisms may capture features that a single architecture does not fully capture.The ensemble strategy uses probability outputs from the five base models.Source: D. Heterogeneous Ensemble Learning… · p. 3
  • The five base learners span convolutional, state-space, recurrent, time-frequency/transformer, and ECG-specific pre-trained approaches; equal-weight voting, validation-weighted voting, and logistic-regression stacking were evaluated.All five models were retained regardless of validation performance, and stacking used validation predictions and labels before application to independent base-model predictions.Source: D. Heterogeneous Ensemble Learning… · p. 3

E. Experimental Setup and Evaluation Criteria

1) Training Protocol and Optimization:

Training used class-weighted loss and validation-based early stopping, with preprocessing and hyperparameters adapted to each architecture. The experiments used a fixed seed, and no data augmentation or mixup was applied.

1) Training Protocol and Optimization:

  • Class imbalance was addressed with BCEWithLogitsLoss weights calculated from training-set negative-to-positive sample ratios for each class.Validation and test labels were excluded from the weight calculation.Source: 1) Training Protocol and Optimization · p. 3
  • Models used AdamW and were trained for at most 50 epochs, with early stopping after seven epochs without validation Macro AUPRC improvement.The checkpoint with the highest validation Macro AUPRC was saved for evaluation.Source: 1) Training Protocol and Optimization · pp. 3–4
  • ECGFounder was fine-tuned on native 500 Hz, 12-lead, 10-second records after 0.5–40 Hz filtering and training-set lead-wise normalization, followed by per-record global z-score standardization.No resampling, linear interpolation, data augmentation, or mixup was used.Source: 1) Training Protocol and Optimization · p. 4

2) Model Hyperparameters:

  • The 1D-ResNet18 used batch size 64 and learning rate 1×10−3; Bidirectional Mamba used batch size 32, learning rate 5×10−4, d_model 128, and d_state 16.Both models used weight decay of 1×10−4.Source: 2) Model Hyperparameters · p. 4
  • xLSTM used sequential sLSTM and mLSTM components; CWT-ViT-KAN processed 32-scale Morlet-wavelet scalograms at 224 × 224 with a ViT-tiny backbone.ECGFounder used pre-trained ECG weights and was fine-tuned on the PTB-XL training set; the three models used batch size 32, 16, and 16, respectively.Source: 2) Model Hyperparameters · p. 4
  • A fixed random seed of 42 was used, and the ensemble strategies were built from probability outputs of the individually evaluated models.Individual models shared the same data splits, label definitions, and evaluation protocol.Source: 2) Model Hyperparameters · p. 4

3) Evaluation Criteria:

Evaluation combined threshold-independent and threshold-dependent measures, prioritizing Macro AUPRC alongside Macro AUROC. Secondary measures assessed per-class performance and exact-label and decision-level errors.

3) Evaluation Criteria:

  • Macro AUPRC was the primary model-selection criterion, with Macro AUROC used as a primary performance metric alongside it.Source: 3) Evaluation Criteria · p. 4
  • Secondary metrics were Macro F1, Micro F1, Macro Precision, Macro Recall, and Macro Specificity, also calculated separately for the five diagnostic classes.Source: 3) Evaluation Criteria · p. 4
  • Subset Accuracy measured records with all five labels correct, while Hamming Loss measured the proportion of incorrect classification decisions.Source: 3) Evaluation Criteria · p. 4

III. RESULTS

ECGFounder was the strongest individual model, while stacking led the ensemble approaches on the primary macro metrics. The CWT-ViT-KAN pipeline had the lowest individual performance despite its additional computational and representational costs.

A. Comparison of Model Performances

  • CWT-ViT-KAN had the lowest individual-model performance, and its added CWT, ViT, and KAN costs did not yield improvement under these experimental conditions.The authors characterize its complexity-performance trade-off as less favorable.Source: A. Comparison of Model Performances · p. 4
  • Bidirectional Mamba and 1D-ResNet18 followed ECGFounder among individual models, while heterogeneous ensembles showed more consistent performance than individual models.Source: A. Comparison of Model Performances · p. 4
  • Stacking achieved Macro AUROC 0.936, Macro AUPRC 0.836, and Macro F1 0.763; ECGFounder led individual models with Macro AUROC 0.930 and Macro AUPRC 0.823.The Macro AUPRC increase from 0.823 to 0.836 was reported as demonstrating the benefit of integrating complementary predictions against class imbalance.Source: A. Comparison of Model Performances · p. 4

B. Effect of Ensemble Learning on Subset Accuracy

On the PTB-XL test set, heterogeneous ensembles improved performance, particularly on macro metrics. Validation-Weighted Soft Voting led on Subset Accuracy, while Stacking had the lowest Hamming Loss.

B. Effect of Ensemble Learning on Subset Accuracy

  • 0.630 Subset Accuracy was achieved by Validation-Weighted Soft Voting, versus 0.602 for ECGFounder among individual models.Subset Accuracy requires all labels to be predicted correctly for a record.Source: B. Effect of Ensemble Learning on… · p. 4, Table I · p. 5
  • 0.110 was the lowest Hamming Loss, achieved by Stacking, which excelled in macro metrics despite Voting's higher exact-match score.ECGFounder had the lowest Hamming Loss among individual models at 0.116.Source: B. Effect of Ensemble Learning on… · p. 4

IV. DISCUSSION

  • Heterogeneous ensembles improved performance, particularly on macro metrics, while ECGFounder led individual models and Bi-Mamba and 1D-ResNet18 remained competitive.The evaluation compared representation-learning mechanisms and ensemble strategies on the PTB-XL test set.Source: §IV · pp. 4–5

A. Model Performances and Architectural Implications

Stacking led the ensemble models on macro metrics, but Validation-Weighted Soft Voting achieved the highest exact-match accuracy. Among individual models, ECGFounder led AUROC and AUPRC, while Bi-Mamba was competitive and CWT-ViT-KAN had the lowest reported metrics.

A. Model Performances and Architectural Implications

  • 0.930 Macro AUROC and 0.823 Macro AUPRC were ECGFounder’s highest individual-model results; Bi-Mamba reached 0.923 Macro AUROC and 0.816 Macro AUPRC.1D-ResNet18 achieved the best macro sensitivity among individual models.Source: A. Model Performances and… · p. 5
  • 0.874 Macro AUROC, 0.709 Macro AUPRC, and 0.664 Macro F1 were CWT-ViT-KAN’s lowest results across the reported individual-model metrics.Under these experimental conditions, its more complex time-frequency representation did not improve performance.Source: A. Model Performances and… · p. 5

B. Subset Accuracy Analysis of the Ensemble Model

  • 0.936 Macro AUROC, 0.836 Macro AUPRC, and 0.763 Macro F1 were achieved by Stacking; Validation-Weighted Soft Voting led Subset Accuracy at 0.630.Stacking scored 0.625 on Subset Accuracy.Source: B. Subset Accuracy Analysis of the… · p. 5

C. Limitations

The study’s comparisons are bounded by differences in input sampling rates and repeated validation-set use, alongside single-seed evaluation on PTB-XL alone. External validation on independent datasets and noisy clinical ECGs is needed to establish broader generalizability.

C. Limitations

  • 500 Hz ECGFounder inputs versus 100 Hz inputs for other individual models mean performance differences cannot be attributed solely to architecture or pre-training.The validation set was also used for ensemble construction and threshold selection, which may increase overfitting risk.Source: C. Limitations · p. 5
  • Single-seed experiments limit assessment of performance variability across training iterations.All experiments used one seed value.Source: C. Limitations · p. 5
  • PTB-XL-only evaluation leaves generalizability to noisier, diverse-center real-world clinical ECGs unestablished; independent-dataset validation is needed.The conclusion likewise calls for external validation on independent datasets and noisy clinical ECG records.Source: C. Limitations · p. 5, §V · p. 6

V. CONCLUSION

  • Stacking led the primary macro metrics, while validation-weighted voting led Subset Accuracy; CWT-ViT-KAN underperformed other individual architectures.The conclusion describes heterogeneous ensembles as potentially advantageous for multi-label ECG classification.Source: §V · pp. 5–6
Loading 2609.12803v1…