PaperlayerView exact revision on arXiv

Source-linked AI summary

Label-Efficient Deep Learning for ECG Delineation: A Multi-Dataset Benchmark against Widely Used Delineation Tools

Jeonghwa Lim, Minje Park, Yeongyeon Na, Yujin Eom, Soyeon Lim, Young Ho Lee, Yu Jeong Kim, Sunghoon Joo, Ki Hong Lee

arXiv:2610.07885v1cs.LGcs.AIcs.CVeess.SP

TL;DR

ECG delineation can produce clinically important measurements, but deep learning depends on costly expert annotations, and evidence for label-efficient delineation and comparisons with tools used in practice has been limited. The study compares self-supervised objectives and fine-tuning strategies, then benchmarks the selected model across datasets. Self-supervised pretraining benefits delineation, and the selected model outperforms the compared tools across datasets and metrics while remaining robust under arrhythmia.

  • Problem

    Whether self-supervised pretraining helps dense ECG delineation and how label-efficient models compare with tools used in practice remained unclear, amid costly expert annotations and challenges from waveform variability and signal degradation.

    Source: §I · p. 1

  • Method

    The study compares self-supervised pretraining objectives with supervised or semi-supervised fine-tuning, then benchmarks the selected model against delineation tools across datasets.

    Source: 2) Training data · p. 3, 3) Evaluation data · p. 3, §VII · p. 9

  • Results

    Self-supervised pretraining benefits delineation, and the MAE-pretrained model outperforms every tool on all three metrics across datasets under both fine-tuning configurations, with its margin widening under arrhythmia.

    Source: §VII · p. 9

  • Takeaways & Limitations

    The findings support wider adoption of label-efficient learning and lay groundwork for reliable ECG delineation in clinical practice.

    Source: §VII · p. 10

    The study examines only three self-supervised objectives with a single established backbone, so the identified strategies are practical directions rather than an optimal choice.

    Source: E. Limitations and Future Work · p. 9

Abstract

from arXiv · show

Electrocardiogram (ECG) delineation, the identification of waveform boundaries, is a foundational step that translates raw ECG signals into clinically interpretable measurements. Deep learning has advanced this task but remains dependent on costly expert annotations. Label-efficient strategies such as self-supervised pretraining and semi-supervised learning are expected to ease this burden, yet it remains unclear whether they yield reliable delineation and whether the deep models they produce outperform the delineation tools used in practice. We address this in two stages. First, comparing self-supervised objectives with supervised or semi-supervised fine-tuning across one internal and four external datasets, we find that pretraining helps but the objective matters, and that the value of semi-supervised fine-tuning depends on the pretraining objective. Second, we benchmark the selected deep learning model against widely used open-source (NeuroKit2, Prominence, ECGdeli) and commercial (CalECG) tools using three complementary metrics. The model ranks best on every metric and dataset, outperforming the strongest tool by a clear margin on the rhythm-diverse set (mIoU 71.3 vs. 54.8%; averaged point-wise sensitivity 92.6 vs. 76.4%), and degrades the least from sinus to arrhythmia. A rhythm-stratified and point-wise analysis further characterizes the distinctive behavior of each tool, yielding practical guidance for tool selection. These results provide systematic, multi-dataset evidence that self-supervised pretraining is effective for ECG delineation and enables a label-efficiently trained deep learning model to outperform widely used delineation tools by leveraging abundant unlabeled data. This supports adopting such models in diverse, real-world clinical settings.

I. INTRODUCTION

ECG delineation supports clinically important measurements, but annotation costs and waveform variability complicate reliable, scalable learning. The study compares label-efficient training strategies and benchmarks the selected model against established tools.

A. Classical ECG Delineation Tools

  • Rule-based signal processing is widely used, but its accuracy tends to fall on atypical beats and noisy signals that distort waveform features.Deep learning learns morphology from data and has achieved improved accuracy, but depends on costly, time-consuming expert annotations.Source: §I · p. 1, A. Classical ECG Delineation Tools · p. 2

I. INTRODUCTION

  • Whether self-supervised pretraining helps dense, time-point ECG delineation, how objectives compare, and how objectives interact with fine-tuning had remained unexamined.The authors also identify a gap in direct comparisons with open-source and commercial tools used in research and clinical practice.Source: §I · p. 2
  • The study compares self-supervised pretraining objectives and supervised versus semi-supervised fine-tuning, then benchmarks the selected model against established tools.The benchmark includes open-source and commercial tools and rhythm-stratified analysis across diverse external datasets.Source: §I · p. 2
  • The paper provides systematic, multi-dataset evidence that self-supervised pretraining benefits ECG delineation.It presents a benchmark of label-efficient deep models against multiple open-source libraries and a commercial package.Source: §I · p. 2
  • The deep learning model achieves the best overall performance and remains robust under arrhythmia, while the study characterizes distinct tool behavior across datasets.The authors identify costly expert labels as a constraint on scaling fully supervised deep learning.Source: §I · pp. 1–2

B. Deep Learning for ECG Delineation

Prior deep learning and self-supervised approaches motivate label-efficient ECG delineation, while the literature leaves gaps in task coverage and benchmarking. The study develops and evaluates models using distinct labeled, unlabeled, and test datasets.

B. Deep Learning for ECG Delineation

  • Deep learning treats ECG delineation as dense prediction, assigning each time point to a wave class or background with encoder-decoder segmentation networks.These models can extend to diverse arrhythmias, but rely on costly dense annotations.Source: B. Deep Learning for ECG Delineation · pp. 2–3

C. Self-Supervised Learning

  • Self-supervised learning transfers representations learned from unlabeled ECG data to downstream tasks, but its benefit for delineation and interaction with fine-tuning remained underexamined.Existing ECG applications had largely focused on classification tasks such as arrhythmia detection and disease screening.Source: C. Self-Supervised Learning · p. 3

D. Benchmarking Gaps in ECG Delineation

  • Prior benchmarks had not jointly compared rule-based and deep learning methods across multiple external datasets while assessing clinical interval measurements.Earlier studies omitted deep learning, used only one external dataset, or left generalization to unseen datasets underexplored.Source: D. Benchmarking Gaps in ECG Delineation · p. 3

A. Datasets and Preprocessing

  • The study uses seven public datasets and a private mobile ECG dataset, following a standardized benchmark configuration and processing two datasets under the same protocol.The included public datasets are QTDB, ISP, PTB-XL, MIMIC-IV-ECG, LUDB, Zhejiang, and RDB.Source: A. Datasets and Preprocessing · p. 3
  • MIMIC-IV-ECG provides large-scale raw ECGs paired with clinical text reports for self-supervised pretraining.After filtering, the study retains 761,139 recordings from 154,094 subjects.Source: 1) Pretraining data · p. 3

1) Pretraining data:

  • QTDB and ISP provide labeled development data, while PTB-XL supplies unlabeled data for semi-supervised training.QTDB is an established delineation benchmark; ISP adds scale, realistic noise, and rhythm diversity, while PTB-XL spans diverse morphologies and rhythms.Source: 2) Training data · p. 3

2) Training data:

The experimental setup develops delineation models by varying encoder initialization and use of unlabeled data, then evaluates the selected model against existing tools. Its datasets and metrics cover internal and external testing, segmentation, clinical intervals, and fiducial localization.

3) Evaluation data:

  • The internal test set combines QTDB and ISP test splits; external sets cover delineation benchmarks, arrhythmias, diverse rhythms, and portable-device recordings.mECGDB supplies interval-level labels only, unlike the dense time-point labels in the other external sets.Source: 3) Evaluation data · pp. 3–4
  • Each ECG is cropped or zero-padded to 10 seconds, resampled to 250 Hz, band-pass filtered, and Z-score normalized before model input.The filter range is 0.67–40 Hz.Source: 4) Preprocessing · p. 4

B. Evaluation Protocol

  • Evaluation uses mIoU, averaged interval error, and averaged point-wise sensitivity to measure segmentation, clinical intervals, and fiducial localization.Sensitivity is evaluated at 40 ms tolerance; the detailed point-wise analysis also reports PPV and localization error at 150 ms tolerance.Source: B. Evaluation Protocol · p. 4
  • Learning-strategy comparisons report means and standard deviations across random-seed runs and use paired bootstrap and Wilcoxon tests with Holm correction.The test set is paired for these comparisons, with a seed-level robustness check also reported.Source: B. Evaluation Protocol · p. 4

IV. EXPERIMENTAL SETUP

  • The study develops models under supervised, semi-supervised, and self-supervised strategies, selects a pretraining objective, and benchmarks the selected model against existing tools.The experimental data include an internal test set and four external test sets chosen to span sources, devices, and populations.Source: 3) Evaluation data · p. 3, §IV · p. 4

A. Comparing Learning Strategies for ECG Delineation

  • The training-strategy comparison varies both self-supervised encoder initialization versus training from scratch and labeled-only versus semi-supervised training.Crossing these factors is intended to isolate the effects of pretraining and semi-supervised training.Source: A. Comparing Learning Strategies for… · p. 4
  • The study compares MAE, MoCo, and MERL pretraining objectives, and uses Mean Teacher with weak-strong augmentation for semi-supervised learning.The objectives represent generative, contrastive, and multimodal approaches, respectively.Source: A. Comparing Learning Strategies for… · p. 4

B. Benchmarking Against Existing Delineation Tools

The benchmark compares the selected first-stage model against established open-source and commercial delineation tools, using a fixed architecture across learning strategies. Tool configurations and evaluation scope differ, particularly for CalECG.

B. Benchmarking Against Existing Delineation Tools

  • The benchmark includes NeuroKit2, ECGdeli, and Prominence as open-source tools, plus commercial CalECG.NeuroKit2 and ECGdeli use wavelets; Prominence uses peak prominence.Source: B. Benchmarking Against Existing… · p. 4

C. Implementation Details

  • CalECG is evaluated only on P onset, QRS onset, QRS offset, T offset, and interval-level metrics, unlike the other tools’ broader fiducial-point evaluation.Tools use default configurations without manual correction.Source: C. Implementation Details · pp. 4–5
  • A ViT-Tiny encoder with a two-layer FCN decoder is held fixed across learning strategies, so performance differences reflect the strategy rather than architecture.The decoder uses a 128-dimensional hidden layer and dropout rate p = 0.1.Source: C. Implementation Details · p. 5

V. RESULTS

The selected DL DELINEATOR configurations outperform benchmarked tools across metrics and datasets, while learning-strategy results show that pretraining effects depend on objective and fine-tuning choice.

A. Comparison of Learning Strategies

  • Without pretraining, semi-supervised learning raises average mIoU from 63.5 to 71.4% and improves mIoU on every dataset, while interval error improves everywhere except RDB.On Zhejiang, mIoU rises from 53.3 to 66.0%.Source: A. Comparison of Learning Strategies · p. 5
  • MAE and MoCo outperform no pretraining, but MERL does not; MoCo gains from semi-supervised fine-tuning while MAE performs comparably with either fine-tuning strategy.Under semi-supervised fine-tuning, average mIoU is 68.7% for MERL, 75.2% for MAE, and 75.0% for MoCo; MAE is selected for its higher average mIoU, better average rank, and stability.Source: A. Comparison of Learning Strategies · p. 5

B. Comparison with Existing Tools

  • The DL DELINEATOR significantly surpasses benchmarked tools across every metric and dataset, with supervised and semi-supervised configurations ranking first and second overall.Their average ranks are 1.3 and 1.7, respectively.Source: Table III · p. 6, B. Comparison with Existing Tools · p. 6
  • On RDB, DL DELINEATOR reaches mIoU 71.3 vs. 54.8% and averaged point-wise sensitivity 92.6 vs. 76.4% for ECGdeli.On Zhejiang, mIoU is 75.1 vs. 42.7% for Prominence; averaged interval error is 15.3 vs. 30.4 ms and averaged point-wise sensitivity is 86.0 vs. 56.3% for ECGdeli.Source: Table III · p. 6, B. Comparison with Existing Tools · p. 6
  • On mECGDB, DL DELINEATOR reduces average interval error to 15.0 ms from 25.4 ms for Prominence, the best tool.For QT interval error, both configurations achieve 19.1 ms versus 21.7 ms for Prominence.Source: Table IV · p. 6, B. Comparison with Existing Tools · p. 6

C. Point-wise Performance Characteristics

Point-wise results distinguish tools by fiducial point and rhythm: the DL DELINEATOR leads nearly everywhere and shows the smallest sinus-to-arrhythmia degradation.

C. Point-wise Performance Characteristics

  • NeuroKit2’s CWT variant has low Pon sensitivity (43.7 and 14.7% on LUDB and Zhejiang), while DWT struggles at QRSon and Toff.Their mECGDB interval errors align with these weaknesses: CWT has 207.4 ms PR error and DWT has 91.5 ms QT error.Source: C. Point-wise Performance… · p. 6
  • CalECG reports only four interval-defining fiducials and its overall sensitivity drops from 68.1% on LUDB to 7.0% on Zhejiang.It does not provide Poff or Ton.Source: C. Point-wise Performance… · p. 6, Table V · p. 7
  • ECGdeli leads QRSon sensitivity on LUDB and Zhejiang (97.3 and 83.2%), while Prominence leads at Pon (79.2 and 65.4%).For the remaining points, the leading tool depends on the dataset.Source: C. Point-wise Performance… · p. 6, Table V · p. 7
  • On Zhejiang, both DL DELINEATOR configurations lead at every fiducial point, including Toff sensitivity 85.4 vs. 45.4% for ECGdeli.On LUDB, supervised fine-tuning falls short of the best tool at Poff and Ton, while semi-supervised fine-tuning exceeds all tools at both.Source: C. Point-wise Performance… · p. 6, Table V · p. 7

D. Rhythm-stratified Subgroup Analysis

  • Under arrhythmia on RDB, DL DELINEATOR achieves mIoU 64.9 vs. 38.7%, interval error 28.5 vs. 38.2 ms, and sensitivity 80.4 vs. 67.3% for the best tools.Its sinus-to-arrhythmia degradation is smallest for mIoU (14.7 and 16.2 vs. 41.6 and 48.5%) and sensitivity (16.7 vs. 17.1% for ECGdeli).Source: Table VI · p. 7, D. Rhythm-stratified Subgroup Analysis · p. 7

E. Qualitative Analysis

Qualitative examples show existing tools becoming unreliable at waveform boundaries and under arrhythmia, while the DL DELINEATOR remains aligned with reference annotations. Pretraining improves delineation, but its value depends on the objective and on whether semi-supervised fine-tuning has room to help.

E. Qualitative Analysis

  • The qualitative examples span methods and datasets, with rhythm classes reflecting each dataset’s characteristics and the plotted detections visually aligning with quantitative results.Fig. 2 groups rows into reference annotations, rule-based tools, and the DL DELINEATOR, with shaded bands for detected P, QRS, and T waves.Source: E. Qualitative Analysis · p. 7, Figure 2 · p. 8
  • On sinus rhythms, most methods recover overall P, QRS, and T structure, but NeuroKit2 still misses waves or narrows the T wave.These failures align with low point-wise sensitivity at Pon for CWT and at QRS and T boundaries for DWT.Source: E. Qualitative Analysis · pp. 7–8
  • Across datasets, existing tools break down under arrhythmia, whereas the DL DELINEATOR stays close to reference annotations across rhythm types.On Zhejiang with PVCs, both NeuroKit2 variants fail, ECGdeli misses P waves, and Prominence misses P and QRS waves preceding an ectopic beat; on RDB, ECGdeli and Prominence produce spurious P waves under AFIB and SVT.Source: E. Qualitative Analysis · p. 8

A. Effectiveness of Label-Efficient Deep Learning

  • Self-supervised pretraining improves performance across test sets, with the clearest gains on external sets, but transfer depends on the objective.MAE and MoCo transfer well, whereas MERL underperforms; the authors suggest that ECG–text alignment may encourage invariance to fine morphology needed for delineation.Source: A. Effectiveness of Label-Efficient… · p. 8
  • Semi-supervised fine-tuning helps most without pretraining, remains beneficial for MoCo, and adds negligibly for MAE, whose supervised performance is already saturated.Its benefits do not simply add to those of pretraining; across the examined objectives, it helps when room remains and at least does not degrade performance.Source: A. Effectiveness of Label-Efficient… · pp. 8–9

B. The Advantage of DL DELINEATOR over Existing Tools

Across the tested datasets and metrics, the DL DELINEATOR outperforms existing tools, including under arrhythmia. Boundary-specific differences and rhythm-dependent tool trade-offs motivate a single robust delineator, while the study’s tested objectives and data impose scope limits.

B. The Advantage of DL DELINEATOR over Existing Tools

  • Across the internal and three external sets, the DL DELINEATOR outperforms every tool on all three metrics, with statistically significant improvements; on mECGDB it again has the lowest error.The advantage holds under both fine-tuning strategies, and the comparison provides evidence of a consistent advantage over open-source and commercial tools.Source: B. The Advantage of DL DELINEATOR over… · p. 9

C. Performance across Fiducial Points and Rhythms

  • Semi-supervised fine-tuning closes supervised residual gaps at Poff and Ton on LUDB, while T-wave boundaries—especially Ton—remain the model’s weakest region.The single-model comparison supports reading fine-tuning differences as tendencies; T-wave boundaries are difficult for every tool because of the gradual, low-amplitude onset.Source: C. Performance across Fiducial Points… · p. 9
  • The DL DELINEATOR remains the most accurate method on every metric under arrhythmia, whereas existing tools lose accuracy in abnormal rhythms.The authors suggest learned representations capture morphological variability that fixed heuristics cannot; rule-based tool performance on predominantly sinus recordings may not carry over to arrhythmia.Source: C. Performance across Fiducial Points… · p. 9

D. From Tool Selection to a Single Robust Delineator

  • Prominence is a reasonable choice for predominantly sinus recordings and Pon localization, while ECGdeli is preferable for QRS targets or mixed and unknown rhythms.Tool selection under arrhythmia is difficult because both degrade sharply and choosing between them requires knowing rhythm in advance; the DL DELINEATOR avoids switching tools and runs faster than these tools.Source: D. From Tool Selection to a Single… · p. 9

E. Limitations and Future Work

  • The study tests only three self-supervised objectives with one backbone, and mECGDB has interval-level labels alone, limiting uncontrolled-condition evidence to one metric.The authors identify broader objective exploration and recordings annotated across more devices as next steps.Source: E. Limitations and Future Work · p. 9

VII. CONCLUSION

The study finds that MAE-pretrained DL DELINEATOR models outperform established tools across datasets and metrics, with a widening margin under arrhythmia. Its conclusions cover one internal and four external test sets, while uncontrolled-condition evidence is limited by interval-only labels.

VII. CONCLUSION

  • With MAE pretraining, the DL DELINEATOR outperforms every tool on all three metrics across datasets under both fine-tuning configurations, with a widening margin under arrhythmia.The improvement is statistically significant in nearly all comparisons; tool differences concentrate at P and T waves and under arrhythmia, while the model remains robust.Source: §VII · pp. 9–10

IX. EVALUATION METRICS

  • The comparison covers one internal and four external test sets and uses three complementary metrics derived consistently from the same predicted segmentations.The metrics are mIoU, averaged interval error for PR, QRS, and QT, and averaged point-wise sensitivity, with corresponding PPV and localization error at 40 ms and 150 ms tolerances.Source: §VII · p. 9, §IX · p. 12

DATA AVAILABILITY

  • All datasets except mECGDB are publicly available; mECGDB is private and requestable from the corresponding author subject to institutional approval and user consent.Its privacy and ethical restrictions prevent sharing.Source: DATA AVAILABILITY · p. 10

VIII. DATASET DETAILS

  • The internal QTDB split is subject-wise at 6:2:2, while ISP uses its predefined test split and divides remaining recordings 6:2 between training and validation.The provided supplementary tables report internal sample counts and rhythm composition for external test sets.Source: §VIII · p. 12, Table S1 · p. 12, Table S2 · p. 12

A. Rhythm-stratified Subgroup Analysis

The rhythm-stratified analysis defines degradation by comparing sinus and arrhythmia subgroup metric values, with sign conventions preserving the interpretation that positive values mean worse arrhythmic performance. The supplementary details also describe the tool implementations, CalECG’s unedited algorithmic baseline, and training configurations.

A. Rhythm-stratified Subgroup Analysis

  • Relative degradation compares metric values on sinus and arrhythmia subgroups; positive Δ denotes arrhythmia degradation, with the sign reversed for averaged interval error because lower is better.A negative Δ under this convention denotes improved performance on the arrhythmia subgroup.Source: A. Rhythm-stratified Subgroup Analysis · p. 12

A. Open-source tools

  • NeuroKit2, ECGdeli, and Prominence use publicly available implementations configured according to their documented examples.Source: A. Open-source tools · p. 12

B. CalECG (AMPS LLC, New York, NY)

  • CalECG results are unedited algorithmic output, not the tool’s intended-use accuracy, because its annotation environment expects human inspection and correction.The initial output indicates substantially more manual correction under arrhythmia than in sinus rhythm.Source: B. CalECG (AMPS LLC, New York, NY) · p. 13

XI. MODEL TRAINING CONFIGURATIONS

  • The experiments use Mean Teacher for semi-supervised learning and evaluate MAE, MoCo, and MERL for self-supervised pretraining.Optimization and architecture-specific hyperparameters are detailed in supplementary tables.Source: §XI · p. 13, Table S4 · p. 13, Table S5 · p. 13

A. Statistical Analysis and Seed Robustness

The full benchmarking results report that MAE pretraining generally performs best and the DL DELINEATOR has the smallest performance degradation from sinus rhythm to arrhythmia. Across three seeds, results are robust in nearly all tested comparisons, and the DL DELINEATOR is fastest on LUDB.

A. Statistical Analysis and Seed Robustness

  • Paired bootstrap tests assess per-ECG metrics, while paired Wilcoxon signed-rank tests compare pretraining objectives and average ranks; p-values are Holm-corrected.The objective tests pair dataset-by-fine-tuning-configuration blocks, and the average-rank tests pair metric-by-dataset conditions.Source: A. Statistical Analysis and Seed… · p. 13
  • 94 of 96 Table III comparisons and 38 of 40 Table IV comparisons remained significantly superior across three seeds; mIoU was significantly superior in every combination and seed.Exceptions involved Prominence; only averaged interval error on RDB reversed direction, by 0.6 ms in one seed.Source: A. Statistical Analysis and Seed… · p. 13

B. Computational Cost

  • 17.1 ms per sample is the DL DELINEATOR’s mean LUDB runtime, ahead of Prominence (20.0 ms), NeuroKit2 variants (56.3 and 63.6 ms), and ECGdeli (141.0 ms).Its runtime SD is 1.9 ms versus at least 6 ms for the other methods; NeuroKit2 produced no output on 7 samples, excluded from its statistics.Source: B. Computational Cost · p. 14, Table S7 · p. 14

C. Full Benchmarking Results

  • The DL DELINEATOR maintains the smallest performance degradation from sinus rhythm to arrhythmia, while MAE pretraining generally performs best among learning strategies.The full results also compare interval error, point-wise sensitivity, tool performance, and per-fiducial localization error; CalECG is reported separately because it outputs only fiducial points required for interval computation.Source: C. Full Benchmarking Results · p. 14, Table S14 · p. 17

XIII. QUALITATIVE RESULTS

The qualitative figures compare reference annotations, rule-based tools, and the DL DELINEATOR on representative 2.5-second Lead II segments, with shaded bands marking detected P, QRS, and T waves.

XIII. QUALITATIVE RESULTS

  • For LUDB, Fig. S1 shows reference, rule-based, and DL delineations on the same representative Lead II segment.Shaded bands mark detected P, QRS, and T waves.Source: Figure S1 · p. 18
  • For Zhejiang, Fig. S2 shows reference, rule-based, and DL delineations on the same representative Lead II segment.Shaded bands mark detected P, QRS, and T waves.Source: Figure S2 · p. 19
  • For RDB, Fig. S3 shows reference, rule-based, and DL delineations on the same representative Lead II segment.Shaded bands mark detected P, QRS, and T waves; examples include sinus rhythm, atrial fibrillation, and supraventricular tachycardia.Source: Figure S3 · p. 20
Loading 2610.07885v1…