Source-linked AI summary
Beat-Synchronous Tokenization for ECG Transformers
Ahmed Sameh, Nolan Wilson, Max Enderlein, Yogatheesan Varatharajah
TL;DR
Fixed temporal patching can split heartbeat structures, so this paper evaluates beat-synchronous tokenization as a physiologically grounded alternative under matched pretraining and downstream protocols. Across PTB-XL and Icentia11k, the strongest beat-based variants retain competitive performance with substantially shorter token sequences, while results depend on the metric and beat encoder design.
Problem
Fixed temporal patches can split cardiac cycles and represent different physiological content across recordings, while the isolated effect of tokenization remains unclear.
Method
The study compares fixed patches with resampled, adaptive-pooled, and R–R-augmented beat tokens across matched Transformer evaluations on PTB-XL and Icentia11k.
Results
Across PTB-XL and Icentia11k, beat-synchronous tokenizers remain competitive with fixed patching while using fewer tokens; Tok1 reaches 0.8945 mean macro AUROC on PTB-XL.
Takeaways & Limitations
Beat-synchronous tokenization is a compact alternative to fixed temporal patching, but practical gains depend on beat-encoder design, padding strategy, and evaluation metric.
Abstract
from arXiv · showhide
Transformer-based electrocardiogram (ECG) models commonly tokenize waveforms into fixed temporal patches. Though convenient, fixed patching can split heartbeat structures across token boundaries. We study beat-synchronous tokenization as a physiologically grounded alternative, comparing fixed patches with three beat-aligned strategies: resampled beats, adaptive pooled beats, and resampled beats augmented with R--R interval information. Experiments span two settings: 10-second 12-lead diagnostic classification on PTB-XL after MIMIC-IV-ECG masked pretraining, and 60-second single-lead rhythm classification on Icentia11k after patient-level contrastive pretraining. On PTB-XL, resampled beat tokens achieve the highest mean macro Area Under the ROC Curve (AUROC; 0.8945) and nearly match the best fixed-patch macro Area Under the Precision-Recall Curve (AUPRC; 0.7414), reducing average sequence length from 100 to 11.2 tokens. On Icentia11k, beat-synchronous tokenizers obtain comparable AUPRC to fixed patching with better stability across runs. These results suggest morphology-preserving beat tokenization is a compact, competitive alternative to fixed temporal patching.
1. INTRODUCTION
Fixed temporal patches can disrupt physiologically meaningful ECG structure, motivating a controlled comparison of beat-synchronous tokenizers across diagnostic and rhythm classification settings.
- Motivation: Fixed patches may split cardiac cycles or mix adjacent beats, while identical patch lengths represent different physiological content as heart rate varies.ECG interpretation can depend on beat morphology, inter-beat timing, and rhythm-level organization.
- Research gap: Prior beat-aware ECG studies often combine tokenization with architectural or training changes, leaving the isolated effect of tokenization unclear.
- Approach: The study compares fixed patches with resampled, adaptive-pooled, and R–R-augmented beat tokens under matched Transformer pretraining and evaluation protocols.
- Evaluation: The evaluation covers 12-lead PTB-XL diagnostic classification after MIMIC-IV-ECG masked reconstruction pretraining and 60-second Icentia11k rhythm classification after contrastive pretraining.
- Findings: On PTB-XL, resampled beat tokens achieve the highest mean macro AUROC, while beat-synchronous models use substantially shorter sequences and retain strong downstream performance.
2. RELATED WORK
Related work shows growing interest in physiologically structured ECG representations, but it does not isolate whether beat-synchronous token boundaries themselves improve performance and efficiency.
- Self-supervised learning: Self-supervised ECG representation learning uses contrastive objectives or masked reconstruction to learn from large unlabeled datasets.
- Beat-aware representations: Beat-aware studies model morphology, lead-specific spatial information, rhythm structure, or ECG-text alignment through specialized representations and training schemes.
- Position of this work: Because prior approaches entangle tokenization with other modeling changes, the independent effect of tokenization on the performance–efficiency tradeoff remains unresolved.
3. METHODOLOGY
The methodology treats tokenization as an isolated design choice, mapping ECG segments into Transformer-ready sequences using fixed temporal patches or cardiac-cycle tokens defined by R-peaks.
- Common pipeline: Each tokenizer maps an ECG segment X to a token sequence {z_i} processed by a Transformer and pooled into a segment-level representation.
- Controlled comparison: The comparison keeps the Transformer-style encoder and downstream evaluation protocol comparable across tokenizers.
- Beat-token designs: Tok3 adds R–R interval timing information to beat tokens, whereas Tok2 uses adaptive pooling as an alternative beat encoder.
- Fixed temporal patches: Fixed tokenization partitions ECGs into non-overlapping patches of length p and embeds them with a one-dimensional convolution using kernel size and stride p.
- Beat-synchronous tokens: Beat-synchronous tokenization extracts each beat from the interval [r_i, r_i+1) between consecutive R-peaks, so sequence length follows detected beat count.
- Beat-token designs: Tok1 resamples each variable-length beat to a fixed representation, using 12 × 300 segments for 12-lead data and 1 × 160 segments for single-lead data.
4. EXPERIMENTAL SETUP
Experiments use public ECG datasets spanning unlabeled 12-lead pretraining, long-context single-lead rhythm evaluation, and publicly released code and pretrained models.
- Pretraining data: MIMIC-IV-ECG supplies unlabeled 10-second, 12-lead recordings sampled at 500 Hz for pretraining.Records have tensor shape 12×5000 and undergo corruption filtering, interpolation, amplitude clipping, and per-lead normalization.
- Downstream evaluation: Icentia11k provides 60-second single-lead rhythm data sampled at 250 Hz from 11,000 patients for long-context evaluation.Patients are divided into SSL pretraining, supervised training, validation, and held-out test groups.
- Reproducibility: The study uses patient-level data organization for Icentia11k and reports publicly available datasets, code, and pretrained models.
5. RESULTS AND DISCUSSION
Across PTB-XL and Icentia11k, morphology-preserving beat-synchronous tokenizers remain competitive with fixed patches while using shorter sequences, though performance depends on beat encoding and metric.
- PTB-XL diagnostic classification: 0.8945 macro AUROC: Tok1 achieves the highest mean PTB-XL diagnostic score and nearly matches fixed p = 50 on macro AUPRC.Fixed p = 50 reaches 0.8903 macro AUROC and 0.7419 macro AUPRC; Tok1’s macro AUPRC is 0.7414.
- Token efficiency: 100 to 11.2 tokens: Tok1 and Tok3 reduce average PTB-XL sequence length by almost an order of magnitude while retaining comparable performance.Fixed p = 50 uses 100 tokens, whereas Tok1 and Tok3 use 11.2 beat tokens on average.
- Model-side efficiency: 33.0% inference-time reduction and 22.4% peak-GPU-memory reduction: Tok1 improves model-side PTB-XL efficiency relative to fixed p = 50.On Icentia11k, dense batch padding largely removes the real-token reduction’s inference-time and memory advantages.
- Beat-token design: Beat alignment alone is insufficient: Tok2 performs substantially worse than Tok1 and Tok3, indicating that beat encoding must preserve diagnostically useful morphology.Tok2 adaptively pools each variable-length beat into one token after convolutional processing, whereas Tok1 and Tok3 resample each cardiac cycle before convolution.
- Icentia11k rhythm classification: 0.8515 mean AUPRC: Tok3 essentially matches fixed patching on imbalanced Icentia11k rhythm classification, while fixed patching retains the highest AUROC at 0.9888.Tok1 reaches 0.8514 mean AUPRC, and Tok3’s difference from fixed patching is negligible.
- Stability under class imbalance: 0.0676 to 0.0076: Tok1 has lower Icentia11k AUPRC run-to-run variability than fixed patching.Tok3 also has a smaller standard deviation of 0.0202, with identical frozen validation and test windows across tokenizers.
6. CONCLUSION
The study finds that beat-synchronous tokenization can provide competitive ECG classification with shorter, physiologically meaningful sequences across diagnostic and rhythm settings. However, reliable R-peak localization is a scope boundary because sensitivity to missed or shifted peaks was not evaluated.
- 6. CONCLUSION: Tok1 and Tok3 match fine fixed-patch PTB-XL performance with substantially fewer tokens and achieve comparable, more stable AUPRC on imbalanced Icentia11k rhythm classification.The adaptive-pooling variant performs poorly, showing that beat-token design must preserve morphology and, when relevant, timing information.
- 6. CONCLUSION: Reliable R-peak localization is required, but sensitivity to missed or shifted R-peaks was not evaluated.Noisy recordings may require robust beat detection or fallback fixed-patch tokenization.