Source-linked AI summary
Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Model
Jiarui Jin, Haoyu Wang, Hongyan Li, Jun Li, Jiahui Pan, Shenda Hong
TL;DR
ECG self-supervised learning commonly segments signals with fixed windows, which can overlook heartbeat form, rhythm, and latent semantic relationships. HeartLang treats heartbeats as words and rhythms as sentences, using QRS-Tokenizer and multi-level self-supervised objectives; it remained competitive across six public datasets, with strong PTB-XL gains over other eSSL methods. The approach is constrained by QRS irregularities and baseline drift that can impair tokenization.
Problem
Existing eSSL methods use fixed-size, fixed-step windows that can ignore ECG form, rhythm, and latent semantic relationships, limiting generalized representation learning.
Method
HeartLang uses QRS-Tokenizer-generated ECG sentences, heartbeat vocabulary construction, vector-quantized reconstruction, and masked sentence pre-training to learn form- and rhythm-level representations.
Results
HeartLang remained highly competitive across six public ECG datasets and improved average macro AUC over the second-best eSSL method by 8.14 on PTB-XL linear probing.
Takeaways & Limitations
Treating heartbeats as words and rhythms as sentences provides a supported framework for learning ECG representations from unlabeled recordings at multiple levels.
Takeaways & Limitations
QRS-Tokenizer performance may degrade for irregular QRS complexes and datasets with substantial baseline drift, partly because zero padding can disrupt heartbeat characteristics.
Abstract
from arXiv · showhide
Electrocardiogram (ECG) is essential for the clinical diagnosis of arrhythmias and other heart diseases, but deep learning methods based on ECG often face limitations due to the need for high-quality annotations. Although previous ECG self-supervised learning (eSSL) methods have made significant progress in representation learning from unannotated ECG data, they typically treat ECG signals as ordinary time-series data, segmenting the signals using fixed-size and fixed-step time windows, which often ignore the form and rhythm characteristics and latent semantic relationships in ECG signals. In this work, we introduce a novel perspective on ECG signals, treating heartbeats as words and rhythms as sentences. Based on this perspective, we first designed the QRS-Tokenizer, which generates semantically meaningful ECG sentences from the raw ECG signals. Building on these, we then propose HeartLang, a novel self-supervised learning framework for ECG language processing, learning general representations at form and rhythm levels. Additionally, we construct the largest heartbeat-based ECG vocabulary to date, which will further advance the development of ECG language processing. We evaluated HeartLang across six public ECG datasets, where it demonstrated robust competitiveness against other eSSL methods. Our data and code are publicly available at https://github.com/PKUDigitalHealth/HeartLang.
1 INTRODUCTION
Existing ECG self-supervised methods often overlook heartbeat form, rhythm, and latent semantic relationships by using fixed time-window segmentation. HeartLang addresses these gaps by treating heartbeats as words and rhythms as sentences, with components for ECG sentence generation and multi-level representation learning.
- Motivation: Fixed-size, fixed-step segmentation can ignore ECG form and rhythm characteristics that support diagnoses across single heartbeats and overall cardiac rhythms.The passage contrasts myocardial-infarction diagnosis from ST-segment elevation in one heartbeat with atrial-fibrillation diagnosis from overall rhythm.
- Motivation: Subject- and sample-level heart-rate differences make fixed windows inconsistent, disrupting semantic relationships between heartbeats.These discrepancies can negatively affect learning generalized representations in self-supervised learning.
- Proposed framework: HeartLang treats heartbeats as words and rhythms as sentences to learn unlabeled ECG representations at both form and rhythm levels.This perspective is implemented as ECG language processing rather than ordinary time-series segmentation.
- Proposed framework: The QRS-Tokenizer generates ECG sentences from raw signals, while ST-ECGFormer uses spatio-temporal information for latent semantic extraction.The framework also includes heartbeat vocabulary construction and masked ECG sentence pre-training.
- Contributions: HeartLang constructs a heartbeat-based ECG vocabulary spanning morphological representations across cardiac conditions.The authors present this vocabulary as the largest ECG vocabulary to date.
2 RELATED WORK
ECG self-supervised learning uses contrastive or reconstruction objectives, while ECG language processing models heartbeat structure as semantic units. Existing ECG language methods face waveform-segmentation difficulties and small vocabularies, motivating HeartLang’s heartbeat-based approach.
- ECG self-supervised learning: eSSL methods broadly use contrastive learning, which compares positive and negative pairs, or reconstruction learning from partial or transformed inputs.These approaches aim to learn representations from unlabeled ECG signals.
- ECG self-supervised learning: Contrastive ECG methods model relationships such as cross-space, temporal, patient-level, inter-subject, and intra-subject similarities.Examples include CLOCS, ASTCL, ISL, and BTFS.
- ECG language processing: ECG language processing treats heartbeat information as semantically meaningful and applies approaches related to natural-language processing.Prior methods segment waveform components within heartbeats to construct vocabularies.
- ECG language processing: Existing ECG language methods may struggle to segment fine-grained waveforms across varying signal quality and use vocabularies of no more than 70 clusters.The related-work passage describes ECG language processing as an emerging and sparsely studied field.
- ECG language processing: HeartLang instead directly treats heartbeats as words and constructs a heartbeat-based vocabulary of 5,394 words.The paper presents this vocabulary as addressing limitations in vocabulary richness and waveform segmentation.
3 METHOD
HeartLang converts raw multi-lead ECG recordings into heartbeat-based ECG sentences, then learns form- and rhythm-level representations through tokenization, vector quantization, reconstruction, and masked pre-training.
- Framework overview: The framework processes multi-lead ECG data through QRS-based sentence generation, vocabulary construction, representation learning, and downstream fine-tuning.The framework overview defines four sequential stages, beginning with QRS-Tokenizer sentence generation and continuing through vocabulary construction and model training.
- Generating ECG sentences: QRS-Tokenizer detects QRS complexes after filtering and wavelet-based moving-wave integration, then uses detected indices to segment heartbeat patches.The I-lead signal is bandpass filtered between 5 and 20 Hz; local maxima satisfying refractory-period and threshold criteria are classified as QRS complexes.
- Generating ECG sentences: The segmented patches are treated as ECG words and concatenated across 12 leads into ECG sentences, padded or truncated to l = 256 with t = 96.Heartbeat patches smaller than t are zero-padded before concatenation, while sentence lengths are normalized to the maximum sequence length.
- ST-ECGFormer backbone: ST-ECGFormer uses a pre-layer-normalized transformer encoder to capture spatio-temporal features and latent semantic relationships in ECG sentences.The backbone supports vector-quantized heartbeat reconstruction, masked ECG sentence pre-training, and downstream fine-tuning.
- ST-ECGFormer backbone: A convolutional mapping transforms each ECG word into a token embedding, while learnable spatial, temporal, and positional embeddings preserve lead and sequence information.The resulting token features have dimension D, and zero-filled patches receive dedicated spatial and temporal embeddings.
- Vector-quantized heartbeat reconstruction: The ECG vocabulary maps interval representations to nearest collective heartbeat embeddings, enabling cross-subject discrete representations for reconstruction training.Vector quantization uses cosine similarity after ST-ECGFormer encoding; the decoder reconstructs the ECG sentence, with EMA updates and MSE loss stabilizing training.
- Masked ECG sentence pre-training: Masked ECG sentence pre-training hides individual ECG words and predicts their collective vocabulary words from the remaining sentence context to learn rhythm-level representations.The procedure consists of individual-word masking followed by prediction of the corresponding collective ECG word indices.
4 EXPERIMENTS
The experiments evaluate HeartLang using public ECG datasets, standardized preprocessing, and linear probing under multiple data-resource settings. Comparisons include eSSL baselines and a multimodal benchmark that is not directly compared.
- Benchmark comparison: HeartLang is compared with other eSSL methods rather than MERL because MERL uses additional ECG-Text clinical-report supervision.
- Pre-training dataset: MIMIC-IV-ECG contains 800,035 12-lead recordings from 161,352 subjects, sampled at 500 Hz for 10 seconds each.
- Implementation: Records are downsampled to 100 Hz and converted into unified ECG sentences with the QRS-Tokenizer before training or fine-tuning.
- HeartLang is evaluated against eSSL methods using linear probing results summarized in Table 1.
- Downstream datasets: The downstream evaluation covers PTB-XL, CPSC2018, and Chapman-Shaoxing-Ningbo datasets spanning multiple cardiac-condition labels.
- Implementation: Linear probing uses frozen ST-ECGFormer features and tests 1%, 10%, and 100% of each task’s training data.
5 RESULTS AND DISCUSSIONS
HeartLang performs strongly across ECG representation-learning evaluations, especially when heartbeat- and rhythm-aware slicing is used. Its vocabulary captures morphology together with contextual spatio-temporal and positional information, while results vary across datasets and training perspectives.
- Linear probing results: 8.14 average macro AUC improvement over the second-best eSSL method is reported on PTB-XL across 1% to 100% training data.
- Signal slicing perspective: Traditional slicing achieves higher masked prediction accuracy partly because its fixed 120 patches create more zero-filled patches than HeartLang’s dynamic, heart-rate-dependent patch count.
- Linear probing results: HeartLang remains highly competitive on CPSC2018 and CSN, but it surpasses other methods on CSN only with 100% training data.
- Signal slicing perspective: The heartbeat-as-words perspective yields clearer semantic patches, stronger VQ-HBR training, and 5.36 average macro AUC improvement in downstream PTB-XL evaluation.
- Signal slicing perspective: 8.38 average macro AUC improvement is observed on PTB-XL Superclass and Subclass subsets with the proposed slicing perspective.
- ECG vocabulary visualization: Vocabulary visualization shows morphology-specific indices, including ST-segment depression at index 1193 and a wide QRS complex at index 5171.
- ECG vocabulary visualization: Spatio-temporal and positional embeddings let similar heartbeats map to different collective words according to context, enriching the vocabulary’s semantic representation.
6 CONCLUSION
HeartLang frames ECGs as language, using heartbeat words and rhythm sentences to learn form- and rhythm-level representations. The paper also reports a heartbeat-based vocabulary and evaluation across six public ECG datasets.
- HeartLang treats heartbeats as words and rhythms as sentences for ECG language processing.The framework uses this perspective to model ECG signals at form and rhythm levels.
- The framework learns form-level representations through vector-quantized heartbeat reconstruction and rhythm-level representations through masked ECG sentence pre-training.
- The authors construct the largest heartbeat-based ECG vocabulary to date, covering morphological representations across cardiac conditions.
- HeartLang is evaluated across six public ECG datasets and demonstrates robust competitiveness against other eSSL methods.
- VQ-HBR training and ablation pre-training use four NVIDIA GeForce RTX 4090 GPUs with a batch size of 64 per GPU.
A.2 DATASET SPLIT
The study specifies dataset splits for MIMIC-IV-ECG, PTBXL, CPSC2018, and CSN using dataset-specific or established partitioning procedures.
- MIMIC-IV-ECG training and validation sets are split at a 9:1 ratio.The validation set is used during VQ-HBR training but not during pre-training.
- PTBXL uses the official split from its original work.
- CPSC2018 and CSN follow the division method provided by MERL.
B.1 EVALUATION ON DIFFERENT VOCABULARY SIZES
Larger ECG vocabularies improve downstream performance compared with a reduced vocabulary of 64, which limits semantic expression.
- A vocabulary size of 64 limits semantic expressions compared with larger vocabulary configurations.
- Larger vocabulary sizes lead to significant downstream performance improvements.The authors attribute this to richer semantic representations and increased pre-training complexity.
B.2 EVALUATION ON FEWER LEAD CONFIGURATION
HeartLang generally benefits from more ECG leads, particularly for disease-diagnosis subsets, while retaining strong performance in the single-lead setting.
- Downstream performance generally improves as the number of ECG leads increases.
- The largest gains occur in the Superclass and Subclass disease-diagnosis subsets.
- Even with a single lead, HeartLang surpasses most baseline methods in downstream tasks.This result supports its adaptation to single-lead configurations.
B.3 LIMITATIONS AND FUTURE WORKS
The QRS-Tokenizer has limitations with irregular QRS complexes and baseline drift, motivating improvements for broader ECG conditions.
- Irregular QRS complexes can cause the QRS-Tokenizer to represent atypical heartbeat patterns inaccurately, degrading performance.The tokenizer relies on QRS complex features as a critical component of its functionality.
- Zero-padding intervals shorter than 96 samples can partially disrupt heartbeat characteristics in datasets with substantial baseline drift.Baseline deviations from zero make this padding strategy less reliable for such datasets.
- Future work should improve robustness to irregular QRS complexes and baseline drift across diverse cardiac conditions.
C MORE VISUALIZATION RESULTS
The visualizations illustrate semantically rich ECG words and sentences, including morphology-preserving reconstructions and heart-rate-adaptive segmentation.
- Additional individual and collective ECG word visualizations demonstrate the semantic richness of the constructed ECG vocabulary.
- The ECG sentence reconstruction is smoother while capturing heartbeat morphology and showing strong generalization capabilities.Figures 6 visualizes original signals in blue and reconstructed signals in red.
- Fast heart rates use the full ECG sentence length of 256 for individual ECG words.Figure 7 illustrates this adaptive segmentation behavior.
- Slow heart rates use only part of the ECG sentence, with the remaining sections zero-filled.Figure 8 illustrates the shorter occupied portion produced by slower rhythms.