Source-linked AI summary
Does the Heart Show Your Pain? Tackling the X-ITE Pain Challenge with Self-Supervised ECG Representation Learning
Dominika Kunc, Przemysław Kazienko, Stanisław Saganowski
TL;DR
Pain recognition from ECG is difficult because wearable-friendly single-modality evidence remains limited. This paper evaluates self-supervised ECG representations, with multimodal ECG–accelerometer pretraining, finding limited overall performance but improved representations and strong subject-to-subject variation.
Problem
Wearable pain assessment needs evidence on whether an accessible single modality such as ECG can support recognition outside laboratory settings.
Method
The study pretrains self-supervised representations on unlabeled ECG, with an additional multimodal ECG–accelerometer setup, then fine-tunes for low-versus-medium pain classification.
Results
ECG-only classification performance was limited overall, while multimodal pretraining improved downstream performance and model results varied substantially across subjects.
Takeaways & Limitations
The findings support exploring cross-modal pretraining and personalized or subject-adaptive models for ECG-based pain recognition.
Takeaways & Limitations
Strong inter-subject differences make personalized or adaptive modeling an important future direction.
Abstract
from arXiv · showhide
Accurate recognition of pain using physiological signals remains a challenging problem due to pain's subjective nature and high inter-individual variability. In this study, we investigate self-supervised representation learning (SSL) methods applied to unimodal electrocardiogram (ECG), complemented by multimodal pretraining, including accelerometer (ACC) signals from the chest. We focus on classifying low versus medium pain levels on the X-ITE Pain dataset. Our results reveal that while ECG-based models show limited classification performance, multimodal pretraining improves learned representations by capturing cross-modal dependencies. Notably, we observe substantial inter-subject variability in model performance, suggesting that pain-related ECG patterns may be subject-specific. Visualizations indicate distinct subject-specific clustering but no clear separation by pain levels, highlighting the complexity of pain detection from ECG alone. We discuss limitations of unimodal input, label noise, and generalization across subjects and propose future directions. This work advances the understanding of physiological signal representation learning for pain recognition and sets the stage for more robust, clinically relevant wearable pain monitoring solutions.
I. INTRODUCTION AND RELATED WORK
Wearable physiological signals offer a promising but challenging route to objective pain assessment, motivating ECG-only self-supervised representation learning for low-versus-medium pain classification. The study finds that average performance is only slightly above chance, with substantial inter-subject variability suggesting potential for personalized modeling.
- Motivation: Wearable physiological signals could provide continuous, objective pain assessment when self-reports are subjective or infeasible.They reflect autonomic nervous system activity and have been explored for pain recognition.
- Problem: Real-world deployment is constrained because many experimental modalities are unavailable in commercial wearables and multimodal data collection is impractical.These constraints motivate investigating a single accessible modality such as ECG.
- Problem: ECG is widely available in smartwatches, chest straps, and health patches, but stress, activity, and cardiovascular differences can obscure pain-related patterns.ECG can be monitored relatively unobtrusively and raises fewer privacy concerns than cameras or microphones.
- Approach: The study evaluates self-supervised learning for ECG representation learning and fine-tunes on X-ITE Pain for binary low-versus-medium pain classification.Pretraining uses unlabeled ECG data to learn general-purpose features that may capture subtle pain-related dynamics.
- Findings: Average F1 scores across subjects are only slightly above random guessing, while performance varies substantially between individuals.Strong performance for some subjects motivates personalized and subject-specific modeling or adaptation.
II. METHODOLOGY · A. X-ITE Pain dataset
The study uses the X-ITE Pain Database, a multimodal physiological and audiovisual dataset recorded from 134 participants during short and one-minute pain stimuli. Analysis is restricted to its challenge subset of 26 training and four testing subjects, using only ECG to distinguish low from medium pain.
- A. X-ITE Pain dataset: The X-ITE Pain Database was recorded from 134 participants at Otto von Guericke University Magdeburg in collaboration with the University Clinic Ulm.
- A. X-ITE Pain dataset: The dataset includes frontal and side-view color face video, frontal thermal video, and color and depth body video.
- A. X-ITE Pain dataset: It also contains audio, EDA, ECG, and sEMG recorded at the trapezius, corrugator, and zygomaticus muscles.
- A. X-ITE Pain dataset: Signals and audiovisual data were recorded during phasic short pain and tonic pain lasting 1 minute.
- A. X-ITE Pain dataset: The study uses only the challenge subset, comprising 26 subjects for training and four subjects for testing.
- A. X-ITE Pain dataset: The classification task focuses exclusively on low and medium pain levels.
- A. X-ITE Pain dataset: The approach uses only the ECG signal from the X-ITE Pain dataset.
B. SSL – Representation Learning Model · III. RESULTS
The study uses ECG-focused self-supervised representation learning with TS-TCC adaptations, then evaluates unimodal and multimodal pretraining for low-versus-medium pain classification under subject-independent validation.
- B. SSL – Representation Learning Model: TS-TCC learns general-purpose representations from 1-lead ECG without labels using temporal and contextual contrastive objectives.Temporal contrasting predicts future representations from past context, while contextual contrasting promotes consistency across augmented views.
- B. SSL – Representation Learning Model: Pretraining uses continuous ECG recordings from 24 individuals, segmented into 10second windows, resampled to 100 Hz, and z-scored within subject.Signals were cleaned with NeuroKit2, with minimal preprocessing intended to preserve natural signal characteristics.
- B. SSL – Representation Learning Model: The ECG encoder adds two convolutional layers to the original TS-TCC architecture and is trained with weak and strong augmentations.Training remains guided by temporal and contextual contrastive losses.
- B. SSL – Representation Learning Model: Pretrained representations are fine-tuned for downstream pain classification using a two-layer ReLU MLP classification head.LOSO cross-validation uses 23 training subjects, two validation subjects, and one test subject per split.
- III. RESULTS: The evaluation targets binary low-versus-medium pain classification on the X-ITE Pain dataset using SSL-pretrained ECG models.The experiment compares unimodal and multimodal encoder pretraining variants.
- III. RESULTS: Classification training is evaluated with either a frozen encoder or a fine-tuned encoder.Frozen training updates only the classification head, whereas fine-tuned training updates all model weights.
A. Overall Performance
Overall, fine-tuned and multimodally pretrained models outperform their frozen and unimodally pretrained counterparts, respectively. Nevertheless, the best model achieves only modest ECG-based pain-classification performance, averaging 0.51 macro F1 and 0.54 accuracy.
- Overall Performance: Fine-tuned models perform better than frozen models in each comparison.The comparison concerns overall macro-averaged F1-score and classification accuracy.
- Overall Performance: Models pretrained multimodally perform better than models pretrained unimodally, indicating benefits from cross-modal learning.Chest ACC pretraining may capture breathing-related chest movement, which could enhance pain recognition because pain affects respiration.
- Overall Performance: 0.51 average F1-score and 0.54 average accuracy were achieved by the best-performing model, only slightly exceeding random guessing.These metrics summarize binary pain classification across all subjects using leave-one-subject-out cross-validation.
B. Subject-Level Performance · C. Performance Distribution
Per-subject results showed substantial variability, with performance ranging from strong to near-random levels. The long-tailed distributions suggest subject-specific ECG patterns and motivate personalized modeling, although limited individual-level data prevented explaining poor performance.
- B. Subject-Level Performance: Per-subject F1-scores exceeded 0.7 for some individuals but fell close to random levels for others.These findings demonstrate substantial inter-subject variability in classification performance.
- B. Subject-Level Performance: The study could not determine why performance was poor for specific individuals because demographic and pain-related information was unavailable.Missing data included demographic information, pain-induction indicators, and pain-perception scales.
- B. Subject-Level Performance: Incorporating personalized information into the models was proposed as a way to improve performance.The authors identify individual-level differences as a promising direction for future research.
- B. Subject-Level Performance: Table I reports each subject’s F1-score and accuracy for the best-performing model.Subjects with stronger performance are highlighted in bold in the table.
- C. Performance Distribution: Figure 2 shows long-tailed distributions of per-subject macro F1-scores and accuracies.The distributions visualize substantial variability across subjects.
- C. Performance Distribution: The long tail suggests that the model may capture meaningful pain-related ECG patterns for specific individuals despite struggling overall.This interpretation supports personalized or subject-adaptive modeling approaches.
D. Representation space interpretability
TSNE visualization was used to inspect structure in learned representations relative to ground-truth labels. The embeddings formed subject-specific clusters but showed no clear separation by pain level, indicating that individual physiological patterns outweighed pain-related ECG differences.
- Visualization method: TSNE projects high-dimensional embeddings into a lower-dimensional space for visual inspection of structure, patterns, or groupings relative to ground-truth labels.This enables representation assessment beyond downstream classification and clustering tasks.
- Subject-specific structure: The representations often formed subject-specific clusters, indicating that they captured individual physiological characteristics.Subject-level separability aligns with known inter-individual variability in physiological responses.
- Pain-level structure: No clear grouping or separation emerged when embeddings were examined by pain level, making ECG-only distinction between pain states challenging.Pain-induced ECG differences may be subtle or masked by stronger individual patterns.
IV. DISCUSSION
ECG-only pain recognition remained challenging despite state-of-the-art self-supervised learning, with performance varying substantially across individuals. Multimodal pretraining with chest accelerometer data improved ECG representations by leveraging cross-modal physiological dependencies.
- ECG-only recognition: ECG-only pain recognition remained challenging, with limited overall classification performance despite state-of-the-art self-supervised learning methods.The discussion suggests ECG may contain little direct pain information or that pain-related features may be obscured by dominant physiological components.
- Inter-subject variability: Model performance showed high inter-subject variability, with some individuals achieving notably better results than the relatively low average accuracy.This suggests pain-related ECG patterns may be highly individual-specific and motivates personalized or subject-adaptive modeling.
- Multimodal pretraining: ACC data improved downstream performance when incorporated during multimodal pretraining.Because signals were collected from the chest, ACC may have captured subtle respiration-related body movements.
- Multimodal pretraining: Multimodal contrastive pretraining may associate respiratory cues from ACC with ECG patterns, enhancing ECG-derived representations at inference.This demonstrates the potential of cross-modal learning to uncover latent dependencies underutilized by unimodal setups.
V. LIMITATIONS AND FUTURE WORK
The study is limited by ECG-only input, inter-subject variability, and subjective, noisy pain labels. Future work should pursue personalized models, multimodal fusion, improved SSL, and methods that preserve ordinal pain relationships.
- Limitations: ECG-only input restricts available physiological information, while EDA or respiration could capture pain-related changes more effectively.Adding these modalities may improve pain-recognition performance.
- Limitations: Substantial inter-subject variability in ECG responses limits generalization and challenges universal pain-detection models.The findings motivate personalized or adaptive approaches.
- Limitations: Participant self-reports introduce inevitable label noise and inconsistencies, hindering learning of clear, consistent pain-related patterns.This remains a challenge even with the high-quality X-ITE Pain dataset.
- Future Work: Personalized or adaptive models that establish individual baselines may better account for unique physiological patterns and improve per-subject pain recognition.Strong inter-subject differences support this direction.
- Future Work: Multimodal fusion of EDA or respiration, including signals accessible in low-cost wearables, could disambiguate pain-related changes from confounding factors.Complementary physiological sources may strengthen recognition models.
- Future Work: Task-specific pretraining, pseudo-labels for weak supervision, and ordinal-aware methods such as regression or custom loss functions are proposed future directions.These approaches target relevant pain features and preserve relationships across low, medium, and high pain levels.
VI. CONCLUSIONS
The study finds that ECG-based pain recognition has limited overall accuracy, while substantial inter-subject variability points to the potential value of personalized approaches. Future progress requires addressing pain’s subjectivity, physiological confounders, label noise, and the ordinal and temporal structure of pain through multimodal and methodological advances.
- Conclusions: ECG-based models showed limited overall accuracy, but substantial inter-subject variability suggests personalized approaches may reveal more meaningful pain-related physiological patterns.The findings concern ECG signals collected in real-life conditions.
- Conclusions: Pain recognition remains difficult because pain is subjective and affected by confounding physiological factors and label noise.These challenges complicate learning reliable pain-related patterns from wearable physiological signals.
- Conclusions: Future research should pursue personalized modeling, multimodal fusion with EDA and respiration, and methods capturing pain’s ordinal and temporal nature.Advancing wearable pain recognition also requires methodological innovation and a nuanced understanding of individual differences and underlying physiology.
VII. ETHICAL IMPACT STATEMENT
The work examines non-invasive, privacy-preserving pain assessment from wearable ECG and self-supervised learning, while emphasizing subjectivity, deployment risks, human oversight, and fairness. Data were de-identified and handled in compliance with ethical standards.
- Potential Benefits: Wearable ECG enables passive, continuous pain monitoring without relying on sensitive personal content such as facial expressions or speech.The approach is presented as a less intrusive alternative to video, audio, or direct clinical observation.
- Risks and Safeguards: Pain is subjective, and physiological signals can be affected by emotional state, activity, and environmental conditions.These factors may limit how accurately automated systems reflect individual pain experiences.
- Risks and Safeguards: False positives and false negatives could cause inappropriate interventions or neglect, requiring human oversight and personalized calibration during deployment.The passage identifies both errors as risks for automated pain detection.
- Data Governance and Fairness: The study used de-identified data in compliance with ethical standards and did not infer sensitive attributes beyond those explicitly studied.Future research should examine fairness across age, gender, and health status groups to avoid exacerbating healthcare disparities.