Source-linked AI summary
Emotion Recognition from Physiological Signals Using Machine Learning Algorithms Under Controlled Emotional Stimuli
Aditi Site, Annariina Lohiranta, Tarmo Lipping
TL;DR
Emotion recognition from physiological signals is studied for well-being analysis and affective computing, with the paper examining whether ECG and GSR can support classification across discrete emotions, valence, and arousal. The study extracts features from recorded signals and evaluates traditional supervised models under controlled emotional stimuli. ECG+GSR performs best across the reported comparisons, while lower-granularity discrete emotions and arousal achieve the strongest results.
Problem
The study asks whether ECG and GSR can support emotion recognition and which target representation and model achieve better classification performance.
Method
The study records ECG and GSR during controlled emotional stimuli, extracts time- and frequency-domain features, and evaluates supervised traditional machine-learning models.
Results
ECG+GSR performed best in the reported comparisons, with arousal reaching 53.4% and four-class discrete emotions reaching 53.0% using Random Forest.
Takeaways & Limitations
The results support the feasibility of using Shimmer-recorded ECG and GSR with traditional models for recognizing discrete emotions and valence/arousal.
Abstract
from arXiv · showhide
Emotion recognition using physiological signals plays a crucial role in well-being analysis, affective computing and human-computer interaction. This study investigates the performance of multiple machine learning models in classifying targets such as discrete emotions with varying granularity, valence and arousal using physiological signals such as Electrocardiogram (ECG) and Galvanic Skin Response (GSR). In here, we extracted various time- and frequency-domain features from the ECG and GSR data to train machine learning models. The results indicate that categorizing discrete emotions with fewer emotions and categorizing arousal achieves good classification accuracy with tree-based models. XGBoost achieved accuracy of 52.8 % for classifying discrete emotions and Random Forest achieved accuracy of 53.6% for classifying arousal. In both the cases, the combination of ECG+GSR feature sets achieved best performance. These findings highlight the effectiveness of physiological signals in capturing emotional states and support their use for emotion recognition systems.
I. INTRODUCTION
The study examines physiological-signal emotion recognition using ECG and GSR, comparing target representations and traditional machine-learning models. It focuses on whether these signals and lightweight models can support emotion classification across valence, arousal, and discrete-emotion classes.
- Motivation: Physiological signals such as ECG and GSR offer objective measurements for emotion recognition, complementing behavior-based cues that can be consciously manipulated or culturally sensitive.The broader motivation includes applications in human–computer interaction, healthcare, and well-being analysis.
- Study objectives: The study introduces a newly collected dataset of physiological signals recorded while participants watched emotion-eliciting stimuli.It also evaluates how different targets affect emotion-classification performance.
- Research questions: The study asks whether ECG and GSR acquired from Shimmer devices can support emotion-recognition models.This question is evaluated alongside target representation and model-selection questions.
- Research questions: The research compares valence, arousal, and discrete-emotion classes as alternative target representations for emotion classification.Discrete emotions are considered at different class granularities.
- Research questions: The study evaluates whether lightweight traditional machine-learning models can achieve good accuracy for emotion recognition.The motivating model scope includes algorithms such as Naive Bayes, SVM, ANN, Random Forest, and ensemble methods.
II. EXPERIMENTAL FRAMEWORK
The experimental framework is designed to collect physiological responses associated with different emotional stimuli in a controlled environment. It covers participants, stimulus arrangements, measured signals, devices, and the data-collection procedure.
- II. EXPERIMENTAL FRAMEWORK: The framework collects physiological signals associated with different emotional stimuli under controlled environmental conditions.Its design includes participant details, emotional-stimulus arrangements, physiological measurements, devices, and collection procedures.
A. Participants
The study recruited 22 participants and exposed them to emotion-eliciting video clips followed by neutral clips and self-assessment ratings. The protocol used participant ratings to verify the intended emotional responses.
- Participants: Twenty-two participants were recruited, including 15 males and 7 females from three age groups and two nationality groups.The sample included 11 international and 11 Finnish participants.
- Emotional stimuli: The stimulus set used short video clips representing positive, negative, and neutral emotional content.Nine clips represented positively valenced emotions, while negative emotions included anger, disgust, fear, sadness, and surprise.
- Experimental procedure: Neutral clips lasting about 60 seconds followed emotional stimuli to reduce emotional carryover effects.Participants then completed a self-assessment questionnaire after each emotional stimulus.
- Experimental procedure: The questionnaire collected discrete-emotion, valence, and arousal ratings using the Self-Assessment Manikin scale.These ratings were used to verify whether the stimuli elicited the intended emotions.
C. Acquisition of Physiological Signals
ECG and GSR were recorded with wearable Shimmer devices while participants watched movie clips. The synchronized signals captured cardiac and sympathetic nervous-system responses at 256 Hz.
- C. Acquisition of Physiological Signals: ECG was recorded with a Shimmer 3 EXG module using a three-lead configuration and four chest electrodes.The ECG recordings were used to capture heart rate and heart-rate variability during emotional arousal.
- C. Acquisition of Physiological Signals: GSR was recorded from fingers of the non-dominant hand using a Shimmer 3 GSR module.GSR was used to capture sympathetic nervous-system activity during emotional arousal.
- C. Acquisition of Physiological Signals: Both ECG and GSR signals were sampled at 256 Hz and synchronized to stimulus presentation time.Synchronization aligned physiological responses with the viewed movie clips.
D. Experimental Procedures
The experimental session used a controlled environment, beginning with participant preparation and sensor calibration before a video-based emotional-stimulus protocol.
- Participants received study and equipment briefings, signed written consent, and completed the session in a quiet, temperature- and light-controlled room.Devices were calibrated and attached before the video protocol began.
III. DATA ANALYSIS FRAMEWORK
The data-analysis framework processed physiological recordings and extracted features before training machine-learning models for emotion classification. Performance was assessed with repeated stratified cross-validation using Python-based tools.
- Data from 22 participants were processed through preprocessing, labeling, signal preprocessing, feature extraction, model training, and evaluation.
- Repeated stratified cross-validation was used to evaluate machine-learning model performance.
- Python libraries supported ECG and GSR preprocessing, feature extraction, machine-learning analysis, visualization, and additional synchronization and signal processing.The workflow used SciPy, NeuroKit2, NumPy, Pandas, Scikit-learn, and Matplotlib.
A. Data Pre-processing and Exploratory Data Analysis
The analysis examined ECG and GSR integrity alongside subjective responses from 22 participants. Most stimuli occupied their intended valence–arousal regions, although carry-over effects and some deviations were observed.
- The raw ECG and GSR recordings were preprocessed and explored alongside subjective self-assessment data collected after emotional stimuli.The selected ECG lead configuration was LL–RA (Left Leg–Right Arm).
- The analysis visualized heart rate and skin conductance level across video segments to assess signal integrity and response reliability.
- Carry-over effects from intense emotional stimuli were observed in skin conductance for a few participants.
- Most video stimuli aligned with the intended valence–arousal space, except sadness and surprise, indicating that the stimuli generally elicited targeted emotions.The assessment used Self-Assessment Manikin valence and arousal ratings and Russell’s circumplex model.
B. Feature Extraction
Feature extraction used ECG and GSR segments corresponding to emotional stimuli, supplemented by self-reported valence and arousal values. ECG features captured autonomic activity through time- and frequency-domain measures.
- ECG and GSR recordings were segmented into non-overlapping windows aligned with emotional stimuli for feature extraction.Self-reported valence and arousal values were also used for classification.
- ECG processing extracted heart rate, heart rate variability, RR intervals, LF and HF band power, and the LF/HF ratio.These features were intended to quantify physiological changes during emotional stimulation and complement time-domain measures with frequency-domain measures.
- 12 features were obtained from the ECG modality.
C. Emotion Recognition Model and Evaluation
The study trains supervised models on physiological and subjective features to classify emotional states, using repeated stratified cross-validation and accuracy evaluation.
- The framework uses supervised SVM, Random Forest, XGBoost, and neural-network models for emotion classification.
- Valence and arousal features are included because they can capture subtle variations in mixed emotional states.
- Repeated stratified cross-validation preserves minority emotion classes across folds and estimates performance on unseen data.
- Model performance is assessed with accuracy, and feature importance is calculated for the top-performing model.
IV. RESULTS
The results compare emotion-classification setups across ECG, GSR, and combined sensor features, with performance generally favoring fewer emotion classes and combined physiological inputs.
- For ECG alone, Random Forest reached 84.7% with subjective data, while a neural network reached 47.6% using only physiological data for four emotion classes.
- For GSR alone, Random Forest reached 86.15% with subjective data, while XGBoost reached 51.1% without subjective data.
- 53.0% was the maximum accuracy for four-class discrete-emotion classification using ECG+GSR features and Random Forest.
- ECG+GSR features outperformed individual ECG and GSR sensors across algorithms and classification cases using physiological data.
- Four-class discrete-emotion classification achieved higher accuracies than nine-class discrete-emotion classification across algorithms.
V. CONCLUSION
The study evaluates ECG and GSR features with traditional machine-learning models for discrete emotions, valence, and arousal. The reported results support physiological-signal-based emotion recognition, particularly for arousal and four-class emotion classification.
- For individual sensors, XGBoost with GSR reached 51.1% and a neural network with ECG reached 47.6% for four-class discrete emotions.
- The authors report that traditional models exceeded random accuracy, indicating meaningful patterns in the physiological data.
- Future work could add more participants, expand features with EEG and PPG, and replicate the models in real-world environments.