Source-linked AI summary
Why Personalization Matters: Cross-Subject Challenges in EMG-IMU-based HRI Activity Recognition
Ruan Rithelle Chagas de Faria Carminati, Giovanni Braglia, Luigi Biagiotti, Ronnier Frates Rohrich, Andre Schneider de Oliveira, Mikael Nedel Hartmann, André Eugenio Lazzaretti
TL;DR
The paper addresses whether wearable EMG/IMU signals can recognize many complex HRI activities and generalize to unseen users. It introduces and evaluates the MAGIC-HRI dataset with cross-subject testing and personalization, finding a substantial generalization gap but marked improvement after adding new-user samples. The results support personalization as a practical strategy for wearable HRI deployment, while online robot-in-the-loop validation remains limited.
Problem
The study examines reliable classification of many complex HRI interactions, the effect of inter-subject variability on LOSO generalization, and strategies for adapting to new users.
Method
The paper introduces MAGIC-HRI and evaluates a wearable EMG/IMU recognition pipeline across subjects using benchmark classification and personalized adaptation experiments.
Results
A discrepancy of more than 60% occurred between 75/25 split and LOSO mean F1-scores, while personalization enabled recognition of more than 65% of signals after at least 7 samples.
Takeaways & Limitations
Personalization can bridge the gap between laboratory performance and robust real-world wearable HRI operation.
Takeaways & Limitations
The approach lacks an in-depth online robot-in-the-loop evaluation, and LOSO results still limit practical use.
Abstract
from arXiv · showhide
This paper investigates wearable-based recognition of human activities and gestures to support Human-Robot Interaction (HRI) in object-handover and assembly-like scenarios. Electromyography (EMG) and Inertial Measurement Unit (IMU) signals were collected using a Myo armband, culminating in a novel dataset introduced as MAGIC-HRI (Multimodal Activity, Gesture and Intention Collection) with a large taxonomy of 53 movement classes, including Brazilian Sign Language (LIBRAS) numbers (0-9), hand gestures, object/tool handover actions (pick up/give/hold), tool-manipulation tasks, and generic assembly/idle motions, collected from 11 participants with 10 samples per class (530 samples per participant). Signals are segmented by detecting muscle activation via an EMG energy envelope, then processed using sliding windows; time- and frequency-domain features are extracted. Multiple classical classifiers are tuned via cross-validated grid search, with Random Forest as the strongest baseline. A Leave-One-Subject-Out (LOSO) protocol reveals a large generalization gap, indicating substantial subject dependence. A personalized adaptation experiment suggests that injecting a small number of samples from a new user can markedly improve recognition. Overall, the study contributes a broad, HRI-driven multimodal dataset, a rigorous evaluation emphasizing generalization, and practical evidence that personalization is likely required for robust deployment in practical HRI.
I. INTRODUCTION
The paper targets reliable wearable recognition of complex human intentions in HRI, especially object handover, while addressing inter-subject generalization and adaptation to new users. It introduces a 53-class EMG/IMU dataset and evaluates generalization and personalization for practical applicability.
- Motivation: Object handover requires precise human–robot synchronization and intention recognition to coordinate grasping and release safely.The interaction includes pre-handover communication and a physical transfer phase.
- Research questions: Wearable EMG/IMU recognition must be evaluated for complex handover interactions, unseen-user generalization, and performance recovery for new operators.These are framed as the paper’s three central research questions.
- Contributions: 53 movement classes collected from 11 subjects define a broader wearable HRI recognition setting than the small class counts common in prior studies.The comparison emphasizes classification scope and wearable sensing rather than camera- or pose-based inputs.
- Contributions: The study benchmarks the 53-class dataset for predicting complex human intents and analyzes model generalization across subjects.It also includes a methodological approach for adapting models to new users.
II. WEARABLE-BASED ACTIVITY AND GESTURE RECOGNITION
The paper applies a data-collection, signal-processing, and pattern-recognition methodology to wearable activity and gesture recognition.
- Recognition pipeline: The methodology combines data collection, signal processing, and pattern recognition in the evaluated wearable recognition pipeline.Figure 2 presents an overview of this pipeline, and the dataset is publicly available as MAGIC-HRI.
A. Sensing Setup
The sensing setup combines a Myo armband for EMG/IMU acquisition with a collaborative robotic arm that performs object handover actions during data collection.
- Wearable interface: The Myo Armband records eight-channel EMG, 3D acceleration, and 3D gyroscope data for wearable gesture recognition.EMG is sampled at up to 200 Hz and IMU at up to 50 Hz.
- Robotic setup: A six-axis collaborative robot delivered objects to participants and retrieved them during data acquisition.The robot was an OMRON TM5-700 system.
B. MAGIC-HRI Dataset Description
MAGIC-HRI contains EMG/IMU recordings from 11 right-handed participants performing 53 predefined movement classes spanning sign-language numbers, gestures, object interaction, tool manipulation, and generic motions.
- Dataset scope: 11 participants contributed EMG/IMU recordings collected at 50 Hz while performing 53 predefined movement classes.Participants were 9 men and 2 women, with mean age 33 years and ages ranging from 22 to 54.
- Numbers: LIBRAS numbers 0 through 9 provide armband-only commands for selecting among multiple tools or other specific robot actions.The number category supports more complex commands than simpler gestures.
- Gestures: Gesture classes include spread fingers, open palm, waving, thumb-to-middle, closed hand, and thumbs-up configurations.These gestures represent general or frequent commands.
- Object interaction: Object-interaction classes cover holding, giving, and picking up objects or tools from the robot.These actions directly represent handover-related behavior.
- Manipulation and generic motions: Tool-manipulation classes include screwing, unscrewing, and active use of electric and heat-related tools, while generic movements include idle, empty-hand, and assembly motions.Generic movements were included with the possibility of reducing classification noise.
C. Signal Processing and Feature Extraction
Signal processing detects muscle activation from an EMG energy envelope, segments the signals, and extracts time- and frequency-domain features from EMG and IMU data.
- Signal segmentation: Raw EMG channels are squared, smoothed with a 50 ms moving average, averaged into one energy envelope, and segmented above 10% of peak energy.The envelope represents total muscle activity across all eight EMG channels.
- Time-domain features: Time-domain extraction produces 10 EMG features and 8 IMU features.The listed EMG features include RMS, MAV, zero crossings, waveform length, variance, and sample entropy; IMU features include mean, standard deviation, skewness, kurtosis, and signal energy.
- Frequency-domain features: Welch’s Method estimates the power spectral density before frequency-domain feature extraction.This transforms the signals from the time domain to the frequency domain.
- Frequency-domain features: Frequency-domain features include mean, median, and peak frequency, spectral entropy, and total power for both EMG and IMU signals.The EMG and IMU frequency-domain feature sets use the same five named measures.
D. Classification Methods
The study tunes multiple machine-learning and deep-learning classifiers through parameter-grid search and evaluates them across MAGIC-HRI’s 53 classes, with Random Forest achieving the strongest reported performance.
- Model selection: Parameter grids span functional settings such as estimator counts, kernel types, and hidden-layer architectures.These grids support cross-validated model selection across the evaluated classifier families.
- Model selection: 5-fold cross-validation is used to reduce dependence on any single data split during classifier tuning.Models are evaluated using accuracy and F1-score across all 53 classes.
E. Training and Evaluation Protocol
The evaluation combines a conventional 75/25 train-test split with Leave-One-Subject-Out testing to measure performance on unseen participants.
- LOSO evaluation: The LOSO protocol trains on 10 subjects and tests on the remaining subject, repeating this process across all 11 participants.Average accuracy and F1-score are calculated over the 11 held-out-subject iterations.
- Evaluation metrics: Models are evaluated using accuracy and F1-score under both the 75/25 split and LOSO protocols.The paired protocols contrast within-dataset testing with cross-subject generalization.
III. EXPERIMENTAL EVALUATION
Experiments identify Random Forest as the strongest baseline, reveal a major gap between split-based and LOSO performance, and test feature reduction and personalized sample injection as possible remedies.
- Classifier comparison: Random Forest achieves the best performance among the evaluated classifiers.The reported configuration uses 150 estimators, maximum depth 30, minimum sample split 2, and square-root feature selection.
- Generalization evaluation: More than 60% separates mean F1-scores under the 75/25 split and LOSO evaluation, exposing poor cross-subject generalization.Increasing window overlap improves the 75/25 split but does not produce comparable LOSO gains.
- Generalization evaluation: Increasing overlap alone is insufficient to address difficulty generalizing to previously unseen subjects.This conclusion follows the comparison across the tested overlap configurations.
- Feature reduction: SHAP-based feature reduction produces no significant improvement in model performance after reducing the initial 198 features to 74.The analysis removes highly correlated features before selecting features by SHAP importance.
- Personalized adaptation: The personalization experiment incrementally adds samples from a left-out subject to the training set, progressing from 0 to 8 cumulative samples.Two samples per movement are reserved as an external test set, and 40 features are used during adaptation.
- Personalized adaptation: More than 65% of signals are recognized once at least 7 personalization samples are added, although performance remains below the 75/25 baseline.Figure 4 reports significant improvement as new subject samples are injected.
IV. DISCUSSION
The discussion emphasizes that prior high accuracies often come from smaller, clearly distinct class sets, while this study examines the harder 53-class setting and LOSO generalization. It also identifies limited online robot-in-the-loop validation and subject variability as boundaries for practical deployment.
- Comparison with prior work: Prior work reported accuracies above 90% across six handover-intention classes, but its smaller and clearly distinct class set limits direct comparison with this study.The related study used the same six subjects for online and offline evaluation, while excluded-subject performance was not discussed in detail.
- Cross-subject generalization: The 53-class setting and LOSO evaluation make cross-subject generalization a central challenge for practical wearable HRI recognition.The discussion contrasts the present broader class setting with prior work and highlights the difficulty of comparing performance when subjects are excluded from training.
- Practical evaluation: A more in-depth robot-in-the-loop evaluation during actual online HRI remains necessary, although initial inference tests found no hardware bottlenecks.The current evidence supports potential real-time integration but does not yet assess classification during online robot interaction.
- Practical evaluation: Personalized samples can increase model accuracy, while collecting data from additional subjects may help mitigate overfitting.These strategies address the practical limitations observed in LOSO results, but their deployment value still requires further evaluation.
V. CONCLUSIONS AND FUTURE WORK
The paper introduces MAGIC-HRI as a broad wearable EMG/IMU dataset and evaluates recognition through a complete pipeline and cross-subject benchmark. Results show that subject variability limits deployment, while personalization markedly improves recognition and supports a brief user-calibration strategy.
- Contributions: MAGIC-HRI provides a 53-class EMG/IMU dataset collected from 11 subjects for object-handover and assembly-oriented industrial scenarios.The dataset broadens wearable HRI recognition beyond the small, highly distinct class sets common in prior studies.
- Contributions: The study combines muscle-activation segmentation, time- and frequency-domain feature extraction, classical and deep learning classifiers, and a benchmark across multiple models.This pipeline is evaluated in a recognition setting involving real interactions with a collaborative robot.
- Findings: Cross-subject variability is the main practical bottleneck, and split-based performance can substantially overestimate real deployment performance.The paper reports that neither increased overlap nor SHAP-based feature reduction may be sufficient to overcome this limitation.
- Findings: More than 65% of signals were recognized once at least 7 subject-specific samples were added to training.The result supports personalization as an effective deployment strategy for wearable HRI recognition.
- Future work: Future work should add subjects and movements, investigate models for cross-subject transfer, and validate the approach in online robot-in-the-loop experiments.These directions target dataset coverage, transfer performance, and real-time HRI validation.