PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning

Ulysse Côté-Allard, Cheikh Latyr Fall, Alexandre Drouin, Alexandre Campeau-Lecours, Clément Gosselin, Kyrre Glette, François Laviolette, Benoit Gosselin

arXiv:1801.07756v5cs.LGstat.ML

TL;DR

sEMG gesture recognition lacks the large single-user datasets commonly needed by deep learning. The paper uses transfer learning from aggregated multi-user recordings with ConvNets across three signal modalities, and reports improved performance across datasets, including 98.31% and 68.98% average accuracies. The study’s main scope boundary is evaluation only with able-bodied participants and related recording conditions.

  • Problem

    Single users cannot reasonably produce the tens of thousands of examples needed for deep learning, motivating the use of aggregated recordings to learn features for new subjects.

    Source: §I · p. 1

  • Method

    The paper pre-trains ConvNets on aggregated inter-user sEMG data and transfers them across raw EMG, spectrogram, and CWT input modalities.

    Source: §I · p. 1, §VI · p. 5

  • Results

    Transfer-learning ConvNets systematically and significantly outperform non-augmented versions, reaching 98.31% average accuracy over 17 participants and 68.98% over 10 participants.

    Source: §IX · pp. 9–10, §X · p. 11

  • Takeaways & Limitations

    The learned features generalize across participants and never-seen-before gestures, supporting reuse of pre-trained weights for Myo Armband tasks with different gestures.

    Source: §IX · p. 11, §X · p. 11

    The study does not test amputees, electrode shifts, varied limb positions, or users wearing fewer electrodes than the pre-training group.

    Source: §IX · p. 11

Abstract

from arXiv · show

In recent years, deep learning algorithms have become increasingly more prominent for their unparalleled ability to automatically learn discriminant features from large amounts of data. However, within the field of electromyography-based gesture recognition, deep learning algorithms are seldom employed as they require an unreasonable amount of effort from a single person, to generate tens of thousands of examples. This work's hypothesis is that general, informative features can be learned from the large amounts of data generated by aggregating the signals of multiple users, thus reducing the recording burden while enhancing gesture recognition. Consequently, this paper proposes applying transfer learning on aggregated data from multiple users, while leveraging the capacity of deep learning algorithms to learn discriminant features from large datasets. Two datasets comprised of 19 and 17 able-bodied participants respectively (the first one is employed for pre-training) were recorded for this work, using the Myo Armband. A third Myo Armband dataset was taken from the NinaPro database and is comprised of 10 able-bodied participants. Three different deep learning networks employing three different modalities as input (raw EMG, Spectrograms and Continuous Wavelet Transform (CWT)) are tested on the second and third dataset. The proposed transfer learning scheme is shown to systematically and significantly enhance the performance for all three networks on the two datasets, achieving an offline accuracy of 98.31% for 7 gestures over 17 participants for the CWT-based ConvNet and 68.98% for 18 gestures over 10 participants for the raw EMG-based ConvNet. Finally, a use-case study employing eight able-bodied participants suggests that real-time feedback allows users to adapt their muscle activation strategy which reduces the degradation in accuracy normally experienced over time.

I. INTRODUCTION

The paper addresses limited single-user training data in sEMG gesture recognition by transferring features learned from aggregated inter-user recordings. It introduces ConvNet architectures using raw signals, spectrograms, and CWT representations, alongside a publicly available Myo dataset.

Motivation

  • Single users cannot realistically generate the tens of thousands of examples often needed to train deep learning gesture recognizers.Aggregating recordings from multiple participants may provide data for learning mappings that generalize to new subjects.Source: §I · p. 1

Contribution

  • The proposed transfer-learning scheme uses inter-user data with ConvNets for sEMG-based hand-gesture recognition.The work also reduces computational load and improves performance relative to an earlier conference version.Source: §I · p. 1
  • Three ConvNet input modalities are evaluated: raw sEMG, short-time Fourier transform spectrograms, and continuous wavelet transforms.The authors identify CWT as a newly applied feature representation for sEMG-based hand-gesture classification.Source: §I · p. 1, §II · p. 2
  • The paper releases a public Myo dataset with 19 participants for pre-training and 17 participants for final evaluation.The authors describe it as the largest published dataset using the commercially available Myo Armband.Source: §I · p. 2, A. Myo Dataset · p. 2
  • The study extends the work with a fourteen-day use-case experiment examining real-time feedback and online classifier performance without recalibration.Experiments are generally reported as averages over 20 runs because the algorithms are stochastic.Source: §I · p. 2

Data and setting

  • The Myo Armband offers convenient, non-invasive acquisition but records lower-quality signals at 200 Hz, below the recommended sEMG sampling range.Dry electrodes are less robust to motion artifacts, and the limited sampling rate can impair gesture discrimination.Source: 1) sEMG Recording Hardware · pp. 2–3

2) Time-Window Length:

The study defines gesture acquisition and real-time processing around seven Myo Dataset gestures, variable armband placement, and a 260 ms input window. It also evaluates NinaPro DB5 using a broader gesture set.

2) Time-Window Length:

3) Labeled Data Acquisition Protocol:

NinaPro DB5

  • NinaPro DB5 contributes 10 participants performing 17 gestures plus a neutral gesture in its second exercise set.That exercise set contains the gestures considered in the rest of the work and adds movements such as wrist pronation and supination.Source: B. NinaPro DB5 · p. 3

A. Feature Sets

The paper compares conventional engineered feature sets and classifiers with deep-learning approaches. It uses dimensionality reduction to support computationally inexpensive and robust sEMG classification.

Feature Sets

  • Four literature-based feature sets provide comparison baselines for the proposed deep-learning methods.They are evaluated with SVM, ANN, RF, KNN, and linear discriminant classifiers.Source: A. Feature Sets · p. 4

Dimensionality Reduction

  • LDA is used for feature projection because it is computationally inexpensive, hyperparameter-free, and associated with robust sEMG gesture-classification accuracy.The paper compares accuracy with and without dimensionality reduction in an appendix.Source: A. Feature Sets · p. 4

Feature Sets

Network Design

B. Proposed Convolutional Network Architectures

The paper develops ConvNets for raw EMG, spectrogram, and CWT inputs, adapting slow-fusion designs to the time-structured representations of multichannel sEMG. Transfer learning aggregates multiple users’ data to learn features applicable to new subjects.

Architecture motivation

  • Slow-fusion ConvNets separate temporal information into parallel branches and gradually fuse those branches throughout the network.This design was adapted from video classification because spectrograms and CWTs encode time alongside channel and frequency or scale.Source: B. Proposed Convolutional Network… · pp. 4–5, Figure 3 · p. 4

Input modalities

  • Spectrogram inputs are represented as Time x Channel x Frequency matrices after windowed Hann-transform processing across eight Myo channels.The resulting spectrogram matrix is 4x8x14 after removing the first frequency band.Source: 1) ConvNet for Spectrograms · p. 5
  • A raw ConvNet processes raw EMG with a distinct architecture because raw signals constitute a different input modality from time-frequency representations.The raw architecture was selected from prior work using a Myo Armband hand-gesture dataset; an enhanced version uses 549 091 learnable parameters.Source: 3) ConvNet for raw EMG · p. 5, Figure 7 · p. 6

Transfer-learning preparation

  • The proposed approach aggregates multiple individuals’ data so ConvNets can learn more general features without requiring one person to record tens of thousands of examples.Transfer learning is used to apply features learned from aggregated users’ signals to new subjects.Source: §VI · p. 5
  • Automatic channel alignment is required for transfer learning because unconstrained armband orientation varies widely between subjects.Channels are circularly shifted using the most active gesture-related channel of the first subject as the reference.Source: §VI · p. 5

Input modalities

  • The spectrogram ConvNet splits inputs into two temporal parts, while the CWT ConvNet splits them into four and fuses branch feature maps by element-wise summation.The spectrogram and CWT architectures use 67 179 and 30 219 learnable parameters, respectively.Source: Figure 4 · p. 6, Figure 5 · p. 6

B. Adaptive Batch Normalization

The adaptive batch-normalization scheme shares a pretrained network across participants while allowing batch-normalization parameters to adapt to new users. Its target network adds a second network through scaled, layer-wise residual connections.

Adaptive Batch Normalization

  • AdaBatch freezes pretrained network weights during target training except for batch-normalization parameters, which are treated as domain-related information.The same network is used for source and target tasks, unlike the separate-network PNN architecture.Source: B. Adaptive Batch Normalization · p. 7
  • The shared Source Network learns the general mapping between muscle activity and gestures across participants.This shared source-task formulation avoids training one source network per participant.Source: C. Proposed Transfer Learning… · p. 7

Target architecture

  • The Second Network receives scaled Source Network outputs through layer-wise element-wise summation to learn the residual between source and target tasks.Learnable scalar coefficients control the transferred outputs before each sum connection.Source: C. Proposed Transfer Learning… · p. 7, Figure 8 · p. 7
  • The combined Source and Second Networks form the Target Network, with different dropout rates used during source pretraining and target training.MC Dropout is set to 35% for Source Network pretraining and 50% for Target Network training.Source: C. Proposed Transfer Learning… · p. 7

Evaluation

B. NinaPro Dataset

The NinaPro evaluation tests transfer learning across participants, including an out-of-sample-gesture setting and a 14-day real-time case study. Feedback was associated with counteracting accuracy degradation over time, while muscle fatigue did not appear to reduce accuracy.

NinaPro evaluation

  • The NinaPro DB5 evaluation averages accuracy over ten participants, using the other nine participants for each participant’s pretraining.ConvNets and their transfer-learning versions are evaluated for one to four training cycles.Source: 1) Comparison with Transfer Learning · p. 8, 2) Comparison with State of the art · p. 8
  • A separate NinaPro experiment evaluates eleven gestures absent from the pretraining dataset, directly testing transfer to never-seen-before gestures.The CWT ConvNet is compared with its transfer-learning-augmented version in this out-of-sample setting.Source: 3) Out-of-Sample Gestures · p. 8

Real-time case study

  • The real-time case study follows eight participants for fourteen days using the CWT+TL classifier, with four receiving feedback and four receiving none.Each daily session contains 30 randomly selected gestures, and the first second after each requested gesture is excluded from results.Source: §VIII · p. 8
  • 95.42% was the average accuracy across participants during the first recording session.The reported value provides the initial-session reference for the 14-day comparison.Source: §VIII · p. 8
  • Without feedback, accuracy degraded over fourteen days, whereas the feedback group was seemingly able to counteract that degradation.The comparison is reported as an observation from Fig. 10 rather than as a definitive causal result.Source: §VIII · p. 8

Fatigue analysis

  • A positive accuracy-regression slope suggests that muscle fatigue did not negatively affect the proposed ConvNet’s accuracy during the five-minute sessions.Many participants nevertheless reported experiencing muscular fatigue during recordings.Source: §VIII · pp. 8–9

IX. DISCUSSION

The proposed transfer-learning ConvNets consistently outperformed non-augmented versions and generalized across participants and gestures, while several deployment and scope limitations remain. Evaluation reached 98.31% on seven gestures across 17 participants and 68.98% on 18 gestures across 10 participants.

Performance comparison

  • TL-augmented ConvNets significantly outperformed non-augmented versions across training-cycle conditions.The advantage held across tested datasets and remained significant with a single training cycle.Source: §IX · pp. 9–10

Generalization

  • The TL method also outperformed non-augmented ConvNets in the out-of-sample experiment, with significance reported for all training repetitions.The authors associate this with learning features that generalize across participants and previously unseen gestures.Source: §IX · pp. 10–11

Limitations and future work

  • Real-time accuracy averaged 95.42%, below the Evaluation Dataset’s 98.31%, likely because gesture transitions were absent from training data.The paper notes that transition data may be poorly classified and proposes unsupervised transition detection as future work.Source: §IX · p. 11
  • The study did not test amputees, electrode shifts, or variability from different limb positions, limiting its evaluated scope.The proposed transfer-learning scheme also has difficulty adapting when a new user wears a different number of electrodes.Source: §IX · p. 11, §X · p. 11

Performance comparison

  • 98.31% average accuracy was achieved on the evaluation dataset across 17 participants.This result was reported for the TL-augmented ConvNet on seven gestures.Source: §X · p. 11
  • 68.98% average accuracy was achieved on NinaPro DB5 across 10 participants for 18 hand/wrist gestures.The proposed classifier used a single Myo Armband.Source: §X · p. 11

APPENDIX A DATA AUGMENTATION

The appendix evaluates signal-specific augmentation methods to address limited sEMG training data and overfitting, selecting sliding windows as the only statistically improved technique. It also considers computational feasibility for embedded deployment, with CWT and spectrogram networks requiring different computational budgets.

Augmentation methods

  • The baseline uses non-overlapping 260ms windows, while sliding windows add overlapping temporal translations without synthetic examples.This preserves recorded signal content while increasing the number of training examples.Source: Appendix A · pp. 13–14
  • Other tested augmentations emulate muscle fatigue, electrode displacement, Gaussian noise, or their aggregation.These transformations modify spectrogram frequency content, redistribute power across channels, add white Gaussian noise, or combine methods.Source: Appendix A · p. 13

Augmentation evaluation

  • Sliding-window augmentation was the only tested technique that significantly improved accuracy over the baseline.The appendix therefore used sliding windows as the paper’s sole data-augmentation technique.Source: Appendix A · p. 14

Embedded implementation

  • CWT and spectrogram ConvNets require approximately 2,274,000 and 14,728,000 FLOPs per classification, respectively.The paper estimates that suitable hardware and network compression could support embedded systems below 100mW.Source: REAL-TIME CLASSIFICATION · p. 14
  • Without optimization, CWT classification takes 21.42ms per example, compared with 2.94ms for spectrogram and 3.70ms for raw EMG ConvNets.Transfer learning adds 0.57ms, 0.90ms, and 0.14ms to the CWT, spectrogram, and raw EMG pipelines, respectively.Source: REAL-TIME CLASSIFICATION · p. 14

APPENDIX C FEATURE ENGINEERING

The section introduces windowed feature engineering for sEMG signals, covering time-domain descriptors and their definitions. It also includes a comparison table for proposed data augmentation techniques.

5) Skewness:

This section describes distributional and signal-dynamics features, including skewness, RMS, Hjorth parameters, and integrated EMG. These features characterize asymmetry, amplitude, frequency, complexity, and rectified signal magnitude.

  • Skewness measures the overall asymmetry of a distribution as its third central moment.Its calculation uses the signal distribution and standard deviation σ.Source: 5) Skewness · p. 15
  • RMS, also called the quadratic mean, is closely related to standard deviation and equals it when the signal mean is zero.Source: 6) Root Mean Square (RMS) [2] · p. 15

Hjorth parameters

9) Autoregression Coefficient (AR):

The section presents autoregressive, entropy, histogram, cepstral, and multiresolution wavelet features. It also documents the implementation and a boundary-effect limitation of the mDWT feature.

Autoregression Coefficient (AR)

Sample Entropy (SampEn)

Histogram (HIST)

  • HIST quantifies how often EMG amplitude deviations from baseline occur across n equal-width amplitude bins.The symmetric amplitude range is centered around the baseline, with n as a hyperparameter.Source: 11) EMG Histogram (HIST) [77] · p. 15

Cepstral coefficients

Modified Discrete Wavelet Transform (mDWT)

C. Time-Frequency Domain Features

Time-frequency features address the non-stationarity of sEMG by analyzing localized signal segments. STFT spectrograms use a fixed window, whereas wavelets analyze signals across multiple scales with frequency-dependent resolution.

Short Term Fourier Transform based Spectrogram (Spectrogram)

Continuous Wavelet Transform (CWT)

APPENDIX D HYPERPARAMETERS SELECTION FOR STATE OF THE ART

The appendix describes classifier hyperparameter searches and compares feature-set accuracies with and without dimensionality reduction. It also reports a transfer-learning adaptation for differing electrode counts, which significantly improves modified ConvNet accuracy.

Hyperparameter selection

  • SVM hyperparameters included RBF and linear kernels, with logarithmic searches over C and γ.C ranged from 10^-3 to 10^3, while γ ranged from 10^-5 to 10^2 for the RBF kernel.Source: Appendix D · p. 16
  • ANN hyperparameters covered hidden-layer sizes, sigmoid, tanh, and ReLU activations, learning rates, L2 penalties, Adam optimization, and early stopping.Ten percent of the training data was used for validation during early stopping.Source: Appendix D · p. 16
  • KNN searches varied neighbor counts and evaluated Manhattan, Euclidean, and third- or fourth-degree Minkowski distances.The candidate neighbor counts were 1, 2, 3, 4, 5, 10, 15, and 20.Source: Appendix D · pp. 16–17

Feature-set evaluation

Transfer-learning adaptation

  • 67.65% average accuracy was achieved by the transfer-learning-enhanced modified ConvNet versus 61.47% without augmentation across 17 participants.The difference was significant under a one-tail Wilcoxon signed-rank test (p-value=0.00494), and the adaptation accommodates different electrode counts.Source: TARGET DATASET · p. 17

Related papers

Loading 1801.07756v5…