Source-linked AI summary
A Deep Learning Framework for Assessing Physical Rehabilitation Exercises
Y. Liao, A. Vakanski, M. Xian
TL;DR
Rehabilitation assessment needs more versatile and robust ways to evaluate home-exercise performance, where continuous professional oversight is unavailable. The paper proposes a deep-learning framework combining learned movement representations, probabilistic performance metrics, score mapping, and spatio-temporal neural networks; across ten exercises, generated quality scores closely follow ground-truth scores, while validation remains limited by healthy-subject data, expensive optical capture, and incomplete clinician-labeled quality assessments.
Problem
Existing rehabilitation-assessment approaches lack versatility, robustness, and practical relevance despite the prevalence of home-based exercise and limited continuous professional oversight.
Method
The framework uses deep autoencoders for dimensionality reduction, GMM log-likelihood for performance metrics, scoring functions for quality values, and supervised deep spatio-temporal neural networks.
Results
Across a dataset of 10 rehabilitation exercises, the generated quality scores closely follow the ground-truth movement-quality scores.
Takeaways & Limitations
Deep neural networks show potential for rehabilitation assessment by modeling human movements hierarchically across spatial and temporal abstraction levels.
Takeaways & Limitations
Validation primarily uses healthy subjects, expensive optical motion capture, and movement data without clinician-assessed ground-truth quality; patient validation is partial.
Abstract
from arXiv · showhide
Computer-aided assessment of physical rehabilitation entails evaluation of patient performance in completing prescribed rehabilitation exercises, based on processing movement data captured with a sensory system. Despite the essential role of rehabilitation assessment toward improved patient outcomes and reduced healthcare costs, existing approaches lack versatility, robustness, and practical relevance. In this paper, we propose a deep learning-based framework for automated assessment of the quality of physical rehabilitation exercises. The main components of the framework are metrics for quantifying movement performance, scoring functions for mapping the performance metrics into numerical scores of movement quality, and deep neural network models for generating quality scores of input movements via supervised learning. The proposed performance metric is defined based on the log-likelihood of a Gaussian mixture model, and encodes low-dimensional data representation obtained with a deep autoencoder network. The proposed deep spatio-temporal neural network arranges data into temporal pyramids, and exploits the spatial characteristics of human movements by using sub-networks to process joint displacements of individual body parts. The presented framework is validated using a dataset of ten rehabilitation exercises. The significance of this work is that it is the first that implements deep neural networks for assessment of rehabilitation performance.
I. INTRODUCTION
Home-based rehabilitation is widespread, but limited continuous professional oversight and insufficiently versatile automated assessment motivate a deep-learning framework for scoring exercise quality.
- I. INTRODUCTION: More than 90% of rehabilitation sessions occur in home-based settings, where patients perform prescribed exercises outside direct clinical supervision.Home programs require patients to record progress and periodically visit clinics for functional assessment.
- I. INTRODUCTION: Low adherence to home exercise regimens is associated with prolonged treatment times and increased healthcare costs.The reported major contributor is the absence of continuous professional feedback and oversight.
- I. INTRODUCTION: Existing rehabilitation tools still lack versatility and robustness for automatic monitoring and assessment of patient performance.Examples include robotic assistive systems, virtual-reality and gaming interfaces, and Kinect-based assistants.
- I. INTRODUCTION: The proposed framework combines performance metrics, scoring functions, and deep-learning models to generate numerical movement-quality scores.It uses probabilistic skeletal-joint modeling, dimensionality reduction, and supervised learning of quality scores.
- I. INTRODUCTION: The deep neural architecture models spatial and temporal variability by hierarchically processing body-part joint displacements and movement sequences at multiple time scales.Its design draws on temporal pyramids and hierarchical recurrent approaches to motion classification.
- I. INTRODUCTION: The paper presents its framework as the first to employ deep neural networks for rehabilitation-exercise assessment.The stated contributions include the framework, a deep spatio-temporal quality-scoring model, and a probabilistic autoencoder-based performance metric.
B. Movement Assessment
Movement assessment methods derive exercise quality from comparisons, distances, or probabilistic models, while the proposed pipeline transforms sensed skeletal coordinates into scores for neural-network training and later automatic assessment.
- B. Movement Assessment: Quantifying correctness in prescribed exercises supports tools for home-based rehabilitation by comparing patient performance with desired performance from healthy participants.The comparison provides the basis for assessing movement quality.
- B. Movement Assessment: Several existing methods classify exercise repetitions as correct or incorrect, producing discrete 0-or-1 outputs.Reported classifiers include AdaBoost, k-nearest neighbors, Bayesian classifiers, and multilayer-perceptron ensembles.
- B. Movement Assessment: Distance-based approaches use measures such as Mahalanobis distance and dynamic time warping to derive movement-quality scores.These methods compare patient repetitions with repetitions performed by healthy individuals or calculate temporal movement distances.
- B. Movement Assessment: Probabilistic approaches assess quality using the likelihood that movement sequences were generated by trained hidden-Markov or Gaussian-mixture models.Their stated advantage is handling variability arising from the stochastic character of human movements.
- B. Movement Assessment: The proposed pipeline processes skeletal joint coordinates through dimensionality reduction, performance quantification, and score mapping before training a neural network.The trained network subsequently generates movement-quality scores from new sensor-acquired movement data.
A. Notation
The framework represents rehabilitation movements as temporal sequences of joint measurements and reduces their dimensionality with an LSTM autoencoder that compresses and reconstructs the data.
- Data representation: Each repetition is represented as a temporal sequence of multidimensional joint-displacement measurements.The data include reference movements from healthy subjects and patient movements, with each measurement containing values for all tracked joints.
- Data representation: Motion-capture measurements typically span 45–120 dimensions because systems track 15–40 joints with three-dimensional positions or orientations.Dimensionality reduction is used to suppress unimportant, redundant, or highly correlated dimensions.
- Dimensionality reduction: The framework uses autoencoder neural networks for nonlinear dimensionality reduction instead of relying only on linear methods such as PCA.Deep autoencoders can increase representational capacity by stacking multiple hidden layers.
- Dimensionality reduction: An autoencoder encodes input data into a lower-dimensional code and decodes that code to reconstruct the output, minimizing mean squared deviation during training.The encoder and decoder are learned as paired mapping functions.
- Autoencoder architecture: The adopted LSTM autoencoder maps 117-dimensional joint-coordinate sequences to a temporal sequence of 4-dimensional code vectors.Its encoder and decoder each contain three intermediate LSTM layers, with encoder widths 30, 10, and 4 and decoder widths 10, 30, and 117.
C. Performance Metric
The performance metric uses probabilistic modeling to evaluate rehabilitation movements through GMM log-likelihood, after representing movement data in a suitable feature space.
- Metric classes: Movement-performance metrics are divided into model-less distance-based metrics and model-based probabilistic metrics.Model-less examples include Euclidean distance, Mahalanobis distance, and DTW, whereas model-based metrics use log-likelihood.
- Proposed metric: The proposed metric is based on Gaussian mixture model log-likelihood.The choice is motivated by statistical methods’ ability to represent inherent variability in human movements and handle spatio-temporal variation.
- Proposed metric: A GMM represents multidimensional movement data as a mixture of Gaussian probability density functions.Its parameters include component mixing coefficients, means, and covariances, commonly estimated with expectation maximization.
- Performance evaluation: For patient repetitions, the framework uses negative log-likelihood under the fitted GMM as the performance metric.The metric evaluates how well movement data instances are represented by the probabilistic model.
D. Scoring Function
The scoring function converts performance-metric values into movement-quality scores between 0 and 1 while preserving the distributions of reference and patient movements.
- Score mapping: A scoring function maps performance-metric values to movement-quality scores in the range 0 to 1.This provides a normalized interpretation of the metric values.
- Score interpretation: The resulting scores support intuitive patient feedback and self-monitoring of progress toward functional recovery.The paper gives an 88% movement-quality score as an example of an easily understood presentation.
- Score mapping: The proposed scoring function is monotonically decreasing and preserves the distributions of performance-metric values.Separate sequences are defined for reference movements and patient movements.
- Score mapping: Reference values are scaled by μ + 3δ so values within (μ − 3δ, μ + 3δ) produce scores close to 1.The patient-movement mapping is likewise designed to preserve the patient metric distribution.
E. Deep Learning Architecture for Rehabilitation Assessment
The rehabilitation-assessment network models skeletal movements across body-part hierarchies and multiple temporal scales, combining convolutional and recurrent processing.
- Spatial processing: The model processes joint displacements from individual body parts through dedicated sub-networks that progressively merge feature vectors.This hierarchical organization is designed to exploit spatial characteristics of human movements.
- Spatio-temporal modeling: Strided one-dimensional convolutional layers learn spatial dependencies, followed by LSTM layers that model temporal correlations in the learned representations.The architecture therefore combines hierarchical spatial processing with recurrent temporal modeling.
- Temporal processing: The network arranges movement sequences into temporal pyramids to process multiple scaled versions of each repetition.The proposed pyramid uses the full sequence plus versions with one-half, one-quarter, and one-eighth temporal length.
- Convolutional blocks: The convolutional blocks use three branches of one-dimensional filters with different lengths, whose outputs are concatenated for subsequent processing.Dropout layers follow the convolutional layers, using a rate of 0.25.
- Model scope: The model is intentionally relatively shallow because the dataset is relatively small and larger, deeper networks could overfit.This design choice is presented as a dataset-size consideration rather than a general architectural rule.
IV. EXPERIMENTAL RESULTS
The framework is validated on the UI-PRMD dataset, which contains skeletal movement data from healthy subjects performing ten rehabilitation exercises.
- Dataset: UI-PRMD contains skeletal data collected from 10 healthy subjects performing 10 repetitions of 10 rehabilitation exercises.The data were acquired with a Vicon optical tracking system as 117-dimensional sequences of angular joint displacements.
B. Performance Quantification
Performance quantification compares GMM log-likelihood with distance-based metrics using scaled values and separation degree to distinguish correct from incorrect repetitions.
- Metric comparison: The evaluation compares GMM log-likelihood with Euclidean, Mahalanobis, and DTW distance metrics.Metrics are evaluated on the UI-PRMD dataset for rehabilitation-exercise assessment.
- Data scaling: Performance metric values are linearly scaled to the range [1, 20] before comparison.The scaling uses separate values for correct and incorrect movement repetitions.
- Separation degree: 0.384 for E1 and 0.497 for E2 show greater Euclidean-distance separation for exercise E2 in the between-subject case.The figure compares scaled Euclidean-distance values for correct and incorrect repetitions of E1 and E2.
- Results: The GMM log-likelihood metric with autoencoder-reduced data produced the largest separation between correct and incorrect movements in both subject-comparison cases.Within-subject repetitions showed improved separation, while GMM log-likelihood was not reported for the raw 117-dimensional data.
C. Neural Networks Performance
Neural networks predict movement quality scores from rehabilitation movement data, using a spatio-temporal architecture evaluated through prediction, ablation, comparison, and cross-validation experiments.
- Training setup: Training uses GMM log-likelihood quality scores from autoencoder-reduced data and considers only the between-subject case because within-subject repetitions are too few for neural-network training.The scoring function maps performance metrics into movement-quality scores between 0 and 1.
- Quality scoring: For E1, Fig. 5 maps GMM log-likelihood values to quality scores, with correct movements near 1 and most incorrect movements between 0.7 and 0.9.The figure presents the metric values and corresponding movement-quality scores for deep squat exercise E1.
- Training setup: Each exercise has a separately trained supervised-regression network that predicts movement quality from an input repetition.Each model is run five times, and performance is reported as average absolute deviation from ground-truth scores.
- Ablation study: The ablation results support contributions from multi-branch layers, temporal pyramids, hierarchical structure, and combined convolutional-recurrent units.Lower average absolute deviation indicates lower prediction error.
- Model comparison: The proposed model outperformed adapted deep-learning movement-classification models in quality assessment across all 10 exercises.The comparison included Hierarchical LSTM, Deep CNN, and Deep LSTM architectures.
- Validation: Predicted quality scores closely followed ground-truth values on both random validation data and leave-one-subject-out cross-validation.The reported cross-validation performance was comparable to the random-test results.
V. DISCUSSION
The discussion reports that GMM log-likelihood outperformed model-less metrics and attributes the framework’s performance to hierarchical spatio-temporal modeling. It also identifies validation limitations involving healthy subjects, expensive motion capture, and limited clinician-labeled patient data.
- Metric comparison: GMM log-likelihood outperformed Euclidean, Mahalanobis, and DTW metrics on the UI-PRMD dataset.The authors link this result to the importance of modeling human movements rather than relying on model-less metrics.
- Representation learning: Autoencoders provided richer lower-dimensional representations than PCA and maximum-variance dimensionality reduction.The stated advantage comes from nonlinear neuron activations and their greater representational capacity.
- Network design: The proposed architecture combines hierarchical body-part feature merging, temporal pyramids, convolutional blocks, and later LSTM units.The convolutional design is motivated partly by the dataset’s small size, with fewer than 200 repetitions per exercise.
- Limitations: Validation primarily used healthy-subject exercises recorded with an expensive optical motion-capture system and often lacked clinician-assigned ground-truth quality scores.The KIMORE deep-squat evaluation supplied only partial validation on patient data collected with a low-cost sensor.
- Future work: Future work targets patient exercises labeled by clinicians, muscle-activity measurements, and home-based assessment using a Kinect sensor.These plans directly address the study’s stated validation and sensing limitations.
VI. CONCLUSION
The conclusion presents a deep learning framework that combines dimensionality reduction, movement metrics, scoring functions, and neural models for rehabilitation-exercise assessment. Across ten exercises, generated quality scores closely followed ground-truth scores, while hierarchical spatial and temporal modeling is identified as the framework’s central advantage.
- Framework: The framework combines dimensionality-reduction algorithms, performance metrics, scoring functions, and deep learning models.It is evaluated on a dataset containing 10 rehabilitation exercises.
- Evaluation: 10 rehabilitation exercises were used to evaluate the framework, whose generated quality scores closely followed the movements’ ground-truth scores.This is the principal outcome reported in the conclusion.
- Implication: Deep learning models can outperform distance-function and single-level probabilistic approaches for rehabilitation-exercise assessment.The authors attribute the advantage to hierarchical modeling of complex spatial and temporal movement correlations.