PaperlayerView exact revision on arXiv

Source-linked AI summary

Impact of Patient Orientation in Single- and Multi-View Camera Environments for AI-based Rehabilitation Monitoring

Miriama Jánošová, Andreas Lang, Petra Budikova, Jan Sedmidubsky

arXiv:2609.35726v2cs.CV

TL;DR

Camera viewpoint can affect the pose data used to identify clinically relevant rehabilitation errors, yet its impact has been insufficiently studied. The paper introduces a multi-angle dataset and separability metric to compare pose-estimation approaches. An optimal 2D view improves separability over the frontal baseline, and an orthogonal two-view setup improves performance over the best single-camera setup.

  • Problem

    The impact of camera placement on detecting clinically relevant rehabilitation movement errors remains insufficiently explored, despite viewpoint-related changes in joint visibility and self-occlusion.

    Source: §1.2 · p. 3, §2 · p. 5

  • Method

    The paper introduces REHAB26-ViewAngles and a separability metric for evaluating how well personalized exercise assessment distinguishes correct from incorrect repetitions.

    Source: §1.2 · p. 3, §5.1 · p. 14

  • Results

    16.9% improvement in separability over the commonly used 0° frontal view; orthogonal views improve performance by about 13.1% over the best single-camera setup.

    Source: §6 · p. 20

  • Takeaways & Limitations

    The findings offer actionable guidance for camera placement in real-world vision-based rehabilitation monitoring.

    Source: §6 · p. 20

    The separability framework does not yet explicitly account for error duration; future work proposes dynamic temporal weighting.

    Source: §6 · p. 20

Abstract

from arXiv · show

Automated quality assessment of rehabilitation exercises relies heavily on accurate human pose estimation from video data. Although numerous RGB-based pose estimation methods have been proposed, the impact of camera placement on detecting clinically relevant movement errors remains insufficiently explored. To address this gap, we introduce REHAB26-ViewAngles, a dataset comprising correct and incorrect rehabilitation exercise executions captured from a wide range of camera angles. Furthermore, we propose a novel separability metric to quantify an algorithm's ability to distinguish between valid and faulty exercise repetitions. Using these tools, we analyze how various RGB-based pose-estimation strategies are suitable for exercise quality assessment under varying camera placements. In particular, we analyze single-camera 2D and 3D pose estimation and four multi-camera strategies: a combination of two orthogonal 2D views, 3D triangulation, weighted 3D fusion, and an AI-based pose-estimation transformer model specifically trained from two synchronized cameras. Our findings reveal that an optimally placed 2D camera can improve the separability by 16.9% over the commonly used 0{\deg} frontal view and frequently outperforms single-camera 3D estimation, while combining two views can further improve accuracy by up to 13.1%. These results offer practical guidance for deploying rehabilitation monitoring in both home and clinical settings.

1 Introduction

Rehabilitation monitoring must assess how accurately patients execute exercises, but camera viewpoint can affect the skeleton data used to detect clinically relevant errors. The paper studies this issue through a personalized monitoring pipeline, a viewpoint-diverse dataset, and a separability metric.

1.1 Personalized Rehabilitation Monitoring

  • The personalized pipeline compares home repetitions with a patient-specific reference using temporally aligned features and individualized confidence intervals.Correct repetitions recorded with a physiotherapist establish the model and reference repetition.Source: §1.1 · pp. 2–3

1.2 Patient Orientation Challenge

  • Camera viewpoint changes joint visibility and self-occlusion, which can affect movement analysis of slow, limited-range rehabilitation exercises.Pose-estimation errors of only a few degrees may be clinically significant in this setting.Source: §1.2 · p. 3
  • The paper introduces REHAB26-ViewAngles, a dataset of correct and erroneous exercise executions recorded across a wide range of camera angles.The dataset is designed to assess how patient orientation affects exercise quality assessment.Source: §1.2 · p. 3
  • A novel separability metric quantifies how well an algorithm distinguishes correct from incorrect repetitions using personalized models of correct execution.The evaluation also investigates monocular and multi-view 2D and 3D pose strategies, including the RUMPL cross-view transformer.Source: §1.2 · p. 3

2 Related Work

Prior work establishes that viewpoint affects pose estimation and movement analysis, but systematic viewpoint selection for rehabilitation remains limited by available datasets. The literature also motivates comparisons of monocular and multi-view pose methods for personalized monitoring.

Pose Estimation Methods

  • Monocular 3D methods commonly lift 2D detections, while 2D estimates may be more stable but can distort out-of-plane motion and joint angles.The cited literature describes both the viewpoint constraints of single-camera 3D estimation and the limitations of 2D skeletons.Source: §2 · p. 4
  • Multi-view methods combine synchronized camera feeds through triangulation or learned fusion, while RUMPL supports flexible camera placements but still depends on valid calibration.The suitability of these methods for subtle clinical movements with sparse two-camera arrays remains largely unexamined.Source: §2 · p. 4

Exercise Correctness Analysis

  • Personalized rehabilitation systems often compare interpretable skeletal features with a patient-specific reference because generic neural models can require substantial annotated data.Monocular RGB pipelines such as MediaPipe Pose are attractive for consumer devices because of real-time performance and deployment efficiency.Source: §2 · pp. 4–5

Role of Camera Viewpoint

  • Camera viewpoint affects joint visibility and self-occlusion, and the best view can depend on the movement being assessed.Lateral views better reveal squat depth and knee alignment, while frontal views better capture arm symmetry and shoulder positioning.Source: §2 · p. 5
  • Existing general-motion and rehabilitation datasets typically cover only a few predefined viewpoints, limiting systematic study of patient orientation and camera placement.This gap motivates REHAB26-ViewAngles.Source: §2 · pp. 5–6

3 REHAB26-ViewAngles Rehabilitation Dataset

REHAB26-ViewAngles was designed to evaluate how patient orientation affects pose estimation and personalized exercise assessment under realistic camera conditions. It combines representative exercises and clinical errors with dense viewpoint coverage and synchronized smartphone recordings.

3 REHAB26-ViewAngles Rehabilitation Dataset

  • The dataset emphasizes dense viewpoint sampling, correct and erroneous executions, and simultaneous consumer-phone recordings to support single- and multi-camera evaluation.Its exercise selection favors representative movement patterns over maximizing the number of exercise types.Source: §3 · pp. 6–7

3.1 Exercise Selection and Clinical Error Modeling

  • The three selected exercises cover different body regions and movement directions, with five physiotherapist-defined clinical errors linked to interpretable skeletal features.The set includes upper- and lower-body movements that can require different viewpoints for reliable observation.Source: §3.1 · pp. 7–8

3.2 Acquisition Protocol

  • Three synchronized smartphones recorded participants across nominal orientations spanning the frontal hemisphere, while preserving small deviations from instructed positions.These natural offsets allow viewpoint robustness to be evaluated under realistic placement conditions.Source: §3.2 · p. 8

3.3 Dataset Statistics

  • REHAB26-ViewAngles contains 1,650 annotated repetitions from three exercises, five participants, and 11 nominal orientations, evenly divided between correct and incorrect executions.Synchronized recordings from three cameras provide 29 unique viewing angles and more than six hours of RGB video.Source: §3.3 · pp. 8–9

4 Viewpoint Optimization for Rehabilitation Monitoring

This section frames viewpoint and pose-acquisition choices for personalized rehabilitation monitoring, then defines a separability metric and describes the single- and multi-camera approaches used to investigate them. The study considers both single-camera deployment and two-camera configurations.

4 Viewpoint Optimization for Rehabilitation Monitoring

  • The study evaluates camera placement and pose-acquisition strategies for personalized monitoring in single-camera home settings and two-camera clinical or home settings.The section introduces the assessment framework, research questions, and acquisition methods used in that evaluation.Source: §4 · p. 10

4.1 Movement Quality Analysis

  • The feature separability metric compares erroneous-repetition errors with errors in correct repetitions, and the exercise-level score averages across erroneous repetitions and their features.Its operational range is (0, 1]; zero marks indistinguishability from normal execution noise, while negative values indicate inverted error detection.Source: §4.1 · p. 11

4.2 Problem Formulation and Research Questions

  • The framework asks how viewpoint affects features and exercise assessment, how to choose and tolerate a viewpoint, whether 2D or 3D is more reliable, and how two views should be fused.It also asks what improvement two cameras provide over one.Source: §4.2 · p. 12

4.3 Single Camera Environments

  • Single-camera assessment compares 2D and 3D poses, balancing 3D's all-directional potential against depth uncertainty and 2D's dependence on the chosen projection.MediaPipe Pose supplies both representations, and YOLO26 cross-validates the 2D findings; 3D feature angles are measured after hip alignment.Source: §4.3 · p. 13

4.4 Multiple Camera Environments

  • For two synchronized, approximately orthogonal views, the study considers feature-specific 2D view selection, calibrated geometric triangulation, weighted 3D fusion, and the RUMPL multi-view transformer.Triangulation requires calibration; weighted fusion uses joint visibility confidence, while RUMPL is adapted to MediaPipe detections and trained using synthetic AMASS poses.Source: §4.4 · pp. 13–14

5 Experimental Evaluation

The evaluation compares camera viewpoints and pose representations for detecting incorrect rehabilitation repetitions, using separability scores computed against personalized models built from correct repetitions. It finds that camera placement and exercise-specific movement features matter, while two-view gains are balanced by deployment demands.

5.1 Methodology

  • Separability is computed per exercise sequence and skeleton acquisition method by comparing each incorrect repetition with a personalized model built from five correct repetitions.Overall scores aggregate relevant features identified using ground-truth error annotations; viewpoints span −90° to 90° for single-camera experiments.Source: §5.1 · pp. 14–15

5.2 Single Camera Environments

  • Feature-level optima vary: shoulder elevation is best assessed frontally, while knee-pass-toes features favor oblique or lateral views, often at 45° or greater.Feature separability also depends on error duration: briefly occurring errors can average to lower scores than errors persisting through more of a repetition.Source: §5.2 · pp. 15–16

5.3 Multiple Camera Environments

  • 0%, 26.6%, and 12.7% are the separability gains for the orthogonal two-view 2D method over single-camera 2D MPP in Exercises 1, 2, and 3, respectively.Triangulation gains are 1.5%, 23.3%, and -1.8%; multi-view deployment requires more space, calibration, and synchronized cameras.Source: Table 3 · p. 17, §5.3 · p. 18

5.2 Single Camera Environments

  • 16.9% is the separability improvement of an optimally placed 2D camera over the commonly used 0° frontal view; optimal angles are exercise-specific.Consensus angles across monocular models are 15° for Exercise 1, 20° for Exercise 2, and 30° for Exercise 3.Source: §5.2 · p. 16, Table 3 · p. 17
  • Features remain reasonably stable within about 5°–10° of the optimal orientation, but multiple features deteriorate rapidly beyond a 15° shift.The evaluation tested incremental angle deviations from −35° to +35° around exercise-specific best angles.Source: §5.2 · p. 17
  • 2D representations are more consistent overall; they provide clearly better separability for Exercise 3, while 3D is slightly better for Exercise 2.The authors attribute poorer 3D performance in some cases to depth-estimation errors; Figure 7 also describes substantial noise in MediaPipe 3D estimates.Source: §5.2 · pp. 17–18, Figure 7 · p. 19

5.3 Multiple Camera Environments

  • The multi-camera approaches vary in stability: triangulation gives the most stable 3D representations, while weighted merging and RUMPL perform inconsistently.The best reported camera placement is roughly −40° and 50°, a side-on pairing described as balancing self-occlusion and geometric information.Source: §5.3 · p. 18

5.4 Computational Complexity and Real-Time Viability

  • Temporal synchronization is the primary deployment challenge for evaluating joint features from two smartphone video streams.Lightweight 2D extraction on two smartphones is described as within mobile hardware capabilities.Source: §5.4 · pp. 19–20

6 Conclusions

The conclusions identify camera viewpoint as an overlooked factor and report gains from optimal single-camera placement and orthogonal two-view fusion. They also note deployment constraints and future work on accounting for error duration.

6 Conclusions

  • Slight lateral and half-profile views improve separability by 16.9% versus the commonly used 0° frontal view; placement tolerance is approximately 10°, with larger deviations substantially reducing accuracy.The study introduced REHAB26-ViewAngles and a dedicated methodology for quantifying exercise-error separability.Source: §6 · p. 20
  • 2D pose estimation yields more accurate quality assessment than 3D methods, with exercise separability better by an average 9.2%; orthogonal fusion improves performance about 13.1% over the best single-camera setup.Triangulation provides a more modest gain of approximately 7.6%.Source: §6 · p. 20
  • Orthogonal fusion is lightweight and needs no additional training, but robust temporal synchronization remains a practical challenge; merged fusion is sensitive to 3D inaccuracies and MV-HPE struggles with occlusions.The authors note that reliably resolving occlusions typically demands denser arrays, such as four or more viewpoints.Source: §6 · p. 20
  • Future work will extend separability scoring to account more explicitly for error duration, including weighting errors at movement apexes over minor fluctuations during resting phases.Source: §6 · p. 20

Declaration of Generative AI and AI-assisted technologies in the writing process

The authors used Claude and Gemini to improve the writing, then reviewed and edited the text and accepted responsibility for its content.

Declaration of Generative AI and AI-assisted technologies in the writing process

Loading 2609.35726v2…