Source-linked AI summary
Active Perception of Upper-Limb Functional Mobility through Human-Robot Handovers
Francesco Semeraro, Marta Lagomarsino, Alberto Ranavolo, Arash Ajoudani
TL;DR
Robots need to estimate workers' task-relevant upper-limb mobility during interaction, without relying on external clinical assessments. This paper models functional joint limits with impairment coefficients and estimates them through repeated handovers; simulation and physical validation report coefficient recovery, with the simulation using a matched human-model optimizer.
Problem
Robots need quantitative joint-level estimates of residual mobility to adapt workplace interactions, while clinical scores lack a quantitative joint-level link and may require external updates.
Source: §I · p. 1Method
The robot models task-available upper-limb ranges of motion with impairment coefficients and updates a belief over them from user kinematics during repeated handovers.
Source: §I · p. 2, A. Problem definition · pp. 2–3Results
In simulation, mean coefficient-estimation error was on average up to a tenth of the possible-value range, achieved in fewer than 25 handover requests; physical validation estimated shoulder and elbow flexion coefficients with suitable accuracy in up to 25 handovers.
Source: A. Estimation performance and… · p. 6, §VI · p. 7Takeaways & Limitations
The reported results support using the pipeline to update a robot's internal representation of a user's functional joint limits when needed for more personalized collaboration.
Source: A. Estimation performance and… · p. 6Simulation used a human model with the same underlying inverse-kinematics optimization as the estimator.
Source: A. Estimation performance and… · p. 6
Abstract
from arXiv · showhide
The reintegration of people in the workplace after an upper-limb injury is often a long process. Collaborative robots and humanoids operating in shared workspaces must adapt their behavior to the physical capabilities of their human partners. Achieving this requires robots to autonomously estimate task-relevant human mobility during interaction, without relying on external clinical assessments. In this work, we formulate the estimation of upper-limb functional mobility as an active perception problem. We model task-available upper-limb functional ranges of motion through impairment coefficients that parameterize reductions of healthy joint limits, and integrate this representation into a human model used by the robot during interaction. Based on this formulation, we propose an online active perception pipeline in which the robot actively probes the user through repeated handover requests while progressively updating a Bayesian belief over the impairment coefficients. This enables the robot to continuously refine its internal model of the user's joint-level capabilities through interaction rather than passive observation. The proposed framework was evaluated through simulation studies and a proof-of-concept physical validation with a robotic platform and enacted human impairments, demonstrating that the pipeline can accurately recover the impairment coefficients during real human-robot handovers.
I. INTRODUCTION
The introduction argues that workplace robots need joint-level estimates of residual upper-limb mobility to adapt interactions after injury. It proposes estimating task-available mobility through robot-initiated handovers and impairment coefficients.
I. INTRODUCTION
- Industrial human-robot collaboration lacks a system that explicitly models and adapts to user-specific joint-level limitations; existing approaches instead substitute movements or reduce the need for arm motion.The introduction contrasts this gap with robotic solutions that account for compensatory movements in rehabilitation settings.Source: §I · p. 1
- Clinical scores do not quantitatively link assessments to joint-level mobility for motion planning, and clinical ROM measurements may use movements unlike work activity.Prior approaches also rely on available and updated clinical scores, which can be demanding to obtain during recovery outside clinical settings.Source: §I · pp. 1–2
- The proposed pipeline repeatedly hands over an object, updates a belief over impairment coefficients from the user's kinematics, and requires no previous learning.The coefficients parameterize modeled functional range-of-motion bounds; estimation continues until the system finds an accurate estimate.Source: §I · p. 2
- The work contributes a robot-centric formulation of functional upper-limb mobility, an online active-perception pipeline, and validation on a real robotic platform with enacted impairments.The introduction motivates this approach by noting that workers returning after injury may not have fully regained mobility.Source: §I · pp. 1–2
II. MATERIALS AND METHODS
The active-perception loop uses repeated handovers to compare the robot's kinematic estimate with the user's recorded reach and update its belief about functional ranges of motion.
II. MATERIALS AND METHODS
- The robot holds an object at a designated handover pose and waits for the user to reach for it.This request-and-reach cycle is repeated until the termination condition is met.Source: §II · p. 2
- At the handover point, the robot estimates user kinematics from the object pose and its current estimate of the user's functional ranges of motion.The user's kinematics are recorded when the hand reaches as close as possible to the handover point.Source: §II · p. 2
- The recorded kinematics and the robot's estimate update the belief over the ranges of motion until the estimate-based stopping condition is met.Source: §II · p. 2
A. Problem definition
The problem definition represents task-available upper-limb mobility by impairment coefficients that reduce healthy joint ROM bounds. The pipeline estimates selected coefficients rather than directly observing ROM boundaries.
A. Problem definition
- Impairment coefficients α reduce healthy joint ROM bounds to represent modeled task-available functional ranges of motion; α=0 denotes full mobility and α=1 full impairment.The model's kinematic chain and rendering are shown in Figure 3.Source: A. Problem definition · pp. 2–3, Figure 3 · p. 3
- The coefficients parameterize functional joint limits, and the pipeline estimates their components rather than directly observing each joint's ROM boundaries.Source: A. Problem definition · p. 3
- The estimation focuses on shoulder and elbow flexion because their coordinated motion contributes principally to forward reaching, the movement elicited by handovers.Source: A. Problem definition · p. 3
B. Human model
The human model encodes impairment through coefficients that reduce healthy joint ranges of motion, and an active perception pipeline updates beliefs about those coefficients from handover interactions.
B. Human model
- Each considered degree of freedom has an impairment coefficient that reduces its healthy range of motion to model functional limits.The model incorporates the coefficients into its inverse kinematics.Source: B. Human model · p. 3
- The optimization penalizes motion of more impaired joints while softly encouraging comfortable configurations for healthy joints.The diagonal matrix A = diag(α) weights motion penalties, and γ weights the comfortable-posture term.Source: B. Human model · p. 3
- Task constraints match the object pose to a desired pose that is iteratively updated to elicit informative upper-limb motions.These configurations are used to improve estimation of the impairment coefficients.Source: B. Human model · p. 3
- The model assumes upper-arm motion is sufficient for estimating joint ranges, so users are instructed to remain stationary and avoid trunk compensation.The active perception process is conducted outside regular work activity to support reproducible conditions.Source: B. Human model · p. 3
C. Active perception
- At each handover, the robot records the user's reaching configuration and updates a particle-filter belief over impairment coefficients using the observed kinematics.The particle set represents the prior, and candidate inverse-kinematics solutions are evaluated against the observation to update particle weights.Source: C. Active perception · pp. 3–4
- The pipeline can update its estimate even when the user cannot reach the object, because its inverse-kinematics optimization runs when the pose constraint cannot be met exactly.The optimizer satisfies the object-pose constraint to the best extent allowed by the modeled impairment.Source: C. Active perception · p. 4
- The estimate is accepted after the variance criterion is met for three consecutive iterations, or after 25 handovers if that criterion is unmet.Acceptance uses the weighted mean of the impairment belief.Source: C. Active perception · p. 4
- When the effective-particle ratio falls below 0.4, particles are resampled with replacement according to their weights and perturbed with Gaussian noise.Resampling addresses cases where many particles have negligible influence on the estimate.Source: C. Active perception · p. 4
III. EXPERIMENTS
The evaluation combines simulations across shoulder and elbow flexion impairment configurations with physical handovers involving enacted impairments. The physical setup used a robotic arm, motion capture, and handover poses tailored to the participant's reach.
III. EXPERIMENTS
- Simulation estimated shoulder- and elbow-flexion impairment coefficients over 1,600 combinations, with other upper-limb degrees of freedom fixed to zero.The two coefficients ranged from 0 to 1 in increments of 0.025641.Source: §III · p. 4
- Physical trials used one participant who moved freely or enacted half or full impairment of shoulder or elbow flexion.A tunable harness constrained elbow flexion; handovers were performed by a Franka Research 3 robot.Source: §III · p. 4
- The robot selected handover poses from a workspace cube tailored to the participant's height and reach, while markerless motion capture supplied joint values at each reach.The mug handle was moved to the selected pose, and the human model was scaled to the participant before the trials.Source: §III · p. 5, Figure 5 · p. 5
IV. RESULTS
Simulation supported repeated trials across impairment settings, whereas the fewer real-life recordings showed how active perception progressed over time.
IV. RESULTS
- Simulation experiments provided statistics on pipeline performance by allowing repeated trials under many impairment settings.Real-life recordings covered fewer instances and were used to show progression over time.Source: §IV · p. 5
- The results section distinguishes broad simulation statistics from the smaller set of real-life progression examples.The real-life recordings relate to fewer instances than the simulations.Source: §IV · p. 5
- The physical recordings were used to show the progression of active perception over time.The source distinguishes this purpose from the simulation-based performance statistics.Source: §IV · p. 5
A. Simulation
Across shoulder and elbow flexion impairment configurations, simulation performance was poorer for cases with low elbow-flexion impairment, which had higher mean error and more requested iterations.
A. Simulation
- The performance metrics cover shoulder and elbow flexion impairment configurations, with each configuration evaluated over 400 trials.The error is the mean of the components of |α̂ − α*|.Source: Table I · p. 5, A. Simulation · p. 5
- Low α*_Ey cases had higher mean error and required more iterations than higher α*_Ey cases.The passage also reports fewer resamplings in cases with high α*_Ey.Source: A. Simulation · p. 5, Table I · p. 5
- Cases with low elbow-flexion impairment met the termination condition fewer times, corresponding to more requested iterations.The results also note fewer resamplings for cases with high α*_Ey.Source: A. Simulation · p. 5
B. Physical validation
Physical handovers produced progressively refined impairment estimates, with convergence depending on which joint limitation the handover poses revealed. Performance was weaker when shoulder mobility was high and elbow flexion was unimpaired.
B. Physical validation
- 11 iterations yielded early convergence for α*_Ey = 1.0, while α*_Sy = 1.0 required 22 iterations for an accurate shoulder estimate.For α*_Ey = 1.0, the elbow limitation was identified by iteration two and the reconstructed posture closely matched the participant by iteration eight.Source: B. Physical validation · pp. 5–6
- When α*_Sy is low and α*_Ey = 0, estimated αEy is off by 0.4 relative to α*_Ey, while estimated αSy is much more accurate at process termination.Source: B. Physical validation · p. 6
- Three recordings converged to precise estimates before the iteration limit, and the pipeline identified the relative magnitude of the impairments from early iterations.The plotted shaded areas represent uncertainty; traces ending before iteration 25 met the early-termination criterion.Source: B. Physical validation · pp. 5–6, Figure 6 · p. 6
V. DISCUSSION
The discussion frames the results as demonstrating the pipeline’s potential while acknowledging limitations that affect estimation performance.
V. DISCUSSION
- The reported results demonstrate the potential of the proposed active perception pipeline.Source: §V · p. 6
- The authors also identify limitations that currently affect estimation performance.Source: §V · p. 6
- The discussion considers these limitations before turning to broader considerations.Source: §V · p. 6
A. Estimation performance and limitations
The pipeline estimates impairment coefficients from handover kinematics without prior learning. In simulation, it achieved errors averaging up to one tenth of the possible coefficient range in fewer than 25 handover requests, while physical validation exposed a condition that limits elbow estimation.
A. Estimation performance and limitations
- Average error reached up to one tenth of the possible-value range for shoulder and elbow flexion in simulation, in fewer than 25 handover requests on average.The estimates relied on online handover kinematics without prior learning.Source: A. Estimation performance and… · p. 6
- The results support updating a robot’s internal representation of functional joint limits when needed for more personalized collaboration.Source: A. Estimation performance and… · p. 6
- When α*_Ey = 0 and α*_Sy was low, shoulder movement reduced elbow excursion, leaving the elbow impairment coefficient less accurately constrained.The discussion attributes this to insufficient elicited elbow flexion during requested handovers.Source: A. Estimation performance and… · p. 6
B. Future developments and deployment on humanoids
The pipeline estimates shoulder and elbow impairment coefficients through handovers, recovering them with suitable accuracy in up to 25 handovers. The discussion identifies limits in pose selection and action-space coverage, while outlining how guided reaching could extend to humanoids and other embodied platforms.
B. Future developments and deployment on humanoids
- The current action space may be too limited, and informative target poses must also lie within the robotic platform’s reachable workspace.The authors suggest redesigning its geometry to elicit greater elbow-flexion contributions.Source: B. Future developments and deployment… · p. 6
- The study randomly selected handover poses and evaluated shoulder and elbow flexion, while broader impairment estimates and additional modeled degrees of freedom remain future extensions.Adding degrees of freedom would require reliable sensing of the corresponding joint variables.Source: B. Future developments and deployment… · p. 7
- The authors frame fROM estimation as platform-independent guided reaching, with alternative reachable poses possible when a desired target is inaccessible.The embodied system presents target poses and observes the resulting human motion; its kinematic constraints shape which poses it can realize.Source: B. Future developments and deployment… · p. 7
- Humanoids and mobile manipulators may offer broader workspaces for target presentation, while embodied robots can manipulate objects and integrate mobility estimates into later actions.The paper also notes augmented reality as a possible complementary interface and describes humanoid posture and positioning as adaptable to user limitations.Source: B. Future developments and deployment… · p. 7, §VI · p. 7
VI. CONCLUSIONS
- The conclusion describes a particle-filter active-perception pipeline for estimating impairment coefficients that parameterize goal-oriented upper-limb functional ranges of motion.The intended future use is to tune interaction to users’ actual capabilities and support an active recuperation process.Source: §VI · p. 7
- Suitable-accuracy estimates of shoulder and elbow flexion impairment coefficients were obtained in real handovers with up to 25 handovers.A participant wore harnesses that partially restricted movement within delimited ranges while interacting with a robotic manipulator.Source: §VI · p. 7