PaperlayerView exact revision on arXiv

Source-linked AI summary

Goal Set Inverse Optimal Control and Iterative Re-planning for Predicting Human Reaching Motions in Shared Workspaces

Jim Mainprice, Rafi Hayne, Dmitry Berenson

arXiv:1606.02111v1cs.RO

TL;DR

Predicting human motion in shared workspaces matters for human-robot collaboration, but collaborative reaching is difficult to model. This paper learns a motion cost from human demonstrations and uses iterative replanning to predict reaching paths. The authors report that it outperforms baselines in most cases, including a shared human-robot workspace.

  • Problem

    Predicting how people move during collaborative tasks matters for shared-workspace collaboration, while the paper identifies human-motion prediction as an important step toward more fluid human-robot collaboration.

    Source: §I · p. 1

  • Method

    The authors learn a cost function from collaborative-motion demonstrations with inverse optimal control, then use iterative STOMP-based replanning to predict reaching motions.

    Source: §I · p. 2

  • Results

    The learned cost function outperforms baseline methods in most cases, and the method also outperforms baselines for prediction in a shared human-robot workspace.

    Source: §I · p. 2

  • Takeaways & Limitations

    Two-human collaborative motion provides a baseline for judging when a person deviates from a natural motion in response to a robot’s actions.

    Source: §I · p. 1

    The learned cost need not be the human’s true criterion, and its result is not informative if that criterion is absent from the selected basis functions.

    Source: C. Inverse Optimal Control · p. 3

Abstract

from arXiv · show

To enable safe and efficient human-robot collaboration in shared workspaces it is important for the robot to predict how a human will move when performing a task. While predicting human motion for tasks not known a priori is very challenging, we argue that single-arm reaching motions for known tasks in collaborative settings (which are especially relevant for manufacturing) are indeed predictable. Two hypotheses underlie our approach for predicting such motions: First, that the trajectory the human performs is optimal with respect to an unknown cost function, and second, that human adaptation to their partner's motion can be captured well through iterative re-planning with the above cost function. The key to our approach is thus to learn a cost function which "explains" the motion of the human. To do this, we gather example trajectories from pairs of participants performing a collaborative assembly task using motion capture. We then use Inverse Optimal Control to learn a cost function from these trajectories. Finally, we predict reaching motions from the human's current configuration to a task-space goal region by iteratively re-planning a trajectory using the learned cost function. Our planning algorithm is based on the trajectory optimizer STOMP, it plans for a 23 DoF human kinematic model and accounts for the presence of a moving collaborator and obstacles in the environment. Our results suggest that in most cases, our method outperforms baseline methods when predicting motions. We also show that our method outperforms baselines for predicting human motion when a human and a robot share the workspace.

I. INTRODUCTION

The paper addresses predicting collaborative reaching in shared workspaces, where human motion around obstacles and another person is difficult to model. It proposes learning a cost function from demonstrations and using it to predict reaching motions.

I. INTRODUCTION

  • Human motion in collaborative tasks is difficult to model because social, interference, and comfort criteria are unclear, and few works address collaborative manipulation among obstacles.The authors state that no framework exists for predicting such motion among obstacles.Source: §I · p. 1
  • The framework focuses on reaching motions, a category relevant to manufacturing pick-and-place tasks and a step toward safe, efficient human-robot collaboration.Two-human collaboration provides a baseline for judging when a person deviates in response to a robot’s actions; using predictions in the robot’s planner is outside this paper’s scope.Source: §I · p. 1
  • The approach assumes human trajectories are locally optimal for an unknown cost and models adaptation through iterative replanning under that cost.Training motions are recorded from two people collaborating, then manually segmented into reaches; PIIRL learns feature weights used by a STOMP-based replanner that searches task-space goal regions.Source: §I · pp. 1–2
  • The learned cost function outperforms baseline methods in most cases, and iterative replanning better captures adaptation when partners’ motions interfere significantly.The method also outperforms baselines for predicting human motion in a shared human-robot workspace.Source: §I · p. 2
  • This version adds task-space goal-set optimization, an expanded human-subject study, generalization tests, and a human-robot workspace-sharing experiment.The goal-set change addresses settings where the human’s goal configuration is unknown.Source: §I · p. 2

A. Probabilistic Graphical Models

  • Graphical models encode relations among activities, objects, and motions but handle obstacle constraints poorly; the authors report outperforming their employed GMM approach on these reaching motions.Source: A. Probabilistic Graphical Models · p. 2

B. Optimal Control

The paper treats human reaching as behavior that can be modeled by an optimality criterion, then uses inverse optimal control to learn a practical predictive cost. Its approach combines collaborative-motion demonstrations with iterative replanning.

B. Optimal Control

  • An optimality criterion can model human motor behavior, and this work seeks one for predicting reaching in shared workspaces without a subject-specific musculoskeletal model.Source: B. Optimal Control · p. 2

C. Inverse Optimal Control

  • Inverse Optimal Control recovers a cost or reward function optimized by an agent from demonstrated trajectories or policies.The paper notes that this problem is also called Inverse Reinforcement Learning.Source: C. Inverse Optimal Control · p. 2
  • The learned cost is a practical prediction model, not necessarily the human’s true criterion; if the true criterion is absent from the selected basis functions, the result is not informative.Source: C. Inverse Optimal Control · p. 3

III. APPROACH

  • The approach segments collaborative-motion libraries into elementary reaches, learns a cost with IOC, then predicts motion near a human or robot through iterative replanning.The trajectory optimizer is based on STOMP.Source: §III · p. 3, D. Trajectory Optimization · p. 3

A. Inverse Optimal Control algorithm

The algorithm represents reaching trajectories with kinematic features and learns their relative weights from demonstrations and sampled alternatives. Goal-set trajectory sampling supports learning when the final configuration is unspecified.

A. Inverse Optimal Control algorithm

  • Trajectory cost is parameterized as a weighted sum of user-defined features, making learning the weight vector tractable.The trajectory is represented as a sequence of configurations; its vector space has dimension M×N, with 23 degrees of freedom and 100 waypoints in this work.Source: A. Inverse Optimal Control algorithm · p. 3
  • PIIRL solves IOC locally using sampled trajectories and requires only local optimality of the demonstrations in high-dimensional continuous state-action spaces.Its fixed-distribution prior encodes a kinematic motion property; the demonstrations need not be globally optimal.Source: A. Inverse Optimal Control algorithm · p. 3
  • PIIRL samples around demonstrations with a Gaussian distribution shaped by finite differences, choosing K so samples have low summed accelerations.The learned feature cost combines a smoothness term with a general configuration-dependent term.Source: A. Inverse Optimal Control algorithm · pp. 3–4
  • Goalset-PIIRL learns weights for task-space goal sets where the final configuration is unspecified, using a modified trajectory-sampling distribution and a convex minimization objective.The objective compares features of demonstrations with those of sampled trajectories around each demonstration.Source: A. Inverse Optimal Control algorithm · p. 4
  • The sampling process permits variation in the final configuration, then projects samples to joint limits and the goal set; colliding samples are discarded.The projection uses metric R, and collision detection considers the environment and the other human.Source: A. Inverse Optimal Control algorithm · p. 4

B. Iterative re-planning

The method segments collaborative motions into reaching examples for inverse learning, then uses goal-set STOMP to replan against a changing workspace. Its planner accounts for trajectory smoothness, obstacles, posture, and the other human.

B. Iterative re-planning

  • Demonstrations are manually segmented into elementary motions and then into shorter segments advanced along each trajectory for the training set.Initial velocity, acceleration, and jerk, plus the other human’s configuration and obstacle positions, contribute features for each segment and its samples.Source: B. Iterative re-planning · p. 4

C. Goal Set STOMP

  • STOMP iteratively deforms a trajectory with fixed initial and goal configurations, stochastically estimating the gradient from Gaussian trajectory samples.It does not require an analytical cost-function gradient and generally converges to a local minimum within 100 iterations.Source: C. Goal Set STOMP · pp. 4–5
  • Goal-set STOMP samples trajectories with different goal configurations while conditioning them to meet the task-space goal region.Jacobian-based inverse kinematics seeds the optimizer because analytical methods cannot be applied to the redundant human kinematics.Source: C. Goal Set STOMP · p. 5
  • The planner combines obstacle and smoothness costs, adds task-space smoothness and a third criterion for posture and the other human, and uses prior replanning waypoints to estimate initial derivatives.The obstacle-cost weight is manually tuned.Source: C. Goal Set STOMP · p. 5

D. Goal Set Trajectory Sampling

Goal-set trajectory sampling adapts trajectory optimization and inverse learning to goals specified as task-space regions rather than fixed endpoint configurations. It modifies the sampling covariance and projects samples to the goal set, with the stated formulation restricted to a 3D point with free rotation.

D. Goal Set Trajectory Sampling

  • Standard STOMP and PIIRL sample from a Gaussian centered on the current trajectory or demonstration, with a fixed endpoint configuration.The goal-set version changes this sampling to permit trajectories meeting a goal region.Source: D. Goal Set Trajectory Sampling · p. 5
  • Modified covariance sampling and projection to the goal set allow different postures at the goal configuration.The formulation defines the goal constraint through the final configuration placing the hand at a given point.Source: D. Goal Set Trajectory Sampling · p. 5
  • The goal-set definition is restricted to a 3D point with free rotation, though a similar algorithm can be derived for broader task-space regions.Source: D. Goal Set Trajectory Sampling · p. 5
  • Figure 4 visualizes collision-free, joint-limit-respecting samples; its colors encode sample density, and larger standard deviation explores more around the goal region.The figure uses 1,000 samples for a kernel-density estimate of hand-frame roll and pitch, with 50 samples showing yaw as black arrows.Source: D. Goal Set Trajectory Sampling · p. 6, Figure 4 · p. 6

E. Human kinematic model description

The human model represents upper-body and right-arm motion with a 23-DoF kinematic model. Its joint bounds come from motion-capture observations, with offsets to allow optimization beyond observed values.

E. Human kinematic model description

F. Feature functions

The feature functions encode interpersonal distance, trajectory smoothness, and distance from a predefined resting posture. Together these features represent avoidance, movement regularity, and posture preferences in the learned cost.

F. Feature functions

  • Interpersonal features use 16 pairwise distances along the arms and pelvis to model avoidance while allowing close interaction.The feature types are inspired by proxemics theory and neuroscience experiments.Source: F. Feature functions · p. 6

2) Smoothness:

  • Smoothness features measure configuration- and task-space length, squared velocity, acceleration, and jerk along the trajectory.Source: 2) Smoothness · p. 6

3) Distance to a resting posture:

IV. HUMAN COLLABORATION EXPERIMENT SETUP

The study used a structured packing task to collect collaborative reaching motions from pairs of participants. Motion capture recorded their movements, which were converted into 23 DoF arm-and-torso configurations.

IV. HUMAN COLLABORATION EXPERIMENT SETUP

  • Pairs stood shoulder to shoulder at a shared table and placed colored balls on matching pegs in a specified order.The order prevented participants from switching tasks mid-motion, isolating manipulation planning.Source: §IV · pp. 6–7, A. Experiment flow · p. 7

B. Recording method

  • Eight Bonita cameras in a Vicon motion-capture system recorded the participants’ interactions.Source: B. Recording method · p. 7
  • Markers on the suit identified joint centers used to derive a 23 DoF configuration of each participant’s right arm and torso.Marker locations included the waist, head, hand, wrist, elbow, shoulder, sternum, and xyphoid process.Source: B. Recording method · p. 7

C. Instructions to participants

V. HUMAN COLLABORATION EXPERIMENT RESULTS

The section evaluates learned cost weights and predicted trajectories using leave-one-out tests. The IOC weights and baseline 0 generally achieve lower DTW scores than baseline 1, while replanning offers little benefit when interference is absent.

1) Validation scores:

  • The evaluation reports joint-center and task-space distances; configuration-space distance is excluded because kinematic redundancy makes it unfair.Source: 1) Validation scores · p. 8

1) Leave-one-out testing:

  • The leave-one-out test predicts seven reaching motions, using 33 segmented demonstrations for IOC and comparing predictions with DTW.DTW compares temporal sequences that may differ in timing or speed.Source: 1) Validation scores · p. 7, 1) Leave-one-out testing · p. 8
  • Normalized feature weights indicate that smoothness and posture matter more than distance features; acceleration and jerk dominate, while velocity and joint-space path length contribute little.Wrist-to-wrist distances have no influence, and pelvic distance weights have limited effect because participants move their pelvises less than their arms.Source: 1) Leave-one-out testing · p. 8
  • Baseline 0 and IOC predictions have lower DTW scores than baseline 1 across both metrics, with and without replanning.Without significant interference in this dataset, no-replanning tends to perform slightly better.Source: 1) Leave-one-out testing · p. 8

2) Generalization Among Participants:

Across the reported predictions, IOC-tuned weights usually outperform baseline methods on task-space distance. In the broader participant and goal-region study, the advantage depends on the metric and baseline.

2) Generalization Among Participants:

  • For generalization testing, 73 training and 20 testing motions from three participant pairs were selected, and training augmentation produced 461 demonstrations.The selected motions covered diverse start configurations and goal regions.Source: 2) Generalization Among Participants · p. 8
  • IOC outperforms baselines on task-space distances in this study, but not baseline 0 on joint-center distances.The segmented motions were generally shorter than the leave-one-out motions, making direct-motion approximation easier for the baselines.Source: 2) Generalization Among Participants · pp. 8–9
  • Across 27 predicted motions, IOC-tuned weights outperform baselines on task-space distance in all but one test, and lead in six of eight similarity tests.Baseline 0 outperforms the alternatives in the other two tests.Source: 2) Generalization Among Participants · p. 9

3) Significant interference:

Replanning better predicts motion in the selected case of significant partner interference, while the broader smoothness and GMM-GMR comparisons reveal distinct limits and trade-offs.

3) Significant interference:

  • With significant interference, replanning better predicts the active human’s motion because single-shot trajectories collide with the passive human’s arm.The replanned trajectories also have smaller average DTW values for both joint-center and task-space metrics.Source: 3) Significant interference · p. 9
  • The significant-interference comparison does not use goal-set algorithms and uses different task-space metric tuning from the other experiments.Source: 3) Significant interference · p. 9

B. Smoothness Analysis

  • For the larger multi-participant dataset, predicted motions tend to be 30% smoother than observed motions.The authors attribute this to greater motion diversity and possible marker occlusions during recording.Source: B. Smoothness Analysis · p. 10

C. Comparison to GMM-GMR prediction

  • IOC and motion-planning predictions outperform GMM-GMR on the study data, with no significant improvement from including both humans’ configurations in the joint distribution.GMM classifications were tested after 5%, 10%, and 30% of trajectory execution.Source: C. Comparison to GMM-GMR prediction · p. 10

VI. HUMAN-ROBOT WORKSPACE SHARING EXPERIMENT

In a human–robot workspace-sharing study, the learned IOC cost function outperformed baseline tunings, and predictions improved as participants acclimated to the robot. The study used motion-capture reaching data and leaves richer features and PR2 retargeting for future work.

VI. HUMAN-ROBOT WORKSPACE SHARING EXPERIMENT

  • The study enrolled 16 participants, retained 15 reliable recordings, and assembled 2,120 human reaching motions from 120 trials for the robot-collaboration experiment.The PR2 executed predetermined straight-line trajectories intended to occlude the workspace while allowing the human to complete the task.Source: §VI · pp. 10–11

A. Results

  • IOC outperformed baseline tunings across all 2120 elementary motions; the conclusion reports gains over hand-tuning in task-space and joint-center-distance DTW.The comparison ran without replanning.Source: Figure 12 · p. 11, A. Results · p. 11, §VII · p. 11
  • For robot predictions, the method reused weights learned in the human study and mapped link-distance features to PR2 wrist, elbow, shoulder, and pelvis kinematics.Source: A. Results · p. 11
  • After excluding the highest-scoring 5%, IOC predictions improved from the first, least-acclimated run to the seventh, with lower mean and standard deviation.The authors attribute early unpredictability to hesitation and note that the method assumes humans continue toward the goal.Source: Figure 14 · p. 11, A. Results · p. 11

VII. CONCLUSION AND FUTURE WORK

  • Future work targets richer prediction features and retargeting those features for motion planning on a PR2.Source: §VII · p. 11

Related papers

Loading 1606.02111v1…