PaperlayerView exact revision on arXiv

Source-linked AI summary

Human-Robot Mutual Adaptation in Shared Autonomy

Stefanos Nikolaidis, Yu Xiang Zhu, David Hsu, Siddhartha Srinivasa

arXiv:1701.07851v1cs.RO

TL;DR

Shared autonomy must balance task performance with user trust when human intent may be suboptimal and only robot actions change the world. The paper integrates a bounded-memory human adaptation model into a partially observable decision process, and a human-subject experiment found improved team performance while retaining high trust compared with following user preferences.

  • Problem

    Shared autonomy must handle partially observable, potentially changing human goals while preserving trust when users choose suboptimal strategies.

  • Method

    The paper integrates a Bounded-memory Adaptation Model into a mixed-observability Markov decision process for mutual human-robot adaptation.

  • Results

    The proposed formalism significantly improved human-robot team performance while retaining high human trust compared with following participants’ preferences.

  • Takeaways & Limitations

    Mutual adaptation balances optimizing task performance with retaining user trust by guiding adaptable users and complying with non-adaptable users.

  • Takeaways & Limitations

    The formulation assumes human adaptability remains fixed throughout the task.

Abstract

from arXiv · show

Shared autonomy integrates user input with robot autonomy in order to control a robot and help the user to complete a task. Our work aims to improve the performance of such a human-robot team: the robot tries to guide the human towards an effective strategy, sometimes against the human's own preference, while still retaining his trust. We achieve this through a principled human-robot mutual adaptation formalism. We integrate a bounded-memory adaptation model of the human into a partially observable stochastic decision model, which enables the robot to adapt to an adaptable human. When the human is adaptable, the robot guides the human towards a good strategy, maybe unknown to the human in advance. When the human is stubborn and not adaptable, the robot complies with the human's preference in order to retain their trust. In the shared autonomy setting, unlike many other common human-robot collaboration settings, only the robot actions can change the physical state of the world, and the human and robot goals are not fully observable. We address these challenges and show in a human subject experiment that the proposed mutual adaptation formalism improves human-robot team performance, while retaining a high level of user trust in the robot, compared to the common approach of having the robot strictly following participants' preference.

1. INTRODUCTION

Shared autonomy can ease difficult teleoperation, but conventional assistance may follow a user toward a poor strategy. The paper proposes mutual adaptation that guides adaptable users while complying with stubborn users, improving team performance while retaining trust.

  • Motivation: Shared autonomy combines direct teleoperation with autonomous assistance for complex robotic-arm tasks.Teleoperation through interfaces such as joysticks can make complex tasks difficult or tedious.
  • Motivation: Users may choose suboptimal strategies, such as an unstable grasp or an inefficient object arrangement.The paper illustrates these risks with unstable grasping and incorrect stacking order.
  • Mutual adaptation: A robot should comply with a stubborn user to preserve trust but insist on an optimal goal when the user is adaptable.Failing to preserve trust may lead to system disuse, whereas adaptation can support mutually adaptive behavior.
  • Problem: Shared autonomy differs from collaboration because human actions do not affect the task state, so performance-only optimization can ignore human inputs and always pursue the optimal goal.This setting therefore requires explicit consideration of human adaptability and trust.
  • Contribution: The proposed formalism models mutual adaptation across collaboration and shared-autonomy settings.It incorporates the operator goal as a latent variable in a mixed-observability Markov decision process and tracks uncertainty over user goals.
  • Evaluation: In a human-subject experiment with n = 51, the formalism significantly improved team performance over following participants’ preferences while retaining high human trust.The evaluation used an assistive robotic arm on a table-clearing task.

2. PROBLEM SETTING

The paper models shared autonomy as a partially observable multi-agent decision problem in which human intent can change and only robot actions alter the world state. Bounded-memory adaptation simplifies the human model while retaining adaptability and disagreement costs.

  • Problem representation: The team is modeled with world state, human and robot actions, latent human goals, and modal policies associated with those goals.Human actions provide observations from which the robot infers the user’s modal policy.
  • Problem representation: Unlike static-intent models, the human mode can change over time as a function of interaction history, world state, and robot actions.The transition function Tmh models these mode changes.
  • Objective: The reward captures both the relative cost of goals and disagreement between human and robot modes.The robot maximizes discounted expected reward over time.
  • Bounded-memory adaptation: The Bounded-memory Adaptation Model replaces the full interaction history with the last k interactions and parameterizes human adaptation with α.This simplifies both the mode-transition and reward functions.
  • Shared-autonomy dynamics: In shared autonomy, joystick inputs do not affect the world state, while robot actions deterministically change the robot configuration.The resulting transition function depends only on the robot configuration and robot action.
  • Comparison with collaboration: The collaboration formalism instead allows both human and robot actions to affect the world state and can omit disagreement penalties when modes are observable.This distinguishes the shared-autonomy formulation from the collaboration setting.

3. HUMAN AND ROBOT MODE INFERENCE

The paper represents uncertain human and robot intentions with stochastic modal policies and infers them from recent interaction history. Because different policies can produce the same action, the modes remain partially observable and require belief updates.

  • Mode inference: Robot inference uses joystick inputs and robot behavior to update beliefs about the human and robot modes.In the table-clearing example, movement toward a bottle increases belief in the corresponding modal policy.
  • Stochastic modal policies: Multiple modal policies may lead to the same goal, so the paper uses stochastic modal policies to compactly represent feasible paths.This reduces the need to enumerate every path while preserving probabilistic uncertainty about future actions.
  • Partial observability: A large number of partially observable modes increases computational cost and makes exact modal-policy inference impossible without prior information in ambiguous cases.The same action, such as upward movement, can be consistent with policies toward different goals.
  • Bounded-memory inference: The robot models mutual inference because humans attribute recent robot actions to a robot mode using bounded memory.The human recalls only the last k time-steps, while the robot estimates the human mode from observations.
  • Modal-policy representation: The mode representation maps the current robot state to probability distributions over human and robot actions.The paper restricts modal policies to generate actions from the current world state.
  • Human adaptation: The human-mode transition model assigns switching probability using adaptability α, with α = 1 implying certain switching and α = 0 implying no adaptation.The robot combines this transition model with its estimate of the robot mode to predict human switching.
  • Human-mode inference: The robot estimates the human mode with an observation function that defines a probability distribution over human actions.This belief update is performed with a Bayes filter.

4. DISAGREEMENT BETWEEN MODES

The robot penalizes disagreement between human and robot modes to balance reaching the optimal goal with retaining trust. It follows a stubborn human’s preference but guides an adaptable human toward the optimal goal.

  • The robot infers the human goal and estimates how likely the human is to switch goals after a robot action.
  • For a non-adaptable human, disagreement costs outweigh the optimal-goal reward, so the robot follows the human preference to retain trust.
  • For an adaptable human, the robot moves toward the optimal goal because it expects the human to change modes.
  • The reward function penalizes the robot for following a mode perceived as different from the human’s mode.

5. HUMAN-ROBOT MUTUAL ADAPTATION FORMALISM

The formalism models hidden human adaptability and mode within a MOMDP, enabling the robot to update beliefs and choose actions that balance task reward with human adaptation. Simulations show compliance with stubborn users and movement toward the optimal goal for more adaptable users.

  • 5.1 MOMDP Formulation: The MOMDP includes robot configuration and action history as observable variables.
  • 5.1 MOMDP Formulation: Human adaptability α and human mode mh are partially observable variables in the MOMDP.
  • 5.1 MOMDP Formulation: Robot actions form a finite set of transitions between discrete robot configurations, with deterministic state updates.
  • 5.1 MOMDP Formulation: The model assumes human adaptability remains fixed throughout the task.
  • 5.1 MOMDP Formulation: The human-mode transition depends on history, robot state, human adaptability, and robot action, while the reward function assigns immediate action rewards.
  • 5.2 Belief Update: The robot maintains a joint distribution over adaptability and human mode, then computes an optimal policy using the MOMDP belief.
  • 5.3 Robot Policy: When robot and human modes likely match, the robot matches human input; otherwise, it executes the MOMDP policy.
  • 5.4 Simulations: With bounded memory k = 1, simulations show compliance for α = 0.0 and adaptation toward mR for α = 0.75.

6. HUMAN SUBJECT EXPERIMENT

The experiment compares fixed, mutual-adaptation, and one-way adaptation policies in shared autonomy, testing whether mutual adaptation balances task performance with trust and perceived collaboration.

  • Experimental design: The experiment tests whether mutual adaptation improves team performance while retaining high perceived collaboration and trust.The hypotheses predict performance differences across conditions and evaluate trust and perceived robot performance as teammate measures.
  • Experimental design: The study compares No-adaptation, Mutual-adaptation, and One-way adaptation conditions in a shared-autonomy table-clearing task.The robot either follows a fixed optimal policy, adapts mutually using the MOMDP policy, or assists the user’s preferred goal.
  • Experimental design: The Mutual-adaptation condition uses the MOMDP policy, whereas the No-adaptation condition always pursues the optimal goal and One-way adaptation follows estimated user preferences.The conditions operationalize different assumptions about human adaptability and robot assistance.
  • Experimental design: Fifty-one participants completed the shared-autonomy experiment, using joystick inputs to clear a table with two bottles.Participants first trained with the joystick, selected a preferred bottle, and then interacted with a robot whose optimal goal differed from that preference.

7. ANALYSIS

Mutual adaptation caused many participants to switch toward the optimal goal and produced significantly better objective performance than one-way adaptation, while trust and perceived collaboration remained high.

  • Objective measures: 56% of Mutual-adaptation participants selected the optimal goal in the final trial, and 75% did so in at least one trial.Only two participants in One-way adaptation changed goals, while 15 of 17 consistently pursued their preferred suboptimal goal.
  • Objective measures: Mean team reward differed significantly across conditions, with significant pairwise differences between No-adaptation and Mutual-adaptation and between Mutual-adaptation and One-way adaptation.The omnibus test yielded χ2(2) = 39.84, p < 0.001; pairwise tests yielded U = 28.5, p < 0.001 and U = 49.5, p = 0.001, respectively.
  • Subjective measures: Mutual-adaptation participants reported high trust (M = 3.94, SE = 0.18) and perceived collaboration (M = 3.91, SE = 0.12).Both subjective scales had good consistency and were combined into scores.
  • Subjective measures: Trust and perceived collaboration differed significantly across conditions, while Mutual-adaptation did not differ significantly from One-way adaptation on either measure.The ANOVA results were trust F(2, 48) = 8.370, p = 0.001 and perceived collaboration F(2, 48) = 9.552, p < 0.001.

8. DISCUSSION

The discussion presents mutual adaptation as a way to balance optimal task performance with human trust, while noting computational growth as tasks, actions, and memory length expand.

  • Discussion: Mutual adaptation balances performance and trust between a fixed optimal policy and an assistance policy that follows user preferences.The experiment compares the formalism with both endpoint policies.
  • Discussion: The fixed policy achieved maximum performance but reduced trust, whereas One-way adaptation produced the worst performance by following suboptimal goals.Mutual adaptation performed better than One-way adaptation and earned more trust than the fixed policy.
  • Discussion: Changing adaptability to 0 with equal goal costs reproduces One-way adaptation, while setting adaptability to 1 reproduces the fixed optimal policy.These parameter settings make the MOMDP policy identical to the two endpoint policies.
  • Scope and scalability: The table-clearing formulation can extend to tasks with many goals, inputs, and robot actions, but observable-state growth is polynomial in robot actions and memory length.For large action sets and memory lengths, the authors suggest feature-based representations for approximating history.
  • Future applications: The authors propose exploring applications beyond assistive robotic arms, including powered wheelchairs and remote manipulators.These settings combine human inputs with robot autonomy.

Funding

The work was supported by several government agencies and the Onassis Foundation.

  • Funding came from DARPA SIMPLEX through ARO, the National Institutes of Health, the National Science Foundation, the Office of Naval Research, and the Onassis Foundation.The listed support included ARO contract 67904LSDRP, NIH grant R01EB019335, and NSF CPS award 1544797.
Loading 1701.07851v1…