PaperlayerView exact revision on arXiv

Source-linked AI summary

Controlling Assistive Robots with Learned Latent Actions

Dylan P. Losey, Krishnan Srinivasan, Ajay Mandlekar, Animesh Garg, Dorsa Sadigh

arXiv:1909.09674v3cs.RO

TL;DR

Assistive arms are dexterous but difficult to teleoperate through low-dimensional inputs. The paper learns latent actions from task demonstrations and evaluates them in user studies, where participants performed eating and cooking tasks more efficiently and found the approach easier.

  • Problem

    Assistive arms have more degrees of freedom than users can directly coordinate through low-dimensional controls, making teleoperation challenging.

    Source: §I · p. 1

  • Method

    The paper learns low-dimensional latent actions from demonstrations and evaluates embeddings for controllability, consistency, and scalability.

    Source: §I · p. 1, §VII · p. 8

  • Results

    In eating and cooking user studies, latent actions improved objective and subjective performance compared with shared-autonomy baselines and an assistive-arm teleoperation strategy.

    Source: §VII · p. 8

  • Takeaways & Limitations

    Latent actions supported tasks with both discrete and continuous goal spaces, including user control of goals and movement style in the continuous task.

    Source: §VII · p. 8

    Latent actions became erratic in robot configurations beyond the demonstration distribution, and users did not clearly prefer them despite improved performance.

    Source: §VII · p. 8

Abstract

from arXiv · show

Assistive robotic arms enable users with physical disabilities to perform everyday tasks without relying on a caregiver. Unfortunately, the very dexterity that makes these arms useful also makes them challenging to teleoperate: the robot has more degrees-of-freedom than the human can directly coordinate with a handheld joystick. Our insight is that we can make assistive robots easier for humans to control by leveraging latent actions. Latent actions provide a low-dimensional embedding of high-dimensional robot behavior: for example, one latent dimension might guide the assistive arm along a pouring motion. In this paper, we design a teleoperation algorithm for assistive robots that learns latent actions from task demonstrations. We formulate the controllability, consistency, and scaling properties that user-friendly latent actions should have, and evaluate how different low-dimensional embeddings capture these properties. Finally, we conduct two user studies on a robotic arm to compare our latent action approach to both state-of-the-art shared autonomy baselines and a teleoperation strategy currently used by assistive arms. Participants completed assistive eating and cooking tasks more efficiently when leveraging our latent actions, and also subjectively reported that latent actions made the task easier to perform. The video accompanying this paper can be found at: https://youtu.be/wjnhrzugBj4.

I. INTRODUCTION

Assistive arms offer dexterity but are difficult to control through low-dimensional interfaces because users must coordinate many robot degrees of freedom. The paper proposes learning low-dimensional latent actions from task demonstrations for continuous teleoperation, alongside formal properties, model comparisons, and user studies.

I. INTRODUCTION

  • Assistive arms can help users perform everyday tasks without caregiver help, but their high dexterity makes them difficult to teleoperate through low-dimensional interfaces.Users must coordinate interconnected robot degrees of freedom through an unintuitive mapping from joystick inputs to robot actions.Source: §I · p. 1

II. RELATED WORK

  • Prior shared-autonomy approaches address discrete goals, while this work proposes learned low-DoF actions for continuous control.Related work also distinguishes methods that use predefined input mappings, same-dimensional human inputs, or latent representations without a human in the loop.Source: §I · p. 1, §II · p. 2

I. INTRODUCTION

  • Learned latent actions let users control high-DoF robot behavior through low-DoF inputs, and users completed eating and cooking tasks with improved objective and subjective performance.The paper compares its approach with shared-autonomy baselines and an assistive-arm teleoperation strategy.Source: §I · p. 1
  • The paper formalizes controllability and consistency as desirable properties, and finds state-conditioned autoencoders reconstruct high-DoF actions from human-controllable low-DoF inputs.Its stated contributions also include comparing models against these properties and evaluating the approach in user studies.Source: §I · p. 1

II. RELATED WORK

  • The paper builds on autoencoder-based latent representations and compares its method with PCA, previously explored for reducing teleoperation input dimensions.Prior latent-representation work covered dynamics, trajectories, plans, policies, and reinforcement-learning skills, typically without human-in-the-loop teleoperation.Source: §II · p. 2

III. PROBLEM STATEMENT

The paper frames learning latent actions from demonstration state-action pairs as finding a lower-dimensional control space that maps to robot actions. It defines controllability, consistency, and scaling as properties relevant to whether that space is useful to users.

III. PROBLEM STATEMENT

  • The task is modeled as a discrete-time MDP with state, action, transition, sparse task-success reward, discount factor, and initial-state distribution.The reward is one when the task is solved in a state.Source: §III · p. 2
  • A latent action space maps lower-dimensional actions to robot actions and is learned from a dataset of task-demonstration state-action pairs.The latent dimension d is smaller than the original action-space dimension m.Source: §III · p. 2
  • Controllability means that some sequence of latent actions can move the robot between any pair of demonstration states.The definition concerns start and goal states drawn from the demonstration dataset.Source: §III · p. 2
  • Consistency means the same latent action has similar effects in nearby states, measured with a task-dependent metric.The formal condition bounds the difference in resulting states under that metric for sufficiently nearby starting states.Source: §III · p. 2
  • Scaling means larger latent actions should produce larger changes in the robot’s state.The stated property relates the norm of the latent action to the resulting state change.Source: §III · p. 2

IV. METHODS

The method uses autoencoders to compress demonstrated high-dimensional robot actions into low-dimensional latent actions and reconstruct them. It conditions decoding on the robot’s current state so the same user input can be interpreted in context.

IV. METHODS

  • The method seeks embeddings that reconstruct high-DoF actions while remaining controllable, consistent, and scalable.The authors argue that decoding must reason about the robot’s state to interpret latent actions accurately and intuitively.Source: §IV · p. 3

A. Models

  • An encoder maps state-action pairs into a latent representation, and a decoder reconstructs robot actions; reconstruction error penalizes differences from demonstrated actions.The method defines the encoder over state and action, and the decoder over the latent representation.Source: A. Models · p. 3
  • VAE models add a latent-space normalization penalty that trades off against reconstruction error.The penalty is the divergence between the encoder’s latent distribution and a normal distribution.Source: A. Models · p. 3
  • Conditional autoencoders decode using both the latent input and current robot state, allowing an input’s meaning to depend on what the robot is doing.The paper contrasts conditional AE and VAE models with non-state-conditioned counterparts.Source: A. Models · p. 3

B. Algorithm

The algorithm trains a low-dimensional embedding from demonstrations, aligns its latent dimensions with joystick directions, and decodes joystick inputs into robot actions. Simulations use planar arms and four tasks to assess reconstruction, controllability, consistency, and scalability.

B. Algorithm

  • The algorithm trains an embedding from demonstrations, manually aligns latent dimensions with joystick degrees of freedom, and decodes each user input into a robot action.The demonstrations may come from a caregiver or another end user.Source: B. Algorithm · p. 3, A. Setup · p. 3

A. Setup

  • The simulation setup uses one- and two-arm planar robots with five joints per arm and 10,000 state-action pairs per task.The simulated state is joint position, the action is joint velocity, and the robot advances by a discrete step.Source: A. Setup · p. 3
  • The four simulated tasks are sine-wave motion, box rotation, circular motion at different radii, and reaching a goal region.The tasks use one- or two-dimensional latent actions.Source: A. Setup · p. 3
  • The simulations compare PCA, AE, VAE, cAE, and cVAE models, averaging results across 10 trained models of each type.The stated hypotheses predict better reconstruction and user-friendly latent-space properties for state-conditioned models.Source: A. Setup · pp. 3–4
  • Action reconstruction accuracy is measured by mean-squared error on test state-action pairs drawn from the training distribution.The setup also tests controllability between test-set start and goal states and assesses consistency and scalability using fixed latent inputs across task states.Source: A. Setup · pp. 3–4

B. Sine Task

On the sine task, state-conditioned autoencoders reconstruct actions more accurately and control movement between states better than unconditioned models. The models’ latent actions also relate approximately linearly to robot behavior.

B. Sine Task

  • cAE and cVAE reduce reconstruction loss to 1.37 ± 1.2% and 3.74 ± 0.4% of PCA loss, versus 98.0±0.6% and 100±0.8% for AE and VAE.Source: B. Sine Task · p. 4
  • For movement between 1,000 randomly selected sine-wave state pairs, cAE and cVAE end 5.6% and 11.1% as far from the goal as PCA, while AE and VAE have 104% and 106% error.Source: B. Sine Task · p. 4
  • Latent actions relate approximately linearly to robot behavior across models, with R2 = 0.99 for PCA and R2 = 0.94 ± 0.04 and R2 = 0.95 ± 0.01 for cAE and cVAE.Source: B. Sine Task · p. 4

C. Rotate Task

In the two-arm rotate task, state-conditioned models substantially improve reconstruction and controllability over their unconditioned counterparts. The measured relation between latent input and end-effector orientation is approximately linear.

C. Rotate Task

  • cAE and cVAE reconstruct actions at 0.65 ± 0.05% and 0.84 ± 0.07% of PCA baseline loss, versus 28.7 ± 4.8% and 38.0 ± 5.8% for AE and VAE.Source: C. Rotate Task · p. 4
  • For box rotation, cAE and cVAE reach 5.4 ± 0.1% and 5.9 ± 0.1% of PCA end-effector error, compared with 56.8±9% for AE and 71.5±8% for VAE.Source: C. Rotate Task · p. 4
  • The measured latent-input-to-orientation relationship is approximately linear, with R2 = 0.995±0.004 and R2 = 0.996 ± 0.002 reported for cVAE.Source: C. Rotate Task · p. 4

D. Circle Task

Across the circle and reach tasks, state-conditioned models better align latent dimensions with task motion and improve goal-directed movement. The paper’s combined task results also support the stated properties of state-conditioned models.

D. Circle Task

  • The angle between latent directions is 27 ± 20◦ and 34 ± 15◦ for AE and VAE, versus 72 ± 9◦ and 74 ± 12◦ for cAE and cVAE, approaching the ideal 90◦.The desired dimensions correspond to tangential and orthogonal end-effector motion; the figure describes cVAE dimensions as moving tangent or orthogonal to the circle.Source: D. Circle Task · p. 4, Figure 4 · p. 5

E. Reach Task

  • For 100 sampled reach goals, cVAE reduces final L2 distance from 0.57±0.38 under VAE to 0.48 ± 0.5 and shortens average paths from 5.1 ± 2.8 to 3.1 ± 0.5 units.Source: E. Reach Task · p. 5, Figure 5 · p. 5
  • Together, the Sine, Rotate, Circle, and Reach results support that state-conditioned models reconstruct actions more accurately and exhibit controllability, consistency, and scalability.The latent dimensions also aligned with task degrees of freedom, and the models could embed robot trajectories.Source: E. Reach Task · p. 5

VI. USER STUDIES

The paper evaluates learned latent actions in two user studies on a 7-DoF robotic arm, using a handheld joystick. One study covers discrete eating goals and the other continuous cooking goals.

VI. USER STUDIES

  • Both user studies use a 7-DoF Fetch robotic arm controlled through a low-DoF handheld joystick.Source: §VI · p. 5
  • The eating study compares the proposed approach with state-of-the-art shared-autonomy methods for discrete goals.Source: §VI · p. 5
  • The cooking study compares learned latent actions with a teleoperation method currently used by assistive arms in continuous goal spaces.Source: §VI · p. 5

A. Discrete Goal Space: Latent Actions vs. Shared Autonomy

The eating study tested whether learned latent actions could help users reach a chosen food morsel in a discrete-goal setting. cVAE users reached their desired morsel more accurately than shared-autonomy baselines while using less time, input, and robot movement.

A. Discrete Goal Space: Latent Actions vs. Shared Autonomy

  • The task involved guiding a fork above one of three food morsels, with cVAE compared against four HARMONIC shared-autonomy conditions.High Assist was the most effective of those four conditions; participants in this study completed only the cVAE condition, while the other strategies were benchmarked in prior work.Source: A. Discrete Goal Space: Latent Actions… · pp. 5–6
  • In cVAE, joystick inputs served as latent actions, and the model was trained on HARMONIC demonstrations.The ten participants each completed five recorded cVAE trials after familiarizing themselves with the task and joystick.Source: A. Discrete Goal Space: Latent Actions… · pp. 5–6
  • 44 of 50 trials reached the desired morsel with cVAE, and versus High Assist, completion time, joystick input, and trajectory length were statistically lower.The objective-results figure summarizes the eating study; its caption also reports faster completion with less user input and end-effector motion.Source: Figure 8 · p. 6, A. Discrete Goal Space: Latent Actions… · p. 6
  • Example cVAE trajectories moved the robot’s end-effector into a region above the plate.The trajectory figure shows examples for High Assist and cVAE, with the robot starting at the black dot and moving over the plate.Source: A. Discrete Goal Space: Latent Actions… · p. 6, Figure 7 · p. 6

B. Continuous Goal Space: Latent Actions vs. End-Effector

The cooking study compared learned latent actions with direct end-effector teleoperation in a task with continuous goals and movement styles. cVAE users completed the full recipe more quickly and with less joystick input, and rated it easier and more natural, though they did not clearly prefer either strategy.

B. Continuous Goal Space: Latent Actions vs. End-Effector

  • The cooking task required handling ingredients, containers, and stirring across continuous goal regions, so shared autonomy methods for discrete goals were not suited to this setting.The comparison used End-Effector teleoperation, where joystick inputs alternately controlled end-effector position and orientation.Source: B. Continuous Goal Space: Latent… · p. 6
  • The cVAE used one 2-DoF joystick and was trained on kinesthetic demonstrations of related subtasks, with less than 7 minutes of demonstration data.End-Effector control used two 2-DoF joysticks and a button to toggle between linear and angular motion.Source: B. Continuous Goal Space: Latent… · p. 6
  • t(10) = −6.9, p < .001 for completion time and t(10) = −5.1, p < .001 for joystick input favored cVAE over End-Effector on the full cooking task.The objective-results figure covers the full task, combining the recipe’s subtasks.Source: Figure 10 · p. 7, B. Continuous Goal Space: Latent… · p. 7
  • Participants rated cVAE as requiring less effort, easier to use, and producing more natural motion than End-Effector, but reported no clear preference between strategies.The other subjective scales were not significantly different.Source: B. Continuous Goal Space: Latent… · p. 7, Figure 11 · p. 8

VII. DISCUSSION AND CONCLUSION

The authors conclude that state-conditioned autoencoders can produce accurate, controllable, consistent, and scalable latent spaces. Across user studies, latent actions supported tasks with discrete and continuous goals, but could become erratic outside the demonstration distribution.

VII. DISCUSSION AND CONCLUSION

  • State-conditioned autoencoders accurately reconstructed intended actions and produced controllable, consistent, and scalable latent spaces.The paper evaluated five models for learning latent actions.Source: §VII · p. 8
  • Across discrete- and continuous-goal studies, latent actions improved performance against shared-autonomy baselines and the assistive-arm teleoperation strategy, respectively.For cooking, the reported improvements included objective and subjective performance; for discrete goals, cVAE produced higher success than shared-autonomy baselines.Source: §VII · p. 8
  • The cooking cVAE was trained with less than 7 minutes of kinesthetic demonstrations, using a standard cVAE trained on the robot’s onboard computer.The authors attribute data efficiency partly to the simplicity of the model structure.Source: §VII · p. 8
  • Outside the training distribution, latent actions could become erratic; one participant unintentionally rotated the arm upside-down while pouring flour and could not guide it back.Although users found latent actions easier, they also enjoyed the increased freedom of End-Effector control; the authors suggest interweaving the strategies.Source: §VII · p. 8

VIII. APPENDIX: DEXTEROUS MANIPULATION

The appendix applies learned latent actions to dexterous grasps, where the robot autonomously reaches toward an object and the user controls grasp style. The demonstrations encode varied grasp styles along a continuous latent dimension.

VIII. APPENDIX: DEXTEROUS MANIPULATION

  • In three object tasks, cVAE embedded kinesthetic demonstrations with different grasp styles into a 1-DoF latent space.The objects were a water bottle, drill, and spray bottle; each task used less than 2 minutes of demonstrations.Source: §VIII · p. 9
  • The robot autonomously performed high-level reaching, while the user’s left-or-right joystick input controlled the fine-grained grasp pose.For the drill, tilting right guided the robot toward a base grasp, while tilting left guided it toward a bit grasp.Source: §VIII · pp. 9–10
  • The authors describe the grasp style as varying along a continuous spectrum, rather than the joystick selecting between discrete grasp options.They present these results as further evidence for latent-action collaboration in dexterous manipulation tasks.Source: §VIII · p. 10
Loading 1909.09674v3…