PaperlayerView exact revision on arXiv

Source-linked AI summary

Accurately and Efficiently Interpreting Human-Robot Instructions of Varying Granularities

Dilip Arumugam, Siddharth Karamcheti, Nakul Gopalan, Lawson L. S. Wong, Stefanie Tellex

arXiv:1704.06616v2cs.AI

TL;DR

Prior robot language-grounding methods typically assume a fixed abstraction level, while planning across large state-action spaces can be inefficient. This paper jointly grounds commands to tasks at different levels in a hierarchical planner, and reports more accurate grounding and faster response than its baselines.

  • Problem

    Prior robot language-grounding methods assume a fixed abstraction level, despite human commands ranging from abstract tasks to fine-grained actions.

    Source: §I · p. 1

  • Method

    A deep neural language model jointly maps each command to a reward function and its level in a hierarchical planning framework.

    Source: §I · pp. 1–2

  • Results

    The evaluation reports more accurate reward-function inference by neural models than statistical baselines and faster, more accurate grounding with hierarchy.

    Source: §I · p. 2

  • Takeaways & Limitations

    The Turtlebot demonstration shows commands at multiple abstraction levels can be grounded while planning subtasks within the same hierarchy.

    Source: §VII · p. 8

    Some cardinal-direction commands were confused because users disagreed about the robot’s orientation; 31.25% of north commands were classified as south and 15% of east commands as west.

    Source: §VI · p. 8

Abstract

from arXiv · show

Humans can ground natural language commands to tasks at both abstract and fine-grained levels of specificity. For instance, a human forklift operator can be instructed to perform a high-level action, like "grab a pallet" or a low-level action like "tilt back a little bit." While robots are also capable of grounding language commands to tasks, previous methods implicitly assume that all commands and tasks reside at a single, fixed level of abstraction. Additionally, methods that do not use multiple levels of abstraction encounter inefficient planning and execution times as they solve tasks at a single level of abstraction with large, intractable state-action spaces closely resembling real world complexity. In this work, by grounding commands to all the tasks or subtasks available in a hierarchical planning framework, we arrive at a model capable of interpreting language at multiple levels of specificity ranging from coarse to more granular. We show that the accuracy of the grounding procedure is improved when simultaneously inferring the degree of abstraction in language used to communicate the task. Leveraging hierarchy also improves efficiency: our proposed approach enables a robot to respond to a command within one second on 90% of our tasks, while baselines take over twenty seconds on half the tasks. Finally, we demonstrate that a real, physical robot can ground commands at multiple levels of abstraction allowing it to efficiently plan different subtasks within the same planning hierarchy.

I. INTRODUCTION

Human instructions range from abstract tasks to fine-grained actions, and prior grounding methods generally assume one abstraction level. The paper combines language grounding across a planning hierarchy with level inference, while related work motivates hierarchical planning as a way to manage large state-action spaces.

I. INTRODUCTION

  • The proposed system maps commands of varying complexity to reward functions at different abstraction levels and jointly infers the appropriate level and task.It uses a deep neural language model and a hierarchical planner to interpret and execute the grounded task.Source: §I · pp. 1–2
  • The evaluation reports that neural language models ground reward functions more accurately than statistical baselines, while hierarchy improves planning speed and grounding accuracy.The authors also demonstrate command grounding at varied abstraction levels on a Turtlebot.Source: §I · p. 2

II. RELATED WORK

  • Prior language-grounding approaches map commands to fixed action sequences or a single abstraction level, limiting their representation of varied commands.Fixed action sequences can be brittle in stochastic environments, while the cited reward-function approach uses one abstraction level.Source: §II · p. 2
  • Hierarchical planning decomposes large planning problems into subtasks, but bottom-up methods can be slow because rewards must be backed up through the hierarchy.AMDPs provide a top-down alternative with reward and transition functions for subtasks.Source: §II · p. 2

III. TECHNICAL APPROACH

The approach represents tasks with reward functions in an OO-MDP and uses lifted functions to separate task meaning from environment-specific object bindings. It jointly predicts a command’s abstraction level and lifted reward function, then grounds the function and plans at that level.

III. TECHNICAL APPROACH

  • An OO-MDP represents the robot’s states, actions, transitions, rewards, and discount horizon; propositional functions provide semantic reward functions for tasks.The evaluation uses the Cleanup World mobile-manipulator domain.Source: §III · p. 3
  • Binding language directly to specific room instances can fail when environment configurations change, so the approach lifts propositional functions to describe attributes such as room color.An environment-specific grounding module binds lifted functions to actual instances before planning.Source: §III · p. 3
  • The framework assumes a predefined hierarchy with reward functions for relevant tasks and subtasks, and assumes commands come from a single fixed abstraction level.AMDPs are selected for fast planning and solving individual subtasks without solving the entire planning problem.Source: §III · p. 3
  • For each command, the model jointly selects an abstraction level and lifted reward function, then the grounding module completes the function before hierarchical planning.Abstract commands map to higher levels and fine-grained commands to the finest level; a parallel corpus pairs commands with reward functions at particular levels.Source: §III · p. 3

IV. LANGUAGE MODELS

The section compares four language-model approaches for grounding commands, spanning a statistical translation baseline and three neural architectures.

IV. LANGUAGE MODELS

  • The comparison includes IBM Model 2, a deep neural bag-of-words model, and two recurrent neural network models.Source: §IV · p. 3

A. IBM Model 2

The IBM Model 2 baseline treats task grounding as machine translation from natural language commands to semantic task representations. It scores reward functions given commands and is trained with the standard EM algorithm.

A. IBM Model 2

  • IBM Model 2 frames grounding as machine translation from natural language commands to lifted reward-function representations.The language is the source and the semantic task representation is the target.Source: A. IBM Model 2 · p. 3
  • The model scores candidate reward functions using the prior-weighted likelihood of the command.Its objective includes Pr(l, m) · Pr(c | l, m).Source: A. IBM Model 2 · p. 3
  • The IBM2 baseline is trained with the standard EM algorithm.The grounding formulation follows MacGlashan et al..Source: A. IBM Model 2 · p. 4

B. Neural Network Language Models

The neural language models map commands to abstraction levels and reward functions using feed-forward or recurrent architectures. The Multi-NN variant uses a bag-of-words input and separate output distributions for level selection and reward functions.

B. Neural Network Language Models

  • The three neural model classes are bag-of-words Multi-NN, sequence-aware Multi-RNN with separate parameters, and sequence-aware Single-RNN with shared parameters.The Multi-NN and Multi-RNN use separate parameters for each abstraction-level output.Source: B. Neural Network Language Models · p. 4

1) Multi-NN: Multiple Output Feed-Forward Network:

  • Multi-NN takes a command as a bag-of-words vector and factorizes its prediction into abstraction-level and reward-function probabilities.It models Pr(l, m | c) as Pr(l | c) · Pr(m | l, c).Source: 1) Multi-NN: Multiple Output… · p. 4
  • Multi-NN uses a shared embedding and hidden layer, followed by output-specific hidden and read-out layers.Its level output has k categories, while each level's reward-function output has r_i categories.Source: 1) Multi-NN: Multiple Output… · p. 4
  • Multi-NN is trained by minimizing cross-entropy with backpropagation and Adam, using dropout with probability p = 0.5.The minibatch size is 16 and the learning rate is 0.001.Source: 1) Multi-NN: Multiple Output… · p. 4

2) Multi-RNN: Multiple Output Recurrent Network:

Multi-RNN processes a command as a word sequence and predicts both abstraction level and reward function. Its recurrent encoder uses a GRU, while its outputs and training objective retain the Multi-NN model's factorized structure.

2) Multi-RNN: Multiple Output Recurrent Network:

  • Multi-RNN reads commands as word sequences and predicts probabilities for abstraction levels and reward functions at each level.Its objective sums the log probabilities of the level and reward-function terms.Source: 2) Multi-RNN: Multiple Output… · p. 4
  • The recurrent encoder replaces the lower feed-forward layers and uses a gated recurrent unit (GRU).The GRU maps raw input tokens into a fixed-size state vector.Source: 2) Multi-RNN: Multiple Output… · p. 4
  • Multi-RNN minimizes the sum of cross-entropy losses for its two output terms, using the Multi-NN optimizer setup and dropout.Source: 2) Multi-RNN: Multiple Output… · p. 4

3) Single-RNN: Single Output Recurrent Network:

Single-RNN directly predicts the joint probability of an abstraction level and reward function from a command sequence, rather than explicitly selecting a level first. Sharing parameters across abstraction levels improves model efficiency and permits the possibility of information transfer between levels.

3) Single-RNN: Single Output Recurrent Network:

C. Grounding Module

The evaluation uses Cleanup World, a three-level planning hierarchy, and tests whether hierarchy improves grounding speed and accuracy. Environment-specific variables are bound through a grounding module that is a lookup table in this domain.

C. Grounding Module

  • The grounding module binds lifted reward-function templates to environment-specific variables; in Cleanup World, a lookup table maps constraints to object-ID tokens.A more complex grounding module could replace the lookup table when constraints are ambiguous, such as when multiple chairs could match “chair.”Source: C. Grounding Module · p. 5

V. EVALUATION

  • The study tests whether hierarchical structure improves language-grounding speed and accuracy using corpus-based simulation and a Turtlebot demonstration.Source: §V · p. 5

A. Mobile-Manipulation Robot Domain

  • Cleanup World models a mobile manipulator moving objects among rooms, and its combinatorially large state space simulates real-world complexity.Source: A. Mobile-Manipulation Robot Domain · p. 5
  • Cleanup World uses a three-level AMDP hierarchy, with primitive movement at L0, room-and-door regions at L1, and room-only regions at L2.Higher-level actions are subtasks for moving the robot or blocks between regions, subject to adjacency and transition constraints.Source: A. Mobile-Manipulation Robot Domain · p. 5

B. Procedure

The AMT study collected commands for tasks at three abstraction levels and evaluated both inferred abstraction level and reward-function grounding. Results show that models trained at only one level do not generalize across the full range of commands, while neural models that infer level jointly with grounding perform more robustly.

C. Robot Task Grounding

  • Figure 4 evaluates IBM2 and Single-RNN trained on one abstraction level against commands from alternate levels, illustrating the effect of language abstraction on grounding accuracy.Source: Figure 4 · p. 6
  • Figure 5 reports the models’ AMT-dataset accuracy under 10-fold cross-validation averaged over three runs.Source: Figure 5 · p. 6

B. Procedure

  • The AMT procedure collected commands for ten demonstrated tasks, constrained to fine-grained, medium, or coarse language, and assessed correct level and reward-function inference.The corpus contained 3,047 commands: 1,309 L0, 872 L1, and 866 L2; fewer than 30 unclear commands were removed.Source: B. Procedure · p. 6

C. Robot Task Grounding

  • IBM2 scores were uniformly poor, and models trained on L0 or L2 data substantially outperformed those trained on alternate levels.The authors speculate that L1's high command variance and overlapping, repetitive tokens contribute to IBM2's weakness; these models also lacked a task or behavior model.Source: C. Robot Task Grounding · p. 6
  • Single-RNN grounding accuracy is maximized when training and evaluation use the same abstraction level, showing that a single-level model does not cover the full command scope.Single-RNN also grounds more robustly than statistical machine translation.Source: C. Robot Task Grounding · p. 6
  • Single-RNN achieved near-perfect abstraction-level prediction and improved grounding over Multi-RNN, despite a slight loss relative to oracle-level results.The authors attribute the gain over Multi-RNN to parameter sharing across training samples, which allows positive information transfer between abstraction levels.Source: C. Robot Task Grounding · pp. 6–7

D. Robot Response Time

Hierarchical planning is generally faster than flat planning, especially in the doubled Cleanup World domain, where AMDP combines hierarchy with heuristics. However, hierarchy can add overhead in smaller domains and is not sufficient by itself for the larger-domain speed gains.

D. Robot Response Time

  • Response-time measurements included command processing, reward-function mapping, and MDP planning, using Single-RNN and only its 2,634 correctly grounded commands.Source: D. Robot Response Time · p. 7
  • The comparison used BASE, a flat BRTDP planner; AMDP, a hierarchical planner with a heuristic; and NH, the same hierarchical approach without that heuristic.BASE was also used at AMDP’s primitive level for comparable planning times.Source: D. Robot Response Time · p. 7
  • In the original domain, AMDP was twice as fast as BASE in over half the cases, four times as fast in a quarter, and reached 20 times speedup, but was slower in 23%.The slowdowns were attributed to overhead from instantiating multiple planning tasks; the worst absolute difference was less than 2s.Source: D. Robot Response Time · p. 7
  • AMDP took < 1s on 90% of tasks in the larger domain, while BASE took > 20s on half the tasks.AMDP was 8 times as fast as BASE in over half the cases, 100 times as fast in a quarter, and reached up to three orders of magnitude in speedup; it was slower in 10% of cases, all within < 4% of BASE time.Source: D. Robot Response Time · p. 7, Figure 6 · p. 7
  • The larger-domain results show that hierarchy alone was insufficient: NH outperformed BASE but was much less efficient than AMDP, where heuristics mitigated hierarchical-planning outliers.AMDP was slower than BASE in only 10% of larger-domain cases, each within < 4% of BASE time.Source: D. Robot Response Time · p. 7

E. Robot Demonstration

A Turtlebot demonstration applied the trained Single-RNN grounding model and AMDP hierarchy to spoken commands spanning low- to high-level tasks.

E. Robot Demonstration

  • The Turtlebot planned and executed spoken commands from “Go north” to taking a block to a room, with imperceivable delays after speech-to-text conversion.The demonstration adapted L0 actions for the robot’s continuous action space while leaving higher AMDP levels unchanged.Source: E. Robot Demonstration · p. 8

VI. DISCUSSION

The discussion traces grounding errors to ambiguity in object references, video orientation, and human-generated data. The conclusion reports strong model performance in real-world Turtlebot tests while identifying mixed-level inputs and learned hierarchies as future directions.

VI. DISCUSSION

  • At the lowest abstraction level, navigation and manipulation commands confused the model when users omitted object mentions; 83.3% of green room navigation tasks were correct, while 16.7% became manipulation.Source: §VI · p. 8
  • Video-orientation disagreement caused directional errors: 31.25% of north commands were classified as south, and 15% of east commands as west.Source: §VI · p. 8
  • Collected commands also contained typos and abstraction-level mismatches; authors left them uncorrected to avoid strong priors and suggest human validation.Source: §VI · p. 8

VII. CONCLUSION

  • The proposed models significantly outperform the previous state-of-the-art, and Turtlebot evaluation demonstrates operation in real-world environments.Source: §VII · p. 8
  • Future work would extend models to mixed-abstraction commands and learn hierarchies informed by user inputs rather than constrained by existing planner hierarchies.Source: §VII · p. 8
  • The authors envision applications in real-world settings where abstraction levels naturally occur, including surgical, manufacturing, and household robotics.Source: §VII · p. 8
Loading 1704.06616v2…