PaperlayerView exact revision on arXiv

Source-linked AI summary

Neural Task Programming: Learning to Generalize Across Hierarchical Tasks

Danfei Xu, Suraj Nair, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, Silvio Savarese

arXiv:1710.01813v2cs.AIcs.LGcs.RO

TL;DR

Complex manipulation requires policies that adapt to new objectives and compose primitives over long-term interactions. NTP recursively decomposes demonstrations into sub-task specifications and delegates them to hierarchical neural programs. It generalizes across unseen tasks with varying lengths, topologies, and semantics in three robot manipulation domains, including visual-input and adversarial evaluations.

  • Problem

    Complex task planning must generalize to new objectives while hierarchically composing primitives for long-term environment interactions.

  • Method

    NTP is a meta-learning framework that recursively decomposes task demonstrations into sub-task specifications and assigns neural programs, including executable primitive actions, to those sub-tasks.

  • Results

    NTP generalizes across unseen tasks with increasing lengths, varying topologies, and changing semantics across three robot manipulation tasks, including visual-input and adversarial settings.

  • Takeaways & Limitations

    NTP provides modular and reusable neural programs for hierarchical tasks and demonstrates one-shot, demonstration-based generalization across sequential manipulation tasks.

Abstract

from arXiv · show

In this work, we propose a novel robot learning framework called Neural Task Programming (NTP), which bridges the idea of few-shot learning from demonstration and neural program induction. NTP takes as input a task specification (e.g., video demonstration of a task) and recursively decomposes it into finer sub-task specifications. These specifications are fed to a hierarchical neural program, where bottom-level programs are callable subroutines that interact with the environment. We validate our method in three robot manipulation tasks. NTP achieves strong generalization across sequential tasks that exhibit hierarchal and compositional structures. The experimental results show that NTP learns to generalize well to- wards unseen tasks with increasing lengths, variable topologies, and changing objectives.

I. INTRODUCTION

Neural Task Programming (NTP) addresses complex manipulation by recursively decomposing task specifications into hierarchical neural programs with executable primitive actions. Across three manipulation tasks, it targets generalization to changing task lengths, topologies, and semantics through modular task representations.

  • Complex manipulation requires sequential decision making over prolonged robot-environment interactions while adapting to new objectives and initial conditions.
  • Object sorting illustrates the challenge: four object categories and four containers yield 256 category-container mappings, alongside variable instances and task orders.
  • NTP interprets task specifications and instantiates hierarchical neural programs whose bottom-level programs are executable primitive actions interacting with the environment.Task specifications can include demonstrations, videos, or language instructions, while this work uses task demonstrations.
  • Hierarchical decomposition promotes information hiding and modularization, with lower-level modules accessing only sub-task specifications relevant to their functionality.The paper connects this design to improved reusability and reduced spurious dependencies on training data.
  • NTP generalizes across task length, topology, and semantics, and is evaluated on Block Stacking, Object Sorting, and Table Clean-up in simulated and real-robot settings.The contributions also report one-shot demonstration-based generalization and end-to-end training with visual input.

II. BACKGROUND & RELATED WORK

The paper motivates NTP by combining learning from demonstrations with hierarchical neural programming to generalize across complex task structures. It addresses adaptation to new objectives and reusable composition of primitives.

  • Skill Learning and Hierarchical Skill Composition: Learning-based methods struggle to generalize across task objectives, while hierarchical policies do not guarantee modularity or reuse.The related work contrasts task-specific rewards and monolithic policies with the need for reusable sub-policies.
  • Neural Task Programming: NTP recursively decomposes overall objectives into simpler sub-tasks and delegates neural programs to execute them.The decomposition mechanism and neural programs are trained end-to-end.
  • Neural Task Programming: NTP extends one-shot task generalization beyond longer sequences to sub-task permutations and changing success conditions.These correspond to task length, topology, and semantics variations.

III. PROBLEM FORMULATION

The formulation treats task execution as interaction with an environment under task-specific success conditions. NTP receives a task specification and generates a policy intended to reach a task-completion state.

  • Task and Environment: Each task has a Boolean success function g(s,t) that indicates whether task t is completed in state s.The task space may be infinite, motivating a versatile task representation.
  • Task Specification: A task specification ψ(t) is a sequence describing the procedure and final objective of a task.The formal representation is ψ(t) = {x1,x2,...,xN}.
  • Task Specification: NTP uses object-location trajectories or raw video sequences as task specifications in the experiments.These specifications provide input when the agent may not directly observe the underlying environment state.
  • Policy Generation: Given a new specification, NTP generates a policy π(a|o;ψ(t)) that maps observations toward a state satisfying g(sT,t) = 1.The policy is represented as a neural program rather than a monolithic goal-driven network.

A. Neural Programmer-Interpreter (NPI)

NTP adapts neural program induction to robot tasks by conditioning recursive execution on task specifications. Its interpreter selects sub-programs, scopes specifications, predicts API arguments, and stops or returns through program calls.

  • NPI: NPI recursively invokes programs or stops the current program using observations, program embeddings, and arguments.Its controller maintains a program call stack and can execute primitive actions.
  • NTP Overview: NTP adds task specification encoding and interpretation to a neural program core that selects sub-programs and predicts termination.The program returns to its caller when the end-of-program probability exceeds 0.5.
  • Robot APIs: At the bottom level, NTP uses robot APIs such as move_to, grip, and release, jointly selecting functions and their arguments.These APIs connect neural program execution to robot motion and gripper control.
  • Task Specification Interpreter: For non-primitive programs, NTP predicts a contiguous sub-task specification so lower-level modules access only relevant information.The interpreter uses the environment state, current program, and current specification to determine the next specification.
  • Subsequence Selection: Convolutional temporal processing assigns scoping labels at sequence locations to select the next specification subsequence.The labels identify Start, End, Inside, or Outside positions.
  • Training: Training uses execution traces with ground-truth hierarchical decompositions and cross-entropy supervision for scoping labels.An adaptive curriculum is also adopted from NPI.

V. EXPERIMENTAL SETUP

The experiments test whether NTP generalizes across task length, topology, and semantics, supports image-based input, and transfers to real-world manipulation tasks.

  • Evaluation goals: The evaluation asks whether NTP generalizes across task length, topology, and semantics, uses image-based input without ground-truth state, and works in real-world tasks.Experiments cover Object Sorting, Block Stacking, and Table Clean-up.
  • Task variations: Object Sorting varies object counts from 1 to 10 per category, while Block Stacking varies sub-task permutations and held-out objectives.Success rates are reported for simulation, with additional failure and completion analyses for real-robot evaluation.
  • Task length evaluation: Figure 5 plots mean success rate over 100 evaluations against the number of objects in unseen Object Sorting instances.The figure evaluates whether performance is maintained as task instances become longer.
  • Baselines: The study compares NTP with Flat, Flat (GRU), NTP (no scope), and NTP (GRU) architectural variants.Flat directly predicts primitive APIs, whereas NTP uses hierarchical programs; NTP (no scope) removes the scoping constraint.

VI. EXPERIMENT 1: OBJECT SORTING

In Object Sorting, NTP is evaluated on unseen category-container assignments and retains high performance as the number of transported objects increases, while manipulation failures remain possible.

  • Simulation: NTP retains performance above 90% as Object Sorting tasks grow from 4 to 40 transported objects, outperforming flat baselines.Training uses four tasks covering all category-container pairs, while 252 combinations remain unseen for evaluation.
  • Real robot: Two of 10 randomly selected unseen Object Sorting trials failed because of manipulation failures.The failures were a grasp failure and a collision-checking failure.

VII. EXPERIMENT 2: BLOCK STACKING

Block Stacking evaluates generalization to variable sub-task orderings, novel objectives, and visual observations. NTP improves with training-task diversity and benefits from scoped hierarchical representations.

  • Task topology: NTP generalizes better to variable Block Stacking topologies as the number of training tasks increases.Topology variations randomly shuffle the order in which block towers are built while preserving the same end configuration.
  • Task semantics: NTP and its recurrent variant improve on unseen task objectives as training tasks increase, approaching seen-task performance with 1000 tasks.The held-out objectives correspond to unseen block configurations demonstrated by novel task demonstrations.
  • Scoping constraint: The performance gap between NTP and NTP (no scope) widens with task size, supporting the value of scoping for modularized and reusable semantics.NTP (no scope) feeds the entire demonstration to subprograms rather than restricting them to corresponding sub-task specifications.
  • Visual state: NTP visual models use either detected object state or end-to-end image processing, with partial video observation showing similar trends but lower performance than full state.NTPVID (Detector) uses an object detector, while NTP (E2E) is trained directly on images without low-level state.

A. Adversarial Dynamics

Under adversarial forces, the reactive NTP core is more robust than its recurrent variation in recovering from disrupted Block Stacking executions.

  • Adversarial robustness: 20% versus 46%: NTP’s success rate decreases less than NTP with a GRU core under the same adversary.An adversary applies force after each block is stacked with 25% probability, potentially knocking down towers.

B. NTP with Visual State

NTP is evaluated with video demonstrations and image-based state representations, using either end-to-end feature learning or a detector-based pipeline. The end-to-end model outperforms the detector-based alternative but remains inferior to the full-state model, while requiring substantially more training computation.

  • NTPVID (E2E) jointly learns a policy and task-relevant features from video demonstrations and single-image states without auxiliary supervision.NTPVID (Detector) instead uses a separately trained CNN as a state preprocessor.
  • NTPVID (E2E) and NTPVID (Detector) are compared with the full-state NTP model across varying numbers of tasks, using 100 demonstrations per task.The end-to-end model uses a 7-layer convolutional network on 64×64 images, while the detector predicts object positions from 224×224 images.
  • NTPVID (E2E) outperforms NTPVID (Detector) and achieves a higher success rate despite using only partial state information.
  • Both visual methods remain inferior to full-state NTP, while detector-specific representations and cascading detection errors reduce NTPVID (Detector) generalization.Detector errors have a Gaussian distribution with a standard deviation of 2 cm.
  • NTPVID (E2E) required 10 days of training on 8 Nvidia Titan X GPUs, whereas NTPVID (Detector) required 24 hours on one GPU.Because of this computational cost, NTPVID (E2E) was evaluated only on 400 and 1000 training tasks.

VIII. EXPERIMENT 3: TABLE CLEAN-UP

The Table Clean-up task tests NTP on real-world manipulation with variation in task length and topology. In simulation, performance ranges from 55% to 100%, with increasing errors attributed to collision checking rather than incorrect NTP decisions.

  • Setup: Table Clean-up requires placing up to 4 bowls and 20 forks into a bin so the resulting stack can be carried away in a tray.The setup combines variable numbers of utensils with varied bowl-stacking orders.
  • Setup: Task variation spans length, through changing utensil counts, and topology, through changing the order in which bowls are stacked.
  • Simulator: 55%-100% performance was observed in simulation for Table Clean-up, with errors increasing as the number of objects grew.The passage attributes these errors to failures in collision checking, not incorrect decisions from NTP.
  • Real Robot Evaluation: The real-robot evaluation table reports results for 20 unseen Block Stacking trials and 10 unseen sorting trials on a Sawyer robot.It distinguishes algorithmic NTP failures from physical manipulation failures such as grasping errors and collisions.
  • Adversarial Dynamics: The adversarial-dynamics evaluation reports that NTP with GRU performs markedly worse under intermittent failures.

IX. DISCUSSION & FUTURE WORK

The paper concludes that NTP learns modular, reusable neural programs for hierarchical robot tasks and generalizes across task length, topology, and semantics. Future work targets richer state representations, expanded controller APIs, and more complex real-robot tasks.

  • NTP is presented as a meta-learning framework that learns modular and reusable neural programs for hierarchical tasks.
  • Across three robot manipulation tasks, NTP generalizes to changes in task length, topology, and semantics.
  • Future Work: Future work includes improving state encoders to extract task-salient information such as object relationships.
  • Future Work: The authors also plan richer APIs, including velocity- and torque-based controllers, and extensions to more complex real-robot tasks.
Loading 1710.01813v2…