Source-linked AI summary
DemoStream: Batched Task-and-Motion-Plan Optimization for Automated Learning from Demonstration
Caelan Garrett, Neel Jawale, Fabio Ramos
TL;DR
Robot learning from demonstration requires large datasets, while existing collection methods involve substantial human effort or are task-specific, and TAMP demonstrations can be difficult to learn from. DemoStream uses batched TAMP to generate and optimize demonstrations, then evaluates the effects of action spaces, costs, and batch sizes on policy learning. In real-world evaluation, policies trained on 75 demonstrations achieved a 67% success rate across three tasks.
Problem
Robot demonstration datasets are costly to collect at scale, while prior TAMP-generated demonstrations have been difficult to learn from because of suboptimality and multi-modality.
Source: §I · p. 1Method
DemoStream uses batched TAMP sampling and computation-graph optimization to explore many solutions in parallel and generate demonstrations for imitation learning.
Source: §I · p. 1Results
75 real-world demonstrations supported visuomotor policies with a 67% success rate across Stack-Two, Banana-Bin, and Apple-Lime.
Source: §I · p. 1Takeaways & Limitations
Task-space costs and task-space motion planning yield the highest success rates across the evaluated action spaces.
Source: A. Action Space and Cost Function · p. 6
Abstract
from arXiv · showhide
Learning from demonstration is an effective strategy for endowing robots with manipulation capabilities but requires training on datasets comprised of massive amounts of demonstrations. Existing demonstration collection approaches either require constant human involvement, for example, via teleoperation or are specialized for a single task. Task and Motion Planning (TAMP) systems are able to autonomously solve and therefore generate demonstrations for general-purpose tasks; however, learning from their demonstrations has proven difficult due to multi-modality in their solutions. In this work, we build on recent work leveraging GPU acceleration for TAMP and propose DemoStream, a TAMP system that optimizes demonstration quality in order to improve behavior cloning success rates. First, we introduce batched TAMP sampling operations along with a new TAMP algorithm that correspondingly performs batched constraint satisfaction in order to explore many solutions in parallel. We conduct simulation experiments ablating different action spaces, cost functions, and batch sizes to analyze the effect of these choices on downstream policy success rates. Finally, we show that we can leverage the safety guarantees of TAMP to directly collect real-world demonstrations at an average rate of 66 demos per hour. See our website for policy rollout videos: demostream.github.io.
I. INTRODUCTION
Robot learning needs large demonstration datasets, while existing collection approaches require substantial human effort or remain task-specific. DemoStream uses batched TAMP to optimize demonstrations for learnability and reports real-world policy success across three tasks.
I. INTRODUCTION
- Prior TAMP-generated demonstrations were difficult to learn from because of suboptimality and multi-modality in stochastic planning solutions.DemoStream investigates GPU acceleration for both faster generation and demonstration optimization.Source: §I · p. 1
- 75 real-world demonstrations supported visuomotor policies with a 67% success rate.The demonstrations covered Stack-Two, Banana-Bin, and Apple-Lime.Source: §I · p. 1
- DemoStream introduces batched samplers and computation-graph optimization to explore thousands of TAMP solutions in parallel.The paper analyzes how action space, cost function, and batch size affect policy-learning success rates.Source: §I · p. 1
- The paper presents automated demonstration generation for imitation learning and evaluates long-horizon, multi-robot, and open-world simulation capabilities.It also identifies safe automated real-world demonstration collection as a contribution.Source: §I · p. 1
II. RELATED WORK
Prior demonstration-generation approaches either retain human involvement or specialize adaptation to particular tasks, while autonomous TAMP demonstrations had not been optimized for learnability. DemoStream extends batched TAMP planning to generate demonstrations across tasks and objects, with support for multiple robots.
II. RELATED WORK
- Prior adaptation methods remain task-specific and need new human demonstrations when action sequences or objects change, while earlier autonomous TAMP demonstrations were not optimized for learnability.Human-in-the-loop TAMP still includes a human on every episode.Source: §II · p. 2
- Unlike ScheduleStream's GPU focus on multi-arm collision checking, DemoStream batches conditional sampling to generate many independent values per operation.Prior GPU-accelerated TAMP work searched continuous action parameters, but earlier autonomous demonstration generators used CPU-only algorithms.Source: §II · p. 2
III. BATCHED TAMP
- DemoStream introduces batched streams and computation-graph optimization to explore thousands of TAMP solutions and study how cost functions affect demonstration and policy outcomes.The stated measures include demonstration success rate, demonstration length, and downstream policy success rate.Source: §III · p. 2
A. TAMP Preliminaries
- The TAMP preliminaries represent domains with states, predicates, functions, and actions whose conditions govern applicability and whose effects update state.The running example is a two-object packing task, with continuous robot configurations and object poses alongside discrete object names.Source: A. TAMP Preliminaries · pp. 2–3
- DemoStream supports multiple robots, extending beyond prior TAMP demonstration generators, [5].Source: A. TAMP Preliminaries · p. 2
B. Batch Streams
Batch streams extend single-input generators to process multiple constraint-satisfaction problems together. DemoStream batches placement generation, inverse kinematics, and motion operations for continuous inputs.
B. Batch Streams
- BatchStream takes a list of input tuples and returns output tuples grouped by input, enabling simultaneous solution of multiple inverse-kinematics subproblems.The example ik_qs stream takes 128 grasp-and-placement pairs as input.Source: B. Batch Streams · p. 3, Figure 2 · p. 3
- Unlike ordinary streams, which consume one input at a time, batch streams support parallel generation for continuous-valued inputs.Ordinary streams specify a predicate, its input parameters, and a generator that yields outputs satisfying the predicate.Source: B. Batch Streams · p. 3
- DemoStream batches placement generation, inverse kinematics, and motion streams because each accepts continuous-valued inputs.Source: B. Batch Streams · p. 3
C. Batch Planning
BATCH-STREAM combines lazy planning with batched stream calls to bind action parameters. Its optimization subroutine evaluates costs and returns the lowest-cost satisfying binding when one is within the maximum cost.
C. Batch Planning
- BATCH-STREAM lazily searches for a plan, retraces its parameter-generating streams, and invokes batched optimization to bind parameters before returning the plan.If no satisfying binding is found, it updates the lazy streams to remove those attempted in the stream plan.Source: C. Batch Planning · p. 4
- BATCH-OPTIMIZATION propagates bindings through stream instances by batching their inputs, joining outputs to matching inputs, and evaluating costs across bindings.Source: C. Batch Planning · p. 4
- BATCH-OPTIMIZATION returns the minimum-cost binding when bindings exist and their minimum cost does not exceed cmax; otherwise, it fails.In practice, costs are evaluated as soon as their parameters are bound, prioritizing low-cost bindings in subsequent batch streams.Source: C. Batch Planning · p. 4
- The implementation runs BATCH-OPTIMIZATION on the CPU and stream and function calls on the GPU, though moving all operations to the GPU could maximize performance at the expense of flexibility.Source: C. Batch Planning · p. 4
IV. DEMONSTRATION GENERATION
DemoStream uses TAMP to produce demonstration datasets for imitation learning, with its design motivated by studying how demonstration quality affects visuomotor policy learning.
IV. DEMONSTRATION GENERATION
- DemoStream solves TAMP problems to produce imitation-learning datasets and investigates how demonstration quality affects visuomotor policy learning.The paper describes its implementation, cost functions, motion planners, and integration with coding agents.Source: §IV · p. 4
A. TAMP Demonstrations
DemoStream generates successful observation-action demonstrations by solving TAMP plans and executing them open-loop. Its datasets span several robot action spaces, and batched simulation evaluates many solutions in parallel.
A. TAMP Demonstrations
- Demonstrations contain proprioception and camera images as observations, paired with actions in joint-space absolute, joint-space relative, or task-space relative action spaces.The task-space relative actions use an inverse-kinematics controller.Source: A. TAMP Demonstrations · pp. 4–5
- The illustrated computation graph represents a plan to pick the red block and place it on the plate, with binding counts shown after each batch stream call.Source: Figure 3 · p. 5
- For each goal, DemoStream samples an initial state, solves a TAMP plan, executes it open-loop, and retains trajectories that reach the goal.After collecting 1,000 successful demonstrations, the authors train supervised models mapping observations to actions.Source: A. TAMP Demonstrations · p. 5
- Batched simulated environments try hundreds of TAMP solutions in parallel across initial states, while finite GPU memory constrains batching multiple TAMP problems.Simulation batching leaves TAMP as the bottleneck, taking several seconds per successful demonstration.Source: A. TAMP Demonstrations · p. 5
B. Stream Implementation
DemoStream accelerates grasp, placement, and inverse-kinematics generation with batched GPU operations, then filters candidates for contact and collision constraints.
B. Stream Implementation
- The grasp sampler generates 10,000 antipodal points, samples rolls around them, and filters grasps for contact and open-gripper collision.Only grasps that succeed in trial-and-error execution are retained for policy learning.Source: B. Stream Implementation · p. 5
- DemoStream uses cuRoboV2 for batched multi-pose inverse kinematics and checks end-effector collision spheres when filtering grasp candidates.Source: B. Stream Implementation · p. 5
- The placement sampler generates 10,000 candidates above a support object and filters out placements without downward contact or that collide with the support.Source: B. Stream Implementation · p. 5
C. Cost Functions
DemoStream evaluates how TAMP cost choices shape solution and demonstration distributions, alongside alternative motion-planning costs and planners.
C. Cost Functions
- The cost function affects which grasps, placements, and configurations are selected, shaping the demonstrated-action distribution and downstream policy learning.The authors hypothesize that policies learn more easily from unimodal, concentrated action distributions.Source: C. Cost Functions · p. 5
- The ablation compares zero cost, joint-space distance, and weighted end-effector translation-and-rotation distance.Joint-space distance correlates with execution duration; end-effector task-space distance also minimizes motion of rigidly attached links such as wrist cameras.Source: C. Cost Functions · p. 5
D. Motion Planning
- The batched sampling-based motion planner combines global PRM exploration with local shortcutting, using a cost function for collision and motion-edge costs.The comparison also includes cuRoboV2’s joint-space trajectory optimizer.Source: D. Motion Planning · p. 5
- The planner variants are joint-space PRM with joint-space costs, joint-space PRM with task-space costs, and task-space PRM with task-space costs.Task-space planning samples end-effector poses from uniformly sampled configurations, checks end-effector spheres for collisions, and lazily maps plans to joint space using iterative IK.Source: D. Motion Planning · p. 6
E. Agentic Goals
For Stack-Three, task-space TAMP and motion costs with T-PRM produce the highest BC-RNN success rates across action spaces. DemoStream also describes adapting batched goal tests for novel language-specified goals and evaluates policy success across several experimental factors.
E. Agentic Goals
- For novel open-world goals, an LLM coding agent can implement batched goal-predicate tests offline for known tasks or lazily online for unseen goals.For simulation tasks, agents can adapt programmatic success tests for GPU-accelerated batched planning.Source: E. Agentic Goals · p. 6
V. EXPERIMENTS
- The experiments vary action space and cost function, demonstration generator, batch size, task variation, object complexity, and real-world constraints.Unless otherwise indicated, the setup uses IsaacLab, IK-Rel actions, 1,000 demonstrations, and 50 policy rollouts.Source: §V · p. 6
A. Action Space and Cost Function
- Task-space TAMP costs with T-PRM yield the highest BC-RNN success across action spaces; zero cost has no positive Joint-Abs or Joint-Rel success.Table I covers Stack-Three success rates across action spaces, cost functions, and motion planners; zero cost represents prior work, while joint-space costs with cuRoboV2 represent cuTAMP.Source: Table I · p. 6, A. Action Space and Cost Function · p. 6
B. MimicGen Comparison
DemoStream outperformed MimicGen on demonstration and policy success rates without task-specific annotated human demonstrations. The section also examines batch-size effects and evaluates performance across increasingly difficult TAMP and RoboLab tasks.
B. MimicGen Comparison
- Compared with IsaacLab’s default MimicGen using reference demonstrations, DemoStream had higher success (92% vs. 35%) and lower time per successful demo (6s vs. 9s), without annotated human demonstrations.Source: B. MimicGen Comparison · p. 6
C. Batch Size Scaling
- As IK batch size increased from 1 to 256, demonstration success and policy success rates increased while mean demonstration length and its dispersion decreased.Halving and quartering execution speed slightly increased policy success to 90% and 92%, respectively; coefficient of variation correlated meaningfully with policy success, unlike mean length alone.Source: C. Batch Size Scaling · pp. 6–7, Figure 4 · p. 7
D. TAMP Tasks
- DemoStream trained high-performing policies despite increased task difficulty across TAMP tasks spanning long horizons (Stack-Four), collision-aware motion (Bin-Stack) and placement (Pack-Two), varying plans (Pack-Either), and multiple robots (Multi-Arm).Source: D. TAMP Tasks · p. 7
E. RoboLab Tasks
- On RoboLab tasks, DemoStream required more attempts to stably grasp objects but still distilled effective policies.The tasks included placing objects in or near a bowl, placing mustard on raisins, and dropping mustard in a bin.Source: Figure 7 · p. 7, Table IV · p. 7
F. Real World
DemoStream collected real-world demonstrations on a Franka Panda using a DROID-style setup and open-world object geometry estimates from point clouds. Across three tasks, it collected 75 demonstrations per task at 66 per hour and trained policies with an average success rate of 67%.
F. Real World
- For open-world TAMP, object geometries were estimated generically from SAM3-segmented point clouds because TAMP could not directly read object geometry and pose.The experiment used a command-line natural-language task prompt on a DROID-style Franka Panda.Source: F. Real World · pp. 7–8
- 75 demonstrations per task were collected at 66 demonstrations per hour, and policies trained on real-world data succeeded 67% of the time on average.The policies were evaluated over 20 trials per task across stacking, banana disposal, and packing tasks.Source: F. Real World · p. 8, Table V · p. 8
- The pipeline ran without intervention except for human-performed randomized resets and success labeling, which TAMP could automate for some tasks.Policies used proprioception and 200x200 wrist and left-external camera images.Source: F. Real World · p. 8