PaperlayerView exact revision on arXiv

Source-linked AI summary

Where To Start? Transferring Simple Skills to Complex Environments

Vitalis Vosylius, Edward Johns

arXiv:2212.06111v1cs.ROcs.LG

TL;DR

Skills trained in open environments face the challenge of collision-free use in new cluttered scenes. The paper optimizes a graph-based affordance model at deployment to select a starting configuration for an acquired skill. Experiments report transfer to unseen clutter across grasping and placing in simulation and the real world.

  • Problem

    Data-driven skills are typically trained in open environments, while training across the huge range of environments needed for collision avoidance is described as impractical.

    Source: §1 · p. 2

  • Method

    A heterogeneous graph-based affordance model predicts skill success from scene point clouds and robot configuration, and deployment optimization selects a starting configuration.

    Source: §1 · p. 2, §3.2 · p. 3, §3.2.1 · p. 4

  • Results

    Experiments validate transfer to previously unseen cluttered environments for grasping and placing in simulation and the real world, and the method outperforms various baselines.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    The method can generalize a priori acquired skills to cluttered and constrained environments by selecting configurations from which they can be executed without collision.

    Source: §6 · p. 8

    The method may fail when successful execution requires contact with the environment, including long-horizon skills involving substantial whole-robot movement.

    Source: §5 · p. 8

Abstract

from arXiv · show

Robot learning provides a number of ways to teach robots simple skills, such as grasping. However, these skills are usually trained in open, clutter-free environments, and therefore would likely cause undesirable collisions in more complex, cluttered environments. In this work, we introduce an affordance model based on a graph representation of an environment, which is optimised during deployment to find suitable robot configurations to start a skill from, such that the skill can be executed without any collisions. We demonstrate that our method can generalise a priori acquired skills to previously unseen cluttered and constrained environments, in simulation and in the real world, for both a grasping and a placing task.

1 Introduction

The paper addresses how to transfer skills trained in open environments to cluttered settings without collisions. It proposes optimizing a separate affordance model at deployment and reports successful transfer to unseen clutter in simulation and the real world.

1 Introduction

  • Skills learned from data-driven methods are typically trained in open environments because training across the huge range of environments needed for collision avoidance is impractical.The resulting question is how to retain skill performance while avoiding collisions in new cluttered environments.Source: §1 · pp. 1–2
  • Reactive control adjustments may be insufficient in complex scenes, motivating a separate model that finds a collision-free starting configuration for an acquired skill.Deployment optimizes predicted affordance, plans a path to the selected configuration, and then executes the skill.Source: §1 · p. 2
  • Experiments validate transfer of a priori acquired skills to previously unseen cluttered environments across grasping and placing, in simulation and the real world.The evaluation also compares the approach with direct end-to-end training in cluttered environments and studies alternative starting-configuration methods and graph representations.Source: §1 · p. 2

2 Related Work

Prior work addresses cluttered manipulation, inference-time model optimization, and scene representation through distinct approaches. This paper emphasizes whole-robot configurations and a heterogeneous graph representation for its affordance model.

Robotic Manipulation in Cluttered Environments

  • Many cluttered-manipulation methods focus on grasping and avoid end-effector collisions without modeling the robot’s complete configuration [4] [5] [6] [7] [8]; [9] instead samples latent plans for target-driven end-effector policies.The paper considers the complete configuration and uses kinematic planning to reach an optimal starting configuration.Source: §2 · p. 2

Optimising Learnt Models at Test Time

  • Prior inference-time optimization work includes energy functions, learned representation choices, grasp-success models, and trajectory constraints [11] [12] [13] [15] [16]; this paper optimizes affordance to select a configuration for a specific skill.Source: §2 · p. 2

Representation Learning for Robot Manipulation

  • Earlier manipulation representations include semantic keypoints [17] [18] [19], dense descriptors [20] [21], and scene graphs [22] [23] [24]; this work uses heterogeneous graphs to represent underlying structure.The cited heterogeneous-graph examples also include academic, recommendation, and economic graphs [27] [28] [29].Source: §2 · p. 2

3 Method

The method combines an acquired skill with a learned affordance model that selects a promising robot configuration from an initial scene observation. A heterogeneous graph represents the target, obstacles, and robot; deployment optimizes the configuration, plans a collision-free path, and executes the skill.

3.1 Overview

  • At deployment, the method optimizes a learned affordance model for a starting configuration, plans a path to it, and executes the previously acquired skill.The skill may be acquired by different means; the paper evaluates grasping and placing skills trained with behavioural cloning in simulation.Source: §3.1 · p. 3, Figure 2 · p. 3

3.2 Affordance Model

  • For each skill, the affordance model predicts the probability of successful, collision-free execution from segmented target and obstacle point clouds and the robot’s joint configuration.The method assumes a distribution of configurations exists from which the acquired skill can succeed, and that the target object and skill are specified.Source: §3.2 · p. 3

3.2.1 Structured Graph Representation

  • The heterogeneous graph jointly represents target, obstacles, and robot; differentiable robot keypoints and relative edge features support gradients from affordance predictions to joint angles.Keypoints address the mismatch between Cartesian point clouds and joint-angle configurations; relative transformations avoid absolute positions and support task-space generalization.Source: §3.2.1 · p. 4, Figure 3 · p. 4

3.2.2 Training

  • The affordance network is trained end-to-end on positive and negative skill rollouts from procedurally generated simulated scenes, labeling collisions as failures.Rollouts use noisy depth images and distorted segmentations to better mimic real-world inputs; a binary cross-entropy loss trains the classifier.Source: §3.2.2 · pp. 4–5

3.3 Deployment

  • Deployment solves a constrained, non-convex affordance optimization with joint-limit and collision constraints, then uses RRT-Connect to reach the selected configuration before skill execution.The collision constraints use a coarse environment estimate of the table and segmented-object bounding boxes; optimization took an average of 3.2 seconds and ran once per initial observation.Source: §3.3 · p. 5

4 Experiments

The experiments test whether the three-step approach transfers skills to cluttered, constrained scenes, comparing it with end-to-end policies and alternative starting-configuration methods. Ablations and real-world trials examine the role of representation and transfer beyond simulation.

4.1 Experimental Setup

  • The evaluation covers grasping novel objects and placing objects in novel containers, using skills trained by behavioral cloning in obstacle-free simulation and six randomized scene types.Methods use the same observation space and number of training examples; simulated evaluation uses unseen target objects and checks that a collision-free solution exists.Source: §4.1 · pp. 5–6

4.2 Comparison Against Learning a Single End-to-End Policy

  • The three-step method performs well across environments with only a minor decline as clutter increases, while its gains over single-step end-to-end policies grow with complexity.The comparison uses success rates for the three-step method and single-step policies trained end-to-end in constrained environments.Source: Table 1 · p. 6, §4.2 · pp. 6–7

4.3 Different Ways of Finding Starting Configurations

  • Alternative starting-configuration methods struggle as clutter increases; joint-space baselines can start skill execution when the wrist camera cannot see the target.The tested alternatives include naive, generative, and two-stage behavior-cloning approaches.Source: Table 2 · p. 7, §4.3 · p. 7

4.4 Ablations

  • Concatenated scene-and-robot representations perform poorly, while end-effector-only input works better in Env 1 but poorly in cluttered, constrained scenes where the whole robot configuration matters.The ablation compares success rates across scene and robot representations.Source: Table 3 · p. 7, §4.4 · p. 7

4.5 Real World Evaluation

  • Trained only in simulation, the method adapted to all tested real-world clutter levels without major performance loss; the Naive baseline struggled in more complex scenes.Trials covered grasping and placing with five objects unseen during training.Source: Table 4 · p. 8, §4.5 · p. 8

5 Limitations, Failure Modes and Future Work

The method has practical constraints involving data collection, skill changes, perception, optimization, and the kinds of skills it can execute without environmental contact.

5 Limitations, Failure Modes and Future Work

  • For skills trained in reality, collecting affordance-model data may be cumbersome, and the model must be retrained whenever the skill changes.These are author-identified deployment burdens.Source: §5 · p. 8
  • Segmentation failures can affect the method, and affordance-score optimization is sensitive to initialization and can become stuck in bad local minima.Source: §5 · p. 8
  • The method may fail when a skill cannot be executed without environmental contact, including long-horizon skills requiring substantial whole-robot movement.Source: §5 · p. 8
  • Future work includes testing tasks beyond grasping and placing and extending the approach to longer-horizon tasks by chaining skills.Source: §5 · p. 8

6 Conclusions

The paper combines previously acquired manipulation skills with a deployment-optimized affordance model to find collision-free starting configurations, and reports transfer to unseen cluttered and constrained scenes. Its experiments consider grasping novel objects and placing held objects in novel containers.

6 Conclusions

  • The method outperforms various baselines, generalises to unseen cluttered and constrained scenes, and transfers from simulation to reality.The evaluation covers both grasping and placing.Source: §6 · p. 8
  • A heterogeneous graph jointly represents the target object, scene obstacles, and robot to support deployment-time selection of suitable skill-starting configurations.Starting the acquired skill from a suitable configuration is intended to enable successful completion without environmental collisions.Source: §6 · p. 8

A Behaviour Cloning Skills

  • The tested skills grasp novel objects and place held objects inside novel containers; both were acquired through behavioural cloning solely in simulation.The skills use 6D end-effector action space and segmented point-cloud observations.Source: A Behaviour Cloning Skills · p. 12

A.1 Training Data

The authors create simulated expert demonstrations with scripted policies and use object collections and obstacle-free tabletop scenes to train the grasping and placing skills.

A.1 Training Data

  • Expert demonstrations are generated in simulation using scripted policies with privileged information such as object geometry and pose.Source: §A.1 · p. 12
  • Grasping uses 40 ShapeNet object categories and ACRONYM annotated grasps, for 200 objects total.Source: §A.1 · p. 12
  • Placing uses 200 ShapeNet containers, including bowls, pans, trash bins, and baskets.Source: §A.1 · p. 12
  • Each skill has 1,000 demonstrations from close to the target in obstacle-free tabletop scenes, with noise added to depth images, camera extrinsics, and segmentations.Source: §A.1 · p. 12

A.2 Training

The behavioural-cloning policies predict relative end-effector actions and gripper commands from target point clouds, are trained with pose and binary-classification losses, and run closed-loop at deployment.

A.2 Training

  • The policies map target point clouds in the end-effector frame to relative transformations between successive SE(3) end-effector poses and binary gripper actions.Source: §A.2 · p. 12
  • Training combines Point Matching loss for translation and orientation with Binary Cross-Entropy loss for gripper-action prediction.The Point Matching loss is defined in Equation 2.Source: §A.2 · p. 12
  • The loss components are scaled with α = 1 and β = 1e−3, and positive BCE examples receive 10× weight because a*_eef is 1 only at trajectory end.Source: §A.2 · p. 12
  • At deployment, end-effector velocities are applied at 30 Hz, and a gripper prediction stops closed-loop execution to open or close the gripper.Source: §A.2 · p. 12
  • Grasp poses are adjusted and revalidated in physics simulation because the gripper differs from the one used for ACRONYM.Source: §A.2 · p. 12

A.3 Architectures

Both skill policies use the same PointNet++–LSTM architecture, with separate models trained for grasping and placing.

A.3 Architectures

  • Each policy uses two PointNet++ set-abstraction layers and a global PointNet layer to compress point-cloud observations into a 512-dimensional vector, followed by an LSTM with 128 hidden dimensions.Source: §A.3 · p. 13
  • Separate linear layers predict 3D translation and a 6D rotation representation; each model has around 2M trainable parameters.Source: §A.3 · p. 13
  • The models are trained with Adam at a constant 1e−4 learning rate; experiments with other rates and schedulers found no significant improvement.Source: §A.3 · p. 13

B.1 Architecture

The affordance model combines point-cloud coarsening with a heterogeneous graph neural network to estimate affordance scores from the scene and robot configuration.

B.1 Architecture

  • PointNet++ Set Abstraction layers coarsen the target and obstacle point clouds before graph processing.Two layers produce 256-dimensional local-neighbourhood features and reduce each point cloud to approximately 3% of its original size.Source: §B.1 · p. 13
  • The fully connected heterogeneous graph combines coarsened scene points with eight robot key points encoded by link identity and Cartesian position.Edges encode relative XYZ translations between nodes.Source: §B.1 · p. 13
  • Two GATv2 layers and global attention pooling produce a graph representation that a sigmoid maps to an affordance-score estimate.The end-to-end model has around 3.5M trainable parameters.Source: §B.1 · p. 13

B.2 Training

The paper trains an affordance model to select skill-start configurations, optimizes those configurations during deployment, and evaluates the approach against end-to-end policies and alternative starting-configuration methods.

B.2 Training

  • The affordance model is trained on positive and negative skill-rollout examples from randomized environments, including negatives where the wrist camera points away from the target.Training uses binary cross-entropy and Adam with a constant learning rate of 1e-4.Source: §B.2 · p. 13

B.3 Deployment

  • At deployment, gradient-based optimization maximizes predicted affordance under nonlinear constraints, using three collision-free, target-visible initializations to reduce local-minimum risk.The selected configuration is reached with kinematic planning; full trajectory optimization was unnecessary and could be more susceptible to local minima.Source: §B.3 · pp. 13–14

C Simulated Environments

  • Simulated environments randomize object poses, obstacle counts, and clutter structures; collision-free oracle rollouts check whether tasks remain feasible.The oracle was designed for open environments and does not account for obstacles.Source: C Simulated Environments · p. 14, Figure 6 · p. 14

D.1 Comparison Against Learning a Single End-to-End Policy

  • The end-to-end Behaviour Cloning baselines control the full trajectory using either end-effector or joint velocities, unlike the approach’s separate start-selection and skill-execution networks.The comparison uses Behaviour Cloning for both the acquired skills and the end-to-end baselines.Source: §D.1 · p. 15
  • Performance gains over single end-to-end policies grow with environment complexity; Figure 7 compares six-seed mean success rates with standard-deviation bars.Source: Figure 7 · p. 15, §D.1 · p. 15

D.2 Different Ways of Finding Starting Configurations

  • The starting-configuration comparisons include a naive collision-free, target-visible start and a CVAE that models multimodal suitable-start distributions.Source: §D.2 · pp. 15–16
  • BC2 jointly learns suitable-start distributions and how to reach them, testing the added motion-planning step; Figure 8 plots success rates across six seeds.The paper excludes baselines that adapt already acquired skills to the new environment.Source: §D.2 · pp. 15–16, Figure 8 · p. 16

D.3 Ablations

The ablation presents success-rate curves for different affordance-model architectures.

D.3 Ablations

  • Figure 9 plots success rates for different affordance-model architectures, with means over six random seeds and standard-deviation bars.Source: Figure 9 · p. 16

Related papers

Loading 2212.06111v1…