PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Object-Centric Representations for Generalizable Robot Learning

Coline Devin, Pieter Abbeel, Trevor Darrell, Sergey Levine

arXiv:1708.04225v3cs.ROcs.AIcs.CV

TL;DR

Robotic manipulation needs visual perception that generalizes beyond the data and labels available to conventional detectors or end-to-end policies. This paper uses pretrained object-centric priors and task-specific attention learned from a few trajectories; it reports generalization across instances and better generalization than end-to-end training.

  • Problem

    Robotic vision remains challenging because computer-vision benchmark gains do not necessarily translate to complex real-world tasks, and end-to-end methods depend on training-data variety to generalize.

    Source: §I · p. 1

  • Method

    The method combines pretrained object-centric priors with task-independent meta-attention and task-specific attention learned from a few trajectories.

    Source: §I · p. 2, §III · p. 3, B. Task-Specific Attention · p. 3

  • Results

    The method generalizes across object instances with few samples and achieves superior generalization to end-to-end training; in mug pouring, distractor demonstrations yielded 100% of the time versus 50% success without distractors.

    Source: §I · p. 2, C. Learning to ignore distractors · p. 6, §V · p. 7

  • Takeaways & Limitations

    Demonstrations can adjust attention’s scope, allowing users to indicate which objects to ignore or which diverse objects should be treated as relevant.

    Source: §I · p. 2, §V · p. 7

    The policy’s image-space object-coordinate representation may be insufficient for tasks requiring fine-grained configuration of articulated or deformable objects.

    Source: §V · p. 7

Abstract

from arXiv · show

Robotic manipulation in complex open-world scenarios requires both reliable physical manipulation skills and effective and generalizable perception. In this paper, we propose a method where general purpose pretrained visual models serve as an object-centric prior for the perception system of a learned policy. We devise an object-level attentional mechanism that can be used to determine relevant objects from a few trajectories or demonstrations, and then immediately incorporate those objects into a learned policy. A task-independent meta-attention locates possible objects in the scene, and a task-specific attention identifies which objects are predictive of the trajectories. The scope of the task-specific attention is easily adjusted by showing demonstrations with distractor objects or with diverse relevant objects. Our results indicate that this approach exhibits good generalization across object instances using very few samples, and can be used to learn a variety of manipulation tasks using reinforcement learning.

I. INTRODUCTION

The introduction frames robotic vision as a generalization challenge: pretrained detectors can mislabel or miss task objects, while end-to-end learning depends heavily on its training distribution. The paper combines generic object-centric priors with task-specific attention learned from a few demonstrations, which can also let users control the scope of generalization.

I. INTRODUCTION

  • Robotic perception remains challenging because benchmark gains in computer vision do not necessarily translate to complex, unstructured real-world tasks.Pretrained detectors can miss or mislabel objects outside their label space, while end-to-end methods depend on the training-data distribution for generalization.Source: §I · p. 1
  • Task learning need not use precise labels or segmentation, but semantic visual cues are needed for policies to transfer between similar object instances.The cues should consistently localize visual features that correlate with objects needed for the task.Source: §I · p. 2
  • A two-stage vision system uses a task-independent object-centric prior, then focuses it on task-relevant objects with fewer than 15 example trajectories.Demonstrations can include distractors to narrow attention or varied relevant objects to broaden category-level generalization.Source: §I · p. 2
  • The framework combines general-purpose object-centric priors with task-specific attention that can be learned from a few demonstrations or specified by a human.It aims to support generalization over objects and environments with minimal per-task data or supervision.Source: §I · p. 2
  • The approach achieves superior generalization to end-to-end training and lets users control generalization scope through the objects included in demonstrations.The paper also reports using reinforcement learning to learn vision-based manipulation skills.Source: §I · p. 2

II. RELATED WORK

Related work contrasts instance-level robotic perception and category-level computer vision, then positions this method as combining pretrained objectness with task-specific detection without per-task object-level supervision. It also contrasts this approach with end-to-end learning, whose generalization depends on the variety of scenes and objects seen during policy training.

II. RELATED WORK

  • Robotic perception often localizes instances, whereas category-level computer vision can generalize across instances but is limited by dataset labels.The level of generalization in this method can be learned and modified from objects shown in task trajectories.Source: §II · p. 2
  • Prior robotics methods combined objectness priors with specific detection, but this approach uses pretrained visual features to select task-relevant proposals without object-level supervision for each task.The authors report reduced engineering burden and generalization over appearance, lighting, and scale.Source: §II · pp. 2–3
  • Unlike end-to-end approaches that rely on varied training scenes and objects for generalization, this method uses pretrained object knowledge while training the policy on one object instance in one scene.The target is generalization to appearance, lighting, and other nuisance variables.Source: §II · p. 3

III. DEEP OBJECT CENTRIC REPRESENTATIONS

The method compresses an image into object descriptors and uses a two-level attention hierarchy to select objects for policy learning. Shared meta-attention proposes possible objects, while task-specific attention selects those relevant to the task.

III. DEEP OBJECT CENTRIC REPRESENTATIONS

  • The representation compresses an image into a vector of object descriptors from which task-relevant objects are selected.The object-centric structure is learned from prior visual data in standard computer vision datasets.Source: §III · p. 3
  • Meta-attention is shared across tasks and identifies possible objects in a scene regardless of the task.It forms the high level of the attention hierarchy.Source: §III · p. 3
  • Task-specific attention is learned per task to identify which possible objects are relevant to the task being performed.It forms the lower level of the attention hierarchy.Source: §III · p. 3

A. Meta-Attention

The method reuses task-independent object hypotheses and learns task-specific attention from trajectories to select objects predictive of movement. The attended regions then inform a policy trained with the learned attention held fixed.

A. Meta-Attention

  • A task-independent meta-attention maps each image to reusable object hypotheses with semantic identity features and image positions.The semantic features are mean-pooled convolutional features from proposal crops, pretrained on ImageNet classification.Source: A. Meta-Attention · p. 3

B. Task-Specific Attention

  • Task-specific attention selects relevant proposals by learning weights over their semantic features from trajectories.A vector w scores proposals, and multiple attention vectors can be stacked into W to attend to different objects.Source: B. Task-Specific Attention · p. 3
  • The selected convolutional features balance flexibility and generalization better than class labels alone or overly general features such as raw pixels.The paper uses upper-layer features from an image-classification network as a general-purpose visual representation, [29].Source: B. Task-Specific Attention · pp. 3–4
  • Trajectories train attention to select objects predictive of future robot configurations, using next-action or end-effector-motion prediction.The trajectories can come from kinesthetic demonstrations or reinforcement learning.Source: B. Task-Specific Attention · p. 4
  • Soft attention uses Boltzmann probabilities over proposals and a weighted sum of object locations as input to movement prediction.The attended location is concatenated with robot joint and end-effector states.Source: B. Task-Specific Attention · p. 4
  • Low-entropy regularization encourages more discrete attention, and policy learning holds attention weights fixed while using the arg-max box.The overview contrasts policy-time arg-max coordinates with the weighted coordinates used in attention training.Source: B. Task-Specific Attention · p. 4, Figure 3 · p. 4

IV. EXPERIMENTS

The experiments assess representation reliability for robotic learning and generalization across visual changes, object instances, and environments. They also examine whether demonstrations can adjust which objects receive attention.

IV. EXPERIMENTS

  • The evaluation targets both reliability for robotic learning and generalization to visual changes in the environment.The authors expect ImageNet-trained features to generalize across visual changes and hard attention to provide robustness to distractors.Source: §IV · p. 4
  • In the illustrated examples, pouring attention selects the mug, while sweeping uses separate attention vectors for the orange and dustpan.The figure identifies the mug’s position as defining the pouring trajectory and notes variable starting positions for the orange and dustpan.Source: Figure 4 · p. 5
  • Demonstrations with different objects can modify the scope of generalization and help correct attention mistakes.The authors consider demonstrations containing distractors or diverse relevant objects.Source: §IV · p. 5

A. Training details

The implementation uses pretrained object proposals and image features, then learns policies with guided policy search across object positions.

A. Training details

  • A region proposal network trained on MSCOCO supplies the task-independent meta-attention proposals.The RPN is trained on the MSCOCO dataset.Source: A. Training details · p. 5
  • Each proposal’s semantic representation is a normalized 256-dimensional feature vector from AlexNet conv5.The network is pretrained for classification, and the feature is taken for each region proposal.Source: A. Training details · p. 5
  • Guided policy search trains local controllers for different object positions, then supervised learning fits one global policy that outputs PR2 joint torques.The policy receives robot state and attended bounding-box coordinates; the authors note the representation can also be used with other reinforcement-learning algorithms.Source: A. Training details · p. 5

B. Generalizing across visual changes

The pouring policy is tested on mugs and environments not seen during training. It generalizes across mug appearances more broadly than a raw-pixel policy, using classification features that support semantic generalization.

B. Generalizing across visual changes

  • The evaluation tests whether classification-trained high-level features generalize across mug instances, with success requiring the policy to locate the mug without being given its location.It uses different mugs for training and test and compares against a raw-pixel policy with a spatial-softmax architecture [23].Source: B. Generalizing across visual changes · p. 5
  • The policy trained on one brown mug in a plain environment generalizes to mugs of various colors, including pouring into a pink mug in clutter.With all four mugs present, it chose the pink mug rather than the brown training mug; the no-vision baseline poured toward the average target position.Source: B. Generalizing across visual changes · p. 5, Figure 5 · p. 6
  • Deep classification features trained on varied objects, including mugs, support generalization to new mug appearances, unlike raw-pixel policies that may attend to non-semantic features.The raw-pixel policy performed on the training mug and another brown mug but ignored other mugs; the paper attributes its limited generalization to features without an incentive to be semantically meaningful.Source: B. Generalizing across visual changes · p. 5

C. Learning to ignore distractors

Adding a pink distractor mug to demonstrations narrows attention toward the brown target. The finetuned policy chooses the brown mug over distractors and achieves higher test success than attention trained without distractors.

C. Learning to ignore distractors

  • Six additional demonstrations with a pink mug as a distractor finetune attention to the brown mug, because the pink mug does not predict the demonstrated arm movements.This provides a way to narrow attention when generalization to other mugs is unwanted.Source: C. Learning to ignore distractors · p. 6
  • 100% success with pink or black-and-white distractors contrasts with 50% for attention trained without distractors.After finetuning, attention stays on the brown training mug; the initial attention was distracted by the other mug.Source: C. Learning to ignore distractors · p. 6
  • The evaluation includes a pink mug the policy successfully pours into and a cluttered scene with the brown training mug.Only the brown mug was seen during training.Source: Figure 6 · p. 6

D. Increasing the scope of generalization

Demonstrations containing diverse relevant objects can broaden task-specific attention beyond its initial object scope. Finetuning on citrus examples expands attention from oranges to limes and lemons.

D. Increasing the scope of generalization

  • Adding demonstrations with limes and lemons can broaden attention beyond oranges when generalizing across citrus is desired.The paper notes that an orange-attending vector may not always generalize to other citrus fruit; finetuning W with a couple of lime and lemon demonstrations is the proposed adjustment.Source: D. Increasing the scope of… · p. 6
  • Finetuning expands attention from oranges to include the lime and lemon shown in demonstrations.The initial attention selected oranges only; the resulting sweeping policy handles an apple, apricot, and purple cup as distractors, but is confused by orange and green cups.Source: D. Increasing the scope of… · p. 6

E. Learning to attend to multiple objects

The sweeping task uses two attention vectors to track an orange and dustpan whose positions vary. Ten demonstrations support a policy that succeeds in most trials and tolerates distractors, including a person holding the dustpan.

E. Learning to attend to multiple objects

  • The policy sweeps the orange into the dustpan in 77% of trials using two attention vectors learned from ten kinesthetic demonstrations.Both objects can start in different positions, and the approach angle must depend on their relative positions as the orange changes appearance while rolling.Source: E. Learning to attend to multiple… · p. 6
  • The sweeping policy is robust to distractors and works when a person holds the dustpan, while a no-image baseline succeeds at 11% of test positions.The baseline result indicates that visual perception is necessary for this task.Source: E. Learning to attend to multiple… · p. 6

V. DISCUSSION

The proposed visual representation uses pretrained object-centric priors and adjustable attention to support robust perception and generalization across object instances. Demonstrations can broaden or narrow attention, while the system remains limited by image-space coordinates and its staged reliance on auxiliary vision data.

  • After distractor demonstrations, attention locks onto the brown mug instead of the pink background mug.Source: §V · p. 7
  • The representation generalizes across mug instances from one mug, yet a handful of additional trajectories can make it instance-specific.Source: §V · p. 7
  • Region proposals provide task-independent candidate objects, while visual-feature attention selects task-relevant ones from few trajectories; invariant features support new instances.Source: §V · p. 7
  • Demonstrations can broaden attention to diverse relevant objects, and multiple attention vectors can target distinct objects in multi-object tasks.Source: §V · p. 7
  • Image-space object coordinates may be insufficient for tasks requiring fine-grained configurations of articulated or deformable objects.Source: §V · p. 7
  • The system requires pretraining on standard computer-vision datasets and currently trains region proposals, attention, and policy in stages.Source: §V · p. 8

Related papers

Loading 1708.04225v3…