PaperlayerView exact revision on arXiv

Source-linked AI summary

Grounding Language with Visual Affordances over Unstructured Data

Oier Mees, Jessica Borja-Diaz, Wolfram Burgard

arXiv:2210.01911v3cs.ROcs.AIcs.CLcs.CVcs.LG

TL;DR

Language-conditioned robot learning commonly depends on large-scale, intervention-heavy data collection, limiting scalable real-world skill acquisition. HULC++ addresses this gap with a hierarchical visuo-lingual affordance model and low-level policy trained from unstructured, offline, reset-free data, achieving strong simulated and real-world results, including long-horizon tasks with substantially less data.

  • Problem

    Learning diverse language-conditioned visuomotor skills generally requires large-scale data collection, frequent human intervention, or simplified simulated environments.

  • Method

    HULC++ combines a self-supervised language-conditioned affordance model with a 7-DoF low-level visuomotor policy trained from unstructured, offline, reset-free data.

  • Results

    HULC++ achieves an average 65.2% success rate over 25 distinct real-world manipulation tasks and executes an average of 6.4 subgoals for abstract instructions.

  • Takeaways & Limitations

    The approach supports long-horizon, multistage language-conditioned manipulation while requiring as little as 1% language annotation and an order of magnitude less data than previous approaches.

  • Takeaways & Limitations

    Real-world task sequencing uses a fixed time horizon, implicitly assuming that tasks require approximately similar numbers of timesteps.

Abstract

from arXiv · show

Recent works have shown that Large Language Models (LLMs) can be applied to ground natural language to a wide variety of robot skills. However, in practice, learning multi-task, language-conditioned robotic skills typically requires large-scale data collection and frequent human intervention to reset the environment or help correcting the current policies. In this work, we propose a novel approach to efficiently learn general-purpose language-conditioned robot skills from unstructured, offline and reset-free data in the real world by exploiting a self-supervised visuo-lingual affordance model, which requires annotating as little as 1% of the total data with language. We evaluate our method in extensive experiments both in simulated and real-world robotic tasks, achieving state-of-the-art performance on the challenging CALVIN benchmark and learning over 25 distinct visuomotor manipulation tasks with a single policy in the real world. We find that when paired with LLMs to break down abstract natural language instructions into subgoals via few-shot prompting, our method is capable of completing long-horizon, multi-tier tasks in the real world, while requiring an order of magnitude less data than previous approaches. Code and videos are available at http://hulc2.cs.uni-freiburg.de

I. INTRODUCTION

HULC++ addresses the challenge of learning diverse language-conditioned robot skills from scalable, unstructured real-world data by hierarchically combining language-conditioned affordances with visuomotor control. The approach uses offline, reset-free teleoperated data, minimal language annotation, and LLM-generated subgoals to support long-horizon manipulation.

  • Motivation: Learning diverse language-conditioned visuomotor skills typically requires large-scale data collection, frequent intervention, or simplified simulated environments.These constraints motivate scalable real-world learning for instruction following.
  • Hierarchical control: HULC++ guides the robot toward language-referred actionable regions before switching to a single 7-DoF language-conditioned visuomotor policy.The policy is trained from offline, unstructured interaction data.
  • Approach: HULC++ combines HULC’s task-agnostic 7-DoF control with VAPO’s object-centric, self-supervised visual affordance understanding.The hierarchical design separates semantic grounding from low-level spatial interaction.
  • Data efficiency: The method learns from unstructured, reset-free teleoperated play data and requires as little as 1% of the total data to receive language annotations.This collection scheme avoids relying on costly expert demonstrations and fully annotated datasets.
  • Long-horizon tasks: When paired with LLMs that decompose abstract instructions into subgoals, HULC++ supports long-horizon, multistage manipulation tasks in the real world.The paper also reports state-of-the-art performance on the CALVIN benchmark.

III. METHOD

HULC++ first predicts language-conditioned visual affordances and uses model-based planning to reach the predicted region, then applies a local language-conditioned policy for interaction. The same unstructured dataset trains both modules, with as little as 1% language annotation.

  • Affordance prediction: The method trains a language-conditioned affordance model to predict 3D locations of objects relevant to an input instruction.The affordance model uses visual observations and language to produce a pixel-wise affordance heatmap and a depth estimate.
  • Hierarchical control: Model-based planning moves the robot toward the predicted location before control switches to a local learning-based policy πfree.The switch occurs when the end-effector is sufficiently near the predicted affordance.
  • Policy switching: The policy selector uses the pixel distance between the projected end-effector and predicted affordance, comparing it with a threshold ϵ.Distances above the threshold select πmod; otherwise the learning-based policy remains active.
  • Sample efficiency: Restricting the model-free policy to regions near language-referred interactions makes learning more sample efficient because it learns local behaviors.The affordance prediction is conditioned on each new language instruction.
  • Training setup: Both the affordance module and the 7-DoF language-conditioned imitation policy learn from the same free-form, unstructured dataset with as little as 1% language annotation.The low-level agent receives RGB observations from gripper and static cameras.

A. Extracting Human Affordances from Unstructured Data

The method extracts language-grounded visual affordances from long, unsegmented teleoperated play data, using gripper-closing actions to identify task-relevant object locations. It combines sparse language annotation with projected end-effector positions and depth information to supervise affordance prediction.

  • The affordance model is trained from a long, unsegmented dataset of semantically meaningful teleoperated behaviors.Play data is described as cheap, scalable, structured by human affordances, and not random.
  • Less than 1% of the dataset is annotated with language instructions describing completed tasks.
  • A gripper-closing signal identifies an object relevant to the instructed task at the end-effector position.
  • Projected end-effector positions label previous camera frames with pixels and language instructions for affordance learning.
  • At test time, predicted pixels and depth information provide the 3D position toward which the model-based policy should move.The approach estimates depth from visual-linguistic features rather than relying only on sensor depth, helping adapt to partial occlusions.

B. Language-Conditioned Visual Affordances

The language-conditioned visual affordance model maps an image and instruction to a pixel-wise affordance heatmap and a depth estimate. It trains the visual module against projected target pixels and selects the highest-probability image location during inference.

  • The affordance model maps visual observation I and language instruction l to a heatmap A and depth estimate d.Its encoder-decoder architecture has two decoder heads conditioned on the instruction.
  • The visual module applies a softmax over heatmap pixels to form an image distribution whose values sum to one.
  • Training constructs a one-hot target map at the projected pixel corresponding to the current state input.
  • During inference, the highest-valued dense pixel prediction identifies the most likely image location for the instruction.
  • The affordance prediction uses a U-Net architecture with language conditioning applied repeatedly in decoder layers.

2) Depth Module:

The depth module derives supervision from the camera-frame depth of the projected affordance pixel and models measurement uncertainty with a Gaussian prediction. Its visuo-lingual features produce distribution parameters that are sampled at inference.

  • The target depth is obtained by transforming the affordance pixel into the camera frame and taking its z coordinate.
  • The module models depth-sensor error by fitting a Gaussian distribution N(µ, σ) through maximum likelihood.The formulation accounts for measurement errors in typical depth sensors.
  • Linear layers process encoded visuo-lingual features and output Gaussian parameters for the depth prediction.Language conditioning is introduced by concatenating the natural-language encoding to the first two multilayer-perceptron layers.
  • The predicted depth d is sampled from d ∼ N(µ, σ) during inference.

C. Low-Level Language-Conditioned Policy

The low-level component is a goal-conditioned language policy that maps visual observations and free-form instructions to actions using unstructured behavior data. Because language goals cannot directly relabel visited states, sparse after-the-fact language annotations support broad task learning.

  • The policy πθ(at | st, l) outputs actions conditioned on visual state observations and free-form language instructions.The agent does not access the environment’s true state and uses a HULC-based goal-reaching policy.
  • The policy is trained with multi-context imitation learning on a long, unstructured dataset of semantically meaningful user behaviors.
  • Visited states cannot be relabeled directly with natural-language goals because language and observation goal spaces are not equivalent.
  • A small number of randomly selected windows paired with after-the-fact instructions can train one language-conditioned visuomotor policy across varied manipulation tasks.

D. Decomposing Instructions with LLMs

The paper uses a pretrained LLM with few-shot examples to convert abstract robot instructions into executable sequences of feasible subtasks, enabling long-horizon task composition.

  • D. Decomposing Instructions with LLMs: The framework supports chaining several language-conditioned instructions into longer sequences of robot actions.
  • D. Decomposing Instructions with LLMs: The example prompt distinguishes context, input task commands, and generated outputs through gray, magenta, and highlighted formatting.
  • D. Decomposing Instructions with LLMs: Table I reports CALVIN performance and ablations across three seeded runs.
  • D. Decomposing Instructions with LLMs: LLMs decompose abstract high-level instructions into sequences of feasible subtasks rather than requiring continual low-level language commands.The approach targets commands such as tidying a workspace and turning off lights.
  • D. Decomposing Instructions with LLMs: Few-shot prompting pairs natural-language commands with robot code examples to generate executable Python code for the requested subtasks.The method leverages pretrained LLM code-writing capabilities.
  • IV. EXPERIMENTS: The resulting subgoals are evaluated as part of experiments testing whether LLMs enable generalization to new behaviors.

A. Simulation Experiments

The experiments evaluate hierarchical language-conditioned control using CALVIN data and compare it with language-conditioned baselines. The approach improves sequence performance, uses less data, and supports diverse long-horizon manipulation in the real world.

  • Evaluation Protocol: The CALVIN protocol uses 6 hours of teleoperated play data, with language annotations on only 1% of the dataset, for sequences of up to five subtasks.The benchmark evaluates natural-language task sequences using onboard sensing.
  • Baselines: The method separates high-level semantic grounding from low-level 3D spatial interaction control and compares against HULC and BC-Z.HULC evaluates hierarchical affordance-based control, while BC-Z uses only language-annotated data.
  • Results and Ablations: 2.93 average sequence length is achieved with the affordance model, compared with 2.69 for the retrained HULC agent.The comparison attributes the increase to decoupling control into a hierarchical structure.
  • Results and Ablations: Up to 50% greater sample efficiency is obtained on smaller 50% and 25% play-data splits than the baseline.Additional ablations examine how affordance and low-level policy data quantities contribute to this gain.
  • Real-World Evaluation: 65.2% average success is achieved across 25 distinct real-world manipulation tasks with HULC++.For abstract instructions decomposed by GPT-3, the method executes 6.4 subgoals on average, while baselines typically fail after 2 to 3.

V. CONCLUSION AND LIMITATIONS

The paper introduces language-conditioned affordances for learning robot skills from unstructured offline reset-free data with as little as 1% language annotation. It reports promising results but identifies fixed-horizon sequencing and code-generation issues as limitations.

  • Conclusion: The approach extracts language-conditioned affordances from diverse human teleoperated data and combines them with model-based control components.This creates a semantic prior over where interaction should occur given a natural-language instruction.
  • Conclusion: The method targets general-purpose language-conditioned skills from unstructured, offline, reset-free data.The conclusion frames efficient learning with limited language annotation as the central contribution.
  • Limitations: Real-world task sequencing uses a fixed time horizon, implicitly assuming that tasks require approximately equal numbers of timesteps.The authors identify tracking task progress and deciding when to advance as an open question.
  • Limitations: The code-generation module for translating abstract language into subgoal sequences is another stated limitation.The supplied passage introduces this limitation but does not provide its full description.

APPENDIX

Appendix ablations examine data efficiency in the affordance, policy, and depth modules. Performance remains relatively stable with reduced data, while depth errors increase only modestly when training on 25% of the dataset.

  • Data-Efficiency Ablations: Pairing affordance and policy models trained with 25% and 100% of the data produces little performance change.The ablation supports the sample efficiency of the visuo-lingual affordance model.
  • Depth Ablations: The depth prediction ablation reports mean pixel distance error and mean depth error across models trained with 25%, 50%, and 100% of the dataset.Validation pixel-distance error is plotted in Figure 5.
  • Depth Ablations: Approximately 3 pixels is the increase in validation pixel distance error when training the depth module with 25% rather than the full dataset.The comparison uses the validation split.
  • Depth Ablations: Approximately 2 cm is the increase in depth error when training with 25% rather than the full dataset.The authors interpret these results as evidence of sample efficiency for real-world robotic applications.
  • Implementation: The low-level policy is trained with distributed data parallelism using eight GPUs, an effective batch size of 512, and the Adam optimizer.Training samples windows of length 16 to 32 and pads them to length 32.

1) Low-Level Policy:

The approach combines visual preprocessing and affordance modeling with model-based and model-free policies to execute language-conditioned commands. A CALVIN rollout visualization traces this interaction from instruction and predicted affordance to intermediate and final states.

  • Real-world static-camera images are 150 × 200, while gripper-camera images are resized from 200 × 200 to 84 × 84 before augmentation and normalization.
  • Figure 6 organizes each rollout column by instruction, predicted affordance, reached state after model-based execution, and final state.
  • Input images are normalized to pixel values between −1.0 and 1.0 after preprocessing.
  • 1e−4 learning rate and batch size 32 are used to train the affordance network end-to-end with Adam on a single GPU.
  • Given a language instruction and visual observation, the visuo-lingual affordance model predicts a location that affords the instruction.
  • The model-based policy guides the robot toward the afforded region before the model-free policy completes the task.
Loading 2210.01911v3…