PaperlayerView exact revision on arXiv

Source-linked AI summary

Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents

Wenlong Huang, Fei Xia, Dhruv Shah, Danny Driess, Andy Zeng, Yao Lu, Pete Florence, Igor Mordatch, Sergey Levine, Karol Hausman, Brian Ichter

arXiv:2303.00855v2cs.ROcs.AIcs.CLcs.CVcs.LG

TL;DR

LLMs can lack the environmental and modality-specific grounding needed for embodied plans. Grounded Decoding guides LLM token generation with grounded functions, and the authors report complex, long-horizon task solving across three embodied domains. The paper notes that obtaining capable, general grounding functions remains an open question.

  • Problem

    LLMs lack environmental interaction, non-language observability, and knowledge of what is safe or possible for a particular embodiment, so plans generated from language alone may be ungrounded in the physical state.

    Source: §1 · p. 1, §3.2 · p. 4

  • Method

    Grounded Decoding guides LLM token sequences with embodiment-conditioned grounding functions, jointly favoring tokens likely under the LLM and consistent with the physical state.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Across three embodied domains, Grounded Decoding is reported to solve complex, long-horizon tasks and to perform strongly across a wide range of tasks while significantly outperforming prior methods in efficiency.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    The approach combines LLM long-horizon planning with grounding that guides plans toward what is possible in the concrete embodied world.

    Source: §3.2 · p. 4

    Whether a capable and general grounding function can be obtained remains an open question.

    Source: §6 · p. 10

Abstract

from arXiv · show

Recent progress in large language models (LLMs) has demonstrated the ability to learn and leverage Internet-scale knowledge through pre-training with autoregressive models. Unfortunately, applying such models to settings with embodied agents, such as robots, is challenging due to their lack of experience with the physical world, inability to parse non-language observations, and ignorance of rewards or safety constraints that robots may require. On the other hand, language-conditioned robotic policies that learn from interaction data can provide the necessary grounding that allows the agent to be correctly situated in the real world, but such policies are limited by the lack of high-level semantic understanding due to the limited breadth of the interaction data available for training them. Thus, if we want to make use of the semantic knowledge in a language model while still situating it in an embodied setting, we must construct an action sequence that is both likely according to the language model and also realizable according to grounded models of the environment. We frame this as a problem similar to probabilistic filtering: decode a sequence that both has high probability under the language model and high probability under a set of grounded model objectives. We demonstrate how such grounded models can be obtained across three simulation and real-world domains, and that the proposed decoding strategy is able to solve complex, long-horizon embodiment tasks in a robotic setting by leveraging the knowledge of both models. The project's website can be found at grounded-decoding.github.io.

1. Introduction

LLMs bring semantic knowledge to robotic planning but lack grounding in the robot and its environment. Grounded Decoding combines their token probabilities with grounded functions, and the paper reports strong performance across three embodied domains.

1. Introduction

  • LLMs lack environmental interaction, access to non-language observations, and knowledge of what is safe or possible for a particular robot.Long-horizon instructions also require sequencing steps in light of the robot’s capabilities and the current environment.Source: §1 · pp. 1–2
  • The paper frames planning as decoding a sequence that is probable under both the LLM and grounded models of its applicability to the current scene.This formulation combines semantic likelihood with constraints grounded in the planning context.Source: §1 · p. 2
  • Grounded Decoding jointly uses LLM token probabilities and token-conditioned robotic functions, with contributions including techniques for grounding affordances and safety requirements.The method guides generation at the LLM’s output; Figure 1 states that it does not require expensive LLM fine-tuning.Source: Figure 1 · p. 1, §1 · p. 2
  • Across three simulation and real-world domains, Grounded Decoding performs strongly on a wide range of tasks and significantly outperforms prior methods in efficiency.The authors report empirical evidence across the three domains.Source: §1 · p. 2

2. Related Work

Related work spans language-model decoding, embodied language models, and task-and-motion planning. Grounded Decoding distinguishes itself from SayCan through token-level open-vocabulary decoding and a broader range of grounding functions.

Guided Decoding for Language Models; Embodied and Multimodal Language Models

  • Prior work includes decoding heuristics and external classifiers for language-space utilities, while embodied language-model research includes multimodal models, action fine-tuning, and downstream models using language-model embeddings.The paper investigates frozen LLMs combined with grounded models for domain-specific grounding during decoding.Source: §2 · p. 2

Comparison to SayCan

  • Unlike SayCan [40], which selects skills from a fixed, enumerated primitive set, Grounded Decoding jointly decodes at the token level for open-vocabulary tasks.The paper notes that SayCan’s constrained set limits applicability when many skills are possible.Source: §2 · p. 3
  • Grounded Decoding explores grounding functions beyond SayCan’s RL-trained value functions and represents planning constraints as learned probabilities supplied by domain-specific models.These probabilities are incorporated into decoding.Source: §2 · p. 3

Task and Motion Planning

  • Task-and-motion planning commonly specifies constraints explicitly, whereas Grounded Decoding uses learned probabilities provided by domain-specific grounded models.Figure 2 depicts language and grounded models jointly scoring candidate tokens, with grounding objectives including affordance, preferences, and safety.Source: §2 · p. 3, Figure 2 · p. 3

3. Grounded Decoding

Grounded Decoding combines an LLM’s language-based planning with state-conditioned grounded scores to generate plans consistent with an instruction and physical state. The section describes its token-level decoding process and several ways to obtain grounding models.

3.1. LLMs and Grounding Models

  • LLMs generate text autoregressively, while grounding functions score token sequences against the robot’s state, including affordances, safety, costs, or preferences.The state may include images, proprioception, or environment state.Source: §3.1 · pp. 3–4, Grounding Functions · p. 4

3.2. Problem formulation.

  • GD combines instruction-conditioned LLM token probabilities with state-conditioned grounding scores so long-horizon language plans can be guided toward physical feasibility.The grounding function need not reason over the full instruction’s long horizon.Source: §3.2 · p. 4

3.3. Algorithm – Grounded Decoding.

  • Greedy GD selects tokens using both models, stops at a terminal token, then sends the partial command to a language-conditioned policy; the derivation assumes independence conditions and a uniform response prior.The grounding function must score partial commands; the grounded scoring function can also support beam search or top-k sampling.Source: §3.3 · pp. 4–5

3.4. Techniques for Obtaining Grounding

  • Grounding can use token-conditioned Q-functions learned through reinforcement or supervised learning, multimodal models when suitable, or expert-designed scores for hard constraints or scarce data.For affordance grounding, partial commands can remain plausible until an object token makes feasibility specific.Source: §3.4 · p. 5

3.5. Comparisons to Prompt-based Methods

  • Prompting scene information may not capture changing feasibility or robot capabilities; GD instead injects continuous probabilities, while multimodal models may lack embodiment conditioning.Figure 3 presents the shared GD formulation across tabletop, Minigrid, and kitchen domains, each with different prompts, grounded models, and primitives.Source: §3.5 · p. 5, Figure 3 · p. 6

4. Experiments

The experiments evaluate Grounded Decoding in tabletop manipulation, 2D mazes, and a physical kitchen. Across these settings, results highlight the value of grounding language-model plans in affordances and task context.

4.1. Long-Horizon Tabletop Manipulation

  • Tabletop evaluation uses 20 seen and unseen tasks spanning letters, blocks and bowls, and box packing, comparing GD search variants with no-grounding and CLIPort baselines.Box-packing safety and preference requirements are enforced only during evaluation.Source: §4.1 · pp. 6–7
  • GD decodes free-form step commands from language and top-down images, then executes them with CLIPort; affordance, safety, and preference scores guide decoding.Affordance scores use CLIPort's pick-and-place predictions; rule-based functions encode safety and preferences.Source: §4.1 · pp. 6–7
  • On tabletop tasks, Grounded Decoding achieves the best results, while beam search further improves performance over greedy search.Language-model planners generalize better to unseen tasks than supervised methods but fall short without grounding.Source: §4.1 · p. 7, Table 1 · p. 7

4.2. 2D Maze

  • Minigrid tasks range from fully described 10–30-step instructions to ambiguous tasks over 100 steps; a PPO-trained goal-conditioned value function scores partially decoded skills.The value function evaluates skill feasibility given the current observation and decoded skill.Source: §4.2 · pp. 7–8
  • In 2D mazes, grounded affordances inform language-model plans for successful long-horizon reasoning, and beam search modestly improves performance, particularly on long-horizon tasks.The affordance function is a goal-conditioned value function.Source: §4.2 · p. 8

4.3. Mobile Manipulation in a Physical Kitchen

  • Kitchen evaluation separates instructions that name the object from abstract ones; visually grounded chain-of-thought brackets when grounded decoding is active before SayCan scores plans.The method uses an open-vocabulary object detector and constrains its influence because it is not trained on robot-domain data.Source: §4.3 · pp. 8–9
  • In the physical kitchen, Grounded Decoding recovers similar performance on Unambiguous tasks and gains 25% in planning performance on Ambiguous tasks.The authors report that visually grounded chain-of-thought can disambiguate abstract tasks.Source: §4.3 · p. 9, Table 3 · p. 9

5. Analysis

Grounded Decoding (GD) is two orders of magnitude more efficient than SayCan with comparable performance. Analysis also finds that GD reduces planning failures and that grounded affordances identify feasible actions that language alone does not reliably group.

5.1. Comparison to SayCan

  • GD is two orders of magnitude more efficient than SayCan, with comparable performance.SayCan scores pre-specified options, while GD considers possible language tokens during autoregressive decoding, independent of action-space size.Source: §5.1 · p. 9, Table 4 · p. 9

5.2. Breakdown of Failure Reasons

  • GD significantly reduces planning failures by incorporating grounded scene information, while beam search outperforms greedy search despite sharing the affordance function.All planning-based methods share the same low-level policy; CLIPort baselines have policy failures only.Source: §5.2 · p. 9

5.3. Grounded Action Manifold

  • Grounded models identify achievable tabletop skills, since language embeddings alone do not perfectly group actionable skills.The analysis enumerates meaningful instructions, scores them for four scenes, and groups them using t-SNE and BERT embeddings; the full language space is much larger than this manual enumeration.Source: §5.3 · p. 10, Figure 7 · p. 10

6. Conclusions, Limitations, & Future

Grounded Decoding (GD) guides language-model decoding with grounding functions that represent token probabilities for an embodiment, and it solves complex, long-horizon tasks across three embodied domains. The authors identify grounding-function generality, prompt steering, and token-by-token fusion as limitations or open questions.

Conclusions

  • GD guides language-model decoding with grounding functions, selecting tokens probable under both the LLM and grounded models.The approach is framed as probabilistic filtering and demonstrated on three embodied domains with complex, long-horizon tasks.Source: §6 · p. 10

Limitations

  • Whether a capable, general grounding function can be obtained remains an open question despite techniques demonstrated across different domains.The authors suggest large-scale robotics models may remove this bottleneck.Source: §6 · p. 10
  • Prompt engineering is often required to steer LLMs toward the desired action space, such as likely verbs and present objects.Source: §6 · p. 10

Limitations & Future

  • Fusing grounding information after each token may be limiting; future work could integrate it more elegantly during decoding.The authors also note joint decoding may be limiting compared with one model capable of both grounding and language reasoning.Source: §6 · p. 10

A.1. Grounded Decoding Implementation Details

The appendix describes three Grounded Decoding implementations across experimental domains, with domain-specific choices of language and grounded models. The implementations use affordance, safety, preference, and open-vocabulary detection functions.

A.1. Grounded Decoding Implementation Details

  • Three Grounded Decoding implementations apply the method to long-horizon planning and behavior synthesis, using different language and grounded models across domains.The comparison of implementations is presented in Table 5.Source: §A.1 · p. 16, Table 5 · p. 16
  • The grounded-model components include Affordance Functions, Safety Functions, Preference Functions, and Open-Vocabulary Object Detectors.Source: §A.1 · p. 16

A.2.1. Tasks

The tabletop tasks comprise 20 language-instruction templates across three categories and 57 objects. Their setup randomizes object positions and uses task-specific binary rewards and scripted demonstrations for the CLIPort baseline.

A.2.1. Tasks

  • The 20 instruction templates span Letters, Blocks&Bowls, and Box Packing, with 57 objects shared across categories.Letters tasks rearrange letter objects to satisfy orders specified by the instructions.Source: §A.2.1 · p. 16
  • Episodes randomize object positions and, except in Letters, include one to three randomly sampled distractors; objects start at least 15 cm apart.Source: §A.2.1 · p. 16
  • Each task has a binary reward based on ground-truth object state, and scripted task policies provide demonstrations for the CLIPort baseline.Source: §A.2.1 · p. 16
  • Instruction attributes are randomized for some tasks using specified word and corner-or-side options.Options include words such as “left” and “right,” and locations such as “top left corner.”Source: §A.2.1 · p. 16

A.2.2. Low-level Primitives

CLIPort serves as the low-level primitive invoked by the language-model planner. It is trained on demonstrations of pick-and-place tasks, with substring augmentation intended to help it handle partial commands.

A.2.2. Low-level Primitives

  • CLIPort is the low-level primitive the language-model planner can invoke.Its per-pixel affordance predictions can also be repurposed as grounded models for long-horizon planning.Source: §A.2.2 · p. 16
  • The single primitive policy is trained on 50,000 demonstrations across 10 tasks, each pairing an instruction and RGB-D scene with expert pick and place coordinates.Expert actions come from simulator ground-truth object poses.Source: §A.2.2 · p. 16
  • Substring augmentation helps the policy perform on partial commands, such as “pick up [x]” from a full pick-and-place instruction.The passage reports this augmentation helps performance on partial commands.Source: §A.2.2 · p. 16

A.2.3. Language Model

The appendix describes experimental protocols and grounding implementations across simulated tabletop, MiniGrid, and real-world kitchen settings. It also reports that MiniGrid prompts generally worked on seen and unseen tasks, while detailing the baselines and policies used in those domains.

A.2.4. Grounding Functions

  • The tabletop affordance function scores the strongest combined pick-and-place pixel logits; separate grounding functions encode safety and user preferences.The safety function penalizes instructions involving knives or red boxes and is used for three Box Packing tasks; the preference function favors two selected objects with weights 0.5 versus 0.1 and is used for two tasks.Source: §A.2.4 · p. 17

A.2.3. Language Model / A.2.5. CLIPort Baseline

  • The InstructGPT setup accompanies CLIPort baselines trained either on decomposed single-step commands or high-level instructions for long-horizon evaluation.Both CLIPort variants use 50,000 demonstrations; the comparison tests whether language-conditioned policies can handle long-horizon tasks and generalize to new instructions.Source: §A.2.3 · p. 16, §A.2.5 · p. 17

A.2.6. Full Experimental Results in Simulated Tabletop Domain

  • Table 6 reports average success rates across 20 rollouts for seen and unseen simulated tabletop tasks, with Box Packing constraints enforced only at evaluation.Its caption identifies the table’s scope but does not provide the results values.Source: Table 6 · p. 18, §A.2.6 · p. 18

A.3.1. Environment Setup / A.3.2. Tasks

  • MiniGrid uses full-grid observations instead of the default 7 × 7 egocentric window, removing partial observability; tasks span easy, medium, and hard categories.The categories range from short, fully described instructions to ambiguous tasks exceeding 100 steps; Table 7 lists example instructions.Source: §A.3.1 · p. 19, §A.3.2 · p. 19, Table 7 · p. 19

A.3.3. Language Model / A.3.4. Low-level Primitives / A.3.5. Grounded Affordance Function

  • MiniGrid prompts generally sufficed for seen and unseen tasks, with no observed improvement from using more than three examples.The authors hypothesize that shared low-level primitives explain the limited benefit of additional examples. A multi-task policy learns short skills, uses substring masking for partial strings, and supplies value estimates for a visually grounded affordance function.Source: §A.3.3 · p. 19, §A.3.4 · pp. 19–20, §A.3.5 · p. 20

A.4.1. Tasks / A.4.2. Language Model

  • For SayCan experiments, PaLM plans after InstructGPT generates grounded chain of thought, and prompts use square brackets to mark grounded decoding.Tables 8 and 9 list unambiguous and ambiguous SayCan instructions, respectively.Source: §A.4.2 · p. 21, Table 8 · p. 21, Table 9 · p. 21

A.4.3. Low-level Primitives / A.4.4. Open-Vocabulary Detector Grounding Function

  • Real-world kitchen manipulation combines learned picking with scripted navigation and uses OWL-ViT’s maximum detection score as a grounding function.Picking policies were trained on demonstrations and autonomous successes; placing follows precomputed motions after navigation. Figure 9 presents additional detector-grounding examples.Source: §A.4.3 · p. 21, §A.4.4 · p. 22, Figure 9 · p. 22

A.5. Prompts

  • The appendix prompts illustrate tabletop packing, MiniGrid navigation, and kitchen requests such as finding snacks or moving fruit to the trash.They include both stepwise task plans and natural-language requests for the robot.Source: §A.5 · pp. 23, 25, 26
Loading 2303.00855v2…