Source-linked AI summary
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
Wenlong Huang, Pieter Abbeel, Deepak Pathak, Igor Mordatch
TL;DR
The paper asks whether pretrained LLMs can use stored world knowledge to plan actionable behavior in interactive environments without further training. It combines demonstration-based prompting with semantic action translation and autoregressive correction, raising executability from 18% to 79% while revealing a correctness trade-off. The results support extracting actionable knowledge from large language models, but within a mid-level, household setting and with important evaluation and expressivity limitations.
Problem
The paper asks whether LLMs can expand high-level task knowledge into executable actions for embodied environments without training on environment-specific demonstrations.
Method
The method uses prompted planning, semantic translation to admissible actions, autoregressive correction, and dynamically selected similar demonstrations while keeping models frozen.
Results
Executability improves from 18% to 79% without invasive model modifications, although human evaluation finds a drop in correctness.
Takeaways & Limitations
The findings provide a promising sign that pretrained LLMs can yield actionable knowledge for common-sense planning in embodied environments.
Takeaways & Limitations
The approach assumes a low-level controller, is evaluated in VirtualHome’s household setting, and suffers correctness loss partly because the environment cannot express every necessary action.
Abstract
from arXiv · showhide
Can world knowledge learned by large language models (LLMs) be used to act in interactive environments? In this paper, we investigate the possibility of grounding high-level tasks, expressed in natural language (e.g. "make breakfast"), to a chosen set of actionable steps (e.g. "open fridge"). While prior work focused on learning from explicit step-by-step examples of how to act, we surprisingly find that if pre-trained LMs are large enough and prompted appropriately, they can effectively decompose high-level tasks into mid-level plans without any further training. However, the plans produced naively by LLMs often cannot map precisely to admissible actions. We propose a procedure that conditions on existing demonstrations and semantically translates the plans to admissible actions. Our evaluation in the recent VirtualHome environment shows that the resulting method substantially improves executability over the LLM baseline. The conducted human evaluation reveals a trade-off between executability and correctness but shows a promising sign towards extracting actionable knowledge from language models. Website at https://huangwl18.github.io/language-planner
1 Introduction
The paper asks whether pretrained LLMs can turn high-level goals into actionable plans without additional training. It finds that large models produce plausible plans, then improves their executability through inference-time translation and prompting, while correctness declines somewhat.
- The study tests whether LLMs already contain enough world knowledge to plan actions for embodied environments without additional training.
- Large GPT-3 and Codex models generate plausible action plans from a single task-and-plan example, but this effect appears only in larger models.
- Naive plans are often non-executable because generated phrases may not match admissible actions or may be linguistically ambiguous.
- The proposed tools translate outputs to admissible actions, condition later generation on corrected actions, and select similar demonstrations for prompting.
- Executability rises from 18% to 79% without modifying model parameters or using extra gradient or internal information, but human-judged correctness drops.
- The paper evaluates multiple techniques and models to characterize the trade-off between executability and semantic correctness.
2 Evaluation Framework
The evaluation uses VirtualHome programs and assesses generated plans along executability and correctness. Executability is mechanically defined by syntax and environment constraints, whereas correctness relies mainly on human judgments because open-ended tasks lack gold-standard evaluation.
- 2.1 Evaluated Environment: VirtualHome: VirtualHome represents activities as programs composed of textual action steps using 42 atomic actions and object arguments.
- 2.1 Evaluated Environment: VirtualHome: The evaluation uses 2,821 ActivityPrograms entries containing task names, natural-language instructions, and executable programs.
- 2.1 Evaluated Environment: VirtualHome: The planning procedure selects a similar demonstration, samples candidate action phrases, ranks admissible actions by semantic similarity plus language-model probability, and appends the highest-scoring action.
- 2.2 Metrics: Executability measures valid parsing and satisfaction of environmental preconditions and postconditions, such as opening a fridge before grabbing milk.
- 2.2 Metrics: Correctness is primarily assessed by human evaluation because natural-language task specifications are ambiguous and multimodal, making gold-standard measurement impractical.
- 2.2 Metrics: The normalized longest-common-subsequence metric is used as a proxy because it can imperfectly measure correctness and replace unscalable human evaluation.
3 Method
The method queries pretrained language models for high-level task plans, then uses inference-time translation, trajectory correction, and task-example selection to improve executability in VirtualHome.
- 3.1 Querying LLMs for Action Plans: Pretrained language models generate action plans by completing a prompt containing one example task and its annotated plan.The approach uses autoregressive sampling with temperature and nucleus sampling; the resulting baseline is called Vanilla <LM>.
- 3.2 Admissible Action Parsing by Semantic Translation: Free-form plans often fail because their phrases do not map unambiguously to the environment’s admissible actions.Failures include mismatched formats, unrecognized action or object names, and lexical ambiguity.
- 3.2 Admissible Action Parsing by Semantic Translation: Semantic translation maps each generated action phrase to the most similar admissible environment action using embedding cosine similarity.The method embeds both the predicted phrase and candidate environment actions, enabling exhaustive enumeration in the discrete VirtualHome action space.
- 3.3 Trajectory Correction: Trajectory correction interleaves plan generation and action translation so subsequent generation is conditioned on admissible actions.The Translation LM can also detect out-of-distribution actions and terminate generation early using a similarity threshold.
- 3.4 Dynamic Example Selection for Improved Knowledge Extraction: Dynamic example selection provides weak supervision by choosing the demonstration task most semantically similar to the query task.The selected task and example plan replace the fixed prompt example, adapting the conditioning context to the query.
- 3.5 Final Method: The final method, Translated <LM>, combines the proposed inference-time improvements without modifying model parameters.The method uses a separate Translation LM from the GPT-style Planning LM.
4 Results
Large language models can generate plausible action plans without additional training, but naive plans are often difficult to execute. Translating plans toward admissible actions improves executability while introducing correctness trade-offs and environment-related limitations.
- Action-plan generation: Human annotators found that sufficiently large LLMs could produce highly realistic plans whose correctness sometimes surpassed human-written plans.Correctness was averaged across tasks and annotators.
- Executability: Naive LLM plans were generally not executable across all 7 VirtualHome household scenes, while smaller models’ apparent executability often came from repeating the prompt example.Smaller models had lower LCS than larger models, indicating this failure mode.
- Action translation: The proposed procedure translates generated phrases to admissible actions and conditions later generation on corrected actions to keep plans anchored to the environment.It uses a bank of allowed actions and sentence-embedding similarity for translation.
- Trade-offs and limitations: Translated programs were judged less correct than vanilla plans because compounded instructions were mapped poorly and some programs terminated too early.Some necessary actions or objects were not implemented in the environment, limiting achievable completeness.
5 Analysis and Discussions
The analysis evaluates ablations, grounding quality, translation models, step-by-step prompting, and program length. Action translation is central to executability, while executable plans can remain incorrect and performance varies with prompting and model size.
- 5.1 Ablation of design decisions: Removing any proposed component decreases executability and LCS, with action translation causing the largest executability drop.Removing trajectory correction slightly improves LCS for Translated GPT-3 but considerably reduces executability.
- 5.2 Are the generated action plans grounded in the environment?: 65.91% of human-written plans are both executable and correct, while action translation improves grounding but remains below human-level performance.Correctness requires at least 70% of human annotators to judge a plan correct.
- 5.3 Effect of Different Translation LMs: Sentence-BERT and Sentence-RoBERTa variants show no significant executability or LCS differences, whereas averaged GloVe embeddings significantly reduce performance.The authors suggest reasonably trained language models suffice for single-step phrase translation.
- 5.4 Can LLMs generate actionable programs by following step-by-step instructions?: Step-by-step instructions are included in prompts to test whether LLMs can translate high-level tasks into executable programs without additional training.This setting reduces ambiguity about how to perform tasks when multiple solutions are possible.
- 5.5 Program length: Smaller LMs tend to generate shorter programs, but shorter plans can be more executable while also becoming incomplete.Average program length is measured across 88 evaluated tasks to test whether improvements simply reflect shorter outputs.
6 Related Works
Related work grounds language in embodied environments using rules, lexical analysis, or domain-specific fine-tuning. This paper instead probes actionable knowledge already present in pretrained LLMs without additional training on free-form human activities.
- Prior embodied-language methods often use formal logic or lexical analysis, requiring hand-designed rules that scale inadequately to complex tasks and environments.
- Many prior studies use smaller LMs fine-tuned with considerable domain-specific data rather than full pretrained LLMs.
- Existing work has rarely evaluated pretrained LLMs in embodiment settings that expose actionable knowledge learned from large unstructured corpora.
- This paper differs by using pretrained LLM knowledge without additional training and evaluating free-form human activity tasks.
7 Conclusion, Limitations & Future Work
The paper investigates actionable knowledge in pretrained LLMs and develops techniques for commonsense grounding in complex human-activity planning. Its scope is limited by correctness loss, mid-level grounding assumptions, absent environment feedback, and indirect evaluation.
- Conclusion: The paper extracts actionable knowledge from pretrained LLMs without additional training to plan actions for complex human activities.
- Limitations: The approach improves executability but causes a considerable correctness drop, partly because VirtualHome may lack actions needed to complete some tasks.
- Limitations: The study evaluates high-level-to-mid-level grounding and assumes a low-level controller can execute actions such as “grab cup.”
- Limitations: The models do not use observation context or feedback and assume plans refer to only one instance of each object class.
- Limitations: Plan quality is assessed using separate executability and human-judged correctness metrics because semantic correctness is difficult to evaluate computationally for open-ended multimodal tasks.
- Future Work: The authors identify these shortcomings as directions for future work on pretrained LMs, goal-driven decision-making, and embodied grounding.
A.1 Hyperparameter Search
The hyperparameter search varies sampling, termination, sample-count, translation, and repetition settings, while comparing several randomly chosen fixed demonstrations.
- The search varies epsilon, temperature, and k, with values {0, 0.4, 0.8}, {0.1, 0.3, 0.6}, and {1, 10}, respectively.
- The beta coefficient controls the trade-off between semantic and translation correctness in action translation.
- OpenAI and Hugging Face implementations search frequency, presence, and repetition penalties when applicable.
- Methods using a fixed example search over three randomly chosen examples.
A.2 Details of Human Evaluations
Human evaluations compare generated plans with expert references across all 88 high-level tasks, using randomized, method-blinded questionnaires. The interface presents the plans for annotators to judge without revealing their generating methods.
- Evaluators generated plans for all 88 high-level tasks and included VirtualHome expert plans as references.
- Questionnaires randomly shuffled action plans and concealed the corresponding methods from annotators.
- The human evaluation was conducted through a Google Forms questionnaire interface on Amazon Mechanical Turk.
A.3 All Evaluated Tasks
The evaluation covers 88 high-level activities from the ActivityPrograms dataset, spanning household routines, food preparation, organization, entertainment, and other everyday tasks. Some task names contain misspellings.
- The evaluated tasks come from Puig et al.’s ActivityPrograms dataset and include 88 high-level activities.
- Some task names in the dataset contain misspellings.
- The task set also covers household activities including hanging pictures, ironing shirts, making beds, organizing closets, and putting away groceries.
- Tasks include routines such as breakfast, brushing teeth, changing clothes, doing homework, and eating snacks or cereal.
- Additional activities include reading oneself to sleep, scrubbing living-room tile floors weekly, turning on a TV with a remote, and emptying pocket items into a bowl.
A.4 Natural Language Templates for All Atomic Actions
The appendix maps VirtualHome’s structured atomic actions into natural-language templates so language models can generate and translate plans in text. Sample plans illustrate both plausible action sequences and omissions or redundant steps.
- A.4 Natural Language Templates for All Atomic Actions: VirtualHome requires specially formatted action steps, so the method defines a natural-language template for each atomic action.Language models see converted natural-language actions during autoregressive generation and action translation before plans are converted back to environment steps.
- A.4 Natural Language Templates for All Atomic Actions: The templates verbalize atomic operations such as close, cut, drink, eat, find, grab, look at, move, open, and pour.
- A.5 Random Samples of Action Plans: One sample plan for changing sheets includes walking between rooms, washing the sheets, placing them on the bed, and sitting on a couch.
- A.5 Random Samples of Action Plans: Other changing-sheets plans are shorter, focusing on finding and grabbing a pillow or finding and grabbing sheets and a pillow.
- A.5 Random Samples of Action Plans: Two eat-cheese plans retrieve cheese from a refrigerator or freezer before eating it.
- A.5 Random Samples of Action Plans: A third eat-cheese plan instead ends after placing unspecified food on a plate following several table, chair, and food actions.
- A.5 Random Samples of Action Plans: One take-jacket-off plan includes finding and grabbing a jacket, sitting on a chair, and removing it.
- A.5 Random Samples of Action Plans: Other jacket plans contain repeated jacket-finding steps, stop after finding the jacket, or add opening and closing a dresser.