Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning

Lin Guan; Karthik Valmeekam; Sarath Sreedharan; Subbarao Kambhampati

Source-linked AI summary

Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning

Lin Guan, Karthik Valmeekam, Sarath Sreedharan, Subbarao Kambhampati

arXiv:2305.14909v2May 24, 2023cs.AI

TL;DR

Direct LLM planning often produces plans with limited correctness and executability. The paper constructs and corrects explicit PDDL world models for use with external planners, and reports that this pairing significantly outperforms existing methods across the evaluated IPC and household domains.

Problem
LLM-generated plans often have limited correctness and executability, leaving direct LLM planning an unreliable approach to planning tasks.
Source: §1 · p. 2
Method
The framework uses LLMs to construct PDDL action models, incorporates natural-language corrective feedback, and uses the resulting models with external domain-independent planners.
Source: §1 · p. 2, §4 · p. 4
Results
Pairing LLMs with an external planner significantly outperforms existing methods across two IPC domains and a household-robot domain.
Source: §6 · p. 10
Takeaways & Limitations
The evaluated domains show that explicit PDDL models can support reliable planning in multiple ways, including through external planners.
Source: §6 · p. 10
GPT-4 can misunderstand causal relationships between actions, particularly for tasks requiring spatial reasoning, and may sometimes produce contradictory effects.
Source: §5.1 · p. 8

Abstract

from arXiv · show

There is a growing interest in applying pre-trained large language models (LLMs) to planning problems. However, methods that use LLMs directly as planners are currently impractical due to several factors, including limited correctness of plans, strong reliance on feedback from interactions with simulators or even the actual environment, and the inefficiency in utilizing human feedback. In this work, we introduce a novel alternative paradigm that constructs an explicit world (domain) model in planning domain definition language (PDDL) and then uses it to plan with sound domain-independent planners. To address the fact that LLMs may not generate a fully functional PDDL model initially, we employ LLMs as an interface between PDDL and sources of corrective feedback, such as PDDL validators and humans. For users who lack a background in PDDL, we show that LLMs can translate PDDL into natural language and effectively encode corrective feedback back to the underlying domain model. Our framework not only enjoys the correctness guarantee offered by the external planners but also reduces human involvement by allowing users to correct domain models at the beginning, rather than inspecting and correcting (through interactive prompting) every generated plan as in previous work. On two IPC domains and a Household domain that is more complicated than commonly used benchmarks such as ALFWorld, we demonstrate that GPT-4 can be leveraged to produce high-quality PDDL models for over 40 actions, and the corrected PDDL models are then used to successfully solve 48 challenging planning tasks. Resources, including the source code, are released at: https://guansuns.github.io/pages/llm-dm.

1 Introduction

The paper proposes extracting PDDL world models from LLMs and using external planners or model validation rather than relying on LLMs alone to plan. LLMs also mediate natural-language feedback into model corrections, enabling users to refine the model without needing PDDL expertise.

1 Introduction

LLM-generated plans can be incorrect or inexecutible, while online feedback may be costly without a faithful simulator and repeated errors can frustrate users.Prior approaches collect feedback during execution, including sensory information [20], human corrections [60], or unmet preconditions [42] [56]; LLMs also struggle to produce correct executable plans even with detailed action descriptions [55].Source: §1 · p. 2
The framework extracts PDDL action models from LLMs for use with sound domain-independent planners, separating action-causality modeling from action sequencing.The models can also validate and correct heuristic plans generated by an LLM planner.Source: §1 · p. 2
GPT-4 produced high-quality PDDL models covering over 400 literals and 41 actions across two IPC domains and a household domain, and corrected all errors using natural-language feedback.Feedback came from PDDL validators and humans; users can inspect natural-language renderings of PDDL and return corrections without prior PDDL knowledge.Source: §1 · p. 2
The models support either planning from PDDL goal specifications with standard planners or validating LLM-suggested plans and returning unmet-precondition or goal-condition feedback.In the latter use, the PDDL model acts as an inexpensive high-level simulator or human proxy for plan correctness.Source: §1 · p. 2
Plan validation improves correctness on average, but the LLM planner's planning capability remains a limit; using it can flexibly accommodate user ordering constraints without extra PDDL state variables.The model-based approach also reduces reliance on faithful simulators or extensive expert plan inspection.Source: §1 · p. 3

2 Related Work

Related work explores LLMs for planning, trained sequence models, symbolic domain-model acquisition, and external-tool use. This paper uses off-the-shelf LLMs to construct symbolic world models and employs sound planners for more reliable plan generation.

LLMs and planning

Off-the-shelf LLMs often produce inaccurate plans [55], prompting work that uses them as heuristics or seeds [55], or improves them through prompting and feedback [60] [51] [20] [34] [46] [42] [56].SayCan [1] and Text2Motion [29] use LLMs to score high-level actions before low-level planners ground them; other work generates Python-style plans [28] [50].Source: §2 · p. 3

Training transformers for sequential decision-making tasks

Unlike work that fine-tunes LLMs [55] [38] or trains sequence models [27] [7] [43], this study uses off-the-shelf LLMs without extra training to construct symbolic world models.Prior experiments suggest task-specific sequence-model training can produce an internal world representation [26].Source: §2 · p. 3

Learning/acquiring symbolic domain models

Prior symbolic domain-model acquisition includes learning-based methods [4] and interactive editors; this work instead leverages LLM world knowledge and in-context learning.Related LLM work translates natural language into formal descriptions [35] or constructs PDDL goals from instructions [58] [32].Source: §2 · p. 3
Unlike a symbolic model that simulates outcomes with guaranteed correctness, an LLM used directly as a world model can add errors; autoregressive models also show limits in reasoning about action effects and detecting plan errors [3] [31] [53] [54].A contemporary approach uses an LLM as a parametric world model and plan critic [15].Source: §2 · p. 3

Language models with access to external tools

Because LLMs may underperform specialized tools on downstream tasks, prior frameworks give them external tools for arithmetic [45] and logical reasoning [39] [57]; this work uses sound planners for more reliable plan generation.The passage characterizes this approach as augmenting LLM capacity with external sound planners.Source: §2 · p. 3

3 Problem Setting and Background

The section frames planning as choosing executable high-level skills to satisfy user goals, with human feedback allowed during domain-model construction and plan execution. It then introduces classical planning and PDDL as the formalism and encoding language used to specify such problems.

3 Problem Setting and Background

The agent uses skills from a library to achieve user-specified goals, and human feedback may support both domain-model construction and plan execution.A planner may be an LLM or an external planner.Source: §3 · p. 3

3.1 Classical planning problems

A classical planning problem specifies a domain, initial state, and goal; actions have preconditions and add/delete effects, and a plan reaches the goal.The domain comprises fluents and actions; actions may be grounded with objects or represented as lifted models.Source: §3.1 · p. 4

3.2 PDDL

PDDL is the standard language for classical planning; its Blocksworld put-down action requires holding a block and makes it clear, on the table, and releases it.The action’s effects also make the robot’s hand empty.Source: §3.2 · p. 4

4 Methodology

The methodology builds an explicit PDDL world model with LLMs, corrects it using validator or human feedback, and uses the resulting model to generate plans with domain-independent planners or support iterative LLM planning.

4.1 Constructing PDDL models with LLMs

LLMs generate action models from domain and action descriptions, examples, and a shared predicate list, then repeat generation with all extracted predicates to catch overlooked preconditions.Each action output includes arguments, PDDL preconditions and effects, and natural-language descriptions of new predicates; those descriptions also support model inspection and grounding initial-state predicates.Source: §4.1 · pp. 4–5

4.2 Correcting errors in the initial PDDL models

An LLM mediates between PDDL and natural-language feedback: validators flag syntax errors, while human experts identify factual errors such as missing effects.Users can inspect natural-language versions of action models and provide corrections during construction or after planning; feedback is integrated by continuing the construction dialogue, avoiding repeated similar corrections.Source: §4.2 · pp. 6–7

4.3 Generating plans with the extracted PDDL models

Given a grounded initial state and symbolic goal, a domain-independent planner can find a satisficing or optimal plan using the extracted PDDL model.LLM-generated plans may seed local-search planners such as LPG to accelerate search; alternatively, extracted PDDL can act as a symbolic simulator for VAL-guided iterative LLM plan refinement [42].Source: §4.3 · p. 7
The extracted domain model can also guide skill learning or exploration in reinforcement-learning settings, even when it is not fully situated.The initial state can be grounded by translating environment descriptions with LLMs or querying pre-trained vision-language models, and user commands can be parsed into symbolic goals as in [30] [58] [32].Source: §4.3 · p. 7

5 Empirical Evaluation

Experiments across Household, Tyreworld, and Logistics evaluate LLM-generated PDDL construction, correction, and downstream planning, finding that GPT-4-derived models support effective symbolic planning while LLM plan generation remains error-prone.

5.1 Constructing PDDL

With partially specified constraints, GPT-4 produced 59 PDDL errors versus GPT-3.5-Turbo’s more than 350; GPT-4’s open-ended Household models had around 45 errors.Experts assessed correctness by annotating generated models; annotation counts approximate the distance to corrected models. GPT-4’s 59 errors included three syntax errors, with the rest factual.Source: §5.1 · pp. 7–8, Table 1 · p. 8
GPT-4’s PDDL models sometimes missed causal relationships, especially for spatial reasoning, and could include contradictory effects.For example, its pickup model omitted possible objects stacked above the target, while a blender-mashing model listed both an object’s removal from and presence in a receptacle.Source: §5.1 · p. 8

5.2 Correcting PDDL with domain experts

For PDDL models generated with partially given constraint information, GPT-4 used 59 feedback messages to address 56 factual errors, while GPT-3.5-Turbo failed to correct all errors.Three instances required additional feedback: in one, the user reiterated the error; in two, GPT-4 introduced new errors.Source: §5.2 · pp. 8–9

5.3 Generating plans with the extracted PDDL models

Fast Downward found valid plans in 95% of cases, while GPT-4 ReAct with VAL feedback remained below 50% success.Fast Downward failures were due only to goal-translation errors; ReAct was limited to eight feedback rounds per task.Source: §5.3 · pp. 9–10, Table 2 · p. 9
GPT-4 ReAct plans could get stuck repeating plans or introduce new errors, and the approach allowed only object types specified in the prompt.For ReAct, GPT-4 received natural-language descriptions translated from the PDDL action models and at most eight VAL feedback messages per task.Source: §5.3 · pp. 9–10
Household LLM plans followed specified ordering constraints despite being incorrect or non-executable, while classical planners sometimes produced unconventional actions that LLM plans rarely did.The authors suggest LLM plans may reflect implicit human preferences and note that combining this with symbolic-model correctness warrants further exploration.Source: §5.3 · p. 10

6 Conclusion

The paper proposes maintaining an explicit world model rather than mapping prompts directly to plans, using LLMs to construct and correct PDDL models and external planners to plan. Experiments report that this pairing significantly outperforms existing methods across two IPC domains and a household-robot domain.

6 Conclusion

The LLM–external-planner approach significantly outperforms existing methods on two IPC domains and a household-robot domain with more action-wise constraints than commonly used benchmarks such as ALFWorld.The conclusion does not provide quantitative comparison values.Source: §6 · p. 10
The proposed paradigm maintains an explicit world model instead of mapping user prompts directly to plans, reflecting the view that LLMs are better suited to supplying models than combinatorial search.The authors state that LLMs are incapable of the combinatorial search needed to produce correct plans.Source: §6 · p. 10
The pipeline generates high-quality PDDL models with GPT-4, corrects them using natural-language feedback, and uses the extracted domain models for reliable planning in multiple ways.Source: §6 · p. 10
Future work should test scaling to more intricate planning logic, support partial observability, and account for noisy perception rather than assuming perfect predicate grounding.The evaluated domains are less complex than many in classical planning; the current framework assumes full observability and perfect grounding of predicate values.Source: §6 · p. 10

A Appendix · A.1 Broader impact on using LLMs

Using LLMs to generate plans can create downstream safety risks, while extracting explicit domain models and planning with sound planners may mitigate them. Explicit models also create opportunities for explanation and debugging, though human verification cannot eliminate the risk of certifying an undesirable model.

A.1 Broader impact on using LLMs

Directly generated LLM plans cannot be guaranteed correct, creating potential downstream safety and security issues.Source: §A.1 · p. 16
Using extracted domain models with external sound planners aims to mitigate those risks, but humans may inadvertently certify an incorrect or undesirable model.Such certification could lead to undesirable plans and agent behaviors.Source: §A.1 · p. 16
Intermediate symbolic models offer additional opportunities to explain generated behaviors, drawing on explainable AI [23].Source: §A.1 · p. 16
When users’ understanding differs from the transcribed model, debugging and model reconciliation methods such as D3wa+ [52] can be applied directly.Source: §A.1 · p. 16

A.2 Additional discussion on alternative to action-by-action PDDL construction

The paper considers building the entire PDDL domain model in one dialogue by providing descriptions of all actions, but retains action-by-action construction because of prompt-length constraints and its satisfactory results.

A.2 Additional discussion on alternative to action-by-action PDDL construction

A single-dialogue alternative would provide descriptions of all actions at once, potentially helping the LLM establish a global view of the domain.The authors do not pursue this approach.Source: §A.2 · pp. 16–17
All-action prompts risk exceeding context windows, impede continued corrective-feedback dialogue, and are unnecessary given action-by-action construction's satisfactory results.The prompt must also be short enough to support smaller models such as GPT-3.5-Turbo and training specialized models.Source: §A.2 · p. 17

A.3 Examples of feedback messages that capture syntax errors

The examples show how low-cost checks and VAL feedback identify common PDDL syntax errors and guide LLM revisions.

A.3 Examples of feedback messages that capture syntax errors

Simple syntax errors detectable by Python scripts were sent directly to the LLM and excluded from Table 1 results.The examples include unsupported keywords, name clashes, and invalid object types.Source: §A.3 · p. 17
When an LLM used an unsupported keyword such as ‘forall’, feedback asked it to express the logic in standard STRIPS style, using existing predicates where possible.Source: §A.3 · p. 17
Feedback flags newly created predicates whose names clash with existing object types and asks the LLM to rename them.Source: §A.3 · p. 17
Predicate-name clashes, including mistakenly listing existing predicates as new, were GPT-4’s most common syntax error in the experiments.Feedback asks the LLM to reuse existing predicates or give genuinely new predicates distinct names.Source: §A.3 · p. 17
Feedback also identifies invalid object types, while VAL can detect predicate-parameter type mismatches and suggest using a correct or newly devised predicate.The listed error types covered most GPT-4 syntax errors; predicate-use problems from mismatched object types were less common.Source: §A.3 · p. 17

A.4 Techniques that assist users to locate errors in PDDL models

The section outlines a starting approach for helping users unfamiliar with PDDL tools locate domain-model errors through natural-language review and corrective feedback during model construction and planning.

A.4 Techniques that assist users to locate errors in PDDL models

Tools such as GIPO can visualize action causal dependencies, but the work leaves these advanced techniques out of scope and offers a starting solution for unfamiliar users.Source: §A.4 · p. 17
Users can review natural-language descriptions of predicates, parameters, preconditions, and effects before planning, though some factual errors may only become apparent when they inspect plans.This initial review is not expected to catch every error because users may not know certain constraints until they see the final plans.Source: §A.4 · p. 18
When no plan is found, users can suggest an executable goal-satisficing plan, which the model validates to locate the first unmet precondition and present relevant actions for inspection.The method flags potentially incorrect models or preconditions; for example, a slicing action incorrectly required both an object on a cutting board and on furniture.Source: §A.4 · p. 18
If a model produces a rejected plan or one that fails during execution, users can flag actions that may have missing preconditions or effects.Potential errors may occur in actions taken before or during the execution failure.Source: §A.4 · p. 18

A.5 Detailed description of the Household domain

The Household domain models a single-arm robot with realistic manipulation limits and uses a prompt describing its tasks, objects, and morphology to construct the PDDL model.

A.5 Detailed description of the Household domain

The robot cannot grasp multiple objects at once or manipulate while holding irrelevant items, and some tasks require open, flexible surfaces such as countertops.The PDDL-construction prompt gives a general description of the tasks, object types, and robot morphology.Source: §A.5 · p. 18

A.6 Household: Constructing PDDL Models

The Household PDDL domain models a robot manipulating small household objects around large, normally immovable furniture and appliances. Its one-gripper design imposes constraints on holding and manipulation, while furniture and appliances determine object locations.

A.6 Household: Constructing PDDL Models

The domain distinguishes the robot, large normally immovable furnitureAppliance objects, and householdObject items, including the smallReceptacle subtype.Examples include stoves, tables, drawers, cabinets, and microwaves as furnitureAppliance objects, and bowls, plates, and lunch boxes as smallReceptacle objects.Source: §A.6 · p. 18
Because the robot has one gripper, it can hold only one object and must not hold irrelevant objects during manipulation tasks such as opening drawers or closing windows.Source: §A.6 · p. 18
Manipulation of small household items requires a flat furniture surface, and furniture or appliances determine the locations of the robot and small items.Source: §A.6 · p. 18

A.6.1 An example prompt for constructing PDDL models of the action "close a small receptacle"

The appendix illustrates a prompt for generating PDDL preconditions and effects for “close a small receptacle,” using example actions and domain and predicate context. It instructs the model to list conditions separately and define predicates with unique names and explicit parameters.

A.6.1 An example prompt for constructing PDDL models of the action "close a small receptacle"

The prompt requires separate precondition and effect conditions, unique predicate names, and explicit predicate parameters.It recommends intuitive, readable predicate names.Source: §A.6.1 · p. 19
Two BlocksWorld actions illustrate the requested input-output format, with domain information, action descriptions, parameters, and predicate definitions.The examples cover putting a block on the table and picking one up.Source: §A.6.1 · p. 19
The target household domain describes a one-gripper robot and asks it to close an openable small receptacle, such as a lunch box or storage box.The prompt supplies existing predicates including robot-at, object-on, pickupable, and closed-receptacle.Source: §A.6.1 · p. 20

A.6.2 Navigate to a furniture piece or an appliance

The action lets the robot move from one normally immovable furniture piece or appliance to another by changing its reported location.

A.6.2 Navigate to a furniture piece or an appliance

The action enables navigation between normally immovable furniture pieces or appliances, including tables, cabinets, sinks, refrigerators, coffee makers, microwaves, and washers.Examples in the passage include dining and side tables as well as washers.Source: §A.6.2 · p. 20
The action takes the furniture piece or appliance the robot is currently at as ?x and its desired destination as ?y.Both parameters have type furnitureAppliance.Source: §A.6.2 · p. 20
Its effects remove the robot-at relation for ?x and establish robot-at ?y.Source: §A.6.2 · p. 20

A.6.3 Pick up an object on or in a furniture piece or an appliance … A.13 Translating user instructions into PDDL goal specifications

These sections present examples of PDDL action-model construction and correction across household, logistics, and Tyreworld tasks, including errors addressed through feedback. They also illustrate translating between natural-language instructions and structured planning representations and correcting planner plans with validation feedback.

A.6.3 Pick up an object on or in a furniture piece or an appliance

Pickup requires a pickupable object at the robot’s location, an open container when openable, and a clear stackable target; its effects make the robot hold the object.The feedback flags the need to require that no other object is stacked on a stackable target.Source: §A.6.3 · p. 21

A.6.4 Put an object on or in a furniture piece or an appliance

Putting an object in or on furniture requires the robot to be there and holding it; the feedback flags an unnecessary clear-object condition and an illegal if keyword.The furniture or appliance should be open if openable.Source: §A.6.4 · pp. 21–22

A.6.5 Stack Objects; A.6.6 Unstack Objects

Stacking requires stackable objects, a clear target, and a flat surface; unstacking requires the specified object-on-object relation, which its effects must remove.The stack schema incorrectly uses pickupability and omits the effect that places the top object on its target; the unstack schema also misses that relation as a precondition.Source: §A.6.5 · p. 22, §A.6.6 · p. 23

A.6.7 Open a furniture piece or an appliance; A.6.8 Close a furniture piece or an appliance

Opening and closing openable furniture require the robot to be present with an empty hand and the furniture in the corresponding prior state; each action changes that state.Opening requires the item to be closed and makes it opened; closing requires it to be opened and makes it closed.Source: §A.6.7 · p. 23, §A.6.8 · p. 24

A.6.9 Toggle a small appliance on; A.6.10 Toggle a small appliance off

Turning a small appliance on requires it to be off and accessible with an empty hand, while turning it off requires it to be on and makes it off.The on-action feedback identifies a missing requirement that the appliance be toggleable.Source: §A.6.9 · p. 24, §A.6.10 · pp. 24–25

A.6.11 Slice an object

Slicing requires a sliceable whole object on a cutting board, a held knife, and a flat surface; the feedback corrects the object’s placement from furniture to board.The feedback also says the cutting board should be treated as a small receptacle.Source: §A.6.11 · p. 25

A.6.12 Heat food with a microwave; A.6.13 Heat food with pan

Microwave heating uses food in a small receptacle inside a closed microwave, while pan heating uses food on a pan placed on a stove; both schemas make heated food no longer pickupable.The feedback flags incorrect placement conditions and missing microwave, pan, and stove type requirements.Source: §A.6.12 · pp. 25–26, §A.6.13 · p. 26

A.6.14 Transfer food from one small receptacle to another

Food transfer moves food between small receptacles, commonly when it cannot be easily picked up, and assumes an open flat surface with receptacles opened if openable and clear if stackable.Examples include pouring milk from a milk box into a mug and transferring heated pizza from a pan to a plate.Source: §A.6.14 · p. 27
The robot can transfer food between two small receptacles on a flat furniture surface when the receptacles satisfy the applicable open-state and stacking constraints.The action removes the food from the source receptacle and puts it in the destination; stacking constraints should apply only to stackable receptacles, with no object stacked atop them.Source: §A.6.14 · p. 27

A.6.15 Put an object onto or into a small receptacle like a bowl and plate

The robot can put an object into or onto a small receptacle on a flat furniture surface, provided the receptacle is open if openable and is not stacked when stackable.The action makes the robot release the object and places it in the receptacle; the proposed model marks holding the object as an unnecessary precondition.Source: §A.6.15 · pp. 27–28

A.6.16 Pick up an object on or in a small receptacle like a bowl and plate

The robot can pick up an object in a small receptacle when it is at the supporting furniture, has an empty hand, and the object and receptacle meet the relevant placement constraints.The proposed model incorrectly treated the object as directly on the furniture and restricted openable receptacles; it also notes that stackability-specific constraints should apply only when relevant.Source: §A.6.16 · pp. 28–29

A.6.17 Open a small receptacle such as a lunch box with a lid; A.6.18 Close a small receptacle such as a lunch box with a lid

The robot can open or close an openable small receptacle placed on an open, flat furniture surface, with stacking constraints applying when the receptacle is stackable.Opening requires the receptacle to be closed and makes it opened; closing requires it to be opened and makes it closed.Source: §A.6.17 · p. 29, §A.6.18 · p. 30

A.6.19 Mash food with a blender

The robot can mash sliced food inside a blender that is off before and after the action, and the food remains inside afterward.The proposed model identifies the blender as a small receptacle, requires the food to be in it, and notes that the food is no longer pickupable after mashing.Source: §A.6.19 · pp. 30–31

A.6.20 Wash an object

The robot can wash a washable object while holding it at a sink or basin, after which the object is no longer dirty and is marked washed.The proposed model flags the sink-or-basin type requirement as missing.Source: §A.6.20 · p. 31

A.6.21 Wipe a surface

The robot can wipe a furniture or appliance surface with a clean, washed cloth it holds, leaving the surface clean and the cloth dirty.The proposed model says the surface need not be flat and the cloth should be typed as a cloth.Source: §A.6.21 · pp. 31–32

A.6.22 Vacuum a carpet; A.6.23 Empty a vacuum cleaner

The robot can vacuum a carpet with a held handheld vacuum whose dust bin is not full, then empty the bin while holding the vacuum beside a trash can.Vacuuming marks the carpet clean and fills the bin; the emptying-action passage supplies the holding-and-location requirement.Source: §A.6.22 · p. 32, §A.6.23 · p. 33

A.6.23 Empty a vacuum cleaner

The vacuum-emptying example flags a trash-can condition and missing object-type preconditions, while the action should leave the robot holding the vacuum and empty its dust bin.The trash can should be open only if it is openable; the reported model instead has an inaccurate opening precondition and omits that it must be a trash can and the object a vacuum cleaner.Source: §A.6.23 · p. 33

A.6.24 Examples of PDDL action models constructed by GPT-3.5-Turbo

GPT-3.5-Turbo’s navigation, pickup, and stacking examples contain irrelevant or missing conditions, unknown parameters, and incorrect effects.The examples flag an omitted robot-location effect for navigation, parameter and precondition errors for pickup, and a missing furniture parameter plus syntax and condition errors for stacking.Source: §A.6.24 · pp. 33–34

A.6.25 The initial set of predicates extracted by GPT-4

GPT-4’s initial predicate inventory covers robot location and grip state, object placement and properties, and appliance or furniture states and task attributes.The listed predicates include robot-at, robot-holding, object-on, object-in, openable/opened, food-heated, washable, dustbin-full, and furniture-cleaned.Source: §A.6.25 · pp. 34–35

A.7.1 Heat food with a pan

For heating food with a pan, corrective feedback retypes the pan as a small receptacle and specifies pan and stove-burner conditions, object-in placement, and a heated-pan effect.The feedback also removes the requirement that the food be pickupable and that the burner have a flat surface; the revised schema uses the existing object-in predicate.Source: §A.7.1 · pp. 35–36

A.7.2 Slice an object

For slicing, feedback changes the cutting board to a small receptacle and places the object on the board rather than directly on the furniture.The revised model represents the object-board relation with object-in and the board-furniture relation with object-on.Source: §A.7.2 · p. 37

A.7.3 GPT-3.5-Turbo trying to correct the action model of "slice an object"

GPT-3.5-Turbo’s slice-model revision uses object-in for the object and board, retains the board’s furniture placement, and adds an is-small-receptacle condition.The suggested predicates include cutting-board and is-small-receptacle for the board.Source: §A.7.3 · p. 38

A.8 Logistics: Constructing PDDL Models

The logistics examples model package transport among cities and locations using trucks and airplanes, with truck-load and unload actions changing package location and truck containment.The domain has directly connected locations within each city, city airports, and one truck per city; the airplane example introduces package loading between cities.Source: §A.8 · p. 39, §A.8.1 · p. 39, §A.8.2 · p. 39

A.8.3 Load a package into an airplane

Loading requires the package and plane at the same location, while unloading requires the package aboard; the effects switch between location and onboard predicates.Both actions are specified with package, plane and location parameters.Source: §A.8.3 · p. 40, §A.8.4 · p. 40

A.8.7 Examples of PDDL action models constructed by GPT-3.5-Turbo

GPT-3.5-Turbo’s initial action examples contain incorrect predicates, syntax and effects, and omit the truck’s location parameter for unloading.Its flight model also lacks airport preconditions.Source: §A.8.7 · pp. 41–42

A.8.8 The initial set of predicates extracted by GPT-4

GPT-4’s logistics predicate set represents package and vehicle locations, package containment in trucks or planes, city membership, and movement of carried packages.The movement predicates encode that packages travel with their truck or airplane.Source: §A.8.8 · p. 42

A.9.1 Fly an airplane from one city to another

GPT-4’s revised flight action requires both airports and different cities, adding an airport predicate after feedback identified the missing constraints.The initial model allowed flights between any locations in different cities; the feedback specifies that planes can land only at airports.Source: §A.9.1 · p. 43

A.9.2 GPT-3.5-Turbo trying to correct the action model of "fly an airplane from one city to another"

After correction feedback, GPT-3.5-Turbo adds an airport condition for the destination but still omits the departure airport condition.The revised model introduces an airport predicate but does not fully satisfy the feedback.Source: §A.9.2 · p. 44

A.10 Tyreworld: Constructing PDDL Models

The Tyreworld model represents a robot replacing a flat tire using tools, wheels, nuts, containers and hubs, with no limit on how many objects it can carry.Each hub has one nut; small objects are divided into tools, wheels and nuts.Source: §A.10 · p. 44

A.10.3 Fetch an object from a container

Opening and closing switch container state; removing an object requires an open container and changes it from contained to held, while putting it in requires an open container and changes it from held to contained.Source: §A.10.1 · p. 45, §A.10.2 · p. 45, §A.10.3 · p. 45

A.10.5 Loosen a nut in a hub

Tyreworld nut actions require the hub on the ground and the wrench held, while jacking actions require the jack held and switch the hub between ground and raised states.Tightening additionally requires the nut to be loose; jacking down restores the hub-on-ground state.Source: §A.10.5 · p. 46, §A.10.6 · p. 46, §A.10.7 · p. 47

A.10.9 Unfasten a hub; A.10.10 Fasten a hub

Unfastening a hub requires it to be jacked up and the nut to be loose; fastening installs a held nut on a jacked-up, unfastened hub and makes the nut loose.The examples flag holding the nut before unfastening as incorrect, and an existing nut-in-hub condition before fastening as unnecessary.Source: §A.10.9 · pp. 47–48, §A.10.10 · p. 48

A.10.11 Remove wheel from hub; A.10.12 Put wheel on hub

Wheel removal and placement both require a jacked-up, unfastened hub; removal leaves the robot holding the wheel, while placement puts a held wheel onto the hub.The examples describe removal from a hub and putting a wheel on an unfastened hub.Source: §A.10.11 · pp. 48–49, §A.10.12 · p. 49

A.10.13 Inflate wheel

The inflation action requires a held, intact, deflated wheel and changes it to inflated, but its initial model omits the required pump.The identified missing precondition is that the robot should have a pump.Source: §A.10.13 · p. 49

A.10.14 Examples of PDDL action models constructed by GPT-3.5-Turbo

GPT-3.5-Turbo examples contain malformed or irrelevant PDDL, including syntax errors, unknown or missing parameters, and incorrect preconditions and effects.The examples include irrelevant predicates and an unknown parameter in an “Open a container” model, and a missing wrench parameter plus syntax errors in “Loosen a nut in a hub.”Source: §A.10.14 · p. 50

A.10.15 The initial set of predicates extracted by GPT-4

GPT-4’s initial predicate set covers containers, object holding, hubs, nuts, wheels, and their relevant states, including whether a wheel is intact, deflated, or inflated.The listed predicates also include pump and jack types and whether a hub is jacked up or unfastened.Source: §A.10.15 · pp. 50–51

A.11.1 Unfasten a hub

For unfastening a hub, corrective feedback removes the erroneous precondition that the robot holds the nut and adds the missing effect that it holds the detached nut.The revised model retains the hub and nut conditions and adds robot-holding of the nut as an effect.Source: §A.11.1 · p. 51

A.11.2 Inflate wheel

For wheel inflation, feedback requires the robot to hold a pump and changes its parameter type after a validator flags the unknown type pump.The revised action uses a small_object parameter identified as a pump.Source: §A.11.2 · pp. 52–53

A.12.1 Examples of prompts for LLM planners

Household prompts specify one-gripper and flat-surface constraints, with pickup requiring access to the object and an empty gripper; a Logistics prompt describes package transport by trucks and planes.The household example plans to put an orange in a fridge, while the Logistics prompt specifies loading a package into a truck at a location.Source: §A.12.1 · pp. 53–54
Planner prompts pair action descriptions with parameters, preconditions, and effects, then provide object states and a transport task for plan generation.An example plan uses drive, load, and unload actions to transport a package to its destination.Source: §A.12.1 · pp. 54–55

A.12.2 Translation to admissible actions

GPT-4 translates free-form plans into admissible action formats and requests plan regeneration when parameters are missing or an action cannot be translated.The translation prompt specifies that missing information should yield “Not able to translate due to missing information.”Source: §A.12.2 · pp. 55–56

A.12.3 Back-prompting LLM planners with validation feedback by VAL

VAL feedback covers unsatisfied action preconditions, unmet goals, and invalid or illegal action parameters, with PDDL conditions translated into natural language.An LLM translates PDDL into natural language so corrective feedback can be provided to the planner.Source: §A.12.3 · p. 56

A.12.4 Examples of validation feedback and instructions with ordering constraints

Examples illustrate ordered household goals (mash last; fork first and knife last) and a plan revision after VAL reports an unmet precondition.One example’s feedback says the robot gripper is not holding an object; another reports conditions involving the fridge and lunch box.Source: §A.12.4 · pp. 57–58

A.13 Translating user instructions into PDDL goal specifications

The goal-translation prompt restricts PDDL goals to supplied predicates; its BlocksWorld example encodes positive stacking goals and a negated table-location goal.The example expresses that block_1 is on block_2, block_2 is on block_3, and block_5 is not on the table.Source: §A.13 · p. 59