Source-linked AI summary
Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning
Lin Guan, Karthik Valmeekam, Sarath Sreedharan, Subbarao Kambhampati
TL;DR
Directly using LLMs as planners is limited by unreliable plans and dependence on extensive environmental or human feedback. The paper instead has LLMs construct and correct explicit PDDL world models, then uses external planners, significantly outperforming existing methods across two IPC domains and a household-robot domain.
Problem
LLM-generated plans often have limited correctness and require substantial simulator, environment, or human feedback to become reliable.
Method
The framework uses LLMs to extract PDDL action models, translates feedback between PDDL and natural language, and applies external domain-independent planners for reliable planning.
Results
Pairing LLMs with an external planner significantly outperforms existing methods across two IPC domains and a household-robot domain.
Takeaways & Limitations
Explicit corrected world models shift LLMs from directly searching for plans toward supplying symbolic models that external planners can use reliably.
Takeaways & Limitations
GPT-4 still shows shallow causal understanding, especially for spatial reasoning, and can occasionally produce contradictory action effects.
Abstract
from arXiv · showhide
There is a growing interest in applying pre-trained large language models (LLMs) to planning problems. However, methods that use LLMs directly as planners are currently impractical due to several factors, including limited correctness of plans, strong reliance on feedback from interactions with simulators or even the actual environment, and the inefficiency in utilizing human feedback. In this work, we introduce a novel alternative paradigm that constructs an explicit world (domain) model in planning domain definition language (PDDL) and then uses it to plan with sound domain-independent planners. To address the fact that LLMs may not generate a fully functional PDDL model initially, we employ LLMs as an interface between PDDL and sources of corrective feedback, such as PDDL validators and humans. For users who lack a background in PDDL, we show that LLMs can translate PDDL into natural language and effectively encode corrective feedback back to the underlying domain model. Our framework not only enjoys the correctness guarantee offered by the external planners but also reduces human involvement by allowing users to correct domain models at the beginning, rather than inspecting and correcting (through interactive prompting) every generated plan as in previous work. On two IPC domains and a Household domain that is more complicated than commonly used benchmarks such as ALFWorld, we demonstrate that GPT-4 can be leveraged to produce high-quality PDDL models for over 40 actions, and the corrected PDDL models are then used to successfully solve 48 challenging planning tasks. Resources, including the source code, are released at: https://guansuns.github.io/pages/llm-dm.
1 Introduction
Direct LLM planners remain unreliable because their plans can be incorrect, require costly online feedback, and repeatedly burden users with corrections. The paper instead extracts PDDL world models from LLMs, corrects them through natural-language feedback, and delegates search or validation to external planners.
- LLM-generated plans often overlook physical plausibility and long-term dependencies, limiting their correctness and executability.
- Online feedback is available only after execution begins, making correction costly when faithful simulators are unavailable or expensive.
- Repeatedly supplying feedback can frustrate users because LLM planners may repeat similar mistakes across scenarios.
- The proposed model-based paradigm extracts symbolic PDDL action models, then uses external domain-independent planners to search for feasible plans or validate LLM-generated plans.
- LLMs translate PDDL and corrective feedback between symbolic models and natural language, allowing humans and validators to help correct initially imperfect models.
- Validation feedback improves plan correctness on average, but LLM planning capability still limits the backprompted approach.
2 Related Work
Related work covers direct, trained, and tool-augmented uses of language models for sequential decision-making, alongside prior methods for acquiring symbolic domain models. This paper uses off-the-shelf LLMs to construct symbolic world models and external sound planners for reliability.
- Prior studies use LLMs as heuristics or seeds for external planners or human-in-the-loop planning because off-the-shelf LLM plans are often inaccurate.
- Other work fine-tunes LLMs or trains sequence models for sequential decision-making, whereas this paper performs no extra training.
- Classical planning research includes learning-based and interactive editor-based approaches for acquiring symbolic domain models.
- Tool-augmented LLM frameworks delegate specialized subtasks to external tools; this work similarly uses sound planners to improve planning reliability.
3 Problem Setting and Background
The paper studies deterministic, goal-directed planning with a finite skill library, formalizing domains through predicates and actions whose preconditions and effects define state transitions.
- The agent receives high-level user instructions and can execute only skills in a library, with corresponding low-level control policies assumed available.
- A classical planning problem is represented as P = ⟨D, I, G⟩, comprising a domain, initial state, and goal specification.
- The domain D = ⟨F, A⟩ contains fluents defining states and actions that can be executed when their preconditions are satisfied.
- Each action has preconditions describing when it is executable and effects describing the resulting state changes.
- In the Blocksworld example, putting a block on the table requires holding it and produces effects including an empty hand and table placement.
4 Methodology
The methodology prompts LLMs to construct PDDL action models incrementally, translates symbolic models and feedback into natural language for correction, and uses the corrected models for classical or LLM-assisted planning.
- PDDL supplies a standardized world-model representation that domain-independent planners can use to search from initial states toward goals.
- Prompts combine generation instructions, examples, domain and embodiment context, action descriptions, and a dynamically updated predicate list.
- The LLM generates each action model separately, returning arguments, PDDL preconditions and effects, and descriptions of newly defined predicates.
- Natural-language predicate descriptions let users inspect generated models and support translating environment descriptions into grounded initial states.
- The correction interface translates PDDL into natural language, incorporates human or validator feedback, and archives corrections back into the symbolic models.
- A standard planner can search using the LLM-acquired model, while LLM plans may seed local search; alternatively, VAL feedback can drive iterative LLM replanning.
- The extracted PDDL model can also guide skill learning or exploration when reinforcement learning is permitted.
5 Empirical Evaluation
The evaluation tests GPT-4’s PDDL construction and correction across Household and IPC domains, then examines planning with the resulting models. GPT-4 produces high-quality models, incorporates natural-language feedback, and supports planning while retaining limitations in causal reasoning and plan correction.
- Evaluation setup: Experiments cover a household-robot domain with 22 mobile and manipulation skills plus the Tyreworld and Logistics IPC domains.The Household domain imposes more rigorous physical-plausibility constraints than commonly used benchmarks.
- Constructing PDDL: The evaluation counts annotations as an approximate distance between generated and corrected PDDL models, distinguishing functional errors from merely redundant predicates.Redundant predicates affect conciseness rather than functionality.
- Constructing PDDL: GPT-4 produced PDDL models with significantly fewer errors than GPT-3.5-Turbo under partial constraint information.Among 59 GPT-4 errors, three were syntax errors and the remainder were factual errors such as missing preconditions and effects.
- Limitations: GPT-4 still shows shallow causal understanding, including missed stacking dependencies and occasional contradictory effects when constructing action models.The experiments identify spatial reasoning and internally inconsistent effects as persistent construction limitations.
- Correcting PDDL with domain experts: GPT-4 corrected syntax and factual PDDL errors from validator and expert feedback, requiring 59 feedback messages for 56 factual errors.The model handled feedback phrased in various ways, though three factual-error cases required additional feedback.
- Generating plans with the extracted PDDL models: The extracted models support planning in Household and Logistics, while LLM plans preserve ordering constraints and rarely produce unconventional actions observed from classical planners.These observations suggest complementary strengths between LLM planners’ implicit preferences and symbolic models’ correctness guarantees.
6 Conclusion
The paper proposes maintaining explicit PDDL world models rather than directly mapping user prompts to plans. Across two IPC domains and a constrained household-robot domain, pairing LLMs with external planners outperforms existing methods, while domain complexity and scalability remain open questions.
- 6 Conclusion: The proposed paradigm uses LLMs to generate and correct explicit PDDL world models, then applies them for reliable planning in multiple ways.The pipeline combines model generation, natural-language feedback correction, and external planning.
- 6 Conclusion: Pairing LLMs with an external planner significantly outperforms existing methods across two IPC domains and a more constrained household-robot domain.The conclusion characterizes the household domain as having more action-wise constraints than commonly used benchmarks such as ALFWorld.
- 6 Conclusion: The evaluation domains remain less complex than many domains in the classical planning literature, leaving scalability of LLM-generated PDDL models unresolved.The paper explicitly identifies scaling to more complex domains as future work.
A.1 Broader impact on using LLMs
The framework improves explainability by exposing explicit domain models, but incorrect or undesirable models may still be mistakenly certified as correct.
- Human verification remains necessary because an incorrect or undesirable extracted domain model could be certified as correct and produce undesirable plans or agent behaviors.
- Explicit symbolic domain models provide additional opportunities to explain generated behaviors compared with directly querying LLMs for plans.Model reconciliation and debugging techniques can also be applied when users’ understanding differs from the transcribed model.
A.2 Additional discussion on alternative to action-by-action PDDL construction
The paper discusses constructing the entire PDDL model in one dialogue as an alternative to action-by-action generation, but does not pursue it because of context-length and feedback-integration concerns.
- A single dialogue containing all action descriptions might help the LLM establish a global view of the actions.
- The authors do not pursue this alternative because including all actions may create lengthy prompts that exceed context windows, especially for smaller models.The stated concern includes practical use with GPT-3.5-Turbo and training smaller specialized models.
- The alternative is also avoided because the framework’s corrective-feedback integration relies on continuing the construction process.
A.3 Examples of feedback messages that capture syntax errors
The feedback system combines lightweight scripts and VAL to catch common PDDL syntax errors, including unsupported constructs, naming conflicts, invalid types, and mismatched predicate arguments.
- Simple Python scripts detect several low-cost syntax errors before feedback is sent to the LLM.These script-detected errors are excluded from the results reported in Table 1.
- Unsupported PDDL keywords such as ‘forall’ trigger feedback requesting an equivalent formulation compatible with standard STRIPS-style models.
- Predicate names conflicting with object types or existing predicates trigger requests to rename new predicates or reuse existing ones.The existing-predicate naming conflict is reported as GPT-4’s most common syntax error in the experiments.
- Invalid parameter object types generate feedback identifying the invalid type and requesting a valid one.
- VAL captures less-common predicate-usage errors caused by mismatched object types and supplies corrective messages specifying the expected type.
A.4 Techniques that assist users to locate errors in PDDL models
Users can review natural-language renderings of PDDL models to pre-screen factual errors, while advanced visualization and error-location tools remain outside the paper’s scope.
- Graphical tools such as GIPO can visualize causal dependencies among actions to help locate errors in PDDL models.The paper notes that such advanced tools are beyond its scope.
- The proposed starting point lets users review predicates and parameters through natural-language descriptions and identify potential factual errors during model construction.
A.5 Detailed description of the Household domain
The Household domain models a single-arm robot with physical manipulation constraints aligned to real-world robotic capabilities.
- The robot cannot grasp multiple objects simultaneously.
- It cannot perform manipulation actions while holding irrelevant items.The example is opening a fridge door while holding a mug.
- The domain constrains the robot according to real-world robotic capabilities, including less flexible robot arms than human arms.
A.6 Household: Constructing PDDL Models
The Household domain represents a single-gripper robot navigating among furniture and appliances to manipulate household objects under explicit spatial and grasping constraints.
- The robot can hold only one object with its single gripper.
- Manipulation of small household items must occur on flat furniture surfaces.The flat surface provides enough space for manipulation.
- The domain distinguishes robots, furnitureAppliance objects, householdObject objects, and smallReceptacle objects.FurnitureAppliance covers large immovable furnishings or appliances; smallReceptacle is a householdObject subtype for items such as bowls and plates.
A.6.1 An example prompt for constructing PDDL models of the action "close a small receptacle"
The example prompt guides an LLM to construct PDDL action models from natural-language domain and action descriptions, using demonstrations and reusable predicates.
- Prompt structure: The prompt asks for action arguments, PDDL preconditions and effects, and any newly defined predicates with natural-language descriptions.
- PDDL constraints: The prompt requires separate conditions and unique predicate names with explicitly defined parameters.
- Prompt structure: Two BlocksWorld examples illustrate the input and output formats for PDDL generation.
- Household action: The prompt describes closing an open small receptacle such as a lunch box or storage box.
- Household action: Reusable predicates encode robot location, object placement, pickupability, and receptacle closure.
A.6.2 Navigate to a furniture piece or an appliance
The navigation action moves the household robot between two furniture or appliance locations by updating the robot's location predicate.
- The action navigates from one normally immovable furniture piece or appliance to another.Examples include dining tables, cabinets, refrigerators, and microwaves.
- Its parameters identify the robot's current location and desired destination.
- The effects remove robot-at ?x and add robot-at ?y.
A.6.3 Pick up an object on or in a furniture piece or an appliance
The action models picking up household objects from furniture or appliances, with user-provided constraints covering pickupability, access, stacking, and robot state. The generated PDDL includes parameters, preconditions, and effects, but the examples contain several identified inaccuracies and omissions.
- Action description: The action picks up an object from a furniture piece or appliance such as a table, drawer, cabinet, dishwasher, or refrigerator.Examples include picking up a coffee mug, fork, or apple from these locations.
- Action description: The action takes an object and a furniture or appliance as parameters, typed respectively as householdObject and furnitureAppliance.The object is the item to pick up, and the second parameter is where it is located.
- Constraints: Its intended preconditions require robot access, object placement, pickupability, an empty gripper, and an open receptacle when the destination is openable.The user also specifies that the object should not have another household item stacked on top of it.
- Model corrections: The examples identify an overly broad stacking precondition and a missing requirement that no other object be stacked on top of the target.The stacking restriction should apply only when the target is stackable.
- Effects: The intended effects remove the object from the furniture or appliance, place it in the robot’s gripper, and make the gripper nonempty.The model also removes robot-hand-empty and object-on facts.