Learning adaptive planning representations with natural language guidance

• Generalized, long-horizon planning remains a core challenge across symbolic planning, reinforcement learning, and generative AI.Source: §1 · p. 1
• Useful high-level actions must support efficient planning, feasible low-level execution, and composition across new tasks.Source: §1 · p. 1
• Most advanced planning and robotics systems still rely on human expertise to engineer domain-specific planning representations.Source: §1 · p. 2
• Ada combines language-derived domain knowledge with hierarchical planning to build symbolic high-level actions and associated low-level controllers.Source: §1 · p. 2, Figure 1 · p. 2
• Ada solves dramatically more tasks than three LM-based sequential-decision baselines on Mini Minecraft and ALFRED, while its abstractions compose for complex unseen tasks.Source: §1 · p. 2

• The environment has raw states, low-level actions, deterministic transitions, and a fixed mapping into predicate-based abstract states.Source: §2 · p. 2
• The agent must execute low-level actions from an initial state so the abstract state satisfies the goal predicate expressed by a language instruction.Source: §2 · pp. 2–3
• Natural language bootstraps a high-level action space to make long-horizon planning more efficient than direct reflex policies.Source: §2 · p. 3
• Each learned action contains a name, arguments, symbolic preconditions and effects, and a controller that realizes the effects.Source: §2 · p. 3
• A symbolic planner first finds an abstract action sequence, which is refined through associated controllers into a low-level plan and decomposes long horizons into shorter subproblems.Source: §2 · p. 3

3 ACTION ABSTRACTIONS FROM LANGUAGE

• At each iteration, Ada proposes operators and symbolic goals, uses bi-level planning to execute unsolved tasks, updates controllers, and filters operators by execution results.Source: §3 · pp. 3–4
• Ada’s central objective is to refine LLM-proposed abstractions into operators that support efficient, accurate planning and generalize to future tasks.Source: §3 · p. 4

3.1 OPERATOR PROPOSAL

• The LLM first generates symbolic task decompositions, after which undefined action names are converted into PDDL-style operator definitions.Source: §3.1 · pp. 4–5

3.2 GOAL PROPOSAL AND BI-LEVEL PLANNING

• Candidate goal formulas are inferred from language, and a symbolic planner searches the operator library for high-level plans satisfying those formulas.Source: §3.2 · p. 5
• Each high-level action induces a local subgoal, whose low-level actions are searched and executed before environment feedback evaluates task success.Source: §3.2 · p. 5

3.3 POLICY LEARNING AND GUIDED LOW-LEVEL SEARCH

• Ada learns shared subgoal-conditioned controllers using environment feedback and hindsight relabeling, including a count-based model over states and subgoals.Source: §3.3 · p. 6

3.4 SCORING LLM OPERATOR PROPOSALS

• Operators are retained only after sufficient usage and successful low-level execution, using empirical success rates to score their accuracy.Source: §3.4 · p. 6

4 EXPERIMENTS

• Ada is evaluated on Mini Minecraft and ALFRED against low-level planning, subgoal prediction, and imperative code-policy baselines.The setup uses iterative continual learning, GPT-3.5, sampled proposals, and best-of accuracy over three replications.Source: §4 · pp. 6–8
• Mini Minecraft tests mining, crafting, and compositional goals, including tasks requiring up to 26 operators and generalization from simpler benchmarks.The compositional evaluation reuses actions learned during Mining and Crafting without additional training.Source: §4 · pp. 6–7
• ALFRED tests household tasks in varied 3D layouts, with ground-truth plans of 5–10 operators and trajectories averaging 50 low-level actions.Its human-annotated goals can be ambiguous, underspecified, or erroneous relative to formally verifiable tasks.Source: §4 · p. 6

4.1 RESULTS

• Ada recovers appropriate mining and crafting operators in Mini Minecraft and semantically identical household operators in ALFRED, enabling generalization to complex tasks.In ALFRED, it reliably recovers all ground-truth operators except that the disjunctive Slice operator is sometimes split or only partially recovered.Source: §4.1 · p. 8, Table 1 · p. 8
• Interactive verification rejects redundant, inaccurate, and underspecified operator proposals before they impair high-level planning or low-level execution.Examples include proposals that omit key preconditions or assume cleaning with a towel rather than washing in a sink with water.Source: §4.1 · p. 8
• Ada solves all Mini Minecraft evaluation variations and vastly outperforms other baselines on ALFRED, where learned operators compose effectively for complex planning.On ALFRED, failures include goal misspecification, low-level policy inaccuracies, and occasional operator overspecification; the ground-truth goal appears among the top four proposals 92% of the time.Source: §4.1 · pp. 8–9, Table 1 · p. 8
• Ada substantially outperforms direct goal and subgoal prediction because those baselines accumulate sequencing and object-grounding errors over longer horizons.Subgoal Prediction declines on compositional Minecraft tasks and handles little beyond basic pick-and-place in ALFRED.Source: §4 · p. 8, §4.1 · p. 9
• Code Policy Prediction performs unevenly because imperative skills leave planning to the LLM, hinder optimized symbolic planning, and produce large, less compact libraries.The baseline learns hundreds of code definitions, whereas Ada targets a compact library suited to structured hierarchical planning.Source: §4 · p. 8, §4.1 · p. 9

Planning for language goals

• Direct language-model action prediction often struggles with generalization and correct plans as task complexity increases, motivating structured and iterative prompting.Other approaches predict formal goals or actions that symbolic planners verify or solve.Source: §5 · pp. 9–10

Learning planning domain and action representations from language

• Ada differs from prior latent-action work by learning planning-compatible action abstractions from language models rather than human demonstrations and annotated step-by-step instructions.Compared with Wang et al. (2023a), Ada learns abstractions integrated with hierarchical planning rather than imperative code-based policies.Source: §5 · p. 10

Language and code

• Prior code-synthesis systems learn imperative or functional libraries, whereas Ada targets representations suited specifically to long-range planning.This distinction includes work using LISP, Python, and lifted program abstractions.Source: §5 · p. 10

Hierarchical planning abstractions

• Most hierarchical planning approaches require manually specified abstractions or learn them from demonstrations and interactions; Ada uses human language to guide their acquisition.The related frameworks include hierarchical task networks, hierarchical goal networks, abstract PDDL domains, and domain control knowledge.Source: §5 · p. 10

6 DISCUSSION AND FUTURE WORK

• Language models provide background knowledge for discovering action abstractions that support complex, long-horizon planning.This knowledge includes priors about how the world works, how to decompose tasks, and which tasks an agent may want to solve.Source: §6 · p. 10
• Ada combines linguistic knowledge with hierarchical planning’s search, grounding, and verification toolkits.Source: §6 · p. 10
• The current framework assumes predefined symbolic predicates, omits fine-grained geometric motor planning, and uses a general rather than planning-specialized LLM.Proposed extensions include deriving predicates from perceptual inputs and transferring planning to another model after bootstrapping abstractions.Source: §6 · p. 10

A APPENDIX

• The authors plan to release the full algorithm implementation, all baselines, and benchmark tasks.Source: A APPENDIX · p. 14

A.1.1 LLM PROMPTING

• Prompts for task decomposition use constructed examples that avoid skills needing to be learned, then incorporate sampled decompositions from solved tasks.This progressively makes the prompt better represent the task distribution without exposing the target skills in advance.Source: §A.1.1 · p. 14
• At each iteration, Ada prompts an LLM to propose high-level task decompositions and operator definitions for unsolved tasks.Operator proposals contain arguments, preconditions, and effects conditioned on undefined operator names in the proposed decompositions.Source: §A.1.1 · p. 14
• Few-shot prompts include environment predicates, valid argument values, PDDL examples, and previously validated operators as the library grows.Including current operators helps represent shared operator patterns over successive iterations.Source: §A.1.1 · p. 14
• The method samples multiple decompositions and sequentially requests up to N distinct operator definitions because direct token-probability sampling lacked sufficient diversity.The experiments use temperature=1.0, with n=3 decomposition samples for operator prompting and n=4 for goal prompting.Source: §A.1.1 · pp. 14–15
• With GPT-4, Ada recovers all required operators on Mini Minecraft and ALFRED, and the performance trends relative to Code as Policies (Voyager) remain unchanged.The table covers both benchmarks and reports generalization to more complex compositional tasks.Source: Table 2 · p. 15
• GPT-3.5 often generates invalid predicates, types, or argument signatures, so Ada removes invalid predicates, reconstructs typed arguments, and rejects operators without valid postconditions.The full learner can still verify and reject underspecified operators produced by this correction process.Source: §A.1.1 · p. 15
• Ada also prompts the LLM to propose candidate first-order-logic goal definitions over the environment predicate vocabulary.Training prompts include a randomly sampled 0.1 fraction of goal-definition supervision and later add examples from successfully solved tasks.Source: §A.1.1 · p. 15
• Proposed goals undergo analogous syntactic post-processing that removes invalid predicates and rejects empty formulas.Source: §A.1.1 · p. 15

A.1.2 POLICY LEARNING AND GUIDED LOW-LEVEL SEARCH

• The low-level search stores candidate trajectories for subgoals in a dictionary and reuses them when matching subgoals recur.For unseen subgoals, it performs brute-force breadth-first search over possible action trajectories.Source: §A.1.2 · p. 15
• Ada lifts observed subgoals and trajectories by replacing concrete objects with variables, enabling reuse across different object instances.The dictionary is updated with sampled trajectories during breadth-first search, even when they do not achieve the searched subgoal.Source: §A.1.2 · p. 16
• When several trajectories match a lifted subgoal, Ada prioritizes the trajectory with the highest observed success frequency.Source: §A.1.2 · p. 16

Learned Operator Libraries on Minecraft

• The full PDDL definition combines symbolic environment constants and predicates, initial pick-and-place operators, an example operator, and all learned operators from Mining and Crafting.Source: §A.2 · p. 16
• The domain includes movement operators that update the agent’s tile location and predicates representing object locations.Source: §A.2 · p. 17
• Inventory operators encode picking up and placing objects, while learned tool-use operators require held tools and empty target inventories before producing held targets.Source: §A.2 · p. 17
• The operator specification includes ingredient and target inventory parameters, requires a station object, identifies it as a WorkStation, and checks whether the target inventory is empty.Source: §A.2 · pp. 20–21

Implementation schemas

• The cited fragments identify two action declarations, one labeled c r a f t −s t i c k 0 at entry 518 and another c r a f t −i r o n −i n g o t 2 at entry 599.Source: §A.2 · pp. 24–25
• The crafting schemas require the station to be occupied and typed as a WorkStation, with inventory conditions governing available ingredients and targets.The repeated preconditions include object-at, object-of-type WorkStation, and inventory predicates.Source: §A.2 · pp. 23–24
• Their effects place the crafted target in the target inventory while removing required ingredients from source inventories.The effect fragments repeatedly mark the target inventory nonempty and negate ingredient-holding predicates.Source: §A.2 · pp. 23–24

• The ALFRED domain definition combines typed symbolic predicates with initial pick-and-place operators and subsequent learned operators.The vocabulary represents agents, locations, receptacles, objects, and properties such as containment, cleanliness, toggleability, and slicability.Source: §A.2 · pp. 26, 28
• The cited fragments identify primitive PickupObjectNotInReceptacle and PutObjectInReceptacle actions and mention objectAtLocation, holds, and inReceptacle predicates.Source: §A.2 · pp. 28–29
• The cited operator definitions include actions for rinsing, turning on, cooling, and slicing objects, with predicates indicating clean and toggled states.Source: §A.2 · pp. 29–30