PaperlayerView exact revision on arXiv

Source-linked AI summary

MemPlan: Memory-Conditioned PDDL Planning for Partially Observable Text Environments

Tengwei Song, Robert Hoehndorf

arXiv:2610.30379v1cs.CLcs.AI

TL;DR

Partially observable text environments require long-horizon planning from incomplete natural-language observations, while LLMs used as primary planners can be costly and unreliable. MemPlan rebuilds fixed-domain PDDL problems using parsed facts, candidate hypotheses, and interaction memory, with an offline-trained interface for parsing and grounding. Across TextWorld, ALFWorld, and Robotouille, the paper reports improved task completion and step efficiency over language-only and PDDL-aware baselines.

  • Problem

    Partially observable text environments require long-horizon agents to act from natural-language observations without access to the full state, while LLM-led planning can be costly and unreliable.

    Source: §1 · pp. 1–2

  • Method

    MemPlan rebuilds fixed-domain PDDL problems from parsed facts, candidate hypotheses, and positive and negative interaction memory, using a constrained language–symbol interface.

    Source: §1 · p. 2

  • Results

    Across ALFWorld, TextWorld, and Robotouille, MemPlan improves task completion and step efficiency over language-only and PDDL-aware baselines.

    Source: §6 · p. 10

  • Takeaways & Limitations

    The results show that interaction memory incorporated into PDDL problem construction can support classical planning in partially observable text environments.

    Source: §6 · p. 10

    MemPlan assumes an available task-level PDDL domain that covers relevant types, predicates, schemas, preconditions, and effects; hard exclusions can also remove valid candidates when feedback is ambiguous or misparsed.

    Source: §5 · p. 10

Abstract

from arXiv · show

LLM agents are flexible in language interaction, but using them as long-horizon planners is costly and prone to hallucinated or invalid actions. This paper targets a different role for LLMs: constrained bridges between language and symbolic planning. We propose MemPlan, a memory-conditioned PDDL planning framework for partially observable interactive text environments. MemPlan keeps the PDDL domain fixed and reconstructs the PDDL problem at each step from parsed observations, candidate hypotheses, and interaction memory. Positive graph memory assigns relevance-based action costs to candidate hypotheses for missing objects, locations, or conditions, guiding the planner toward more plausible checks. Negative memory records execution-refuted assumptions and compiles them into exclusion constraints, preventing repeated attempts at candidates already contradicted by feedback. A unified schema-conditioned language--symbol interface, trained offline with supervised fine-tuning, connects text environments to the planning loop through observation-to-fact parsing and symbolic-action-to-command grounding. Experiments on TextWorld, ALFWorld, and Robotouille show that MemPlan improves task completion and step efficiency over language-only and PDDL-aware baselines, while substantially reducing online token cost compared with LLM-planning baselines such as ReAct. The same fine-tuned interface is also reused across TextWorld task types without task-specific fine-tuning. Our code and data are available at: https://github.com/tewiSong/MemPlan.

1 Introduction

Partially observable text environments demand long-horizon planning from language observations and commands, but LLM-led planning can be costly and unreliable. MemPlan instead combines fixed-domain symbolic planning with interaction memory and a constrained language–symbol interface.

1 Introduction

  • Interactive text environments combine partial observability and delayed consequences with natural-language observations and commands, making long-horizon tasks challenging.LLM agents may hallucinate actions, lose track of earlier observations, or fail to maintain a consistent plan.Source: §1 · p. 1
  • Classical planners search explicitly over action preconditions and effects, and prior work reports approximately 93.1% fewer LLM tokens when replacing an LLM planner with a classical planner.Partially observable environments do not directly provide the complete symbolic state required by a closed-world planner.Source: §1 · pp. 1–2
  • Existing completion methods can require committing to a guessed state, while iterative PDDL refinement depends on repeated online formalization that can introduce errors or forgotten observations.Missing objects, locations, or conditions may have multiple plausible completions in text environments.Source: §1 · p. 2
  • MemPlan builds a PDDL problem at each step from candidate hypotheses and interaction memory, avoiding a single guessed world state and online PDDL rewriting.Positive graph memory assigns costs to prioritize plausible hypothesis checks; negative memory excludes assumptions refuted by feedback.Source: §1 · p. 2
  • The framework uses an offline-trained schema-conditioned interface to parse observations and ground symbolic actions, while keeping online LLM use to constrained interface calls.It is evaluated on TextWorld, ALFWorld, and Robotouille across task completion, step efficiency, token cost, and cross-task interface use.Source: §1 · p. 2

2 Related Work

Prior work connects language agents, memory, feedback, and symbolic planning in several ways. MemPlan differs by compiling interaction memory into a fixed-domain PDDL problem through candidate hypotheses, action costs, and exclusions.

2 Related Work

  • Memory and feedback methods improve LLM agents’ state tracking, action generation, or feedback use, while MemPlan compiles graph-memory relevance into PDDL action costs and refutations into exclusions.Examples include QuBE, WKM, AriGraph, and AdaPlanner [Kim et al., 2024, Qiao et al., 2024, Anokhin et al., 2025, Sun et al., 2023].Source: §2 · p. 2
  • Figure 1 depicts the online loop in which the interface extracts facts and grounds actions, the planner uses memory-conditioned PDDL, and feedback updates memory before replanning.The loop uses the fixed domain, candidate hypotheses, positive memory, and negative memory.Source: Figure 1 · p. 3
  • LLM-to-PDDL work translates, completes, or iteratively refines planning representations; MemPlan instead keeps the domain fixed and compiles online memory into the problem.The compared approaches include LLM+P, LLM-DP, PDDLEGO, and PDDLEGO+ [Liu et al., 2023, Dagan et al., 2023, Zhang et al., 2024, Gong et al., 2025].Source: §2 · pp. 2–3
  • Open-world planning motivates placeholders for entities discovered during execution, which MemPlan adapts to text environments with parsing, command grounding, and memory updates from textual feedback.Related structured embodied systems use symbolic or graph representations, while MemPlan focuses on partially observable text environments.Source: §2 · p. 3

3 Method

MemPlan rebuilds a closed PDDL problem from partial observations, hypotheses, and memory while keeping the domain fixed. It uses a constrained interface for text-to-fact parsing and action grounding, then updates memory and replans after feedback.

3 Method

  • The online loop receives a goal, observation, and admissible commands, builds a PDDL problem from parsed facts, hypotheses, and memory, then executes an action and replans after feedback.The domain is assumed fixed and supplied to the agent rather than generated or repaired online.Source: §3 · p. 3, §3.4 · p. 6

3.1 Offline Schema-Conditioned Language–Symbol Interface

  • The offline-trained, frozen interface parses observations into schema-validated symbolic facts and grounds the first symbolic action to an admissible command.If no admissible command realizes that action, the system selects an observation command, updates memory from feedback, and replans.Source: §3.1 · p. 4

3.2 Memory and Candidate Hypotheses

  • For unresolved goal entities, MemPlan introduces typed placeholders and generates candidate hypotheses from observed entities, memory neighborhoods, visited areas, and goal-compatible values.After feedback resolves a placeholder, later PDDL problems use the concrete entity.Source: §3.2 · p. 4

3.3 Memory-Conditioned PDDL Construction

  • Positive graph memory assigns lower PDDL action costs to more relevant hypothesis checks, while negative memory excludes candidates refuted by targeted checks.The fixed domain and action schemas remain unchanged; the problem incorporates placeholders, candidate checks, and memory-derived exclusions.Source: §3.3 · pp. 4–5

3.4 Online Planning and Memory Update

  • MemPlan executes only the first grounded action of each plan, updates positive memory with supported observations and resolved placeholders, records refutations in negative memory, and replans.The online loop rebuilds the problem after feedback rather than executing the entire plan open-loop.Source: §3.3 · p. 5, §3.4 · p. 6

4 Experiments

Experiments evaluate MemPlan on ALFWorld, TextWorld, and Robotouille, with comparisons covering task completion, efficiency, memory-agent performance, and component ablations. Across the reported settings, MemPlan combines shorter ALFWorld trajectories with favorable TextWorld memory-agent scores and feedback-driven Robotouille replanning.

4.1 Experimental Setup

  • The evaluation covers TextWorld, ALFWorld, and Robotouille; the setup uses fixed environment PDDL domains and an offline-trained unified language–symbol interface.The experiments address completion and step efficiency, memory-agent comparisons, synchronous and asynchronous planning, and component contributions.Source: §4.1 · p. 6, §4 · p. 6

4.2 Q1: Main Results on ALFWorld

  • Across ALFWorld task types, MemPlan produces shorter trajectories than GPT-4o with domain descriptions and compared methods, while LLM-DP uses fewer online tokens.The ALFWorld comparisons also report improved completion over GPT-4o with domain descriptions; frontier ReAct approaches ceiling-level completion, making steps and tokens the more informative differences.Source: §4.2 · p. 7, Table 1 · p. 7, Table 2 · p. 7

4.3 Q2: TextWorld Memory-Agent Comparison

  • MemPlan achieves the strongest normalized-score performance among the compared TextWorld memory-agent methods, which use different memory modules with the same LLM planning backbone.The comparison covers seven task instances, so the result is evidence on the reported TextWorld setting rather than all text-game distributions.Source: §4.3 · p. 7, Figure 2 · p. 8

4.4 Q3: Synchronous and Asynchronous Planning on Robotouille

  • MemPlan’s lead over frontier ReAct and LLM-DP is more pronounced asynchronously, where delayed effects and stale observations make feedback-driven replanning relevant.Robotouille success is macro-averaged over task configurations after averaging across seeds.Source: §4.4 · p. 8, Table 3 · p. 8

4.5 Q4: Ablation Study

  • Removing placeholders causes structural planning failure, while removing negative memory lets the agent revisit hypotheses already contradicted by feedback.The ablations evaluate seven TextWorld task instances with three seeds per instance.Source: §4.5 · p. 8, Table 4 · p. 9
  • Removing the parser weakens symbolic state and removing the grounder increases invalid commands; uniform-cost hypotheses preserve feasibility but require more interaction steps.Removing both parser and grounder gives the weakest interface variant; graph-memory costs mainly improve search efficiency.Source: §4.5 · p. 8

4.6 Efficiency Analysis

  • MemPlan uses more online tokens than LLM-DP but achieves shorter episodes and higher task completion; its online LLM calls handle parsing and command grounding.MemPlan replans after each executed action, while planner calls do not consume LLM tokens.Source: §4.6 · p. 8, Table 5 · p. 9

4.7 Case Study: Recovering from a Missing Cooking Device

  • In a representative Cooking Hard run, MemPlan uses a placeholder for an unresolved grill, replans after an unrealized action, then grounds bbq_0 to the executable BBQ command.The task requires collecting ingredients, preparation and cooking, assembly, and eating; the reported run completes in 31 steps with normalized score 1.00.Source: §4.7 · p. 9, Figure 3 · p. 9

5 Limitations

MemPlan depends on a supplied, sufficiently complete task-level PDDL domain, uses hard exclusions for refuted hypotheses, and is evaluated only in constrained simulated text environments.

5 Limitations

  • MemPlan assumes an available task-level PDDL domain covering relevant types, predicates, actions, preconditions, and effects; incorrect or incomplete rules can yield invalid symbolic plans.Domain construction is not automated inside the online agent.Source: §5 · p. 10
  • Hard exclusions can prematurely remove a valid candidate when execution feedback is ambiguous or incorrectly parsed.The paper proposes confidence-weighted negative memory as future work.Source: §5 · p. 10
  • Evaluation is limited to simulated text environments with benchmark-specific action spaces and, when available, admissible commands; safety-critical deployment, physical robotics, and unconstrained open-ended settings are unevaluated.Source: §5 · p. 10

6 Conclusion

The conclusion presents memory-conditioned PDDL problem construction as effective for partially observable text environments, and the appendix outlines the experimental details it documents.

6 Conclusion

  • Across ALFWorld, TextWorld, and Robotouille, MemPlan improves task completion and step efficiency over language-only and PDDL-aware baselines.Source: §6 · p. 10
  • MemPlan represents unresolved task information as candidate hypotheses, assigns planner-visible costs through positive graph memory, and excludes execution-refuted assumptions in later planning calls.Source: §6 · p. 10
  • The design keeps the LLM out of online planning and uses it for constrained parsing and grounding while interaction memory becomes part of PDDL problem construction.Source: §6 · p. 10

A Experimental Details

  • The experimental-details appendix covers benchmark scope, episode budgets, metrics, statistical reporting, PDDL and planner settings, hypothesis-cost construction, and online token accounting.Source: A Experimental Details · p. 13

A.1 Environment Roles and Evaluation Scope

The appendix assigns ALFWorld, TextWorld, and Robotouille distinct evaluation roles and describes the scope of each benchmark.

A.1 Environment Roles and Evaluation Scope

  • ALFWorld is the main end-to-end benchmark, covering 134 household-manipulation episodes across six task types, with results by task type and macro-average.The tasks are clean, cool, examine, heat, put, and puttwo.Source: §A.1 · p. 13
  • TextWorld supports comparison with memory-augmented LLM agents and evaluation of cross-task use of the learned language–symbol interface across seven task instances.The instances are grouped as Treasure Hunt Hard, Cooking Hard, and Cleaning.Source: §A.1 · p. 13
  • Robotouille tests synchronous and asynchronous long-horizon planning using 151 task-seed runs, with success macro-averaged across task configurations after averaging within each configuration.The two settings are reported separately, and compared methods share each task's budget and termination rule.Source: §A.1 · p. 13

A.2 Episode Budgets and Evaluation Units

The appendix specifies a shared-budget condition, task-specific success rules, and tables documenting evaluation budgets and TextWorld grouping.

A.2 Episode Budgets and Evaluation Units

  • Compared methods use the same episode budget within each environment and task.Source: §A.2 · p. 13
  • Table 6 reports evaluation runs and episode budgets, while Table 7 reports TextWorld evaluation grouping.Source: Table 6 · p. 13, Table 7 · p. 13
  • Success is defined by task completion within 150 environment steps for ALFWorld, task-dependent limits for TextWorld, and task-specific budgets for Robotouille.Source: §A.2 · p. 13

A.3 Metric Definitions and Statistical Reporting

The appendix defines evaluation measures and statistical reporting, then details the fixed-domain planning setup, memory-conditioned problem construction, graph-based hypothesis costs, and language–symbol interface. It also reports positive-memory sensitivity and specifies what counts toward online LLM-token usage.

A.3 Metric Definitions and Statistical Reporting

  • The appendix defines task success, episode length, TextWorld normalized score, invalid-action rate, repeated hypothesis checks, and planner failures through explicit completion, command-count, scoring, rejection, repetition, and planning criteria.Task success uses episode-budget completion (ALFWorld macro-averages six task types; Robotouille averages task configurations after seeds by setting); episode length counts executed commands, including observation fallbacks. TextWorld scores are computed per instance then aggregated by task group; invalid actions are executed commands rejected by the environment; repeated checks revisit earlier-refuted candidates; planner failures include no plan, timeout, or an unmatched first action without admissible observation replacement.Source: §A.3 · pp. 13–14

A.4 PDDL Domains and Planner

  • MemPlan keeps the PDDL domain fixed and reconstructs the problem file online with current objects, facts, goals, candidate hypotheses, action costs, and exclusions.The domain supplies object types, predicates, action schemas, preconditions, and effects.Source: §A.4 · p. 14

A.5 Example Compiled PDDL Problem

  • Negative-memory refutations become domain-compatible exclusion predicates, while graph-derived costs bias the planner toward more relevant hypothesis checks.A refuted room candidate can be represented by searched(r), blocking later checks of that room.Source: §A.5 · p. 15, §A.6 · pp. 17–18

A.6 Graph-Memory Hypothesis Cost Construction

  • Candidate relevance is computed from cosine similarity between a goal-and-missing-item query and a candidate document built from the candidate’s one-hop positive-memory neighborhood.Similarities are min–max normalized over the current candidate set and converted into PDDL action costs; the costs guide satisficing search but do not guarantee a globally minimum-cost plan.Source: §A.6 · pp. 17–18

A.7 Positive-Memory Hyperparameter Sensitivity

  • All nine positive-memory settings achieved a TextWorld normalized score of 1.00, with mean episode lengths ranging from 31.0 to 34.1 steps.The tested combinations used τ ∈ {0.40, 0.50, 0.60}, λ ∈ {5, 10, 20}, and κbase = 1; task completion also depends on placeholders, negative memory, and the fixed PDDL domain.Source: §A.7 · p. 18

A.8 Online Token Accounting

  • Online LLM-token counts include test-time prompt and completion tokens for parsing and grounding, but exclude fine-tuning, offline domain generation, deterministic PDDL construction, and planner calls.Fine-tuning and other offline costs are reported separately from online interaction costs.Source: §A.8 · p. 18, Table 9 · p. 18

B.1 Unified Interface Model

  • The unified Qwen2.5-7B interface handles observation-to-fact parsing and symbolic-action-to-command grounding, with outputs checked against the domain schema or admissible commands.Training uses 17,000 observation–fact examples and 9,600 symbolic-action–command examples.Source: §B.1 · p. 19, §B.2 · p. 19

B.3 TextWorld Interface Transfer Setting

  • For TextWorld, the frozen interface is fine-tuned only on cook and hunt and then used on other task types without target-task examples.ALFWorld and Robotouille validation splits are diagnostic only, not used for fine-tuning, checkpoint selection, tuning, or test-time adaptation.Source: §B.3 · p. 19, §B.4 · p. 19

B.6 Interface Validation Metrics

The interface is evaluated separately on fact parsing and command grounding, with error categories distinguishing omissions, mismatches, and unsafe grounding. The section also describes baseline configurations and two PDDLEGO+ feedback conditions.

B.6 Interface Validation Metrics

  • Interface validation measures accepted-fact precision, recall, and F1 for parsing, and top-1 admissible-command accuracy for grounding, alongside schema and environment rejection rates.ALFWorld and Robotouille validation examples are diagnostic only and were not used for training, checkpoint selection, prompt or threshold tuning, or test-time adaptation.Source: §B.6 · p. 20

B.7 Interface Error Categories

  • The error taxonomy separates parser omissions and hallucinations, entity and predicate mismatches, grounding mismatches, and cases with no safe grounding.When no safe grounding is available, MemPlan may execute an admissible observation command and update memory only from environment feedback.Source: §B.7 · p. 20

C.1 Baseline Configurations

  • The baseline set spans language-only agents, PDDL-aware methods, and TextWorld memory agents, with prompts, model versions, command access, and stopping rules fixed before evaluation.The supplied passages identify GPT-4o, GPT-4o with domain descriptions, ReAct, LLM-DP, and AriGraph among the comparisons.Source: §C.1 · pp. 20–21

C.2 PDDLEGO and PDDLEGO+

  • PDDLEGO+ is evaluated on the same 134 ALFWorld episodes with either its native fine-grained repair feedback or shared standard textual feedback.Both configurations retain iterative PDDL formalization, planning, execution, and refinement; online PDDL generation and repair calls count toward token usage.Source: §C.2 · p. 21

C.3 Common Evaluation Protocol

The evaluation protocol holds goals, observations, episode limits, and admissible-command access consistent where available, while fixing prompts and decoding settings before testing. Comparisons cover language-only, PDDL-aware, and memory-agent baselines, followed by TextWorld ablations of core components.

C.3 Common Evaluation Protocol

  • Methods share task goals, textual observations, episode limits, and admissible-command access when available, while differing in language-model, memory, domain, and planning use.Prompt templates, model versions, decoding parameters, history windows, and domain-description inputs are fixed before evaluation and not tuned on test episodes.Source: §C.3 · p. 21
  • The comparisons are system-level, and MemPlan specifically assumes a fixed PDDL domain while compiling interaction memory into online PDDL problem construction.This scope differs from methods that generate, complete, or refine planning representations.Source: §C.3 · p. 21

C.4 Language-Only Baselines

  • Language-only baselines generate commands from task context, ReAct also counts reasoning text as online tokens, and LLM-DP uses LLM-generated belief states for symbolic planning.PDDLEGO methods iteratively construct or refine PDDL representations; TextWorld memory baselines use memory to support LLM decision-making.Source: §C.4 · p. 22, §C.5 · p. 22, §C.6 · p. 22

D.1 Ablation Variables

  • TextWorld ablations remove graph-based hypothesis costs, negative memory, placeholders, the observation parser, the action grounder, or both learned interface components.The seven task instances are run with three seeds per variant, for 21 runs per variant and 147 runs total.Source: D Ablation Details · p. 22, §D.1 · p. 23

D.2 Ablation Metrics

The ablation metrics track normalized score, executed steps, repeated checks, planner dead ends, and invalid actions. The grounding case study distinguishes planner-side object identifiers from the command arguments used by the text environment.

D.2 Ablation Metrics

  • Ablation outcomes are measured by normalized score, executed commands, repeated refuted-hypothesis checks, planner dead ends, and environment-rejected commands.Runs that exhaust the interaction budget without completion receive the budgeted step count.Source: §D.2 · p. 23

E Case-Study Grounding Details

  • In the grounding case study, unresolved unknown_grill_1 is resolved to planner object bbq_0, while the executable command uses the surface argument BBQ.The grounding module maps symbolic actions and arguments to a command from the admissible set, so planner object names need not match command text.Source: E Case-Study Grounding Details · p. 23

F Additional Reproducibility Details

The appendix documents compute, released materials, asset licenses, and LLM use; the checklist reports evaluation and training details alongside the work’s scope and limitations.

F Additional Reproducibility Details

  • Approximately 500 GPU-hours were used to fine-tune the interface, and final evaluation campaigns took approximately 21 wall-clock hours.Fast Downward planning ran on CPU, while language–symbol interface calls used the fine-tuned Qwen2.5-7B model.Source: F Additional Reproducibility Details · p. 23
  • The released repository provides code, configuration and PDDL construction scripts, evaluation scripts, and reproduction instructions.It covers experiments using TextWorld, ALFWorld, and Robotouille.Source: F Additional Reproducibility Details · p. 24, §2 · p. 27
  • The appendix lists licenses for the released code and existing environments, models, and planning tools, and notes that some baseline implementations were local.MemPlan code uses the MIT License; Fast Downward uses GPLv3 and Qwen2.5-7B-Instruct uses Apache License 2.0.Source: F Additional Reproducibility Details · p. 24, §2 · p. 26
  • MemPlan uses a fine-tuned interface for observation parsing and action grounding, with no LLM generating or repairing its PDDL domain during evaluation.GPT-4o and ReAct-style baselines use LLMs for online action generation.Source: F Additional Reproducibility Details · p. 24, §2 · p. 27
  • The work is evaluated in simulated text environments and is not evaluated for safety-critical deployment, physical robotics, or decisions about people.The appendix identifies incorrect parsing or grounding, over-reliance on generated symbolic states, and misuse of autonomous planning systems as potential risks.Source: F Additional Reproducibility Details · p. 24, NeurIPS Paper Checklist · p. 25, §2 · p. 25

NeurIPS Paper Checklist

  • Section 4 specifies the main setup, while appendices provide evaluation, training, baseline, compute, and reproduction details.Reported significance procedures include two-sided 95% bootstrap confidence intervals with 10,000 resamples for ALFWorld and Robotouille; TextWorld uses per-instance scores and group averages across seven instances.Source: §2 · pp. 25–26

Related papers

Loading 2610.30379v1…