PaperlayerView exact revision on arXiv

Source-linked AI summary

SGG-ReflAct: Sub-Goal Guided ReflAct with Structured Planning for Reliable Long-Horizon Reasoning

Jaeho Jung, Sung Hoon Jung

arXiv:2609.34548v1cs.AI

TL;DR

Long-horizon agents can drift as ungrounded reasoning compounds, and end-goal reflection may offer little guidance early in a task. SGG-ReflAct anchors reflection to planned sub-goals; it outperforms REFLACT in nearly all settings, while results show plan quality matters to search-based gains.

  • Problem

    In long-horizon tasks, ungrounded steps can compound, while reflecting on a distant end-goal may provide little concrete guidance early in an episode.

    Source: §1 · p. 2

  • Method

    SGG-ReflAct generates a sub-goal plan once and anchors each turn’s reflection to the active sub-goal, with BeamSGG-ReflAct substituting a beam-search planner.

    Source: §1 · pp. 2–3

  • Results

    Across three benchmarks, SGG-ReflAct outperformed REACT, Plan-and-Act, and REFLACT in nearly all settings, including a +14.9 pp gain over REFLACT on ALFWorld for Llama-3.1-8B-Instruct.

    Source: §6 · p. 9

  • Takeaways & Limitations

    Plan quality is fundamental: search benefits require candidate plans to explicitly include the operations needed to achieve the goal.

    Source: §6 · p. 9

    The sub-goal plan is fixed at t = 0, so the method cannot generate new sub-goals mid-episode.

    Source: §6 · p. 10

Abstract

from arXiv · show

Recent advances in reasoning backbones have empowered large language model (LLM)agentstotackle complex, multi-step tasks. However, as reasoning horizons grow, inconsistent internal beliefs induce intermediate errors that cause agents to drift from their goals. This limitation also persists in REFLACT, which reflects only on the end-goal at each step without explicitly considering intermediate sub goals. To address this problem, we propose SGG-ReflAct (Sub-Goal Guided Re flAct), a reasoning backbone that integrates sub-goals generated through a single path LLM planner into the reflection process. We further extend this framework to BeamSGG-ReflAct, which replaces the single-path planner with a beam search based LLM planner for structured plan exploration. We run experiments on ALF World, ScienceWorld, and Jericho with multiple LLM models. SGG-ReflAct out performs REFLACT in nearly all settings, achieving best success rate gains of 14.9 percentage points on ALFWorld and 8.0 percentage points on ScienceWorld with Llama-3.1-8B-Instruct. Our experimental analysis shows that SGG-ReflAct re duces hallucinated actions and achieves its largest gains on procedurally ordered tasks. Furthermore, experimental results with BeamSGG-ReflAct show that the backbone's effectiveness depends on plan quality: explicitly specifying the re quired operations recovers gains that plan searching alone cannot achieve. These results demonstrate that SGG-ReflAct offers a practical and highly effective rea soning backbone, enabling LLM agents to achieve reliable performance in com plex, long-horizon tasks through easy integration.

1 INTRODUCTION

Long-horizon agents can drift when ungrounded reasoning compounds, and reflecting only on a distant end-goal may offer little guidance early in a task. SGG-ReflAct instead anchors reflection to active sub-goals, outperforming REFLACT in nearly all settings and reducing invalid actions.

1 INTRODUCTION

  • The method plans ordered sub-goals once, then anchors each per-turn reflection to the active sub-goal instead of the distant end-goal.The planner is modular, enabling a beam-search planner variant while keeping reflection and acting constant.Source: §1 · pp. 2–3, Figure 1 · p. 2
  • SGG-ReflAct outperforms REACT, Plan-and-Act, and REFLACT in nearly all settings, with gains of +14.9 pp on ALFWorld and +8.0 pp on ScienceWorld for Llama-3.1-8B-Instruct.The evaluations span three models and three interactive text benchmarks.Source: §1 · p. 2
  • Anchoring reflections to active sub-goals reduces invalid-action rate from 43.3% to 29.9% on Llama-3.1-8B-Instruct.The grounding analysis attributes gains to reflection anchoring, not merely adding plans to context.Source: §1 · p. 3
  • Plan quality determines BeamSGG-ReflAct’s effectiveness: plans must specify required operations, and explicit prompting recovers gains that search alone cannot achieve.Inadequate plans can turn gains into degradation.Source: §1 · p. 3

2 RELATED WORK

Related work spans reasoning-action backbones, planning and search, and reflection methods. SGG-ReflAct combines per-turn reflection with sub-goal targets, distinguishing it from methods that add plans without redirecting reflection.

LLM agent reasoning backbones

  • REACT ties thoughts to recent observations, while WKM and KnowAgent add knowledge modules; these approaches leave the core thought-action loop unchanged.The passage argues that such additions help little when the backbone itself produces ungrounded thoughts.Source: §2 · p. 3

Agent planning and search

  • Plan-and-Act executes a high-level plan, whereas SGG-ReflAct uses its fixed plan as a reflection target rather than an action script, adding one planning call.Pre-Act updates plans during execution; other work structures plans as trees, sub-goal orderings, or trajectory rollouts.Source: §2 · p. 3
  • BeamSGG-ReflAct adapts self-evaluation-guided beam search to build sub-goal plans, scoring sequences by feasibility and order rather than answer correctness.Prior search methods are described as not transferring directly to sequential, partially observable environments.Source: §2 · p. 3

Reflection mechanisms

  • Unlike reflection methods that revise past outputs or verify actions after proposal, SGG-ReflAct keeps per-turn reflection and targets the active sub-goal.The passage contrasts this with plan-injection methods that add a plan to context but leave the reflection backbone unchanged.Source: §2 · p. 4

3 METHOD

The method formalizes sub-goal-guided reflection as a two-phase backbone: generate an ordered plan once, then identify the active sub-goal and reflect before acting at each turn. BeamSGG-ReflAct changes only how the plan is built, using self-evaluation-guided search.

3 METHOD

  • The framework formalizes agents as acting from observation history under partial observability, with a reasoning backbone adding a natural-language thought before each action.REACT attends to history, REFLACT reflects on state relative to the end-goal, and SGG-ReflAct reflects on the active sub-goal.Source: §3 · p. 4

3.1 SGG-REFLACT FRAMEWORK

  • The planner runs once at episode start, mapping the initial context and an in-context example to an ordered sequence of high-level sub-goals.The plan is fixed at four sub-goals across environments and costs one additional LLM call per episode.Source: §3.1 · p. 4
  • At every turn, the agent selects the active sub-goal, reflects on its state relative to that target, and produces the action in the same model call.The end-goal remains in the system prompt, while the sub-goal target can be closer to the current state.Source: §3.1 · p. 5

3.2 BEAMSGG-REFLACT: SEARCH-ENHANCED PLANNING

  • BeamSGG-ReflAct replaces only the single-path planner with beam search over candidate sub-goal sequences, using LLM self-evaluation to score prefixes.The search requires at most 42 calls per episode; its reflection-and-acting phase is unchanged.Source: §3.2 · p. 5

4 EXPERIMENTS

The experiments test whether sub-goal-guided per-turn reflection improves long-horizon agents, using three interactive benchmarks and three model capacity levels. The results compare SGG-ReflAct and its beam-search variant with baselines that isolate planning and reflection.

4 EXPERIMENTS

  • The experiments ask whether sub-goal-guided per-turn reflection improves performance and which of the plan and reflection ingredients contributes.Baselines remove one ingredient at a time; tasks are long-horizon, and models span three capacity levels.Source: §4 · p. 6

4.1 BASELINES

  • The baseline design spans systems with neither ingredient, per-turn reflection without sub-goals, and a plan without per-turn reflection.The experiment rationale says baselines remove one ingredient at a time.Source: §4 · p. 6

4.2 BENCHMARKS AND METRICS

  • The evaluation covers ALFWorld, ScienceWorld, and Jericho, using Success Rate across all benchmarks and Average Reward for ScienceWorld and Jericho.The benchmarks vary in domain and planning demands; Jericho’s 20-game set pairs large action spaces with sparse rewards.Source: §4.2 · p. 6

4.3 MODELS AND IMPLEMENTATION

  • The model suite includes GPT-4o, GPT-4o-mini, and resource-constrained, open-weight Llama-3.1-8B-Instruct.The paper identifies Llama-3.1-8B-Instruct as most likely to benefit from explicit sub-goals and focuses its analyses on that model.Source: §4.3 · p. 6

4.4 MAIN RESULTS

  • SGG-ReflAct leads every model group in ScienceWorld SR and all but GPT-4o in AR, where BeamSGG-ReflAct scores 74.9 against 73.5.On Llama-3.1-8B-Instruct, its improvements over REFLACT are +8.0 pp SR and +7.2 pp AR.Source: §4.4 · p. 6
  • With Llama-3.1-8B-Instruct, SGG-ReflAct reaches 50.7% on ALFWorld and exceeds REFLACT by +8.0 pp SR on ScienceWorld; it leads every ALFWorld model group.Table 1 compares performance across benchmarks and models.Source: §4.4 · p. 6, Table 1 · p. 7
  • The reported result is 13.3%, consistent with Kim et al. (2025)’s finding that smaller models struggle to follow a preset plan.Source: §4.4 · p. 7
  • On Jericho, SGG-ReflAct has the best SR in every model group and the best AR on GPT-4o and GPT-4o-mini; BeamSGG-ReflAct never beats the single-path planner.On GPT-4o, SGG-ReflAct reaches 40.0% SR against REFLACT’s 35.0%, while BeamSGG-ReflAct falls to 25.0%.Source: §4.4 · p. 7

5 DISCUSSION

The discussion attributes SGG-ReflAct’s effectiveness to reflection anchored to active sub-goals and to plan content. It reports reduced grounding errors in some settings, particularly on procedurally ordered tasks, while noting that weak plans and search scoring constrain performance.

5.1 REFLACT VS. SGG-REFLACT: TASK-LEVEL SUCCESS AND GROUNDING (C1)

  • On ScienceWorld with Llama-3.1-8B-Instruct, anchoring reflection to the active sub-goal accounts for +6.1 pp of SGG-ReflAct’s +8.0 pp success-rate gain over REFLACT.Adding the plan alone accounts for +1.9 pp; anchoring also adds +3.9 pp average reward, reaching 48.7%.Source: §5.1 · p. 7
  • For Llama-3.1-8B-Instruct on ScienceWorld, SGG-ReflAct lowers overall invalid-action rate from 43.3% with REFLACT to 29.9% and late-stage rate from 71.4% to 49.7%.The reduction is not uniform across models: GPT-4o-mini’s overall rate is effectively unchanged, 32.6% versus 32.7%, and its late-third rate is worse.Source: §5.1 · p. 8
  • SGG-ReflAct’s gains are largest on strict-prerequisite tasks, but it underperforms REFLACT on Lifespan retrieval and all agents score at most 5.0% on Grow.The authors attribute the Lifespan result to over-decomposition and identify the step budget as the primary Grow bottleneck.Source: §5.1 · p. 8

5.2 PLAN CONTENT AS A FUNDAMENTAL DETERMINANT OF BACKBONE PERFORMANCE (C2)

  • Across nine model-benchmark cells, SGG-ReflAct beats BeamSGG-ReflAct in all nine, while BeamSGG-ReflAct beats REFLACT in five.The authors suggest that search cannot help when candidate plans omit necessary steps.Source: §5.2 · pp. 8–9
  • Deficient plans can restate a goal instead of specifying the needed physical operation; on Llama-3.1-8B-Instruct, 14 of 18 Freeze and Melt episodes earned partial reward but none were solved.The authors conclude that search-based reordering cannot supply critical steps that are absent from the plan.Source: §5.2 · p. 9
  • Explicit procedural prompting improved targeted state-change success by up to 16.7 points on GPT-4o, but the Llama single-path planner lost 5.6 points on those episodes.The prompt is given only to the Planner; the authors attribute the Llama result to its inability to turn the added rule into a correctly ordered plan. Search scoring also favors locally consistent prefixes over future completion, and wider beams may preserve plans missing essential operations.Source: §5.2 · p. 9

6 CONCLUSION

SGG-ReflAct plans sub-goals and anchors each reflection to the active sub-goal, outperforming REACT, Plan-and-Act, and REFLACT in nearly all tested settings. The analyses identify reflection anchoring and plan content as important to performance, while the fixed plan and limited episode counts constrain the evaluation.

6 CONCLUSION

  • Across ALFWorld, ScienceWorld, and Jericho, SGG-ReflAct outperformed REACT, Plan-and-Act, and REFLACT in nearly all settings, with gains of +14.9 pp on ALFWorld and +8.0 pp on ScienceWorld using Llama-3.1-8B-Instruct.These were the largest gains, achieved on the smallest model.Source: §6 · p. 9
  • Ablations indicate that improvements stem from anchoring reflection to the active sub-goal rather than merely adding a plan, with grounding gains especially pronounced on procedurally ordered tasks.The analysis links sub-goal anchoring to reduced invalid actions and improved performance on procedurally ordered tasks.Source: §6 · p. 9
  • Plan quality conditions the benefits of search: candidate plans must specify required operations, and improving sub-goal descriptions is a prerequisite for effective plan search.Search alone cannot recover gains when plans merely restate the goal.Source: §6 · pp. 9–10
  • The findings suggest refining reflection targets in reasoning backbones and prioritizing sub-goal content when optimizing planners.Source: §6 · p. 10
  • The plan is fixed at episode start, plan length is set to N = 4 for every task, and some reported margins rely on few episodes.A natural next step is a trigger for generating new sub-goals mid-episode.Source: §6 · p. 10

REPRODUCIBILITY STATEMENT

The evaluation re-implements agents, environments, and code in one codebase under shared conditions. Appendices provide detailed settings, prompts, algorithms, and per-group results.

REPRODUCIBILITY STATEMENT

  • All agents, environments, and evaluation code were re-implemented in one codebase and run under identical conditions.Runs shared a base LLM per group, stop-word set, flat-text in-context learning format, and random seed 42.Source: REPRODUCIBILITY STATEMENT · p. 10
  • Appendices E and D report exact settings, corrected success criteria, and per-task step budgets, while Appendix C gives the beam-search algorithm and hyperparameters.Appendix C also includes self-evaluation scoring values and its prompt.Source: REPRODUCIBILITY STATEMENT · p. 10
  • Appendices N through R list system prompts, in-context examples, and planner prompts; Appendices F and G give per-group results and dispersions behind Table 1 aggregates.Source: REPRODUCIBILITY STATEMENT · p. 10

ETHICS STATEMENT

The study evaluates LLM agents in three simulated text environments and does not involve human subjects, sensitive data, or physical or safety-critical deployment. The authors identify no direct harmful-application path beyond general risks associated with more capable LLM agents.

ETHICS STATEMENT

  • The work evaluates agents only in simulated text environments—ALFWorld, ScienceWorld, and Jericho—with no physical or safety-critical deployment.Source: ETHICS STATEMENT · p. 10
  • The study involves no human subjects or personal or sensitive data and uses publicly released benchmarks under their original licenses.Source: ETHICS STATEMENT · p. 10
  • The method changes how an agent reasons but grants no new capabilities or tool access; the authors see no direct harmful-application path beyond general risks from more capable LLM agents.Source: ETHICS STATEMENT · p. 10

AI USE STATEMENT

The statement discloses assistants' roles in analysis, writing, citation checking, and scripts, while authors verified reported work. The accompanying materials describe the framework comparison, plan-length rationale, beam-search procedure, and episode termination rules.

AI USE STATEMENT

  • Assistants supported analysis framing, robustness checks, script writing, interpretation, citation retrieval and checking, prose editing, and most analysis and plotting scripts; authors reviewed outputs and verified reported numbers against experiment logs.Authors designed, executed, and verified all agent runs and wrote and checked the agent implementations and evaluation harness.Source: AI USE STATEMENT · pp. 10–11

A FRAMEWORK COMPARISON

  • Table 2 compares frameworks by upfront planning, per-turn reflection, and whether the plan anchors that reflection; MPO provides a meta-plan but does not change the reflection target.Source: Table 2 · p. 14, A FRAMEWORK COMPARISON · p. 14

B CHOOSING THE PLAN LENGTH N

  • The shared plan length N = 4 follows ScienceWorld's weighted mean ordered-action-chain length of 3.79 and median and mode of 4, despite task-type values ranging from 2 to 8.The authors use one plan length across tasks rather than tuning it per environment.Source: B CHOOSING THE PLAN LENGTH N · p. 14

C BEAMSGG-REFLACT PLANNING DETAILS

  • BeamSGG-ReflAct scores partial plans with a judge's continuous 0–1 rubric-based rating, multiplies scores across steps, and retains top-K candidates at each depth.The rubric checks ordering, prerequisites, objective completion, and generic descriptions; deterministic candidate generation is sequential and self-evaluation runs in parallel.Source: C BEAMSGG-REFLACT PLANNING DETAILS · pp. 14–16

D STEP BUDGETS AND EPISODE TERMINATION

  • Episodes end on task completion or step-budget exhaustion; ALFWorld and Jericho have 30-step caps, while ScienceWorld uses task-specific budgets.ScienceWorld also ends after ten consecutive actions the environment cannot parse; a valid action resets that counter.Source: D STEP BUDGETS AND EPISODE TERMINATION · p. 16
  • Malformed responses missing the Action: tag consume a step and produce an Error Input observation without advancing the environment state.This rule applies uniformly across all three benchmarks; on ScienceWorld, these responses do not count toward the unparsable-action termination rule.Source: D STEP BUDGETS AND EPISODE TERMINATION · p. 16

E EXPERIMENTAL SETTINGS

The appendix specifies shared evaluation conventions, episode stopping rules, and method exclusions that shape the experimental comparisons.

E EXPERIMENTAL SETTINGS

  • Models decode at temperature 0 with 512-token limits and unchanged weights; success uses progress thresholds in ScienceWorld and Jericho, and ALFWorld’s native binary flag.ScienceWorld success requires at least 0.7 progress and Jericho exactly 1.0; BeamSGG-ReflAct adds at most 42 planner calls per episode.Source: E EXPERIMENTAL SETTINGS · p. 16
  • Reflexion is excluded because its multi-episode reset conflicts with the single-episode setting, while LATS exceeds the one-call, no-environment-interaction budget.MPO is discussed rather than rerun because prior work found that adding meta-plans barely changed performance and slightly lowered Llama-3.1-8B-Instruct results.Source: E EXPERIMENTAL SETTINGS · pp. 16–17
  • Episodes stop on task completion or budget exhaustion; ScienceWorld also stops after ten consecutive unparsable actions and uses task-specific step budgets.ALFWorld and Jericho have 30-step episode caps.Source: Table 4 · p. 17

F ALFWORLD SUCCESS RATE BY TASK TYPE

The ALFWorld task-type breakdown identifies Pick Two & Place as the hardest type across models, with the weakest backbones collapsing on it.

F ALFWORLD SUCCESS RATE BY TASK TYPE

G SCIENCEWORLD SUCCESS RATE BY TASK CATEGORY

ScienceWorld results vary most across methods on state-change tasks, while the task-level breakdown highlights both planner differences and remaining procedural bottlenecks.

G SCIENCEWORLD SUCCESS RATE BY TASK CATEGORY

  • The state-change category has the widest spread between methods, while Grow stays near zero across agents and models because its bottleneck is the step budget.Source: G SCIENCEWORLD SUCCESS RATE BY TASK… · p. 17
  • Figure 3 splits state-change into its task types and separates paint-mixing; BeamSGG-ReflAct’s nonzero Other rate comes from four of 47 episodes, and no agent solves life-stage episodes.Source: G SCIENCEWORLD SUCCESS RATE BY TASK… · p. 17
  • Table 6 reports success rates for all three models across nine categories grouping ScienceWorld’s 24 task types, with overall rates and category dispersion matching Table 1.Source: Table 6 · p. 18
  • On Melt, BeamSGG-ReflAct solves one of nine episodes while SGG-ReflAct solves none; the reported difference is driven by stating the required procedure in the plan, not planner identity.Source: G SCIENCEWORLD SUCCESS RATE BY TASK… · p. 18

H REFLECTION ANCHORING: REFLACT+PLAN VS. SGG-REFLACT (EXAMPLE)

In a matched ScienceWorld example, SGG-ReflAct solves a mercury-boiling task that REFLACT+Plan fails, with the contrast centered on per-turn reflection instructions.

H REFLECTION ANCHORING: REFLACT+PLAN VS. SGG-REFlACT (EXAMPLE)

I ADDITIONAL GROUNDING AND PLAN-QUALITY ANALYSES

These analyses test whether lower invalid-action rates reflect grounding, examine how procedural prompting shifts performance across ScienceWorld categories, and compare planner variants and trajectory lengths. They also identify unresolved limits in interpreting the prompting ablation.

Grounding analysis

  • 32.5% for SGG-ReflAct and 32.3% for BeamSGG-ReflAct, versus 39.0% for REACT and 38.8% for REFLACT, under the broader hallucinated-action rate.The authors treat invalid-action rate as a proxy and leave fine-grained annotation of syntax-versus-grounding failures to future work.Source: I ADDITIONAL GROUNDING AND… · p. 20

Ordering-rule effects

  • The ordering rule’s effects vary: three of six model/planner pairs change primarily on 36 targeted state-change episodes, while the other three change mostly elsewhere.Table 8 compares base and +proc across nine ScienceWorld categories, including pooled state-change and chemistry-mixing categories.Source: I ADDITIONAL GROUNDING AND… · p. 20, Table 8 · p. 21

Category volatility

  • For GPT-4o-mini Measurement, the single-path planner rises from 10.0% to 50.0%, while the Beam Search Planner falls from 45.0% to 15.0%.The authors attribute this reversal to the category’s 20-episode sample, where a handful of flipped episodes can move the rate by ten points or more.Source: I ADDITIONAL GROUNDING AND… · p. 21

Interpretation and limitations

  • The ablation cannot distinguish whether the ordering rule teaches a specific state-change fact or improves general planning competence.The authors leave this distinction for future work.Source: I ADDITIONAL GROUNDING AND… · p. 21

Planner search configurations

  • At comparable planner budgets, greedy and beam search agree on six of nine categories and differ by at most four points on the others, except chemistry mixing.On chemistry mixing, the single-path planner reaches 42.3%, versus 3.8% for Beam Search and 7.7% for greedy; Table 9 compares the three planners at comparable budgets.Source: Table 9 · p. 21, I ADDITIONAL GROUNDING AND… · p. 21

Trajectory length

  • On shared solved episodes, SGG-ReflAct’s mean steps to success remain within about one step of REFLACT across models, though Jericho results are inconclusive.The comparison covers episodes solved by both methods; Table 10 defines n as the number of commonly solved episodes.Source: I ADDITIONAL GROUNDING AND… · p. 21, Table 10 · p. 22

J GROUNDING: A HALLUCINATED-ACTION CASCADE (EXAMPLE)

This section presents examples of reflections using active sub-goals and observed state to guide sequential task actions, alongside planner prompts with ordered milestones and task-specific constraints. It also documents grounding, ablation, and procedural-prompting analyses, action-count and invalid-action measures, and shared prompts and in-context examples; the supplied captions do not provide outcome values.

J GROUNDING: A HALLUCINATED-ACTION CASCADE (EXAMPLE)

  • On the same violet-paint task, REFLACT repeats rejected commands, while SGG-ReflAct uses sub-goals to ground its actions and succeeds with 0 rejected actions versus 9 for REFLACT.SGG-ReflAct gathers materials and then mixes them, finishing in eight steps; REFLACT fixates on unparseable phrasing and fails to recover.Source: J GROUNDING: A HALLUCINATED-ACTION… · p. 22

K ACTIONS PER EPISODE

  • NoThinking and Plan-and-Act execute 3.7 to 7.5 more actions per episode than SGG-ReflAct, while BeamSGG-ReflAct takes as few or fewer actions yet solves less.Table 11 covers 211 ScienceWorld episodes; the passage cautions that fewer actions alone do not indicate efficiency.Source: K ACTIONS PER EPISODE · p. 23, Table 11 · p. 23

L INVALID-ACTION RATE FOR ALL AGENTS

  • Table 12 reports invalid-action rates for six agents across three models, overall and by early, middle, and late episode thirds; lower rates mean fewer rejected actions, not necessarily better performance.The thirds pool equal portions of each episode’s executed actions.Source: L INVALID-ACTION RATE FOR ALL AGENTS · p. 23, Table 12 · p. 24

3. Find the heat source to melt the caesium

  • On melt-caesium, BeamSGG-ReflAct succeeds by specifying “find a heat source” and “apply heat,” whereas SGG-ReflAct’s plan restates the goal without naming an executable transformation.Without an explicit apply-heat step, SGG-ReflAct wanders after locating the blast furnace and exhausts its step budget without melting caesium.Source: §4 · p. 24

3. Cause the mercury to boil or combust

  • On boil-mercury, SGG-ReflAct’s concise plan succeeds while BeamSGG-ReflAct’s extra container and heat-source steps mislead the agent and prevent the outcome.The examples indicate that naming a missing operation can help, while unnecessary decomposition can hurt; on freeze-tin, BeamSGG-ReflAct also gives the wrong-direction instruction to apply heat.Source: §4 · p. 24, §3 · pp. 24–25

N AGENT PROMPTS (ALFWORLD)

  • The ALFWorld SGG-ReflAct and BeamSGG-ReflAct prompts share the same agent instruction, which asks agents to identify the active sub-goal, reflect on progress toward it, and then act.ScienceWorld uses the same instruction; Jericho adds a role prefix, and the appendix reproduces prompts verbatim from source code.Source: N AGENT PROMPTS (ALFWORLD) · pp. 25–26

O SGG-REFLACT IN-CONTEXT EXAMPLES

  • The in-context examples illustrate agents tracking sub-goals through locating, obtaining, transporting, and placing an object, with a separate cleaning step in a pick-clean-place task.The examples are single-shot and shared with BeamSGG-ReflAct; ALFWorld provides one example per task type.Source: O SGG-REFLACT IN-CONTEXT EXAMPLES · pp. 26–27
  • For the lettuce task, the agent marks cleaning complete, carries the clean lettuce to the dining table, and places it there.The example includes an observation confirming the placement.Source: O SGG-REFLACT IN-CONTEXT EXAMPLES · p. 28
  • For the heating task, the agent follows sub-goals to find an egg, heat it in the microwave, carry it to the dining table, and place it.The example shows the egg being heated and placed after the agent locates it on a countertop.Source: O SGG-REFLACT IN-CONTEXT EXAMPLES · pp. 28–29
  • For the cooling task, the agent finds a pan on a stove burner, cools it in the fridge, returns it to a burner, and places it.The example’s sub-goals explicitly separate obtaining, cooling, transport, and placement.Source: O SGG-REFLACT IN-CONTEXT EXAMPLES · pp. 29–30
  • For the two-credit-card task, the agent places the first card in the dresser before returning for and placing a second card.The plan makes obtaining and placing each card a separate sequence of sub-goals.Source: O SGG-REFLACT IN-CONTEXT EXAMPLES · pp. 30–31
  • For the bowl-examination task, the agent locates and obtains a bowl, searches for a desk lamp, then uses the lamp for the examination step.The supplied example ends with the lamp turning on; it does not show a subsequent examination observation.Source: O SGG-REFLACT IN-CONTEXT EXAMPLES · pp. 31–32
  • In the chemistry example, the plan lists gathering materials, mixing green paint, and focusing on it after teleporting to the art studio.Only the initial teleport action and observation are shown in this excerpt.Source: O SGG-REFLACT IN-CONTEXT EXAMPLES · p. 32
  • In the art-studio example, the agent gathers blue and yellow paint, mixes them, then focuses on the resulting green paint.Each reflection tracks the current sub-goal and the agent’s state before choosing an action.Source: O SGG-REFLACT IN-CONTEXT EXAMPLES · pp. 32–33
  • In the castle-rescue example, the agent obtains a sword, enters the castle, reaches the princess’s bedroom, and wakes her to finish the task.The reflections mark sub-goals complete and use available actions to progress toward the next one.Source: O SGG-REFLACT IN-CONTEXT EXAMPLES · pp. 33–34

P ALFWORLD PLANNER PROMPTS AND IN-CONTEXT EXAMPLE

  • The ALFWorld planner requires four ordered, outcome-focused sub-goals, and its spray-bottle example progresses from locating and obtaining the object to placing it on the toilet.The prompt orders sub-goals by dependencies and makes the final sub-goal directly achieve the task; beam search generates candidates one step at a time.Source: P ALFWORLD PLANNER PROMPTS AND… · pp. 34–35

Q SCIENCEWORLD PLANNER PROMPTS AND IN-CONTEXT EXAMPLE

  • The ScienceWorld planner orders green-paint milestones as reaching the materials, gathering them, mixing them, and focusing on the result, with focus placed after obtaining the object.Its prompts require task-specific ordering and constrain beam-search candidates to respect prerequisites and place focus after finding the object.Source: Q SCIENCEWORLD PLANNER PROMPTS AND… · p. 36
  • For state-change tasks, +proc orders obtaining and focusing on the substance before reaching and applying heat or cold, excludes chemistry and paint tasks, and adapts beam rules for single-path prompts.The prompts explicitly reject a vague state-change sub-goal and require focusing on the substance as step two.Source: Q SCIENCEWORLD PLANNER PROMPTS AND… · p. 37

R JERICHO PLANNER PROMPTS AND IN-CONTEXT EXAMPLE

  • Jericho prompts require exactly four dependency-ordered outcome milestones, generic descriptions rather than invented specifics, and a final milestone that achieves the objective; the example ends by waking the princess.The one-shot example progresses from a useful starting-area item through reaching the princess to completing the rescue.Source: R JERICHO PLANNER PROMPTS AND… · pp. 37–38

S ABLATION, GROUNDING, AND PROCEDURAL-PROMPTING FIGURES/TABLES

  • The materials cover reflection-anchoring ablation, invalid-action rates by episode third, bucketed ScienceWorld success rates, and base-versus-+proc planner rates, but the captions give no cell values.Figure 3 covers ten task buckets and 211 episodes; Table 14 distinguishes 36 targeted state-change episodes from the overall set.Source: Table 13 · p. 40, Figure 2 · p. 40, Figure 3 · p. 40
Loading 2609.34548v1…