Source-linked AI summary
Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks
Lutfi Eren Erdogan, Nicholas Lee, Sehoon Kim, Suhong Moon, Hiroki Furuta, Gopala Anumanchipalli, Kurt Keutzer, Amir Gholami
TL;DR
Long-horizon agents struggle to maintain coherent strategies while translating high-level goals into executable actions. Plan-and-Act separates planning from execution and uses scalable synthetic data generation to train the Planner; it achieves a state-of-the-art 53.94% success rate on WebArena-Lite.
Problem
Long-horizon agents struggle to decompose high-level goals and maintain coherent strategies while adapting across many steps.
Method
Plan-and-Act uses a Planner for structured high-level plans, an Executor for environment-specific actions, and synthetic data generated from successful trajectories plus augmented examples.
Results
53.94% success rate on WebArena-Lite, achieving a state-of-the-art result for web navigation.
Takeaways & Limitations
Synthetic data generation can rapidly produce training examples and support state-of-the-art web-navigation performance with supervised fine-tuning.
Takeaways & Limitations
Trajectory generation depends on a baseline model that can successfully complete web tasks, especially when datasets lack training data.
Abstract
from arXiv · showhide
Large language models (LLMs) have shown remarkable advancements in enabling language agents to tackle simple tasks. However, applying them for complex, multi-step, long-horizon tasks remains a challenge. Recent work have found success by separating high-level planning from low-level execution, which enables the model to effectively balance high-level planning objectives and low-level execution details. However, generating accurate plans remains difficult since LLMs are not inherently trained for this task. To address this, we propose Plan-and-Act, a novel framework that incorporates explicit planning into LLM-based agents and introduces a scalable method to enhance plan generation through a novel synthetic data generation method. Plan-and-Act consists of a Planner model which generates structured, high-level plans to achieve user goals, and an Executor model that translates these plans into environment-specific actions. To train the Planner effectively, we introduce a synthetic data generation method that annotates ground-truth trajectories with feasible plans, augmented with diverse and extensive examples to enhance generalization. We evaluate Plan-and-Act using web navigation as a representative long-horizon planning environment, demonstrating a state-of-the-art 57.58% success rate on the WebArena-Lite benchmark as well as a text-only state-of-the-art 81.36% success rate on WebVoyager.
1. Introduction
Plan-and-Act addresses the difficulty of planning long-horizon agent tasks by separating high-level planning from low-level execution and training the Planner with scalable synthetic data. In web navigation, it reports a 53.94% WebArena-Lite success rate.
- Motivation: Long-horizon agents struggle to decompose user goals into actionable steps and maintain coherent strategies as tasks become more complex.These difficulties are amplified by limited high-quality training data for effective planning.
- Motivation: Single-model approaches must balance high-level strategy with low-level execution, often losing sight of objectives and behaving inconsistently.
- Framework: Plan-and-Act separates planning and execution into a PLANNER that generates high-level plans and an EXECUTOR that translates them into environment-specific actions.The architecture can also feed environment feedback back to the Planner for replanning.
- Synthetic data: The training pipeline grounds plans in successful action trajectories and synthetically augments them with diverse planning examples.This approach targets the scarcity of real-world planning data without requiring manual annotation or a sandbox environment.
- Evaluation: 53.94% WebArena-Lite success rate is reported as a state-of-the-art result for web navigation.
2. Related Work
Prior web-agent research uses hierarchical systems, synthetic or on-policy trajectory collection, and other enhancements such as HTML pretraining, vision, or reinforcement learning. Plan-and-Act emphasizes a simpler two-agent framework and planning-data generation for training open-source LLMs.
- Web agents: Web-agent research has developed systems for navigating pages through GUI or API interaction alongside dedicated datasets and benchmarks.
- Planning: Several prior approaches use hierarchical planning, but rely on prompting closed-source models such as GPT-4o.Plan-and-Act instead provides a systematic way to generate training data for open-source LLMs on web tasks.
- Planning: Plan-and-Act uses a two-agent framework, whereas cited systems include tree-like planning, six-agent infrastructure, or separate refinement and skill-discovery modules.
- Synthetic data: Other web-agent data methods collect diverse trajectories through more complex techniques and external simulators, but generally do not produce planning data.
- Adjacent work: Related robotics and embodied-agent work uses hierarchical LLM agents but does not provide a framework for synthetic-data collection or generation for open-source LLM training.
- Web agents: Other web agents augment pretrained models with HTML understanding, multimodal visual understanding, or reinforcement learning.
- Synthetic data: Synthetic data generation has been used broadly to augment LLM training, including through Self-Instruct and Alpaca-style approaches.
- Trajectory collection: Web-agent training commonly collects on-policy trajectories in reusable environments and filters or expands them using failed instances.
3. System Architecture
Plan-and-Act assigns strategic decomposition to a PLANNER and concrete environment interaction to an EXECUTOR. Dynamic replanning updates the strategy after actions, addressing static-plan weaknesses when web environments change.
- Motivation: A single model handling long-horizon planning and multiple low-level actions faces cognitive load that can produce suboptimal or inconsistent behavior.
- Architecture: The PLANNER makes strategic decisions while the EXECUTOR translates them into concrete environment actions.This separation lets each component focus on its core responsibility.
- Architecture: For web tasks, the PLANNER decomposes a query into manageable steps and the EXECUTOR converts those steps into clicks or typed inputs using HTML observations.
- PLANNER: The PLANNER provides a structured roadmap that guides execution while retaining flexibility for the EXECUTOR.
- EXECUTOR: The EXECUTOR runs the Planner’s steps by calling tools, retrieving data, or changing the environment as required.
- Static planning: Static plans are vulnerable to dynamic content and unexpected failures because the EXECUTOR may lack information available only after execution.
- Dynamic replanning: Dynamic replanning has the PLANNER update the strategy after each EXECUTOR step using the current state, prior plans, and actions.
- Dynamic replanning: The evolving plan can retain newly discovered information, such as the identity of a top contributor, for subsequent actions.
4. Synthetic Data Generation
The paper builds synthetic planning data by grounding high-level plans in executable trajectories and expanding them with diverse query-plan examples. The pipeline addresses limited off-the-shelf planning ability, the cost of manual annotation, and the scarcity of planner data.
- 14.21% WebArena-Lite success with PLAN-AND-ACT improved on the 9.85% off-the-shelf baseline but remained far from satisfactory.
- Prompting and in-context examples help on simple tasks but do not resolve performance on non-trivial queries.
- Manual planner and executor data collection is costly and time-consuming because it requires annotating plan breakdowns, HTML states, and desired web actions.
- 4.2. Grounded Plan Generation: Grounded plan generation reverse-engineers structured plans from completed trajectories, linking each high-level step to explicit low-level actions so plans match executable behavior.
- Synthetic replanning data supplies original plans and partial trajectories for queries whose required decisions depend on observations available during execution.
- 4.3. Synthetic Plan Expansion: 10,000 additional user query-plan pairs were generated with GPT-4o in under an hour, increasing diversity while addressing planner-data imbalance and overfitting.
5. Results
PLAN-AND-ACT results show that explicit, well-trained planning improves web-agent success across Executors, with further gains from synthetic data, dynamic replanning, and Chain-of-Thought reasoning. The framework reaches 53.94% on WebArena-Lite and outperforms prior open-source work on text-only WebVoyager.
- Static PLANNER Results: 29.63% success rate is achieved by adding a properly trained PLANNER to a base EXECUTOR, up from 9.85% without planning.The improvement is reported as evidence that explicit planning bridges high-level intentions and low-level actions.
- Static PLANNER Results: Synthetic data expansion improves performance incrementally, with 10,000 generated plans adding approximately 10 percentage points and targeted examples adding 4–5 points.Executor scaling with more training data shows diminishing returns after the initial 1,113 examples, suggesting plan quality is a stronger bottleneck.
- REPLANNING RESULTS: 53.94% WebArena-Lite accuracy is achieved with dynamic replanning, improving 10.31% over the static PLANNER and surpassing WebRL-3.1-70B by 4.84%.Dynamic planning enables the PLANNER to refine and improve plans during task execution.
- REPLANNING RESULTS: 44.24% accuracy is achieved with an unfinetuned base EXECUTOR when supplied with a high-quality dynamic plan, a 34.39% improvement.This result indicates that dynamic plan quality can substantially improve performance even without Executor fine-tuning.
- CHAIN OF THOUGHT RESULTS: 4.36% performance improvement from adding Chain-of-Thought reasoning establishes a new WebArena-Lite state of the art.An 8B model using CoT performs on par with the non-CoT 70B model on the same data.
- WebArena and WebVoyager Results: PLAN-AND-ACT performs better or on-par with most prior work on full WebArena and sets a new text-only SOTA on WebVoyager.The WebVoyager result includes an 8B model outperforming previous open-source models and a 32B model outperforming all prior work.
6. Conclusion
PLAN-AND-ACT separates planning from execution and uses scalable synthetic data generation to improve long-horizon web agents. The framework achieves strong web-navigation performance while exposing limitations in static planning, data requirements, and replanning efficiency.
- Separating the PLANNER from the EXECUTOR improves alignment between user queries and executable actions, supporting consistency and adaptability in dynamic environments.
- 15,000 synthetic training examples were generated in under an hour with GPT-4o, whereas comparable environment-interaction data collection would take days or weeks.
- PLAN-AND-ACT combines synthetic data generation, plan expansion, targeted failure refinement, and dynamic replanning to improve WebArena-Lite success rates.
- The framework demonstrates potential even with a standard WebArena-Lite-style EXECUTOR, leaving executor improvements such as chain-of-thought reasoning for future work.
- The current approach depends on a baseline model that can successfully complete web tasks, especially for datasets without training data such as WebVoyager.
- Dynamic replanning after every action can be inefficient and slow performance, motivating executor-controlled replanning or delegated subagents.
A.1. Planner and Executor Output Examples
The examples show Planner outputs as ordered, high-level steps and Executor behavior as concrete interaction sequences for web tasks. Plans specify navigation, inputs, report generation, and result analysis.
- Planner and Executor Output Examples: A directions task is decomposed into initiating directions, entering the starting location, entering the destination, and searching for a route.
- Planner and Executor Output Examples: The final directions step analyzes the results page to determine the estimated driving time between the specified locations.
- Planner and Executor Output Examples: A shipping-report task includes setting the requested date range, generating the report, verifying visible details, and exiting with a completion message.
- Planner and Executor Output Examples: The report workflow generates the shipping report by clicking “Show Report,” then confirms that complete report details are visible before completion.
A.2.1. QUERY REFINEMENT
The query-refinement example shows a replanner responding to failed retrieval by replacing an unsuccessful search with a broader query and then restructuring the downstream workflow.
- QUERY REFINEMENT: The example illustrates refinement from a broad “analyze the search results” instruction into concrete actions after execution failure.
- QUERY REFINEMENT: After refinement, the plan searches for libraries, reviews results, selects a library, and initiates directions.
- QUERY REFINEMENT: The refined workflow continues by entering Carnegie Museum of Art as the origin and the selected CMU library as the destination.
- QUERY REFINEMENT: The plan specifies walking directions and analyzes the resulting travel time between Carnegie Museum of Art and the selected library.
- QUERY REFINEMENT: When “Library at CMU” returns no results, the replanner changes the query to “Library near CMU” to improve retrieval.
A.2.2. ANALYZING SEARCH RESULTS AND MEMORY (AGGREGATING RESULTS ACROSS MULTIPLE PAGES)
The example demonstrates refining broad report-analysis instructions into structured extraction across multiple visible pages. The executor progressively scrolls, records monthly counts, and formats the final response.
- ANALYZING SEARCH RESULTS AND MEMORY (AGGREGATING RESULTS ACROSS MULTIPLE PAGES): The report task requires presenting successful-order counts for January through December 2022 in MM:COUNT format.
- ANALYZING SEARCH RESULTS AND MEMORY (AGGREGATING RESULTS ACROSS MULTIPLE PAGES): The report workflow sets the 2022 date range, selects monthly aggregation, generates the report, and then extracts its visible results.
- ANALYZING SEARCH RESULTS AND MEMORY (AGGREGATING RESULTS ACROSS MULTIPLE PAGES): The replanner refines the broad instruction to analyze the report into step-by-step structured data extraction.
- ANALYZING SEARCH RESULTS AND MEMORY (AGGREGATING RESULTS ACROSS MULTIPLE PAGES): The initial visible rows yield counts 01:11, 02:16, 03:14, and 04:8.
- ANALYZING SEARCH RESULTS AND MEMORY (AGGREGATING RESULTS ACROSS MULTIPLE PAGES): Scrolling reveals additional counts: 05:8, 06:13, 07:9, 08:8, 09:10, and 10:4.
- ANALYZING SEARCH RESULTS AND MEMORY (AGGREGATING RESULTS ACROSS MULTIPLE PAGES): A final scroll retrieves the remaining counts 11:11 and 12:10.
A.3. Planner Prompt
The Planner Prompt instructs a model to analyze the user query and initial HTML state, then generate a structured high-level plan for an Executor. Plans group related actions into specific logical steps while avoiding unnecessary low-level interaction details.
- A.3.1. PLANNER SYSTEM PROMPT: The Planner analyzes the user query and initial HTML state to generate a structured, step-by-step global plan.
- A.3.1. PLANNER SYSTEM PROMPT: The required output starts with “## Step 1,” increments step numbers, and includes Reasoning and Step sections for every step.
- A.3.1. PLANNER SYSTEM PROMPT: Each plan step contains a high-level justification and a concise description of the logical work being undertaken.
- A.3.1. PLANNER SYSTEM PROMPT: Plans should combine related actions into logical units while preserving strict action order and avoiding unnecessary granularity.
- A.3.1. PLANNER SYSTEM PROMPT: Step descriptions should specify intended outcomes and concrete parameters rather than UI mechanics such as individual clicks or typing actions.
- A.3.1. PLANNER SYSTEM PROMPT: The Planner may include conditional instructions, but they should remain clear and avoid overly complex or ambiguous guidance.
- A.3.1. PLANNER SYSTEM PROMPT: The initial HTML grounds the plan in the webpage context, and the first-step reasoning must explicitly observe relevant elements, properties, and interactions.
A.4. Executor Prompt
The Executor Prompt defines the Executor as the low-level action model: it uses the global plan, prior trajectory, and current HTML to choose the next web action. The plan serves as a roadmap for locating the current task position and progressing toward completion.
- A.4. Executor Prompt: The prompt is an addition to the WebArena-Lite executor prompt, where each user-assistant pair represents an HTML-action round.
- A.4.1. EXECUTOR SYSTEM PROMPT: It translates high-level plan steps into concrete actions such as clicking, typing, selecting, or other web interactions.
- A.4.1. EXECUTOR SYSTEM PROMPT: The Executor receives the task instruction, global plan, previous action trajectory, and current HTML state.
- A.4.1. EXECUTOR SYSTEM PROMPT: Its goal is to output the next immediate environment action that advances completion of the assigned web task.
- A.4.1. EXECUTOR SYSTEM PROMPT: The Global Plan is a structured, step-by-step roadmap whose steps contain reasoning and high-level actions.
- A.4.1. EXECUTOR SYSTEM PROMPT: The Executor identifies its current plan position by consulting the previous trajectory and current webpage observation.
A.5. Plan Data Annotator Prompt
The Plan Data Annotator Prompt converts action trajectories into structured global plans by grouping sequential actions into specific logical steps. It preserves exact action mappings so the Executor can align plans with executable behavior.
- A.5.1. PLAN DATA ANNOTATOR SYSTEM PROMPT: The annotator receives the initial HTML state and a sequential web-navigation trajectory as inputs for plan construction.
- A.5.1. PLAN DATA ANNOTATOR SYSTEM PROMPT: The required format includes Reasoning, Description, Step, and Actions under numbered plan steps beginning with “## Step 1.”
- A.5.1. PLAN DATA ANNOTATOR SYSTEM PROMPT: The Description field must copy trajectory action descriptions exactly, including comments and corresponding do functions.
- A.5.1. PLAN DATA ANNOTATOR SYSTEM PROMPT: The Step field states the task-level objective, while the Actions field lists the exact trajectory indices assigned to that step.
- A.5.1. PLAN DATA ANNOTATOR SYSTEM PROMPT: Each plan step groups one or more trajectory actions into a broader logical unit without focusing on fine-grained UI interactions.
- A.5.1. PLAN DATA ANNOTATOR SYSTEM PROMPT: Assigned actions must remain sequential and cannot skip or misalign the trajectory order.
- A.5.1. PLAN DATA ANNOTATOR SYSTEM PROMPT: The annotator should make steps specific and outcome-oriented, while leaving translation of granular interactions to the Executor.
- A.5.1. PLAN DATA ANNOTATOR SYSTEM PROMPT: Scrolling and similar implementation details should be abstracted into goals such as locating or finding an item.
A.6. Synthetic Plan Generator Prompt
The Synthetic Plan Generator creates new query–HTML–plan examples from seed data while grounding them in feasible website states. It emphasizes diversity without copying examples or inventing unsupported website capabilities.
- A.6. Synthetic Plan Generator Prompt: It uses seed examples to generate new data and receives the requested number of examples in the user message.
- A.6.1. SYNTHETIC PLAN GENERATOR SYSTEM PROMPT: The generator produces synthetic training pairs consisting of a user query, an example initial HTML-state index, and a high-level global plan.
- A.6.1. SYNTHETIC PLAN GENERATOR SYSTEM PROMPT: Each generated example selects an existing initial HTML state and must include a detailed first-step observation grounded in that state.
- A.6.1. SYNTHETIC PLAN GENERATOR SYSTEM PROMPT: Global plans describe high-level task steps that may encompass multiple low-level actions without becoming step-by-step interaction instructions.
- A.6.1. SYNTHETIC PLAN GENERATOR SYSTEM PROMPT: The generator formats each plan as numbered steps containing Reasoning and Step fields.
- A.6.1. SYNTHETIC PLAN GENERATOR SYSTEM PROMPT: Another Executor agent translates the generated high-level plan into granular web actions.
- A.6.1. SYNTHETIC PLAN GENERATOR SYSTEM PROMPT: Generated queries should be diverse and grounded in the capabilities, elements, and actions represented by the example websites.