PaperlayerView exact revision on arXiv

Source-linked AI summary

Hierarchical Prompting Assists Large Language Model on Web Navigation

Abishek Sridhar, Robert Lo, Frank F. Xu, Hao Zhu, Shuyan Zhou

arXiv:2305.14257v3cs.CLcs.AIcs.LG

TL;DR

Prior prompting work gave little attention to encoding states in diverse, lengthy interactive tasks. ASH summarizes observations before the actor predicts actions, and on Webshop it improved task success over the previous state-of-the-art prompting approach by 6.8%.

  • Problem

    Prior prompting research focused on action-space shaping, while effective state encoding received little attention despite its importance in diverse, lengthy environments.

    Source: §1 · p. 1

  • Method

    ASH uses a dedicated SUMMARIZER to condense raw observations into goal-relevant representations, then an ACTOR predicts actions from summarized interaction history.

    Source: §1 · pp. 1–2

  • Results

    6.8% higher task success rate than the previous state-of-the-art prompting approach (29% relative gain); ASH also surpassed REACT in average score and success rate on Webshop.

    Source: §1 · p. 2, §5 · p. 3, §7 · p. 5

  • Takeaways & Limitations

    On Webshop, ASH excels on complex tasks with lengthy trajectories, reduces LLM hallucinations, and mitigates the need for explicit reasoning.

    Source: §7 · p. 5

    The approach was not tested on more realistic web-navigation benchmarks because none were available when the work was completed.

    Source: Limitations · p. 5

Abstract

from arXiv · show

Large language models (LLMs) struggle on processing complicated observations in interactive decision making tasks. To alleviate this issue, we propose a simple hierarchical prompting approach. Diverging from previous prompting approaches that always put the full observation (e.g. a web page) to the prompt, we propose to first construct an action-aware observation which is more condensed and relevant with a dedicated SUMMARIZER prompt. The ACTOR prompt then predicts the next action based on the summarized observation. While our method has broad applicability, we particularly demonstrate its efficacy in the complex domain of web navigation where a full observation often contains redundant and irrelevant information. Our approach outperforms the previous state-of-the-art prompting mechanics by 6.2% on task success rate, demonstrating its potential on interactive decision making tasks with long observation traces.

1 Introduction

Interactive tasks require sequential actions based on both the environment and a goal, but prior prompting work focused more on action-space design than on encoding diverse, lengthy states. The paper introduces ASH, which summarizes observations before action prediction, and reports higher Webshop task success than the previous state-of-the-art prompting approach.

1 Introduction

  • Interactive tasks such as household duties and web navigation require sequential actions based on the environment state and a natural-language objective.Source: §1 · p. 1
  • Prior work shaped action spaces or coordinated reasoning and acting, while effective state encoding received little attention despite its importance in diverse, lengthy environments.Source: §1 · p. 1
  • ASH uses a SUMMARIZER to create goal-relevant observations and an ACTOR to predict the next action from trajectory history.The hierarchical design avoids sending all information to a single prompt.Source: §1 · pp. 1–2
  • 6.8% higher task success rate than the previous state-of-the-art prompting approach (29% relative gain) on Webshop.The authors focus on product purchasing in this real-world-scale e-commerce simulator.Source: §1 · p. 2

2 The Web Navigation Task

Web navigation is framed as reaching a state specified by a natural-language goal through a sequence of actions. The agent acts from its interaction history, and the final state is scored against the goal state.

2 The Web Navigation Task

  • The task is to reach a target environment state defined by a natural-language instruction, starting from an initial state and observation.Source: §2 · p. 2
  • At each time step, the agent chooses an action using the interaction history of observations and prior actions.Source: §2 · p. 2
  • After a stop action, task score is the similarity between the final state and target state, on a scale from zero to one.Source: §2 · p. 2

3 ASH Prompting

ASH separates observation summarization from action prediction: a stateless SUMMARIZER condenses each observation using the previous action, and a stateful ACTOR predicts actions from summarized history. Dedicated, scenario-specific instructions guide the summarizer, while the actor receives the natural-language instruction and interaction history.

3.1 SUMMARIZER

  • The paper illustrates the summarizer prompt with history, raw observation, and chain-of-thought instruction alongside the summarizer’s output.Source: Figure 2 · p. 2

3 ASH Prompting

  • ASH decomposes action prediction into a stateless SUMMARIZER stage and a stateful ACTOR stage.The summarizer produces a condensed observation from the current observation and previous action; the actor predicts the next action using updated history.Source: §3 · p. 2

3.1 SUMMARIZER

  • The SUMMARIZER uses in-context examples to retain relevant information and remove irrelevant content from the current observation.Source: §3.1 · p. 2
  • Its prompt supplies the previous action, instruction, current raw observation, and customized step-by-step summarization guidance.Source: §3.1 · pp. 2–3
  • Scenario-specific examples guide tasks such as checking product desirability and filtering irrelevant options on a product page.The paper notes that other instructions cover improving search queries and compressing search results.Source: §3.1 · p. 3

3.2 ACTOR

  • The ACTOR prompt presents the natural-language instruction and a history of alternating actions and summarized observations, then asks for the current action.Source: §3.2 · p. 3

4 Experiment

The experiments evaluate ASH on Webshop, an e-commerce benchmark requiring agents to purchase products from natural-language descriptions. The setup defines the available actions, scoring, models, and trajectory limits.

4 Experiment

  • Webshop is a real-world-scale e-commerce benchmark in which agents purchase products matching natural-language descriptions.Its catalog contains 1,181,436 distinct products with varied features, producing a complex state space.Source: §4 · p. 3
  • Agents can search with text queries, click interactable elements, or use a state-preserving think action.Source: §4 · p. 3
  • A purchased product receives a score from zero to one based on the percentage of required attributes it satisfies; invalid-action streaks or exceeding the maximum length yield zero.Source: §4 · p. 3
  • Experiments use CODE-DAVINCI-002 and GPT-3.5-TURBO with greedy decoding and a 500-token maximum; trajectories end after 20 steps or five consecutive invalid actions.Webshop trajectories average 6.5 steps.Source: §4 · p. 3

5 Results

ASH outperforms REACT on web-navigation results, with gains in success rate and average score; further analyses report advantages on longer trajectories and fewer repeated-invalid-action failures. The authors also find that ASH lets the simpler DIRECT prompting mechanism outperform REACT.

5 Results

  • Experiments sampled 500 trajectories from Webshop’s full product space while retaining webpage complexity and unrestricted search results.The comparison differs from prior work that restricted search results to the top three.Source: §5 · p. 3
  • ASH improves success rate over REACT by 6.8% (29% relative gain) and average score by over 3.3% with CODE-DAVINCI-002, and consistently outperforms REACT across both tested base models.The models are CODE-DAVINCI-002 and GPT-3.5-TURBO.Source: §5 · p. 3, Table 1 · p. 4
  • For trajectories longer than 11 steps, ASH scores 38.2 on average versus REACT’s 7.4, and ASH scores higher across trajectory lengths.Figure 4 plots average score by trajectory length and shows trajectory counts.Source: §5 · p. 4, Figure 4 · p. 4
  • ASH reduces the repeated-invalid-action failure rate from REACT’s 12.7% to 10.4%, a relative reduction of 18%.Figure 5a reports the percentage of failures due to repeated invalid actions.Source: §5 · p. 4, Figure 5 · p. 4

ASH mitigates the need of explicit reasoning

6 Related Work

Prior prompting work has shaped action representations and added explicit reasoning, while most approaches use raw environment observations. ASH differs from REACT by separating observation summarization from acting and by using separately developed prompts.

Prompting-based Decision Making

  • Some prompting-based decision-making work turns actions into program components rather than linear sequences; another adds a verbal think step, while most works use vanilla raw observations from the environment.Source: §6 · p. 5
  • ASH differs from REACT by condensing observations before acting and by developing the ACTOR and SUMMARIZER prompts separately.REACT performs observation processing internally and relies on the ACTOR prompt.Source: §6 · p. 5

Multi-stage Prompting

  • Multi-stage prompting has also been applied to knowledgeable dialogue and translation, with reported generalization beyond a knowledge base and adaptation to downstream tasks.The passage attributes these findings to Liu et al. (2022) and Tan et al. (2021).Source: §6 · p. 5

7 Conclusion

ASH is a hierarchical method for LLM-based web navigation. On Webshop, it surpassed REACT in average score and success rate, including on complex tasks with lengthy trajectories.

7 Conclusion

  • ASH is a hierarchical method tailored to LLM-based sequential decision-making in web navigation.Source: §7 · p. 5
  • On Webshop, ASH surpassed REACT in average score and success rate.The conclusion also reports that ASH excels in complex tasks with lengthy trajectories.Source: §7 · p. 5
  • ASH excels in complex tasks with lengthy trajectories.Source: §7 · p. 5
  • ASH reduces LLM hallucinations and mitigates the need for explicit reasoning.Source: §7 · p. 5

Limitations

The authors did not use detailed manual or automatic prompt engineering for the ACTOR and SUMMARIZER prompts. They also did not test ASH on more realistic web-navigation benchmarks, which were unavailable when the work was completed.

Limitations

  • The ACTOR and SUMMARIZER prompts did not receive detailed manual or automatic prompt engineering.Source: Limitations · p. 5
  • ASH was not tested on more realistic web-navigation benchmarks because such benchmarks were unavailable when the work was completed.Source: Limitations · p. 5

Ethics Statement

The authors state that their prompting paradigm could apply to various sequential planning tasks and does not, to their knowledge, discriminate against or harm humans. They warn that automated web navigation may risk privacy through unintended access to or scraping of sensitive information.

Ethics Statement

  • The authors state that their prompting paradigm could apply to various sequential planning tasks.Source: Ethics Statement · p. 5
  • The authors state that, to their knowledge, the paradigm does not discriminate against or harm humans.Source: Ethics Statement · p. 5
  • Automated web navigation with models may unintentionally access or scrape sensitive or private information, risking privacy breaches.Source: Ethics Statement · p. 5

A Performance on other LLMS and Sensitivity to In-context Examples

The section examines ASH across tested models and presents model-specific SUMMARIZER and ACTOR prompts with web-navigation examples. These examples show how summarization condenses webpage information and supports the actor’s choices.

A Performance on other LLMS and Sensitivity to In-context Examples

  • With both TEXT-DAVINCI-002 and CODE-DAVINCI-002, ASH outperformed REACT; CODE-DAVINCI-002 also performed better than TEXT-DAVINCI-002, so the authors used it for further analysis.The initial test used a 50-sample subset; the supplied passages do not give the table’s numerical results.Source: Table 2 · p. 7, A Performance on other LLMS and… · p. 7

B.1 REACT

  • REACT selected a product despite a desired property being wrong, illustrating the suboptimal choice in the example trajectory.The instruction required a particular color, size, and price; the shown REACT actions click a product and its color and size options.Source: B REACT vs ASH Prompting Trajectory · p. 7, §B.1 · pp. 7–8

B.2 ACTOR-SUMMARIZER

  • ASH detected that its initial product lacked the requested color option, reformulated the search, and proceeded to a product offering color13.The example illustrates how retaining fewer options in the summary helped the actor identify a better product and avoid a longer trajectory.Source: B REACT vs ASH Prompting Trajectory · p. 7, §B.2 · pp. 9–10

C ASH Prompts

  • The authors provide SUMMARIZER and ACTOR prompt templates and note that the SUMMARIZER prompt can be adapted to the environment and desired behavior.They describe using it to guide the ACTOR toward behavior suited to a particular use case.Source: C ASH Prompts · p. 10

C.1 SUMMARIZER Prompt for CODE-DAVINCI-002

  • The SUMMARIZER prompt asks what information would help the actor buy the correct product and directs it to retain about three likely matches, discarding the rest.If none appears to match, it should suggest searching with a different keyword.Source: §C.1 · p. 12
  • For CODE-DAVINCI-002, the summarizer identifies instruction-relevant product details, filters mismatched options, and flags requested attributes missing from a description for further checking.The examples also direct the actor to check the previous page for missing attributes such as machine-washability, fit, color, size, and price.Source: §C.1 · pp. 12–14

C.2 SUMMARIZER Prompt for GPT-3.5-TURBO

  • For GPT-3.5-TURBO, the prompt conditions summarization on the instruction, previous action, and current webpage, then returns a condensed observation in an info[...] format.It describes the summarizer as adding or removing relevant information to help the navigation agent buy the desired product.Source: §C.2 · p. 14
  • GPT-3.5-TURBO’s prompt preserves navigation buttons, retains only a few instruction-matched products, and adds a hint when an invalid action reflects a missing search button.In the example, it tells the agent to go to the search page while keeping navigation controls and nearby product options.Source: §C.2 · pp. 14, 16, 17
  • The summarizer prompt removes the already-given instruction, filters mismatched options, flags missing product information or options, and retains navigation buttons.Source: §C.2 · p. 17

C.3 ACTOR Prompt for CODE-DAVINCI-002

  • The search targets 66 x 66 blackout shades; the stated need is white, 66-by-66-inch, easy-to-install shades under $110, and a later thought proposes buying an item with matching size and white blackout options.Source: §C.3 · pp. 18–19

C.4 ACTOR Prompt for GPT-3.5-TURBO

  • The GPT-3.5-TURBO actor prompt defines the instruction-observation-action history, available search, click, and think actions, and rules for issuing supported, non-repeated actions.It also advises matching product meaning rather than exact wording and following observation hints.Source: §C.4 · pp. 20–21
  • The GPT-3.5-TURBO ACTOR example follows a deodorant request by selecting the bright-citrus option and 3-ounce size, then clicking Buy Now.The request specifies a 3-ounce bright citrus deodorant for sensitive skin under $50.Source: §C.4 · pp. 23–24

D SUMMARIZER Output Example

  • The SUMMARIZER output retains the instruction and three relevant products from a search-results page while removing irrelevant information.The example starts with a page of 50 results and shows a condensed output with three product entries.Source: D SUMMARIZER Output Example · p. 24

E.1 SUMMARIZER Failure Example 1

The SUMMARIZER fails on this example because the raw observation is too long, repeating many options without effectively summarizing them. The example contains a pants request and a product page with numerous color and size choices.

E.1 SUMMARIZER Failure Example 1

  • The SUMMARIZER fails because the raw observation is too long, repeating many options without effectively summarizing them.Source: §E.1 · p. 25
  • The request specifies black pants in size 29w x 30l priced below $50, while the page lists many color and size options.Source: §E.1 · pp. 25–26
  • The shown summarizer output repeats the request and retains product information rather than condensing the page's options.Source: §E.1 · p. 26

E.2 SUMMARIZER Failure Example 2

A second failure example shows the SUMMARIZER hallucinating a size that does not exist on the product page. The environment setup uses text-rich observations, disables rendering, and tests seeded sessions.

E.2 SUMMARIZER Failure Example 2

  • The SUMMARIZER hallucinates the required size 34w x 34l, which is not an available option on the product page.Source: §E.2 · pp. 27–28

F Environment Setup Code

  • The environment is configured for text-rich observations with rendering off and the full product space, then tested across seeds 501–999 using randomized 10-letter sessions.Source: F Environment Setup Code · p. 28
Loading 2305.14257v3…