PaperlayerView exact revision on arXiv

Source-linked AI summary

WebRollback: Enhancing Web Agents with Explicit Rollback Mechanisms

Zhisong Zhang, Tianqing Fang, Kaixin Ma, Wenhao Yu, Hongming Zhang, Haitao Mi, Dong Yu

arXiv:2504.11788v3cs.CLcs.AI

TL;DR

Dynamic web environments can trap agents in erroneous states that one-way greedy search may struggle to escape. WebRollback lets models decide when and where to return to a previous state, and experiments report effectiveness across two benchmarks in zero-shot and fine-tuning settings.

  • Problem

    Dynamic web pages and changing layouts can trap agents in erroneous states, while one-way greedy search may struggle to escape unexpected or unpromising states.

    Source: §1 · p. 1

  • Method

    WebRollback adds explicit rollback, letting the model decide when to revert and which previous state to return to.

    Source: §1 · p. 2, §2 · p. 2

  • Results

    Experiments across two web navigation benchmarks demonstrate effectiveness in both zero-shot and fine-tuning settings.

    Source: §1 · p. 2, §4 · p. 5

  • Takeaways & Limitations

    The authors conclude that model-directed rollback provides a flexible and efficient scheme for web agents.

    Source: §4 · p. 5

    The study focuses on web navigation, tunes models around 7B, and uses URL redirection despite real-world actions sometimes being difficult or infeasible to reverse.

    Source: Limitations · p. 5

Abstract

from arXiv · show

With recent advancements in large language models, web agents have been greatly improved. However, dealing with complex and dynamic web environments requires more advanced planning and search abilities. Previous studies usually adopt a greedy one-way search strategy, which may struggle to recover from erroneous states. In this work, we enhance web agents with an explicit rollback mechanism, enabling the agent to revert back to a previous state in its navigation trajectory. This mechanism gives models the flexibility to directly control the search process, leading to an effective and efficient web navigation method. We conduct experiments on two live web navigation benchmarks with zero-shot and fine-tuning settings. The results demonstrate the effectiveness of our proposed approach.

1 Introduction

Dynamic web environments can trap agents in erroneous states, and one-way greedy search may struggle to escape them. WebRollback gives models control over when and where to roll back, and is reported to achieve the best overall results while maintaining efficiency.

  • Dynamic pages and changing layouts can leave agents in erroneous states where planned actions fail to produce expected outcomes.Most web agents use one-way greedy search, which may struggle to escape unexpected or unpromising states.Source: §1 · p. 1
  • Parallel search methods are costly for web navigation because they require maintaining multiple environments, while BestFirst can switch states frequently.The text describes serialized search with flexibility to switch paths as a suitable balance for web navigation.Source: §1 · pp. 1–2
  • WebRollback lets the model decide when to revert from an erroneous or unpromising state and where to return in the trajectory.It supports multi-step rollbacks in a single operation, unlike the cited previous backtracking methods.Source: §1 · p. 2
  • Li et al. (2025) study rollback in cleaner agent-task environments, whereas this work explores more complex real-world web tasks.Source: §1 · p. 2
  • Across two real-world web navigation benchmarks, WebRollback achieves the best overall results while maintaining efficiency.The experiments cover zero-shot prompting with strong LLMs and fine-tuning smaller LLMs using demonstrations from larger models.Source: §1 · p. 2

2 Method

The method adds model-based critique and rollback modules to a web agent's action loop. The critique decides whether to continue or roll back, and the rollback module selects a prior state to restore.

  • The agent repeatedly observes the current webpage, takes an action that transitions the environment, and receives a new observation until the task is resolved or its navigation budget is exhausted.Source: §2 · p. 2
  • After each action, a critique module decides whether to continue from the new state or roll back.The binary decision lets the model determine when rollback should occur.Source: §2 · p. 2
  • When rollback is selected, a rollback module chooses a preceding state and resets the browser and state trajectory to it.The agent can select a specific prior state, enabling multi-step rollbacks.Source: §2 · pp. 2–3

3 Experiments

Across zero-shot and fine-tuning evaluations, the experiments find that rollback offers a strong balance of effectiveness and efficiency. They also examine how search budgets affect performance and why explicit rollback can help agents use those budgets more effectively.

3.1 Settings

  • The experiments use two live benchmarks, compare OneWay, BestFirst, and Rollback, and average results across three runs scored by GPT-4o.Zero-shot settings use LLAMA-3.3-70B-INSTRUCT and QWEN2.5-72B-INSTRUCT; the default maximum step budget is 16.Source: §3.1 · p. 3, §3.2 · p. 3

3.3 Fine-tuning

  • Fine-tuning targets smaller LLMs using demonstrations from QWEN2.5-72B-INSTRUCT and training data from Mind2Web-Live and OpenWebVoyager.The tuned models are LLAMA-3.1-8B-INSTRUCT and QWEN2.5-7B-INSTRUCT.Source: §3.3 · p. 4
  • Fine-tuning results show that Rollback achieves the overall best results while balancing effectiveness and efficiency at reasonable cost.The reported patterns are similar to those for the zero-shot teacher models.Source: §3.3 · p. 4
  • Rollback has the best overall performance as step budgets increase, while rollback strategies improve faster than OneWay when more steps are available.With an 8-step budget, methods perform similarly; the analysis uses 100 randomly selected instances from the two benchmarks.Source: §3.3 · pp. 4–5, Figure 4 · p. 5

3.4 Why Explicit Rollback Helps

  • OneWay requires recovery more often than Rollback (19% vs 7%), while BestFirst can make unnecessary state switches; Rollback explicitly chooses when and where to revert.The authors report that fewer unnecessary switches help Rollback use search budgets more effectively.Source: §3.4 · p. 5

4 Conclusion and Discussion

Across two web navigation benchmarks, the approach is effective in both zero-shot and fine-tuning settings. It equips agents to decide when and where to roll back, while irreversible actions remain a challenge.

4 Conclusion and Discussion

  • The framework enables agents to revert to previous states and directly decide when and where to roll back.The authors describe the scheme as flexible and efficient.Source: §4 · p. 5
  • Experiments across two web navigation benchmarks demonstrate the approach’s effectiveness in both zero-shot and fine-tuning settings.Source: §4 · p. 5
  • Rollback execution remains challenging for irreversible actions, which may require greater caution and richer state representations before commitment.The authors identify processing payments and sending emails as examples of irreversible actions.Source: §4 · p. 5

Limitations

The study’s evaluation and rollback implementation have defined scope boundaries.

Limitations

  • Evaluation is limited to web navigation; tuning covers roughly 7B models, and URL-redirection rollback may be difficult or infeasible after irreversible changes.Source: Limitations · p. 5

Ethics Statement

The authors flag possible ethics concerns in the text data and models used.

Ethics Statement

  • The study did not apply aggressive text filtering or modify open-source models for safety or bias, and acknowledges possible offensiveness, toxicity, fairness, or bias issues.Source: Ethics Statement · p. 5

A Web Agent Prompts

The appendix specifies prompt workflows for acting, evaluating intermediate web-navigation steps, and selecting rollback points with lessons from the trace.

A Web Agent Prompts

  • The acting agent receives the target task, prior trace, rollback experience, previous tryings, and current accessibility tree, then emits one action in the prescribed format.Source: A Web Agent Prompts · pp. 7–8
  • The acting prompt directs one valid action at a time and discourages repeating actions when the page remains unchanged.Source: A Web Agent Prompts · p. 7
  • The evaluator compares before-and-after observations and records useful current-page details for later navigation.Source: A Web Agent Prompts · p. 8
  • Its critic judges intermediate progress and assigns 1.0 to stuck or erroneous states, 3.0 to reasonable steps, and 5.0 to critical successful steps before choosing continue or back.Source: A Web Agent Prompts · pp. 8–9
  • Rollback is cautious: continue trying promising states, but consider reverting after several unsuccessful attempts.Source: A Web Agent Prompts · p. 9
  • The rollback prompt selects a previous checkpoint and summarizes lessons from the trace between the current state and return point.Source: A Web Agent Prompts · pp. 9–10

B Evaluation Prompt

The evaluation prompt gives an LLM the task, navigation trace, predicted response, and optional gold information. It asks for a criteria-based assessment of task completion and an overall score.

  • The evaluator receives the task, navigation trajectory, predicted response, and optional gold information as inputs.Source: B Evaluation Prompt · p. 10
  • The evaluator should judge completion against the accessibility tree and response, without assuming information absent from the tree.When the response contradicts the accessibility tree, the tree takes precedence; when information is absent from the tree, the response is believed.Source: B Evaluation Prompt · p. 10
  • The prompt includes placeholders for the original query, navigation trajectory, predicted response, and gold information, plus the starting URL.Source: B Evaluation Prompt · p. 10
  • The prompt asks the evaluator to summarize the trajectory, explain its rationale, and list task criteria with whether each was met.Source: B Evaluation Prompt · p. 10
  • The evaluator assigns one overall score from 0.0 to 1.0 based on criteria, with intermediate values representing partial correctness.Acknowledging failures does not increase the score.Source: B Evaluation Prompt · p. 11

C Detailed Settings

The experiments use filtered subsets of Mind2Web-Live and WebVoyager after excluding sites with unstable connections. Fine-tuning uses a 455-instance training set and a multi-task supervised fine-tuning recipe.

  • Evaluation uses testing subsets of Mind2Web-Live and WebVoyager.Source: C Detailed Settings · p. 11
  • Unstable connections caused by network problems and anti-scraping measures led the authors to test connectivity and exclude affected websites.The excluded-site set is enumerated in the paper.Source: C Detailed Settings · p. 11
  • After filtering, the test sets contain 42 Mind2Web-Live and 342 WebVoyager instances, while the fine-tuning training set contains 455 instances.Training data comes from Mind2Web-Live and OpenWebVoyager.Source: C Detailed Settings · p. 11
  • Fine-tuning uses full-model supervised fine-tuning with one multi-task model supporting different modules through their corresponding prompts.The recipe uses OpenRLHF.Source: C Detailed Settings · p. 11

D Extra Results

Appendix D reports zero-shot results for smaller language models, whose performance is much worse than that of 70B-level models.

  • Zero-shot performance is much worse for LLAMA-3.1-8B-INSTRUCT and QWEN2.5-7B-INSTRUCT than for 70B-level models.Source: Table 4 · p. 11
  • Table 4 presents zero-shot results for LLAMA-3.1-8B-INSTRUCT and QWEN2.5-7B-INSTRUCT.Source: Table 4 · p. 11
  • The table caption attributes the lower performance of these smaller models to evaluation without fine-tuning.Source: Table 4 · p. 11
Loading 2504.11788v3…