PaperlayerView exact revision on arXiv

Source-linked AI summary

Go-Browse: Training Web Agents with Structured Exploration

Apurva Gandhi, Graham Neubig

arXiv:2506.03533v2cs.CL

TL;DR

Web agents can struggle to understand unfamiliar websites, and collecting high-quality interaction data at scale is costly. GO-BROWSE uses graph-structured exploration to collect web-agent trajectories while reusing discoveries across episodes. Fine-tuning a 7B model reaches 21.7% WebArena success, beating GPT-4O-MINI by 2.4% and NNETNAV by 2.9%.

  • Problem

    Collecting high-quality web-agent demonstrations is costly, while agents need data grounded in the websites they will encounter.

    Source: §1 · p. 1

  • Method

    GO-BROWSE explores websites as graphs, reusing discovered pages across episodes while collecting task-solving trajectories.

    Source: §1 · pp. 1–2, §3 · p. 3

  • Results

    21.7% WebArena success for fine-tuned QWEN-2.5-7B-INSTRUCT beats GPT-4O-MINI by 2.4% and NNETNAV by 2.9%.

    Source: §1 · p. 2, §5 · p. 6

  • Takeaways & Limitations

    GO-BROWSE-WA trajectories support models that navigate explored websites more deeply, including to specific product attributes and order details.

    Source: §6 · p. 7, §8 · p. 9

    The current collection covers five WebArena domains, and model- and prompt-introduced biases may propagate to agent behavior.

    Source: §8 · pp. 9–10

Abstract

from arXiv · show

One of the fundamental problems in digital agents is their lack of understanding of their environment. For instance, a web browsing agent may get lost in unfamiliar websites, uncertain what pages must be visited to achieve its goals. To address this, we propose Go-Browse, a method for automatically collecting diverse and realistic web agent data at scale through structured exploration of web environments. Go-Browse achieves efficient exploration by framing data collection as a graph search, enabling reuse of information across exploration episodes. We instantiate our method on the WebArena benchmark, collecting a dataset of 10K successful task-solving trajectories and 40K interaction steps across 100 URLs. Fine-tuning a 7B parameter language model on this dataset achieves a success rate of 21.7% on the WebArena benchmark, beating GPT-4o mini by 2.4% and exceeding current state-of-the-art results for sub-10B parameter models by 2.9%.

1 INTRODUCTION

Web agents perform poorly without interaction-specific training data, while collecting high-quality demonstrations at scale is costly. GO-BROWSE structures website exploration as graph search and reports improved WebArena results after fine-tuning a 7B model.

1 INTRODUCTION

  • Pretrained models score below models trained specifically for GUI interaction on WebArena, while humans achieve 78%.GPT-4O scores 38%, GPT-4O-MINI 19%, QWEN-2.5-7B-INSTRUCT 8%, CLAUDE-3.7-SONNET 45.4%, and COMPUTER-USING AGENT 58%.Source: §1 · p. 1
  • Human trajectory collection is expensive and time-consuming, motivating automated alternatives for scaling web-agent data.Prior approaches include scaling human-generated data, tutorial-based synthetic demonstrations, and website-interaction exploration.Source: §1 · p. 1
  • Directly exploring target environments outperforms tutorial-derived demonstrations in cited work: 16% versus 6% success.The passage argues that demonstrations from one website may not transfer to the many other sites an agent must use.Source: §1 · p. 1
  • GO-BROWSE builds a graph of visited URLs and reuses promising pages across episodes, separating navigation from local task solving.The authors say this reset-and-explore strategy can help weaker pretrained LLMs collect higher-quality data.Source: §1 · p. 2
  • 21.7% WebArena success rate beats GPT-4O-MINI by 2.4% and NNETNAV by 2.9% after fine-tuning QWEN-2.5-7B-INSTRUCT on GO-BROWSE data.The collection spans 100 URLs and includes approximately 10K successful and 17K unsuccessful trajectories.Source: §1 · p. 2

2 BACKGROUND

The background defines the paper’s web-agent interaction and reward setup, then contrasts interaction-first and instruction-first data-collection policies. Each policy offers benefits but has limitations in efficiency or grounding.

2.1 LLM WEB AGENTS

  • Web agents follow a ReAct loop: an LLM receives the current state, produces an action, and the browser returns the next state.The state includes the task, page representation, action space, action history, and recent errors.Source: §2.1 · pp. 2–3
  • A trajectory is a sequence of states and actions, and a binary reward records whether it completed the task successfully.Trajectories end at a maximum horizon or a terminal action.Source: §2.1 · p. 3

2.2 EXPLORATION POLICIES FOR DATA COLLECTION

  • Interaction-first policies explore with general instructions, then use a prompted LLM to assign concrete tasks to the collected rollouts.Independent episodes can redundantly revisit website regions and spend time on trajectories that yield no useful tasks, despite potentially reaching deeper pages.Source: §2.2 · p. 3
  • Instruction-first policies propose tasks from observed page content, attempt them, and retain successful task-trajectory pairs.Task proposals are usually limited to the observed page and may hallucinate infeasible tasks about unseen parts of the site.Source: §2.2 · p. 3

3 GO-BROWSE

GO-BROWSE explores websites as URL graphs, repeatedly selecting frontier pages and combining task proposal, feasibility checking, and trajectory sampling. Its modules seek both local page coverage and discovery of new pages while reusing prior exploration.

3 GO-BROWSE

  • GO-BROWSE maintains a frontier of discovered pages and expands it by proposing and solving tasks for local exploration and discovery of new webpages.Its graph nodes are unique URLs and edges are trajectories between them; the outer loop traverses the graph while the inner loop explores a selected page.Source: §3 · pp. 3–4
  • NavExplorer interacts with a page to discover neighboring webpages and propose navigational tasks grounded in its observations.It prioritizes destinations likely to support common, useful user tasks.Source: §3 · p. 4
  • PageExplorer proposes plausible tasks for the current webpage to cover its local functionality.Its tasks complement navigation-oriented proposals with within-page exploration.Source: §3 · p. 4
  • The FeasibilityChecker retains tasks with at least one successful trajectory, after an agent attempts them and a VLM judge checks completion.Solvers then sample additional trajectories for feasible tasks, from both the current page and the website root.Source: §3 · pp. 4–5
  • Reusing previously discovered pages can reduce redundant exploration and shift effort toward novel website regions.Prefixed sampling eases task solving by separating navigation from local execution, while unprefixed sampling retains long-horizon and exploratory behavior.Source: §3 · p. 5

4 DATA COLLECTION

The WebArena data collection covers 100 URLs across five domains, using model-driven exploration, feasibility checks, and trajectory sampling. The released dataset includes successful and failed trajectories, while the reported fine-tuning experiments use successful steps.

4 DATA COLLECTION

  • The collection explores 20 URLs in each of five WebArena domains, covering 100 distinct URLs.The domains are Shopping Admin, Shopping, Reddit, Gitlab, and Map; the authors note the pipeline can also run on other websites.Source: §4 · pp. 5–6
  • NavExplorer uses CLAUDE-3.7-SONNET for up to 15 interaction steps, while PageExplorer uses GPT-4O for up to 20 and Claude for up to 10.The FeasibilityChecker uses Claude for up to three solving attempts and a GPT-4O-based VLM judge; each URL is capped at 30 feasible tasks.Source: §4 · p. 6
  • Task solving used QWEN-2.5-7B-INSTRUCT with a maximum horizon length of 10 steps; all dataset-collection interaction steps used temperature 0.7, and GO-BROWSE-WA collection cost ∼$975.57.Source: §4 · p. 6
  • GO-BROWSE-WA releases failures and alternative page representations, but fine-tuning uses successful steps and accessibility trees.Table 1 reports dataset statistics for the five WebArena domains, with 20 pages explored per domain.Source: §4 · p. 6, Table 1 · p. 6

5 FINE-TUNING SETUP AND RESULTS

The authors fine-tune QWEN-2.5-7B-INSTRUCT on successful trajectories and evaluate it on WebArena and Online-Mind2Web. The resulting model leads the reported WebArena comparisons and retains an advantage over NNETNAV-7B on the out-of-domain benchmark.

Fine-tuning and evaluation setup

  • QWEN-2.5-7B-INSTRUCT is fine-tuned with supervised learning on successful dataset trajectories, and the models are evaluated on 812 WebArena tasks using task-specific reward functions.The comparison model is trained on NNETNAV-WA using the same parameters; benchmarking uses temperature 0.Source: §5 · p. 6

Domain results

  • GO-BROWSE-7B scores higher than GPT-4O-MINI and NNETNAV-7B in every WebArena domain except Gitlab.Its advantage over NNETNAV-7B is 11% on Shopping Admin and 7% on Reddit.Source: §5 · p. 6

Out-of-domain evaluation

  • On Online-Mind2Web, GO-BROWSE-7B leads NNETNAV-7B; on In-Domain-Adjacent sites, it is within <1% SR difference of GPT-4O-MINI and ahead of NNETNAV-7B by 3%.Online-Mind2Web has 300 tasks across 136 live websites, and models perform worse there than on in-domain WebArena.Source: Table 2 · p. 6, §5 · p. 6

WebArena results

  • 21.7% overall WebArena success rate for GO-BROWSE-7B, exceeding NNETNAV-7B by 2.9% and GPT-4O-MINI by 2.4%.It also outperforms the pretrained QWEN-2.5-7B-INSTRUCT model by 13.4%.Source: §5 · p. 6, Table 3 · p. 7

6 ANALYSIS

The analysis finds that Go-Browse produces more diverse task data and succeeds on deeper navigation, while its outer loop expands URL coverage. Prefixed sampling particularly benefits weaker models, and feasibility filtering reduces collection effort.

Task diversity

  • GO-BROWSE-WA has a more balanced distribution across domains than NNETNAV-WA, whose task data is disproportionately Gitlab-heavy and Reddit-light.The authors say this distribution aligns with NNETNAV-7B outperforming GO-BROWSE-7B on Gitlab but performing worse on Shopping Admin and Reddit.Source: §6 · p. 7, Figure 4 · p. 7

Successful trajectories go deeper

  • Go-Browse-only successes skew toward greater URL depth, while NNETNAV-only successes show no significant depth difference; Go-Browse wins include longer-horizon tasks and deeper URL visits.Examples include product-attribute editing and Reddit searches; Go-Browse visited those URLs 9 and 7 more times, respectively, while NNETNAV had 1 or 0 visits.Source: §6 · pp. 7–8, Figure 5 · p. 8, Table 4 · p. 8

Prefixed sampling

  • Prefixed sampling yields higher success rates, especially for deeper nodes and weaker models such as QWEN-2.5-7B-INSTRUCT.Node depth is the shortest trajectory length from the root, calculated with Dijkstra’s algorithm; the authors say prefixed sampling can bootstrap weaker pretrained models into producing higher-quality data.Source: Figure 6 · p. 8, §6 · p. 8

Feasibility filtering

  • 403 tasks filtered by the FeasibilityChecker avoided 3.2K trajectory rollouts (∼29.4k steps), a 13% step reduction for the same amount of positive data.Source: §6 · p. 9

Outer-loop coverage

  • As the number of reset nodes increases, the number of unique URLs visited grows steadily, showing the outer loop's contribution to broader website coverage.The comparison fixes proposed tasks at 30 per domain; the 1/30 setting omits the outer loop.Source: §6 · p. 9

7 RELATED WORK

Go-Browse is positioned as an unsupervised instruction-first data-generation method that gathers context through website exploration and explicitly leverages a web graph. Its design also draws on reset-then-explore strategies and differs from web agents that augment actions or interaction context.

Exploration methods

  • Table 5 compares unique URLs visited across five domains under varying numbers of resets.Source: Table 5 · p. 9

LLM web agents

Synthetic data generation

  • Go-Browse is an unsupervised instruction-first method that gathers its own context through exploration and explicitly builds and leverages a web graph.The authors connect this focus on deep exploration with collecting high-quality, high-coverage data for WebArena.Source: §7 · p. 9

Exploration methods

  • Go-Browse takes inspiration from Go-Explore’s reset-then-explore strategy for sharing information across episodes in Atari games (Ecoffet et al., 2019; 2021).Source: §7 · p. 9

8 CONCLUSION AND LIMITATIONS

The paper presents GO-BROWSE-WA as a diverse dataset for training web agents and reports improved WebArena performance and deeper navigation. The experiments are limited to five WebArena domains, and the authors flag possible bias propagation from LLMs and prompts.

8 CONCLUSION AND LIMITATIONS

  • Supervised fine-tuning of a 7B model on GO-BROWSE-WA improves success rates over the prior sub-10B state of the art and GPT-4O-MINI.The paper also reports that the dataset contains 9.5K successful and 39K unsuccessful trajectories.Source: §8 · p. 9
  • GO-BROWSE-WA contains high-quality, diverse trajectories, and trained models navigate explored websites better and more deeply.Source: §8 · p. 9
  • The WebArena experiments cover five domains; expanding collection to more websites is identified as a future direction.Source: §8 · p. 9
  • The current method trains a 7B model on successful trajectories, while using the 39K unsuccessful trajectories with alternative objectives and larger models remains future work.Source: §8 · pp. 9–10
  • LLM- and prompt-derived data may introduce biases that propagate to agent behavior, so the authors call for auditing and mitigation before deployment.Source: §8 · p. 10

A WEB AGENT IMPLEMENTATION DETAILS

The appendix describes the web-agent action space and prompt components, then compares explorer models and task types. It reports complementary task-proposal behavior and a larger contribution of navigation tasks from NavExplorer.

A.1 WEB AGENT ACTION SPACE

A.2 PROMPTS FOR LM COMPONENTS

  • The prompts guide agents to propose feasible, concrete tasks on current pages or linked pages, including information-seeking, navigation, and content-modification tasks.Navigation prompts prioritize useful destinations and return to the current page after exploring links.Source: §A.2 · pp. 13–14

B.1.1 GPT-4O VS. CLAUDE-3.7-SONNET FOR PAGEEXPLORER TASK PROPOSAL

  • Claude proposes almost twice as many PageExplorer tasks with half the maximum step budget, while GPT-4O proposes more diverse tasks, especially content-modifying tasks.Source: §B.1.1 · p. 15, Table 7 · p. 15

B.1.2 NAVEXPLORER VS. PAGEEXPLORER TASKS

  • NavExplorer, added alongside PageExplorer to support website coverage, more than doubles the dataset’s navigational tasks.Source: §B.1.2 · p. 15, Table 8 · p. 15

B.2 DATASET COLLECTION COST ANALYSIS

The cost analysis reports the overall expense of collecting GO-BROWSE-WA and describes cached-token use. The paired bootstrap analysis finds a statistically significant advantage over QWEN2.5-7B-INSTRUCT, with more moderate evidence against two other baselines.

B.2 DATASET COLLECTION COST ANALYSIS

  • $975.57 was the overall cost of collecting GO-BROWSE-WA.About 53% of input tokens were cache reads on average, reducing rollout costs for Claude and GPT models.Source: §B.2 · p. 16, Table 9 · p. 16

B.3 PAIRED BOOTSTRAP TEST FOR WEBARENA RESULTS

  • The paired bootstrap test used 10,000 samples of WebArena benchmark results to assess model differences.Source: §B.3 · p. 16, Table 10 · p. 16
  • GO-BROWSE-7B was significantly better than QWEN2.5-7B-INSTRUCT (p < 0.001), while its comparisons with GPT-4O-MINI (p = 0.108) and NNETNAV-7B (p = 0.094) indicated moderate confidence.Source: §B.3 · p. 16, Table 10 · p. 16

B.4 OOD ANALYSIS ON ONLINE-MIND2WEB (OM2W)

The OM2W analysis separates tasks from websites adjacent to WebArena domains from tasks on truly out-of-domain websites. GO-BROWSE-7B is competitive on IDA tasks, while NNETNAV-7B performs better on truly OOD tasks, particularly easy ones.

B.4 OOD ANALYSIS ON ONLINE-MIND2WEB (OM2W)

  • The authors interpret the results as showing that GO-BROWSE-trained models continue to perform well on out-of-domain websites similar to those explored during data collection.Source: §B.4 · p. 17
  • On truly OOD tasks, NNETNAV-7B performs better than GO-BROWSE-7B, with most of its increase coming from easy tasks.Source: §B.4 · p. 17, Figure 8 · p. 17
  • Figure 7 reports model success-rate breakdowns on the Online-Mind2Web benchmark, separating tasks from In-Domain-Adjacent (IDA) websites from those on truly Out-of-Domain (OOD) websites.Source: Figure 7 · p. 17

C HYPERPARAMETERS AND ADDITIONAL EXPERIMENT DETAILS

The appendix reports fine-tuning settings and computational resources, notes use of NNetNav-WA as a baseline dataset, and describes how LLMs were used in data collection and paper preparation.

C HYPERPARAMETERS AND ADDITIONAL EXPERIMENT DETAILS

D LLM USAGE

  • LLMs were used to explore and interact with websites for data collection, and to suggest prose revisions and iterate on figure code; authors reviewed generated revisions.Source: D LLM USAGE · p. 20

Related papers

Loading 2506.03533v2…