Source-linked AI summary
Improving LLM-based Autonomous Web Agents with Filtering
Zhitong Guo, Jing Yu Koh, Ruiyu Li
TL;DR
LLM web agents face a webpage-input challenge: raw HTML is verbose and can consume limited context with irrelevant details. The paper tests retrieval-based filtering, including fine-tuned DeBERTa and T5 models and a zero-shot ColBERT retriever. The DeBERTa-based model raises LLaMA-2-70B’s WebArena test success rate from 1.97% to 2.96%.
Problem
Raw HTML is verbose and can fill limited LLM context windows with details irrelevant to web tasks.
Source: §1 · pp. 1–2Method
The authors compare ColBERT and DeBERTa retrieval and fine-tune DeBERTa and T5 on Mind2Web trajectory data for transfer to WebArena.
Source: §1 · p. 2, §4.1 · p. 5, §4.2 · p. 6Results
Takeaways & Limitations
The experiments show that retrieval-based filtering can improve LLaMA-2-70B’s WebArena success rate.
Source: §6 · p. 8Filtering can remove essential elements, and downstream language models may be unable to correct the resulting errors.
Source: §5.2 · p. 7
Abstract
from arXiv · showhide
Autonomous web agents, powered by Large Language Models (LLMs), have garnered significant attention for automating various web-based tasks with multi-step reasoning and decision-making capabilities. An open research question in the development of these agents lies in the format of the webpage input. Raw HTML source code, with its extensive and often irrelevant details, poses difficulties for LLMs with limited context windows. To address this challenge, we first reproduce baseline models such as GPT-3.5 and LLaMA-2-70B on the WebArena (Zhou et al., 2023) benchmark, identifying common failure modes. We then propose two retrieval strategies to filter out irrelevant context for LLM agents. We develop DeBERTa-based and T5-based models that rank HTML elements by their relevance to the task. We fine-tune them on Mind2Web trajectory data and transfer them to WebArena. Experiments show that our DeBERTa-based model improves the success rate of the LLaMA-2-70B LLM agent on WebArena from 1.97% to 2.96%. Moreover, we develop a zero-shot ColBERT-based retriever that is able to retrieve the ground-truth element with a recall of 0.52 on Mind2Web and 0.47 on WebArena.
1 Introduction
Raw HTML can overwhelm LLM context with verbose, task-irrelevant details, motivating input filtering for web agents. The paper reproduces WebArena baselines and explores retrieval strategies, reporting a DeBERTa-based improvement for LLaMA-2-70B.
1 Introduction
- Raw HTML can fill LLM context windows with verbose, irrelevant code and elements that are not visible in the rendered page.Source: §1 · p. 1
- WebArena’s accessibility tree is a more concise representation than raw HTML, and the paper investigates further filtering by user intent.Source: §1 · pp. 1–2
- The authors compare GPT-3.5 and LLaMA-2-70B baselines on WebArena and analyze common failure modes across task types.Source: §1 · p. 2
- Retrieval strategies using ColBERT and DeBERTa select relevant accessibility-tree elements, including DeBERTa fine-tuned on Mind2Web data for transfer to WebArena.Source: §1 · p. 2
- Experiments show that the retrieval strategy improves LLaMA-2-70B on WebArena tasks, increasing its test-set success rate from 1.97% to 2.96%.Source: §1 · p. 2
- The paper also proposes GPT-4 subquery generation and planning methods intended to generalize beyond simulated web environments.Source: §1 · p. 2
2 Related Work
Prior web-agent work spans language-model agents, filtering and summarization approaches, and learning-based agents, while the paper situates its benchmark study in WebArena. It also reviews pretrained language models, retrieval, and prompt-based LLM agents.
2.1 Web Agents Controlled by Natural Language
- Earlier web-agent research includes natural-language agents and learning-based approaches, but WebShop focuses mainly on item selection and does not modify websites.Source: §2.1 · p. 3
- WebArena covers diverse web tasks, while earlier agents and filtering approaches differ in task scope, input representation, and planning strategy.Mind2Web filters DOM inputs and presents five candidates per iteration; WebAgent summarizes HTML snippets and uses predefined sub-instructions, unlike WebArena’s dynamically planned steps.Source: §2.1 · p. 3
2.2 Pre-trained Language Model and Fine-tuning
- Prompt-based WebArena agents use few-shot examples with accessibility-tree inputs, while fine-tuning adapts pretrained models to particular tasks or datasets.Source: §2.2 · p. 4
- DeBERTa uses content and position vectors with disentangled attention, while ColBERT ranks query-document pairs through late interaction between contextual embeddings.Source: §2.2 · p. 4
3 Reproduced Baseline Results
The authors reproduce the GPT-3.5 WebArena baseline and evaluate LLaMA-2-70B, with Table 1 comparing their success rates across WebArena sites.
3.2 LLaMA-2-70B
- They evaluated LLaMA-2-70B on WebArena, which the original WebArena paper had not reported.They matched GPT-3.5 settings except for LLaMA decoding temperature 0.6 and top-p 0.95.Source: §3 · p. 4, §3.2 · p. 4
3.1 GPT-3.5
- The authors reran GPT-3.5 using the original WebArena instructions and settings, combining chain-of-thought prompting with the no-UA-hint prompt.The setup used temperature 1.0, top-p 0.9, and a maximum of 30 state transitions.Source: §3.1 · p. 4
3 Reproduced Baseline Results
- Table 1 compares GPT-3.5 and LLaMA model success rates across WebArena sites.The supplied caption identifies the comparison scope but provides no success-rate values.Source: Table 1 · p. 5
3.2 LLaMA-2-70B
- Figure 2 depicts DeBERTa classifying HTML elements for task relevance, then retaining the top-k elements by descending logits.The model is fine-tuned on Mind2Web data.Source: Figure 2 · p. 5
4 Methods
The methods train relevance retrievers on Mind2Web, apply retrieval-based filtering to LLM inputs, and compare retrievers and retention choices on cleaned WebArena trajectories.
4.1 Finetuning DeBERTa and T5 as retrievers
- Mind2Web ground-truth element IDs label the next human-trajectory element positive and other elements negative; training oversamples positives to balance labels and minimizes binary cross-entropy.WebArena lacks ground-truth labels, so the retrievers are fine-tuned using Mind2Web training data.Source: §4.1 · pp. 5–6
4.2 Zero-shot ColBERT retriever
- Zero-shot ColBERT scores query-document pairs by aggregating each query token’s maximum cosine similarity to document-token embeddings, then uses retrieved elements to construct observations.The two observation heuristics select neighboring elements around the top five results or use the topmost and bottom-most retrieved elements as bounds.Source: §4.2 · p. 6
4.3 Augmenting LLMs with Filtering
- During evaluation, DeBERTa ranks accessibility-tree elements by logits and passes the top-scoring 50% to the LLM, reducing context length by half on average.The model processes each element in the current observation before ranking.Source: §4.3 · p. 6
4.4 Collecting Data and Choosing Parameters
- The authors cleaned 207 WebArena trajectories for step-wise data, including trees that could extend beyond the viewport-bounded trees used in WebArena.They compared DeBERTa and zero-shot ColBERT on these trajectories.Source: §4.4 · p. 6
- As more top results are retained, DeBERTa and ColBERT achieve higher task-success recall, with DeBERTa consistently outperforming ColBERT; k = 50% performs better than k = 50.The authors choose k = 50% for subsequent experiments and identify better selection of the retained-element count as future work.Source: Figure 3 · p. 6, Figure 4 · p. 6, §4.4 · p. 6
5 Results and Discussions
The fine-tuned DeBERTa and T5 retrievers place the Mind2Web ground-truth element in the top 10 results 81% of the time, and DeBERTa filtering improves LLaMA-2-70B’s WebArena success rate. Results also show task and site variation, alongside qualitative examples of filtering helping agents and cases where filtering can remove essential elements or disrupt layout.
5.1 Retrieval Results and Model Comparison
- 81% of the time, the ground-truth element is among the top 10 results for the fine-tuned models on Mind2Web validation.DeBERTa and T5 outperform zero-shot ColBERT; DeBERTa slightly outperforms ColBERT and is transferred to WebArena.Source: §5.1 · p. 6
5.2 Quantitative Analysis
- The authors report that filtering inputs with DeBERTa improves the overall results, increasing them from 1.97% to 2.96%.The results are summarized in Table 1.Source: §5.2 · p. 7
- Accessibility-tree element counts correlate with task complexity: shopping and map observations average around 50 and 70 elements, while GitLab, Reddit, and shopping administration typically have around 100.GitLab sometimes exceeds 400 elements; retrieval improves GPT-3.5 on GitLab and LLaMA on shopping administration and shopping.Source: §5.2 · p. 7
- DeBERTa filtering can remove essential elements, causing downstream errors that the language model cannot correct.The authors suggest a recall-oriented training objective as a possible remedy.Source: §5.2 · p. 7
5.3 Qualitative Analysis
- In several examples, DeBERTa was capable of preserving most useful page elements, allowing the downstream language model, in principle, to operate without loss of useful information.The authors suggest the approach may benefit language models with shorter context windows: LLaMA has 4K tokens, while GPT-3.5 has 16K tokens.Source: §5.3 · p. 8
- For GPT-3.5 Task 152, filtering prevents a repeated action and allows the agent to proceed to the correct answer.Before filtering, the same typing action occurred three times and led to early termination.Source: §5.3 · p. 8
6 Conclusion and Future Work
The paper concludes that retrieval can filter irrelevant context for web agents, with fine-tuned DeBERTa and T5 retrievers trained on Mind2Web and tested on WebArena, alongside zero-shot ColBERT experiments. It reports improved LLaMA-2-70B WebArena success and proposes further work on trajectory fine-tuning and subquery generation and planning.
6 Conclusion and Future Work
- DeBERTa filtering improves LLaMA-2-70B’s WebArena test success rate from 1.97% to 2.96%.The fine-tuned models place the ground-truth element in their top 10 results 81% of the time on Mind2Web.Source: §6 · p. 8
- The work proposes DeBERTa and T5 retrievers fine-tuned on Mind2Web trajectory data, and evaluates zero-shot ColBERT on Mind2Web and WebArena HTML data.The fine-tuned retrievers are transferred to WebArena.Source: §6 · p. 8
- Future work includes fine-tuning retriever-enhanced agents on cleaned trajectories and exploring subquery generation and prompt-based planning for open-domain websites.The authors state this as a route toward generalizing beyond simulated web environments.Source: §6 · p. 8
A.1 Improvement of Repetitive Actions
Filtering the context changes GPT-3.5’s repeated, prematurely terminating action into a correct answer on Task 152.
A.1 Improvement of Repetitive Actions
- After filtering, GPT-3.5 avoids repeating the same typing action three times and reaches the correct answer on Task 152.Before filtering, the repeated action led to early stopping.Source: Figure 10 · p. 11
A.2 Subquery Generation
The example asks an agent to compare walking and driving travel times between Randyland and Carnegie Mellon University using OpenStreetMap. Its generated subtasks specify entering the locations, selecting each travel mode, obtaining each route, recording the estimates, and comparing them.
A.2 Subquery Generation
- The task is to compare walking and driving route times from Randyland to Carnegie Mellon University in OpenStreetMap.The observation includes location fields and Foot and Car travel-mode options.Source: §A.2 · pp. 11–12
- The generated plan enters the start and destination, selects Foot (OSRM), and searches for the walking route.It then records the walking route’s estimated time.Source: §A.2 · p. 12
- The plan repeats the route search with Car (OSRM), records its estimated time, and compares the walking and driving estimates.It also suggests optionally noting route details such as distance, directions, landmarks, or turns.Source: §A.2 · p. 12