Source-linked AI summary
SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills
Boyuan Zheng, Michael Y. Fatemi, Xiaolong Jin, Zora Zhiruo Wang, Apurva Gandhi, Yueqi Song, Yu Gu, Jayanth Srinivasa, Gaowen Liu, Graham Neubig, Yu Su
TL;DR
Web agents struggle to generalize across complex websites and to abstract reusable procedural knowledge. SkillWeaver addresses this by exploring websites, synthesizing and honing reusable APIs, and achieves relative success-rate gains on WebArena and real-world websites, including benefits for weaker agents.
Problem
Web agents face complex, diverse websites and limited ability to abstract reusable procedural knowledge, refine skills, and compose them.
Method
SkillWeaver autonomously explores websites, practices discovered skills, distills successful trajectories into APIs, and tests and debugs those APIs.
Results
31.8% relative success-rate gains were reported on WebArena and 39.8% on real-world websites, while weaker agents improved by up to 54.3% on WebArena.
Takeaways & Limitations
Skills synthesized by stronger agents can transfer as lightweight, plug-and-play modules that enhance weaker agents.
Takeaways & Limitations
API-augmented agents remain vulnerable to selecting the wrong API or generating incorrect parameters, especially weaker language models.
Abstract
from arXiv · showhide
To survive and thrive in complex environments, humans have evolved sophisticated self-improvement mechanisms through environment exploration, hierarchical abstraction of experiences into reuseable skills, and collaborative construction of an ever-growing skill repertoire. Despite recent advancements, autonomous web agents still lack crucial self-improvement capabilities, struggling with procedural knowledge abstraction, refining skills, and skill composition. In this work, we introduce SkillWeaver, a skill-centric framework enabling agents to self-improve by autonomously synthesizing reusable skills as APIs. Given a new website, the agent autonomously discovers skills, executes them for practice, and distills practice experiences into robust APIs. Iterative exploration continually expands a library of lightweight, plug-and-play APIs, significantly enhancing the agent's capabilities. Experiments on WebArena and real-world websites demonstrate the efficacy of SkillWeaver, achieving relative success rate improvements of 31.8% and 39.8%, respectively. Additionally, APIs synthesized by strong agents substantially enhance weaker agents through transferable skills, yielding improvements of up to 54.3% on WebArena. These results demonstrate the effectiveness of honing diverse website interactions into APIs, which can be seamlessly shared among various web agents.
1 Introduction
SkillWeaver addresses web agents’ difficulty with complex, diverse websites by enabling autonomous exploration, reusable skill abstraction, and iterative API refinement. Its synthesized APIs improve performance across simulated and real-world web environments.
- Complex websites create large action spaces and make generalization to new task types and websites difficult for web agents.
- Human-like self-improvement motivates exploring websites and abstracting repeated routines into reusable procedural skills.
- Trajectory-based methods often leave procedural knowledge implicit, increasing training demands and limiting generalization to new websites and tasks.
- SkillWeaver explores websites, practices candidate skills, converts successful trajectories into reusable APIs, and tests and debugs them for reliability.
- 39.8% relative improvement in success rate was achieved on real-world websites, while WebArena success-rate gains ranged from 25% to 38% after 160 iterations.
2 SkillWeaver
SkillWeaver represents website skills as reusable Python-based browser-automation APIs and builds them through proposal, synthesis, and honing. The framework emphasizes diverse, short-horizon skills that can be composed and tested for reliability.
- In this paper, an API is a Python function containing Playwright browser-automation code rather than a REST interface.
- Skill Proposal discovers novel skills, Skill Synthesis converts practiced trajectories into reusable functions, and Skill Honing tests and debugs them.
- LLMs propose short-horizon, reusable skills beyond the current repertoire and progress from simple skills toward complex compositions.
- The task types include procedural workflows, navigational exploration, and information seeking through specialized scraping APIs.
- Successful practice is evaluated by a reward model before actions are encapsulated into generalized Python implementations with static analysis.
- Synthesized APIs are tested directly when possible or with LLM-generated parameter values when additional parameters are required.
3 Experiments
The experiments evaluate SkillWeaver on WebArena and live websites, comparing browsing agents with synthesized or human-crafted APIs. Exploration is conducted separately per website to build API libraries that can be used by the evaluated agents.
- SkillWeaver is evaluated on WebArena and real-world live websites to assess performance in simulated and dynamic environments.
- WebArena is a sandboxed benchmark spanning five application domains and 812 tasks with automated functional-correctness evaluation.
- The evaluation compares synthesized APIs with human-crafted official APIs available from WebArena’s source code and documentation.
- Online-Mind2Web contributes 300 realistic tasks across 136 popular live websites spanning diverse domains.
- The baseline agent uses atomic Playwright browsing actions, while the skills agent extends its action space with synthesized and human-crafted API calls.
- Exploration uses 160 GPT-4o iterations per website, including skill attempts, existing-skill tests, and composition of previously synthesized APIs.
4 Results and Analysis
SKILLWEAVER improves web-agent performance across benchmark and live websites, transfers synthesized APIs across agents, and supports comparisons with human-crafted APIs and emergent compositional skills. Results also expose limits in API use and in handling tasks requiring strong environment understanding, visual reasoning, or long-range planning.
- WebArena Results: 39.8% average relative success-rate improvement was observed for GPT-4o on WebArena, compared with 54.3% for GPT-4o-mini.The improvements were measured across evaluated WebArena websites.
- Comparisons: SKILLWEAVER achieved higher average success rates than AutoEval and better or comparable performance across domains except Shopping, while outperforming human-crafted workflows on CMS and Map.The Shopping exception reflects websites requiring more extensive interaction with partially observable information.
- Live Website Results: 39.8% average relative success-rate improvement was reported across four live websites, although the Car website’s final success rate remained unchanged.Synthesized APIs guided four of nine Car tasks to their required final states, but failures remained in final steps requiring strong environment understanding and visual reasoning.
- Generalization across Agents: GPT-4o-mini agents gained relative success-rate improvements ranging from 40% to 133% across five websites after integrating synthesized APIs.The improved GPT-4o-mini agent achieved a better average success rate than the GPT-4o WebArena agent particularly on Map, Shopping, and Reddit.
- Comparison with Human-Crafted APIs: Synthesized APIs were comparable to human-crafted APIs on Reddit and Shopping but underperformed on high-API-support websites such as GitLab and Maps.The comparison grouped websites by low, medium, and high API support.
- Emergence of Compositional APIs: After sufficient exploration, SKILLWEAVER generated compositional APIs that sequentially called simpler APIs to apply multiple search filters.This behavior was associated with increasing abstraction and efficiency in synthesizing and integrating discovered skills.
5 Related Work
Prior web-agent research uses prompting, tools, trajectories, rewards, and training, but these approaches do not fully provide explicit, reusable procedural skills for generalization.
- Web Agents: Web agents use powerful LLMs and prompting strategies to predict actions from webpage representations.Inputs include HTML content, accessibility trees, and screenshots.
- Tool Use and Synthesis: Tool-synthesis research applies language models across reasoning, theorem proving, data analysis, and digital-agent automation.
- Self-Improvement: Self-improvement methods collect experience trajectories, evaluate success with reward models, and use results for policy training.These methods include both parametric and non-parametric approaches.
6 Conclusion
SkillWeaver enables web agents to self-improve through exploration and API synthesis, with transferable skills improving both benchmark performance and weaker agents.
- 6 Conclusion: 31.8% relative success-rate gains were achieved on WebArena, while real-world websites showed 39.8% gains.
- 6 Conclusion: 54.3% improvements were observed when weaker agents used skills from more powerful counterparts.The paper characterizes these skills as lightweight, transferable modules supporting knowledge distillation.
- 6 Conclusion: The framework points toward web agents improving without parameter tuning by evolving a collective skill base.
Ethics Statement
The paper identifies privacy, sensitive operations, and harmful actions as safety concerns for deploying generalist web agents in real-world settings.
- Ethics Statement: Real-world deployment raises privacy concerns when agents access personal profiles.
- Ethics Statement: Financial transactions and application submissions are examples of sensitive operations requiring caution.
- Ethics Statement: Online evaluation revealed that web agents may generate harmful actions on the web.
Table of Content:
The appendix materials describe SkillWeaver’s prompts, state representations, Playwright action generation, API synthesis, validation, and safety constraints.
- Safety: The framework uses safety instructions that discourage side-effectful exploration, including account creation and human interaction.
- Skill Proposal: Exploration proposes realistic multi-action website shortcuts, prioritizing useful creation, modification, filtering, and querying skills.
- State Representation: Website states combine screenshots, titles, URLs, and accessibility-tree hierarchies serialized into structured representations.
- Action Synthesis: Action generation produces Playwright code using accessibility-tree-centric selectors and contextual filtering to identify elements.
A.3 Code Generation Agent Error Detection
The code-generation agent uses an error-detection prompt template to identify and correct implementation problems. The listed checks cover function signatures, disabled functions, Playwright usage, and selector choice.
- The error-detection prompt is activated when the agent’s code contains errors.
- Checks include the required act function, its single page argument, and whether referenced functions are disabled.
- The template flags incorrect Playwright calls and instructs the agent to use accessibility-tree functions.
B WebArena Benchmark
The WebArena benchmark includes web tasks spanning medical information retrieval, recipes, flights, and vehicle searches.
- WebArena benchmark tasks cover multiple website domains and interaction types.
Drugs (23 tasks)
The benchmark section lists diverse drug-information and other web tasks, alongside examples of synthesized skills that automate multi-step shopping actions. These skills can complete checkout workflows, but execution depends on conditions such as a pre-filled cart.
- Drugs (23 tasks): The drug-task set includes medication reviews, FDA alerts, risk summaries, interaction checks, dosage information, side effects, and pill identification.
- Correctly-Implemented Skills: Synthesized APIs can automate multiple atomic actions as one action with richer input parameters.
- Correctly-Implemented Skills: An express checkout skill navigates from the cart through shipping and review steps to place an order.
- Correctly-Implemented Skills: A successful express checkout produced an order confirmation page with order number 000000191.
- Correctly-Implemented Skills: The checkout skill requires the cart to be pre-filled and assumes the Proceed to Checkout button is visible and clickable.
D.1.2 Debugging Process Fixing a Skill’s Implementation
The debugging process fixes skill implementations by diagnosing failures, refining DOM interactions, and validating recovered APIs through execution. Examples show both successful repairs and remaining robustness limitations.
- Recovery workflow: The debugging process can recover APIs by identifying bugs, applying fixes, and recording recovery instructions for subsequent execution.The example explicitly describes finding an API bug and making recoveries through a knowledge-base patch.
- Diagnosing implementation failures: The agent diagnoses a strict-mode error caused by an ambiguous textbox selector resolving to two elements.The conflicting elements are the GitLab search box and the delete-merged-branches input.
- Fixing a skill’s implementation: The repair scopes textbox selection to the confirmation dialog before filling it and clicking “Delete merged branches.”This replaces the ambiguous global textbox locator with a dialog-specific selector.
- Validating the repair: The corrected API successfully deletes merged branches in GitLab after navigating to the project’s branches page and confirming the action.The usage log records successful execution in the tested project, while also noting that the user must be logged in and have project access.
- Verification limitations: Verification may be unreliable when it only checks whether an API runs without exceptions, allowing malfunctioning APIs to be marked as verified.This evaluation criterion can produce unintended consequences.
- Robustness limitations: Conditional checks can suppress exceptions when DOM elements are missing, but they do not necessarily improve API robustness.A Magento inventory example checks for button and input availability before proceeding, while the accompanying analysis warns that this can avoid atomic actions rather than correct misuse.