Source-linked AI summary
Online Skill Learning for Web Agents via State-Grounded Dynamic Retrieval
Jiaxi Li, Ke Deng, Yun Wang, Jingyuan Huang, Yucheng Shi, Qiaoyu Tan, Jin Lu, Ninghao Liu
TL;DR
Existing web-agent skill reuse retrieves a fixed set of skills from the initial task instruction, although useful skills can depend on the evolving webpage state. SGDR instead retrieves executable sub-procedures dynamically at each step using both task and state information, and consistently improves performance over strong baselines on WebArena across five domains.
Problem
Existing methods retrieve skills once from the initial task instruction, despite skill usefulness depending on the current webpage state during interactive execution.
Method
SGDR extracts intermediate sub-procedures as text–code pairs and dynamically retrieves them at each step using both the task instruction and current webpage state.
Results
SGDR consistently improves overall success rates and step efficiency over strong online skill-learning baselines on WebArena across five website domains and two backbone models.
Takeaways & Limitations
State-grounded retrieval is a practical approach for improving web agents across proprietary and open-source backbone models.
Takeaways & Limitations
Experiments use WebArena’s limited set of website domains, interaction patterns, and agent actions, leaving SGDR’s generality in broader web environments unvalidated.
Abstract
from arXiv · showhide
Language agents increasingly rely on reusable skills to improve multi-step web automation across related tasks. A growing line of work studies online skill learning, where agents continually induce skills from previous task trajectories and reuse them in future tasks on the fly. However, existing methods mainly reuse skills at the task-level: a fixed set of skills is retrieved based on the initial task instruction and then held fixed throughout execution. This static strategy is misaligned with web execution, where the appropriate next action depends not only on the task goal but also on the current webpage state, which often transitions into situations that the initial skills fail to cover. To address this gap, we propose State-Grounded Dynamic Retrieval (SGDR), an online skill learning method that enables stepwise skill reuse for web agents. SGDR consists of three components: a sliding-window extraction process that turns completed trajectories into reusable sub-procedures invokable at intermediate execution states, a dual text-code representation that connects skill retrieval with executable action, and a state-grounded dynamic retrieval mechanism that matches skills to both the task goal and the current webpage state. Experiments on WebArena across five domains show that SGDR consistently outperforms strong baselines, achieving average success rates of 37.5% with GPT-4.1 and 24.3% with Qwen3-4B, corresponding to relative gains of 10.6% and 10.0% over the strongest baseline, respectively. The code is available at https://github.com/plusnli/skill-dynamic-retrieval.
1 Introduction
SGDR addresses the mismatch between static task-level skill reuse and interactive web execution by retrieving executable sub-procedures dynamically according to the task and evolving webpage state. It combines sliding-window extraction with dual text–code representations and improves WebArena performance and step efficiency over strong online skill-learning baselines.
- Motivation: Recurring web-task procedures motivate reusable skill learning for language agents across information seeking, form filling, forum interaction, navigation, filtering, and submission.These tasks differ in goals but share procedural patterns that can be summarized and reused.
- Online Skill Learning: Online skill learning continually induces skills from completed executions and updates the skill library as tasks arrive sequentially.This setting more closely matches deployment than relying on a fixed offline library.
- Problem: Existing methods retrieve skills once from the initial task instruction and keep them fixed, although useful skills depend on the current webpage state.This task-level one-shot design is insufficient for interactive web automation.
- Problem: Dynamic retrieval requires intermediate-granularity skills because full-trajectory procedures may not fit intermediate states, while single-action skills may be too primitive.Skill-library granularity therefore affects retrieval quality and applicability.
- SGDR: SGDR retrieves skills at each decision step using the task instruction and evolving webpage state, extracting reusable sub-procedures with sliding windows and representing them as text–code pairs.The representation supports natural-language retrieval and executable browser actions.
- Evaluation: On WebArena across five website domains and two backbone models, SGDR consistently improves success rates and step efficiency over strong online skill-learning baselines.The evaluation tests the method across multiple domains and model backbones.
2 Related Work
Related work has progressed from simulated browser interaction toward realistic, dynamic, long-horizon web automation. In parallel, language agents increasingly self-improve by discovering and accumulating reusable skills from prior executions.
- Web Agents: Early web-agent research studied language-model interaction with browsers for information retrieval and task completion in simulated environments.The cited studies include Liu et al. (2018), Nakano et al. (2021), and Yao et al. (2022).
- Web Agents: Recent work scales web agents toward generalist navigation on real-world websites and more dynamic, long-horizon web automation.The passage identifies generalist navigation on real-world websites as one axis of progress and characterizes the broader shift from controlled browser interaction.
- Online Skill Learning: A parallel research direction enables language agents to self-improve by discovering and accumulating reusable skills from past executions.Early approaches store procedural knowledge in natural language and adapt it non-parametrically through verbal reflections or distilled experiential insights.
3 Preliminaries
Online skill learning is formulated as sequential task solving in which an agent acts through webpage observations, updates a reusable skill library from completed trajectories, and applies past skills to future tasks. Task success is evaluated externally, while skill updates rely on the instruction, trajectory, and evaluator-produced proxy judgment rather than ground truth.
- Task-stream setting: Each task pairs a natural-language goal gi with a multi-step trajectory Ti formed by interleaved webpage observations and actions.The trajectory has length Hi, and tasks are indexed across a stream of N tasks.
- Skill library: The agent maintains a library Si of reusable procedural skills induced from previous task executions, starting from an empty S0.After task gi, the agent may induce new skills ∆Si and update the library.
- Online access constraints: During task gi, only the previously accumulated library Si−1 is available, while the current task’s ground-truth signal yi remains unavailable during execution and library updates.This prevents ground-truth success from guiding skill induction or action selection for the current task.
- Skill update: Skill induction after gi uses the instruction gi, trajectory Ti, and evaluator proxy judgment ˆyi, with newly induced skills available only to subsequent tasks.The evaluator judgment is binary: ˆyi = 1 denotes a correctly solved trajectory and ˆyi = 0 denotes failure.
- Objective: The online learning objective is to maximize cumulative ground-truth task success yi over the task stream.The overall agent π includes its action policy, skill induction, and skill reuse rules.
4 Proposed Method
SGDR learns reusable web procedures from successful trajectories and dynamically retrieves them at each execution step. Its method combines sliding-window extraction and verification, text–code skill representations, and task-state relevance retrieval with MMR reranking.
- Method overview: SGDR addresses skill granularity and evolving webpage states through sliding-window extraction, text–code representation, and state-grounded dynamic retrieval with reranking.These components form the overall SGDR pipeline.
- Skill representation: Each skill pairs a natural-language description for retrieval with executable code for carrying out the corresponding web operations.The description abstracts intent and applicable state, while the code implements the selected procedure.
- Skill extraction: After successful tasks, SGDR divides trajectories into sliding-window segments and retains only LLM-induced skills whose substituted executions remain successful.Windowed segments target reusable intermediate-granularity subroutines rather than overly specific full tasks or overly fine-grained individual actions.
- Dynamic retrieval: At each step, SGDR summarizes the current webpage and combines its state query with the task instruction to score and retrieve relevant skills.The relevance score balances task-goal alignment and current-page applicability through a hyperparameter α.
- Dynamic retrieval: MMR reranking reduces near-duplicate candidates from overlapping windows by balancing relevance with procedural coverage in the step-specific activated set.The selected skills are exposed only for the current decision, together with their descriptions and callable code.
5 Experiments
Experiments on WebArena across five domains show that SGDR improves success and execution efficiency over skill-free and online skill-learning baselines under GPT-4.1 and QWEN3-4B. Ablations further show that combining task and webpage-state retrieval, diverse reranking, and sliding-window extraction is most effective.
- Benchmark: WebArena evaluates SGDR across five domains—Shopping, Admin, Reddit, Gitlab, and Map—whose shared within-domain interfaces support continual skill acquisition.Tasks completed within a domain provide trajectories from which skills are extracted and reused for subsequent tasks in that domain.
- Overall results: 37.5% and 24.3% average success rates make SGDR the strongest method with GPT-4.1 and QWEN3-4B, improving over CER by 3.6 and 2.2 points, respectively.CER is the strongest baseline in this comparison, and the gains indicate benefits beyond static task-level skill reuse.
- Domain results: 47.7% on Admin versus 41.4% is a notable GPT-4.1 domain gain, while SGDR performs best on four of five domains and Gitlab remains the main exception.The paper attributes Gitlab’s weaker transfer to persistent repository preconditions and SGDR’s local rather than whole-task skills.
- Execution efficiency: 4.8 average steps with GPT-4.1 is fewer than Vanilla’s 6.0, ASI’s 5.2, and CER’s 6.4; with QWEN3-4B, SGDR reduces steps by 11.1% versus Vanilla and 13.8% versus CER.A skill can execute a short multi-action browser procedure, replacing several primitive clicks and fills.
- Ablation studies: 34.6% and 35.9% are among the best retrieval-ablation results for α = 0.5, while combined task-and-state retrieval outperforms task-only and state-only retrieval across all three domains.MMR improves over relevance-only retrieval by promoting procedural diversity, and sliding-window skills outperform full-trajectory and single-action alternatives.
6 Case Study
Representative case studies show that SGDR induces reusable skills from successful trajectories across different web domains. Examples include filling and submitting Map driving-direction queries and GitLab merge-request comments.
- 6 Case Study: SGDR induces reusable skills from judged-as-successful trajectories across several domains, including Map and GitLab.The case studies are presented in Appendix B.
- 6 Case Study: One Map skill fills start and destination fields to submit a driving-directions query.
- 6 Case Study: One GitLab skill fills and submits a merge-request comment.
7 Conclusion
SGDR addresses limitations of task-level skill reuse by dynamically retrieving skills using both task and webpage-state information. Its sliding-window extraction and adaptive retrieval provide support throughout online web-agent execution, with strong WebArena performance across five domains and two backbone models.
- SGDR addresses core limitations of task-level skill reuse for online skill learning.
- Sliding-window extraction converts evaluator-assessed trajectories into skills for reuse during execution.
- Dynamic retrieval matches skills to both the task and current state, rather than retrieving them only at task start.
- SGDR shows strong performance across five WebArena domains with two backbone models.
Limitations
The study’s evaluation is limited to WebArena’s finite coverage of domains, interaction patterns, and agent actions, and it does not examine integration with model fine-tuning or long-term personalization.
- Evaluation scope: SGDR is evaluated only on WebArena, whose website domains, interaction patterns, and agent action set remain limited.Broader web environments are needed to further validate the method’s generality.
- Skill integration: The study focuses on nonparametric skill accumulation and reuse without exploring model fine-tuning or long-term agent personalization.
Ethical Considerations
The study evaluates SGDR in WebArena without human subjects, private user data, or live third-party website interactions, while recognizing deployment risks for more capable web agents. These risks include unintended actions, sensitive-information access, and violations of website usage policies, motivating appropriate safeguards.
- The experiments use WebArena and involve neither human subjects nor private user data or live third-party website interactions.
- Deploying more capable web agents without safeguards could enable unintended actions, sensitive-information access, or violations of website usage policies.
A Experiment Details · A.1 Agent Action Space
The WebArena experiments use a shared primitive action space across SGDR and all baselines. It includes navigation, tab management, user messaging, and infeasibility reporting actions.
- A.1 Agent Action Space: The default base action space is identical for vanilla, AWM, ASI, CER, and SGDR in all WebArena experiments.This shared action space is defined as the primitive action set used by the web navigation agents.
- A.1 Agent Action Space: goto(url) navigates to a URL, while go_back() and go_forward() move between the previous and next pages.These are the action space’s page-navigation primitives.
- A.1 Agent Action Space: new_tab() opens a tab, tab_close() closes the current tab, and tab_focus(index) brings a tab to the front.These primitives support multi-tab web interaction.
- A.1 Agent Action Space: The action space includes an operation to send a message to the user.This action enables direct user communication during web-agent execution.
- A.1 Agent Action Space: report_infeasible(reason) notifies the user when the instruction is infeasible.The action takes a reason argument and communicates infeasibility to the user.
A.2 Task Indices for Website Domains … B.1 Driving Directions Form Submission
The appendix documents the domain-filtered WebArena task split, LLM prompts, and parameter-configuration materials, followed by representative SGDR skills. The case study shows a parameterized driving-directions procedure extracted from a successful Map trajectory and reusable across related tasks.
- A.2 Task Indices for Website Domains: 764 single-domain WebArena tasks remain after removing cross-site tasks: 187 Shopping, 182 Admin, 106 Reddit, 180 GitLab, and 109 Map.The filtering prevents cross-domain skill transfer from confounding evaluation.
- A.3 Prompts for LLM-Based Components: The trajectory evaluator judges success or failure from the user intent, action history, final webpage state, and agent response.The evaluator is used for SGDR and the AWM, ASI, and CER baselines.
- A.3 Prompts for LLM-Based Components: Skill induction processes successful-trajectory windows of 2, 3, 4, or 5 steps into reusable, parameterized Python subroutines.Reusable windows represent recognizable operations, generalize variable inputs as arguments, and contain 2 to 5 action steps.
- A.3 Prompts for LLM-Based Components: Skill callability requires every element-ID argument to be visible in the current single accessibility tree before invocation.Skills depending on IDs that appear only after a transition are rejected, while descriptions pair action verbs and page context for retrieval.
- A.3 Prompts for LLM-Based Components: Webpage summaries name the operational page type and actions enabled in the current state, using the same vocabulary as skill descriptions for cosine matching.The summarizer avoids enumerating every element, pure visual descriptions, task instructions, and speculation about future steps.
- A.3 Prompts for LLM-Based Components: Retrieved skills are preferred as single actions when their intent matches the needed subroutine and their required arguments are visible; otherwise, the agent uses primitive actions.In either case, the prompt instructs the agent to continue making progress toward the goal.
- A.4 Parameter Configuration: Table 6 summarizes SGDR’s main parameter configuration and experimental setup, with blank entries marking parameters mentioned but not explicitly specified.The provided material identifies the table but does not include its parameter rows.
- B Case Study: Representative skills from Map, GitLab, Shopping, Reddit, and Admin are extracted from judged-successful trajectories as parameterized code functions paired with natural-language descriptions.The driving-directions example fills start and destination fields and clicks Go, separating structural webpage IDs from task-specific locations for reuse.
B.2 Merge Request Comment Submission … B.5 Shipping Carrier Selection
Across four WebArena case studies, SGDR induces compact, parameterized procedures that separate webpage-specific structural arguments from task-specific content and support diverse web interactions. The examples span merge-request commenting, product search and wishlist addition, comment replies, and shipping-carrier selection, with skills grounded in the page state for reuse when the required controls are visible.
- B.2 Merge Request Comment Submission: SGDR induces a merge-request commenting skill that fills a comment textbox and clicks the submit button.The procedure takes comment_box_id, submit_button_id, and comment as arguments.
- B.2 Merge Request Comment Submission: The merge-request example separates webpage element identifiers from task-dependent comment content and is useful when the required fields and button are visible.This illustrates why state-grounded retrieval matters for intermediate execution states.
- B.4 Comment Reply Submission: Together, the GitLab and Reddit examples show that similar fill-and-submit patterns emerge across code-collaboration and forum domains.Both procedures distinguish task-specific text from structural webpage arguments.
- B.3 Product Search and Wishlist Addition: SGDR induces a shopping subroutine that searches for a product, submits the query, and adds the first search result to a wishlist.Its arguments include product_query together with search_box_id, search_button_id, and add_to_wishlist_button_id.
- B.3 Product Search and Wishlist Addition: The shopping skill abstracts over repeated interactions by separating product_query from structural webpage arguments.The example demonstrates a longer e-commerce procedure rather than a simpler two-step fill-and-submit skill.
- B.4 Comment Reply Submission: SGDR induces a forum-reply skill that fills a reply textbox and submits a message on a comment thread page.The task-specific message is separated from reply_box_id and post_button_id.
- B.5 Shipping Carrier Selection: SGDR induces an Admin shipping skill that clicks Add Tracking Number and selects a carrier from a dropdown on an order-details page.The procedure uses select_option and separates carrier_name from add_tracking_btn_id and carrier_dropdown_id.
- B.5 Shipping Carrier Selection: Across all five WebArena domains, induced skills separate structural webpage arguments from task-specific content and cover form submission, comment posting, product search, wishlist addition, and dropdown selection.The case studies therefore combine page-state grounding with reuse across future tasks.